From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5BFE2550AF for ; Mon, 1 Dec 2025 12:42:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592932; cv=none; b=rUCySxWEMN9LH9grA+WqeDIT2e1lre0CnKn+iZDR3auyEtwzcsWB27RVpt7VB/0qJPJis2tJGQW9iiMes/706wsItExR0bRWSW5e/LkOUg7pVcuXtQ9FI0w3ZvAilBETqn5SFO0JTslhXqxcQc1NcKriD36Nl8LFtznukWpG5XU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592932; c=relaxed/simple; bh=jbA548yJQDkqAtQzfJUj6On8p4dvfo9kGRb+k7p2L6I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oUB5JVOHL5ziiZhpz5rkN3xt1AxIOPTq4jL7t8/d9CB90be0DDXZoo1TBoXolPHH4IRYdKz1Wr4JC2mlKMvoLdcitaAYH9gBFHd47u8xdj9aCEKiHBPzmip6TZmzCPmNkWL37qkmPzK169oTPlsWbp03r3/4nOCx9MzJV3oELak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Mbd2z+GX; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Mbd2z+GX" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b735ce67d1dso687545866b.3 for ; Mon, 01 Dec 2025 04:42:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592928; x=1765197728; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MfZWhz4tfpGGO+7FZPEiHA0UbhTazLSPBY31aWjLIRo=; b=Mbd2z+GXyOFUzKlmGBe7GZwJmRymKI7OBcqe2XN7ul2/OUfb3IKNwQpiGDV+sW85CN 1htnjdYFxpw1c4mDPTOd0IeygNoHngg/Wy3kEs0dbhTB+tnG2JRjEBEXttOfb4uOK043 0tZh3euE4BFJjyaeORt0tLUfSQDOwk7MYVDtU+XytxwQIWj5MnVTDiDRQqmqeb3TeDD2 kpORSgj2GwzvrlGfwxD0yi3Y+RRtT0lhecFO4yvuANFxLeZoGZBSYoymWyrjNYJkOjDx 3sOQYzwZFB6ep+L1p1Q+K18+LzEYCIq+gbt3hPBQstfLd6zmlq+3Iah6enMOq3wKsbxn 98mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592928; x=1765197728; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=MfZWhz4tfpGGO+7FZPEiHA0UbhTazLSPBY31aWjLIRo=; b=q56N6XHyOORVyLdXEe+/dCmWWUbUD2PDEqTm6wMu1OPGd6uDWqIPKBzcGPqxo9TxqK 8kMUSkIHEgdWFutvgSF2TdqV7QfnWWgYmsTPPO6NJaS6jdAmhRX7TfoUIlW2DSCZMQTL 6PcJylmcSjIRSog0M/4xhD2nmpWbHOuz/dOLfGSBjzqoTN+4Tlg/u445hsTIL366q5wl Bw98ky8JJT4GA817YAYXYqZ8XjPfVcI6cpNnAN79gHS8nyxkxCmghq6tqI9JgNmAn5qX ZIpHqHtRtGcus+0RamiSkdqcN/cwhBa/Eb+3TlT+GhIGyT4MFT1hwpSZYjm2IQGh5bK5 1DQQ== X-Gm-Message-State: AOJu0YxIpl7fpQ0q+ue4wt4xb43d/Ei2k9G+u3iUY3r+MPKL5wMc1BAe jkp3K7a02yKYkkTW2M6b8tzUIO07/O6O6YXA5e6gpB/pxaDIvn3FhGlt X-Gm-Gg: ASbGncvoQ2VCViTOoflLqbngUhCnnJogZtgfDTuTsu0XpA+zLf/g0/WqXhFEQh2XSkx yWnx+oflwU/c9zqZnqeTfTojnrQAMisQZiSEZSQ7UsQ+VadIFKiGHUGPD22YyvctyZXBztfc/Bj bQu6hx2W/nSQpejz0xEUNOLGpyhVem7rVaF/FJvjWVzooc/9vNUQ5LAb0GCSZ9pYRA6/DlUDte0 y3QGVKuIJRpFmZxtek09TfhV27HmNw4pBrTAtfxa2F/+atx6fdBPP7nIoK3XNKvhu2IDLnLJ5pu Is55chNa/H4K52Gc86n2YHyuU6AAiATA9X8qxMRAN7vEncuxBkAj98EsQ8yLAR9eR+f+CkxdAXy bLk0kzQWgkRGsk0JOlwEOGwwe4XNZGO1NJkqpV9SFmV1hm/eSUlBOo8X2N6TxDXiew6X4VV+njD m6+jrAb7rm X-Google-Smtp-Source: AGHT+IHpngQZ3ykuqVkwLPiQyqKVbbW0LFmvmVbm39ZWFBWeY7IKsSnO7jI8da+/ocCOoGZdDaInyA== X-Received: by 2002:a17:907:809:b0:b73:7280:7821 with SMTP id a640c23a62f3a-b76c558f338mr2667024366b.60.1764592927725; Mon, 01 Dec 2025 04:42:07 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:07 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 01/28] sched/deadline: Do not access dl_se->rq directly Date: Mon, 1 Dec 2025 13:41:34 +0100 Message-ID: <20251201124205.11169-2-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make deadline.c code access the runqueue of a scheduling entity saved in the sched_dl_entity data structure. This allows future patches to save different runqueues in sched_dl_entity other than the global runqueues. Move dl_server_apply_params call in sched_init_dl_servers as the rq_of_dl_se function will return the correct deadline entity only if the dl_server flag is set. Add a WARN_ON on the return value of dl_server_apply_params in sched_init_dl_servers as this function may fail if the kernel is not configured correctly. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/deadline.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 7b7671060b..d7940c45f7 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -874,7 +874,7 @@ static void replenish_dl_entity(struct sched_dl_entity = *dl_se) * and arm the defer timer. */ if (dl_se->dl_defer && !dl_se->dl_defer_running && - dl_time_before(rq_clock(dl_se->rq), dl_se->deadline - dl_se->runtime)= ) { + dl_time_before(rq_clock(rq), dl_se->deadline - dl_se->runtime)) { if (!is_dl_boosted(dl_se)) { =20 /* @@ -1176,11 +1176,11 @@ static enum hrtimer_restart dl_server_timer(struct = hrtimer *timer, struct sched_ * of time. The dl_server_min_res serves as a limit to avoid * forwarding the timer for a too small amount of time. */ - if (dl_time_before(rq_clock(dl_se->rq), + if (dl_time_before(rq_clock(rq), (dl_se->deadline - dl_se->runtime - dl_server_min_res))) { =20 /* reset the defer timer */ - fw =3D dl_se->deadline - rq_clock(dl_se->rq) - dl_se->runtime; + fw =3D dl_se->deadline - rq_clock(rq) - dl_se->runtime; =20 hrtimer_forward_now(timer, ns_to_ktime(fw)); return HRTIMER_RESTART; @@ -1191,7 +1191,7 @@ static enum hrtimer_restart dl_server_timer(struct hr= timer *timer, struct sched_ =20 enqueue_dl_entity(dl_se, ENQUEUE_REPLENISH); =20 - if (!dl_task(dl_se->rq->curr) || dl_entity_preempt(dl_se, &dl_se->rq->cu= rr->dl)) + if (!dl_task(rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl)) resched_curr(rq); =20 __push_dl_task(rq, rf); @@ -1459,7 +1459,7 @@ static void update_curr_dl_se(struct rq *rq, struct s= ched_dl_entity *dl_se, s64 =20 hrtimer_try_to_cancel(&dl_se->dl_timer); =20 - replenish_dl_new_period(dl_se, dl_se->rq); + replenish_dl_new_period(dl_se, rq); =20 /* * Not being able to start the timer seems problematic. If it could not @@ -1572,23 +1572,24 @@ void dl_server_update(struct sched_dl_entity *dl_se= , s64 delta_exec) { /* 0 runtime =3D fair server disabled */ if (dl_se->dl_runtime) - update_curr_dl_se(dl_se->rq, dl_se, delta_exec); + update_curr_dl_se(rq_of_dl_se(dl_se), dl_se, delta_exec); } =20 void dl_server_start(struct sched_dl_entity *dl_se) { - struct rq *rq =3D dl_se->rq; + struct rq *rq; =20 if (!dl_server(dl_se) || dl_se->dl_server_active) return; =20 + rq =3D rq_of_dl_se(dl_se); if (WARN_ON_ONCE(!cpu_online(cpu_of(rq)))) return; =20 dl_se->dl_server_active =3D 1; enqueue_dl_entity(dl_se, ENQUEUE_WAKEUP); - if (!dl_task(dl_se->rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl)) - resched_curr(dl_se->rq); + if (!dl_task(rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl)) + resched_curr(rq); } =20 void dl_server_stop(struct sched_dl_entity *dl_se) @@ -1628,9 +1629,9 @@ void sched_init_dl_servers(void) =20 WARN_ON(dl_server(dl_se)); =20 - dl_server_apply_params(dl_se, runtime, period, 1); - dl_se->dl_server =3D 1; + WARN_ON(dl_server_apply_params(dl_se, runtime, period, 1)); + dl_se->dl_defer =3D 1; setup_new_dl_entity(dl_se); } @@ -1655,7 +1656,7 @@ int dl_server_apply_params(struct sched_dl_entity *dl= _se, u64 runtime, u64 perio { u64 old_bw =3D init ? 0 : to_ratio(dl_se->dl_period, dl_se->dl_runtime); u64 new_bw =3D to_ratio(period, runtime); - struct rq *rq =3D dl_se->rq; + struct rq *rq =3D rq_of_dl_se(dl_se); int cpu =3D cpu_of(rq); struct dl_bw *dl_b; unsigned long cap; @@ -1732,7 +1733,7 @@ static enum hrtimer_restart inactive_task_timer(struc= t hrtimer *timer) p =3D dl_task_of(dl_se); rq =3D task_rq_lock(p, &rf); } else { - rq =3D dl_se->rq; + rq =3D rq_of_dl_se(dl_se); rq_lock(rq, &rf); } =20 --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFFF430C37D for ; Mon, 1 Dec 2025 12:42:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592932; cv=none; b=lFSw2pn9r2YpIvAHDjmffm+H4bjJmG+5q4qQlj6iun6amHtuSTxocZDuU6DUumqqLmeyKdlkVwHsfLqIAKz1WDIPGTgfmHnK8uqnhoV3Afp3BJ184PZudP209FVisRuxq7bAzXIcwzs6Oj6cFh7sjWnTj08jAOdz8/roESuhxsg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592932; c=relaxed/simple; bh=/yJIMbw0pN7Tv4Sc1jnGyXAcHZjShnRJ61RDLzkuL64=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j4jun1blhBeUuul4WP3hPbzWGvD4JFdrhW/2gb1MA+30A3lP0JIUAERI0iQWor9FquB7OY+J2UlxKBAZAiJp/QFCWTa3LjJNO4w1Z+rziGo/cKTQO9wpQEJWksK2/sdd2Mz+zmL/g72z+T5UHhha/wNCRmkVKbfGHudcoDaBlWE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GWXHLoR2; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GWXHLoR2" Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-b735ce67d1dso687549766b.3 for ; Mon, 01 Dec 2025 04:42:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592929; x=1765197729; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zMOxQNbTJbyJkfx5C97heHQ9owdvgbQ7zmiPOjpmg/E=; b=GWXHLoR2pF4fCkxdfJKVQJ/NKhZpyCayT4LLL16miDCMs6JZt1AWLlTzXYwwH1N5cR 2ZzL3PMPsjiu2WMpcWAEEE6NSaRH1TIPyNSTCqjMACMIGRd3m4fhY1iXMzK3ybbLMlfq pGuOWArzXTWQx1VD/aQCXD2oSNzWoSMb1qtUpv3eXv13nSxrQP7Kr6E9XGYO7FTORu8N dGAtE7P9asFs48sKS7Xz6/0MMZAYKH2ubsrvq3ApW9LB1M5N3dMGEc9ooTbyXtzZJpqC EnAB4A6OX3jvqdOLM1V8u8nYJmYlVkopPuCOOsv9qBn5vRSbG4s5NTB1ClHeR6q09coe cOrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592929; x=1765197729; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=zMOxQNbTJbyJkfx5C97heHQ9owdvgbQ7zmiPOjpmg/E=; b=i2FyR4em4YOv5ZTT6SpGySSMDt2RaJcd0uWyw6AQ1awZlyrUSBGaEUH8jDa/xwBLNf ffMncIsSdKFzhHnz6+AvabwpVlNRBAnBusWmw17rA6KkgKMEHbDa8oYB67l1VbzYFAHs 9vMnwb1eSYA30GcYHrJuk3RwYXA4HMwVyuBDKu3uz2Brgr9HtfksTvDbbGMBQvDziywY RUAA4bQS8i6lrQtKp0b8d7fptr/igslQ7SwEoSxBE8qUvPlqOEkxYAz1tEQnWA0CV6TN JgomUmHRx472eCszrLWPrFJoTmR4BUJUrM4EeY4LdefsMi1eJ7XHjf3LollYZciqxwgW qF6w== X-Gm-Message-State: AOJu0YzjT7FGOz4f6i39Rc9FbLlme4QRjIl08ABsFZkcdfHU1mCUA5pM FVuQX7VNE23inhC5Daia7CsSJEGq7f059G944MTIjOTMsbnN0rLLnZVm X-Gm-Gg: ASbGnct0N9glnA4iySlgt9fd2kwEVJqOujDQaEjsGGPxMSdVh4U3lLShuksOCvz+6V8 qkAkak7uwOq7gAp8t5nk8qRTb4ndtWZw4wSA3N81W1iaeP8ul+RZp/dZ/H6rCMOqxZjaC0jCQWn UHhXf/BJGjeJfmnbSAlF3CUzC6WMHj/zlcmFEVtAFFZpwRFiPl5WctlBaSScKnJb+4kt/L2hIt2 pgXpcirBeinDHHsKPrI5xirEVtKroZ7qAN3zQkss0vBkjjYslojxP9ElnPUxmA15LA+U7S9LZiJ XBWvaM/Y9XDbdeKd4i0dvFIIeaTun1KrAMYyXWaS7guZzu85S9YcA0Q+OsLwrnfKUjygJXTBzoL HpoCDbxSOfG2tLbPPHbxB2KP3AuBerOuw7E7mH1DueoPI+dlSBktl1kon66ex11b62M377kKHDc o7CQk7KurZ X-Google-Smtp-Source: AGHT+IGV39T80ugkBft9UgM6j80yH3kMGRsNgzwIw/vB3ynxeWNqU+pUiOGDJs59WkZnQQSKfe3BWQ== X-Received: by 2002:a17:907:9496:b0:b73:2aa1:f9fb with SMTP id a640c23a62f3a-b76c55131ecmr2716290166b.32.1764592928894; Mon, 01 Dec 2025 04:42:08 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:08 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 02/28] sched/deadline: Distinct between dl_rq and my_q Date: Mon, 1 Dec 2025 13:41:35 +0100 Message-ID: <20251201124205.11169-3-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Split the single runqueue pointer in sched_dl_entity into two separate pointers, following the existing pattern used by sched_rt_entity: - dl_rq: Points to the deadline runqueue where this entity is queued (global runqueue). - my_q: Points to the runqueue that this entity serves (for servers). This distinction is currently redundant for the fair_server (both point to the same CPU's structures), but is essential for future RT cgroup support where deadline servers will be queued on the global dl_rq while serving tasks from cgroup-specific runqueues. Update rq_of_dl_se() to use container_of() to recover the global rq from dl_rq, and update fair.c to explicitly use my_q (local rq) when accessing the served runqueue. Update dl_server_init() to take a dl_rq pointer (use to retrieve the global runqueue where the dl_server is scheduled) and a rq pointer (for the local runqueue served by the server). Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- include/linux/sched.h | 6 ++++-- kernel/sched/deadline.c | 11 +++++++---- kernel/sched/fair.c | 4 ++-- kernel/sched/sched.h | 3 ++- 4 files changed, 15 insertions(+), 9 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index b469878de2..000aa3b2b1 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -729,12 +729,14 @@ struct sched_dl_entity { * Bits for DL-server functionality. Also see the comment near * dl_server_update(). * - * @rq the runqueue this server is for + * @dl_rq the runqueue on which this entity is (to be) queued + * @my_q the runqueue "owned" by this entity * * @server_has_tasks() returns true if @server_pick return a * runnable task. */ - struct rq *rq; + struct dl_rq *dl_rq; + struct rq *my_q; dl_server_pick_f server_pick_task; =20 #ifdef CONFIG_RT_MUTEXES diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index d7940c45f7..e36d98eb5f 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -75,11 +75,12 @@ static inline struct rq *rq_of_dl_rq(struct dl_rq *dl_r= q) =20 static inline struct rq *rq_of_dl_se(struct sched_dl_entity *dl_se) { - struct rq *rq =3D dl_se->rq; + struct rq *rq; =20 if (!dl_server(dl_se)) rq =3D task_rq(dl_task_of(dl_se)); - + else + rq =3D container_of(dl_se->dl_rq, struct rq, dl); return rq; } =20 @@ -1604,10 +1605,12 @@ void dl_server_stop(struct sched_dl_entity *dl_se) dl_se->dl_server_active =3D 0; } =20 -void dl_server_init(struct sched_dl_entity *dl_se, struct rq *rq, +void dl_server_init(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq, + struct rq *served_rq, dl_server_pick_f pick_task) { - dl_se->rq =3D rq; + dl_se->dl_rq =3D dl_rq; + dl_se->my_q =3D served_rq; dl_se->server_pick_task =3D pick_task; } =20 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 5b75232427..9c724d8232 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8959,7 +8959,7 @@ static struct task_struct *__pick_next_task_fair(stru= ct rq *rq, struct task_stru =20 static struct task_struct *fair_server_pick_task(struct sched_dl_entity *d= l_se) { - return pick_task_fair(dl_se->rq); + return pick_task_fair(dl_se->my_q); } =20 void fair_server_init(struct rq *rq) @@ -8968,7 +8968,7 @@ void fair_server_init(struct rq *rq) =20 init_dl_entity(dl_se); =20 - dl_server_init(dl_se, rq, fair_server_pick_task); + dl_server_init(dl_se, &rq->dl, rq, fair_server_pick_task); } =20 /* diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index adfb6e3409..10733b7cb7 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -408,7 +408,8 @@ extern s64 dl_scaled_delta_exec(struct rq *rq, struct s= ched_dl_entity *dl_se, s6 extern void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec= ); extern void dl_server_start(struct sched_dl_entity *dl_se); extern void dl_server_stop(struct sched_dl_entity *dl_se); -extern void dl_server_init(struct sched_dl_entity *dl_se, struct rq *rq, +extern void dl_server_init(struct sched_dl_entity *dl_se, struct dl_rq *dl= _rq, + struct rq *served_rq, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); =20 --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9C9830DEAD for ; Mon, 1 Dec 2025 12:42:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592934; cv=none; b=d7v9bgEf7i9q8/2LOmk3V+Zbc+gTAv+VilZaWFqNyUmZ6iQpqeLK3oQF+6UfpZFwez0QRp1DBGsbYEU6uhMd4hW3+GhuZxx0DNI5Y7/fl8otpfUD9/EdFcoIdXflljjCarh9ZRIJY6HbPBzTxZm4VAzHYILTePqFz7q1eyFT8z0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592934; c=relaxed/simple; bh=lWa65bQkTwSYd1W45tRe4xFLtKOdyORrO5xSjj11SEM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ChZliWBOPF6gzLMS+QMqjFOFOxthln8SfLdQrOnw/zM8+RmC8aTZGDmx5A0SkDP/BecoUIr6fh7SXIbs0j1N9iPzrH0xI2M9atcaWSeVpKPLOtFN3nTwTOqu0S3BHhJcHpO8yIT9Sa9Zpc/hrzjAtqyrOkmRWfczwEi+oWMkQIM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MZUOePba; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MZUOePba" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b7355f6ef12so824519766b.3 for ; Mon, 01 Dec 2025 04:42:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592930; x=1765197730; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8sAirH3HXHVSEKC4rvVYd3trYR/HU2YcPNGLniUqCIw=; b=MZUOePba5yPd1iZc6BT4y0V1VGzAlhq6XZFDF+EXBRPkfyjL2MSRJI7jnpqo+knrnu GutzidCPJrMo+8mdJp105+CJ0Gz//sKR+xV1OQpf1/NL+iGdZnXAw333aIDfFm2/XigX /HDOpXCexftNoEuqe4qYWCT9fs7/9Mp3oSRPaLuXFFGjTUKiL+5tJbokWgx1F+afXOQi TPTnt2dG2wiUUnWKfy+ESEgattlaTeigrSrUiwTm1Frs0lDi2B2ZWvwRouKpE+aG2dFo x1IaN0Xeh8huvnDght8l+XeryC44nOAigkH1grJUbPPmguDuNF1tsF4yfEVkwD7DVvZJ 2b7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592930; x=1765197730; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=8sAirH3HXHVSEKC4rvVYd3trYR/HU2YcPNGLniUqCIw=; b=wDbUb30Tw7QA1vb7Zx/3kpu4o8JWS0QKnp5lEffvkpAXoEH1T2QfmH0Xpj9Qwtb2qJ cd6mj6UiaF8qosbaYuQ+4zlFywAYHbgffDXoOkOHezZZeX7B//571w/kEqXaAnJiVTXx G5cVIH8sarjb0C9icKO7SetbIhlfr5g32wKf/8HtgXOHDcPx1e/WFlie+jm3Tk7CiC5c AsIVQqR4Vk/hb+9o2Jg4NacketRCVxDZyZuAbfILP5cZvvEmrqZC0H0RHsk4dr8uDxiw /RokJa2LGaMY/3IO6/lDPe7c+11c5MgcHo1G6iP2lMWvBV/4xSYzZz1z/m2EYYxZinif dK9g== X-Gm-Message-State: AOJu0YwsLMRUemsda4pudhCL6igunKjbN9JeIFk7bAKSY3VNECHQYbOR +ZpP0DPARNDFVKn/2ieF4muORO0vsbhuMOp1iD0HT8a1hfl+APjcBsrj X-Gm-Gg: ASbGncvER1uz9fDOw1KvEM2KmqWlkOfXanvWTDtGvZDI15DSZ9C380aQDdkgQwEVmh0 v/NZNdjmcWIGIrQQ7oB6ReV/erj7kCiRyJquW2SEkXwdWOhSPI+Gv84oq3yGkvKVVrZCLQFBpTh BXvTLRp9DqxpuFVRHU64F67egoLgzwRAPxr5xnsV3+h4UbAylC0OOckN4qfHryo6g5eMRNUiZBc hO8WNm+/qWU2cVI+DPYDizD+Gv2VmyeuFBKQ4F8ep9yYf46lAlM3MNtAQSQT8x9KCohUjo2zWBI 87nJ9tAEi0RTq+3cbBCgEcRFl8P1sJa900ZADJirI8nmFzaQfrvMsxK8ZJ23EgaJutuSn3Is5OD WhFKQdtGOnSxZwa117ewBFli3lMf72x19dKbWOH+o/HjQJ58TSqfRHdTs1fTEU9TYRVBy7874+w 37RFR2Zwg3 X-Google-Smtp-Source: AGHT+IHIKl5+K15T00TMzCGeXTdEyLLGL7K0fymq22J65ryJ+SbkjIbIWhSdr60TaY3sy0uYge3yeg== X-Received: by 2002:a17:906:fe49:b0:b72:b289:6de3 with SMTP id a640c23a62f3a-b76719d0982mr4368147966b.58.1764592929789; Mon, 01 Dec 2025 04:42:09 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:09 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 03/28] sched/rt: Pass an rt_rq instead of an rq where needed Date: Mon, 1 Dec 2025 13:41:36 +0100 Message-ID: <20251201124205.11169-4-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make rt.c code access the runqueue through the rt_rq data structure rather than passing an rq pointer directly. This allows future patches to define rt_rq data structures which do not refer only to the global runqueue, but also to local cgroup runqueues (as rt_rq will not be always equal to &rq->rt). Add checks in rt_queue_{push/pull}_tasks to make sure that the given rt_rq object refers to a global runqueue and not any local one. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 99 ++++++++++++++++++++++++++--------------------- 1 file changed, 54 insertions(+), 45 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 7936d43337..b7bf80a571 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -370,9 +370,9 @@ static inline void rt_clear_overload(struct rq *rq) cpumask_clear_cpu(rq->cpu, rq->rd->rto_mask); } -static inline int has_pushable_tasks(struct rq *rq) +static inline int has_pushable_tasks(struct rt_rq *rt_rq) { - return !plist_head_empty(&rq->rt.pushable_tasks); + return !plist_head_empty(&rt_rq->pushable_tasks); } static DEFINE_PER_CPU(struct balance_callback, rt_push_head); @@ -381,50 +381,54 @@ static DEFINE_PER_CPU(struct balance_callback, rt_pul= l_head); static void push_rt_tasks(struct rq *); static void pull_rt_task(struct rq *); -static inline void rt_queue_push_tasks(struct rq *rq) +static inline void rt_queue_push_tasks(struct rt_rq *rt_rq) { - if (!has_pushable_tasks(rq)) + struct rq *rq =3D container_of(rt_rq, struct rq, rt); + + if (!has_pushable_tasks(rt_rq)) return; queue_balance_callback(rq, &per_cpu(rt_push_head, rq->cpu), push_rt_tasks= ); } -static inline void rt_queue_pull_task(struct rq *rq) +static inline void rt_queue_pull_task(struct rt_rq *rt_rq) { + struct rq *rq =3D container_of(rt_rq, struct rq, rt); + queue_balance_callback(rq, &per_cpu(rt_pull_head, rq->cpu), pull_rt_task); } -static void enqueue_pushable_task(struct rq *rq, struct task_struct *p) +static void enqueue_pushable_task(struct rt_rq *rt_rq, struct task_struct = *p) { - plist_del(&p->pushable_tasks, &rq->rt.pushable_tasks); + plist_del(&p->pushable_tasks, &rt_rq->pushable_tasks); plist_node_init(&p->pushable_tasks, p->prio); - plist_add(&p->pushable_tasks, &rq->rt.pushable_tasks); + plist_add(&p->pushable_tasks, &rt_rq->pushable_tasks); /* Update the highest prio pushable task */ - if (p->prio < rq->rt.highest_prio.next) - rq->rt.highest_prio.next =3D p->prio; + if (p->prio < rt_rq->highest_prio.next) + rt_rq->highest_prio.next =3D p->prio; - if (!rq->rt.overloaded) { - rt_set_overload(rq); - rq->rt.overloaded =3D 1; + if (!rt_rq->overloaded) { + rt_set_overload(rq_of_rt_rq(rt_rq)); + rt_rq->overloaded =3D 1; } } -static void dequeue_pushable_task(struct rq *rq, struct task_struct *p) +static void dequeue_pushable_task(struct rt_rq *rt_rq, struct task_struct = *p) { - plist_del(&p->pushable_tasks, &rq->rt.pushable_tasks); + plist_del(&p->pushable_tasks, &rt_rq->pushable_tasks); /* Update the new highest prio pushable task */ - if (has_pushable_tasks(rq)) { - p =3D plist_first_entry(&rq->rt.pushable_tasks, + if (has_pushable_tasks(rt_rq)) { + p =3D plist_first_entry(&rt_rq->pushable_tasks, struct task_struct, pushable_tasks); - rq->rt.highest_prio.next =3D p->prio; + rt_rq->highest_prio.next =3D p->prio; } else { - rq->rt.highest_prio.next =3D MAX_RT_PRIO-1; + rt_rq->highest_prio.next =3D MAX_RT_PRIO-1; - if (rq->rt.overloaded) { - rt_clear_overload(rq); - rq->rt.overloaded =3D 0; + if (rt_rq->overloaded) { + rt_clear_overload(rq_of_rt_rq(rt_rq)); + rt_rq->overloaded =3D 0; } } } @@ -1431,6 +1435,7 @@ static void enqueue_task_rt(struct rq *rq, struct task_struct *p, int flags) { struct sched_rt_entity *rt_se =3D &p->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); if (flags & ENQUEUE_WAKEUP) rt_se->timeout =3D 0; @@ -1444,17 +1449,18 @@ enqueue_task_rt(struct rq *rq, struct task_struct *= p, int flags) return; if (!task_current(rq, p) && p->nr_cpus_allowed > 1) - enqueue_pushable_task(rq, p); + enqueue_pushable_task(rt_rq, p); } static bool dequeue_task_rt(struct rq *rq, struct task_struct *p, int flag= s) { struct sched_rt_entity *rt_se =3D &p->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); update_curr_rt(rq); dequeue_rt_entity(rt_se, flags); - dequeue_pushable_task(rq, p); + dequeue_pushable_task(rt_rq, p); return true; } @@ -1639,14 +1645,14 @@ static void wakeup_preempt_rt(struct rq *rq, struct= task_struct *p, int flags) static inline void set_next_task_rt(struct rq *rq, struct task_struct *p, = bool first) { struct sched_rt_entity *rt_se =3D &p->rt; - struct rt_rq *rt_rq =3D &rq->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); p->se.exec_start =3D rq_clock_task(rq); if (on_rt_rq(&p->rt)) update_stats_wait_end_rt(rt_rq, rt_se); /* The running task is never eligible for pushing */ - dequeue_pushable_task(rq, p); + dequeue_pushable_task(rt_rq, p); if (!first) return; @@ -1659,7 +1665,7 @@ static inline void set_next_task_rt(struct rq *rq, st= ruct task_struct *p, bool f if (rq->donor->sched_class !=3D &rt_sched_class) update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 0); - rt_queue_push_tasks(rq); + rt_queue_push_tasks(rt_rq); } static struct sched_rt_entity *pick_next_rt_entity(struct rt_rq *rt_rq) @@ -1710,7 +1716,7 @@ static struct task_struct *pick_task_rt(struct rq *rq) static void put_prev_task_rt(struct rq *rq, struct task_struct *p, struct = task_struct *next) { struct sched_rt_entity *rt_se =3D &p->rt; - struct rt_rq *rt_rq =3D &rq->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); if (on_rt_rq(&p->rt)) update_stats_wait_start_rt(rt_rq, rt_se); @@ -1726,7 +1732,7 @@ static void put_prev_task_rt(struct rq *rq, struct ta= sk_struct *p, struct task_s * if it is still active */ if (on_rt_rq(&p->rt) && p->nr_cpus_allowed > 1) - enqueue_pushable_task(rq, p); + enqueue_pushable_task(rt_rq, p); } /* Only try algorithms three times */ @@ -1736,16 +1742,16 @@ static void put_prev_task_rt(struct rq *rq, struct = task_struct *p, struct task_s * Return the highest pushable rq's task, which is suitable to be executed * on the CPU, NULL otherwise */ -static struct task_struct *pick_highest_pushable_task(struct rq *rq, int c= pu) +static struct task_struct *pick_highest_pushable_task(struct rt_rq *rt_rq,= int cpu) { - struct plist_head *head =3D &rq->rt.pushable_tasks; + struct plist_head *head =3D &rt_rq->pushable_tasks; struct task_struct *p; - if (!has_pushable_tasks(rq)) + if (!has_pushable_tasks(rt_rq)) return NULL; plist_for_each_entry(p, head, pushable_tasks) { - if (task_is_pushable(rq, p, cpu)) + if (task_is_pushable(rq_of_rt_rq(rt_rq), p, cpu)) return p; } @@ -1845,14 +1851,15 @@ static int find_lowest_rq(struct task_struct *task) return -1; } -static struct task_struct *pick_next_pushable_task(struct rq *rq) +static struct task_struct *pick_next_pushable_task(struct rt_rq *rt_rq) { + struct rq *rq =3D rq_of_rt_rq(rt_rq); struct task_struct *p; - if (!has_pushable_tasks(rq)) + if (!has_pushable_tasks(rt_rq)) return NULL; - p =3D plist_first_entry(&rq->rt.pushable_tasks, + p =3D plist_first_entry(&rt_rq->pushable_tasks, struct task_struct, pushable_tasks); BUG_ON(rq->cpu !=3D task_cpu(p)); @@ -1905,7 +1912,7 @@ static struct rq *find_lock_lowest_rq(struct task_str= uct *task, struct rq *rq) */ if (unlikely(is_migration_disabled(task) || !cpumask_test_cpu(lowest_rq->cpu, &task->cpus_mask) || - task !=3D pick_next_pushable_task(rq))) { + task !=3D pick_next_pushable_task(&rq->rt))) { double_unlock_balance(rq, lowest_rq); lowest_rq =3D NULL; @@ -1939,7 +1946,7 @@ static int push_rt_task(struct rq *rq, bool pull) if (!rq->rt.overloaded) return 0; - next_task =3D pick_next_pushable_task(rq); + next_task =3D pick_next_pushable_task(&rq->rt); if (!next_task) return 0; @@ -2014,7 +2021,7 @@ static int push_rt_task(struct rq *rq, bool pull) * run-queue and is also still the next task eligible for * pushing. */ - task =3D pick_next_pushable_task(rq); + task =3D pick_next_pushable_task(&rq->rt); if (task =3D=3D next_task) { /* * The task hasn't migrated, and is still the next @@ -2202,7 +2209,7 @@ void rto_push_irq_work_func(struct irq_work *work) * We do not need to grab the lock to check for has_pushable_tasks. * When it gets updated, a check is made if a push is possible. */ - if (has_pushable_tasks(rq)) { + if (has_pushable_tasks(&rq->rt)) { raw_spin_rq_lock(rq); while (push_rt_task(rq, true)) ; @@ -2231,6 +2238,7 @@ static void pull_rt_task(struct rq *this_rq) int this_cpu =3D this_rq->cpu, cpu; bool resched =3D false; struct task_struct *p, *push_task; + struct rt_rq *src_rt_rq; struct rq *src_rq; int rt_overload_count =3D rt_overloaded(this_rq); @@ -2260,6 +2268,7 @@ static void pull_rt_task(struct rq *this_rq) continue; src_rq =3D cpu_rq(cpu); + src_rt_rq =3D &src_rq->rt; /* * Don't bother taking the src_rq->lock if the next highest @@ -2268,7 +2277,7 @@ static void pull_rt_task(struct rq *this_rq) * logically higher, the src_rq will push this task away. * And if its going logically lower, we do not care */ - if (src_rq->rt.highest_prio.next >=3D + if (src_rt_rq->highest_prio.next >=3D this_rq->rt.highest_prio.curr) continue; @@ -2284,7 +2293,7 @@ static void pull_rt_task(struct rq *this_rq) * We can pull only a task, which is pushable * on its rq, and no others. */ - p =3D pick_highest_pushable_task(src_rq, this_cpu); + p =3D pick_highest_pushable_task(src_rt_rq, this_cpu); /* * Do we have an RT task that preempts @@ -2390,7 +2399,7 @@ static void switched_from_rt(struct rq *rq, struct ta= sk_struct *p) if (!task_on_rq_queued(p) || rq->rt.rt_nr_running) return; - rt_queue_pull_task(rq); + rt_queue_pull_task(rt_rq_of_se(&p->rt)); } void __init init_sched_rt_class(void) @@ -2426,7 +2435,7 @@ static void switched_to_rt(struct rq *rq, struct task= _struct *p) */ if (task_on_rq_queued(p)) { if (p->nr_cpus_allowed > 1 && rq->rt.overloaded) - rt_queue_push_tasks(rq); + rt_queue_push_tasks(rt_rq_of_se(&p->rt)); if (p->prio < rq->donor->prio && cpu_online(cpu_of(rq))) resched_curr(rq); } @@ -2448,7 +2457,7 @@ prio_changed_rt(struct rq *rq, struct task_struct *p,= int oldprio) * may need to pull tasks to this runqueue. */ if (oldprio < p->prio) - rt_queue_pull_task(rq); + rt_queue_pull_task(rt_rq_of_se(&p->rt)); /* * If there's a higher priority task waiting to run -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA76030BBBB for ; Mon, 1 Dec 2025 12:42:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592934; cv=none; b=cqiCpaOHpaeQ/bTPkEBr+nFsFP/988jCvrCUhtZ/LOkZ8q61Kc3MDPZfOarGzO6fc/OhiU0MvMEteYTTTL39fHXHN0lfe0rQvBG4UK19Smgr4pjqU8DhwQWci96KpqIx2oWE0j4U/HtS+hiRz3UxhTh8vOUlqpeATHPLf99X8AI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592934; c=relaxed/simple; bh=eKQmw+YKYRUrzNEnspCBhJYfWsdCTd+kyXQy35hAu9U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rjhdM5vWbMAILne+3uPmxqkOuXip39hRZ+SpoMlYHsL8gzGPG1oYVH5h4lIb9SiBmqKMNr7L7Pm4WeAcZskm/WWJuIEDRPTs5B7h0mbKfQDNvyPvdjgLp/KYUzQ1rU5SALWjlIkgvGhQpWhKe8uaUEuZdmZJ+QO+ZGyHciLs/08= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cAX5reQQ; arc=none smtp.client-ip=209.85.208.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cAX5reQQ" Received: by mail-ed1-f42.google.com with SMTP id 4fb4d7f45d1cf-64198771a9bso8038735a12.2 for ; Mon, 01 Dec 2025 04:42:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592931; x=1765197731; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gg5gfp1wMHo7BZByH/VSJvF4QUqT0jh+BRltGi2JdnU=; b=cAX5reQQFEAzzk2K75wJiF8hRRl7kiwe+FBuf2TEOLstIz/FDvMD14E4LiieRF4Xda 9p/rOYWhb4Cr3GbZH8bQx+lFp8GDDqmfVpBNCcTBOH4yjxgYLqpLf6ZNGuG43msnroAO ij1lIWBKM71CMI4SNgHPtjwYZPTB2Qng0IoBUyLbsFW1n1hVBRPJF7hwWYggKZljnSAU /bFOCLNyLvH5P6btWDtGhYxaO4uC1YXr1xh6hGBOe1h6NVjWashhrS6iLKJU8JUhCuQM bndPLBzGBCFI9yTp4vXkGT+/RDab/WfswIFy3S+m4YMYcVIsJCgGcN1PBpzZoYoJLGRH 4f5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592931; x=1765197731; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=gg5gfp1wMHo7BZByH/VSJvF4QUqT0jh+BRltGi2JdnU=; b=DEVMdGn/qKyC4rScdJDn64468TIpw9UAux8Em9n35oSAYG81jXJGwo9WxpG0pT12Jp jqmYNdJo4F/6+GaqxchrnTX8zqkGlWMgent81zsqJtYd1JZDfABrhjT1KT0GV8uvvyLw JvQqK2yKcu3/bLvjikEd/h5ipOzY4ausBBNCpH9UFdxSSY4eA4scJLDPDbe+RRm6VfM3 jXzeQ4n3TIh4+nFUgpI3wskYf8EapFY3wK78Kzeo1QjOLt7zePXRXLT2HXJRcfZcR+Gz NmPv4A+wNShugStEo9ot88y6wFBaEkrQf1Ms76X25yDL6naI3/fISt467gZ1WZ8VWcyq alug== X-Gm-Message-State: AOJu0YyvHLoRbnoqdnU2CdQ1Qz2p1Ob89u01Evsxgnks2zKuvR4qI69x yiTDQy37VmuG9m76dorB2FwVhALzUxEjCEJ991bWuWtZs0QXTdFD7SeH X-Gm-Gg: ASbGncsHL42FiUclBb0vM+0z3Uk33o77jI4wWXMcmibYbwSB5v2Vh/Qh2GT9tQ25XyU T8qkq6oCQdyvkkkVzu2D9aQcesTCnXaJxvg6fbTQknqMQFXgeYRTczybk0GwtMEtjVG3ChkFyua zfq1NxbeCNWMnLor34x5ltAqKIkonl+pD2bGpD7Z2qFnsdwCbRFgZtEN1aK+t7HGeiJsjxBVA0X AVZreCtGcHfzTiyckNLfvKaoliJJA5X2w1mLlkWJ9W8yxK0824S3Fwzw+ndxX+Y0BUUOc/imEOy PcdFbsCL9NK6stamxK6Kqh9mF0PBuxwsFsXTAGMtDIWgDY7zQ15tIquYm/nxJUVCnKJl6jtXLnC ZXGfSIJDLRkbJ1DuNOTdrwy3V+jQfb4uE9lJXoKOl6hqeXKJB4cA37fNM11TbL3+/3c/UpW8l8H W5ZNuX+SDs X-Google-Smtp-Source: AGHT+IFXV/QwkOcHoYKisyYle/kvP8ENJFSSh7+Y2OAD1tMutjVmmjxGE7NdHqFlJc+HXE3FwUNMcA== X-Received: by 2002:a17:907:928e:b0:b73:3d15:6337 with SMTP id a640c23a62f3a-b767158b529mr4329817566b.21.1764592930888; Mon, 01 Dec 2025 04:42:10 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:10 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 04/28] sched/rt: Move some functions from rt.c to sched.h Date: Mon, 1 Dec 2025 13:41:37 +0100 Message-ID: <20251201124205.11169-5-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make the following functions/macros be non-static and move them in sched.h, so that they can be also used in other source files: - rt_entity_is_task() - rt_task_of() - rq_of_rt_rq() - rt_rq_of_se() - rq_of_rt_se() There are no functional changes. This is needed by future patches. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 56 ------------------------------------------ kernel/sched/sched.h | 58 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 56 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index b7bf80a571..b6264e715c 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -166,36 +166,6 @@ static void destroy_rt_bandwidth(struct rt_bandwidth *= rt_b) hrtimer_cancel(&rt_b->rt_period_timer); } -#define rt_entity_is_task(rt_se) (!(rt_se)->my_q) - -static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) -{ - WARN_ON_ONCE(!rt_entity_is_task(rt_se)); - - return container_of(rt_se, struct task_struct, rt); -} - -static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) -{ - /* Cannot fold with non-CONFIG_RT_GROUP_SCHED version, layout */ - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); - return rt_rq->rq; -} - -static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) -{ - WARN_ON(!rt_group_sched_enabled() && rt_se->rt_rq->tg !=3D &root_task_gro= up); - return rt_se->rt_rq; -} - -static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) -{ - struct rt_rq *rt_rq =3D rt_se->rt_rq; - - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); - return rt_rq->rq; -} - void unregister_rt_sched_group(struct task_group *tg) { if (!rt_group_sched_enabled()) @@ -294,32 +264,6 @@ int alloc_rt_sched_group(struct task_group *tg, struct= task_group *parent) #else /* !CONFIG_RT_GROUP_SCHED: */ -#define rt_entity_is_task(rt_se) (1) - -static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) -{ - return container_of(rt_se, struct task_struct, rt); -} - -static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) -{ - return container_of(rt_rq, struct rq, rt); -} - -static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) -{ - struct task_struct *p =3D rt_task_of(rt_se); - - return task_rq(p); -} - -static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) -{ - struct rq *rq =3D rq_of_rt_se(rt_se); - - return &rq->rt; -} - void unregister_rt_sched_group(struct task_group *tg) { } void free_rt_sched_group(struct task_group *tg) { } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 10733b7cb7..964d596d8e 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3071,6 +3071,64 @@ extern void set_rq_offline(struct rq *rq); extern bool sched_smp_initialized; +#ifdef CONFIG_RT_GROUP_SCHED +#define rt_entity_is_task(rt_se) (!(rt_se)->my_q) + +static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) +{ + WARN_ON_ONCE(!rt_entity_is_task(rt_se)); + + return container_of(rt_se, struct task_struct, rt); +} + +static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) +{ + /* Cannot fold with non-CONFIG_RT_GROUP_SCHED version, layout */ + WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); + return rt_rq->rq; +} + +static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) +{ + WARN_ON(!rt_group_sched_enabled() && rt_se->rt_rq->tg !=3D &root_task_gro= up); + return rt_se->rt_rq; +} + +static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) +{ + struct rt_rq *rt_rq =3D rt_se->rt_rq; + + WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); + return rt_rq->rq; +} +#else +#define rt_entity_is_task(rt_se) (1) + +static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) +{ + return container_of(rt_se, struct task_struct, rt); +} + +static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) +{ + return container_of(rt_rq, struct rq, rt); +} + +static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) +{ + struct task_struct *p =3D rt_task_of(rt_se); + + return task_rq(p); +} + +static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) +{ + struct rq *rq =3D rq_of_rt_se(rt_se); + + return &rq->rt; +} +#endif + DEFINE_LOCK_GUARD_2(double_rq_lock, struct rq, double_rq_lock(_T->lock, _T->lock2), double_rq_unlock(_T->lock, _T->lock2)) -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CBA830E836 for ; Mon, 1 Dec 2025 12:42:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592937; cv=none; b=H5cETookAWyw7jJA3BFc3uyRdkW8UqIAdG1gNbRyTN7GmIvZECvWnxMW5cDkmZKJehQqi82nehhyFIgEtXckmRnnSWTam4Uonh6ig9uPvqjzTMjeHPQwZTLra4kM2Lx8uH6RQcEl8v0CqwgDyGSQ4MtYPNI9BJOALeom37it8JU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592937; c=relaxed/simple; bh=Be4A+C6ilCxeqjWYZxa0FGuu1YE9Z8RG5O0cQZ3KffY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=heYwzBZT9b48KBuXM9WxvkKYQkzpzvraRr34/ekWG8LQq2mldWBHjzuvwG3Kibf9Ihjl1T1wIEe14vvYpDqkXc+mGNZpdSq74MYmSYhOBAoaER5WzWRPTChLP9WoPrxJ4Y8rMEDYbsqkOlT+W5fz+VvDdmaadU0dhkba357zHws= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=krzjjxKM; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="krzjjxKM" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-b7380f66a8bso630043766b.2 for ; Mon, 01 Dec 2025 04:42:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592932; x=1765197732; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uHKvQQZiL4btHWgaovyVSuzcpH/4eJODl/d58zMXNDY=; b=krzjjxKMI9HCCy4v6Pk8ykgfVz1ozbICjDFvgm1f+nHrurpJO2Ly8/BE4kqqsIAWes ZBVNwyD0DFqBSMOh0amsOmo2w+ubWtBY9povVy36Yqo6YKWyzQ4OY0BwiUg6idJR+gyy 8CZ/wsZTQMUEFhTSNx5ZJRRxp2t3wuA16ObvpedOYNhraV8vFJQ+9mTske0jdfkBbaRT /w8HS7M1VlAibVSLHiy90qvt5maHjpkSBqmP+nrj5DSl+99TBK8BPTPGkpIQ1Gbq2qaX HPar8w24sYZ7cxHbZQS9ES6/S18bZHH3Or9dfRphY3rPCuyy0y5D/sAoasxGaLEcSIuv e13g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592932; x=1765197732; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=uHKvQQZiL4btHWgaovyVSuzcpH/4eJODl/d58zMXNDY=; b=btrifduyKkR8mAqwebMiFW6Igs0LX3CuDaTHRlsaP2ySA8CJ827MizPoZ0GMmW+pUj 4UzUdEvlSIBirKDi5hiXScVN6HDoL1HKy2XKoI9RZmVx5sRZBhsXjnOsocjCwdbH9SXc y6lOOU//fu4B1JSiETGol0QuYFJMf/dcevJn3mHUCFgPakvaPMDNna9n5UZ6+ezZkt0G KS6nSiiabTqWxYbqwYE5xVMwY/j86GzkkXtU9OpBFY2U5ItnjqJIpAxwYnU63Er7qett WnoNhGeRA6HsczzovZYFshlWrZSTn2bIfo1MZeSmDzxMKT6GDQ7TXHj12+GVpD1aycyb I22w== X-Gm-Message-State: AOJu0YypzFhjPry6QLJ57eDf+ynBMtdI2fHsR6yjgdTET2nbktiOjy/6 NJlLzLzYpiI3680b+u4vWfiPGxB7QX3Rge5RfHLp+Gwof40mYKajG9On X-Gm-Gg: ASbGncscLZKf+NNMSF6q2NsWgEDeSGgVmtemI98pLjLYVoJ0EU/NvjKirpbvd638Ox7 JReHPt3Oqp5BdE7jLMap0vnuDjbZLrly53ArTWmqpnUYW+GIJEB/i52HRKdPSVZqr1oIW7NYsXQ saitySnZ3DOz+X6uWBr6J4yJ3e0koU/MhqBMHCo3zHpXBBl7CilzoIW1qMd0KZ8QV3v4cIVL/PO 7bb7d0ZrOl2mCMp/gaJ+Y8YXQEQthw72sFnh1U3pl+o5kXd4iOn8/mfkf1GwhjkNT/2RKXYsudL t0iNbOUmGUSUiev7Arv/FTzq9vlpGNyH0iZgvrT00LaRzdKyG8JtXleoMEuWOChgHOLstb8HlWP Obt0PrU5uXZOudTqGjNpP1z9VqApwJdOKo2NHOvQqFnfJOe+qGmWm6G475/h8KnsqxzahM/FQnI AAJQwAh+A6 X-Google-Smtp-Source: AGHT+IEwTXs7SmjYzQgzciFwUKd5xuCDT+pxxOOMNkdvfFh7tLhdVhOwitQ5TXU1AFxBVElvoGsQQg== X-Received: by 2002:a17:907:3f9b:b0:b70:af93:b32d with SMTP id a640c23a62f3a-b7671a17160mr4347830466b.53.1764592931829; Mon, 01 Dec 2025 04:42:11 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:11 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 05/28] sched/rt: Disable RT_GROUP_SCHED Date: Mon, 1 Dec 2025 13:41:38 +0100 Message-ID: <20251201124205.11169-6-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Disable the old RT_GROUP_SCHED scheduler. Note that this does not completely remove all the RT_GROUP_SCHED functionality, just unhooks it and removes most of the relevant functions. Some of the RT_GROUP_SCHED functions are kept because they will be adapted for the HCBS scheduling. Most notably: - Disable the initialization of the rt_bandwidth for group scheduling. - Unhook any functionality for RT_GROUP_SCHED in normal rt.c code, leaving only non-group functionality. - Remove group related field initialization in init_rt_rq(). - Remove all the unhooked (and so unused) functions from RT_GROUP_SCHED. - Remove all allocation/deallocation code for rt-groups, always returning failure on allocation. - Update inc/dec_rt_tasks active tasks' counters, as rt scheduling entities now only represent a single task, and not a group of tasks anymore. Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 6 - kernel/sched/deadline.c | 34 -- kernel/sched/debug.c | 6 - kernel/sched/rt.c | 862 ++-------------------------------------- kernel/sched/sched.h | 15 +- kernel/sched/syscalls.c | 13 - 6 files changed, 29 insertions(+), 907 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index f754a60de8..ac5820b18c 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8691,11 +8691,6 @@ void __init sched_init(void) =20 init_defrootdomain(); =20 -#ifdef CONFIG_RT_GROUP_SCHED - init_rt_bandwidth(&root_task_group.rt_bandwidth, - global_rt_period(), global_rt_runtime()); -#endif /* CONFIG_RT_GROUP_SCHED */ - #ifdef CONFIG_CGROUP_SCHED task_group_cache =3D KMEM_CACHE(task_group, 0); =20 @@ -8747,7 +8742,6 @@ void __init sched_init(void) * starts working after scheduler_running, which is not the case * yet. */ - rq->rt.rt_runtime =3D global_rt_runtime(); init_tg_rt_entry(&root_task_group, &rq->rt, NULL, i, NULL); #endif rq->sd =3D NULL; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index e36d98eb5f..501121d58b 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1500,40 +1500,6 @@ static void update_curr_dl_se(struct rq *rq, struct = sched_dl_entity *dl_se, s64 if (!is_leftmost(dl_se, &rq->dl)) resched_curr(rq); } - - /* - * The fair server (sole dl_server) does not account for real-time - * workload because it is running fair work. - */ - if (dl_se =3D=3D &rq->fair_server) - return; - -#ifdef CONFIG_RT_GROUP_SCHED - /* - * Because -- for now -- we share the rt bandwidth, we need to - * account our runtime there too, otherwise actual rt tasks - * would be able to exceed the shared quota. - * - * Account to the root rt group for now. - * - * The solution we're working towards is having the RT groups scheduled - * using deadline servers -- however there's a few nasties to figure - * out before that can happen. - */ - if (rt_bandwidth_enabled()) { - struct rt_rq *rt_rq =3D &rq->rt; - - raw_spin_lock(&rt_rq->rt_runtime_lock); - /* - * We'll let actual RT tasks worry about the overflow here, we - * have our own CBS to keep us inline; only account when RT - * bandwidth is relevant. - */ - if (sched_rt_bandwidth_account(rt_rq)) - rt_rq->rt_time +=3D delta_exec; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - } -#endif /* CONFIG_RT_GROUP_SCHED */ } =20 /* diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 02e16b70a7..efcf8d82f8 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -890,12 +890,6 @@ void print_rt_rq(struct seq_file *m, int cpu, struct r= t_rq *rt_rq) =20 PU(rt_nr_running); =20 -#ifdef CONFIG_RT_GROUP_SCHED - P(rt_throttled); - PN(rt_time); - PN(rt_runtime); -#endif - #undef PN #undef PU #undef P diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index b6264e715c..74038b27f8 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -82,115 +82,18 @@ void init_rt_rq(struct rt_rq *rt_rq) rt_rq->highest_prio.next =3D MAX_RT_PRIO-1; rt_rq->overloaded =3D 0; plist_head_init(&rt_rq->pushable_tasks); - /* We start is dequeued state, because no RT tasks are queued */ - rt_rq->rt_queued =3D 0; - -#ifdef CONFIG_RT_GROUP_SCHED - rt_rq->rt_time =3D 0; - rt_rq->rt_throttled =3D 0; - rt_rq->rt_runtime =3D 0; - raw_spin_lock_init(&rt_rq->rt_runtime_lock); - rt_rq->tg =3D &root_task_group; -#endif } =20 #ifdef CONFIG_RT_GROUP_SCHED =20 -static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun= ); - -static enum hrtimer_restart sched_rt_period_timer(struct hrtimer *timer) -{ - struct rt_bandwidth *rt_b =3D - container_of(timer, struct rt_bandwidth, rt_period_timer); - int idle =3D 0; - int overrun; - - raw_spin_lock(&rt_b->rt_runtime_lock); - for (;;) { - overrun =3D hrtimer_forward_now(timer, rt_b->rt_period); - if (!overrun) - break; - - raw_spin_unlock(&rt_b->rt_runtime_lock); - idle =3D do_sched_rt_period_timer(rt_b, overrun); - raw_spin_lock(&rt_b->rt_runtime_lock); - } - if (idle) - rt_b->rt_period_active =3D 0; - raw_spin_unlock(&rt_b->rt_runtime_lock); - - return idle ? HRTIMER_NORESTART : HRTIMER_RESTART; -} - -void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime) -{ - rt_b->rt_period =3D ns_to_ktime(period); - rt_b->rt_runtime =3D runtime; - - raw_spin_lock_init(&rt_b->rt_runtime_lock); - - hrtimer_setup(&rt_b->rt_period_timer, sched_rt_period_timer, CLOCK_MONOTO= NIC, - HRTIMER_MODE_REL_HARD); -} - -static inline void do_start_rt_bandwidth(struct rt_bandwidth *rt_b) -{ - raw_spin_lock(&rt_b->rt_runtime_lock); - if (!rt_b->rt_period_active) { - rt_b->rt_period_active =3D 1; - /* - * SCHED_DEADLINE updates the bandwidth, as a run away - * RT task with a DL task could hog a CPU. But DL does - * not reset the period. If a deadline task was running - * without an RT task running, it can cause RT tasks to - * throttle when they start up. Kick the timer right away - * to update the period. - */ - hrtimer_forward_now(&rt_b->rt_period_timer, ns_to_ktime(0)); - hrtimer_start_expires(&rt_b->rt_period_timer, - HRTIMER_MODE_ABS_PINNED_HARD); - } - raw_spin_unlock(&rt_b->rt_runtime_lock); -} - -static void start_rt_bandwidth(struct rt_bandwidth *rt_b) -{ - if (!rt_bandwidth_enabled() || rt_b->rt_runtime =3D=3D RUNTIME_INF) - return; - - do_start_rt_bandwidth(rt_b); -} - -static void destroy_rt_bandwidth(struct rt_bandwidth *rt_b) -{ - hrtimer_cancel(&rt_b->rt_period_timer); -} - void unregister_rt_sched_group(struct task_group *tg) { - if (!rt_group_sched_enabled()) - return; - - if (tg->rt_se) - destroy_rt_bandwidth(&tg->rt_bandwidth); } =20 void free_rt_sched_group(struct task_group *tg) { - int i; - if (!rt_group_sched_enabled()) return; - - for_each_possible_cpu(i) { - if (tg->rt_rq) - kfree(tg->rt_rq[i]); - if (tg->rt_se) - kfree(tg->rt_se[i]); - } - - kfree(tg->rt_rq); - kfree(tg->rt_se); } =20 void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq, @@ -200,66 +103,19 @@ void init_tg_rt_entry(struct task_group *tg, struct r= t_rq *rt_rq, struct rq *rq =3D cpu_rq(cpu); =20 rt_rq->highest_prio.curr =3D MAX_RT_PRIO-1; - rt_rq->rt_nr_boosted =3D 0; rt_rq->rq =3D rq; rt_rq->tg =3D tg; =20 tg->rt_rq[cpu] =3D rt_rq; tg->rt_se[cpu] =3D rt_se; - - if (!rt_se) - return; - - if (!parent) - rt_se->rt_rq =3D &rq->rt; - else - rt_se->rt_rq =3D parent->my_q; - - rt_se->my_q =3D rt_rq; - rt_se->parent =3D parent; - INIT_LIST_HEAD(&rt_se->run_list); } =20 int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) { - struct rt_rq *rt_rq; - struct sched_rt_entity *rt_se; - int i; - if (!rt_group_sched_enabled()) return 1; =20 - tg->rt_rq =3D kcalloc(nr_cpu_ids, sizeof(rt_rq), GFP_KERNEL); - if (!tg->rt_rq) - goto err; - tg->rt_se =3D kcalloc(nr_cpu_ids, sizeof(rt_se), GFP_KERNEL); - if (!tg->rt_se) - goto err; - - init_rt_bandwidth(&tg->rt_bandwidth, ktime_to_ns(global_rt_period()), 0); - - for_each_possible_cpu(i) { - rt_rq =3D kzalloc_node(sizeof(struct rt_rq), - GFP_KERNEL, cpu_to_node(i)); - if (!rt_rq) - goto err; - - rt_se =3D kzalloc_node(sizeof(struct sched_rt_entity), - GFP_KERNEL, cpu_to_node(i)); - if (!rt_se) - goto err_free_rq; - - init_rt_rq(rt_rq); - rt_rq->rt_runtime =3D tg->rt_bandwidth.rt_runtime; - init_tg_rt_entry(tg, rt_rq, rt_se, i, parent->rt_se[i]); - } - return 1; - -err_free_rq: - kfree(rt_rq); -err: - return 0; } =20 #else /* !CONFIG_RT_GROUP_SCHED: */ @@ -377,9 +233,6 @@ static void dequeue_pushable_task(struct rt_rq *rt_rq, = struct task_struct *p) } } =20 -static void enqueue_top_rt_rq(struct rt_rq *rt_rq); -static void dequeue_top_rt_rq(struct rt_rq *rt_rq, unsigned int count); - static inline int on_rt_rq(struct sched_rt_entity *rt_se) { return rt_se->on_rq; @@ -426,16 +279,6 @@ static inline bool rt_task_fits_capacity(struct task_s= truct *p, int cpu) =20 #ifdef CONFIG_RT_GROUP_SCHED =20 -static inline u64 sched_rt_runtime(struct rt_rq *rt_rq) -{ - return rt_rq->rt_runtime; -} - -static inline u64 sched_rt_period(struct rt_rq *rt_rq) -{ - return ktime_to_ns(rt_rq->tg->rt_bandwidth.rt_period); -} - typedef struct task_group *rt_rq_iter_t; =20 static inline struct task_group *next_task_group(struct task_group *tg) @@ -461,457 +304,20 @@ static inline struct task_group *next_task_group(str= uct task_group *tg) iter && (rt_rq =3D iter->rt_rq[cpu_of(rq)]); \ iter =3D next_task_group(iter)) =20 -#define for_each_sched_rt_entity(rt_se) \ - for (; rt_se; rt_se =3D rt_se->parent) - -static inline struct rt_rq *group_rt_rq(struct sched_rt_entity *rt_se) -{ - return rt_se->my_q; -} - static void enqueue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags); static void dequeue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags); =20 -static void sched_rt_rq_enqueue(struct rt_rq *rt_rq) -{ - struct task_struct *donor =3D rq_of_rt_rq(rt_rq)->donor; - struct rq *rq =3D rq_of_rt_rq(rt_rq); - struct sched_rt_entity *rt_se; - - int cpu =3D cpu_of(rq); - - rt_se =3D rt_rq->tg->rt_se[cpu]; - - if (rt_rq->rt_nr_running) { - if (!rt_se) - enqueue_top_rt_rq(rt_rq); - else if (!on_rt_rq(rt_se)) - enqueue_rt_entity(rt_se, 0); - - if (rt_rq->highest_prio.curr < donor->prio) - resched_curr(rq); - } -} - -static void sched_rt_rq_dequeue(struct rt_rq *rt_rq) -{ - struct sched_rt_entity *rt_se; - int cpu =3D cpu_of(rq_of_rt_rq(rt_rq)); - - rt_se =3D rt_rq->tg->rt_se[cpu]; - - if (!rt_se) { - dequeue_top_rt_rq(rt_rq, rt_rq->rt_nr_running); - /* Kick cpufreq (see the comment in kernel/sched/sched.h). */ - cpufreq_update_util(rq_of_rt_rq(rt_rq), 0); - } - else if (on_rt_rq(rt_se)) - dequeue_rt_entity(rt_se, 0); -} - -static inline int rt_rq_throttled(struct rt_rq *rt_rq) -{ - return rt_rq->rt_throttled && !rt_rq->rt_nr_boosted; -} - -static int rt_se_boosted(struct sched_rt_entity *rt_se) -{ - struct rt_rq *rt_rq =3D group_rt_rq(rt_se); - struct task_struct *p; - - if (rt_rq) - return !!rt_rq->rt_nr_boosted; - - p =3D rt_task_of(rt_se); - return p->prio !=3D p->normal_prio; -} - -static inline const struct cpumask *sched_rt_period_mask(void) -{ - return this_rq()->rd->span; -} - -static inline -struct rt_rq *sched_rt_period_rt_rq(struct rt_bandwidth *rt_b, int cpu) -{ - return container_of(rt_b, struct task_group, rt_bandwidth)->rt_rq[cpu]; -} - -static inline struct rt_bandwidth *sched_rt_bandwidth(struct rt_rq *rt_rq) -{ - return &rt_rq->tg->rt_bandwidth; -} - -bool sched_rt_bandwidth_account(struct rt_rq *rt_rq) -{ - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - - return (hrtimer_active(&rt_b->rt_period_timer) || - rt_rq->rt_time < rt_b->rt_runtime); -} - -/* - * We ran out of runtime, see if we can borrow some from our neighbours. - */ -static void do_balance_runtime(struct rt_rq *rt_rq) -{ - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - struct root_domain *rd =3D rq_of_rt_rq(rt_rq)->rd; - int i, weight; - u64 rt_period; - - weight =3D cpumask_weight(rd->span); - - raw_spin_lock(&rt_b->rt_runtime_lock); - rt_period =3D ktime_to_ns(rt_b->rt_period); - for_each_cpu(i, rd->span) { - struct rt_rq *iter =3D sched_rt_period_rt_rq(rt_b, i); - s64 diff; - - if (iter =3D=3D rt_rq) - continue; - - raw_spin_lock(&iter->rt_runtime_lock); - /* - * Either all rqs have inf runtime and there's nothing to steal - * or __disable_runtime() below sets a specific rq to inf to - * indicate its been disabled and disallow stealing. - */ - if (iter->rt_runtime =3D=3D RUNTIME_INF) - goto next; - - /* - * From runqueues with spare time, take 1/n part of their - * spare time, but no more than our period. - */ - diff =3D iter->rt_runtime - iter->rt_time; - if (diff > 0) { - diff =3D div_u64((u64)diff, weight); - if (rt_rq->rt_runtime + diff > rt_period) - diff =3D rt_period - rt_rq->rt_runtime; - iter->rt_runtime -=3D diff; - rt_rq->rt_runtime +=3D diff; - if (rt_rq->rt_runtime =3D=3D rt_period) { - raw_spin_unlock(&iter->rt_runtime_lock); - break; - } - } -next: - raw_spin_unlock(&iter->rt_runtime_lock); - } - raw_spin_unlock(&rt_b->rt_runtime_lock); -} - -/* - * Ensure this RQ takes back all the runtime it lend to its neighbours. - */ -static void __disable_runtime(struct rq *rq) -{ - struct root_domain *rd =3D rq->rd; - rt_rq_iter_t iter; - struct rt_rq *rt_rq; - - if (unlikely(!scheduler_running)) - return; - - for_each_rt_rq(rt_rq, iter, rq) { - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - s64 want; - int i; - - raw_spin_lock(&rt_b->rt_runtime_lock); - raw_spin_lock(&rt_rq->rt_runtime_lock); - /* - * Either we're all inf and nobody needs to borrow, or we're - * already disabled and thus have nothing to do, or we have - * exactly the right amount of runtime to take out. - */ - if (rt_rq->rt_runtime =3D=3D RUNTIME_INF || - rt_rq->rt_runtime =3D=3D rt_b->rt_runtime) - goto balanced; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - - /* - * Calculate the difference between what we started out with - * and what we current have, that's the amount of runtime - * we lend and now have to reclaim. - */ - want =3D rt_b->rt_runtime - rt_rq->rt_runtime; - - /* - * Greedy reclaim, take back as much as we can. - */ - for_each_cpu(i, rd->span) { - struct rt_rq *iter =3D sched_rt_period_rt_rq(rt_b, i); - s64 diff; - - /* - * Can't reclaim from ourselves or disabled runqueues. - */ - if (iter =3D=3D rt_rq || iter->rt_runtime =3D=3D RUNTIME_INF) - continue; - - raw_spin_lock(&iter->rt_runtime_lock); - if (want > 0) { - diff =3D min_t(s64, iter->rt_runtime, want); - iter->rt_runtime -=3D diff; - want -=3D diff; - } else { - iter->rt_runtime -=3D want; - want -=3D want; - } - raw_spin_unlock(&iter->rt_runtime_lock); - - if (!want) - break; - } - - raw_spin_lock(&rt_rq->rt_runtime_lock); - /* - * We cannot be left wanting - that would mean some runtime - * leaked out of the system. - */ - WARN_ON_ONCE(want); -balanced: - /* - * Disable all the borrow logic by pretending we have inf - * runtime - in which case borrowing doesn't make sense. - */ - rt_rq->rt_runtime =3D RUNTIME_INF; - rt_rq->rt_throttled =3D 0; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - raw_spin_unlock(&rt_b->rt_runtime_lock); - - /* Make rt_rq available for pick_next_task() */ - sched_rt_rq_enqueue(rt_rq); - } -} - -static void __enable_runtime(struct rq *rq) -{ - rt_rq_iter_t iter; - struct rt_rq *rt_rq; - - if (unlikely(!scheduler_running)) - return; - - /* - * Reset each runqueue's bandwidth settings - */ - for_each_rt_rq(rt_rq, iter, rq) { - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - - raw_spin_lock(&rt_b->rt_runtime_lock); - raw_spin_lock(&rt_rq->rt_runtime_lock); - rt_rq->rt_runtime =3D rt_b->rt_runtime; - rt_rq->rt_time =3D 0; - rt_rq->rt_throttled =3D 0; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - raw_spin_unlock(&rt_b->rt_runtime_lock); - } -} - -static void balance_runtime(struct rt_rq *rt_rq) -{ - if (!sched_feat(RT_RUNTIME_SHARE)) - return; - - if (rt_rq->rt_time > rt_rq->rt_runtime) { - raw_spin_unlock(&rt_rq->rt_runtime_lock); - do_balance_runtime(rt_rq); - raw_spin_lock(&rt_rq->rt_runtime_lock); - } -} - -static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun) -{ - int i, idle =3D 1, throttled =3D 0; - const struct cpumask *span; - - span =3D sched_rt_period_mask(); - - /* - * FIXME: isolated CPUs should really leave the root task group, - * whether they are isolcpus or were isolated via cpusets, lest - * the timer run on a CPU which does not service all runqueues, - * potentially leaving other CPUs indefinitely throttled. If - * isolation is really required, the user will turn the throttle - * off to kill the perturbations it causes anyway. Meanwhile, - * this maintains functionality for boot and/or troubleshooting. - */ - if (rt_b =3D=3D &root_task_group.rt_bandwidth) - span =3D cpu_online_mask; - - for_each_cpu(i, span) { - int enqueue =3D 0; - struct rt_rq *rt_rq =3D sched_rt_period_rt_rq(rt_b, i); - struct rq *rq =3D rq_of_rt_rq(rt_rq); - struct rq_flags rf; - int skip; - - /* - * When span =3D=3D cpu_online_mask, taking each rq->lock - * can be time-consuming. Try to avoid it when possible. - */ - raw_spin_lock(&rt_rq->rt_runtime_lock); - if (!sched_feat(RT_RUNTIME_SHARE) && rt_rq->rt_runtime !=3D RUNTIME_INF) - rt_rq->rt_runtime =3D rt_b->rt_runtime; - skip =3D !rt_rq->rt_time && !rt_rq->rt_nr_running; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - if (skip) - continue; - - rq_lock(rq, &rf); - update_rq_clock(rq); - - if (rt_rq->rt_time) { - u64 runtime; - - raw_spin_lock(&rt_rq->rt_runtime_lock); - if (rt_rq->rt_throttled) - balance_runtime(rt_rq); - runtime =3D rt_rq->rt_runtime; - rt_rq->rt_time -=3D min(rt_rq->rt_time, overrun*runtime); - if (rt_rq->rt_throttled && rt_rq->rt_time < runtime) { - rt_rq->rt_throttled =3D 0; - enqueue =3D 1; - - /* - * When we're idle and a woken (rt) task is - * throttled wakeup_preempt() will set - * skip_update and the time between the wakeup - * and this unthrottle will get accounted as - * 'runtime'. - */ - if (rt_rq->rt_nr_running && rq->curr =3D=3D rq->idle) - rq_clock_cancel_skipupdate(rq); - } - if (rt_rq->rt_time || rt_rq->rt_nr_running) - idle =3D 0; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - } else if (rt_rq->rt_nr_running) { - idle =3D 0; - if (!rt_rq_throttled(rt_rq)) - enqueue =3D 1; - } - if (rt_rq->rt_throttled) - throttled =3D 1; - - if (enqueue) - sched_rt_rq_enqueue(rt_rq); - rq_unlock(rq, &rf); - } - - if (!throttled && (!rt_bandwidth_enabled() || rt_b->rt_runtime =3D=3D RUN= TIME_INF)) - return 1; - - return idle; -} - -static int sched_rt_runtime_exceeded(struct rt_rq *rt_rq) -{ - u64 runtime =3D sched_rt_runtime(rt_rq); - - if (rt_rq->rt_throttled) - return rt_rq_throttled(rt_rq); - - if (runtime >=3D sched_rt_period(rt_rq)) - return 0; - - balance_runtime(rt_rq); - runtime =3D sched_rt_runtime(rt_rq); - if (runtime =3D=3D RUNTIME_INF) - return 0; - - if (rt_rq->rt_time > runtime) { - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - - /* - * Don't actually throttle groups that have no runtime assigned - * but accrue some time due to boosting. - */ - if (likely(rt_b->rt_runtime)) { - rt_rq->rt_throttled =3D 1; - printk_deferred_once("sched: RT throttling activated\n"); - } else { - /* - * In case we did anyway, make it go away, - * replenishment is a joke, since it will replenish us - * with exactly 0 ns. - */ - rt_rq->rt_time =3D 0; - } - - if (rt_rq_throttled(rt_rq)) { - sched_rt_rq_dequeue(rt_rq); - return 1; - } - } - - return 0; -} - -#else /* !CONFIG_RT_GROUP_SCHED: */ +#else /* !CONFIG_RT_GROUP_SCHED */ =20 typedef struct rt_rq *rt_rq_iter_t; =20 #define for_each_rt_rq(rt_rq, iter, rq) \ for ((void) iter, rt_rq =3D &rq->rt; rt_rq; rt_rq =3D NULL) =20 -#define for_each_sched_rt_entity(rt_se) \ - for (; rt_se; rt_se =3D NULL) - -static inline struct rt_rq *group_rt_rq(struct sched_rt_entity *rt_se) -{ - return NULL; -} - -static inline void sched_rt_rq_enqueue(struct rt_rq *rt_rq) -{ - struct rq *rq =3D rq_of_rt_rq(rt_rq); - - if (!rt_rq->rt_nr_running) - return; - - enqueue_top_rt_rq(rt_rq); - resched_curr(rq); -} - -static inline void sched_rt_rq_dequeue(struct rt_rq *rt_rq) -{ - dequeue_top_rt_rq(rt_rq, rt_rq->rt_nr_running); -} - -static inline int rt_rq_throttled(struct rt_rq *rt_rq) -{ - return false; -} - -static inline const struct cpumask *sched_rt_period_mask(void) -{ - return cpu_online_mask; -} - -static inline -struct rt_rq *sched_rt_period_rt_rq(struct rt_bandwidth *rt_b, int cpu) -{ - return &cpu_rq(cpu)->rt; -} - -static void __enable_runtime(struct rq *rq) { } -static void __disable_runtime(struct rq *rq) { } - -#endif /* !CONFIG_RT_GROUP_SCHED */ +#endif /* CONFIG_RT_GROUP_SCHED */ =20 static inline int rt_se_prio(struct sched_rt_entity *rt_se) { -#ifdef CONFIG_RT_GROUP_SCHED - struct rt_rq *rt_rq =3D group_rt_rq(rt_se); - - if (rt_rq) - return rt_rq->highest_prio.curr; -#endif - return rt_task_of(rt_se)->prio; } =20 @@ -931,67 +337,8 @@ static void update_curr_rt(struct rq *rq) if (unlikely(delta_exec <=3D 0)) return; =20 -#ifdef CONFIG_RT_GROUP_SCHED - struct sched_rt_entity *rt_se =3D &donor->rt; - if (!rt_bandwidth_enabled()) return; - - for_each_sched_rt_entity(rt_se) { - struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); - int exceeded; - - if (sched_rt_runtime(rt_rq) !=3D RUNTIME_INF) { - raw_spin_lock(&rt_rq->rt_runtime_lock); - rt_rq->rt_time +=3D delta_exec; - exceeded =3D sched_rt_runtime_exceeded(rt_rq); - if (exceeded) - resched_curr(rq); - raw_spin_unlock(&rt_rq->rt_runtime_lock); - if (exceeded) - do_start_rt_bandwidth(sched_rt_bandwidth(rt_rq)); - } - } -#endif /* CONFIG_RT_GROUP_SCHED */ -} - -static void -dequeue_top_rt_rq(struct rt_rq *rt_rq, unsigned int count) -{ - struct rq *rq =3D rq_of_rt_rq(rt_rq); - - BUG_ON(&rq->rt !=3D rt_rq); - - if (!rt_rq->rt_queued) - return; - - BUG_ON(!rq->nr_running); - - sub_nr_running(rq, count); - rt_rq->rt_queued =3D 0; - -} - -static void -enqueue_top_rt_rq(struct rt_rq *rt_rq) -{ - struct rq *rq =3D rq_of_rt_rq(rt_rq); - - BUG_ON(&rq->rt !=3D rt_rq); - - if (rt_rq->rt_queued) - return; - - if (rt_rq_throttled(rt_rq)) - return; - - if (rt_rq->rt_nr_running) { - add_nr_running(rq, rt_rq->rt_nr_running); - rt_rq->rt_queued =3D 1; - } - - /* Kick cpufreq (see the comment in kernel/sched/sched.h). */ - cpufreq_update_util(rq, 0); } =20 static void @@ -1062,58 +409,11 @@ dec_rt_prio(struct rt_rq *rt_rq, int prio) dec_rt_prio_smp(rt_rq, prio, prev_prio); } =20 -#ifdef CONFIG_RT_GROUP_SCHED - -static void -inc_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) -{ - if (rt_se_boosted(rt_se)) - rt_rq->rt_nr_boosted++; - - start_rt_bandwidth(&rt_rq->tg->rt_bandwidth); -} - -static void -dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) -{ - if (rt_se_boosted(rt_se)) - rt_rq->rt_nr_boosted--; - - WARN_ON(!rt_rq->rt_nr_running && rt_rq->rt_nr_boosted); -} - -#else /* !CONFIG_RT_GROUP_SCHED: */ - -static void -inc_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) -{ -} - -static inline -void dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) {} - -#endif /* !CONFIG_RT_GROUP_SCHED */ - -static inline -unsigned int rt_se_nr_running(struct sched_rt_entity *rt_se) -{ - struct rt_rq *group_rq =3D group_rt_rq(rt_se); - - if (group_rq) - return group_rq->rt_nr_running; - else - return 1; -} - static inline -unsigned int rt_se_rr_nr_running(struct sched_rt_entity *rt_se) +unsigned int is_rr_task(struct sched_rt_entity *rt_se) { - struct rt_rq *group_rq =3D group_rt_rq(rt_se); struct task_struct *tsk; =20 - if (group_rq) - return group_rq->rr_nr_running; - tsk =3D rt_task_of(rt_se); =20 return (tsk->policy =3D=3D SCHED_RR) ? 1 : 0; @@ -1122,26 +422,21 @@ unsigned int rt_se_rr_nr_running(struct sched_rt_ent= ity *rt_se) static inline void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) { - int prio =3D rt_se_prio(rt_se); - - WARN_ON(!rt_prio(prio)); - rt_rq->rt_nr_running +=3D rt_se_nr_running(rt_se); - rt_rq->rr_nr_running +=3D rt_se_rr_nr_running(rt_se); + WARN_ON(!rt_prio(rt_se_prio(rt_se))); + rt_rq->rt_nr_running +=3D 1; + rt_rq->rr_nr_running +=3D is_rr_task(rt_se); =20 - inc_rt_prio(rt_rq, prio); - inc_rt_group(rt_se, rt_rq); + inc_rt_prio(rt_rq, rt_se_prio(rt_se)); } =20 static inline void dec_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) { WARN_ON(!rt_prio(rt_se_prio(rt_se))); - WARN_ON(!rt_rq->rt_nr_running); - rt_rq->rt_nr_running -=3D rt_se_nr_running(rt_se); - rt_rq->rr_nr_running -=3D rt_se_rr_nr_running(rt_se); + rt_rq->rt_nr_running -=3D 1; + rt_rq->rr_nr_running -=3D is_rr_task(rt_se); =20 dec_rt_prio(rt_rq, rt_se_prio(rt_se)); - dec_rt_group(rt_se, rt_rq); } =20 /* @@ -1170,10 +465,6 @@ static void __delist_rt_entity(struct sched_rt_entity= *rt_se, struct rt_prio_arr static inline struct sched_statistics * __schedstats_from_rt_se(struct sched_rt_entity *rt_se) { - /* schedstats is not supported for rt group. */ - if (!rt_entity_is_task(rt_se)) - return NULL; - return &rt_task_of(rt_se)->stats; } =20 @@ -1186,9 +477,7 @@ update_stats_wait_start_rt(struct rt_rq *rt_rq, struct= sched_rt_entity *rt_se) if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); stats =3D __schedstats_from_rt_se(rt_se); if (!stats) return; @@ -1205,9 +494,7 @@ update_stats_enqueue_sleeper_rt(struct rt_rq *rt_rq, s= truct sched_rt_entity *rt_ if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); stats =3D __schedstats_from_rt_se(rt_se); if (!stats) return; @@ -1235,9 +522,7 @@ update_stats_wait_end_rt(struct rt_rq *rt_rq, struct s= ched_rt_entity *rt_se) if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); stats =3D __schedstats_from_rt_se(rt_se); if (!stats) return; @@ -1254,9 +539,7 @@ update_stats_dequeue_rt(struct rt_rq *rt_rq, struct sc= hed_rt_entity *rt_se, if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); if ((flags & DEQUEUE_SLEEP) && p) { unsigned int state; =20 @@ -1275,21 +558,8 @@ static void __enqueue_rt_entity(struct sched_rt_entit= y *rt_se, unsigned int flag { struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); struct rt_prio_array *array =3D &rt_rq->active; - struct rt_rq *group_rq =3D group_rt_rq(rt_se); struct list_head *queue =3D array->queue + rt_se_prio(rt_se); =20 - /* - * Don't enqueue the group if its throttled, or when empty. - * The latter is a consequence of the former when a child group - * get throttled and the current group doesn't have any other - * active members. - */ - if (group_rq && (rt_rq_throttled(group_rq) || !group_rq->rt_nr_running)) { - if (rt_se->on_list) - __delist_rt_entity(rt_se, array); - return; - } - if (move_entity(flags)) { WARN_ON_ONCE(rt_se->on_list); if (flags & ENQUEUE_HEAD) @@ -1319,57 +589,18 @@ static void __dequeue_rt_entity(struct sched_rt_enti= ty *rt_se, unsigned int flag dec_rt_tasks(rt_se, rt_rq); } =20 -/* - * Because the prio of an upper entry depends on the lower - * entries, we must remove entries top - down. - */ -static void dequeue_rt_stack(struct sched_rt_entity *rt_se, unsigned int f= lags) -{ - struct sched_rt_entity *back =3D NULL; - unsigned int rt_nr_running; - - for_each_sched_rt_entity(rt_se) { - rt_se->back =3D back; - back =3D rt_se; - } - - rt_nr_running =3D rt_rq_of_se(back)->rt_nr_running; - - for (rt_se =3D back; rt_se; rt_se =3D rt_se->back) { - if (on_rt_rq(rt_se)) - __dequeue_rt_entity(rt_se, flags); - } - - dequeue_top_rt_rq(rt_rq_of_se(back), rt_nr_running); -} - static void enqueue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags) { - struct rq *rq =3D rq_of_rt_se(rt_se); - update_stats_enqueue_rt(rt_rq_of_se(rt_se), rt_se, flags); =20 - dequeue_rt_stack(rt_se, flags); - for_each_sched_rt_entity(rt_se) - __enqueue_rt_entity(rt_se, flags); - enqueue_top_rt_rq(&rq->rt); + __enqueue_rt_entity(rt_se, flags); } =20 static void dequeue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags) { - struct rq *rq =3D rq_of_rt_se(rt_se); - update_stats_dequeue_rt(rt_rq_of_se(rt_se), rt_se, flags); =20 - dequeue_rt_stack(rt_se, flags); - - for_each_sched_rt_entity(rt_se) { - struct rt_rq *rt_rq =3D group_rt_rq(rt_se); - - if (rt_rq && rt_rq->rt_nr_running) - __enqueue_rt_entity(rt_se, flags); - } - enqueue_top_rt_rq(&rq->rt); + __dequeue_rt_entity(rt_se, flags); } =20 /* @@ -1432,10 +663,8 @@ static void requeue_task_rt(struct rq *rq, struct tas= k_struct *p, int head) struct sched_rt_entity *rt_se =3D &p->rt; struct rt_rq *rt_rq; =20 - for_each_sched_rt_entity(rt_se) { - rt_rq =3D rt_rq_of_se(rt_se); - requeue_rt_entity(rt_rq, rt_se, head); - } + rt_rq =3D rt_rq_of_se(rt_se); + requeue_rt_entity(rt_rq, rt_se, head); } =20 static void yield_task_rt(struct rq *rq) @@ -1630,21 +859,6 @@ static struct sched_rt_entity *pick_next_rt_entity(st= ruct rt_rq *rt_rq) return next; } =20 -static struct task_struct *_pick_next_task_rt(struct rq *rq) -{ - struct sched_rt_entity *rt_se; - struct rt_rq *rt_rq =3D &rq->rt; - - do { - rt_se =3D pick_next_rt_entity(rt_rq); - if (unlikely(!rt_se)) - return NULL; - rt_rq =3D group_rt_rq(rt_se); - } while (rt_rq); - - return rt_task_of(rt_se); -} - static struct task_struct *pick_task_rt(struct rq *rq) { struct task_struct *p; @@ -1652,7 +866,7 @@ static struct task_struct *pick_task_rt(struct rq *rq) if (!sched_rt_runnable(rq)) return NULL; =20 - p =3D _pick_next_task_rt(rq); + p =3D rt_task_of(pick_next_rt_entity(&rq->rt)); =20 return p; } @@ -2311,8 +1525,6 @@ static void rq_online_rt(struct rq *rq) if (rq->rt.overloaded) rt_set_overload(rq); =20 - __enable_runtime(rq); - cpupri_set(&rq->rd->cpupri, rq->cpu, rq->rt.highest_prio.curr); } =20 @@ -2322,8 +1534,6 @@ static void rq_offline_rt(struct rq *rq) if (rq->rt.overloaded) rt_clear_overload(rq); =20 - __disable_runtime(rq); - cpupri_set(&rq->rd->cpupri, rq->cpu, CPUPRI_INVALID); } =20 @@ -2481,12 +1691,12 @@ static void task_tick_rt(struct rq *rq, struct task= _struct *p, int queued) * Requeue to the end of queue if we (and all of our ancestors) are not * the only element on the queue */ - for_each_sched_rt_entity(rt_se) { - if (rt_se->run_list.prev !=3D rt_se->run_list.next) { - requeue_task_rt(rq, p, 0); - resched_curr(rq); - return; - } + if (rt_se->run_list.prev !=3D rt_se->run_list.next) { + requeue_task_rt(rq, p, 0); + resched_curr(rq); + // set_tsk_need_resched(p); + + return; } } =20 @@ -2504,16 +1714,7 @@ static unsigned int get_rr_interval_rt(struct rq *rq= , struct task_struct *task) #ifdef CONFIG_SCHED_CORE static int task_is_throttled_rt(struct task_struct *p, int cpu) { - struct rt_rq *rt_rq; - -#ifdef CONFIG_RT_GROUP_SCHED // XXX maybe add task_rt_rq(), see also sched= _rt_period_rt_rq - rt_rq =3D task_group(p)->rt_rq[cpu]; - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); -#else - rt_rq =3D &cpu_rq(cpu)->rt; -#endif - - return rt_rq_throttled(rt_rq); + return 0; } #endif /* CONFIG_SCHED_CORE */ =20 @@ -2761,13 +1962,7 @@ long sched_group_rt_period(struct task_group *tg) #ifdef CONFIG_SYSCTL static int sched_rt_global_constraints(void) { - int ret =3D 0; - - mutex_lock(&rt_constraints_mutex); - ret =3D __rt_schedulable(NULL, 0, 0); - mutex_unlock(&rt_constraints_mutex); - - return ret; + return 0; } #endif /* CONFIG_SYSCTL */ =20 @@ -2802,10 +1997,6 @@ static int sched_rt_global_validate(void) return 0; } =20 -static void sched_rt_do_global(void) -{ -} - static int sched_rt_handler(const struct ctl_table *table, int write, void= *buffer, size_t *lenp, loff_t *ppos) { @@ -2833,7 +2024,6 @@ static int sched_rt_handler(const struct ctl_table *t= able, int write, void *buff if (ret) goto undo; =20 - sched_rt_do_global(); sched_dl_do_global(); } if (0) { diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 964d596d8e..5c48d6a5e6 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -815,7 +815,7 @@ struct scx_rq { =20 static inline int rt_bandwidth_enabled(void) { - return sysctl_sched_rt_runtime >=3D 0; + return 0; } =20 /* RT IPI pull logic requires IRQ_WORK */ @@ -855,7 +855,7 @@ struct rt_rq { =20 static inline bool rt_rq_is_runnable(struct rt_rq *rt_rq) { - return rt_rq->rt_queued && rt_rq->rt_nr_running; + return rt_rq->rt_nr_running; } =20 /* Deadline class' related fields in a runqueue */ @@ -2572,7 +2572,7 @@ static inline bool sched_dl_runnable(struct rq *rq) =20 static inline bool sched_rt_runnable(struct rq *rq) { - return rq->rt.rt_queued > 0; + return rq->rt.rt_nr_running > 0; } =20 static inline bool sched_fair_runnable(struct rq *rq) @@ -2683,9 +2683,6 @@ extern void resched_curr(struct rq *rq); extern void resched_curr_lazy(struct rq *rq); extern void resched_cpu(int cpu); =20 -extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 r= untime); -extern bool sched_rt_bandwidth_account(struct rt_rq *rt_rq); - extern void init_dl_entity(struct sched_dl_entity *dl_se); =20 extern void init_cfs_throttle_work(struct task_struct *p); @@ -3072,12 +3069,8 @@ extern void set_rq_offline(struct rq *rq); extern bool sched_smp_initialized; =20 #ifdef CONFIG_RT_GROUP_SCHED -#define rt_entity_is_task(rt_se) (!(rt_se)->my_q) - static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) { - WARN_ON_ONCE(!rt_entity_is_task(rt_se)); - return container_of(rt_se, struct task_struct, rt); } =20 @@ -3102,8 +3095,6 @@ static inline struct rq *rq_of_rt_se(struct sched_rt_= entity *rt_se) return rt_rq->rq; } #else -#define rt_entity_is_task(rt_se) (1) - static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) { return container_of(rt_se, struct task_struct, rt); diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index 77ae87f36e..93a9c03b28 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -626,19 +626,6 @@ int __sched_setscheduler(struct task_struct *p, change: =20 if (user) { -#ifdef CONFIG_RT_GROUP_SCHED - /* - * Do not allow real-time tasks into groups that have no runtime - * assigned. - */ - if (rt_group_sched_enabled() && - rt_bandwidth_enabled() && rt_policy(policy) && - task_group(p)->rt_bandwidth.rt_runtime =3D=3D 0 && - !task_group_is_autogroup(task_group(p))) { - retval =3D -EPERM; - goto unlock; - } -#endif /* CONFIG_RT_GROUP_SCHED */ if (dl_bandwidth_enabled() && dl_policy(policy) && !(attr->sched_flags & SCHED_FLAG_SUGOV)) { cpumask_t *span =3D rq->rd->span; --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBDA630AAA6 for ; Mon, 1 Dec 2025 12:42:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592936; cv=none; b=b78eKdRrfF5UMKhRiQ1Z88ccYSuVGehuGPhDcrVdqb5cHyByGUrGSw6B13IBw+4xpn8CES0pOxLj1V7Oq15fIq7w7FxYSi76Wh69SvzMZ0IHBQPQP+zNRYyrRNwdpuvjLYdVivuzBVWZaMZSKYVD7Eye+A/1FP1q6Bd9LloUxWU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592936; c=relaxed/simple; bh=VUhS8rm+rja3StXdNa9mR6SKz5XBV85CgB2D1j/ju7Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NXbS30R79l2gDS2h44GijK+3GXNL4rglv1SIQe3/bK58yyds7nq/wAVTNpM2DaU8dPqhj72G8vR44ZVlkrhTer5Dmxc3uD7UmpSntuN5TdwRJyIT7Z3UchPAaKEmdz/31njWfSGJjR8Q5kRVgm2lZk+7iESoVCwpy7pW7aIaYJs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RhGOxGiL; arc=none smtp.client-ip=209.85.208.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RhGOxGiL" Received: by mail-ed1-f42.google.com with SMTP id 4fb4d7f45d1cf-64149f78c0dso6267343a12.3 for ; Mon, 01 Dec 2025 04:42:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592933; x=1765197733; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=f/jmZMFqfo9o66d41r23lJRa7nTeBjZM9UfI4eBF/48=; b=RhGOxGiLhXFdK4PBkEW2k73fWxetRtIKIcAkkV42OgC2msSwC9cX+5IsL7WLFTRmnT qlpYbuYkKMLyG+ntaQmZsbLECf5Y/bhv2bdn9Sq+BwHhVtZJgc3FLpY/1Q4NYK1Td/so D0TT6d1dKhf5SfZ6zZtXFM/EGMAziyesoMo38cUU+MKkIRUMQAX1To2r8ps2O4jj+POo 07Xz8sVRxy9lEahzaAQXZ1tpZ3XfmwmSV9Ee4bvhr3VobajPZVkabZR9Xc/aqZYixFtY EYqd60o/s80ue64IEN3kzPMiTnuahB4z8LqWVOwbZx8ZsYJ+bvK6HXTTRnN0w3WN04pH RbnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592933; x=1765197733; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=f/jmZMFqfo9o66d41r23lJRa7nTeBjZM9UfI4eBF/48=; b=WLUzI5spAS/rSFFwu22s3mgiueHEwe6Qnn0JemCb4ThZNEkoKvN5VwyFOnRQbnBstZ 58xPVM4z2p18B4AA0TIfW+6Q5ZVu7kaapQwyq2Byf+qvF5h0kSnb5N85uIVBjA11uPkX i/cD6yRLJkzChPWNCaIu6vOVnJ0Nxv8AHhYVFpxez2DQwzSk0figIu6wofGr2P8/jPYt BY1Q1oZfsDT6TxWr6nzpmU7xNPoe07QoBK99Soyr5lnummSTr8klT6iYA+oZTpgXf61Y 5DgIOhXX0bvEpuhbATz9/fVfuPRlltxIYjhsTKArAK7cWOHxQgZudiclXirF7d28vo0i l/eg== X-Gm-Message-State: AOJu0YwV9Nn+TdjD2VvuM1PqgGYy9G7MNTNTRhXmQXLNQo8Mz1tqaGP7 CX/xWoDSLVQpjJZTSVEGscoC3loLzmiFXJW7WZkUQhA4IgR81LCkgzim X-Gm-Gg: ASbGncu0iQ1X1wVOd5iTEgdHMHaGvPR3UwhCnVh1H1nObWUxCtos4xKEepyM+zZA3Jo dPh5Rk2BAdrV0igpDC59WvgJE+nIkrcobJhedNoagYpRJP23EGuM0MTtE2hV04vaK3UkNICXiGy uoWQoZN8+BE07qmbpvJG66RY+iKGikzWTUY/JDktV0oVvbMXkmfiAfMxwjAo+TFTiPWmhXR+0BK Ukc9/iEH59TdnHcY52AdiHHkmi2Cve8VP6LBBh6dmUUlwhTo163bTrKX+li9M9AecxKxu2B9T38 twxPiTdTCKzxiywmfF64Vpl9RJjzmtBsuSxS8IGyOd65eXjxaj3tGGody48F02jlZ2WYGBtNe3t Ndn2AmnB5ERB6wlbCkXexgU55ojcppSGUYGbKfdoycQsPKFNSpxllrjUF9SCAwszjvyedoKb98S r6hBCn+vRt X-Google-Smtp-Source: AGHT+IG1jSgRDCuqUjxGZ7n7RiNdge5AVicNN3jNxZBeY5iEsyKJHGlJ4W6A4TwJcoerBVxD770tLA== X-Received: by 2002:a17:907:3f0a:b0:b73:780d:2bcf with SMTP id a640c23a62f3a-b76715ab995mr4431341566b.16.1764592932879; Mon, 01 Dec 2025 04:42:12 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:12 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 06/28] sched/rt: Remove rq field in struct rt_rq Date: Mon, 1 Dec 2025 13:41:39 +0100 Message-ID: <20251201124205.11169-7-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The rq field now is just caching the pointer to the global runqueue of the given rt_rq, so it is unnecessary as the global runqueue can be retrieved in other ways. Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 7 ++----- kernel/sched/sched.h | 19 +++++++++++++------ 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 74038b27f8..21f4e94235 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -100,10 +100,7 @@ void init_tg_rt_entry(struct task_group *tg, struct rt= _rq *rt_rq, struct sched_rt_entity *rt_se, int cpu, struct sched_rt_entity *parent) { - struct rq *rq =3D cpu_rq(cpu); - rt_rq->highest_prio.curr =3D MAX_RT_PRIO-1; - rt_rq->rq =3D rq; rt_rq->tg =3D tg; =20 tg->rt_rq[cpu] =3D rt_rq; @@ -183,7 +180,7 @@ static void pull_rt_task(struct rq *); =20 static inline void rt_queue_push_tasks(struct rt_rq *rt_rq) { - struct rq *rq =3D container_of(rt_rq, struct rq, rt); + struct rq *rq =3D served_rq_of_rt_rq(rt_rq); =20 if (!has_pushable_tasks(rt_rq)) return; @@ -193,7 +190,7 @@ static inline void rt_queue_push_tasks(struct rt_rq *rt= _rq) =20 static inline void rt_queue_pull_task(struct rt_rq *rt_rq) { - struct rq *rq =3D container_of(rt_rq, struct rq, rt); + struct rq *rq =3D served_rq_of_rt_rq(rt_rq); =20 queue_balance_callback(rq, &per_cpu(rt_pull_head, rq->cpu), pull_rt_task); } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 5c48d6a5e6..1896c4e247 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3074,11 +3074,16 @@ static inline struct task_struct *rt_task_of(struct= sched_rt_entity *rt_se) return container_of(rt_se, struct task_struct, rt); } =20 +static inline struct rq *served_rq_of_rt_rq(struct rt_rq *rt_rq) +{ + WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); + return container_of(rt_rq, struct rq, rt); +} + static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) { /* Cannot fold with non-CONFIG_RT_GROUP_SCHED version, layout */ - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); - return rt_rq->rq; + return cpu_rq(served_rq_of_rt_rq(rt_rq)->cpu); } =20 static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) @@ -3089,10 +3094,7 @@ static inline struct rt_rq *rt_rq_of_se(struct sched= _rt_entity *rt_se) =20 static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) { - struct rt_rq *rt_rq =3D rt_se->rt_rq; - - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); - return rt_rq->rq; + return rq_of_rt_rq(rt_se->rt_rq); } #else static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) @@ -3100,6 +3102,11 @@ static inline struct task_struct *rt_task_of(struct = sched_rt_entity *rt_se) return container_of(rt_se, struct task_struct, rt); } =20 +static inline struct rq *served_rq_of_rt_rq(struct rt_rq *rt_rq) +{ + return container_of(rt_rq, struct rq, rt); +} + static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) { return container_of(rt_rq, struct rq, rt); --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7694F30F535 for ; Mon, 1 Dec 2025 12:42:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592937; cv=none; b=b67iFE5hTckyQOS/IzUk6a7emvlAiirDaRumvdPSvRiUnzCO/HTzqa4YhjiAEKfrMTc4BknRQ9x5RKMmk/EgFwsLTXQgoFGZdnd4Tx3hICtE7NsP5EZNmonxdHET/VnC8b5hCF4hnO/iqReQyZcr6r5Nu3coRrflNawAe+iGErI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592937; c=relaxed/simple; bh=hLLyRP9tIpkrqh2yyZW48M1b4Ru0WaHejoUu70RERXo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sbdx8HY6iusZCWqREajOuvWYx+SL4q+io4eveiWLwO8q5DWWcm2+PuCqB2eQN6dCQcRHGvGEGRAxhI9C7hcZGzS+3e2Fy0k/dXgQ2tYJzBKjugsN5JmfcE7QK4Sl+o3NjwaWvXY4JwRzFLgtJgDmJ/2nTKzX5Jt0qUEK+nTFjdQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KEy+CjBO; arc=none smtp.client-ip=209.85.218.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KEy+CjBO" Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-b7636c96b9aso566743466b.2 for ; Mon, 01 Dec 2025 04:42:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592934; x=1765197734; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GaKgYA75OwYmMB1zX5+FGUb+nUV3NZxKAt8kgtn0VYk=; b=KEy+CjBOVpR9rMLfgoccgn5jB+XrJQdXUQOi/pmYV6sd4Lr+y+edxtJeMHx+1I8Kh4 ae8Og+hf9G4qP61jRjvkA740XEpcUzr+cyjW7g2+YNNI+0sc//oCaVNQF5veReO1q3A2 rpH5KdpvgnJlXlz0y6q7Qiku2oqWgGwSYDnnzkBOiQ8vprUghLP8buz/aWDeymOvRhl0 3cGhIAy8eXaqDubcw/9cBfcXc9YsLBM6wnNABy7fObzimQOZbyTE9nQeqdTdTjjuueh6 W/cCb4D3ULz74Z9O2OxlWdwWZwzdvyYAq4h+1SYBq4pnWlWQD3IDWfF4Zy9EAwQOk81F 9avQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592934; x=1765197734; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=GaKgYA75OwYmMB1zX5+FGUb+nUV3NZxKAt8kgtn0VYk=; b=nvmx6r+gRz1asRdFZBM4nImc84AyHq5jSnjZfGR4h2q8RMsCioizYKudtk+9iCVtuj wBIvPfsSj72EcubS0ysN/sqFkyUXchrs9KJzZRJwfFm1+W19qqRWyfE3isCENibjBaU3 8hnOC5rmMlvoEjUdzUkzh4WpG1zs43Qq3LtzXwp/3XcW545tBrTYgrvBGZvBd8KmoNAT xRGcTiBIZfqo3tvAWKJ9tT1x7qtUe6dBVzYtZjEC5xl9GEWDclnfXkeAKsoDvnSTchG2 B4Cl3bubfkreDGMp/GWkzL9pWnEr9RZ06qUAvmbNAGEROs5TosmTtI6aVOtcICIS1kTf SEiw== X-Gm-Message-State: AOJu0YyV7Vtad5zAV3C6mlfRTfoiKIgkbGlLxbi2dp3zJoWQV6d0aRjI Z0hSVK+fxGRkyYXXpLCVw9s/qz8XAPRb80gLfqz6WejGMoLvAMxZ19OD X-Gm-Gg: ASbGnctUtAK8ib4TYX0TxmYUJNv1/kbF3DqfK+JpWY0HKLhDcfmPv7Bmu/ImjZcYVCp gGa7cGMQc7Fc8kdo+xYfaHBSgsHDqAaMuE1L4AiTNQiibvDmdaaUmS1za/8ETEuL38+Cdgyjb3F wubRMRRBeHEmWzJk5IXlKWvTZQoYMkxtJZXY/5EbFHj3KMjg0fEuBRDVQYlUuMHx21BhuCwuDQR BiPWtceuDqS/tbBErolqGfdJ8xCYHEavg1G73Qs+mLPjKPYWIHYCtBfoA+QJAXaxQ9sT7JXl/w3 PIXTr6m80vJ7igUKHBNQAyvR5PG5YwJlTnPJ/zLcqfoEnbUJTcmhIXKRW+C+CyaVvrK2N6d0aoL ShcloGv+QJdIwlFsTQSe50aYj1tN+GJAm5LlyYUDkYragB/zk1D27YHbowDy1xVjDKnBX/GbSyV T6EAG17BXx3jocsppM/iM= X-Google-Smtp-Source: AGHT+IG1ZGDl/bPCRWjzFrD8xRhb6EcOsOngpgcxa2bWBz0vm/gOuvAM4sEsYJ7SL+JrNgDiAkWeMg== X-Received: by 2002:a17:907:9611:b0:b73:8b79:a322 with SMTP id a640c23a62f3a-b7671549d5amr4259954166b.11.1764592933784; Mon, 01 Dec 2025 04:42:13 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:13 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 07/28] sched/rt: Introduce HCBS specific structs in task_group Date: Mon, 1 Dec 2025 13:41:40 +0100 Message-ID: <20251201124205.11169-8-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Each task_group manages a number of new objects: - a sched_dl_entity/dl_server for each CPU - a dl_bandwidth object to keep track of its allocated dl_bandwidth Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/sched.h | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 1896c4e247..75a3dc475b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -319,6 +319,13 @@ struct rt_bandwidth { unsigned int rt_period_active; }; +struct dl_bandwidth { + raw_spinlock_t dl_runtime_lock; + u64 dl_runtime; + u64 dl_period; +}; + + static inline int dl_bandwidth_enabled(void) { return sysctl_sched_rt_runtime >=3D 0; @@ -492,10 +499,17 @@ struct task_group { #endif /* CONFIG_FAIR_GROUP_SCHED */ #ifdef CONFIG_RT_GROUP_SCHED + /* + * Each task group manages a different scheduling entity per CPU, i.e. a + * different deadline server, and a runqueue per CPU. All the dl-servers + * share the same dl_bandwidth object. + */ struct sched_rt_entity **rt_se; + struct sched_dl_entity **dl_se; struct rt_rq **rt_rq; struct rt_bandwidth rt_bandwidth; + struct dl_bandwidth dl_bandwidth; #endif struct scx_task_group scx; @@ -845,12 +859,16 @@ struct rt_rq { raw_spinlock_t rt_runtime_lock; unsigned int rt_nr_boosted; - - struct rq *rq; /* this is always top-level rq, cache? */ #endif #ifdef CONFIG_CGROUP_SCHED struct task_group *tg; /* this tg has "this" rt_rq on given CPU for runna= ble entities */ #endif + + /* + * The cgroup's served runqueue if the rt_rq entity belongs to a cgroup, + * otherwise the top-level global runqueue. + */ + struct rq *rq; }; static inline bool rt_rq_is_runnable(struct rt_rq *rt_rq) -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5FED30F924 for ; Mon, 1 Dec 2025 12:42:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592939; cv=none; b=GZp51n5RaqURSfwKvm/hveOGXyI1vRGg0VtEahwwdr/ILsetw1K5f8p5dHJooaHhI1GAgtloUDFlk6g5TstcYs5WQLdxqaPw9ntpAcah2VKjOwt6iWempv5jJHpVfCLN2g9UxxFw+0dIZS/92XLsto14Ig7hfQW1WtGZScROxZQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592939; c=relaxed/simple; bh=LWVL0V1CMyaANTn5Z44oxog+SfIvK0iqhIrhT3E77MA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cLOTCvI8mBrfbPl34gpJIcbyc/3JrZKewWVyZS+ssTa79xQusu6TBcmj6cObKZX09Ji4SomvI1n0r7+3zGeAUzDEG6zvCWMT39AJwuvIkpFSPpvZoPhN/jLITatbagUl1pTddqVbXqDGPYlzCzgX8XYk+3hjbOm/6EVgdmYlwRE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=VWzybl9b; arc=none smtp.client-ip=209.85.218.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VWzybl9b" Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-b7636c96b9aso566747366b.2 for ; Mon, 01 Dec 2025 04:42:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592935; x=1765197735; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=04AxSFOwxQMQiMuntsx/5ZGU1FwJkihpErkM8vIVFio=; b=VWzybl9bQ+GfVjQKmQ38G3MPNW/8L0JV5Vb7AkGsEIHxK47u+xyHMi5/HqO2CqXRiv TpeXgOgwJelgA1y9buTzDofi6GHIfaWPCkVcrxoF24QCxnID2yJbQKlAKOTnrniEZyac zmCp3J39kieFEtGXahioT7PpF2y/QAfwHFKgszlZYhmnSpuMdZm+AwNnd1QJ5t/t02uO lgIrT83un5CsDTDiJden19EmNIglpXQHJ2WSEQWggPAuqcAvUOF7boQ4JAjiklbK9nAe 5+DkIi8MDNTSkeMFaNA0Ud+0J/WhfLpDxr5P8sQ1eusqbb/yotorTYaEyPWckVRXfesj Fu5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592935; x=1765197735; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=04AxSFOwxQMQiMuntsx/5ZGU1FwJkihpErkM8vIVFio=; b=m/UoyoLsuexT3S6O8T+q0ordEgldHowy61a3zt2NPc86FvHYYegOaxceERgAAH6O23 W4W9DLbXY3n3yqv59AEaSk2cc9VGigvdxyOBjGWscSV7cGCbxKFGH0z3cvN/zmqLQkfs Xb1RYJ6bqh/8wysNwTzoRTechvdy7jzlGul3KLq6DL0baYnXXQvpkW7MeYvfLJQM/35q /hU/p9fIR/agRmOa/FAh22RC+8d5r96rNgVJexi76/TUvSXX4G5JCeViEmriEYF5c4VS zKNgDtR3LsVNXsyYUE5qP7MZUXikjWLnMjSNVFe7vDWx0ODkZnoglCuQ1DTtbv2Oi4Iy cDUg== X-Gm-Message-State: AOJu0YxFh3qNYKW4ViGQCTZf+LbeNt77LbnqOK/Ta3jfFPYa0b1tkkfv m1pjc+5rMFkC0lnCl66/MS9GMa2iqfUEg8/qMNbbjuVwNhk7H+yxsEQg X-Gm-Gg: ASbGnctuNjSrmw/SBo7fxGcmbeCtWCX4T6Fvm70k+CX99XAcRU+XtP40Gm60E2WthoP EynTAt+Q6IA4FoUl9nVg9ovIKWCvo/eoqkA15dvS2vuAOY7Ssw3fC15lVVnTttTcoaojRCxxdxW +mlc+DbKH7r77FCYlRmcOaq+Yh6K8/7Xm6VTizh03/leCsKAF7u8RV9dkySt2tLcnZGI51Pfu2u 0dj3SNRZJwPY87DFYdgO/o+RBxll1JjBWBYV2LepOVloS4Az1csJbjo/ZYec9Kv4DSWPWxgoctN wdmaHOKbEhjzKo2S7wJozILAOFeP0dH+uewPa3Oo4Amp7t3uRqB9Yo7elP1xU9vF3H4AS5gF1le Ho6h5rieve4Q5pVA3HXz+oNuYBCaiopxyZSjJ6FHqFF3iVMXUQ5fr4CP++9r0wj/v9xiNR0A+Mr Du2Qgd7tAC X-Google-Smtp-Source: AGHT+IFofmYDQIzDHpmspVKfKEjJ8aYV/NPOl+VwSiIdmApwpti0dXqINXJ7JpshQYzNKuaUaO/byQ== X-Received: by 2002:a17:907:9628:b0:b4a:e11a:195b with SMTP id a640c23a62f3a-b767184bc82mr3844451666b.44.1764592934803; Mon, 01 Dec 2025 04:42:14 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:14 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 08/28] sched/core: Initialize HCBS specific structures. Date: Mon, 1 Dec 2025 13:41:41 +0100 Message-ID: <20251201124205.11169-9-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni - Update autogroups' creation/destruction to use the new data structures. - Initialize the default bandwidth for rt-cgroups (sched_init). - Initialize rt-scheduler's specific data structures for the root control group (sched_init). - Change signature of init_tg_rt_entry to accept a rq pointer for the local runqueue and a dl_se pointer for the global runqueue / dl_server entity, update the function to initialize the newly added HCBS fields. - Add utility functions to check (and get) if a rt_rq entity is connected to a rt-cgroup. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/autogroup.c | 4 ++-- kernel/sched/core.c | 11 +++++++++-- kernel/sched/deadline.c | 8 ++++++++ kernel/sched/rt.c | 11 ----------- kernel/sched/sched.h | 30 +++++++++++++++++++++++++++--- 5 files changed, 46 insertions(+), 18 deletions(-) diff --git a/kernel/sched/autogroup.c b/kernel/sched/autogroup.c index cdea931aae..017eadc0a0 100644 --- a/kernel/sched/autogroup.c +++ b/kernel/sched/autogroup.c @@ -52,7 +52,7 @@ static inline void autogroup_destroy(struct kref *kref) =20 #ifdef CONFIG_RT_GROUP_SCHED /* We've redirected RT tasks to the root task group... */ - ag->tg->rt_se =3D NULL; + ag->tg->dl_se =3D NULL; ag->tg->rt_rq =3D NULL; #endif sched_release_group(ag->tg); @@ -109,7 +109,7 @@ static inline struct autogroup *autogroup_create(void) * the policy change to proceed. */ free_rt_sched_group(tg); - tg->rt_se =3D root_task_group.rt_se; + tg->dl_se =3D root_task_group.dl_se; tg->rt_rq =3D root_task_group.rt_rq; #endif /* CONFIG_RT_GROUP_SCHED */ tg->autogroup =3D ag; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index ac5820b18c..495cbdfdc5 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8680,7 +8680,7 @@ void __init sched_init(void) scx_tg_init(&root_task_group); #endif /* CONFIG_EXT_GROUP_SCHED */ #ifdef CONFIG_RT_GROUP_SCHED - root_task_group.rt_se =3D (struct sched_rt_entity **)ptr; + root_task_group.dl_se =3D (struct sched_dl_entity **)ptr; ptr +=3D nr_cpu_ids * sizeof(void **); =20 root_task_group.rt_rq =3D (struct rt_rq **)ptr; @@ -8691,6 +8691,11 @@ void __init sched_init(void) =20 init_defrootdomain(); =20 +#ifdef CONFIG_RT_GROUP_SCHED + init_dl_bandwidth(&root_task_group.dl_bandwidth, + global_rt_period(), global_rt_runtime()); +#endif /* CONFIG_RT_GROUP_SCHED */ + #ifdef CONFIG_CGROUP_SCHED task_group_cache =3D KMEM_CACHE(task_group, 0); =20 @@ -8742,7 +8747,9 @@ void __init sched_init(void) * starts working after scheduler_running, which is not the case * yet. */ - init_tg_rt_entry(&root_task_group, &rq->rt, NULL, i, NULL); + rq->rt.tg =3D &root_task_group; + root_task_group.rt_rq[i] =3D &rq->rt; + root_task_group.dl_se[i] =3D NULL; #endif rq->sd =3D NULL; rq->rd =3D NULL; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 501121d58b..3046148c94 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -513,6 +513,14 @@ static inline int is_leftmost(struct sched_dl_entity *= dl_se, struct dl_rq *dl_rq =20 static void init_dl_rq_bw_ratio(struct dl_rq *dl_rq); =20 +void init_dl_bandwidth(struct dl_bandwidth *dl_b, u64 period, u64 runtime) +{ + raw_spin_lock_init(&dl_b->dl_runtime_lock); + dl_b->dl_period =3D period; + dl_b->dl_runtime =3D runtime; +} + + void init_dl_bw(struct dl_bw *dl_b) { raw_spin_lock_init(&dl_b->lock); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 21f4e94235..6a51232de6 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -96,17 +96,6 @@ void free_rt_sched_group(struct task_group *tg) return; } =20 -void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq, - struct sched_rt_entity *rt_se, int cpu, - struct sched_rt_entity *parent) -{ - rt_rq->highest_prio.curr =3D MAX_RT_PRIO-1; - rt_rq->tg =3D tg; - - tg->rt_rq[cpu] =3D rt_rq; - tg->rt_se[cpu] =3D rt_se; -} - int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) { if (!rt_group_sched_enabled()) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 75a3dc475b..6f6e39a628 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -601,9 +601,6 @@ extern void start_cfs_bandwidth(struct cfs_bandwidth *c= fs_b); extern void unthrottle_cfs_rq(struct cfs_rq *cfs_rq); extern bool cfs_task_bw_constrained(struct task_struct *p); =20 -extern void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq, - struct sched_rt_entity *rt_se, int cpu, - struct sched_rt_entity *parent); extern int sched_group_set_rt_runtime(struct task_group *tg, long rt_runti= me_us); extern int sched_group_set_rt_period(struct task_group *tg, u64 rt_period_= us); extern long sched_group_rt_runtime(struct task_group *tg); @@ -2701,6 +2698,7 @@ extern void resched_curr(struct rq *rq); extern void resched_curr_lazy(struct rq *rq); extern void resched_cpu(int cpu); =20 +void init_dl_bandwidth(struct dl_bandwidth *dl_b, u64 period, u64 runtime); extern void init_dl_entity(struct sched_dl_entity *dl_se); =20 extern void init_cfs_throttle_work(struct task_struct *p); @@ -3114,6 +3112,22 @@ static inline struct rq *rq_of_rt_se(struct sched_rt= _entity *rt_se) { return rq_of_rt_rq(rt_se->rt_rq); } + +static inline int is_dl_group(struct rt_rq *rt_rq) +{ + return rt_rq->tg !=3D &root_task_group; +} + +/* + * Return the scheduling entity of this group of tasks. + */ +static inline struct sched_dl_entity *dl_group_of(struct rt_rq *rt_rq) +{ + if (WARN_ON_ONCE(!is_dl_group(rt_rq))) + return NULL; + + return rt_rq->tg->dl_se[served_rq_of_rt_rq(rt_rq)->cpu]; +} #else static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) { @@ -3143,6 +3157,16 @@ static inline struct rt_rq *rt_rq_of_se(struct sched= _rt_entity *rt_se) =20 return &rq->rt; } + +static inline int is_dl_group(struct rt_rq *rt_rq) +{ + return 0; +} + +static inline struct sched_dl_entity *dl_group_of(struct rt_rq *rt_rq) +{ + return NULL; +} #endif =20 DEFINE_LOCK_GUARD_2(double_rq_lock, struct rq, --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE1C930F953 for ; Mon, 1 Dec 2025 12:42:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592939; cv=none; b=S5j49u+YuToWitD+j+jLFtQ7O9u8WtlyadzJ+9IC47xqYLksSvV8VmcpkMO4UPKCHpo7uudOJ1Ah9dy8IG74PTD2NF+y/ya20W82VqcTmPgZcLhjCTNPWLZIc3NA+f1miwVc3cA4nnpjp1kmHiebiL/xB61bIaRKeEXlxELP61w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592939; c=relaxed/simple; bh=Vixvn7Q4a1AVo94e3HWWNzPHgX2j24uJpRCie+/se3c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=vDedfYAPuqsPRC+efMkWcdqTvU4BPdukflWU8STb38JBl8VQV16g8IpB/dofebeaVeh08Ia+fgjVdgkDXjq07sYoWwJNyJHdYId7k/w6QNzxfI7Y5uVPwUNXOQPBJ9JtCKgqX5S+1UtgRtZuiJUKjMU4JbRIOCXp0L/Kpotyw+g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mxJnJlh/; arc=none smtp.client-ip=209.85.208.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mxJnJlh/" Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-64080ccf749so6124437a12.2 for ; Mon, 01 Dec 2025 04:42:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592936; x=1765197736; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xtxY05+0ptwGNb1eFLw3jgalAvFnWd4waUi1SCenMgo=; b=mxJnJlh/T1GtVTvepKtqdksfm7UJ6Niv0iZJmFtxgFpyi0mDKkigI23D/7FQsLrD9m a80gMlWmG0RPRT1XuOAND6rA3UDuZMdBbLUV95MBEca/OYa+PAIWsr49eNyeJvyKHaNX zF4W7R6/qMl6BterUS11LYwU7JOJKmWz0CbQ04AHFALfaJCCxw/vDkyawrUIheIWL75Y 4cNlj4kWPKWQiNZhC4N3Ke+XapbSBij8i6Qcipy9nF8X1utTrg/Kxzte3SjAE/Rx1KNF Wc3EvH3a9aC1N+lPLXb35twpAL+a4k5ogw5mWVCksDhaCiyao92sHMdwkBx0mVH253jT hzSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592936; x=1765197736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=xtxY05+0ptwGNb1eFLw3jgalAvFnWd4waUi1SCenMgo=; b=MTK/k688M+eNnHlANzwwVuuLKkJUddQEjX60J+zOC28ZXbBGpRcF8GE6EUQSWrUJlQ fIpz/DHTNE1oTaWiYMOAScJTqZ71riEqzZmwPKMvoyKqxW2pI7EbHp0Yw/hqwxeUJf54 lIWeNaDkpQgl9dK4mmBzkrMS9CeuWIs2asjUq/3QDwie2CbtAsahFbFppLbBPzDE+j61 jEQXD8QHTwE1pk5YygS/UrtbuKzVOjGoygAyoFJ1epQeTO6uszvVftYLtWgm0xqglI+O BwaZnv8A4C/zrxUXM7Uptbad5THvd384voKnpUNuMOITV6XEF7iOvcvMYj5Ni8T9N2PX 8k7w== X-Gm-Message-State: AOJu0YyvbxVCgoAXQMEGuYhWxgoC9xGy3Wxzh+H+nuWXrKou7JVvpEUM 056ePri58JhRlzvPb3O09xCI4KK408YV/FwHDZZM+Ho8SkHxlHtVOSj5 X-Gm-Gg: ASbGncuugtnhyrvv9iMCoAfQaYfHNHs7ZAPbru3aZEViQCUSBOq8e6BokWA1Cbo5War zhMkpZOJ67sRycZ7bv/q0mFyvC4ZL4O9bCYWpznlcm6s8t7J7IDLHPGm808tDSvF8lCO4vGkbDk K9Ov5+nMU1Kms0ZrrTmPEnXSntA4+9NLPxiQt/cyysLOhGae+alrCEVSnlM6iCOdGiMaghwTazN hVR8bbakPvIgfKkUCVj5CNff9VSiRD95Ha7wmQwTAOzCEywy7tDZ6yS29bJhfxOoPVfQzGf6cxl 4JPWvncUOVEBgap81MkiB3UjkN9CvAv+JUpbeVBh5k0Nptyqk1YjIoUhUBP6NzJHNd+X6qHk41x aKqhfYkwVWJld65BGOBCBuYoResaFINT3nhkDQZ/xx14cyfF+jBoAhc33lvR4vF8LNP+cFOzkyW sH1nqCQLFE3co1zuiYPQs= X-Google-Smtp-Source: AGHT+IFOPSLX19T+ZVI0LQjXvLWDsUpwT0AWj00+J6Hp+nOaVxObfZBu7K90KZ0iHXWID3XczCXztQ== X-Received: by 2002:a17:907:3f9c:b0:b6d:5bc3:e158 with SMTP id a640c23a62f3a-b76715abd36mr4002165266b.17.1764592935731; Mon, 01 Dec 2025 04:42:15 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:15 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 09/28] sched/deadline: Add dl_init_tg Date: Mon, 1 Dec 2025 13:41:42 +0100 Message-ID: <20251201124205.11169-10-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add dl_init_tg to initialize and/or update a rt-cgroup dl_server and to also account the allocated bandwidth. This function is currently unhooked and will be later used to allocate bandwidth to rt-cgroups. Add lock guard for raw_spin_rq_lock_irq for cleaner code. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/deadline.c | 31 +++++++++++++++++++++++++++++++ kernel/sched/sched.h | 5 +++++ 2 files changed, 36 insertions(+) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 3046148c94..4f43883a65 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -340,6 +340,37 @@ void cancel_inactive_timer(struct sched_dl_entity *dl_= se) cancel_dl_timer(dl_se, &dl_se->inactive_timer); } =20 +#ifdef CONFIG_RT_GROUP_SCHED +void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 rt_peri= od) +{ + struct rq *rq =3D container_of(dl_se->dl_rq, struct rq, dl); + int is_active; + u64 new_bw; + + guard(raw_spin_rq_lock_irq)(rq); + is_active =3D dl_se->my_q->rt.rt_nr_running > 0; + + update_rq_clock(rq); + dl_server_stop(dl_se); + + new_bw =3D to_ratio(rt_period, rt_runtime); + dl_rq_change_utilization(rq, dl_se, new_bw); + + dl_se->dl_runtime =3D rt_runtime; + dl_se->dl_deadline =3D rt_period; + dl_se->dl_period =3D rt_period; + + dl_se->runtime =3D 0; + dl_se->deadline =3D 0; + + dl_se->dl_bw =3D new_bw; + dl_se->dl_density =3D new_bw; + + if (is_active) + dl_server_start(dl_se); +} +#endif + static void dl_change_utilization(struct task_struct *p, u64 new_bw) { WARN_ON_ONCE(p->dl.flags & SCHED_FLAG_SUGOV); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 6f6e39a628..f42bef06a9 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -419,6 +419,7 @@ extern void dl_server_init(struct sched_dl_entity *dl_s= e, struct dl_rq *dl_rq, struct rq *served_rq, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); +extern void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 = rt_period); =20 extern void dl_server_update_idle_time(struct rq *rq, struct task_struct *p); @@ -1922,6 +1923,10 @@ DEFINE_LOCK_GUARD_1(rq_lock_irqsave, struct rq, rq_unlock_irqrestore(_T->lock, &_T->rf), struct rq_flags rf) =20 +DEFINE_LOCK_GUARD_1(raw_spin_rq_lock_irq, struct rq, + raw_spin_rq_lock_irq(_T->lock), + raw_spin_rq_unlock_irq(_T->lock)) + static inline struct rq *this_rq_lock_irq(struct rq_flags *rf) __acquires(rq->lock) { --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D4F130FC2D for ; Mon, 1 Dec 2025 12:42:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592941; cv=none; b=pf6Jbtad7F6wNKNU/aAO0iomjv1g1IL0aG5B29HWsrRwmVFJsK4J9GG66twnRzdaysspqQ8RfMer7zX1BzXUkCgAJZ4uotwQ/Z639Gs/+PXjAqJDvGDYo2UiX4sBOsl1fGtBd+4LzAwp6NDJXmjvFk3KH9ZrNEIoGg4ZFUjFEIw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592941; c=relaxed/simple; bh=1mcgOHYRO2xHgEprO4OO6ROTvUomqwHwdE7/L7UpgqA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=g+tcgxD8PapLDxhCqXmJopmdw/53bpVFfo/ylSQ3a2Fq/XiJ0dO2xR/Tbpe4zOS4g6IqrL9ZM8CMK/b+unx369IomARzzeI3Co9D/b3Q4+A1HpDXXcKqKtf4B042N+K5RxqWhPmfQQ26CrydhDBXU8+wZnJbJNsw6insXEWOR6s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=c93dizpW; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="c93dizpW" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b735b89501fso510454366b.0 for ; Mon, 01 Dec 2025 04:42:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592937; x=1765197737; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uj6RNmz5NRTJ9FELpWwEPrdAIia3oMPoBC/Wu0k17QA=; b=c93dizpWhR9vfsnb5tjUvV8/xeAKlHPd2tBYXEEsBMtKapzoX50srnJRhTs6o6glSd EPwKdqQQIuaG6aHKTqvUEBRgzYEda0Us4o7rgHt5s7cMxmSaYrWdQejWaay4Mr+Rmfo9 0J59YYKReK70IsuTSjP7ajGPGPzU1M9KGX+rs0IR5zTf3VmfHtAa8fBXBfFuN6phIHu8 qoU5tpRXPjPcX+bCz1+l4WLqzU1J2UX3WH1DaXjLjEXaAo/BdNVgLBceoltMugMuzZ8L SCvKNXAWUORPU9GmEgLKo5Xsxl2NAQew2WdMeqh1IWxakos1/t1jMHMqO0CwRNxnTAs0 tTuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592937; x=1765197737; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=uj6RNmz5NRTJ9FELpWwEPrdAIia3oMPoBC/Wu0k17QA=; b=Vma4ACAwyDTwdiHdD4/Z3G2hwnq0dfrGMdxB8czexCUt7VgSrkhDjw32YDOg0QfoVN R4/wAzKhyOZXJkuXBhnLmJiMwvU0TAj7i2SxtVUheBala+piYAGrCHz+EqALRz+Sz4wf ZLCUmPZ08JSipxIZUOYaksLd7g9GVTCLlBYY4IBEkRbwU1iFj4CJkGASG3qDwxldhgRU ulq1GEDWHPSgZw4SKAyUKWLtKvEq2hg92zHOJDwOX6twPawNMZnxPvD2ZIa2zJFt/ioO w48MR2UPXNkbSev9gz6QMDwQH/JeofI9llxgbRXHeXJhk1yjrGpQH6SsE0lx/hgkth3O YvLA== X-Gm-Message-State: AOJu0YzI1XkxJLm0Z9yrIeSoy1GaAuJeWqDsPHj5IJO0XL2UycY8Mp4t FDhLGsz83iY2M7CnjuIvZ5BzOJDgoIyGYCqxDpH2Xc2BZ04H++bF0sdx X-Gm-Gg: ASbGncuyt3CY1yE+uMIsAxRBCsXv4P3cbkXapdChCxr9WBaGkFB/ISPDOE6GLVzjAjt cCRYoda74cEc4+ETZCD96Au8wXw/jBaqz/A3HptIkQNNpscoikx0AMRa8cnedb3zVfYQnBFYr9H hnK0w+LavNsQ2JrUL6fK7081HHivMmeF7d42FmEocBUF8N7S0NaEsrfIOD6l8P1NNhKCtrVsIna TjdY6t90pR4LQ+JLdsJd+hfSOe+4q3p1tp7Jb123vIN/oCClbpHGBxRLuQ5UKMtnhGMZawit+6f UUZ2qNTLJDr2O2NUv9NuPxKTTUSJOtlUeK9gRNSLNCDX0PG8JpiON/OKm5tyChm2seUEaz39yrw hJxihtDBsu2n2v6zNJdsZfLU7dNooPc7E8VmZ6LtTmYSaj7AphTYip8EBr1hfO3YxLFxieCvFCr ZtUiv0he3h X-Google-Smtp-Source: AGHT+IGxVGwC/sKYUwqRKG8lXLIscGWazq91V21BiqeQVaA3olRdaG4t9Av2Vc0WTgyCeiUBd+OaoA== X-Received: by 2002:a17:907:9690:b0:b76:8074:344b with SMTP id a640c23a62f3a-b76c5355c41mr2315128366b.8.1764592936640; Mon, 01 Dec 2025 04:42:16 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:16 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 10/28] sched/rt: Add {alloc/free}_rt_sched_group Date: Mon, 1 Dec 2025 13:41:43 +0100 Message-ID: <20251201124205.11169-11-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni - Add allocation and deallocation code for rt-cgroups. - Declare dl_server specific functions (only skeleton, but no implementation yet), needed by the deadline servers to be called when trying to schedule. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/rt.c | 125 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 123 insertions(+), 2 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 6a51232de6..2301efc03f 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -92,19 +92,140 @@ void unregister_rt_sched_group(struct task_group *tg) =20 void free_rt_sched_group(struct task_group *tg) { + int i; + unsigned long flags; + if (!rt_group_sched_enabled()) return; + + if (!tg->dl_se || !tg->rt_rq) + return; + + for_each_possible_cpu(i) { + if (!tg->dl_se[i] || !tg->rt_rq[i]) + continue; + + /* + * Shutdown the dl_server and free it + * + * Since the dl timer is going to be cancelled, + * we risk to never decrease the running bw... + * Fix this issue by changing the group runtime + * to 0 immediately before freeing it. + */ + dl_init_tg(tg->dl_se[i], 0, tg->dl_se[i]->dl_period); + + raw_spin_rq_lock_irqsave(cpu_rq(i), flags); + hrtimer_cancel(&tg->dl_se[i]->dl_timer); + raw_spin_rq_unlock_irqrestore(cpu_rq(i), flags); + kfree(tg->dl_se[i]); + + /* Free the local per-cpu runqueue */ + kfree(served_rq_of_rt_rq(tg->rt_rq[i])); + } + + kfree(tg->rt_rq); + kfree(tg->dl_se); +} + +static struct task_struct *rt_server_pick(struct sched_dl_entity *dl_se) +{ + return NULL; +} + +static inline void __rt_rq_free(struct rt_rq **rt_rq) +{ + int i; + + for_each_possible_cpu(i) { + kfree(served_rq_of_rt_rq(rt_rq[i])); + } + + kfree(rt_rq); +} + +DEFINE_FREE(rt_rq_free, struct rt_rq **, if (_T) __rt_rq_free(_T)) + +static inline void __dl_se_free(struct sched_dl_entity **dl_se) +{ + int i; + + for_each_possible_cpu(i) { + kfree(dl_se[i]); + } + + kfree(dl_se); } =20 +DEFINE_FREE(dl_se_free, struct sched_dl_entity **, if (_T) __dl_se_free(_T= )) + int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) { + /* Instantiate automatic cleanup in event of kalloc fail */ + struct rt_rq **tg_rt_rq __free(rt_rq_free) =3D NULL; + struct sched_dl_entity *dl_se, **tg_dl_se __free(dl_se_free) =3D NULL; + struct rq *s_rq; + int i; + if (!rt_group_sched_enabled()) return 1; =20 + /* Allocate all necessary resources beforehand */ + tg_rt_rq =3D kcalloc(nr_cpu_ids, sizeof(struct rt_rq *), GFP_KERNEL); + if (!tg_rt_rq) + return 0; + + tg_dl_se =3D kcalloc(nr_cpu_ids, + sizeof(struct sched_dl_entity *), GFP_KERNEL); + if (!tg_dl_se) + return 0; + + for_each_possible_cpu(i) { + s_rq =3D kzalloc_node(sizeof(struct rq), + GFP_KERNEL, cpu_to_node(i)); + if (!s_rq) + return 0; + + dl_se =3D kzalloc_node(sizeof(struct sched_dl_entity), + GFP_KERNEL, cpu_to_node(i)); + if (!dl_se) { + kfree(s_rq); + return 0; + } + + tg_rt_rq[i] =3D &s_rq->rt; + tg_dl_se[i] =3D dl_se; + } + + tg->rt_rq =3D no_free_ptr(tg_rt_rq); + tg->dl_se =3D no_free_ptr(tg_dl_se); + + /* Initialize the allocated resources now. */ + init_dl_bandwidth(&tg->dl_bandwidth, 0, 0); + + for_each_possible_cpu(i) { + s_rq =3D served_rq_of_rt_rq(tg->rt_rq[i]); + dl_se =3D tg->dl_se[i]; + + init_rt_rq(&s_rq->rt); + s_rq->cpu =3D i; + s_rq->rt.tg =3D tg; + + init_dl_entity(dl_se); + dl_se->dl_runtime =3D tg->dl_bandwidth.dl_runtime; + dl_se->dl_period =3D tg->dl_bandwidth.dl_period; + dl_se->dl_deadline =3D dl_se->dl_period; + dl_se->dl_bw =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); + dl_se->dl_density =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); + dl_se->dl_server =3D 1; + + dl_server_init(dl_se, &cpu_rq(i)->dl, s_rq, rt_server_pick); + } + return 1; } =20 -#else /* !CONFIG_RT_GROUP_SCHED: */ +#else /* !CONFIG_RT_GROUP_SCHED */ =20 void unregister_rt_sched_group(struct task_group *tg) { } =20 @@ -114,7 +235,7 @@ int alloc_rt_sched_group(struct task_group *tg, struct = task_group *parent) { return 1; } -#endif /* !CONFIG_RT_GROUP_SCHED */ +#endif /* CONFIG_RT_GROUP_SCHED */ =20 static inline bool need_pull_rt_task(struct rq *rq, struct task_struct *pr= ev) { --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95D5930C37D for ; Mon, 1 Dec 2025 12:42:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592941; cv=none; b=S8/RlKRnIlAAoZx/h/pp9yV0W9N3zNraEl6NpkENND7ATGqrDknZQGMl1u9JfU7G7yVo1HPU5L0oOwMjCLVQe3WAGr0z+9/BrV65xGd9AA0tuAuZ2XraqYehcPEz6UjbUwK1hvx/w/nkx9A3SiKutpFkUPPx5buX1sno5S5mq6k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592941; c=relaxed/simple; bh=oI7Z2jgP+psbwMMvyJAHJRLGccfzMoF+ggwgRyx44zM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H+an/GIFPz9vED4I2N+D/w5nDag0zZV/nlmICYXFousJU43C9KQS3ext8OiXulyN8KPckLnE4gsLaMZExDipPEjwt+sMJ14tyzt9JuACDu1+5LAualyZp3m9yaQndoOrN6msFd9iXi/PHC7cEMenDVpElpzIWlOuliGdivI3g/Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fMqkDt8O; arc=none smtp.client-ip=209.85.218.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fMqkDt8O" Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-b7291af7190so652573866b.3 for ; Mon, 01 Dec 2025 04:42:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592938; x=1765197738; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5QdJ43HNE6kz3Z9ogRwg7j6TKHxVoh152ftOQ0IpmLE=; b=fMqkDt8OjZfyydsYbM1USGQIMdqg+VFjWhiLCGlBn2tbKirHcznAfSemHR/atcvG82 s7iuxug9WnPi1EWSXUb36OC+Y2a0EMzH/al6/d8B8fBWR3rK+arIqa52SWdrIOasKrpS WSmFd0F7KRzbJTRDZ6+sn+2l5ENFyr+EjSBnXydhkGS5D+rFaXfLOEvyAp6INVCCkOZf ymnzJoxgeAwHlGbAl7Ahce5ZmG/ixLdu8ewTD45pGnqBqthbn7/4tOzYzNi9rqacTB8t rcZ1GXGsfW6atmx9vzMAhdx9VCg6XLdCjedplhsHC7b5gghbK1l+2/7ThDe8LBEsduTj E1hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592938; x=1765197738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=5QdJ43HNE6kz3Z9ogRwg7j6TKHxVoh152ftOQ0IpmLE=; b=HDxCp6swR3hEWx3K9tAPhgrNqfznneAiOjkEmP3huWpnHv+rnYPYd2AQZFtEwXMzuP cLbehOAchJ4j6oNzZA/Db9Mq2XSmWuCia1tBU4vkSrYlklRCSR43bX61o8Bz6jS8Ks0x OjQnKU7jH6wss911cnGLjuJXI1uw59PDrsz+EQZck2Ex8ltnVF9imQungj+20eFn61yT C7JEGS+nyRJlEL+cUxNfi/GDHEVrrFOfCl9Y+zI3nEqyK5DDxpnDcc0zRd1/AJ3IZ4Mc /RHMaAiL8QhI6YfHuzkqhPj3f8vCkCVF9p0kZ/OWSN6acP6zgG+CdTzQMm/UsndDzAw+ mSMQ== X-Gm-Message-State: AOJu0Yxrjt5K+uDLp6Q4YMqS43rLGX7MuKJ2pLL3JvTf24DV2YfhQmJB BDjkjCWXV3eeZu5H48X6etBu9n/0IeNmG6+gRKroyhHT8r2aL5/JB2uS X-Gm-Gg: ASbGncuSo1PVeVQpSSSQxP7rJ8VjuB3EfO+gMij+h19cWguRHo2vDeTbgVhMOpYCvpb XSLSzd35EFAiJyRI4E08ib6fVpQAOh2QCwHWD6VSVqGgg5yJVdQPHlJYyoKHqtFsycNm/Xu0mBQ OsXMtdcFjydbFZELVi/hGaL8zIkLTxMBHeL2R9TEzrmgKUy7T20L6NkRen0eVuZNfvooN5En+5U ihzAJAy1K3r9ZUAr7WMxiIOkzHneLG058EhQ3Ad21joIJOsi9oJqkx1oahkpcy8l5gc1T4ZCEp8 Hv3ti0I2UmI5u+PiyXLol7O1dPloRsYUvgPkQ4drG19DDleEQzUP+vFmiiSfN6FnkEEkfY8z+pS U6xifYS/ikUPApUhrICPbGsvlSZzKw5FU0Fo8G1GzRav3noy1/Ya61xhf0/6Xt/fub8YArUh3W7 tlPmHoLzQp X-Google-Smtp-Source: AGHT+IEVpk9DL6TxkXebttdCeVrtf5oJ8gsJabYJxnIs5bCdglkZBLjaxpgaQH+NNFkzvgAntlLgJQ== X-Received: by 2002:a17:906:fd8a:b0:b73:70db:4994 with SMTP id a640c23a62f3a-b76716954e5mr3957474466b.34.1764592937469; Mon, 01 Dec 2025 04:42:17 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:17 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 11/28] sched/deadline: Account rt-cgroups bandwidth in deadline tasks schedulability tests. Date: Mon, 1 Dec 2025 13:41:44 +0100 Message-ID: <20251201124205.11169-12-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Account the rt-cgroups hierarchy's reserved bandwidth in the schedulability test of deadline entities. This mechanism allows to completely reserve portion of the rt-bandwidth to rt-cgroups even if they do not use all of it. Account for the rt-cgroups' reserved bandwidth also when changing the total dedicated bandwidth for real time tasks. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/deadline.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 4f43883a65..089fd2c9b7 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -213,8 +213,15 @@ void __dl_add(struct dl_bw *dl_b, u64 tsk_bw, int cpus) static inline bool __dl_overflow(struct dl_bw *dl_b, unsigned long cap, u64 old_bw, u64 new_b= w) { + u64 dl_groups_root =3D 0; + +#ifdef CONFIG_RT_GROUP_SCHED + dl_groups_root =3D to_ratio(root_task_group.dl_bandwidth.dl_period, + root_task_group.dl_bandwidth.dl_runtime); +#endif return dl_b->bw !=3D -1 && - cap_scale(dl_b->bw, cap) < dl_b->total_bw - old_bw + new_bw; + cap_scale(dl_b->bw, cap) < dl_b->total_bw - old_bw + new_bw + + cap_scale(dl_groups_root, cap); } static inline @@ -3147,10 +3154,16 @@ int sched_dl_global_validate(void) u64 period =3D global_rt_period(); u64 new_bw =3D to_ratio(period, runtime); u64 cookie =3D ++dl_cookie; + u64 dl_groups_root =3D 0; struct dl_bw *dl_b; - int cpu, cpus, ret =3D 0; + int cpu, cap, cpus, ret =3D 0; unsigned long flags; +#ifdef CONFIG_RT_GROUP_SCHED + dl_groups_root =3D to_ratio(root_task_group.dl_bandwidth.dl_period, + root_task_group.dl_bandwidth.dl_runtime); +#endif + /* * Here we want to check the bandwidth not being set to some * value smaller than the currently allocated bandwidth in @@ -3163,10 +3176,12 @@ int sched_dl_global_validate(void) goto next; dl_b =3D dl_bw_of(cpu); + cap =3D dl_bw_capacity(cpu); cpus =3D dl_bw_cpus(cpu); raw_spin_lock_irqsave(&dl_b->lock, flags); - if (new_bw * cpus < dl_b->total_bw) + if (new_bw * cpus < dl_b->total_bw + + cap_scale(dl_groups_root, cap)) ret =3D -EBUSY; raw_spin_unlock_irqrestore(&dl_b->lock, flags); -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B0AD3101B6 for ; Mon, 1 Dec 2025 12:42:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592942; cv=none; b=fsCewXOqSEDCfEtWggia6tgQB+aej/P+hX965p8N/Vc8D49dTna1cu2lusfBI51cUDzRSpUw8zwo+zKGQbYoxW8g2qvEf58B4BUZMn0hkJU6743hvELC+n5GLVt2AlDK9txq20AmCCywEtTkcEpR3Pr5nJtemeL/R3lzYnDbVQE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592942; c=relaxed/simple; bh=k6vGTH/kCHJ8gl9GzuNz7ZmOXkBbH4lEhlurPAgcnoY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V4A1F/a26Ux3bg/Coi7PEeZMYiIAImB5Tga/Bu7DqFuU6yXGulSE89wN7I2D/CPccV/d462hxdnHHCggbeQx83uuFdPPDu6K15vsZX3WxLwXncz45v55pjBee095feMhNDKqtPcc2yiqOHXRu6KvFIVVZ2jTGhVHb6oggftKlas= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=i/NuzH18; arc=none smtp.client-ip=209.85.208.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="i/NuzH18" Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-64165cd689eso7659466a12.0 for ; Mon, 01 Dec 2025 04:42:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592938; x=1765197738; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lm8Cv/YoLZ7BtJ1e+O4Z4YDATB0lJ0X3mzV7WuVpHIA=; b=i/NuzH18oGQBXSZmAMrt/lkq0djxJvqRn8Z9hrZAVDX7s70wRumQQhAZwthS2QADLM s72HuH3ymw6n9wO+uRh4uKK1Igf8I8kBf1EV3nI84Yj9Psbf5HcgQRoHuRQcgv9itUku U+f1hIiB7x47CIcfdFkrwc54FTtU3uLriuK+ETPL8uWuN/SwiLKnUs9MaVjvC7gEUGEr dB/PsWnJgAc6tCvs1rffEFM3eJUAQT5HMYByCvDjj7hdn9y6L3p//F9tY8F9VQYmSvoc IEE9JjRyqRV3sC4Mbvr6taxuw1W25Re27ZGG+4adadRLwoIHkYu0iQiCSSRFaOQQ1mBr t3sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592938; x=1765197738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=lm8Cv/YoLZ7BtJ1e+O4Z4YDATB0lJ0X3mzV7WuVpHIA=; b=OR/tNFFsZlke5EGer/qnz8R3Jhf6X+DlfjNIfTAcI5xCx7rJNbv28CL7YCkREJdpj3 4Smmb31yszarP84clCjL8Qe2HO6LzGIR0fqt8qpXJ3YAobDOQ/Hz5d5Q5Nur2H1e1Fvq vVGbA6DHxJ5dvbWFPhULzH+e1mTd7MHNBm2VmZRnXVer3mBTIsfigWkWWGS7BpWhobuR LCn93/lXm7lSVfcuog6XWFZwHSD2ebN0DEd2HfwXNe6NiUwn7dT8VYJsAAHn2pYN1J+S 2xs5L1Qsk/ZvZnLMadqcVSp/XmG872rmID2RqrAsey6S+bT2cXEqU5VR4uV4r1WraFyt Z0FA== X-Gm-Message-State: AOJu0YzmI2CURM7ucjkLKX7goufXYeXJsW5R3tBhSbxMekfGDmmZ5DHO mjgEBLgX1FThrWSfEYIQqNvitEcgxqjlC6AfXoqFuzSNtwM7YQ9zZN0A X-Gm-Gg: ASbGncstJNI5/uzLK8ppbI4jsPtkPhLduT7+C6weLyBmlui0lp2WP9qzEpsIULX7S/g ksOrwHz8aiyTxLvShOHShHfYsPq8Nz4ZitGVoLK4UvzXnFnTJWo8goXW0huf4+2AW/mK69bNz5J g/FhGfIS6IikROhny4iaWekxfjgjXqdmXtkyxkpDGnSP0mCkjS5Y3jx/trd2GAeUFTqClVDCFh1 6ivLuy1n01H+frqAJvWosWPz03YkX3SNsHzUNWBMdPQ4hfPBpeBU0dOilL3sAwK6jQpD4u9KbzS d+hgTv8jBqFw/S3d2oGXvBzDSdbbRwjvcCuT0hUQmMJ58298nTwpAwB3k0iYRynLIDBK8ZDqxfh rkfs0p1TugydCnWIZaK/Wn0XkQk7A92SDXrya6wSDsVk0Cjwlu4lanrxiSawAaQhmNAJ6E3z53Y VQs0F/wQT6 X-Google-Smtp-Source: AGHT+IEHuWswqEwdC+9qn+WugVNjfLuveVPkB153iRgBZ/W32v7K/V5rcY3WDNKmnO3F1ehvE+cDsQ== X-Received: by 2002:a17:907:6d1c:b0:b72:e158:8229 with SMTP id a640c23a62f3a-b766ef1d289mr4861178866b.15.1764592938303; Mon, 01 Dec 2025 04:42:18 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:18 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 12/28] sched/rt: Implement dl-server operations for rt-cgroups. Date: Mon, 1 Dec 2025 13:41:45 +0100 Message-ID: <20251201124205.11169-13-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" - Implement rt_server_pick, the callback that deadline servers use to pick a task to schedule. - rt_server_pick(): pick the next runnable rt task and tell the scheduler that it is going to be scheduled next. - Let enqueue/dequeue_task_rt function start/stop the attached deadline server when the first/last task is enqueued/dequeued on a specific rq/server. - Change update_curr_rt to perform a deadline server update if the updated task is served by non-root group. - Update inc/dec_dl_tasks to account the number of active tasks in the local runqueue for rt-cgroups servers, as their local runqueue is different from the global runqueue, and thus when a rt-group server is activated/deactivated, the number of served tasks' must be added/removed. This uses nr_running to be compatible with future dl-server interfaces. - Update inc/dec_rt_prio_smp to change a rq's cpupri only if the rt_rq is the global runqueue, since cgroups are scheduled via their dl-server priority. - Update inc/dec_rt_tasks to account for waking/sleeping tasks on the global runqueue, when the task runs on the root cgroup, or its local dl server is active. The accounting is not done when servers are throttled, as they will add/sub the number of tasks running when they get enqueued/dequeued. For rt cgroups, account for the number of active tasks in the nr_running field of the local runqueue (add/sub_nr_running), as this number is used when a dl server is enqueued/dequeued. - Update set_task_rq to record the dl_rq, tracking which deadline server manages a task. - Update set_task_rq to not use the parent field anymore, as it is unused by this patchset's code. Remove the unused parent field from sched_rt_entity. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: luca abeni Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- include/linux/sched.h | 1 - kernel/sched/deadline.c | 8 +++++ kernel/sched/rt.c | 68 ++++++++++++++++++++++++++++++++++++++--- kernel/sched/sched.h | 8 ++++- 4 files changed, 79 insertions(+), 6 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 000aa3b2b1..3f1f15b6d2 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -629,7 +629,6 @@ struct sched_rt_entity { struct sched_rt_entity *back; #ifdef CONFIG_RT_GROUP_SCHED - struct sched_rt_entity *parent; /* rq on which this entity is (to be) queued: */ struct rt_rq *rt_rq; /* rq "owned" by this entity/group: */ diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 089fd2c9b7..b890fdd4b2 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1847,6 +1847,10 @@ void inc_dl_tasks(struct sched_dl_entity *dl_se, str= uct dl_rq *dl_rq) if (!dl_server(dl_se)) add_nr_running(rq_of_dl_rq(dl_rq), 1); + else if (rq_of_dl_se(dl_se) !=3D dl_se->my_q) { + WARN_ON(dl_se->my_q->rt.rt_nr_running !=3D dl_se->my_q->nr_running); + add_nr_running(rq_of_dl_rq(dl_rq), dl_se->my_q->nr_running); + } inc_dl_deadline(dl_rq, deadline); } @@ -1859,6 +1863,10 @@ void dec_dl_tasks(struct sched_dl_entity *dl_se, str= uct dl_rq *dl_rq) if (!dl_server(dl_se)) sub_nr_running(rq_of_dl_rq(dl_rq), 1); + else if (rq_of_dl_se(dl_se) !=3D dl_se->my_q) { + WARN_ON(dl_se->my_q->rt.rt_nr_running !=3D dl_se->my_q->nr_running); + sub_nr_running(rq_of_dl_rq(dl_rq), dl_se->my_q->nr_running); + } dec_dl_deadline(dl_rq, dl_se->deadline); } diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 2301efc03f..7ec117a18d 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -128,9 +128,22 @@ void free_rt_sched_group(struct task_group *tg) kfree(tg->dl_se); } +static struct sched_rt_entity *pick_next_rt_entity(struct rt_rq *rt_rq); +static inline void set_next_task_rt(struct rq *rq, struct task_struct *p, = bool first); + static struct task_struct *rt_server_pick(struct sched_dl_entity *dl_se) { - return NULL; + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + struct rq *rq =3D rq_of_rt_rq(rt_rq); + struct task_struct *p; + + if (dl_se->my_q->rt.rt_nr_running =3D=3D 0) + return NULL; + + p =3D rt_task_of(pick_next_rt_entity(rt_rq)); + set_next_task_rt(rq, p, true); + + return p; } static inline void __rt_rq_free(struct rt_rq **rt_rq) @@ -435,6 +448,7 @@ static inline int rt_se_prio(struct sched_rt_entity *rt= _se) static void update_curr_rt(struct rq *rq) { struct task_struct *donor =3D rq->donor; + struct rt_rq *rt_rq; s64 delta_exec; if (donor->sched_class !=3D &rt_sched_class) @@ -444,8 +458,18 @@ static void update_curr_rt(struct rq *rq) if (unlikely(delta_exec <=3D 0)) return; - if (!rt_bandwidth_enabled()) + if (!rt_group_sched_enabled()) return; + + if (!dl_bandwidth_enabled()) + return; + + rt_rq =3D rt_rq_of_se(&donor->rt); + if (is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + dl_server_update(dl_se, delta_exec); + } } static void @@ -456,7 +480,7 @@ inc_rt_prio_smp(struct rt_rq *rt_rq, int prio, int prev= _prio) /* * Change rq's cpupri only if rt_rq is the top queue. */ - if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && &rq->rt !=3D rt_rq) + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) return; if (rq->online && prio < prev_prio) @@ -471,7 +495,7 @@ dec_rt_prio_smp(struct rt_rq *rt_rq, int prio, int prev= _prio) /* * Change rq's cpupri only if rt_rq is the top queue. */ - if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && &rq->rt !=3D rt_rq) + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) return; if (rq->online && rt_rq->highest_prio.curr !=3D prev_prio) @@ -534,6 +558,16 @@ void inc_rt_tasks(struct sched_rt_entity *rt_se, struc= t rt_rq *rt_rq) rt_rq->rr_nr_running +=3D is_rr_task(rt_se); inc_rt_prio(rt_rq, rt_se_prio(rt_se)); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (!dl_se->dl_throttled) + add_nr_running(rq_of_rt_rq(rt_rq), 1); + add_nr_running(served_rq_of_rt_rq(rt_rq), 1); + } else { + add_nr_running(rq_of_rt_rq(rt_rq), 1); + } } static inline @@ -544,6 +578,16 @@ void dec_rt_tasks(struct sched_rt_entity *rt_se, struc= t rt_rq *rt_rq) rt_rq->rr_nr_running -=3D is_rr_task(rt_se); dec_rt_prio(rt_rq, rt_se_prio(rt_se)); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (!dl_se->dl_throttled) + sub_nr_running(rq_of_rt_rq(rt_rq), 1); + sub_nr_running(served_rq_of_rt_rq(rt_rq), 1); + } else { + sub_nr_running(rq_of_rt_rq(rt_rq), 1); + } } /* @@ -725,6 +769,14 @@ enqueue_task_rt(struct rq *rq, struct task_struct *p, = int flags) check_schedstat_required(); update_stats_wait_start_rt(rt_rq_of_se(rt_se), rt_se); + /* Task arriving in an idle group of tasks. */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + is_dl_group(rt_rq) && rt_rq->rt_nr_running =3D=3D 0) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + dl_server_start(dl_se); + } + enqueue_rt_entity(rt_se, flags); if (task_is_blocked(p)) @@ -744,6 +796,14 @@ static bool dequeue_task_rt(struct rq *rq, struct task= _struct *p, int flags) dequeue_pushable_task(rt_rq, p); + /* Last task of the task group. */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + is_dl_group(rt_rq) && rt_rq->rt_nr_running =3D=3D 0) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + dl_server_stop(dl_se); + } + return true; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index f42bef06a9..fb4dcb4551 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2203,7 +2203,7 @@ static inline void set_task_rq(struct task_struct *p,= unsigned int cpu) if (!rt_group_sched_enabled()) tg =3D &root_task_group; p->rt.rt_rq =3D tg->rt_rq[cpu]; - p->rt.parent =3D tg->rt_se[cpu]; + p->dl.dl_rq =3D &cpu_rq(cpu)->dl; #endif /* CONFIG_RT_GROUP_SCHED */ } @@ -2750,6 +2750,9 @@ static inline void add_nr_running(struct rq *rq, unsi= gned count) unsigned prev_nr =3D rq->nr_running; rq->nr_running =3D prev_nr + count; + if (rq !=3D cpu_rq(rq->cpu)) + return; + if (trace_sched_update_nr_running_tp_enabled()) { call_trace_sched_update_nr_running(rq, count); } @@ -2763,6 +2766,9 @@ static inline void add_nr_running(struct rq *rq, unsi= gned count) static inline void sub_nr_running(struct rq *rq, unsigned count) { rq->nr_running -=3D count; + if (rq !=3D cpu_rq(rq->cpu)) + return; + if (trace_sched_update_nr_running_tp_enabled()) { call_trace_sched_update_nr_running(rq, -count); } -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 076C531062E for ; Mon, 1 Dec 2025 12:42:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592943; cv=none; b=AZJmbnsc+R6OzvEVuRf0qmEpYw7iuaAPIaLoAg3AxykQx/REm1DJLJmEKRtUEkWG8cZjFFifx+f+emNAOL48GCrf0IAemXyvOLdeSueLzaKXzVlWNfH9kOJYjF643K8vNCJ8gPucwdh+TtzxCQ89i6Gc70Ph/YJoDQQ0v20cVNg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592943; c=relaxed/simple; bh=UG3K+wxMNW7BTgQ31fHXNhk62O0mzoq3tbNazZI8W30=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FRXIgWZ8/31+o+gtBwKT6C4SR8xT8weEGHKIgYqkMZZUkVYjT8KPtgOIHbfDd7qs94eq+sufesyDQLt5BM3o1X4Ps9MHaZMxvDhg3Ee6EBkTmk02IkqE0mhCSOkkAHAikl55MF4KGTIb9rtH4/rakkGqKZ7GXH5ZJdSgTD7LypA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kzsmnKGv; arc=none smtp.client-ip=209.85.218.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kzsmnKGv" Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-b713c7096f9so673286166b.3 for ; Mon, 01 Dec 2025 04:42:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592939; x=1765197739; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F8g1afQO1rtVJZT9hlKagPzN9gZNeZwBORiXsRXGHT0=; b=kzsmnKGvl/EL6m0Pb374s3wn9NQTd0tCLjr8Unza+UW7E/8EIEOcIIBng2S/WTTHci URN8XCIq/CuZtPgiGXa16kPJjDkPwH8Ng0WSeIKqppDWcSmlLtY70tbHhloFt/QtFNp+ 0ZE6+LMSt9LRySzrtwVhNIupKJw7F1nWHFmRdeJj1oxI8xlQPDgaPrcOzNEzc7VaTrKq VT0PhqsMLF24blbJwZas2Qa7vBSmhdZJN37FIICmf+E0w5TxmAT7PmLfNq7uuj4OoMHS ZkKwNT9uAc9EhZ4S1RkNzd4tv8UM+I60xulccHXrM6B/Q8bNUeA23mnKVQV5fcI2onPo 8exw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592939; x=1765197739; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=F8g1afQO1rtVJZT9hlKagPzN9gZNeZwBORiXsRXGHT0=; b=kkuPVCcelXd1k8BuY4oZNw1cw3I9I4c1HslbxQtZuZHAn7EnolgDNBmSAU2Zx0q3wG 6I25lIZUJbFbY9wbVCSPYZdWI2E8GZyDh/lyGcIL3Ehaky/ot7ACvzi+8OayZHOR0AdA IGZbdOvoL3OTKRk6nl97neslDoShTk9X+9j25mrVf+nJaT9iUXIs7MCHlIJ0pu7YPX2i bpgpQUvkwsLvQSs4N9gfxxjdNX21qDkbZ/xe5ak8ImyXCgy5wwAFSapdgXDYOGV6lRf8 86TtB0hRC1PgljzifwGDk6qQMYQDz89642gPqmDgAh21kq9Qi9CFnmXj+jSJK1/N8wPB 4XFg== X-Gm-Message-State: AOJu0Yydv7L1pqf6mGyweTTjguefEroUvIZFu1OeMCkLXcRe5u4kFF1m vcEU95WRX0V4Vzo60hKF0I3Deac+qDaicSkiwcSKEJs7Mq0LY+0MS4Bb X-Gm-Gg: ASbGncttsm5AOIMmLHtnlYT8dOria6oRo/HlUKJ6PDA/Zh/pAqDXJdL3LcBEMTfBCpb kceXUVoCeO5WU5KOIdE+hJYIpVXKaIVJlfhno3pNhRHHbpJa0COC+iHCQkxhnD6P9rKj/zPuXjH uX4ZHHsRxRPgn/HryPbiOX47N94SpxRyU6iJbg4dPbK0EeyOdEO9Q1AexfpwRvQB0lUM8YXg75X 85lZhSQ/AoWWl3+ZpfvNqwjQvwg7Zaie6R1BUC7dxIG1kcMtnas1c6bxKt59BvukmT+B8F7est6 woeWvZtv96F2FtSXG1hb11ID0KYYQbdtR/NZT7tudWh8ho0LfSHS6Pu9HS236SdduBWgnJHcFN4 H52R//ugvYBjUMJOiw061QXrrDhPXxtRb1Jdyf6j/w6r8B5l2q0MR2GSM4ZAv85L2PqjFMIRdjK uc2j+hA7DemtYbWBf/DpI= X-Google-Smtp-Source: AGHT+IG8a8W7ew5ByJ+3rmg/WGQAR19MtvriaAtoQ1OfKaaIVKmszSnTnu5ZLbmHwhDKzdVBdr9cLw== X-Received: by 2002:a17:907:26c2:b0:b4e:f7cc:72f1 with SMTP id a640c23a62f3a-b76715aba47mr3966683166b.22.1764592939084; Mon, 01 Dec 2025 04:42:19 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:18 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 13/28] sched/rt: Update task event callbacks for HCBS scheduling Date: Mon, 1 Dec 2025 13:41:46 +0100 Message-ID: <20251201124205.11169-14-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Update wakeup_preempt_rt, switched_{from/to}_rt and prio_changed_rt with rt-cgroup's specific preemption rules: - In wakeup_preempt_rt(), whenever a task wakes up, it must be checked if it is served by a deadline server or it lives on the global runqueue. Preemption rules (as documented in the function), change based on the current task and woken task runqueue: - If both tasks are FIFO/RR tasks on the global runqueue, or the same cgroup, run as normal. - If woken is inside a cgroup, but curr is a FIFO task on the global runqueue, always preempt. If curr is a DEADLINE task, check if the dl server preempts curr. - If both tasks are FIFO/RR tasks in served but different groups, check whether the woken server preempts the current server. - In switched_from_rt(), perform a pull only on the global runqueue, and do nothing if the task is inside a group. This will change when migrations will be added. - In switched_to_rt(), queue a push only on the global runqueue, while perform a priority check when the task switching is inside a group. This will change also when migrations will be added. - In prio_changed_rt(), queue a pull only on the global runqueue, if the task is not queued. If the task is queued, run preemption checks only if both the prio changed task and curr are in the same cgroup. Update sched_rt_can_attach() to check if a task can be attached to a given cgroup. For now the check only consists in checking if the group has non-zero bandwidth. Remove the tsk argument from sched_rt_can_attach, as it is unused. Change cpu_cgroup_can_attach() to check if the attachee is a FIFO/RR task before attaching it to a cgroup. Update __sched_setscheduler() to perform checks when trying to switch to FIFO/RR for a task inside a cgroup, as the group needs to have runtime allocated. Update task_is_throttled_rt() for SCHED_CORE, returning the is_throttled value of the server if present, while global rt-tasks are never throttled. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: luca abeni Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 2 +- kernel/sched/rt.c | 104 +++++++++++++++++++++++++++++++++++----- kernel/sched/sched.h | 2 +- kernel/sched/syscalls.c | 13 +++++ 4 files changed, 108 insertions(+), 13 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 495cbdfdc5..d7fc83cdae 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9318,7 +9318,7 @@ static int cpu_cgroup_can_attach(struct cgroup_taskse= t *tset) goto scx_check; cgroup_taskset_for_each(task, css, tset) { - if (!sched_rt_can_attach(css_tg(css), task)) + if (rt_task(task) && !sched_rt_can_attach(css_tg(css))) return -EINVAL; } scx_check: diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 7ec117a18d..2b7c4b7754 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -960,7 +960,58 @@ static int balance_rt(struct rq *rq, struct task_struc= t *p, struct rq_flags *rf) static void wakeup_preempt_rt(struct rq *rq, struct task_struct *p, int fl= ags) { struct task_struct *donor =3D rq->donor; + struct sched_dl_entity *woken_dl_se =3D NULL; + struct sched_dl_entity *curr_dl_se =3D NULL; + if (!rt_group_sched_enabled()) + goto no_group_sched; + + /* + * Preemption checks are different if the waking task and the current task + * are running on the global runqueue or in a cgroup. The following rules + * apply: + * - dl-tasks (and equally dl_servers) always preempt FIFO/RR tasks. + * - if curr is a FIFO/RR task inside a cgroup (i.e. run by a + * dl_server), or curr is a DEADLINE task and waking is a FIFO/RR t= ask + * on the root cgroup, do nothing. + * - if waking is inside a cgroup but curr is a FIFO/RR task in the r= oot + * cgroup, always reschedule. + * - if they are both on the global runqueue, run the standard code. + * - if they are both in the same cgroup, check for tasks priorities. + * - if they are both in a cgroup, but not the same one, check whether = the + * woken task's dl_server preempts the current's dl_server. + * - if curr is a DEADLINE task and waking is in a cgroup, check whether + * the woken task's server preempts curr. + */ + if (is_dl_group(rt_rq_of_se(&p->rt))) + woken_dl_se =3D dl_group_of(rt_rq_of_se(&p->rt)); + if (is_dl_group(rt_rq_of_se(&rq->curr->rt))) + curr_dl_se =3D dl_group_of(rt_rq_of_se(&rq->curr->rt)); + else if (task_has_dl_policy(rq->curr)) + curr_dl_se =3D &rq->curr->dl; + + if (woken_dl_se !=3D NULL && curr_dl_se !=3D NULL) { + if (woken_dl_se =3D=3D curr_dl_se) { + if (p->prio < rq->curr->prio) + resched_curr(rq); + + return; + } + + if (dl_entity_preempt(woken_dl_se, curr_dl_se)) + resched_curr(rq); + + return; + + } else if (woken_dl_se !=3D NULL) { + resched_curr(rq); + return; + + } else if (curr_dl_se !=3D NULL) { + return; + } + +no_group_sched: if (p->prio < donor->prio) { resched_curr(rq); return; @@ -1710,6 +1761,8 @@ static void rq_offline_rt(struct rq *rq) */ static void switched_from_rt(struct rq *rq, struct task_struct *p) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); + /* * If there are other RT tasks then we will reschedule * and the scheduling of the other RT tasks will handle @@ -1717,10 +1770,11 @@ static void switched_from_rt(struct rq *rq, struct = task_struct *p) * we may need to handle the pulling of RT tasks * now. */ - if (!task_on_rq_queued(p) || rq->rt.rt_nr_running) + if (!task_on_rq_queued(p) || rt_rq->rt_nr_running) return; - rt_queue_pull_task(rt_rq_of_se(&p->rt)); + if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED) || !is_dl_group(rt_rq)) + rt_queue_pull_task(rt_rq); } void __init init_sched_rt_class(void) @@ -1740,6 +1794,8 @@ void __init init_sched_rt_class(void) */ static void switched_to_rt(struct rq *rq, struct task_struct *p) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); + /* * If we are running, update the avg_rt tracking, as the running time * will now on be accounted into the latter. @@ -1755,8 +1811,14 @@ static void switched_to_rt(struct rq *rq, struct tas= k_struct *p) * then see if we can move to another run queue. */ if (task_on_rq_queued(p)) { - if (p->nr_cpus_allowed > 1 && rq->rt.overloaded) - rt_queue_push_tasks(rt_rq_of_se(&p->rt)); + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + if (p->prio < rq->curr->prio) + resched_curr(rq); + } else { + if (p->nr_cpus_allowed > 1 && rq->rt.overloaded) + rt_queue_push_tasks(rt_rq_of_se(&p->rt)); + } + if (p->prio < rq->donor->prio && cpu_online(cpu_of(rq))) resched_curr(rq); } @@ -1769,6 +1831,8 @@ static void switched_to_rt(struct rq *rq, struct task= _struct *p) static void prio_changed_rt(struct rq *rq, struct task_struct *p, int oldprio) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); + if (!task_on_rq_queued(p)) return; @@ -1777,16 +1841,25 @@ prio_changed_rt(struct rq *rq, struct task_struct *= p, int oldprio) * If our priority decreases while running, we * may need to pull tasks to this runqueue. */ - if (oldprio < p->prio) - rt_queue_pull_task(rt_rq_of_se(&p->rt)); + if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED) && oldprio < p->prio) + rt_queue_pull_task(rt_rq); /* * If there's a higher priority task waiting to run * then reschedule. */ - if (p->prio > rq->rt.highest_prio.curr) + if (p->prio > rt_rq->highest_prio.curr) resched_curr(rq); } else { + /* + * This task is not running, thus we check against the currently + * running task for preemption. We can preempt only if both tasks are + * in the same cgroup or on the global runqueue. + */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + rt_rq_of_se(&p->rt)->tg !=3D rt_rq_of_se(&rq->curr->rt)->tg) + return; + /* * This task is not running, but if it is * greater than the current running task @@ -1881,7 +1954,16 @@ static unsigned int get_rr_interval_rt(struct rq *rq= , struct task_struct *task) #ifdef CONFIG_SCHED_CORE static int task_is_throttled_rt(struct task_struct *p, int cpu) { +#ifdef CONFIG_RT_GROUP_SCHED + struct rt_rq *rt_rq; + + rt_rq =3D task_group(p)->rt_rq[cpu]; + WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); + + return dl_group_of(rt_rq)->dl_throttled; +#else return 0; +#endif } #endif /* CONFIG_SCHED_CORE */ @@ -2133,16 +2215,16 @@ static int sched_rt_global_constraints(void) } #endif /* CONFIG_SYSCTL */ -int sched_rt_can_attach(struct task_group *tg, struct task_struct *tsk) +int sched_rt_can_attach(struct task_group *tg) { /* Don't accept real-time tasks when there is no way for them to run */ - if (rt_group_sched_enabled() && rt_task(tsk) && tg->rt_bandwidth.rt_runti= me =3D=3D 0) + if (rt_group_sched_enabled() && tg->dl_bandwidth.dl_runtime =3D=3D 0) return 0; return 1; } -#else /* !CONFIG_RT_GROUP_SCHED: */ +#else /* !CONFIG_RT_GROUP_SCHED */ #ifdef CONFIG_SYSCTL static int sched_rt_global_constraints(void) @@ -2150,7 +2232,7 @@ static int sched_rt_global_constraints(void) return 0; } #endif /* CONFIG_SYSCTL */ -#endif /* !CONFIG_RT_GROUP_SCHED */ +#endif /* CONFIG_RT_GROUP_SCHED */ #ifdef CONFIG_SYSCTL static int sched_rt_global_validate(void) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index fb4dcb4551..bc3ed02e40 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -606,7 +606,7 @@ extern int sched_group_set_rt_runtime(struct task_group= *tg, long rt_runtime_us) extern int sched_group_set_rt_period(struct task_group *tg, u64 rt_period_= us); extern long sched_group_rt_runtime(struct task_group *tg); extern long sched_group_rt_period(struct task_group *tg); -extern int sched_rt_can_attach(struct task_group *tg, struct task_struct *= tsk); +extern int sched_rt_can_attach(struct task_group *tg); extern struct task_group *sched_create_group(struct task_group *parent); extern void sched_online_group(struct task_group *tg, diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index 93a9c03b28..0b75126019 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -626,6 +626,19 @@ int __sched_setscheduler(struct task_struct *p, change: if (user) { +#ifdef CONFIG_RT_GROUP_SCHED + /* + * Do not allow real-time tasks into groups that have no runtime + * assigned. + */ + if (rt_group_sched_enabled() && + dl_bandwidth_enabled() && rt_policy(policy) && + !sched_rt_can_attach(task_group(p)) && + !task_group_is_autogroup(task_group(p))) { + retval =3D -EPERM; + goto unlock; + } +#endif if (dl_bandwidth_enabled() && dl_policy(policy) && !(attr->sched_flags & SCHED_FLAG_SUGOV)) { cpumask_t *span =3D rq->rd->span; -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B39B310779 for ; Mon, 1 Dec 2025 12:42:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592944; cv=none; b=rjfBo4f38ut9VyT8FWhZDYn/+Ceeyk0grGNzIKwDRbqrkOIDT9weaP9IwUgZjyGbGVxJVYHlby0l0JMYUyqqnH//OFpBJKBhPvfpVsJGZoKxtD/wQmbZcD2F9NIIkkiy4mnUmxizPnrbgq5Ix+Xdh/X3HodEPvLp+J0B1jeXTK8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592944; c=relaxed/simple; bh=cZM88g+rNBDskUGro8OBr43p/6kNCy7TkduCagsSQpM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=soiUonlgtN+T49Wo6PLCL2yHoWyYXZ6TsF+ss5c6punNGBbITOmEm9i1CxQKuf8somipuCFOIMT+OJEyNamn4ekGleMAqNIfRCWCgqjgsHYpix354FdjCA9+2zKh27cfK4FrjQgo79i+aV0QqicbRmcmjqyYQ//t7Rbde6ThXmQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jkHc5EXx; arc=none smtp.client-ip=209.85.218.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jkHc5EXx" Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-b7277324054so608712666b.0 for ; Mon, 01 Dec 2025 04:42:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592940; x=1765197740; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=azOgCvS8LK0s63UIy7dDLbReRbwXHsNSs/wFWEXEJIQ=; b=jkHc5EXxMmWcPq4kyRGcuRohQ9RH60oYQ0jxNuW3enJgioSP77IjiogQgM2s1+1yR5 gmERr28M1Tz2/fH9bBZToIw26bnjOF0RwuIusXGKC4HkkjDFAiCz7XDfWBlDtV87gV7R /kknlrP53OEEI7NFscVwvbLGt0Ui/e0mWxbqo8avOKS4W7V4WCQWiUgk/qyVgwyNMT0n JG+MxfVtxueDa5dKwXPHE8Ai6oxIVg7quOV1Drh0ANDKbZ66yPUJJY2b0ZiOM8gJv7y0 4fLs3ExtMn3sU4sIRBm82QZIO8Pf58WN5WbHZxoXBP+UmewJl6kRr0y8eSINrZkXX6rX 0fUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592940; x=1765197740; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=azOgCvS8LK0s63UIy7dDLbReRbwXHsNSs/wFWEXEJIQ=; b=C51LKr709wuJGt+KA7zZzvyUoR2MwoH/FozViZ+iQ4ttuzd/6XaNLZYJxqxyvvAaBD rqOJmPBWpRd9tG2ayMkOQPNoi4VhQfLPiadUvFZOsN4ZWej/7RqidgLgQEzpLhomNZwu 0vHQ7lRw2lsWBXipK0ZJXYiTvsHP3PMppfwLft1nXSgm4wssgJCnqsKX9XYCmtbiBr2S GxrVIX1uB3edsfjoTq2xnYMmHce6xwuhiQNSiXgIzmSnDX25PK+MOHU/0HxCuxYDW9Jv 1wD2IvXHRaGZwX7DyQoy7WSvWkd3aoiJRBirraQ2eCIVu1r3bWtFquPUXmYczP7O3ryP ugNA== X-Gm-Message-State: AOJu0YyXH0XPAuxDk4VfR64v1n1osZ3lC5u3lMf3IJB9P8AWteDcvJ6S pK8sqHBPUL3gBf6x0u7hX7l7qbxUnYKZAs1SDdPOBtjX5XaDmOzCVOTA X-Gm-Gg: ASbGncsrGTbAdULesljWi+2ko2tt8PQVBn87gZKajDlPqsRW3YgPtAMxe8mFKdSp0+P 58jX5fgN3FYoc9Cl0Vr8H5ZrkHIaXC8xjf0NzaOnW0b1g3R6nqlpQvyVGQQWCIN/z/254y5el2Z oxc8jTi91odPHvCx4d0GxoWCgaQbw412RrqlDMN43yeFA0Z1ZifuMhQ/xb6YSys6xmV2AhFsZaM WEcR6fsirZKc+VzDbvtZzc8l4MSBdoR02QuNgckoi7TLhTiWmmY52WrEA58d3CUXnmlYeanMxwO eqPnt0SZLOHOdI/bJZX2p3Xi5aMFQBwT7D4Rc5rCIBkchl1KuBJiW5ix+PWIG0n1oMWReA6wSBT 6jpvuNEL+kra37lVw1SMoN/vtw5sCKYu7rVGYTWu65Gu6qatWkku2BeFO5t/Py+5C0/hy3l/kZ2 mM95AES75G X-Google-Smtp-Source: AGHT+IEeeWdrYH6wYTf161KuxLaASkweGnpW+0dHKqgfemmWoxi70t7n1z9kZrest0L5EnxbFP1lHw== X-Received: by 2002:a17:907:9628:b0:b4a:e11a:195b with SMTP id a640c23a62f3a-b767184bc82mr3844480166b.44.1764592939958; Mon, 01 Dec 2025 04:42:19 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:19 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 14/28] sched/rt: Update rt-cgroup schedulability checks Date: Mon, 1 Dec 2025 13:41:47 +0100 Message-ID: <20251201124205.11169-15-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Update sched_group_rt_runtime/period and sched_group_set_rt_runtime/period to use the newly defined data structures and perform necessary checks to update both the runtime and period of a given group. The set functions call tg_set_rt_bandwidth() which is also updated: - Use the newly added HCBS dl_bandwidth structure instead of rt_bandwidth. - Update __rt_schedulable() to check for numerical issues: - Prevent a non-zero runtime that is too small, since a non-zero very small runtime will make the servers behave as they had zero runtime. - Since some computation use signed integers, the period might be so big that when read as a signed integer becomes a negative number, and we don't want that. If the period satisfies this prerequisite, also the runtime will do, since the runtime is always less than or equal to the period. - Update tg_rt_schedulable(), used when walking the cgroup tree to check if all invariants are met: - Update most of the instructions to obtain data from the newly added data structures (dl_bandwidth). - If the task group is the root group, run a total bandwidth check with the newly added dl_check_tg() function. - After all checks are successful, if the changed group is not the root cgroup, update the assigned runtime and period to all the local deadline servers. - Additionally use a mutex guard instead of manually locking/unlocking. Add dl_check_tg(), which performs an admission control test similar to __dl_overflow, but this time we are updating the cgroup's total bandwidth rather than scheduling a new DEADLINE task or updating a non-cgroup deadline server. Finally, prevent creation of a cgroup hierarchy with depth greater than two, as this will be addressed in a future patch. A depth two hierarchy is sufficient for now for testing the patchset. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/core.c | 6 ++++ kernel/sched/deadline.c | 46 +++++++++++++++++++++++---- kernel/sched/rt.c | 70 +++++++++++++++++++++++------------------ kernel/sched/sched.h | 1 + 4 files changed, 87 insertions(+), 36 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d7fc83cdae..bdf1bebe52 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9253,6 +9253,12 @@ cpu_cgroup_css_alloc(struct cgroup_subsys_state *par= ent_css) return &root_task_group.css; } =20 + /* Do not allow cpu_cgroup hierachies with depth greater than 2. */ +#ifdef CONFIG_RT_GROUP_SCHED + if (parent !=3D &root_task_group) + return ERR_PTR(-EINVAL); +#endif + tg =3D sched_create_group(parent); if (IS_ERR(tg)) return ERR_PTR(-ENOMEM); diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index b890fdd4b2..7ed157dfa6 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -347,7 +347,47 @@ void cancel_inactive_timer(struct sched_dl_entity *dl_= se) cancel_dl_timer(dl_se, &dl_se->inactive_timer); } =20 +/* + * Used for dl_bw check and update, used under sched_rt_handler()::mutex a= nd + * sched_domains_mutex. + */ +u64 dl_cookie; + #ifdef CONFIG_RT_GROUP_SCHED +int dl_check_tg(unsigned long total) +{ + unsigned long flags; + int which_cpu; + int cap; + struct dl_bw *dl_b; + u64 gen =3D ++dl_cookie; + + for_each_possible_cpu(which_cpu) { + rcu_read_lock_sched(); + + if (!dl_bw_visited(which_cpu, gen)) { + cap =3D dl_bw_capacity(which_cpu); + dl_b =3D dl_bw_of(which_cpu); + + raw_spin_lock_irqsave(&dl_b->lock, flags); + + if (dl_b->bw !=3D -1 && + cap_scale(dl_b->bw, cap) < dl_b->total_bw + cap_scale(total, cap)) { + raw_spin_unlock_irqrestore(&dl_b->lock, flags); + rcu_read_unlock_sched(); + + return 0; + } + + raw_spin_unlock_irqrestore(&dl_b->lock, flags); + } + + rcu_read_unlock_sched(); + } + + return 1; +} + void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 rt_peri= od) { struct rq *rq =3D container_of(dl_se->dl_rq, struct rq, dl); @@ -3150,12 +3190,6 @@ DEFINE_SCHED_CLASS(dl) =3D { #endif }; =20 -/* - * Used for dl_bw check and update, used under sched_rt_handler()::mutex a= nd - * sched_domains_mutex. - */ -u64 dl_cookie; - int sched_dl_global_validate(void) { u64 runtime =3D global_rt_runtime(); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 2b7c4b7754..b0a6da20b5 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2007,11 +2007,6 @@ DEFINE_SCHED_CLASS(rt) =3D { }; =20 #ifdef CONFIG_RT_GROUP_SCHED -/* - * Ensure that the real time constraints are schedulable. - */ -static DEFINE_MUTEX(rt_constraints_mutex); - static inline int tg_has_rt_tasks(struct task_group *tg) { struct task_struct *task; @@ -2045,8 +2040,8 @@ static int tg_rt_schedulable(struct task_group *tg, v= oid *data) unsigned long total, sum =3D 0; u64 period, runtime; =20 - period =3D ktime_to_ns(tg->rt_bandwidth.rt_period); - runtime =3D tg->rt_bandwidth.rt_runtime; + period =3D tg->dl_bandwidth.dl_period; + runtime =3D tg->dl_bandwidth.dl_runtime; =20 if (tg =3D=3D d->tg) { period =3D d->rt_period; @@ -2062,8 +2057,7 @@ static int tg_rt_schedulable(struct task_group *tg, v= oid *data) /* * Ensure we don't starve existing RT tasks if runtime turns zero. */ - if (rt_bandwidth_enabled() && !runtime && - tg->rt_bandwidth.rt_runtime && tg_has_rt_tasks(tg)) + if (dl_bandwidth_enabled() && !runtime && tg_has_rt_tasks(tg)) return -EBUSY; =20 if (WARN_ON(!rt_group_sched_enabled() && tg !=3D &root_task_group)) @@ -2077,12 +2071,17 @@ static int tg_rt_schedulable(struct task_group *tg,= void *data) if (total > to_ratio(global_rt_period(), global_rt_runtime())) return -EINVAL; =20 + if (tg =3D=3D &root_task_group) { + if (!dl_check_tg(total)) + return -EBUSY; + } + /* * The sum of our children's runtime should not exceed our own. */ list_for_each_entry_rcu(child, &tg->children, siblings) { - period =3D ktime_to_ns(child->rt_bandwidth.rt_period); - runtime =3D child->rt_bandwidth.rt_runtime; + period =3D child->dl_bandwidth.dl_period; + runtime =3D child->dl_bandwidth.dl_runtime; =20 if (child =3D=3D d->tg) { period =3D d->rt_period; @@ -2108,6 +2107,20 @@ static int __rt_schedulable(struct task_group *tg, u= 64 period, u64 runtime) .rt_runtime =3D runtime, }; =20 + /* + * Since we truncate DL_SCALE bits, make sure we're at least + * that big. + */ + if (runtime !=3D 0 && runtime < (1ULL << DL_SCALE)) + return -EINVAL; + + /* + * Since we use the MSB for wrap-around and sign issues, make + * sure it's not set (mind that period can be equal to zero). + */ + if (period & (1ULL << 63)) + return -EINVAL; + rcu_read_lock(); ret =3D walk_tg_tree(tg_rt_schedulable, tg_nop, &data); rcu_read_unlock(); @@ -2118,6 +2131,7 @@ static int __rt_schedulable(struct task_group *tg, u6= 4 period, u64 runtime) static int tg_set_rt_bandwidth(struct task_group *tg, u64 rt_period, u64 rt_runtime) { + static DEFINE_MUTEX(rt_constraints_mutex); int i, err =3D 0; =20 /* @@ -2137,34 +2151,30 @@ static int tg_set_rt_bandwidth(struct task_group *t= g, if (rt_runtime !=3D RUNTIME_INF && rt_runtime > max_rt_runtime) return -EINVAL; =20 - mutex_lock(&rt_constraints_mutex); + guard(mutex)(&rt_constraints_mutex); err =3D __rt_schedulable(tg, rt_period, rt_runtime); if (err) - goto unlock; + return err; =20 - raw_spin_lock_irq(&tg->rt_bandwidth.rt_runtime_lock); - tg->rt_bandwidth.rt_period =3D ns_to_ktime(rt_period); - tg->rt_bandwidth.rt_runtime =3D rt_runtime; + guard(raw_spinlock_irq)(&tg->dl_bandwidth.dl_runtime_lock); + tg->dl_bandwidth.dl_period =3D rt_period; + tg->dl_bandwidth.dl_runtime =3D rt_runtime; =20 - for_each_possible_cpu(i) { - struct rt_rq *rt_rq =3D tg->rt_rq[i]; + if (tg =3D=3D &root_task_group) + return 0; =20 - raw_spin_lock(&rt_rq->rt_runtime_lock); - rt_rq->rt_runtime =3D rt_runtime; - raw_spin_unlock(&rt_rq->rt_runtime_lock); + for_each_possible_cpu(i) { + dl_init_tg(tg->dl_se[i], rt_runtime, rt_period); } - raw_spin_unlock_irq(&tg->rt_bandwidth.rt_runtime_lock); -unlock: - mutex_unlock(&rt_constraints_mutex); =20 - return err; + return 0; } =20 int sched_group_set_rt_runtime(struct task_group *tg, long rt_runtime_us) { u64 rt_runtime, rt_period; =20 - rt_period =3D ktime_to_ns(tg->rt_bandwidth.rt_period); + rt_period =3D tg->dl_bandwidth.dl_period; rt_runtime =3D (u64)rt_runtime_us * NSEC_PER_USEC; if (rt_runtime_us < 0) rt_runtime =3D RUNTIME_INF; @@ -2178,10 +2188,10 @@ long sched_group_rt_runtime(struct task_group *tg) { u64 rt_runtime_us; =20 - if (tg->rt_bandwidth.rt_runtime =3D=3D RUNTIME_INF) + if (tg->dl_bandwidth.dl_runtime =3D=3D RUNTIME_INF) return -1; =20 - rt_runtime_us =3D tg->rt_bandwidth.rt_runtime; + rt_runtime_us =3D tg->dl_bandwidth.dl_runtime; do_div(rt_runtime_us, NSEC_PER_USEC); return rt_runtime_us; } @@ -2194,7 +2204,7 @@ int sched_group_set_rt_period(struct task_group *tg, = u64 rt_period_us) return -EINVAL; =20 rt_period =3D rt_period_us * NSEC_PER_USEC; - rt_runtime =3D tg->rt_bandwidth.rt_runtime; + rt_runtime =3D tg->dl_bandwidth.dl_runtime; =20 return tg_set_rt_bandwidth(tg, rt_period, rt_runtime); } @@ -2203,7 +2213,7 @@ long sched_group_rt_period(struct task_group *tg) { u64 rt_period_us; =20 - rt_period_us =3D ktime_to_ns(tg->rt_bandwidth.rt_period); + rt_period_us =3D tg->dl_bandwidth.dl_period; do_div(rt_period_us, NSEC_PER_USEC); return rt_period_us; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index bc3ed02e40..334ab6d597 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -419,6 +419,7 @@ extern void dl_server_init(struct sched_dl_entity *dl_s= e, struct dl_rq *dl_rq, struct rq *served_rq, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); +extern int dl_check_tg(unsigned long total); extern void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 = rt_period); =20 extern void dl_server_update_idle_time(struct rq *rq, --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A110B3101BF for ; Mon, 1 Dec 2025 12:42:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592944; cv=none; b=aurJQcp93UXLBYeJDOf1a6XF6QduMOde5RKhzB7VEqjpMXqurwX7zfQPLmlsbUXqzWqYtI/Ay5ZI7xZTZ7LUDqif+9sQ0HteQmGHuPGDazQkFeIyv813ge99W3nAybXSIQxyA2jY6STHbuMLR8MwWIYVSOIwt/pLrjwTC5O/HBo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592944; c=relaxed/simple; bh=kRAV6Sip3ySWqNXeYhykc+OTIYsWrhmYevq/Ew7UNKE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nk5P3sCK/E0r6YPTm46KSP/g/uUvvmX+pbtBB6dJxgTUJ6LTAIM7Z8+Ix+AqDqVjMMfhqm15TFP4WfTAWZ8cZgZj+2Ej67NtXV86Pj05epeCuR0qbb1idYRRWvX1TtKUb0riit5WNbO69CznzNstbKw3LFTD/qwYfUFR8D4ilLo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PiiK99UR; arc=none smtp.client-ip=209.85.218.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PiiK99UR" Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-b734fcbf1e3so503911266b.3 for ; Mon, 01 Dec 2025 04:42:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592941; x=1765197741; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VfOen1xVY5o2YZPHr+01crG7DyOloUam5Y6+uPIg3QY=; b=PiiK99UR3qqBEFID0pIKWiFAD4iVDqHfEKe9uA0sKQDzUyJInOVSSiQm1qhkjUHhEM E1lxc/3g2TwofIVPSTpJoRZkdC9olxNzzh+eXHj8JfaelKQcbYe56HSkZTqqapcVCCAx A3+Dnyr1UAImtvPIDFIfWA4kFPcHQewSrOuyFXYjzRqcBau8GUM00XM8u39v88f3Nxkq V/P+MZuvTRAMaI5IVi42irETvgwrFZefVxTmrAJcFpSYWtYCy5kpD9nF6tCdIOPBXSkL EuUOruvhqg1c6qlua0iCuQNwDkcEwXY4o7aA1ZB9ahBWtjNH95IViiei6P/Qlnfi7r78 fYZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592941; x=1765197741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=VfOen1xVY5o2YZPHr+01crG7DyOloUam5Y6+uPIg3QY=; b=dCd6FrOWKC7dUfE/LMA/SsxWxK6L4YkX0jBsYBYgJE/miJlC1qibx/lus7OCsZvN7C s2wy5pHfBP3IriwGbVJfWVDc1eGiXKzuOkWsVUyTUXEo7ueoYHp+oWgRF9SL88hTJta5 N9DI6ctKiXgYGWiaUppOMZ+eZE73i9ap9+OiGyKRhky2T7Gm0dIM7o7CQBTffBuJ1+PT co5gx6bE6lJ4BrraXzDbosS0ZkacyxR7szG7aDW8hCbWp2hERHoIIJC7EAo+z+lKNdUP MbjqLtLmLEiR4IAA6qwBjf+nbEqzVRZ5eKm/FJWtT2M3iL0d4TtovWTeYG3Jyb6kejcx ZtPQ== X-Gm-Message-State: AOJu0Yzdp3YVKzkyYjmKpScxTHINLOKK4Q6GWCtMZy6dtYpN3v/MqJpD WVJE305ixwwMyKP0U5qjINZJjaUdbHlrXdelpRPJigwKs/+gij3fv96x X-Gm-Gg: ASbGncuad4mrSospnKThKt8G0eJRiDnJfcK9TwGLwXXcoAMzEj39CK36zecOe7iQNrR EqpGZMuaygfuP8w7LIdZ6vq5FiD5TbiVaUF0FBb0U+LHYgtrv44Qmws20VKMatNXYxW9wpHgTlb hl0a8c9fvC0oqDTn4/EAR9dcktERx4rkljxGrhq36A4DIOVm94YxBGKjnQ7IJd48RcC/Z7W0J19 Zjg0AnfPV56Ci9DI2oF39+KK+46vaIks3xs3lQHypTUltIZGUUNr7A2SMTovGoBCKGwJndK899q oCHgDo/U2Un3rUZmxMEtp+Xs3SK+jY5ahXvmG4Sc5q8qQZTk3f7dki5G9UvnfpFbK0MeMsZcRVB EckhcZe8yYui03PHEnbrIIf94kTvPwf3cZhQloaWEJQdvJ1QpzxIbbsx7mRq4FPVRhyAJJ62D8/ 3q+X2a9oNR X-Google-Smtp-Source: AGHT+IFdNSsgg3zRaL/0BOhdvYKvXkkw/hl5etSqVwSeQlm5MysW2vLwi3lhern/ERSLTj7Ef2fVCg== X-Received: by 2002:a17:907:948e:b0:b76:6aca:f1f3 with SMTP id a640c23a62f3a-b7671589e26mr4355707166b.19.1764592940736; Mon, 01 Dec 2025 04:42:20 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:20 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 15/28] sched/rt: Allow zeroing the runtime of the root control group Date: Mon, 1 Dec 2025 13:41:48 +0100 Message-ID: <20251201124205.11169-16-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In HCBS, the root control group follows the already existing rules for rt-task scheduling. As such, it does not make use of the deadline servers to account for runtime, or any other HCBS specific code and features. While the runtime of SCHED_DEADLINE tasks depends on the global bandwidth reserved for rt_tasks, the runtime of SCHED_FIFO/SCHED_RR tasks is limited by the activation of fair-servers (as the RT_THROTTLING mechanism has been removed in favour of them), thus their maximum bandwidth depends solely on the fair-server settings (which are thightly related to the global bandwidth reserved for rt-tasks) and the amount of SCHED_OTHER workload to run (recall that if no SCHED_OTHER tasks are running, the FIFO/RR tasks may fully utilize the CPU). The values of runtime and period in the root cgroup's cpu controller do not affect, by design of HCBS, the fair-server settings and similar (consequently they do not affect the scheduling of FIFO/RR tasks in the root cgroup), but they are just used to reserve a portion of the SCHED_DEADLINE bandwidth to the scheduling of rt-cgroups. These values only affect child cgroups, their deadline servers and their assigned FIFO/RR tasks. Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index b0a6da20b5..928f53c1b0 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2134,13 +2134,6 @@ static int tg_set_rt_bandwidth(struct task_group *tg, static DEFINE_MUTEX(rt_constraints_mutex); int i, err =3D 0; - /* - * Disallowing the root group RT runtime is BAD, it would disallow the - * kernel creating (and or operating) RT threads. - */ - if (tg =3D=3D &root_task_group && rt_runtime =3D=3D 0) - return -EINVAL; - /* No period doesn't make any sense. */ if (rt_period =3D=3D 0) return -EINVAL; @@ -2227,6 +2220,10 @@ static int sched_rt_global_constraints(void) int sched_rt_can_attach(struct task_group *tg) { + /* Allow executing in the root cgroup regardless of allowed bandwidth */ + if (tg =3D=3D &root_task_group) + return 1; + /* Don't accept real-time tasks when there is no way for them to run */ if (rt_group_sched_enabled() && tg->dl_bandwidth.dl_runtime =3D=3D 0) return 0; -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 607E5311963 for ; Mon, 1 Dec 2025 12:42:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592945; cv=none; b=nIApj/OgPXDTeEeeefKteWSIlOpqLwOU1b1InMubOBfCOLUpJ3JQxUAcBJlUD/Yh2pLWsUlDrMihW5ZDbNTFnKRis16BIT6e9/zvdPLmGM4MAT8qWBZoXPYuqDbpUNMdDiCoAVmCW3vlju+MDIZKRAbYYXcFq6eNvPcoz/tJLXg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592945; c=relaxed/simple; bh=T1qGbhRTkC28d7o63YuT93RIlLek17T4jLGGswVSUIU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UraFwGufqKE4JSrOu3juUnKLj+RtmLteXMseqTvTy7en39bu5wXa37GVr1r8b9qq8GWF8sYvqVqrFQvutaQa+52HF95+azP8+liTub/SF5nkNLlJHA7+FD9EEAeJA+xWnQCMwd187JhayCvytYwLuzx43DqMMprgneN4AilMhBQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UXdLJLDW; arc=none smtp.client-ip=209.85.208.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UXdLJLDW" Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-640b0639dabso7349475a12.3 for ; Mon, 01 Dec 2025 04:42:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592942; x=1765197742; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/44OW1N0N075v+z8jY5sCVSRCT5RDByQx5KR+2xJP54=; b=UXdLJLDWmCbwqdCfxPnlzOY4ycSQykLE62aDbw1R2DC9l4IxVflxeByoTnPlifR44L kuccZUHpai06inQnIuJCsFbBG6pA6My/RQRNcRJRpgRQqceAA8EVzaxq1uWr8r/xG6+L kZx1wPDTLQ6lJgX0tXdqy9NHDpsnb+jZTBZLV10QDAG4DEdguBrZaX4hNP8zmwfdUk5x p8wBy0eytP0Yanwo89qmOL1+PbvwYrtx0fDInW8/PszwCgyj8+5Uz+brfM2raL3U/Yqk UA0ODK/2P3VofktexlslfxY6gjvf4au4lWvz6A9aDGxeSmK57Uc1SSy/lg88tXchbRH0 RvHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592942; x=1765197742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=/44OW1N0N075v+z8jY5sCVSRCT5RDByQx5KR+2xJP54=; b=plyrtKzQZedbU9FGgWo+5QsbkGRje4+gh9leUcAgK3ofo2NF49nfVAHqcokU/GfBZ+ wSyE968flBADrQ+qxm1giat6VM+Gt43s7+MmtFjhY0QdjNrzrcClLZYGJoEjMAzUSrVu frlxUyue1+gicBIyb2u+qDQcfnU1pMOJOG0e0/NG7oq2147t1J8d5cKhmW/CvzrzShCA 9N5Lt3OR8gYx7n2NrNSnZAYg2oNqo8/tQDIx+itOglPrGHkaMYk/tp6AQAMK4E2RCTOr EOsNaoUWWP3/Jr6OfgQHFQVXl67Sp57kLeOzMjauq6iCFdjQJjRVFLJQ4hFqhEcJKTAk haVA== X-Gm-Message-State: AOJu0YyD/pYxWGDbJ0Z287wkvh0KF+fqNdF5gosRBkr65S71AOQX053J U4iMKkVCZpZabnhg0SEGjOaoyInqcfJ3e2aagUV8ixmkQUsZpMjErHy1 X-Gm-Gg: ASbGncuTH+iFKrV+GyF19UvE8Xq4tCfSyRE9ahzGfbt8dgMrC8LK6K0hTqq9wZwMakO /aNtTUHDeuTd80xSLRzm/13em23VRxX8isWR37iuRu5HsjDEk9XSRsbEsGG7HY+tcoD45VYyU9A YqKXucbsriJRem5jv9wcNdwbpj0WksF1k/Rto0E3h4FzCoihTKwgllu8R9Y0t+Grrupi15E1Yn0 JkRFo3Rsm6vWXix9CGm60hk0gsJuUUtWi6Tg5bJowUhrOvAIk/bnqYhWfXLqI1Z84EjZYclS6z0 eBXNuDHzUDVCRsnYVSN6sY8nrCsm9rbW1/cP3pIvjlwT0MrAkDi/rLAM84Y5AqgTr3D/2Cx6GQ6 I7qVUcUIsTz2sOSXm8PxaWKHqaUAJbPShTV6wcFoGYwxo0rdFHSlz2oUgb5Eyr6MWVZks2BYJwE w4P6rl2cnO X-Google-Smtp-Source: AGHT+IG8uRxxyzrug3ziU1U0GaXYiQte2r1cGKsfHBaoKby5xGiWan7osRBaz+XToC1s8/2YTdaUaw== X-Received: by 2002:a17:907:971e:b0:b73:769d:350d with SMTP id a640c23a62f3a-b767158b537mr4180137366b.22.1764592941514; Mon, 01 Dec 2025 04:42:21 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:21 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 16/28] sched/rt: Remove old RT_GROUP_SCHED data structures Date: Mon, 1 Dec 2025 13:41:49 +0100 Message-ID: <20251201124205.11169-17-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Completely remove the old RT_GROUP_SCHED's functions and data structures: - Remove the fields back and my_q from sched_rt_entity. - Remove the rt_bandwidth data structure. - Remove the field rt_bandwidth from task_group. - Remove the rt_bandwidth_enabled function. - Remove the fields rt_queued, rt_throttled, rt_time, rt_runtime, rt_runtime_lock and rt_nr_boosted from rt_rq. All of the removed fields and data are similarly represented in previously added fields in rq, rt_rq, dl_bandwidth and in the dl server themselves. Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- include/linux/sched.h | 3 --- kernel/sched/sched.h | 26 -------------------------- 2 files changed, 29 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 3f1f15b6d2..9ef7797983 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -627,12 +627,9 @@ struct sched_rt_entity { unsigned short on_rq; unsigned short on_list; - struct sched_rt_entity *back; #ifdef CONFIG_RT_GROUP_SCHED /* rq on which this entity is (to be) queued: */ struct rt_rq *rt_rq; - /* rq "owned" by this entity/group: */ - struct rt_rq *my_q; #endif } __randomize_layout; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 334ab6d597..4b65775ada 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -310,15 +310,6 @@ struct rt_prio_array { struct list_head queue[MAX_RT_PRIO]; }; -struct rt_bandwidth { - /* nests inside the rq lock: */ - raw_spinlock_t rt_runtime_lock; - ktime_t rt_period; - u64 rt_runtime; - struct hrtimer rt_period_timer; - unsigned int rt_period_active; -}; - struct dl_bandwidth { raw_spinlock_t dl_runtime_lock; u64 dl_runtime; @@ -510,7 +501,6 @@ struct task_group { struct sched_dl_entity **dl_se; struct rt_rq **rt_rq; - struct rt_bandwidth rt_bandwidth; struct dl_bandwidth dl_bandwidth; #endif @@ -826,11 +816,6 @@ struct scx_rq { }; #endif /* CONFIG_SCHED_CLASS_EXT */ -static inline int rt_bandwidth_enabled(void) -{ - return 0; -} - /* RT IPI pull logic requires IRQ_WORK */ #if defined(CONFIG_IRQ_WORK) && defined(CONFIG_SMP) # define HAVE_RT_PUSH_IPI @@ -848,17 +833,6 @@ struct rt_rq { bool overloaded; struct plist_head pushable_tasks; - int rt_queued; - -#ifdef CONFIG_RT_GROUP_SCHED - int rt_throttled; - u64 rt_time; /* consumed RT time, goes up in update_curr_rt */ - u64 rt_runtime; /* allotted RT time, "slice" from rt_bandwidth, RT shar= ing/balancing */ - /* Nests inside the rq lock: */ - raw_spinlock_t rt_runtime_lock; - - unsigned int rt_nr_boosted; -#endif #ifdef CONFIG_CGROUP_SCHED struct task_group *tg; /* this tg has "this" rt_rq on given CPU for runna= ble entities */ #endif -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3ABE1311C21 for ; Mon, 1 Dec 2025 12:42:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592946; cv=none; b=MdQvhFIlkn/HcHzpcloFAwE73uTCFoCMguJnYLTmkFPYTksvSnPyhTgGY+hoNOoWnlMUdLxmWS5DvdJxSYdcyGqd75g1aqYSgh0vEJwdT9kW7mP3q73IuMQzwc04hrL+68ceUbsB/Uty7eOTl2uybXS+hXzNBD+PpKfRMr9j9Hg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592946; c=relaxed/simple; bh=9NhK29Uoc5Bfnina0HvslEebOxLopEwveTmyahyHzA0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QN/cVAuSvdi5LfFGesb3wDNxxhT6gvCIMMQU5pEoqvC2emYST63GOwAxL1dKqKwSeq5VaJR6f0G00froFM25To1Br0qcYiXVP1bqTOnRnOR9vImNdj9WeAMTcZiI/QVTEo8zBGi/LkYkd9xBcwP5KoPuNXaEl24nMRdXKoX2alk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=M63Pm7b5; arc=none smtp.client-ip=209.85.208.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="M63Pm7b5" Received: by mail-ed1-f41.google.com with SMTP id 4fb4d7f45d1cf-64312565c10so6289513a12.2 for ; Mon, 01 Dec 2025 04:42:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592942; x=1765197742; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9t8gWspL8tSTeY/JmdJvUlRQ8yVj/mJL00mFbBMP5qk=; b=M63Pm7b51oCbVXxe1mKdqv0Jl8fKgo60Tx2bOrqOMu45B2udb5ru7D7kouq3wdMpUQ nUb8ZiWoPVSuGVbfrWp/Lr4YG/XB3XFjmpZ2KHZ8h8G2cjxgOmLT/pqkvv6tlUfMreeJ DA5C5LuUzTyOTm9YgBhnon+a5qKcb5x01hQODtclszydkChHN1vjCvwG9hcOW+5PypAX smi+CaxR8nFKsZqeLaZtkW25VUWetUewEewhT9MsO+2lbLbCHVpID9am4wpIAKGPZSOr an7OspsaaRCj1uOC9MbPenna2vhQ44i2q/feY1e2sJF6ADPdykbXUrh1stwapQpjvg8J AQmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592942; x=1765197742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=9t8gWspL8tSTeY/JmdJvUlRQ8yVj/mJL00mFbBMP5qk=; b=JFGYCLaq4kTa1WMxtq8RmWTTRY8xZ1H8SQh8nLjJ5dtR6Qky2VRd55dDt9ZMl4uTEo +iirbggV+7sCl10qTL6bzy7QJ19wvnG8P26omaxncr1EerfivQc9Qdag7bjlCS3VzNUw tlucwiqNmpuJfVTboabOMmBxZbMdO3784CCQApHR8i3VUwSwU07uMPHoTJjxbvf125Ye I5h8b7/xtaqhTVcjRO8bTUTMiZiduSxDEsQD7lDO1ATahV9iM7BL0BpPDICE67ixvEsn fKMd6NptTFw0ZzvOeadNbKGHu6HxU5tvtAs7Z/iM12NMp4N8RJYjm8IV/FB8EbwT0UKJ nT4Q== X-Gm-Message-State: AOJu0YwqJMCsexMacqXYIDkfyIjxLIdckx5XLV+8Vf3QssqHyhw98wP9 Rbk2IMwIfuPVkQID42rZ/WQYci2VUmCyoAG/fqCXGW1o1FamiCz7PYBx X-Gm-Gg: ASbGncv/A+6fXN4OB9CsQg3tnp+IUOy0x+Ky9iDXHaZtqOZf7iUPCeOV/ExK2SVOLm0 FCEIUxTxld8yL4a7v/YHP5m27PihVJoB8vnRGSWTE8gH0DrO1Jz60HQOAnK0xa2yZYR5VLXKzt7 LVneLhjK68F2cuhdmRuso3xsYeIudxLn7dsbe494Chr/JcHSkEjw01Y7by7d1z8aFzcMMBLoUbJ aszWjm/40OzI/U+pyu5LKYcSkva6rwLjb1OB3aLOvWn4l2io587wVqTqdeBcp1p/PMrZZQsxH8W cXRdY9BRN19IzDPnLAeEar+f7WhwAVDiBr4WbTmr+wps/b9Cu0ZSAUQMaNdKjQEAjy66Pqa0asb /4C+f9WN7FgORrWqTnFZZqMCz5vUbthg97XlT8COzXHlhHEk+XXGDr/6cTXVd9FoKYGSSKBaiMq coIXnALpxQ X-Google-Smtp-Source: AGHT+IFQfzxrMCBbksufl+AYaZj3E+VZvonfHYmrW/noARqON3/t4zHw0sSnhDGghXpwcHZfPI13Mw== X-Received: by 2002:a17:907:3cc2:b0:b73:1e09:7377 with SMTP id a640c23a62f3a-b76c5593daemr2745141566b.58.1764592942396; Mon, 01 Dec 2025 04:42:22 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:22 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 17/28] sched/core: Cgroup v2 support Date: Mon, 1 Dec 2025 13:41:50 +0100 Message-ID: <20251201124205.11169-18-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make rt_runtime_us and rt_period_us virtual files accessible also to the cgroup v2 controller, effectively enabling the RT_GROUP_SCHED mechanism to cgroups v2. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index bdf1bebe52..6278f203b4 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -10293,6 +10293,18 @@ static struct cftype cpu_files[] =3D { .write =3D cpu_uclamp_max_write, }, #endif /* CONFIG_UCLAMP_TASK_GROUP */ +#ifdef CONFIG_RT_GROUP_SCHED + { + .name =3D "rt_runtime_us", + .read_s64 =3D cpu_rt_runtime_read, + .write_s64 =3D cpu_rt_runtime_write, + }, + { + .name =3D "rt_period_us", + .read_u64 =3D cpu_rt_period_read_uint, + .write_u64 =3D cpu_rt_period_write_uint, + }, +#endif /* CONFIG_RT_GROUP_SCHED */ { } /* terminate */ }; =20 --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55AB73126C0 for ; Mon, 1 Dec 2025 12:42:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592947; cv=none; b=MSya1SXt3K9GnznsQQq+FB8uHGOn4ApwS9RUlawQw9LLgC3LUkdMaJ1qDy5LsqbIOq3njCNLfdiK+Ao91ifZ4C5/ccghLZNK6PLuF6hAs1HqCznYm7krSxcFAns7cy1f7/BsV2njQmnvc8zYATLbkyaOD+e483yegr2vt0XxNBw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592947; c=relaxed/simple; bh=+UM1VV7t/K6zNWiYuhbTHU/3k4YJUIqmIzu87rU3UGE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pVdAP/CUrzto5Osz1DNo3ZNzFra4rzaLb8b74EkTDuiRP/P0N+b1uIg98L9LpIEl9hsCgJrRGf557FeYiUzF9QIWfSh3ITeJaweDoYrtcvRzplsLRCpIKzk46MZRSnVms7duFzosm2naPRYQwFOgLG2GWccXvaNBS/lR9Qhfjrg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CJNyn9zH; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CJNyn9zH" Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-b736cd741c1so640447966b.0 for ; Mon, 01 Dec 2025 04:42:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592943; x=1765197743; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HhpS2oKxZvZfaQrgep/KpSTvkTmB1tmZ6ahjhloadqk=; b=CJNyn9zHthYvVpwEDUqRZkMpJ7bzXWnh5MYTmum2H17OPY0fq8kL59LcKpSOMbKstA XKZS7OKJlGZRyZ+8HES/LtbgzCpppgQSbvO9BE4/s/ZA+3Q9tp8RUMYpSyxD5DzgSPfH kCdLB+UVwQRstz/aMnRISxOdzzQIAhBSfzJ6DBJ/0+cE85sHy1/M/FUDJgLZLXW6C9jU 0jqf72j0vEmIiyIdhHrOZxSX8gQeh6Hpdh2GG6F5AmVnwImgCZhJWn0fjKzSd/QJ9GNV AbMZibhw4y0cwIzhz8ZlXyUjIgg1IOpnj7mpZJuL9HwFo4Q8qbQnOflsuLLrjBzLzfqI gnHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592943; x=1765197743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=HhpS2oKxZvZfaQrgep/KpSTvkTmB1tmZ6ahjhloadqk=; b=sFxihFWF17yD8WXlUVlQ0g9xCmnZAr/B8iCZ5mxnvCbcWSR52K+hdQHLcZYDCimrNF AfGpy00Q1tmwBx2AAEmt8OJ/+HlhYE1pS1oRqJC2pA7/8j2ej3GAICOEmzOM/CEzF1bb xsvC6ejmg+LTDA99Tu687muRfx2hu60Q6LJNBc9r+GITwfBbxrYlFizlgkAluPHtC2bn kE16AW6rcOWa7rh+FMYZ71HggnAOBR038OrLqNFfaILrZwWrZTlVatTUAjF2MjmDTIrv k6f/+EfBhRWPUZOQb1fCAIhrdSnl5nMnA/HAQII5bFVsi1Nzezyyt/E0QNGaBH3Gtymh HpDw== X-Gm-Message-State: AOJu0Yz9KeYm7eIKOCHK4jgCPbdz1i6hkHpjIR0TqTzY8ABqG5Vr6X7p lEaZ0emXuAxDbzXk1u1f2arvRRq8ByFZ2GbVd3dGMjiNGVmeeu9V/JBLgDfZU5If X-Gm-Gg: ASbGnctue6LQ9iD0fbN1GDEOsNeRDIFkG//DpMWICG08YNe37QNMe8ZXBQ9KOTuJTtk 3KDHq4FlSfQ2TurSodcH3o7WVMQFVLEriMPBRo3cYRcRAakK7NBbEhH4geizQCNT6okfwlY+8vg z1/+JIrr8r2gztzifpCnN5qp3/jix/QEBGoc/YDcuscgFPTsGT6XZadhT+XPBYgSdqwFzzix66v YOEzT0NSXQLkuH08vEpvZM4BANIwYUBOIhYhdxebveX5sfSVbiSCsSN3o6Xkw6NG6OaMGTULhob 44vJZ9tILKCUGsgUqnEe18tMSnj9dXRfIXPEqdWpLVRoimluShU+AdyTUYSR3AeIAf5Xks3UYV1 3GWsz5Nka8bkDDLfB+fQY0E6cLFD2cJnQ5Erue2W1h+KeNNLTrprSNLNHYzVi0izxc862tscuWP a4ADc4QJ2+ X-Google-Smtp-Source: AGHT+IGGfo8Vf8AHW2kGZy8RU+Uaroi9UtSuWHSFVj0cFB4gA+SuN7TJhDLaJkNSlmxIc9bbrIrslQ== X-Received: by 2002:a17:907:724a:b0:b73:3f7d:3df with SMTP id a640c23a62f3a-b76717099demr2564337666b.41.1764592943384; Mon, 01 Dec 2025 04:42:23 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:22 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 18/28] sched/rt: Remove support for cgroups-v1 Date: Mon, 1 Dec 2025 13:41:51 +0100 Message-ID: <20251201124205.11169-19-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Disable control files for cgroups-v1, and allow only cgroups-v2. This should simplify maintaining the code, also because cgroups-v1 are deprecated. Set the default rt-cgroups runtime to zero, otherwise a cgroup-v1 kernel will not be able to start SCHED_DEADLINE tasks. The bandwidth for rt-cgroups must then be manually assigned after the kernel boots. Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 6278f203b4..cfb39050a2 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8693,7 +8693,7 @@ void __init sched_init(void) =20 #ifdef CONFIG_RT_GROUP_SCHED init_dl_bandwidth(&root_task_group.dl_bandwidth, - global_rt_period(), global_rt_runtime()); + global_rt_period(), 0); #endif /* CONFIG_RT_GROUP_SCHED */ =20 #ifdef CONFIG_CGROUP_SCHED @@ -10032,20 +10032,6 @@ static struct cftype cpu_legacy_files[] =3D { }; =20 #ifdef CONFIG_RT_GROUP_SCHED -static struct cftype rt_group_files[] =3D { - { - .name =3D "rt_runtime_us", - .read_s64 =3D cpu_rt_runtime_read, - .write_s64 =3D cpu_rt_runtime_write, - }, - { - .name =3D "rt_period_us", - .read_u64 =3D cpu_rt_period_read_uint, - .write_u64 =3D cpu_rt_period_write_uint, - }, - { } /* Terminate */ -}; - # ifdef CONFIG_RT_GROUP_SCHED_DEFAULT_DISABLED DEFINE_STATIC_KEY_FALSE(rt_group_sched); # else @@ -10071,10 +10057,6 @@ __setup("rt_group_sched=3D", setup_rt_group_sched); =20 static int __init cpu_rt_group_init(void) { - if (!rt_group_sched_enabled()) - return 0; - - WARN_ON(cgroup_add_legacy_cftypes(&cpu_cgrp_subsys, rt_group_files)); return 0; } subsys_initcall(cpu_rt_group_init); --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52C7F312825 for ; Mon, 1 Dec 2025 12:42:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592949; cv=none; b=JzM9K96ZPukg/A0jij2SB8u0/0i7tG3kE1aNPerW9X4XR/uEaXS23TuKoVhZz026wJe0ZMnqiqGLuFrXIFQTuLVt2jxkzctufswcLacEhTICRAICdqt8NAkZZ+1YGBG2qFkgzB9TbsolBTc2NlrbM66ljwX4DEzs6rv+AvhvicE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592949; c=relaxed/simple; bh=nwzCzmQCbRGpt9RS1KfZi6qB3vg44wTz++4vlAs636o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RdT9zLdkmj6/u9YRy03F7uPHpx4WWwfSSgsDTPG6LkxYCaMfccWs8lwQ9mW4O77ZtZbkUyhoGQiw/Nwq6mfSFtUrSBcxzRjYoiF+THgEhEJLZ5quz/wqDLjPpAowqJmpnL5O4WUE6i4FrBe3Pq4bt90CsBbw2Tx1Fe51Bf3QIrk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=T/CwAZ5Q; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="T/CwAZ5Q" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-b735487129fso584182466b.0 for ; Mon, 01 Dec 2025 04:42:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592944; x=1765197744; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8tAH7XbatC4BT69CgiRsmyLJQ+32HQcwrTzhuC5J9t8=; b=T/CwAZ5QCEmI+dyKC1RDw2lMBoLnZlU+Tq6JpWZFMI7SPvK6aDgnDI4MkyFpckT1Lv 7zr7ylh5yT5rX1dv13pbP97RBROS7jyYDgV4jpHDdUlbEQdDsqc3XBm0vw6Eg5Lzxodv zC/wh7soPg1IgfiDTQmc/6QVDxiJuUvLQCB7+KU6Z3YUGDNAsBkRDut/ts2WnVJca6WB gx7ltXdN2Gdp4lhspx5+FGJmsOAu3F6h7V7Y0Pz/K7O9zNPwnOCY+wHu25LN/a8vpkIq 03Kn2fzNtaiK1XGPUE2y5t8193rtQMdWoPVby6Y5TliNKf3Vj1ot8yBigxfiQs/2fU5d GWhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592944; x=1765197744; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=8tAH7XbatC4BT69CgiRsmyLJQ+32HQcwrTzhuC5J9t8=; b=URlM53JsGuWI5XgYRlP4BxsdfWpaqwW8ch3x+KZbQ9zSuJ+p4x0YjeDy5A0/hk89Eg Sv3tBXIPMq+smUYYON/fEIJ+bzB/iQBmERElZid4FwiQDoY8BvmurUHlz3Cu7sXFUpWf 4fTmXISYae3ccShHpRtDqd0J5GwARTa0vze8gg2zR9InnsX4Fd4+c72hl1Q7vXpgbWfM Un7ELjku8ElNTZUMFYuBqx2bMDfXgjdbzs2sLhmRWbtq6flRfakXVHLx8Lj1wzDV71uX 7smew3SwtuU99zTGPWd1bFC5QHiV907UeyaSbd3475PIzZ/9Adrt693j4CfwgXlvXnrM v4qw== X-Gm-Message-State: AOJu0YzX60BHtU1d5T2Dv9j3bckQ3/YzR08VPCJ+EfC34kQ1IqeDM+AV Sbi9J4j9K0I+bBiVKas+koa0WA9BYQQC/Lm7uvxISUh+ZWTk3Nqy83+/ X-Gm-Gg: ASbGncs5EmTEpGVtpDsXMZzf92lCGO0fhrF7/SUCPSIgjNAqV6mpH1T6uEvPC7PkQWt uWdkuPcwfadcdG1IWorSsqiBI/TkfPdDbRL8tyvtzmIkr8Ww2XE/hT+mxrGXGjL3DzlIEIaDD8l pEjvAKuuDLJnRzEVOegnakcF6ZniSoajBiP+9QZ756lNiK+i5rBsP/MyFJgag3gcUN9SLSEf7Cr 0pjVS3lbDnhEH7U+K3jR2DIeh2O0+WFE/JWZbU6My1HJwRpgNMl+gVCKZNrkL+Z2nrBUzSvDj5w aSy/b31ZWrpw1t5T+Yv4AJ6XuYtn0clmR5G4nA3m03ZHSHmB0tofezDr/VKX4VeFINN0g4Z1YYO D0L1vWBQHDcjv8dKwv2TSrgGJLwsPjkJjlfOaOMflp2T6LgqnJWURGIC2KUdFbzpy/K4UJ2qHOp PvCmW/6W6N X-Google-Smtp-Source: AGHT+IHTC7WDKAqdxr7e0OTaE2LS3M+JyLcS2denqML8KOowz4Z6ZSZ6FRBkMKrACALKXl/nCJCuOg== X-Received: by 2002:a17:907:2d11:b0:b73:853d:540e with SMTP id a640c23a62f3a-b767170c823mr4429015966b.30.1764592944271; Mon, 01 Dec 2025 04:42:24 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:23 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 19/28] sched/deadline: Allow deeper hierarchies of RT cgroups Date: Mon, 1 Dec 2025 13:41:52 +0100 Message-ID: <20251201124205.11169-20-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Remove the check for depth two only cgroup hierachies. Introduce the concept of live and active groups: - A group is live if it is a leaf group or if all its children have zero runtime. - A live group with non-zero runtime can be used to schedule tasks. - A live group with running tasks is deemed active. - A non-live group cannot be used to run tasks, but it is only used for bandwidth accounting, i.e. the sum of its children bandwidth must be less than or equal to the bandwidth of the parent. This change allows to use cgroups for bandwidth management for different users. - While the root cgroup specifies the total allocatable bandwidth of rt cgroups, a further accounting is performed to keep track of the live bandwidth, i.e. the sum of the bandwidth of live groups. The hierarchy invariant states that the live bandwidth must always be less than or equal to the total allocatable bw. Add is_live_sched_group() and sched_group_has_live_siblings() in deadline.c. These utility functions are used by dl_init_tg to perform updates only when necessary: - Only live groups may update the active dl bandwidth of dl entities (call to dl_rq_change_utilization), while non-live groups must not use servers, and thus must not change the active dl bandwidth. - The total bandwidth accounting must be changed to follow the live/non-live rules: - When disabling (runtime zero) the last child of a group, the parent becomes a live group, and so the parent's bw must be accounted back. - When enabling (runtime non-zero) the first child, the parent becomes a non-live group, and so the parent's bandwidth must be removed. Update free_rt_sched_group() to only zero out the runtime of non-zeroed servers. This is also necessary to force the bandwidth accounting of live groups. Update tg_set_rt_bandwidth() to change the runtime of a group to a non-zero value only if its parent is inactive, thus forcing it to become non-live if it was precedently (it would've already been non-live if a sibling cgroup was live). Update sched_rt_can_attach() to allow attaching only on live groups. Update dl_init_tg() to take a task_group pointer and a cpu's id rather than passing directly the pointer to the cpu's deadline server. The task_group pointer is necessary to check and update the live bandwidth accounting. Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/core.c | 6 ---- kernel/sched/deadline.c | 61 ++++++++++++++++++++++++++++++++++++++--- kernel/sched/rt.c | 16 +++++++++-- kernel/sched/sched.h | 3 +- 4 files changed, 72 insertions(+), 14 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index cfb39050a2..983cd1b478 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9253,12 +9253,6 @@ cpu_cgroup_css_alloc(struct cgroup_subsys_state *par= ent_css) return &root_task_group.css; } - /* Do not allow cpu_cgroup hierachies with depth greater than 2. */ -#ifdef CONFIG_RT_GROUP_SCHED - if (parent !=3D &root_task_group) - return ERR_PTR(-EINVAL); -#endif - tg =3D sched_create_group(parent); if (IS_ERR(tg)) return ERR_PTR(-ENOMEM); diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 7ed157dfa6..082bccc30b 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -388,11 +388,44 @@ int dl_check_tg(unsigned long total) return 1; } -void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 rt_peri= od) +/* + * A cgroup is deemed live if: + * - It is a leaf cgroup. + * - All it's children have zero runtime. + */ +bool is_live_sched_group(struct task_group *tg) +{ + struct task_group *child; + bool is_active =3D 1; + + /* if there are no children, this is a leaf group, thus it is live */ + list_for_each_entry_rcu(child, &tg->children, siblings) { + if (child->dl_bandwidth.dl_runtime > 0) + is_active =3D 0; + } + return is_active; +} + +static inline bool sched_group_has_live_siblings(struct task_group *tg) +{ + struct task_group *child; + bool has_active_siblings =3D 0; + + list_for_each_entry_rcu(child, &tg->parent->children, siblings) { + if (child !=3D tg && child->dl_bandwidth.dl_runtime > 0) + has_active_siblings =3D 1; + } + return has_active_siblings; +} + +void dl_init_tg(struct task_group *tg, int cpu, u64 rt_runtime, u64 rt_per= iod) { + struct sched_dl_entity *dl_se =3D tg->dl_se[cpu]; struct rq *rq =3D container_of(dl_se->dl_rq, struct rq, dl); - int is_active; - u64 new_bw; + int is_active, is_live_group; + u64 old_runtime, new_bw; + + is_live_group =3D is_live_sched_group(tg); guard(raw_spin_rq_lock_irq)(rq); is_active =3D dl_se->my_q->rt.rt_nr_running > 0; @@ -400,8 +433,10 @@ void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_= runtime, u64 rt_period) update_rq_clock(rq); dl_server_stop(dl_se); + old_runtime =3D dl_se->dl_runtime; new_bw =3D to_ratio(rt_period, rt_runtime); - dl_rq_change_utilization(rq, dl_se, new_bw); + if (is_live_group) + dl_rq_change_utilization(rq, dl_se, new_bw); dl_se->dl_runtime =3D rt_runtime; dl_se->dl_deadline =3D rt_period; @@ -413,6 +448,24 @@ void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_= runtime, u64 rt_period) dl_se->dl_bw =3D new_bw; dl_se->dl_density =3D new_bw; + /* + * Handle parent bandwidth accounting when child runtime changes: + * - When disabling the last child, the parent becomes a leaf group, + * and so the parent's bandwidth must be accounted back. + * - When enabling the first child, the parent becomes a non-leaf group, + * and so the parent's bandwidth must be removed. + * Only leaf groups (those without active children) have non-zero bandwid= th. + */ + if (tg->parent && tg->parent !=3D &root_task_group) { + if (rt_runtime =3D=3D 0 && old_runtime !=3D 0 && + !sched_group_has_live_siblings(tg)) { + __add_rq_bw(tg->parent->dl_se[cpu]->dl_bw, dl_se->dl_rq); + } else if (rt_runtime !=3D 0 && old_runtime =3D=3D 0 && + !sched_group_has_live_siblings(tg)) { + __sub_rq_bw(tg->parent->dl_se[cpu]->dl_bw, dl_se->dl_rq); + } + } + if (is_active) dl_server_start(dl_se); } diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 928f53c1b0..a2084e9dc5 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -113,7 +113,8 @@ void free_rt_sched_group(struct task_group *tg) * Fix this issue by changing the group runtime * to 0 immediately before freeing it. */ - dl_init_tg(tg->dl_se[i], 0, tg->dl_se[i]->dl_period); + if (tg->dl_se[i]->dl_runtime) + dl_init_tg(tg, i, 0, tg->dl_se[i]->dl_period); raw_spin_rq_lock_irqsave(cpu_rq(i), flags); hrtimer_cancel(&tg->dl_se[i]->dl_timer); @@ -2134,6 +2135,14 @@ static int tg_set_rt_bandwidth(struct task_group *tg, static DEFINE_MUTEX(rt_constraints_mutex); int i, err =3D 0; + /* + * Do not allow to set a RT runtime > 0 if the parent has RT tasks + * (and is not the root group) + */ + if (rt_runtime && tg !=3D &root_task_group && + tg->parent !=3D &root_task_group && tg_has_rt_tasks(tg->parent)) + return -EINVAL; + /* No period doesn't make any sense. */ if (rt_period =3D=3D 0) return -EINVAL; @@ -2157,7 +2166,7 @@ static int tg_set_rt_bandwidth(struct task_group *tg, return 0; for_each_possible_cpu(i) { - dl_init_tg(tg->dl_se[i], rt_runtime, rt_period); + dl_init_tg(tg, i, rt_runtime, rt_period); } return 0; @@ -2228,7 +2237,8 @@ int sched_rt_can_attach(struct task_group *tg) if (rt_group_sched_enabled() && tg->dl_bandwidth.dl_runtime =3D=3D 0) return 0; - return 1; + /* tasks can be attached only if the taskgroup has no live children. */ + return (int)is_live_sched_group(tg); } #else /* !CONFIG_RT_GROUP_SCHED */ diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 4b65775ada..6c3fbfe84f 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -411,7 +411,8 @@ extern void dl_server_init(struct sched_dl_entity *dl_s= e, struct dl_rq *dl_rq, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); extern int dl_check_tg(unsigned long total); -extern void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 = rt_period); +extern void dl_init_tg(struct task_group *tg, int cpu, u64 rt_runtime, u64= rt_period); +extern bool is_live_sched_group(struct task_group *tg); extern void dl_server_update_idle_time(struct rq *rq, struct task_struct *p); -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3EB003128D0 for ; Mon, 1 Dec 2025 12:42:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592950; cv=none; b=IkXvnaPZglLXdbpF+fNGh0gvBHV3cBBjuOf3lgSq9+vjq0qNEGeQBdgbrZmw7RQvyblwIdH4Rr61cQN8GStvCy3W3J6Nh7lbVuyZpHvc2i/M5QHPTOQ7oVtb4QrQ280FEktPe6Z5jHHiz1AB0oun/L8LeWy0JLOOESXtK4tMsVw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592950; c=relaxed/simple; bh=SAsxqv1eu+ikY7V3M7DAW8ruQkbpwq7SFxB5Gi8mQrw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qHtYlzm3tRdcuDfWDjFYFBW2n/4gHRssODyT9Z9N1jJtFF1oujRxgerZD465P6K47cw/nu83dBhmAOEZjfoNPDb8lipQ0axLxVcrMe2xU9rLcJQLHRk49Bbk3sQhJbTcFl9XmhVAc+RVRPobol576dMcEcX0PKUSgt4jUPlfSFU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fMe7tgvP; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fMe7tgvP" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b73161849e1so999062366b.2 for ; Mon, 01 Dec 2025 04:42:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592945; x=1765197745; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Daq6uMLStCAxEufGh71WVrlqgO7omdYzhIWDJZWeUOs=; b=fMe7tgvPFmHdeb8ia6lp+DBJidw4UbVSAtfIQnjJFK1Hl8dzE7vDM4sk+Ahthx8BSh GkenpkSpWSNXcBRlMX1YPv+ZFESRgz5SqWLb2h1bG0cHy9nXSpwh2DpQ56j4GCqhMKBu duC4064sdYn6WZ1Zdrr+PfEf4I7s0eotBgeBbTEY8eQGT5yPSunB6mOBHJnNmqhlFVaB d9C7+nJLMW1qK6KC+IGNSyl0M3gUj2V+HaXtTr0sDhTVH/uBx0007+3GzClBHBy4iIHK 0MCfdLm+0/vZIwz7njGt85g+eH7hDwjeihlkzhCgA7kuvlp2toM0hHBqCuW2OiyVkxCv yoHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592945; x=1765197745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Daq6uMLStCAxEufGh71WVrlqgO7omdYzhIWDJZWeUOs=; b=rY1UgpoKBZd76IfJWDlI2Hs7EQRiFT2yW0oeq+eTIucFP7CHm/gJyFx1GE/IYh7+LT 3iN1O54y3GGIq3nTJsZLJDA3j7AMyPIAhBoynaLjDqPfT8Pq0yNaNsQkzDPVt7xKM02y cedpAxbpm6Y6Ug14uxvSuFF4y21fysOo0RLzjBr+3ssqFYqBdBVCLKg23bpWoO7EfMDR BzvJSTzswRZn1YwFJd538UfuJ8p5BzeW2XUbxGqd6vcUS+QD4Ql6bArRnPVbM+bm0aCC enndcGEXr4Q7r6QbgSau/ixdOeSAEghgjYMtuaRJMONkqM9KXnrzmzW1Tui1fNpA6bzO 5Gzg== X-Gm-Message-State: AOJu0YwX+QZVOMefKBNZf7WBveUa7Nf8BH8ItTlFEO67rH97TxfNugrm oxXQb9F206+N2ykwoOSpneSepIbKlxRcqOeXG9lLKduLJGGAgJxp2/yP X-Gm-Gg: ASbGncvtYVW7WQCeZITBTJ/I+scKxhulVHrXFdbjMYpyznZ4K9oZI/j+CPVm37AvZh2 1iY4hDFOiIoG0HK8CANhokzAtbQKMcjgobQkObgtGN+TT/mJLfQYF7KHcr9cLSv9uZ44/ArTnVj /KHGdia8s+finO+/8K0lEelx7pMAXC5VVbQpu1RgtOCWVUBu/i7Ywx+Iec5k98o5OwhYlj4YFq1 4MlUNhTQg1B9rKOzYbUq3h6KECvDprWQKyXz2DN2Fvz+ikk4Gni8auzwlr3I7tkZX1P0ISDyt5X fS7scZYj1jwHhTHrDJJxg5v5S8bbu6o0Y/L0cxGx0vc0WKmeeSFxqA2jFs4GcXvrghnQBLE6RO9 dk+H00NAvtb2jqqR7R6YC2VZBeoL9ELiAQm+EWEN0cOURDCWlTEHs0h05KBVe225MzwslC2g1m8 nT/yc0f9kL X-Google-Smtp-Source: AGHT+IGPcAp6OD/mWCK0LW3yBUEoBKqsa6x4FF4vzrTz2ai9pGk0qdJo1hdlPtDHuObSgc78v3n4iA== X-Received: by 2002:a17:907:d93:b0:b73:3028:93a1 with SMTP id a640c23a62f3a-b767150b284mr3482698166b.9.1764592945081; Mon, 01 Dec 2025 04:42:25 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:24 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 20/28] sched/rt: Add rt-cgroup migration Date: Mon, 1 Dec 2025 13:41:53 +0100 Message-ID: <20251201124205.11169-21-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni When the runtime is exhausted in a RT CGroup, the scheduler checks for another non-throttled runqueue and, if available, migrates the tasks. The bandwidth (runtime/period) chosen for a certain cgroup is replicated on every core of the system, therefore, in an SMP system with M cores, the total available bandwidth is the given runtime/period multiplied by M. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/rt.c | 469 +++++++++++++++++++++++++++++++++++++++++++ kernel/sched/sched.h | 10 + 2 files changed, 479 insertions(+) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index a2084e9dc5..86f5602f30 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,3 +1,4 @@ +#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR @@ -84,6 +85,8 @@ void init_rt_rq(struct rt_rq *rt_rq) plist_head_init(&rt_rq->pushable_tasks); } =20 +static void group_pull_rt_task(struct rt_rq *this_rt_rq); + #ifdef CONFIG_RT_GROUP_SCHED =20 void unregister_rt_sched_group(struct task_group *tg) @@ -319,6 +322,46 @@ static inline void rt_queue_pull_task(struct rt_rq *rt= _rq) queue_balance_callback(rq, &per_cpu(rt_pull_head, rq->cpu), pull_rt_task); } =20 +#ifdef CONFIG_RT_GROUP_SCHED +static DEFINE_PER_CPU(struct balance_callback, rt_group_push_head); +static DEFINE_PER_CPU(struct balance_callback, rt_group_pull_head); +static void group_push_rt_tasks_callback(struct rq *); +static void group_pull_rt_task_callback(struct rq *); + +static void rt_queue_push_from_group(struct rt_rq *rt_rq) +{ + struct rq *rq =3D served_rq_of_rt_rq(rt_rq); + struct rq *global_rq =3D cpu_rq(rq->cpu); + + if (global_rq->rq_to_push_from) + return; + + if (!has_pushable_tasks(rt_rq)) + return; + + global_rq->rq_to_push_from =3D rq; + queue_balance_callback(global_rq, &per_cpu(rt_group_push_head, global_rq-= >cpu), + group_push_rt_tasks_callback); +} + +static void rt_queue_pull_to_group(struct rt_rq *rt_rq) +{ + struct rq *rq =3D served_rq_of_rt_rq(rt_rq); + struct rq *global_rq =3D cpu_rq(rq->cpu); + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (dl_se->dl_throttled || global_rq->rq_to_pull_to) + return; + + global_rq->rq_to_pull_to =3D rq; + queue_balance_callback(global_rq, &per_cpu(rt_group_pull_head, global_rq-= >cpu), + group_pull_rt_task_callback); +} +#else /* !CONFIG_RT_GROUP_SCHED */ +static inline void rt_queue_push_from_group(struct rt_rq *rt_rq) {}; +static inline void rt_queue_pull_to_group(struct rt_rq *rt_rq) {}; +#endif /* CONFIG_RT_GROUP_SCHED */ + static void enqueue_pushable_task(struct rt_rq *rt_rq, struct task_struct = *p) { plist_del(&p->pushable_tasks, &rt_rq->pushable_tasks); @@ -1721,6 +1764,432 @@ static void pull_rt_task(struct rq *this_rq) resched_curr(this_rq); } =20 +#ifdef CONFIG_RT_GROUP_SCHED +/* + * Find the lowest priority runqueue among the runqueues of the same + * task group. Unlike find_lowest_rt(), this does not mean that the + * lowest priority cpu is running tasks from this runqueue. + */ +static int group_find_lowest_rt_rq(struct task_struct *task, struct rt_rq = *task_rt_rq) +{ + struct sched_domain *sd; + struct cpumask mask, *lowest_mask =3D &mask; + struct sched_dl_entity *dl_se; + struct rt_rq *rt_rq; + int prio, lowest_prio; + int cpu, this_cpu =3D smp_processor_id(); + + if (task->nr_cpus_allowed =3D=3D 1) + return -1; /* No other targets possible */ + + lowest_prio =3D task->prio - 1; + cpumask_clear(lowest_mask); + for_each_cpu_and(cpu, cpu_online_mask, task->cpus_ptr) { + dl_se =3D task_rt_rq->tg->dl_se[cpu]; + rt_rq =3D &dl_se->my_q->rt; + prio =3D rt_rq->highest_prio.curr; + + /* + * If we're on asym system ensure we consider the different capacities + * of the CPUs when searching for the lowest_mask. + */ + if (dl_se->dl_throttled || !rt_task_fits_capacity(task, cpu)) + continue; + + if (prio >=3D lowest_prio) { + if (prio > lowest_prio) { + cpumask_clear(lowest_mask); + lowest_prio =3D prio; + } + + cpumask_set_cpu(cpu, lowest_mask); + } + } + + if (cpumask_empty(lowest_mask)) + return -1; + + /* + * At this point we have built a mask of CPUs representing the + * lowest priority tasks in the system. Now we want to elect + * the best one based on our affinity and topology. + * + * We prioritize the last CPU that the task executed on since + * it is most likely cache-hot in that location. + */ + cpu =3D task_cpu(task); + if (cpumask_test_cpu(cpu, lowest_mask)) + return cpu; + + /* + * Otherwise, we consult the sched_domains span maps to figure + * out which CPU is logically closest to our hot cache data. + */ + if (!cpumask_test_cpu(this_cpu, lowest_mask)) + this_cpu =3D -1; /* Skip this_cpu opt if not among lowest */ + + rcu_read_lock(); + for_each_domain(cpu, sd) { + if (sd->flags & SD_WAKE_AFFINE) { + int best_cpu; + + /* + * "this_cpu" is cheaper to preempt than a + * remote processor. + */ + if (this_cpu !=3D -1 && + cpumask_test_cpu(this_cpu, sched_domain_span(sd))) { + rcu_read_unlock(); + return this_cpu; + } + + best_cpu =3D cpumask_any_and_distribute(lowest_mask, + sched_domain_span(sd)); + if (best_cpu < nr_cpu_ids) { + rcu_read_unlock(); + return best_cpu; + } + } + } + rcu_read_unlock(); + + /* + * And finally, if there were no matches within the domains + * just give the caller *something* to work with from the compatible + * locations. + */ + if (this_cpu !=3D -1) + return this_cpu; + + cpu =3D cpumask_any_distribute(lowest_mask); + if (cpu < nr_cpu_ids) + return cpu; + + return -1; +} + +/* + * Find and lock the lowest priority runqueue among the runqueues + * of the same task group. Unlike find_lock_lowest_rt(), this does not + * mean that the lowest priority cpu is running tasks from this runqueue. + */ +static struct rt_rq *group_find_lock_lowest_rt_rq(struct task_struct *task= , struct rt_rq *rt_rq) +{ + struct rq *rq =3D rq_of_rt_rq(rt_rq); + struct rq *lowest_rq; + struct rt_rq *lowest_rt_rq =3D NULL; + struct sched_dl_entity *lowest_dl_se; + int tries, cpu; + + for (tries =3D 0; tries < RT_MAX_TRIES; tries++) { + cpu =3D group_find_lowest_rt_rq(task, rt_rq); + + if ((cpu =3D=3D -1) || (cpu =3D=3D rq->cpu)) + break; + + lowest_dl_se =3D rt_rq->tg->dl_se[cpu]; + lowest_rt_rq =3D &lowest_dl_se->my_q->rt; + lowest_rq =3D cpu_rq(cpu); + + if (lowest_rt_rq->highest_prio.curr <=3D task->prio) { + /* + * Target rq has tasks of equal or higher priority, + * retrying does not release any lock and is unlikely + * to yield a different result. + */ + lowest_rt_rq =3D NULL; + break; + } + + /* if the prio of this runqueue changed, try again */ + if (double_lock_balance(rq, lowest_rq)) { + /* + * We had to unlock the run queue. In + * the mean time, task could have + * migrated already or had its affinity changed. + * Also make sure that it wasn't scheduled on its rq. + * It is possible the task was scheduled, set + * "migrate_disabled" and then got preempted, so we must + * check the task migration disable flag here too. + */ + if (unlikely(is_migration_disabled(task) || + lowest_dl_se->dl_throttled || + !cpumask_test_cpu(lowest_rq->cpu, &task->cpus_mask) || + task !=3D pick_next_pushable_task(rt_rq))) { + + double_unlock_balance(rq, lowest_rq); + lowest_rt_rq =3D NULL; + break; + } + } + + /* If this rq is still suitable use it. */ + if (lowest_rt_rq->highest_prio.curr > task->prio) + break; + + /* try again */ + double_unlock_balance(rq, lowest_rq); + lowest_rt_rq =3D NULL; + } + + return lowest_rt_rq; +} + +static int group_push_rt_task(struct rt_rq *rt_rq, bool pull) +{ + struct rq *rq =3D rq_of_rt_rq(rt_rq); + struct task_struct *next_task; + struct rq *lowest_rq; + struct rt_rq *lowest_rt_rq; + int ret =3D 0; + + if (!rt_rq->overloaded) + return 0; + + next_task =3D pick_next_pushable_task(rt_rq); + if (!next_task) + return 0; + +retry: + if (is_migration_disabled(next_task)) { + struct task_struct *push_task =3D NULL; + int cpu; + + if (!pull || rq->push_busy) + return 0; + + /* + * If the current task does not belong to the same task group + * we cannot push it away. + */ + if (rq->curr->sched_task_group !=3D rt_rq->tg) + return 0; + + /* + * Invoking group_find_lowest_rt_rq() on anything but an RT task doesn't + * make sense. Per the above priority check, curr has to + * be of higher priority than next_task, so no need to + * reschedule when bailing out. + * + * Note that the stoppers are masqueraded as SCHED_FIFO + * (cf. sched_set_stop_task()), so we can't rely on rt_task(). + */ + if (rq->curr->sched_class !=3D &rt_sched_class) + return 0; + + cpu =3D group_find_lowest_rt_rq(rq->curr, rt_rq); + if (cpu =3D=3D -1 || cpu =3D=3D rq->cpu) + return 0; + + /* + * Given we found a CPU with lower priority than @next_task, + * therefore it should be running. However we cannot migrate it + * to this other CPU, instead attempt to push the current + * running task on this CPU away. + */ + push_task =3D get_push_task(rq); + if (push_task) { + preempt_disable(); + raw_spin_rq_unlock(rq); + stop_one_cpu_nowait(rq->cpu, push_cpu_stop, + push_task, &rq->push_work); + preempt_enable(); + raw_spin_rq_lock(rq); + } + + return 0; + } + + if (WARN_ON(next_task =3D=3D rq->curr)) + return 0; + + /* We might release rq lock */ + get_task_struct(next_task); + + /* group_find_lock_lowest_rq locks the rq if found */ + lowest_rt_rq =3D group_find_lock_lowest_rt_rq(next_task, rt_rq); + if (!lowest_rt_rq) { + struct task_struct *task; + /* + * group_find_lock_lowest_rt_rq releases rq->lock + * so it is possible that next_task has migrated. + * + * We need to make sure that the task is still on the same + * run-queue and is also still the next task eligible for + * pushing. + */ + task =3D pick_next_pushable_task(rt_rq); + if (task =3D=3D next_task) { + /* + * The task hasn't migrated, and is still the next + * eligible task, but we failed to find a run-queue + * to push it to. Do not retry in this case, since + * other CPUs will pull from us when ready. + */ + goto out; + } + + if (!task) + /* No more tasks, just exit */ + goto out; + + /* + * Something has shifted, try again. + */ + put_task_struct(next_task); + next_task =3D task; + goto retry; + } + + lowest_rq =3D rq_of_rt_rq(lowest_rt_rq); + + move_queued_task_locked(rq, lowest_rq, next_task); + resched_curr(lowest_rq); + ret =3D 1; + + double_unlock_balance(rq, lowest_rq); +out: + put_task_struct(next_task); + + return ret; +} + +static void group_pull_rt_task(struct rt_rq *this_rt_rq) +{ + struct rq *this_rq =3D rq_of_rt_rq(this_rt_rq); + int this_cpu =3D this_rq->cpu, cpu; + bool resched =3D false; + struct task_struct *p, *push_task =3D NULL; + struct rt_rq *src_rt_rq; + struct rq *src_rq; + struct sched_dl_entity *src_dl_se; + + for_each_online_cpu(cpu) { + if (this_cpu =3D=3D cpu) + continue; + + src_dl_se =3D this_rt_rq->tg->dl_se[cpu]; + src_rt_rq =3D &src_dl_se->my_q->rt; + + if (src_rt_rq->rt_nr_running <=3D 1 && !src_dl_se->dl_throttled) + continue; + + src_rq =3D rq_of_rt_rq(src_rt_rq); + + /* + * Don't bother taking the src_rq->lock if the next highest + * task is known to be lower-priority than our current task. + * This may look racy, but if this value is about to go + * logically higher, the src_rq will push this task away. + * And if its going logically lower, we do not care + */ + if (src_rt_rq->highest_prio.next >=3D + this_rt_rq->highest_prio.curr) + continue; + + /* + * We can potentially drop this_rq's lock in + * double_lock_balance, and another CPU could + * alter this_rq + */ + push_task =3D NULL; + double_lock_balance(this_rq, src_rq); + + /* + * We can pull only a task, which is pushable + * on its rq, and no others. + */ + p =3D pick_highest_pushable_task(src_rt_rq, this_cpu); + + /* + * Do we have an RT task that preempts + * the to-be-scheduled task? + */ + if (p && (p->prio < this_rt_rq->highest_prio.curr)) { + WARN_ON(p =3D=3D src_rq->curr); + WARN_ON(!task_on_rq_queued(p)); + + /* + * There's a chance that p is higher in priority + * than what's currently running on its CPU. + * This is just that p is waking up and hasn't + * had a chance to schedule. We only pull + * p if it is lower in priority than the + * current task on the run queue + */ + if (src_rq->curr->sched_task_group =3D=3D this_rt_rq->tg && + p->prio < src_rq->curr->prio) + goto skip; + + if (is_migration_disabled(p)) { + /* + * If the current task does not belong to the same task group + * we cannot push it away. + */ + if (src_rq->curr->sched_task_group !=3D this_rt_rq->tg) + goto skip; + + push_task =3D get_push_task(src_rq); + } else { + move_queued_task_locked(src_rq, this_rq, p); + resched =3D true; + } + /* + * We continue with the search, just in + * case there's an even higher prio task + * in another runqueue. (low likelihood + * but possible) + */ + } +skip: + double_unlock_balance(this_rq, src_rq); + + if (push_task) { + preempt_disable(); + raw_spin_rq_unlock(this_rq); + stop_one_cpu_nowait(src_rq->cpu, push_cpu_stop, + push_task, &src_rq->push_work); + preempt_enable(); + raw_spin_rq_lock(this_rq); + } + } + + if (resched) + resched_curr(this_rq); +} + +static void group_push_rt_tasks(struct rt_rq *rt_rq) +{ + while (group_push_rt_task(rt_rq, false)) + ; +} + +static void group_push_rt_tasks_callback(struct rq *global_rq) +{ + struct rt_rq *rt_rq =3D &global_rq->rq_to_push_from->rt; + + if ((rt_rq->rt_nr_running > 1) || + (dl_group_of(rt_rq)->dl_throttled =3D=3D 1)) { + + while (group_push_rt_task(rt_rq, false)) + ; + } + + global_rq->rq_to_push_from =3D NULL; +} + +static void group_pull_rt_task_callback(struct rq *global_rq) +{ + struct rt_rq *rt_rq =3D &global_rq->rq_to_pull_to->rt; + + group_pull_rt_task(rt_rq); + global_rq->rq_to_pull_to =3D NULL; +} +#else /* !CONFIG_RT_GROUP_SCHED */ +static void group_pull_rt_task(struct rt_rq *this_rt_rq) { } +static void group_push_rt_tasks(struct rt_rq *rt_rq) { } +#endif /* CONFIG_RT_GROUP_SCHED */ + /* * If we are not running and we are not going to reschedule soon, we should * try to push tasks away now diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 6c3fbfe84f..c8eac719eb 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1311,6 +1311,16 @@ struct rq { call_single_data_t cfsb_csd; struct list_head cfsb_csd_list; #endif + +#ifdef CONFIG_RT_GROUP_SCHED + /* + * Balance callbacks operate only on global runqueues. + * These pointers allow referencing cgroup specific runqueues + * for balancing operations. + */ + struct rq *rq_to_push_from; + struct rq *rq_to_pull_to; +#endif }; =20 #ifdef CONFIG_FAIR_GROUP_SCHED --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E53B831280D for ; Mon, 1 Dec 2025 12:42:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592950; cv=none; b=nGs/Ov4HTbwlz55jOo9oRhMPOSALQsmRO7rzAwNXgEbQRJfjOuhttVO8aIMJHo1CLwkyjsrdJU08LE1UqWeKg3mkBkZ27SSRGTkbgGZz1A6dB7ZgryMm6AHcNTdaIWJmbzjGaiAAPvXAF88Sg01NCJRgHkk/J0nf9F0C+wKjLFU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592950; c=relaxed/simple; bh=IxTICo5qwmnsAN3xzEAKO2lNXpDqF33T1HI5DrjBBMI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uiIFvYeFrmxxejuxNOz7gMDbUEUFwjzLFo7q9WnvIqmuVmMC2CFRWMzNP+px5JMvJiEBPRnCDsDEeUuDJF/LJ4x7TRwRNzAeQDRkT1ULZ1XejgSQrUsHGB3zOvQzNzTSx+Ownm5hijXsb9br5rbK4eqv1YWt2/NzKb/W6Du6igc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BZuLh3D4; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BZuLh3D4" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-b73875aa527so645875866b.3 for ; Mon, 01 Dec 2025 04:42:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592946; x=1765197746; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TbcefNdzzZCkICFoQKdfBmi4Qmz+Ypj1mBGRZo7Yg94=; b=BZuLh3D4IXb0nghp5wTqh1491ad+kWKLB5HKimhcAUFuJ4rQ+FyLVN/LLgHjZoJUWS NcfIDTT48415rgWSZFCojNdLK3rCJbBE8j3IQDjcfBYNNaTYOvhhpFQEm8ob3Hxis72j SB4o5VFRATMntPAULgbX3+vV0UT7IkSvEQ2NlsYO6Eb44WJDfj6f1Mg8MkywtSXlmKwB qTGuL+RxLBNrCi96vSIaj7RZKhPNikerBBttvUyd7F6xeDU/RW9wKn9qNuBw/HxDSYDE XTiprXVJB/hDUURVuZW/NdXM3znb44AkB+B7bUv29C1AbDG4eaqutD3kvdM/6M+Mcbs3 bvTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592946; x=1765197746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=TbcefNdzzZCkICFoQKdfBmi4Qmz+Ypj1mBGRZo7Yg94=; b=BZ3i+pp8pqD7evgATw3zbHK9gK9rsg092HqKq0XBv0vMMtvisamfm4wOqJaWRLknE/ BF7RcV9Dzgb+0b2pGi6ivO8bjEvMk8w5VzfF5hDEfizNbU0MEy4erKxyKXAJ1/g0bUX5 5TY4Bymgp0Q5mTNMC8dzXEI69k6INLIoj3oIpj+OS06UkBwSauYg4nPY6ykQO90uxA+Q JMksIGshFIcXk7DvNFH8UGvFVsS9AZWN6kbtDEnV1DhP5eBz1Sj0UxESJMihu0Sr76SA /yFXhtfpkFZSFN3c7WfCOJAytGmSWBVxvPMvPg/j2PaNzIi6xz7Q1+S1jOB/OS5OuHl4 m69Q== X-Gm-Message-State: AOJu0Yzs4R0Wbr0oPeXteZsQ0jdSs+C5qgOewnbwdImBCTRcmYXaXZlU b6XyU8EiwCnT7FzNTM4HVkJofLygghxpmmRRSROW0fRCeTe/JzbkUN4a X-Gm-Gg: ASbGncvO8cUX2MjO+bIwj/AM3+R+4HRDDe3ei9hzb6x+RdykAWfZdYOzM+TRj8bf7UD /H6sdt8YloQHPe2JuIODfroUkK7ermPgRvIVnBwHuD3NvcjtoeVLjv38/i/6X+63Uf1zY66lBRL pGkiQf5q8i2JGA8+MaF9yFzWf/PItp40ewstmSgJ664bRt37hg1HwKas2ug6LcF0aOaJx/BkrsY 4rmP0E6YTZTlofU8B4crK3xExxUHAqZrQ6sCQHKNe2GfGdrUco4ViQSMaenQs/dilmKqWjnilR1 rBilR4LI0Bt/HrCEO/aGwyWl7ZvWFIT2rtuSZw0wH4Hk2sNYWXhMwW6sCrnL0QVh+lG6mRGQpTb 4fCrWbXW2BNo+naUiRXrAhqrht0L5Rz4CDYj7D/T4WRFtr2x6YmF9gxPUQVeWpVAxrg7UbCL6gk d31hbrXxOe X-Google-Smtp-Source: AGHT+IFsLM/zJmyvcHdu8bOsoaw55txlpe9h8r3zmjaM68jw6WPLTyBd/JMG4KBAZ5R2gZU2pOWC4Q== X-Received: by 2002:a17:907:3dac:b0:b72:fd32:a463 with SMTP id a640c23a62f3a-b76715abd0dmr3799043566b.23.1764592945994; Mon, 01 Dec 2025 04:42:25 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:25 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 21/28] sched/rt: Add HCBS migration related checks and function calls Date: Mon, 1 Dec 2025 13:41:54 +0100 Message-ID: <20251201124205.11169-22-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add HCBS related checks and operations to allow rt-task migration, differentiating between cgroup's tasks or tasks that run on the global runqueue. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/rt.c | 61 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 47 insertions(+), 14 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 86f5602f30..e2b67f8309 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,4 +1,3 @@ -#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR @@ -892,6 +891,11 @@ select_task_rq_rt(struct task_struct *p, int cpu, int = flags) struct rq *rq; bool test; + /* Just return the task_cpu for processes inside task groups */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + is_dl_group(rt_rq_of_se(&p->rt))) + goto out; + /* For anything but wake ups, just return the task_cpu */ if (!(flags & (WF_TTWU | WF_FORK))) goto out; @@ -991,7 +995,10 @@ static int balance_rt(struct rq *rq, struct task_struc= t *p, struct rq_flags *rf) * not yet started the picking loop. */ rq_unpin_lock(rq, rf); - pull_rt_task(rq); + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq_of_se(&p->rt)= )) + group_pull_rt_task(rt_rq_of_se(&p->rt)); + else + pull_rt_task(rq); rq_repin_lock(rq, rf); } @@ -1100,7 +1107,10 @@ static inline void set_next_task_rt(struct rq *rq, s= truct task_struct *p, bool f if (rq->donor->sched_class !=3D &rt_sched_class) update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 0); - rt_queue_push_tasks(rt_rq); + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + rt_queue_push_from_group(rt_rq); + else + rt_queue_push_tasks(rt_rq); } static struct sched_rt_entity *pick_next_rt_entity(struct rt_rq *rt_rq) @@ -1153,6 +1163,13 @@ static void put_prev_task_rt(struct rq *rq, struct t= ask_struct *p, struct task_s */ if (on_rt_rq(&p->rt) && p->nr_cpus_allowed > 1) enqueue_pushable_task(rt_rq, p); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (dl_se->dl_throttled) + rt_queue_push_from_group(rt_rq); + } } /* Only try algorithms three times */ @@ -2196,6 +2213,7 @@ static void group_push_rt_tasks(struct rt_rq *rt_rq) = { } */ static void task_woken_rt(struct rq *rq, struct task_struct *p) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); bool need_to_push =3D !task_on_cpu(rq, p) && !test_tsk_need_resched(rq->curr) && p->nr_cpus_allowed > 1 && @@ -2203,7 +2221,12 @@ static void task_woken_rt(struct rq *rq, struct task= _struct *p) (rq->curr->nr_cpus_allowed < 2 || rq->donor->prio <=3D p->prio); - if (need_to_push) + if (!need_to_push) + return; + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + group_push_rt_tasks(rt_rq); + else push_rt_tasks(rq); } @@ -2243,7 +2266,9 @@ static void switched_from_rt(struct rq *rq, struct ta= sk_struct *p) if (!task_on_rq_queued(p) || rt_rq->rt_nr_running) return; - if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED) || !is_dl_group(rt_rq)) + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + rt_queue_pull_to_group(rt_rq); + else rt_queue_pull_task(rt_rq); } @@ -2272,6 +2297,13 @@ static void switched_to_rt(struct rq *rq, struct tas= k_struct *p) */ if (task_current(rq, p)) { update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 0); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq_of_se(&p->rt)= )) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq_of_se(&p->rt)); + + p->dl_server =3D dl_se; + } + return; } @@ -2281,13 +2313,10 @@ static void switched_to_rt(struct rq *rq, struct ta= sk_struct *p) * then see if we can move to another run queue. */ if (task_on_rq_queued(p)) { - if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { - if (p->prio < rq->curr->prio) - resched_curr(rq); - } else { - if (p->nr_cpus_allowed > 1 && rq->rt.overloaded) - rt_queue_push_tasks(rt_rq_of_se(&p->rt)); - } + if (!is_dl_group(rt_rq) && p->nr_cpus_allowed > 1 && rq->rt.overloaded) + rt_queue_push_tasks(rt_rq); + else if (is_dl_group(rt_rq) && rt_rq->overloaded) + rt_queue_push_from_group(rt_rq); if (p->prio < rq->donor->prio && cpu_online(cpu_of(rq))) resched_curr(rq); @@ -2311,8 +2340,12 @@ prio_changed_rt(struct rq *rq, struct task_struct *p= , int oldprio) * If our priority decreases while running, we * may need to pull tasks to this runqueue. */ - if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED) && oldprio < p->prio) - rt_queue_pull_task(rt_rq); + if (oldprio < p->prio) { + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + rt_queue_pull_to_group(rt_rq); + else + rt_queue_pull_task(rt_rq); + } /* * If there's a higher priority task waiting to run -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E26B231352B for ; Mon, 1 Dec 2025 12:42:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592951; cv=none; b=BLHXK7rgdRXrDWglmqnupXgo9QCw7beLsSb43+aEXs6i0ifUo+KOt9LdrJgYeK3uv8BXPwlcE68izvE67Ox+8GC3jVssoo3mqukNYsV5ngz6/F1CPMwvu7t8uaiIeCA43i/WgKkxobdZuOh09t829OrPB5XzHMMPU9rqKTXFhA8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592951; c=relaxed/simple; bh=nl/zj7XOo2Pt4AyqQMas/N8xeke7DeDOLWstQrQbvtM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oXO9pAgH9IsKWexU6Tdb51miPkzh9Zfm4fzjIlBsaxRVMehrVNZvfkwsesctoYg+bochgf4jnSZiJbxqde7+RLb9/iQERthBnDDmoF8GGRk2U7v2H71Dd5MnF4geuQhFSVofJrlx9eiBvRil21zQ9CYQyBbORbIW/MLnl769OCw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ijinxqWx; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ijinxqWx" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b7380f66a8bso630077266b.2 for ; Mon, 01 Dec 2025 04:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592947; x=1765197747; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CA7ZGgm2S1Wr54pv6s8ICnbsqpB7VrG8KXCm9pXfumM=; b=ijinxqWxNYsmyXqgNiuqv26iH6SW39bvB7dnP319xWaAktw84WuvJUzjNHAFywxbDG tkJH2z/000uKy3coGzWNaKg3150TyBN2qun9ddFQKGrb89Gq/PIfb61+A0Rmvnv3OSOJ fmzrlUdttHYSNhUDKnJ2DdyGjP8Ihoga1tziGiRpvDUnQWx5WDcnz2UrcNy6kL7V5ZeH +X0VuGTAN62kQtjD+XnIT5ctJLJhDvMdGTJEiW4J9A6ml/NMippSxFQlV9/0QwIkbO7Y rZ29q4Dv/Aub7hkfJjOTZ6snQuoGkOex1wbGrCfNcUdRLIHde/Mh1Ko4vhC6ZbBDbyPf kJig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592947; x=1765197747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CA7ZGgm2S1Wr54pv6s8ICnbsqpB7VrG8KXCm9pXfumM=; b=DxMOfOssNU6VkEgDNF9nYwzTlSP2I5h9BZ4kSD+ZNxB5Ngklw/5QXG7aLMlZ7kxLSz QUZrmL0HW5MRiTEpn7RYr+Mv7+GMvUH3TqdZkBQKPuqRiGrpq+VhKXv56k77j+ElFzcM Y2bFRGL8WQP3ls2/cXKOmvUHFb89x9o2oPN5DSnFr3x8zemO2rbc5OiaX0V+5EFRsKYg qVgAxlLAktUbqvYmf0Xk5E3Vc7dZWaSfTe/XohY5fa2M90M2oaQDlg0gUGv1K2I2bYzO 3g8oec9IrdNYZz0HkQBjiYlGWNQ/iUkraaR4xNRORQqDQVOVWw4adnWxp6LFy1V0uWFn sk2g== X-Gm-Message-State: AOJu0YzICbF7F6RKa48txhV2kVHOKOXFmSoe/yb/NDtzRK/cWPGmE09M FqGBtzprXJesj1HAyEmTS/xxO3ikSlUP4fcPCeW5pIWpOgBkTSVjNBwg X-Gm-Gg: ASbGncs7p9ewxgnDYHUfz51wZ7d6hcpKruhxLEutCzazOAAwJBd713FsR+QKs5J7agl DbS5T7MlOxIJv0ee///RqpQA0r1fj+Svx9Vo3hrOEa1gCPVE88bJqWNkqXgOO7FEL1jSeu2MLs7 wfQaLyeoul5Up+SiHELkqEAq773KDwAsskP/9dI/rbI09YP6QhNjbwuG/Qnkys/shEiQ+MjbNkU wcyOFvKBIUFvp9loHcXTM8Ps4XdteS4rWalFnaueb6JOYgIxmPHtnh0nhGHShQ7E2XpnbIa0Scy ylFYpv1NPq+h9Bmx/ngV+B9Gggl4OX34uUq1JTo71AdiUaw/kaeu4NoCiQu76fYecM9v1LuF7vf 9XBajPG0P4xVqL5V8sPhgNc32qsGH8nyXgsV7Q0YdflBy7avKxmnBrJ3/JIo6ifRUx9gcp4qz96 3+YK7K2kVz X-Google-Smtp-Source: AGHT+IGRn2UPC9lfEI3hwJk8oqsutLo9dymbtCkWVGRO+g7eEa/RaIxY5NwR5lINfRuzFrtzEsrFWw== X-Received: by 2002:a17:907:7203:b0:b76:3548:cdee with SMTP id a640c23a62f3a-b76715e630bmr4113962566b.25.1764592946869; Mon, 01 Dec 2025 04:42:26 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:26 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 22/28] sched/deadline: Introduce dl_server_try_pull_f Date: Mon, 1 Dec 2025 13:41:55 +0100 Message-ID: <20251201124205.11169-23-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce a new deadline server callback, dl_server_try_pull_f, that attempts to pull some tasks from other runqueues and returns true if after this operation the runqueue is not empty. This function is needed by some scheduling algorithms to guarantee that they are work-conserving (i.e. whenever there is an idle CPU and a ready task, this must be immediately scheduled there) or to enforce some other properties of the scheduling algorithm used on the served runqueue (for example, for a fixed-priority scheduler the m highest-priority tasks must be scheduled). The function is called whenever the dl_server_timer (the runtime replenishment timer) expires and the deadline server is recharged. The idea behind this callback is that since the deadline server is being unthrottled and is becoming able to serve its runqueue, it should pull tasks from the other runqueues (if there are no runnable tasks on its own runqueue, or if the tasks on its runqueue have low priority). The interface of the function provides a sched_dl_entity pointer, i.e. the deadline server and expects to return true if there are runnable tasks on that server (also just pulled from other runqueues), false otherwise. The replenishment timer callback, if there are no runnable tasks for a given server, will replenish its bandwidth and then stop it. This callback is not relevant for fair deadline servers. This fixes the test case where a single hog process, in a cgroup with reservation 10ms/100ms, runs (without this patch) on only one CPU, while (with this patch), by definition of the global Fixed Priority scheduling algorithm, it must run on all (up to a utilization of 100ms/100ms) the CPUs of the machine. Co-developed-by: luca abeni Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- include/linux/sched.h | 3 ++- kernel/sched/deadline.c | 12 ++++++++++++ kernel/sched/fair.c | 8 +++++++- kernel/sched/rt.c | 13 ++++++++++++- kernel/sched/sched.h | 6 ++++++ 5 files changed, 39 insertions(+), 3 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 9ef7797983..62b8586d4f 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -633,7 +633,7 @@ struct sched_rt_entity { #endif } __randomize_layout; =20 -typedef bool (*dl_server_has_tasks_f)(struct sched_dl_entity *); +typedef bool (*dl_server_try_pull_f)(struct sched_dl_entity *); typedef struct task_struct *(*dl_server_pick_f)(struct sched_dl_entity *); =20 struct sched_dl_entity { @@ -734,6 +734,7 @@ struct sched_dl_entity { struct dl_rq *dl_rq; struct rq *my_q; dl_server_pick_f server_pick_task; + dl_server_try_pull_f server_try_pull_task; =20 #ifdef CONFIG_RT_MUTEXES /* diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 082bccc30b..a588fe3bbf 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1295,6 +1295,7 @@ static const u64 dl_server_min_res =3D 1 * NSEC_PER_M= SEC; static enum hrtimer_restart dl_server_timer(struct hrtimer *timer, struct = sched_dl_entity *dl_se) { struct rq *rq =3D rq_of_dl_se(dl_se); + bool is_active; u64 fw; =20 scoped_guard (rq_lock, rq) { @@ -1309,6 +1310,15 @@ static enum hrtimer_restart dl_server_timer(struct h= rtimer *timer, struct sched_ if (!dl_se->dl_runtime) return HRTIMER_NORESTART; =20 + rq_unpin_lock(rq, rf); + is_active =3D dl_se->server_try_pull_task(dl_se); + rq_repin_lock(rq, rf); + if (!is_active) { + replenish_dl_entity(dl_se); + dl_server_stop(dl_se); + return HRTIMER_NORESTART; + } + if (dl_se->dl_defer_armed) { /* * First check if the server could consume runtime in background. @@ -1712,10 +1722,12 @@ void dl_server_stop(struct sched_dl_entity *dl_se) =20 void dl_server_init(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq, struct rq *served_rq, + dl_server_try_pull_f try_pull_task, dl_server_pick_f pick_task) { dl_se->dl_rq =3D dl_rq; dl_se->my_q =3D served_rq; + dl_se->server_try_pull_task =3D try_pull_task; dl_se->server_pick_task =3D pick_task; } =20 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 9c724d8232..dad46f6bd4 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8957,6 +8957,11 @@ static struct task_struct *__pick_next_task_fair(str= uct rq *rq, struct task_stru return pick_next_task_fair(rq, prev, NULL); } =20 +static bool fair_server_try_pull_task(struct sched_dl_entity *dl_se) +{ + return true; +} + static struct task_struct *fair_server_pick_task(struct sched_dl_entity *d= l_se) { return pick_task_fair(dl_se->my_q); @@ -8968,7 +8973,8 @@ void fair_server_init(struct rq *rq) =20 init_dl_entity(dl_se); =20 - dl_server_init(dl_se, &rq->dl, rq, fair_server_pick_task); + dl_server_init(dl_se, &rq->dl, rq, + fair_server_try_pull_task, fair_server_pick_task); } =20 /* diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index e2b67f8309..80580b48ab 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -134,6 +134,16 @@ void free_rt_sched_group(struct task_group *tg) static struct sched_rt_entity *pick_next_rt_entity(struct rt_rq *rt_rq); static inline void set_next_task_rt(struct rq *rq, struct task_struct *p, = bool first); =20 +static bool rt_server_try_pull(struct sched_dl_entity *dl_se) +{ + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + + if (dl_se->my_q->rt.rt_nr_running =3D=3D 0) + group_pull_rt_task(rt_rq); + + return dl_se->my_q->rt.rt_nr_running > 0; +} + static struct task_struct *rt_server_pick(struct sched_dl_entity *dl_se) { struct rt_rq *rt_rq =3D &dl_se->my_q->rt; @@ -235,7 +245,8 @@ int alloc_rt_sched_group(struct task_group *tg, struct = task_group *parent) dl_se->dl_density =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); dl_se->dl_server =3D 1; =20 - dl_server_init(dl_se, &cpu_rq(i)->dl, s_rq, rt_server_pick); + dl_server_init(dl_se, &cpu_rq(i)->dl, s_rq, + rt_server_try_pull, rt_server_pick); } =20 return 1; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index c8eac719eb..c069f6fef0 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -363,6 +363,11 @@ extern s64 dl_scaled_delta_exec(struct rq *rq, struct = sched_dl_entity *dl_se, s6 * * dl_se::rq -- runqueue we belong to. * + * dl_se::server_try_pull() -- used on bandwidth enforcement; the server= has a + * chance to pull tasks from the other runqueues, + * otherwise it is stopped if there is no task to + * run. + * * dl_se::server_pick() -- nested pick_next_task(); we yield the period = if this * returns NULL. * @@ -408,6 +413,7 @@ extern void dl_server_start(struct sched_dl_entity *dl_= se); extern void dl_server_stop(struct sched_dl_entity *dl_se); extern void dl_server_init(struct sched_dl_entity *dl_se, struct dl_rq *dl= _rq, struct rq *served_rq, + dl_server_try_pull_f try_pull_task, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); extern int dl_check_tg(unsigned long total); --=20 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93FD6313551 for ; Mon, 1 Dec 2025 12:42:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592951; cv=none; b=c3Ga4F2j8FEV/mIWnH6km2mOfuWyONqSqjW01ow10KFcvWm189vRBxncki3GIWHHi/HEKUEggENibK62RDogrRwSVGsAHIctWiNNBtOqgO50QfbFBq3DsNVBJwYOCu6Gl+kulUN4y/8oW0pWlc1DyTY8g+vblmI1IDOL1TTtVN0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592951; c=relaxed/simple; bh=zpl9lrHwMZiCuquS504AZENL8Zm2jy92zLoYZHBtoLA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kH6NX6IzZBfSyvqzHqxjCjKTiTEhgKSciHDQtA3YBiwodRiWX+fe4wxcxx3E6j/2I6xwToLqdVlOAvbnRNx8IL5aoAvAKUjBT/b0Y5VD0BuPTz+f+FRMHV6aK71XoZ2eYOrkLiCT8zNJPR0aC/6YNrTFc/hwLa2oMbe25rUtci0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=k58KmzFx; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="k58KmzFx" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b735ce67d1dso687606366b.3 for ; Mon, 01 Dec 2025 04:42:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592948; x=1765197748; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SmeURNsimak6rY7zL0sw+Dxd4FLsw174r8/qwrg2kaU=; b=k58KmzFxJhB0PEMXY8UEfoAa/sy8I3RGHRTtptSG7FTAaIHVMkjvuv0zlVh+hTaBVx rmKvb4qPI5jFSHV8EjBf8dhHRYcAO4rdu4XzHsRpFgYaQ/tW4YwV1tBHcy8EzBo5JPvc V7xC8qrVTslzbSXVNeIGTxxUx6U3PsLVLNY1r7PG+pOIPgAVHhbsh1gjMwFHhdXJYYyv 62prn1FDBCZ1JGnIidI9lgwO/6GJOowl9W6k2grHjjS65XIoIWJ4yMI5NlX6ug2IKE/f l0Xu8lhp/lDofc49FyIVwHb+nS+YqhWvSrSBdKdglt6bIj+O8SrYj4qfbBTDW/TGDbgh 73jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592948; x=1765197748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=SmeURNsimak6rY7zL0sw+Dxd4FLsw174r8/qwrg2kaU=; b=kUcjhMk9GZP89BGC1h7wC5EBGoKywnQe5GqQZE3cHI637p7eUCk7vSADweDsyHmENa qGCVVgYb8652HlVonJSN26GA/9OrK+XPD5s1pTXmW5MnIavIanTr6v7pR1u1V5dBH70L R+0aPcPJxYhbJS5nG+jgzUzU8s+L+2ojFNKJDjfx3rI//LkNZ1n/0DE241GXm00aX+yo oKmMCcbsP3YYTFwh+fUiI74J2CwMEkv7VmfsmGKB4LT6Wciri88s2r9Fb4r6Co7ToMV8 /yB6QECb+36v1L0un9CN0HiIOzQbFQxZSoVvq4V+vzaksQ410OWSlvllgjiFgoan6anq cIqA== X-Gm-Message-State: AOJu0YwrlzCx4RLqWO3EV5a7GzijniLZd1CQtHGteCZ8AfwVjbqyswuc qhvpPcG9eQLXcK68dN/FvgqcNMQg6cB/rZ9CR2kjjUYPs3U4SHb51Aaw X-Gm-Gg: ASbGnctnz/h2DOkZ7S9L2rXGiXv21Zb4APPnmWCnI4NcbjJwpKo0jtR6jvPe4byBD7a m5NkQQaGyFI45Gki+Q003uCrWBer1kXWNt8ovfPT5xw2kPBwNct1ZoHiMf6kF74UUmHp7du5Oaf TI2BxKL7304ninn8I6KIeD9Pf+3pdaA2EIoFtKUS/b3LLqtKX/H2lmRklN6ULI1v9y/+M5hK9rS tEuNuLtiFTISY1+AteW7DjNp3IDw4TMB9DCwqkIgJz4pW+oaCPwyrRHjgDkY90QEO0dfS2yOo0M rkB3f1TQN+iftYMrWs8nZeE9Z8PeXPUqDiuFrLaJ21ipj9ZBFpmZqIIbruj6Mgv42UdJwFohBI4 7aA+HQhJ3JVd+zSIpkTuYf3WlY19zE9kN4bS9AAaDarJVuhL1QOobRM+VZ9Xn79hNs6s/tdLlED AjwbF/tReC X-Google-Smtp-Source: AGHT+IEL9vsbaY5D5W2VAlsrYJ2xX/+d6m6QRI0IP1Cnq8iwRDjrnjoA/a+jsWNZthl0oaxbVICgQg== X-Received: by 2002:a17:907:6d10:b0:b77:1964:1881 with SMTP id a640c23a62f3a-b7719642737mr1280772066b.58.1764592947694; Mon, 01 Dec 2025 04:42:27 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:27 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 23/28] sched/deadline: Fix HCBS migrations on server stop Date: Mon, 1 Dec 2025 13:41:56 +0100 Message-ID: <20251201124205.11169-24-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Do not unthrottle a non-fair-server dl_server_stop(), since it ends up being stopped when throttled (we try to migrate all the RT tasks away from it). Notes: This is a temporary workaround, but it will be hopefully removed in favor of less invasive code. Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/deadline.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index a588fe3bbf..91588f3d2e 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1714,9 +1714,11 @@ void dl_server_stop(struct sched_dl_entity *dl_se) return; dequeue_dl_entity(dl_se, DEQUEUE_SLEEP); - hrtimer_try_to_cancel(&dl_se->dl_timer); + if (dl_se =3D=3D &rq_of_dl_se(dl_se)->fair_server) { + hrtimer_try_to_cancel(&dl_se->dl_timer); + dl_se->dl_throttled =3D 0; + } dl_se->dl_defer_armed =3D 0; - dl_se->dl_throttled =3D 0; dl_se->dl_server_active =3D 0; } -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B521A31062E for ; Mon, 1 Dec 2025 12:42:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592953; cv=none; b=Qp8uITlr8Xe6NeAW1BdWqoxZ9zCERUasCIz5kZVXFwMwRD4nCYnoSZ4x6xLhtr9PqJknaoBD6WO4OeWxaG/QMI57S5EEghVyRFav+KOGYjejavoV0bpmwwbNc2MMv07UIP5Cqx2Gg70RjR5K01DMxh1+2riE54Tata78v9MorYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592953; c=relaxed/simple; bh=apBDXQWtxNCJR6bL8tBtybLIUn1s8pXxXJIoKUni/D0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hg+WCHidWDL3y3L1/C4d3uUx0mZNh2ZUPMaZtltXPm1nVDkINS4mvY0f0oGv+CxYnP58rXN133l1XCOgpwn9MmPCMx4E3qs9CmTKtf7OoWpc566nkUswgKkVX0nM6U5xwy2azqg7kyRfR1Tw+KnoZvjkorcvwuDk6wQpyWECB7s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TiyR0csk; arc=none smtp.client-ip=209.85.208.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TiyR0csk" Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-6419aaced59so6187581a12.0 for ; Mon, 01 Dec 2025 04:42:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592949; x=1765197749; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/gXTXiM+VuRz2Zb4Y6AlTWBmDw4CXW0a5IFGDZK1PxY=; b=TiyR0csklwS/mgc9Vc/ubkODHX619enhyacoawswB5HKBEOD+YLIjd1augJ5rK9p3D 21XzcMwzyRuTdAvCk4Cs2zTB080V/SC5llAgnO6TaU17gBZ3IRNqMEd6dObbIWDx9ji2 lq4MQA2hkDcYPdoUo7SHIoOGYxGxNMjB/LzDEjkUY2vo5xb1Qbxu33h1g8vQaFMa2VkL 2avqnzyRgmyxVTK6JcrAOzhsHISLXGz68OMVXlDMp7/34pv34nXdy3vmF5QsZdsxIkin ZBUdP7sF1Gq9TlufGhfT/1RtR1bdSzHiG2GE8CXD1wt7JjcfOQeJnD60iS9EhC6PY7rY UJuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592949; x=1765197749; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=/gXTXiM+VuRz2Zb4Y6AlTWBmDw4CXW0a5IFGDZK1PxY=; b=DFKut2jyfN1BKpLmsxFF2jDxgq6sr8LlT0ktHd6YNMwUt+Qvq/fC66QXXyizzw2GrI hUyrurjOHZ/84LhY1+iucm0nLEbyq+3A/G2E+3l56VD9ldktDnls/h3tEnxMP3eG8WD1 FqngL9T1s3tdBTU6BPoceKJyGCZQ9hVA2kgI0VdI5/D89uvkpdhYn4ayWDf6+8cIbuJb HcWmwmy1o1zF1z+VVGeUiuY6BBmPEqSOM7LJSXiIXka/QMZ0rWAlLHqn1zdreBBRgPhH +C1D3cR0h8imk1bw5xiK9hwH34oA0TaWAIyLwG0uPhophCsdyo4iWHNX8h0sPxsipCXW +9mw== X-Gm-Message-State: AOJu0YymmlzDNXQQZ1ZJJFCj4FkHsgxPP5Qt1aORYo7206rq1OeP3+ls 09O5eUd+Y0UBmO9dOS08IuGztphLc61RNoyRWQIof+n5RIfuSo+yg4IH X-Gm-Gg: ASbGncu2tWf2Dakk25zmiwultK8q7QePPPLEuoWOZ+uyDzKUMn1WWhr+tHNRH6nzrVB TUAXA/4uuFroLu1Y0p0bVH1Jv9yN3ZshrJCNXP/0VcMB1pigW38GjkL6s2LvqrE4qYAqcawWrga +D6ZlCTw/teWiBEVaRcYW3ePfbNh8G8JPSfELMtvanDF+XkOuMfkiGB7f0tEdid/feDjPRmtlIO mlQ4UdYR6AF1tD3CIjsvFF+DtCoobagUIOUM+RZUGczlkfQOoL54hgUG48vdo7AkfAYQfhoc38Z jEvLj6UnD1ziIStHhGjEwZjwPvGf0iIHZXHrpzw5tVJJ7VlYm1nxJ+xY9/dSbsyL4R8O7nHPB4s R3Tdx+QKiMVR8QhYpeinMJ+k4epcKltOBDyQJHexyLbUa6AdrL5vkbdgzE3/3AeZjBDMxNhMypY VCuGg67L7D X-Google-Smtp-Source: AGHT+IHu/oPE1WrdASzcdY8P4h5akCVYBd7T6j6/eOXhOSygoG1UjI/TlrE7ZF0iJejgDtwd6/dN8Q== X-Received: by 2002:a17:907:3f14:b0:b71:5079:9702 with SMTP id a640c23a62f3a-b76715ab943mr3856301066b.21.1764592948587; Mon, 01 Dec 2025 04:42:28 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:28 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 24/28] sched/core: Execute enqueued balance callbacks when changing allowed CPUs Date: Mon, 1 Dec 2025 13:41:57 +0100 Message-ID: <20251201124205.11169-25-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Execute balancing callbacks when setting the affinity of a task, since the HCBS scheduler may request balancing of throttled dl_servers to fully utilize the server's bandwidth. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 983cd1b478..2c08f31d3d 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2915,6 +2915,7 @@ static int affine_move_task(struct rq *rq, struct tas= k_struct *p, struct rq_flag if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask) || (task_current_donor(rq, p) && !task_current(rq, p))) { struct task_struct *push_task =3D NULL; + struct balance_callback *head; if ((flags & SCA_MIGRATE_ENABLE) && (p->migration_flags & MDF_PUSH) && !rq->push_busy) { @@ -2933,11 +2934,13 @@ static int affine_move_task(struct rq *rq, struct t= ask_struct *p, struct rq_flag } preempt_disable(); + head =3D splice_balance_callbacks(rq); task_rq_unlock(rq, p, rf); if (push_task) { stop_one_cpu_nowait(rq->cpu, push_cpu_stop, p, &rq->push_work); } + balance_callbacks(rq, head); preempt_enable(); if (complete) @@ -2992,6 +2995,8 @@ static int affine_move_task(struct rq *rq, struct tas= k_struct *p, struct rq_flag } if (task_on_cpu(rq, p) || READ_ONCE(p->__state) =3D=3D TASK_WAKING) { + struct balance_callback *head; + /* * MIGRATE_ENABLE gets here because 'p =3D=3D current', but for * anything else we cannot do is_migration_disabled(), punt @@ -3005,16 +3010,19 @@ static int affine_move_task(struct rq *rq, struct t= ask_struct *p, struct rq_flag p->migration_flags &=3D ~MDF_PUSH; preempt_disable(); + head =3D splice_balance_callbacks(rq); task_rq_unlock(rq, p, rf); if (!stop_pending) { stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, &pending->arg, &pending->stop_work); } + balance_callbacks(rq, head); preempt_enable(); if (flags & SCA_MIGRATE_ENABLE) return 0; } else { + struct balance_callback *head; if (!is_migration_disabled(p)) { if (task_on_rq_queued(p)) @@ -3025,7 +3033,12 @@ static int affine_move_task(struct rq *rq, struct ta= sk_struct *p, struct rq_flag complete =3D true; } } + + preempt_disable(); + head =3D splice_balance_callbacks(rq); task_rq_unlock(rq, p, rf); + balance_callbacks(rq, head); + preempt_enable(); if (complete) complete_all(&pending->done); -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 832DE30E0F3 for ; Mon, 1 Dec 2025 12:42:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592953; cv=none; b=nE5f7F6eeZ9rPtzFfKaNPTA5ppbzUg5Qn0gDOKtHFZQuKNbWdd1OO4k36Sv2dpbvt+qRekFKMsBHTjSZCh5s3UnrxbpuUvi5jRj7nPgFJoLkFQrDMQ6mKp7sGEckIaQ9Mcpb0G+Z+3LRtbhBBEMYXVf2T5BT/2pX/1iyskw1rCU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592953; c=relaxed/simple; bh=RddOlNBZm39rECcZNVnsP3ouHtvi5PSny7clRLUCeE0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NewvXIulejCTjhxaXS84AwMV5U8GUr7JvWlMqmHWRcUNiNx7ykHDX2PE99AKiHexRlASt7V4iOmVmExol2/aFvJBz0hoeVkifLo8nVlAprazuwgoBBuwPPsnei5nEBWRiqwU3A3xMoaLnGrcuKse7I8s2Js2Tm2zdOUZhJXjP3c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gNapT3RW; arc=none smtp.client-ip=209.85.218.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gNapT3RW" Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-b7697e8b01aso809499066b.2 for ; Mon, 01 Dec 2025 04:42:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592950; x=1765197750; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CSTJW8ky8tnddPe6hnHZZVmAi+gkFOihScO4c+NAyRg=; b=gNapT3RWZjLhfZdgkx1QLprZIGawpMp6XY6BDCU3ybq72m/YKVrwLZp1ulMuQHpD+m 0xBcWRC8Zfp7rk9G0xebvj7HSooE0XbS36XmYL0Sxnn1Dxwatg0yXo4jJkpXuuvlwJdK dRNsAy7pCUk0YzKona5V8lpDgzU0nMT0FWZw3BMX3+DAaMsv9H9JaHoqMvzk/pW8+9/4 86p/TZvC8AaCA0NbyhYD+7AJfKC36eINMvALCDUCLlEwZ0BmeB+dE0L7t9qa4U8zoab8 lkzlQh3IugP1RAEVTmI/oDkRMip+MrJob8UvictEvnlalVxujVA6H3tKJmxdxtBjpikn Wgrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592950; x=1765197750; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CSTJW8ky8tnddPe6hnHZZVmAi+gkFOihScO4c+NAyRg=; b=Sp6vDQpMHHv5B6xo3jd945BU/AYPRI0dh+Yek1GBh/do7aOpIUqTSpkT44GkXneKK8 CHNdbqpbg+A9/+3pogbfzPnflv3IzdRVIcUIRwBF5SRWSr4eHqjUfe9DFyqLLJ589fL+ XTmvyQdU9SMHQkXilQFM0TmvTN7t1acnZFg6+ShJea606A3URQBwVX7fl++fboNIQqcl RP9VuthvlBdmVjGGjLT3ZYuIc9GnycnPxTLaP/fdSt1WhbgKaR6nAOTBTxj+y2UOHeoL 93AF2TzZAO+xn+vnBE44nGIsrbpyKwCJfsHM2rcwit0H6rblfqKgak++M4ezzkCf9YOV Ds4w== X-Gm-Message-State: AOJu0YxEYWkiPreYbvbnjDE850r5mtUW3bIIjKNAXAAWB6CEGMHyeejF 1k74uRzQgnXJHznunRVes2ysuZldJ4nb6KrgoBxADlOhxoAkZmKj61Ha X-Gm-Gg: ASbGncuJwC62bfGZjPUhhLKp+lA/i76690IwH/mJIV8V+inc4pAtIFkSuN//QffC7Y0 hAR/jmtF3I8xI4TwZ0Ir39QiYMhPNVX17hfeq9T0FtFKZhm42qWdPEYe+MY1cvAugvxJEGmUXfC 2e7YgasfSWDk67gVmn6Za5kEJ3wIWsFOXsQnZD9uMbOnQUP96Gb8c5QddepPR/9qIWWDArFmgsg hzv8EmLYr/5UNHhYpPZ/z7oeVGOmraUswL/0D2UHGLIO/iRzNWPik8nu7yNTRxNL7puuaawR5II QJKdFrqvsJ+pjAl0hiHIs9VVjijaCYejJyf5bo8DZUhhKkokZPcHl0QHMwrXJxKEd9Gyolw9caj vIneKXxZvT9ZfMWcPLAopmBWt38bNr4hUiMf9xaF53+QjrJZQkzwP98gQAC4CjEMCnTbX67WROR 4oTMAnYenK X-Google-Smtp-Source: AGHT+IGcXHNyqXQv0U0T+TdsGsB/Kd7MUOeNAH0e2XntWuL+BfzEULA/yo18IKrNgBAXXml2+0Z+GQ== X-Received: by 2002:a17:907:3da8:b0:b6d:50f7:a805 with SMTP id a640c23a62f3a-b76719d0791mr4388941066b.59.1764592949430; Mon, 01 Dec 2025 04:42:29 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:29 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 25/28] sched/core: Execute enqueued balance callbacks when migrating task betweeen cgroups Date: Mon, 1 Dec 2025 13:41:58 +0100 Message-ID: <20251201124205.11169-26-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Execute balancing callbacks when migrating task between cgroups, since the HCBS scheduler, similarly to the previous patch, may request balancing of throttled dl_servers to fully utilize the server's bandwidth. Introduce the RELEASE_LOCK helper macro to explicitly unlock a guard-based lock. The macro calls the destructor function of the lock and invalidates it so that the lock is not unlocked twice when the lock variable goes out of scope. Signed-off-by: Yuri Andriaccio --- include/linux/cleanup.h | 3 +++ kernel/sched/core.c | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/include/linux/cleanup.h b/include/linux/cleanup.h index 2573585b7f..65c222b308 100644 --- a/include/linux/cleanup.h +++ b/include/linux/cleanup.h @@ -518,4 +518,7 @@ __DEFINE_LOCK_GUARD_0(_name, _lock) #define DEFINE_LOCK_GUARD_1_COND(X...) CONCATENATE(DEFINE_LOCK_GUARD_1_CON= D_, COUNT_ARGS(X))(X) +#define RELEASE_LOCK(_name, _var) \ + ({ class_##_name##_destructor(&_var); no_free_ptr(_var.lock); }) + #endif /* _LINUX_CLEANUP_H */ diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2c08f31d3d..f69480243e 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9223,6 +9223,7 @@ void sched_move_task(struct task_struct *tsk, bool fo= r_autogroup) { int queued, running, queue_flags =3D DEQUEUE_SAVE | DEQUEUE_MOVE | DEQUEUE_NOCLOCK; + struct balance_callback *head; struct rq *rq; CLASS(task_rq_lock, rq_guard)(tsk); @@ -9253,6 +9254,12 @@ void sched_move_task(struct task_struct *tsk, bool f= or_autogroup) */ resched_curr(rq); } + + preempt_disable(); + head =3D splice_balance_callbacks(rq); + RELEASE_LOCK(task_rq_lock, rq_guard); + balance_callbacks(rq, head); + preempt_enable(); } static struct cgroup_subsys_state * -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0B6F314A8F for ; Mon, 1 Dec 2025 12:42:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592956; cv=none; b=HARzl6N7+8IsPZ4KB8eb8GlY3D6BMfJdZtI8foaBZasztcrT+aoT7wJelhrwl5ESEAzI+Yr6h0mSoRk03TF0VPuxPGlDt2dCGmGm66LutiYmXAx8UzOCjxxpOozZg3UfT6nM7XTHdMo+U+BdCXuyHYUCMQ9ABKoOPWF8mSCDdhQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592956; c=relaxed/simple; bh=3kD8/0vC8COcMppQkPHYbD3ew+AHoKKbw49D25FD9f0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EHfTgCkUzYS7oUvI8uv18UrpixaRG/BCJV6Vp1m4kb7WJWKNJOo8jEjtimHNs9qQXZ2avXBTqFTkz4iXoJHxAuUtEFvTqwck+VTMcuogqV6cGjUkw5eFz1PrQhjb+Nky4gJyqmKmQY6a+J93p8Dkf3yqSJDQ+H0jvDcz43fnDeg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=b++cUUVj; arc=none smtp.client-ip=209.85.208.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="b++cUUVj" Received: by mail-ed1-f50.google.com with SMTP id 4fb4d7f45d1cf-640ca678745so7183443a12.2 for ; Mon, 01 Dec 2025 04:42:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592951; x=1765197751; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VynGhLyLnCtRDFI80gOjWbt3RNlLcXHqh6uWjY2TOIo=; b=b++cUUVj28b7QdfYwFvDD/r9qD+NWReLJiROwMinwhDmA36Umga/1sMQJMDDiYM6Yb auNxxC6j8wcYIRH0HtWL3xe9MoJFihULWzVWbQnuxkJPCqqjIbh3aPD/WQE8fY+joRop ezTnoMjhN7G/no+Mp/zdVnh71b3cZSilwwyyh1hgGm2VRV5vFGL5RchsSQ1QHmOGtZ2Q gnRxBPSFvdIYZERS89I/+JjnnSa08/+xHemc9Ek9LncfZJGsZi9NhACYTBCi8uhBhsTM oC0xrREK88FqC1zl6fbfaoL1kw7ECcTZolaWpl4YEwDnW7gvgIYmTzyYKNaWvuhdPO+q LsOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592951; x=1765197751; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=VynGhLyLnCtRDFI80gOjWbt3RNlLcXHqh6uWjY2TOIo=; b=SOghgJgBgL0hCPNsS+OBseByFpWE2zfT/oyBhTMjPM/xCCI6qW8H4+/4afHBtmGYmf vpZENxmtQZf3Z6x3cFSvP28kuMIi7uQl+oBJ787cHayx3pHu4CYGO8j1+6M2Pyeoc33v 67ST7VUsUfHQqIgLRBl328j94mUPxvWqZ77iHW6+lW3oNfHxthFpLdmFVmjxL4OtUoI+ FXPkWtrzp8FckHccP6kJ9gNuUOKkFvVEHyhETwnJHxOwtEuZZXcw6Z50u6l/Lzlzl9x6 G9LiyGzMcLv2oJO3CyV8w4t3167y1iaPpnySJVfHuc1ztJ9/svD7Xz8ejHsC21Xb7Pq0 1trg== X-Gm-Message-State: AOJu0YzpN41Bp8yYYF3kYaOPpHRnisvIiiFRiyQ2IKDRxSKQzfxg0cGZ k1zI1kDBnnxgzPEQzVrwDe81YPMRy5blb6m9va9NqYnjplvEeVqM3rPX X-Gm-Gg: ASbGncv1f3jEtSQfPb0YLLrZllsXSM07bnFGHS8W+jgdOiFPJxnaD7XZ/RvXmdYglo+ DuJWCZHAjW+HA1HjyykjUyRfPfPc2r8CvkKM4KtEguGoeIBiUJgg+l6EFAll5GxBJZsC5ayryp/ HC85Rw7ZGYdw4CD8Oq3jlN229q/KEp/5mQSJBoybUAPqBPcBPJPDoGi+tw2J7Ds+Ld56LgLxEeC 0FlZbbt76HdV19ds2vtuGy5BtUVDinBBHh0rJyv0mG/nlj+9mfYiNKxHq4pNUyzJbUQvEwd5WcN EhABpzhplCsjimaX4WTtz8+VAfy3lTztnV7qSV6rQ/iDzS6oSRRxNh8cNjP4Gi5hq2mkcPp9Hd9 TFMecI0iAqRmAyVXVqCDIS82ZGK9cOawzdL/LiGS2vpnXKNi+4FlntlAvOJKDb9AqkM6qmQSLhP lguSsCpfr2CnQJX9oJ/QU= X-Google-Smtp-Source: AGHT+IFEdo2RA+warFuEtNhkPY15DCVdmxNetZsYDhkWI9DDCFflVPPNXSfKQ7D3gl/EfdkdPIAWbQ== X-Received: by 2002:a17:907:7282:b0:b73:278a:a499 with SMTP id a640c23a62f3a-b76715654bbmr4582001566b.15.1764592950575; Mon, 01 Dec 2025 04:42:30 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:29 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 26/28] Documentation: Update documentation for real-time cgroups Date: Mon, 1 Dec 2025 13:41:59 +0100 Message-ID: <20251201124205.11169-27-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Update the RT_GROUP_SCHED specific documentation. Give a brief theoretical background for Hierarchical Constant Bandwidth Server (HCBS). Document how the HCBS is implemented in the kernel and how the RT_GROUP_SCHED behaves now compared to the version which this patchset replaces. Signed-off-by: Yuri Andriaccio --- Documentation/scheduler/sched-rt-group.rst | 500 ++++++++++++++++++--- 1 file changed, 426 insertions(+), 74 deletions(-) diff --git a/Documentation/scheduler/sched-rt-group.rst b/Documentation/sch= eduler/sched-rt-group.rst index ab464335d3..a5a9203355 100644 --- a/Documentation/scheduler/sched-rt-group.rst +++ b/Documentation/scheduler/sched-rt-group.rst @@ -53,9 +53,12 @@ CPU time is divided by means of specifying how much time= can be spent running in a given period. We allocate this "run time" for each real-time group wh= ich the other real-time groups will not be permitted to use. -Any time not allocated to a real-time group will be used to run normal pri= ority -tasks (SCHED_OTHER). Any allocated run time not used will also be picked u= p by -SCHED_OTHER. +Each real-time group runs at the same priority as SCHED_DEADLINE, thus they +share and contend the SCHED_DEADLINE allowed bandwidth. Any time not alloc= ated +to a real-time group (and SCHED_DEADLINE tasks) will be used to run both +SCHED_FIFO/SCHED_RR, normal priority tasks (SCHED_OTHER), and SCHED_EXT ta= sks, +following the usual priorities. Any allocated run time not used will also = be +picked up by the other scheduling classes, in the same order as before. Let's consider an example: a frame fixed real-time renderer must deliver 25 frames a second, which yields a period of 0.04s per frame. Now say it will= also @@ -73,10 +76,6 @@ The remaining CPU time will be used for user input and o= ther tasks. Because real-time tasks have explicitly allocated the CPU time they need to perform their tasks, buffer underruns in the graphics or audio can be eliminated. -NOTE: the above example is not fully implemented yet. We still -lack an EDF scheduler to make non-uniform periods usable. - - 2. The Interface =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D @@ -86,40 +85,92 @@ lack an EDF scheduler to make non-uniform periods usabl= e. The system wide settings are configured under the /proc virtual file syste= m: -/proc/sys/kernel/sched_rt_period_us: +``/proc/sys/kernel/sched_rt_period_us``: The scheduling period that is equivalent to 100% CPU bandwidth. -/proc/sys/kernel/sched_rt_runtime_us: - A global limit on how much time real-time scheduling may use. This is al= ways - less or equal to the period_us, as it denotes the time allocated from the - period_us for the real-time tasks. Without CONFIG_RT_GROUP_SCHED enabled, - this only serves for admission control of deadline tasks. With - CONFIG_RT_GROUP_SCHED=3Dy it also signifies the total bandwidth availabl= e to - all real-time groups. +``/proc/sys/kernel/sched_rt_runtime_us``: + A global limit on how much time real-time scheduling may use (SCHED_DEAD= LINE + tasks + real-time groups). This is always less or equal to the period_us= , as + it denotes the time allocated from the period_us for the real-time tasks. + Without **CONFIG_RT_GROUP_SCHED** enabled, this only serves for admission + control of deadline tasks. With **CONFIG_RT_GROUP_SCHED=3Dy** it also si= gnifies + the total bandwidth available to both real-time groups and deadline task= s. * Time is specified in us because the interface is s32. This gives an operating range from 1us to about 35 minutes. - * sched_rt_period_us takes values from 1 to INT_MAX. - * sched_rt_runtime_us takes values from -1 to sched_rt_period_us. + * ``sched_rt_period_us`` takes values from 1 to INT_MAX. + * ``sched_rt_runtime_us`` takes values from -1 to ``sched_rt_period_us``. * A run time of -1 specifies runtime =3D=3D period, ie. no limit. - * sched_rt_runtime_us/sched_rt_period_us > 0.05 inorder to preserve + * ``sched_rt_runtime_us/sched_rt_period_us`` > 0.05 inorder to preserve bandwidth for fair dl_server. For accurate value check average of - runtime/period in /sys/kernel/debug/sched/fair_server/cpuX/ - - -2.2 Default behaviour ---------------------- - -The default values for sched_rt_period_us (1000000 or 1s) and -sched_rt_runtime_us (950000 or 0.95s). This gives 0.05s to be used by -SCHED_OTHER (non-RT tasks). These defaults were chosen so that a run-away -real-time tasks will not lock up the machine but leave a little time to re= cover -it. By setting runtime to -1 you'd get the old behaviour back. - -By default all bandwidth is assigned to the root group and new groups get = the -period from /proc/sys/kernel/sched_rt_period_us and a run time of 0. If you -want to assign bandwidth to another group, reduce the root group's bandwid= th -and assign some or all of the difference to another group. + runtime/period in ``/sys/kernel/debug/sched/fair_server/cpuX/`` + +The default value for ``sched_rt_period_us`` is 1000000 (or 1s) and for +``sched_rt_runtime_us`` is 950000 (or 0.95s). This gives a minimum of 0.05= s to +be used by SCHED_FIFO/SCHED_RR and non-RT tasks (SCHED_OTHER, SCHED_EXT), = while +0.95s are the maximum to be used by SCHED_DEADLINE, and rt-cgroups if enab= led. + +2.2 Cgroup settings +------------------- + +Enabling **CONFIG_RT_GROUP_SCHED** lets you explicitly allocate real CPU +bandwidth to task groups. + +This uses the cgroup virtual file system and the CPU controller for cgroup= s. +Enabling the controller for the hierarchy creates two files: + +* ``/cpu.rt_period_us``, the scheduling period of the group. +* ``/cpu.rt_runtime_us``, the maximum runtime each CPU will provide + every period. + + .. tip:: + For more information on working with control groups, you should read + *Documentation/admin-guide/cgroup-v1/cgroups.rst* as well. + .. + +By default the root cgroup has the same period of +``/proc/sys/kernel/sched_rt_period_us``, which is 1s, and a runtime of zer= o, so +that rt-cgroup is *soft-disabled* by default, and all the runtime is avail= able +for SCHED_DEADLINE tasks only. New groups instead get both period and runt= ime of +zero. + +2.3 Cgroup Hierarchy and Behaviours +----------------------------------- + +With HCBS, cgroups may act either as task runners or bandwidth reservation: + +* A bandwidth reservation cgroup (such as the root control group), has the + purpose to reserve a portion of the total real-time bandwidth for its su= b-tree + of groups. A group in this state cannot run SCHED_FIFO/SCHED_RR tasks. + + .. important:: + The *root control group* behaviour is different from the other cgroups= , as + its job is to reserve bandwidth for the whole group hierarchy, but it = can + also run rt tasks. This is an exception: FIFO/RR tasks running in the + root cgroup follow the same rules as FIFO/RR tasks in a kernel which h= as + **CONFIG_RT_GROUP_SCHED=3Dn**, and the bandwidth reservation is instea= d a + feature connected to HCBS, that acts on the cgroup tree. + .. + +* A *live* group instead can be used to run FIFO/RR tasks, with the given + bandwidth parameters: each CPU is served a *potentially continuous* runt= ime of + ``/cpu.rt_runtime_us`` every period ``/cpu.rt_period_us`= `. It + is important to notice that increasing the period but leaving the bandwi= dth + constant changes the behaviour of the cgroup's servers, as the bandwidth= given + overall is the same, but it is given in longer bursts (and longer slices= of no + bandwidth). + +More specifically on *live* and non-*live*: + +* A group is deemed *live* if it is a leaf of the groups' hierarchy or all= of + its children have runtime 0. +* *Live* groups are the only groups allowed to run real-time tasks. A SCHE= D_FIFO + task cannot be migrated in a non-*live* group, neither a task inside this + group can change scheduling policy to SCHED_FIFO/SCHED_RR if the group i= s not + *live*. +* Non-*live* groups are only used for bandwidth reservation. +* Group's bandwidth follow this invariant: the sum of the bandwidths of a + group's children is always less than or equal to the group's bandwidth. Real-time group scheduling means you have to assign a portion of total CPU bandwidth to the group before it will accept real-time tasks. Therefore yo= u will @@ -128,63 +179,364 @@ done that, even if the user has the rights to run pr= ocesses with real-time priority! -2.3 Basis for grouping tasks ----------------------------- +3. Theoretical Background +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D + + + .. BIG FAT WARNING ****************************************************** + + .. warning:: + + This section contains a (not-thorough) summary on deadline/hierarchical + scheduling theory, and how it applies to real-time control groups. + The reader can "safely" skip to Section 4 if only interested in seeing + how the scheduling policy can be used. Anyway, we strongly recommend + to come back here and continue reading (once the urge for testing is + satisfied :P) to be sure of fully understanding all technical details. + + .. **********************************************************************= ** + +The real-time cgroup scheduler is based upon the **Hierarchical Constant +Bandwidth Server** (HCBS) [1] *Compositional Scheduling Framework* (CSF). A +**CSF** is a framework where global (system-level) timing properties can be +established by composing independently (specified and) analyzed local +(component-level) timing properties [5]. + +For HCBS (related to the Linux kernel), the compositional framework consis= ts of +two parts: + +* The *scheduling components*, which are the basic units of the scheduling= . In + the kernel these are the single cgroups along with the tasks that must b= e run + inside. + +* The *scheduling resources*, which are the CPUs of the machine. + +HCBS is a *hierarchical scheduling framework*, where the scheduling compon= ents +form a hierarchy and resources are allocated from parent components to its= child +components in the hierarchy. + +The Chapter is organized as follows: **Section 3.1** gives basic real-time +theory definitions that are used throughout the whole section. **Section 3= .2** +talks about the HCBS framework, giving a general idea on how this is struc= tured. +**Section 3.3** introduces the MPR model, one of the many models which may= be +used for the analysis of the scheduling components and the computation of = the +minimum required scheduling resources for a given component. **Section 3.4= ** +shows the schedulability test for MPR on the HCBS framework. **Section 3.5= ** +shows how to convert a MPR interface to a HCBS compatible resource reserva= tion +for a component. Finally, **Section 3.6** lists other interesting models w= hich +could be used for the component analysis in HCBS. + +3.1 Basic Definitions +--------------------- + +*We borrow the same definitions given in the* ``sched_deadline`` *document= , which +are very briefly summarized here, and new ones, needed by the following co= ntent, +are added.* + +A typical real-time task is composed of a repetition of computation phases= (task +instances, or jobs) which are activated on a periodic or sporadic fashion.= For +our purposes, real-time tasks are characterized by three parameters: + +* Worst Case Execution Time (WCET): the maximum execution time among all j= obs. +* Relative Deadline (D): the maximum time each job must be completed, rela= tive + to the release time of the job. +* Inter-Arrival Period (P): the exact/minimum (for periodic/sporadic tasks= ) time + between each consecutive job. + +3.2 Hierarchical Constant Bandwidth Server (HCBS) [1] +----------------------------------------------------- + +As mentioned, HCBS is a *hierarchical scheduling framework*: + +* The framework hierarchy follows the same hierarchy of cgroups. Cgroups m= ay + have two roles, either bandwidth reservation for children cgroups, or th= ey may + be *live*, i.e. run tasks (but not both). The root cgroup, for the kerne= l's + implementation of HCBS, acts only as bandwidth reservation (but as writt= en in + this document it has also different uses outside of the hierarchical + framework). +* The cgroup tree is internally flattened, for ease of scheduling, to a + two-level hierarchy, since only the *live* groups are of interest and al= l the + necessary information for their scheduling lies in their interface (ther= e is + no need for the reservation components). +* The hierarchical framework, now on two levels, consists then of a first = level + of cgroups, and a second level of tasks that are run inside these groups. +* The scheduling of components is performed using global Earliest Deadline= First + (gEDF), SCHED_DEADLINE in the kernel, following the bandwidth reservatio= n of + each group. +* Whenever a component is scheduled, a local scheduler picks which of the = tasks + of the cgroup to run. The scheduling policy is global Fixed Priority (gF= P), + SCHED_FIFO/SCHED_RR in the kernel. + +3.3 Multiprocessor Periodic Resource (MPR) model +------------------------------------------------ + +A Multiprocessor Periodic Resource (MPR) model [2] **u =3D = ** +specifies that an identical, unit-capacity multiprocessor platform collect= ively +provides **Theta** units of resource every **Pi** time units, where the +**Theta** time units are supplied with concurrency at most **m'**. + +This theoretical model is one of the many models that can abstract the +interface of our real-time cgroups: let **m'** be the number of CPUs of the +machine, let **Theta** be **m' * /cpu.rt_runtime_us** and **Pi** be +**/cpu.rt_period_us**. -Enabling CONFIG_RT_GROUP_SCHED lets you explicitly allocate real -CPU bandwidth to task groups. +Let's introduce the concept of Supply Bound Function (SBF). A SBF is a fun= ction +which outputs a lower bound for the processor supply provided in a given t= ime +interval, given a resource supply model. For a completely dedicated CPU, t= he SBF +function is simply the identity function, as it will always provide **t** = units +of computation for an interval of length **t**. The situation gets slightl= y more +complicated for the MPR model or any of the other model listed in section = 3.6. -This uses the cgroup virtual file system and "/cpu.rt_runtime_us" -to control the CPU time reserved for each control group. +The **SBF(t)** for a MPR model **u =3D ** is:: -For more information on working with control groups, you should read -Documentation/admin-guide/cgroup-v1/cgroups.rst as well. + | 0 if t' < 0 + | + SBF_u(t) =3D | floor(t' / PI) * Theta + | + max(0, m' * x - (m' * Pi - Theta) if t' >=3D 0 and 1 = <=3D x <=3D y + | + | floor(t' / PI) * Theta + | + max(0, m' * x - (m' * Pi - Theta) else + | - (m' - beta) -Group settings are checked against the following limits in order to keep t= he -configuration schedulable: +where:: - \Sum_{i} runtime_{i} / global_period <=3D global_runtime / global_period + alpha =3D floor(Theta / m') + beta =3D Theta - m' * alpha + t' =3D t - (Pi - ceil(Theta / m')) + x =3D t' - (Pi * floor(t' / Pi)) + y =3D Pi - floor(Theta / m') -For now, this can be simplified to just the following (but see Future plan= s): +Briefly, this function models that the server's bandwidth is given as late= as +possible, so describing the worst case possible for the supplied bandwidth. - \Sum_{i} runtime_{i} <=3D global_runtime +3.4 Schedulability for MPR on global Fixed-Priority +--------------------------------------------------- +Let's introduce the concept of Demand Bound Function (DBF). A DBF is a fun= ction +that, given a taskset, a scheduling algorithm and an interval of time, out= puts +the worst resource demand for that interval of time. -3. Future plans -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +It is easy to see that, given a DBF and a SBF, we can deem a component/tas= kset +schedulable if, for every time interval t >=3D 0, it is possible to demons= trate +that: -There is work in progress to make the scheduling period for each group -("/cpu.rt_period_us") configurable as well. + DBF(t) <=3D SBF(t) -The constraint on the period is that a subgroup must have a smaller or -equal period to its parent. But realistically its not very useful _yet_ -as its prone to starvation without deadline scheduling. +We have the Supply Bound Function for our given MPR model, so we are missi= ng the +Demand Bound Function for a given taskset that is being scheduled using gl= obal +Fixed Priority. -Consider two sibling groups A and B; both have 50% bandwidth, but A's -period is twice the length of B's. +3.4.1 Schedulability Analysis for global Fixed Priority +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -* group A: period=3D100000us, runtime=3D50000us +Bertogna, Cirinei and Lipari [6] have derived a schedulability test for gl= obal +Fixed Priority (gFP) on multi-processor platforms. In this test (called +*BCL_gFP* test) we can consider all the CPUs to be dedicated to the schedu= ling. - - this runs for 0.05s once every 0.1s + A taskset **Tau** is schedulable with gFP on a multiprocessor platform + composed of **m'** identical processors if for each task **tau_k in Tau*= *: -* group B: period=3D 50000us, runtime=3D25000us + Sum(for i < k)( min(W_i(D_k), D_k - C_k + 1) ) < m' * (D_k - C_k + 1) - - this runs for 0.025s twice every 0.1s (or once every 0.05 sec). + where **W_i(t)** is the workload of task **tau_i** over a time interval = **t**: -This means that currently a while (1) loop in A will run for the full peri= od of -B and can starve B's tasks (assuming they are of lower priority) for a who= le -period. + W_i(t) =3D N_i(t) * C_i + min(C_i, t + D_i - C_i - N_i(t) * P_i) -The next project will be SCHED_EDF (Earliest Deadline First scheduling) to= bring -full deadline scheduling to the linux kernel. Deadline scheduling the above -groups and treating end of the period as a deadline will ensure that they = both -get their allocated time. + and **N_i(t)** is the number of activations of task **tau_i** that compl= ete in + a time interval **t**: -Implementing SCHED_EDF might take a while to complete. Priority Inheritanc= e is -the biggest challenge as the current linux PI infrastructure is geared tow= ards -the limited static priority levels 0-99. With deadline scheduling you need= to -do deadline inheritance (since priority is inversely proportional to the -deadline delta (deadline - now)). + N_i(t) =3D floor( (t + D_i - C_i) / P_i ) + + while the **min** term is the contribution of the carried-out job in the + interval **t**, i.e. that job that does not completely fit in the interv= al + **t**, but starts inside the interval after all the jobs that complete. + +3.4.2 From BCL_gFP to the Demand Bound Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We can then derive the DBF from this test: + + DBF_gFP(tau_k) =3D Sum(for i < k)( min(W_i(D_k), D_k - C_k + 1) ) + m' *= (C_k - 1) + +Briefly, the first sum component, the same in the BCL_gFP test, describes = the +maximum interference that higher priority task give to the analysed task. = The +workload is upperbounded by ``(D_k - C_K + 1)`` because we are only intere= sted +in the interference in the slack time, while for the ``C_k`` time we are +requiring that all the CPUs are fully available, as the single job needs `= C_k` +(non overlapping) time units to run. + +The demand bound function from Bertogna et al. is only defined on a single= time +(i.e. the deadline of the task in analysis) instead of all possible times = as +this is the minimum argument to demonstrate schedulability on global Fixed +Priority. + +3.4.3 Putting it all togheter +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A component **C**, on **m'** processors, running a taskset **Tau =3D { tau= _1 =3D +(C_1, D_1, P_1), ..., tau_n =3D (C_n, D_n, P_n) }** of **n** sporadic task= s, is +schedulable under gFP using an MPR model **u =3D **, if for= all +tasks **tau_k in Tau**: + + DBF_gFP(tau_k) <=3D SBF_u(D_K) + +3.5 From MPR to deadline servers +-------------------------------- + +Since there exist no algorithm to schedule MPR interfaces, a tecnique was +developed to transform MPR interfaces into periodic tasks, so that a +number of periodic servers which respect the tasks requirements can be use= d for +the scheduling of the MPR interface and associated tasks. + +Let **u =3D ** be a MPR interface, let **a =3D Theta - m * f= loor(Theta +/ m)**, let **k =3D floor(a)**. Define a transformation from **u** to a pe= riodic +taskset **Tau_u =3D { tau_1 =3D (C_1, D_1, P_1), ..., tau_m' =3D (C_m', D_= m', P_m') +}**, where: + + **tau_1 =3D ... =3D tau_k =3D (floor(Theta / m') + 1, Pi, Pi)** + + **tau_k+1 =3D (floor(Theta / m') + a - k * floor(a/k), Pi, Pi)** + + **tau_k+2 =3D ... =3D tau_m' =3D (floor(Theta / m'), Pi, Pi)** + +This periodic taskset of servers **Tau_u** can be scheduled on any number = of +processors with concurrency at most **m'**. + +For real-time control groups, it is possible to just consider a slightly m= ore +demanding taskset **Tau_u'**, where each task **tau_i** is defined as foll= ows: + + **tau_i =3D (ceil(Theta / m'), Pi, Pi)** + +3.6 Other models +---------------- + +There exist many other theoretical models in literature which are used to +describe a hierarchical scheduling framework on multi-core architectures. +Notable examples are the Multi Supply Function (MSF) abstraction [3], the +Parallel Supply Function (PSF) abstraction [4] and the Bounded Delay +Multipartition (BDM) [7]. + +3.7 References +-------------- + 1 - L. Abeni, A. Balsini, and T. Cucinotta, =E2=80=9CContainer-based rea= l-time + scheduling in the Linux kernel,=E2=80=9D SIGBED Rev., vol. 16, no. 3= , pp. 33-38, + Nov. 2019, doi: 10.1145/3373400.3373405. + 2 - A. Easwaran, I. Shin, and I. Lee, =E2=80=9COptimal virtual cluster-b= ased + multiprocessor scheduling,=E2=80=9D Real-Time Syst, vol. 43, no. 1, = pp. 25-59, + Sept. 2009, doi: 10.1007/s11241-009-9073-x. + 3 - E. Bini, G. Buttazzo, and M. Bertogna, =E2=80=9CThe Multi Supply Fun= ction + Abstraction for Multiprocessors,=E2=80=9D in 2009 15th IEEE Internat= ional + Conference on Embedded and Real-Time Computing Systems and Applicati= ons, + Aug. 2009, pp. 294-302. doi: 10.1109/RTCSA.2009.39. + 4 - E. Bini, B. Marko, and S. K. Baruah, =E2=80=9CThe Parallel Supply Fu= nction + Abstraction for a Virtual Multiprocessor,=E2=80=9D in Scheduling, S.= Albers, S. K. + Baruah, R. H. M=C3=B6hring, and K. Pruhs, Eds., in Dagstuhl Seminar = Proceedings + (DagSemProc), vol. 10071. Dagstuhl, Germany: Schloss Dagstuhl - + Leibniz-Zentrum f=C3=BCr Informatik, 2010, pp. 1-14. doi: + 10.4230/DagSemProc.10071.14. + 5 - I. Shin and I. Lee, =E2=80=9CCompositional real-time scheduling fram= ework,=E2=80=9D in + 25th IEEE International Real-Time Systems Symposium, Dec. 2004, pp. = 57-67. + doi: 10.1109/REAL.2004.15. + 6 - M. Bertogna, M. Cirinei, and G. Lipari, =E2=80=9CSchedulability Anal= ysis of Global + Scheduling Algorithms on Multiprocessor Platforms,=E2=80=9D IEEE Tra= nsactions on + Parallel and Distributed Systems, vol. 20, no. 4, pp. 553-566, Apr. = 2009, + doi: 10.1109/TPDS.2008.129. + 7 - G. Lipari and E. Bini, =E2=80=9CA Framework for Hierarchical Schedul= ing on + Multiprocessors: From Application Requirements to Run-Time Allocatio= n,=E2=80=9D in + 2010 31st IEEE Real-Time Systems Symposium, Nov. 2010, pp. 249-258. = doi: + 10.1109/RTSS.2010.12. + + +4. Using Real-Time cgroups +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +4.1 CGroup Setup +---------------- -This means the whole PI machinery will have to be reworked - and that is o= ne of -the most complex pieces of code we have. +The following is a brief guide to the use of Real-Time Control Groups. + +Of course, real-time control groups require mounting of the cgroup file sy= stem. +We have decided to only support cgroups v2, so make sure you mount the v2 +controller for the cgroup hierarchy. + +Additionally the real-time cgroups require the CPU controller for the cgro= ups to +be enabled:: + + # Assume the cgroup file system is mounted at /sys/fs/cgroup + > echo "+cpu" > /sys/fs/cgroup/cgroup.subtree_control + +The CPU controller can only be mounted if there is no SCHED_FIFO/SCHED_RR = task +scheduled in any cgroup other than the root control group. + +The root control group has no bandwidth allocated by default, so make sure= to +allocate some bandwidth so that it can be used by the other cgroups. More = on +that in the following section... + +4.2 Bandwidth Allocation for groups +----------------------------------- + +Allocating bandwidth to a cgroup is a fundamental step to run real-time +workload. The cgroup filesystem exposes two files: + +* ``/cpu.rt_runtime_us``: which specifies the cgroups' runtime in + microseconds. +* ``/cpu.rt_period_us``: which specifies the cgroups' period in + microseconds. + +Both files are readable and writable, and their default value is zero. By +definition, the specified runtime must be always less than or equal to the +period. Additionally, an admission test checks if the bandwidth invariant = is +respected (i.e. sum of children's bandwidth <=3D parent's bandwidth). + +The root control group files instead control and reserve the SCHED_DEADLINE +bandwidth allocated to real-time cgroups, since real-time groups compete a= nd +share the same bandwidth allocated to SCHED_DEADLINE tasks. + +4.3 Running real-time tasks in groups +------------------------------------- + +To run tasks in real-time groups it is just necessary to change a tasks +scheduling policy to SCHED_FIFO/SCHED_RR and migrate it into the group. If= the +group is not allowed to run real-time tasks because of incorrect configura= tion, +either migrating a SCHED_FIFO/SCHED_RR task into the group or changing +scheduling policy to a task already inside the group will fail:: + + # assume there is a task of PID 42 running + # change its scheduling policy to SCHED_FIFO, priority 99 + > chrt -f -p 99 42 + + # migrate the task to a cgroup + > echo 42 > /sys/fs/cgroup//cgroup.procs + +4.4 Special case: the root control group +---------------------------------------- + +The root cgroup is special, compared to the other cgroups, as its tasks ar= e not +managed by the HCBS algorithm, rather they just use the original +SCHED_FIFO/SCHED_RR policies (as if CONFIG_RT_GROUP_SCHED was disabled). As +mentioned, its bandwidth files are just used to control how much of the +SCHED_DEADLINE bandwidth is allocated to cgroups. + +4.5 Guarantees and Special Behaviours +------------------------------------- + +Real-time cgroups are run at the same priority level of SCHED_DEADLINE tas= ks. +Since this is the highest priority scheduling policy, and since the Consta= nt +Bandwidth Server (CBS) enforces that the specified bandwidth requirements = for +both groups and tasks cannot be overrun, real-time groups have the same +guarantees that SCHED_DEADLINE tasks have, i.e. they will be necessarily +supplied by the amount of bandwidth requested (whenever the admission tests +pass). + +This means that, since SCHED_FIFO/SCHED_RR tasks (scheduled in the root co= ntrol +group) are not subject to bandwidth controls, they are run at a lower prio= rity +than the cgroups' counterparts. Nonetheless, a minimum amount of bandwidth= , if +reserved, will always be available to run SCHED_FIFO/SCHED_RR workloads in= the +root cgroup, while they will be able to use more runtime if any of the +SCHED_DEADLINE tasks or servers use less than their specified amount of +bandwidth. SCHED_OTHER tasks are instead scheduled as normal, at lower pri= ority +than real-time workloads. + +The aforementioned behaviour differs from the preceding RT_GROUP_SCHED +implementation, but this is necessary to give actual guarantees to the amo= unt of +bandwidth given to rt-cgroups. \ No newline at end of file -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34595314B88 for ; Mon, 1 Dec 2025 12:42:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592955; cv=none; b=dWTnsxpqSqEc0U5iONWzRNW7dksptGzTrKxkCirUpKbsMzfx7mA+U8L+EhcYj82U37lUrGqNDgDF/YrrP0mLum3fGrkyOoBeCrlI/OWWu+DixApf34eXnbg4vHAPXOkvT00Nid7meIefgQSh/BwlVBa+BZMhawM48EYmeZXF05Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592955; c=relaxed/simple; bh=N1u9L/y0OeTwTWyVG5yMMeGfCQdJNWgFMNiykahcouU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gzk/aO7TOSmaeq+60UyvVq7Oo1tudts4/4Mn3j17ufKwqoH9l3s40av1J5TRn8OmPV1vTt0VagFwpxfcU/3ewvjlWKxRYaWxT0JzEW3kjB7os8za9lkNfMG2IZop7K71OfOTcw4NgpA25VKQhgAQBl7F8lTOvLiZfjZPWNcOI8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RV3nyDnY; arc=none smtp.client-ip=209.85.208.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RV3nyDnY" Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-640860f97b5so6387265a12.2 for ; Mon, 01 Dec 2025 04:42:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592951; x=1765197751; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=w+Co3MGUQifN0gjXqWsbfxvWbgtFzfweeKd5vgRFBu4=; b=RV3nyDnYjPxMk60Iiievdlgg33lKG6CtzSZzy5Vkx1SBCC31VFNA7QVMSmQYVqbv2a fRwnAA7z/SD9UM+CJXS9avnNZMXpmRWum/oQRL+p9fVL+dAPfwRtrSDo6sq0F3dCZj9u xvWaHrZv6RkDDKsCYq5W8SMJ5qtq2vPzo7VWXdDTXTKN+4MjHUtDgKYPnKSQ0iQFT0Fd PeZv3bWxVQNZ2gAjDI+Kxz5yx+87v3i0hM7dfuO+TQBHDwWtEfJfopjpHj4WvCf5X0lZ s2FB/3a/yvwa0qTgPCY0D7mPfH92y8ze6ZBuKQSgGBjMptqmQwrnx5D+h2c03pNrgj/D vvNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592951; x=1765197751; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=w+Co3MGUQifN0gjXqWsbfxvWbgtFzfweeKd5vgRFBu4=; b=YAcmS6dmqMcQ3nkmYr+VfBMoiWgVhSNtcx1ZCNopuA+RsvGWLiZyMYzfyt5HhgO6LM po7JMGzjvFtGJJAZhiK8t+FTzoSb0iKx6oC+icctHh4UIJrJNlH6fP7T05Dxj5bsCkEx L+ms7EPQLgNzGuk11gAR2ggYFBIVdhC8Eh+IkzMU0pJIhjQi8lFtVvTyPYyWoMTqkWrW MjtaBCYOHEYbXnk89wSm2f1C44CMEINsRh7nkuiD8P4y2hMaBETLt6EnDVNDggAT3xyv YR27d/abVQe2RD5zfgr4Wldl7Q/+yd7zyGCuNqJRzkW8ILC0ajwKqvycN5rH/7F7S8B8 owhA== X-Gm-Message-State: AOJu0YzYuRwnV/zsj+OX59m7YvVAkrLltSaJN3t+oTlzP7wazIZfjoql nDKhcqHztT5NyObrwSRo+sE6lhjv9yEUMD1ZULzuUxcP33va/lNLAYHb X-Gm-Gg: ASbGncsjh+9dHaoozU09P8da5gMusNZDziV1nJ1FBE81Y7a9jW0+GcZGD8D9qycTNSZ Jypa0JFCx3WDEVKeQ9GFwng95neEmMLnrj/tK37m5/Owze8SD9O7zDsC1snQDhY4fDhDztYgOEa JUfgkSNgFFNOJ1VLOQJdQDg4eBHjt84PXDrOn4Gy3D/egb364wrEIrRohp6GG9hWF/n/Lwcrfsd cWWvvzvbWQTn4i/l+09V2JywBIrd3sRIZxrD4vCOHXYMxkGAT/EpbeetY35L/So5v/94ud1eHic /AX88g+/fUOJB+kJgUBqbV/oU6F1q/q6//+cGiIAi2TxKVhbHRuho37HRPU+6YKBMFuU6NAzWPU M26cARtCDOXoS0bwAWF2ta0l77G/+es4fz/hePvxeIzvQ1sYuxI0leFA5PNKG4SvJ5XgDuYammR iaX0wJiQAS X-Google-Smtp-Source: AGHT+IFrfYLO2XTDxnXnydC0Y9gAkzKFOozVWSg7sJB1fuojutHeigkaRX+tQ9eDwv0Vnrw4sRrzPw== X-Received: by 2002:a17:907:6d10:b0:b77:1964:1881 with SMTP id a640c23a62f3a-b7719642737mr1280796066b.58.1764592951328; Mon, 01 Dec 2025 04:42:31 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:31 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 27/28] [DEBUG] sched/rt: Add debug BUG_ONs for pre-migration code Date: Mon, 1 Dec 2025 13:42:00 +0100 Message-ID: <20251201124205.11169-28-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add debug BUG_ONs in rt_queue_push/pull_task(s). Can be safely added after all the pre-migration patches. These are extra asserts which are only useful to debug the kernel code and are not meant to be part of the final patchset. Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 80580b48ab..435d147aa5 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -319,6 +319,9 @@ static inline void rt_queue_push_tasks(struct rt_rq *rt= _rq) { struct rq *rq =3D served_rq_of_rt_rq(rt_rq); + BUG_ON(rt_rq =3D=3D NULL); + BUG_ON(rq !=3D cpu_rq(rq->cpu)); + if (!has_pushable_tasks(rt_rq)) return; @@ -329,6 +332,9 @@ static inline void rt_queue_pull_task(struct rt_rq *rt_= rq) { struct rq *rq =3D served_rq_of_rt_rq(rt_rq); + BUG_ON(rt_rq =3D=3D NULL); + BUG_ON(rq !=3D cpu_rq(rq->cpu)); + queue_balance_callback(rq, &per_cpu(rt_pull_head, rq->cpu), pull_rt_task); } -- 2.51.0 From nobody Mon Dec 1 21:30:06 2025 Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2723D315D25 for ; Mon, 1 Dec 2025 12:42:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592956; cv=none; b=tN1TAGhaJYwhv1eMpoc0bQ9pIz1wtHpDzAN0LMZWdV0OLqhe7V4e1Sts75cTBzk5eZZXTGbAN2R+ksBiozPUJyr2ON5tJrCo2PLqLslfnJ7EvJWHY993/O1zPthoUf94kPEdL/wwKB3tOQZlH3DNP24PgKzL+q29b5fMPSS3sXY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764592956; c=relaxed/simple; bh=Yd3Dq1tLzigZLg//Q9L61Hu8w0/rKwSmpeiSv+tqbGA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QVDWvTC+K2QutTNW0RP/JQ2NTFKUyUwhGopiC9mZ8AL8h/g1q5HJlJZSQf8c3gapgTzOTewqKlA5iy26L8H6mGnfc/poT/Impqks1LhzlLlM2Ef84oQRDQNJmj8A4+oLuZbdKozCZkZKAm4Z0iSjumT/H9+apJyBq+NKAm1f27o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OWa6oaoS; arc=none smtp.client-ip=209.85.208.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OWa6oaoS" Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-641677916b5so456340a12.0 for ; Mon, 01 Dec 2025 04:42:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764592952; x=1765197752; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=foxXJq8kko4Pdvtrz8obXKgDpDxe+CJjQmDp5W5Tpqk=; b=OWa6oaoSFeE7SnT6FMijNVbhL1vXpsq2UohCidDHRx4vCz4tZVtulzW275jSpXH3JU 1F0jfFWartcLM/VIxZs/BOfhpCIIwwkRKt7JfWcurHv9PG0usLPokAxqH/YIX2zz6EPJ brY1CP2brCcq9fJOY12ARnSP/185gVXb80x8vpWEshOx6kMv5Dz3TPCKvuGDhRA9bGBO DYQN1CQooWfMCD/3ut2VNn3B9+Q15qI6v0RPQvM0yu3l4I9u+3UUfGu524GD9OnziVMj 97cZSdVuBvxz9SljF3yGnNhviv97iY/6oFPm9h1Jy2NukqXOIinw4+0q+fnAa99BDqAt 1ZxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764592952; x=1765197752; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=foxXJq8kko4Pdvtrz8obXKgDpDxe+CJjQmDp5W5Tpqk=; b=nDnVGXaLF0HXAA5bUQs6CDMBDeydMhz8W4IreUhnSLPmGHqj1wCm1QDl9w8XzaxlQK CnZMltO7Xv9VW7GoXEq+PvprYEhdEJFWFnfcDAveBjlqOeNhlaz9jNC0Fg0a0Jdcx9Dv CbBJfLk3v0eOFWo8WnsRfa1ffMat+o62P7+aMiF2urw3/joRmn4HNHk5fU4nliGnb5TT llOTvPhVQaxv8FR0IsPmVh8S159voEMwdQoEZNUvT9xTWqNmrc6iz7/dd3c8sCd0RShG XCLh70Oon2wTRj924iykZefm23Ev4ogrRAA2cUtoZkzZDyNCUzywHTJP4sJnDzaDsMnG xlBQ== X-Gm-Message-State: AOJu0YzDHIH1UTmBIC/uXE3VYQh63d5tm+GjOrsnBYeVlIwxa74zkpP8 4uupKW1lUZ6HiyISBg2nB7UU2sXEe6jVy2koF0Lhtl6V8NnDuxDji55i X-Gm-Gg: ASbGncukpZAuLavV/f/mvYnFGussr+IamhJU6XqHZ9F2wvhAJg562w9esDJ8n/47/c9 gnZIWOpv22kaYYu8wGD5YA4mV1XfkHyksPmqbkUphd6OmB+BO7DXGYdvAIFbckYGOk335J5y2+W OeempYVivyhBerbmdtaoVaFHAcyYKP6RkvOXAVAWlGw5xXiqEHGaTawpVxqvvnr4/7hYAcmtugZ rJiIoLAWaK8T7djNFyojv6LxmufTZg6PgmEoDdVVrU61hjQB3+dcSsLmgb8Hh2U90OyAVbXezUa klNoobu/oTTRbqod2DLhxtzyo6zpOvUkwxPZB8pdIc9KIUEesLUgTjDVEh3xqJ9jQRCwT16u82b yy+Km8pRIIRmoUphijXQJWNTj3h/fLPZe7o8qHAEbegaPykFUSLHxssl+MXT8MQHnNd1plQ+wIO 7deBOUBipt X-Google-Smtp-Source: AGHT+IE42JKm1EwLkuqFDGu2uLgr1WZC8Hf7xmZTi+z3kQukdee6q6AljHCaYlB79fDBNcN8GPuwOw== X-Received: by 2002:a17:907:3cc5:b0:b71:51b:bd4c with SMTP id a640c23a62f3a-b767170c7dbmr3710910766b.50.1764592952144; Mon, 01 Dec 2025 04:42:32 -0800 (PST) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b76f59e8612sm1173738266b.52.2025.12.01.04.42.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 04:42:31 -0800 (PST) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v4 28/28] [DEBUG] sched/rt: Add debug BUG_ONs in migration code. Date: Mon, 1 Dec 2025 13:42:01 +0100 Message-ID: <20251201124205.11169-29-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201124205.11169-1-yurand2000@gmail.com> References: <20251201124205.11169-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add debug BUG_ONs in group specific migration functions. Can be safely added after all the migration patches. These are extra asserts which are only useful to debug the kernel code and are not meant to be part of the final patchset. Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 435d147aa5..1a777298f9 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -349,6 +349,9 @@ static void rt_queue_push_from_group(struct rt_rq *rt_r= q) struct rq *rq =3D served_rq_of_rt_rq(rt_rq); struct rq *global_rq =3D cpu_rq(rq->cpu); + BUG_ON(rt_rq =3D=3D NULL); + BUG_ON(rq =3D=3D global_rq); + if (global_rq->rq_to_push_from) return; @@ -366,6 +369,10 @@ static void rt_queue_pull_to_group(struct rt_rq *rt_rq) struct rq *global_rq =3D cpu_rq(rq->cpu); struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + BUG_ON(rt_rq =3D=3D NULL); + BUG_ON(!is_dl_group(rt_rq)); + BUG_ON(rq =3D=3D global_rq); + if (dl_se->dl_throttled || global_rq->rq_to_pull_to) return; @@ -1393,6 +1400,8 @@ static struct rq *find_lock_lowest_rq(struct task_str= uct *task, struct rq *rq) */ static int push_rt_task(struct rq *rq, bool pull) { + BUG_ON(is_dl_group(&rq->rt)); + struct task_struct *next_task; struct rq *lowest_rq; int ret =3D 0; @@ -1689,6 +1698,8 @@ void rto_push_irq_work_func(struct irq_work *work) static void pull_rt_task(struct rq *this_rq) { + BUG_ON(is_dl_group(&this_rq->rt)); + int this_cpu =3D this_rq->cpu, cpu; bool resched =3D false; struct task_struct *p, *push_task; @@ -1813,6 +1824,8 @@ static int group_find_lowest_rt_rq(struct task_struct= *task, struct rt_rq *task_ int prio, lowest_prio; int cpu, this_cpu =3D smp_processor_id(); + BUG_ON(task->sched_task_group !=3D task_rt_rq->tg); + if (task->nr_cpus_allowed =3D=3D 1) return -1; /* No other targets possible */ @@ -1915,6 +1928,8 @@ static struct rt_rq *group_find_lock_lowest_rt_rq(str= uct task_struct *task, stru struct sched_dl_entity *lowest_dl_se; int tries, cpu; + BUG_ON(task->sched_task_group !=3D rt_rq->tg); + for (tries =3D 0; tries < RT_MAX_TRIES; tries++) { cpu =3D group_find_lowest_rt_rq(task, rt_rq); @@ -1971,6 +1986,8 @@ static struct rt_rq *group_find_lock_lowest_rt_rq(str= uct task_struct *task, stru static int group_push_rt_task(struct rt_rq *rt_rq, bool pull) { + BUG_ON(!is_dl_group(rt_rq)); + struct rq *rq =3D rq_of_rt_rq(rt_rq); struct task_struct *next_task; struct rq *lowest_rq; @@ -2090,6 +2107,8 @@ static int group_push_rt_task(struct rt_rq *rt_rq, bo= ol pull) static void group_pull_rt_task(struct rt_rq *this_rt_rq) { + BUG_ON(!is_dl_group(this_rt_rq)); + struct rq *this_rq =3D rq_of_rt_rq(this_rt_rq); int this_cpu =3D this_rq->cpu, cpu; bool resched =3D false; @@ -2202,6 +2221,9 @@ static void group_push_rt_tasks_callback(struct rq *g= lobal_rq) { struct rt_rq *rt_rq =3D &global_rq->rq_to_push_from->rt; + BUG_ON(global_rq->rq_to_push_from =3D=3D NULL); + BUG_ON(served_rq_of_rt_rq(rt_rq) =3D=3D global_rq); + if ((rt_rq->rt_nr_running > 1) || (dl_group_of(rt_rq)->dl_throttled =3D=3D 1)) { @@ -2216,6 +2238,9 @@ static void group_pull_rt_task_callback(struct rq *gl= obal_rq) { struct rt_rq *rt_rq =3D &global_rq->rq_to_pull_to->rt; + BUG_ON(global_rq->rq_to_pull_to =3D=3D NULL); + BUG_ON(served_rq_of_rt_rq(rt_rq) =3D=3D global_rq); + group_pull_rt_task(rt_rq); global_rq->rq_to_pull_to =3D NULL; } -- 2.51.0