From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8577274659 for ; Mon, 29 Sep 2025 09:22:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137747; cv=none; b=QuL+yCWGiUnqqyrtEte/KYO5g4xjS21KLPr1IEifEi+og/BJ8Dvw2LZ+807Nw92Q2kos22XaUMdvqzZiWHVCkJ32IB0ZGfiUM+7rhexB1UG1gHI9ONXlPF7sI8XN/DaQNqOPdXZOomnwxA4/ny7UcYT07A3VHrnuaKz83PMTT+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137747; c=relaxed/simple; bh=/Nvv3+IoLrE1qokv5D7XIZ4rmVNSJbEEYyhfkWbXFsQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MwtSwUYp3BqpaC8Vt5K41eWTyk0ceJlf9+DrkzGivDLfUl+Brrozfl0VdDQMFjk7Hyc/HuL73HbtJCvBYMtke97llHZg1fz/2RTp6ah5ONujImWi1uh1+LOwmlT3hZmODYh0slYutA3xAOBx0hdeKcOA8jCmM+sVb9w5qAjPoek= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UqKK9BGk; arc=none smtp.client-ip=209.85.208.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UqKK9BGk" Received: by mail-ed1-f52.google.com with SMTP id 4fb4d7f45d1cf-62ec5f750f7so6356885a12.3 for ; Mon, 29 Sep 2025 02:22:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137744; x=1759742544; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=spCan/N9K4QxlTmBD9/alPS6yosYlCXKA4lq5unkhdM=; b=UqKK9BGkgHAjVqpZpWtoseIFDMQA+tIT3r/bUxfFDsGb01Xqi/rhIYEw7tp9tnyqHG pysvCnuC6lFQQzj/r54aKcdUBeYoQrN/RUYVl7haZggjuW2l1kpogKtSlHhKmpMaZS5t VqDf17CR7j34gso/+C7tWu3vAhyeBn50HwRC9YFH8/+6cv5592iLDHvraJq1+Rkn2cT6 ha76/nWJmBBegsu2ORAzvRn0dQRb50b5jLQZDdbZajNgZditwJSn6i1TiklLa8y8f6b6 qFblhjVWFHIchFBlBrPhkjKJSMf6a6QVWCDHsby/ubc14Cz9mwvPkvpg6LjjtEVaDT+s pyrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137744; x=1759742544; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=spCan/N9K4QxlTmBD9/alPS6yosYlCXKA4lq5unkhdM=; b=NTmQ2cR48WeOdWQi5BYGh2Ge246a2ZQpjoV2cROAkWEycG4Lebofa4JBClAkFAVRDH +458RYPMbuLXS5cdioj8nmo7Lgjgi2eMZTY5IsSh3ULwE6ZSUmBQB91QbgN7BLt4mrGd t9ZSG1O7mosFvsomEDDcplELWgSio5PzTJ276/cWCyH15qAUQBZ3P1unSSQYHKLfSPvK X5EvMTfRuY3vCjWpjf3xlZkoIKx36rOpj2oQlAMblT+P7/hMut144TIG/c7rDcob5N00 VobvTzpXP+4Onk/UnMdfRVY5q5PrxTGSsxI9E7i0ygVTWNqCNSBuIFmCyliGEAOurXBS +yKA== X-Gm-Message-State: AOJu0Yyui6K5SIk57ZaDOq61KnLwwbOHqHXzycGj5hiaNXKbrDatTWOY ydWtaBNg1/gJ4FhnGu8qB43pyQ5YfSeRwuxit3NEUEEQkNP+FK1WsFiD X-Gm-Gg: ASbGnctIH4C2T4WxZduZkMkMdJn5ZXl+Vd4GuQeN8JHHKI31gzdPEVG9wgvlOYEGRSd pB1BvFUU4NMeVY3Vu6EsmGGScPXnW87HcnjjSN89TRWg9qmvm/kr7ZtVBOo1VJG9ZpjM9D5lP+w xZnLT0/0XJKjJ78edSzvbnIY/4IikloCqQJk4Z4kXYRzSAU7CICyeqle5xZgGgZSnVVDXCR4fUL nV5s5YSX5hACW/uJtHDOsaqgyn1Oy9o0y/HyMx21VpEDt02/mM9g8verch6e4qEA/swmTQi32UP JoIVn8dOU2rNitPw2OnERPzOR2ybycM6KjZWUJiZ4QzqGlcHDdLroi3u+QwhH2V0sF/reg6O6Ub oY12BnyKUw3H6VvKkmV0vlZby3h/8lg== X-Google-Smtp-Source: AGHT+IGjA+ST+aacUoohlmqOoXtrfNZh2pdwSqRCZXFbbGwhKyw22UgOknjetLUulTKWkNEUVBcvFw== X-Received: by 2002:a17:907:3fa0:b0:b04:67f3:890f with SMTP id a640c23a62f3a-b34bbbda1e6mr1910135666b.33.1759137743961; Mon, 29 Sep 2025 02:22:23 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:23 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 01/24] sched/deadline: Do not access dl_se->rq directly Date: Mon, 29 Sep 2025 11:21:58 +0200 Message-ID: <20250929092221.10947-2-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make deadline.c code access the runqueue of a scheduling entity saved in the sched_dl_entity data structure. This allows future patches to save different runqueues in sched_dl_entity other than the global runqueues. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/deadline.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index f25301267e4..5113c3a2633 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -874,7 +874,7 @@ static void replenish_dl_entity(struct sched_dl_entity = *dl_se) * and arm the defer timer. */ if (dl_se->dl_defer && !dl_se->dl_defer_running && - dl_time_before(rq_clock(dl_se->rq), dl_se->deadline - dl_se->runtime)= ) { + dl_time_before(rq_clock(rq), dl_se->deadline - dl_se->runtime)) { if (!is_dl_boosted(dl_se) && dl_se->server_has_tasks(dl_se)) { =20 /* @@ -1184,11 +1184,11 @@ static enum hrtimer_restart dl_server_timer(struct = hrtimer *timer, struct sched_ * of time. The dl_server_min_res serves as a limit to avoid * forwarding the timer for a too small amount of time. */ - if (dl_time_before(rq_clock(dl_se->rq), + if (dl_time_before(rq_clock(rq), (dl_se->deadline - dl_se->runtime - dl_server_min_res))) { =20 /* reset the defer timer */ - fw =3D dl_se->deadline - rq_clock(dl_se->rq) - dl_se->runtime; + fw =3D dl_se->deadline - rq_clock(rq) - dl_se->runtime; =20 hrtimer_forward_now(timer, ns_to_ktime(fw)); return HRTIMER_RESTART; @@ -1199,7 +1199,7 @@ static enum hrtimer_restart dl_server_timer(struct hr= timer *timer, struct sched_ =20 enqueue_dl_entity(dl_se, ENQUEUE_REPLENISH); =20 - if (!dl_task(dl_se->rq->curr) || dl_entity_preempt(dl_se, &dl_se->rq->cu= rr->dl)) + if (!dl_task(rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl)) resched_curr(rq); =20 __push_dl_task(rq, rf); @@ -1467,7 +1467,7 @@ static void update_curr_dl_se(struct rq *rq, struct s= ched_dl_entity *dl_se, s64 =20 hrtimer_try_to_cancel(&dl_se->dl_timer); =20 - replenish_dl_new_period(dl_se, dl_se->rq); + replenish_dl_new_period(dl_se, rq); =20 /* * Not being able to start the timer seems problematic. If it could not @@ -1581,21 +1581,22 @@ void dl_server_update(struct sched_dl_entity *dl_se= , s64 delta_exec) /* 0 runtime =3D fair server disabled */ if (dl_se->dl_runtime) { dl_se->dl_server_idle =3D 0; - update_curr_dl_se(dl_se->rq, dl_se, delta_exec); + update_curr_dl_se(rq_of_dl_se(dl_se), dl_se, delta_exec); } } =20 void dl_server_start(struct sched_dl_entity *dl_se) { - struct rq *rq =3D dl_se->rq; + struct rq *rq; =20 if (!dl_server(dl_se) || dl_se->dl_server_active) return; =20 dl_se->dl_server_active =3D 1; enqueue_dl_entity(dl_se, ENQUEUE_WAKEUP); - if (!dl_task(dl_se->rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl)) - resched_curr(dl_se->rq); + rq =3D rq_of_dl_se(dl_se); + if (!dl_task(rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl)) + resched_curr(rq); } =20 void dl_server_stop(struct sched_dl_entity *dl_se) @@ -1651,9 +1652,9 @@ void sched_init_dl_servers(void) =20 WARN_ON(dl_server(dl_se)); =20 - dl_server_apply_params(dl_se, runtime, period, 1); - dl_se->dl_server =3D 1; + WARN_ON(dl_server_apply_params(dl_se, runtime, period, 1)); + dl_se->dl_defer =3D 1; setup_new_dl_entity(dl_se); } @@ -1678,7 +1679,7 @@ int dl_server_apply_params(struct sched_dl_entity *dl= _se, u64 runtime, u64 perio { u64 old_bw =3D init ? 0 : to_ratio(dl_se->dl_period, dl_se->dl_runtime); u64 new_bw =3D to_ratio(period, runtime); - struct rq *rq =3D dl_se->rq; + struct rq *rq =3D rq_of_dl_se(dl_se); int cpu =3D cpu_of(rq); struct dl_bw *dl_b; unsigned long cap; @@ -1755,7 +1756,7 @@ static enum hrtimer_restart inactive_task_timer(struc= t hrtimer *timer) p =3D dl_task_of(dl_se); rq =3D task_rq_lock(p, &rf); } else { - rq =3D dl_se->rq; + rq =3D rq_of_dl_se(dl_se); rq_lock(rq, &rf); } =20 --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C00C21EE7B9 for ; Mon, 29 Sep 2025 09:22:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137748; cv=none; b=nc1hDOl/O7jUuYwuyPK8zmrNuUfGxt5YAF5WgrI1x6gPDxUfW9T2h22MKpOjVjwEQ4vIsBQM+xIIIZJSfVnOk6c5WtiE7oRenVrjmXaWuaygUz2HW1F/kGJaJSTeH72pAeXVCy1HgBkZCKjKfFr0mC+iu8wVKK/s/9xAl/Ad5X4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137748; c=relaxed/simple; bh=S5MxWR5GoGZBnStHQQUs/qHKLb2YQcNYZa0d7u7q6bc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JLZDgEVxNbaw7Ec3NQbTNBS+3mmvgFpCFlgyJjhdFaFn4Z2ay36NFFl+XEqZQywqMsAbXREJ9F+g/okB91mSASubncA48xkHCWAAy+T4Cpydj5QCGoOV94X15H2EzVpQ9W7FOAb81It5Bb5oCDgiAJwdX1mWHEC/VbzEHWiFHf4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gL1xQMHu; arc=none smtp.client-ip=209.85.218.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gL1xQMHu" Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-b3da3b34950so175871366b.3 for ; Mon, 29 Sep 2025 02:22:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137745; x=1759742545; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iQL/+PGJfM/ECDwo+SvhH2uE9jNxwr9mlcuoVIiRG2I=; b=gL1xQMHufgzinmaiNZo66jbsXvYNRs9sxMpcyAdnwvXPSWAYOIPrjaRD8oSd91lfjV lcYuTV7FJOBLqPcEf9gHMVRJ9Iw4jVmaFgD3g5xZDj1JzAqLWkudYWhXYCOJUbw3x+Uf 9kZW89kxv0yGiCJkw0DsQW3g2dUZtDE96Eo+kC1md015IMaQcz5iT7+1Hk9RqjXgf8cx uNiWr6SSR4rphApTVquqIxW8EShhVCum+Km6QYEwoaQg5/yXpzG9hl93bgL5Yl0xTUqb fYyQYWz+D388HOk15Lq3UUdKxOIyUSngsHkumg/1eMD+Uep/SSVMn44rKui/7bBd01jA JoQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137745; x=1759742545; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iQL/+PGJfM/ECDwo+SvhH2uE9jNxwr9mlcuoVIiRG2I=; b=QcuqyEP9RcVkYE8XcsrpN/IL0n0ZGu7fz19iEkyqX5eh+6Pg+WVhlJ1MiyLPcMDtto ssXZzNEPRanmXTOPEJGiUFxk4GR7MY2y+jDi57g7QN5b7iLQW5PvoWdUPknOEpTZtVq5 P+4n6+v4X5z6Oinhpxto4N9OXx+hjGauLLZCGDTAVSkioqXk/qgWYm9ctWt7rwdUJS0+ rV9SplvpwSfC5VLcVrceuzumzVndKUXLGxCHbQTO3rNBboSHoeRqwOyaX+0hx/u6M3jQ QG5OR/nnKJ8OTOS6p4Xz5TBeecrJ+hsc4FQgBaOq6AvyWCmoyAlzNip7eLRleEU3jqGO V8DA== X-Gm-Message-State: AOJu0Yw7bFxSN5d5yX2zJU2MHzEEuEuQ/us2UAMeG/TwaUqzGzz3Vej5 OfoLWdedOE0dbxQYUvmur+zdz7s7ttccRldLHdSf++FpNe4+fhAFJPPZ X-Gm-Gg: ASbGncvA8U/048SS1oUXc7q2Ue8KS+BSqCx5RVVcoIcvdYktxbTOUmWbfeRN7Oc7ApU W2ag2GH6hp9/nBmcbfJWeRGXOOB9M+3XR4GkDJhYN7qI0aE1uAt8cqzB4SH7PSpX41BmrWEzTCS 51YlbI6FxKuSdsmfD7YVaQ9DzD/MLZkCCssubved5+1IcwRARZh+kAbfycxiXgPSXPdHZqOJHTY JZ971RV7Xeht5FXuvWazNWqx7GUXIwLDXbBao+3oKY+1Y4G8eNIJ2+fPOdZYNj84kj1j71EdiMP lrc0tAPIERqdu1Kv5pOmCbBQLZZLs2/YDZBboKFjdPFuLE9751ewDRXK5d/FW4mmLOulf0NY5HW aRPY5rXf56LGjTJZXp94= X-Google-Smtp-Source: AGHT+IGuX8WBXuHMb9hvcHGd4mXpxHtHFZCkkRcro1SywNYuKbNRTtYgE5xsvvMIJKJOqzM80+2Tew== X-Received: by 2002:a17:906:794d:b0:b3c:bb7a:c366 with SMTP id a640c23a62f3a-b3cbb7ac5bdmr498051166b.7.1759137744724; Mon, 29 Sep 2025 02:22:24 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:24 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 02/24] sched/deadline: Distinct between dl_rq and my_q Date: Mon, 29 Sep 2025 11:21:59 +0200 Message-ID: <20250929092221.10947-3-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Create two fields for runqueues in sched_dl_entity to make a distinction be= tween the global runqueue and the runqueue which the dl_server serves. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- include/linux/sched.h | 6 ++++-- kernel/sched/deadline.c | 11 +++++++---- kernel/sched/fair.c | 6 +++--- kernel/sched/sched.h | 3 ++- 4 files changed, 16 insertions(+), 10 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index f8188b83335..5022ff99c5d 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -727,12 +727,14 @@ struct sched_dl_entity { * Bits for DL-server functionality. Also see the comment near * dl_server_update(). * - * @rq the runqueue this server is for + * @dl_rq the runqueue on which this entity is (to be) queued + * @my_q the runqueue "owned" by this entity * * @server_has_tasks() returns true if @server_pick return a * runnable task. */ - struct rq *rq; + struct dl_rq *dl_rq; + struct rq *my_q; dl_server_has_tasks_f server_has_tasks; dl_server_pick_f server_pick_task; =20 diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 5113c3a2633..6ff00f71041 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -75,11 +75,12 @@ static inline struct rq *rq_of_dl_rq(struct dl_rq *dl_r= q) =20 static inline struct rq *rq_of_dl_se(struct sched_dl_entity *dl_se) { - struct rq *rq =3D dl_se->rq; + struct rq *rq; =20 if (!dl_server(dl_se)) rq =3D task_rq(dl_task_of(dl_se)); - + else + rq =3D container_of(dl_se->dl_rq, struct rq, dl); return rq; } =20 @@ -1625,11 +1626,13 @@ static bool dl_server_stopped(struct sched_dl_entit= y *dl_se) return false; } =20 -void dl_server_init(struct sched_dl_entity *dl_se, struct rq *rq, +void dl_server_init(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq, + struct rq *served_rq, dl_server_has_tasks_f has_tasks, dl_server_pick_f pick_task) { - dl_se->rq =3D rq; + dl_se->dl_rq =3D dl_rq; + dl_se->my_q =3D served_rq; dl_se->server_has_tasks =3D has_tasks; dl_se->server_pick_task =3D pick_task; } diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b173a059315..2723086538b 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8861,12 +8861,12 @@ static struct task_struct *__pick_next_task_fair(st= ruct rq *rq, struct task_stru =20 static bool fair_server_has_tasks(struct sched_dl_entity *dl_se) { - return !!dl_se->rq->cfs.nr_queued; + return !!dl_se->my_q->cfs.nr_queued; } =20 static struct task_struct *fair_server_pick_task(struct sched_dl_entity *d= l_se) { - return pick_task_fair(dl_se->rq); + return pick_task_fair(dl_se->my_q); } =20 void fair_server_init(struct rq *rq) @@ -8875,7 +8875,7 @@ void fair_server_init(struct rq *rq) =20 init_dl_entity(dl_se); =20 - dl_server_init(dl_se, rq, fair_server_has_tasks, fair_server_pick_task); + dl_server_init(dl_se, &rq->dl, rq, fair_server_has_tasks, fair_server_pic= k_task); } =20 /* diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index be9745d104f..2499b36a3c5 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -382,7 +382,8 @@ extern s64 dl_scaled_delta_exec(struct rq *rq, struct s= ched_dl_entity *dl_se, s6 extern void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec= ); extern void dl_server_start(struct sched_dl_entity *dl_se); extern void dl_server_stop(struct sched_dl_entity *dl_se); -extern void dl_server_init(struct sched_dl_entity *dl_se, struct rq *rq, +extern void dl_server_init(struct sched_dl_entity *dl_se, struct dl_rq *dl= _rq, + struct rq *served_rq, dl_server_has_tasks_f has_tasks, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7993128641F for ; Mon, 29 Sep 2025 09:22:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137749; cv=none; b=u8/Ay7tn7jmXr/Q1A39tzSW4266B6Z2Ag7Lxt5KyTlaNzOaA7MQxI3o61yjmz2MKvMVk9A+XB4csXbXn3C34htAOd8F5VL5xXQ2cUx9f+vCLN/4i7Dv1s1tSpSCfRX13n8O9cjB2/gHu4dCgHQHE8EZ2N1Z1kbhav46jA92UJMw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137749; c=relaxed/simple; bh=jciHxDfXeBXjUy/dxJ8pfOaiWCeNGNngLrWuDvXssKw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NIeascRYUfXcVm312//hGUP1E/lM0elRUoQQvwpQ/ii8D3/RX0gC8dBOmWquvwhdkDq9XGusH7nbxPy5/fV3xTlJYss9p92Eww1FZiBY1Sk9EF8iZipDCapN2eBluXJtc3ZCTAYEwkgVR5o7jDq/ZdMQC8hufN8PEJy1fGyoJp4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nRMk+o9b; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nRMk+o9b" Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-b07e3a77b72so897926166b.0 for ; Mon, 29 Sep 2025 02:22:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137746; x=1759742546; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gggYPJJA1Zkq3x0Mc0f9JOTzS0urDfAEbiSwWlZGS5s=; b=nRMk+o9bhNbu2WYi+kViyz0gmgV5zfFd39rvzua4r3eQPG2HUUWWvicgwK8OA3avqd t8HllAIfbeIsfq/Yq+Pn8TA5sNlH+oRJtng99r5IWkqRFitS8T0ja4Ac8gFiSSivolAV o1vJAjQJH1NgxCmfvKAJUpfVZOFrUT7qjQ0zJQQ5zIstKVecii/2/i6jEmkUmNvH4HG1 ChHL22xFySGr/fbYwmM3oJSFitGoCsl1CTXiIAGeEbhDy+qardltkb+M11V8NsqP4sdQ 9HFQJOBXnJM+dfPi/28IOy6LesAR1MnsVxrQb7w+Ay3ykrVsKDe42n4c578ZRsrA2Xai Te6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137746; x=1759742546; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gggYPJJA1Zkq3x0Mc0f9JOTzS0urDfAEbiSwWlZGS5s=; b=vglfQkukuF2v/1UqaNH6a3zM6ZJDgVV9XZv7fP+O9uR52yA21+/bsWnhJi10MKEkA/ JtAuzWBvScbT6Qayfy1UwrniueFjeBy0QxSd1a1ydODmOCsUcvJ77oxhP/Q2/XkEH/MJ ua3RhE+mQxeB1wYdgHdylKP4UBFIrSoEzczUoJvor20SseXsCQi2BbOnePaCxW8ELiKa uN0oCyceUNBbvNFDLUqtNg7MGzcvETIuHozgazEBOIrbjzqWfL96L6cBSA3tNpzkzjz1 c4M8in6/qFTo5vttz4Qu4jiJwjSQsz97KtEMte/kPr6fmduiA7YMKGLMC3/jHfo2N0j5 ErIw== X-Gm-Message-State: AOJu0YzMjT5Pu0nq4N3Sg7IH53DHU4mbltMSWpbnMxe+exUx3UHgKNLQ 0D6hOltGDOJLhJikr2RgaLB6kOvh9r3oBW2PdShIslyaZKpY8UpM32w6 X-Gm-Gg: ASbGncvlFTBTSji0NeRMFvg4yVS1VVqmyQdq6FEaTvYKO2a8c/YgzAtFlB3xPc51rY3 +pjALus8Agd2p2A+fpisgAwAJtpZB+tQ37FO+f25A7gpAA2zyjENsgV4o8QkMk0mdBGWFCT0ffq SgdDseeYal35NaraL+s4jAPESM2t4xGbXAeuHLknQTNLdZiZ4N/FmFlryfiWVf4STgfMQvLnvPr kTmB09c4KxxSzgiZg5uy/1HMpt7e0kZzSXax1eBeQvZOsjnzWdX9GlotU75dFlcGR6/RbvWBDe+ updNyvJKwlQO1niKuECpzbAjB0rjNGBomag5PptVTeTnMOmuW4Tl2ks6Mc3RRzJjxrlk/IIfsGz K7AivdkZmBpHzLjZOA7BZ19fVdMOkMQ== X-Google-Smtp-Source: AGHT+IFcawAFjjtrBu72ZmutGJat4fOVwMnAHz+aI0KZt43YXA2uZneYZ92vjrSPuYCxKe+Amm5iqQ== X-Received: by 2002:a17:907:971c:b0:af9:8739:10ca with SMTP id a640c23a62f3a-b3a12358b98mr811166766b.28.1759137745462; Mon, 29 Sep 2025 02:22:25 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:25 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 03/24] sched/rt: Pass an rt_rq instead of an rq where needed Date: Mon, 29 Sep 2025 11:22:00 +0200 Message-ID: <20250929092221.10947-4-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make rt.c code access the runqueue through the rt_rq data structure rather = than passing an rq pointer directly. This allows future patches to define rt_rq = data structures which do not refer only to the global runqueue, but also to local cgroup runqueues (rt_rq is not always equal to &rq->rt). Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 99 ++++++++++++++++++++++++++--------------------- 1 file changed, 54 insertions(+), 45 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 7936d433373..a7d063d2303 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -370,9 +370,9 @@ static inline void rt_clear_overload(struct rq *rq) cpumask_clear_cpu(rq->cpu, rq->rd->rto_mask); } =20 -static inline int has_pushable_tasks(struct rq *rq) +static inline int has_pushable_tasks(struct rt_rq *rt_rq) { - return !plist_head_empty(&rq->rt.pushable_tasks); + return !plist_head_empty(&rt_rq->pushable_tasks); } =20 static DEFINE_PER_CPU(struct balance_callback, rt_push_head); @@ -381,50 +381,54 @@ static DEFINE_PER_CPU(struct balance_callback, rt_pul= l_head); static void push_rt_tasks(struct rq *); static void pull_rt_task(struct rq *); =20 -static inline void rt_queue_push_tasks(struct rq *rq) +static inline void rt_queue_push_tasks(struct rt_rq * rt_rq) { - if (!has_pushable_tasks(rq)) + struct rq *rq =3D container_of(rt_rq, struct rq, rt); + + if (!has_pushable_tasks(rt_rq)) return; =20 queue_balance_callback(rq, &per_cpu(rt_push_head, rq->cpu), push_rt_tasks= ); } =20 -static inline void rt_queue_pull_task(struct rq *rq) +static inline void rt_queue_pull_task(struct rt_rq * rt_rq) { + struct rq *rq =3D container_of(rt_rq, struct rq, rt); + queue_balance_callback(rq, &per_cpu(rt_pull_head, rq->cpu), pull_rt_task); } =20 -static void enqueue_pushable_task(struct rq *rq, struct task_struct *p) +static void enqueue_pushable_task(struct rt_rq *rt_rq, struct task_struct = *p) { - plist_del(&p->pushable_tasks, &rq->rt.pushable_tasks); + plist_del(&p->pushable_tasks, &rt_rq->pushable_tasks); plist_node_init(&p->pushable_tasks, p->prio); - plist_add(&p->pushable_tasks, &rq->rt.pushable_tasks); + plist_add(&p->pushable_tasks, &rt_rq->pushable_tasks); =20 /* Update the highest prio pushable task */ - if (p->prio < rq->rt.highest_prio.next) - rq->rt.highest_prio.next =3D p->prio; + if (p->prio < rt_rq->highest_prio.next) + rt_rq->highest_prio.next =3D p->prio; =20 - if (!rq->rt.overloaded) { - rt_set_overload(rq); - rq->rt.overloaded =3D 1; + if (!rt_rq->overloaded) { + rt_set_overload(rq_of_rt_rq(rt_rq)); + rt_rq->overloaded =3D 1; } } =20 -static void dequeue_pushable_task(struct rq *rq, struct task_struct *p) +static void dequeue_pushable_task(struct rt_rq *rt_rq, struct task_struct = *p) { - plist_del(&p->pushable_tasks, &rq->rt.pushable_tasks); + plist_del(&p->pushable_tasks, &rt_rq->pushable_tasks); =20 /* Update the new highest prio pushable task */ - if (has_pushable_tasks(rq)) { - p =3D plist_first_entry(&rq->rt.pushable_tasks, + if (has_pushable_tasks(rt_rq)) { + p =3D plist_first_entry(&rt_rq->pushable_tasks, struct task_struct, pushable_tasks); - rq->rt.highest_prio.next =3D p->prio; + rt_rq->highest_prio.next =3D p->prio; } else { - rq->rt.highest_prio.next =3D MAX_RT_PRIO-1; + rt_rq->highest_prio.next =3D MAX_RT_PRIO-1; =20 - if (rq->rt.overloaded) { - rt_clear_overload(rq); - rq->rt.overloaded =3D 0; + if (rt_rq->overloaded) { + rt_clear_overload(rq_of_rt_rq(rt_rq)); + rt_rq->overloaded =3D 0; } } } @@ -1431,6 +1435,7 @@ static void enqueue_task_rt(struct rq *rq, struct task_struct *p, int flags) { struct sched_rt_entity *rt_se =3D &p->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); =20 if (flags & ENQUEUE_WAKEUP) rt_se->timeout =3D 0; @@ -1444,17 +1449,18 @@ enqueue_task_rt(struct rq *rq, struct task_struct *= p, int flags) return; =20 if (!task_current(rq, p) && p->nr_cpus_allowed > 1) - enqueue_pushable_task(rq, p); + enqueue_pushable_task(rt_rq, p); } =20 static bool dequeue_task_rt(struct rq *rq, struct task_struct *p, int flag= s) { struct sched_rt_entity *rt_se =3D &p->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); =20 update_curr_rt(rq); dequeue_rt_entity(rt_se, flags); =20 - dequeue_pushable_task(rq, p); + dequeue_pushable_task(rt_rq, p); =20 return true; } @@ -1639,14 +1645,14 @@ static void wakeup_preempt_rt(struct rq *rq, struct= task_struct *p, int flags) static inline void set_next_task_rt(struct rq *rq, struct task_struct *p, = bool first) { struct sched_rt_entity *rt_se =3D &p->rt; - struct rt_rq *rt_rq =3D &rq->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); =20 p->se.exec_start =3D rq_clock_task(rq); if (on_rt_rq(&p->rt)) update_stats_wait_end_rt(rt_rq, rt_se); =20 /* The running task is never eligible for pushing */ - dequeue_pushable_task(rq, p); + dequeue_pushable_task(rt_rq, p); =20 if (!first) return; @@ -1659,7 +1665,7 @@ static inline void set_next_task_rt(struct rq *rq, st= ruct task_struct *p, bool f if (rq->donor->sched_class !=3D &rt_sched_class) update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 0); =20 - rt_queue_push_tasks(rq); + rt_queue_push_tasks(rt_rq); } =20 static struct sched_rt_entity *pick_next_rt_entity(struct rt_rq *rt_rq) @@ -1710,7 +1716,7 @@ static struct task_struct *pick_task_rt(struct rq *rq) static void put_prev_task_rt(struct rq *rq, struct task_struct *p, struct = task_struct *next) { struct sched_rt_entity *rt_se =3D &p->rt; - struct rt_rq *rt_rq =3D &rq->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); =20 if (on_rt_rq(&p->rt)) update_stats_wait_start_rt(rt_rq, rt_se); @@ -1726,7 +1732,7 @@ static void put_prev_task_rt(struct rq *rq, struct ta= sk_struct *p, struct task_s * if it is still active */ if (on_rt_rq(&p->rt) && p->nr_cpus_allowed > 1) - enqueue_pushable_task(rq, p); + enqueue_pushable_task(rt_rq, p); } =20 /* Only try algorithms three times */ @@ -1736,16 +1742,16 @@ static void put_prev_task_rt(struct rq *rq, struct = task_struct *p, struct task_s * Return the highest pushable rq's task, which is suitable to be executed * on the CPU, NULL otherwise */ -static struct task_struct *pick_highest_pushable_task(struct rq *rq, int c= pu) +static struct task_struct *pick_highest_pushable_task(struct rt_rq *rt_rq,= int cpu) { - struct plist_head *head =3D &rq->rt.pushable_tasks; + struct plist_head *head =3D &rt_rq->pushable_tasks; struct task_struct *p; =20 - if (!has_pushable_tasks(rq)) + if (!has_pushable_tasks(rt_rq)) return NULL; =20 plist_for_each_entry(p, head, pushable_tasks) { - if (task_is_pushable(rq, p, cpu)) + if (task_is_pushable(rq_of_rt_rq(rt_rq), p, cpu)) return p; } =20 @@ -1845,14 +1851,15 @@ static int find_lowest_rq(struct task_struct *task) return -1; } =20 -static struct task_struct *pick_next_pushable_task(struct rq *rq) +static struct task_struct *pick_next_pushable_task(struct rt_rq *rt_rq) { + struct rq *rq =3D rq_of_rt_rq(rt_rq); struct task_struct *p; =20 - if (!has_pushable_tasks(rq)) + if (!has_pushable_tasks(rt_rq)) return NULL; =20 - p =3D plist_first_entry(&rq->rt.pushable_tasks, + p =3D plist_first_entry(&rt_rq->pushable_tasks, struct task_struct, pushable_tasks); =20 BUG_ON(rq->cpu !=3D task_cpu(p)); @@ -1905,7 +1912,7 @@ static struct rq *find_lock_lowest_rq(struct task_str= uct *task, struct rq *rq) */ if (unlikely(is_migration_disabled(task) || !cpumask_test_cpu(lowest_rq->cpu, &task->cpus_mask) || - task !=3D pick_next_pushable_task(rq))) { + task !=3D pick_next_pushable_task(&rq->rt))) { =20 double_unlock_balance(rq, lowest_rq); lowest_rq =3D NULL; @@ -1939,7 +1946,7 @@ static int push_rt_task(struct rq *rq, bool pull) if (!rq->rt.overloaded) return 0; =20 - next_task =3D pick_next_pushable_task(rq); + next_task =3D pick_next_pushable_task(&rq->rt); if (!next_task) return 0; =20 @@ -2014,7 +2021,7 @@ static int push_rt_task(struct rq *rq, bool pull) * run-queue and is also still the next task eligible for * pushing. */ - task =3D pick_next_pushable_task(rq); + task =3D pick_next_pushable_task(&rq->rt); if (task =3D=3D next_task) { /* * The task hasn't migrated, and is still the next @@ -2202,7 +2209,7 @@ void rto_push_irq_work_func(struct irq_work *work) * We do not need to grab the lock to check for has_pushable_tasks. * When it gets updated, a check is made if a push is possible. */ - if (has_pushable_tasks(rq)) { + if (has_pushable_tasks(&rq->rt)) { raw_spin_rq_lock(rq); while (push_rt_task(rq, true)) ; @@ -2231,6 +2238,7 @@ static void pull_rt_task(struct rq *this_rq) int this_cpu =3D this_rq->cpu, cpu; bool resched =3D false; struct task_struct *p, *push_task; + struct rt_rq *src_rt_rq; struct rq *src_rq; int rt_overload_count =3D rt_overloaded(this_rq); =20 @@ -2260,6 +2268,7 @@ static void pull_rt_task(struct rq *this_rq) continue; =20 src_rq =3D cpu_rq(cpu); + src_rt_rq =3D &src_rq->rt; =20 /* * Don't bother taking the src_rq->lock if the next highest @@ -2268,7 +2277,7 @@ static void pull_rt_task(struct rq *this_rq) * logically higher, the src_rq will push this task away. * And if its going logically lower, we do not care */ - if (src_rq->rt.highest_prio.next >=3D + if (src_rt_rq->highest_prio.next >=3D this_rq->rt.highest_prio.curr) continue; =20 @@ -2284,7 +2293,7 @@ static void pull_rt_task(struct rq *this_rq) * We can pull only a task, which is pushable * on its rq, and no others. */ - p =3D pick_highest_pushable_task(src_rq, this_cpu); + p =3D pick_highest_pushable_task(src_rt_rq, this_cpu); =20 /* * Do we have an RT task that preempts @@ -2390,7 +2399,7 @@ static void switched_from_rt(struct rq *rq, struct ta= sk_struct *p) if (!task_on_rq_queued(p) || rq->rt.rt_nr_running) return; =20 - rt_queue_pull_task(rq); + rt_queue_pull_task(rt_rq_of_se(&p->rt)); } =20 void __init init_sched_rt_class(void) @@ -2426,7 +2435,7 @@ static void switched_to_rt(struct rq *rq, struct task= _struct *p) */ if (task_on_rq_queued(p)) { if (p->nr_cpus_allowed > 1 && rq->rt.overloaded) - rt_queue_push_tasks(rq); + rt_queue_push_tasks(rt_rq_of_se(&p->rt)); if (p->prio < rq->donor->prio && cpu_online(cpu_of(rq))) resched_curr(rq); } @@ -2448,7 +2457,7 @@ prio_changed_rt(struct rq *rq, struct task_struct *p,= int oldprio) * may need to pull tasks to this runqueue. */ if (oldprio < p->prio) - rt_queue_pull_task(rq); + rt_queue_pull_task(rt_rq_of_se(&p->rt)); =20 /* * If there's a higher priority task waiting to run --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 001D42868AD for ; Mon, 29 Sep 2025 09:22:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137749; cv=none; b=co4j5T+3bslNW4Ga0uJDm84P7oTVxnhabyH/cpdxf9eL3NmIekvJz+0JhLZa4yEV3TQJkIkbJ0Z+mhgkpNmstqF1pu60DsEdmVyzzpRaB6Px5RvBjfHmzqQut1GAYtiTtCMLlNylXsNLFzworb8iKG7sjBYRgVgk/XGRElzlCX0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137749; c=relaxed/simple; bh=48+i5rW7eRi70EVsLTYHWmDFwwj22lT+qIPuNDi/IYg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HhVIm4FvyOIQZpgD4dcrkJPZCDPuNCNATTe8K3MmifbizRjIrt456mifNvvXVT4E2HrhvW3HYtetqUl/Y7AE8TdQ42lL1gUiK5//Qeune9g51J7v9NTL1laYhbAs9aSu1bcmx1xO94pbo3Bq2a7pHc3IZ3TcZN9VcJm2HdvqwaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Bf/Q5C5x; arc=none smtp.client-ip=209.85.208.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Bf/Q5C5x" Received: by mail-ed1-f50.google.com with SMTP id 4fb4d7f45d1cf-634c01ca9dcso4426051a12.3 for ; Mon, 29 Sep 2025 02:22:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137746; x=1759742546; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hoGXqYKPgkRonEfbC7CzVFU8BRc0pw6mQcE9m7FqR9c=; b=Bf/Q5C5xSMkA0nY83lpw0v43kPnmqLpq3DoH3l5RYPU+lP0KG2ri5KgrJnIxt7uvvO TLAcKuHEsQoHw0/or+XTpUfYa75kuRmKU4fKQWb6u55/XjQDg4+zjLJPoMsNb3UE8xs9 xJF4VeK3Fb0R3rWTQvO0VMI6XojX8Gq5iQcDhpqIHDWjGOTesSJ9z8xYsFL9iLk9DfuU pF4iVDX9EXxz6mf+UZ075r5LwcX9hH/urIsa+Z2Cw/xzeVb2SJjZnReOIsBZFf6OtnSS hnN0871duZFvMnJ91IcNOjXE3euL3vpV9SMDUG0IS25zcqwgDanOZJACbUcycxDTjmeb 8g3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137746; x=1759742546; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hoGXqYKPgkRonEfbC7CzVFU8BRc0pw6mQcE9m7FqR9c=; b=ODsln4R61hxf9RxW5BR2Xc4yT6HTDpXMUYC50B6YoPqTlLOrQlyagKT2DLtOSZqpUu W7pJqkTrRf0xe4afsBK7atL+2BTMWUEhTxZz4lFAc7JMEuU/BXEDh4mkwL0NlEyWv+Ku Vxqjf3SHk3F+oY6YwiySm4i3mY/idVgo4LLG/vrTyDIVmi0O/iENJ75Hj0S8uAZrmHQI 1H5faECWa/6T2ylsVKIjZfXuLnEiCjRz1KoIKyOcra+oz5mKdbLcEpl8e9lHnLYemytF r6Qj0bgCScICWxw0qzBSC1YRaQOJIvXAdMF+C9Ovd01+WDfRbzrAcRn8ef4U18q0v/S8 8/cA== X-Gm-Message-State: AOJu0YztLHWt5eprLSP4es9PBcfN3Hee5dQiOnvFlQ9CqWXOFDjIXTXW rUeM8c7Y7sT9U5BdJrBzDILWiGAn16oOScEtTzmh1bOLwZchTKz/zE4cHwBjOn2T X-Gm-Gg: ASbGnctxyUObyvXManIQKUq6dxmC/9MLXh31FSH/nOaSKApyWGNLQELrOL3/218cirv TAcU+baXfL5rgPCjlFqYYWo9GUDgR1hbobbavPQW8ooiU2Hw3S79SQnieJJqPn2Jn28/rghsQsZ BYI+8bUMMywYdMZUU2b8/b7J39q3v31c6YVQ6adceWxonqml9BZtWEYQiRGLeasa5wPO5Xr4CTg B7WwU8WJKyEDhtmJPjIqbSWDBAKPS+5rcXs1natpxWpzBeYoEnShvhjcEQPggqD3BHBClK38QPi pEGwAvXYyefWs6hn3XFA1dj+5QfzUMoW2Pa2F20Qib7H8g1ZaqF3eN7z9xLeIsee/zczFWDP4gg mEYPYhdzTW/0SVcC8mj36r0FjzvUjezSuczhM6z5t X-Google-Smtp-Source: AGHT+IEhLbBZpwHYi84vsUqDkJBVR0XSQ1j4ZTzHZHefq61fvkR7VVVp7HZzaryM4lzsIFXoNetFNw== X-Received: by 2002:a17:907:9305:b0:b10:9e5d:d173 with SMTP id a640c23a62f3a-b34be3ebc89mr1786875766b.41.1759137746192; Mon, 29 Sep 2025 02:22:26 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:25 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 04/24] sched/rt: Move some functions from rt.c to sched.h Date: Mon, 29 Sep 2025 11:22:01 +0200 Message-ID: <20250929092221.10947-5-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make the following functions be non-static and move them in sched.h, so that they can be used also in other source files: - rt_task_of() - rq_of_rt_rq() - rt_rq_of_se() - rq_of_rt_se() There are no functional changes. This is needed by future patches. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 52 ------------------------------------------ kernel/sched/sched.h | 54 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+), 52 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index a7d063d2303..a599f63bf7f 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -168,34 +168,6 @@ static void destroy_rt_bandwidth(struct rt_bandwidth *= rt_b) =20 #define rt_entity_is_task(rt_se) (!(rt_se)->my_q) =20 -static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) -{ - WARN_ON_ONCE(!rt_entity_is_task(rt_se)); - - return container_of(rt_se, struct task_struct, rt); -} - -static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) -{ - /* Cannot fold with non-CONFIG_RT_GROUP_SCHED version, layout */ - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); - return rt_rq->rq; -} - -static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) -{ - WARN_ON(!rt_group_sched_enabled() && rt_se->rt_rq->tg !=3D &root_task_gro= up); - return rt_se->rt_rq; -} - -static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) -{ - struct rt_rq *rt_rq =3D rt_se->rt_rq; - - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); - return rt_rq->rq; -} - void unregister_rt_sched_group(struct task_group *tg) { if (!rt_group_sched_enabled()) @@ -296,30 +268,6 @@ int alloc_rt_sched_group(struct task_group *tg, struct= task_group *parent) =20 #define rt_entity_is_task(rt_se) (1) =20 -static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) -{ - return container_of(rt_se, struct task_struct, rt); -} - -static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) -{ - return container_of(rt_rq, struct rq, rt); -} - -static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) -{ - struct task_struct *p =3D rt_task_of(rt_se); - - return task_rq(p); -} - -static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) -{ - struct rq *rq =3D rq_of_rt_se(rt_se); - - return &rq->rt; -} - void unregister_rt_sched_group(struct task_group *tg) { } =20 void free_rt_sched_group(struct task_group *tg) { } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 2499b36a3c5..831ab576842 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3040,6 +3040,60 @@ extern void set_rq_offline(struct rq *rq); =20 extern bool sched_smp_initialized; =20 +#ifdef CONFIG_RT_GROUP_SCHED +static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) +{ + WARN_ON_ONCE(rt_se->my_q); + + return container_of(rt_se, struct task_struct, rt); +} + +static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) +{ + /* Cannot fold with non-CONFIG_RT_GROUP_SCHED version, layout */ + WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); + return rt_rq->rq; +} + +static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) +{ + WARN_ON(!rt_group_sched_enabled() && rt_se->rt_rq->tg !=3D &root_task_gro= up); + return rt_se->rt_rq; +} + +static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) +{ + struct rt_rq *rt_rq =3D rt_se->rt_rq; + + WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); + return rt_rq->rq; +} +#else +static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) +{ + return container_of(rt_se, struct task_struct, rt); +} + +static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) +{ + return container_of(rt_rq, struct rq, rt); +} + +static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) +{ + struct task_struct *p =3D rt_task_of(rt_se); + + return task_rq(p); +} + +static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) +{ + struct rq *rq =3D rq_of_rt_se(rt_se); + + return &rq->rt; +} +#endif + DEFINE_LOCK_GUARD_2(double_rq_lock, struct rq, double_rq_lock(_T->lock, _T->lock2), double_rq_unlock(_T->lock, _T->lock2)) --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18C2E2877F4 for ; Mon, 29 Sep 2025 09:22:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137752; cv=none; b=FAbgNefwmJrEolhADIzgoEa2lnlKR5/raDtwvAlU/iiRBzkg6clPL5JdVUycFFOTBYdgG0E9aVc3eMeHcGCatlqf+uLI2phUgU19VThPwoNWwVnLLwXWYWVWNcrwOw0eqU7mCQ2UrMCV7ogbAHhWllEd+TC2ZrkBdqW92GOTkK8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137752; c=relaxed/simple; bh=zL1bHqEMUQn5i45uZxHN7e6EDkXqoRh7+FVEdcP6sDY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TM4JSnKCmeZwyTu19+MUMT9B4/atcb1YEKOKMIFa0Nt3nPmLya3u9vJUkLLW+uLsMk4apSEO+8iGwxOAeXFooo7hYLevNQAK+T3bf29uWKu1FW4Ki44KjtGPRCF0WIPkvRswNaur/Tc8NnuefPZoPGQ7/Xz5XxaDYMCj3hNA5jw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=L5msoRZR; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="L5msoRZR" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-b3b3a6f4dd4so290477266b.0 for ; Mon, 29 Sep 2025 02:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137747; x=1759742547; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/FOfooFC2bX7cd8gntjpv3C3EQC9mZMfkaQz0RD3iJs=; b=L5msoRZRPuCDUSlZfzO2pU8e/Dd2SOo56HAr9DLvM3UZUB1MQ5uWc1hC9m99hCpj+g S4TPiKybLYVHnGO/foZEfg5tiiQLIGqhw2r88o91YbbYvJk4oAPORvidyGxoZ2jxfOQv cvA2a6dws9MexNIq43iP3lxr+cBbxnCfXEdNGw9p5LrGzf61Sq6kinmph+2Qi6TAmceq fQgbNr+AkmP37tSE4A1nJ057Ezyuqb0Mm5oSyZCkuBmdLtmwNofKKMiGIeQTW0pifjuG RAhMg6+MlBhYyJoq6s7MLZ3Fi8xEOgDldbAZlhvvOqYNeE9E4vQ7gZgjO4ICaEXKIs21 aWtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137747; x=1759742547; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/FOfooFC2bX7cd8gntjpv3C3EQC9mZMfkaQz0RD3iJs=; b=a2pKv9JxA+JSS9+09Q1HppIIX7sZfO4csEKVqv5SpFax6gBEf5Se86Ban8FPpZf0Iw 6bFzWe1dqiqkosl4+hyZiT/kH+Wcs76J9PwkdS03TRsWmXInCsykzeWvORh/RhjDCvpN k+EnrHUID0g6UQE7o55oG9uVOLlsm1BoFHHysfJTqkJSvFkAV/KMIv6aQRUeZQgI0sTg zkcaKAy/kxJHjOYeT4OxgMnaC0azL2G8+EnrpXd49L/dSu5UmSRn8RkmfqhHkB5EHlAL QJmfd6Woe4nutjpsg02jy/qgHEslqqk8Madqw1oaC/OAkdyXdsfsGYzNBlkhhgqznysm eZ7w== X-Gm-Message-State: AOJu0YxiOGNsZ9bLPKTkZwqC3kCb8vm8x6aEbSKtRfLQVTq+9U2cmBGc J5pPEHk1J7vbNwaSd7bncEgA1KR3BPzHWalHr8xzFR2xzleW7FtTXLm1 X-Gm-Gg: ASbGncsfPv95NPMYxgV0emcTX5uTbeNsriLFln6JIH0/9rU6xdyC//P106a1+gyH64P nez3lw/VBeZig/ZB4WEdrMkv4d2VV+rxHAvjP3RIouzCGUiLJVFfrNe0kDysAulYavdNUht0ROJ cAdCqn9BpNm99ZZXFFaF0w88CeIwWIl8hBn2j8rTCPiYSR21SALStNlwdUHuXHDzu2lXX7XGpkC ohphERORZUDYbn6YN3XcxTCMLw2QcxwbmBZxiAZj22JWV+RovpIj4V408Rmpdn1ae3NAozMhVcY Fm40R6slqT+yaMlXE7nBTOEfpBbgKBNVtmQeXd5KBnPihbS/kHQgzi4cUSeLWac5F/WB3B5JS1g wf0sgpLoqDoXjUe4XgVKjeS6bLU7pRg== X-Google-Smtp-Source: AGHT+IEdG7PWyuPLmx6MU1SDC6j5PTNN+RDGsbGfRn8AxWOX42LCLVxwprv/TPqkrn1JsyPlffI+dA== X-Received: by 2002:a17:907:d18:b0:b3d:7914:6bbd with SMTP id a640c23a62f3a-b3d7914713fmr458559866b.22.1759137747050; Mon, 29 Sep 2025 02:22:27 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:26 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 05/24] sched/rt: Disable RT_GROUP_SCHED Date: Mon, 29 Sep 2025 11:22:02 +0200 Message-ID: <20250929092221.10947-6-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Disable the old RT_GROUP_SCHED scheduler. Note that this does not completely remove all the RT_GROUP_SCHED functionality, just unhooks it and removes mo= st of the relevant functions. Some of the RT_GROUP_SCHED functions are kept becau= se they will be adapted for the HCBS scheduling. Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 6 - kernel/sched/deadline.c | 34 -- kernel/sched/debug.c | 6 - kernel/sched/rt.c | 848 ++-------------------------------------- kernel/sched/sched.h | 11 +- kernel/sched/syscalls.c | 13 - 6 files changed, 26 insertions(+), 892 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index ccba6fc3c3f..5791aa1f8c8 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8721,11 +8721,6 @@ void __init sched_init(void) =20 init_defrootdomain(); =20 -#ifdef CONFIG_RT_GROUP_SCHED - init_rt_bandwidth(&root_task_group.rt_bandwidth, - global_rt_period(), global_rt_runtime()); -#endif /* CONFIG_RT_GROUP_SCHED */ - #ifdef CONFIG_CGROUP_SCHED task_group_cache =3D KMEM_CACHE(task_group, 0); =20 @@ -8777,7 +8772,6 @@ void __init sched_init(void) * starts working after scheduler_running, which is not the case * yet. */ - rq->rt.rt_runtime =3D global_rt_runtime(); init_tg_rt_entry(&root_task_group, &rq->rt, NULL, i, NULL); #endif rq->sd =3D NULL; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 6ff00f71041..277fbaff8b5 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1508,40 +1508,6 @@ static void update_curr_dl_se(struct rq *rq, struct = sched_dl_entity *dl_se, s64 if (!is_leftmost(dl_se, &rq->dl)) resched_curr(rq); } - - /* - * The fair server (sole dl_server) does not account for real-time - * workload because it is running fair work. - */ - if (dl_se =3D=3D &rq->fair_server) - return; - -#ifdef CONFIG_RT_GROUP_SCHED - /* - * Because -- for now -- we share the rt bandwidth, we need to - * account our runtime there too, otherwise actual rt tasks - * would be able to exceed the shared quota. - * - * Account to the root rt group for now. - * - * The solution we're working towards is having the RT groups scheduled - * using deadline servers -- however there's a few nasties to figure - * out before that can happen. - */ - if (rt_bandwidth_enabled()) { - struct rt_rq *rt_rq =3D &rq->rt; - - raw_spin_lock(&rt_rq->rt_runtime_lock); - /* - * We'll let actual RT tasks worry about the overflow here, we - * have our own CBS to keep us inline; only account when RT - * bandwidth is relevant. - */ - if (sched_rt_bandwidth_account(rt_rq)) - rt_rq->rt_time +=3D delta_exec; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - } -#endif /* CONFIG_RT_GROUP_SCHED */ } =20 /* diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 02e16b70a79..efcf8d82f85 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -890,12 +890,6 @@ void print_rt_rq(struct seq_file *m, int cpu, struct r= t_rq *rt_rq) =20 PU(rt_nr_running); =20 -#ifdef CONFIG_RT_GROUP_SCHED - P(rt_throttled); - PN(rt_time); - PN(rt_runtime); -#endif - #undef PN #undef PU #undef P diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index a599f63bf7f..c625ea45ca7 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,3 +1,4 @@ +#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR @@ -82,117 +83,18 @@ void init_rt_rq(struct rt_rq *rt_rq) rt_rq->highest_prio.next =3D MAX_RT_PRIO-1; rt_rq->overloaded =3D 0; plist_head_init(&rt_rq->pushable_tasks); - /* We start is dequeued state, because no RT tasks are queued */ - rt_rq->rt_queued =3D 0; - -#ifdef CONFIG_RT_GROUP_SCHED - rt_rq->rt_time =3D 0; - rt_rq->rt_throttled =3D 0; - rt_rq->rt_runtime =3D 0; - raw_spin_lock_init(&rt_rq->rt_runtime_lock); - rt_rq->tg =3D &root_task_group; -#endif } =20 #ifdef CONFIG_RT_GROUP_SCHED =20 -static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun= ); - -static enum hrtimer_restart sched_rt_period_timer(struct hrtimer *timer) -{ - struct rt_bandwidth *rt_b =3D - container_of(timer, struct rt_bandwidth, rt_period_timer); - int idle =3D 0; - int overrun; - - raw_spin_lock(&rt_b->rt_runtime_lock); - for (;;) { - overrun =3D hrtimer_forward_now(timer, rt_b->rt_period); - if (!overrun) - break; - - raw_spin_unlock(&rt_b->rt_runtime_lock); - idle =3D do_sched_rt_period_timer(rt_b, overrun); - raw_spin_lock(&rt_b->rt_runtime_lock); - } - if (idle) - rt_b->rt_period_active =3D 0; - raw_spin_unlock(&rt_b->rt_runtime_lock); - - return idle ? HRTIMER_NORESTART : HRTIMER_RESTART; -} - -void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime) -{ - rt_b->rt_period =3D ns_to_ktime(period); - rt_b->rt_runtime =3D runtime; - - raw_spin_lock_init(&rt_b->rt_runtime_lock); - - hrtimer_setup(&rt_b->rt_period_timer, sched_rt_period_timer, CLOCK_MONOTO= NIC, - HRTIMER_MODE_REL_HARD); -} - -static inline void do_start_rt_bandwidth(struct rt_bandwidth *rt_b) -{ - raw_spin_lock(&rt_b->rt_runtime_lock); - if (!rt_b->rt_period_active) { - rt_b->rt_period_active =3D 1; - /* - * SCHED_DEADLINE updates the bandwidth, as a run away - * RT task with a DL task could hog a CPU. But DL does - * not reset the period. If a deadline task was running - * without an RT task running, it can cause RT tasks to - * throttle when they start up. Kick the timer right away - * to update the period. - */ - hrtimer_forward_now(&rt_b->rt_period_timer, ns_to_ktime(0)); - hrtimer_start_expires(&rt_b->rt_period_timer, - HRTIMER_MODE_ABS_PINNED_HARD); - } - raw_spin_unlock(&rt_b->rt_runtime_lock); -} - -static void start_rt_bandwidth(struct rt_bandwidth *rt_b) -{ - if (!rt_bandwidth_enabled() || rt_b->rt_runtime =3D=3D RUNTIME_INF) - return; - - do_start_rt_bandwidth(rt_b); -} - -static void destroy_rt_bandwidth(struct rt_bandwidth *rt_b) -{ - hrtimer_cancel(&rt_b->rt_period_timer); -} - -#define rt_entity_is_task(rt_se) (!(rt_se)->my_q) - void unregister_rt_sched_group(struct task_group *tg) { - if (!rt_group_sched_enabled()) - return; - - if (tg->rt_se) - destroy_rt_bandwidth(&tg->rt_bandwidth); } =20 void free_rt_sched_group(struct task_group *tg) { - int i; - if (!rt_group_sched_enabled()) return; - - for_each_possible_cpu(i) { - if (tg->rt_rq) - kfree(tg->rt_rq[i]); - if (tg->rt_se) - kfree(tg->rt_se[i]); - } - - kfree(tg->rt_rq); - kfree(tg->rt_se); } =20 void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq, @@ -202,72 +104,23 @@ void init_tg_rt_entry(struct task_group *tg, struct r= t_rq *rt_rq, struct rq *rq =3D cpu_rq(cpu); =20 rt_rq->highest_prio.curr =3D MAX_RT_PRIO-1; - rt_rq->rt_nr_boosted =3D 0; rt_rq->rq =3D rq; rt_rq->tg =3D tg; =20 tg->rt_rq[cpu] =3D rt_rq; tg->rt_se[cpu] =3D rt_se; - - if (!rt_se) - return; - - if (!parent) - rt_se->rt_rq =3D &rq->rt; - else - rt_se->rt_rq =3D parent->my_q; - - rt_se->my_q =3D rt_rq; - rt_se->parent =3D parent; - INIT_LIST_HEAD(&rt_se->run_list); } =20 int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) { - struct rt_rq *rt_rq; - struct sched_rt_entity *rt_se; - int i; - if (!rt_group_sched_enabled()) return 1; =20 - tg->rt_rq =3D kcalloc(nr_cpu_ids, sizeof(rt_rq), GFP_KERNEL); - if (!tg->rt_rq) - goto err; - tg->rt_se =3D kcalloc(nr_cpu_ids, sizeof(rt_se), GFP_KERNEL); - if (!tg->rt_se) - goto err; - - init_rt_bandwidth(&tg->rt_bandwidth, ktime_to_ns(global_rt_period()), 0); - - for_each_possible_cpu(i) { - rt_rq =3D kzalloc_node(sizeof(struct rt_rq), - GFP_KERNEL, cpu_to_node(i)); - if (!rt_rq) - goto err; - - rt_se =3D kzalloc_node(sizeof(struct sched_rt_entity), - GFP_KERNEL, cpu_to_node(i)); - if (!rt_se) - goto err_free_rq; - - init_rt_rq(rt_rq); - rt_rq->rt_runtime =3D tg->rt_bandwidth.rt_runtime; - init_tg_rt_entry(tg, rt_rq, rt_se, i, parent->rt_se[i]); - } - return 1; - -err_free_rq: - kfree(rt_rq); -err: - return 0; } =20 #else /* !CONFIG_RT_GROUP_SCHED: */ =20 -#define rt_entity_is_task(rt_se) (1) - void unregister_rt_sched_group(struct task_group *tg) { } =20 void free_rt_sched_group(struct task_group *tg) { } @@ -381,9 +234,6 @@ static void dequeue_pushable_task(struct rt_rq *rt_rq, = struct task_struct *p) } } =20 -static void enqueue_top_rt_rq(struct rt_rq *rt_rq); -static void dequeue_top_rt_rq(struct rt_rq *rt_rq, unsigned int count); - static inline int on_rt_rq(struct sched_rt_entity *rt_se) { return rt_se->on_rq; @@ -430,16 +280,6 @@ static inline bool rt_task_fits_capacity(struct task_s= truct *p, int cpu) =20 #ifdef CONFIG_RT_GROUP_SCHED =20 -static inline u64 sched_rt_runtime(struct rt_rq *rt_rq) -{ - return rt_rq->rt_runtime; -} - -static inline u64 sched_rt_period(struct rt_rq *rt_rq) -{ - return ktime_to_ns(rt_rq->tg->rt_bandwidth.rt_period); -} - typedef struct task_group *rt_rq_iter_t; =20 static inline struct task_group *next_task_group(struct task_group *tg) @@ -465,457 +305,20 @@ static inline struct task_group *next_task_group(str= uct task_group *tg) iter && (rt_rq =3D iter->rt_rq[cpu_of(rq)]); \ iter =3D next_task_group(iter)) =20 -#define for_each_sched_rt_entity(rt_se) \ - for (; rt_se; rt_se =3D rt_se->parent) - -static inline struct rt_rq *group_rt_rq(struct sched_rt_entity *rt_se) -{ - return rt_se->my_q; -} - static void enqueue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags); static void dequeue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags); =20 -static void sched_rt_rq_enqueue(struct rt_rq *rt_rq) -{ - struct task_struct *donor =3D rq_of_rt_rq(rt_rq)->donor; - struct rq *rq =3D rq_of_rt_rq(rt_rq); - struct sched_rt_entity *rt_se; - - int cpu =3D cpu_of(rq); - - rt_se =3D rt_rq->tg->rt_se[cpu]; - - if (rt_rq->rt_nr_running) { - if (!rt_se) - enqueue_top_rt_rq(rt_rq); - else if (!on_rt_rq(rt_se)) - enqueue_rt_entity(rt_se, 0); - - if (rt_rq->highest_prio.curr < donor->prio) - resched_curr(rq); - } -} - -static void sched_rt_rq_dequeue(struct rt_rq *rt_rq) -{ - struct sched_rt_entity *rt_se; - int cpu =3D cpu_of(rq_of_rt_rq(rt_rq)); - - rt_se =3D rt_rq->tg->rt_se[cpu]; - - if (!rt_se) { - dequeue_top_rt_rq(rt_rq, rt_rq->rt_nr_running); - /* Kick cpufreq (see the comment in kernel/sched/sched.h). */ - cpufreq_update_util(rq_of_rt_rq(rt_rq), 0); - } - else if (on_rt_rq(rt_se)) - dequeue_rt_entity(rt_se, 0); -} - -static inline int rt_rq_throttled(struct rt_rq *rt_rq) -{ - return rt_rq->rt_throttled && !rt_rq->rt_nr_boosted; -} - -static int rt_se_boosted(struct sched_rt_entity *rt_se) -{ - struct rt_rq *rt_rq =3D group_rt_rq(rt_se); - struct task_struct *p; - - if (rt_rq) - return !!rt_rq->rt_nr_boosted; - - p =3D rt_task_of(rt_se); - return p->prio !=3D p->normal_prio; -} - -static inline const struct cpumask *sched_rt_period_mask(void) -{ - return this_rq()->rd->span; -} - -static inline -struct rt_rq *sched_rt_period_rt_rq(struct rt_bandwidth *rt_b, int cpu) -{ - return container_of(rt_b, struct task_group, rt_bandwidth)->rt_rq[cpu]; -} - -static inline struct rt_bandwidth *sched_rt_bandwidth(struct rt_rq *rt_rq) -{ - return &rt_rq->tg->rt_bandwidth; -} - -bool sched_rt_bandwidth_account(struct rt_rq *rt_rq) -{ - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - - return (hrtimer_active(&rt_b->rt_period_timer) || - rt_rq->rt_time < rt_b->rt_runtime); -} - -/* - * We ran out of runtime, see if we can borrow some from our neighbours. - */ -static void do_balance_runtime(struct rt_rq *rt_rq) -{ - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - struct root_domain *rd =3D rq_of_rt_rq(rt_rq)->rd; - int i, weight; - u64 rt_period; - - weight =3D cpumask_weight(rd->span); - - raw_spin_lock(&rt_b->rt_runtime_lock); - rt_period =3D ktime_to_ns(rt_b->rt_period); - for_each_cpu(i, rd->span) { - struct rt_rq *iter =3D sched_rt_period_rt_rq(rt_b, i); - s64 diff; - - if (iter =3D=3D rt_rq) - continue; - - raw_spin_lock(&iter->rt_runtime_lock); - /* - * Either all rqs have inf runtime and there's nothing to steal - * or __disable_runtime() below sets a specific rq to inf to - * indicate its been disabled and disallow stealing. - */ - if (iter->rt_runtime =3D=3D RUNTIME_INF) - goto next; - - /* - * From runqueues with spare time, take 1/n part of their - * spare time, but no more than our period. - */ - diff =3D iter->rt_runtime - iter->rt_time; - if (diff > 0) { - diff =3D div_u64((u64)diff, weight); - if (rt_rq->rt_runtime + diff > rt_period) - diff =3D rt_period - rt_rq->rt_runtime; - iter->rt_runtime -=3D diff; - rt_rq->rt_runtime +=3D diff; - if (rt_rq->rt_runtime =3D=3D rt_period) { - raw_spin_unlock(&iter->rt_runtime_lock); - break; - } - } -next: - raw_spin_unlock(&iter->rt_runtime_lock); - } - raw_spin_unlock(&rt_b->rt_runtime_lock); -} - -/* - * Ensure this RQ takes back all the runtime it lend to its neighbours. - */ -static void __disable_runtime(struct rq *rq) -{ - struct root_domain *rd =3D rq->rd; - rt_rq_iter_t iter; - struct rt_rq *rt_rq; - - if (unlikely(!scheduler_running)) - return; - - for_each_rt_rq(rt_rq, iter, rq) { - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - s64 want; - int i; - - raw_spin_lock(&rt_b->rt_runtime_lock); - raw_spin_lock(&rt_rq->rt_runtime_lock); - /* - * Either we're all inf and nobody needs to borrow, or we're - * already disabled and thus have nothing to do, or we have - * exactly the right amount of runtime to take out. - */ - if (rt_rq->rt_runtime =3D=3D RUNTIME_INF || - rt_rq->rt_runtime =3D=3D rt_b->rt_runtime) - goto balanced; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - - /* - * Calculate the difference between what we started out with - * and what we current have, that's the amount of runtime - * we lend and now have to reclaim. - */ - want =3D rt_b->rt_runtime - rt_rq->rt_runtime; - - /* - * Greedy reclaim, take back as much as we can. - */ - for_each_cpu(i, rd->span) { - struct rt_rq *iter =3D sched_rt_period_rt_rq(rt_b, i); - s64 diff; - - /* - * Can't reclaim from ourselves or disabled runqueues. - */ - if (iter =3D=3D rt_rq || iter->rt_runtime =3D=3D RUNTIME_INF) - continue; - - raw_spin_lock(&iter->rt_runtime_lock); - if (want > 0) { - diff =3D min_t(s64, iter->rt_runtime, want); - iter->rt_runtime -=3D diff; - want -=3D diff; - } else { - iter->rt_runtime -=3D want; - want -=3D want; - } - raw_spin_unlock(&iter->rt_runtime_lock); - - if (!want) - break; - } - - raw_spin_lock(&rt_rq->rt_runtime_lock); - /* - * We cannot be left wanting - that would mean some runtime - * leaked out of the system. - */ - WARN_ON_ONCE(want); -balanced: - /* - * Disable all the borrow logic by pretending we have inf - * runtime - in which case borrowing doesn't make sense. - */ - rt_rq->rt_runtime =3D RUNTIME_INF; - rt_rq->rt_throttled =3D 0; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - raw_spin_unlock(&rt_b->rt_runtime_lock); - - /* Make rt_rq available for pick_next_task() */ - sched_rt_rq_enqueue(rt_rq); - } -} - -static void __enable_runtime(struct rq *rq) -{ - rt_rq_iter_t iter; - struct rt_rq *rt_rq; - - if (unlikely(!scheduler_running)) - return; - - /* - * Reset each runqueue's bandwidth settings - */ - for_each_rt_rq(rt_rq, iter, rq) { - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - - raw_spin_lock(&rt_b->rt_runtime_lock); - raw_spin_lock(&rt_rq->rt_runtime_lock); - rt_rq->rt_runtime =3D rt_b->rt_runtime; - rt_rq->rt_time =3D 0; - rt_rq->rt_throttled =3D 0; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - raw_spin_unlock(&rt_b->rt_runtime_lock); - } -} - -static void balance_runtime(struct rt_rq *rt_rq) -{ - if (!sched_feat(RT_RUNTIME_SHARE)) - return; - - if (rt_rq->rt_time > rt_rq->rt_runtime) { - raw_spin_unlock(&rt_rq->rt_runtime_lock); - do_balance_runtime(rt_rq); - raw_spin_lock(&rt_rq->rt_runtime_lock); - } -} - -static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun) -{ - int i, idle =3D 1, throttled =3D 0; - const struct cpumask *span; - - span =3D sched_rt_period_mask(); - - /* - * FIXME: isolated CPUs should really leave the root task group, - * whether they are isolcpus or were isolated via cpusets, lest - * the timer run on a CPU which does not service all runqueues, - * potentially leaving other CPUs indefinitely throttled. If - * isolation is really required, the user will turn the throttle - * off to kill the perturbations it causes anyway. Meanwhile, - * this maintains functionality for boot and/or troubleshooting. - */ - if (rt_b =3D=3D &root_task_group.rt_bandwidth) - span =3D cpu_online_mask; - - for_each_cpu(i, span) { - int enqueue =3D 0; - struct rt_rq *rt_rq =3D sched_rt_period_rt_rq(rt_b, i); - struct rq *rq =3D rq_of_rt_rq(rt_rq); - struct rq_flags rf; - int skip; - - /* - * When span =3D=3D cpu_online_mask, taking each rq->lock - * can be time-consuming. Try to avoid it when possible. - */ - raw_spin_lock(&rt_rq->rt_runtime_lock); - if (!sched_feat(RT_RUNTIME_SHARE) && rt_rq->rt_runtime !=3D RUNTIME_INF) - rt_rq->rt_runtime =3D rt_b->rt_runtime; - skip =3D !rt_rq->rt_time && !rt_rq->rt_nr_running; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - if (skip) - continue; - - rq_lock(rq, &rf); - update_rq_clock(rq); - - if (rt_rq->rt_time) { - u64 runtime; - - raw_spin_lock(&rt_rq->rt_runtime_lock); - if (rt_rq->rt_throttled) - balance_runtime(rt_rq); - runtime =3D rt_rq->rt_runtime; - rt_rq->rt_time -=3D min(rt_rq->rt_time, overrun*runtime); - if (rt_rq->rt_throttled && rt_rq->rt_time < runtime) { - rt_rq->rt_throttled =3D 0; - enqueue =3D 1; - - /* - * When we're idle and a woken (rt) task is - * throttled wakeup_preempt() will set - * skip_update and the time between the wakeup - * and this unthrottle will get accounted as - * 'runtime'. - */ - if (rt_rq->rt_nr_running && rq->curr =3D=3D rq->idle) - rq_clock_cancel_skipupdate(rq); - } - if (rt_rq->rt_time || rt_rq->rt_nr_running) - idle =3D 0; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - } else if (rt_rq->rt_nr_running) { - idle =3D 0; - if (!rt_rq_throttled(rt_rq)) - enqueue =3D 1; - } - if (rt_rq->rt_throttled) - throttled =3D 1; - - if (enqueue) - sched_rt_rq_enqueue(rt_rq); - rq_unlock(rq, &rf); - } - - if (!throttled && (!rt_bandwidth_enabled() || rt_b->rt_runtime =3D=3D RUN= TIME_INF)) - return 1; - - return idle; -} - -static int sched_rt_runtime_exceeded(struct rt_rq *rt_rq) -{ - u64 runtime =3D sched_rt_runtime(rt_rq); - - if (rt_rq->rt_throttled) - return rt_rq_throttled(rt_rq); - - if (runtime >=3D sched_rt_period(rt_rq)) - return 0; - - balance_runtime(rt_rq); - runtime =3D sched_rt_runtime(rt_rq); - if (runtime =3D=3D RUNTIME_INF) - return 0; - - if (rt_rq->rt_time > runtime) { - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - - /* - * Don't actually throttle groups that have no runtime assigned - * but accrue some time due to boosting. - */ - if (likely(rt_b->rt_runtime)) { - rt_rq->rt_throttled =3D 1; - printk_deferred_once("sched: RT throttling activated\n"); - } else { - /* - * In case we did anyway, make it go away, - * replenishment is a joke, since it will replenish us - * with exactly 0 ns. - */ - rt_rq->rt_time =3D 0; - } - - if (rt_rq_throttled(rt_rq)) { - sched_rt_rq_dequeue(rt_rq); - return 1; - } - } - - return 0; -} - -#else /* !CONFIG_RT_GROUP_SCHED: */ +#else /* !CONFIG_RT_GROUP_SCHED */ =20 typedef struct rt_rq *rt_rq_iter_t; =20 #define for_each_rt_rq(rt_rq, iter, rq) \ for ((void) iter, rt_rq =3D &rq->rt; rt_rq; rt_rq =3D NULL) =20 -#define for_each_sched_rt_entity(rt_se) \ - for (; rt_se; rt_se =3D NULL) - -static inline struct rt_rq *group_rt_rq(struct sched_rt_entity *rt_se) -{ - return NULL; -} - -static inline void sched_rt_rq_enqueue(struct rt_rq *rt_rq) -{ - struct rq *rq =3D rq_of_rt_rq(rt_rq); - - if (!rt_rq->rt_nr_running) - return; - - enqueue_top_rt_rq(rt_rq); - resched_curr(rq); -} - -static inline void sched_rt_rq_dequeue(struct rt_rq *rt_rq) -{ - dequeue_top_rt_rq(rt_rq, rt_rq->rt_nr_running); -} - -static inline int rt_rq_throttled(struct rt_rq *rt_rq) -{ - return false; -} - -static inline const struct cpumask *sched_rt_period_mask(void) -{ - return cpu_online_mask; -} - -static inline -struct rt_rq *sched_rt_period_rt_rq(struct rt_bandwidth *rt_b, int cpu) -{ - return &cpu_rq(cpu)->rt; -} - -static void __enable_runtime(struct rq *rq) { } -static void __disable_runtime(struct rq *rq) { } - -#endif /* !CONFIG_RT_GROUP_SCHED */ +#endif /* CONFIG_RT_GROUP_SCHED */ =20 static inline int rt_se_prio(struct sched_rt_entity *rt_se) { -#ifdef CONFIG_RT_GROUP_SCHED - struct rt_rq *rt_rq =3D group_rt_rq(rt_se); - - if (rt_rq) - return rt_rq->highest_prio.curr; -#endif - return rt_task_of(rt_se)->prio; } =20 @@ -935,67 +338,8 @@ static void update_curr_rt(struct rq *rq) if (unlikely(delta_exec <=3D 0)) return; =20 -#ifdef CONFIG_RT_GROUP_SCHED - struct sched_rt_entity *rt_se =3D &donor->rt; - if (!rt_bandwidth_enabled()) return; - - for_each_sched_rt_entity(rt_se) { - struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); - int exceeded; - - if (sched_rt_runtime(rt_rq) !=3D RUNTIME_INF) { - raw_spin_lock(&rt_rq->rt_runtime_lock); - rt_rq->rt_time +=3D delta_exec; - exceeded =3D sched_rt_runtime_exceeded(rt_rq); - if (exceeded) - resched_curr(rq); - raw_spin_unlock(&rt_rq->rt_runtime_lock); - if (exceeded) - do_start_rt_bandwidth(sched_rt_bandwidth(rt_rq)); - } - } -#endif /* CONFIG_RT_GROUP_SCHED */ -} - -static void -dequeue_top_rt_rq(struct rt_rq *rt_rq, unsigned int count) -{ - struct rq *rq =3D rq_of_rt_rq(rt_rq); - - BUG_ON(&rq->rt !=3D rt_rq); - - if (!rt_rq->rt_queued) - return; - - BUG_ON(!rq->nr_running); - - sub_nr_running(rq, count); - rt_rq->rt_queued =3D 0; - -} - -static void -enqueue_top_rt_rq(struct rt_rq *rt_rq) -{ - struct rq *rq =3D rq_of_rt_rq(rt_rq); - - BUG_ON(&rq->rt !=3D rt_rq); - - if (rt_rq->rt_queued) - return; - - if (rt_rq_throttled(rt_rq)) - return; - - if (rt_rq->rt_nr_running) { - add_nr_running(rq, rt_rq->rt_nr_running); - rt_rq->rt_queued =3D 1; - } - - /* Kick cpufreq (see the comment in kernel/sched/sched.h). */ - cpufreq_update_util(rq, 0); } =20 static void @@ -1066,58 +410,17 @@ dec_rt_prio(struct rt_rq *rt_rq, int prio) dec_rt_prio_smp(rt_rq, prio, prev_prio); } =20 -#ifdef CONFIG_RT_GROUP_SCHED - -static void -inc_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) -{ - if (rt_se_boosted(rt_se)) - rt_rq->rt_nr_boosted++; - - start_rt_bandwidth(&rt_rq->tg->rt_bandwidth); -} - -static void -dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) -{ - if (rt_se_boosted(rt_se)) - rt_rq->rt_nr_boosted--; - - WARN_ON(!rt_rq->rt_nr_running && rt_rq->rt_nr_boosted); -} - -#else /* !CONFIG_RT_GROUP_SCHED: */ - -static void -inc_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) -{ -} - -static inline -void dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) {} - -#endif /* !CONFIG_RT_GROUP_SCHED */ - static inline unsigned int rt_se_nr_running(struct sched_rt_entity *rt_se) { - struct rt_rq *group_rq =3D group_rt_rq(rt_se); - - if (group_rq) - return group_rq->rt_nr_running; - else - return 1; + return 1; } =20 static inline unsigned int rt_se_rr_nr_running(struct sched_rt_entity *rt_se) { - struct rt_rq *group_rq =3D group_rt_rq(rt_se); struct task_struct *tsk; =20 - if (group_rq) - return group_rq->rr_nr_running; - tsk =3D rt_task_of(rt_se); =20 return (tsk->policy =3D=3D SCHED_RR) ? 1 : 0; @@ -1126,26 +429,21 @@ unsigned int rt_se_rr_nr_running(struct sched_rt_ent= ity *rt_se) static inline void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) { - int prio =3D rt_se_prio(rt_se); - - WARN_ON(!rt_prio(prio)); + WARN_ON(!rt_prio(rt_se_prio(rt_se))); rt_rq->rt_nr_running +=3D rt_se_nr_running(rt_se); rt_rq->rr_nr_running +=3D rt_se_rr_nr_running(rt_se); =20 - inc_rt_prio(rt_rq, prio); - inc_rt_group(rt_se, rt_rq); + inc_rt_prio(rt_rq, rt_se_prio(rt_se)); } =20 static inline void dec_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) { WARN_ON(!rt_prio(rt_se_prio(rt_se))); - WARN_ON(!rt_rq->rt_nr_running); rt_rq->rt_nr_running -=3D rt_se_nr_running(rt_se); rt_rq->rr_nr_running -=3D rt_se_rr_nr_running(rt_se); =20 dec_rt_prio(rt_rq, rt_se_prio(rt_se)); - dec_rt_group(rt_se, rt_rq); } =20 /* @@ -1174,10 +472,6 @@ static void __delist_rt_entity(struct sched_rt_entity= *rt_se, struct rt_prio_arr static inline struct sched_statistics * __schedstats_from_rt_se(struct sched_rt_entity *rt_se) { - /* schedstats is not supported for rt group. */ - if (!rt_entity_is_task(rt_se)) - return NULL; - return &rt_task_of(rt_se)->stats; } =20 @@ -1190,9 +484,7 @@ update_stats_wait_start_rt(struct rt_rq *rt_rq, struct= sched_rt_entity *rt_se) if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); stats =3D __schedstats_from_rt_se(rt_se); if (!stats) return; @@ -1209,9 +501,7 @@ update_stats_enqueue_sleeper_rt(struct rt_rq *rt_rq, s= truct sched_rt_entity *rt_ if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); stats =3D __schedstats_from_rt_se(rt_se); if (!stats) return; @@ -1239,9 +529,7 @@ update_stats_wait_end_rt(struct rt_rq *rt_rq, struct s= ched_rt_entity *rt_se) if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); stats =3D __schedstats_from_rt_se(rt_se); if (!stats) return; @@ -1258,9 +546,7 @@ update_stats_dequeue_rt(struct rt_rq *rt_rq, struct sc= hed_rt_entity *rt_se, if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); if ((flags & DEQUEUE_SLEEP) && p) { unsigned int state; =20 @@ -1279,21 +565,8 @@ static void __enqueue_rt_entity(struct sched_rt_entit= y *rt_se, unsigned int flag { struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); struct rt_prio_array *array =3D &rt_rq->active; - struct rt_rq *group_rq =3D group_rt_rq(rt_se); struct list_head *queue =3D array->queue + rt_se_prio(rt_se); =20 - /* - * Don't enqueue the group if its throttled, or when empty. - * The latter is a consequence of the former when a child group - * get throttled and the current group doesn't have any other - * active members. - */ - if (group_rq && (rt_rq_throttled(group_rq) || !group_rq->rt_nr_running)) { - if (rt_se->on_list) - __delist_rt_entity(rt_se, array); - return; - } - if (move_entity(flags)) { WARN_ON_ONCE(rt_se->on_list); if (flags & ENQUEUE_HEAD) @@ -1323,57 +596,18 @@ static void __dequeue_rt_entity(struct sched_rt_enti= ty *rt_se, unsigned int flag dec_rt_tasks(rt_se, rt_rq); } =20 -/* - * Because the prio of an upper entry depends on the lower - * entries, we must remove entries top - down. - */ -static void dequeue_rt_stack(struct sched_rt_entity *rt_se, unsigned int f= lags) -{ - struct sched_rt_entity *back =3D NULL; - unsigned int rt_nr_running; - - for_each_sched_rt_entity(rt_se) { - rt_se->back =3D back; - back =3D rt_se; - } - - rt_nr_running =3D rt_rq_of_se(back)->rt_nr_running; - - for (rt_se =3D back; rt_se; rt_se =3D rt_se->back) { - if (on_rt_rq(rt_se)) - __dequeue_rt_entity(rt_se, flags); - } - - dequeue_top_rt_rq(rt_rq_of_se(back), rt_nr_running); -} - static void enqueue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags) { - struct rq *rq =3D rq_of_rt_se(rt_se); - update_stats_enqueue_rt(rt_rq_of_se(rt_se), rt_se, flags); =20 - dequeue_rt_stack(rt_se, flags); - for_each_sched_rt_entity(rt_se) - __enqueue_rt_entity(rt_se, flags); - enqueue_top_rt_rq(&rq->rt); + __enqueue_rt_entity(rt_se, flags); } =20 static void dequeue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags) { - struct rq *rq =3D rq_of_rt_se(rt_se); - update_stats_dequeue_rt(rt_rq_of_se(rt_se), rt_se, flags); =20 - dequeue_rt_stack(rt_se, flags); - - for_each_sched_rt_entity(rt_se) { - struct rt_rq *rt_rq =3D group_rt_rq(rt_se); - - if (rt_rq && rt_rq->rt_nr_running) - __enqueue_rt_entity(rt_se, flags); - } - enqueue_top_rt_rq(&rq->rt); + __dequeue_rt_entity(rt_se, flags); } =20 /* @@ -1436,10 +670,8 @@ static void requeue_task_rt(struct rq *rq, struct tas= k_struct *p, int head) struct sched_rt_entity *rt_se =3D &p->rt; struct rt_rq *rt_rq; =20 - for_each_sched_rt_entity(rt_se) { - rt_rq =3D rt_rq_of_se(rt_se); - requeue_rt_entity(rt_rq, rt_se, head); - } + rt_rq =3D rt_rq_of_se(rt_se); + requeue_rt_entity(rt_rq, rt_se, head); } =20 static void yield_task_rt(struct rq *rq) @@ -1636,17 +868,7 @@ static struct sched_rt_entity *pick_next_rt_entity(st= ruct rt_rq *rt_rq) =20 static struct task_struct *_pick_next_task_rt(struct rq *rq) { - struct sched_rt_entity *rt_se; - struct rt_rq *rt_rq =3D &rq->rt; - - do { - rt_se =3D pick_next_rt_entity(rt_rq); - if (unlikely(!rt_se)) - return NULL; - rt_rq =3D group_rt_rq(rt_se); - } while (rt_rq); - - return rt_task_of(rt_se); + return NULL; } =20 static struct task_struct *pick_task_rt(struct rq *rq) @@ -2315,8 +1537,6 @@ static void rq_online_rt(struct rq *rq) if (rq->rt.overloaded) rt_set_overload(rq); =20 - __enable_runtime(rq); - cpupri_set(&rq->rd->cpupri, rq->cpu, rq->rt.highest_prio.curr); } =20 @@ -2326,8 +1546,6 @@ static void rq_offline_rt(struct rq *rq) if (rq->rt.overloaded) rt_clear_overload(rq); =20 - __disable_runtime(rq); - cpupri_set(&rq->rd->cpupri, rq->cpu, CPUPRI_INVALID); } =20 @@ -2485,12 +1703,12 @@ static void task_tick_rt(struct rq *rq, struct task= _struct *p, int queued) * Requeue to the end of queue if we (and all of our ancestors) are not * the only element on the queue */ - for_each_sched_rt_entity(rt_se) { - if (rt_se->run_list.prev !=3D rt_se->run_list.next) { - requeue_task_rt(rq, p, 0); - resched_curr(rq); - return; - } + if (rt_se->run_list.prev !=3D rt_se->run_list.next) { + requeue_task_rt(rq, p, 0); + resched_curr(rq); + // set_tsk_need_resched(p); + + return; } } =20 @@ -2508,16 +1726,7 @@ static unsigned int get_rr_interval_rt(struct rq *rq= , struct task_struct *task) #ifdef CONFIG_SCHED_CORE static int task_is_throttled_rt(struct task_struct *p, int cpu) { - struct rt_rq *rt_rq; - -#ifdef CONFIG_RT_GROUP_SCHED // XXX maybe add task_rt_rq(), see also sched= _rt_period_rt_rq - rt_rq =3D task_group(p)->rt_rq[cpu]; - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); -#else - rt_rq =3D &cpu_rq(cpu)->rt; -#endif - - return rt_rq_throttled(rt_rq); + return 0; } #endif /* CONFIG_SCHED_CORE */ =20 @@ -2765,13 +1974,7 @@ long sched_group_rt_period(struct task_group *tg) #ifdef CONFIG_SYSCTL static int sched_rt_global_constraints(void) { - int ret =3D 0; - - mutex_lock(&rt_constraints_mutex); - ret =3D __rt_schedulable(NULL, 0, 0); - mutex_unlock(&rt_constraints_mutex); - - return ret; + return 0; } #endif /* CONFIG_SYSCTL */ =20 @@ -2806,10 +2009,6 @@ static int sched_rt_global_validate(void) return 0; } =20 -static void sched_rt_do_global(void) -{ -} - static int sched_rt_handler(const struct ctl_table *table, int write, void= *buffer, size_t *lenp, loff_t *ppos) { @@ -2837,7 +2036,6 @@ static int sched_rt_handler(const struct ctl_table *t= able, int write, void *buff if (ret) goto undo; =20 - sched_rt_do_global(); sched_dl_do_global(); } if (0) { diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 831ab576842..2d373a1ba67 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -787,7 +787,7 @@ struct scx_rq { =20 static inline int rt_bandwidth_enabled(void) { - return sysctl_sched_rt_runtime >=3D 0; + return 0; } =20 /* RT IPI pull logic requires IRQ_WORK */ @@ -827,7 +827,7 @@ struct rt_rq { =20 static inline bool rt_rq_is_runnable(struct rt_rq *rt_rq) { - return rt_rq->rt_queued && rt_rq->rt_nr_running; + return rt_rq->rt_nr_running; } =20 /* Deadline class' related fields in a runqueue */ @@ -2543,7 +2543,7 @@ static inline bool sched_dl_runnable(struct rq *rq) =20 static inline bool sched_rt_runnable(struct rq *rq) { - return rq->rt.rt_queued > 0; + return rq->rt.rt_nr_running > 0; } =20 static inline bool sched_fair_runnable(struct rq *rq) @@ -2654,9 +2654,6 @@ extern void resched_curr(struct rq *rq); extern void resched_curr_lazy(struct rq *rq); extern void resched_cpu(int cpu); =20 -extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 r= untime); -extern bool sched_rt_bandwidth_account(struct rt_rq *rt_rq); - extern void init_dl_entity(struct sched_dl_entity *dl_se); =20 #define BW_SHIFT 20 @@ -3043,8 +3040,6 @@ extern bool sched_smp_initialized; #ifdef CONFIG_RT_GROUP_SCHED static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) { - WARN_ON_ONCE(rt_se->my_q); - return container_of(rt_se, struct task_struct, rt); } =20 diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index 77ae87f36e8..93a9c03b28e 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -626,19 +626,6 @@ int __sched_setscheduler(struct task_struct *p, change: =20 if (user) { -#ifdef CONFIG_RT_GROUP_SCHED - /* - * Do not allow real-time tasks into groups that have no runtime - * assigned. - */ - if (rt_group_sched_enabled() && - rt_bandwidth_enabled() && rt_policy(policy) && - task_group(p)->rt_bandwidth.rt_runtime =3D=3D 0 && - !task_group_is_autogroup(task_group(p))) { - retval =3D -EPERM; - goto unlock; - } -#endif /* CONFIG_RT_GROUP_SCHED */ if (dl_bandwidth_enabled() && dl_policy(policy) && !(attr->sched_flags & SCHED_FLAG_SUGOV)) { cpumask_t *span =3D rq->rd->span; --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79579288525 for ; Mon, 29 Sep 2025 09:22:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137751; cv=none; b=PqfK1q7gmTWoGMbQex5Tr3CotfMM2PUIjGa52yWzd3D2fqCgM1xE9twRqH+cXAUfVgRHI99Y1QS8j850rH69eNAoQ4Y+AtNys917lxZK92SzQDZXkqMdJT8JZxX3IvKumuWepxSjGDsMCQ0W4vnIqeofxmlSOxw0xGlSrmeqOLQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137751; c=relaxed/simple; bh=92LgpsC9S+J3KClZDsE1ZiYjCcrnI7wmdf0SpM6xyHU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=g52WMXnrx4U7Mp/I8aqJSz3RPp8y/+gqdbknFBj3ZTy86hPLD4HLh/CrttLj405djnwDdxycOGQgcBXhbomVKHsiB+vm+ucx9SCRyeXifwR3Xl4oiDfZ/AfDsk4IJM4PIuGQ6IK7ej5TdICH6o2gzou8HRRG4aVd6kDkpQGElbA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SdYB71c6; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SdYB71c6" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-b3b27b50090so304299266b.0 for ; Mon, 29 Sep 2025 02:22:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137748; x=1759742548; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7k5FST/O+5lHGgu9ZsVOdWQJG6/wJtNjeDkBuwWUIAo=; b=SdYB71c6EfDxx4EcDCPekb7rlQQJGRv/KJszpPfW9aoVW8nPRXRvLc0ZOQ+hTP6YMp I2IOzo3gonuDuxRHNfTYsrGbma/ZksMmvUVJ0E2aibTnwVva0BfChyBO8qQbmJFz8GNn RPZoDTNF0W+0r2/vqawCG+NYw+19JzZ62nmZCvTFnXyLqxEpKhw/xsL7s7mDOpYS/Nce U9iVBT+f/f5E/UlBBs7RDvxZCOP1QPIeC4WfXDOxrTwrqLRseCaFuf8aVkcy09WLglQb ZiKLvRbREyYIo2AIq0kfEWkgRLkAaRuK3JfpqdaJCgoP77bFptdZ39N02HTU/x5lM1K5 KcGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137748; x=1759742548; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7k5FST/O+5lHGgu9ZsVOdWQJG6/wJtNjeDkBuwWUIAo=; b=OZrWFsJ/UtuQa3GmYKFhGnbMToD2Wmvug6c1OTHMlWMGVvOiX9iwHnoP+IENN5Dhua lZxfvTEDeEkHxOoChJY2QnPOQrkqWwrJ8YSvEJWPdD+ItJw0N5Gx+xMVTJTAOsvkM8UK y/nsnF4GL54SEOgTU3Ozme1xyocZlxost2jYWJW+G5p2S5G/QY9m2YqPnAe95gIrql0R XzKfFSOsx5KoYRzFTHoV/0Yz4QMkQ5Xhz+U17B3IdrDgSl6iWQn2QK9SMaCIh84V42+V +QNixducbbD0SDV6qlAezG49ip92ABv0FQy3xaQbjR5dQ+BQibQYLMmUirO+CUQogotJ jICw== X-Gm-Message-State: AOJu0YxyEOQPuWa6SUD5BHFpgh0bY0hDtOfQUuhyA+fJSN94fNDgUW3i O1kVLB4IlGnwxsE3KhZLFwWJ8UIMrX0RqGStrJ3MarzYDoHa6pLclwVf X-Gm-Gg: ASbGncu0+AHlf6s7RZemyBYoYa0/rMJ3KeSt5w1XH3vJmfk7Comudir1D63cj3lCuvB FtVP+orM7pRC34IkoLUgPuDt+vUpFgL0Ywp8qL9o3cO4IDbcwX4RJISjDhnlDE5pazaZlg7qzxF H8eJTiUFKXMCoPphwlZudCBiSr/LdyXx+A+w21JIw4RhtBiuDXqI+CdMsy2MeZi9ZtVswgEVPna rXr4t7PATCUCFa+FKH4MZDaw6XJOK1WEkqqsxnN06SWUnlx40PD9T3W1P1L68j3qgTOIm/Mn7Kc r51Syvn7mKWSi4w/vFCBXBZGAR54g+6WjdXxZ3fUCj/lLzy/HZeZP7xSbxpTpbHOl3zl/fbUu9x Uzpn1zSuzrHZ4CtF9rrKFlbUwUDRx+g== X-Google-Smtp-Source: AGHT+IEPElphIZVl7S3799FuvysqXTO6qLcHaSp2KgeBoPSdMH41LOb/sP7J/XpospKPVklkqFRv4Q== X-Received: by 2002:a17:906:2713:b0:b3c:a161:683b with SMTP id a640c23a62f3a-b3ca1617c55mr460797566b.60.1759137747751; Mon, 29 Sep 2025 02:22:27 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:27 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 06/24] sched/rt: Introduce HCBS specific structs in task_group Date: Mon, 29 Sep 2025 11:22:03 +0200 Message-ID: <20250929092221.10947-7-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Each task_group manages a number of new objects: - a sched_dl_entity/dl_server for each CPU - a dl_bandwidth object to keep track of its allocated dl_bandwidth Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/sched.h | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 2d373a1ba67..59a154505d8 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -319,6 +319,13 @@ struct rt_bandwidth { unsigned int rt_period_active; }; =20 +struct dl_bandwidth { + raw_spinlock_t dl_runtime_lock; + u64 dl_runtime; + u64 dl_period; +}; + + static inline int dl_bandwidth_enabled(void) { return sysctl_sched_rt_runtime >=3D 0; @@ -467,10 +474,17 @@ struct task_group { #endif /* CONFIG_FAIR_GROUP_SCHED */ =20 #ifdef CONFIG_RT_GROUP_SCHED + /* + * Each task group manages a different scheduling entity per CPU, i.e. a + * different deadline server, and a runqueue per CPU. All the dl-servers + * share the same dl_bandwidth object. + */ struct sched_rt_entity **rt_se; + struct sched_dl_entity **dl_se; struct rt_rq **rt_rq; =20 struct rt_bandwidth rt_bandwidth; + struct dl_bandwidth dl_bandwidth; #endif =20 struct scx_task_group scx; @@ -817,12 +831,12 @@ struct rt_rq { raw_spinlock_t rt_runtime_lock; =20 unsigned int rt_nr_boosted; - - struct rq *rq; /* this is always top-level rq, cache? */ #endif #ifdef CONFIG_CGROUP_SCHED struct task_group *tg; /* this tg has "this" rt_rq on given CPU for runna= ble entities */ #endif + + struct rq *rq; /* cgroup's runqueue if the rt_rq entity belongs to a cgr= oup, otherwise top-level rq */ }; =20 static inline bool rt_rq_is_runnable(struct rt_rq *rt_rq) --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB3E7291C07 for ; Mon, 29 Sep 2025 09:22:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137752; cv=none; b=CQCjKal020WazboEQH/biXm2kER/HeDm1JeXZcKvMCF6Fd3/jJS1M0s7MTsmPKzJyFUaBWwMgbLCOBS+hm5r40AbvpI5le4R5dUbXY41qqbBrLwsbMwnPZS2j8jIz3RxRvvMxmNtkce2qypZHjpUh7F/hNz8fiu2CfbDPPGKAyQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137752; c=relaxed/simple; bh=2vobuS75wZ3iblpIpVxl8dpvoj9aaH5YDpIN5T7o2tk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kzyysaTTvO+9le1GMaCe66yBBNZGnUa41WXIEOuVRyHlmX4lg5Pad6FkL0vCx6SczpXCe8Aq+MDZYRVP+bk+7h0PkLfR0Ka3zI+WBAR6+Csab0G5QBPWSOpv/PcWKlwWg+1jQqoFQkkPrHfhAMsodsFy/SkRjWNr80Ikfqw2lBc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=C2ksFtZN; arc=none smtp.client-ip=209.85.208.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="C2ksFtZN" Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-62fa0653cd1so6393675a12.0 for ; Mon, 29 Sep 2025 02:22:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137749; x=1759742549; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uVIdjl25oNdyF+4lq9AqQ5rIdrsLQhC1m1Oil1Kj7Y0=; b=C2ksFtZNknBp5VwFgR0/g5hJbpiWmA/vWzrcYUVRSGflJnQwk3qn6RDgo3Spb7euJq 9wFYPf4zIIESc66xuNatOsci2othcoyJZCVqSCOGmnRJ8E9gDs24T/DmWN42MCc5jtTU YOUSKnj9oF0te5zVkpdJ1+HX9BG2BCKyKU0b15KA9qzEqVPkuDAEE2YJKpuEUdcBIRd7 +LLOTZ7G/0b+8efYFGaRMnFnG4WCrW4J/hM3a5/18MdJR8F4pBi7r300Z/MNf9/cOvSy fVnIq7xx55WjL2Qmx9ZfhvZMZ2CITUmHc4tKD4vOYBUDBzPWlud6O6sSJ5LIBY58TaQy RiXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137749; x=1759742549; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uVIdjl25oNdyF+4lq9AqQ5rIdrsLQhC1m1Oil1Kj7Y0=; b=dOAAmTsXilLZE1Pe3uEm0MWKQIToW9WphkfK8LwO0Z9Rp09gTRrfhVZE2LN907p9T1 jCUBmSAOb0+cjUHYUtlkkjWSOFa27jSSvmo0OSI4BYASQtss4zahHcGDtMw18R4HZnnQ P6JFG3tx/qt40mHw72bDILr7IzVJRwH66A5X3YTD283mYICcSx3hmxS4JJAK1qIII8JB z3xAnuhKm3147CKTrmb9fTz05UGcpdIrQNpiSrhZSRktmgHXBCw2WA1i+m9Esz/gcGrO 0r7SjUM1RAN7g6HIegZid47cVM/oUjVrmn4+NRIP3UxXygxFveqS/0TKTz1FYO3N/7WH yqjw== X-Gm-Message-State: AOJu0YzhkDNZhNxVwbcyG6YMjiMHqMkSKf22v/JK5+iPReoVQwmCTW2/ nFVsGVjI+mO+mhyZSeE+mSIe3RDUfMumaQ5el2HDKf+rNgM7IDvPxndP X-Gm-Gg: ASbGncv7w5nYa4rGmfm4PKrm920IzjnWgQPGRZmbTr2rZrQxiGPj3yD0CSAXnhdT+9B zPzSCDlV0FS6Il9SSGC+LvXAdwynD1v6TUMhofBOIzfwTSJCDfcx4QrG9uK/sjkxEDLlg0mDhlK 81yKB53DWWCQrpc8qVFv6jDCIkmUm9q7h/5341+5MBvzViyNZVIXnauSNsYI6AaubhumwIaWLTK yxlzBR5c8lYgysYalPE2MfRVA5cNCKycuPJPV1fTG6Xp/LiYeT0FSXNjmWqTdmMMzwcgIN5O/Ec 7SVpqdlW9hZvrnzWQtRQF6Pm3S55/eS8aHjvpLei4B14Xy0YrjYBlbe11Wqt8cojjmYNQvbV85L /sWqfsQ1f+IRgSUa7gMk= X-Google-Smtp-Source: AGHT+IEI2xpAGbvRpY4BIV8ijaBZbVioGgATkA9IGkZ6XLAvSiH2w7owNTO9dsg+KYQVkVCJDO0NFw== X-Received: by 2002:a17:907:6e8c:b0:b3f:c562:faf7 with SMTP id a640c23a62f3a-b3fc562fe27mr194433466b.55.1759137748476; Mon, 29 Sep 2025 02:22:28 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:28 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 07/24] sched/core: Initialize root_task_group Date: Mon, 29 Sep 2025 11:22:04 +0200 Message-ID: <20250929092221.10947-8-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add the initialization function for task_group's dl_servers. Initialize the default bandwidth for rt-cgroups and the root control group. Add utility functions to check (and get) if a rt_rq entity is connected to a real-time cgroup. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/autogroup.c | 4 ++-- kernel/sched/core.c | 9 +++++++-- kernel/sched/deadline.c | 8 ++++++++ kernel/sched/rt.c | 18 ++++++++---------- kernel/sched/sched.h | 32 +++++++++++++++++++++++++++++--- 5 files changed, 54 insertions(+), 17 deletions(-) diff --git a/kernel/sched/autogroup.c b/kernel/sched/autogroup.c index cdea931aae3..017eadc0a0a 100644 --- a/kernel/sched/autogroup.c +++ b/kernel/sched/autogroup.c @@ -52,7 +52,7 @@ static inline void autogroup_destroy(struct kref *kref) =20 #ifdef CONFIG_RT_GROUP_SCHED /* We've redirected RT tasks to the root task group... */ - ag->tg->rt_se =3D NULL; + ag->tg->dl_se =3D NULL; ag->tg->rt_rq =3D NULL; #endif sched_release_group(ag->tg); @@ -109,7 +109,7 @@ static inline struct autogroup *autogroup_create(void) * the policy change to proceed. */ free_rt_sched_group(tg); - tg->rt_se =3D root_task_group.rt_se; + tg->dl_se =3D root_task_group.dl_se; tg->rt_rq =3D root_task_group.rt_rq; #endif /* CONFIG_RT_GROUP_SCHED */ tg->autogroup =3D ag; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 5791aa1f8c8..e5b4facee24 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8710,7 +8710,7 @@ void __init sched_init(void) scx_tg_init(&root_task_group); #endif /* CONFIG_EXT_GROUP_SCHED */ #ifdef CONFIG_RT_GROUP_SCHED - root_task_group.rt_se =3D (struct sched_rt_entity **)ptr; + root_task_group.dl_se =3D (struct sched_dl_entity **)ptr; ptr +=3D nr_cpu_ids * sizeof(void **); =20 root_task_group.rt_rq =3D (struct rt_rq **)ptr; @@ -8721,6 +8721,11 @@ void __init sched_init(void) =20 init_defrootdomain(); =20 +#ifdef CONFIG_RT_GROUP_SCHED + init_dl_bandwidth(&root_task_group.dl_bandwidth, + global_rt_period(), global_rt_runtime()); +#endif /* CONFIG_RT_GROUP_SCHED */ + #ifdef CONFIG_CGROUP_SCHED task_group_cache =3D KMEM_CACHE(task_group, 0); =20 @@ -8772,7 +8777,7 @@ void __init sched_init(void) * starts working after scheduler_running, which is not the case * yet. */ - init_tg_rt_entry(&root_task_group, &rq->rt, NULL, i, NULL); + init_tg_rt_entry(&root_task_group, rq, NULL, i, NULL); #endif rq->sd =3D NULL; rq->rd =3D NULL; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 277fbaff8b5..5c8fe80b585 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -513,6 +513,14 @@ static inline int is_leftmost(struct sched_dl_entity *= dl_se, struct dl_rq *dl_rq =20 static void init_dl_rq_bw_ratio(struct dl_rq *dl_rq); =20 +void init_dl_bandwidth(struct dl_bandwidth *dl_b, u64 period, u64 runtime) +{ + raw_spin_lock_init(&dl_b->dl_runtime_lock); + dl_b->dl_period =3D period; + dl_b->dl_runtime =3D runtime; +} + + void init_dl_bw(struct dl_bw *dl_b) { raw_spin_lock_init(&dl_b->lock); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index c625ea45ca7..84dbb4853b6 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -97,18 +97,16 @@ void free_rt_sched_group(struct task_group *tg) return; } =20 -void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq, - struct sched_rt_entity *rt_se, int cpu, - struct sched_rt_entity *parent) +void init_tg_rt_entry(struct task_group *tg, struct rq *served_rq, + struct sched_dl_entity *dl_se, int cpu, + struct sched_dl_entity *parent) { - struct rq *rq =3D cpu_rq(cpu); + served_rq->rt.highest_prio.curr =3D MAX_RT_PRIO-1; + served_rq->rt.rq =3D cpu_rq(cpu); + served_rq->rt.tg =3D tg; =20 - rt_rq->highest_prio.curr =3D MAX_RT_PRIO-1; - rt_rq->rq =3D rq; - rt_rq->tg =3D tg; - - tg->rt_rq[cpu] =3D rt_rq; - tg->rt_se[cpu] =3D rt_se; + tg->rt_rq[cpu] =3D &served_rq->rt; + tg->dl_se[cpu] =3D dl_se; } =20 int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 59a154505d8..d433abe1c9c 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -576,9 +576,9 @@ extern void start_cfs_bandwidth(struct cfs_bandwidth *c= fs_b); extern void unthrottle_cfs_rq(struct cfs_rq *cfs_rq); extern bool cfs_task_bw_constrained(struct task_struct *p); =20 -extern void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq, - struct sched_rt_entity *rt_se, int cpu, - struct sched_rt_entity *parent); +extern void init_tg_rt_entry(struct task_group *tg, struct rq *s_rq, + struct sched_dl_entity *rt_se, int cpu, + struct sched_dl_entity *parent); extern int sched_group_set_rt_runtime(struct task_group *tg, long rt_runti= me_us); extern int sched_group_set_rt_period(struct task_group *tg, u64 rt_period_= us); extern long sched_group_rt_runtime(struct task_group *tg); @@ -2668,6 +2668,7 @@ extern void resched_curr(struct rq *rq); extern void resched_curr_lazy(struct rq *rq); extern void resched_cpu(int cpu); =20 +void init_dl_bandwidth(struct dl_bandwidth *dl_b, u64 period, u64 runtime); extern void init_dl_entity(struct sched_dl_entity *dl_se); =20 #define BW_SHIFT 20 @@ -3077,6 +3078,21 @@ static inline struct rq *rq_of_rt_se(struct sched_rt= _entity *rt_se) WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); return rt_rq->rq; } + +static inline int is_dl_group(struct rt_rq *rt_rq) +{ + return rt_rq->tg !=3D &root_task_group; +} + +/* + * Return the scheduling entity of this group of tasks. + */ +static inline struct sched_dl_entity *dl_group_of(struct rt_rq *rt_rq) +{ + BUG_ON(!is_dl_group(rt_rq)); + + return rt_rq->tg->dl_se[cpu_of(rt_rq->rq)]; +} #else static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) { @@ -3101,6 +3117,16 @@ static inline struct rt_rq *rt_rq_of_se(struct sched= _rt_entity *rt_se) =20 return &rq->rt; } + +static inline int is_dl_group(struct rt_rq *rt_rq) +{ + return 0; +} + +static inline struct sched_dl_entity *dl_group_of(struct rt_rq *rt_rq) +{ + return NULL; +} #endif =20 DEFINE_LOCK_GUARD_2(double_rq_lock, struct rq, --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0116A29D289 for ; Mon, 29 Sep 2025 09:22:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137753; cv=none; b=a06JGexzRc6FLnPXzRh+Vsj/gVG3Uqz2mWuwCnElaBp4yR3MWi3SaioEcpg9XKmgtlrCMxRi38DZy95vAd3lxQPbteywB3QCoXiUixbuXP3TLR+4cpQNcwlf+2bFrdBUuO9+9/EM1KLJdcKpsMg5fUb0jDCY+NLCQW+ROqeyeso= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137753; c=relaxed/simple; bh=PWg23h/Ek2S/NeMbR69Z43bG27wBpoR47R8x2RBYyUQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lqHnsTgM7hxydGFI4xRZqUFGxrReMxDYJjUkg/s/qtZXchhCPNUrdR9QZAhE2TK88U4YAnWw784chi1+CKfdTw7ZPb9r/WWv2pgIvC7Xg7heLCHr83qian6sQ6ixLBIj+CJPg5C8rtdp3L/CxMzAgduqjjr0EcpIkNd7ONq7UHw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=DqdNjNCT; arc=none smtp.client-ip=209.85.218.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DqdNjNCT" Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-b3e234fcd4bso145344466b.3 for ; Mon, 29 Sep 2025 02:22:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137749; x=1759742549; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VzinNLU2Xw+DwWnbw6AoHDjNZrWmaSiy854G2AxZOYQ=; b=DqdNjNCTktLdIjyB1jRnvlSS1AIMMPmIn5aQ2fMnkfUnMrYEvX4Z2nWShvWkKkP8m1 +HQLDfYnjE1/rLDh/7FaHms5rBL5wGW9RTnp0U/gVnYouhHVqv36BlSjmwwkoGcq2d8x 4N+ZraRRyn/2Y5hvHM1MQe8BIBIK6sOh3KFb7sZuuTHrOFOAl9FDzbleQv/FACPpCxG8 MZeKS8+dkY4WdwK78O1qe34bt+6AHberXjgmxPgDx3iZkY84pW9EYQZpuw3okAMPwdbZ pSlkm/S8z40r0DAPbLQFBQLMxbfq92VgVqOEn2cOrfTzGqp8+Mnqe/7Db0Vw0/NCG51l mt6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137749; x=1759742549; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VzinNLU2Xw+DwWnbw6AoHDjNZrWmaSiy854G2AxZOYQ=; b=rFbd/7ah+oJLCTkdXZkXN8/vb5CyZNL/4K9Df2I8Pg0upGHPPWOBqitlpZF6mVUXEO HNtkRHOdP0x/Uk9DgZ+xtWfZdWja6oCl1C11ZeFWE1mmRYf5XDJwxqlVjhyMKgcQwnTU xIoCBYoDylnHhjnbVUHvi+Xe9H1QNKVu9XonrI8nqZMCAeEjGt5F4HCuGfkMacgj6gtH mkqYSieXmKo9jKVA5U1k0LQNkNXSZJChk1JAu8StX7nbjYSBRpBP2U6kwRQCz4V+hCIT vmaSgdPfMUV5S+pO8peschIbLdZqDqNcdxnvoQoY9EjUo/MDMHXImaa+6pilcDZS0n0f rghg== X-Gm-Message-State: AOJu0YyTfcLJF30Y3yi3yuWR8IUKsmLD6zqd+V6Bz5eYs2H1i6N8fkNG 5zwlOSf8aI/DNeBrAlAlOKGkBZpSA8ncekFSD0mTVfT0T4kbJ8miHAfk X-Gm-Gg: ASbGncs4TNg/TKMrklQV4+f3m3soME1YehGCDQ0H//4uGiAXyuMBSRux959qa+ZJnY3 wBDvHaI57xZnI+rrLPEEp6iBbpabbj5fav0+Mw/DU6HMkQgmhEsqt1uJh+BEeLx8+dOUKiQjesB 0WdD/e+HPRWhbkhU+fOr3FDc2bKORpYfO+wkWsLiqe+j2ZPrNhHGOdqzuklIEqOG+64m0MIs3vM BXjA2tVzSxpbCs5CSVkGAoXOZaxj6xcdoOjrJq/gegLLxmCvFwUsTy43JIYbbesJM0TNBwkUCkv z2N2gj+qm5/Dw6RzZpEhlNHKuXyAFtK44ETr+pdDpD8PeJYz30+5pMcrabXtwBWH3W7Lg8vJPtx z6BM/jJuYOy+v6f+HhBE= X-Google-Smtp-Source: AGHT+IHrDJdhCrtJ10dnGENNvCxknjjLx5jfwMqeniBulAcmXqVH9OCnuD6UTbGtexmXPkGEG/tv6w== X-Received: by 2002:a17:907:b1b:b0:b3c:e14c:e24 with SMTP id a640c23a62f3a-b3ce15b5f15mr450740866b.13.1759137749207; Mon, 29 Sep 2025 02:22:29 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:28 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 08/24] sched/deadline: Add dl_init_tg Date: Mon, 29 Sep 2025 11:22:05 +0200 Message-ID: <20250929092221.10947-9-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add dl_init_tg to initialize and/or update a rt-cgroup dl_server and to also account the allocated bandwidth. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/deadline.c | 33 +++++++++++++++++++++++++++++++++ kernel/sched/sched.h | 1 + 2 files changed, 34 insertions(+) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 5c8fe80b585..34a1494d782 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -340,6 +340,39 @@ void cancel_inactive_timer(struct sched_dl_entity *dl_= se) cancel_dl_timer(dl_se, &dl_se->inactive_timer); } =20 +#ifdef CONFIG_RT_GROUP_SCHED +void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 rt_peri= od) +{ + struct rq *rq =3D container_of(dl_se->dl_rq, struct rq, dl); + int is_active; + u64 new_bw; + + raw_spin_rq_lock_irq(rq); + is_active =3D dl_se->my_q->rt.rt_nr_running > 0; + + update_rq_clock(rq); + dl_server_stop(dl_se); + + new_bw =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); + dl_rq_change_utilization(rq, dl_se, new_bw); + + dl_se->dl_runtime =3D rt_runtime; + dl_se->dl_deadline =3D rt_period; + dl_se->dl_period =3D rt_period; + + dl_se->runtime =3D 0; + dl_se->deadline =3D 0; + + dl_se->dl_bw =3D new_bw; + dl_se->dl_density =3D new_bw; + + if (is_active) + dl_server_start(dl_se); + + raw_spin_rq_unlock_irq(rq); +} +#endif + static void dl_change_utilization(struct task_struct *p, u64 new_bw) { WARN_ON_ONCE(p->dl.flags & SCHED_FLAG_SUGOV); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index d433abe1c9c..9853f321363 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -394,6 +394,7 @@ extern void dl_server_init(struct sched_dl_entity *dl_s= e, struct dl_rq *dl_rq, dl_server_has_tasks_f has_tasks, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); +extern void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 = rt_period); =20 extern void dl_server_update_idle_time(struct rq *rq, struct task_struct *p); --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD7FF2C032E for ; Mon, 29 Sep 2025 09:22:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137753; cv=none; b=N+PabZDLsBsUMq3fbL9IFd63tsoPo/3XoN8opIvDTTGXJrrSsCfPNbEQ92sJd4bNg3a1ekutHg70l5WrcXlf4vFBEQj5SCZGKHXN50het/liYKlHfDj2ZHKd50ybIWd1uXurUBMR+lkVe7AHqpwS3x9Diqqn4IKyDUOEFtzoOlI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137753; c=relaxed/simple; bh=SXheP6/xZI9zCv3GztLGmkKg9TdEclhTg9jkgRMvDHI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ECpEb4PBLcl5d1YRL56W4lu3vAdIFhucKYdp6iaWQezA1cdl1gLfZOEeKwvgkvbJuuLgzaxkAnxjzv2CX5FgnAh8O9kHgf0DIaZ/e/l5DbB/xt7owunI5HM83kyrzx4rDfOJ88fidHxHWDJ/glXjbJ+F93doxG6tqi8KUOKzdC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AZTpFgTc; arc=none smtp.client-ip=209.85.208.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AZTpFgTc" Received: by mail-ed1-f50.google.com with SMTP id 4fb4d7f45d1cf-62ec5f750f7so6357045a12.3 for ; Mon, 29 Sep 2025 02:22:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137750; x=1759742550; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KGK2m7a0HEycJlq9ku+st1/ytMVTFoHVuVKupFmTy4A=; b=AZTpFgTcQ6EPqDL9E2wEF+eY6uQ5j18iacynR/0wxqouEi8waPs0S2uAptIKnsCacc oS5Q9ODJPEv9kPQ96acQqMBRKgN+FEJBDkKPAAHIncTVG4rtqQxFBDOlPX2/LGdG/7QP iqJi9Z5u0VjYykt66w81xwlg7+HAj7iAkif1i4wJ5fRiNN5h5i8juR8BxbSMDD+Mzot6 l+wyyt2OLTlCTUZpoE8wgHLn2w58qDpFWXbCwkQPOGp93pXTEtwxOcs/E0n0CQ5Xhedv A83bMU7tgwJNuSMC84Cp/1A5o44lmbsn8bv2mkbBnSUWWlRRfABUcFwWtE/9YJmZRN4t TSFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137750; x=1759742550; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KGK2m7a0HEycJlq9ku+st1/ytMVTFoHVuVKupFmTy4A=; b=QRPd2UDjdjkg9Hrip/2NcohppJXKXHyopCMr5ykPuziK9PENGujf8jJ09ZJYVbdK2U lGuzX2XNC/7dXVfYionRH3dfQIcJUjtBU2Lc5cly4a9ub1u1gBFldWAm91LTW5+qGWJ7 M0uBpuYnuTRvwNh58MF0kXjDzhpYiesXPF74XCN5zMBzse5nBik8rHBRNVRaup2jxFp6 T0f4tZmZEXZ46+4LGlsqRTypS3YMv3Ijvi1H2NwE3dXosLgmg2xztckgbbGQA+hx2ATp XV2aU13VfPYeVc/oM3sSFMh0eirpzAEKuv7qsAD0OPicLQrAFoOYxmcRHiVkWKJBDhKV oOmA== X-Gm-Message-State: AOJu0YyJ3ItcRFmb1DUZIP4hYIIvHTo5gXJ8DraOzQ8Q49y9Za09E4Ib JrgSL/pwnS38iGXRmjUfKCQQLsq7RLSnzEf9aFpesm91gvb72X1098tC X-Gm-Gg: ASbGnct8i63C6sxSP7xDBXNiTHi1f9lGS4Lslp/ey5pX6zhjeA152/L1lS4QnE8/CwJ kUJUoR1XOG6AP6QRm0jvqP2bHWEh8IhMd8bLQTI2cszE88LWlzMBfHJZu5meTR4QbLXBlNootPl i3ic0hbZ5G47y892vVnUGwgrTqSRruOy5p6ixaSTYsY63hEq9vYXfq6DeNeThEyE+nX3GhP5MHO zdyDuqGBF0uBA36pEihW4k9d9pf+VnhxIKGxiB5VI5/ky6ADZl2IS50FFbcs7hqH796iUE/uzgd Tin7WOzD6ymWVBG3atuowmF6hNNPZJXjXqe/+WHzkgag5tWtvV0Vza7if1dvf6Y8Tv/AVsE1+Sr 4hXM5X8LYqxXhpEw14uiSkvk7fKgDWA== X-Google-Smtp-Source: AGHT+IEHhc2vnZ95ykOQEE9lhp4XhTYXnTApSwqPZRq7uMqEWUjgr0DvOUn5SQERoFKV0hhfVI9atg== X-Received: by 2002:a17:906:478f:b0:b2c:bf1e:2788 with SMTP id a640c23a62f3a-b34bbbd9e1dmr1619575966b.39.1759137749950; Mon, 29 Sep 2025 02:22:29 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:29 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 09/24] sched/rt: Add {alloc/free}_rt_sched_group Date: Mon, 29 Sep 2025 11:22:06 +0200 Message-ID: <20250929092221.10947-10-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add allocation and deallocation code for rt-cgroups. Declare dl_server specific functions (no implementation yet), needed by the allocation code. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/rt.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 84dbb4853b6..3094f59d0c8 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -93,8 +93,43 @@ void unregister_rt_sched_group(struct task_group *tg) =20 void free_rt_sched_group(struct task_group *tg) { + int i; + unsigned long flags; + struct rq *served_rq; + if (!rt_group_sched_enabled()) return; + + if (!tg->dl_se || !tg->rt_rq) + return; + + for_each_possible_cpu(i) { + if (!tg->dl_se[i] || !tg->rt_rq[i]) + continue; + + /* + * Shutdown the dl_server and free it + * + * Since the dl timer is going to be cancelled, + * we risk to never decrease the running bw... + * Fix this issue by changing the group runtime + * to 0 immediately before freeing it. + */ + dl_init_tg(tg->dl_se[i], 0, tg->dl_se[i]->dl_period); + + raw_spin_rq_lock_irqsave(cpu_rq(i), flags); + BUG_ON(tg->rt_rq[i]->rt_nr_running); + hrtimer_cancel(&tg->dl_se[i]->dl_timer); + raw_spin_rq_unlock_irqrestore(cpu_rq(i), flags); + kfree(tg->dl_se[i]); + + /* Free the local per-cpu runqueue */ + served_rq =3D container_of(tg->rt_rq[i], struct rq, rt); + kfree(served_rq); + } + + kfree(tg->rt_rq); + kfree(tg->dl_se); } =20 void init_tg_rt_entry(struct task_group *tg, struct rq *served_rq, @@ -109,11 +144,65 @@ void init_tg_rt_entry(struct task_group *tg, struct r= q *served_rq, tg->dl_se[cpu] =3D dl_se; } =20 +static bool rt_server_has_tasks(struct sched_dl_entity *dl_se) +{ + return false; +} + +static struct task_struct *rt_server_pick(struct sched_dl_entity *dl_se) +{ + return NULL; +} + int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) { + struct rq *s_rq; + struct sched_dl_entity *dl_se; + int i; + if (!rt_group_sched_enabled()) return 1; =20 + tg->rt_rq =3D kcalloc(nr_cpu_ids, sizeof(struct rt_rq *), GFP_KERNEL); + if (!tg->rt_rq) + return 0; + + tg->dl_se =3D kcalloc(nr_cpu_ids, sizeof(dl_se), GFP_KERNEL); + if (!tg->dl_se) { + kfree(tg->rt_rq); + tg->rt_rq =3D NULL; + return 0; + } + + init_dl_bandwidth(&tg->dl_bandwidth, 0, 0); + + for_each_possible_cpu(i) { + s_rq =3D kzalloc_node(sizeof(struct rq), + GFP_KERNEL, cpu_to_node(i)); + if (!s_rq) + return 0; + + dl_se =3D kzalloc_node(sizeof(struct sched_dl_entity), + GFP_KERNEL, cpu_to_node(i)); + if (!dl_se) { + kfree(s_rq); + return 0; + } + + init_rt_rq(&s_rq->rt); + init_dl_entity(dl_se); + dl_se->dl_runtime =3D tg->dl_bandwidth.dl_runtime; + dl_se->dl_period =3D tg->dl_bandwidth.dl_period; + dl_se->dl_deadline =3D dl_se->dl_period; + dl_se->dl_bw =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); + dl_se->dl_density =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); + dl_se->dl_server =3D 1; + + dl_server_init(dl_se, &cpu_rq(i)->dl, s_rq, rt_server_has_tasks, rt_serv= er_pick); + + init_tg_rt_entry(tg, s_rq, dl_se, i, parent->dl_se[i]); + } + return 1; } =20 --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CD1B2C11C5 for ; Mon, 29 Sep 2025 09:22:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137754; cv=none; b=R5FunJsnxVaoBYt5GwBDNIIrS7N7/deglizFTjkEw0KA4iuyIvr9V1FHrWHa64gUz7xlwvFgl//ya2Gw3PPMI9Bg/FgttaPDWjDMhEtghnh3YvScQN/WcVb9E8sn0HpKZZW7D1jMmV8rsN1qHLMKfLZabkLc6BUZdR9nARWDQ4s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137754; c=relaxed/simple; bh=h/pz6zvc6qYAU8k60RuBoW4qESIFMe3VdTt5+BJwZ1M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OCYLv4fCRucUCvPebGRvvcBd3iWt79dsRptvqSgqY63a3TOE+b+iMi9Rh0swmPT2aRx/+7q0wqjEqnPPO817Estuo24pSSeNmHqh9kp8SHKHJMyWhGSH1oJyHu34qN1/Es0hwzOUc3qsxpAihW+PJXxY24qE/4mj/J+sFLkC1+U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gJp8Xscn; arc=none smtp.client-ip=209.85.208.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gJp8Xscn" Received: by mail-ed1-f47.google.com with SMTP id 4fb4d7f45d1cf-6364eb32535so681488a12.1 for ; Mon, 29 Sep 2025 02:22:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137751; x=1759742551; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ewFloRXAVA2+FWERqq7N2JT61DFtN9Zpod7cZzsTfrk=; b=gJp8XscnUNlx8uD+3wKFvzFoyLjZT9IImK/pqnGSRsxiaFYdl3S4MjXrBqL37/+uK3 DmvooXhp0pPBoPW4V2cTef0HuMp66sR6qHzL96JgA0sTSqow+6LlvlgxO6kPLl6iYSvY mj4XXnbECx2TMo3G3wEtriGgnk3D1S65ioIYm0Z3fgix62hJfqIcV6Z86MLtPP27UaZf kRyAXUP+my143CA8vv6jy+eZrS/G4U6yoVPS00SuTANghw2uHHldiq11eGvFAjfUxepL 3wh4HoW8Bms1u7boBbj4/+o1t7CZoQPvI+HbWlINE2cHmrMXcQKiApYYZqQSUBoNjUzE bDeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137751; x=1759742551; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ewFloRXAVA2+FWERqq7N2JT61DFtN9Zpod7cZzsTfrk=; b=H0QNBZYj2cielAm7ha4c9jo2FccznXizBmSR9nGLEK3n+HhzH5vTC2SWN83teLIDTF cp6VSAhR4OhvgfoUQlUGPwLb+DSobewU93HSyZPmbN7LaunKRWnBqFA6N6ADuHpNlPfI 5uYFU8nxVJPBu0Qkev0cL7i5yx/tSwffI7g2ZUMDtpdzGmDzwQGBnnuQw2aCrsZQA1nt l6iNwt+H17E2ROp6fDOf1s/eVQh2pFXLD+7yEv0rq7HxRB1BZLTgfu7feQfC5VSScM2V WBPLmJ2mnjGTAR5H4OCGxyMALzZVp0mV1owtKYCvZvqnPMHSD/YdPr2AqgRHj/pd/AU1 uV1Q== X-Gm-Message-State: AOJu0Ywe864J6r9HJ0cCoSFCgbeOyO4VY0tlQi0LTEVfxq7uOOAYu+pc w/G4mK0VSxbYEWPejDOHl1tCZ0Nlwj6lEHmmjEbA4K0Q01uOM2pNhCIz X-Gm-Gg: ASbGncsG+sEAW0QjouPKgH7sNwPeYqJN9qeq6uUHFyK5Rpc1KbZWMMdXOtpZp6qtO+L VyMB/sRDh0QiwFaaQSBNhVZe7eDwARj/1WQ/wFWMxVjYFjx99LlPzCJSCNwSg5pddQXitArvlNl krqoHuUqaaK8P4+ZJZh5kgUqR7Gbw5R/iXrDs4pVsYmeLZ0nCTOUN1yUWj+3FpPpNnmcWjaeExd AlvMW1iXxlJqMAIXtTc3tL1VP424Vui0WTYF7S+quuOSVIfsreHONmrcO1xBFLZ6jkTyWQbd9OE +7U0cG7BO+70d/8x9Ch7FMTSB4hFusU6qgBzeFw/ju7nIGAkPYE/0YpqNi8baMsVMKlbgMyToHs lEZKOaBuldxoktBLCXO12PLnZ/RZp/A== X-Google-Smtp-Source: AGHT+IHBWp+fRHg6YJhpsfypwX85hWi5tD2FmRz3bBtNRkCmaEIip3zV15sacPRM3XW5RsKr85Ce4g== X-Received: by 2002:a17:907:3f87:b0:b32:2b60:f13 with SMTP id a640c23a62f3a-b34bf94d3f9mr1899327066b.54.1759137750656; Mon, 29 Sep 2025 02:22:30 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:30 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 10/24] sched/deadline: Account rt-cgroups bandwidth in deadline tasks schedulability tests. Date: Mon, 29 Sep 2025 11:22:07 +0200 Message-ID: <20250929092221.10947-11-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Account the rt-cgroups hierarchy's reserved bandwidth in the schedulability test of deadline entities. This mechanism allows to completely reserve port= ion of the rt-bandwidth to rt-cgroups even if they do not use all of it. Account for the rt-cgroups' reserved bandwidth also when changing the total dedicated bandwidth for real time tasks. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/deadline.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 34a1494d782..754bfe231b4 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -213,8 +213,15 @@ void __dl_add(struct dl_bw *dl_b, u64 tsk_bw, int cpus) static inline bool __dl_overflow(struct dl_bw *dl_b, unsigned long cap, u64 old_bw, u64 new_b= w) { + u64 dl_groups_root =3D 0; + +#ifdef CONFIG_RT_GROUP_SCHED + dl_groups_root =3D to_ratio(root_task_group.dl_bandwidth.dl_period, + root_task_group.dl_bandwidth.dl_runtime); +#endif return dl_b->bw !=3D -1 && - cap_scale(dl_b->bw, cap) < dl_b->total_bw - old_bw + new_bw; + cap_scale(dl_b->bw, cap) < dl_b->total_bw - old_bw + new_bw + + cap_scale(dl_groups_root, cap); } =20 static inline @@ -3150,10 +3157,16 @@ int sched_dl_global_validate(void) u64 period =3D global_rt_period(); u64 new_bw =3D to_ratio(period, runtime); u64 cookie =3D ++dl_cookie; + u64 dl_groups_root =3D 0; struct dl_bw *dl_b; - int cpu, cpus, ret =3D 0; + int cpu, cap, cpus, ret =3D 0; unsigned long flags; =20 +#ifdef CONFIG_RT_GROUP_SCHED + dl_groups_root =3D to_ratio(root_task_group.dl_bandwidth.dl_period, + root_task_group.dl_bandwidth.dl_runtime); +#endif + /* * Here we want to check the bandwidth not being set to some * value smaller than the currently allocated bandwidth in @@ -3166,10 +3179,12 @@ int sched_dl_global_validate(void) goto next; =20 dl_b =3D dl_bw_of(cpu); + cap =3D dl_bw_capacity(cpu); cpus =3D dl_bw_cpus(cpu); =20 raw_spin_lock_irqsave(&dl_b->lock, flags); - if (new_bw * cpus < dl_b->total_bw) + if (new_bw * cpus < dl_b->total_bw + + cap_scale(dl_groups_root, cap)) ret =3D -EBUSY; raw_spin_unlock_irqrestore(&dl_b->lock, flags); =20 --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F5662C11EE for ; Mon, 29 Sep 2025 09:22:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137755; cv=none; b=EXT5O6nq3BXOpYnbd2iKt4n7qXDkcGNGsladLWhALotOCLqhLIEi3ms2osQjWRXBhKyVq0O82XjtTVftWkjnzmx7K+xzX5H4euCTVWST2ajVxhx+ZtmrZBs6mO0I78YKK9c6p1k0RBjRVsoS3vmuCtzwaeuYSavFY2UTWeGspLg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137755; c=relaxed/simple; bh=V4wngCUgQ/YZWcT/RBmIppyvJKtBN2bYG/baemQJjbc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ky7648TudT1PWDrgtO7y5WvLmUYXKNgGngtI0rclHkD3dve7Y20bQzYv+M0In8RgB5YnvwkyBfMvXsxVVU57fJgt690/yiPzQWxVBZz0fr1uPDlJiz6PLtXGoQ0UmRwgiQnGvZRdsIc7SVPXE3OTZV5vQZJF/4X7skqZ/8r1KCc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SWDcNhYh; arc=none smtp.client-ip=209.85.218.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SWDcNhYh" Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-b3727611d76so593431666b.2 for ; Mon, 29 Sep 2025 02:22:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137752; x=1759742552; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2dKw2ZUbysIQMVVKyG+B5HZc8IIYQ0mINr1zfzu0iXM=; b=SWDcNhYhvrxrRzjdpwBr/m/NXcVNRsBtbZMbBrzERB0frDXjwMt7/Vo8bjbgWWxe5u sIhkkPzDlkqs+lFuPQKXqmpcPGqMdvorfXeO0xATJZ4eeeyqezwESbofPZAkQ7lwlRpc 57pTiv+I62f9fqyCCwtvATKt/zqj3YVjKDytgOdFD5xLfK03W9ouSIiMRbHyG0e2JB7/ jfDwk9vXaRjWJicWAL4HrvS2bbAuOqkIH7j3W7RmrYaTbZ18UPqb+SXAO0fqR9YmINLQ lagyihkL3YwuyQnq//T6BvTCbSOCsx5xYLFGPdM7QQLKOse/JYNhTvwq8zecA2fZPw2j yyxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137752; x=1759742552; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2dKw2ZUbysIQMVVKyG+B5HZc8IIYQ0mINr1zfzu0iXM=; b=i7XX5bJiGM86ptjQl9vte+kSEX2MVi4lxy4gqpPYsXupFalsDTXEy17PNXXj6IjNNH /K5QXMCq888Wivv/SAtOfnJ7wBtcmCEG52ahzh2IWZMjVeIYz02MdxE+Otn9GRf07Hp7 ygMtzzmczTPjz1JODhCVnOB6NffQ2CcwUGLFWCIPzG3WGAPVXTbU3YwbbCtjeU3ab2+m /uRUamOtT7WcBPeNYp2SJsPkMC6dkeQF6Asjp5ZCs8XnF/QnxXSETeamEnF7zP5xJkur qsUclrGCyeS+i+MYeemDBlcGIxFfIIMe9Ix57B05v5RcEAmLlpP+p5st6ukKWFW5/4TR 9QzA== X-Gm-Message-State: AOJu0Yxpqt71g2YkIXy2rM5SkDOUbgVoxPpecPsHImNV2HML96C2rOAc dmxnNfjDEeRx4uI1PjUJOO8Si/m2r5RSu/WAlAcXykqqYvN1JC127t/m X-Gm-Gg: ASbGnctOVT55TXLoQoALga48BNgouXdXODHbNg6ezm5y6KqR8Q5dPr8j4fwJh4M0nVT qWt7TWtzKCHwa5e+OYjreH3+dOtJfo1zUQecXMLaNV03SwKz8rGYRTxt1GSjAOInJ9eah0NCEjD Mwq9grr3MJZ7QWNLar0L6+3NKfLEp+lwVAoNHQ8i8j34v0v4R+sSbfXkn9AulgrWR8IN/swrWln Rpq43BUSYYC4uiLfftaZjr/9Za8xVhqJS2xe6+H+fR+CrZSWBbV+DcMvqnN7aC8i4gefBlWBbE2 iX2L2k3fcHLw3l1y+NYfM28wn5U/Asmv05J1ebxfjYvjC4IIvHZhDLIfQl6IjigjD/ER34R7R/I wW0IcAI6rTE7hPGh0vvXV8zJJolI+gZGrguxHeaOt X-Google-Smtp-Source: AGHT+IGWOU4/onOPYt9ps0wf3cHeWpNMN3b9aoRdpnIwYN2/m2SasnTRFEqxZCFa27kB0CeiMFmK1A== X-Received: by 2002:a17:907:1c15:b0:afe:87bd:da59 with SMTP id a640c23a62f3a-b34be2f5318mr1753833066b.42.1759137751408; Mon, 29 Sep 2025 02:22:31 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:31 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 11/24] sched/rt: Add rt-cgroups' dl-servers operations. Date: Mon, 29 Sep 2025 11:22:08 +0200 Message-ID: <20250929092221.10947-12-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Implement the servers' functions that pick the next eligible task to run. Enable/Disable dl-servers on task enqueue/dequeue when necessary. Update dl-servers on task update. Account the number of active rt-tasks in the cgroups' specific runqueue. Account the number of active rt-tasks on the global counter of active tasks= when a cgroup is enqueued/dequeued (dl-server started/stopped). Update rq's cpuprio only if the cgroup's is root control group. Record which dl_server is managing a task when it changes runqueue. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: luca abeni Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/deadline.c | 16 ++++++--- kernel/sched/rt.c | 79 ++++++++++++++++++++++++++++++++++++----- kernel/sched/sched.h | 3 +- 3 files changed, 85 insertions(+), 13 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 754bfe231b4..1293b9a252b 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1869,9 +1869,13 @@ void inc_dl_tasks(struct sched_dl_entity *dl_se, str= uct dl_rq *dl_rq) u64 deadline =3D dl_se->deadline; =20 dl_rq->dl_nr_running++; - - if (!dl_server(dl_se)) + if (!dl_server(dl_se)) { add_nr_running(rq_of_dl_rq(dl_rq), 1); + } else if (dl_se !=3D &rq_of_dl_rq(dl_rq)->fair_server) { + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + + add_nr_running(rq_of_dl_rq(dl_rq), rt_rq->rt_nr_running); + } =20 inc_dl_deadline(dl_rq, deadline); } @@ -1881,9 +1885,13 @@ void dec_dl_tasks(struct sched_dl_entity *dl_se, str= uct dl_rq *dl_rq) { WARN_ON(!dl_rq->dl_nr_running); dl_rq->dl_nr_running--; - - if (!dl_server(dl_se)) + if (!dl_server(dl_se)) { sub_nr_running(rq_of_dl_rq(dl_rq), 1); + } else if (dl_se !=3D &rq_of_dl_rq(dl_rq)->fair_server) { + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + + sub_nr_running(rq_of_dl_rq(dl_rq), rt_rq->rt_nr_running); + } =20 dec_dl_deadline(dl_rq, dl_se->deadline); } diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 3094f59d0c8..d9442f64c6b 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -144,14 +144,27 @@ void init_tg_rt_entry(struct task_group *tg, struct r= q *served_rq, tg->dl_se[cpu] =3D dl_se; } =20 +static struct task_struct *_pick_next_task_rt(struct rt_rq *rt_rq); +static inline void set_next_task_rt(struct rq *rq, struct task_struct *p, = bool first); + static bool rt_server_has_tasks(struct sched_dl_entity *dl_se) { - return false; + return !!dl_se->my_q->rt.rt_nr_running; } =20 static struct task_struct *rt_server_pick(struct sched_dl_entity *dl_se) { - return NULL; + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + struct rq *rq =3D rq_of_rt_rq(rt_rq); + struct task_struct *p; + + if (dl_se->my_q->rt.rt_nr_running =3D=3D 0) + return NULL; + + p =3D _pick_next_task_rt(rt_rq); + set_next_task_rt(rq, p, true); + + return p; } =20 int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) @@ -416,6 +429,7 @@ static inline int rt_se_prio(struct sched_rt_entity *rt= _se) static void update_curr_rt(struct rq *rq) { struct task_struct *donor =3D rq->donor; + struct rt_rq *rt_rq; s64 delta_exec; =20 if (donor->sched_class !=3D &rt_sched_class) @@ -425,8 +439,18 @@ static void update_curr_rt(struct rq *rq) if (unlikely(delta_exec <=3D 0)) return; =20 - if (!rt_bandwidth_enabled()) + if (!rt_group_sched_enabled()) + return; + + if (!dl_bandwidth_enabled()) return; + + rt_rq =3D rt_rq_of_se(&donor->rt); + if (is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + dl_server_update(dl_se, delta_exec); + } } =20 static void @@ -437,7 +461,7 @@ inc_rt_prio_smp(struct rt_rq *rt_rq, int prio, int prev= _prio) /* * Change rq's cpupri only if rt_rq is the top queue. */ - if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && &rq->rt !=3D rt_rq) + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) return; =20 if (rq->online && prio < prev_prio) @@ -452,7 +476,7 @@ dec_rt_prio_smp(struct rt_rq *rt_rq, int prio, int prev= _prio) /* * Change rq's cpupri only if rt_rq is the top queue. */ - if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && &rq->rt !=3D rt_rq) + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) return; =20 if (rq->online && rt_rq->highest_prio.curr !=3D prev_prio) @@ -521,6 +545,15 @@ void inc_rt_tasks(struct sched_rt_entity *rt_se, struc= t rt_rq *rt_rq) rt_rq->rr_nr_running +=3D rt_se_rr_nr_running(rt_se); =20 inc_rt_prio(rt_rq, rt_se_prio(rt_se)); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (!dl_se->dl_throttled) + add_nr_running(rq_of_rt_rq(rt_rq), 1); + } else { + add_nr_running(rq_of_rt_rq(rt_rq), 1); + } } =20 static inline @@ -531,6 +564,15 @@ void dec_rt_tasks(struct sched_rt_entity *rt_se, struc= t rt_rq *rt_rq) rt_rq->rr_nr_running -=3D rt_se_rr_nr_running(rt_se); =20 dec_rt_prio(rt_rq, rt_se_prio(rt_se)); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (!dl_se->dl_throttled) + sub_nr_running(rq_of_rt_rq(rt_rq), 1); + } else { + sub_nr_running(rq_of_rt_rq(rt_rq), 1); + } } =20 /* @@ -712,6 +754,14 @@ enqueue_task_rt(struct rq *rq, struct task_struct *p, = int flags) check_schedstat_required(); update_stats_wait_start_rt(rt_rq_of_se(rt_se), rt_se); =20 + /* Task arriving in an idle group of tasks. */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + is_dl_group(rt_rq) && rt_rq->rt_nr_running =3D=3D 0) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + dl_server_start(dl_se); + } + enqueue_rt_entity(rt_se, flags); =20 if (task_is_blocked(p)) @@ -731,6 +781,14 @@ static bool dequeue_task_rt(struct rq *rq, struct task= _struct *p, int flags) =20 dequeue_pushable_task(rt_rq, p); =20 + /* Last task of the task group. */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + is_dl_group(rt_rq) && rt_rq->rt_nr_running =3D=3D 0) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + dl_server_stop(dl_se); + } + return true; } =20 @@ -953,9 +1011,14 @@ static struct sched_rt_entity *pick_next_rt_entity(st= ruct rt_rq *rt_rq) return next; } =20 -static struct task_struct *_pick_next_task_rt(struct rq *rq) +static struct task_struct *_pick_next_task_rt(struct rt_rq *rt_rq) { - return NULL; + struct sched_rt_entity *rt_se; + + rt_se =3D pick_next_rt_entity(rt_rq); + BUG_ON(!rt_se); + + return rt_task_of(rt_se); } =20 static struct task_struct *pick_task_rt(struct rq *rq) @@ -965,7 +1028,7 @@ static struct task_struct *pick_task_rt(struct rq *rq) if (!sched_rt_runnable(rq)) return NULL; =20 - p =3D _pick_next_task_rt(rq); + p =3D _pick_next_task_rt(&rq->rt); =20 return p; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 9853f321363..b2c87541257 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2170,7 +2170,7 @@ static inline void set_task_rq(struct task_struct *p,= unsigned int cpu) if (!rt_group_sched_enabled()) tg =3D &root_task_group; p->rt.rt_rq =3D tg->rt_rq[cpu]; - p->rt.parent =3D tg->rt_se[cpu]; + p->dl.dl_rq =3D &cpu_rq(cpu)->dl; #endif /* CONFIG_RT_GROUP_SCHED */ } =20 @@ -2726,6 +2726,7 @@ static inline void add_nr_running(struct rq *rq, unsi= gned count) =20 static inline void sub_nr_running(struct rq *rq, unsigned count) { + BUG_ON(rq->nr_running < count); rq->nr_running -=3D count; if (trace_sched_update_nr_running_tp_enabled()) { call_trace_sched_update_nr_running(rq, -count); --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FF152C15BC for ; Mon, 29 Sep 2025 09:22:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137756; cv=none; b=uT36RWpt/k/PDuB58+n7+xYqsCKkh2V/I/Mn7h+PCoc1dSjOYFoA4TvJGVs4zBriaut3Kt+VtuiZHuiu0zVE4nJcLYkCT+NTQ81eY2WrRL10Y/vYOM72Lam2+eXpNWZA0u42H7Lrg/fQTd3wa6soMU8tIuiG6fL9BC80MYaAUMk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137756; c=relaxed/simple; bh=BwH/PxaUKt6ws7RHR35k6Zp2DnWln8HMf2+ObNT5XMc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PXq9gB9ISccGZoB43zbuwAYhSEyvJpUyvI3QUJQ3yQczkh0EwdJL6b3yMb5oW+y9MuUcnGdDtUl6+sVy+GY7UM31sEKgcJ6K+1k9rJpZoOed81YE5l3lAfwaAxxTCFF2raESEtwLpdiBBhyy5X4fDBD6wWLkH2MsMBGvPwlK3pM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=P+QECEvO; arc=none smtp.client-ip=209.85.208.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="P+QECEvO" Received: by mail-ed1-f47.google.com with SMTP id 4fb4d7f45d1cf-62fc2d92d34so7980172a12.2 for ; Mon, 29 Sep 2025 02:22:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137752; x=1759742552; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2Q0EwLJfZBwPemnG8QeZQUc6RbkgXm11UO0MNIXaVYo=; b=P+QECEvO1hdPs7RosuYOKFsLZTbx7C9oju/YhK9uAjlULn/TCj+tg0Ti4vyJiS7CAr WBITf3KlDbjy76i/CXTJMML1rPKzZGcX2Cu5Lel03PTfs/ZdULgJ+HYaAQgsDSGeApJY fcJyg/aKU9guov9N4owqDRWpKCmMupM4b63mO5huvik04MLdNONJm5TCO2tP9wsMtCyA xyCo8M/7K/6sfOEqMgfQeFQvd/YP4VOnrwhy/Zu3wzS4xbo8+lts7BN4+Xc1q8YuwhTg Rq2L14l+PHb4xEOd73f8cxWcLPt7ELfKQ6dK8c7s4Vf2B59lsDqQG8RAyyiS63EhJFez 06TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137752; x=1759742552; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2Q0EwLJfZBwPemnG8QeZQUc6RbkgXm11UO0MNIXaVYo=; b=QVwvnoRb4giz7Kjm9zP2x1MyVnaR9VwC8IIWyAVtvoaFOHVK/UVR9d1hyKiCbwBfxe QIBdHvQ0OuS+x9FvsTkSK2r/AuK7dFUO2J5swNfEKscfUftLO4uKCNtdzoyg89pREo32 nq9apKMkAlidnLxnnvBQyHfnTxXuKtrSDa7h1LsYRoyp68G7y/XMF7CG5myRniGy3cHR Q5DZsTSRyJvt6lKlb5l9nqwrWPJ1/Sn7yY2MTU7Uqgq22myjj2Jh8Tzro/0s3o36Qs6M Ezl/Be4sJ1fsGYtqksTdeTc4oLFFXmAnhecOnQXThCDN0kld+oHyYbgyDpI4WOpNDHja 6WAA== X-Gm-Message-State: AOJu0YzmgNbBc2Nxueksjkf9XzFN348LtchV0DUqefemq4CUOMmjiGkX igCWmnlLWCqEh9P8xSSPHHRsm7w8wv53p89HIamtTMLPTVT/ITiI1ulA X-Gm-Gg: ASbGnctzW6GY+nKTVGXdh0X266bwpyjMuuWGHyT7jVAX9LePohMeg5eVnH4cGT8cI45 rrbNJq3mtgdn0obkVqM8zpAuuJSrpy1iiCLT7wVJWZWcxC3iZj2L8307b3jpkDxrZjX5v8zmhuz re4vOwJGpi0Y3P27AqLwXgBVxcirQVPcw3f8HIsT4K70wWzXm2aIlyUxRhfSivVNrLv3+yDVkZ1 xlrnOhhaQxQ2jVxuxZThILqChU1OYnd8QyQDu/vt3pKO4BjKJdcLJiJ+0atbAEB2AYOX7x1dA/r b9QqoDexLQDwlRz4WvtTL4FdbZMS6Dctj28qqPl96Y4/2paDyDz8tqlH62+mAzJIVOTKadIQV1V Url8YjY3eNeqLM5OCHyw= X-Google-Smtp-Source: AGHT+IG6Ja8q+uUOfXqXgMp6HaNsgVjHaNzUOnk5CK5fJWUFd3O1nCODhM1aQM2bbdLHZWvGNr763A== X-Received: by 2002:a17:907:9446:b0:b04:aa9d:2240 with SMTP id a640c23a62f3a-b34bde135e3mr1686360266b.39.1759137752416; Mon, 29 Sep 2025 02:22:32 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:32 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 12/24] sched/rt: Update task event callbacks for HCBS scheduling Date: Mon, 29 Sep 2025 11:22:09 +0200 Message-ID: <20250929092221.10947-13-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Update wakeup_preempt_rt, switched_{from/to}_rt and prio_changed_rt with rt-cgroup's specific preemption rules. Add checks whether a rt-task can be attached or not to a rt-cgroup. Update task_is_throttled_rt for SCHED_CORE. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: luca abeni Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 2 +- kernel/sched/rt.c | 88 ++++++++++++++++++++++++++++++++++++++--- kernel/sched/syscalls.c | 13 ++++++ 3 files changed, 96 insertions(+), 7 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index e5b4facee24..2cfbe3b7b17 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9346,7 +9346,7 @@ static int cpu_cgroup_can_attach(struct cgroup_taskse= t *tset) goto scx_check; =20 cgroup_taskset_for_each(task, css, tset) { - if (!sched_rt_can_attach(css_tg(css), task)) + if (rt_task(task) && !sched_rt_can_attach(css_tg(css), task)) return -EINVAL; } scx_check: diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index d9442f64c6b..ce114823fe7 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -946,6 +946,50 @@ static void wakeup_preempt_rt(struct rq *rq, struct ta= sk_struct *p, int flags) { struct task_struct *donor =3D rq->donor; =20 + if (!rt_group_sched_enabled()) + goto no_group_sched; + + /* + * Preemption checks are different if the waking task and the current + * task are running on the global runqueue or in a cgroup. + * The following rules apply: + * - dl-tasks (and equally dl_servers) always preempt FIFO/RR tasks. + * - if curr is inside a cgroup (i.e. run by a dl_server) and + * waking is not, do nothing. + * - if waking is inside a cgroup but not curr, always reschedule. + * - if they are both on the global runqueue, run the standard code. + * - if they are both in the same cgroup, check for tasks priorities. + * - if they are both in a cgroup, but not the same one, check whether + * the woken task's dl_server preempts the current's dl_server. + */ + if (is_dl_group(rt_rq_of_se(&p->rt)) && + is_dl_group(rt_rq_of_se(&rq->curr->rt))) { + struct sched_dl_entity *woken_dl_se, *curr_dl_se; + + woken_dl_se =3D dl_group_of(rt_rq_of_se(&p->rt)); + curr_dl_se =3D dl_group_of(rt_rq_of_se(&rq->curr->rt)); + + if (rt_rq_of_se(&p->rt)->tg =3D=3D rt_rq_of_se(&rq->curr->rt)->tg) { + if (p->prio < rq->curr->prio) + resched_curr(rq); + + return; + } + + if (dl_entity_preempt(woken_dl_se, curr_dl_se)) + resched_curr(rq); + + return; + + } else if (is_dl_group(rt_rq_of_se(&p->rt))) { + resched_curr(rq); + return; + + } else if (is_dl_group(rt_rq_of_se(&rq->curr->rt))) { + return; + } + +no_group_sched: if (p->prio < donor->prio) { resched_curr(rq); return; @@ -1705,6 +1749,8 @@ static void rq_offline_rt(struct rq *rq) */ static void switched_from_rt(struct rq *rq, struct task_struct *p) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); + /* * If there are other RT tasks then we will reschedule * and the scheduling of the other RT tasks will handle @@ -1712,10 +1758,11 @@ static void switched_from_rt(struct rq *rq, struct = task_struct *p) * we may need to handle the pulling of RT tasks * now. */ - if (!task_on_rq_queued(p) || rq->rt.rt_nr_running) + if (!task_on_rq_queued(p) || rt_rq->rt_nr_running) return; =20 - rt_queue_pull_task(rt_rq_of_se(&p->rt)); + if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED)) + rt_queue_pull_task(rt_rq); } =20 void __init init_sched_rt_class(void) @@ -1750,8 +1797,17 @@ static void switched_to_rt(struct rq *rq, struct tas= k_struct *p) * then see if we can move to another run queue. */ if (task_on_rq_queued(p)) { + +#ifndef CONFIG_RT_GROUP_SCHED if (p->nr_cpus_allowed > 1 && rq->rt.overloaded) rt_queue_push_tasks(rt_rq_of_se(&p->rt)); +#else + if (rt_rq_of_se(&p->rt)->overloaded) { + } else { + if (p->prio < rq->curr->prio) + resched_curr(rq); + } +#endif if (p->prio < rq->donor->prio && cpu_online(cpu_of(rq))) resched_curr(rq); } @@ -1764,6 +1820,8 @@ static void switched_to_rt(struct rq *rq, struct task= _struct *p) static void prio_changed_rt(struct rq *rq, struct task_struct *p, int oldprio) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); + if (!task_on_rq_queued(p)) return; =20 @@ -1772,16 +1830,25 @@ prio_changed_rt(struct rq *rq, struct task_struct *= p, int oldprio) * If our priority decreases while running, we * may need to pull tasks to this runqueue. */ - if (oldprio < p->prio) - rt_queue_pull_task(rt_rq_of_se(&p->rt)); + if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED) && oldprio < p->prio) + rt_queue_pull_task(rt_rq); =20 /* * If there's a higher priority task waiting to run * then reschedule. */ - if (p->prio > rq->rt.highest_prio.curr) + if (p->prio > rt_rq->highest_prio.curr) resched_curr(rq); } else { + /* + * This task is not running, thus we check against the currently + * running task for preemption. We can preempt only if both tasks are + * in the same cgroup or on the global runqueue. + */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + rt_rq_of_se(&p->rt)->tg !=3D rt_rq_of_se(&rq->curr->rt)->tg) + return; + /* * This task is not running, but if it is * greater than the current running task @@ -1876,7 +1943,16 @@ static unsigned int get_rr_interval_rt(struct rq *rq= , struct task_struct *task) #ifdef CONFIG_SCHED_CORE static int task_is_throttled_rt(struct task_struct *p, int cpu) { +#ifdef CONFIG_RT_GROUP_SCHED + struct rt_rq *rt_rq; + + rt_rq =3D task_group(p)->rt_rq[cpu]; + WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); + + return dl_group_of(rt_rq)->dl_throttled; +#else return 0; +#endif } #endif /* CONFIG_SCHED_CORE */ =20 @@ -2131,7 +2207,7 @@ static int sched_rt_global_constraints(void) int sched_rt_can_attach(struct task_group *tg, struct task_struct *tsk) { /* Don't accept real-time tasks when there is no way for them to run */ - if (rt_group_sched_enabled() && rt_task(tsk) && tg->rt_bandwidth.rt_runti= me =3D=3D 0) + if (rt_group_sched_enabled() && tg->dl_bandwidth.dl_runtime =3D=3D 0) return 0; =20 return 1; diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index 93a9c03b28e..71f20be6f29 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -626,6 +626,19 @@ int __sched_setscheduler(struct task_struct *p, change: =20 if (user) { +#ifdef CONFIG_RT_GROUP_SCHED + /* + * Do not allow real-time tasks into groups that have no runtime + * assigned. + */ + if (rt_group_sched_enabled() && + dl_bandwidth_enabled() && rt_policy(policy) && + !sched_rt_can_attach(task_group(p), p) && + !task_group_is_autogroup(task_group(p))) { + retval =3D -EPERM; + goto unlock; + } +#endif if (dl_bandwidth_enabled() && dl_policy(policy) && !(attr->sched_flags & SCHED_FLAG_SUGOV)) { cpumask_t *span =3D rq->rd->span; --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F36102C21E4 for ; Mon, 29 Sep 2025 09:22:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137757; cv=none; b=hDVWPUxNkK8l0q01R6ovzG+PKuJnNO8FNff5KheDzwJ9vqal+qKK3KkPHLZ9Vn8R8XlyrCiOGjJh48hF1F6Vtl6tgDkHYZMHyxgaDPwLh9WejrZqQqDsCviQnkuu4SEDSgqeiNIinSoZbYaIWKjqv2euDd/ll6Mpxe6tJ3tM1LY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137757; c=relaxed/simple; bh=fCqvrO2cKxcNa4ZL1uaDwbPb7ivBVURv4PDyaiKipFk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bBBk61sCs+h1n9VwRgBr5P6jpm4WUPypD+dOWCsxkQhG5rKuhBQj8hGMM2WDlM8uswzoqIv9TfQd3itebh6SSJ4oYc92uiBNbZFQADmC8jU1erW3zqjrf3gyAdqLJx0C5uGN37MUGnzwLD0BzhykrOMpa9AkVmIecblsgsqksOM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=djD0vZoa; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="djD0vZoa" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-b3ee18913c0so133097366b.3 for ; Mon, 29 Sep 2025 02:22:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137753; x=1759742553; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=L3bcCdHXskWPXk6GtHk1u/zclf51+7VczfN2xDqdC/U=; b=djD0vZoayGfg31Fkivq+cwlaAiEvirFgk86ZrXkjrLzl0BG40zuPcq9CNcXmMvjcyy 5JGyK3whJxMp5xtMqetoRlEBYf0UG+1lQpvCwH4c93wkbuSiB44zPWCE3Lml8g0bhwLS /rHxFuhgTl8Tk54LCQl2iTMqw+WrqEGOlc0kDSyEn0x3wmmtMwKax+xlVLRWsjQfEJ/0 H/aTw7vjMAiNCWXwWM5LDqLhJkfee8KAME2hnNCzRkgWU9KdWJJMGSFG81VzN9hgQ8jp Kg94CYQ+PsCkS8sWtguiIhu+3MKT1Z3mVBQT41NhlnA+ZH3OENM7QKST+fyspuwWgGQQ 5R2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137753; x=1759742553; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L3bcCdHXskWPXk6GtHk1u/zclf51+7VczfN2xDqdC/U=; b=UUACrP+blu5+MBKZkH6c98jkb85gZzmU/23bRdrydUj8gNSBOjmj2Rc/XD2bYrUa30 GzRliPj5G2G7WJ133IqwZ/0ZYjMollg/RaFH+0mJjBnsofNSa8E26metKD0ce9hCYfxf 8ZOOZ426nlMGWJQlwJhLnvCdTqoP8gDsg8jWmvouRKDJOAW8h1qCuChCGxhlMy8d0FfM d9LzfiE2N7dFY0LWJPAaRZooxCBMNRjgJ5OMemJbmdMoNQeZYJ+Docav4hhDaxI/8P5b dMtjkUShA6faFukD32S0SChQU6EmE6kJekECi42997ywRs7fwd7sd4LrSVULXMDHzO1U m/BA== X-Gm-Message-State: AOJu0YyYxtWQMd+tT2mu9UggSjJNFdLgVG5gk2SGuJDmH334Sg6XBeOG uu7whbdZcTOCqEseXr3o7aoCU64acfacTYTOle9yWFC3QGsoa6KM+llG X-Gm-Gg: ASbGncv9fAWmCHiYrNwKRqcQQfhg69E685G8E6TnkF4Wr+k7pOpswaK+HWDVmeOB+qk viVzP0tzd3zZBpeLjfqNOp6O9qDS88krRAu4PBLX6Viambu5cNauCiYK2p/vu1lXDTWPCvcqjnb XuA+ub/SIvXKz6K9lJ9vZOI0hOwd5ipMOUIC8Tr9/buB0vaLX9qiqOy4DKidNH97mSBhJMLIaTY cx5r48pm611vRCsUGYzbJL4M6DBSvGAjslotpI34DD0hHNCzim1jSip4b6SvwVzF6X+aRkAYEht pmyWRYBFtb4j18kDd+WYoeiJrv3x7tLP30JlOvmxveXWWaMePwtewMVeHlogPy5/Nb5T5E0YdEF FEdIQdq/6wRHT9S7hGUydvGBauDm7QQ== X-Google-Smtp-Source: AGHT+IEn9FStZky34j+Urt4Ym2Wku9Rm+dkOnekNRZ2Xd7o7lJC2WWohqOTd5aQpvF0vQfr4muGSgw== X-Received: by 2002:a17:907:1c95:b0:b2b:c145:ab8f with SMTP id a640c23a62f3a-b34bdd13351mr1338977966b.65.1759137753142; Mon, 29 Sep 2025 02:22:33 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:32 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 13/24] sched/rt: Update rt-cgroup schedulability checks Date: Mon, 29 Sep 2025 11:22:10 +0200 Message-ID: <20250929092221.10947-14-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Update schedulability checks and setup of runtime/period for rt-cgroups. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/core.c | 6 ++++ kernel/sched/deadline.c | 46 +++++++++++++++++++++++---- kernel/sched/rt.c | 70 +++++++++++++++++++++++------------------ kernel/sched/sched.h | 1 + 4 files changed, 87 insertions(+), 36 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2cfbe3b7b17..1217f714dd2 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9281,6 +9281,12 @@ cpu_cgroup_css_alloc(struct cgroup_subsys_state *par= ent_css) return &root_task_group.css; } =20 + /* Do not allow cpu_cgroup hierachies with depth greater than 2. */ +#ifdef CONFIG_RT_GROUP_SCHED + if (parent !=3D &root_task_group) + return ERR_PTR(-EINVAL); +#endif + tg =3D sched_create_group(parent); if (IS_ERR(tg)) return ERR_PTR(-ENOMEM); diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 1293b9a252b..5d93b3ca030 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -347,7 +347,47 @@ void cancel_inactive_timer(struct sched_dl_entity *dl_= se) cancel_dl_timer(dl_se, &dl_se->inactive_timer); } =20 +/* + * Used for dl_bw check and update, used under sched_rt_handler()::mutex a= nd + * sched_domains_mutex. + */ +u64 dl_cookie; + #ifdef CONFIG_RT_GROUP_SCHED +int dl_check_tg(unsigned long total) +{ + unsigned long flags; + int which_cpu; + int cpus; + struct dl_bw *dl_b; + u64 gen =3D ++dl_cookie; + + for_each_possible_cpu(which_cpu) { + rcu_read_lock_sched(); + + if (!dl_bw_visited(which_cpu, gen)) { + cpus =3D dl_bw_cpus(which_cpu); + dl_b =3D dl_bw_of(which_cpu); + + raw_spin_lock_irqsave(&dl_b->lock, flags); + + if (dl_b->bw !=3D -1 && + dl_b->bw * cpus < dl_b->total_bw + total * cpus) { + raw_spin_unlock_irqrestore(&dl_b->lock, flags); + rcu_read_unlock_sched(); + + return 0; + } + + raw_spin_unlock_irqrestore(&dl_b->lock, flags); + } + + rcu_read_unlock_sched(); + } + + return 1; +} + void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 rt_peri= od) { struct rq *rq =3D container_of(dl_se->dl_rq, struct rq, dl); @@ -3153,12 +3193,6 @@ DEFINE_SCHED_CLASS(dl) =3D { #endif }; =20 -/* - * Used for dl_bw check and update, used under sched_rt_handler()::mutex a= nd - * sched_domains_mutex. - */ -u64 dl_cookie; - int sched_dl_global_validate(void) { u64 runtime =3D global_rt_runtime(); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index ce114823fe7..7c7622303e8 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1996,11 +1996,6 @@ DEFINE_SCHED_CLASS(rt) =3D { }; =20 #ifdef CONFIG_RT_GROUP_SCHED -/* - * Ensure that the real time constraints are schedulable. - */ -static DEFINE_MUTEX(rt_constraints_mutex); - static inline int tg_has_rt_tasks(struct task_group *tg) { struct task_struct *task; @@ -2034,8 +2029,8 @@ static int tg_rt_schedulable(struct task_group *tg, v= oid *data) unsigned long total, sum =3D 0; u64 period, runtime; =20 - period =3D ktime_to_ns(tg->rt_bandwidth.rt_period); - runtime =3D tg->rt_bandwidth.rt_runtime; + period =3D tg->dl_bandwidth.dl_period; + runtime =3D tg->dl_bandwidth.dl_runtime; =20 if (tg =3D=3D d->tg) { period =3D d->rt_period; @@ -2051,8 +2046,7 @@ static int tg_rt_schedulable(struct task_group *tg, v= oid *data) /* * Ensure we don't starve existing RT tasks if runtime turns zero. */ - if (rt_bandwidth_enabled() && !runtime && - tg->rt_bandwidth.rt_runtime && tg_has_rt_tasks(tg)) + if (dl_bandwidth_enabled() && !runtime && tg_has_rt_tasks(tg)) return -EBUSY; =20 if (WARN_ON(!rt_group_sched_enabled() && tg !=3D &root_task_group)) @@ -2066,12 +2060,17 @@ static int tg_rt_schedulable(struct task_group *tg,= void *data) if (total > to_ratio(global_rt_period(), global_rt_runtime())) return -EINVAL; =20 + if (tg =3D=3D &root_task_group) { + if (!dl_check_tg(total)) + return -EBUSY; + } + /* * The sum of our children's runtime should not exceed our own. */ list_for_each_entry_rcu(child, &tg->children, siblings) { - period =3D ktime_to_ns(child->rt_bandwidth.rt_period); - runtime =3D child->rt_bandwidth.rt_runtime; + period =3D child->dl_bandwidth.dl_period; + runtime =3D child->dl_bandwidth.dl_runtime; =20 if (child =3D=3D d->tg) { period =3D d->rt_period; @@ -2097,6 +2096,20 @@ static int __rt_schedulable(struct task_group *tg, u= 64 period, u64 runtime) .rt_runtime =3D runtime, }; =20 + /* + * Since we truncate DL_SCALE bits, make sure we're at least + * that big. + */ + if (runtime !=3D 0 && runtime < (1ULL << DL_SCALE)) + return -EINVAL; + + /* + * Since we use the MSB for wrap-around and sign issues, make + * sure it's not set (mind that period can be equal to zero). + */ + if (period & (1ULL << 63)) + return -EINVAL; + rcu_read_lock(); ret =3D walk_tg_tree(tg_rt_schedulable, tg_nop, &data); rcu_read_unlock(); @@ -2107,6 +2120,7 @@ static int __rt_schedulable(struct task_group *tg, u6= 4 period, u64 runtime) static int tg_set_rt_bandwidth(struct task_group *tg, u64 rt_period, u64 rt_runtime) { + static DEFINE_MUTEX(rt_constraints_mutex); int i, err =3D 0; =20 /* @@ -2126,34 +2140,30 @@ static int tg_set_rt_bandwidth(struct task_group *t= g, if (rt_runtime !=3D RUNTIME_INF && rt_runtime > max_rt_runtime) return -EINVAL; =20 - mutex_lock(&rt_constraints_mutex); + guard(mutex)(&rt_constraints_mutex); err =3D __rt_schedulable(tg, rt_period, rt_runtime); if (err) - goto unlock; + return err; =20 - raw_spin_lock_irq(&tg->rt_bandwidth.rt_runtime_lock); - tg->rt_bandwidth.rt_period =3D ns_to_ktime(rt_period); - tg->rt_bandwidth.rt_runtime =3D rt_runtime; + guard(raw_spinlock_irq)(&tg->dl_bandwidth.dl_runtime_lock); + tg->dl_bandwidth.dl_period =3D rt_period; + tg->dl_bandwidth.dl_runtime =3D rt_runtime; =20 - for_each_possible_cpu(i) { - struct rt_rq *rt_rq =3D tg->rt_rq[i]; + if (tg =3D=3D &root_task_group) + return 0; =20 - raw_spin_lock(&rt_rq->rt_runtime_lock); - rt_rq->rt_runtime =3D rt_runtime; - raw_spin_unlock(&rt_rq->rt_runtime_lock); + for_each_possible_cpu(i) { + dl_init_tg(tg->dl_se[i], rt_runtime, rt_period); } - raw_spin_unlock_irq(&tg->rt_bandwidth.rt_runtime_lock); -unlock: - mutex_unlock(&rt_constraints_mutex); =20 - return err; + return 0; } =20 int sched_group_set_rt_runtime(struct task_group *tg, long rt_runtime_us) { u64 rt_runtime, rt_period; =20 - rt_period =3D ktime_to_ns(tg->rt_bandwidth.rt_period); + rt_period =3D tg->dl_bandwidth.dl_period; rt_runtime =3D (u64)rt_runtime_us * NSEC_PER_USEC; if (rt_runtime_us < 0) rt_runtime =3D RUNTIME_INF; @@ -2167,10 +2177,10 @@ long sched_group_rt_runtime(struct task_group *tg) { u64 rt_runtime_us; =20 - if (tg->rt_bandwidth.rt_runtime =3D=3D RUNTIME_INF) + if (tg->dl_bandwidth.dl_runtime =3D=3D RUNTIME_INF) return -1; =20 - rt_runtime_us =3D tg->rt_bandwidth.rt_runtime; + rt_runtime_us =3D tg->dl_bandwidth.dl_runtime; do_div(rt_runtime_us, NSEC_PER_USEC); return rt_runtime_us; } @@ -2183,7 +2193,7 @@ int sched_group_set_rt_period(struct task_group *tg, = u64 rt_period_us) return -EINVAL; =20 rt_period =3D rt_period_us * NSEC_PER_USEC; - rt_runtime =3D tg->rt_bandwidth.rt_runtime; + rt_runtime =3D tg->dl_bandwidth.dl_runtime; =20 return tg_set_rt_bandwidth(tg, rt_period, rt_runtime); } @@ -2192,7 +2202,7 @@ long sched_group_rt_period(struct task_group *tg) { u64 rt_period_us; =20 - rt_period_us =3D ktime_to_ns(tg->rt_bandwidth.rt_period); + rt_period_us =3D tg->dl_bandwidth.dl_period; do_div(rt_period_us, NSEC_PER_USEC); return rt_period_us; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index b2c87541257..97e1e779df9 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -394,6 +394,7 @@ extern void dl_server_init(struct sched_dl_entity *dl_s= e, struct dl_rq *dl_rq, dl_server_has_tasks_f has_tasks, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); +extern int dl_check_tg(unsigned long total); extern void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 = rt_period); =20 extern void dl_server_update_idle_time(struct rq *rq, --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0905286D63 for ; Mon, 29 Sep 2025 09:22:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137757; cv=none; b=L6DUGp28hJ5ZjZgiyCQiJH9NgPrt771vmQF3AEDbrQAu0uYggL0XOJPfZJ+jXlMff5BxRmdasiMv8GOEmIE6NfEp6XoPyrrUZM33LCA9JArdTlI0FjUvNNb6UmGC42/2jvAbey9TUZQ247XZiPiRh4e6QBdR04fcKxFwES+HfL0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137757; c=relaxed/simple; bh=OE3e7QjLyUEcviZfsBugdK1SdHoykK0TuJW0UPWcFUw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=md/g+R/BLwE5reofGBFU4UBU7mbD1PTUEiXejjy6YesjRxiGJ9IxjS/inFgBTyaBv81Dg9FcAxPW/iSYDIm0LcvW//sfSpeT0HfBhkbFngeC2JZN2CQSfM+SoyEKDdAOUXPlLqtLoxnQykNVS9CdJ5fppghaWjjjDWr9k3K92Mk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KLpmd+8p; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KLpmd+8p" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b2e66a300cbso863894566b.3 for ; Mon, 29 Sep 2025 02:22:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137754; x=1759742554; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yMlwnkxKk+zcMClE+vVN4lHuHMN3457FeNpmPjxk3go=; b=KLpmd+8pHmfI+tIF/XgMcyaZugDGFyD1S3v5z+vnpp+wmxFv9tTjUcOXGVt0epooJj O9AGzGfFfb5i45bbtE/pvoQxbqvB1k4bI+3dARi++U/UmEfn00vBMdcn3I+OoxZvUdPi CfyK0OvLutecw4iGizsaLIXmzRM8G5WlfsLOHbisCW1tZD+VW1tW0fJlVqsW7CZcDYxv gbYeZxNEFB9NciF2qVVQdGXCDHCnCNnnKbABKatYdSMfbv+2xh6HutE82U52gv4/fxDH WvkX/eUSmdMO5WOSEj5EMQweas2pBL8JjHJi/mXjSqWqlofU0uAIkUGWh7Yr9AjofGJ4 F3ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137754; x=1759742554; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yMlwnkxKk+zcMClE+vVN4lHuHMN3457FeNpmPjxk3go=; b=bao+PSM8qM36+Vsgl1bVWzsvokznjdEYduckkY5L1gk0zzRiobtrjFU0JKLiRkWfsb YGu/slu84ylpiQyVe2pFBVPzSHc5DySZ0reC6EYTujoSoojnx6QnULVU54ewkV/fXrqb jDe1ieJFyOshfOwsK/WsJqKQayd8M2RXdzFmw9M9ZHjwqcL7E/G+hhAjhQ28i9FfgRib KvlTe+Yi0LQROWeGTwEzUWRNdKUT+4vRIwru0v44aLc4vaOAxSNsuIouJYtEk8aiyG8v QP7TK5jPl3/BGE6AzcGL+Xc2DqpaYJc0DjLQzxs4+JyjsbL42ZrxesoFd06bHb8HsaiA LWeQ== X-Gm-Message-State: AOJu0YzM7bnMpca7pGlPOGj9Evj5e/CNLrEDPjbLE/DaY/zubvjQOOEe V15sMaBXFnAMA1Svt99qprCkAAQUQizO2ixbh9lkSSeCL2PZWcS3Dr8W X-Gm-Gg: ASbGnctgRgt5J46C/1x4ueb3xpJU9XH609siG4C+4I/cu69akOBpkEimAbJ1UyPCiGC bojbR4haYsQ9bZt2wJWdu4ihQGLgDrV+jOVdO8uDga6zI/gGISdcDHLtKvOPvF0ZTkZ2dd6WKzb KbDsJ6fGRoGwCC9WdmQrb1RAZar9/0gr1jJz76Bdc5kW9p+z39YeJ1QkfhXHOT2+PMP5Brr09y8 HocimpC4HT8YNj6mGU9tVZb4c54JneBpVMM49KXr3kq+gg5L9ML9iAioOACz7NSdXK4XlysWGrh BNbjc8qdN5+6UyaIXFn45/K4YTI1Rq24eJT2iL5t44YgAhSF+1I4j4DD+zkG+X0MrFwOVn/YIYm KylYG9LZsWYenIBJ8QSs= X-Google-Smtp-Source: AGHT+IF476mouee226vXmTR6yINrp78K5Vwc2/xGClwUm27zxjgQfBp+GLTogHG6L4P/AA+lzG+kWA== X-Received: by 2002:a17:907:a088:b0:b3c:8b25:ab74 with SMTP id a640c23a62f3a-b3c8b447df4mr586947866b.10.1759137753919; Mon, 29 Sep 2025 02:22:33 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:33 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 14/24] sched/rt: Allow zeroing the runtime of the root control group Date: Mon, 29 Sep 2025 11:22:11 +0200 Message-ID: <20250929092221.10947-15-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In HCBS, the root control group follows the already existing rules for rt-t= ask scheduling. As such, it does not make use of the deadline servers to accoun= t for runtime, or any other HCBS specific code and features. While the runtime of SCHED_DEADLINE tasks depends on the global bandwidth reserved for rt_tasks, the runtime of SCHED_FIFO/SCHED_RR tasks is limited = by the activation of fair-servers (as the RT_THROTTLING mechanism has been rem= oved in favour of them), thus their maximum bandwidth depends solely on the fair-server settings (which are thightly related to the global bandwdith reserved for rt-tasks) and the amount of SCHED_OTHER workload to run (recall that if no SCHED_OTHER tasks are running, the FIFO/RR tasks may fully utili= ze the CPU). The values of runtime and period in the root cgroup's cpu controller do not affect, by design of HCBS, the fair-server settings and similar (consequent= ly they do not affect the scheduling of FIFO/RR tasks in the root cgroup), buy= they are just used to reserve a portion of the SCHED_DEADLINE bandwidth to the scheduling of rt-cgroups. These values only affect child cgroups, their dea= dline servers and their assigned FIFO/RR tasks. Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 7c7622303e8..94693093b70 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2123,13 +2123,6 @@ static int tg_set_rt_bandwidth(struct task_group *tg, static DEFINE_MUTEX(rt_constraints_mutex); int i, err =3D 0; =20 - /* - * Disallowing the root group RT runtime is BAD, it would disallow the - * kernel creating (and or operating) RT threads. - */ - if (tg =3D=3D &root_task_group && rt_runtime =3D=3D 0) - return -EINVAL; - /* No period doesn't make any sense. */ if (rt_period =3D=3D 0) return -EINVAL; @@ -2216,6 +2209,10 @@ static int sched_rt_global_constraints(void) =20 int sched_rt_can_attach(struct task_group *tg, struct task_struct *tsk) { + /* Allow executing in the root cgroup regardless of allowed bandwidth */ + if (tg =3D=3D &root_task_group) + return 1; + /* Don't accept real-time tasks when there is no way for them to run */ if (rt_group_sched_enabled() && tg->dl_bandwidth.dl_runtime =3D=3D 0) return 0; --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 711882C2374 for ; Mon, 29 Sep 2025 09:22:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137758; cv=none; b=aRrHRZkLtEXWgO2FH0Y8czbz1bDpLywyYosp1LOxIOi5Ce9tXz+i1U2LOifHAvZXDfDGTPUc23fIW9xZV/N4s6w41JsoAToNB8aH0KspUCIUcPbucwi78YzEX0BTRe+k2vmbzZNcSiV+jTRsY4lhzJ0D7KFT7VgWiPalsFcMor0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137758; c=relaxed/simple; bh=wxJSKPgfMQikmHRdXw1g42BneE4R5D1ug59CyIN7q70=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FUjrZOjx45imFwucg5gMViqG0eb3nD0BK3o/0u11CBaEmLsD1HlvuDief92tAZfmEkOciwA9PljDVuX8ndMHzwM78IE59n4pO2RCd6madLrR1NhXhFiy8Qfw4JBGxBMO9MCtEU7GfeJS6/BzuX039XN+mQ30C5Q2GwIhFEgoZng= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=huDRxXDW; arc=none smtp.client-ip=209.85.218.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="huDRxXDW" Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-b2ef8e00becso443211266b.0 for ; Mon, 29 Sep 2025 02:22:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137755; x=1759742555; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pqDwEkWnkj1mNHsS4mS1Mt2o9FVfm/uK0s+31WopnJw=; b=huDRxXDW8PF3Op9kuwTpPDmtf3we9bv2TVIGZkrHl63aImM6lgBopYod+Zq6auzGKo bc+imC7Z8ybivX330LtQXcFWRpbpEzS2ZUf6pIZLttpwoRuCoiZLuocesNbJjFKLsmNo wxX0YNyNSI4wA72fyPBMvk2zHOxGzEqsBxG/KCHRn9g2LYzBNRm4U68dpIVCTED1DH9o MzLb1TyZYsxA7ZbsvwwEAhkYkSlINr8g7fzgpxbWJdDUQL+Y6r8xKq17RUivRj9ouinT QrW8xQlbQfc92uQfx4lAleiQdbiwkKikoImi6y0osaZVpXTnVcRsW0vJ4Q1v+bsZOpBs gbEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137755; x=1759742555; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pqDwEkWnkj1mNHsS4mS1Mt2o9FVfm/uK0s+31WopnJw=; b=OpTToUejcDuT72MyxbjFpyEn1n4IqWVoSItweyLiMFq54fXdbahiCmcSKZkS3HzZdx mUs/gZab2MWssv2srohQiflzY7viX1xeOAc1ktErew3xOj8ykkjoKN7UIr+UIRnuVPr6 9CGWXlDf6yOb9c1k2UmyQmSuOMRkLWP3IrZ956ZIl4yh4GtqWzldX/kyjLejAoPUZAZa Iyl5q8YsFA/ZOS3vTabZHLJTRgK0qHgcL9IB0uT78Q53YCV3zAN54RuYPKQQQzLbg6/a BgKY8M8qm/zBxEnigSwOkDYvyJc6av1RhxQos2VhM74rYFqRAs5zWVcchzzAtWPn0ouK ng7g== X-Gm-Message-State: AOJu0YzfyGnsnwyhoNwBFfTdOjeSaL/IcFhMRhCniDj11y6ksAuhpg7X aRBFgHOpYji8zHJ5Q0MuGCsAkmvz79ktCxfwT7R4uVSEiIngUEzrg8lT X-Gm-Gg: ASbGncs9aQJHxYMRaZ0yMj1NBk1zJCOpdSHXI26Hb66L3zhUFzmFyCCQ2K6M08Ysu+r uIlj5x4zQR7AVCYwNzSC2uCxL6ne0JEOnV2xBDa/3ysNLjVXXwVucfHyH5t2pZ8CXuFg3O5zubW DyXR8BUNfOOI/rDtnOp5zfqpj33VzGjRYdzkCguRy6T2O9v8LGqwI3cMNFeKtS6Sh2yoxwQy6Bm Bb3DSKgaS/DHykWlTuJNCZyNTNt2bB9fWySkrj5kEzbTPh76ZAnpOuPecpTxOV6nNmzt9E0dVXA MgHZ3HnyBrOynJP0orMxmWAhszP9SC7dfBGto3aDGNo8wyzh0iN9iqN11rrxb2F/tlbqfQMlf6d Y8dRPwt92OcixF86GlAk= X-Google-Smtp-Source: AGHT+IH9TF6a9Za+QFK+0NwEXil1aoDVIeWgAtvvlgnFS46GTk2Nulmr0+GwbXErePRC9xssFmqUmA== X-Received: by 2002:a17:907:9689:b0:b30:ed1f:894c with SMTP id a640c23a62f3a-b34be6d7425mr1846527766b.43.1759137754625; Mon, 29 Sep 2025 02:22:34 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:34 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 15/24] sched/rt: Remove old RT_GROUP_SCHED data structures Date: Mon, 29 Sep 2025 11:22:12 +0200 Message-ID: <20250929092221.10947-16-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Completely remove the old RT_GROUP_SCHED's functions and data structures. Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- include/linux/sched.h | 4 ---- kernel/sched/rt.c | 1 - kernel/sched/sched.h | 26 -------------------------- 3 files changed, 31 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 5022ff99c5d..379923eba59 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -624,13 +624,9 @@ struct sched_rt_entity { unsigned short on_rq; unsigned short on_list; =20 - struct sched_rt_entity *back; #ifdef CONFIG_RT_GROUP_SCHED - struct sched_rt_entity *parent; /* rq on which this entity is (to be) queued: */ struct rt_rq *rt_rq; - /* rq "owned" by this entity/group: */ - struct rt_rq *my_q; #endif } __randomize_layout; =20 diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 94693093b70..1cdc699bbb7 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,4 +1,3 @@ -#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 97e1e779df9..fddb171145e 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -310,15 +310,6 @@ struct rt_prio_array { struct list_head queue[MAX_RT_PRIO]; }; =20 -struct rt_bandwidth { - /* nests inside the rq lock: */ - raw_spinlock_t rt_runtime_lock; - ktime_t rt_period; - u64 rt_runtime; - struct hrtimer rt_period_timer; - unsigned int rt_period_active; -}; - struct dl_bandwidth { raw_spinlock_t dl_runtime_lock; u64 dl_runtime; @@ -485,7 +476,6 @@ struct task_group { struct sched_dl_entity **dl_se; struct rt_rq **rt_rq; =20 - struct rt_bandwidth rt_bandwidth; struct dl_bandwidth dl_bandwidth; #endif =20 @@ -801,11 +791,6 @@ struct scx_rq { }; #endif /* CONFIG_SCHED_CLASS_EXT */ =20 -static inline int rt_bandwidth_enabled(void) -{ - return 0; -} - /* RT IPI pull logic requires IRQ_WORK */ #if defined(CONFIG_IRQ_WORK) && defined(CONFIG_SMP) # define HAVE_RT_PUSH_IPI @@ -823,17 +808,6 @@ struct rt_rq { bool overloaded; struct plist_head pushable_tasks; =20 - int rt_queued; - -#ifdef CONFIG_RT_GROUP_SCHED - int rt_throttled; - u64 rt_time; /* consumed RT time, goes up in update_curr_rt */ - u64 rt_runtime; /* allotted RT time, "slice" from rt_bandwidth, RT shar= ing/balancing */ - /* Nests inside the rq lock: */ - raw_spinlock_t rt_runtime_lock; - - unsigned int rt_nr_boosted; -#endif #ifdef CONFIG_CGROUP_SCHED struct task_group *tg; /* this tg has "this" rt_rq on given CPU for runna= ble entities */ #endif --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C5662C3263 for ; Mon, 29 Sep 2025 09:22:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137759; cv=none; b=aQzbELc7S97QOGwpHFldnxaZNM3Bdw4CKV5xqW1hjBKS7ImuYf1MTXjp3IMnH/hR/TRJyOv7OWPL8+YMgVR8wopgenBn33HslbNXsdqpFPi29QKqM6vYtwdsb7E6+0x4VCSHDxwS7+YAgp/jiWH3dWzaU2jT+3um5nEtYZ3d504= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137759; c=relaxed/simple; bh=EnaltF9AtPol8+QofD30sWxNSAUKa7XJAuOqdqgY8qw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=t9fSkG11qd+MytxGUeb28dUnyvqbUtIZeN7S8PpsbH4n8K+IJemcVIHvekwzxukzcU51eIOo4KDcKSLydHLy88wF9H8trVQFHpOri/2HkvFMxroB3VXeVrZhHnHcsaklkvokvQrm2Psfptepb4BDPH6gLT3KPtmlWMh3g9ofrVE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dWWrAnCb; arc=none smtp.client-ip=209.85.208.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dWWrAnCb" Received: by mail-ed1-f47.google.com with SMTP id 4fb4d7f45d1cf-62ecd3c21d3so8133102a12.0 for ; Mon, 29 Sep 2025 02:22:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137755; x=1759742555; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IbDFA1CgYTL7s/j1v3pYiRwiNX7dw5cHY213QrnT9E8=; b=dWWrAnCbdrn83H9NhgENygzTFvTjKLd3bgS1bqKZNRDJqV6COBKWb+2zl+TNic9HCn 85qVlUDDigb0Onf6ZbTCz4GmD3T8h65xOBgZPqOg3T5i5vsYcA0K677OjN0x2W3PoR+F mC3QyrGcA1to9tDt2YYgqQj8L1jiFhM6jU0zTXoU5Mclqj0QD4Cqkg7nWleZx9hmm0MJ hRfV5UeHF7dwReSOgQeguXuQYxWDdqzfQtNMEqmSXXiFoyaVdlZS0nPynqSp91sywdul rkqdpS3B9Vnwcrr+tD8zMaNSUa3iNygITEUym5ovioOnNwadhvvvLvS0J8mku47jv0Pq N4Zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137755; x=1759742555; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IbDFA1CgYTL7s/j1v3pYiRwiNX7dw5cHY213QrnT9E8=; b=bWZ9HUEoTcSEvc4ijrpz9+JtbB7xQdbG1Xb6YHVep8g4v5eo0NNdWzsN1Rs82IdNB9 v+jTjimPqsil81rmK5Byik0SvRep9JmUeJxBmmtA7kk/zAbtvlMi/J/5Rm4FZxJSntw+ LVKB6H3+XFfwUVLDF3Aair/ulUmu0cfiEBhdiEJhPuWrJAh5ZELE29Nq2xvSOvAdE8Ky 1WEVaD9G7FcCwB6TevWVR92zAaVjflJIjgs+KSWoTpsJLd39EldW+7YhmgYsAwEHmYkl wP9RzA3kqglnfJx13tntvLhhHHUaPej2OiBcg3yaSOiQTxZHMcFqQ5l33zZTKte+RJ0+ LNNg== X-Gm-Message-State: AOJu0Yyvjg7gAj8JyY96eAckcTERU5sHgY+1QVlsqnyk1m6kqtD0lLzY M56DU3G1duTnk6U/LLgPvGayKtshZUecGqSzLr7rr6wQEiLnNP0jHJb6 X-Gm-Gg: ASbGncsPB7TEhxqoCN7ViEEd/+AzEvIKCkCN484r3CXyXGQatFWOXPWAEkLFb1wf1sP 2fgBq6UwrMdk6hBjVv5f2o8gbvIWLgZDqcSKAV9ncbyE2TfQqbF4bCYx38Fi9vGMQd25jlf+TiR qgRzOEU62R4rmlZb7DC/0hwZbfYdSxQlboXQ00WtgrcvAxcZpyc1CkNG9b2sEOY52KnuTfa6i2v 7UdsyqtNzmlta01RdNphOxwU2ou6R575VnUYcrDxA2CVHpOdQ0GAYWbIA7LAd+gCdLbeUj/t3zR TJ/TV1A0dAsmRSQNSreYLbHgve9cR5gFDvrOx+7dlNKkD3vAJ5BJ2IyeSxdloTCbJXr7o3oRkUS 2cJ5lsDsCRLq4xX6NUEVmCNlUuEc6hQ== X-Google-Smtp-Source: AGHT+IG1uQtVtnlTg7TWyOjZ1NucFFoxqVtFaN2a1MQ5usOK/APy4aSpI67JS9HCp9tVhlbWACOOjg== X-Received: by 2002:a17:907:9446:b0:b04:590a:a5b5 with SMTP id a640c23a62f3a-b34bb41a8f0mr1741222266b.24.1759137755344; Mon, 29 Sep 2025 02:22:35 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:35 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 16/24] sched/core: Cgroup v2 support Date: Mon, 29 Sep 2025 11:22:13 +0200 Message-ID: <20250929092221.10947-17-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make rt_runtime_us and rt_period_us virtual files accessible also to the cg= roup v2 controller, effectively enabling the RT_GROUP_SCHED mechanism to cgroups= v2. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 1217f714dd2..f6e9a4b22aa 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -10323,6 +10323,18 @@ static struct cftype cpu_files[] =3D { .write =3D cpu_uclamp_max_write, }, #endif /* CONFIG_UCLAMP_TASK_GROUP */ +#ifdef CONFIG_RT_GROUP_SCHED + { + .name =3D "rt_runtime_us", + .read_s64 =3D cpu_rt_runtime_read, + .write_s64 =3D cpu_rt_runtime_write, + }, + { + .name =3D "rt_period_us", + .read_u64 =3D cpu_rt_period_read_uint, + .write_u64 =3D cpu_rt_period_write_uint, + }, +#endif /* CONFIG_RT_GROUP_SCHED */ { } /* terminate */ }; =20 --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D671E2D0601 for ; Mon, 29 Sep 2025 09:22:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137759; cv=none; b=gXT7GfLytv4gopaYl9nIKbS0tbjTNHLrwGLtla5IFPFYjWARe/t+vDSoTJFQ6b8TwU3nj3sbiM4cVADbt+iCBtEuoVZj0cTRdULEceioeYSVpiXrZZUrHQhkbvknXygNG/cGM8n3/JHdZIvF+UuQgdnrORTxKFgh1kEy+wZ3BIk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137759; c=relaxed/simple; bh=AqMfnXGDJbqgHWw3glDpocUaBJ6+Dpq8ZPGwXRgKweg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LrtKhPvi2YrZry2AtJhLA/WBD7A7oJlpf+l7tgs55otr7BLl/pmOBFR3TL+Mr7DnzfGz1OvsfnI+/exmh0e+xoxyPBOjyr6LvvClCzkFcpQP8bW+0mRlQA3h3CtA1i7YJYa5nFZDbybQUpbq1j8p2Rd0KbMTISIokFCxJwyx1Hk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YV5XOHCQ; arc=none smtp.client-ip=209.85.218.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YV5XOHCQ" Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-afcb78ead12so758001966b.1 for ; Mon, 29 Sep 2025 02:22:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137756; x=1759742556; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=emBg48566aQmdaDJEGloD7DA0LLiiyLE4SYZ2DkK16w=; b=YV5XOHCQWytZsEutChQ1+GxVOeRdY+Sq++FgIi9qFqkyXwMh1qcWhfYmTu5qElKG9I 96n3CGC1NZZo+DrAh34pTMPBMp4YKVEe9lufCBOYarDcgOX4G2Kt44hSojhOktE495M3 5vWbADouTRWXwJ3fnG86cjXta5fdRpWEkVjq4VTlsH/wS6MbKHNbECFFaAt+YFVE4x/f GhT3xBEQmyXKttjOUuyBALcMOjcGATct3kvfNIpNVNp0CP7deRIcZ4rkosCpPRjP6UfN BLR9u/t5bhqbBSNR0uIYoaXmS8QUu3DhtEwwOS/kKE0EULLIO+GOh7muD7FQ3Q+dH1hy 47pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137756; x=1759742556; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=emBg48566aQmdaDJEGloD7DA0LLiiyLE4SYZ2DkK16w=; b=VjaSEjqPtnImoJhpegSKZ15UXEFCFy/4AeZkZ7CqIrTfYjGPTO0HzlGT774GOZIp+x HbEO+5l2nuIsXks6m7z2OkL/gdRoZzcjjQQXLEQMZwZNdKQH2t9WqhyxJiy8E+P428Rn 4XQ8xWGhkl0LFs/6eTGZuKPLeYw5gyqBBbqz4VYFHCWZt3Bwc3g2+LyCi/B4uBYYXyaG TwZoAZDufmAw6QLvbVSofeLFQSaYWDzHRKG4h4Wb6C9Mq7kVURqS0KDMEvhmv0YGSRbI 2nOkUSSvymCJ7KZiCmZCUkD9pZMkPqdTZYvzP+F1f4tJu1Gpa6U5nfNaH5U8wyX+KVWm EtGQ== X-Gm-Message-State: AOJu0YzPD7Tw0mHD2w7ksnbTvMoRfvLLVr321QLiHDpNvoHczx9tiUak ZL7I64oE5WCfpJ35XkyhdIJ5vxtTN+dXzjV5GyJ3j3vaUYlEg4iGO8kS X-Gm-Gg: ASbGncsAfH1oe6Iq8Px2m8VKYhHCUfsKYmJ7r/A+XebyQ19OTbOAFUZ0ZmlDDdd8vuq bwQNr6sIF+VYlxYEUEAvUn7QE8wvkdAtNpQ2TNFgx7U7RTyfpJG9EsyS9aLEv6VqspvXEiZ8dAw WNMQW6/8EQn0V5ZMucZDZNCstjf681aYSiMQj8oye/RUg1DejzjMQ2cLOJeyCuKY8DXip1sq8QN XPgmJcf5OYSmVI6novRHt8vuXGkGh8M+az8jR1YlDRft6s5HICi7fTrvxBjsEiOpwSE/8TiGnLU 3v8LoN3evRz0mVCMqYTZppg9MgEMY+gaXGxygU47l2GkLp7uacMNhGhIfnbJQLkbe8eAlqMXw4H bMQXQK9NXujbU8Jhxbv8= X-Google-Smtp-Source: AGHT+IESrnCuYjxXCfrzeWo9ElJeqYwTyvvLe3b45Jp4EisnHBLUFA2Jl5eU7/ETkdStyckFGEcYzg== X-Received: by 2002:a17:907:9713:b0:b40:cfe9:ed2c with SMTP id a640c23a62f3a-b40cfe9f8e1mr55983366b.64.1759137756096; Mon, 29 Sep 2025 02:22:36 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:35 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 17/24] sched/rt: Remove support for cgroups-v1 Date: Mon, 29 Sep 2025 11:22:14 +0200 Message-ID: <20250929092221.10947-18-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Disable control files for cgroups-v1, and allow only cgroups-v2. This should simplify maintaining the code, also because cgroups-v1 are deprecated. Set the default rt-cgroups runtime to zero, otherwise a cgroup-v1 kernel wi= ll not be able to start SCHED_DEADLINE tasks. The bandwidth for rt-cgroups must then be manually assigned after the kernel boots. Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index f6e9a4b22aa..6f516cdc7bb 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8723,7 +8723,7 @@ void __init sched_init(void) =20 #ifdef CONFIG_RT_GROUP_SCHED init_dl_bandwidth(&root_task_group.dl_bandwidth, - global_rt_period(), global_rt_runtime()); + global_rt_period(), 0); #endif /* CONFIG_RT_GROUP_SCHED */ =20 #ifdef CONFIG_CGROUP_SCHED @@ -10062,20 +10062,6 @@ static struct cftype cpu_legacy_files[] =3D { }; =20 #ifdef CONFIG_RT_GROUP_SCHED -static struct cftype rt_group_files[] =3D { - { - .name =3D "rt_runtime_us", - .read_s64 =3D cpu_rt_runtime_read, - .write_s64 =3D cpu_rt_runtime_write, - }, - { - .name =3D "rt_period_us", - .read_u64 =3D cpu_rt_period_read_uint, - .write_u64 =3D cpu_rt_period_write_uint, - }, - { } /* Terminate */ -}; - # ifdef CONFIG_RT_GROUP_SCHED_DEFAULT_DISABLED DEFINE_STATIC_KEY_FALSE(rt_group_sched); # else @@ -10101,10 +10087,6 @@ __setup("rt_group_sched=3D", setup_rt_group_sched); =20 static int __init cpu_rt_group_init(void) { - if (!rt_group_sched_enabled()) - return 0; - - WARN_ON(cgroup_add_legacy_cftypes(&cpu_cgrp_subsys, rt_group_files)); return 0; } subsys_initcall(cpu_rt_group_init); --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCBEA28750B for ; Mon, 29 Sep 2025 09:22:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137761; cv=none; b=Qs3saml5vSmIPb+FBlqbQlBfcTVp2b5UYrdx0YKId+We8fL5Xy3lejEW1iaTZAa8wD35o+k5x7WhFFGj3pVSHfpt1LHUdgQV/FQMNUDCITsAfjMLOw+BuvoZH30h2zGzvh6OfJOOIdsdFUkooN0/nH1JY9GWvHJyNP9/B+l3OWc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137761; c=relaxed/simple; bh=SX06KYe3UfsMlxi07fSgKyO3jk3XpncC2n0/cGMfWZQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fzvhzPJ0h3uA0VQa9AfQbtMuZUlZsh6HFSaUCskJfG3dTDVjFdS1hJsxMTwCp+gZhxxR0LdVSJdiYntP6Q25Ioc3RabCiFCAAF6GkNWqemUnt7uumjGPcegfKnhil+RFZvaVldl3D/2lmi6GMcfA8qi4c8X3vhM2KapnFCBJeL4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EjReApQJ; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EjReApQJ" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-b3b3a6f4dd4so290499366b.0 for ; Mon, 29 Sep 2025 02:22:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137757; x=1759742557; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GCB7d9ViC9MONweDSs0QerFusX73MoEMEy3T8nwoQ+g=; b=EjReApQJettja88f2/BNpa/4+T6MKX5c3VGgzNC9frhfmImCuGukZzDGoEyg7vrz5U WQXiWmdZowfMAKhOx1f/ncPQmSvE3moahDs57+1zqcTRRP1GQZJtO56mDTk472yk8Dir CyVyJmG2WCskA1bdEZz7Px8b9dxYMyhI8Hu7rPnJS8Q6YCgjBSjbpSKIQZYh09JQTlpk baRvoRh6o3FWeUGtoALdUcUctrlfr5iLCQpurBux7w0Im8/IOrjJxouCCiCCyCJBplfF 0Qs8LjwohPu/aKjoCQ0f36eXsjLHxtz3ZAx2PGaiqC6+8tVmvQqKU1NoWO1oOfX0qEDI W3pA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137757; x=1759742557; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GCB7d9ViC9MONweDSs0QerFusX73MoEMEy3T8nwoQ+g=; b=RK6MKmCfbv3Q3TVOIeXIA5zIgW7dvn15QhDtx67ek/IDl0kt/1CiH/udsDz/gYIwXV MP+BnlFzzS2uZDOZAUPD9m7ikOvW5rWA0uKV7ab76B1FYeBL5284dgPdQXe/b2Gf0rAL W7N8cJte4c1X6OZuEGAd1HbyVysePfWYbwrzB/v4LwueMqkUu0MgLhyRUoyM/Ojc+5Z+ Rqx9TikZh7cFyGx0MhLavKzLVOg5DblRXMPq+SHNWc3Xz6asOgL9m2Fuqx7iqgeIa4lb yHhHv7TvhnVRkrvjgKgrZtKuak4evxjbMackdxkqRzH6S/fRGWJH2/WQeY+oSHkEkHjj r/Ow== X-Gm-Message-State: AOJu0YycoMN1qV1RpLKjHwpvyIRcRDp7DXKH2X8tPiBhp3nmqrGRmVPK AJGIQ85MkcqBaqwrUQ/RXuI1dFPF9fuYe6XGmzYE/7EfLSVf7Rblv8tX X-Gm-Gg: ASbGncts9WY4SQ+L0mbgiN6LxQhsbA22BrSnh5SyfQVmRP/cBvBxHJB+aPtKYS+bXqN oeaLzqJyY1jfu64z3hfEeggXMYaXuDj+BAEAy2Ustq7R3PEIFpF7u5UwFGPYq4aCrtIXXCqBy7w nxaVH/ZrEAugT+YtFWAWLkXcW8CJhRSvDEAfhX4vrjUbEdqJIr9+kDUi85vNhHb9ZVPoAX9Vg7J e2oDBC/iUsMe7JOrDo+Me3Gksz8HO+uIyfDO4g1pcSWTZNu6NHiUm/vxZcBGaP7kg7DDwfklEBM 02K1kaBurkQ2hoQhP8jpCAMPCJVqwLuxSiNa38xOIlKfAi0z9rYsSN7Hs3nvgxcM71YGr+gKpBX +qAy6FfBjoElEuJ+Zgb8= X-Google-Smtp-Source: AGHT+IGMktd+ej9fkjLPpg2JFh9TI+04sASGgTJogJZqKRxWN3xpQPwYtvzScBqBiPPBXwaFY/1F2A== X-Received: by 2002:a17:907:86ab:b0:b04:9854:981f with SMTP id a640c23a62f3a-b34bc96d5femr1786364866b.43.1759137756875; Mon, 29 Sep 2025 02:22:36 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:36 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 18/24] sched/deadline: Allow deeper hierarchies of RT cgroups Date: Mon, 29 Sep 2025 11:22:15 +0200 Message-ID: <20250929092221.10947-19-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Allow creation of cgroup hierachies with depth greater than two. Add check to prevent attaching tasks to a child cgroup of an active cgroup = (i.e. with a running FIFO/RR task). Add check to prevent attaching tasks to cgroups which have children with non-zero runtime. Update rt-cgroups allocated bandwidth accounting for nested cgroup hierachi= es. Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/core.c | 6 ----- kernel/sched/deadline.c | 51 +++++++++++++++++++++++++++++++++++++---- kernel/sched/rt.c | 16 ++++++++++--- kernel/sched/sched.h | 3 ++- 4 files changed, 62 insertions(+), 14 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 6f516cdc7bb..d1d7215c4a2 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9281,12 +9281,6 @@ cpu_cgroup_css_alloc(struct cgroup_subsys_state *par= ent_css) return &root_task_group.css; } =20 - /* Do not allow cpu_cgroup hierachies with depth greater than 2. */ -#ifdef CONFIG_RT_GROUP_SCHED - if (parent !=3D &root_task_group) - return ERR_PTR(-EINVAL); -#endif - tg =3D sched_create_group(parent); if (IS_ERR(tg)) return ERR_PTR(-ENOMEM); diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 5d93b3ca030..abe11985c41 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -388,11 +388,42 @@ int dl_check_tg(unsigned long total) return 1; } =20 -void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 rt_peri= od) +bool is_active_sched_group(struct task_group *tg) { + struct task_group *child; + bool is_active =3D 1; + + // if there are no children, this is a leaf group, thus it is active + list_for_each_entry_rcu(child, &tg->children, siblings) { + if (child->dl_bandwidth.dl_runtime > 0) { + is_active =3D 0; + } + } + return is_active; +} + +static inline bool sched_group_has_active_siblings(struct task_group *tg) +{ + struct task_group *child; + bool has_active_siblings =3D 0; + + // if there are no children, this is a leaf group, thus it is active + list_for_each_entry_rcu(child, &tg->parent->children, siblings) { + if (child !=3D tg && child->dl_bandwidth.dl_runtime > 0) { + has_active_siblings =3D 1; + } + } + return has_active_siblings; +} + +void dl_init_tg(struct task_group *tg, int cpu, u64 rt_runtime, u64 rt_per= iod) +{ + struct sched_dl_entity *dl_se =3D tg->dl_se[cpu]; struct rq *rq =3D container_of(dl_se->dl_rq, struct rq, dl); - int is_active; - u64 new_bw; + int is_active, is_active_group; + u64 old_runtime, new_bw; + + is_active_group =3D is_active_sched_group(tg); =20 raw_spin_rq_lock_irq(rq); is_active =3D dl_se->my_q->rt.rt_nr_running > 0; @@ -400,8 +431,10 @@ void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_= runtime, u64 rt_period) update_rq_clock(rq); dl_server_stop(dl_se); =20 + old_runtime =3D dl_se->dl_runtime; new_bw =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); - dl_rq_change_utilization(rq, dl_se, new_bw); + if (is_active_group) + dl_rq_change_utilization(rq, dl_se, new_bw); =20 dl_se->dl_runtime =3D rt_runtime; dl_se->dl_deadline =3D rt_period; @@ -413,6 +446,16 @@ void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_= runtime, u64 rt_period) dl_se->dl_bw =3D new_bw; dl_se->dl_density =3D new_bw; =20 + // add/remove the parent's bw + if (tg->parent && tg->parent !=3D &root_task_group) + { + if (rt_runtime =3D=3D 0 && old_runtime !=3D 0 && !sched_group_has_active= _siblings(tg)) { + __add_rq_bw(tg->parent->dl_se[cpu]->dl_bw, dl_se->dl_rq); + } else if (rt_runtime !=3D 0 && old_runtime =3D=3D 0 && !sched_group_has= _active_siblings(tg)) { + __sub_rq_bw(tg->parent->dl_se[cpu]->dl_bw, dl_se->dl_rq); + } + } + if (is_active) dl_server_start(dl_se); =20 diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 1cdc699bbb7..17ad91261cb 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -114,7 +114,8 @@ void free_rt_sched_group(struct task_group *tg) * Fix this issue by changing the group runtime * to 0 immediately before freeing it. */ - dl_init_tg(tg->dl_se[i], 0, tg->dl_se[i]->dl_period); + if (tg->dl_se[i]->dl_runtime) + dl_init_tg(tg, i, 0, tg->dl_se[i]->dl_period); =20 raw_spin_rq_lock_irqsave(cpu_rq(i), flags); BUG_ON(tg->rt_rq[i]->rt_nr_running); @@ -2122,6 +2123,14 @@ static int tg_set_rt_bandwidth(struct task_group *tg, static DEFINE_MUTEX(rt_constraints_mutex); int i, err =3D 0; =20 + /* + * Do not allow to set a RT runtime > 0 if the parent has RT tasks + * (and is not the root group) + */ + if (rt_runtime && (tg !=3D &root_task_group) && (tg->parent !=3D &root_ta= sk_group) && tg_has_rt_tasks(tg->parent)) { + return -EINVAL; + } + /* No period doesn't make any sense. */ if (rt_period =3D=3D 0) return -EINVAL; @@ -2145,7 +2154,7 @@ static int tg_set_rt_bandwidth(struct task_group *tg, return 0; =20 for_each_possible_cpu(i) { - dl_init_tg(tg->dl_se[i], rt_runtime, rt_period); + dl_init_tg(tg, i, rt_runtime, rt_period); } =20 return 0; @@ -2216,7 +2225,8 @@ int sched_rt_can_attach(struct task_group *tg, struct= task_struct *tsk) if (rt_group_sched_enabled() && tg->dl_bandwidth.dl_runtime =3D=3D 0) return 0; =20 - return 1; + /* tasks can be attached only if the taskgroup has no active children. */ + return (int)is_active_sched_group(tg); } =20 #else /* !CONFIG_RT_GROUP_SCHED: */ diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index fddb171145e..55631d93e02 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -386,7 +386,8 @@ extern void dl_server_init(struct sched_dl_entity *dl_s= e, struct dl_rq *dl_rq, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); extern int dl_check_tg(unsigned long total); -extern void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 = rt_period); +extern void dl_init_tg(struct task_group *tg, int cpu, u64 rt_runtime, u64= rt_period); +extern bool is_active_sched_group(struct task_group *tg); =20 extern void dl_server_update_idle_time(struct rq *rq, struct task_struct *p); --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A348B2D12EB for ; Mon, 29 Sep 2025 09:22:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137762; cv=none; b=QO4V2ksAfBuGLyiSHwFETwPF+QNBqa6f3pHKIELX6dX4gCLbarGUojmokUusSr8AvenIru7DN2t62oY9Zgr2SytZg0tpxxwe1uJ/1nD+HHjWBW6xTMihe+3JemI8jWgR4y5TwwL3XvxE5ZMA7EPlpfu34jiARXBPwbtu+7FGb6o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137762; c=relaxed/simple; bh=fxG+bTxB07LCi9Ts9lL1Yl3nrlWVjIIv8vHQOIlnXQQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KBPzfaDzo2EIWnzAKDoySAQi6OWaa7TWEnY56SAJLUjGUybtsgmVoesM5yCC4FsK16CATHEAF2WtBmGq1WSKZNDiIZCviKAFPJyaLHiYxT6SSuinpJ+EsipxFlmQTmnF4O9VU4zn7IgWWpi0aIwwjTZT7I1EC1g+zBpe8mOSyJ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lsjexBXM; arc=none smtp.client-ip=209.85.218.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lsjexBXM" Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-b3ee18913c0so133110666b.3 for ; Mon, 29 Sep 2025 02:22:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137758; x=1759742558; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sruGDAIuG3pHlYCEj+2RbEEqGhhXQxwPKteoyiFJfm0=; b=lsjexBXMu4H+YBug8YlV7mwJ0j3dvEUBuvgek5UrbJAISHOQyzJY7JE89wPy+oTm2p cuf5flMW2cZ5N+JIIlkSbnL/j4eOenR+9+OFPZ4QtoWJnut+TXhekKBUoFRLvK8g6DX6 Fo7kvywj1chZeSMxXu5jlpWb5N1QF0Vqzm+nVBmyD35JlmHwBPKccNLytDdKWms2gDlL NhEamDtsSVSofyBJgrlY+WdCciXpZuedo39cqWmrk2ETo4T8Q7hzxSuyZOnbdwgnCuvC sEFhG/1YBE2BpBcxyGF/02l1kmbLUyQDgfKj4FpR6O5whRllbnFGXTOqAScab2vj/NZW zWkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137758; x=1759742558; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sruGDAIuG3pHlYCEj+2RbEEqGhhXQxwPKteoyiFJfm0=; b=a+3iTKJnm0QTk3D4dW/0lVRNX+RTBo653heiTdQt7+bgOylBsIl5NK85xNAP76RIRF Vm0g608gJ+km0qKXBT4+keNm8EoVawDkAu0+CRuGJu5gtE9vZ/LhmwLee9dVVUhWXDCS PFT/5pyf6ERQjw3WCz4BxSl70ahFlbjdzgk/8/AKht2O6LCY95B2H602EnJg5aErqQ/Y 4035o2o4T78CTTEHSdJmaw6VZ9w104ByVshImBDl+hxBc5WcqrGsSJFNQ/V4wggetqxr UOzfLb+C6mxh98hLuTp+y9Usc7bHhdKgbb7ouTv8X5cO+nuhndOINko9onkJNSo6AmTJ YFpw== X-Gm-Message-State: AOJu0Yx2TIwe/Rvt41cZdsHYIYSrzP9vD9laDvcdwXujRe+Ba0NZF2Hh oQ4X+YZIyDotpaQcQfi/Yo30I47rPEyjX4puT3wMmKNbTQWi1rTA3ss+ X-Gm-Gg: ASbGncsnGVMITqX70OMeDpqB2LVqihp2H700AMMbAmDzA8xcBjFfOCQB96MBsVlXP9u s/lOzdcLV2pREITjn7+6Y6ggxXLSQLGdgfEBO6Eum8ynZnYco8BlUePGQqIrEjhDD0SNU8TFWlM 2Ojk/hnAPRzMq/eLVC9MWxy7QF/tPJdPHYx17TKjal9oAcanOwREepXoZ5XJLXvEm78QDBhlHtx ADgrU+e9Vm+MSSXfFMDLL/ilDPdt91UCFzb9qtnGJznKfNcQZYXNaPaCXTz1QNUp3+ruzpDdYZB kj6WqH62JyMKJZSF2fQrT92IVx+rZgvAzTvNRM2Qh/vP+jyzPIA2anonOBRXrh3qvZ+5MZgSDle A7V3fm+JJec65L3Askr4Whzp1xg0kcg== X-Google-Smtp-Source: AGHT+IF7xM4d9a5Oa2BSywkR/hoC2YErIuLnwL9Uxt8xadFiyU0NjsTWrgCgLmXYsxeiJSgifomyBg== X-Received: by 2002:a17:907:3d91:b0:b0e:d477:4961 with SMTP id a640c23a62f3a-b34b8d96245mr1725701166b.23.1759137757691; Mon, 29 Sep 2025 02:22:37 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:37 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 19/24] sched/rt: Add rt-cgroup migration Date: Mon, 29 Sep 2025 11:22:16 +0200 Message-ID: <20250929092221.10947-20-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni When the runtime is exhausted in a RT CGroup, the scheduler checks for another non-throttled runqueue and, if available, migrates the tasks. The bandwidth (runtime/period) chosen for a certain cgroup is replicated on every core of the system, therefore, in an SMP system with M cores, the total available bandwidth is the given runtime/period multiplied by M. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/rt.c | 495 +++++++++++++++++++++++++++++++++++++++++++ kernel/sched/sched.h | 10 + 2 files changed, 505 insertions(+) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 17ad91261cb..1fbcac14fc9 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,3 +1,4 @@ +#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR @@ -84,6 +85,8 @@ void init_rt_rq(struct rt_rq *rt_rq) plist_head_init(&rt_rq->pushable_tasks); } =20 +static void group_pull_rt_task(struct rt_rq *this_rt_rq); + #ifdef CONFIG_RT_GROUP_SCHED =20 void unregister_rt_sched_group(struct task_group *tg) @@ -136,6 +139,7 @@ void init_tg_rt_entry(struct task_group *tg, struct rq = *served_rq, struct sched_dl_entity *dl_se, int cpu, struct sched_dl_entity *parent) { + served_rq->cpu =3D cpu; served_rq->rt.highest_prio.curr =3D MAX_RT_PRIO-1; served_rq->rt.rq =3D cpu_rq(cpu); served_rq->rt.tg =3D tg; @@ -299,6 +303,53 @@ static inline void rt_queue_pull_task(struct rt_rq * r= t_rq) queue_balance_callback(rq, &per_cpu(rt_pull_head, rq->cpu), pull_rt_task); } =20 +#ifdef CONFIG_RT_GROUP_SCHED +static DEFINE_PER_CPU(struct balance_callback, rt_group_push_head); +static DEFINE_PER_CPU(struct balance_callback, rt_group_pull_head); +static void group_push_rt_tasks_callback(struct rq *); +static void group_pull_rt_task_callback(struct rq *); + +static void rt_queue_push_from_group(struct rt_rq *rt_rq) +{ + struct rq *rq =3D container_of(rt_rq, struct rq, rt); + struct rq *global_rq =3D cpu_rq(rq->cpu); + + BUG_ON(rt_rq =3D=3D NULL); + BUG_ON(rt_rq->rq !=3D global_rq); + + if (global_rq->rq_to_push_from) + return; + + if (!has_pushable_tasks(rt_rq)) + return; + + global_rq->rq_to_push_from =3D rq; + queue_balance_callback(global_rq, &per_cpu(rt_group_push_head, global_rq-= >cpu), + group_push_rt_tasks_callback); +} + +static void rt_queue_pull_to_group(struct rt_rq *rt_rq) +{ + struct rq *rq =3D container_of(rt_rq, struct rq, rt); + struct rq *global_rq =3D cpu_rq(rq->cpu); + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + BUG_ON(rt_rq =3D=3D NULL); + BUG_ON(!is_dl_group(rt_rq)); + BUG_ON(rt_rq->rq !=3D global_rq); + + if (dl_se->dl_throttled || global_rq->rq_to_pull_to) + return; + + global_rq->rq_to_pull_to =3D rq; + queue_balance_callback(global_rq, &per_cpu(rt_group_pull_head, global_rq-= >cpu), + group_pull_rt_task_callback); +} +#else +static inline void rt_queue_push_from_group(struct rt_rq *rt_rq) {}; +static inline void rt_queue_pull_to_group(struct rt_rq *rt_rq) {}; +#endif + static void enqueue_pushable_task(struct rt_rq *rt_rq, struct task_struct = *p) { plist_del(&p->pushable_tasks, &rt_rq->pushable_tasks); @@ -1303,6 +1354,8 @@ static struct rq *find_lock_lowest_rq(struct task_str= uct *task, struct rq *rq) */ static int push_rt_task(struct rq *rq, bool pull) { + BUG_ON(is_dl_group(&rq->rt)); + struct task_struct *next_task; struct rq *lowest_rq; int ret =3D 0; @@ -1599,6 +1652,8 @@ void rto_push_irq_work_func(struct irq_work *work) =20 static void pull_rt_task(struct rq *this_rq) { + BUG_ON(is_dl_group(&this_rq->rt)); + int this_cpu =3D this_rq->cpu, cpu; bool resched =3D false; struct task_struct *p, *push_task; @@ -1708,6 +1763,446 @@ static void pull_rt_task(struct rq *this_rq) resched_curr(this_rq); } =20 +#ifdef CONFIG_RT_GROUP_SCHED +/* + * Find the lowest priority runqueue among the runqueues of the same + * task group. Unlike find_lowest_rt(), this does not mean that the + * lowest priority cpu is running tasks from this runqueue. + */ +static int group_find_lowest_rt_rq(struct task_struct *task, struct rt_rq*= task_rt_rq) +{ + struct sched_domain *sd; + struct cpumask mask, *lowest_mask =3D &mask; + struct sched_dl_entity *dl_se; + struct rt_rq *rt_rq; + int prio, lowest_prio; + int cpu, this_cpu =3D smp_processor_id(); + + BUG_ON(task->sched_task_group !=3D task_rt_rq->tg); + + if (task->nr_cpus_allowed =3D=3D 1) + return -1; /* No other targets possible */ + + lowest_prio =3D task->prio - 1; + cpumask_clear(lowest_mask); + for_each_cpu_and(cpu, cpu_online_mask, task->cpus_ptr) { + dl_se =3D task_rt_rq->tg->dl_se[cpu]; + rt_rq =3D &dl_se->my_q->rt; + prio =3D rt_rq->highest_prio.curr; + + /* + * If we're on asym system ensure we consider the different capacities + * of the CPUs when searching for the lowest_mask. + */ + if (dl_se->dl_throttled || !rt_task_fits_capacity(task, cpu)) + continue; + + if (prio >=3D lowest_prio) { + if (prio > lowest_prio) { + cpumask_clear(lowest_mask); + lowest_prio =3D prio; + } + + cpumask_set_cpu(cpu, lowest_mask); + } + } + + if (cpumask_empty(lowest_mask)) + return -1; + + /* + * At this point we have built a mask of CPUs representing the + * lowest priority tasks in the system. Now we want to elect + * the best one based on our affinity and topology. + * + * We prioritize the last CPU that the task executed on since + * it is most likely cache-hot in that location. + */ + cpu =3D task_cpu(task); + if (cpumask_test_cpu(cpu, lowest_mask)) + return cpu; + + /* + * Otherwise, we consult the sched_domains span maps to figure + * out which CPU is logically closest to our hot cache data. + */ + if (!cpumask_test_cpu(this_cpu, lowest_mask)) + this_cpu =3D -1; /* Skip this_cpu opt if not among lowest */ + + rcu_read_lock(); + for_each_domain(cpu, sd) { + if (sd->flags & SD_WAKE_AFFINE) { + int best_cpu; + + /* + * "this_cpu" is cheaper to preempt than a + * remote processor. + */ + if (this_cpu !=3D -1 && + cpumask_test_cpu(this_cpu, sched_domain_span(sd))) { + rcu_read_unlock(); + return this_cpu; + } + + best_cpu =3D cpumask_any_and_distribute(lowest_mask, + sched_domain_span(sd)); + if (best_cpu < nr_cpu_ids) { + rcu_read_unlock(); + return best_cpu; + } + } + } + rcu_read_unlock(); + + /* + * And finally, if there were no matches within the domains + * just give the caller *something* to work with from the compatible + * locations. + */ + if (this_cpu !=3D -1) + return this_cpu; + + cpu =3D cpumask_any_distribute(lowest_mask); + if (cpu < nr_cpu_ids) + return cpu; + + return -1; +} + +/* + * Find and lock the lowest priority runqueue among the runqueues + * of the same task group. Unlike find_lock_lowest_rt(), this does not + * mean that the lowest priority cpu is running tasks from this runqueue. + */ +static struct rt_rq* group_find_lock_lowest_rt_rq(struct task_struct *task= , struct rt_rq *rt_rq) +{ + struct rq *rq =3D rq_of_rt_rq(rt_rq); + struct rq *lowest_rq; + struct rt_rq *lowest_rt_rq =3D NULL; + struct sched_dl_entity *lowest_dl_se; + int tries, cpu; + + BUG_ON(task->sched_task_group !=3D rt_rq->tg); + + for (tries =3D 0; tries < RT_MAX_TRIES; tries++) { + cpu =3D group_find_lowest_rt_rq(task, rt_rq); + + if ((cpu =3D=3D -1) || (cpu =3D=3D rq->cpu)) + break; + + lowest_dl_se =3D rt_rq->tg->dl_se[cpu]; + lowest_rt_rq =3D &lowest_dl_se->my_q->rt; + lowest_rq =3D cpu_rq(cpu); + + if (lowest_rt_rq->highest_prio.curr <=3D task->prio) { + /* + * Target rq has tasks of equal or higher priority, + * retrying does not release any lock and is unlikely + * to yield a different result. + */ + lowest_rt_rq =3D NULL; + break; + } + + /* if the prio of this runqueue changed, try again */ + if (double_lock_balance(rq, lowest_rq)) { + /* + * We had to unlock the run queue. In + * the mean time, task could have + * migrated already or had its affinity changed. + * Also make sure that it wasn't scheduled on its rq. + * It is possible the task was scheduled, set + * "migrate_disabled" and then got preempted, so we must + * check the task migration disable flag here too. + */ + if (unlikely(is_migration_disabled(task) || + lowest_dl_se->dl_throttled || + !cpumask_test_cpu(lowest_rq->cpu, &task->cpus_mask) || + task !=3D pick_next_pushable_task(rt_rq))) { + + double_unlock_balance(rq, lowest_rq); + lowest_rt_rq =3D NULL; + break; + } + } + + /* If this rq is still suitable use it. */ + if (lowest_rt_rq->highest_prio.curr > task->prio) + break; + + /* try again */ + double_unlock_balance(rq, lowest_rq); + lowest_rt_rq =3D NULL; + } + + return lowest_rt_rq; +} + +static int group_push_rt_task(struct rt_rq *rt_rq, bool pull) +{ + BUG_ON(!is_dl_group(rt_rq)); + + struct rq *rq =3D rq_of_rt_rq(rt_rq); + struct task_struct *next_task; + struct rq *lowest_rq; + struct rt_rq *lowest_rt_rq; + int ret =3D 0; + + if (!rt_rq->overloaded) + return 0; + + next_task =3D pick_next_pushable_task(rt_rq); + if (!next_task) + return 0; + +retry: + if (is_migration_disabled(next_task)) { + struct task_struct *push_task =3D NULL; + int cpu; + + if (!pull || rq->push_busy) + return 0; + + /* + * If the current task does not belong to the same task group + * we cannot push it away. + */ + if (rq->curr->sched_task_group !=3D rt_rq->tg) + return 0; + + /* + * Invoking group_find_lowest_rt_rq() on anything but an RT task doesn't + * make sense. Per the above priority check, curr has to + * be of higher priority than next_task, so no need to + * reschedule when bailing out. + * + * Note that the stoppers are masqueraded as SCHED_FIFO + * (cf. sched_set_stop_task()), so we can't rely on rt_task(). + */ + if (rq->curr->sched_class !=3D &rt_sched_class) + return 0; + + cpu =3D group_find_lowest_rt_rq(rq->curr, rt_rq); + if (cpu =3D=3D -1 || cpu =3D=3D rq->cpu) + return 0; + + /* + * Given we found a CPU with lower priority than @next_task, + * therefore it should be running. However we cannot migrate it + * to this other CPU, instead attempt to push the current + * running task on this CPU away. + */ + push_task =3D get_push_task(rq); + if (push_task) { + preempt_disable(); + raw_spin_rq_unlock(rq); + stop_one_cpu_nowait(rq->cpu, push_cpu_stop, + push_task, &rq->push_work); + preempt_enable(); + raw_spin_rq_lock(rq); + } + + return 0; + } + + if (WARN_ON(next_task =3D=3D rq->curr)) + return 0; + + /* We might release rq lock */ + get_task_struct(next_task); + + /* group_find_lock_lowest_rq locks the rq if found */ + lowest_rt_rq =3D group_find_lock_lowest_rt_rq(next_task, rt_rq); + if (!lowest_rt_rq) { + struct task_struct *task; + /* + * group_find_lock_lowest_rt_rq releases rq->lock + * so it is possible that next_task has migrated. + * + * We need to make sure that the task is still on the same + * run-queue and is also still the next task eligible for + * pushing. + */ + task =3D pick_next_pushable_task(rt_rq); + if (task =3D=3D next_task) { + /* + * The task hasn't migrated, and is still the next + * eligible task, but we failed to find a run-queue + * to push it to. Do not retry in this case, since + * other CPUs will pull from us when ready. + */ + goto out; + } + + if (!task) + /* No more tasks, just exit */ + goto out; + + /* + * Something has shifted, try again. + */ + put_task_struct(next_task); + next_task =3D task; + goto retry; + } + + lowest_rq =3D rq_of_rt_rq(lowest_rt_rq); + + move_queued_task_locked(rq, lowest_rq, next_task); + resched_curr(lowest_rq); + ret =3D 1; + + double_unlock_balance(rq, lowest_rq); +out: + put_task_struct(next_task); + + return ret; +} + +static void group_pull_rt_task(struct rt_rq *this_rt_rq) +{ + BUG_ON(!is_dl_group(this_rt_rq)); + + struct rq *this_rq =3D rq_of_rt_rq(this_rt_rq); + int this_cpu =3D this_rq->cpu, cpu; + bool resched =3D false; + struct task_struct *p, *push_task =3D NULL; + struct rt_rq *src_rt_rq; + struct rq *src_rq; + struct sched_dl_entity *src_dl_se; + + for_each_online_cpu(cpu) { + if (this_cpu =3D=3D cpu) + continue; + + src_dl_se =3D this_rt_rq->tg->dl_se[cpu]; + src_rt_rq =3D &src_dl_se->my_q->rt; + + if (src_rt_rq->rt_nr_running <=3D 1 && !src_dl_se->dl_throttled) + continue; + + src_rq =3D rq_of_rt_rq(src_rt_rq); + + /* + * Don't bother taking the src_rq->lock if the next highest + * task is known to be lower-priority than our current task. + * This may look racy, but if this value is about to go + * logically higher, the src_rq will push this task away. + * And if its going logically lower, we do not care + */ + if (src_rt_rq->highest_prio.next >=3D + this_rt_rq->highest_prio.curr) + continue; + + /* + * We can potentially drop this_rq's lock in + * double_lock_balance, and another CPU could + * alter this_rq + */ + push_task =3D NULL; + double_lock_balance(this_rq, src_rq); + + /* + * We can pull only a task, which is pushable + * on its rq, and no others. + */ + p =3D pick_highest_pushable_task(src_rt_rq, this_cpu); + + /* + * Do we have an RT task that preempts + * the to-be-scheduled task? + */ + if (p && (p->prio < this_rt_rq->highest_prio.curr)) { + WARN_ON(p =3D=3D src_rq->curr); + WARN_ON(!task_on_rq_queued(p)); + + /* + * There's a chance that p is higher in priority + * than what's currently running on its CPU. + * This is just that p is waking up and hasn't + * had a chance to schedule. We only pull + * p if it is lower in priority than the + * current task on the run queue + */ + if (src_rq->curr->sched_task_group =3D=3D this_rt_rq->tg && + p->prio < src_rq->curr->prio) + goto skip; + + if (is_migration_disabled(p)) { + /* + * If the current task does not belong to the same task group + * we cannot push it away. + */ + if (src_rq->curr->sched_task_group !=3D this_rt_rq->tg) + goto skip; + + push_task =3D get_push_task(src_rq); + } else { + move_queued_task_locked(src_rq, this_rq, p); + resched =3D true; + } + /* + * We continue with the search, just in + * case there's an even higher prio task + * in another runqueue. (low likelihood + * but possible) + */ + } +skip: + double_unlock_balance(this_rq, src_rq); + + if (push_task) { + preempt_disable(); + raw_spin_rq_unlock(this_rq); + stop_one_cpu_nowait(src_rq->cpu, push_cpu_stop, + push_task, &src_rq->push_work); + preempt_enable(); + raw_spin_rq_lock(this_rq); + } + } + + if (resched) + resched_curr(this_rq); +} + +static void group_push_rt_tasks(struct rt_rq *rt_rq) +{ + while (group_push_rt_task(rt_rq, false)) + ; +} + +static void group_push_rt_tasks_callback(struct rq *global_rq) +{ + struct rt_rq *rt_rq =3D &global_rq->rq_to_push_from->rt; + + BUG_ON(global_rq->rq_to_push_from =3D=3D NULL); + BUG_ON(rt_rq->rq !=3D global_rq); + + if ((rt_rq->rt_nr_running > 1) || + (dl_group_of(rt_rq)->dl_throttled =3D=3D 1)) { + + while (group_push_rt_task(rt_rq, false)) + ; + } + + global_rq->rq_to_push_from =3D NULL; +} + +static void group_pull_rt_task_callback(struct rq *global_rq) +{ + struct rt_rq *rt_rq =3D &global_rq->rq_to_pull_to->rt; + + BUG_ON(global_rq->rq_to_pull_to =3D=3D NULL); + BUG_ON(rt_rq->rq !=3D global_rq); + + group_pull_rt_task(rt_rq); + global_rq->rq_to_pull_to =3D NULL; +} +#else /* CONFIG_RT_GROUP_SCHED */ +static void group_pull_rt_task(struct rt_rq *this_rt_rq) { } +static void group_push_rt_tasks(struct rt_rq *rt_rq) { } +#endif /* CONFIG_RT_GROUP_SCHED */ + /* * If we are not running and we are not going to reschedule soon, we should * try to push tasks away now diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 55631d93e02..4e287aaa72f 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1282,6 +1282,16 @@ struct rq { call_single_data_t cfsb_csd; struct list_head cfsb_csd_list; #endif + +#ifdef CONFIG_RT_GROUP_SCHED + /* + * Balance callbacks operate only on global runqueues. + * These pointers allow referencing cgroup specific runqueues + * for balancing operations. + */ + struct rq *rq_to_push_from; + struct rq *rq_to_pull_to; +#endif }; =20 #ifdef CONFIG_FAIR_GROUP_SCHED --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 74FD52882CE for ; Mon, 29 Sep 2025 09:22:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137762; cv=none; b=MYEGeK9pTVUEa9t05znwNsWjgOWKxZeQyTkORcgsyTkcXgZ5w6yuGM8AHxc+yLkbdPWzX6sUUVc3v7vvc0e6kb6e/QW45/XmXP6ypFXnx6I+KbAXNm1pjtFbab4A3Ir61/0DLd8ODzXHuKy8fyjpFBhk4+SurCPrs+kzHR84b9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137762; c=relaxed/simple; bh=Iz7a/+csj2v8C4sn2k5anh+MgzwpbdhqAF5JZBkm4Pc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G2uK/nJAvODNH+KSM/PAZLDnTVFZv7zoOfjhYvxYxBTONTdzCsMU8zjhsPNUfHvSfMQc2jLMwQ5hxBhLVbPSmXYCvTYT3DXQYLIsnLm2WIprvtTUDW7U6ksG3OyywBkShQQvuc+w9u4fc9mPYXgjBj29os6Q+lKOELMWurYw/9Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Xh7/MrDl; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Xh7/MrDl" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-b3727611c1bso786327966b.1 for ; Mon, 29 Sep 2025 02:22:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137759; x=1759742559; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+lRFZDcuR+/tnVgGSLQo/bREaLE1pnyBOicDh/5tuqQ=; b=Xh7/MrDlSXAjGNirnSU1XJ+JZFjkQQRTXUP6Wt2jZcUa4iW98BLQFg0auW2BRAufI3 gw67DeoIA5CukOP9mAtwNZLSX0pt9Hil2OvGbTWt7fmZWzKvFNwS0Armz7/tORQ8E0Hl 6lxQIn3VyRxEsgfTMEqhQ1BCawNuPUaX3nW1x5jauTJei145TJel6L5eRCnowXOF23Zu B46qidhezUDN9rMkl/Cee10v5rU+FU3kRgHfGYctXDAdHUpvYSdKMq9s8GkiRvWshCqB OvC3ETxCzaCAoYE2RKd+s8X00FwARV2x+N4lj0XWnpxGgq9wKqT+LghrNVs/JcL0by9C CodQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137759; x=1759742559; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+lRFZDcuR+/tnVgGSLQo/bREaLE1pnyBOicDh/5tuqQ=; b=u+m3dPv6eTBfF8T7fNzYdp9JwsRGIAfMkLY495BH9sfetjJPVALY6vW5XmfxoxBcwP LkNyYLJr9N720qazKB4CBqS1Nwqs2juJgICpqDySyoAA/yFpbcjEiSM2p07xOzH3i6J/ VKGK1KZxqlyvtXr7Qzmlw9mRWxwwJO8uH+Gqy8xD2HPh0cvbUc3ROn9arSk+/in2P5mr ze24pSC5r9FktQlTmK34CIDRwMu4rgYpJjG4X0l8C8jyxMBuQAY4yzUgtBFIv9qQ8A/V zdeuVdS/q0DHul8aHFLfi801q836jd2875IUXHiY8LOpResJ7Vv6Eh4+y3m51Q5iW8mw o5+g== X-Gm-Message-State: AOJu0Yzv08i9lQCzEiNu00PyUEZUU+qVOI6eMYOdPJk0QifrL6OBWbQk HOXOydsftHfUBbc8dVGG6KhZOWn5E/HZPoVycL2+p28wxRr6dqpf2r59s0HliQ== X-Gm-Gg: ASbGncv34j4Qen1/Vch2q+hqSwXPPXllYMu4SujI4/xq6z9v+CZjAPH5f+vuvk6vtH3 YdvwVGtN1r1kiRKipr9Cnmgt5CQmz2o9BtQZEjzmvK7Yk7/Qb86Ew9zS3k528nZAFTU17DoVGxJ K8mw+eJnk+PSHUWfFWSOOpmXeKOmvx9Uz5OfpYvwbVTFAjts14xPotYqrqEya5dEJJu1+kVrDu5 cZ6ujV1HsXiJyT2bBMXZhcpK05obmQEXJjHQTbCkcUF1VA+phpX1lohNImsFOJL8yD6OA0VYb23 hwl1wNsf/cdpXGrDUsnZ/uIZk2TTSepJ9vwr+Mvmh9GXWvtbIvL8k48SdHG82VYD6DVOFWxAGkn M7++Up668jmryiLB0+igMup8hxNf3x5kSqw2KkHUK X-Google-Smtp-Source: AGHT+IEVILOg+z91fG4SWjeQohPG/tNRPAWTFl9FF7AuDiiUdQbPb54oZ90e624eSQJ3nmcJdqFcUg== X-Received: by 2002:a17:907:f815:b0:b31:ec30:c678 with SMTP id a640c23a62f3a-b34bbebc92amr1982113766b.57.1759137758418; Mon, 29 Sep 2025 02:22:38 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:38 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 20/24] sched/rt: Add HCBS migration related checks and function calls Date: Mon, 29 Sep 2025 11:22:17 +0200 Message-ID: <20250929092221.10947-21-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add HCBS related checks and operations to allow rt-task migration, differentiating between cgroup's tasks or tasks that run on the global runq= ueue. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/rt.c | 69 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 53 insertions(+), 16 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 1fbcac14fc9..a864f569011 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -884,6 +884,11 @@ select_task_rq_rt(struct task_struct *p, int cpu, int = flags) struct rq *rq; bool test; =20 + /* Just return the task_cpu for processes inside task groups */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + is_dl_group(rt_rq_of_se(&p->rt))) + goto out; + /* For anything but wake ups, just return the task_cpu */ if (!(flags & (WF_TTWU | WF_FORK))) goto out; @@ -983,7 +988,10 @@ static int balance_rt(struct rq *rq, struct task_struc= t *p, struct rq_flags *rf) * not yet started the picking loop. */ rq_unpin_lock(rq, rf); - pull_rt_task(rq); + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq_of_se(&p->rt)= )) + group_pull_rt_task(rt_rq_of_se(&p->rt)); + else + pull_rt_task(rq); rq_repin_lock(rq, rf); } =20 @@ -1085,7 +1093,10 @@ static inline void set_next_task_rt(struct rq *rq, s= truct task_struct *p, bool f if (rq->donor->sched_class !=3D &rt_sched_class) update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 0); =20 - rt_queue_push_tasks(rt_rq); + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + rt_queue_push_from_group(rt_rq); + else + rt_queue_push_tasks(rt_rq); } =20 static struct sched_rt_entity *pick_next_rt_entity(struct rt_rq *rt_rq) @@ -1148,6 +1159,13 @@ static void put_prev_task_rt(struct rq *rq, struct t= ask_struct *p, struct task_s */ if (on_rt_rq(&p->rt) && p->nr_cpus_allowed > 1) enqueue_pushable_task(rt_rq, p); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (dl_se->dl_throttled) + rt_queue_push_from_group(rt_rq); + } } =20 /* Only try algorithms three times */ @@ -2209,6 +2227,7 @@ static void group_push_rt_tasks(struct rt_rq *rt_rq) = { } */ static void task_woken_rt(struct rq *rq, struct task_struct *p) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); bool need_to_push =3D !task_on_cpu(rq, p) && !test_tsk_need_resched(rq->curr) && p->nr_cpus_allowed > 1 && @@ -2216,7 +2235,12 @@ static void task_woken_rt(struct rq *rq, struct task= _struct *p) (rq->curr->nr_cpus_allowed < 2 || rq->donor->prio <=3D p->prio); =20 - if (need_to_push) + if (!need_to_push) + return; + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + group_push_rt_tasks(rt_rq); + else push_rt_tasks(rq); } =20 @@ -2256,7 +2280,9 @@ static void switched_from_rt(struct rq *rq, struct ta= sk_struct *p) if (!task_on_rq_queued(p) || rt_rq->rt_nr_running) return; =20 - if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED)) + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + rt_queue_pull_to_group(rt_rq); + else rt_queue_pull_task(rt_rq); } =20 @@ -2277,12 +2303,21 @@ void __init init_sched_rt_class(void) */ static void switched_to_rt(struct rq *rq, struct task_struct *p) { + struct rt_rq *rt_rq; + /* * If we are running, update the avg_rt tracking, as the running time * will now on be accounted into the latter. */ if (task_current(rq, p)) { update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 0); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq_of_se(&p->rt)= )) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq_of_se(&p->rt)); + + p->dl_server =3D dl_se; + } + return; } =20 @@ -2292,17 +2327,15 @@ static void switched_to_rt(struct rq *rq, struct ta= sk_struct *p) * then see if we can move to another run queue. */ if (task_on_rq_queued(p)) { - -#ifndef CONFIG_RT_GROUP_SCHED - if (p->nr_cpus_allowed > 1 && rq->rt.overloaded) - rt_queue_push_tasks(rt_rq_of_se(&p->rt)); -#else - if (rt_rq_of_se(&p->rt)->overloaded) { - } else { - if (p->prio < rq->curr->prio) - resched_curr(rq); + rt_rq =3D rt_rq_of_se(&p->rt); + if (!is_dl_group(rt_rq) && p->nr_cpus_allowed > 1 && rq->rt.overloaded) { + rt_queue_push_tasks(rt_rq); + return; + } else if (is_dl_group(rt_rq) && rt_rq->overloaded) { + rt_queue_push_from_group(rt_rq); + return; } -#endif + if (p->prio < rq->donor->prio && cpu_online(cpu_of(rq))) resched_curr(rq); } @@ -2325,8 +2358,12 @@ prio_changed_rt(struct rq *rq, struct task_struct *p= , int oldprio) * If our priority decreases while running, we * may need to pull tasks to this runqueue. */ - if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED) && oldprio < p->prio) - rt_queue_pull_task(rt_rq); + if (oldprio < p->prio) { + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + rt_queue_pull_to_group(rt_rq); + else + rt_queue_pull_task(rt_rq); + } =20 /* * If there's a higher priority task waiting to run --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51EAA2D24B2 for ; Mon, 29 Sep 2025 09:22:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137763; cv=none; b=ZagSjVjDTky+MD5YHNQF73OjVQuV/DCkUchDjl1xZkDoVfDYkmN1Bc94ZeAIscfSB3e3HZAtz4AuRxrtio1trtq8dEqtzXCAQLUUJY0xj3iN93QCEDwhg0By1MWW5hxQpUP1FQdlMFKMcxXG/YdInd2jg99pY06Bp03tKlTR138= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137763; c=relaxed/simple; bh=m3Pzi6tSvc0JP0+0C3UP/XKTq1slnjF8MBXLhDQOUcw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HxDOmBbgrUbJ8qklRvn0d4riA3Sc5mOV/FdkPR2OjMPKQcf28XjSGSy9nJf9MfyCvVt8CBaFRCV42eE8LQsOc2YlUSVWZcHTGkuAeCAKDk9H0oSEsPLgTiatWGE9TlqmwmpON3o+CU+wjoboQeMsmfVQ53InrvwPv61BLTNiqJk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gimy7TQh; arc=none smtp.client-ip=209.85.218.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gimy7TQh" Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-b3e7cc84b82so171782866b.0 for ; Mon, 29 Sep 2025 02:22:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137760; x=1759742560; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dHFDxjqTBX2TNsqIiC9tqPpLk5YNM8tu+mV6Str/9aY=; b=gimy7TQhJOpOYNcWznj5egl0tOS0vXSt+esoocTIcAp35RjWWynqNzASjwgZnn2qMi 1qQqCF3Fgo/YAv07zfaGl5yDpDtF1p/X9WF6EFvvT/bwZSmLGU0VtFLNjdMUoKwcvbYY 4rRquAgQc/r8TT2A3i9Ifs2aj8PXD4gouPENMvc1SayK4LuIfOjUYXfF0yyG3JC8qGNf +OHohC0Z/CIqlxW2sZ0JWhpE3MEGyZhTq6DHLehrYvmNEuul6Po/yDjqg5Z7lqlkfMeO Kb3Dyl+vcapGDXgyMMfja2FN8fUJTRh4tBFj3jTWP2gagBINgpChHvtAzJEQDI5l+q6r DgWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137760; x=1759742560; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dHFDxjqTBX2TNsqIiC9tqPpLk5YNM8tu+mV6Str/9aY=; b=PSz2E+tCI7GUNQk3s6K5qOZ2S2vEJFJuGaBpvYqaNCQITQgSEZ4rAoTxLYO713zzpQ 99jo9x4FaQBef9xaKgsamNrfy941XEw/2fEuaTWAarhivCyRrT7FY1mPNUYub9YbPLWb U+oDHuEwcSDFSbbZmxIp7AF9sym8I3PbfP/aWn2g/z/EvU8a1bTLffYnhoussb9PxMbc 6sWmd3K4eXBh6XZjkFOLM2kYzzp4OWatuZ1CWwQXOh0ogbM2E4YWXskf+riTq8bdqqwp RR64rIp+S+oLZnONm3lYfCQ+mMMrDJ6a2MOCF2Dpyn8yQ6lz9PXcR0xq3SL7CwmMjZVi w5Sg== X-Gm-Message-State: AOJu0YwNkRwRQmqKrFSbEHYKW95081fV2pdAlRi8QDa+LtBfvM8o2p4U Rtk75OzRaRpmzeJEvD2pasonget5dvaA78OtZzAe5iKM3jvJWLoZfz+j X-Gm-Gg: ASbGncsEg1cqDXHE///s+cb1F/aMTW/ILgDJFsKN4SW1uEEVUoDhUhGVZPNWOMXZnHM xskMljgZl6nVvgfKUzuTL/6k0RM+Qor8KTaoWAjIx5xZ54K7I8CCLi+6cEjbiQUhz40sgPEtIee oClSdjVKKam0MJJHrPv8ptbYc7MiFzpx8r5a/t9sMEjshE6zqf2AEk/dA/5BbPFHyfAwY5SUM1E EUydS2pXXtzd7yusDS5k3GvAFQ1WXuHIvEnVf6xitbEsjrv9UEJiRCtYzKwtyu+xlZyIQDv9sBq xuB4hzBLywEq8VMDLiZANkoSk/zGxgw5etfcuDVcKbxPw6LHFyE+mu8+5sDuxlv+U9z8uj03j9M NCe4Xvu0aU7Dyg0hgUYfqe9fnqfG2tA== X-Google-Smtp-Source: AGHT+IHE7LtsON2OlCpiUTAE7zmHeLcehiCmKUGuR5VjB56xRGNZbhIyKHO+uH4Lp9o6A/bhRPqNBw== X-Received: by 2002:a17:907:7fa9:b0:b30:b2e4:af3f with SMTP id a640c23a62f3a-b34b7fbb4cdmr1789721366b.14.1759137759248; Mon, 29 Sep 2025 02:22:39 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:39 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 21/24] sched/deadline: Make rt-cgroup's servers pull tasks on timer replenishment Date: Mon, 29 Sep 2025 11:22:18 +0200 Message-ID: <20250929092221.10947-22-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/deadline.c | 6 +++++- kernel/sched/rt.c | 6 +++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index abe11985c41..27a5d5ffd39 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1290,6 +1290,7 @@ static enum hrtimer_restart dl_server_timer(struct hr= timer *timer, struct sched_ { struct rq *rq =3D rq_of_dl_se(dl_se); u64 fw; + bool has_tasks; =20 scoped_guard (rq_lock, rq) { struct rq_flags *rf =3D &scope.rf; @@ -1303,7 +1304,10 @@ static enum hrtimer_restart dl_server_timer(struct h= rtimer *timer, struct sched_ if (!dl_se->dl_runtime) return HRTIMER_NORESTART; =20 - if (!dl_se->server_has_tasks(dl_se)) { + rq_unpin_lock(rq, rf); + has_tasks =3D dl_se->server_has_tasks(dl_se); + rq_repin_lock(rq, rf); + if (!has_tasks) { replenish_dl_entity(dl_se); dl_server_stopped(dl_se); return HRTIMER_NORESTART; diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index a864f569011..b09f0887e3b 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,4 +1,3 @@ -#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR @@ -153,6 +152,11 @@ static inline void set_next_task_rt(struct rq *rq, str= uct task_struct *p, bool f =20 static bool rt_server_has_tasks(struct sched_dl_entity *dl_se) { +#ifdef CONFIG_SMP + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + + group_pull_rt_task(rt_rq); +#endif return !!dl_se->my_q->rt.rt_nr_running; } =20 --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0FB12D3217 for ; Mon, 29 Sep 2025 09:22:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137764; cv=none; b=nb78PBf15uNAePrVhGb3A98toTIUFwusF8JRzF9sOxrrz/taM47UvO01gFrdLxF95ONRrdCRIKjVQu04fplZz/O5z2uxFaAozq/QK+7h/FvMhaofU4ToYVPwcDNcr0eFyrLD5TKeXXrlNfp7AJdIT5dGQ4K9/1vWpRZ3krnu3vo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137764; c=relaxed/simple; bh=p6tAXc/Gp8y6wh3gHk8uXHEN1wF4RM0QuacVVWEMKGw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=k0yO3kNJ+ThAQArsR9KdwFNNMW4bjHodrbBWpPfCGiTvfAMis7rO5M4S6ABHfRM7mnZt0gcTZXWg4pIYCGbxJSnfT/meHsFq6FYo7BMDistHQuNMr8hA27TwYA2ASIg1/I7nXHRNUDaOLOqVj6u+HHwSY8KPY43xR0JwNIAAXrA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=M2GM8HBc; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="M2GM8HBc" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-b30ead58e0cso826085066b.0 for ; Mon, 29 Sep 2025 02:22:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137760; x=1759742560; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/O5MDDXltIa3/ZVaUfZG566CA7ZdErQ3dKbXCzebZiM=; b=M2GM8HBcmIzIJBu8cZpljU+LbIByobAHjs3QV1SF0IlhabZNqnTBPS8U/IT/xK3d1E dVXGHYFcUximxq1HWbKV44KtkTUimZVXK+rU+NtFeBJyyqrsy/V8/rTayH6xKiMMZoYz BfcXxb3zQSb9hmhHMA5rlM3zuuLcnyecQFkx28tIkLWm4OM3eCsxRU07hq7S/+LgjvFA AcObJUBJyMcOo78uZZgRYN04zsqRrARN6ObuVGPAHZOwbc3xthNQmGeFXzb1YR24DvKx lTxyt4mZkm7IZThrbvOlHdglXSrSYhPLfeXnNUxy8nLxg2gndSmHmV4O5+fcxCvnthZO f9vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137760; x=1759742560; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/O5MDDXltIa3/ZVaUfZG566CA7ZdErQ3dKbXCzebZiM=; b=py1ZSl4qKO+YkoFI/VzJZOo44BJO5B/6+GHAbODJSCGbj3HwdELD1VcUpIvZfj8X8n 8s2S2H+sTpQMZhlD8wiSL9OC8RGccJUN/I0P0e+JEGo/CDeiObbyoGWOiq3UPIF96w3H Z4eF4MYTtGz5lm8s6Y4qNavxAH2BO1NxA0sqdfe4crUCNXk21lqIldD9Hh6xoDfQo2YP oV2+hHtysD4GubAh8HuVjZi8az79Y68R6n48uic1Fc8cVhRKgISTH2GMghnX0648w+F7 Qwl66ywq6gmPUT6PEItk6M/lc/pDgndopoyAKz1QmYkvS5uhqpy4Gs9JLbDMf0LoNOau ui1A== X-Gm-Message-State: AOJu0YyA7+imtCnifjVpDch6DNfC6GTZVLoDYyserO0sjRV/De+NB96n Q9EPl8mXKUNIuBN/0TCZg/r8fyl4MxxxA4DD1D8G/9D3fHktZ4DgriJd X-Gm-Gg: ASbGnctioJLhfIGNfF3W/JHU5Ecc3XCF8Y/s31MZ9DlIQvVR6Cgo1pYbdR0vz2E/kMH 39eRdEY57jpak0AyTzxPZgKrZPA+rbxHJUpFwqjT2KJNv6yZwqY/s7S1p0FwcCWhTZMmcgul5dM VRzLCagW1/A1nY5g4RBM7GbYF+DoC2JLq3Buz2sOz3x66J7jfdUnduXDWmLzpbtkhG4TGxEiSec p0cuqLDeepmAEA0V4v45yKEAKwkxnur9ntwnzXTqHR4UTTtrLy6K9S5gCgQOA9MKwN5gdXILp8x 3h2gc4Eua63vtc+87+UEz/HYgB6b4I8Dyc2PfekFl8lNA50nNG8d+ZiwPEaWcG98ePGP8j4gk56 YbEzcMTbuCS78mn/FXOaHKcuXSJzJ+UkTU7F9JCld X-Google-Smtp-Source: AGHT+IG/29r9BZ8hOEFCsztkdALclY5lYFDek6KlTxCKlYpwVc9lYrzesovC7SyxIviINs6huBBI/A== X-Received: by 2002:a17:907:d93:b0:b3f:cc6d:e0a8 with SMTP id a640c23a62f3a-b3fcc7d24f9mr204587666b.17.1759137760005; Mon, 29 Sep 2025 02:22:40 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:39 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 22/24] sched/deadline: Fix HCBS migrations on server stop Date: Mon, 29 Sep 2025 11:22:19 +0200 Message-ID: <20250929092221.10947-23-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Do not unthrottle a non-fair-server dl_server_stop(), since it ends up being stopped when throttled (we try to migrate all the RT tasks away from it). Notes: This is a temporary workaround, but it will be hopefully removed in favor of less invasive code. Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/deadline.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 27a5d5ffd39..6435a94159d 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1707,9 +1707,11 @@ void dl_server_stop(struct sched_dl_entity *dl_se) return; =20 dequeue_dl_entity(dl_se, DEQUEUE_SLEEP); - hrtimer_try_to_cancel(&dl_se->dl_timer); + if (dl_se =3D=3D &rq_of_dl_se(dl_se)->fair_server) { + hrtimer_try_to_cancel(&dl_se->dl_timer); + dl_se->dl_throttled =3D 0; + } dl_se->dl_defer_armed =3D 0; - dl_se->dl_throttled =3D 0; dl_se->dl_server_active =3D 0; } =20 --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6284A2D3728 for ; Mon, 29 Sep 2025 09:22:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137764; cv=none; b=fI4o91tEze9CXK8EHrLEj9y7jkknqAo0A0Z/sXuYV3pHaTxAGMf3ryez1uti1piBJQvpfZdI1mJPAhDQfYnFld6nitXCKXg0wFdQRoKTxK6Y0/D8g3F2hFHU/Dqekc+83QfmONiVmtlvCcDrC/NxU7hIPWBOO+E9wNE+AQGv8Qs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137764; c=relaxed/simple; bh=dSTafO0Wfmu1tsE0JuYHYquZxnO+mteeMPPolrhjE5w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qnxG3SiVOyf8M+z6dOO+Ekhl83CHgL5T2topPQTpqg4cUkL0FBjKiHSRyxZ80lzF3p9QVXagLYjoQ3pl0ptUfY/QEJXOugoawRbqgSWnjFtPLGu0sQjHYnJXeiqgglkQuBEOwK3M28xR9K+/UhCANSva5yLYccEMFlJXc9G8CHg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Fxgz0vuJ; arc=none smtp.client-ip=209.85.208.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Fxgz0vuJ" Received: by mail-ed1-f53.google.com with SMTP id 4fb4d7f45d1cf-628f29d68ecso9149933a12.3 for ; Mon, 29 Sep 2025 02:22:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137761; x=1759742561; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=R3P+b1nntEpP8NlkC2PbHvgc9UNBVCKeA0Cy8fPRIVo=; b=Fxgz0vuJpY2oSWco7+W7PiGBJs/Z+vRsWm45+wv1HI3udqyWHncidmW6Ofsi8fc8fR i+No1EUWa33d9EFkfaXgr3BkA1MDcQ8cW1PZikNrqqFz7hYkHz0+CaHyfQwPWhdFjvps 3fkLUKnBiLXYbH+VrucWuvOGh+dg0pzQcTJ5M4+DC9GIcrJ4JhIxGwCPx/RlVtmrpemY ewtPcNSiQcyvaC0sBhCgcjfsRAnmcFVKpGRZtob8HiyaB3DwFWBwGyRcygeD8y1H9xw5 iaGQYjgdbOkdtR1ltAwGB6ymY8xhI2xhP7KjhCV8d09UQeBofBtLzEX3sYCEyEvEctcx ZlvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137761; x=1759742561; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=R3P+b1nntEpP8NlkC2PbHvgc9UNBVCKeA0Cy8fPRIVo=; b=CEzAVON183HTi8XqEUtEuPx7e3QezWpx/BT6rIYOLstGeHeAytI5wfJc+VnEe15QF3 T09MJ1afk2+EHINRq2ypUUw2rsSk72EBlYw3soj5fuCRenLhgo61UpqvYII/CnyHMgBD Hgpg2GhpMiL0ayJaJfeD4mm0PxNucpT0w9oqklsjmVYknM7fMR2O8FOOKlPr3Wdh0ONY 7+rmiN3ZoDBtz6i5D7MufQLodmbqxWsvhlOwh/GHjlZnZpVZK6O4Uqx9cfzY9WwHKU1h RvJ/AJBY+pxOnqlU0NRlbDB7Xf0II5wHm1Dj6aPL0Jk1uE++sTPK/NFVL+D0ClO0ihGl umXg== X-Gm-Message-State: AOJu0YxWUpA8DfHbifHVoslZPYAfnueQShsdDhGdkTGpGDu9ZDIUHNNw IolHfFxN8gs9s1FcMmH9MAo5k26P3MHkJIAYVO6bqlsql4PxEjQ3GUUL X-Gm-Gg: ASbGncv5ceykN/Jk0fl6jIOhxf9IW9qC0/486ZbnpiUEcpjE2hS/M/7szqNXaHSts5v YXBnDpQkcTBYeN6ELxtnJC8I24YpH8Ippyz3CY23MBgVXPVJibQdPhMD0Rgs/zFLYFGHfhqsqX5 StE+mvm1LC0JsuM8pUYTTraKNnUx4slY1NCxSyc8DskjmAmTkF3daY97RZ2SrWHMzFsa74Rc8Uw lBjigu0nixLidW+gPflGs4USz7eA/eBm65H9Ev6mVTHcNXyIN2hPr6kk/vcA4jSkIYu0HwGVyZW yBRDIVIFlK3PUXs0K7Gm5P9+cKXnsIzZC5FjOoNmabGUN6JB6Ot9jcewlHZTjAYPeDQk9W1WU9V qWigXWmlQ49VEe4+ap3uGo4nL2UQFjA== X-Google-Smtp-Source: AGHT+IHgoR0hf5g2HMHuDmYVCAC33uJarphDIsZTc//ymRvUrt3I6JVghWnibJc4SlH583MceQ0OWA== X-Received: by 2002:a17:906:f586:b0:b3d:d6be:4cbe with SMTP id a640c23a62f3a-b3dd6be501fmr494957366b.18.1759137760701; Mon, 29 Sep 2025 02:22:40 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:40 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 23/24] sched/core: Execute enqueued balance callbacks when changing allowed CPUs Date: Mon, 29 Sep 2025 11:22:20 +0200 Message-ID: <20250929092221.10947-24-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Execute balancing callbacks when setting the affinity of a task, since the = HCBS scheduler may request balancing of throttled dl_servers to fully utilize the server's bandwidth. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d1d7215c4a2..7639e9abba1 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2950,6 +2950,7 @@ static int affine_move_task(struct rq *rq, struct tas= k_struct *p, struct rq_flag if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask) || (task_current_donor(rq, p) && !task_current(rq, p))) { struct task_struct *push_task =3D NULL; + struct balance_callback *head; =20 if ((flags & SCA_MIGRATE_ENABLE) && (p->migration_flags & MDF_PUSH) && !rq->push_busy) { @@ -2968,11 +2969,13 @@ static int affine_move_task(struct rq *rq, struct t= ask_struct *p, struct rq_flag } =20 preempt_disable(); + head =3D splice_balance_callbacks(rq); task_rq_unlock(rq, p, rf); if (push_task) { stop_one_cpu_nowait(rq->cpu, push_cpu_stop, p, &rq->push_work); } + balance_callbacks(rq, head); preempt_enable(); =20 if (complete) @@ -3027,6 +3030,8 @@ static int affine_move_task(struct rq *rq, struct tas= k_struct *p, struct rq_flag } =20 if (task_on_cpu(rq, p) || READ_ONCE(p->__state) =3D=3D TASK_WAKING) { + struct balance_callback *head; + /* * MIGRATE_ENABLE gets here because 'p =3D=3D current', but for * anything else we cannot do is_migration_disabled(), punt @@ -3040,16 +3045,19 @@ static int affine_move_task(struct rq *rq, struct t= ask_struct *p, struct rq_flag p->migration_flags &=3D ~MDF_PUSH; =20 preempt_disable(); + head =3D splice_balance_callbacks(rq); task_rq_unlock(rq, p, rf); if (!stop_pending) { stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, &pending->arg, &pending->stop_work); } + balance_callbacks(rq, head); preempt_enable(); =20 if (flags & SCA_MIGRATE_ENABLE) return 0; } else { + struct balance_callback *head; =20 if (!is_migration_disabled(p)) { if (task_on_rq_queued(p)) @@ -3060,7 +3068,12 @@ static int affine_move_task(struct rq *rq, struct ta= sk_struct *p, struct rq_flag complete =3D true; } } + + preempt_disable(); + head =3D splice_balance_callbacks(rq); task_rq_unlock(rq, p, rf); + balance_callbacks(rq, head); + preempt_enable(); =20 if (complete) complete_all(&pending->done); --=20 2.51.0 From nobody Wed Oct 1 22:33:20 2025 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48B612D3A71 for ; Mon, 29 Sep 2025 09:22:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137765; cv=none; b=QgcGzvUw/tqIaT1b4wmWMO6q7tLEJlxbgrCoubuHguWgR12K0ck3C0PinD9Xc5lP9rPTUH7kkMPIeoJoV9FfJqfz6FGBoF6q/mZ1o48oom/xEn/Rt8ftLfLO6xaYo4g5UfrDgWAWO5QcXAXKKCONaafvBlJdITJCboRKdrALd5I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759137765; c=relaxed/simple; bh=3pttDbYbABLZI5Kna45k6qR+8PXaeJRJV25CcDASDwc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gYkaH+ygZ4F+yOEADwfQMLZxknN+s9wnMlWKSou8UT9zqM8sdkSF0+KgzPi9U3QyF3BCbVrlbJKhmfJXh4DpOVDBlPfdVyL7xfXWZ2ClQy3IsltmKtHx1/vMCuBOOONsKH32eubplEXu/vX2JMybXZ2XmB0/2J7rxwJpZnKI6l8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Pt0rYXTK; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Pt0rYXTK" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-b346142c74aso841712766b.2 for ; Mon, 29 Sep 2025 02:22:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759137761; x=1759742561; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hleM5rVXWBZmpwO2/flr+0eqHP7XnDYEg6jHQEy66OI=; b=Pt0rYXTKVK+Rhk9SWfkv+QfoXt62BK35pvQzJPoBwepL+U3+bEnKFnBBC3mzX1uN8x zmQxE4zsJk01Fd7nvfG1awtrp98qRhWNDQm4P2bcAi5ivTrZsIBnVSnfT+S0flwcsXd0 rzg8IIh6qFkF6tuLbIN/3qsx0AssL00+cRbmK+EnemvlNjoE5+HSrqSdmDoQD7Y4yKDi W4/Kdwctx9JyERoMiH/k7gi8T1ssLpqI7jhYaCOhxJ9KRpO6sx1fdupkBgAL8fr9XImZ asrFqL1bFET3m0HyiOoJvDS6S1l2zwOjteHiKTLtNf5CYDOXbe14RceS/F2LFaEVw8rK cvQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759137761; x=1759742561; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hleM5rVXWBZmpwO2/flr+0eqHP7XnDYEg6jHQEy66OI=; b=sm9TLXctQ+E8I2uCQ55PL9YEb0Y7wt2ngpXpyBEudqVHG/2CrnuYTO2q02zrUuCct/ gg+1P0SjB4uA+/0gt64dmGS8905CxPt7HNFc5dVG+dUtGOL5ahECYi5CSbdrbFwMR+a/ pO1G22Q00bSH7x1gjGCcR0v0wBXWw7YYj/zMidBrfM3zrKuPbLcd6f2ECosTpWB4HTEz y0DVYPmxU3cCRdNkVuc6UZtpfC1UAd0Cp69DA3tS6jjRt07n1qLYC+6btxEM4EPgZLd8 KJrnqeyKzzRugWYnfX7ljQEVSFHi0MnwXQb5tOZRIL1V/kZ5zfiArVYBqGyXp0XKgx/G zRRw== X-Gm-Message-State: AOJu0YxSpppnAhuYybCNT17I5fYABPjxdvLhYW7r0qiDt82nDDzaDZIN T7C+EAojmMTAiO5ojdqcswXlxQe8PGZ8GFZec0Ttepo94vwN0ay7XbQa X-Gm-Gg: ASbGncvXN8wm+8z5bJmxM9I8/ji9BsTWZTUbkhh0zp+OYnFopsEcDlGfgxM+AVyqbLz FPPN2VTWCmJ67e0x3qXoGz6Gw8dlILES58fvszpaxqYtdn+Pbb3l2FY2F9nub/KF5CoHnW9FMSc Zirv0XhZs1EgmiPWhkzh2cR6dLXxbjZd1UjZMPgyVkFJHCELnHoqNOPDA3PfVDNx/DbdUuSxJe3 JBZm4x4SAs8wWtbz/wsWkx8rJbZ/CHYg2u5zL2V+3wwtneAtiFYC1F5uWx5NSRMgjZ0S8XqhIQx HHFp9Fc81LhDIoDbGY1VuBNxwL1ADNNClD9XPiQNl/g1fhWs2gk5MEa2adWrC8zotef+omxbPii T/o9v17S6c5KwqnIIXye8Ec4ogIk3Xg== X-Google-Smtp-Source: AGHT+IGgZWI9G5c+HIV6R5gmn/yt7GQkHR4PiGN+xNFOQELEBVnt6zHE006xZsEcouAIhBZ+XhSBvQ== X-Received: by 2002:a17:906:6a14:b0:b32:2b60:ee7 with SMTP id a640c23a62f3a-b34b77d9ed9mr1685371966b.24.1759137761385; Mon, 29 Sep 2025 02:22:41 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b3e89655b09sm181082366b.77.2025.09.29.02.22.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Sep 2025 02:22:41 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v3 24/24] sched/core: Execute enqueued balance callbacks when migrating task betweeen cgroups Date: Mon, 29 Sep 2025 11:22:21 +0200 Message-ID: <20250929092221.10947-25-yurand2000@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250929092221.10947-1-yurand2000@gmail.com> References: <20250929092221.10947-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Execute balancing callbacks when migrating task between cgroups, since the = HCBS scheduler, similarly to the previous patch, may request balancing of thrott= led dl_servers to fully utilize the server's bandwidth. Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 7639e9abba1..8d99a0c20c6 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9251,10 +9251,11 @@ void sched_move_task(struct task_struct *tsk, bool = for_autogroup) { int queued, running, queue_flags =3D DEQUEUE_SAVE | DEQUEUE_MOVE | DEQUEUE_NOCLOCK; + struct balance_callback *head; struct rq *rq; + struct rq_flags rf; =20 - CLASS(task_rq_lock, rq_guard)(tsk); - rq =3D rq_guard.rq; + rq =3D task_rq_lock(tsk, &rf); =20 update_rq_clock(rq); =20 @@ -9281,6 +9282,12 @@ void sched_move_task(struct task_struct *tsk, bool f= or_autogroup) */ resched_curr(rq); } + + preempt_disable(); + head =3D splice_balance_callbacks(rq); + task_rq_unlock(rq, tsk, &rf); + balance_callbacks(rq, head); + preempt_enable(); } =20 static struct cgroup_subsys_state * --=20 2.51.0