From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E0212BD031 for ; Thu, 31 Jul 2025 10:55:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959349; cv=none; b=F0skDp7ayaccIDkbdQBgjWzWFd/v+Treqn7kmQX9EGi7LV8QzWsa0cYtiOwLCWdsRE2ZbVUOdAlNSJKeY0tvTXUC3aEDcWlp6CN/cIwTHHPUjdDhaRV1iSTW03DcV5aAYfV//fejZBd1k55vFflxnQdB6cdiFzr+A+c0GvqM6CI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959349; c=relaxed/simple; bh=vxUvAjFI51ozTe7TYJ58k74HV+/JXtcjWQqC1kYa5n8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WOaeY0AZIt1GOxJdXd98ylk2fIrOCpkvLsR8xHHOyfv7eNndwEAo6I2+/IhYe7xF8KNTeijoXc7TyQ3WBb8eT7veKPbcUBEBtQiZ3SLntZ/kRS37zWH8G4QfwonEpV/giVwg21cVHHWUid28gcvHAEOa9ZVs+6MwEeI3PXAUk1w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iqrfyk22; arc=none smtp.client-ip=209.85.221.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iqrfyk22" Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-3b78310b296so471979f8f.2 for ; Thu, 31 Jul 2025 03:55:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959346; x=1754564146; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YtBTnswHJbWz9qX4CMvDbzP7yVZaNKyeALd7dwNWNtU=; b=iqrfyk22xglFYuTYqZUu+vSIWm60SCppOlFrDb/bsgKngDsPMxY1GwQGt5yt9TVmJ8 DdjWCiC94GPZiJJBeNuSIqTvLoSJAgbuqrrBW5LZmTtL3zborWpTnuM7vxSgUGZ3su0O SeOT7CvLvKg2HcHUDvugQ4GgojGbiOoObR57UImH9UKPg20qeKduXtXwQjFNsFGzYMjz nOE5mm7bEu5CWguFlwz/p5z6Dpt7cbdIIYKSjQj1VLd40lA4wTHufWLEs7/pkJ1ofcwD oTaBFvlrpwSPa2XrK8nAIDucGo0ndSoOQ47sc0oYIzos0JMW5/tHBbHXmoGC/vM0QD3t KqJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959346; x=1754564146; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YtBTnswHJbWz9qX4CMvDbzP7yVZaNKyeALd7dwNWNtU=; b=eZoe/LplvG5iLpTWUYWC88oa6NglUJeiqObyxaJk/cSKUXERZC4OaXAdFdvLUEI2Fs N+7cNS7859EMNXvlt3pysAWiubHjRZWTZHxhSAbaTxCKyPkyOLdO9TmYRmKDZCzFGSAV 3NGdyHZzUFfaDSGQ1mzNzoteN/c3CvdW4VcaUmXhoonMcvpaHsigcxDrXHqYE6oaXDQw HnYbxMMYBrNWOTKkVg74ownQZNqYmATw6FPAliIbK0KO0up65JjEBSyHp4HN1RJOMXmN tx1952EpB1RDQfmfXj4o/NNg+eM7NWcOYOrQMK+wnHdno4H57JxHMBZDbFW6MLhkDuhJ iy7A== X-Gm-Message-State: AOJu0YwRWOH8/7O0XTS0oa/ZE2kYwVGcQn8yP7iP0Dvt4Ct+spn6PyAq 7wahOhn7dymokIkPp13k6XTVsQN3ZhwT4oUOx3N6CQ7pWTQ2udCN5gLE X-Gm-Gg: ASbGncseRzjTNYtqzop8NeFCd+tmcsiRzonnfEM08YyzE8zmhyxHvLHWFl5/z8l5ieO y0wWnyf6N+XOPT4lM5XcIFU3SSgB7jr1zVocSbokIW9ZpxUHb3MqFuifz3QVgJTFxbiLWCDxYTz a/FEQxpb8oziFvV0dYux/f43MAAdKsSad1MAwuoeEsdctn1HOT4wktkFoKHsn2f6fCa/418mwmX u1wA4MuVekJxUcBug+8ZSK+6aQMe6ZsmSLl0v7tQ7ZLNnumJMqJ3pDK5rMMyJhXxQzxbo/M8zXR NhwUxQSuynRrLtt/KCiDN0lWmkst9L6MRcC2viQ+PXvfLNHuD8Wh06pUJCIb67ExHF9dbdQMZ9Q +AwQGcsROq4DyJSMy69k= X-Google-Smtp-Source: AGHT+IEQTOgRE7kWkIhApFMVKh0Arwhz81d+QiYAOCzqSinguylpX20jiLJLkn+G5YKgwD0a1l7MYw== X-Received: by 2002:a5d:5d89:0:b0:3a3:67bb:8f3f with SMTP id ffacd0b85a97d-3b795033f01mr5418674f8f.53.1753959345437; Thu, 31 Jul 2025 03:55:45 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:45 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 01/25] sched/deadline: Remove fair-servers from real-time task's bandwidth accounting Date: Thu, 31 Jul 2025 12:55:19 +0200 Message-ID: <20250731105543.40832-2-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fair-servers are currently used in place of the old RT_THROTTLING mechanism= to prevent the starvation of SCHED_OTHER (and other lower priority) tasks when real-time FIFO/RR processes are trying to fully utilize the CPU. To allow t= he RT_THROTTLING mechanism, the maximum allocatable bandwidth for real-time ta= sks has been limited to 95% of the CPU-time. The RT_THROTTLING mechanism is now removed in favor of fair-servers, which = are currently set to use, as expected, 5% of the CPU-time. Still, they share the same bandwidth that allows to run real-time tasks, and which is still set t= o 95% of the total CPU-time. This means that by removing the RT_THROTTLING mechan= ism, the bandwidth remaning for real-time SCHED_DEADLINE tasks and other dl-serv= ers (FIFO/RR are not affected) is only 90%. This patch reclaims the 5% lost CPU-time, which is definitely reserved for SCHED_OTHER tasks, but should not be accounted togheter with the other real= -time tasks. More generally, the fair-servers' bandwidth must not be accounted wi= th other real-time tasks. Updates: - Make the fair-servers' bandwidth not be accounted into the total allocated bandwidth for real-time tasks. - Remove the admission control test when allocating a fair-server. - Do not account for fair-servers in the GRUB's bandwidth reclaiming mechan= ism. - Limit the max bandwidth to (BW_UNIT - max_rt_bw) when changing the parame= ters of a fair-server, preventing overcommitment. - Add dl_bw_fair, which computes the total allocated bandwidth of the fair-servers in the given root-domain. - Update admission tests (in sched_dl_global_validate) when changing the maximum allocatable bandwidth for real-time tasks, preventing overcommitm= ent. Since the fair-server's bandwidth can be changed through debugfs, it has not been enforced that a fair-server's bw must be always equal to (BW_UNIT - max_rt_bw), rather it must be less or equal to this value. This allows reta= ining the fair-servers' settings changed through the debugfs when chaning the max_rt_bw. This also means that in order to increase the maximum bandwidth for real-ti= me tasks, the bw of fair-servers must be first decreased through debugfs other= wise admission tests will fail, and viceversa, to increase the bw of fair-server= s, the bw of real-time tasks must be reduced beforehand. This v2 version addresses the compilation error on i386 reported at: https://lore.kernel.org/oe-kbuild-all/202507220727.BmA1Osdg-lkp@intel.com/ v1: https://lore.kernel.org/all/20250721111131.309388-1-yurand2000@gmail.co= m/ Signed-off-by: Yuri Andriaccio --- kernel/sched/deadline.c | 66 ++++++++++++++++++----------------------- kernel/sched/sched.h | 1 - kernel/sched/topology.c | 8 ----- 3 files changed, 29 insertions(+), 46 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index e2d51f4306b..8ba6bf3ef68 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -141,6 +141,24 @@ static inline int dl_bw_cpus(int i) return cpus; } =20 +static inline u64 dl_bw_fair(int i) +{ + struct root_domain *rd =3D cpu_rq(i)->rd; + u64 fair_server_bw =3D 0; + + RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held(), + "sched RCU must be held"); + + if (cpumask_subset(rd->span, cpu_active_mask)) + i =3D cpumask_first(rd->span); + + for_each_cpu_and(i, rd->span, cpu_active_mask) { + fair_server_bw +=3D cpu_rq(i)->fair_server.dl_bw; + } + + return fair_server_bw; +} + static inline unsigned long __dl_bw_capacity(const struct cpumask *mask) { unsigned long cap =3D 0; @@ -1657,25 +1675,9 @@ void sched_init_dl_servers(void) } } =20 -void __dl_server_attach_root(struct sched_dl_entity *dl_se, struct rq *rq) -{ - u64 new_bw =3D dl_se->dl_bw; - int cpu =3D cpu_of(rq); - struct dl_bw *dl_b; - - dl_b =3D dl_bw_of(cpu_of(rq)); - guard(raw_spinlock)(&dl_b->lock); - - if (!dl_bw_cpus(cpu)) - return; - - __dl_add(dl_b, new_bw, dl_bw_cpus(cpu)); -} - int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64= period, bool init) { - u64 old_bw =3D init ? 0 : to_ratio(dl_se->dl_period, dl_se->dl_runtime); - u64 new_bw =3D to_ratio(period, runtime); + u64 max_bw, new_bw =3D to_ratio(period, runtime); struct rq *rq =3D dl_se->rq; int cpu =3D cpu_of(rq); struct dl_bw *dl_b; @@ -1688,17 +1690,14 @@ int dl_server_apply_params(struct sched_dl_entity *= dl_se, u64 runtime, u64 perio =20 cpus =3D dl_bw_cpus(cpu); cap =3D dl_bw_capacity(cpu); + max_bw =3D div64_ul(cap_scale(BW_UNIT - dl_b->bw, cap), (unsigned long)cp= us); =20 - if (__dl_overflow(dl_b, cap, old_bw, new_bw)) + if (new_bw > max_bw) return -EBUSY; =20 if (init) { __add_rq_bw(new_bw, &rq->dl); - __dl_add(dl_b, new_bw, cpus); } else { - __dl_sub(dl_b, dl_se->dl_bw, cpus); - __dl_add(dl_b, new_bw, cpus); - dl_rq_change_utilization(rq, dl_se, new_bw); } =20 @@ -2939,17 +2938,6 @@ void dl_clear_root_domain(struct root_domain *rd) rd->dl_bw.total_bw =3D 0; for_each_cpu(i, rd->span) cpu_rq(i)->dl.extra_bw =3D cpu_rq(i)->dl.max_bw; - - /* - * dl_servers are not tasks. Since dl_add_task_root_domain ignores - * them, we need to account for them here explicitly. - */ - for_each_cpu(i, rd->span) { - struct sched_dl_entity *dl_se =3D &cpu_rq(i)->fair_server; - - if (dl_server(dl_se) && cpu_active(i)) - __dl_add(&rd->dl_bw, dl_se->dl_bw, dl_bw_cpus(i)); - } } =20 void dl_clear_root_domain_cpu(int cpu) @@ -3133,9 +3121,10 @@ int sched_dl_global_validate(void) u64 period =3D global_rt_period(); u64 new_bw =3D to_ratio(period, runtime); u64 cookie =3D ++dl_cookie; + u64 fair_bw; struct dl_bw *dl_b; - int cpu, cpus, ret =3D 0; - unsigned long flags; + int cpu, ret =3D 0; + unsigned long cap, flags; =20 /* * Here we want to check the bandwidth not being set to some @@ -3149,10 +3138,13 @@ int sched_dl_global_validate(void) goto next; =20 dl_b =3D dl_bw_of(cpu); - cpus =3D dl_bw_cpus(cpu); + cap =3D dl_bw_capacity(cpu); + fair_bw =3D dl_bw_fair(cpu); =20 raw_spin_lock_irqsave(&dl_b->lock, flags); - if (new_bw * cpus < dl_b->total_bw) + if (cap_scale(new_bw, cap) < dl_b->total_bw) + ret =3D -EBUSY; + if (cap_scale(new_bw, cap) + fair_bw > cap_scale(BW_UNIT, cap)) ret =3D -EBUSY; raw_spin_unlock_irqrestore(&dl_b->lock, flags); =20 diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index d3f33d10c58..8719ab8a817 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -390,7 +390,6 @@ extern void sched_init_dl_servers(void); extern void dl_server_update_idle_time(struct rq *rq, struct task_struct *p); extern void fair_server_init(struct rq *rq); -extern void __dl_server_attach_root(struct sched_dl_entity *dl_se, struct = rq *rq); extern int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64 period, bool init); =20 diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 977e133bb8a..4ea3365984a 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -500,14 +500,6 @@ void rq_attach_root(struct rq *rq, struct root_domain = *rd) if (cpumask_test_cpu(rq->cpu, cpu_active_mask)) set_rq_online(rq); =20 - /* - * Because the rq is not a task, dl_add_task_root_domain() did not - * move the fair server bw to the rd if it already started. - * Add it now. - */ - if (rq->fair_server.dl_server) - __dl_server_attach_root(&rq->fair_server, rq); - rq_unlock_irqrestore(rq, &rf); =20 if (old_rd) --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAE6D2BD5A3 for ; Thu, 31 Jul 2025 10:55:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959349; cv=none; b=fzrV1DBznleiAjKgb1+EFCYb0hFfAxN5mK6mSMceeCLW+GbfVemtJMEgeiGuIaAFXIy3U+RmDZz0ic0y1nyp3+IdTk0NqXnCUTuGz/yLB4+v0CpnB0KcRgyuPHL0mzlD6sfIlJ3LZpcP8JyiWSurqoMImJ3QMTa0cEMgE1QFJcY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959349; c=relaxed/simple; bh=7zHuy3Bzmy3kpmP2kbCBpXVwxrJx8uxdFSO/UfUhAzw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JmimCY/tOY1sMv6dxqhTMCTUbrda+NYzmYmiI3dAHQ0r1DSYjilh9UrqhouF08mT4AJBhjgyOHxZgqINTLnVH89C6zLcOwGI2Rcuoxk3m1G7GsXCuyrDmhhxc38wlnfkfRmJDYTz1YLDYXnOp1GlhEk/1q1iPqIGC4y0DWENe5c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CTTEAsU3; arc=none smtp.client-ip=209.85.221.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CTTEAsU3" Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-3b7823559a5so488411f8f.0 for ; Thu, 31 Jul 2025 03:55:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959346; x=1754564146; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tzi2jPoOSdH5XG/gZrMJeOKkz0/pDG4/KykhFPpTSrE=; b=CTTEAsU3+0Ro2/COg8j4nFQW89eo1OUdfv+6a9mhJZKqODxNr1z7+8w5kp20xT6AKu dn6Ei9JzGVTJlTqyHSSUQ+0eEhqJ7Xdk0rqRZ2jVOTpfXI7DiXFlNbOKE2Bo1/9FbGps jR3plHAHnvWl4bBrbB3XOuWz8WNoxC3IzKN7GE53HwxmOkN2TvFNRy9Zte9oTU6ne80O azav6+jPysbw6L3n+W/S2P5Ir+JMZ33cp3cbWqSp04aB1v3FBOaArDwb94VhSnugtYud wXKv446hO5YGeIqQ5423Jeir8BZKjF7lCU4rFCPQeXx/6YL1dI8i+elLsPPeIkPkKeu3 ZnRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959346; x=1754564146; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tzi2jPoOSdH5XG/gZrMJeOKkz0/pDG4/KykhFPpTSrE=; b=DllsRjkrJVve9Vx/RgbInezSsGdMtmD5mREsZsp3M/HPDqkjr4BdDiWgCP3nC0bC9Z SiuHLS+4UqLwGKxzQv42JWN2rB35Uo3M0fJ6ZQCxcyttzGQmkv10O2AG2gbInpH9QZLM W3ndsVTzFGiqn0ynmdlWt5dGNGbdHu1kHQPtUrkq8zrx12iXAbDnG2Q9WJfhXxcmfW0Q 1Uao25tC2M0NnAcvcOn6XU4T+r8eRhXmrfMOIarbcyx1l/YN+D6Mi7jkRwEkuXae7zcz 8ANVxUzmHSMYOvmUCiBrEu0CXEfrFK0ei1LwHDGvJNf4MgxpkREXYOwLPFqyGKCO24Xm 5v4w== X-Gm-Message-State: AOJu0Yxs1zIIVC1uFcxkSsZAF8VkqQYp5k+aInefLsMEzFkB7P5jtx7h 8STO1RdTtMe4EPkGiSiBhj3RGen2pB4mgyC61IEsond50a0tMXGKdtfP X-Gm-Gg: ASbGncuH7qCIMj42J7+FIjx/hvCZM68vjMxhg0zeCKr8q/uwd5pg+BFp+mgiElKc7GU 1sOdtEJos2kYisss82vSHh6XjLE49/Zj+XXiS2xcTjT5CTeDANcA6XoiQCjy98PChVoJGcwI+1k h+XCbjOn4+FuG7r85Z/MjNjWhJT8pshGAe3P135aeXOLHlkKVxzew70f7Q/YkwU6qR1w6XUNm2q NWPij3Ud/aLPOodfgpbDkhbQxN9iJDPe/qknedjPbT7Y4mpOCUefgh0ox7nKCyR1tkA5uykVPOw NMRdwSn0A2lIEqqOI4W9tBabXjLljSTcrj8nYDRRe5JLKf8ksAzWOkZwhxabNBg6HUGyfgDecs9 gjjrDYE+CvNaDe4sP/sk= X-Google-Smtp-Source: AGHT+IGuhmZKx5E9GRCc9EnhnDq0z3y7OE0+CKyG8WVwqqH7o54scX+YtHNHMWjn7N8aeWjjij8zug== X-Received: by 2002:a05:6000:1889:b0:3b3:9c75:acc6 with SMTP id ffacd0b85a97d-3b79503d7d3mr6049492f8f.59.1753959346211; Thu, 31 Jul 2025 03:55:46 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:45 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 02/25] sched/deadline: Do not access dl_se->rq directly Date: Thu, 31 Jul 2025 12:55:20 +0200 Message-ID: <20250731105543.40832-3-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make deadline.c code access the runqueue of a scheduling entity saved in the sched_dl_entity data structure. This allows future patches to save different runqueues in sched_dl_entity other than the global runqueues. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/deadline.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 8ba6bf3ef68..46b9b78cca2 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -892,7 +892,7 @@ static void replenish_dl_entity(struct sched_dl_entity = *dl_se) * and arm the defer timer. */ if (dl_se->dl_defer && !dl_se->dl_defer_running && - dl_time_before(rq_clock(dl_se->rq), dl_se->deadline - dl_se->runtime)= ) { + dl_time_before(rq_clock(rq), dl_se->deadline - dl_se->runtime)) { if (!is_dl_boosted(dl_se) && dl_se->server_has_tasks(dl_se)) { =20 /* @@ -1202,11 +1202,11 @@ static enum hrtimer_restart dl_server_timer(struct = hrtimer *timer, struct sched_ * of time. The dl_server_min_res serves as a limit to avoid * forwarding the timer for a too small amount of time. */ - if (dl_time_before(rq_clock(dl_se->rq), + if (dl_time_before(rq_clock(rq), (dl_se->deadline - dl_se->runtime - dl_server_min_res))) { =20 /* reset the defer timer */ - fw =3D dl_se->deadline - rq_clock(dl_se->rq) - dl_se->runtime; + fw =3D dl_se->deadline - rq_clock(rq) - dl_se->runtime; =20 hrtimer_forward_now(timer, ns_to_ktime(fw)); return HRTIMER_RESTART; @@ -1217,7 +1217,7 @@ static enum hrtimer_restart dl_server_timer(struct hr= timer *timer, struct sched_ =20 enqueue_dl_entity(dl_se, ENQUEUE_REPLENISH); =20 - if (!dl_task(dl_se->rq->curr) || dl_entity_preempt(dl_se, &dl_se->rq->cu= rr->dl)) + if (!dl_task(rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl)) resched_curr(rq); =20 __push_dl_task(rq, rf); @@ -1485,7 +1485,7 @@ static void update_curr_dl_se(struct rq *rq, struct s= ched_dl_entity *dl_se, s64 =20 hrtimer_try_to_cancel(&dl_se->dl_timer); =20 - replenish_dl_new_period(dl_se, dl_se->rq); + replenish_dl_new_period(dl_se, rq); =20 /* * Not being able to start the timer seems problematic. If it could not @@ -1597,21 +1597,22 @@ void dl_server_update(struct sched_dl_entity *dl_se= , s64 delta_exec) /* 0 runtime =3D fair server disabled */ if (dl_se->dl_runtime) { dl_se->dl_server_idle =3D 0; - update_curr_dl_se(dl_se->rq, dl_se, delta_exec); + update_curr_dl_se(rq_of_dl_se(dl_se), dl_se, delta_exec); } } =20 void dl_server_start(struct sched_dl_entity *dl_se) { - struct rq *rq =3D dl_se->rq; + struct rq *rq; =20 if (!dl_server(dl_se) || dl_se->dl_server_active) return; =20 dl_se->dl_server_active =3D 1; enqueue_dl_entity(dl_se, ENQUEUE_WAKEUP); - if (!dl_task(dl_se->rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl)) - resched_curr(dl_se->rq); + rq =3D rq_of_dl_se(dl_se); + if (!dl_task(rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl)) + resched_curr(rq); } =20 void dl_server_stop(struct sched_dl_entity *dl_se) @@ -1667,9 +1668,9 @@ void sched_init_dl_servers(void) =20 WARN_ON(dl_server(dl_se)); =20 - dl_server_apply_params(dl_se, runtime, period, 1); - dl_se->dl_server =3D 1; + BUG_ON(dl_server_apply_params(dl_se, runtime, period, 1)); + dl_se->dl_defer =3D 1; setup_new_dl_entity(dl_se); } @@ -1678,7 +1679,7 @@ void sched_init_dl_servers(void) int dl_server_apply_params(struct sched_dl_entity *dl_se, u64 runtime, u64= period, bool init) { u64 max_bw, new_bw =3D to_ratio(period, runtime); - struct rq *rq =3D dl_se->rq; + struct rq *rq =3D rq_of_dl_se(dl_se); int cpu =3D cpu_of(rq); struct dl_bw *dl_b; unsigned long cap; @@ -1752,7 +1753,7 @@ static enum hrtimer_restart inactive_task_timer(struc= t hrtimer *timer) p =3D dl_task_of(dl_se); rq =3D task_rq_lock(p, &rf); } else { - rq =3D dl_se->rq; + rq =3D rq_of_dl_se(dl_se); rq_lock(rq, &rf); } =20 --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCEC32BDC09 for ; Thu, 31 Jul 2025 10:55:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959350; cv=none; b=J6OZu4lTYLRjAPEza6lFsP/ihEeAppFFqJPkALFsjCs8KW93CQKcu+og/gPwYZ6l9HCldCKu9XwO9donFqIVHsTPLqBJlKLQg9dB3/oEo/hpPe86vWw/apjthwRO9+jSR4L8qAALwkLwD3dJqYpjaZ7lKj8e1lkGJPpMEjE+VgQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959350; c=relaxed/simple; bh=TDuXtiBRf5XyOqVakO8h9p+u/BxiS5DyyaQsPNw9mBY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XAh2pCYVwpEXAJWb/WJEJ4UaFLBXa4Lhu5Vxrb6GNCyQmi01w+UJKHJptiokCY4hTwOP+159yj3WduUAgo6Y+a51xosxRoDp3sqnkcBRsyL2+Ht3kMIuyzCkgJIuaCqKfuVZDlNHGqtLIyRQoBGv9tuEqVLpbFi/U3kDn0hEmeI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mN6OGzZz; arc=none smtp.client-ip=209.85.221.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mN6OGzZz" Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-3b792b0b829so706136f8f.3 for ; Thu, 31 Jul 2025 03:55:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959347; x=1754564147; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EV5At0v2MxwIlo7gcdYLnxR3OpnsqgCRXMq13jPhmuk=; b=mN6OGzZz9e774aQnhnI3QcqApopcQ1hx//hZb4opFu1R3gLCu1sh+1oKUddcEvhgfj 9nLNTzBJ0A7qz5AAE82Dg6usbVQYyyRhRzSHCOCpdmr6ZzjdqqqG5sc0OfMsrXI3Vi2X CSAwc8NFc+mNlogyy4fkHcviz6pObWiIQO1H5ju59QIQ+w4NqL/i2XdV/ogwcA/pDRtl 8S6PNeVMP9eZoJ+t1vaHKpzUuv1n1BFBx0bMgIz7/qy+NnvNSie6ezre4wxUfKSTpbOI xIO/R62kELzi5u3eIYVhX4zIAl4H/KG3ggnViFTb/Ajn/p/b+s60Xf3kxpl6TUh3Qdlv Bp7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959347; x=1754564147; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EV5At0v2MxwIlo7gcdYLnxR3OpnsqgCRXMq13jPhmuk=; b=BoJwKaTT4VIrrKIlh6oqr2PuZuRnShCW75jLyuj7zZcUixTbw1/lH70khkjwFQxhhA sfozwwXkxiVD+Nw+qtX3pSe0UQzVbJPYc+BaO+D8j2NheiUVg0rrJWYRbfP6W61yd7K4 0tUwSfuA5Q5UeQsYAJC0lg9dfbBwFRgxRzPhdSie2ZZhm4dpejxxaoXqn7IxLHHKe6vz CCA4gRn3T+jfTDmKdijhYoC8vtBNjIvxxmh/R1RUkM71PSn6HT2mzPIM0ZZG5hHykITP QIG/oMNT2fE5vwmDfSo+AdPktCmPwiKM2qS6ukzCIoeIFPocLnoMBeYy0MZnmYX6vgtY 8aiA== X-Gm-Message-State: AOJu0YwwJ7mKDPH4l5bX5ZiqA75WgzQ0dEUiAwKzCw39Lixt/6ZytYPy wIRInZVfVOETDPaz6s/lUOwHK/BxPMhbS133AK2Vpjj9Kd28eNuiLFIQ X-Gm-Gg: ASbGncuGx2qFgtbCJo3q4hgx5peXP0IQ5DdvsQ5ENPP100v6SkTciWVmGi5aksbIYbS EoLNDUUsfcYpcIAm/GprRnaE7eyiS8Hm3PXej7DIMBza2foKiGZzfQR37OVcr9z1lT93T8ufnik mLJDkXAGA0ZIfaMswWOb/3l5cL15S1BIx5MQWHHl7WV5ZwHjxFpfkd+Sg0gaccnUsu8ohjs2jEG guLQ/XVF4aYMJM1MhnbrH7omFT+f2aUFS2LiyV0fHZK7ClMOBECxjyEx6iTlJReeOefTe5yMh3d LawEy6H8Wbe+mVWSF60IQAB3v0nmbvDAyC+PEFiZEZTMv03KUPMz0vK/NfieunEkPnXEl96/ki1 WZ9xhHRJXHBPlaivwWgo2k/Fi6Zfr3g== X-Google-Smtp-Source: AGHT+IGDDa6RRleE5kYPN07mlH8Es4goEc4/iuBLPGtFq5ZYSEu4efaNzW+5pUtOabs8CvDN5zKcTw== X-Received: by 2002:a05:6000:2f85:b0:3a5:2694:d75f with SMTP id ffacd0b85a97d-3b79501df77mr5776287f8f.52.1753959346949; Thu, 31 Jul 2025 03:55:46 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:46 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 03/25] sched/deadline: Distinct between dl_rq and my_q Date: Thu, 31 Jul 2025 12:55:21 +0200 Message-ID: <20250731105543.40832-4-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Create two fields for runqueues in sched_dl_entity to make a distinction be= tween the global runqueue and the runqueue which the dl_server serves. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- include/linux/sched.h | 6 ++++-- kernel/sched/deadline.c | 11 +++++++---- kernel/sched/fair.c | 6 +++--- kernel/sched/sched.h | 3 ++- 4 files changed, 16 insertions(+), 10 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 40d2fa90df4..f0c8229afd1 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -724,12 +724,14 @@ struct sched_dl_entity { * Bits for DL-server functionality. Also see the comment near * dl_server_update(). * - * @rq the runqueue this server is for + * @dl_rq the runqueue on which this entity is (to be) queued + * @my_q the runqueue "owned" by this entity * * @server_has_tasks() returns true if @server_pick return a * runnable task. */ - struct rq *rq; + struct dl_rq *dl_rq; + struct rq *my_q; dl_server_has_tasks_f server_has_tasks; dl_server_pick_f server_pick_task; =20 diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 46b9b78cca2..73ca5c0a086 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -75,11 +75,12 @@ static inline struct rq *rq_of_dl_rq(struct dl_rq *dl_r= q) =20 static inline struct rq *rq_of_dl_se(struct sched_dl_entity *dl_se) { - struct rq *rq =3D dl_se->rq; + struct rq *rq; =20 if (!dl_server(dl_se)) rq =3D task_rq(dl_task_of(dl_se)); - + else + rq =3D container_of(dl_se->dl_rq, struct rq, dl); return rq; } =20 @@ -1641,11 +1642,13 @@ static bool dl_server_stopped(struct sched_dl_entit= y *dl_se) return false; } =20 -void dl_server_init(struct sched_dl_entity *dl_se, struct rq *rq, +void dl_server_init(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq, + struct rq *served_rq, dl_server_has_tasks_f has_tasks, dl_server_pick_f pick_task) { - dl_se->rq =3D rq; + dl_se->dl_rq =3D dl_rq; + dl_se->my_q =3D served_rq; dl_se->server_has_tasks =3D has_tasks; dl_se->server_pick_task =3D pick_task; } diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b173a059315..2723086538b 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8861,12 +8861,12 @@ static struct task_struct *__pick_next_task_fair(st= ruct rq *rq, struct task_stru =20 static bool fair_server_has_tasks(struct sched_dl_entity *dl_se) { - return !!dl_se->rq->cfs.nr_queued; + return !!dl_se->my_q->cfs.nr_queued; } =20 static struct task_struct *fair_server_pick_task(struct sched_dl_entity *d= l_se) { - return pick_task_fair(dl_se->rq); + return pick_task_fair(dl_se->my_q); } =20 void fair_server_init(struct rq *rq) @@ -8875,7 +8875,7 @@ void fair_server_init(struct rq *rq) =20 init_dl_entity(dl_se); =20 - dl_server_init(dl_se, rq, fair_server_has_tasks, fair_server_pick_task); + dl_server_init(dl_se, &rq->dl, rq, fair_server_has_tasks, fair_server_pic= k_task); } =20 /* diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 8719ab8a817..a8073d0824d 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -382,7 +382,8 @@ extern s64 dl_scaled_delta_exec(struct rq *rq, struct s= ched_dl_entity *dl_se, s6 extern void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec= ); extern void dl_server_start(struct sched_dl_entity *dl_se); extern void dl_server_stop(struct sched_dl_entity *dl_se); -extern void dl_server_init(struct sched_dl_entity *dl_se, struct rq *rq, +extern void dl_server_init(struct sched_dl_entity *dl_se, struct dl_rq *dl= _rq, + struct rq *served_rq, dl_server_has_tasks_f has_tasks, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91F512BDC2C for ; Thu, 31 Jul 2025 10:55:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959351; cv=none; b=rc4TZ4UwFVG/waM0AFCuWRJS9VgInx1oOEzYnR9i8cikorydfQU3mtgZvtUdnfmVV+6tmD9mCmf/9J9QPzPV9kS1ZbnPxVzZ/bagWD3qw5gRqGzaMRwpKm+koZA27IQsv8LPBI/Uioxes/j1BcyVbZCZVS+7ATOiHsI8C1jt1Mo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959351; c=relaxed/simple; bh=m9XIExoPBwNvEZE0FWjJD8V4F+TozE8xFK5348SptMg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X7qoTHttA0msqAWnA9E4pQbA3YhW2OUVJAeML50xIqlU1k7GgzWfuAXgu0P2UZiPftSQM+E19RJJhnhrCEQlDj6WmW6hRVEbfQ8EnDvth8KCb5WzFKk34LCfMC2E1NS3iqMVepY+podw2hV2eR9w3/ZFJAu/f8dAtLCZq0d2iJ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KExVs5X+; arc=none smtp.client-ip=209.85.128.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KExVs5X+" Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-45892deb246so1522315e9.2 for ; Thu, 31 Jul 2025 03:55:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959348; x=1754564148; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DswH3BpDRcb2kXY69HQrOvfA/HLUjfYY/+tBjVrNqnw=; b=KExVs5X+OQ4fyoRgrkuFkc60I6l1q4AVo1u5fYN6UyRoJt5ThY01BpSDLQ9o5fNJHk QEeBElSyqgM83SCQLLuwTVJuRp/2aL2xISGUf6xi7o8xXPqkhc+BtpX1CBqtL3EfMpMR HOPCjo4vHvr+IuXHFhhM0vf29lScs2HaFbCyfQr5fo67UB5+2MT4IYmdUrUJwZ+qyshp vOtrVxwrBn+yjbufWCI7fW5Y3/YH4RPOm7taDb/8OO8YemmmqMSzuYKzWGiq1TaJRSiK 8oro7bKKZu+C49g4Bnj4hkr8gFadSh+MQg9nKf17FdYwcACJg8bRqLOjpMm1c1+Mnxfi n+RA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959348; x=1754564148; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DswH3BpDRcb2kXY69HQrOvfA/HLUjfYY/+tBjVrNqnw=; b=F65a/THk7iqQanC218ffmAGLbaBkWfl3h9YNtAr58QbEKzOyZ3qGflHxuuL6rmqEno HDW1RwB8jkWtBNqfDIlTPdrhEbJYdkCf7nKamLRnIyPimvoWRZ61ARUfNTq4zvp3UGsZ U5KCBG2H3UaMnanCbSGLpp2zzEYRLhbCFxY1pjKrNMzt3oIXRirAc+mhnGFK3KbjrnA7 owqlwYQHJ7Flcnr7W4Np1nsgKku8nv8Voii/CdBxwztDls8ZUOvuysckJuxAiDRbIDNY pa7YjGSHbQkUBe+VSeCsjSLoNLN5GK6qgaYDo4VRD18FeqdrfsykJFZfJwG2tTGIIIWG f49w== X-Gm-Message-State: AOJu0YyxKm8DDKwZq3fydPDf+fcluAM2z1KQl1yhy2XlhwIpCz+rUPha MPyn9XLv+NaUWF2M6Jf6/4Srzu/bvH9Lp+uW7gW05mhN5cRFgLBjPVo1 X-Gm-Gg: ASbGncuoWQzlmaai1PYnzxW5z0rzhTvtmJawyw7KHhWXf+kwJOXskkx7rztXONYyocM bNmyeMn+8+FQgq6vWDLyheVaBIoXV+0DFDQXSP/2MBwS72qL6iX471teAI9UDegvTDJ0TTwGVZV AZrDulMBiS7X0ZCw9BnlBvqlphmXVLzOpEonNwWfOGaAveXHhxY8sE+2Az8ZC1hAWoJgRrdYU8z d5CXlxSIqfi1fv/IRR14GR+FV6POBjDHlnFmuwdxIr7Q1tCC1HnCcxSELc1HvzW9hOlxBi3PfSi wY9rVNspDCQvsNWtQCSrZgWRXAVmFYvFzn+tTfw1WMTRDsluHmXBAFtzOCsBJ2wa8/ZpL8p5mAB gXtPb0BaB6Nvi5PP5Pgc= X-Google-Smtp-Source: AGHT+IF4zckHj1HngwZI7aRLjnFn4G32ocR4jamL7mIqk+6zZVfquwfo2LNore8dtKeo4rQ3PmHGKA== X-Received: by 2002:a05:600c:4713:b0:456:2b2a:5f41 with SMTP id 5b1f17b1804b1-45892bdd2a9mr57753805e9.32.1753959347721; Thu, 31 Jul 2025 03:55:47 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:47 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 04/25] sched/rt: Pass an rt_rq instead of an rq where needed Date: Thu, 31 Jul 2025 12:55:22 +0200 Message-ID: <20250731105543.40832-5-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make rt.c code access the runqueue through the rt_rq data structure rather = than passing an rq pointer directly. This allows future patches to define rt_rq = data structures which do not refer only to the global runqueue, but also to local cgroup runqueues (rt_rq is not always equal to &rq->rt). Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 83 +++++++++++++++++++++++++---------------------- 1 file changed, 44 insertions(+), 39 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 7936d433373..945e3d705cc 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -370,9 +370,9 @@ static inline void rt_clear_overload(struct rq *rq) cpumask_clear_cpu(rq->cpu, rq->rd->rto_mask); } =20 -static inline int has_pushable_tasks(struct rq *rq) +static inline int has_pushable_tasks(struct rt_rq *rt_rq) { - return !plist_head_empty(&rq->rt.pushable_tasks); + return !plist_head_empty(&rt_rq->pushable_tasks); } =20 static DEFINE_PER_CPU(struct balance_callback, rt_push_head); @@ -383,7 +383,7 @@ static void pull_rt_task(struct rq *); =20 static inline void rt_queue_push_tasks(struct rq *rq) { - if (!has_pushable_tasks(rq)) + if (!has_pushable_tasks(&rq->rt)) return; =20 queue_balance_callback(rq, &per_cpu(rt_push_head, rq->cpu), push_rt_tasks= ); @@ -394,37 +394,37 @@ static inline void rt_queue_pull_task(struct rq *rq) queue_balance_callback(rq, &per_cpu(rt_pull_head, rq->cpu), pull_rt_task); } =20 -static void enqueue_pushable_task(struct rq *rq, struct task_struct *p) +static void enqueue_pushable_task(struct rt_rq *rt_rq, struct task_struct = *p) { - plist_del(&p->pushable_tasks, &rq->rt.pushable_tasks); + plist_del(&p->pushable_tasks, &rt_rq->pushable_tasks); plist_node_init(&p->pushable_tasks, p->prio); - plist_add(&p->pushable_tasks, &rq->rt.pushable_tasks); + plist_add(&p->pushable_tasks, &rt_rq->pushable_tasks); =20 /* Update the highest prio pushable task */ - if (p->prio < rq->rt.highest_prio.next) - rq->rt.highest_prio.next =3D p->prio; + if (p->prio < rt_rq->highest_prio.next) + rt_rq->highest_prio.next =3D p->prio; =20 - if (!rq->rt.overloaded) { - rt_set_overload(rq); - rq->rt.overloaded =3D 1; + if (!rt_rq->overloaded) { + rt_set_overload(rq_of_rt_rq(rt_rq)); + rt_rq->overloaded =3D 1; } } =20 -static void dequeue_pushable_task(struct rq *rq, struct task_struct *p) +static void dequeue_pushable_task(struct rt_rq *rt_rq, struct task_struct = *p) { - plist_del(&p->pushable_tasks, &rq->rt.pushable_tasks); + plist_del(&p->pushable_tasks, &rt_rq->pushable_tasks); =20 /* Update the new highest prio pushable task */ - if (has_pushable_tasks(rq)) { - p =3D plist_first_entry(&rq->rt.pushable_tasks, + if (has_pushable_tasks(rt_rq)) { + p =3D plist_first_entry(&rt_rq->pushable_tasks, struct task_struct, pushable_tasks); - rq->rt.highest_prio.next =3D p->prio; + rt_rq->highest_prio.next =3D p->prio; } else { - rq->rt.highest_prio.next =3D MAX_RT_PRIO-1; + rt_rq->highest_prio.next =3D MAX_RT_PRIO-1; =20 - if (rq->rt.overloaded) { - rt_clear_overload(rq); - rq->rt.overloaded =3D 0; + if (rt_rq->overloaded) { + rt_clear_overload(rq_of_rt_rq(rt_rq)); + rt_rq->overloaded =3D 0; } } } @@ -1431,6 +1431,7 @@ static void enqueue_task_rt(struct rq *rq, struct task_struct *p, int flags) { struct sched_rt_entity *rt_se =3D &p->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); =20 if (flags & ENQUEUE_WAKEUP) rt_se->timeout =3D 0; @@ -1444,17 +1445,18 @@ enqueue_task_rt(struct rq *rq, struct task_struct *= p, int flags) return; =20 if (!task_current(rq, p) && p->nr_cpus_allowed > 1) - enqueue_pushable_task(rq, p); + enqueue_pushable_task(rt_rq, p); } =20 static bool dequeue_task_rt(struct rq *rq, struct task_struct *p, int flag= s) { struct sched_rt_entity *rt_se =3D &p->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); =20 update_curr_rt(rq); dequeue_rt_entity(rt_se, flags); =20 - dequeue_pushable_task(rq, p); + dequeue_pushable_task(rt_rq, p); =20 return true; } @@ -1639,14 +1641,14 @@ static void wakeup_preempt_rt(struct rq *rq, struct= task_struct *p, int flags) static inline void set_next_task_rt(struct rq *rq, struct task_struct *p, = bool first) { struct sched_rt_entity *rt_se =3D &p->rt; - struct rt_rq *rt_rq =3D &rq->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); =20 p->se.exec_start =3D rq_clock_task(rq); if (on_rt_rq(&p->rt)) update_stats_wait_end_rt(rt_rq, rt_se); =20 /* The running task is never eligible for pushing */ - dequeue_pushable_task(rq, p); + dequeue_pushable_task(rt_rq, p); =20 if (!first) return; @@ -1710,7 +1712,7 @@ static struct task_struct *pick_task_rt(struct rq *rq) static void put_prev_task_rt(struct rq *rq, struct task_struct *p, struct = task_struct *next) { struct sched_rt_entity *rt_se =3D &p->rt; - struct rt_rq *rt_rq =3D &rq->rt; + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); =20 if (on_rt_rq(&p->rt)) update_stats_wait_start_rt(rt_rq, rt_se); @@ -1726,7 +1728,7 @@ static void put_prev_task_rt(struct rq *rq, struct ta= sk_struct *p, struct task_s * if it is still active */ if (on_rt_rq(&p->rt) && p->nr_cpus_allowed > 1) - enqueue_pushable_task(rq, p); + enqueue_pushable_task(rt_rq, p); } =20 /* Only try algorithms three times */ @@ -1736,16 +1738,16 @@ static void put_prev_task_rt(struct rq *rq, struct = task_struct *p, struct task_s * Return the highest pushable rq's task, which is suitable to be executed * on the CPU, NULL otherwise */ -static struct task_struct *pick_highest_pushable_task(struct rq *rq, int c= pu) +static struct task_struct *pick_highest_pushable_task(struct rt_rq *rt_rq,= int cpu) { - struct plist_head *head =3D &rq->rt.pushable_tasks; + struct plist_head *head =3D &rt_rq->pushable_tasks; struct task_struct *p; =20 - if (!has_pushable_tasks(rq)) + if (!has_pushable_tasks(rt_rq)) return NULL; =20 plist_for_each_entry(p, head, pushable_tasks) { - if (task_is_pushable(rq, p, cpu)) + if (task_is_pushable(rq_of_rt_rq(rt_rq), p, cpu)) return p; } =20 @@ -1845,14 +1847,15 @@ static int find_lowest_rq(struct task_struct *task) return -1; } =20 -static struct task_struct *pick_next_pushable_task(struct rq *rq) +static struct task_struct *pick_next_pushable_task(struct rt_rq *rt_rq) { + struct rq *rq =3D rq_of_rt_rq(rt_rq); struct task_struct *p; =20 - if (!has_pushable_tasks(rq)) + if (!has_pushable_tasks(rt_rq)) return NULL; =20 - p =3D plist_first_entry(&rq->rt.pushable_tasks, + p =3D plist_first_entry(&rt_rq->pushable_tasks, struct task_struct, pushable_tasks); =20 BUG_ON(rq->cpu !=3D task_cpu(p)); @@ -1905,7 +1908,7 @@ static struct rq *find_lock_lowest_rq(struct task_str= uct *task, struct rq *rq) */ if (unlikely(is_migration_disabled(task) || !cpumask_test_cpu(lowest_rq->cpu, &task->cpus_mask) || - task !=3D pick_next_pushable_task(rq))) { + task !=3D pick_next_pushable_task(&rq->rt))) { =20 double_unlock_balance(rq, lowest_rq); lowest_rq =3D NULL; @@ -1939,7 +1942,7 @@ static int push_rt_task(struct rq *rq, bool pull) if (!rq->rt.overloaded) return 0; =20 - next_task =3D pick_next_pushable_task(rq); + next_task =3D pick_next_pushable_task(&rq->rt); if (!next_task) return 0; =20 @@ -2014,7 +2017,7 @@ static int push_rt_task(struct rq *rq, bool pull) * run-queue and is also still the next task eligible for * pushing. */ - task =3D pick_next_pushable_task(rq); + task =3D pick_next_pushable_task(&rq->rt); if (task =3D=3D next_task) { /* * The task hasn't migrated, and is still the next @@ -2202,7 +2205,7 @@ void rto_push_irq_work_func(struct irq_work *work) * We do not need to grab the lock to check for has_pushable_tasks. * When it gets updated, a check is made if a push is possible. */ - if (has_pushable_tasks(rq)) { + if (has_pushable_tasks(&rq->rt)) { raw_spin_rq_lock(rq); while (push_rt_task(rq, true)) ; @@ -2231,6 +2234,7 @@ static void pull_rt_task(struct rq *this_rq) int this_cpu =3D this_rq->cpu, cpu; bool resched =3D false; struct task_struct *p, *push_task; + struct rt_rq *src_rt_rq; struct rq *src_rq; int rt_overload_count =3D rt_overloaded(this_rq); =20 @@ -2260,6 +2264,7 @@ static void pull_rt_task(struct rq *this_rq) continue; =20 src_rq =3D cpu_rq(cpu); + src_rt_rq =3D &src_rq->rt; =20 /* * Don't bother taking the src_rq->lock if the next highest @@ -2268,7 +2273,7 @@ static void pull_rt_task(struct rq *this_rq) * logically higher, the src_rq will push this task away. * And if its going logically lower, we do not care */ - if (src_rq->rt.highest_prio.next >=3D + if (src_rt_rq->highest_prio.next >=3D this_rq->rt.highest_prio.curr) continue; =20 @@ -2284,7 +2289,7 @@ static void pull_rt_task(struct rq *this_rq) * We can pull only a task, which is pushable * on its rq, and no others. */ - p =3D pick_highest_pushable_task(src_rq, this_cpu); + p =3D pick_highest_pushable_task(src_rt_rq, this_cpu); =20 /* * Do we have an RT task that preempts --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3DFEF2BE03C for ; Thu, 31 Jul 2025 10:55:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959351; cv=none; b=RLnfTNe/0l75108HjBt9P0bwFEIUDoEe6YgAPwr22IRzoXWpvqsExy5S6v1U9xFMm1pIYyplr0a8qhXWrGMqmemZMTteBatvgUl/nKlGMXaScr0dODA++u+Vl5rR23moEYwcY0rnDx913XQNgM1yAkSb2KM4RPP6jT3UnZrQQxQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959351; c=relaxed/simple; bh=1h1hYV15b/9OrZx9rgbJfHxJeDSJJGbmTMQBxgv3xnA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IY0zpttKK05C4Pi5KZHGrd9XaGRyA3Xgog1fQmyoGkBjuK0B8ue1klGHQBdOOGgcWv382yTTRGW+2CzNi2AtqZnR8i+zj1FsJ6wfeRKnLxIg9vDGbbPkKg6L+fb2e9O0209XRwmC7zaqP48DMooWpJvv+RD86YIf18otaKOdnzA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OUKakwI1; arc=none smtp.client-ip=209.85.221.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OUKakwI1" Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-3b7892609a5so502344f8f.1 for ; Thu, 31 Jul 2025 03:55:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959348; x=1754564148; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RuQIWstDhNu35nB45LKa/AekW382Kc6x1gcIP9pJGZE=; b=OUKakwI1sFhtQBIELJo0nU4Hk0Yf2WJnXrLnKKDuHRfYbz4zBjfOz0Jb9M24EcPwmv QPd8XNRZLYRdr8ySaxQ8xjb3ZJuTmRTH10H0duJzFniDZ7ck4fsRE3vjmMVd3cExVxEf tqZisa7cUxrtVXHkqShRMRxMClKJGfz89g2++X+EpAEjCyejmxAWCOCyZMWc/vRoxZM3 SmPQUuFx++BTg9mGOu17j8i75ZIKq9FIguX8JQOkPM66emur1nE5TJeWruhQLBMEBNCu NocF/QsqXxdfiZeeYobh0PoVVz/RX8JuJWzSxqOqLG65nHAAbUm+NIfY8jjdo2QDHm+G YobQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959348; x=1754564148; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RuQIWstDhNu35nB45LKa/AekW382Kc6x1gcIP9pJGZE=; b=F/EXAIwS7Miwt5nP0PeCN7RuPa30Y5Nt/lGKmoO6JlxQK8I91XJI4gxX/ok/b+ZJJY 1uqR0Gn3mriOkLnPL04PQ9WiaMDDfkfv0r7LTfWyNl3svPGoPRcMXKo1bTkSo/+jXNg6 SWG2BMf9XwCn6zQN6eEEMKCrtMzoILkGcqC9z5mm5m0kavaFzDYIu5GUO2BeT4a/hjp6 7VzWBZlsexTmxFH6fk8GB89Y6gc6/p5nsF0qHoTiGbxjZV/xvCsKuy+4+khJg2T50CAz zrHyfwXGJLGEo27DpeEA1Kxcb8NeIsj7xsd3ZprhWZ+/1G81m+gtZW07tdB2CoaBdCMN 59+g== X-Gm-Message-State: AOJu0Yw18jWy9X3+pWwtUBYlbwJsHsSrr3tC3TXcBbWVxNJgGUfQT6dU WzE2/SYhiKXWAlzuQtftipqIIFcODNhnLaBbPdEfUuU/vyFl+t3Rl9bV X-Gm-Gg: ASbGncsTQs3FfQ5jPLj9viNrucv9FgStzaubHhx6doleptqUi1/Z870D9vLybBhvXtz O6R32/CI+A/U5/PeSv022Tq7WYTVJwOiBY33cwDbva0/ogcyfhs+TKXP5lBnVTdaX91s/nKfV8P OQFXPSy3iJV6c+Bf8gjIJ1+aR8dpuyqFEL59lTZSCMmtXffV6nzNcYhhLsHp+itWtcpecXEh1dh PF1wf/b/r2qzhFi4t/ltdjI+yArVfkUq7Mb5P8XShGMWIq6JBwtdS45Giwn1jl2iJDwJPERv54i P/XKRlHq4J3MiKGZ41TzYIw9bsQ+zv/bkMKo4QmvSGfvF/RVikiszKt1XAnhQkWpIh0oLMSNCyJ ljnQOVttRGOl+qW/HslAhkk03JQQ0hg== X-Google-Smtp-Source: AGHT+IGhoH+yhFWiOeEr25sQ0DFAA5hiD1OLZM9f1MXEwhtU5DDI6Ah6hI2Qk76ptoPCOmdznbwIZQ== X-Received: by 2002:a05:6000:240b:b0:3b7:b3f2:f8c3 with SMTP id ffacd0b85a97d-3b7b3f2fd1fmr1227787f8f.57.1753959348495; Thu, 31 Jul 2025 03:55:48 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:48 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 05/25] sched/rt: Move some functions from rt.c to sched.h Date: Thu, 31 Jul 2025 12:55:23 +0200 Message-ID: <20250731105543.40832-6-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make the following functions be non-static and move them in sched.h, so that they can be used also in other source files: - rt_task_of() - rq_of_rt_rq() - rt_rq_of_se() - rq_of_rt_se() There are no functional changes. This is needed by future patches. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/rt.c | 52 -------------------------------------------- kernel/sched/sched.h | 51 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 52 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 945e3d705cc..3ea92b08a0e 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -168,34 +168,6 @@ static void destroy_rt_bandwidth(struct rt_bandwidth *= rt_b) =20 #define rt_entity_is_task(rt_se) (!(rt_se)->my_q) =20 -static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) -{ - WARN_ON_ONCE(!rt_entity_is_task(rt_se)); - - return container_of(rt_se, struct task_struct, rt); -} - -static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) -{ - /* Cannot fold with non-CONFIG_RT_GROUP_SCHED version, layout */ - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); - return rt_rq->rq; -} - -static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) -{ - WARN_ON(!rt_group_sched_enabled() && rt_se->rt_rq->tg !=3D &root_task_gro= up); - return rt_se->rt_rq; -} - -static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) -{ - struct rt_rq *rt_rq =3D rt_se->rt_rq; - - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); - return rt_rq->rq; -} - void unregister_rt_sched_group(struct task_group *tg) { if (!rt_group_sched_enabled()) @@ -296,30 +268,6 @@ int alloc_rt_sched_group(struct task_group *tg, struct= task_group *parent) =20 #define rt_entity_is_task(rt_se) (1) =20 -static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) -{ - return container_of(rt_se, struct task_struct, rt); -} - -static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) -{ - return container_of(rt_rq, struct rq, rt); -} - -static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) -{ - struct task_struct *p =3D rt_task_of(rt_se); - - return task_rq(p); -} - -static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) -{ - struct rq *rq =3D rq_of_rt_se(rt_se); - - return &rq->rt; -} - void unregister_rt_sched_group(struct task_group *tg) { } =20 void free_rt_sched_group(struct task_group *tg) { } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index a8073d0824d..a1e6d2852ca 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3042,6 +3042,57 @@ extern void set_rq_offline(struct rq *rq); =20 extern bool sched_smp_initialized; =20 +#ifdef CONFIG_RT_GROUP_SCHED +static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) +{ +#ifdef CONFIG_SCHED_DEBUG + WARN_ON_ONCE(rt_se->my_q); +#endif + return container_of(rt_se, struct task_struct, rt); +} + +static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) +{ + return rt_rq->rq; +} + +static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) +{ + return rt_se->rt_rq; +} + +static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) +{ + struct rt_rq *rt_rq =3D rt_se->rt_rq; + + return rt_rq->rq; +} +#else +static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) +{ + return container_of(rt_se, struct task_struct, rt); +} + +static inline struct rq *rq_of_rt_rq(struct rt_rq *rt_rq) +{ + return container_of(rt_rq, struct rq, rt); +} + +static inline struct rq *rq_of_rt_se(struct sched_rt_entity *rt_se) +{ + struct task_struct *p =3D rt_task_of(rt_se); + + return task_rq(p); +} + +static inline struct rt_rq *rt_rq_of_se(struct sched_rt_entity *rt_se) +{ + struct rq *rq =3D rq_of_rt_se(rt_se); + + return &rq->rt; +} +#endif + DEFINE_LOCK_GUARD_2(double_rq_lock, struct rq, double_rq_lock(_T->lock, _T->lock2), double_rq_unlock(_T->lock, _T->lock2)) --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F1BB2BE640 for ; Thu, 31 Jul 2025 10:55:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959354; cv=none; b=JCYwio9IPWeemxsSQC87CjvTAKaPb29vOQc0virNgO6wRm7H82qliGpkxnPEtOFiO/fij2OZQPXElDbM7GEWPGBP0EBsafPtp4iElNc7sPBTw1BJaXbPJk1DGu7jDdtobypujTTzfQM18s8gFbUbCYvzR/nkQMxDO3b2lML6R30= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959354; c=relaxed/simple; bh=NyeFCyZWpNROaZ3UYXVHKZ5Ws4sInZfOFprOudRsDS8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FhFam+TcWE5SsZlzDjwbGpnlg0DnLN8BCF3GLB8rHkoxcup0z6PeXXsVTiq88qsgdbuFWfM9cKvJ8oHf+ZkSOA5JGH5eAisT6xCksHGlZ+4xKEYrrkJkppVk/NbmPKJQJtyGjDaLcGSoUqegsBguTMw72jJWK0eWu6OFY4eh7lQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JEpnI7uh; arc=none smtp.client-ip=209.85.221.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JEpnI7uh" Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-3b7823559a5so488460f8f.0 for ; Thu, 31 Jul 2025 03:55:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959350; x=1754564150; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dYw/d9I482wc8oNxK/Xqfg9Pk68BEa7RMv7CXnvYchk=; b=JEpnI7uhwxE1iLQzqqY/4i94B6or5SqCP+qEvdCRTuD8mWICz7IplfxTKZ8gxpnE42 Os32HptSUUEl8Uuc/oiJYHRMU8umbKg1IAvqjzjW/8EqFbIGPYocXLCHExZeIucWrGKt oU5JpCrzuwtTIY5iXK9wHLONHXk6txP6jzRUNj3mV5eiPI5KZmfd89R5tNntmOGpVJjc lT+D4OWYEcZC78unimK/FFwrc5rvLvI/Bb6sWpvDwgXxNNNwoeR9JNVINiLvovmf87+C DsrS8BbQcS0D1aDu6pn7mLzaUkw2XfwPaRpKd5xHagNPWbCtmOvA6WQI+enFfbLjOgAZ 4qAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959350; x=1754564150; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dYw/d9I482wc8oNxK/Xqfg9Pk68BEa7RMv7CXnvYchk=; b=kDEwu067SWrJg+HXgcU+YhlPSmMh0HWp4to1wZ3TYG+1dKGVbntH13u/0hQetQLS81 Omuhp3PLp96MejGRb0cfUaIVJqOllbRT2GWmHMsUnMro3Dn8je36wknz9edUkCvbsrQk CGiODsKj+aJtqcLhdEtMY1uEaoWiBjF6DiWw7PykByCN5WMjXiNYI9SW7g5QZOeOXCXU gbJlBcV3kfHv46gx8zTYJnP6+KBQBa14qFzUUtw9O9gs4ydVf/TmFKPN7zxPokXERgU9 SAhZM3xAsqAWz9eAkWD4OfN8PSWqIH2jhPv6sjEMnUvtncCNoLzqJU1kQuUpP9gtTMQH UoZQ== X-Gm-Message-State: AOJu0YxwcOS05Ki2DSCyC2bvsF2pQIP+Vt5rwapheDvYb1eDUGzbQpKx FXChAPw6BO1N0TFw1Cx5kVOBiWl9IKt2Mhh3u5cCAZj0YqUwkqcVgB6z X-Gm-Gg: ASbGncup2APzCy0oOw0WT1ZuY48ZmjX+fMyfanP86Fst7q2gzIq9d3DjPwcWGSRH6Ym ATBShQG+YxNX52M6YD2oBxgZKZdWbeNfGxMyTyozZCrsJJucEzNiOJPjliVfPSZPQOVNHBr8Fnh LlHV8se1Y8tggw6RM+NJpF20VYU0mgkmX+KHUtvq5si5DfCh8uQIOR2Q+qMS8CxcjopzslVCbPe t6j84Ek2cLn3pmVdh0oi1kMk3My8TlL8kll12WwwseRwwTMj5naUnOY3r4yz+FPg4W3vMjcmv32 XCCzmMj5TW8nxBfqtAV//cft0araP7Ysjb1XN1pLmvfbH61rMAuw4caG8gJDWIQOxOIzQz0kd52 aPllaXOCpsfh74Z0Q0+s= X-Google-Smtp-Source: AGHT+IEaFWhFQZnIux20vxGqCQ3YcxhCSiNObxQKiK4NoE7xrdmunYH9Vd6R6uBVwxwnuIsS7cQvkA== X-Received: by 2002:a05:6000:290a:b0:3b8:d08c:cde5 with SMTP id ffacd0b85a97d-3b8d08cd0e9mr264258f8f.43.1753959349356; Thu, 31 Jul 2025 03:55:49 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:49 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 06/25] sched/rt: Disable RT_GROUP_SCHED Date: Thu, 31 Jul 2025 12:55:24 +0200 Message-ID: <20250731105543.40832-7-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Disable the old RT_GROUP_SCHED scheduler. Note that this does not completely remove all the RT_GROUP_SCHED functionality, just unhooks it and removes mo= st of the relevant functions. Some of the RT_GROUP_SCHED functions are kept becau= se they will be adapted for the HCBS scheduling. Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 6 - kernel/sched/deadline.c | 34 -- kernel/sched/debug.c | 6 - kernel/sched/rt.c | 848 ++-------------------------------------- kernel/sched/sched.h | 9 +- kernel/sched/syscalls.c | 13 - 6 files changed, 26 insertions(+), 890 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 3ec00d08d46..42587a3c71f 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8714,11 +8714,6 @@ void __init sched_init(void) =20 init_defrootdomain(); =20 -#ifdef CONFIG_RT_GROUP_SCHED - init_rt_bandwidth(&root_task_group.rt_bandwidth, - global_rt_period(), global_rt_runtime()); -#endif /* CONFIG_RT_GROUP_SCHED */ - #ifdef CONFIG_CGROUP_SCHED task_group_cache =3D KMEM_CACHE(task_group, 0); =20 @@ -8770,7 +8765,6 @@ void __init sched_init(void) * starts working after scheduler_running, which is not the case * yet. */ - rq->rt.rt_runtime =3D global_rt_runtime(); init_tg_rt_entry(&root_task_group, &rq->rt, NULL, i, NULL); #endif rq->sd =3D NULL; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 73ca5c0a086..0640d0ca45b 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1524,40 +1524,6 @@ static void update_curr_dl_se(struct rq *rq, struct = sched_dl_entity *dl_se, s64 if (!is_leftmost(dl_se, &rq->dl)) resched_curr(rq); } - - /* - * The fair server (sole dl_server) does not account for real-time - * workload because it is running fair work. - */ - if (dl_se =3D=3D &rq->fair_server) - return; - -#ifdef CONFIG_RT_GROUP_SCHED - /* - * Because -- for now -- we share the rt bandwidth, we need to - * account our runtime there too, otherwise actual rt tasks - * would be able to exceed the shared quota. - * - * Account to the root rt group for now. - * - * The solution we're working towards is having the RT groups scheduled - * using deadline servers -- however there's a few nasties to figure - * out before that can happen. - */ - if (rt_bandwidth_enabled()) { - struct rt_rq *rt_rq =3D &rq->rt; - - raw_spin_lock(&rt_rq->rt_runtime_lock); - /* - * We'll let actual RT tasks worry about the overflow here, we - * have our own CBS to keep us inline; only account when RT - * bandwidth is relevant. - */ - if (sched_rt_bandwidth_account(rt_rq)) - rt_rq->rt_time +=3D delta_exec; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - } -#endif /* CONFIG_RT_GROUP_SCHED */ } =20 /* diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 3f06ab84d53..f05decde708 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -892,12 +892,6 @@ void print_rt_rq(struct seq_file *m, int cpu, struct r= t_rq *rt_rq) =20 PU(rt_nr_running); =20 -#ifdef CONFIG_RT_GROUP_SCHED - P(rt_throttled); - PN(rt_time); - PN(rt_runtime); -#endif - #undef PN #undef PU #undef P diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 3ea92b08a0e..a6282784978 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,3 +1,4 @@ +#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR @@ -82,117 +83,18 @@ void init_rt_rq(struct rt_rq *rt_rq) rt_rq->highest_prio.next =3D MAX_RT_PRIO-1; rt_rq->overloaded =3D 0; plist_head_init(&rt_rq->pushable_tasks); - /* We start is dequeued state, because no RT tasks are queued */ - rt_rq->rt_queued =3D 0; - -#ifdef CONFIG_RT_GROUP_SCHED - rt_rq->rt_time =3D 0; - rt_rq->rt_throttled =3D 0; - rt_rq->rt_runtime =3D 0; - raw_spin_lock_init(&rt_rq->rt_runtime_lock); - rt_rq->tg =3D &root_task_group; -#endif } =20 #ifdef CONFIG_RT_GROUP_SCHED =20 -static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun= ); - -static enum hrtimer_restart sched_rt_period_timer(struct hrtimer *timer) -{ - struct rt_bandwidth *rt_b =3D - container_of(timer, struct rt_bandwidth, rt_period_timer); - int idle =3D 0; - int overrun; - - raw_spin_lock(&rt_b->rt_runtime_lock); - for (;;) { - overrun =3D hrtimer_forward_now(timer, rt_b->rt_period); - if (!overrun) - break; - - raw_spin_unlock(&rt_b->rt_runtime_lock); - idle =3D do_sched_rt_period_timer(rt_b, overrun); - raw_spin_lock(&rt_b->rt_runtime_lock); - } - if (idle) - rt_b->rt_period_active =3D 0; - raw_spin_unlock(&rt_b->rt_runtime_lock); - - return idle ? HRTIMER_NORESTART : HRTIMER_RESTART; -} - -void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime) -{ - rt_b->rt_period =3D ns_to_ktime(period); - rt_b->rt_runtime =3D runtime; - - raw_spin_lock_init(&rt_b->rt_runtime_lock); - - hrtimer_setup(&rt_b->rt_period_timer, sched_rt_period_timer, CLOCK_MONOTO= NIC, - HRTIMER_MODE_REL_HARD); -} - -static inline void do_start_rt_bandwidth(struct rt_bandwidth *rt_b) -{ - raw_spin_lock(&rt_b->rt_runtime_lock); - if (!rt_b->rt_period_active) { - rt_b->rt_period_active =3D 1; - /* - * SCHED_DEADLINE updates the bandwidth, as a run away - * RT task with a DL task could hog a CPU. But DL does - * not reset the period. If a deadline task was running - * without an RT task running, it can cause RT tasks to - * throttle when they start up. Kick the timer right away - * to update the period. - */ - hrtimer_forward_now(&rt_b->rt_period_timer, ns_to_ktime(0)); - hrtimer_start_expires(&rt_b->rt_period_timer, - HRTIMER_MODE_ABS_PINNED_HARD); - } - raw_spin_unlock(&rt_b->rt_runtime_lock); -} - -static void start_rt_bandwidth(struct rt_bandwidth *rt_b) -{ - if (!rt_bandwidth_enabled() || rt_b->rt_runtime =3D=3D RUNTIME_INF) - return; - - do_start_rt_bandwidth(rt_b); -} - -static void destroy_rt_bandwidth(struct rt_bandwidth *rt_b) -{ - hrtimer_cancel(&rt_b->rt_period_timer); -} - -#define rt_entity_is_task(rt_se) (!(rt_se)->my_q) - void unregister_rt_sched_group(struct task_group *tg) { - if (!rt_group_sched_enabled()) - return; - - if (tg->rt_se) - destroy_rt_bandwidth(&tg->rt_bandwidth); } =20 void free_rt_sched_group(struct task_group *tg) { - int i; - if (!rt_group_sched_enabled()) return; - - for_each_possible_cpu(i) { - if (tg->rt_rq) - kfree(tg->rt_rq[i]); - if (tg->rt_se) - kfree(tg->rt_se[i]); - } - - kfree(tg->rt_rq); - kfree(tg->rt_se); } =20 void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq, @@ -202,72 +104,23 @@ void init_tg_rt_entry(struct task_group *tg, struct r= t_rq *rt_rq, struct rq *rq =3D cpu_rq(cpu); =20 rt_rq->highest_prio.curr =3D MAX_RT_PRIO-1; - rt_rq->rt_nr_boosted =3D 0; rt_rq->rq =3D rq; rt_rq->tg =3D tg; =20 tg->rt_rq[cpu] =3D rt_rq; tg->rt_se[cpu] =3D rt_se; - - if (!rt_se) - return; - - if (!parent) - rt_se->rt_rq =3D &rq->rt; - else - rt_se->rt_rq =3D parent->my_q; - - rt_se->my_q =3D rt_rq; - rt_se->parent =3D parent; - INIT_LIST_HEAD(&rt_se->run_list); } =20 int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) { - struct rt_rq *rt_rq; - struct sched_rt_entity *rt_se; - int i; - if (!rt_group_sched_enabled()) return 1; =20 - tg->rt_rq =3D kcalloc(nr_cpu_ids, sizeof(rt_rq), GFP_KERNEL); - if (!tg->rt_rq) - goto err; - tg->rt_se =3D kcalloc(nr_cpu_ids, sizeof(rt_se), GFP_KERNEL); - if (!tg->rt_se) - goto err; - - init_rt_bandwidth(&tg->rt_bandwidth, ktime_to_ns(global_rt_period()), 0); - - for_each_possible_cpu(i) { - rt_rq =3D kzalloc_node(sizeof(struct rt_rq), - GFP_KERNEL, cpu_to_node(i)); - if (!rt_rq) - goto err; - - rt_se =3D kzalloc_node(sizeof(struct sched_rt_entity), - GFP_KERNEL, cpu_to_node(i)); - if (!rt_se) - goto err_free_rq; - - init_rt_rq(rt_rq); - rt_rq->rt_runtime =3D tg->rt_bandwidth.rt_runtime; - init_tg_rt_entry(tg, rt_rq, rt_se, i, parent->rt_se[i]); - } - return 1; - -err_free_rq: - kfree(rt_rq); -err: - return 0; } =20 #else /* !CONFIG_RT_GROUP_SCHED: */ =20 -#define rt_entity_is_task(rt_se) (1) - void unregister_rt_sched_group(struct task_group *tg) { } =20 void free_rt_sched_group(struct task_group *tg) { } @@ -377,9 +230,6 @@ static void dequeue_pushable_task(struct rt_rq *rt_rq, = struct task_struct *p) } } =20 -static void enqueue_top_rt_rq(struct rt_rq *rt_rq); -static void dequeue_top_rt_rq(struct rt_rq *rt_rq, unsigned int count); - static inline int on_rt_rq(struct sched_rt_entity *rt_se) { return rt_se->on_rq; @@ -426,16 +276,6 @@ static inline bool rt_task_fits_capacity(struct task_s= truct *p, int cpu) =20 #ifdef CONFIG_RT_GROUP_SCHED =20 -static inline u64 sched_rt_runtime(struct rt_rq *rt_rq) -{ - return rt_rq->rt_runtime; -} - -static inline u64 sched_rt_period(struct rt_rq *rt_rq) -{ - return ktime_to_ns(rt_rq->tg->rt_bandwidth.rt_period); -} - typedef struct task_group *rt_rq_iter_t; =20 static inline struct task_group *next_task_group(struct task_group *tg) @@ -461,457 +301,20 @@ static inline struct task_group *next_task_group(str= uct task_group *tg) iter && (rt_rq =3D iter->rt_rq[cpu_of(rq)]); \ iter =3D next_task_group(iter)) =20 -#define for_each_sched_rt_entity(rt_se) \ - for (; rt_se; rt_se =3D rt_se->parent) - -static inline struct rt_rq *group_rt_rq(struct sched_rt_entity *rt_se) -{ - return rt_se->my_q; -} - static void enqueue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags); static void dequeue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags); =20 -static void sched_rt_rq_enqueue(struct rt_rq *rt_rq) -{ - struct task_struct *donor =3D rq_of_rt_rq(rt_rq)->donor; - struct rq *rq =3D rq_of_rt_rq(rt_rq); - struct sched_rt_entity *rt_se; - - int cpu =3D cpu_of(rq); - - rt_se =3D rt_rq->tg->rt_se[cpu]; - - if (rt_rq->rt_nr_running) { - if (!rt_se) - enqueue_top_rt_rq(rt_rq); - else if (!on_rt_rq(rt_se)) - enqueue_rt_entity(rt_se, 0); - - if (rt_rq->highest_prio.curr < donor->prio) - resched_curr(rq); - } -} - -static void sched_rt_rq_dequeue(struct rt_rq *rt_rq) -{ - struct sched_rt_entity *rt_se; - int cpu =3D cpu_of(rq_of_rt_rq(rt_rq)); - - rt_se =3D rt_rq->tg->rt_se[cpu]; - - if (!rt_se) { - dequeue_top_rt_rq(rt_rq, rt_rq->rt_nr_running); - /* Kick cpufreq (see the comment in kernel/sched/sched.h). */ - cpufreq_update_util(rq_of_rt_rq(rt_rq), 0); - } - else if (on_rt_rq(rt_se)) - dequeue_rt_entity(rt_se, 0); -} - -static inline int rt_rq_throttled(struct rt_rq *rt_rq) -{ - return rt_rq->rt_throttled && !rt_rq->rt_nr_boosted; -} - -static int rt_se_boosted(struct sched_rt_entity *rt_se) -{ - struct rt_rq *rt_rq =3D group_rt_rq(rt_se); - struct task_struct *p; - - if (rt_rq) - return !!rt_rq->rt_nr_boosted; - - p =3D rt_task_of(rt_se); - return p->prio !=3D p->normal_prio; -} - -static inline const struct cpumask *sched_rt_period_mask(void) -{ - return this_rq()->rd->span; -} - -static inline -struct rt_rq *sched_rt_period_rt_rq(struct rt_bandwidth *rt_b, int cpu) -{ - return container_of(rt_b, struct task_group, rt_bandwidth)->rt_rq[cpu]; -} - -static inline struct rt_bandwidth *sched_rt_bandwidth(struct rt_rq *rt_rq) -{ - return &rt_rq->tg->rt_bandwidth; -} - -bool sched_rt_bandwidth_account(struct rt_rq *rt_rq) -{ - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - - return (hrtimer_active(&rt_b->rt_period_timer) || - rt_rq->rt_time < rt_b->rt_runtime); -} - -/* - * We ran out of runtime, see if we can borrow some from our neighbours. - */ -static void do_balance_runtime(struct rt_rq *rt_rq) -{ - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - struct root_domain *rd =3D rq_of_rt_rq(rt_rq)->rd; - int i, weight; - u64 rt_period; - - weight =3D cpumask_weight(rd->span); - - raw_spin_lock(&rt_b->rt_runtime_lock); - rt_period =3D ktime_to_ns(rt_b->rt_period); - for_each_cpu(i, rd->span) { - struct rt_rq *iter =3D sched_rt_period_rt_rq(rt_b, i); - s64 diff; - - if (iter =3D=3D rt_rq) - continue; - - raw_spin_lock(&iter->rt_runtime_lock); - /* - * Either all rqs have inf runtime and there's nothing to steal - * or __disable_runtime() below sets a specific rq to inf to - * indicate its been disabled and disallow stealing. - */ - if (iter->rt_runtime =3D=3D RUNTIME_INF) - goto next; - - /* - * From runqueues with spare time, take 1/n part of their - * spare time, but no more than our period. - */ - diff =3D iter->rt_runtime - iter->rt_time; - if (diff > 0) { - diff =3D div_u64((u64)diff, weight); - if (rt_rq->rt_runtime + diff > rt_period) - diff =3D rt_period - rt_rq->rt_runtime; - iter->rt_runtime -=3D diff; - rt_rq->rt_runtime +=3D diff; - if (rt_rq->rt_runtime =3D=3D rt_period) { - raw_spin_unlock(&iter->rt_runtime_lock); - break; - } - } -next: - raw_spin_unlock(&iter->rt_runtime_lock); - } - raw_spin_unlock(&rt_b->rt_runtime_lock); -} - -/* - * Ensure this RQ takes back all the runtime it lend to its neighbours. - */ -static void __disable_runtime(struct rq *rq) -{ - struct root_domain *rd =3D rq->rd; - rt_rq_iter_t iter; - struct rt_rq *rt_rq; - - if (unlikely(!scheduler_running)) - return; - - for_each_rt_rq(rt_rq, iter, rq) { - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - s64 want; - int i; - - raw_spin_lock(&rt_b->rt_runtime_lock); - raw_spin_lock(&rt_rq->rt_runtime_lock); - /* - * Either we're all inf and nobody needs to borrow, or we're - * already disabled and thus have nothing to do, or we have - * exactly the right amount of runtime to take out. - */ - if (rt_rq->rt_runtime =3D=3D RUNTIME_INF || - rt_rq->rt_runtime =3D=3D rt_b->rt_runtime) - goto balanced; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - - /* - * Calculate the difference between what we started out with - * and what we current have, that's the amount of runtime - * we lend and now have to reclaim. - */ - want =3D rt_b->rt_runtime - rt_rq->rt_runtime; - - /* - * Greedy reclaim, take back as much as we can. - */ - for_each_cpu(i, rd->span) { - struct rt_rq *iter =3D sched_rt_period_rt_rq(rt_b, i); - s64 diff; - - /* - * Can't reclaim from ourselves or disabled runqueues. - */ - if (iter =3D=3D rt_rq || iter->rt_runtime =3D=3D RUNTIME_INF) - continue; - - raw_spin_lock(&iter->rt_runtime_lock); - if (want > 0) { - diff =3D min_t(s64, iter->rt_runtime, want); - iter->rt_runtime -=3D diff; - want -=3D diff; - } else { - iter->rt_runtime -=3D want; - want -=3D want; - } - raw_spin_unlock(&iter->rt_runtime_lock); - - if (!want) - break; - } - - raw_spin_lock(&rt_rq->rt_runtime_lock); - /* - * We cannot be left wanting - that would mean some runtime - * leaked out of the system. - */ - WARN_ON_ONCE(want); -balanced: - /* - * Disable all the borrow logic by pretending we have inf - * runtime - in which case borrowing doesn't make sense. - */ - rt_rq->rt_runtime =3D RUNTIME_INF; - rt_rq->rt_throttled =3D 0; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - raw_spin_unlock(&rt_b->rt_runtime_lock); - - /* Make rt_rq available for pick_next_task() */ - sched_rt_rq_enqueue(rt_rq); - } -} - -static void __enable_runtime(struct rq *rq) -{ - rt_rq_iter_t iter; - struct rt_rq *rt_rq; - - if (unlikely(!scheduler_running)) - return; - - /* - * Reset each runqueue's bandwidth settings - */ - for_each_rt_rq(rt_rq, iter, rq) { - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - - raw_spin_lock(&rt_b->rt_runtime_lock); - raw_spin_lock(&rt_rq->rt_runtime_lock); - rt_rq->rt_runtime =3D rt_b->rt_runtime; - rt_rq->rt_time =3D 0; - rt_rq->rt_throttled =3D 0; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - raw_spin_unlock(&rt_b->rt_runtime_lock); - } -} - -static void balance_runtime(struct rt_rq *rt_rq) -{ - if (!sched_feat(RT_RUNTIME_SHARE)) - return; - - if (rt_rq->rt_time > rt_rq->rt_runtime) { - raw_spin_unlock(&rt_rq->rt_runtime_lock); - do_balance_runtime(rt_rq); - raw_spin_lock(&rt_rq->rt_runtime_lock); - } -} - -static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun) -{ - int i, idle =3D 1, throttled =3D 0; - const struct cpumask *span; - - span =3D sched_rt_period_mask(); - - /* - * FIXME: isolated CPUs should really leave the root task group, - * whether they are isolcpus or were isolated via cpusets, lest - * the timer run on a CPU which does not service all runqueues, - * potentially leaving other CPUs indefinitely throttled. If - * isolation is really required, the user will turn the throttle - * off to kill the perturbations it causes anyway. Meanwhile, - * this maintains functionality for boot and/or troubleshooting. - */ - if (rt_b =3D=3D &root_task_group.rt_bandwidth) - span =3D cpu_online_mask; - - for_each_cpu(i, span) { - int enqueue =3D 0; - struct rt_rq *rt_rq =3D sched_rt_period_rt_rq(rt_b, i); - struct rq *rq =3D rq_of_rt_rq(rt_rq); - struct rq_flags rf; - int skip; - - /* - * When span =3D=3D cpu_online_mask, taking each rq->lock - * can be time-consuming. Try to avoid it when possible. - */ - raw_spin_lock(&rt_rq->rt_runtime_lock); - if (!sched_feat(RT_RUNTIME_SHARE) && rt_rq->rt_runtime !=3D RUNTIME_INF) - rt_rq->rt_runtime =3D rt_b->rt_runtime; - skip =3D !rt_rq->rt_time && !rt_rq->rt_nr_running; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - if (skip) - continue; - - rq_lock(rq, &rf); - update_rq_clock(rq); - - if (rt_rq->rt_time) { - u64 runtime; - - raw_spin_lock(&rt_rq->rt_runtime_lock); - if (rt_rq->rt_throttled) - balance_runtime(rt_rq); - runtime =3D rt_rq->rt_runtime; - rt_rq->rt_time -=3D min(rt_rq->rt_time, overrun*runtime); - if (rt_rq->rt_throttled && rt_rq->rt_time < runtime) { - rt_rq->rt_throttled =3D 0; - enqueue =3D 1; - - /* - * When we're idle and a woken (rt) task is - * throttled wakeup_preempt() will set - * skip_update and the time between the wakeup - * and this unthrottle will get accounted as - * 'runtime'. - */ - if (rt_rq->rt_nr_running && rq->curr =3D=3D rq->idle) - rq_clock_cancel_skipupdate(rq); - } - if (rt_rq->rt_time || rt_rq->rt_nr_running) - idle =3D 0; - raw_spin_unlock(&rt_rq->rt_runtime_lock); - } else if (rt_rq->rt_nr_running) { - idle =3D 0; - if (!rt_rq_throttled(rt_rq)) - enqueue =3D 1; - } - if (rt_rq->rt_throttled) - throttled =3D 1; - - if (enqueue) - sched_rt_rq_enqueue(rt_rq); - rq_unlock(rq, &rf); - } - - if (!throttled && (!rt_bandwidth_enabled() || rt_b->rt_runtime =3D=3D RUN= TIME_INF)) - return 1; - - return idle; -} - -static int sched_rt_runtime_exceeded(struct rt_rq *rt_rq) -{ - u64 runtime =3D sched_rt_runtime(rt_rq); - - if (rt_rq->rt_throttled) - return rt_rq_throttled(rt_rq); - - if (runtime >=3D sched_rt_period(rt_rq)) - return 0; - - balance_runtime(rt_rq); - runtime =3D sched_rt_runtime(rt_rq); - if (runtime =3D=3D RUNTIME_INF) - return 0; - - if (rt_rq->rt_time > runtime) { - struct rt_bandwidth *rt_b =3D sched_rt_bandwidth(rt_rq); - - /* - * Don't actually throttle groups that have no runtime assigned - * but accrue some time due to boosting. - */ - if (likely(rt_b->rt_runtime)) { - rt_rq->rt_throttled =3D 1; - printk_deferred_once("sched: RT throttling activated\n"); - } else { - /* - * In case we did anyway, make it go away, - * replenishment is a joke, since it will replenish us - * with exactly 0 ns. - */ - rt_rq->rt_time =3D 0; - } - - if (rt_rq_throttled(rt_rq)) { - sched_rt_rq_dequeue(rt_rq); - return 1; - } - } - - return 0; -} - -#else /* !CONFIG_RT_GROUP_SCHED: */ +#else /* !CONFIG_RT_GROUP_SCHED */ =20 typedef struct rt_rq *rt_rq_iter_t; =20 #define for_each_rt_rq(rt_rq, iter, rq) \ for ((void) iter, rt_rq =3D &rq->rt; rt_rq; rt_rq =3D NULL) =20 -#define for_each_sched_rt_entity(rt_se) \ - for (; rt_se; rt_se =3D NULL) - -static inline struct rt_rq *group_rt_rq(struct sched_rt_entity *rt_se) -{ - return NULL; -} - -static inline void sched_rt_rq_enqueue(struct rt_rq *rt_rq) -{ - struct rq *rq =3D rq_of_rt_rq(rt_rq); - - if (!rt_rq->rt_nr_running) - return; - - enqueue_top_rt_rq(rt_rq); - resched_curr(rq); -} - -static inline void sched_rt_rq_dequeue(struct rt_rq *rt_rq) -{ - dequeue_top_rt_rq(rt_rq, rt_rq->rt_nr_running); -} - -static inline int rt_rq_throttled(struct rt_rq *rt_rq) -{ - return false; -} - -static inline const struct cpumask *sched_rt_period_mask(void) -{ - return cpu_online_mask; -} - -static inline -struct rt_rq *sched_rt_period_rt_rq(struct rt_bandwidth *rt_b, int cpu) -{ - return &cpu_rq(cpu)->rt; -} - -static void __enable_runtime(struct rq *rq) { } -static void __disable_runtime(struct rq *rq) { } - -#endif /* !CONFIG_RT_GROUP_SCHED */ +#endif /* CONFIG_RT_GROUP_SCHED */ =20 static inline int rt_se_prio(struct sched_rt_entity *rt_se) { -#ifdef CONFIG_RT_GROUP_SCHED - struct rt_rq *rt_rq =3D group_rt_rq(rt_se); - - if (rt_rq) - return rt_rq->highest_prio.curr; -#endif - return rt_task_of(rt_se)->prio; } =20 @@ -931,67 +334,8 @@ static void update_curr_rt(struct rq *rq) if (unlikely(delta_exec <=3D 0)) return; =20 -#ifdef CONFIG_RT_GROUP_SCHED - struct sched_rt_entity *rt_se =3D &donor->rt; - if (!rt_bandwidth_enabled()) return; - - for_each_sched_rt_entity(rt_se) { - struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); - int exceeded; - - if (sched_rt_runtime(rt_rq) !=3D RUNTIME_INF) { - raw_spin_lock(&rt_rq->rt_runtime_lock); - rt_rq->rt_time +=3D delta_exec; - exceeded =3D sched_rt_runtime_exceeded(rt_rq); - if (exceeded) - resched_curr(rq); - raw_spin_unlock(&rt_rq->rt_runtime_lock); - if (exceeded) - do_start_rt_bandwidth(sched_rt_bandwidth(rt_rq)); - } - } -#endif /* CONFIG_RT_GROUP_SCHED */ -} - -static void -dequeue_top_rt_rq(struct rt_rq *rt_rq, unsigned int count) -{ - struct rq *rq =3D rq_of_rt_rq(rt_rq); - - BUG_ON(&rq->rt !=3D rt_rq); - - if (!rt_rq->rt_queued) - return; - - BUG_ON(!rq->nr_running); - - sub_nr_running(rq, count); - rt_rq->rt_queued =3D 0; - -} - -static void -enqueue_top_rt_rq(struct rt_rq *rt_rq) -{ - struct rq *rq =3D rq_of_rt_rq(rt_rq); - - BUG_ON(&rq->rt !=3D rt_rq); - - if (rt_rq->rt_queued) - return; - - if (rt_rq_throttled(rt_rq)) - return; - - if (rt_rq->rt_nr_running) { - add_nr_running(rq, rt_rq->rt_nr_running); - rt_rq->rt_queued =3D 1; - } - - /* Kick cpufreq (see the comment in kernel/sched/sched.h). */ - cpufreq_update_util(rq, 0); } =20 static void @@ -1062,58 +406,17 @@ dec_rt_prio(struct rt_rq *rt_rq, int prio) dec_rt_prio_smp(rt_rq, prio, prev_prio); } =20 -#ifdef CONFIG_RT_GROUP_SCHED - -static void -inc_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) -{ - if (rt_se_boosted(rt_se)) - rt_rq->rt_nr_boosted++; - - start_rt_bandwidth(&rt_rq->tg->rt_bandwidth); -} - -static void -dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) -{ - if (rt_se_boosted(rt_se)) - rt_rq->rt_nr_boosted--; - - WARN_ON(!rt_rq->rt_nr_running && rt_rq->rt_nr_boosted); -} - -#else /* !CONFIG_RT_GROUP_SCHED: */ - -static void -inc_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) -{ -} - -static inline -void dec_rt_group(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) {} - -#endif /* !CONFIG_RT_GROUP_SCHED */ - static inline unsigned int rt_se_nr_running(struct sched_rt_entity *rt_se) { - struct rt_rq *group_rq =3D group_rt_rq(rt_se); - - if (group_rq) - return group_rq->rt_nr_running; - else - return 1; + return 1; } =20 static inline unsigned int rt_se_rr_nr_running(struct sched_rt_entity *rt_se) { - struct rt_rq *group_rq =3D group_rt_rq(rt_se); struct task_struct *tsk; =20 - if (group_rq) - return group_rq->rr_nr_running; - tsk =3D rt_task_of(rt_se); =20 return (tsk->policy =3D=3D SCHED_RR) ? 1 : 0; @@ -1122,26 +425,21 @@ unsigned int rt_se_rr_nr_running(struct sched_rt_ent= ity *rt_se) static inline void inc_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) { - int prio =3D rt_se_prio(rt_se); - - WARN_ON(!rt_prio(prio)); + WARN_ON(!rt_prio(rt_se_prio(rt_se))); rt_rq->rt_nr_running +=3D rt_se_nr_running(rt_se); rt_rq->rr_nr_running +=3D rt_se_rr_nr_running(rt_se); =20 - inc_rt_prio(rt_rq, prio); - inc_rt_group(rt_se, rt_rq); + inc_rt_prio(rt_rq, rt_se_prio(rt_se)); } =20 static inline void dec_rt_tasks(struct sched_rt_entity *rt_se, struct rt_rq *rt_rq) { WARN_ON(!rt_prio(rt_se_prio(rt_se))); - WARN_ON(!rt_rq->rt_nr_running); rt_rq->rt_nr_running -=3D rt_se_nr_running(rt_se); rt_rq->rr_nr_running -=3D rt_se_rr_nr_running(rt_se); =20 dec_rt_prio(rt_rq, rt_se_prio(rt_se)); - dec_rt_group(rt_se, rt_rq); } =20 /* @@ -1170,10 +468,6 @@ static void __delist_rt_entity(struct sched_rt_entity= *rt_se, struct rt_prio_arr static inline struct sched_statistics * __schedstats_from_rt_se(struct sched_rt_entity *rt_se) { - /* schedstats is not supported for rt group. */ - if (!rt_entity_is_task(rt_se)) - return NULL; - return &rt_task_of(rt_se)->stats; } =20 @@ -1186,9 +480,7 @@ update_stats_wait_start_rt(struct rt_rq *rt_rq, struct= sched_rt_entity *rt_se) if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); stats =3D __schedstats_from_rt_se(rt_se); if (!stats) return; @@ -1205,9 +497,7 @@ update_stats_enqueue_sleeper_rt(struct rt_rq *rt_rq, s= truct sched_rt_entity *rt_ if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); stats =3D __schedstats_from_rt_se(rt_se); if (!stats) return; @@ -1235,9 +525,7 @@ update_stats_wait_end_rt(struct rt_rq *rt_rq, struct s= ched_rt_entity *rt_se) if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); stats =3D __schedstats_from_rt_se(rt_se); if (!stats) return; @@ -1254,9 +542,7 @@ update_stats_dequeue_rt(struct rt_rq *rt_rq, struct sc= hed_rt_entity *rt_se, if (!schedstat_enabled()) return; =20 - if (rt_entity_is_task(rt_se)) - p =3D rt_task_of(rt_se); - + p =3D rt_task_of(rt_se); if ((flags & DEQUEUE_SLEEP) && p) { unsigned int state; =20 @@ -1275,21 +561,8 @@ static void __enqueue_rt_entity(struct sched_rt_entit= y *rt_se, unsigned int flag { struct rt_rq *rt_rq =3D rt_rq_of_se(rt_se); struct rt_prio_array *array =3D &rt_rq->active; - struct rt_rq *group_rq =3D group_rt_rq(rt_se); struct list_head *queue =3D array->queue + rt_se_prio(rt_se); =20 - /* - * Don't enqueue the group if its throttled, or when empty. - * The latter is a consequence of the former when a child group - * get throttled and the current group doesn't have any other - * active members. - */ - if (group_rq && (rt_rq_throttled(group_rq) || !group_rq->rt_nr_running)) { - if (rt_se->on_list) - __delist_rt_entity(rt_se, array); - return; - } - if (move_entity(flags)) { WARN_ON_ONCE(rt_se->on_list); if (flags & ENQUEUE_HEAD) @@ -1319,57 +592,18 @@ static void __dequeue_rt_entity(struct sched_rt_enti= ty *rt_se, unsigned int flag dec_rt_tasks(rt_se, rt_rq); } =20 -/* - * Because the prio of an upper entry depends on the lower - * entries, we must remove entries top - down. - */ -static void dequeue_rt_stack(struct sched_rt_entity *rt_se, unsigned int f= lags) -{ - struct sched_rt_entity *back =3D NULL; - unsigned int rt_nr_running; - - for_each_sched_rt_entity(rt_se) { - rt_se->back =3D back; - back =3D rt_se; - } - - rt_nr_running =3D rt_rq_of_se(back)->rt_nr_running; - - for (rt_se =3D back; rt_se; rt_se =3D rt_se->back) { - if (on_rt_rq(rt_se)) - __dequeue_rt_entity(rt_se, flags); - } - - dequeue_top_rt_rq(rt_rq_of_se(back), rt_nr_running); -} - static void enqueue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags) { - struct rq *rq =3D rq_of_rt_se(rt_se); - update_stats_enqueue_rt(rt_rq_of_se(rt_se), rt_se, flags); =20 - dequeue_rt_stack(rt_se, flags); - for_each_sched_rt_entity(rt_se) - __enqueue_rt_entity(rt_se, flags); - enqueue_top_rt_rq(&rq->rt); + __enqueue_rt_entity(rt_se, flags); } =20 static void dequeue_rt_entity(struct sched_rt_entity *rt_se, unsigned int = flags) { - struct rq *rq =3D rq_of_rt_se(rt_se); - update_stats_dequeue_rt(rt_rq_of_se(rt_se), rt_se, flags); =20 - dequeue_rt_stack(rt_se, flags); - - for_each_sched_rt_entity(rt_se) { - struct rt_rq *rt_rq =3D group_rt_rq(rt_se); - - if (rt_rq && rt_rq->rt_nr_running) - __enqueue_rt_entity(rt_se, flags); - } - enqueue_top_rt_rq(&rq->rt); + __dequeue_rt_entity(rt_se, flags); } =20 /* @@ -1432,10 +666,8 @@ static void requeue_task_rt(struct rq *rq, struct tas= k_struct *p, int head) struct sched_rt_entity *rt_se =3D &p->rt; struct rt_rq *rt_rq; =20 - for_each_sched_rt_entity(rt_se) { - rt_rq =3D rt_rq_of_se(rt_se); - requeue_rt_entity(rt_rq, rt_se, head); - } + rt_rq =3D rt_rq_of_se(rt_se); + requeue_rt_entity(rt_rq, rt_se, head); } =20 static void yield_task_rt(struct rq *rq) @@ -1632,17 +864,7 @@ static struct sched_rt_entity *pick_next_rt_entity(st= ruct rt_rq *rt_rq) =20 static struct task_struct *_pick_next_task_rt(struct rq *rq) { - struct sched_rt_entity *rt_se; - struct rt_rq *rt_rq =3D &rq->rt; - - do { - rt_se =3D pick_next_rt_entity(rt_rq); - if (unlikely(!rt_se)) - return NULL; - rt_rq =3D group_rt_rq(rt_se); - } while (rt_rq); - - return rt_task_of(rt_se); + return NULL; } =20 static struct task_struct *pick_task_rt(struct rq *rq) @@ -2311,8 +1533,6 @@ static void rq_online_rt(struct rq *rq) if (rq->rt.overloaded) rt_set_overload(rq); =20 - __enable_runtime(rq); - cpupri_set(&rq->rd->cpupri, rq->cpu, rq->rt.highest_prio.curr); } =20 @@ -2322,8 +1542,6 @@ static void rq_offline_rt(struct rq *rq) if (rq->rt.overloaded) rt_clear_overload(rq); =20 - __disable_runtime(rq); - cpupri_set(&rq->rd->cpupri, rq->cpu, CPUPRI_INVALID); } =20 @@ -2481,12 +1699,12 @@ static void task_tick_rt(struct rq *rq, struct task= _struct *p, int queued) * Requeue to the end of queue if we (and all of our ancestors) are not * the only element on the queue */ - for_each_sched_rt_entity(rt_se) { - if (rt_se->run_list.prev !=3D rt_se->run_list.next) { - requeue_task_rt(rq, p, 0); - resched_curr(rq); - return; - } + if (rt_se->run_list.prev !=3D rt_se->run_list.next) { + requeue_task_rt(rq, p, 0); + resched_curr(rq); + // set_tsk_need_resched(p); + + return; } } =20 @@ -2504,16 +1722,7 @@ static unsigned int get_rr_interval_rt(struct rq *rq= , struct task_struct *task) #ifdef CONFIG_SCHED_CORE static int task_is_throttled_rt(struct task_struct *p, int cpu) { - struct rt_rq *rt_rq; - -#ifdef CONFIG_RT_GROUP_SCHED // XXX maybe add task_rt_rq(), see also sched= _rt_period_rt_rq - rt_rq =3D task_group(p)->rt_rq[cpu]; - WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); -#else - rt_rq =3D &cpu_rq(cpu)->rt; -#endif - - return rt_rq_throttled(rt_rq); + return 0; } #endif /* CONFIG_SCHED_CORE */ =20 @@ -2761,13 +1970,7 @@ long sched_group_rt_period(struct task_group *tg) #ifdef CONFIG_SYSCTL static int sched_rt_global_constraints(void) { - int ret =3D 0; - - mutex_lock(&rt_constraints_mutex); - ret =3D __rt_schedulable(NULL, 0, 0); - mutex_unlock(&rt_constraints_mutex); - - return ret; + return 0; } #endif /* CONFIG_SYSCTL */ =20 @@ -2802,10 +2005,6 @@ static int sched_rt_global_validate(void) return 0; } =20 -static void sched_rt_do_global(void) -{ -} - static int sched_rt_handler(const struct ctl_table *table, int write, void= *buffer, size_t *lenp, loff_t *ppos) { @@ -2833,7 +2032,6 @@ static int sched_rt_handler(const struct ctl_table *t= able, int write, void *buff if (ret) goto undo; =20 - sched_rt_do_global(); sched_dl_do_global(); } if (0) { diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index a1e6d2852ca..2f9035cb9e5 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -789,7 +789,7 @@ struct scx_rq { =20 static inline int rt_bandwidth_enabled(void) { - return sysctl_sched_rt_runtime >=3D 0; + return 0; } =20 /* RT IPI pull logic requires IRQ_WORK */ @@ -829,7 +829,7 @@ struct rt_rq { =20 static inline bool rt_rq_is_runnable(struct rt_rq *rt_rq) { - return rt_rq->rt_queued && rt_rq->rt_nr_running; + return rt_rq->rt_nr_running; } =20 /* Deadline class' related fields in a runqueue */ @@ -2545,7 +2545,7 @@ static inline bool sched_dl_runnable(struct rq *rq) =20 static inline bool sched_rt_runnable(struct rq *rq) { - return rq->rt.rt_queued > 0; + return rq->rt.rt_nr_running > 0; } =20 static inline bool sched_fair_runnable(struct rq *rq) @@ -2656,9 +2656,6 @@ extern void resched_curr(struct rq *rq); extern void resched_curr_lazy(struct rq *rq); extern void resched_cpu(int cpu); =20 -extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 r= untime); -extern bool sched_rt_bandwidth_account(struct rt_rq *rt_rq); - extern void init_dl_entity(struct sched_dl_entity *dl_se); =20 #define BW_SHIFT 20 diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index 77ae87f36e8..93a9c03b28e 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -626,19 +626,6 @@ int __sched_setscheduler(struct task_struct *p, change: =20 if (user) { -#ifdef CONFIG_RT_GROUP_SCHED - /* - * Do not allow real-time tasks into groups that have no runtime - * assigned. - */ - if (rt_group_sched_enabled() && - rt_bandwidth_enabled() && rt_policy(policy) && - task_group(p)->rt_bandwidth.rt_runtime =3D=3D 0 && - !task_group_is_autogroup(task_group(p))) { - retval =3D -EPERM; - goto unlock; - } -#endif /* CONFIG_RT_GROUP_SCHED */ if (dl_bandwidth_enabled() && dl_policy(policy) && !(attr->sched_flags & SCHED_FLAG_SUGOV)) { cpumask_t *span =3D rq->rd->span; --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EA462BE7B5 for ; Thu, 31 Jul 2025 10:55:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959354; cv=none; b=j7VwJqpdPQyozacSU4g5hO1Bk0ZVfOV8UF1J08zD2CEOa/8vDzjF504Bx9x3CN+Y7H/vR4CoqBO3FI/GV7uEHdIkglZVs2O18zeudLUaelX0N00c0b8aeeWhONVPXtsObQMT3+/8Ti37TAltTvHXNoEu9+y1igHXquXbaFD3mT8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959354; c=relaxed/simple; bh=2nNH86zTsg7R2fmNZl123xTHKQk9SLRGN74jQT1JzLs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nGs/YCGJBwRmJdlatBGZ8x/7f/n9GSC5XMPskDzgOqpoeKwILk1ZTTn0YReXMxzbPw57YSFbqRQmKvp/sOQU1A2UdrHMRwLpNhs++VVfDetQzX+m/qDS20s1rlgNdG0cpjMlWTk5JFFBRo8MYXFWgU70fCN6pWMeTP7Q3o5CnVg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jJsVUMqo; arc=none smtp.client-ip=209.85.221.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jJsVUMqo" Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-3b79bddd604so211518f8f.0 for ; Thu, 31 Jul 2025 03:55:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959350; x=1754564150; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=L0kWGTgavAow+H+0yQxg8Ps6erJvXQE/DQ1ARn7035s=; b=jJsVUMqoJwjxZameiwcj3IYs99SiWiChMMUg1vYTQKGIdJmBQJOnnRROhghR/D75sk bRHfPobIMwnbKV3b6QVV/bJyy5cpsA/xGSrDXnPeu/VrMyTM3RpXz9QlXS0PeU9BJjmQ 4Fu30TC7C08+GnWvVzhdKXeZ3uNo3Cbbfl62ztvANdk6hvWrH1QGBpZVjxKknHoRtHDM DqnQdZG5osBiENIkIDYTKINbvsbGnaF8T/HzOs0u0pBmfwYvmRGxxwZ6QOPSemAM8Ysv ks7MDz+fmXpYn14i1pV4cwR6intNlUY/jQXin2GVmNmeE550e79ycVDXf00I5bZDcbrh 9uJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959350; x=1754564150; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L0kWGTgavAow+H+0yQxg8Ps6erJvXQE/DQ1ARn7035s=; b=mUIkSgt9qSGlsW50yJPhICEk+raeiwmbGi47/BJ8s9Wi2xyQIUrYYFZp5patm9o3oZ wV3smQAMTKmNxyKBpQs8sR3K0r6un1OoFyN14IOvsycrfnf/aJslfR9y3d1/rM/EtLgl kwXgMsla6tJdD8Rr5YTIuYSOcq1XqzcghlzdmpvhniYQ63RQMgDYJvxotD5FzA+Pb4Ec Lv3fBI64dTjUkIbjs81BhkyqjT59suuFYa9tcD825PLB91qOP2b6/yFHvCmSZ1Th4mqL X5h3a3vLFoP4KZ2y2u8yKewTR3cA5SIZ+eFJxEEKyE4iEwnOXClQASJg5KJzgIOy5rWe lQFA== X-Gm-Message-State: AOJu0YxwieaL1kACxu5e0zQUtMxoHe4/07mvEcXHKwxSzBn9GpbWLsy9 ZidxJ31IVb6V/jSwb2B+simDZOfOi/zu2HGnffTxVO+vh+hsGA9jxr0R X-Gm-Gg: ASbGncvbqtHimHbLQ7fvKanp/95sjw1yLNphBehggLaiWLLMk20dOoNNK8FXQJntHIp sjLwozvx9EVF9jwpokPRF7v2SCQHUGy1pkzMg+YrT33Vucgn6GZYs7MGDKLRlf65WvGBHUSUryx o0q1iXlENjDsQr9TMhuAX9xzCMsqzdH+k74/MATHRA/WgWSwOQmy30sQt+ltSuYpAj5rvsVMlba bJwBaaerc6A4aTTM1nLTonPK1I4x8/rxKpUpFTmkK5W4Hn4k9rJ5RL77MB71Dgr90HmELo1+/v3 iQvwa/NotRIUoQembvpXrhmhq+G6x+FtokZs2GJduCl0lwC16McJKdTC7x4ewSpfzS91/ajkZBn pSZZnx3j5TVfGpTSLtns= X-Google-Smtp-Source: AGHT+IER4w7Gv2HT+lC8OIjC2A2wM1LDd+FmczqJH8iIfDQUqrkmZjEiTLub/ovZQlhvGnrzY9zJKA== X-Received: by 2002:a05:6000:2381:b0:391:3aaf:1d5f with SMTP id ffacd0b85a97d-3b795009481mr5578956f8f.52.1753959350129; Thu, 31 Jul 2025 03:55:50 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:49 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 07/25] sched/rt: Introduce HCBS specific structs in task_group Date: Thu, 31 Jul 2025 12:55:25 +0200 Message-ID: <20250731105543.40832-8-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Each task_group manages a number of new objects: - a sched_dl_entity/dl_server for each CPU - a dl_bandwidth object to keep track of its allocated dl_bandwidth Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/sched.h | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 2f9035cb9e5..2a7601d400c 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -319,6 +319,13 @@ struct rt_bandwidth { unsigned int rt_period_active; }; =20 +struct dl_bandwidth { + raw_spinlock_t dl_runtime_lock; + u64 dl_runtime; + u64 dl_period; +}; + + static inline int dl_bandwidth_enabled(void) { return sysctl_sched_rt_runtime >=3D 0; @@ -467,9 +474,15 @@ struct task_group { =20 #ifdef CONFIG_RT_GROUP_SCHED struct sched_rt_entity **rt_se; + /* + * The scheduling entities for the task group are managed as a single + * sched_dl_entity, each of them sharing the same dl_bandwidth. + */ + struct sched_dl_entity **dl_se; struct rt_rq **rt_rq; =20 struct rt_bandwidth rt_bandwidth; + struct dl_bandwidth dl_bandwidth; #endif =20 #ifdef CONFIG_EXT_GROUP_SCHED @@ -819,12 +832,12 @@ struct rt_rq { raw_spinlock_t rt_runtime_lock; =20 unsigned int rt_nr_boosted; - - struct rq *rq; /* this is always top-level rq, cache? */ #endif #ifdef CONFIG_CGROUP_SCHED struct task_group *tg; /* this tg has "this" rt_rq on given CPU for runna= ble entities */ #endif + + struct rq *rq; /* cgroup's runqueue if the rt_rq entity belongs to a cgr= oup, otherwise top-level rq */ }; =20 static inline bool rt_rq_is_runnable(struct rt_rq *rt_rq) --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4B212BEC20 for ; Thu, 31 Jul 2025 10:55:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959354; cv=none; b=kt856km/iLmjrhgY45BvyuROyKXEfC340oGFklWKXBtxbiG/of3vrg1NBE0oWj3QjZ9ANSmB2MnaiLy+0qXTKWx60cFVxqDnJsVJ+TDM6O3Y5YolvHmq/7LoNJWsttjNc+ExrMpjyUSt5wKZitHZKc5kxib4ZAYjZ3r2t+IEl9A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959354; c=relaxed/simple; bh=22/BBPVYAgcEcnh9sb9JTkNEXXz/AMYJeDNiGyEFh6M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kyUgj6FUrdKC1XUmttlD8e+y2eBSpmtO6QTNvIVSKWI80qSgMqAb2G8+oUMUVS4SoH+McAy4x4z/Ub70VRIVvtgIa+PnXLpeei5hl2ajpY4IPpVVC5npCB3QHgy8OCZgRNmx+xRp+MD+0rpEQyq3tWuyynj9JMF9j9/6woXbBQs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=G9pOa4qd; arc=none smtp.client-ip=209.85.221.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="G9pOa4qd" Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-3b780bdda21so609802f8f.3 for ; Thu, 31 Jul 2025 03:55:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959351; x=1754564151; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UwcLfkk7r5x07qaUEcnr0bt4ogsKgYoe5GuJmqfu0Uo=; b=G9pOa4qdNhJ93Mkp3by0M1narOK8zw1X49G8quHE9CZplzOr/pTKUc1AeuUXD8s56u k8w/bUEu9F+5MiAKqwGnHMIIBj7nRAbabo8voP4nCZ98HnSTGCXidRWyqjJ9c/p6yRmN wJwGnT5EdpH7xST9wvNTNJpGd3pERXdmq4828IaRR0IzEYr2/wEFbCexHbLeiYAZUwdS Ky7k5y8LUfA9/bhXyJDcfJ20gzAfc84YrJgmQTr7FhUq40eHgQN6kh2cCuj0j+8+owGW Msxf+toL78bYVlfvFdZdNNkTDYSX6wQgKOikbTJz+Yor/o9uizoxfWSbOehwNnuhHyCk 9t4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959351; x=1754564151; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UwcLfkk7r5x07qaUEcnr0bt4ogsKgYoe5GuJmqfu0Uo=; b=URcJULOaXQDa7OMeoqYBTwAcE4pYAMVAEs1i5+hbYWU4V0z7+ugNIMgdBsYrcIBqSa c+V/ISB2RoXJiu2kwwESF8e29Rrh531Z36JeOpDOogVYmmnG1JcDwgauOeUeePENVtuq PIPfVIlluy8vu0baRKPtm167ca1GU8eX5D9Gou2HWdAUMA6viwQvCVAveY1yQg5FDQFi TudJzFIen1XdL9/onZBJOu88Z66hg8ND7bP1prdQ9ZlNrh1u1KbaudVQ2yuFpAfSalJD Eiyt7+hzGIuGgZ7K1CM5vdOpPZ6DYPqU26BT78QsGMbu8Y2Tz5VhHWHB8Hz9o1lh6iXO RcEQ== X-Gm-Message-State: AOJu0Yy58uMqMVQ8akLBNLKgZ6+jEZJJVMtUrvtFL5zSNfqu7jSBzGM4 8RKND/HK5FwHYBcbTxpGmglgFwuAckM4oURNiiJWcC3OJBdztT5CHbF7 X-Gm-Gg: ASbGncsbgcOXJ2xzH+IhNYSld83FAuTk3BPV573rd12tqMX6GdA9jC7mCeQMXtXBmfg DwcSwvR7X7VCL0/SPOMSrOJsvCGtzqsnoDKFhe7Nnr0IowB8RbDKCtNEddIMsdyK7krj8Htla0L sNvbYEi3DVUhTcZgBtWUqtfWxD9OKYajYQ66pG+kuC4sOjIGx+1yf3qtDjgs5CAFxj2sNCTRnBK Z+2D7/UpV75XwJQnZz62wCuV3TUkE7emLlr04Z0yzpKlu28hUlWt0VrAjZE8U1llHZ2NB4uYf0K 8ll4tWMPn/EMb5b236Ot3FOuMMtwgE2s9JJHZIfc3uSSxZcK0Ehpqi5a9Q2rphfNXcLf9pdBMcH bBr1OgTHL6qLGDlfYKK4= X-Google-Smtp-Source: AGHT+IHIAMJQh9vdqcyDc46LIAaCbP2BCTOf9gMJ11VQgISefC0bY7ZfqeQx9HUNhCSGmDhUHbFiKA== X-Received: by 2002:a05:6000:144b:b0:3b5:f8b3:d4fc with SMTP id ffacd0b85a97d-3b7950190e7mr6416634f8f.53.1753959350947; Thu, 31 Jul 2025 03:55:50 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:50 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 08/25] sched/deadline: Account rt-cgroups bandwidth in deadline tasks schedulability tests. Date: Thu, 31 Jul 2025 12:55:26 +0200 Message-ID: <20250731105543.40832-9-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Account the rt-cgroups hierarchy's reserved bandwidth in the schedulability test of deadline entities. This mechanism allows to completely reserve port= ion of the rt-bandwidth to rt-cgroups even if they do not use all of it. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/deadline.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 0640d0ca45b..43af48038b9 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -231,8 +231,15 @@ void __dl_add(struct dl_bw *dl_b, u64 tsk_bw, int cpus) static inline bool __dl_overflow(struct dl_bw *dl_b, unsigned long cap, u64 old_bw, u64 new_b= w) { + u64 dl_groups_root =3D 0; + +#ifdef CONFIG_RT_GROUP_SCHED + dl_groups_root =3D to_ratio(root_task_group.dl_bandwidth.dl_period, + root_task_group.dl_bandwidth.dl_runtime); +#endif return dl_b->bw !=3D -1 && - cap_scale(dl_b->bw, cap) < dl_b->total_bw - old_bw + new_bw; + cap_scale(dl_b->bw, cap) < dl_b->total_bw - old_bw + new_bw + + cap_scale(dl_groups_root, cap); } =20 static inline --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D0AE2BEC37 for ; Thu, 31 Jul 2025 10:55:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959355; cv=none; b=rqFVuom71Q5hTRnLewxTdIGtPi500ysAF8QrAlAQRRllFVsinVPW1+6tse0+dVbXT1rOq2IBTbRPHCyDDUTzciE2BTm9p3FpgH5AN0oNOGbNbE2Qv8kJF7dSUHJupIrnTHXjTjzrkhiJ2sil217hE9xHRy0GAMu1Z5SRS6cMarI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959355; c=relaxed/simple; bh=5h8kV4D5T34UY7lWfYT38q0co0S9gry1y2pxE4/+dPE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JYVfX02AE2urd3Z4+HJa2se6qjNR3vRquwSgb1OJJxyR3ErXiuFGMp1Gm6/xLYyd0tN5un0C5J/aP/fUxERUeGxaonQH9Oc0T1oP0h+7OUFNFIoSDNJQ8dqQSlzfgEesGLQDQt92nDkgMqlMYBTVfbziKgnc2DXLv1Dj4PWRfuc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WL6XGOZ3; arc=none smtp.client-ip=209.85.128.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WL6XGOZ3" Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-455b002833bso2762315e9.0 for ; Thu, 31 Jul 2025 03:55:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959352; x=1754564152; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DsoVVUXbQcQUiFyMHfd1wlwL2hp1Ean1m0cDIu2kNZg=; b=WL6XGOZ3rfxfxqbIkk6VskkLoyV4KiBVQ/mQkp+IC/DuV1d5DRDK5XcTXvid37HpVm z/S2AfwQYFWxAjL506TqdXNXvR/oFnYoBH0U2qS1uvzK/D4CFeHOGyYiSUOmlvzxNwq1 IzrpAcs53uPZzNCGT1OYmo/7Gk+W7p2sMdyPtrYjYlzGSIKH9R9emGZsUn4wdHpsbNaj mxu/hPzRGsENtveEDrXwnybEBF9koMt8f93HvL0CFvLZhQaTZ0/rut/zO6yrnDxR/1s6 pLhqTJ/GZHzbhmhcG8jkVYJ9gvLC2dYF/c8t2Lbx/xEwcYJFLKICYo+ZgjWS9Ah9C7LM dfIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959352; x=1754564152; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DsoVVUXbQcQUiFyMHfd1wlwL2hp1Ean1m0cDIu2kNZg=; b=nG8LM8+Ty01zsuZTJ636kxXg0qHP2ZwNR5QJPnwQCJ3znhHg4UCdNPavJOitI9lhpQ p+MUl44vsnr4OKn9L33tmmZ0079kJvm2IZ+zE6WqntSibk7HqutCFDuCEg43X+mhDtch 9YnVkqy3z8KtV8Ea/ywT3fFpfDa2HPbUi3EgwflcgXKP6E2R0anQlFIu5cBxILJo0fuq alcc2ro79F4A+QKk9Xx6CTSso+gsiNYxGqEi7ceI0zdPUJd/cMI3A8wDYSBZpmZm9fBM WEULhWqRNBIK648Of6l1e790b3x0dUBzw78fLXDHIA2O/iaXZ2kqPz+MdLj+CPmbHLsF Ic+w== X-Gm-Message-State: AOJu0YzdnRsKvdJ05DXhxvlQIL1QGtUcTLxX8wgiF5wB4Pcjz/lnJJBF LeFI4DC47hyDwwzJ34azfxI1KxnOz/Se8PPWb9xnpqSFULl9og1gje9N X-Gm-Gg: ASbGncsyUfT99naQBNsbU6v3pAteRSwqLPJ7TrsmIl54AsJRC+NshPiQ5yuoz/BUcGE JfhbAJeOWy0gyWVTHPzI5f242ZbRYttFQJDqCGPcq7CxxVxUUM4BiYXwk/ZNXT1cbwzc9PXRjJA iZyLCvm1LSHJQhqnYYdfqBTgzI4EaSKFSd91STpEY4U6L4yKSONl7W/fG4nP8NgOSpceM5YAobb WGp4HTsY0DTRzgp9cLwTzi3Gu7Bh/bdh6xSYh7dOzAuBtwwenkBNcKYDUByKbxehZtwJes7tg64 //NJ+9wE4fNFeOISStNvVpQqICc6fdh6ywRFa4SAPMx8cjhzcQI60RADVXO5IzTYsPhCTzH+NUJ MHhZE8/TXDo27t6xLeN4= X-Google-Smtp-Source: AGHT+IEqruCgsr8I+jh6GHoDrJ/dFtAuwDM180ZpyVnxfnZhiISVrJ+XZ/qcNKXB/tSH11xA2eGhAQ== X-Received: by 2002:a5d:5d0a:0:b0:3b7:7d63:f4f2 with SMTP id ffacd0b85a97d-3b79502d719mr5260838f8f.48.1753959351735; Thu, 31 Jul 2025 03:55:51 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:51 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 09/25] sched/deadline: Account rt-cgroups bandwidth in sched_dl_global_validate Date: Thu, 31 Jul 2025 12:55:27 +0200 Message-ID: <20250731105543.40832-10-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" sched_dl_global_validate (similarly to the previous patch) must take into account the rt-cgroups' reserved bandwidth. Signed-off-by: Yuri Andriaccio --- kernel/sched/deadline.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 43af48038b9..55b7f883815 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -3098,11 +3098,17 @@ int sched_dl_global_validate(void) u64 period =3D global_rt_period(); u64 new_bw =3D to_ratio(period, runtime); u64 cookie =3D ++dl_cookie; + u64 dl_groups_root =3D 0; u64 fair_bw; struct dl_bw *dl_b; int cpu, ret =3D 0; unsigned long cap, flags; =20 +#ifdef CONFIG_RT_GROUP_SCHED + dl_groups_root =3D to_ratio(root_task_group.dl_bandwidth.dl_period, + root_task_group.dl_bandwidth.dl_runtime); +#endif + /* * Here we want to check the bandwidth not being set to some * value smaller than the currently allocated bandwidth in @@ -3119,7 +3125,7 @@ int sched_dl_global_validate(void) fair_bw =3D dl_bw_fair(cpu); =20 raw_spin_lock_irqsave(&dl_b->lock, flags); - if (cap_scale(new_bw, cap) < dl_b->total_bw) + if (cap_scale(new_bw, cap) < dl_b->total_bw + cap_scale(dl_groups_root, = cap)) ret =3D -EBUSY; if (cap_scale(new_bw, cap) + fair_bw > cap_scale(BW_UNIT, cap)) ret =3D -EBUSY; --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49B162BEFF0 for ; Thu, 31 Jul 2025 10:55:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959356; cv=none; b=I4qPK4rmymUlPjf6YGvqyTQ56fKTYBG+WORXtbdG/kluu2Z2Af15FPeia+getTFIFm1WNUIRNTwbXW1g7ooBTOJY58waby/tkuD0ovdQtzmV6sRjKomrskVoFEEtfBbFO8rfq4biGrTQAXYtR+DaippBpc8ys079IZ35ykzl6G8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959356; c=relaxed/simple; bh=y/m1xuIauJkrYvt1YIYbjMXrR9vbZJTyFydAQZV8SOg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HUwVX4rHoSCuTD4Enq/A0fji/WRRE6QUwSiL51NG0daMPeBu6hN0RZG53O/DJYryNoBG/WpB4PjmTHdC7P5ryTsZuW9oXsk7Ojo/o1OoDZ+HyDfEWsvoCcORxoq7B1VtpYiRHqKZqp1Rq1pAhTkmCGP/qFW1Vu28+N6dV4jh8jk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=J4mXsLxU; arc=none smtp.client-ip=209.85.221.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J4mXsLxU" Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-3a6cd1a6fecso684277f8f.3 for ; Thu, 31 Jul 2025 03:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959352; x=1754564152; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9uyGZGqhZLBvXN5S0wAqgq7D0Qe29eTkjq/bpY1CEUY=; b=J4mXsLxUq71LfgkJOW0VGP7tSNK6Rxn4UAAiHkNniqTJE3Hz5p4TgaVL5mhYrd4hGL HavQh8KqIB66nS/JlF7DUTZsh4k4p4e0o5F8uaz92LpUERARr1t7ycJwBUJ/Tk1f9XDY 7kgBZN24OpULlHE1pR7CzAAV0jVe9y2GjoP1ZRk8IPosI7n+yhvBIyV6AU4bgH5nE+bo 0UwRg/rFXD7OJBfCkmJFOv2QNXw7Jzet8Yh1O9jY45okeOuB6eh5f6nj0fY3RA2vU3Jp YlW1CLDHaCtavPW4tsiOPR5dAyqZAEbQHWTHLepqEtWP6NRcV1+0EgDzv8Qm3Tt8ljYW yqrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959352; x=1754564152; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9uyGZGqhZLBvXN5S0wAqgq7D0Qe29eTkjq/bpY1CEUY=; b=TiaPPkkPGuDBVtdNa8QEJbqqrnWmGwVz2nRi7hkzq6cugJuPDXTZ9d1/NhuAI5Ob65 whnXufOIaYBzdB2HaIAypNsN/ejhsZBzvfkMvdriz9Yzvhfjp+XfVu4aScxe8BJ3Ge7l z31YvGonyjwD8yrYC8b1DJowZxdj43PNHgZhwtyuzkbruZAuvRlgT6PBj84svC0nUzme 3xWIpVNQj7fVUbJ0Pu8ul0vGb0HmgnfZVyZn8iwiBm9it0OReFqe06DtRePtZCUz6d0v ihDhHM1881YrtedErHgXq8fwqEaj7/R8JBDjpUFsU6kR6OtG3FbjTURnqRt8aw6gM5Te 5Nzw== X-Gm-Message-State: AOJu0YyvA2ecuo/eh56MPiUywgBvFFfvn+Y60RQokI8yGAuRPgeSe9QS KmzIb/YxE1eF+3KBPuNEpPCNg1YbkfIxxruVZyNUagFtsQC98AnNPpW3 X-Gm-Gg: ASbGnctbNMOOBeKAToCn9tKWMB6d1v70K4w8m6phK95Bz2rqhNzcQJb5qL7SC5rJjh3 OLFdEPaDxuF7wX1HSk2M3ExZJEr5B24L6TOA9QyWS0JVrDq1kFwOoUgknQjfMVVLO7XXw7TxZlc On4oj/UOGc5F36INjdol4WiWhSQWckQO6F3rzXXLiYhYBYe7H+ZbIhecxLndO5gEFkYOyC3K3rJ Ab/26KcuUhFs+ynKxIUbBchsUgkX5yKx5vgxlvOSpFnU6dl8vZ0msQF0iQlJqktBpShBWZb+/Dw KBnv2DfZiJDFEY7O/7+Jw0SyPTytBUFXk9aJnquUaAzLwZuZjBVpfXlqwSrpax7QtBUn1HSl249 YXvj4dZ4tqMHCjUj4ij0ZXe2YlNC0Og== X-Google-Smtp-Source: AGHT+IGvxapxj8Q5c8Lp8FXTmJ8VG/Tz+ri2W75A14HZcpDfDBCoBuLYrT8UVhnKqrBRMS8+t1zXPg== X-Received: by 2002:a5d:64cd:0:b0:3b7:9c38:e8a5 with SMTP id ffacd0b85a97d-3b79c38e968mr1922605f8f.56.1753959352482; Thu, 31 Jul 2025 03:55:52 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:52 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 10/25] sched/core: Initialize root_task_group Date: Thu, 31 Jul 2025 12:55:28 +0200 Message-ID: <20250731105543.40832-11-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add the initialization function for task_group's dl_servers. Initialize the default bandwidth for rt-cgroups and the root control group. Add utility functions to check (and get) if a rt_rq entity is connected to a real-time cgroup. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/autogroup.c | 4 ++-- kernel/sched/core.c | 9 +++++++-- kernel/sched/deadline.c | 8 ++++++++ kernel/sched/rt.c | 18 ++++++++---------- kernel/sched/sched.h | 32 +++++++++++++++++++++++++++++--- 5 files changed, 54 insertions(+), 17 deletions(-) diff --git a/kernel/sched/autogroup.c b/kernel/sched/autogroup.c index cdea931aae3..017eadc0a0a 100644 --- a/kernel/sched/autogroup.c +++ b/kernel/sched/autogroup.c @@ -52,7 +52,7 @@ static inline void autogroup_destroy(struct kref *kref) =20 #ifdef CONFIG_RT_GROUP_SCHED /* We've redirected RT tasks to the root task group... */ - ag->tg->rt_se =3D NULL; + ag->tg->dl_se =3D NULL; ag->tg->rt_rq =3D NULL; #endif sched_release_group(ag->tg); @@ -109,7 +109,7 @@ static inline struct autogroup *autogroup_create(void) * the policy change to proceed. */ free_rt_sched_group(tg); - tg->rt_se =3D root_task_group.rt_se; + tg->dl_se =3D root_task_group.dl_se; tg->rt_rq =3D root_task_group.rt_rq; #endif /* CONFIG_RT_GROUP_SCHED */ tg->autogroup =3D ag; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 42587a3c71f..3a69cb906c3 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8703,7 +8703,7 @@ void __init sched_init(void) scx_tg_init(&root_task_group); #endif /* CONFIG_EXT_GROUP_SCHED */ #ifdef CONFIG_RT_GROUP_SCHED - root_task_group.rt_se =3D (struct sched_rt_entity **)ptr; + root_task_group.dl_se =3D (struct sched_dl_entity **)ptr; ptr +=3D nr_cpu_ids * sizeof(void **); =20 root_task_group.rt_rq =3D (struct rt_rq **)ptr; @@ -8714,6 +8714,11 @@ void __init sched_init(void) =20 init_defrootdomain(); =20 +#ifdef CONFIG_RT_GROUP_SCHED + init_dl_bandwidth(&root_task_group.dl_bandwidth, + global_rt_period(), global_rt_runtime()); +#endif /* CONFIG_RT_GROUP_SCHED */ + #ifdef CONFIG_CGROUP_SCHED task_group_cache =3D KMEM_CACHE(task_group, 0); =20 @@ -8765,7 +8770,7 @@ void __init sched_init(void) * starts working after scheduler_running, which is not the case * yet. */ - init_tg_rt_entry(&root_task_group, &rq->rt, NULL, i, NULL); + init_tg_rt_entry(&root_task_group, rq, NULL, i, NULL); #endif rq->sd =3D NULL; rq->rd =3D NULL; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 55b7f883815..b8228f553fe 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -538,6 +538,14 @@ static inline int is_leftmost(struct sched_dl_entity *= dl_se, struct dl_rq *dl_rq =20 static void init_dl_rq_bw_ratio(struct dl_rq *dl_rq); =20 +void init_dl_bandwidth(struct dl_bandwidth *dl_b, u64 period, u64 runtime) +{ + raw_spin_lock_init(&dl_b->dl_runtime_lock); + dl_b->dl_period =3D period; + dl_b->dl_runtime =3D runtime; +} + + void init_dl_bw(struct dl_bw *dl_b) { raw_spin_lock_init(&dl_b->lock); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index a6282784978..38178003184 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -97,18 +97,16 @@ void free_rt_sched_group(struct task_group *tg) return; } =20 -void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq, - struct sched_rt_entity *rt_se, int cpu, - struct sched_rt_entity *parent) +void init_tg_rt_entry(struct task_group *tg, struct rq *served_rq, + struct sched_dl_entity *dl_se, int cpu, + struct sched_dl_entity *parent) { - struct rq *rq =3D cpu_rq(cpu); + served_rq->rt.highest_prio.curr =3D MAX_RT_PRIO-1; + served_rq->rt.rq =3D cpu_rq(cpu); + served_rq->rt.tg =3D tg; =20 - rt_rq->highest_prio.curr =3D MAX_RT_PRIO-1; - rt_rq->rq =3D rq; - rt_rq->tg =3D tg; - - tg->rt_rq[cpu] =3D rt_rq; - tg->rt_se[cpu] =3D rt_se; + tg->rt_rq[cpu] =3D &served_rq->rt; + tg->dl_se[cpu] =3D dl_se; } =20 int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 2a7601d400c..3283d824859 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -577,9 +577,9 @@ extern void start_cfs_bandwidth(struct cfs_bandwidth *c= fs_b); extern void unthrottle_cfs_rq(struct cfs_rq *cfs_rq); extern bool cfs_task_bw_constrained(struct task_struct *p); =20 -extern void init_tg_rt_entry(struct task_group *tg, struct rt_rq *rt_rq, - struct sched_rt_entity *rt_se, int cpu, - struct sched_rt_entity *parent); +extern void init_tg_rt_entry(struct task_group *tg, struct rq *s_rq, + struct sched_dl_entity *rt_se, int cpu, + struct sched_dl_entity *parent); extern int sched_group_set_rt_runtime(struct task_group *tg, long rt_runti= me_us); extern int sched_group_set_rt_period(struct task_group *tg, u64 rt_period_= us); extern long sched_group_rt_runtime(struct task_group *tg); @@ -2669,6 +2669,7 @@ extern void resched_curr(struct rq *rq); extern void resched_curr_lazy(struct rq *rq); extern void resched_cpu(int cpu); =20 +void init_dl_bandwidth(struct dl_bandwidth *dl_b, u64 period, u64 runtime); extern void init_dl_entity(struct sched_dl_entity *dl_se); =20 #define BW_SHIFT 20 @@ -3077,6 +3078,21 @@ static inline struct rq *rq_of_rt_se(struct sched_rt= _entity *rt_se) =20 return rt_rq->rq; } + +static inline int is_dl_group(struct rt_rq *rt_rq) +{ + return rt_rq->tg !=3D &root_task_group; +} + +/* + * Return the scheduling entity of this group of tasks. + */ +static inline struct sched_dl_entity *dl_group_of(struct rt_rq *rt_rq) +{ + BUG_ON(!is_dl_group(rt_rq)); + + return rt_rq->tg->dl_se[cpu_of(rt_rq->rq)]; +} #else static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) { @@ -3101,6 +3117,16 @@ static inline struct rt_rq *rt_rq_of_se(struct sched= _rt_entity *rt_se) =20 return &rq->rt; } + +static inline int is_dl_group(struct rt_rq *rt_rq) +{ + return 0; +} + +static inline struct sched_dl_entity *dl_group_of(struct rt_rq *rt_rq) +{ + return NULL; +} #endif =20 DEFINE_LOCK_GUARD_2(double_rq_lock, struct rq, --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A2912BF012 for ; Thu, 31 Jul 2025 10:55:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959357; cv=none; b=BB8UYEdekO0NxUwW2flZbL605QzvfXP/9FBn/W6NpNeFeV6vuTk0mbjHHzXXcoQ06zkyPJFLHoHuNI5ZYVuJi+mJbeLRmHVZL3NDZjgn4uEviftuiqER8KechDw+h4iys5jc1vGSj797PZ1bRwj9tbbScKP42DBx2m5mly3yntM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959357; c=relaxed/simple; bh=89Ej/L+xXNc2n2O1xoDK9fP54YtIRq9uSUgx+r+oUnU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Xv1WfuM0vnNBql4tqy9DWo+br1VQnpw9XlvcyYDraX25ZjYontAPqj9UaxRZh+kGFiyyqNEFLW3+/zRNd82z3BJ1aqMkXWbDY/AgCWsuOIGT6SIuTJfxyOQUwkPT6UPg+Q4VpfvPJcZBk4Jjb3GvEE8jauQACJvck+XxSCIRomY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IR2U2bTL; arc=none smtp.client-ip=209.85.221.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IR2U2bTL" Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-3a588da60dfso136501f8f.1 for ; Thu, 31 Jul 2025 03:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959353; x=1754564153; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jv3iRJDYgpMDxRSvFG+C/LPY2Ji1gd5ewL7Azcb/wOA=; b=IR2U2bTLaxNmdj4AMbIya0JtPXbkS0KP6XbMmOIVrmRsAhtf+wnYR9ypYrcq/Hap+S FMP6ahaQ+xRdl1B4W0iVpelyrgftxifMLEcfVhLiLrAG0tgcmUFRpZ1Yka/hTuQFakT3 4twQ/7KDLmyMTsTOE65N2fdJif6S/jFDLWPzC3h5t+GYYrIbAdBlP8b0dhItjDq82iBX GMNYnbe5lvzZQBxXPavyAmkR76RV/YZYpJsXysjY3rgEFmCIyJ8ii44cQLUjNmJeHX0Y NYGeDlCpON/YpEHfJGA6zrXyoTjeERMWlIFVorErXhD6jEXLeM6OGVPBlN86orv90UWl P2Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959353; x=1754564153; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jv3iRJDYgpMDxRSvFG+C/LPY2Ji1gd5ewL7Azcb/wOA=; b=eY+oRgZ2cEO/w6mDgTN9PWzy4QveLJeQvexQDqMEj6GnJ6g/CsPxj1Mr47DdTSxc2b UNZmWlIQJV3qSzXGBtvKw66OWBGpQBoUc7tNfI26k4UU9ZzLvtkMWx/DScPe4ZZe5UP4 9vKBS37TgKBuT8qd9mDKlRyYg3Za4Wicdc83+xhoYOMSa3yT1Uv7EPFSFBTq52oONwtd KB/3rIFI95nTOYah+iF+fuUKZlxirkd21JTMN9WrJvE+Vn74yh+t78iW1LzDjBNAeSqD EXuuRYvIYjy3/FZurCpfjJIpn3SFIKyO1OWm2CJkKFP5+WKVlTsRIAGsaNVFnvGaBElR TXfw== X-Gm-Message-State: AOJu0YzBxmQ+oeRUxn5Z5LZ5/PBymriGSodwXD8kDhkhAZJtcZXUI6yy 5G/hb7Ntw3S04ci6b0JUduE4CObPzqD3RXKQlZo0itnKmpA8vLdpMBqg X-Gm-Gg: ASbGncuSwLCJZg1qgc5PVL9hYHSd8iMJ+04x6MedHngsPaXstb/fzGgaDGyO9dSIhVd r0aHGPfi/UB5IHrU5uD7EJenncxkoa/9XThzfr5XdxN/2RvqvuK1r4TMfEwYLvTQb65jRwZDLXv JWhMCEP5V3iD00YX7b0s2JV+gvzp53D4niNlJ/D1viu2XLVGjx+9d7sCqd84ue1dUgq5OCLl28E WU49XbUH45sDY3pyJ1SOInXYimkIij68FjUn9/ZhSsUg4f8I+0URX5PqEuHDTIoGfntfg9BC+lY jNGrcHzNlJwhXSHlMAVfqK5wIMGcx7ONDU9w9emislPrZB41MPHj+q6K36hxf0RptHWJnK/Ep7C y8c9qoA5nRdoOEmqeBOc= X-Google-Smtp-Source: AGHT+IHwTBBgUxn3U7tBA9I6lZ8ZpnW2PYiM9+n19M7sdQapa9jpb1FYib5G71azbdijP1XtbDK6XQ== X-Received: by 2002:a05:6000:2212:b0:3b7:944d:e5e6 with SMTP id ffacd0b85a97d-3b794ff1593mr5393750f8f.36.1753959353312; Thu, 31 Jul 2025 03:55:53 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:52 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 11/25] sched/deadline: Add dl_init_tg Date: Thu, 31 Jul 2025 12:55:29 +0200 Message-ID: <20250731105543.40832-12-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni This function is used to initialize and/or update a rt-cgroup dl_server, al= so accounting for the allocated bandwidth. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/deadline.c | 33 +++++++++++++++++++++++++++++++++ kernel/sched/sched.h | 1 + 2 files changed, 34 insertions(+) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index b8228f553fe..264838c4a85 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -365,6 +365,39 @@ void cancel_inactive_timer(struct sched_dl_entity *dl_= se) cancel_dl_timer(dl_se, &dl_se->inactive_timer); } =20 +#ifdef CONFIG_RT_GROUP_SCHED +void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 rt_peri= od) +{ + struct rq *rq =3D container_of(dl_se->dl_rq, struct rq, dl); + int is_active; + u64 new_bw; + + raw_spin_rq_lock_irq(rq); + is_active =3D dl_se->my_q->rt.rt_nr_running > 0; + + update_rq_clock(rq); + dl_server_stop(dl_se); + + new_bw =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); + dl_rq_change_utilization(rq, dl_se, new_bw); + + dl_se->dl_runtime =3D rt_runtime; + dl_se->dl_deadline =3D rt_period; + dl_se->dl_period =3D rt_period; + + dl_se->runtime =3D 0; + dl_se->deadline =3D 0; + + dl_se->dl_bw =3D new_bw; + dl_se->dl_density =3D new_bw; + + if (is_active) + dl_server_start(dl_se); + + raw_spin_rq_unlock_irq(rq); +} +#endif + static void dl_change_utilization(struct task_struct *p, u64 new_bw) { WARN_ON_ONCE(p->dl.flags & SCHED_FLAG_SUGOV); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 3283d824859..611e3757fea 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -394,6 +394,7 @@ extern void dl_server_init(struct sched_dl_entity *dl_s= e, struct dl_rq *dl_rq, dl_server_has_tasks_f has_tasks, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); +extern void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 = rt_period); =20 extern void dl_server_update_idle_time(struct rq *rq, struct task_struct *p); --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC2E42BFC8F for ; Thu, 31 Jul 2025 10:55:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959357; cv=none; b=CErVfcA+RBAv63lgE+KvcjnoySa9/y0L/GPVLC3UIeOkQATJ16pBVM8TBoh1CRk5g3uVELBmxmiG/1O9Qudjw/zc9PUCCLFDgH/hcNzbO9qVEDu1lJ7Fxw9rj4hvSGKSOTj4OTi4XM83Tq7KA8F/Hsan4Uq7ek39Hnsf7gUZ3vM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959357; c=relaxed/simple; bh=j3rSyFw1+tmCTEgOl0PEOIDQ8q+5Y19PGpaUbY4ELAY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gwOogvp3ecNe9cXbld8qKZm21ag0MJxU++a5ZrHfbjKgZMMotsjwjjSxpg5f+3FdnQONKr+wBHBvbySOKx8xwDvLVGi5P9pN4i7ZCpUtRA3hplc1LIla+wL790WAmeJEBEMMzKNNUAU2305L4WdwJNUjile3rXzIMii6rKiBX7A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ff0zefVR; arc=none smtp.client-ip=209.85.128.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ff0zefVR" Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-45629702e52so3015585e9.2 for ; Thu, 31 Jul 2025 03:55:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959354; x=1754564154; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MWI0wdc7axH18MCt0Tc2btZEL3SXmvH78G2j6OBM6hw=; b=ff0zefVRkF25JmhDxmE0J/bDXRN3b2k2JUKpCaZg2q6UbpLMO8iy1m/PryH2wAuTLy Eg1rWPts/tmrS6SeiH+u60XkNCdNovzZ86HnQByHg1hMJOMftoDcSucF/+hXEZipxjkX T/sK+fV0Ka2A1ccrqyBfXm/QGpcE532eBHARhwMw0l9rNzfal+jwLbLHGZUih2bmnt56 KK3FYsQsOalfhaG6hSXXXFgwukWlbAjONTeFruAtlOsAk5Lc+XZ31HxCGEGcsxAxekNS /tMMfxxsKflU6rMIFo8mPqTev7DsyW9K4CgHxuU/X98RyIb5qo3tecNswoKfHR/Xy6f3 J4Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959354; x=1754564154; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MWI0wdc7axH18MCt0Tc2btZEL3SXmvH78G2j6OBM6hw=; b=kzZUR9/Rzl6p9Z00Q9HsIU6GtF/X+Hj0pze1v4u5j1fyrxALUzysIy0/oNGcX6z0l9 0GH0a4qmLitgGt2fiXu+872N1rZpOBvO/t9KSbf4yysHJz/HzAj9uO2Cx9r2tJe/et6J JcwcJ25z/q4HbMV9/S8oF53ddFwFfH/CyrcavmC3XE21I2/h/T5066Skzfgpg8QSR8fF Z78e12cDTI92bMXIxqx2hFE+WLfZOHzXFqTMXDO9PyLjJGp+7lJtHkG/+JMMTmmPT4xB zLPvuujJT97sy+h4gXnJPFHaN9rvhiw0XWyOLUkGmVZ112kUfxMrm8th34+pQT/oS1dE B+aQ== X-Gm-Message-State: AOJu0YzZc57gGxXG6iwx7VdNQyg896J5PJ01NkGflGsjIwGqzoJeIhNo gtEt7dFRuP24X/jb3FigDp/YkIFkQiwQ+FNtmRzz7vYfKGIWzXIHri2+ X-Gm-Gg: ASbGncuAe709tiQ9y8TZQVebhk6xJeXqnb1eoi4JQVxwKwykHSZg2dnXjTi92PBXMzs /MUqVyF96+vNhcDCQCdpg4drVbr4nEH/ftch4E5gKYL6rHw4PmuuMocOp8P8Z7k4+vuW8vCXP3h iQqJvnmmCJ4qDEl3X4dQRYFSoDiWCION+nt6XLjVY+hckFUMO3Rodfwt11ykbKEjIZIkmtNw2Js 2jUw0CeL8rf7A+YCs01IxipBdymrka92dazlK7BfmBu1OBJdIoqyOEjjOYjN4MFTI8Dd9ufTv6F N9ptytpxfoM4IiGgB8dvtZGZJataV2SA/mB7DeLIWvNbma/Pua2FnRPoeqFSxpE2umqzqSTC+XU GntgwvQrCx+sB6T/2IEECkTZeyBR68Z6zzod8A9Yo X-Google-Smtp-Source: AGHT+IFhccSoMoOt0clDq1R5CDBIWX8hg30FplKcZE7h5wg3YTN2Hh81/fxYH0wN5lofMkp9y+4Gvg== X-Received: by 2002:a05:600c:4450:b0:456:1442:86e with SMTP id 5b1f17b1804b1-45892bc5fecmr61870885e9.21.1753959354120; Thu, 31 Jul 2025 03:55:54 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:53 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 12/25] sched/rt: Add {alloc/free}_rt_sched_group and dl_server specific functions Date: Thu, 31 Jul 2025 12:55:30 +0200 Message-ID: <20250731105543.40832-13-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add allocation and deallocation code for rt-cgroups. Add rt dl_server's spe= cific functions that pick the next eligible task to run. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/rt.c | 107 ++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 104 insertions(+), 3 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 38178003184..9c4ac6875a2 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -93,8 +93,39 @@ void unregister_rt_sched_group(struct task_group *tg) =20 void free_rt_sched_group(struct task_group *tg) { + int i; + if (!rt_group_sched_enabled()) return; + + for_each_possible_cpu(i) { + if (tg->dl_se) { + unsigned long flags; + + /* + * Since the dl timer is going to be cancelled, + * we risk to never decrease the running bw... + * Fix this issue by changing the group runtime + * to 0 immediately before freeing it. + */ + dl_init_tg(tg->dl_se[i], 0, tg->dl_se[i]->dl_period); + raw_spin_rq_lock_irqsave(cpu_rq(i), flags); + BUG_ON(tg->rt_rq[i]->rt_nr_running); + raw_spin_rq_unlock_irqrestore(cpu_rq(i), flags); + + hrtimer_cancel(&tg->dl_se[i]->dl_timer); + kfree(tg->dl_se[i]); + } + if (tg->rt_rq) { + struct rq *served_rq; + + served_rq =3D container_of(tg->rt_rq[i], struct rq, rt); + kfree(served_rq); + } + } + + kfree(tg->rt_rq); + kfree(tg->dl_se); } =20 void init_tg_rt_entry(struct task_group *tg, struct rq *served_rq, @@ -109,12 +140,77 @@ void init_tg_rt_entry(struct task_group *tg, struct r= q *served_rq, tg->dl_se[cpu] =3D dl_se; } =20 +static bool rt_server_has_tasks(struct sched_dl_entity *dl_se) +{ + return !!dl_se->my_q->rt.rt_nr_running; +} + +static struct task_struct *_pick_next_task_rt(struct rt_rq *rt_rq); +static inline void set_next_task_rt(struct rq *rq, struct task_struct *p, = bool first); +static struct task_struct *rt_server_pick(struct sched_dl_entity *dl_se) +{ + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + struct rq *rq =3D rq_of_rt_rq(rt_rq); + struct task_struct *p; + + if (dl_se->my_q->rt.rt_nr_running =3D=3D 0) + return NULL; + + p =3D _pick_next_task_rt(rt_rq); + set_next_task_rt(rq, p, true); + + return p; +} + int alloc_rt_sched_group(struct task_group *tg, struct task_group *parent) { + struct rq *s_rq; + struct sched_dl_entity *dl_se; + int i; + if (!rt_group_sched_enabled()) return 1; =20 + tg->rt_rq =3D kcalloc(nr_cpu_ids, sizeof(struct rt_rq *), GFP_KERNEL); + if (!tg->rt_rq) + goto err; + tg->dl_se =3D kcalloc(nr_cpu_ids, sizeof(dl_se), GFP_KERNEL); + if (!tg->dl_se) + goto err; + + init_dl_bandwidth(&tg->dl_bandwidth, 0, 0); + + for_each_possible_cpu(i) { + s_rq =3D kzalloc_node(sizeof(struct rq), + GFP_KERNEL, cpu_to_node(i)); + if (!s_rq) + goto err; + + dl_se =3D kzalloc_node(sizeof(struct sched_dl_entity), + GFP_KERNEL, cpu_to_node(i)); + if (!dl_se) + goto err_free_rq; + + init_rt_rq(&s_rq->rt); + init_dl_entity(dl_se); + dl_se->dl_runtime =3D tg->dl_bandwidth.dl_runtime; + dl_se->dl_period =3D tg->dl_bandwidth.dl_period; + dl_se->dl_deadline =3D dl_se->dl_period; + dl_se->dl_bw =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); + dl_se->dl_density =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); + dl_se->dl_server =3D 1; + + dl_server_init(dl_se, &cpu_rq(i)->dl, s_rq, rt_server_has_tasks, rt_serv= er_pick); + + init_tg_rt_entry(tg, s_rq, dl_se, i, parent->dl_se[i]); + } + return 1; + +err_free_rq: + kfree(s_rq); +err: + return 0; } =20 #else /* !CONFIG_RT_GROUP_SCHED: */ @@ -860,9 +956,14 @@ static struct sched_rt_entity *pick_next_rt_entity(str= uct rt_rq *rt_rq) return next; } =20 -static struct task_struct *_pick_next_task_rt(struct rq *rq) +static struct task_struct *_pick_next_task_rt(struct rt_rq *rt_rq) { - return NULL; + struct sched_rt_entity *rt_se; + + rt_se =3D pick_next_rt_entity(rt_rq); + BUG_ON(!rt_se); + + return rt_task_of(rt_se); } =20 static struct task_struct *pick_task_rt(struct rq *rq) @@ -872,7 +973,7 @@ static struct task_struct *pick_task_rt(struct rq *rq) if (!sched_rt_runnable(rq)) return NULL; =20 - p =3D _pick_next_task_rt(rq); + p =3D _pick_next_task_rt(&rq->rt); =20 return p; } --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCA0E2C08C1 for ; Thu, 31 Jul 2025 10:55:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959359; cv=none; b=bBEfz6jD4ALOXLXoPnrpouPsejlhujKQVKsPaOxgZCD1QCmQ4mhvbIN3vFeIkGqycqXO+ZLhCH0dZBEH5RYvLMBrCbmjbCincvUG5GIr4aOoOkH3OGqcbHKV8/pIU74fVCu9WNkL+vc3WYkjyDsvkWu8Px9ZRA8wo1dfyl7vijc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959359; c=relaxed/simple; bh=AVckV5NwmShnHX3/Ld3i0C0oi6LECmS+Pgn/BVXJQn8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p8MSnQDM1jSThfvZWJHupGEwQfPifpQ9JsEx/TDXCHDI5mDUuvlJcQYpff417Gk2Jqum4rLg04ChmY5mlgLn5P1NBDiKrSUEn8YX5o684AovIu7KPD1thej3zpmXUDLcUqWuqaRy+1S7wHBA3lfdVPtuMjZKIeI4xobZ3Sf4eB0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=S3TpfjXn; arc=none smtp.client-ip=209.85.221.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="S3TpfjXn" Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-3b7823559a5so488521f8f.0 for ; Thu, 31 Jul 2025 03:55:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959355; x=1754564155; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nJBdY2a1s0dCZdurrikDScRYILWYnnIvEZxcIlp4hRs=; b=S3TpfjXnwP1n8FzSwtTrnmSrn5NZJZxWsySAu9kzJzDj2EBKx6aZ3u2/9A8kDH7Fpq dKEn8zPV+l8tRYTm2TuEH721QB7hKOqwg+fHk/4qjxrRypJbxtvsHsiLjNhgKoOM+pnF pZ+FQzcKkMYhVxG9tVkvQXbC3fCqdSzmVnz47s9O0bcUr0UjLCHhpjU8exhp8dEWjhxQ KKCrCK7BjoSPEVmZn7WX+cr6m8XUzweRozVny/QcsK8JDuj4ub+sEY2AHTPZUvOiejEV sDZAqhMzrO2FtVRbOLRNmhM5ZewGsKqIZ9tqxzCPVyjRdNfJt6rCED5MaLxaPC/G/neD ecpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959355; x=1754564155; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nJBdY2a1s0dCZdurrikDScRYILWYnnIvEZxcIlp4hRs=; b=uYAJqZ4xnX/NvFAMRf6JVKgseMVHx2oXhs4gPHhE4CVls2bHUV6GatL8gF1YGdQROJ BtxiOZN1pyX6KVqj6FtngdZwORS2xxeGhR8TIFosczz/Ll4r/rUTV3GKRgU/BB+noQAc V1yTgeBhkJ0GVQP53b0kwUYzMZE+H6LSMfC6m86/yjyarNeicbyhF1jUSwnDjTHIsfic dsaKWdHXzcdaQIVAiQkOTBlYVW/h3SlALFX2qI0ebJHE4yqD6zY/uHLbyOaTD/GiFI0m 5QOlLy/MoTyzyIqSn9XEVm/RNOjviYqq5zeovSuU6F1reWa3N18M34ADl8kujAwM6MLN KPmA== X-Gm-Message-State: AOJu0YxgqiIK82NbK1qf8nVdJhR9tLPdG3ZbfeW+ZWhO/nkB+fE9oDpK pTN8EqllE+aCMPJkHAj6fJyW6I4fPO1QCNADThwqkVw72ZJLAvxoqKya X-Gm-Gg: ASbGncvEueUieyvfqPihRxkFOMske+FjB9eAkN4rKbtnmvdUvnmRuHTX7sJli3ZvEhl jQjhNR57M6euPgMdoyP/DpShYWw2Gi/Ouck26Ugib1feit23DdP6liaHLDeotojZQpjXKf40ESo 5MkpWZhwpkptGfp/SJVmyl5/4u3yM+tRcsO2GmPB7hVolFPq/dSgkMoHBORM1uwPbNte165rSaU 2dtAgcN62dIEnov3wjLjGAptQu/Be9fsgs1pcwv4+/qUHu/sENJUeKc+pDwUHYKfFol/eXGovct C0yGzjzvRr+yuAZVoF8TsX/Ew63H+Oey1ydjIac9WycGt3LHbYUVY91kiyoAGifswjs+bSc462P VfYDUjRR1ItBej+FLftw= X-Google-Smtp-Source: AGHT+IHHCwhtDRLgfAHxopC2dYxs4PgDo3S+VW/1bljunXIHJUQIPFl4YAQ22KG1t3RIjmPSR9VMzQ== X-Received: by 2002:a05:6000:2008:b0:3b7:931d:379a with SMTP id ffacd0b85a97d-3b79503d691mr4575955f8f.58.1753959354901; Thu, 31 Jul 2025 03:55:54 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:54 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 13/25] sched/rt: Add HCBS related checks and operations for rt tasks Date: Thu, 31 Jul 2025 12:55:31 +0200 Message-ID: <20250731105543.40832-14-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add checks wheter a task belongs to the root cgroup or a rt-cgroup, since H= CBS reuses the rt classes' scheduler, and operate accordingly where needed. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/core.c | 3 + kernel/sched/deadline.c | 16 ++++- kernel/sched/rt.c | 147 +++++++++++++++++++++++++++++++++++++--- kernel/sched/sched.h | 6 +- kernel/sched/syscalls.c | 13 ++++ 5 files changed, 171 insertions(+), 14 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 3a69cb906c3..6173684a02b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2196,6 +2196,9 @@ void wakeup_preempt(struct rq *rq, struct task_struct= *p, int flags) { struct task_struct *donor =3D rq->donor; =20 + if (is_dl_group(rt_rq_of_se(&p->rt)) && task_has_rt_policy(p)) + resched_curr(rq); + if (p->sched_class =3D=3D donor->sched_class) donor->sched_class->wakeup_preempt(rq, p, flags); else if (sched_class_above(p->sched_class, donor->sched_class)) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 264838c4a85..b948000f29f 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1866,7 +1866,13 @@ void inc_dl_tasks(struct sched_dl_entity *dl_se, str= uct dl_rq *dl_rq) u64 deadline =3D dl_se->deadline; =20 dl_rq->dl_nr_running++; - add_nr_running(rq_of_dl_rq(dl_rq), 1); + if (!dl_server(dl_se) || dl_se =3D=3D &rq_of_dl_rq(dl_rq)->fair_server) { + add_nr_running(rq_of_dl_rq(dl_rq), 1); + } else { + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + + add_nr_running(rq_of_dl_rq(dl_rq), rt_rq->rt_nr_running); + } =20 inc_dl_deadline(dl_rq, deadline); } @@ -1876,7 +1882,13 @@ void dec_dl_tasks(struct sched_dl_entity *dl_se, str= uct dl_rq *dl_rq) { WARN_ON(!dl_rq->dl_nr_running); dl_rq->dl_nr_running--; - sub_nr_running(rq_of_dl_rq(dl_rq), 1); + if (!dl_server(dl_se) || dl_se =3D=3D &rq_of_dl_rq(dl_rq)->fair_server) { + sub_nr_running(rq_of_dl_rq(dl_rq), 1); + } else { + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + + sub_nr_running(rq_of_dl_rq(dl_rq), rt_rq->rt_nr_running); + } =20 dec_dl_deadline(dl_rq, dl_se->deadline); } diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 9c4ac6875a2..83695e11db4 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -419,6 +419,7 @@ static inline int rt_se_prio(struct sched_rt_entity *rt= _se) static void update_curr_rt(struct rq *rq) { struct task_struct *donor =3D rq->donor; + struct rt_rq *rt_rq; s64 delta_exec; =20 if (donor->sched_class !=3D &rt_sched_class) @@ -428,8 +429,18 @@ static void update_curr_rt(struct rq *rq) if (unlikely(delta_exec <=3D 0)) return; =20 - if (!rt_bandwidth_enabled()) + if (!rt_group_sched_enabled()) return; + + if (!dl_bandwidth_enabled()) + return; + + rt_rq =3D rt_rq_of_se(&donor->rt); + if (is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + dl_server_update(dl_se, delta_exec); + } } =20 static void @@ -440,7 +451,7 @@ inc_rt_prio_smp(struct rt_rq *rt_rq, int prio, int prev= _prio) /* * Change rq's cpupri only if rt_rq is the top queue. */ - if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && &rq->rt !=3D rt_rq) + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) return; =20 if (rq->online && prio < prev_prio) @@ -455,7 +466,7 @@ dec_rt_prio_smp(struct rt_rq *rt_rq, int prio, int prev= _prio) /* * Change rq's cpupri only if rt_rq is the top queue. */ - if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && &rq->rt !=3D rt_rq) + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) return; =20 if (rq->online && rt_rq->highest_prio.curr !=3D prev_prio) @@ -524,6 +535,15 @@ void inc_rt_tasks(struct sched_rt_entity *rt_se, struc= t rt_rq *rt_rq) rt_rq->rr_nr_running +=3D rt_se_rr_nr_running(rt_se); =20 inc_rt_prio(rt_rq, rt_se_prio(rt_se)); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (!dl_se->dl_throttled) + add_nr_running(rq_of_rt_rq(rt_rq), 1); + } else { + add_nr_running(rq_of_rt_rq(rt_rq), 1); + } } =20 static inline @@ -534,6 +554,15 @@ void dec_rt_tasks(struct sched_rt_entity *rt_se, struc= t rt_rq *rt_rq) rt_rq->rr_nr_running -=3D rt_se_rr_nr_running(rt_se); =20 dec_rt_prio(rt_rq, rt_se_prio(rt_se)); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (!dl_se->dl_throttled) + sub_nr_running(rq_of_rt_rq(rt_rq), 1); + } else { + sub_nr_running(rq_of_rt_rq(rt_rq), 1); + } } =20 /* @@ -715,6 +744,14 @@ enqueue_task_rt(struct rq *rq, struct task_struct *p, = int flags) check_schedstat_required(); update_stats_wait_start_rt(rt_rq_of_se(rt_se), rt_se); =20 + /* Task arriving in an idle group of tasks. */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + is_dl_group(rt_rq) && rt_rq->rt_nr_running =3D=3D 0) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + dl_server_start(dl_se); + } + enqueue_rt_entity(rt_se, flags); =20 if (task_is_blocked(p)) @@ -734,6 +771,14 @@ static bool dequeue_task_rt(struct rq *rq, struct task= _struct *p, int flags) =20 dequeue_pushable_task(rt_rq, p); =20 + /* Last task of the task group. */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + is_dl_group(rt_rq) && rt_rq->rt_nr_running =3D=3D 0) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + dl_server_stop(dl_se); + } + return true; } =20 @@ -891,6 +936,34 @@ static void wakeup_preempt_rt(struct rq *rq, struct ta= sk_struct *p, int flags) { struct task_struct *donor =3D rq->donor; =20 + if (!rt_group_sched_enabled()) + goto no_group_sched; + + if (is_dl_group(rt_rq_of_se(&p->rt)) && + is_dl_group(rt_rq_of_se(&rq->curr->rt))) { + struct sched_dl_entity *dl_se, *curr_dl_se; + + dl_se =3D dl_group_of(rt_rq_of_se(&p->rt)); + curr_dl_se =3D dl_group_of(rt_rq_of_se(&rq->curr->rt)); + + if (dl_entity_preempt(dl_se, curr_dl_se)) { + resched_curr(rq); + return; + } else if (!dl_entity_preempt(curr_dl_se, dl_se)) { + if (p->prio < rq->curr->prio) { + resched_curr(rq); + return; + } + } + return; + } else if (is_dl_group(rt_rq_of_se(&p->rt))) { + resched_curr(rq); + return; + } else if (is_dl_group(rt_rq_of_se(&rq->curr->rt))) { + return; + } + +no_group_sched: if (p->prio < donor->prio) { resched_curr(rq); return; @@ -1609,12 +1682,36 @@ static void pull_rt_task(struct rq *this_rq) resched_curr(this_rq); } =20 +#ifdef CONFIG_RT_GROUP_SCHED +static int group_push_rt_task(struct rt_rq *rt_rq) +{ + struct rq *rq =3D rq_of_rt_rq(rt_rq); + + if (is_dl_group(rt_rq)) + return 0; + + return push_rt_task(rq, false); +} + +static void group_push_rt_tasks(struct rt_rq *rt_rq) +{ + while (group_push_rt_task(rt_rq)) + ; +} +#else +static void group_push_rt_tasks(struct rt_rq *rt_rq) +{ + push_rt_tasks(rq_of_rt_rq(rt_rq)); +} +#endif + /* * If we are not running and we are not going to reschedule soon, we should * try to push tasks away now */ static void task_woken_rt(struct rq *rq, struct task_struct *p) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); bool need_to_push =3D !task_on_cpu(rq, p) && !test_tsk_need_resched(rq->curr) && p->nr_cpus_allowed > 1 && @@ -1623,7 +1720,7 @@ static void task_woken_rt(struct rq *rq, struct task_= struct *p) rq->donor->prio <=3D p->prio); =20 if (need_to_push) - push_rt_tasks(rq); + group_push_rt_tasks(rt_rq); } =20 /* Assumes rq->lock is held */ @@ -1632,6 +1729,7 @@ static void rq_online_rt(struct rq *rq) if (rq->rt.overloaded) rt_set_overload(rq); =20 + /*FIXME: Enable the dl server! */ cpupri_set(&rq->rd->cpupri, rq->cpu, rq->rt.highest_prio.curr); } =20 @@ -1641,6 +1739,7 @@ static void rq_offline_rt(struct rq *rq) if (rq->rt.overloaded) rt_clear_overload(rq); =20 + /* FIXME: Disable the dl server! */ cpupri_set(&rq->rd->cpupri, rq->cpu, CPUPRI_INVALID); } =20 @@ -1650,6 +1749,8 @@ static void rq_offline_rt(struct rq *rq) */ static void switched_from_rt(struct rq *rq, struct task_struct *p) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); + /* * If there are other RT tasks then we will reschedule * and the scheduling of the other RT tasks will handle @@ -1657,10 +1758,11 @@ static void switched_from_rt(struct rq *rq, struct = task_struct *p) * we may need to handle the pulling of RT tasks * now. */ - if (!task_on_rq_queued(p) || rq->rt.rt_nr_running) + if (!task_on_rq_queued(p) || rt_rq->rt_nr_running) return; =20 - rt_queue_pull_task(rq); + if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED)) + rt_queue_pull_task(rq); } =20 void __init init_sched_rt_class(void) @@ -1695,8 +1797,17 @@ static void switched_to_rt(struct rq *rq, struct tas= k_struct *p) * then see if we can move to another run queue. */ if (task_on_rq_queued(p)) { + +#ifndef CONFIG_RT_GROUP_SCHED if (p->nr_cpus_allowed > 1 && rq->rt.overloaded) rt_queue_push_tasks(rq); +#else + if (rt_rq_of_se(&p->rt)->overloaded) { + } else { + if (p->prio < rq->curr->prio) + resched_curr(rq); + } +#endif if (p->prio < rq->donor->prio && cpu_online(cpu_of(rq))) resched_curr(rq); } @@ -1709,6 +1820,8 @@ static void switched_to_rt(struct rq *rq, struct task= _struct *p) static void prio_changed_rt(struct rq *rq, struct task_struct *p, int oldprio) { + struct rt_rq *rt_rq =3D rt_rq_of_se(&p->rt); + if (!task_on_rq_queued(p)) return; =20 @@ -1717,16 +1830,25 @@ prio_changed_rt(struct rq *rq, struct task_struct *= p, int oldprio) * If our priority decreases while running, we * may need to pull tasks to this runqueue. */ - if (oldprio < p->prio) + if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED) && oldprio < p->prio) rt_queue_pull_task(rq); =20 /* * If there's a higher priority task waiting to run * then reschedule. */ - if (p->prio > rq->rt.highest_prio.curr) + if (p->prio > rt_rq->highest_prio.curr) resched_curr(rq); } else { + /* + * This task is not running, thus we check against the currently + * running task for preemption. We can preempt only if both tasks are + * in the same cgroup or on the global runqueue. + */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + rt_rq_of_se(&p->rt)->tg !=3D rt_rq_of_se(&rq->curr->rt)->tg) + return; + /* * This task is not running, but if it is * greater than the current running task @@ -1821,7 +1943,16 @@ static unsigned int get_rr_interval_rt(struct rq *rq= , struct task_struct *task) #ifdef CONFIG_SCHED_CORE static int task_is_throttled_rt(struct task_struct *p, int cpu) { +#ifdef CONFIG_RT_GROUP_SCHED + struct rt_rq *rt_rq; + + rt_rq =3D task_group(p)->rt_rq[cpu]; + WARN_ON(!rt_group_sched_enabled() && rt_rq->tg !=3D &root_task_group); + + return dl_group_of(rt_rq)->dl_throttled; +#else return 0; +#endif } #endif /* CONFIG_SCHED_CORE */ =20 diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 611e3757fea..8bf8af7064f 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2171,7 +2171,7 @@ static inline void set_task_rq(struct task_struct *p,= unsigned int cpu) if (!rt_group_sched_enabled()) tg =3D &root_task_group; p->rt.rt_rq =3D tg->rt_rq[cpu]; - p->rt.parent =3D tg->rt_se[cpu]; + p->dl.dl_rq =3D &cpu_rq(cpu)->dl; #endif /* CONFIG_RT_GROUP_SCHED */ } =20 @@ -2727,6 +2727,7 @@ static inline void add_nr_running(struct rq *rq, unsi= gned count) =20 static inline void sub_nr_running(struct rq *rq, unsigned count) { + BUG_ON(rq->nr_running < count); rq->nr_running -=3D count; if (trace_sched_update_nr_running_tp_enabled()) { call_trace_sched_update_nr_running(rq, -count); @@ -3057,9 +3058,6 @@ extern bool sched_smp_initialized; #ifdef CONFIG_RT_GROUP_SCHED static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) { -#ifdef CONFIG_SCHED_DEBUG - WARN_ON_ONCE(rt_se->my_q); -#endif return container_of(rt_se, struct task_struct, rt); } =20 diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index 93a9c03b28e..7c1f7649477 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -626,6 +626,19 @@ int __sched_setscheduler(struct task_struct *p, change: =20 if (user) { +#ifdef CONFIG_RT_GROUP_SCHED + /* + * Do not allow real-time tasks into groups that have no runtime + * assigned. + */ + if (rt_group_sched_enabled() && + dl_bandwidth_enabled() && rt_policy(policy) && + task_group(p)->dl_bandwidth.dl_runtime =3D=3D 0 && + !task_group_is_autogroup(task_group(p))) { + retval =3D -EPERM; + goto unlock; + } +#endif if (dl_bandwidth_enabled() && dl_policy(policy) && !(attr->sched_flags & SCHED_FLAG_SUGOV)) { cpumask_t *span =3D rq->rd->span; --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6FE532C1581 for ; Thu, 31 Jul 2025 10:55:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959359; cv=none; b=u9qFVwhYN4XWjTpq3ZMd28qjzG5XhzMP8H+eROk/fMI8TTaeuXRy4/X5nirzIk8a4z63jy3QrU84SumKpaISaiN5hGXdML+mp7T0D36T7XunbV+9TcrX2hNHeNH/BDnFhPmfS+2QgJpzFowqWX97osSOok2nfDo/RhjBV/3+xdU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959359; c=relaxed/simple; bh=xO0PgRp2TDyPC9ly5AjN8AEQNEsBUUHcZB6gNAbif/o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KdPb2WdygR5rWZF25BiakRZrRgTuxCVwEfZvYF1MaBeUac4Lglfjf7LO2Dz+qml29O/k303G6bfH0GYMQlpBKClDMpMivmIWkdwp2H6d0gvn1kUO9uyZtoi4brS2JvevYvmz/0q9sVs4NKs7+YgyWS72qfQ2Z/rVmRrVUcNUbm0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TjPVzWS7; arc=none smtp.client-ip=209.85.221.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TjPVzWS7" Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-3b7848df30cso1103901f8f.0 for ; Thu, 31 Jul 2025 03:55:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959356; x=1754564156; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GPD4AKGlLYy7ld0yhNWiLxKSPOb9hAQPD23s+30e4P8=; b=TjPVzWS7ie/+QT/O/Vpk2hIIuTfr9TGJHWC4hGvE6YH7Jp5oyrBFZD15VwvmFzlW7Q 18ydfnhop3itwVNQLod6QT5zaPi8wp7slijUIA2L9m8PLg7V9EMxESqBv7NLHVH+Nuyp fOYfhCS0xBEOaq5adnMpls54MI1vl0kZ+8bICB2vMYZV6NiQS77OSbegAum3gw6ckRap FNdzxJXqCslMQsHgBt9UzhgIAr/FjH1dxNemnJpZzMfyIuezjvJd6/FtWOcwDT0RaXig u08+LV172H/0l3kv3PBiEOZTzyTvD8SyvXC1SkfpC5ssdWbtQnERxKrjzXKw4PnlI4lS e5xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959356; x=1754564156; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GPD4AKGlLYy7ld0yhNWiLxKSPOb9hAQPD23s+30e4P8=; b=hfUrAxHrr/9zJ/ZzT5LeVJMpyGTS0uqqH7Hndl3pNGVK2/jPK1hM7AoNBhNFf9WhUE HrWmK89b1LQ/VfwMGG7Bi1j0tiQ8H7vyJLCfi7TkFKm6NVKu8aTV6BqirMobd6oFxEQc T+i1SbS2YhtGpUQed8lmJuVbeNGudMGrq4X8kmNoRqhSxNf05PohpARu8Hab4IsP3MBm cRtNdDmnMLUzzFIk7Kk8qNpXOHq/g8a2h5+zGTHnrI84jEfuxrUv/8D4qiMir/2qd42V R8YscC3nwfojv3X9prvThozf16QTtJkOWRB9MKCBXfsXembxTjB8GKcUQ/OrKe43NKsc ITfQ== X-Gm-Message-State: AOJu0YxPTtjHWnTvjANElAnkU3Zk2jzZe+tAk6VgXKEcXsWZMIQmcMZP w5AOyHknBRLTNK1m//BuBq7BudItYVkyJcCHOkLJQGZykkWj7nvCEjeY X-Gm-Gg: ASbGncs5CXjRo0MmF5NAOQ+DKRaXT7GYVh28EiCFY/d9xv6Dl1Ls6+sKk2QvEgBT9cE 5nhx7Dci4XNnp2eEx1o6nINIwDo3lp9O8W+8yrth9JNy/SHhBkGP4+fFgGubacFC8bbesiS/BxH Ms0jO6OnBAVTinM0fMR/hxuTjkPkVuEzcrzLGFd7pft/PCXEPSZz3tnOrm8O2aJCPcXULisAY0b RKp/1XOpxt7g3WKaS4IZxQe0kyoknbyjYqXWAkYcefS4TPMqd95uleeM37UTnmCCGpsTRexzivW B+TxuOOdADRLYiO5mkAcLkPmWM/og2DL6f/4dHIFuKWpkkwQTEefIYFbA3DWbmnX7rCStXbTxOx DVPXQATrGd/RdOhSEwVE= X-Google-Smtp-Source: AGHT+IGkctD7bV3W9py+qBKbrRx/gABjDK9obtz5h/vD52b8y7fAtV43jnezdQfMHn1arsNgvR+OYA== X-Received: by 2002:a5d:5f4c:0:b0:3a5:281b:9fac with SMTP id ffacd0b85a97d-3b79d4b4f0dmr827056f8f.17.1753959355678; Thu, 31 Jul 2025 03:55:55 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:55 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 14/25] sched/rt: Update rt-cgroup schedulability checks Date: Thu, 31 Jul 2025 12:55:32 +0200 Message-ID: <20250731105543.40832-15-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Update schedulability checks and setup of runtime/period for rt-cgroups. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/core.c | 6 ++++ kernel/sched/deadline.c | 46 ++++++++++++++++++++++---- kernel/sched/rt.c | 72 +++++++++++++++++++++++------------------ kernel/sched/sched.h | 1 + 4 files changed, 88 insertions(+), 37 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 6173684a02b..63cb9271052 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9277,6 +9277,12 @@ cpu_cgroup_css_alloc(struct cgroup_subsys_state *par= ent_css) return &root_task_group.css; } =20 + /* Do not allow cpu_cgroup hierachies with depth greater than 2. */ +#ifdef CONFIG_RT_GROUP_SCHED + if (parent !=3D &root_task_group) + return ERR_PTR(-EINVAL); +#endif + tg =3D sched_create_group(parent); if (IS_ERR(tg)) return ERR_PTR(-ENOMEM); diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index b948000f29f..7b131630743 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -365,7 +365,47 @@ void cancel_inactive_timer(struct sched_dl_entity *dl_= se) cancel_dl_timer(dl_se, &dl_se->inactive_timer); } =20 +/* + * Used for dl_bw check and update, used under sched_rt_handler()::mutex a= nd + * sched_domains_mutex. + */ +u64 dl_cookie; + #ifdef CONFIG_RT_GROUP_SCHED +int dl_check_tg(unsigned long total) +{ + unsigned long flags; + int which_cpu; + int cpus; + struct dl_bw *dl_b; + u64 gen =3D ++dl_cookie; + + for_each_possible_cpu(which_cpu) { + rcu_read_lock_sched(); + + if (!dl_bw_visited(which_cpu, gen)) { + cpus =3D dl_bw_cpus(which_cpu); + dl_b =3D dl_bw_of(which_cpu); + + raw_spin_lock_irqsave(&dl_b->lock, flags); + + if (dl_b->bw !=3D -1 && + dl_b->bw * cpus < dl_b->total_bw + total * cpus) { + raw_spin_unlock_irqrestore(&dl_b->lock, flags); + rcu_read_unlock_sched(); + + return 0; + } + + raw_spin_unlock_irqrestore(&dl_b->lock, flags); + } + + rcu_read_unlock_sched(); + } + + return 1; +} + void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 rt_peri= od) { struct rq *rq =3D container_of(dl_se->dl_rq, struct rq, dl); @@ -3139,12 +3179,6 @@ DEFINE_SCHED_CLASS(dl) =3D { #endif }; =20 -/* - * Used for dl_bw check and update, used under sched_rt_handler()::mutex a= nd - * sched_domains_mutex. - */ -u64 dl_cookie; - int sched_dl_global_validate(void) { u64 runtime =3D global_rt_runtime(); diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 83695e11db4..bd11f4a03f7 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1996,11 +1996,6 @@ DEFINE_SCHED_CLASS(rt) =3D { }; =20 #ifdef CONFIG_RT_GROUP_SCHED -/* - * Ensure that the real time constraints are schedulable. - */ -static DEFINE_MUTEX(rt_constraints_mutex); - static inline int tg_has_rt_tasks(struct task_group *tg) { struct task_struct *task; @@ -2034,8 +2029,8 @@ static int tg_rt_schedulable(struct task_group *tg, v= oid *data) unsigned long total, sum =3D 0; u64 period, runtime; =20 - period =3D ktime_to_ns(tg->rt_bandwidth.rt_period); - runtime =3D tg->rt_bandwidth.rt_runtime; + period =3D tg->dl_bandwidth.dl_period; + runtime =3D tg->dl_bandwidth.dl_runtime; =20 if (tg =3D=3D d->tg) { period =3D d->rt_period; @@ -2051,8 +2046,7 @@ static int tg_rt_schedulable(struct task_group *tg, v= oid *data) /* * Ensure we don't starve existing RT tasks if runtime turns zero. */ - if (rt_bandwidth_enabled() && !runtime && - tg->rt_bandwidth.rt_runtime && tg_has_rt_tasks(tg)) + if (dl_bandwidth_enabled() && !runtime && tg_has_rt_tasks(tg)) return -EBUSY; =20 if (WARN_ON(!rt_group_sched_enabled() && tg !=3D &root_task_group)) @@ -2066,12 +2060,17 @@ static int tg_rt_schedulable(struct task_group *tg,= void *data) if (total > to_ratio(global_rt_period(), global_rt_runtime())) return -EINVAL; =20 + if (tg =3D=3D &root_task_group) { + if (!dl_check_tg(total)) + return -EBUSY; + } + /* * The sum of our children's runtime should not exceed our own. */ list_for_each_entry_rcu(child, &tg->children, siblings) { - period =3D ktime_to_ns(child->rt_bandwidth.rt_period); - runtime =3D child->rt_bandwidth.rt_runtime; + period =3D child->dl_bandwidth.dl_period; + runtime =3D child->dl_bandwidth.dl_runtime; =20 if (child =3D=3D d->tg) { period =3D d->rt_period; @@ -2097,6 +2096,20 @@ static int __rt_schedulable(struct task_group *tg, u= 64 period, u64 runtime) .rt_runtime =3D runtime, }; =20 + /* + * Since we truncate DL_SCALE bits, make sure we're at least + * that big. + */ + if (runtime !=3D 0 && runtime < (1ULL << DL_SCALE)) + return -EINVAL; + + /* + * Since we use the MSB for wrap-around and sign issues, make + * sure it's not set (mind that period can be equal to zero). + */ + if (period & (1ULL << 63)) + return -EINVAL; + rcu_read_lock(); ret =3D walk_tg_tree(tg_rt_schedulable, tg_nop, &data); rcu_read_unlock(); @@ -2107,6 +2120,7 @@ static int __rt_schedulable(struct task_group *tg, u6= 4 period, u64 runtime) static int tg_set_rt_bandwidth(struct task_group *tg, u64 rt_period, u64 rt_runtime) { + static DEFINE_MUTEX(rt_constraints_mutex); int i, err =3D 0; =20 /* @@ -2126,34 +2140,30 @@ static int tg_set_rt_bandwidth(struct task_group *t= g, if (rt_runtime !=3D RUNTIME_INF && rt_runtime > max_rt_runtime) return -EINVAL; =20 - mutex_lock(&rt_constraints_mutex); + guard(mutex)(&rt_constraints_mutex); err =3D __rt_schedulable(tg, rt_period, rt_runtime); if (err) - goto unlock; + return err; =20 - raw_spin_lock_irq(&tg->rt_bandwidth.rt_runtime_lock); - tg->rt_bandwidth.rt_period =3D ns_to_ktime(rt_period); - tg->rt_bandwidth.rt_runtime =3D rt_runtime; + guard(raw_spinlock_irq)(&tg->dl_bandwidth.dl_runtime_lock); + tg->dl_bandwidth.dl_period =3D rt_period; + tg->dl_bandwidth.dl_runtime =3D rt_runtime; =20 - for_each_possible_cpu(i) { - struct rt_rq *rt_rq =3D tg->rt_rq[i]; + if (tg =3D=3D &root_task_group) + return 0; =20 - raw_spin_lock(&rt_rq->rt_runtime_lock); - rt_rq->rt_runtime =3D rt_runtime; - raw_spin_unlock(&rt_rq->rt_runtime_lock); + for_each_possible_cpu(i) { + dl_init_tg(tg->dl_se[i], rt_runtime, rt_period); } - raw_spin_unlock_irq(&tg->rt_bandwidth.rt_runtime_lock); -unlock: - mutex_unlock(&rt_constraints_mutex); =20 - return err; + return 0; } =20 int sched_group_set_rt_runtime(struct task_group *tg, long rt_runtime_us) { u64 rt_runtime, rt_period; =20 - rt_period =3D ktime_to_ns(tg->rt_bandwidth.rt_period); + rt_period =3D tg->dl_bandwidth.dl_period; rt_runtime =3D (u64)rt_runtime_us * NSEC_PER_USEC; if (rt_runtime_us < 0) rt_runtime =3D RUNTIME_INF; @@ -2167,10 +2177,10 @@ long sched_group_rt_runtime(struct task_group *tg) { u64 rt_runtime_us; =20 - if (tg->rt_bandwidth.rt_runtime =3D=3D RUNTIME_INF) + if (tg->dl_bandwidth.dl_runtime =3D=3D RUNTIME_INF) return -1; =20 - rt_runtime_us =3D tg->rt_bandwidth.rt_runtime; + rt_runtime_us =3D tg->dl_bandwidth.dl_runtime; do_div(rt_runtime_us, NSEC_PER_USEC); return rt_runtime_us; } @@ -2183,7 +2193,7 @@ int sched_group_set_rt_period(struct task_group *tg, = u64 rt_period_us) return -EINVAL; =20 rt_period =3D rt_period_us * NSEC_PER_USEC; - rt_runtime =3D tg->rt_bandwidth.rt_runtime; + rt_runtime =3D tg->dl_bandwidth.dl_runtime; =20 return tg_set_rt_bandwidth(tg, rt_period, rt_runtime); } @@ -2192,7 +2202,7 @@ long sched_group_rt_period(struct task_group *tg) { u64 rt_period_us; =20 - rt_period_us =3D ktime_to_ns(tg->rt_bandwidth.rt_period); + rt_period_us =3D tg->dl_bandwidth.dl_period; do_div(rt_period_us, NSEC_PER_USEC); return rt_period_us; } @@ -2207,7 +2217,7 @@ static int sched_rt_global_constraints(void) int sched_rt_can_attach(struct task_group *tg, struct task_struct *tsk) { /* Don't accept real-time tasks when there is no way for them to run */ - if (rt_group_sched_enabled() && rt_task(tsk) && tg->rt_bandwidth.rt_runti= me =3D=3D 0) + if (rt_group_sched_enabled() && rt_task(tsk) && tg->dl_bandwidth.dl_runti= me =3D=3D 0) return 0; =20 return 1; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 8bf8af7064f..9f235df4bf1 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -394,6 +394,7 @@ extern void dl_server_init(struct sched_dl_entity *dl_s= e, struct dl_rq *dl_rq, dl_server_has_tasks_f has_tasks, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); +extern int dl_check_tg(unsigned long total); extern void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 = rt_period); =20 extern void dl_server_update_idle_time(struct rq *rq, --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AF0F2C15A5 for ; Thu, 31 Jul 2025 10:55:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959360; cv=none; b=OJGVsNibXSrXfb76eZILwZkGWKm2/v+2+zm3kvt7yIaOeo5cffErbXsri4jUgHBceVCc3378DteSmQrhY5SeyNsYbtSqzApM2R4uthfsAqFHpFIwSbF7augBkqlWz0hmrLKlgkOtog7hfZMIxrR4X2V55tjN/loyGCVXw3pqcCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959360; c=relaxed/simple; bh=YOfu9u/lEMDWROzeoesVEFAQDL2hwPLy1q02eqDMkLc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F8XQqzCKyKs9tLTq1w34V7IuCfBmHiG13/5FIvVemZPKlWaBePDPUSm827ARV0v7LiL3rluJ/7YPC3izITVQ8gPXWqGpPR61RvpikBZgYJz5aLjXiF0iUymp8KdkpHl1jNmSnmPT7liOvyxjkxiCZ7xAA2dcEZ1K1SGn+d6vqbw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OyDCRJRZ; arc=none smtp.client-ip=209.85.221.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OyDCRJRZ" Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-3b782cca9a0so630761f8f.1 for ; Thu, 31 Jul 2025 03:55:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959356; x=1754564156; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4Xqu0xZcMwu5bTVe+RW26RwXnOxaIGhBZuY4+vQOKNA=; b=OyDCRJRZgXfk34x5IuInWuOV+3wbNg6MBlZOpcwHuKCvbFOH9EAsEbanw6S5Q2LuIf HUsT3Nm82nlHlYT8Lc2w2WYnWT7PvVJtysDP9RPFTCon12bLt/AVTuoG4KD2u2xojDuU mavw4wZpijDFhSu8o6uEZm+Jb4MvHq0ei9RN8OvXA/X1/uoVBT9adndsWGhLNMJh90/8 GU8cnw4pAY/6OItiYSl8Yj6c0EkzYbL42Pmtom2xGdTeN/eQFX+4vEoNkdt5xMEYEFjC SbZrF985eHygS5GHObCDlY6XfpxVLEcndrG2NY6Y2FEpynCMfUcMAV50wz0CQ9mn8cTb z6Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959356; x=1754564156; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4Xqu0xZcMwu5bTVe+RW26RwXnOxaIGhBZuY4+vQOKNA=; b=E75gxNoKfHsNW3l7J6W970trFPfjm54FRrgEsLgSlyC7PT+3I2bMTvc50TD+KYR8eh IWUqfKHn0P9dOp18nhLwgdthCz5q1TN2mKq0ZQMZzwq3wUbOie+9xoMUhMotN5zrbxIb +z9PI8xozK/FhCV3V90LdZtxPPXbvk2FnDEXi623SfTKLzrYujqx6Ax2F9ovUdxYtqka Sza0YCqPgwHUUvESWu9wegWT61UOuUD6HL17NqVpxLb6AaJN2MDNp0tJc/ktM6yRGEcq gKBif8iEEO7acf4TCvfFMbIDZvHXNX/aJM/+hiAcSON+QkLxxGMzQ7D2xlsubD8atMt7 sH/w== X-Gm-Message-State: AOJu0YzTWtN7bG54BxmjSQGgoYFm8VjVPNyYCvnUU6TVx7IOmbUhE9Wc 9L0oi7AFkMHVpcc+kqGk8ZOBeSzn/OOSHvSUQFASgd12qbBjk3WFqGYc X-Gm-Gg: ASbGnctewyuAHCoxpwcantOFQbMqxUzS9KwyMANDaQL+oGJd+GbFobaNOVnQgHUorB5 UzBkqcEVSmlhWk1vHn3gYO2PDzoJdCTZ+kfLR6WVc9Tei6wPpEPw7J/65XJWTNH87fYZ/0yUrjU dSHqRx3MQ6U/GyA0XYaU9IUqBNquOo+uoXZ2f4ddsOumifa8tF5HbHajdiJlzAa5/Qo5GdN2AX4 mDoAEA/peEIGsV46In5XuApQ94EGIc5fEXPbIHQhhKpZSGdB1NuuB1ihu9GTRhmTd73+MfyPlbF vrmKDQwo3K1KcwOVSWCOLAMoswrZQjBlUZa/l4O/ENI6zC8yA4Lnz7DpqgAgGhiwajAhD2yLkeL XswHQo4kqMs6uF+vFleI= X-Google-Smtp-Source: AGHT+IGKY0LuxO/c50j3yZ7leqa+8itWfH62iJhEIHQj7AFYzPpKsvn2M9NaW5dXXFnGyHmRLA6yqw== X-Received: by 2002:a05:6000:250f:b0:3b7:94c6:865e with SMTP id ffacd0b85a97d-3b794ffce8bmr4934246f8f.30.1753959356450; Thu, 31 Jul 2025 03:55:56 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:56 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 15/25] sched/rt: Remove old RT_GROUP_SCHED data structures Date: Thu, 31 Jul 2025 12:55:33 +0200 Message-ID: <20250731105543.40832-16-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Completely remove the old RT_GROUP_SCHED's functions and data structures. Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- include/linux/sched.h | 4 ---- kernel/sched/rt.c | 1 - kernel/sched/sched.h | 26 -------------------------- 3 files changed, 31 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index f0c8229afd1..343e8ef5ba1 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -621,13 +621,9 @@ struct sched_rt_entity { unsigned short on_rq; unsigned short on_list; =20 - struct sched_rt_entity *back; #ifdef CONFIG_RT_GROUP_SCHED - struct sched_rt_entity *parent; /* rq on which this entity is (to be) queued: */ struct rt_rq *rt_rq; - /* rq "owned" by this entity/group: */ - struct rt_rq *my_q; #endif } __randomize_layout; =20 diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index bd11f4a03f7..f37ac9100d1 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,4 +1,3 @@ -#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 9f235df4bf1..4a1bbda3720 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -310,15 +310,6 @@ struct rt_prio_array { struct list_head queue[MAX_RT_PRIO]; }; =20 -struct rt_bandwidth { - /* nests inside the rq lock: */ - raw_spinlock_t rt_runtime_lock; - ktime_t rt_period; - u64 rt_runtime; - struct hrtimer rt_period_timer; - unsigned int rt_period_active; -}; - struct dl_bandwidth { raw_spinlock_t dl_runtime_lock; u64 dl_runtime; @@ -483,7 +474,6 @@ struct task_group { struct sched_dl_entity **dl_se; struct rt_rq **rt_rq; =20 - struct rt_bandwidth rt_bandwidth; struct dl_bandwidth dl_bandwidth; #endif =20 @@ -802,11 +792,6 @@ struct scx_rq { }; #endif /* CONFIG_SCHED_CLASS_EXT */ =20 -static inline int rt_bandwidth_enabled(void) -{ - return 0; -} - /* RT IPI pull logic requires IRQ_WORK */ #if defined(CONFIG_IRQ_WORK) && defined(CONFIG_SMP) # define HAVE_RT_PUSH_IPI @@ -824,17 +809,6 @@ struct rt_rq { bool overloaded; struct plist_head pushable_tasks; =20 - int rt_queued; - -#ifdef CONFIG_RT_GROUP_SCHED - int rt_throttled; - u64 rt_time; /* consumed RT time, goes up in update_curr_rt */ - u64 rt_runtime; /* allotted RT time, "slice" from rt_bandwidth, RT shar= ing/balancing */ - /* Nests inside the rq lock: */ - raw_spinlock_t rt_runtime_lock; - - unsigned int rt_nr_boosted; -#endif #ifdef CONFIG_CGROUP_SCHED struct task_group *tg; /* this tg has "this" rt_rq on given CPU for runna= ble entities */ #endif --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0582B2BEC37 for ; Thu, 31 Jul 2025 10:55:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959360; cv=none; b=pynkZYDuCu1T4ZqriHhSyCbA84ubnwc7dIluBPHMTi/a9UV4vhHvJG8Nt90g3b1YcMohaHWQUMMa5cNszD8cckeb+W4ANfQkQ0C0936sfNH1UquqNBjbiiCBrWpsAORxVCgjdwbfKsRLY7jqfi8FLVBeDvRPuOb5cJlC58XEnSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959360; c=relaxed/simple; bh=vT8thxDnBUGhaNRv6bsofTBtiZ0lHkJZ4bEOP0XyyPE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T0GWbk2rsNEUjmx/My1J2dYZN4t7W8oy0c+D7K+BYd912wHE83ZcOI0ueJwwU9xf/g9cNgnsiywdtknZAJQ3FycpLWCdnm8uclgbZEzNiRDIJRJZu6Zr+Rm/MWdBzCR7v+FToGdv5KEENPR1bhCwrXF6Juu2fBsN7B3YcStICfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ASMpT1vN; arc=none smtp.client-ip=209.85.128.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ASMpT1vN" Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-45896cf24ebso8030185e9.1 for ; Thu, 31 Jul 2025 03:55:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959357; x=1754564157; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2gXvphQYm+11VPSF+20asdPzpBx6sInxbMMeHrwqe5s=; b=ASMpT1vNryvQ/OCl+dveCRua1gduYwwm73mg7koLtNHFaV+XdbfgenWb4I1inWKgKJ 9EXTE8MXhAytAoUyf1iab12Kg+MwDTlVlhSZfSGZjVNAvohOKdXZVscOwa9M//t2L5IL riRBNp4AUnwK9Sng+smkK3Hqvf1WJQzCDyoUhqLwE4trD6HUwUM9n4uXDfrzfA3AVgd/ Rcu2nHqVjUNtDmIZePLtDBHA4WLCrKksnQRcaK1efOPlWjEojt5GN9fsjuSr7ZflXO2b 0NYteaovpWyEcLpP31BQLfFkqPSxXFzc8lugbxo7PboURvWYaOOwPIhnpHn82ARdGbGc +Cqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959357; x=1754564157; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2gXvphQYm+11VPSF+20asdPzpBx6sInxbMMeHrwqe5s=; b=Py1gr2hhYn39lWquZNN9Y7I2W0kw8paR8mMV9IOhM90f6EnatLQEhjLeEL+mUtCyHu UeqSmjKpdrPFi19Li03s1GUztWDZtUEn22ckYY780mI5EJeDVLKhXn/yctYlru9JF6iD O0HXt6lBlGqg0BN4wbNhQlGE1u1uXm18JnwccnILqqcLDgSAH+z7KI78r+c7+EJSaIa2 AfffX003LWtPvb9nIzqbWCoekNupmmA8FEIK88aVNkgSj8f+qT/ictXR4HyytMJ6yycG xtMLUxg4fgGCYAIq8qCTPoaCXeGX6xkshqAWLW128LKZQXIi3PdEXRFwHv0m8W0TLDzm d3bw== X-Gm-Message-State: AOJu0YxOrUDNMFbZG9SH6Dhal+0lZi5oGigQnd679WGBNOrLHImdhOtK PK28UgxFyqt1OKgAWy+PNXfpFiejJcTfF49+qijzzb6FoYVab6HoTsLG X-Gm-Gg: ASbGncsGoQe+oJvcguaUWmagv2OgIwlWNjTWdSZpWQyEfdarh5canQ/0pk9HX1WUyq+ TPny5A+IVbZxvl6KKtGrgz9zxdwQVweMlATQpq2snZE+fqTf9epQvIz13ityUJyACYKyr6YTriY zzbW4lAnKWdMc2x8dM9SSaEBFezr0J9++HCxGvek3NqJAPeVHw+zVXefIlr1BzrjoroXLWfFnLI F46B7D2fkEaCkAbw6U9tyuL87MQWJGuyZLOe8tE7fLyWNNxlftj6i5GMmC33bV52QJNQbKD/E22 4DRJ+GafrxK4JLdrYYKbhXkL2G8fStpKC+EdJVTQVjdRyAX6vGAO4Y3S3JE3753Vh10nppK69ps rUR704otXioodxTyP0fI= X-Google-Smtp-Source: AGHT+IGq5SMvbmCocSEtPogcy4KMJShnp1ajdGQDUdrBwgclE/e7ildXHY/DEML0ulK+Q4iVwIcpKQ== X-Received: by 2002:a05:6000:4305:b0:3b7:9173:6964 with SMTP id ffacd0b85a97d-3b79501d723mr5493498f8f.49.1753959357229; Thu, 31 Jul 2025 03:55:57 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:57 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 16/25] sched/core: Cgroup v2 support Date: Thu, 31 Jul 2025 12:55:34 +0200 Message-ID: <20250731105543.40832-17-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Make rt_runtime_us and rt_period_us virtual files accessible also to the cg= roup v2 controller, effectively enabling the RT_GROUP_SCHED mechanism to cgroups= v2. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 63cb9271052..465f44d7235 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -10300,6 +10300,18 @@ static struct cftype cpu_files[] =3D { .write =3D cpu_uclamp_max_write, }, #endif /* CONFIG_UCLAMP_TASK_GROUP */ +#ifdef CONFIG_RT_GROUP_SCHED + { + .name =3D "rt_runtime_us", + .read_s64 =3D cpu_rt_runtime_read, + .write_s64 =3D cpu_rt_runtime_write, + }, + { + .name =3D "rt_period_us", + .read_u64 =3D cpu_rt_period_read_uint, + .write_u64 =3D cpu_rt_period_write_uint, + }, +#endif /* CONFIG_RT_GROUP_SCHED */ { } /* terminate */ }; =20 --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE53A2C3244 for ; Thu, 31 Jul 2025 10:55:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959361; cv=none; b=OFCnc88CkPMg6Jdam8C7FDAlXeLXbd5+MFMXmDZTfLSeWbZoROQ4tY396gTX1o8GMrmKIZiwBbvvOwUNE8t6Wl/oZSMex/pEje8l8fGbEaonToCGBG2A6MPUI40/9DyIrDZiii0/+SIDyCuVbqvdnQF1tpI5esLTuQCvlU+bviY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959361; c=relaxed/simple; bh=0lGiSVg9B9/1r1Abqba1GBms0sXGUPhMMmufol61dqs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DsEBD30ArVqUIXhdelKh2v9wcVzCIOLaSbRTOeq+bRavCs8h9mG1eSdWgNMRkOQnrI859yqb8GeW5jhzpZ9rB/YA5JrlNRb07Qp2Wu60jPbYHgWG83mqDuka+5bZfGTgLlIPbskEg8eJoXKMnOeSH8+5/JR1rnOzwez6LqcKhNw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fJb44Ojf; arc=none smtp.client-ip=209.85.221.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fJb44Ojf" Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-3b7862bd22bso1131392f8f.1 for ; Thu, 31 Jul 2025 03:55:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959358; x=1754564158; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+O4AnU3EwEKBel9AzlHSCXMa3OEVEdjq3202WmSD4KM=; b=fJb44OjfzrgRDskorwkcOtmCUkoavP9WrBQLCJ1hdy/RvvoBPQzRNZvj2UlWpSK5a3 hs39gwnpz8HgKck8a7cpOLqku1wZ6DNSL5LDZJ+nMya7iUkfBTOeRYNUb8d5u3rXJl+O e7UQkngl+911RRZZqnKW8CRo/CMW5iSJ1B4RzbHPB63hHTUAkgxct8r5IJq/yP8y5UjT d/BkDy9BExy7AZ77KBSDEvf3Km0jc37BwUR34KlsAnqDPPmrswvGh1GIeVijurrq69xJ j7y0AJYLyroegsXr6YAePmi3TU4vdPXUf2UHCgklmwDd2ouEYi2rTdP6fwC8FJACNNQR 65rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959358; x=1754564158; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+O4AnU3EwEKBel9AzlHSCXMa3OEVEdjq3202WmSD4KM=; b=O6tz5XhzsMoBgeL2m/fFViGVLt4jbcrz2LqIEfLXyazC+NOzopFUEKgZfurOK4296h i14inEF6CLih+ckeTU8+/Dg78lWCVLQkOsXxa5MJ3bh4Quwgy0MhS2EFJfPazc1ZZxes olAlBHvMFQr1ptZZcC4BXKr0dlcPJOeanYUWxkHaBnam8T1OXPN7n9GyL1rLl4A5Dgof rn3Iwv8dlNd18DH38oyTz4vDYT92jjJ9Ljl5dqIyUx2TnuM6rUO0zNwBmWWUQeZLZq/e UPTSYDhemCW113WrnAizQN9MFyCYbZxy3famALB3qeACeXQ4rGU/Uw+VpsJWq96Sd+uI SDig== X-Gm-Message-State: AOJu0YwoSWTKencLzJFrlvHub2+UYlYirXuTvzudMwtcSHxuAOLfjGXv U0vtQIoQE63sWc+aUoPmSiGQ2iZ2eFIESRGiCJCgPiz1pwVdAmvwYjOt X-Gm-Gg: ASbGncuts7qomMCJvkfBSYZUnXwJtLpgQTyBDLEY5+r+X5R8Db6S19sqSbFdc9bT9ay Ms5VWVaUK3sLimrSN/OBqh6JodcnLPW0uB4Lsnij5O0TD9pE/zKESZ5HMUgsOk9IhBR+Me/j+LV leZEfIUJElQ+/m62XW85HYTWepC+IIftf5pJCBxWglcpiEdARro9F9ik2wvqtPuy/LbejUt9psu dOG1pYk32z6Q7ApJi1otirOoWmr3mjJLZpyCvHjxCVqT1MLdEAoLvR+B0Z+wR3qYbtvdKcRTM24 t+TOif8AVo8SyGg6M5tEEPlWrwo4ZxOG6hFTtYTVFUeuAnMg3juEcvqDEiXxf2HwQz6VWYPLyoy 4u32eBtMyZVRvfsyDSaAzIduoOKraCg== X-Google-Smtp-Source: AGHT+IHGpDsILQ6xEj635dFaDMpEp64oVlwH46UxdQYX+4iZc6IiBwfa+yAB3WHwwGeExUJSaY2iYQ== X-Received: by 2002:adf:a418:0:b0:3b7:91a3:bc9d with SMTP id ffacd0b85a97d-3b79d4e5954mr1169644f8f.23.1753959357955; Thu, 31 Jul 2025 03:55:57 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:57 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 17/25] sched/rt: Remove support for cgroups-v1 Date: Thu, 31 Jul 2025 12:55:35 +0200 Message-ID: <20250731105543.40832-18-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Disable control files for cgroups-v1, and allow only cgroups-v2. This should simplify maintaining the code, also because cgroups-v1 are deprecated. Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 465f44d7235..85950e10bb1 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -10039,20 +10039,6 @@ static struct cftype cpu_legacy_files[] =3D { }; =20 #ifdef CONFIG_RT_GROUP_SCHED -static struct cftype rt_group_files[] =3D { - { - .name =3D "rt_runtime_us", - .read_s64 =3D cpu_rt_runtime_read, - .write_s64 =3D cpu_rt_runtime_write, - }, - { - .name =3D "rt_period_us", - .read_u64 =3D cpu_rt_period_read_uint, - .write_u64 =3D cpu_rt_period_write_uint, - }, - { } /* Terminate */ -}; - # ifdef CONFIG_RT_GROUP_SCHED_DEFAULT_DISABLED DEFINE_STATIC_KEY_FALSE(rt_group_sched); # else @@ -10078,10 +10064,6 @@ __setup("rt_group_sched=3D", setup_rt_group_sched); =20 static int __init cpu_rt_group_init(void) { - if (!rt_group_sched_enabled()) - return 0; - - WARN_ON(cgroup_add_legacy_cftypes(&cpu_cgrp_subsys, rt_group_files)); return 0; } subsys_initcall(cpu_rt_group_init); --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 980932C325F for ; Thu, 31 Jul 2025 10:56:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959362; cv=none; b=Xomafff9Y/ez1dPUtJQnGqgfTSH3ty/8RVafR6p/yu2E+Ei/JmHXeh6P7jFTM6Q7T5LGFLd3E/GriqYcQ5JZEukymyt50KXfJgVew/X4RUspHPCaPlTQ4E/xWSpGG1/AqV2khc4srQMd5dmzf4ouzEEJdMjKM6qlRmTMCeDcnKA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959362; c=relaxed/simple; bh=o3Xn5RS6R+mYhqsW2xGF7xOIDPgstLl2QFHTFT45Z2A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nD4oca8n58GOWD0uGSjo5mME3AoL0qJAtqvtn75CEtFMvEZQPGbYp/nXF71ag88cTQlAm/zct7ff/JaH9pL2M8nJJ5QrfJ9wtuOUwwwEhz6HyT8M1Lk189GlDA/wcP8rYXrBURjI0tx0q0iA/tYUp3KnjoPkhh0SNb2Ssm9rczM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YfCjHFu/; arc=none smtp.client-ip=209.85.221.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YfCjHFu/" Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-3b7834f2e72so485087f8f.2 for ; Thu, 31 Jul 2025 03:56:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959359; x=1754564159; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hga7bxShN6VAEkqm5Kbc7Uh0UaKMqOrTHjafHB8ysB4=; b=YfCjHFu/IGDbj0RI5Cp41jc18+h6nxpXnBd3KWBpf0OSbRm6CAbia3ESA3xRVjv7bX 1FgjsCKLCYdnEgRHGNl7qChoUGVy2E90zrfnB4204v/iiUSnXpkPt9FjnuaQhv4A3OgF czsqe0TKFqJszFhZdcbdiOu6K3aGbJ+A2f0QFC7iA8D2auPZHOc42T0qhoXEPb7l83yh pSAGqWgsr0nBtoV+SphHkQoT8Y9kzZzjcgzdgDI0sNn8a2kSza40jv7SBrHvJieMSdEH l4DahDETblIILKd069t+BwHICBg1qIxJCOnw2KTB2GJsoU5t0Op16LFeiHmI1JaDrmOF QDjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959359; x=1754564159; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hga7bxShN6VAEkqm5Kbc7Uh0UaKMqOrTHjafHB8ysB4=; b=H52R93KOSnXMOgEMFjyRl9W9988z9eSe/Iy8+jE0IojVW7p4csbwQ9wzIzo4MQJu6Q GJFg04hq+g9aLCumU2Xsv7XEG5o5m+7vv/6YiYz7hBw7heWCYCJQb4ns+26PWjXzoCUb oGutgCxrF9E6Uz10Im6ct29BWOIkSl61gHxarQp7kRc6HH+kBTW+oEuWAjF+zAaQ04tR 7nFrZ9+3vTTRO9jnBlWvqXe/PBkpME3xNIZV2lg5nqSzcvYwdF++mpkUdwci10hvkhX8 ZXPETV8qGVO1LM848XYX2bok1eQRF799kD16LEJYKpqeqiL08NHYd4tbBUD7NUG8qPLn zqUw== X-Gm-Message-State: AOJu0Yzzw98bawPiU1H1eHsxRUW6tZh84n6KUvokHuUOi8XKZRDEXgzr ZrD3SdtMMW5Zue2NTxpGrWENh7LxacfiXz8WDja0UhTD9lj0Kr1Ixt+E X-Gm-Gg: ASbGncu7XMEIdpx0f0QMIyWwuKAIFmSZHiKQzseKVYBAtiKB7eCzUbnejPK7qUsILFb zguhgv721d58H8KZmCDXeNZVuy1ZroEir418n9E0k8mmYsYI0etcCUQV1nlUVgh9dbcq+DvxFAB oMn56vpU58jPl2PpDRoKhhvT6PZBBpvSpMabTSA45tteY02OG8AAlDYGhcZXwsigtDfkaJkq5HX g/ovqMdUI8YpJr1/wUvT/WHC0otk+jfL3HLzgcEWUH876RuCwmhXw01rn+gA4AvD1ZchG8OyeKs WtBjuZGUk+MqlodAbu7yrKGIiX4QvrgjxS6DHezEF0gCZcQ1C8xuWebQ0efU/1ldM2dCFqGVwlH ktiEo/uoNty2Hy4ywjTk= X-Google-Smtp-Source: AGHT+IH71lNuvewEWXshWp6RphVFa9Qi7ttHjsyjqaKAWvZYWD+0xxpmzVJ/76N9cq0nHmWFeHpjWA== X-Received: by 2002:a05:6000:4212:b0:3a3:63d3:369a with SMTP id ffacd0b85a97d-3b794fc182dmr6057778f8f.25.1753959358760; Thu, 31 Jul 2025 03:55:58 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:58 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 18/25] sched/rt: Zero rt-cgroups default bandwidth Date: Thu, 31 Jul 2025 12:55:36 +0200 Message-ID: <20250731105543.40832-19-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set the default rt-cgroups runtime to zero, otherwise a cgroup-v1 kernel wi= ll not be able to start SCHED_DEADLINE tasks. The bandwidth for rt-cgroups must then be manually assigned after the kernel boots. Allow zeroing the runtime of the root control group. This runtime only affe= cts the available bandwidth of the rt-cgroup hierarchy but not the SCHED_FIFO / SCHED_RR tasks on the global runqueue. Notes: Disabling the root control group bandwidth should not cause any side effect= , as SCHED_FIFO / SCHED_RR tasks do not depend on it since the introduction of fair_servers. Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 4 ++-- kernel/sched/rt.c | 13 +++++-------- kernel/sched/syscalls.c | 2 +- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 85950e10bb1..3ac65c6af70 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8719,7 +8719,7 @@ void __init sched_init(void) =20 #ifdef CONFIG_RT_GROUP_SCHED init_dl_bandwidth(&root_task_group.dl_bandwidth, - global_rt_period(), global_rt_runtime()); + global_rt_period(), 0); #endif /* CONFIG_RT_GROUP_SCHED */ =20 #ifdef CONFIG_CGROUP_SCHED @@ -9348,7 +9348,7 @@ static int cpu_cgroup_can_attach(struct cgroup_taskse= t *tset) goto scx_check; =20 cgroup_taskset_for_each(task, css, tset) { - if (!sched_rt_can_attach(css_tg(css), task)) + if (rt_task(task) && !sched_rt_can_attach(css_tg(css), task)) return -EINVAL; } scx_check: diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index f37ac9100d1..75a6860c2e2 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2122,13 +2122,6 @@ static int tg_set_rt_bandwidth(struct task_group *tg, static DEFINE_MUTEX(rt_constraints_mutex); int i, err =3D 0; =20 - /* - * Disallowing the root group RT runtime is BAD, it would disallow the - * kernel creating (and or operating) RT threads. - */ - if (tg =3D=3D &root_task_group && rt_runtime =3D=3D 0) - return -EINVAL; - /* No period doesn't make any sense. */ if (rt_period =3D=3D 0) return -EINVAL; @@ -2215,8 +2208,12 @@ static int sched_rt_global_constraints(void) =20 int sched_rt_can_attach(struct task_group *tg, struct task_struct *tsk) { + /* Allow executing in the root cgroup regardless of allowed bandwidth */ + if (tg =3D=3D &root_task_group) + return 1; + /* Don't accept real-time tasks when there is no way for them to run */ - if (rt_group_sched_enabled() && rt_task(tsk) && tg->dl_bandwidth.dl_runti= me =3D=3D 0) + if (rt_group_sched_enabled() && tg->dl_bandwidth.dl_runtime =3D=3D 0) return 0; =20 return 1; diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index 7c1f7649477..71f20be6f29 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -633,7 +633,7 @@ int __sched_setscheduler(struct task_struct *p, */ if (rt_group_sched_enabled() && dl_bandwidth_enabled() && rt_policy(policy) && - task_group(p)->dl_bandwidth.dl_runtime =3D=3D 0 && + !sched_rt_can_attach(task_group(p), p) && !task_group_is_autogroup(task_group(p))) { retval =3D -EPERM; goto unlock; --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 523C92C3745 for ; Thu, 31 Jul 2025 10:56:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959363; cv=none; b=GWIIt/N6gLttPznfhDobIBTK75mNBeh81QAt1OrhctX4Q7S/BAng1EOJdxr6BkZNkzXmxeewoZyqHvXQKNUnLI4gNIHFO75/pS2TdtD9YjDc/21qCT2MjbMT5PolKbAlPqeZhfNLwmNLid6hbuIznsn0zvVpGUYM4SqUf/thsl4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959363; c=relaxed/simple; bh=6zRKlwo23+6u56UEk2LaVGEisTYGXjm0mm3VKe7EI3I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pV41ii1/JGOWvAgPmDU34gFoqKLQzuXAFWVF6UQFQYfIWVuFSuKZ4DZ12vH8WJ/2Wil+hWdF6vW7aUe/mNrC0BeBgBRjhV7rquil1KDYuCv3KNe2448/lAfmHVHnt7du6ZI4jexOJLjWRdj9CE1CH4c9yO8C5y4Yla9QQUO0F5o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cXR5acCe; arc=none smtp.client-ip=209.85.221.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cXR5acCe" Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-3b783ea502eso514353f8f.1 for ; Thu, 31 Jul 2025 03:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959360; x=1754564160; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U/Dh5Zhhr5dqENgdMbOY1ekMDiju3H1epGU1a1pN8No=; b=cXR5acCejyqI5A1/A3YsttxGzcUXaZ5T7qO2fBOemye0EcOdcfQm2GPWfNnzh10a9R q88GDUecqnIWlfTwuL2mwRhhwTBUYVQXS2JaWqmi4YWszoHurxqMqtfWlb2kYFkN1t/7 fFgQC5udX7s6zeAR1eKrel2oeJZFpx7y2gY9j/dPuc+q0j38ni+tAsFyQX077fxL3AsC 9NR0OER0a9l/qLSCL6aJctcSMytgYvG6vKqOfjnwUcTP+Sj9Uegs3XU8DAFFox36TVUy +momvr72uscu8IKxacfVO5fehnD6La0qRPz9HU+e6QS58W0BEGY5CCy/1UUaM2FBoqf3 Z2Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959360; x=1754564160; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U/Dh5Zhhr5dqENgdMbOY1ekMDiju3H1epGU1a1pN8No=; b=M0AnqZ+MoH0DweFsb2yELRF5Uw6c8C2lg/7JOQ5+5DS7MK/8Z2HeC+jHt68ejnjWm9 kAc/5tnS75XuPYRu2PEK6p6S30HTcxiK1P6PnpTXRdXSkgxna+HnAegkuSywg56IQcYI FQdeCTyDA7V6UVJNTnJm81Dftchj5M51Wq2MnQhpyoa86gKSLu/RFrIkXrs5pzVXoGYn m6sE85JvlXeAeEe2YEMQS94PUO+jaA+BF4J91Sx2W2lO3gaK7j9PFpOzecVNw6Wp5UdQ dlUcIuBiKkM37IdOTf/AT3gCD4sR3sJONCGBw2UWCk1U8ZxvyGeCTH1bNAnKDG3rIe+y mDxg== X-Gm-Message-State: AOJu0YyG8G42JSEgKSSmLKp1ZQAewpMielAuiwWLkuESgQRbcZMb0Avi QY7AejfIcOdgn5XuNe/rH+5gYy8WqsjT0PpLv4I+MkybRU1u/AHZuw8p7svZ2w56uNc= X-Gm-Gg: ASbGncsrFn4KyVqEDg1O0G47Nsy0WQg2JsAfIiJfZIddbhuii1eiqn8gVYqF+POQjm2 rl6944YgoKaj/MeUpOrbfsOqxXqq+DJkFNc5ze7B1Y4ZyKgwDtvUL3PaNz751wFC/qMCtWWH+EO LaDfKO6nq2tFZwg9Vwwr8vJU0vQvwqufW20JgmcQqXTUX79pjDl1iLdTOnt6CUM03J7hQbMZbbt +LZy7oHWjfFtYgwmhv26+GhP2sHjFkDbtag6oNTuVIkqwFpQ6GmFdpqFKUK3GdOjYxbQYl+9e1C yqJ9o2+L8RGIZQajNJOVNCQocee6G1fePyXMD01+yquT39Rd5J5VVLFzHJb5R5WCZo1Ajeq03tC T6hsxrzezXt+CCoFlR88= X-Google-Smtp-Source: AGHT+IHKpVdt0z1whnjQjXu62SKd40EHF2IK97jOX0EYjMV2OcCVC9XEWf8Tq59R2n3+AwYknba3aQ== X-Received: by 2002:a05:6000:144a:b0:3b7:931d:37a0 with SMTP id ffacd0b85a97d-3b79d43cd5bmr1391052f8f.9.1753959359552; Thu, 31 Jul 2025 03:55:59 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:55:59 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 19/25] sched/deadline: Allow deeper hierarchies of RT cgroups Date: Thu, 31 Jul 2025 12:55:37 +0200 Message-ID: <20250731105543.40832-20-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Allow creation of cgroup hierachies with depth greater than two. Add check to prevent attaching tasks to a child cgroup of an active cgroup = (i.e. with a running FIFO/RR task). Add check to prevent attaching tasks to cgroups which have children with non-zero runtime. Update rt-cgroups allocated bandwidth accounting for nested cgroup hierachi= es. Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/core.c | 6 ----- kernel/sched/deadline.c | 51 +++++++++++++++++++++++++++++++++++++---- kernel/sched/rt.c | 25 +++++++++++++++++--- kernel/sched/sched.h | 2 +- 4 files changed, 70 insertions(+), 14 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 3ac65c6af70..eb9de8c7b1f 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9277,12 +9277,6 @@ cpu_cgroup_css_alloc(struct cgroup_subsys_state *par= ent_css) return &root_task_group.css; } =20 - /* Do not allow cpu_cgroup hierachies with depth greater than 2. */ -#ifdef CONFIG_RT_GROUP_SCHED - if (parent !=3D &root_task_group) - return ERR_PTR(-EINVAL); -#endif - tg =3D sched_create_group(parent); if (IS_ERR(tg)) return ERR_PTR(-ENOMEM); diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 7b131630743..e263abcdc04 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -406,11 +406,42 @@ int dl_check_tg(unsigned long total) return 1; } =20 -void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 rt_peri= od) +static inline bool is_active_sched_group(struct task_group *tg) { + struct task_group *child; + bool is_active =3D 1; + + // if there are no children, this is a leaf group, thus it is active + list_for_each_entry_rcu(child, &tg->children, siblings) { + if (child->dl_bandwidth.dl_runtime > 0) { + is_active =3D 0; + } + } + return is_active; +} + +static inline bool sched_group_has_active_siblings(struct task_group *tg) +{ + struct task_group *child; + bool has_active_siblings =3D 0; + + // if there are no children, this is a leaf group, thus it is active + list_for_each_entry_rcu(child, &tg->parent->children, siblings) { + if (child !=3D tg && child->dl_bandwidth.dl_runtime > 0) { + has_active_siblings =3D 1; + } + } + return has_active_siblings; +} + +void dl_init_tg(struct task_group *tg, int cpu, u64 rt_runtime, u64 rt_per= iod) +{ + struct sched_dl_entity *dl_se =3D tg->dl_se[cpu]; struct rq *rq =3D container_of(dl_se->dl_rq, struct rq, dl); - int is_active; - u64 new_bw; + int is_active, is_active_group; + u64 old_runtime, new_bw; + + is_active_group =3D is_active_sched_group(tg); =20 raw_spin_rq_lock_irq(rq); is_active =3D dl_se->my_q->rt.rt_nr_running > 0; @@ -418,8 +449,10 @@ void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_= runtime, u64 rt_period) update_rq_clock(rq); dl_server_stop(dl_se); =20 + old_runtime =3D dl_se->dl_runtime; new_bw =3D to_ratio(dl_se->dl_period, dl_se->dl_runtime); - dl_rq_change_utilization(rq, dl_se, new_bw); + if (is_active_group) + dl_rq_change_utilization(rq, dl_se, new_bw); =20 dl_se->dl_runtime =3D rt_runtime; dl_se->dl_deadline =3D rt_period; @@ -431,6 +464,16 @@ void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_= runtime, u64 rt_period) dl_se->dl_bw =3D new_bw; dl_se->dl_density =3D new_bw; =20 + // add/remove the parent's bw + if (tg->parent && tg->parent !=3D &root_task_group) + { + if (rt_runtime =3D=3D 0 && old_runtime !=3D 0 && !sched_group_has_active= _siblings(tg)) { + __add_rq_bw(tg->parent->dl_se[cpu]->dl_bw, dl_se->dl_rq); + } else if (rt_runtime !=3D 0 && old_runtime =3D=3D 0 && !sched_group_has= _active_siblings(tg)) { + __sub_rq_bw(tg->parent->dl_se[cpu]->dl_bw, dl_se->dl_rq); + } + } + if (is_active) dl_server_start(dl_se); =20 diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 75a6860c2e2..29b51251fdc 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -107,7 +107,8 @@ void free_rt_sched_group(struct task_group *tg) * Fix this issue by changing the group runtime * to 0 immediately before freeing it. */ - dl_init_tg(tg->dl_se[i], 0, tg->dl_se[i]->dl_period); + if (tg->dl_se[i]->dl_runtime) + dl_init_tg(tg, i, 0, tg->dl_se[i]->dl_period); raw_spin_rq_lock_irqsave(cpu_rq(i), flags); BUG_ON(tg->rt_rq[i]->rt_nr_running); raw_spin_rq_unlock_irqrestore(cpu_rq(i), flags); @@ -2122,6 +2123,14 @@ static int tg_set_rt_bandwidth(struct task_group *tg, static DEFINE_MUTEX(rt_constraints_mutex); int i, err =3D 0; =20 + /* + * Do not allow to set a RT runtime > 0 if the parent has RT tasks + * (and is not the root group) + */ + if (rt_runtime && (tg !=3D &root_task_group) && (tg->parent !=3D &root_ta= sk_group) && tg_has_rt_tasks(tg->parent)) { + return -EINVAL; + } + /* No period doesn't make any sense. */ if (rt_period =3D=3D 0) return -EINVAL; @@ -2145,7 +2154,7 @@ static int tg_set_rt_bandwidth(struct task_group *tg, return 0; =20 for_each_possible_cpu(i) { - dl_init_tg(tg->dl_se[i], rt_runtime, rt_period); + dl_init_tg(tg, i, rt_runtime, rt_period); } =20 return 0; @@ -2208,6 +2217,9 @@ static int sched_rt_global_constraints(void) =20 int sched_rt_can_attach(struct task_group *tg, struct task_struct *tsk) { + struct task_group *child; + int can_attach =3D 1; + /* Allow executing in the root cgroup regardless of allowed bandwidth */ if (tg =3D=3D &root_task_group) return 1; @@ -2216,7 +2228,14 @@ int sched_rt_can_attach(struct task_group *tg, struc= t task_struct *tsk) if (rt_group_sched_enabled() && tg->dl_bandwidth.dl_runtime =3D=3D 0) return 0; =20 - return 1; + /* If one of the children has runtime > 0, cannot attach RT tasks! */ + list_for_each_entry_rcu(child, &tg->children, siblings) { + if (child->dl_bandwidth.dl_runtime) { + can_attach =3D 0; + } + } + + return can_attach; } =20 #else /* !CONFIG_RT_GROUP_SCHED: */ diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 4a1bbda3720..3dd2ede6d35 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -386,7 +386,7 @@ extern void dl_server_init(struct sched_dl_entity *dl_s= e, struct dl_rq *dl_rq, dl_server_pick_f pick_task); extern void sched_init_dl_servers(void); extern int dl_check_tg(unsigned long total); -extern void dl_init_tg(struct sched_dl_entity *dl_se, u64 rt_runtime, u64 = rt_period); +extern void dl_init_tg(struct task_group *tg, int cpu, u64 rt_runtime, u64= rt_period); =20 extern void dl_server_update_idle_time(struct rq *rq, struct task_struct *p); --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43B482D0C6B for ; Thu, 31 Jul 2025 10:56:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959365; cv=none; b=UFX0UsmYOp/iNfQ6cCxWM+eiueYTAsk5CLEodRLFSuXM9GPVlN1a4pFxNmpnBstD+IVlJsz7xk0iR3ARRfNAEGp1iqW28o+IOqstDLIV+b/bs5yr57Z5CrjfE0h8WQWWD5H+H2Pz2mUY7B4D2lQW5eVQwWNRCFuKZWRhSeZqEKk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959365; c=relaxed/simple; bh=K12u59/Jdto8wGdYTXc18vqTil4fRVAsVNrZ2T5FNPA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JHHR4VcqqmAXaDMEX4LZlvH4jp+LS8+1qvUmfoUAxFoDlpZ83bBYnU3b1j+0bpwT6NaQUHfxNZQ1dp+OXEq6k5wiyo8V8dzfzRDtQepcDkmkbqYu7Q5NuO+sNFf9wEF0aIdlkWZPyNuF0FaKMwpAdThzRnmYGwasN4O1+ihzTeA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=I0t+uOoC; arc=none smtp.client-ip=209.85.221.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="I0t+uOoC" Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-3b78d729bb8so615294f8f.0 for ; Thu, 31 Jul 2025 03:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959360; x=1754564160; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6POCJpohgM1qd3Vfj0hh2gC5FtaPdLsrtNLQFlwev7M=; b=I0t+uOoCO7OjPUczrYzIz5NbB+2XKk1Ldke5HUELZP0ONKk2TT+fAdW6J9Qw0p893r DshDe1RocrkY3KpH8yKIEOJCpCGdgUwTB44ZIonWAyjdY7ucNBRda7gU35orUcr4NKzQ loJYO5Apvmlk2ARU9kHaTeww1tQ9GEE+ePAJF1it0byFybcsFq+IOWdirFf14zh4tC2j ixwMkDTaZolPjmfkS2Ah5o4jnZ5RZ83BH4o+DkpVMmddcltBc/0I1IyZmwOUS+v34T+y Ew+kYc9em3qpB4KtBESdGryOwfMojd4h/6u5+wTD+u4xYZmzl5ToZlCZFFzzPkNPL/IO Qx/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959360; x=1754564160; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6POCJpohgM1qd3Vfj0hh2gC5FtaPdLsrtNLQFlwev7M=; b=isLiU1srC1L/ZZOF4cqlHsQco3MHrtvQt29k7hvMYFnqM+AzkGlskmuN2QYvaYJmbd 9nZ9/ZwDOEDHu+Q3NShI7UE86NGLqlmaGtV/yrdKlPp0xCaNCf0KQVUvVtcZNrizDX1S pM7jIkw3SmYPS01REq1L1GN2vYq9seVmzYWLOprhWoAWirHv6WKZTkijbscq9lDIYF6l IkfJJyitIDSEFDYFK7f0QJuktsPsjj2sQ5S1W2YHnGZS4/1CldxMVrFasLSbCzrqpZfO 8YbionV2f1OBVgLxlpCvEGuJreROjCb8YIEFUAXgrGa9h5sI8J6znzNsOhOUoUHOiNIm Zsiw== X-Gm-Message-State: AOJu0YynE92keMqrsaeHU8v/czYrgtm/Y5SC3/nU3JRmgCEtDbtkWZPZ 1Dp0yAYls97k2EDkr/VYD9KQ8GeAaKARiEi7r/Hpin8B0LLa0AqyBVQA X-Gm-Gg: ASbGncsveErsddCjginDKki90XancBxOvynWJ4eh/W6h5AnCrwRT6HoRY+hLSyPJhJs dGVQ5GHH+9oEDl2iq6ZHH0RQ5IKwkKHaukDZYZjjktMrkRzJ3NvH0FJUeRra485m5Osi/Yl6X/w 0VWlycD6+8yFrGv68n9IAgJEHXPsEOJnwHRtjUSqr2HL19OqQ/lmFhyoauUZNuWfUxHjNQKGuD7 0IEHOuRjP/sq8eX2rTVjkyWkbcCeK8J7ePdA+rOii8h4QKvZkWr+d2Z+Fpb9fmZjyOA5tuQ01ut wdp8ehLq89FlJBBm3ElCziJuZxNQHbwqPinWTGQ2ShzQzVdUKNfS8TjM3ZO2GlPR30+GFKXpF+q G5Pt2oMlVzIhnpFEHLcM= X-Google-Smtp-Source: AGHT+IFTLt3dOQNeazPOHMKBHCZZ9hnVACfWv/yBY7T2W+KYoSLQf39dT6WIFJ+YvM3FLhvm8ml+hA== X-Received: by 2002:a05:6000:2210:b0:3b7:75ee:1eb0 with SMTP id ffacd0b85a97d-3b794ffddc2mr5814369f8f.38.1753959360338; Thu, 31 Jul 2025 03:56:00 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.55.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:56:00 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 20/25] sched/rt: Add rt-cgroup migration Date: Thu, 31 Jul 2025 12:55:38 +0200 Message-ID: <20250731105543.40832-21-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni When the runtime is exhausted in a RT CGroup, the scheduler checks for another non-throttled runqueue and, if available, migrates the tasks. The bandwidth (runtime/period) chosen for a certain cgroup is replicated on every core of the system, therefore, in an SMP system with M cores, the total available bandwidth is the given runtime/period multiplied by M. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/rt.c | 471 ++++++++++++++++++++++++++++++++++++++++++- kernel/sched/sched.h | 5 + 2 files changed, 468 insertions(+), 8 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 29b51251fdc..2fdb2657554 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,3 +1,4 @@ +#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR @@ -84,6 +85,8 @@ void init_rt_rq(struct rt_rq *rt_rq) plist_head_init(&rt_rq->pushable_tasks); } =20 +static void group_pull_rt_task(struct rt_rq *this_rt_rq); + #ifdef CONFIG_RT_GROUP_SCHED =20 void unregister_rt_sched_group(struct task_group *tg) @@ -289,6 +292,45 @@ static inline void rt_queue_pull_task(struct rq *rq) queue_balance_callback(rq, &per_cpu(rt_pull_head, rq->cpu), pull_rt_task); } =20 +#ifdef CONFIG_RT_GROUP_SCHED +static DEFINE_PER_CPU(struct balance_callback, rt_group_push_head); +static DEFINE_PER_CPU(struct balance_callback, rt_group_pull_head); +static void push_group_rt_tasks(struct rq *); +static void pull_group_rt_tasks(struct rq *); + +static void rt_queue_push_from_group(struct rq *rq, struct rt_rq *rt_rq) +{ + BUG_ON(rt_rq =3D=3D NULL); + BUG_ON(rt_rq->rq !=3D rq); + + if (rq->rq_to_push_from) + return; + + rq->rq_to_push_from =3D container_of(rt_rq, struct rq, rt); + queue_balance_callback(rq, &per_cpu(rt_group_push_head, rq->cpu), + push_group_rt_tasks); +} + +static void rt_queue_pull_to_group(struct rq *rq, struct rt_rq *rt_rq) +{ + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + BUG_ON(rt_rq =3D=3D NULL); + BUG_ON(!is_dl_group(rt_rq)); + BUG_ON(rt_rq->rq !=3D rq); + + if (dl_se->dl_throttled || rq->rq_to_pull_to) + return; + + rq->rq_to_pull_to =3D container_of(rt_rq, struct rq, rt); + queue_balance_callback(rq, &per_cpu(rt_group_pull_head, rq->cpu), + pull_group_rt_tasks); +} +#else +static inline void rt_queue_push_from_group(struct rq *rq, struct rt_rq *r= t_rq) {}; +static inline void rt_queue_pull_to_group(struct rq *rq, struct rt_rq *rt_= rq) {}; +#endif + static void enqueue_pushable_task(struct rt_rq *rt_rq, struct task_struct = *p) { plist_del(&p->pushable_tasks, &rt_rq->pushable_tasks); @@ -1277,6 +1319,8 @@ static struct rq *find_lock_lowest_rq(struct task_str= uct *task, struct rq *rq) */ static int push_rt_task(struct rq *rq, bool pull) { + BUG_ON(is_dl_group(&rq->rt)); + struct task_struct *next_task; struct rq *lowest_rq; int ret =3D 0; @@ -1573,6 +1617,8 @@ void rto_push_irq_work_func(struct irq_work *work) =20 static void pull_rt_task(struct rq *this_rq) { + BUG_ON(is_dl_group(&this_rq->rt)); + int this_cpu =3D this_rq->cpu, cpu; bool resched =3D false; struct task_struct *p, *push_task; @@ -1683,27 +1729,436 @@ static void pull_rt_task(struct rq *this_rq) } =20 #ifdef CONFIG_RT_GROUP_SCHED -static int group_push_rt_task(struct rt_rq *rt_rq) +/* + * Find the lowest priority runqueue among the runqueues of the same + * task group. Unlike find_lowest_rt(), this does not mean that the + * lowest priority cpu is running tasks from this runqueue. + */ +static int group_find_lowest_rt_rq(struct task_struct *task, struct rt_rq*= task_rt_rq) +{ + struct sched_domain *sd; + struct cpumask mask, *lowest_mask =3D &mask; + struct sched_dl_entity *dl_se; + struct rt_rq *rt_rq; + int prio, lowest_prio; + int cpu, this_cpu =3D smp_processor_id(); + + BUG_ON(task->sched_task_group !=3D task_rt_rq->tg); + + if (task->nr_cpus_allowed =3D=3D 1) + return -1; /* No other targets possible */ + + lowest_prio =3D task->prio - 1; + cpumask_clear(lowest_mask); + for_each_cpu_and(cpu, cpu_online_mask, task->cpus_ptr) { + dl_se =3D task_rt_rq->tg->dl_se[cpu]; + rt_rq =3D &dl_se->my_q->rt; + prio =3D rt_rq->highest_prio.curr; + + /* + * If we're on asym system ensure we consider the different capacities + * of the CPUs when searching for the lowest_mask. + */ + if (dl_se->dl_throttled || !rt_task_fits_capacity(task, cpu)) + continue; + + if (prio >=3D lowest_prio) { + if (prio > lowest_prio) { + cpumask_clear(lowest_mask); + lowest_prio =3D prio; + } + + cpumask_set_cpu(cpu, lowest_mask); + } + } + + if (cpumask_empty(lowest_mask)) + return -1; + + /* + * At this point we have built a mask of CPUs representing the + * lowest priority tasks in the system. Now we want to elect + * the best one based on our affinity and topology. + * + * We prioritize the last CPU that the task executed on since + * it is most likely cache-hot in that location. + */ + cpu =3D task_cpu(task); + if (cpumask_test_cpu(cpu, lowest_mask)) + return cpu; + + /* + * Otherwise, we consult the sched_domains span maps to figure + * out which CPU is logically closest to our hot cache data. + */ + if (!cpumask_test_cpu(this_cpu, lowest_mask)) + this_cpu =3D -1; /* Skip this_cpu opt if not among lowest */ + + rcu_read_lock(); + for_each_domain(cpu, sd) { + if (sd->flags & SD_WAKE_AFFINE) { + int best_cpu; + + /* + * "this_cpu" is cheaper to preempt than a + * remote processor. + */ + if (this_cpu !=3D -1 && + cpumask_test_cpu(this_cpu, sched_domain_span(sd))) { + rcu_read_unlock(); + return this_cpu; + } + + best_cpu =3D cpumask_any_and_distribute(lowest_mask, + sched_domain_span(sd)); + if (best_cpu < nr_cpu_ids) { + rcu_read_unlock(); + return best_cpu; + } + } + } + rcu_read_unlock(); + + /* + * And finally, if there were no matches within the domains + * just give the caller *something* to work with from the compatible + * locations. + */ + if (this_cpu !=3D -1) + return this_cpu; + + cpu =3D cpumask_any_distribute(lowest_mask); + if (cpu < nr_cpu_ids) + return cpu; + + return -1; +} + +/* + * Find and lock the lowest priority runqueue among the runqueues + * of the same task group. Unlike find_lock_lowest_rt(), this does not + * mean that the lowest priority cpu is running tasks from this runqueue. + */ +static struct rt_rq* group_find_lock_lowest_rt_rq(struct task_struct *task= , struct rt_rq *rt_rq) +{ + struct rq *rq =3D rq_of_rt_rq(rt_rq); + struct rq *lowest_rq; + struct rt_rq *lowest_rt_rq =3D NULL; + struct sched_dl_entity *lowest_dl_se; + int tries, cpu; + + BUG_ON(task->sched_task_group !=3D rt_rq->tg); + + for (tries =3D 0; tries < RT_MAX_TRIES; tries++) { + cpu =3D group_find_lowest_rt_rq(task, rt_rq); + + if ((cpu =3D=3D -1) || (cpu =3D=3D rq->cpu)) + break; + + lowest_dl_se =3D rt_rq->tg->dl_se[cpu]; + lowest_rt_rq =3D &lowest_dl_se->my_q->rt; + lowest_rq =3D cpu_rq(cpu); + + if (lowest_rt_rq->highest_prio.curr <=3D task->prio) { + /* + * Target rq has tasks of equal or higher priority, + * retrying does not release any lock and is unlikely + * to yield a different result. + */ + lowest_rt_rq =3D NULL; + break; + } + + /* if the prio of this runqueue changed, try again */ + if (double_lock_balance(rq, lowest_rq)) { + /* + * We had to unlock the run queue. In + * the mean time, task could have + * migrated already or had its affinity changed. + * Also make sure that it wasn't scheduled on its rq. + * It is possible the task was scheduled, set + * "migrate_disabled" and then got preempted, so we must + * check the task migration disable flag here too. + */ + if (unlikely(is_migration_disabled(task) || + lowest_dl_se->dl_throttled || + !cpumask_test_cpu(lowest_rq->cpu, &task->cpus_mask) || + task !=3D pick_next_pushable_task(rt_rq))) { + + double_unlock_balance(rq, lowest_rq); + lowest_rt_rq =3D NULL; + break; + } + } + + /* If this rq is still suitable use it. */ + if (lowest_rt_rq->highest_prio.curr > task->prio) + break; + + /* try again */ + double_unlock_balance(rq, lowest_rq); + lowest_rt_rq =3D NULL; + } + + return lowest_rt_rq; +} + +static int group_push_rt_task(struct rt_rq *rt_rq, bool pull) { + BUG_ON(!is_dl_group(rt_rq)); + struct rq *rq =3D rq_of_rt_rq(rt_rq); + struct task_struct *next_task; + struct rq *lowest_rq; + struct rt_rq *lowest_rt_rq; + int ret =3D 0; + + if (!rt_rq->overloaded) + return 0; + + next_task =3D pick_next_pushable_task(rt_rq); + if (!next_task) + return 0; + +retry: + if (is_migration_disabled(next_task)) { + struct task_struct *push_task =3D NULL; + int cpu; + + if (!pull || rq->push_busy) + return 0; + + /* + * If the current task does not belong to the same task group + * we cannot push it away. + */ + if (rq->curr->sched_task_group !=3D rt_rq->tg) + return 0; + + /* + * Invoking group_find_lowest_rt_rq() on anything but an RT task doesn't + * make sense. Per the above priority check, curr has to + * be of higher priority than next_task, so no need to + * reschedule when bailing out. + * + * Note that the stoppers are masqueraded as SCHED_FIFO + * (cf. sched_set_stop_task()), so we can't rely on rt_task(). + */ + if (rq->curr->sched_class !=3D &rt_sched_class) + return 0; + + cpu =3D group_find_lowest_rt_rq(rq->curr, rt_rq); + if (cpu =3D=3D -1 || cpu =3D=3D rq->cpu) + return 0; + + /* + * Given we found a CPU with lower priority than @next_task, + * therefore it should be running. However we cannot migrate it + * to this other CPU, instead attempt to push the current + * running task on this CPU away. + */ + push_task =3D get_push_task(rq); + if (push_task) { + preempt_disable(); + raw_spin_rq_unlock(rq); + stop_one_cpu_nowait(rq->cpu, push_cpu_stop, + push_task, &rq->push_work); + preempt_enable(); + raw_spin_rq_lock(rq); + } =20 - if (is_dl_group(rt_rq)) return 0; + } + + if (WARN_ON(next_task =3D=3D rq->curr)) + return 0; + + /* We might release rq lock */ + get_task_struct(next_task); + + /* group_find_lock_lowest_rq locks the rq if found */ + lowest_rt_rq =3D group_find_lock_lowest_rt_rq(next_task, rt_rq); + if (!lowest_rt_rq) { + struct task_struct *task; + /* + * group_find_lock_lowest_rt_rq releases rq->lock + * so it is possible that next_task has migrated. + * + * We need to make sure that the task is still on the same + * run-queue and is also still the next task eligible for + * pushing. + */ + task =3D pick_next_pushable_task(rt_rq); + if (task =3D=3D next_task) { + /* + * The task hasn't migrated, and is still the next + * eligible task, but we failed to find a run-queue + * to push it to. Do not retry in this case, since + * other CPUs will pull from us when ready. + */ + goto out; + } + + if (!task) + /* No more tasks, just exit */ + goto out; + + /* + * Something has shifted, try again. + */ + put_task_struct(next_task); + next_task =3D task; + goto retry; + } + + lowest_rq =3D rq_of_rt_rq(lowest_rt_rq); + + move_queued_task_locked(rq, lowest_rq, next_task); + resched_curr(lowest_rq); + ret =3D 1; + + double_unlock_balance(rq, lowest_rq); +out: + put_task_struct(next_task); + + return ret; +} + +static void group_pull_rt_task(struct rt_rq *this_rt_rq) +{ + BUG_ON(!is_dl_group(this_rt_rq)); + + struct rq *this_rq =3D rq_of_rt_rq(this_rt_rq); + int this_cpu =3D this_rq->cpu, cpu; + bool resched =3D false; + struct task_struct *p, *push_task =3D NULL; + struct rt_rq *src_rt_rq; + struct rq *src_rq; + struct sched_dl_entity *src_dl_se; + + for_each_online_cpu(cpu) { + if (this_cpu =3D=3D cpu) + continue; =20 - return push_rt_task(rq, false); + src_dl_se =3D this_rt_rq->tg->dl_se[cpu]; + src_rt_rq =3D &src_dl_se->my_q->rt; + + if (src_rt_rq->rt_nr_running <=3D 1 && !src_dl_se->dl_throttled) + continue; + + src_rq =3D rq_of_rt_rq(src_rt_rq); + + /* + * Don't bother taking the src_rq->lock if the next highest + * task is known to be lower-priority than our current task. + * This may look racy, but if this value is about to go + * logically higher, the src_rq will push this task away. + * And if its going logically lower, we do not care + */ + if (src_rt_rq->highest_prio.next >=3D + this_rt_rq->highest_prio.curr) + continue; + + /* + * We can potentially drop this_rq's lock in + * double_lock_balance, and another CPU could + * alter this_rq + */ + push_task =3D NULL; + double_lock_balance(this_rq, src_rq); + + /* + * We can pull only a task, which is pushable + * on its rq, and no others. + */ + p =3D pick_highest_pushable_task(src_rt_rq, this_cpu); + + /* + * Do we have an RT task that preempts + * the to-be-scheduled task? + */ + if (p && (p->prio < this_rt_rq->highest_prio.curr)) { + WARN_ON(p =3D=3D src_rq->curr); + WARN_ON(!task_on_rq_queued(p)); + + /* + * There's a chance that p is higher in priority + * than what's currently running on its CPU. + * This is just that p is waking up and hasn't + * had a chance to schedule. We only pull + * p if it is lower in priority than the + * current task on the run queue + */ + if (p->prio < src_rq->curr->prio) + goto skip; + + if (is_migration_disabled(p)) { + /* + * If the current task does not belong to the same task group + * we cannot push it away. + */ + if (src_rq->curr->sched_task_group !=3D this_rt_rq->tg) + goto skip; + + push_task =3D get_push_task(src_rq); + } else { + move_queued_task_locked(src_rq, this_rq, p); + resched =3D true; + } + /* + * We continue with the search, just in + * case there's an even higher prio task + * in another runqueue. (low likelihood + * but possible) + */ + } +skip: + double_unlock_balance(this_rq, src_rq); + + if (push_task) { + preempt_disable(); + raw_spin_rq_unlock(this_rq); + stop_one_cpu_nowait(src_rq->cpu, push_cpu_stop, + push_task, &src_rq->push_work); + preempt_enable(); + raw_spin_rq_lock(this_rq); + } + } + + if (resched) + resched_curr(this_rq); } =20 static void group_push_rt_tasks(struct rt_rq *rt_rq) { - while (group_push_rt_task(rt_rq)) + while (group_push_rt_task(rt_rq, false)) ; } -#else -static void group_push_rt_tasks(struct rt_rq *rt_rq) + +static void push_group_rt_tasks(struct rq *rq) { - push_rt_tasks(rq_of_rt_rq(rt_rq)); + BUG_ON(rq->rq_to_push_from =3D=3D NULL); + + if ((rq->rq_to_push_from->rt.rt_nr_running > 1) || + (dl_group_of(&rq->rq_to_push_from->rt)->dl_throttled =3D=3D 1)) { + group_push_rt_task(&rq->rq_to_push_from->rt, false); + } + + rq->rq_to_push_from =3D NULL; } -#endif + +static void pull_group_rt_tasks(struct rq *rq) +{ + BUG_ON(rq->rq_to_pull_to =3D=3D NULL); + BUG_ON(rq->rq_to_pull_to->rt.rq !=3D rq); + + group_pull_rt_task(&rq->rq_to_pull_to->rt); + rq->rq_to_pull_to =3D NULL; +} +#else /* CONFIG_RT_GROUP_SCHED */ +static void group_pull_rt_task(struct rt_rq *this_rt_rq) { } +static void group_push_rt_tasks(struct rt_rq *rt_rq) { } +#endif /* CONFIG_RT_GROUP_SCHED */ =20 /* * If we are not running and we are not going to reschedule soon, we should diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 3dd2ede6d35..10e29f37f9b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1282,6 +1282,11 @@ struct rq { call_single_data_t cfsb_csd; struct list_head cfsb_csd_list; #endif + +#ifdef CONFIG_RT_GROUP_SCHED + struct rq *rq_to_push_from; + struct rq *rq_to_pull_to; +#endif }; =20 #ifdef CONFIG_FAIR_GROUP_SCHED --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3BEE2D0C89 for ; Thu, 31 Jul 2025 10:56:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959364; cv=none; b=VaXLaGoxmtLiqY/vYFlmYxY6JxaeS5/wbmD+wkNX9fI3DIt2tY47vlFmBFx2qKd3INZAQLqrHOMrXr0UKUXCnsLIJ6DU9emOKHjR6jMBzlQkC2qFTDik9aUTtGd7r4upy+V6WaP8MsmsumV6gaDC3i0S+2IJVgEIaAUqsbcDbxk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959364; c=relaxed/simple; bh=rX4+5bu2PZZ9SUBwzZ8uhPmAteJnql3vA02pjPtir7U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Pil1eP+28zguls5icskySvx3fe5Y9KbWdDXsNRtRlDlTqGTVel5a6dVeqnqFPUDvGk668tKpNUjo2urfqiXCDibj1v0b+fdap0aQTRMlTeZN68RtqwhWLwHg2Bj0N9BH2l7qJsYOaq9BLwDqWi4hrEZmkzATmsG2HBIIKa2OwOE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZbW0gtV5; arc=none smtp.client-ip=209.85.128.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZbW0gtV5" Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-4560add6cd2so5031095e9.0 for ; Thu, 31 Jul 2025 03:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959361; x=1754564161; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W9Xf/zncsK0C6ZDulQirQipsNhzEPIaovYvVkcMe4d0=; b=ZbW0gtV5IuSnv3auHfFvZ0DjiH4yATevigbC5d606R6Zzk6tsJjnduy6jzbwMl1+eu Ru0Y9//c3jz28uWRUDK6X1wPPNaRucJHfzxQjsqroMXqvkE7wcl24R4j9kkKk1vmSytM n98FqTailYQRQh3FvfG5bOfzj1C6LziDBaq+MDSr38NaYWlxNo/xv9uMEISAxSjUn/6F IqzAFeDUCT6zE0Lq8/eyC+Mn2TlzjA+VZOoZnR9I8f84Ru+wOULwmjwhetbcQcEiI63A SVY5IuMkw4/9g4cUDKF4Pj+8c6NkWkoMU+QzF/DRWDVIPltVfgDKDYvRHU9/R+KH5qGS dY4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959361; x=1754564161; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W9Xf/zncsK0C6ZDulQirQipsNhzEPIaovYvVkcMe4d0=; b=tDnZhQKuekJr80xCc1WGghe1LMKmvK94/DllXfBMiNO6Xv6cssPpKZo2mt78dCFOiR 6rzeBaUkwANYmVXv+Uj/kj3ZwBCZyeBhKN5BnMtM5DKP67YNWBGXbjPWdE5pUhMoPIRD l+euM6v8ohblX4TsBt2pR6LSX7w9Keg44qRPwxYO1WkPum+p6M4gdLSJE0GEmX3BwkSD t0QK54/d6FQcjngPo3JWmAGHfie/F9A5cI/D+ncWhstpjNnf9aXeAGn1jO9BNNt1/TAB RQ4deuT59T1f5UgRjediLJ5qfnlwAM7gyHKwvzTj+V3C8tNeyLkbAs3I3kVq3Ps59d5r PXvQ== X-Gm-Message-State: AOJu0Yyq3jSMQpT4Ki8fvl+HJaAdaYhxYWUOLH3Rj5+4GjhjJSQ5W7cA Awfq7F5CUyxBhT5m5yl5N27R0WIhMc+hfyFFoWUSRXBS7b6Ae5hr670G X-Gm-Gg: ASbGncv2qtefY602Rl79MOAaiQtQtkre2eXZXfvoZLzg/CgMRguJUGN6lxFh77yPx/W AkOqJSaYOC4E0adsIbQC/TDRrmfOfKXaAFBfROfDrKI9GXXj5FzwAVquLeJJgjYkqrch3OCuDZT XbaBHKdY7ebuGX9f3iFHzN6qM1hS0IJCTOf679INxroJA5USnKhkFwKaK5HF2iMcNVthrMAkyb4 Slu/CqTO8/HUAof7WozovVNSIamjF+qByEmbW5MwNWpYu3zTFseg+AES94pt56CIN9LwWxODCgR YSkf8Q8qAgojiWHBo9bwSDUWlEYDUmgDXiK3BH3/CyU81MgE22bN+oeos8WaQ6a+268/pxb+ndC LG+Zg424LFGX7aMW16sg= X-Google-Smtp-Source: AGHT+IHf6AKSb9RscPmZGD57JpqKv90J23PCflYXkYWObEj1yT0k2mVVPC0on5btjQdD5EuCvdcOsw== X-Received: by 2002:a05:600c:4ab0:b0:456:22f8:3aa1 with SMTP id 5b1f17b1804b1-458a1e0c00fmr11349765e9.2.1753959361111; Thu, 31 Jul 2025 03:56:01 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.56.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:56:00 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 21/25] sched/rt: add HCBS migration related checks and function calls Date: Thu, 31 Jul 2025 12:55:39 +0200 Message-ID: <20250731105543.40832-22-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Add HCBS related checks and operations to allow rt-task migration, differentiating between cgroup's tasks or tasks that run on the global runq= ueue. Co-developed-by: Alessio Balsini Signed-off-by: Alessio Balsini Co-developed-by: Andrea Parri Signed-off-by: Andrea Parri Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/rt.c | 63 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 48 insertions(+), 15 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 2fdb2657554..677ab9e8aa4 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -865,6 +865,11 @@ select_task_rq_rt(struct task_struct *p, int cpu, int = flags) struct rq *rq; bool test; =20 + /* Just return the task_cpu for processes inside task groups */ + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && + is_dl_group(rt_rq_of_se(&p->rt))) + goto out; + /* For anything but wake ups, just return the task_cpu */ if (!(flags & (WF_TTWU | WF_FORK))) goto out; @@ -964,7 +969,10 @@ static int balance_rt(struct rq *rq, struct task_struc= t *p, struct rq_flags *rf) * not yet started the picking loop. */ rq_unpin_lock(rq, rf); - pull_rt_task(rq); + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq_of_se(&p->rt)= )) + group_pull_rt_task(rt_rq_of_se(&p->rt)); + else + pull_rt_task(rq); rq_repin_lock(rq, rf); } =20 @@ -1050,7 +1058,10 @@ static inline void set_next_task_rt(struct rq *rq, s= truct task_struct *p, bool f if (rq->donor->sched_class !=3D &rt_sched_class) update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 0); =20 - rt_queue_push_tasks(rq); + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + rt_queue_push_from_group(rq, rt_rq); + else + rt_queue_push_tasks(rq); } =20 static struct sched_rt_entity *pick_next_rt_entity(struct rt_rq *rt_rq) @@ -1113,6 +1124,13 @@ static void put_prev_task_rt(struct rq *rq, struct t= ask_struct *p, struct task_s */ if (on_rt_rq(&p->rt) && p->nr_cpus_allowed > 1) enqueue_pushable_task(rt_rq, p); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq); + + if (dl_se->dl_throttled) + rt_queue_push_from_group(rq, rt_rq); + } } =20 /* Only try algorithms three times */ @@ -2174,8 +2192,13 @@ static void task_woken_rt(struct rq *rq, struct task= _struct *p) (rq->curr->nr_cpus_allowed < 2 || rq->donor->prio <=3D p->prio); =20 - if (need_to_push) + if (!need_to_push) + return; + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) group_push_rt_tasks(rt_rq); + else + push_rt_tasks(rq); } =20 /* Assumes rq->lock is held */ @@ -2216,7 +2239,9 @@ static void switched_from_rt(struct rq *rq, struct ta= sk_struct *p) if (!task_on_rq_queued(p) || rt_rq->rt_nr_running) return; =20 - if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED)) + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + rt_queue_pull_to_group(rq, rt_rq); + else rt_queue_pull_task(rq); } =20 @@ -2243,6 +2268,13 @@ static void switched_to_rt(struct rq *rq, struct tas= k_struct *p) */ if (task_current(rq, p)) { update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 0); + + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq_of_se(&p->rt)= )) { + struct sched_dl_entity *dl_se =3D dl_group_of(rt_rq_of_se(&p->rt)); + + p->dl_server =3D dl_se; + } + return; } =20 @@ -2252,17 +2284,14 @@ static void switched_to_rt(struct rq *rq, struct ta= sk_struct *p) * then see if we can move to another run queue. */ if (task_on_rq_queued(p)) { - -#ifndef CONFIG_RT_GROUP_SCHED - if (p->nr_cpus_allowed > 1 && rq->rt.overloaded) + if (!is_dl_group(rt_rq_of_se(&p->rt)) && p->nr_cpus_allowed > 1 && rq->r= t.overloaded) { rt_queue_push_tasks(rq); -#else - if (rt_rq_of_se(&p->rt)->overloaded) { - } else { - if (p->prio < rq->curr->prio) - resched_curr(rq); + return; + } else if (is_dl_group(rt_rq_of_se(&p->rt)) && rt_rq_of_se(&p->rt)->over= loaded) { + rt_queue_push_from_group(rq, rt_rq_of_se(&p->rt)); + return; } -#endif + if (p->prio < rq->donor->prio && cpu_online(cpu_of(rq))) resched_curr(rq); } @@ -2285,8 +2314,12 @@ prio_changed_rt(struct rq *rq, struct task_struct *p= , int oldprio) * If our priority decreases while running, we * may need to pull tasks to this runqueue. */ - if (!IS_ENABLED(CONFIG_RT_GROUP_SCHED) && oldprio < p->prio) - rt_queue_pull_task(rq); + if (oldprio < p->prio) { + if (IS_ENABLED(CONFIG_RT_GROUP_SCHED) && is_dl_group(rt_rq)) + rt_queue_pull_to_group(rq, rt_rq); + else + rt_queue_pull_task(rq); + } =20 /* * If there's a higher priority task waiting to run --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 943232D1301 for ; Thu, 31 Jul 2025 10:56:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959365; cv=none; b=pagsJyl6EWTCYlPJu/rOZF0RRPrio+waH+NWhbHLbgfDPhxUmUtj0tm20ouvjEqkQ7NjYlEr09qi7h6m04Naaf7QdDz3DZhXAf3Def9I4ZtJl0GbRiRmSgqZu/o8v2hjganY6Qyz1+PsmolK6W8ZOOLosdipp58HJCz3bCDSolk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959365; c=relaxed/simple; bh=33Z2Y3YOQwE+2eVhBbmvS1yb+JUXhQFuskeiQAMZ/eI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mX3PETAgi4bXR1j3y5K5rmflG3dRwezwhCWF+67zMfIyW6VeOzdiq0Hq5QD4wbz2r6seESvLHRfpPEmzMOQzuj06+A5oGhmBYsCEVYSONNNqST15rXP3IbauUy0ULqRIIuP+87mGJ0Tl8HFXuvBy97cvB9PzFzxciVyRn5IHPLQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ScXvXLmk; arc=none smtp.client-ip=209.85.221.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ScXvXLmk" Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-3b78a034f17so658456f8f.2 for ; Thu, 31 Jul 2025 03:56:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959362; x=1754564162; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UExeYnjsYht+iw9f1NhGR2WA3gC27Fs4pKhtTeEITRw=; b=ScXvXLmkN8UP52N82Gs6YPT0ps9j7/hc0PRYkd1ScJ8FTrVvRnfwSv8yKF8OQ41tdw VU3R/feYFysU4QMswkdvUjiqkdIsCon8eXjzs9pyUWle0rIp//yIHJYJN5p2lRfU9gZU voz9I+niHWEErNgHOQE+WZYG0/Lffr8i85LJbDxDqvJhzoixZKF8a7t9C0IiMh6Pk/4U owH83ueuLFD3F2reToQODX1RxeNUEDHvpeck+gdE3r4u1Sw++NKnCMlwS98/0SQvhpJX dnTXYuZjW52O/HsnhXdGKvCLE4RPEy8FRlq4r5FsFB2IkHF7omo5HhTIbWWVsLODWNth GMjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959362; x=1754564162; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UExeYnjsYht+iw9f1NhGR2WA3gC27Fs4pKhtTeEITRw=; b=fV4cTjcffV8K91wsoHxu1W7Rg9ZSZGWZurY1uDh4zNXeyY7DHoj+6W0YkqmmsmI/dT 9xArt+X7MrpsDr3fGj5IxDirlh8MB/a2H4th4VyOmgCMqiBUIciSvDa3IoRJlmOkqYii jdJdoCzqoPnGZDLkQfjnbM5mVXihUqPjyeMKvYmAyP+twLXbjLVh1/bgvNyyrL5G0gdV 2jZQY3tYEEI6qHCruUyLR8u8BQtTfpm4+ukKVXVYUPAy1ec+EYQtl1lFryEYpe7EE2fB l69fXJ29uATODv6n3O6Ny8z6+/Em78B2rcIusnDes93Pwp+9HYBi4vIWecVHh6bpRZt5 AjGQ== X-Gm-Message-State: AOJu0YzkEDluRFDtslk4qLXNCmmfJXB2lA8+AJeUwyAYzFUqBHuuUumb v2+DdplDd3lz3976Dy4U2ij+SQpn483mJCb62H3yIbPzVbNgdFQFpbD8BAPfRLY4LKs= X-Gm-Gg: ASbGncslzoFYUkFxlcvaitylmIV1m9K8lmqQVk51fUirLN8NhqrBeiaqN84C/kuvsAG 70rovltisFfi93I7csvt7admG2OAVq5yJcqA856rIqIZ6LraBTLA7jZaBX4ZhWCY3EDs6o8F0HH TkWLaMEF2fiVwYw1MpWH4fCdMEy+YNQ9vBGCZq0hcTjpVG7qUCs5hKH1pnzVB4p723kKXbMbSG1 FCmSu2AHg3sXkVDE2ku/SZT/SQ6Aal7hQjDiScZXBhrWsaQyETlQDosNBPYYwAZT10oarJaObEZ 1Go5RgMo7TFSv3Pyy1XfBrNzrUhJYrOGRo75Pp9OKJM3vsfl4Hn/8RxFKE8DweqtCafqZSexqCu 7RRp8RE0/Odz8khUQt1PwQ6USl5vLKA== X-Google-Smtp-Source: AGHT+IFMnFvjaD1MozpayBu55S0JsGLwx1gfUJPUN3Iab9yjfsTa279BuV0zO+Jai5iJv9FjhBCtyw== X-Received: by 2002:a05:6000:288b:b0:3b7:9350:44d4 with SMTP id ffacd0b85a97d-3b794fc2be0mr5414578f8f.11.1753959361857; Thu, 31 Jul 2025 03:56:01 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.56.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:56:01 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 22/25] sched/deadline: Make rt-cgroup's servers pull tasks on timer replenishment Date: Thu, 31 Jul 2025 12:55:40 +0200 Message-ID: <20250731105543.40832-23-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/deadline.c | 6 +++++- kernel/sched/rt.c | 6 +++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index e263abcdc04..021d7349897 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1308,6 +1308,7 @@ static enum hrtimer_restart dl_server_timer(struct hr= timer *timer, struct sched_ { struct rq *rq =3D rq_of_dl_se(dl_se); u64 fw; + bool has_tasks; =20 scoped_guard (rq_lock, rq) { struct rq_flags *rf =3D &scope.rf; @@ -1321,7 +1322,10 @@ static enum hrtimer_restart dl_server_timer(struct h= rtimer *timer, struct sched_ if (!dl_se->dl_runtime) return HRTIMER_NORESTART; =20 - if (!dl_se->server_has_tasks(dl_se)) { + rq_unpin_lock(rq, rf); + has_tasks =3D dl_se->server_has_tasks(dl_se); + rq_repin_lock(rq, rf); + if (!has_tasks) { replenish_dl_entity(dl_se); dl_server_stopped(dl_se); return HRTIMER_NORESTART; diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 677ab9e8aa4..116fa0422b9 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1,4 +1,3 @@ -#pragma GCC diagnostic ignored "-Wunused-function" // SPDX-License-Identifier: GPL-2.0 /* * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR @@ -145,6 +144,11 @@ void init_tg_rt_entry(struct task_group *tg, struct rq= *served_rq, =20 static bool rt_server_has_tasks(struct sched_dl_entity *dl_se) { +#ifdef CONFIG_SMP + struct rt_rq *rt_rq =3D &dl_se->my_q->rt; + + group_pull_rt_task(rt_rq); +#endif return !!dl_se->my_q->rt.rt_nr_running; } =20 --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8CBE32D238D for ; Thu, 31 Jul 2025 10:56:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959366; cv=none; b=aWrPphHs+oxjWLf5NrHsFOfTdsNeC6uOanmgehYlo4tIHmov0+7nBaCc60d7Iu6ixkMUM1ZfIJz7LTmeewwhuzrhC+KQKImU4oP4l7aMCBcDQER/cCbr7v/8HToYA39zMUl8+TiUNUQwbdbIkboDoGTpwQgIiavLlJKG/lS5zKg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959366; c=relaxed/simple; bh=OpUMuETLodfrnIf+F1HVlkdpkuJLOZnropoVxSeakd0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GXlzW/ahXtZ9V3s8i9+3zZBu+TzRGaUYGSNNHDCWkbxuG6WYszSHf+rQ3NW355lQI0bBXcYVYXh4EzdSpJJ4lA7lljfImFNcEaZ0NmCwVAbIxmUPYKvnpSp7HRJ6FzRltk9WYEIcaH+CaSFS3dYdueEJNjTB7L3wXSIu/E4aat0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cxYxu848; arc=none smtp.client-ip=209.85.128.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cxYxu848" Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-45634205adaso3468135e9.2 for ; Thu, 31 Jul 2025 03:56:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959363; x=1754564163; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kIm3rlyPz0rfM/eD/RBrN306alHhzSHRloZOtfBSX6s=; b=cxYxu848Tb5n1L/RNiiCH4btunaC5NV272alBxmqbFj47fVh9xeTKoE1N4zWYhXMqc xpKNH6B+PBIZxSdR0/zeGLzgWMFS3t26il9ByHh/z8OYsM3Wae1+C+uzaptgteroH+hY mMsrPjuEsZTUeYz5mxgDT+Wjw9Wu209wjV/QU4bJtgD06mWdqMO9N8tQ8CdzE0jjOXYV UNv79ROUssgGaji1tpDQTiKG46zzLR1dcD9Nmo+DcVYFhfdLMqHLPGfIGJIbcN0EdjGr swBJ+RgSt7psQVqicX0T6GHYFtRSUAL2cxwi3h+cyDTnOk6XFAOb4P2O1nPCJiXQzeBE byDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959363; x=1754564163; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kIm3rlyPz0rfM/eD/RBrN306alHhzSHRloZOtfBSX6s=; b=N4wP8kQ2m1/7xPuKqZ+VPWHdH9tNfF96a6PMVsQGU9qYqFAfcvzq3qHXCuIldn4oN+ Kwm5XWWQOJJ3xohAz9U8MxmyN4YNGOnMdrK0n1pchY1QdiJ/tqxMzvgIBl2HkuzBh3Lu znCkmPQCLw/G8uRZKrwstEk1v8st3l5Wth+sg2ptOh4ApRTHcv/zAWLDC5xEcohIy2/X HdMki7bMZX+CdKuuyfDrGyPxtBaMP3hFt2LrbtBxo9ByjPIQAW4hG3//3kQlS3PwJUCQ TmcZR+BoCDOT8re3XGkmyWYKt8fxtXQ+xLwVoBP7fkduj1USLmxy+T0xB2Cix5zG+QK+ 86lA== X-Gm-Message-State: AOJu0YyB6Clp3PFXYthnmNBx6D2PUqDwWDdkRmaGlxCpAGW0FI7sSxQG ZT1NxRopjEjYkpyw5IHSyNisMFvrcD6CFyWEXYBhvJekCX8tRC3WELi5 X-Gm-Gg: ASbGncsbPvS+vezb2ga+ZTWY5lVSOBdzlbDWz+h7hGBitymMwVto5A6+6utmYFeSZ99 m+LvAPn6XoY3Iq5re1PQRRQPStXyuUEFGTXvdrVLaTTNza+MWSkzCbiSI2Uhh4pvWOaRIPA4Urw IdBm/Gh2/Ty2W34TK5OSga/xBMBAD8aPdh9OsfLJEdrzyky5A+0f5mfqwXx821mPs1QRZSOZby9 ewhYXZdcSTZ4240ySXDoRdFgbyXRAN/OkAUcsGsO3iwhtodVzfqTxxdzhPMBk8N2Fj7sMsFJZn/ +YjVCygeIZiK/ZD/gGYir1PHRodzunvY9zlXlR5zGHnSmjM9I2Q18HV8Mcp6mWQRQMk6fZpCPYG lUTbuHu3ah4uXcxsVqUo= X-Google-Smtp-Source: AGHT+IHRBj19Y/8c4RtBt8R1QreJkcQ2oxrrOP8mZ0lIqy+GRRcjW42OC0INVfegbbJW2IPN3jmfLw== X-Received: by 2002:a05:600c:3ba6:b0:450:cf42:7565 with SMTP id 5b1f17b1804b1-45892bce8c7mr50859405e9.23.1753959362600; Thu, 31 Jul 2025 03:56:02 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.56.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:56:02 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 23/25] sched/deadline: Fix HCBS migrations on server stop Date: Thu, 31 Jul 2025 12:55:41 +0200 Message-ID: <20250731105543.40832-24-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Do not unthrottle a non-fair-server dl_server_stop(), since it ends up being stopped when throttled (we try to migrate all the RT tasks away from it). Notes: This is a temporary workaround, but it will be hopefully removed in favor of less invasive code. Co-developed-by: Yuri Andriaccio Signed-off-by: Yuri Andriaccio Signed-off-by: luca abeni --- kernel/sched/deadline.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 021d7349897..d9ab209a492 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1723,9 +1723,11 @@ void dl_server_stop(struct sched_dl_entity *dl_se) return; =20 dequeue_dl_entity(dl_se, DEQUEUE_SLEEP); - hrtimer_try_to_cancel(&dl_se->dl_timer); + if (dl_se =3D=3D &rq_of_dl_se(dl_se)->fair_server) { + hrtimer_try_to_cancel(&dl_se->dl_timer); + dl_se->dl_throttled =3D 0; + } dl_se->dl_defer_armed =3D 0; - dl_se->dl_throttled =3D 0; dl_se->dl_server_active =3D 0; } =20 --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2118B2D23BC for ; Thu, 31 Jul 2025 10:56:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959367; cv=none; b=VHwroZLH1kxHketKRGv4EHMhRlMHstTnnauCKJMoJxGOJavM35/4ZUmF1/8tlKILZL4kIWzUxccN2Sf/DtvuNZ7CMqdkopC5xixFgBcUwwrQZdRywd0YK04oQkbHYnKSQnDF1ocKeieW3v6rurS+3YQkQaAGE84VxBnCVM9OICE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959367; c=relaxed/simple; bh=CXH/nbHLh4BZqInfPZ2i20418s2hQeb0ON9avKB+N0g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dBqdD++Z/HTdRYi8EvP/rwp5hYFaTCTnjdq6UQGrNFncD+344O6MvxH4YXhaxbAqg5SGkcMFN1swTmH/S6V2YGc/snBSn/qxAawqHcOC8vPu5GXSQkqeHqJb8yrm9kjQUluqr4E+smFLm2+h2xjQ1B5MMdiSaiyscwksBItkCac= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EmDDztlT; arc=none smtp.client-ip=209.85.221.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EmDDztlT" Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-3b783d851e6so837752f8f.0 for ; Thu, 31 Jul 2025 03:56:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959363; x=1754564163; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aloRYfRZHcWQKm9mHZFlIFmOkTIhwAj/KWqtVi8IvZo=; b=EmDDztlTGrXPH6PfrT8ynX8uXIon1lgGft6GGL0m3toaRbk9g5Y/Kqg66nL8zAk9eC LC1ecbDpVHg/UtLnFiZyFhX2UrX/i8bdr731zA9+DzmjdrfJUN3nrqzNBgH8IXd6Fr0c ZrUhqkelVbJeVCWweHz8PfCGVAiYEN0h6ddemlXhO9lojLNYSM801jxY7VsCkdISj3h+ Hqe8/cgL9Ppu+5oJ2u4JaspnoiNiH6JPBya0Elbflb56ku5CSYo4GaJnvZCI6wdnqBmT 23nD2KQaA7/zXWHfp/CXce2Vrzgn+mZ2/ygtAzgCx6M0NgPMsdOtdIk/O5HMYnwR54DS m2Tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959363; x=1754564163; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aloRYfRZHcWQKm9mHZFlIFmOkTIhwAj/KWqtVi8IvZo=; b=UvNgXGAE7d4A6/VkxJanIP2Nz1OA4jRJGrditE0WQCy2RXivRWIe/BQyF9NOiFwTlD lYtG+9bCCCYEgmthGnoFzumQ+ukRxo0E9mk5PKeay1kdvnxL959P7BlW0dGjtkv1mhwI RBoisAcuGKkvRJ2dSR5gjM7U2LNERNTIL79AbUVztpE30DDCCwvSxvfiow/hxeiptQrd FEreB+uVERmf02iszz3M2p5Zinp4hOplFfCeaE8OeBE7vx30jdvWRsU0JQsIrySPUDtg 36SzcLyDb0MrtDbSgF3zwqT6LpzdVrFCWJY1P/xyyGcF106hSniXD5HNsy/Syuvj71nD xlRA== X-Gm-Message-State: AOJu0Yz8FgL4qBrij/Ps8IxcnW42ohDgUNMYwyK9De+w47sgOM4cTttG 0sr4+s2BMkIYK8/Wv8Cne/q2nvuaqM62b35L0q/gjjfBgYiToeuoG/Ty X-Gm-Gg: ASbGncuFY2rfd+xIOLjWZ0lOgULTRuEOM2SqrmpUOK6obmc1m0BcpKY7rAru4R8Pe1s 85BySvFAY31j6676mak2d1MW3pI6gIDtVrI3WDlpEN7bIwLk12lHdbrMj4J7ko9i/Vl+wM90Jlg 4YiLt2kFq1SOgNVuU/WIJnDTnH+Gw3nY/H3nNfHD7qx7AoFeLlpoeHFiUQdT5jh8+KLXlQBQV2+ //Y03FLETZjkSdv7Kk39dV7G1d+L8HmW4ZengWDYLLAcdUbm1nJy+FGIe3eH7aZT6hGBko2zGkO l06B6k7u2tjrVsqjMlauAAX7Ch4pNlnD9ZWIazd2zOKfLF98yf4YiCtVXPWZ9kPymd6xkljirhO pMIkIc6CCeK7GVwNWhNw= X-Google-Smtp-Source: AGHT+IE2ZLE2HztsKXVGjNeZQAHYrlUVm+ngZJmjVVC6PQHo9dM2yJf52WCYd3LGT2NraaPiE+AosQ== X-Received: by 2002:a05:6000:4305:b0:3a4:f70d:a65e with SMTP id ffacd0b85a97d-3b794ffe750mr4987689f8f.37.1753959363375; Thu, 31 Jul 2025 03:56:03 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.56.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:56:03 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 24/25] sched/core: Execute enqueued balance callbacks when changing allowed CPUs Date: Thu, 31 Jul 2025 12:55:42 +0200 Message-ID: <20250731105543.40832-25-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: luca abeni Execute balancing callbacks when setting the affinity of a task, since the = HCBS scheduler may request balancing of throttled dl_servers to fully utilize the server's bandwidth. Signed-off-by: luca abeni Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index eb9de8c7b1f..c8763c46030 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2947,6 +2947,7 @@ static int affine_move_task(struct rq *rq, struct tas= k_struct *p, struct rq_flag if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask) || (task_current_donor(rq, p) && !task_current(rq, p))) { struct task_struct *push_task =3D NULL; + struct balance_callback *head; =20 if ((flags & SCA_MIGRATE_ENABLE) && (p->migration_flags & MDF_PUSH) && !rq->push_busy) { @@ -2965,11 +2966,13 @@ static int affine_move_task(struct rq *rq, struct t= ask_struct *p, struct rq_flag } =20 preempt_disable(); + head =3D splice_balance_callbacks(rq); task_rq_unlock(rq, p, rf); if (push_task) { stop_one_cpu_nowait(rq->cpu, push_cpu_stop, p, &rq->push_work); } + balance_callbacks(rq, head); preempt_enable(); =20 if (complete) @@ -3024,6 +3027,8 @@ static int affine_move_task(struct rq *rq, struct tas= k_struct *p, struct rq_flag } =20 if (task_on_cpu(rq, p) || READ_ONCE(p->__state) =3D=3D TASK_WAKING) { + struct balance_callback *head; + /* * MIGRATE_ENABLE gets here because 'p =3D=3D current', but for * anything else we cannot do is_migration_disabled(), punt @@ -3037,16 +3042,19 @@ static int affine_move_task(struct rq *rq, struct t= ask_struct *p, struct rq_flag p->migration_flags &=3D ~MDF_PUSH; =20 preempt_disable(); + head =3D splice_balance_callbacks(rq); task_rq_unlock(rq, p, rf); if (!stop_pending) { stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, &pending->arg, &pending->stop_work); } + balance_callbacks(rq, head); preempt_enable(); =20 if (flags & SCA_MIGRATE_ENABLE) return 0; } else { + struct balance_callback *head; =20 if (!is_migration_disabled(p)) { if (task_on_rq_queued(p)) @@ -3057,7 +3065,12 @@ static int affine_move_task(struct rq *rq, struct ta= sk_struct *p, struct rq_flag complete =3D true; } } + + preempt_disable(); + head =3D splice_balance_callbacks(rq); task_rq_unlock(rq, p, rf); + balance_callbacks(rq, head); + preempt_enable(); =20 if (complete) complete_all(&pending->done); --=20 2.50.1 From nobody Sun Oct 5 17:56:34 2025 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0D712C3244 for ; Thu, 31 Jul 2025 10:56:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959368; cv=none; b=KDW+a/VCXRnWhmvdm74gtKEVwdA0mIyUf97J4S3xUacssUQWaLs1ZLDmlxkDiAVTCzcm/RaxHICnbyNeAiG32oDQLTiIrF8Oh75V564JInQncWIaVAQ58Kxdbig01SxzBWtxRl6gZ6Z4k/roo/+7Fvdfs0hws7myPjACq7gx+5k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753959368; c=relaxed/simple; bh=4n9dDvibvFF2K8duQIxb/c28VkIW1FuNIozyqhLdMH0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GGdXHFO/sOZfyF+HNmN4hxYn6HVDjHgsDKj5ICgC5QcLyGDDKY4WJf6dVi3LfCyjGQCpMJs3mjpY2DvHpo7HTicXz210dwODDKSIds3Cu326ZSIH7F+AAqv0DIYyyM4zeUC0m+eoYAlzp2TtFTSlluvAWCKV2VlrAcI/LmPnOyY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UCu68OE6; arc=none smtp.client-ip=209.85.128.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UCu68OE6" Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-4563cfac19cso7808415e9.2 for ; Thu, 31 Jul 2025 03:56:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753959364; x=1754564164; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sIWR8F3615anV5iq4cLOiQAGxUyyEmR9Z0MfkybRjAQ=; b=UCu68OE6O1dtk/EcpkCLNBVwhVzGy3WYrXDQfEho/Bk/uZkrPqo5ndbtETyOvPt7Sp dcUqI0cvh6+7epZghgNti1XS0XYgCMbHQl09QRO8TlAG4bdSO0r30sIXrUxwNEp82fVh qpH1iboObM65Tm+rrAXEYBUk3CW3H+Zsb8SKbkeswZRPVPOvAtAC/si3MOokd4XFu68T 2/hjcngzVRCpwpmDzu4UKLyDEvMBVuRqX2obBF7ZS06L0CQ3H6YDSw4B+pKCoEF74tAw y/whcjjGQiqR8uqtqBNR+RfYYTJz070zgbwoLbp9eVSMrI5srMFCe1KcM3IwMxq2rm8R MXeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753959364; x=1754564164; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sIWR8F3615anV5iq4cLOiQAGxUyyEmR9Z0MfkybRjAQ=; b=p/CyyucwBC22J7uQ6Iz3P04gHkKsyNh3BYqGcN1VdzH4OBxc3vM2CuAfRwk9Dco6Go jNxmfKR7FSTgB0OXFzY4OiDX+Jhn9ru7OEf0fkIEifqQ6WFfdyF7VfgXhOTTFyoiMhOX zd7IYoF7Yq59RC+07gDg6HcmMr/fZ7wk1hvW0gCDtUCghkvNCpM8VXFG0wR/Tf6rx4Bd b37RFqhTCkt8jeDDFSLXGQqex6hLxYpxAe6MRjO6V0XXqoSdkhslXJdogFs3ZV17EeL9 hB7Vc8LIwrNNyOtudnBvLRfsXnoHOg+NjhhieBcM3krTYbhUbxabko7rE6zi9+OE6PqX kfSA== X-Gm-Message-State: AOJu0YwKRJG+1vr1zQayR7A7KuYRap488jgMqdwZLna0bDtR4b9qpiqD XjiTj7ZyJ1wxOGan/J+LYgEudg+97BGff3+QjvLo8qXwmjsIrzV20PlQ X-Gm-Gg: ASbGncv76nzJAASj49qnWe3GG+QRPdqnuHExryygfEzFb/4fQjbMCIiGXj2y79laksV 2HPPqITIYnB75LCvysAKFxhbXJuRiLJrhT7UHEIgWP/tpI3c3iVwNGjvH8k1isBx6B14+mxqZbC AVcqLTjwFL7WJKEP8m2YlmZo2BnUGfm5GfV2WvqtKY5t8+gDposXU64cHeQfJntxda+Oo6UbsNE SoAsB1jQTRpJQIFZeea+LJndNxmWnxByw5V57GkRiNyfW09o25emWjiaYwePaFKUmx7e6v6fxIp vlbs4WJCOK3u5F7sePTi6SxtJrc9CEm3OOCMoxR7UWxdStRXO8/WbJRg+KJL/lI+ftjxKZy2PEo YI0Yo4VhDQZBoN+0m8zA= X-Google-Smtp-Source: AGHT+IGe9POE29jIFgrNmwcAbP7onYa5j4KecD1gmJUY2Z0ojhLUraXguYOR0AQ7z1kOaJ7MTk17Qw== X-Received: by 2002:a05:600c:1da2:b0:456:1ac8:cace with SMTP id 5b1f17b1804b1-45892b9e27emr79953285e9.12.1753959364156; Thu, 31 Jul 2025 03:56:04 -0700 (PDT) Received: from victus-lab ([193.205.81.5]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b79c4696c8sm1946025f8f.55.2025.07.31.03.56.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 03:56:03 -0700 (PDT) From: Yuri Andriaccio To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Cc: linux-kernel@vger.kernel.org, Luca Abeni , Yuri Andriaccio Subject: [RFC PATCH v2 25/25] sched/core: Execute enqueued balance callbacks when migrating task betweeen cgroups Date: Thu, 31 Jul 2025 12:55:43 +0200 Message-ID: <20250731105543.40832-26-yurand2000@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731105543.40832-1-yurand2000@gmail.com> References: <20250731105543.40832-1-yurand2000@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Execute balancing callbacks when migrating task between cgroups, since the = HCBS scheduler, similarly to the previous patch, may request balancing of thrott= led dl_servers to fully utilize the server's bandwidth. Signed-off-by: Yuri Andriaccio --- kernel/sched/core.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index c8763c46030..65896f46e50 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9247,10 +9247,11 @@ void sched_move_task(struct task_struct *tsk, bool = for_autogroup) { int queued, running, queue_flags =3D DEQUEUE_SAVE | DEQUEUE_MOVE | DEQUEUE_NOCLOCK; + struct balance_callback *head; struct rq *rq; + struct rq_flags rf; =20 - CLASS(task_rq_lock, rq_guard)(tsk); - rq =3D rq_guard.rq; + rq =3D task_rq_lock(tsk, &rf); =20 update_rq_clock(rq); =20 @@ -9277,6 +9278,12 @@ void sched_move_task(struct task_struct *tsk, bool f= or_autogroup) */ resched_curr(rq); } + + preempt_disable(); + head =3D splice_balance_callbacks(rq); + task_rq_unlock(rq, tsk, &rf); + balance_callbacks(rq, head); + preempt_enable(); } =20 static struct cgroup_subsys_state * --=20 2.50.1