From nobody Wed Oct 8 13:21:51 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BECA82BEFED for ; Fri, 27 Jun 2025 11:51:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025121; cv=none; b=HJHHpLZ8akLKJrA8AFW+QouJ7Y47fIFWXxtCuAsX4RC4Itci+RpIh3Mq5ZZ9PJGcQJuyc78KdgXNo426X/aQiNqZypF3gAsinAygpxkHmYKlnW97C4D7QNVP5a6KTiP63QKfvY0iPvBVgBYGl1K7v6aDUwoWuRYKBO9DMHsRB6o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025121; c=relaxed/simple; bh=yy9+hyWkYP4tu1FrbCNzXA6zJDfHt7+xCgmHezVuEBs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=srqTNe5syjkcLzXgT2RKx5MARQmPLasKAZN68/vcZXuKE/GZxyMxy6tXONqFwkRo3vkYyUbCoGFS1OsyjgFO5ganAvXyXDqt7l1+T7Qtt7vx75meMxM2UraW/8yeNSdLxgqL3hSxPYOH561CJGtiJwWPYkoY4GvltAHqTSI2ZB4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=O2VfLTmc; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="O2VfLTmc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751025117; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hgBU5EWmfQae5vywUt8Oy8bOTjAoEHoV5AIpgE2mQsQ=; b=O2VfLTmcBot4RNsu9JH9RsV4IVWwQKb6Y/7NXN0k4GB7QdXOQgSZ2njg6ZMlcHvpVIapgy F235wwaekPn/FmXVKSN+LxbHo1RbhQ76KWp7djJkNkkYOKisxweGzjFv1MeDUVofd/n3nk avT6hLtkSBGBuqywFzUoIFWoohMrazg= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-66-pzEuy2HsMsW8mA7AV8t7UA-1; Fri, 27 Jun 2025 07:51:55 -0400 X-MC-Unique: pzEuy2HsMsW8mA7AV8t7UA-1 X-Mimecast-MFC-AGG-ID: pzEuy2HsMsW8mA7AV8t7UA_1751025114 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9FA6E1944A83; Fri, 27 Jun 2025 11:51:53 +0000 (UTC) Received: from jlelli-thinkpadt14gen4.remote.csb (unknown [10.44.32.84]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2685819560A7; Fri, 27 Jun 2025 11:51:47 +0000 (UTC) From: Juri Lelli To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Waiman Long Cc: linux-kernel@vger.kernel.org, Marcel Ziswiler , Luca Abeni , Juri Lelli Subject: [PATCH 1/5] sched/deadline: Initialize dl_servers after SMP Date: Fri, 27 Jun 2025 13:51:14 +0200 Message-ID: <20250627115118.438797-2-juri.lelli@redhat.com> In-Reply-To: <20250627115118.438797-1-juri.lelli@redhat.com> References: <20250627115118.438797-1-juri.lelli@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" dl-servers are currently initialized too early at boot when CPUs are not fully up (only boot CPU is). This results in miscalculation of per runqueue DEADLINE variables like extra_bw (which needs a stable CPU count). Move initialization of dl-servers later on after SMP has been initialized and CPUs are all online, so that CPU count is stable and DEADLINE variables can be computed correctly. Fixes: d741f297bceaf ("sched/fair: Fair server interface") Reported-by: Marcel Ziswiler Signed-off-by: Juri Lelli Tested-by: Marcel Ziswiler # nuc & rock5b --- kernel/sched/core.c | 2 ++ kernel/sched/deadline.c | 50 ++++++++++++++++++++++++++--------------- kernel/sched/sched.h | 1 + 3 files changed, 35 insertions(+), 18 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2f8caa9db78d5..89b3ed637465b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8371,6 +8371,8 @@ void __init sched_init_smp(void) init_sched_rt_class(); init_sched_dl_class(); =20 + sched_init_dl_servers(); + sched_smp_initialized =3D true; } =20 diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 0f30697ad7956..c1f223f372968 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -761,6 +761,8 @@ static inline void setup_new_dl_entity(struct sched_dl_= entity *dl_se) struct dl_rq *dl_rq =3D dl_rq_of_se(dl_se); struct rq *rq =3D rq_of_dl_rq(dl_rq); =20 + update_rq_clock(rq); + WARN_ON(is_dl_boosted(dl_se)); WARN_ON(dl_time_before(rq_clock(rq), dl_se->deadline)); =20 @@ -1580,23 +1582,7 @@ void dl_server_start(struct sched_dl_entity *dl_se) { struct rq *rq =3D dl_se->rq; =20 - /* - * XXX: the apply do not work fine at the init phase for the - * fair server because things are not yet set. We need to improve - * this before getting generic. - */ - if (!dl_server(dl_se)) { - u64 runtime =3D 50 * NSEC_PER_MSEC; - u64 period =3D 1000 * NSEC_PER_MSEC; - - dl_server_apply_params(dl_se, runtime, period, 1); - - dl_se->dl_server =3D 1; - dl_se->dl_defer =3D 1; - setup_new_dl_entity(dl_se); - } - - if (!dl_se->dl_runtime) + if (!dl_server(dl_se)) return; =20 dl_se->dl_server_active =3D 1; @@ -1607,7 +1593,7 @@ void dl_server_start(struct sched_dl_entity *dl_se) =20 void dl_server_stop(struct sched_dl_entity *dl_se) { - if (!dl_se->dl_runtime) + if (!dl_server(dl_se) || !dl_server_active(dl_se)) return; =20 dequeue_dl_entity(dl_se, DEQUEUE_SLEEP); @@ -1626,6 +1612,32 @@ void dl_server_init(struct sched_dl_entity *dl_se, s= truct rq *rq, dl_se->server_pick_task =3D pick_task; } =20 +void sched_init_dl_servers(void) +{ + int cpu; + struct rq *rq; + struct sched_dl_entity *dl_se; + + for_each_online_cpu(cpu) { + u64 runtime =3D 50 * NSEC_PER_MSEC; + u64 period =3D 1000 * NSEC_PER_MSEC; + + rq =3D cpu_rq(cpu); + + guard(rq_lock_irq)(rq); + + dl_se =3D &rq->fair_server; + + WARN_ON(dl_server(dl_se)); + + dl_server_apply_params(dl_se, runtime, period, 1); + + dl_se->dl_server =3D 1; + dl_se->dl_defer =3D 1; + setup_new_dl_entity(dl_se); + } +} + void __dl_server_attach_root(struct sched_dl_entity *dl_se, struct rq *rq) { u64 new_bw =3D dl_se->dl_bw; @@ -1652,6 +1664,8 @@ int dl_server_apply_params(struct sched_dl_entity *dl= _se, u64 runtime, u64 perio int retval =3D 0; int cpus; =20 + guard(rcu)(); + dl_b =3D dl_bw_of(cpu); guard(raw_spinlock)(&dl_b->lock); =20 diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 105190b180203..3058fb6246dab 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -385,6 +385,7 @@ extern void dl_server_stop(struct sched_dl_entity *dl_s= e); extern void dl_server_init(struct sched_dl_entity *dl_se, struct rq *rq, dl_server_has_tasks_f has_tasks, dl_server_pick_f pick_task); +extern void sched_init_dl_servers(void); =20 extern void dl_server_update_idle_time(struct rq *rq, struct task_struct *p); --=20 2.49.0 From nobody Wed Oct 8 13:21:51 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 273952BD59B for ; Fri, 27 Jun 2025 11:52:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025126; cv=none; b=P5aUwoA5bMD1ecL3QilSwcv5w5grYlPFNScB1Ddgi+94FVhNqkTTLFBKLiZtqBQv+lF6LBFnylClKJ2ZnDwvXRqTLKDDlUlW0QUeKwzwosFz13rtagdu0R6fRDQFDuIyKfBYIHrUVg742l0qaGhYo5oV0tyXGsQQAclItTZJDQs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025126; c=relaxed/simple; bh=eLabGndv5WbLVWZBWuCa784Nl9PyS/LDkKRTF1WtRSc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pv7CgNJ6lYUyMUfa5CGfl0Gzw8ZMVBsL/pXedKVaF4lNm7WlQwvxIVsuTSTfpHUsaBn7FXgMsJn7zs5/u+S63LLRrfSTBz03zz0Nw3oP5lYYhHx/mcpNz7vZ1nhyiQNrIiTqdVAXdtspKBc2Dvs6s6nPfSV8K6gz5m0YNBBQfbc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=a1N4YoGN; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a1N4YoGN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751025124; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OYPEz3mUS07c5nVCrOLOsn4ePZ2xxtEuXLNx/SjYWkQ=; b=a1N4YoGNxPtn6AevYODb0O106jXBzVaEk42lBYWLo7NuF25DJQOWmtCT8GY7D57W2gXk+C UCVxtAUD0O6yxgdB4AGGLL72NwEer0G3tH8VHa5YUIpOMOrcLnYALauFaFhDGgGEMG1LH0 bz9MIwqEF8qFriXz4NcyssQwrrQxu6I= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-618-UTx3Z_1INq-JNb5Xn6jw_A-1; Fri, 27 Jun 2025 07:52:02 -0400 X-MC-Unique: UTx3Z_1INq-JNb5Xn6jw_A-1 X-Mimecast-MFC-AGG-ID: UTx3Z_1INq-JNb5Xn6jw_A_1751025121 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 764A418011DF; Fri, 27 Jun 2025 11:52:00 +0000 (UTC) Received: from jlelli-thinkpadt14gen4.remote.csb (unknown [10.44.32.84]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C98DA19560A7; Fri, 27 Jun 2025 11:51:54 +0000 (UTC) From: Juri Lelli To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Waiman Long Cc: linux-kernel@vger.kernel.org, Marcel Ziswiler , Luca Abeni , Juri Lelli Subject: [PATCH 2/5] sched/deadline: Reset extra_bw to max_bw when clearing root domains Date: Fri, 27 Jun 2025 13:51:15 +0200 Message-ID: <20250627115118.438797-3-juri.lelli@redhat.com> In-Reply-To: <20250627115118.438797-1-juri.lelli@redhat.com> References: <20250627115118.438797-1-juri.lelli@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" dl_clear_root_domain() doesn't take into account the fact that per-rq extra_bw variables retain values computed before root domain changes, resulting in broken accounting. Fix it by resetting extra_bw to max_bw before restoring back dl-servers contributions. Fixes: 2ff899e351643 ("sched/deadline: Rebuild root domain accounting after= every update") Reported-by: Marcel Ziswiler Signed-off-by: Juri Lelli Tested-by: Marcel Ziswiler # nuc & rock5b --- kernel/sched/deadline.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index c1f223f372968..7a3b556d45a99 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2907,7 +2907,14 @@ void dl_clear_root_domain(struct root_domain *rd) int i; =20 guard(raw_spinlock_irqsave)(&rd->dl_bw.lock); + + /* + * Reset total_bw to zero and extra_bw to max_bw so that next + * loop will add dl-servers contributions back properly, + */ rd->dl_bw.total_bw =3D 0; + for_each_cpu(i, rd->span) + cpu_rq(i)->dl.extra_bw =3D cpu_rq(i)->dl.max_bw; =20 /* * dl_servers are not tasks. Since dl_add_task_root_domain ignores --=20 2.49.0 From nobody Wed Oct 8 13:21:51 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 255A72BF3FC for ; Fri, 27 Jun 2025 11:52:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025133; cv=none; b=HOfRtZeQI4AofDA+2h7Pm7iN8D2nLA4kqNV0ZPIb9VVNw+9wDWkuG8juBqbdTRLcTliapBdASIrMoyMHA9PeJlbf7JfuBhH/1YWzglFhz1LgXr196RX/yAP8+xLm3We3FuF1L5MOCZFlLSBcytpP/GofuySSvJj9XncWUx9UXPc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025133; c=relaxed/simple; bh=B1fDgTPvcW5CNOENUIG20E3kZIfrYCCjaLOj12iKS54=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JekGIRmPCEwfuRUgL2NyPKcX7NIbkMGKokAWetPimHxrx12ZKpkLxxlNYTzk2Ffm33t3v8WH9ug8sj0obqlaVc1wt2VBDFzMiyibd6RTenA1DetXPpMMIYwgTXqarGi42lEqEs+HLP2wnAF3Kbu0vhzeKNHWjv1EVxAa5GfvlaY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cbi11tNe; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cbi11tNe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751025131; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I5vSNy0YZG1hcCr6zeagT5BwkVdXA07NqtiRR9wtVcQ=; b=cbi11tNe1dd1XRQxM2MfkM37lL5URIhj9LbdCUyRych2epZmbIy/sq0hYP/gJpfaDfcK2a 0MBCec1uQ+w/E7gRBRaeECAfYGQhClqelzmzfzR2kd73Rij/qQDDxwIy3WT/Md+OiCl5T9 xJGYXmvA84ag5CURYR8uM7EHgquQf3E= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-684-hv7pZnlCN1mjkOBSAXu05Q-1; Fri, 27 Jun 2025 07:52:07 -0400 X-MC-Unique: hv7pZnlCN1mjkOBSAXu05Q-1 X-Mimecast-MFC-AGG-ID: hv7pZnlCN1mjkOBSAXu05Q_1751025126 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 147421801BD8; Fri, 27 Jun 2025 11:52:06 +0000 (UTC) Received: from jlelli-thinkpadt14gen4.remote.csb (unknown [10.44.32.84]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0E10E19560AB; Fri, 27 Jun 2025 11:52:00 +0000 (UTC) From: Juri Lelli To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Waiman Long Cc: linux-kernel@vger.kernel.org, Marcel Ziswiler , Luca Abeni , Juri Lelli Subject: [PATCH 3/5] sched/deadline: Fix accounting after global limits change Date: Fri, 27 Jun 2025 13:51:16 +0200 Message-ID: <20250627115118.438797-4-juri.lelli@redhat.com> In-Reply-To: <20250627115118.438797-1-juri.lelli@redhat.com> References: <20250627115118.438797-1-juri.lelli@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" A global limits change (sched_rt_handler() logic) currently leaves stale and/or incorrect values in variables related to accounting (e.g. extra_bw). Properly clean up per runqueue variables before implementing the change and rebuild scheduling domains (so that accounting is also properly restored) after such a change is complete. Reported-by: Marcel Ziswiler Signed-off-by: Juri Lelli Tested-by: Marcel Ziswiler # nuc & rock5b --- kernel/sched/deadline.c | 4 +++- kernel/sched/rt.c | 6 ++++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 7a3b556d45a99..187f324565f92 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -3166,6 +3166,9 @@ void sched_dl_do_global(void) if (global_rt_runtime() !=3D RUNTIME_INF) new_bw =3D to_ratio(global_rt_period(), global_rt_runtime()); =20 + for_each_possible_cpu(cpu) + init_dl_rq_bw_ratio(&cpu_rq(cpu)->dl); + for_each_possible_cpu(cpu) { rcu_read_lock_sched(); =20 @@ -3181,7 +3184,6 @@ void sched_dl_do_global(void) raw_spin_unlock_irqrestore(&dl_b->lock, flags); =20 rcu_read_unlock_sched(); - init_dl_rq_bw_ratio(&cpu_rq(cpu)->dl); } } =20 diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 15d5855c542cb..be6e9bcbe82b6 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2886,6 +2886,12 @@ static int sched_rt_handler(const struct ctl_table *= table, int write, void *buff sched_domains_mutex_unlock(); mutex_unlock(&mutex); =20 + /* + * After changing maximum available bandwidth for DEADLINE, we need to + * recompute per root domain and per cpus variables accordingly. + */ + rebuild_sched_domains(); + return ret; } =20 --=20 2.49.0 From nobody Wed Oct 8 13:21:51 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B2D822C08B8 for ; Fri, 27 Jun 2025 11:52:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025139; cv=none; b=BsHJtl2Tl2kjVnWbdq+UxQ/zmw+AShIY0yYZNwSLjmPByyltjyad9Lw+S11dpQI9Vs5G2wEKFAzeBReAPAzQOW53rLkDMpAvPLFzI5KSZAy9aMUMUCRdPrgOaI0VibZUfYrg0+qWH5B4ypSe+fNqSk4RnWRgXa1N8x+zVCKP2Lw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025139; c=relaxed/simple; bh=oe/djtKSTdSzpUdhBo9TuVrBDGew6WT12dilaDKbJ6g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iL7dtcJwRW4aPXJHxBwBxJ6lhS+NxrZwYlITuHpNiH48MACsL9XhTM8fWwaWP66+5rjdWQZ6vz23Q5yWKwZ31KAUwCAXNdL5IrdkIjqc2sXnte3AKEcshPqbV9L050rdnUVxYP4IZu2gKUG2sFpPVreugjNgaw/gC+xV4Jy29OU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UNXA1+ZP; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UNXA1+ZP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751025136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OUoybRgoNWhIkWcgJ2UioTXoFAnH9jtRNvXiFttaDg0=; b=UNXA1+ZPM0YMv2tWZRFJQnsLPdR04Y6hIptB73QlPj3DMeXMjYW4tltMm15UYITH2HLpjE 1J8t3Wu0W8tZhMUSeqjwKvHfoqzSogiMcDaDzakAJSp8eVJz3KOAiYR32saWlKS0SmrUvA 5BMB3U7IQIE7aNefjKInyPId/yTH7jE= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-644-nQlPTTL4NhWbScIck-psfg-1; Fri, 27 Jun 2025 07:52:13 -0400 X-MC-Unique: nQlPTTL4NhWbScIck-psfg-1 X-Mimecast-MFC-AGG-ID: nQlPTTL4NhWbScIck-psfg_1751025132 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B81341954198; Fri, 27 Jun 2025 11:52:11 +0000 (UTC) Received: from jlelli-thinkpadt14gen4.remote.csb (unknown [10.44.32.84]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A0A2619560A7; Fri, 27 Jun 2025 11:52:06 +0000 (UTC) From: Juri Lelli To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Waiman Long Cc: linux-kernel@vger.kernel.org, Marcel Ziswiler , Luca Abeni , Juri Lelli Subject: [PATCH 4/5] tools/sched: Add root_domains_dump.py which dumps root domains info Date: Fri, 27 Jun 2025 13:51:17 +0200 Message-ID: <20250627115118.438797-5-juri.lelli@redhat.com> In-Reply-To: <20250627115118.438797-1-juri.lelli@redhat.com> References: <20250627115118.438797-1-juri.lelli@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Root domains information is somewhat hard to access at runtime. Even with sched_debug and sched_verbose, such information is only printed on kernel console when domains are modified. Add a simple drgn script to more easily retrieve root domains information at runtime. Since tools/sched is a new directory, add it to MAINTAINERS as well. Signed-off-by: Juri Lelli Tested-by: Marcel Ziswiler # nuc & rock5b --- MAINTAINERS | 1 + tools/sched/root_domains_dump.py | 68 ++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) create mode 100755 tools/sched/root_domains_dump.py diff --git a/MAINTAINERS b/MAINTAINERS index a92290fffa163..b986a49383c9c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -22165,6 +22165,7 @@ F: include/linux/wait.h F: include/uapi/linux/sched.h F: kernel/fork.c F: kernel/sched/ +F: tools/sched/ =20 SCHEDULER - SCHED_EXT R: Tejun Heo diff --git a/tools/sched/root_domains_dump.py b/tools/sched/root_domains_du= mp.py new file mode 100755 index 0000000000000..56dc91f017b20 --- /dev/null +++ b/tools/sched/root_domains_dump.py @@ -0,0 +1,68 @@ +#!/usr/bin/env drgn +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2025 Juri Lelli +# Copyright (C) 2025 Red Hat, Inc. + +desc =3D """ +This is a drgn script to show the current root domains configuration. For = more +info on drgn, visit https://github.com/osandov/drgn. + +Root domains are only printed once, as multiple CPUs might be attached to = the +same root domain. +""" + +import os +import argparse + +import drgn +from drgn import FaultError +from drgn.helpers.common import * +from drgn.helpers.linux import * + +def print_root_domains_info(): + + # To store unique root domains found + seen_root_domains =3D set() + + print("Retrieving (unique) Root Domain Information:") + + runqueues =3D prog['runqueues'] + def_root_domain =3D prog['def_root_domain'] + + for cpu_id in for_each_possible_cpu(prog): + try: + rq =3D per_cpu(runqueues, cpu_id) + + root_domain =3D rq.rd + + # Check if we've already processed this root domain to avoid d= uplicates + # Use the memory address of the root_domain as a unique identi= fier + root_domain_cast =3D int(root_domain) + if root_domain_cast in seen_root_domains: + continue + seen_root_domains.add(root_domain_cast) + + if root_domain_cast =3D=3D int(def_root_domain.address_): + print(f"\n--- Root Domain @ def_root_domain ---") + else: + print(f"\n--- Root Domain @ 0x{root_domain_cast:x} ---") + + print(f" From CPU: {cpu_id}") # This CPU belongs to this root= domain + + # Access and print relevant fields from struct root_domain + print(f" Span : {cpumask_to_cpulist(root_domain.span[0]= )}") + print(f" Online : {cpumask_to_cpulist(root_domain.span[0]= )}") + + except drgn.FaultError as fe: + print(f" (CPU {cpu_id}: Fault accessing kernel memory: {fe})") + except AttributeError as ae: + print(f" (CPU {cpu_id}: Missing attribute for root_domain (ke= rnel struct change?): {ae})") + except Exception as e: + print(f" (CPU {cpu_id}: An unexpected error occurred: {e})") + +if __name__ =3D=3D "__main__": + parser =3D argparse.ArgumentParser(description=3Ddesc, + formatter_class=3Dargparse.RawTextHel= pFormatter) + args =3D parser.parse_args() + + print_root_domains_info() --=20 2.49.0 From nobody Wed Oct 8 13:21:51 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 244F82D9798 for ; Fri, 27 Jun 2025 11:52:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025145; cv=none; b=qu9z/3cbWsJ/OmJmNrGkjAOUpO6K2MP1bAwI0bpNxiY1loHs+SbvKXn2nsxgs28GH2h1WcvxR+48NUofxUop3gJXQy4VjgrtCDaX6AsSKwkHqYbjMniVZFDx5GvpuzyyPcdUOiFofaJcySQ530DIRHK/zrc0qbETBzx5nj9oQzo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751025145; c=relaxed/simple; bh=+wUi4GXukNo2pvBcAQ46Qj1TJ38IJNJg5MzaCi/0apg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PzC3rIFNWEM7JTVLRxIP0gZYQhqMbYVNTMKjvgjXrqky3B6wcS48ggV+c62Af4Q/NFw6YFXtvC53lGgQCytO8Ce6iwqV/aMoWyudCv33TBKDiMUk5SCbgfP+W3D3urxFSCO1v5p7xPqObl7Wd3RuZ8lDhNMBO0ATnloDo4PAIHU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cj0e/SEr; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cj0e/SEr" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751025143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NK5HahFlaS1XZ4t7oWyohvLCLN5zar3OEBM1RLwvCbE=; b=cj0e/SErSkZH4Z6JId9ow2YmRIePhFsbOYq8x7I0Ks7177ozapDr222DRyrPScJ5xN/LcD 6cbbd6In2KaK8PEH+5pyUjxTOVXd+2t55KNMexugJqQSpdwmMCfkC4Y8cDmLKDnpPpfcz/ +9hNYe15appDWyTT2GKxThVT8MLdXH4= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-468-bJHnK-l7PnaOJpZJ83RoXg-1; Fri, 27 Jun 2025 07:52:19 -0400 X-MC-Unique: bJHnK-l7PnaOJpZJ83RoXg-1 X-Mimecast-MFC-AGG-ID: bJHnK-l7PnaOJpZJ83RoXg_1751025138 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 418DA1801212; Fri, 27 Jun 2025 11:52:18 +0000 (UTC) Received: from jlelli-thinkpadt14gen4.remote.csb (unknown [10.44.32.84]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4F82819560A7; Fri, 27 Jun 2025 11:52:13 +0000 (UTC) From: Juri Lelli To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Waiman Long Cc: linux-kernel@vger.kernel.org, Marcel Ziswiler , Luca Abeni , Juri Lelli Subject: [PATCH 5/5] tools/sched: Add dl_bw_dump.py for printing bandwidth accounting info Date: Fri, 27 Jun 2025 13:51:18 +0200 Message-ID: <20250627115118.438797-6-juri.lelli@redhat.com> In-Reply-To: <20250627115118.438797-1-juri.lelli@redhat.com> References: <20250627115118.438797-1-juri.lelli@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" dl_rq bandwidth accounting information is crucial for the correct functioning of SCHED_DEADLINE. Add a drgn script for accessing that information at runtime, so that it's easier to check and debug issues related to it. Signed-off-by: Juri Lelli Tested-by: Marcel Ziswiler # nuc & rock5b --- tools/sched/dl_bw_dump.py | 57 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100755 tools/sched/dl_bw_dump.py diff --git a/tools/sched/dl_bw_dump.py b/tools/sched/dl_bw_dump.py new file mode 100755 index 0000000000000..aae4e42b17690 --- /dev/null +++ b/tools/sched/dl_bw_dump.py @@ -0,0 +1,57 @@ +#!/usr/bin/env drgn +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2025 Juri Lelli +# Copyright (C) 2025 Red Hat, Inc. + +desc =3D """ +This is a drgn script to show dl_rq bandwidth accounting information. For = more +info on drgn, visit https://github.com/osandov/drgn. + +Only online CPUs are reported. +""" + +import os +import argparse + +import drgn +from drgn import FaultError +from drgn.helpers.common import * +from drgn.helpers.linux import * + +def print_dl_bws_info(): + + print("Retrieving dl_rq bandwidth accounting information:") + + runqueues =3D prog['runqueues'] + + for cpu_id in for_each_possible_cpu(prog): + try: + rq =3D per_cpu(runqueues, cpu_id) + + if rq.online =3D=3D 0: + continue + + dl_rq =3D rq.dl + + print(f" From CPU: {cpu_id}") + + # Access and print relevant fields from struct dl_rq + print(f" running_bw : {dl_rq.running_bw}") + print(f" this_bw : {dl_rq.this_bw}") + print(f" extra_bw : {dl_rq.extra_bw}") + print(f" max_bw : {dl_rq.max_bw}") + print(f" bw_ratio : {dl_rq.bw_ratio}") + + except drgn.FaultError as fe: + print(f" (CPU {cpu_id}: Fault accessing kernel memory: {fe})") + except AttributeError as ae: + print(f" (CPU {cpu_id}: Missing attribute for root_domain (ke= rnel struct change?): {ae})") + except Exception as e: + print(f" (CPU {cpu_id}: An unexpected error occurred: {e})") + +if __name__ =3D=3D "__main__": + parser =3D argparse.ArgumentParser(description=3Ddesc, + formatter_class=3Dargparse.RawTextHel= pFormatter) + args =3D parser.parse_args() + + print_dl_bws_info() --=20 2.49.0