From nobody Mon Sep 28 06:37:32 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020133.outbound.protection.outlook.com [52.101.195.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99689481650 for ; Tue, 25 Aug 2026 14:14:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.133 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667261; cv=fail; b=ULT7xT4nTcNou3iIqV2LuS7O1qDCIpQussFwVAQdkov/m60wAAAE5T0BL9qpbLlnQWLDoS96kioqvw7zYO7L8jJYLCXKfCTcad6Cdq2EO7e+hcqSLlrVNrwgNHppBdyKORG3TpiRTyCv0NIKK3QzMHkEI/lvMehTeWuJQpiFLP4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667261; c=relaxed/simple; bh=/04n7Rklv3vsBtw9V0KImzvkzUNR/ihHFstg0xQn+ws=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=NZV765Y5KsBUuKqjn9MECtXUanViXmEfDrQVLustf3IiaoozW/TKLQQG8DWFL9Ww/tYOzR4rJaRUbVxG8SN2Vxne4XXHi2h/td5U+3bE3vEpqN82TTExhZs+5GwJ2Q58vk6g4QyWx/LHucv5IeC48Gu3NDBQM8Z3pwxF5mwAxJI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=b+24q8x4cuNx/UXn28oBW0lt+i5f4Y6rpyRrRB2iAFY2Qfs4PkQS1wJnuMVJAKNCuKWlQm6GHgxsWlsECpfyNijffU8nyzmdUoHsu9xx532JJSlBURLjMoyNBcGheGLDckeF81Z9/EGzgKEEbq3/0RYVVnJAJGFdtn6gu8BhQQI1T3hwkdqjmp7f/YnoNjCfFwDzYY1GyXVNE7rj9luTTW13D4Ofce9gS/nNCPSUZNXCWFHAj+hQS5rAInnT8MneaEvsrAjowGj45f3ZzdrGuBl0Zd7i47wExa3tVgfRLd197awkMqI7FmPM9cgDmwXlvIqLv1yN3yds+G5T2a5pgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=17JQVpo/+BsVuU+LUtteUok96dPjyPiTZoer+3IVwiU=; b=s5vqFDSlg7qg6+JYnC3CJTLeNQQpX+aW7qLyY9x6VlBmqsRkwPlb3JsjLS3DdcKnvjIbz1473LErybyEwbCGCw+8H3OiTM8kM+QfaOZIlytB81WhmgQkForsejH7molmyYQ5mGMhNcAkoFuUIP5KMwquAf2KkKj/UEN77j3DUQr1cWEv2uL9qbmf2K0QqtCIIypG2GXguCvA60T0nkIp6wPsDaSPXYBIPR846+rHXnbLPj263InGly1Ff/ad5PY1ajCJKcuc7RmWHNGApjKZ+rE6WQPBWr/jGoJaC0ZVDwdu48CFewtayhipH3s5vYFg23sXadWDtxhpaQ4b6iYq+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) by CWYP123MB8971.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:286::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.6; Tue, 25 Aug 2026 14:14:16 +0000 Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230]) by CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230%4]) with mapi id 15.21.0339.012; Tue, 25 Aug 2026 14:14:16 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v5 1/6] sched: Annotate rq->rd with __rcu and update lockless readers Date: Tue, 25 Aug 2026 10:14:08 -0400 Message-ID: <20260825141413.868997-2-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825141413.868997-1-atomlin@atomlin.com> References: <20260825141413.868997-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0200.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:318::15) To CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLP123MB6607:EE_|CWYP123MB8971:EE_ X-MS-Office365-Filtering-Correlation-Id: 1d2db888-0a6f-457c-8702-08df02b32579 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|7416014|376014|366016|1800799024|10067099003|22082099003|18002099003|56012099006; X-Microsoft-Antispam-Message-Info: 8EJyyLHYbceE3jTVCZMvTrH9BsKBqK5SuyHt5plHmtQr5nVVn50ehJfkxH6a1a9bi1yo205n0gZT6yExuK/CLhS3vucRb3Z4LncHt/1PJVWlBcYmd7pgvsK24WygW8GahfnYwba2cBIykvCaJKkEQFn4VadzH0463zdg6LopDHKoQx0HFvFrZPZ63evav9Uj+GI4JIy4t6JxEtz66z227jXLmGGWochPqhI8rJPuzXi+FgtaIZZFf8RqALkDNVxCXFB/qAILDJJJGv5lEzGjPw1KORtbYCjGgoP2qXmyhceV4GyvVGE8vq/6/09GB5ZyQDALaGd+WOqxL9jNjkvVUoKbtP+6pwFegpJPZetlewwx3GpJu+0CIzGMA9tzAIOjboY7ZeWrDAWyKZMuRC8fJBPOzoIsJWJdUeOQHYp6QeGX2Kl1mrMNFn21h/r92Odb2kMyiWYJxpCrS650JGhfxdcuUTJscC+V1HaLxkwvBLL8yjGS1NsFCcQ6UwTTp7l3aMrj+YSxMC2kC65yHdMy7Of56FXXsvYVoXuWWYUMt21Z8o7xeXaZnrznJyuZNE/YndYjC1XFFTSGRrCXo3XLs1IUPQIn5+9xrA4a6+GZBWBDW8vBjELluSZuG0kf0AW1kgSdHp3VlsPTQ8hMNIleFNSdhWr8ANxsADyuos4bC44= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(7416014)(376014)(366016)(1800799024)(10067099003)(22082099003)(18002099003)(56012099006);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?5mVHKRVp0SllOcTPikWFSp/Xr/LVPKPRlEZSeYGV7yDB67pqaQX9AAlTeObb?= =?us-ascii?Q?cXHqDqXrLTFNRg8KcyQ+Fqv1s9qOKNSzZtTBP0F5wPllbolRaDeYr5Dmb2K1?= =?us-ascii?Q?ua1Rfo1OHeXdKFRaKNYucglViTQnPQHXBmA1OCf+OBbY8TzIN9Vn1nZUax3Q?= =?us-ascii?Q?kasNLF2jxbNlKaRIpufldLLDa+LylYzJjnNWhcKNC8F3yBUj6lSMwBE2XsCy?= =?us-ascii?Q?gyZ074g919Nyei8mR+nBM2yYjiyWWYg/EaJwbP4GWGKi5mFW+zeFFjqMkbr2?= =?us-ascii?Q?Ayq5YK00U1Kb3yIe1XXr29857BWaoNahgbhuBVpsNualaHop8n6KaJX093Ue?= =?us-ascii?Q?oWZPDPFvia96tgxeEjZPZxhG34srJDWseVtg935R6hcu1T/dtRn8YYeVwM4m?= =?us-ascii?Q?D+ndEdganBh09C3B53U0TBTUFvGUAiDleAhysIylh280BphsCQWBRRO8pRVF?= =?us-ascii?Q?cFsdWZoXCHbjc4Kr2STSERE/tWRiWgF6YBHSbmFSSSXOmMcRd+XSSO251ltP?= =?us-ascii?Q?X5raVvZEs+y+fZKUbryMKsTSQIjShpvYEnkrxAd/MNzT/1KL4HZGi4D4YWw0?= =?us-ascii?Q?zCDwfFK9Lux5asOyEKnFx8N/Y21hCTV53qNOihbAVPYFE5Jxmr/D+1gsd/rD?= =?us-ascii?Q?F46e6GOm4aKwFNyLLbKdIHqPnSWwAcOz7YQ83o1xGkdWGmvuBhpqjn05beJw?= =?us-ascii?Q?KpLckA9Dt0I0RucSbcSGSuAeb6qdW8U5gHj0MOVigzT9LZ7Urvn6nH851HZi?= =?us-ascii?Q?pfiN4rSWCLW0W1rFyvKFc95fhYbSiXawZOhZOw0Sb/PNkGumMBKd77JGFqK+?= =?us-ascii?Q?aEz0e1dao6gjxpN57v0xcj3hTXNNjXgL2uhBMb/lBzFAa0/418nyqm/PgSn5?= =?us-ascii?Q?lv5MHhuJBdbEutePQeMy9lZxW1CEnvsQ0ljZW8s63x0fJrkAPcxC3lL51URZ?= =?us-ascii?Q?o67MPeJ3cbFG2E/RRSWsRy90ysgV0BT5LT2n8BA87TgJT3sx7nbbOrF1cYJG?= =?us-ascii?Q?aJ6n5JsuuObliwYHJX5gNSmCnKxY6NzjXcch/ZI5lOYHwoVWk67VkmqftOM2?= =?us-ascii?Q?fMm4mkNXsx7tMs83uFF+Zwn7cNRKFAIzbNr3ULY2Q1gjn//Rr01HiKX/2qu9?= =?us-ascii?Q?gntAAgmS2LqdpPgjRR4t/UqESNILboDWYBLiQl9qLaRJcX7lwnWP0c5O8ttV?= =?us-ascii?Q?Wv2NQfWq65Lv533Yol9mTNoilblZmMusm3OL+imezInR2TSTts+COh1AKMLv?= =?us-ascii?Q?DmrT41Mrjen1qdO1YnnXxlz1x/N5sJAINXsIxzlrPHrmHcHxz5uMPBcAW7cT?= =?us-ascii?Q?/63IwRMM/MeN9w9sa7QkfXwiwWXfCpHm7F39Sye/XZ5DWmQpLLvXHEdOErZd?= =?us-ascii?Q?+PbGAK9cP85pbNtR8nDV8WLNlc/wPeRfRD27yYxmuzr7dVHKtHvG0pXv5yJA?= =?us-ascii?Q?oD8NYFUiIMULoP4isJ/sAmBgxWUOvRYMS4lAR5Q66P2cBIa24seiOfvytdCG?= =?us-ascii?Q?bKFopRj2tID775NkR+1CpY7Qrtu/Q8aiQKWyKam0VTd4+c1cMAPSrjmqgSxO?= =?us-ascii?Q?nOZqqyft0YBYHRpBun0MFgcpU4HZ+1GilFU0HH3zcKcxRsxQviNohjhdhOSq?= =?us-ascii?Q?kfCH9jFg0nJlg6wxx8r6Kf6RZuQ6gCaoVUAHdp9vhqqpC6UoA8DW/YaqfihE?= =?us-ascii?Q?R7Q12u3cXHfLac2VoR+Pjc/8fKrJjdTdebssnVr/fT8Uz4V2?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1d2db888-0a6f-457c-8702-08df02b32579 X-MS-Exchange-CrossTenant-AuthSource: CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 14:14:16.2348 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ci64ateeuZ5QXoGOozcOaGQrWSMcRawSqUGzIbv788o0Uk+gxhGIz8W7Veih1zNNYnlmF0dJwWG7M+cmgRoRDw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CWYP123MB8971 Content-Type: text/plain; charset="utf-8" The root_domain pointer rd field in struct rq is updated dynamically using RCU, and its memory reclamation is deferred via call_rcu() in rq_attach_root(). However, struct rq's rd field was missing the __rcu compiler annotation, and several lockless readers across the scheduler subsystem accessed rq->rd directly without using RCU dereference primitives. Add the __rcu annotation to struct rq's rd field in kernel/sched/sched.h. Update lockless readers across kernel/sched/ to use rcu_dereference(), rcu_dereference_sched() or rcu_access_pointer() appropriately. This ensures proper data-dependency barriers on all architectures, enables Sparse static analysis validation, and documents RCU read-side ownership contracts. Signed-off-by: Aaron Tomlin --- kernel/sched/core.c | 16 ++++++++++------ kernel/sched/deadline.c | 8 ++++---- kernel/sched/fair.c | 29 +++++++++++++++-------------- kernel/sched/sched.h | 2 +- 4 files changed, 30 insertions(+), 25 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 96226707c2f6..3882aa99e2f1 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8580,10 +8580,12 @@ void set_rq_offline(struct rq *rq) static inline void sched_set_rq_online(struct rq *rq, int cpu) { struct rq_flags rf; + struct root_domain *rd; =20 rq_lock_irqsave(rq, &rf); - if (rq->rd) { - BUG_ON(!cpumask_test_cpu(cpu, rq->rd->span)); + rd =3D rcu_dereference_protected(rq->rd, lockdep_is_held(&rq->__lock)); + if (rd) { + BUG_ON(!cpumask_test_cpu(cpu, rd->span)); set_rq_online(rq); } rq_unlock_irqrestore(rq, &rf); @@ -8592,10 +8594,12 @@ static inline void sched_set_rq_online(struct rq *r= q, int cpu) static inline void sched_set_rq_offline(struct rq *rq, int cpu) { struct rq_flags rf; + struct root_domain *rd; =20 rq_lock_irqsave(rq, &rf); - if (rq->rd) { - BUG_ON(!cpumask_test_cpu(cpu, rq->rd->span)); + rd =3D rcu_dereference_protected(rq->rd, lockdep_is_held(&rq->__lock)); + if (rd) { + BUG_ON(!cpumask_test_cpu(cpu, rd->span)); set_rq_offline(rq); } rq_unlock_irqrestore(rq, &rf); @@ -9012,8 +9016,8 @@ void __init sched_init(void) #endif rq->next_class =3D &idle_sched_class; =20 - rq->sd =3D NULL; - rq->rd =3D NULL; + RCU_INIT_POINTER(rq->sd, NULL); + RCU_INIT_POINTER(rq->rd, NULL); rq->cpu_capacity =3D SCHED_CAPACITY_SCALE; rq->balance_callback =3D &balance_push_callback; rq->active_balance =3D 0; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 0f858b98c9aa..8e1e8337aba0 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -122,12 +122,12 @@ static inline struct dl_bw *dl_bw_of(int i) { RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held(), "sched RCU must be held"); - return &cpu_rq(i)->rd->dl_bw; + return &rcu_dereference_sched(cpu_rq(i)->rd)->dl_bw; } =20 static inline int dl_bw_cpus(int i) { - struct root_domain *rd =3D cpu_rq(i)->rd; + struct root_domain *rd =3D rcu_dereference_sched(cpu_rq(i)->rd); =20 RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held(), "sched RCU must be held"); @@ -159,13 +159,13 @@ static inline unsigned long dl_bw_capacity(int i) RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held(), "sched RCU must be held"); =20 - return __dl_bw_capacity(cpu_rq(i)->rd->span); + return __dl_bw_capacity(rcu_dereference_sched(cpu_rq(i)->rd)->span); } } =20 bool dl_bw_visited(int cpu, u64 cookie) { - struct root_domain *rd =3D cpu_rq(cpu)->rd; + struct root_domain *rd =3D rcu_dereference_sched(cpu_rq(cpu)->rd); =20 if (rd->visit_cookie =3D=3D cookie) return true; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d78467ec6ee1..ad367a542eb0 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7739,13 +7739,10 @@ static inline void set_rd_overutilized(struct root_= domain *rd, bool flag) =20 static inline void check_update_overutilized_status(struct rq *rq) { - /* - * overutilized field is used for load balancing decisions only - * if energy aware scheduler is being used - */ + struct root_domain *rd =3D rcu_dereference(rq->rd); =20 - if (!is_rd_overutilized(rq->rd) && cpu_overutilized(rq->cpu)) - set_rd_overutilized(rq->rd, 1); + if (rd && !is_rd_overutilized(rd) && cpu_overutilized(rq->cpu)) + set_rd_overutilized(rd, 1); } =20 /* Runqueue only has SCHED_IDLE tasks enqueued */ @@ -9358,7 +9355,7 @@ static int find_energy_efficient_cpu(struct task_stru= ct *p, int prev_cpu) unsigned long prev_delta =3D ULONG_MAX, best_delta =3D ULONG_MAX; unsigned long p_util_min =3D uclamp_is_used() ? uclamp_eff_value(p, UCLAM= P_MIN) : 0; unsigned long p_util_max =3D uclamp_is_used() ? uclamp_eff_value(p, UCLAM= P_MAX) : 1024; - struct root_domain *rd =3D this_rq()->rd; + struct root_domain *rd =3D rcu_dereference(this_rq()->rd); int cpu, best_energy_cpu, target =3D -1; int prev_fits =3D -1, best_fits =3D -1; unsigned long best_actual_cap =3D 0; @@ -9562,7 +9559,7 @@ select_task_rq_fair(struct task_struct *p, int prev_c= pu, int wake_flags) cpumask_test_cpu(cpu, p->cpus_ptr)) return cpu; =20 - if (!is_rd_overutilized(this_rq()->rd)) { + if (!is_rd_overutilized(rcu_dereference(this_rq()->rd))) { new_cpu =3D find_energy_efficient_cpu(p, prev_cpu); if (new_cpu >=3D 0) return new_cpu; @@ -12554,13 +12551,15 @@ static inline void update_sd_lb_stats(struct lb_e= nv *env, struct sd_lb_stats *sd env->fbq_type =3D fbq_classify_group(&sds->busiest_stat); =20 if (!env->sd->parent) { + struct root_domain *rd =3D rcu_dereference(env->dst_rq->rd); + /* update overload indicator if we are at root domain */ - set_rd_overloaded(env->dst_rq->rd, sg_overloaded); + set_rd_overloaded(rd, sg_overloaded); =20 /* Update over-utilization (tipping point, U >=3D 0) indicator */ - set_rd_overutilized(env->dst_rq->rd, sg_overutilized); + set_rd_overutilized(rd, sg_overutilized); } else if (sg_overutilized) { - set_rd_overutilized(env->dst_rq->rd, sg_overutilized); + set_rd_overutilized(rcu_dereference(env->dst_rq->rd), sg_overutilized); } =20 update_idle_cpu_scan(env, sum_util); @@ -12806,8 +12805,10 @@ static struct sched_group *sched_balance_find_src_= group(struct lb_env *env) if (busiest->group_type =3D=3D group_misfit_task) goto force_balance; =20 - if (!is_rd_overutilized(env->dst_rq->rd) && - rcu_dereference_all(env->dst_rq->rd->pd)) + struct root_domain *rd =3D rcu_dereference(env->dst_rq->rd); + + if (rd && !is_rd_overutilized(rd) && + rcu_dereference_all(rd->pd)) goto out_balanced; =20 /* ASYM feature bypasses nice load balance check */ @@ -14386,7 +14387,7 @@ static int sched_balance_newidle(struct rq *this_rq= , struct rq_flags *rf) if (!sd) goto out; =20 - if (!get_rd_overloaded(this_rq->rd) || + if (!get_rd_overloaded(rcu_dereference(this_rq->rd)) || this_rq->avg_idle < sd->max_newidle_lb_cost) { =20 update_next_balance(sd, &next_balance); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 56acf502ba26..aca352e2f4a8 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1253,7 +1253,7 @@ struct rq { int membarrier_state; #endif =20 - struct root_domain *rd; + struct root_domain __rcu *rd; struct sched_domain __rcu *sd; =20 struct balance_callback *balance_callback; --=20 2.55.0 From nobody Mon Sep 28 06:37:32 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020133.outbound.protection.outlook.com [52.101.195.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB41D481FCE for ; Tue, 25 Aug 2026 14:14:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.133 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667265; cv=fail; b=Adqi1uh7cKZNw+k8R8HOUQeXrKsh5sBoNzYc0zjNOUeOqvV6kcU78NVeoMhcjQ8GX1jIZ74kro1q6Ge7CmDe1Pt6AIrafdHwwNivodf0e7PBRkrnj2tWarlpuj7FAPwaj3V3+cfjGoNE74dGToYHQ9ZR9zZr2rr3ehni8Zs/b30= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667265; c=relaxed/simple; bh=YOXT5xhmbHbCiG1dJsjKr8ALDa9U1IHzsmyHP7vVVio=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=JfsA6msdzUc1k1zID6DDqGGOK63qgT4gUcTPxgsqtujWX/OZRQNAvy3XI5hkaJt97YgqcesJzWmJA1uA3S+Bgbyf64q6W9hXgFQO/4atcPJX02DNBgAXVpLwU/wSafmPu8HJgsfkSo99j4ePNOzPgReiJASVWMoEQCjr+uhJ7KU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=O37NIVGQfJzhlhnL6SavN0ZtsFISKRAiCwpQ7nbO1yhp1xgXwQ/HD//ZgKgiT+yM8mOfx5Yilkyfnu0hY7jB7pZni9oJwKYjmOdl3VKDcHG1mHMKIT1XQrlvdwPkC7Nwooyp3N2VJvJ4Ankkr7xXgiaTl7uJ4sAtYkruNRD7TOjVVVdEK/NSVlzThMEAd07MOQbluUJC442HAw0nTSrzg7W6qXBScuuRzfFoKSG0RXB99FQq1jmhLy5z8mswQlDQdiPG0LrYPPB/6o20w6ReYz71bh5ASE2YfIg8v8xkS5stxFf2I15Kpb8d1fB+Jju6icFX440NoXQUZy/zS0QY/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=dAB+NFGSr/yzNokZfAlX8DQyn0cs8RAEgEigIJeSk9k=; b=WWaBWpadAn3MG4X/9Mt/l4t2ZdI1VH/+GxRXpeeFxC0j67gOLPOyeFKNhBWTuKjtZ/wC/WRxj48flgm/77UA+8ROgabjPAXq8fl3M2QZDONaLetHSvZ0Krpc71spJfks6MkXNuLHH3u9vDmipiyaXna/PnIUT0lTIYgCZ14BZzGhVLFJkQwK8/VtZlSpb5fKQCA4m1VEQaWitnMzsESSnURnjfiYWTfR7zv9qba5FaUhcaUFp6npMXgl1PloNUjGW58oHB/mTnJBhZRF047mTmy+QY7lbqv68WWUkZ+LNKr+Mp9QnMlXNZcU8TwOpnajGolSZ1062CGUCWobtLrP4g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) by CWYP123MB8971.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:286::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.6; Tue, 25 Aug 2026 14:14:17 +0000 Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230]) by CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230%4]) with mapi id 15.21.0339.012; Tue, 25 Aug 2026 14:14:17 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v5 2/6] sched/debug: Protect lockless rq->rd access in print_dl_rq() Date: Tue, 25 Aug 2026 10:14:09 -0400 Message-ID: <20260825141413.868997-3-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825141413.868997-1-atomlin@atomlin.com> References: <20260825141413.868997-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0428.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18b::19) To CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLP123MB6607:EE_|CWYP123MB8971:EE_ X-MS-Office365-Filtering-Correlation-Id: 7127c7f4-d544-4998-af06-08df02b3264a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|7416014|376014|366016|1800799024|10067099003|6133799003|22082099003|18002099003|56012099006; X-Microsoft-Antispam-Message-Info: 1l6F+ixZ+X5Dbu6A67k9jg2wvDmmw9qGNzzLexIq9zcdkBScgf7zzBstdo/AtRrtlD/grg1MXKQlvZbNVGEmB92xFWCbCURoP/ezS4ix36X0j6VqMkYHwia3PRVWtqN3DYdKDaV6ELgyjBLSEsSxI1RgkL3JwUt7u/pxVrb8RapyWeMBUSoGdx+S5/I7B16LB2QpomwL7sqvPEubhbRgw2TvJCnx8TnDLbeEOqv9taWrTmV3rIRMdiOzpGsiKkn/Aq6kM6EDfZKCTv/mOp/bDiK+HLNxc1G68kvdkYerHBkt9tjfI0qu5EB4A6zIc92k8hzmtZSsNPSG7cB4NB9DRJuhsYJkQ1Ta50QGDWqqQHcrhVxwrnUTHNUTxWCY0LOPDNUGrZb9EYguFC68117LAxEzQg7jpc3oBGBbjZUsBMhSFduODKXuNSzZJ43y4Hfn649sTbol99Q64jBTv8mswENHcofxB/4RSqMTInhE8PUUCFQ8TRio7CV2l/6Vp5LRjnCcAXYKiqCx7XGiIn1eNPd2G0SwcVA7vAm8uI5y5tnuttjPxzeq1j4pOiHCo827DjsZhI0ifMXTsLQSY4XkewyPw1zPCuuSoOSm1Jcd4ZFfu6lXhvshaWcCWJljJHLTCsPBgtTa4Vypw3nmqTbpMVPDwpCTj3SzVeci3mxai8o= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(7416014)(376014)(366016)(1800799024)(10067099003)(6133799003)(22082099003)(18002099003)(56012099006);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?4ZcJF7CVKFtRByoy8SVofYe426OPS9JV/N3J8SZeAEmJzv0w/+FdItEA05pO?= =?us-ascii?Q?U3Mn5XY1xK8xG6ykyfNcsq0jh2fXjFz1sij1nNUTTd26yjbGAq4YV8WNu9+W?= =?us-ascii?Q?MptKQGafiB2HFOj5TzB2wWFyQi9rk0XYM60CRnLKx6Ut8tZOv12Q5na18BrS?= =?us-ascii?Q?r4OBlSZUldzJ8GuTD75cR/4UZB1/X4cT5W9f054J9qepQ0RBu55GzZwN2cUr?= =?us-ascii?Q?0mdoMNUdZsmkOHqrSWZWDOpGDqFJblJMaJicvmMG+oeCSFc5nTrroAV6t6GG?= =?us-ascii?Q?mburZPFBTfOFa/B8UPlpPx1XJXEOJJp6kMket3DTzjvrM6+MEoL+d55am0dd?= =?us-ascii?Q?1Z4FbZy2cVOrE34MuK9sETaRU8anY1aHwbPp1zZ+K5Um4evKP8hp9sbTvfXJ?= =?us-ascii?Q?GDuV//3qW41kJHFx2ey5eGDccPuLJ2RQcxYcZdCtonsapvVhKSCkGnBc9JtI?= =?us-ascii?Q?LQJbz78s482G+b8PbkaeuYx/kwO/eEKurLZwGCsolTYRXkhXcu/CTblny1jy?= =?us-ascii?Q?6TDiFK60DeE+TOzZ0ONaIk9r1PPHD0/rmTK3DbU0os12jAnh8H2CQkeJhKFF?= =?us-ascii?Q?1w+S6gkUgyX6FcLkrquJY9PM6I/94TJsdRbmcd7ThjJJcptVU2JtsHPAwelV?= =?us-ascii?Q?F8NuYXrNhC7+U1HEZHKWngB1ckmNpwJN5Now17xSx1VfXS7RIAn48qqzc7i/?= =?us-ascii?Q?7Nw+oq0bFausvNJo8NCtuL6uKVYzEX0xBfkPy2GFPb4wWSL7zU40zifJOLSK?= =?us-ascii?Q?i4TXxeI7nqrfp6z3M6LibcpoUcH0yzPrsARI05jYB/1fjMrTGFGyr4wPYIhP?= =?us-ascii?Q?x349O6pjThfxzzCAfm68SRGjkWH8eVgR9DB4RE4BbTxSGhGGnQZ7xAZkjPPR?= =?us-ascii?Q?eCzaFH+HDxEb26QLfXHsTRMceq5aQQqYe9duuWflXjDybrmaQGdEagsmngTB?= =?us-ascii?Q?RkLtZK72YvXlBresUMZY9Qft90YHVZybiM+woqQkCiJL5lQSbfJNdCvVr8Er?= =?us-ascii?Q?nP/bg9nnkExmtUu6tfGCTJmIK/hfuwm2rOQSiQhzef6qJLTa1tJQfaf1Z3Fc?= =?us-ascii?Q?1VjupJrD47YGMO0e1C8AIn2LkV8uHR+SY8fe+yQYhVcvvBE5bfJLyqa+81u2?= =?us-ascii?Q?sMK3jW9ChrY/MpZCe6DVUuQ4jVzZm+QWeo41xoaYM4eWR2/0idiPZbsgMPcJ?= =?us-ascii?Q?GQjc7iyIGLB3Lux5sDmpkC8EvYyoKusIAkzawUdKddq9SLTgGW5TfViWO7N+?= =?us-ascii?Q?gxTmZ0D9F1SHWcu3tSfAiiXT4BswrqmSO+LjouhrNFv9HSFzrxLX7n9+tIAH?= =?us-ascii?Q?NBfOxo7wUfwIEzK3vT07qe75hqj8iSRonxyxW+0iQ4pU4+DyU6DYDDBZyto9?= =?us-ascii?Q?D0j4B+se/r43SV/SjFEp9hC2/nOl7LtrhmXZKpDzYyJIJAhtabr3dtyVAa+p?= =?us-ascii?Q?3iKBnCllQ1aWL5kzRf7aobngeyIXaTi+p0+H3FTSsBZTHYFRfh4RAcKy1zyj?= =?us-ascii?Q?LZ1wRIsDRmT1UQpxQ1ULWeXmZKbJdTOwgR9XMY0b1NX37WMktwl6Fi6euUOJ?= =?us-ascii?Q?2vV4n6yx3hM/6pwl3bKXs5Y7ajksz44HFS7bJ1EtTpYb1U1HjZM9ri2lpnH+?= =?us-ascii?Q?YZwrlTn1DfuKfeDrgI9NHG8UeLITtWyONaTIAHhIVyAETY2bISeFR4F9f1Co?= =?us-ascii?Q?MfWHi/kn7WCE7zsHxlohPXGfRddg6jS6kWikp0iNK4ubcQb9?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7127c7f4-d544-4998-af06-08df02b3264a X-MS-Exchange-CrossTenant-AuthSource: CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 14:14:17.6021 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qpOcuf/vToodQz9cCVWe11hFunNCTLTGHPdLIQHLq+nLZsPy3+wGyCLa4oAwcZpF0KHTylSw5xmSUrmXKMZ/9g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CWYP123MB8971 Content-Type: text/plain; charset="utf-8" In print_dl_rq(), cpu_rq(cpu)->rd is dereferenced locklessly to display deadline bandwidth statistics. During CPU hot-unplug or cgroup cpuset repartitioning events, partition_sched_domains() calls cpu_attach_domain(), which executes rq_attach_root() to detach the CPU from its root_domain. When the reference count of the detached root_domain drops to zero, rq_attach_root() calls call_rcu(&old_rd->rcu, free_rootdomain) to schedule memory teardown after an RCU grace period. However, rq_attach_root() previously updated rq->rd using a plain C store without an RCU publication barrier (i.e., rcu_assign_pointer()). Without a release memory barrier on the writer side, CPU or compiler reordering could allow the new rq->rd pointer store to become visible to other CPUs before the initialization writes to rd->dl_bw are committed. Furthermore, because print_dl_rq() did not hold an RCU read lock while dereferencing cpu_rq(cpu)->rd, an RCU grace period could elapse concurrently while debugfs is reading the file, allowing free_rootdomain() to execute kfree(old_rd) and causing a use-after-free race condition when print_dl_rq() reads dl_bw->bw. Resolve this by using rcu_assign_pointer(rq->rd, rd) in rq_attach_root() to guarantee a release memory barrier when publishing a root_domain. Finally, fetch rq->rd using guard(rcu)() and rcu_dereference() in print_dl_= rq(). Fixes: 02968ccf7b80 ("sched: add /proc/sched_debug file") Reported-by: sashiko-bot Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 3 ++- kernel/sched/topology.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 40584b27ea0c..632d5fe9e470 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -1089,7 +1089,8 @@ void print_dl_rq(struct seq_file *m, int cpu, struct = dl_rq *dl_rq) SEQ_printf(m, " .%-30s: %lu\n", #x, (unsigned long)(dl_rq->x)) =20 PU(dl_nr_running); - dl_bw =3D &cpu_rq(cpu)->rd->dl_bw; + guard(rcu)(); + dl_bw =3D &rcu_dereference(cpu_rq(cpu)->rd)->dl_bw; SEQ_printf(m, " .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw); SEQ_printf(m, " .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw); =20 diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 622e2e01974c..b411cc00029c 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -496,7 +496,7 @@ void rq_attach_root(struct rq *rq, struct root_domain *= rd) } =20 atomic_inc(&rd->refcount); - rq->rd =3D rd; + rcu_assign_pointer(rq->rd, rd); =20 cpumask_set_cpu(rq->cpu, rd->span); if (cpumask_test_cpu(rq->cpu, cpu_active_mask)) --=20 2.55.0 From nobody Mon Sep 28 06:37:32 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020133.outbound.protection.outlook.com [52.101.195.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 401FA48383C for ; Tue, 25 Aug 2026 14:14:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.133 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667267; cv=fail; b=E/erBjsTY7OoWVOzpV56sDf06+i8Ah32K3J1PTVUCoTVYoDKHy++exMDX2vDn9687uhOkwj/hDAxJPoKgEDL98B7Zr1ucX44C7ECUobHAXFDAJ+pbASne2UunAFFhVWO4UFEfJ6TdtgZpb96I/suJqzBo37Y9O74VBfZcZFbSAo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667267; c=relaxed/simple; bh=wiGXR6q2zWq4AcnibKRhSbwHra0/Lq1G5nPxxEhSXZg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=OAVSiNv0wwow4xnuX7KClDl+VqAqiG0VUUZcsngAbG23uGmHb/fg9iDiyqkR2xUUPM6STZ3fwlIDEDRkvr3XxGnmUzl4RA0Q1kBjZODMZygtXZmkf3CrETp8e2vhhGZe/MvL0l6DWB+7lxFdjWUqDiyHiNqRt5YgtRD4oqJ6U64= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ExQmStRXmkep42pijBlrilxZ19ZNr4aGAmRTgG2XycAdrd0HAfknjFslinkWyLIPCiJtXK6hdmARroKWDCK2LV8nW7+R2NOdt4eEhM/7Tb/NLz4JRtNaSKfu2u9f7LYXL3x+ZM/L7M0yHeM5xxJ9S4mpRhShDXH2RU7vLq3xhdMu41magcba9lX6fIyZpwh9PvUAfWfsf57sJyT/BuBvkLVnDsU6HYBGIURCkeqMX1/yFQn5nPnrdanVeqcEvgf3SC+7gxM094ZSZGVvLTdOinHaw5ab96xxaiwKY1Wgysl9l4G+GhwlIEHdMpOL0r51zBwWg8BMNzGQZH13CFM3Cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=GKF/J09G2GKf9X8pBqg3rP2feH1dN6DIG/dK3UbQ8d0=; b=OyjT8BYNdNGmGHtrnXN9hDWs86q64PRs23tTA0ENdHs5Uvlt1tKMiAvq+UekZY+F9Eh8KF/DUmf59c7tafGpa0daNhPvPUS8fI4iVmzEdyJYvAW90zxj1UNjzhB4x6LJH0EoKe2h9OAfHjzJ6rcFudqnPEjzPcio3+WO7zF61lIva5HrYgUaekVD1iC/fhEyLTftihXaPlxlo01AJexhZjvtBCGMlPhQiSNuNSAO70DraKWzdO/inzM4qGp1fnyLfPeQyjKZXsd4XYhQhe5YuoYT1xm8Y4s5UMgb8LH5/kLQqoR4QF6+hHZ6Gm4ZvR2ole41jtp1qUamKGR/8Q4x0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) by CWYP123MB8971.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:286::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.6; Tue, 25 Aug 2026 14:14:19 +0000 Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230]) by CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230%4]) with mapi id 15.21.0339.012; Tue, 25 Aug 2026 14:14:19 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v5 3/6] sched/debug: Protect lockless rq->curr access in print_cpu() Date: Tue, 25 Aug 2026 10:14:10 -0400 Message-ID: <20260825141413.868997-4-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825141413.868997-1-atomlin@atomlin.com> References: <20260825141413.868997-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0491.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1ab::10) To CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLP123MB6607:EE_|CWYP123MB8971:EE_ X-MS-Office365-Filtering-Correlation-Id: b5202807-0edb-4dd0-adf8-08df02b32730 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|7416014|376014|366016|1800799024|10067099003|22082099003|18002099003|56012099006; X-Microsoft-Antispam-Message-Info: Rp4vxtV6Us3zzAXCLkJrbn24u6+zMin/1Pt/hn9fJ+2AsXJ/WQKDJ81SjVOuXRbDm8TbrYog7WhXjZckF+7fCaur0bs448x+Y7mnaG/wHJNhTY4FohWcAnt5WMbAvU6q3eJOOphxz6EZnVWE+PhyZbnzVqqkxusd5nZwRifgOgsIe7jHDT0b1HhhqFqdojC3v8CL0MwWnMkUbTjjZf7SNDZBttsdEeDsSWTHr4tJ0wUfQLnl4RDLpaYT9VT/CoQhKkEPFXGjKwFZuCTsw5dpFTRqXM7iwwnV+DQVL5DCWc7JOjbsRIrC6DK7ts88m6G9AcCsGv9Z3kXLB05ZTAM13xtgywv0CCSz+uh0/O18U82u5phCWEKKzPbOgcFBYrf9NqqHJlwBu0UkMOQJc+xbVSaUDSX3N24n2DEYI2R1A0ZNP9tCG6U3mo7F7EcP4P+a7EanxQDHesuS5hTjd7fAI09DejrTLX8FtjxkQHI9li/8COGpPvGBbPDZVjCDod7VJ931bI1y76fNUyjJDg2YaEv36nDnkHLODH8aTA2W5RQ3WsslqeSAvSHAlVCb8X3yaVYwd+pQWoAkWHdywgpEPX4FubFLgWOtIf5qUlxQhwgYFhbqUFCTTR6JtgNUou3aeZGVXXHc0aJ2a2WgX5s9pznS9cAbpM6NBJSFU98dV8o= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(7416014)(376014)(366016)(1800799024)(10067099003)(22082099003)(18002099003)(56012099006);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?zaTqcjunN1NPnLf6t0Bc7zxU+0wdFR2+KCypVGOYJKHNVfWeSFSfFl4J5p95?= =?us-ascii?Q?ZpVgDCuw7hhCsiLvRa2cMLG0jdVdHBWkQhb57yoNV8DVjZ46VV+apqPV0tLu?= =?us-ascii?Q?nBOdZEn7vN6npjyctkSMXITdzyF4YPzwlvnLGqOhYcdN/U4fboTCPXhgwTYW?= =?us-ascii?Q?6PHIj8HMBd09euB/118i82wgDhcbucx/xyedd9tD6L5t4fFoGdUgSiS0fd4X?= =?us-ascii?Q?xDwJ5nOwkSonE0cDCDGU0VQ7BRpEMA1xTCS65ZCmNAHPwCqpb/+sdNvS7P/d?= =?us-ascii?Q?hJ0V1pT9MXl50teFF4GfXfJ6IoaHMWRkb55jUSZC/62C5n9PfIC+JFs/vyR8?= =?us-ascii?Q?YbK+7ABeYOj5gTd7+5MEkXCHBrM19YhK7u5YkEPyiDQcdTAxNdAfCdzdVaza?= =?us-ascii?Q?E8WgAIDy7hpwYTfi15ryKEKaVf/iTy/OESDSYjV8tntH0ZlQ0pbDSMZXOC2H?= =?us-ascii?Q?HLxxRoXhQf7xs3mSU3m8LKTUX/2hGk5V/R8K75pZ6aHC/WIh932+DZ3lhhLP?= =?us-ascii?Q?VLS0k8siA13D0Bect1AdH1QdGg2xIO/Cou4s3BDPiNe/8M9PFcsSzZakVUlr?= =?us-ascii?Q?AKUvER7ssq48naNv0b0B4DRaN0iLvQ02kJxMxX9DidyMkaPEcXmAF/v1rSfa?= =?us-ascii?Q?sriUmVLDlQidF9NQa9T0JYQP0DsnL9Ki/VoDnGoz7G+bEXeuqUVWS9CN0/Qq?= =?us-ascii?Q?cl5g2yj/KrD79FjMDEYlMldFU109yTwjo9PSggjdZL6p9PJQEd5YujLs3PWp?= =?us-ascii?Q?xsj5YcoFtcaspsqWUeHBFEXAO42qw6BAhuQMPyVJtgnUJanNcnvjWMJg+J/t?= =?us-ascii?Q?GVAiiSgur8sZLxUKbtN7zSGe8NF2CTlyDTDw07CwX8eQ1fe8k355qCb7v2jy?= =?us-ascii?Q?8kJ1L5pKo4GMXT/QCNUMAEhebVh6C2Fwn40WwnD2+jYy7VTHUF+lB6dcPjiB?= =?us-ascii?Q?zkj6qMaVCuwDbiA+BKLb7anPGjtOKrOaFwnEKoxELwBYQ7cnYFZ6hVe4DTRi?= =?us-ascii?Q?KFlZkc1SEvkU5GaLkNoFuCFSEeHGNhlqCP//5yq3Qob2lEE1v/aQMva2gnGC?= =?us-ascii?Q?5DHAhPqqMMpc3TJ0pbAFvbgZ+yk9+rb+gSB/pEldWSusJU1fGNzNry5a6yLd?= =?us-ascii?Q?zFY2d19lV+Djb3l4NZuyNuqYGkUq60JxhxmaylzUdXHPYm4JF3LElDH95eM3?= =?us-ascii?Q?B+A2VIVz9tjcbneyEUNnedVG5UAqRnk/sasm/MatH8Nao4eqW8niO6kX6Vff?= =?us-ascii?Q?vLmdArcDVln6bhKEhwL3HmmrzYXtcDX+9lcvCytSJltMDt9U1CyagEoVdzyv?= =?us-ascii?Q?LvVXTwcZ6sQE8ODZzHWP+lZVLTOw1jUnefP5Byam7189Sn2opQ21MgxjjfAb?= =?us-ascii?Q?ADpSOh1vpQ0Jo1t2UmCy7fqxVm6IN5TIB2KFSgPvqmHFOIOHZ43kOcCYPnPq?= =?us-ascii?Q?CyJkHCqxpgHYTPo5L343AGiLtLhwqbBGLJCYfItsjOxEQ7LpRb0dxnD7Bv53?= =?us-ascii?Q?u4eRVQivo8RQ2lSmz/i4ARjv0VorafZu+MJei+QgrT6Qo9VRV5ue2tVTuKif?= =?us-ascii?Q?/s910XZ58Xued3F5zZ18+G4F+Vu3flogU/KFfg70pHWvWoqWFgNBqcSI92CF?= =?us-ascii?Q?YcF096QLXWjJkP8qKAXxoKpOHmCRDbeX5d/Hngau5Em7BEToRkEuoo5jHw3w?= =?us-ascii?Q?b0RLf9WZJIy8Fi9Y+AMVO+da5D7UHhhnyJdSklcP+qnDmXV6?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: b5202807-0edb-4dd0-adf8-08df02b32730 X-MS-Exchange-CrossTenant-AuthSource: CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 14:14:19.1579 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tTRHMongfNcbpoLBMU8RouB+DFun2KdrfAkYfowSu1qvNsXB7tFXy5NozjgsNcgkVu1/slLQNYnDc+Cp+WfkJw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CWYP123MB8971 Content-Type: text/plain; charset="utf-8" In print_cpu(), rq->curr is dereferenced locklessly to print the current task's PID via task_pid_nr(rq->curr). While accessing /sys/kernel/debug/sched/debug is inherently best-effort only; rq->curr is indeed expected to change dynamically while print_cpu() is executing. However, if the task currently running on the CPU exits concurrently and its reference count drops to zero, put_task_struct() calls call_rcu() to schedule __put_task_struct_rcu_cb(). Because print_cpu() does not hold the RCU read lock while dereferencing rq->curr, an RCU grace period can complete concurrently and free the task structure via free_task(), creating a potential use-after-free race condition. Resolve this by reading rq->curr using rcu_dereference(rq->curr) inside an RCU read-side critical section. Holding the RCU read lock delays the invocation of __put_task_struct_rcu_cb() until after rcu_read_unlock(), ensuring that the struct task_struct memory remains valid while being accessed. Fixes: 02968ccf7b80 ("sched: add /proc/sched_debug file") Reported-by: sashiko-bot Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 632d5fe9e470..27e0840ba9be 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -1127,7 +1127,10 @@ do { \ P(nr_switches); P(nr_uninterruptible); PN(next_balance); - SEQ_printf(m, " .%-30s: %ld\n", "curr->pid", (long)(task_pid_nr(rq->curr= ))); + rcu_read_lock(); + SEQ_printf(m, " .%-30s: %ld\n", "curr->pid", + (long)(task_pid_nr(rcu_dereference(rq->curr)))); + rcu_read_unlock(); PN(clock); PN(clock_task); #undef P --=20 2.55.0 From nobody Mon Sep 28 06:37:32 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020133.outbound.protection.outlook.com [52.101.195.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECB0848165C for ; Tue, 25 Aug 2026 14:14:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.133 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667269; cv=fail; b=DO0ilP0CF9a73kwKKkWg3di6jQJ3dVR+dULVO4Mjrw7HKXwmO0w3uMZ2qmixtXolbn6nZmHR/7Ti/C1hrxqejkQlQov16aLH6fwDsV02k9lvK428BYZs4ekKjq1dd+DgRdlrulJTLULv2A+M96/x8am5ye5O7caRZ05XOeDxTag= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667269; c=relaxed/simple; bh=3lhMzHas/uh4/DrY0pO9kuckfK2pC3KlOv/G7+zWHcw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=YlxSid0TMi7K0y6i/Kb0ZhcZzd6rqCucy1SOl8njxvrfhU6KOheqncUvGzADBmbMtYdaGrhqg90jIp+vbitt8DWI0q2ERi8aCyvPerZencjoEcsBrEKNNxFZzjQkPt5W7iaOzRB2SKEzz3kqYSPDrL+MSs7N1pTGqSWQfgqa8PQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=E4XcTJV182zVbWA7O+5SwpoH+c9K5tnh7uz3/oEzALdUwPbS+neMIhKE0rIwlfhRkKWueClhdA6AdVEGLP0AzWAhXLG0vxus7zJLaH26YhGf33kisF519aQ2uAzXmvxaffpH8a78ocpr3B36vD5ngT9KtLZ990w8rCGzUF7JoHgHLMlNLUFzSAuUoC8YVVZ63GQpozEJprB+daLkVMI7mzbbWj7D1bNlR0PBGVw3sigm0acNlea8Opkoi/0ZBLQ9VujHv+fNPvCG+Lgijo1fI3Cko6nvRF+y9wKWahnA6GBts+SnlsaIYP+/3+3EkLoHI+wUuvIx5qs4sb4ERFCUOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=yhNziI3nbRiOwj+s+zdt0uNKjwfvhmYMdUwsNiOUiYE=; b=Rk3JRP5OpsRp/bBDkGW2jsXx7rIIJd+ygFhyKmEJNFHuls/ECZVuCDMJAZ+fGlT6zo4q9PMN9yCqBVUbX6PkOEOZE1eG8etg7vIMARCcQtHjGEG1Ntg5o5ekW5vXorlsy2lhbMxD0U3IB3WigWnAKxfWyn4uWGo2YfzvO7B1kLC6zBnaQhwJ8m3QBfwGQ9WOJ9iciGaBXQyd9p2fC6r+3bebHfvtA9Y6SMmLGjQMt5WfyANTR7cy3pDqN8/XzXUoNbmYqNniZLnvepIg/sK099H84WBldnVqAU5vAk+cmjIVVYgNHVmO6D4FecP4Oj1Vu1xql+KiubBOW4e571HEDw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) by CWYP123MB8971.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:286::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.6; Tue, 25 Aug 2026 14:14:20 +0000 Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230]) by CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230%4]) with mapi id 15.21.0339.012; Tue, 25 Aug 2026 14:14:20 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v5 4/6] sched/debug: Protect p->mm access in sched_show_numa() Date: Tue, 25 Aug 2026 10:14:11 -0400 Message-ID: <20260825141413.868997-5-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825141413.868997-1-atomlin@atomlin.com> References: <20260825141413.868997-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0199.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:318::16) To CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLP123MB6607:EE_|CWYP123MB8971:EE_ X-MS-Office365-Filtering-Correlation-Id: d8621cee-5afd-439d-70d1-08df02b3282a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|7416014|376014|366016|1800799024|10067099003|22082099003|18002099003|56012099006; X-Microsoft-Antispam-Message-Info: VR7CZ1jgt+Qcan7K+fWIfJWDtLeWHXQpBd1x+QmJNsZynWJH61kj9vihm8FQzAwec54C3lhckPLiFKrxkfNRUfTS4Btf5GuEW4QHYJqWVXcXw2CQIs4hRTZMjTscVZmGAQwgyjwAf3If0mLtaAmOTc0mSlY2i0fSdKBIdoEx0kc1A7fJqhUBlZXSYRSuXW+VD6ECzIjle6/d5EaK0NACgurypSV7i/Fze725fgY7yg1/ppnnj9/JVAbS/jpPcsVjAoOun5VE7SObYT/MV6fl0Rs4LeOHbenGOTkLnB2SutlEJhwaxb6OSg97vC9Ik4d4ZOjq3Eqer0cOO4Q+U9ixTmTmizNiCP5gcZSF2q9+TI0BzYIVgA3hfek+OTnwpuJD8kdPpIRzYxdhAWqXZueKUSZTVT5cXOX5apXOBw0WToEaiBieHBmpPp3PpDOBtRswLMlOc7n0Np4DINohxdB61NKcoCeFBW76fIx6O4URGwfhUKR1h13h91ko6gS/YfFsiTYf5m7JKtJaJ+jRApR5LYgIFrP4fvMURnFdGLHrUCSfywcBIW6eOABSAmc8OWWIZJ98Cw+jJY9EIS2OtG6lDpPVDVIgCkd4DXcDcwDlm/vJmM0pyM4toqkighkud6DgPTuRe3JXK7+k2uMxBglhQVse891uagrLppBwOtT8hZA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(7416014)(376014)(366016)(1800799024)(10067099003)(22082099003)(18002099003)(56012099006);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?l1/MejymtWyAlCp2fU6g0OLWCMb81yxTUqBZqzVU+erYRbZRWfNAAV656CJT?= =?us-ascii?Q?EZ5tn2KBtcuArdC/Z4Dmdx08C+qRqrVWJhz6neyxUOH2oYxL0sU50f1T/z7f?= =?us-ascii?Q?1Fvflt870OlZOK5y/OEfkZAmOc0c8u3CheBb1n8XrTIuRCG8Erl1+lq6fuF3?= =?us-ascii?Q?iW1PoruxwGN0wLCop/+8BqjgHVTI7HDF8PUOAWNX3M5dzOUEqEnTSKfPMPM3?= =?us-ascii?Q?8f86K4jGyyxFkHDb8H0iljx/TEGyAw1vl7US7YeO34XY8bT4g6JN0BwLzD4p?= =?us-ascii?Q?whqhHJIahep45b/PRZ8cDccpAln90aFIrkr/O7KCuCn4V0kz/5Wn5AceYiM4?= =?us-ascii?Q?9BPqDALq/FMjM33eh9gGshJXUmpAexsMWLxLUdr28RxV5NdvjznIt0Vjqt+y?= =?us-ascii?Q?t+0eIlTJbZBbd7XrofHcLfimC5U2aiMTKxmtGxibGJXPuVB3pnREUxYTlsoM?= =?us-ascii?Q?laCMxQ/5TPADPY8hmtn9ToOy4LfaCiAsrOdKTEHJGV/UO/ZOMtBovZHs0pei?= =?us-ascii?Q?7tvHdaHj5zWCHiHEMsh+/yQiY+4Pab3AA9JdXLVdOLU+v3/l8nykT7SgeLRb?= =?us-ascii?Q?e/p4fdnBqvRKX9bJcktZurApGMYEOWL7HL1wUxNBGRhN7E6zRZRcst0R3fG0?= =?us-ascii?Q?hOAiL8ZLzvURAzFdBsp+O9lKS9LkkEBHFN0PUs9pUBgACkl3XUMPxDDGUPvI?= =?us-ascii?Q?wwqTVIe0kniEYN0eLqjtqU6ZzyMjJvNifutatU5+1ofXzPWneZYcthJtQ2rc?= =?us-ascii?Q?YyTZ9crEJURHciYIi1uROoHPlX3pKe0M/J09MuA6ACezjniI1CcIfJb1nP++?= =?us-ascii?Q?PW8ak0yyfrXiwAbXzPBzvR6amflEubJYRPbj1RX9GBKzIS6v9+S/y2fbTMNv?= =?us-ascii?Q?dX+BNPy0snvgw1KU1HblRcG4UUjPJIT35x9LlxXMvCWYRlZAEli3JORiufEq?= =?us-ascii?Q?COwYHp+VgdzunbhPQy6Eqe+F0plL/PdLePnAg//aq/0I9hY6tS0YZUsrpL4u?= =?us-ascii?Q?4y9bxXbVk0HpAs+qUqeX6RAoxYdNQyctGGFS74lsOvvDAtBM74ecUEZRMU6M?= =?us-ascii?Q?19mBOzQlGgIv3YLmxdnHiXzYAC3WiUisKGLiRKRNB8B3HocGkf0K2t85fXWF?= =?us-ascii?Q?s/WuykuFK4VNa8h+GEFK8nIo7OwQw/Gf4EfLd8sCsoVsqPnywI3f6rXaUWKc?= =?us-ascii?Q?XsBujEzC1K9BzbCgzdC6faOfjXC7yVlSG+AiORtj8TGNRpYeF5CC2ababclu?= =?us-ascii?Q?FC+AUVNOvPf06hrNaQMqpxapjC+PXUyAGuuSvrUsDxo5WWqXQmMEIlNwwolh?= =?us-ascii?Q?IWU/XpOdymNtD7fdf24Nq4SJha61dk6hfiiFU/2iwSt4zyvdRwQJkvbOH22S?= =?us-ascii?Q?ZAhc8sWiglA1pULSJGL973Ubdefq1drhvp/PL7ounVishXf+JjYiEu1BaM9h?= =?us-ascii?Q?O81zVdIfScb506t2i/uWcgaaWH/WZvSJurDBBtVFs4Dx1eb6fXN7/vAimoVG?= =?us-ascii?Q?ycpyUbVnnrpMZI+K+FJFmnpxiA6kDGb/xGOFeCywJf6LhT4G7DOy7xW9twnh?= =?us-ascii?Q?qO4NNt9qJXgh/+HVEK/WrQ5mzC64zYf0AlnFSOimrsppgtXxCFTMF1glNf2H?= =?us-ascii?Q?pidxDxFYh4AbQ7gNqW6fvWO7/XLzLUR5Hc259M2r2WNCS0hs3RIQQ9dOKRID?= =?us-ascii?Q?DNJ+paOLlCGhBDix/JjEzt1f4lYdfd6ElfQ8vtXOas8LQQDb?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: d8621cee-5afd-439d-70d1-08df02b3282a X-MS-Exchange-CrossTenant-AuthSource: CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 14:14:20.7955 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rLPHDjOvTJgP1V7SzZJnbo61BGCVZjaW1JYrxp3R7BMxCvKELo/8Ur4JDCBun2zExCoTfCYGx0f6Zz3nnqyFzQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CWYP123MB8971 Content-Type: text/plain; charset="utf-8" In sched_show_numa(), p->mm is checked locklessly and then passed to the P(mm->numa_scan_seq) macro. This presents both a time-of-change to time-of-use race and a potential use-after-free vulnerability. If a task exits concurrently via exit_mm(p), another CPU can set p->mm to NULL and call mmput(mm) to free the struct mm_struct. Dereferencing mm->numa_scan_seq without holding task_lock(p) can access freed memory if mmput() runs immediately after the check. Fix this by wrapping the p->mm check and macro dereference in task_lock(p) and task_unlock(p). In exit_mm(), current->mm is set to NULL under task_lock(p) before mmput() is called, guaranteeing that p->mm cannot be set to NULL or freed while task_lock(p) is held. Fixes: b32e86b4301e ("sched/numa: Add debugging") Reported-by: sashiko-bot Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 27e0840ba9be..c05ba4d8b169 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -1312,8 +1312,10 @@ void print_numa_stats(struct seq_file *m, int node, = unsigned long tsf, static void sched_show_numa(struct task_struct *p, struct seq_file *m) { #ifdef CONFIG_NUMA_BALANCING + task_lock(p); if (p->mm) P(mm->numa_scan_seq); + task_unlock(p); =20 P(numa_pages_migrated); P(numa_preferred_nid); --=20 2.55.0 From nobody Mon Sep 28 06:37:32 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020133.outbound.protection.outlook.com [52.101.195.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90F80484226 for ; Tue, 25 Aug 2026 14:14:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.133 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667271; cv=fail; b=ALTodD9hjhW5hizIzAmNITPb7i6kQj9gcRwrT6S52kkLbuhavBsCbbKV3n+3jnQ5i7sTooXLOWB2zrgG97NdXrpN96wq21X3pKDZvqjoVq94t72pSXqEMtzCjuzot4mCWabZ+hNW7pbDJ474CAH8BIQtIjeuBOOxqHYpHZqgse8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667271; c=relaxed/simple; bh=R0kOzotFKAirMS+Bp7OPuqe5NoqjLAifLYR5xdmq63I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=OqG+iRO7EIZvCuVvOk8CDu9QWEHyKJhH+3xOl4JvsnhMKpJ0ELbcjKTeqFao/QtMaJSXOyPfcru93u1ILYNYivIBawNQ7Pd8Q1PKWLCOFpGYvDO5OSKe9jHAaKeGaxxXJAUBzONdWX1Wb/gJ5SxkUwQunSp7V9kkDjhOZxZ6kTE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JGm9NWY5777LJTrOH6uHwQYQezuYUkSwjc4CkwUHBTADMjciiinL75laXf46CihNPT+9YgEy45/qQHzJ2G+dYt8KJTHpjC3d49xFQwtKIL8hmOdsUXPS7Kz52QtGgDLAZn8nEAFublTtM/gnCsHiXZ0qIsiRYhm/2jXhlUpv1MTYQBTgsWneKP7h/n0wClwj4DZ0QnQGYJOEqwLMuRWCqc5q6XssxdCY62tVCYPr5Q419mWKPflYClm694K8ZcuVLAnZsBJ0qDNvNpftwKoyO0/c9Sg4WLV82xmGkSEIkN/oWyMvVOp4K55Ez/dht5AtFMlX8XsVxfyjHnlhN+69rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=lT3gQobiWONXCK/KHz2DeumzA21Z9K3EkWLavANwepU=; b=OkXzFBLSJpbDcgyezyoVWNu5TZGw3YJREyrvU64nlUQlXQVXtTWdM+eF9UjygXRafVPRQF3H+M9CK4gvqpDy/d7hS51C69FrKBnWL6om66HLZb3xeTUKSoj+ONPq9otCWuZ4+JCoeq1iZ1mk2vRKSLZJRizPhUwjS0qSnJ+WbGjjgTtjwzfUu2w0ZuDPP019bPU65yR4TFVLtxmj9yFLH0dCr637Vg5pihSVh4ccCD6slsFXOnIUg0fuihTx16kZdVzTZeS80gMbJzxYESMGNnHg2VkaO2tBFzNSnxmjhbouHL+RdSP/bpZHOqRClqYo15M1/BZli7S89linICabVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) by CWYP123MB8971.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:286::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.6; Tue, 25 Aug 2026 14:14:22 +0000 Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230]) by CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230%4]) with mapi id 15.21.0339.012; Tue, 25 Aug 2026 14:14:22 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v5 5/6] sched/fair: Use list_for_each_entry_rcu() in print_cfs_stats() Date: Tue, 25 Aug 2026 10:14:12 -0400 Message-ID: <20260825141413.868997-6-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825141413.868997-1-atomlin@atomlin.com> References: <20260825141413.868997-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0418.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18b::9) To CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLP123MB6607:EE_|CWYP123MB8971:EE_ X-MS-Office365-Filtering-Correlation-Id: 003d8fde-7d36-4545-6ad4-08df02b32921 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|7416014|376014|366016|1800799024|10067099003|6133799003|3023799007|22082099003|18002099003|56012099006; X-Microsoft-Antispam-Message-Info: WZW5BGmc9bnnpfX8EVFA+ydJBih/HiihWkUQuWdWDdNKCtFUV67djgJES6pVhZdSteGVOTLK1OS4BQxxoOjpcNK4pef8vP0v/LxadgiL+ezR4Jhrew0txVMahoqCTnCDuKGnjdDlcS3DtbQsPf5FNsRpsffaT4YgYvg1sYmnd85qs0Ra2q3t+LRUY6fyj+dlNvZCzl/t9njgNDjIOr2bv5TGurIS/fMSxfQgFpw59Ra9Q8ghOedXkKQ/Ck8M9YzfHCHREVDVlU/egoTzLclij8L7O8trizU18LHSSZlzKmLITjyz6rxx9D79kZCFBbMmkUqytm1FuOkeMYepuIUQZPylGpUwInpNo3OxfimAMV8S7+cd9AZdk0A1rmeJWgDGux47OZZ5jaqDTmnRGNXGUPR+/idkodQX0mLnWTvsquiPFKHIJ0H1XmA8hYLuU8cGscdiMcAjHVh9WYEhKJr/2mVq7SIt3mgAGCA1HTBWBQO+krI/wgvqWxUKTmE9M8fCCDSPVvTZn0eFwq/Uso8fR0gfmL/BfewNTfi1dlI+dt2PrD5bg5OvluksmdvpUAu4yvaygEGpRSdK6YEwJadvUgX+UqWxpNiLAYfJN8YJ3B0p6DngROHlXY+5CNaHUCPyB8zsctfMHNMM01hyuuNXzbnI/nTdl1nq0Jlsgw7h3ZM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(7416014)(376014)(366016)(1800799024)(10067099003)(6133799003)(3023799007)(22082099003)(18002099003)(56012099006);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?OI4L0/1COqXlnUqdWY/UmFhrJWooJsCsCLt/CazYD/Uz/9AVgSAVT6T3pVA6?= =?us-ascii?Q?haKXGx7C9mfvYilxisIgxEGTQIBfaI9S82On8zq1a1G43lbvOt7K4+k/3Foa?= =?us-ascii?Q?u5Jbd6+4XE9zmB4TWBfk+05lYkdWXA2xx2Ix8iEhJ6MCS2y641ct5M96YHqH?= =?us-ascii?Q?+wMGGb6tbH6d6nu7dTwcRTrP59WRX+xGBZxh5x4TVFWzo+Xbag/KNbbfUWSu?= =?us-ascii?Q?nYzYo6XID93gcAY111u3RFMnzTQTD5aXsraoY4xRojHRG9+5v24tdEtLCp05?= =?us-ascii?Q?4L0dpMxB8ghfmjXP6JFclEXNgxNmN5Ogfjjcxakl2esZmrQVvVfWUplseYdO?= =?us-ascii?Q?WnEeqPIG4s6RmT9Si7cFVgflIA12zfjxPR0BB7OD1axUTUD5RB0RBsRBbhH3?= =?us-ascii?Q?t3ykin576luaTIVEmjOE/qIWZjMv2G82NktPLmEQopjSej6a10ziHthkEddC?= =?us-ascii?Q?bykA2ThDO32mksqGaHKZ44BDLECMFvHFjitRV5v44ACVW2j5e0wW/qL7QcNB?= =?us-ascii?Q?LdIVXH8wyJjiAmb/D67CJ+my+OsQbAxBk/o5PGYA3Yi0vFkvRghpv142fpeL?= =?us-ascii?Q?tBUaxxXFOdK/mN4NmgDtADv/Lg4OESCA3UxnHFBx2GtkN6VZj6LCVXVX/8ev?= =?us-ascii?Q?M+xY2FvMd46ZaQgHQxBx6obY0t0zhn2p61L4A9nz0jckE5nzep+24M897Ih8?= =?us-ascii?Q?StVfgkby4GRY9oITDj8LLblYYjfJOfapJI/sFBgxZL8/UbVLT21G8JJop2ox?= =?us-ascii?Q?XsdrrMfw0vmk+PwXEFToPN+PvHOJKRc2/5o+LrJuezm0pjgEKcOm4B3Q0SXX?= =?us-ascii?Q?L3bREM7uxuf7E0eVeJoTq6E7EwyX7bzuZnayMEdr98Aa9WSUes6y5cK5Kk8+?= =?us-ascii?Q?XLRdR5Nsw2ndQuYMW8HL33sb74I9m4TP25R8q8lHPyEddopfKg1U1t/+NP3R?= =?us-ascii?Q?8F1UsCQQdZYGpp6NS3TcS6xOAjRa2i00LLApt4cHY4CFuLbQqOoPGRdfHg6e?= =?us-ascii?Q?yuL3Xgsu5o4l099+dZyCQPjtHM9apV+SLAElXiKt8hz534MBv44HDE6bCR6Z?= =?us-ascii?Q?Hp4rxytXLxtbDVdITdioxnQrqZmQk7KlVDs6/01dLeF1YiLe+Fm77amuLo+h?= =?us-ascii?Q?Uj6DFvzhI9TAc6thV1w3zaMWQryUkTcJ1CIueqlLzTnanDHoVJh+lB6za0t4?= =?us-ascii?Q?EyYZZ/Y1bEK69u1fHS6dngKnxtXUEQSIDAcX0y+89mOrmQXq0vVsAWEuFVWE?= =?us-ascii?Q?3NBNbIIz8y7Ey9b8qN0zrf1XoT1pctG3teiW16VS42Jt7LfCKMqCvuvsLlgX?= =?us-ascii?Q?1EBTc8AeEBS84vjN7aZJS1lqEGkM8s2SG07RaAZtrIEC+ZWyWK0Wg9zJwM8o?= =?us-ascii?Q?4jQ8W3zdyLnFXlnfQ30p+vwdV+r9tFq+bMHFEdQanvUbXX7iiIP4TvhQOTBO?= =?us-ascii?Q?dPf6Vi52tHxPYqXdLK1CDIZosNKwDO4vr5ADGApmoeoscTEImu2f7c1jSBos?= =?us-ascii?Q?E6OURsKUX+F3ybcyZan0Au89oUUeCGn96VZcPOR0w0laEbbc3l+Gz048g989?= =?us-ascii?Q?41Rrc/NeGTVzEaGW9IR9AyHkYniQ6UHufXwDGl0EaPWdyhlwlv/cqkgo6f/t?= =?us-ascii?Q?P/qHxhyYspLlI2k4ckuRQSu2nZj5WRN/4dP/BFLBPTGgKDQe7R3fh9rHlx3T?= =?us-ascii?Q?ejNgqXY6NPsq81VFZEUruiPjVUM9F/9yhUVRcSaTgliIBnzY?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: 003d8fde-7d36-4545-6ad4-08df02b32921 X-MS-Exchange-CrossTenant-AuthSource: CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 14:14:22.4959 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +oVrRXCvb/fSRlbb6v/m5OKJzUMkwxHF4Ev5+OTV0OhfUqfJc38cv4VMFMh6YHyUPQJVP51p84GNQAbLc/uugA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CWYP123MB8971 Content-Type: text/plain; charset="utf-8" In print_cfs_stats(), rq->leaf_cfs_rq_list is traversed using for_each_leaf_cfs_rq_safe(), which expands to list_for_each_entry_safe(). Although rq->leaf_cfs_rq_list is RCU-protected, list_for_each_entry_safe() is a non-RCU iteration macro. It dereferences pointer links without READ_ONCE() and pre-fetches the next pointer. When a writer concurrently adds a new cfs_rq to the list using list_add_rcu(), a reader traversing with list_for_each_entry_safe() lacks READ_ONCE() protection. Without READ_ONCE(), the compiler is free to re-fetch pointers or reorder instructions. As a result, the reader can observe a newly inserted cfs_rq's pointer before its internal fields are fully visible, leading to reading uninitialised data or dereferencing invalid pointers. Additionally, because print_cfs_rq() drops rq->lock during seq_file I/O, concurrent cfs_rq list removals and re-insertions can modify cfs_rq->next. If cfs_rq is re-inserted near the head of the list while rq->lock is dropped, lockless readers can jump backward in the list. Under sufficient load it could lead to unbounded list traversal inside the RCU read-side critical section and trigger an RCU stall. Fix this by introducing for_each_leaf_cfs_rq_rcu(), which expands to list_for_each_entry_rcu(). This uses READ_ONCE() during list traversal. Finally, capping print_cfs_stats() lockless list traversal with a hard iteration ceiling to guarantee loop termination and prevent RCU stalls under continuous list churn. Fixes: 039ae8bcf7a5 ("sched/fair: Fix O(nr_cgroups) in the load balancing p= ath") Reported-by: sashiko-bot Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 39 +++------------------------------ kernel/sched/fair.c | 33 ++++++++++++++++++++++++---- kernel/sched/sched.h | 51 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 83 insertions(+), 40 deletions(-) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index c05ba4d8b169..8793059688d7 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -11,17 +11,6 @@ #include #include "sched.h" =20 -/* - * This allows printing both to /sys/kernel/debug/sched/debug and - * to the console - */ -#define SEQ_printf(m, x...) \ - do { \ - if (m) \ - seq_printf(m, x); \ - else \ - pr_cont(x); \ - } while (0) =20 /* * Ease the printing of nsec fields: @@ -853,38 +842,16 @@ static void print_cfs_group_stats(struct seq_file *m,= int cpu, struct task_group #endif /* CONFIG_FAIR_GROUP_SCHED */ =20 #ifdef CONFIG_CGROUP_SCHED -static DEFINE_SPINLOCK(sched_debug_lock); -static char group_path[PATH_MAX]; +DEFINE_SPINLOCK(sched_debug_lock); +char sched_debug_group_path[PATH_MAX]; =20 -static void task_group_path(struct task_group *tg, char *path, int plen) +void task_group_path(struct task_group *tg, char *path, int plen) { if (autogroup_path(tg, path, plen)) return; =20 cgroup_path(tg->css.cgroup, path, plen); } - -/* - * Only 1 SEQ_printf_task_group_path() caller can use the full length - * group_path[] for cgroup path. Other simultaneous callers will have - * to use a shorter stack buffer. A "..." suffix is appended at the end - * of the stack buffer so that it will show up in case the output length - * matches the given buffer size to indicate possible path name truncation. - */ -#define SEQ_printf_task_group_path(m, tg, fmt...) \ -{ \ - if (spin_trylock(&sched_debug_lock)) { \ - task_group_path(tg, group_path, sizeof(group_path)); \ - SEQ_printf(m, fmt, group_path); \ - spin_unlock(&sched_debug_lock); \ - } else { \ - char buf[128]; \ - char *bufend =3D buf + sizeof(buf) - 3; \ - task_group_path(tg, buf, bufend - buf); \ - strcpy(bufend - 1, "..."); \ - SEQ_printf(m, fmt, buf); \ - } \ -} #endif =20 static void diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index ad367a542eb0..b85f826be450 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -412,6 +412,10 @@ static inline void assert_list_leaf_cfs_rq(struct rq *= rq) list_for_each_entry_safe(cfs_rq, pos, &rq->leaf_cfs_rq_list, \ leaf_cfs_rq_list) =20 +#define for_each_leaf_cfs_rq_rcu(rq, cfs_rq) \ + list_for_each_entry_rcu(cfs_rq, &(rq)->leaf_cfs_rq_list, \ + leaf_cfs_rq_list) + /* Do the two (enqueued) entities belong to the same group ? */ static inline struct cfs_rq * is_same_group(struct sched_entity *se, struct sched_entity *pse) @@ -497,6 +501,9 @@ static inline void assert_list_leaf_cfs_rq(struct rq *r= q) #define for_each_leaf_cfs_rq_safe(rq, cfs_rq, pos) \ for (cfs_rq =3D &rq->cfs, pos =3D NULL; cfs_rq; cfs_rq =3D pos) =20 +#define for_each_leaf_cfs_rq_rcu(rq, cfs_rq) \ + for (cfs_rq =3D &rq->cfs; cfs_rq; cfs_rq =3D NULL) + static inline struct sched_entity *parent_entity(struct sched_entity *se) { return NULL; @@ -15400,14 +15407,32 @@ DEFINE_SCHED_CLASS(fair) =3D { #endif }; =20 +#define SCHED_DEBUG_MAX_ITER 4096 +#define SCHED_DEBUG_TRUNCATED_MSG \ + "stats truncated at " __stringify(SCHED_DEBUG_MAX_ITER) " iterations\n" + void print_cfs_stats(struct seq_file *m, int cpu) { - struct cfs_rq *cfs_rq, *pos; + struct cfs_rq *cfs_rq; + int max_iter =3D SCHED_DEBUG_MAX_ITER; =20 - rcu_read_lock(); - for_each_leaf_cfs_rq_safe(cpu_rq(cpu), cfs_rq, pos) + guard(rcu)(); + for_each_leaf_cfs_rq_rcu(cpu_rq(cpu), cfs_rq) { + if (--max_iter < 0) { + SEQ_printf(m, "\n"); + if (IS_ENABLED(CONFIG_FAIR_GROUP_SCHED)) { + SEQ_printf_task_group_path(m, cfs_rq_tg(cfs_rq), + "cfs_rq[%d]:%s ... " + SCHED_DEBUG_TRUNCATED_MSG, + cpu); + } else { + SEQ_printf(m, "cfs_rq[%d]: " + SCHED_DEBUG_TRUNCATED_MSG, cpu); + } + break; + } print_cfs_rq(m, cpu, cfs_rq); - rcu_read_unlock(); + } } =20 #ifdef CONFIG_NUMA_BALANCING diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index aca352e2f4a8..e25b3fd35972 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -774,6 +774,18 @@ struct cfs_rq { #endif /* CONFIG_FAIR_GROUP_SCHED */ }; =20 +#ifdef CONFIG_FAIR_GROUP_SCHED +static inline struct task_group *cfs_rq_tg(struct cfs_rq *cfs_rq) +{ + return cfs_rq->tg; +} +#else +static inline struct task_group *cfs_rq_tg(struct cfs_rq *cfs_rq) +{ + return NULL; +} +#endif + #ifdef CONFIG_SCHED_CLASS_EXT /* scx_rq->flags, protected by the rq lock */ enum scx_rq_flags { @@ -3393,6 +3405,45 @@ extern struct sched_entity *__pick_root_entity(struc= t cfs_rq *cfs_rq); extern struct sched_entity *__pick_first_entity(struct cfs_rq *cfs_rq); extern struct sched_entity *__pick_last_entity(struct cfs_rq *cfs_rq); =20 +/* + * This allows printing both to /sys/kernel/debug/sched/debug and + * to the console + */ +#define SEQ_printf(m, x...) \ +do { \ + if (m) \ + seq_printf(m, x); \ + else \ + pr_cont(x); \ +} while (0) + +#ifdef CONFIG_CGROUP_SCHED +extern spinlock_t sched_debug_lock; +extern char sched_debug_group_path[PATH_MAX]; +extern void task_group_path(struct task_group *tg, char *path, int plen); + +#define SEQ_printf_task_group_path(m, tg, fmt...) \ +{ \ + if (spin_trylock(&sched_debug_lock)) { \ + task_group_path(tg, sched_debug_group_path, sizeof(sched_debug_group_pat= h)); \ + SEQ_printf(m, fmt, sched_debug_group_path); \ + spin_unlock(&sched_debug_lock); \ + } else { \ + char buf[128]; \ + char *bufend =3D buf + sizeof(buf) - 3; \ + task_group_path(tg, buf, bufend - buf); \ + strscpy(bufend - 1, "...", sizeof("...")); \ + SEQ_printf(m, fmt, buf); \ + } \ +} +#else +static inline void __printf(3, 4) +SEQ_printf_task_group_path(struct seq_file *m, struct task_group *tg, + const char *fmt, ...) +{ +} +#endif + extern bool sched_debug_verbose; =20 extern void print_cfs_stats(struct seq_file *m, int cpu); --=20 2.55.0 From nobody Mon Sep 28 06:37:32 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020133.outbound.protection.outlook.com [52.101.195.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82A1C484257 for ; Tue, 25 Aug 2026 14:14:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.133 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667272; cv=fail; b=ovV/UdS/8eVfUNcr2pTBgCZkUHcQcsPoD1UtS7g3x0AC8OvtzK1lAbF09YvJz8OCDR1lF1EOhTG0iOOKCzzvdNwOIL5FUc+C0y1+G7tOuK1r9a8yPCYimlcayWkUxmo0WlFGTrhQdjkepoYrHlhg/ZjSmPjkGTTlR0Bi9MtKMrA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787667272; c=relaxed/simple; bh=HYtOwjWt4OfQ66/OaYLqaYwltMKiLcVGjuJlLqHnvGA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Sg9jVGnJJvFP5SWMUd8DKyiZ1//dUejP6a0UvuBodwDP48iv4T52hbdHUDNNhdqg3yv8Q4jy0EQFEqZMTTt9qu+5z29CzkGPM2llIFrGO7A7bQcyEwsixDP/tSMz1wvOCeQE+P8elWU1KA32o+bJCY91HShmz9gXXHBqwnm6RiA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NJeRjeRgAmtfchaYPe/PrYBo8tE/Ql6UA0fvyDSieSmfuGTT9c2CMZUNlxXiMd3wpkFbUdJqn64wwDog+JMRRQv75rOejR1eelY0+2BUSIANgrzz4QfeUwNbZGO0BDQAQP+5ZnbEtJLQdcQ49a4gHzHA4Ed0lk60jHFL6xP7SBTaux3/8oNg+A7FsLkYifu9Dn3PmJd8euKfGYhA4aCh9uL4/yVbnBnnkdT5tuf4WtAGldi2SRNgWIChuqBtCVYQrkTrUd3s9VnNjtDZsbG2z64Y8eiA8PdWKEJ12ZGAi9Ti53K3P4qZXlD6r0rKcHmCXcI2G/HZVPmANcCkrSwVqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=xyV3jG1qNudPQbbyRel66uw9BP5zokC12X/izJbsCMQ=; b=NxDJryPVBu2qpsHHT+s/R9W/ZCFE66NC5Ufg467MUxsZHcnhtvsEnl4m+5na2iWkUUP/DRWjkS4JrGtbHfwbg89XdORX7UuC3gw1UOxPOse22r7hoR04VE4QThZc+ygqj2HRQfKdK/HjaqlqlractSVJte5BtJb7MVkCSY15KJnef/RnwKetMQH7fTZEGjKZSUMFE4Alk+fJ+YFtM5cqthIcIhmTV8jupj6kO7v7GATvRAY6UQQakgaIGW59+qZH4WkfMmcXwwIRV9ELDhfMLMlYbX2BFtjIqfr99cx97DSkGPxaOcINq+TmtG0mKw4wSBzgEwMOJaGqCM2ipl9Ytw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) by CWYP123MB8971.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:286::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.6; Tue, 25 Aug 2026 14:14:26 +0000 Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230]) by CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230%4]) with mapi id 15.21.0339.012; Tue, 25 Aug 2026 14:14:24 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v5 6/6] sched/debug: Introduce per-CPU debugfs files Date: Tue, 25 Aug 2026 10:14:13 -0400 Message-ID: <20260825141413.868997-7-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825141413.868997-1-atomlin@atomlin.com> References: <20260825141413.868997-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO6P123CA0030.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:313::10) To CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLP123MB6607:EE_|CWYP123MB8971:EE_ X-MS-Office365-Filtering-Correlation-Id: d03913aa-d06f-4751-8208-08df02b32a38 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|7416014|376014|366016|1800799024|10067099003|3023799007|22082099003|18002099003|5023799004|56012099006; X-Microsoft-Antispam-Message-Info: 575v2T60E2aHkL8V5q8uikuZZAtEGos67jZquiE+qQyCuXR4J6xW2Z8J89w7jYHus5j3h2lqdpo0PmKW62A9kVMTit6xEyjsrharevkkYiAQZf6ih/JEbhs/S23pRNb0KbAV3l3JXADXWekLnUtJ82A2LeiUa+fIx5XDm7LZBpU4KyfhleU2B4QryEhTblM8/bEHxrGSzt/lI47mR5Zm0RIdeVD7EYfrG/io3Wa6aDGJO0y5WEARyTXiJQtsjDcC/eCgDU+xN8T/6V6lQXdi2Tzv8BCE9vR0l1Lo0LvSvejYdsbDqYzFtty9ptaeJ2gv/3yzaVbzzdazm9MvUounQN6SRDa2maFVr3HbBf5+kZLhSYS2bpYwVudO+qcp9DWWO2ZV3CqaEngRizzY+kEkftBkXXs/k3JUuyJadtzEEglsZwo95ttJPVN8dPX6/L0ix0e2pUuJOs0iPeyEVg3N269JhdwLwIeI0XGy/fExdQwb8BPpWs9ERrsiiE0dSB6Sag8NklTfoDOPozaw9F1nJgIVif0/1NjLVUppalpegsZq0Pp4L4XEQp9If8aiR8n40qeuGOh5YKslG8zYI3e3rkTWgkKayQY5OaUQNE7b5dzXyJNqLx2RxMv45LtKxhdgjLcnrixNd9wPb+4xiTAhlrpRpqGjH+Cj3w8OkfmmBqY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(7416014)(376014)(366016)(1800799024)(10067099003)(3023799007)(22082099003)(18002099003)(5023799004)(56012099006);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?DC7kt6+dRyqswcIyb546ulU/2GD55aMi/fxGxKVr0yfhWm5F7PZ5tLz0fae7?= =?us-ascii?Q?BFQx8kjCMvRTSU6C1rztmILuLbfYAk14KDuEfnN9kYOF4wrLYVVilrvek7bA?= =?us-ascii?Q?fqiXPgQasrSzlwZQwrpPgMWPdR5bG5HW1g9yhauPi2DqJNzvWhwZkK3szXKT?= =?us-ascii?Q?RpTxS8UUNWcw4bvB+E0aq1Z22dIwt7tS3NfYFReJ0E3WZmCk2jCiu/BrLcji?= =?us-ascii?Q?8aIOrn/lZo0LbiiZ6anAD7JE2GDyOVkFqSTGeWGIMw+2lYPXYqSZDLIRAfyo?= =?us-ascii?Q?6vyGSrgqB/sBtMBs0F2RmuP3tzrgNkHLrYXfVMaQLqfYoLhd1V9LXGx+6Tny?= =?us-ascii?Q?5PutkcUxRIURuY5/+sPCDdcqjUMDEZ55qmdeaBy7RP13b794mP1ZDLwq7MNp?= =?us-ascii?Q?Pl0acAS3oyUixclGQu2Ib/BST/H9sTwLgLs9Uq6CNjfq//2x6eLUZtpscO2x?= =?us-ascii?Q?PtVk1VUxgO75IRS2lCyV4JXAdQCiBzC8pHQc9CGytTnFnDqJGpoEWpKkEhwS?= =?us-ascii?Q?+vHQGPyhrQvJkBmjDAc7Z7YXkyOZBRqBDeVzAZGouX//ZxgzN1PPIUtbPbAw?= =?us-ascii?Q?OLD2gpcWfbLSTsPDLvyZ0FfiIxWTWoFL28kyS6sDg1EuQbMBnwWP8i3y+P5B?= =?us-ascii?Q?R09boFQ2+cCMozEdOHQ8yyU4nKWQbtErKAf3CsrUzGKBtSUSr6RdzBDhqxNA?= =?us-ascii?Q?YDbBGgtPbEpk3ruwqhxYz0kUOO/7PuD9X5vErc2siexydWWz5qL/2yPQ9RvB?= =?us-ascii?Q?jU2sjSWmKef6jbex7s4XPt0Oawr2uwv4qn5QOwSz68AVw+4rbKKrq/l6p8QA?= =?us-ascii?Q?6YU9HvVWAt2yKilf8GiNeXsWW5MVjXWrJRghpoySbIOUqNcyuImrzWgLv35H?= =?us-ascii?Q?EyLgdNMpoqIOqNLvdLWtRJkn5y8JciV+5BL3qBAPl+FJGxNI5GK7RXLXUjTy?= =?us-ascii?Q?E3CK0/z3EEz7sBNBa5Kt053lQhda47HC1s8DgfC6uKk/+QT+q6VA4MwY3V8r?= =?us-ascii?Q?8e/KDp3l0oUibIeIUCtYXCqjTtKYyGmRri2feoOzghxvg16GPonGwQT8Y4Se?= =?us-ascii?Q?xfgNblxgbirGUFKh1Lt/91u7tg7z5MKltmMyXAHQAuOSBitSWuurRbIWB+1F?= =?us-ascii?Q?8cr/DTBV/7z3I1GrN+MdnhlmMUNfoMOy8X6Se8PcO2zwGHNIjy95MB4ribtv?= =?us-ascii?Q?sBWVQJ7wClTPufkYPiwYH8zenPRfhbzMaBequOJ8uw6F8AeydWvklwlmE0QE?= =?us-ascii?Q?4aXocG/KZQv1UlaVO7xn7QVwEll2NDEJ9FQq3M5SHGFEnamzdTjGjvj2M1z/?= =?us-ascii?Q?eKkvsIMaE/9zHXZB2Dj6Y5EzWz3DlQHhmzzzDkTvQ9bxsny7S7lt8swq9maQ?= =?us-ascii?Q?HUmBMP4S0yYkP33eczND9jeCHXD5k/A38O6IS+XI+/oyLjrTWASorkqzugFT?= =?us-ascii?Q?e2nJkFVpljuSst39+hPJ2gntjD//Guv2mUTJ484HLdpG3wVaf9VMDlcR3Dv/?= =?us-ascii?Q?RXgL9M4BFJrmliCFBhXvPfd5MSFL1l8Hbqo467UeMhYLyjhxUCH1CFneVdVE?= =?us-ascii?Q?jP79w/qkxURNDp3NufVhKHD94fR2/RIbfzb+YgWSdRLiVV0BNzrNqEKVlno+?= =?us-ascii?Q?JkXdp/3/yQmwNUFtIC04WFRJPaCQ2FCB+ExDkEUuFnpmk26DqysMEPfLQ3Us?= =?us-ascii?Q?9DnqScX5uVVFbAI/pdpHIFsLmTXrNvJDoOCfLljA6iBcTvaS?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: d03913aa-d06f-4751-8208-08df02b32a38 X-MS-Exchange-CrossTenant-AuthSource: CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 14:14:24.2976 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: d8+9vU9GFGAjlTEnK0BS/0bRx97SjZmbSOiUfrO7AiO/5AQImuDQbDUaQ8rOQHogYcr52CWTwwD0MrCVBGTudQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CWYP123MB8971 Content-Type: text/plain; charset="utf-8" Currently, accessing scheduler debugging details for a specific CPU requires reading /sys/kernel/debug/sched/debug, which outputs information for all online CPUs. When investigating a latency anomaly or scheduling issue isolated to a specific CPU, accessing /sys/kernel/debug/sched/cpu/cpu/debug provides an immediate, targeted view of that runqueue. Add support for per-CPU debug files under: /sys/kernel/debug/sched/cpu/cpu/debug. Reading /sys/kernel/debug/sched/cpu/cpu/debug calls print_cpu() specifically for CPU , exposing CPU-specific runqueue details on demand. If the target CPU is currently offline, reading its file returns -ENODEV. Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 8793059688d7..a3f5b105b7ea 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -345,6 +345,7 @@ static const struct file_operations sched_verbose_fops = =3D { }; =20 static const struct seq_operations sched_debug_sops; +static void print_cpu(struct seq_file *m, int cpu); =20 static int sched_debug_open(struct inode *inode, struct file *filp) { @@ -622,6 +623,47 @@ static void debugfs_fair_server_init(void) } } =20 +static int sched_debug_cpu_show(struct seq_file *m, void *v) +{ + unsigned long cpu =3D (unsigned long) m->private; + + if (!cpu_online(cpu)) + return -ENODEV; + + print_cpu(m, cpu); + return 0; +} + +static int sched_debug_cpu_open(struct inode *inode, struct file *filp) +{ + return single_open(filp, sched_debug_cpu_show, inode->i_private); +} + +static const struct file_operations sched_debug_cpu_fops =3D { + .open =3D sched_debug_cpu_open, + .read =3D seq_read, + .llseek =3D seq_lseek, + .release =3D single_release, +}; + +static __init void debugfs_cpu_init(void) +{ + struct dentry *d_cpu_dir; + unsigned long cpu; + char buf[16]; + + d_cpu_dir =3D debugfs_create_dir("cpu", debugfs_sched); + + for_each_possible_cpu(cpu) { + struct dentry *d_cpu; + + snprintf(buf, sizeof(buf), "cpu%lu", cpu); + d_cpu =3D debugfs_create_dir(buf, d_cpu_dir); + + debugfs_create_file("debug", 0444, d_cpu, (void *) cpu, &sched_debug_cpu= _fops); + } +} + static __init int sched_init_debug(void) { struct dentry __maybe_unused *numa, *llc; @@ -679,6 +721,7 @@ static __init int sched_init_debug(void) #ifdef CONFIG_SCHED_CLASS_EXT debugfs_ext_server_init(); #endif + debugfs_cpu_init(); =20 return 0; } --=20 2.55.0