From nobody Mon Sep 28 05:42:23 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020115.outbound.protection.outlook.com [52.101.195.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A601F3AD531 for ; Tue, 25 Aug 2026 18:46:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.115 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683609; cv=fail; b=tXiSPehjIg4J89h2GadMOE4uoCEsIkss4Bd6FOvA1wfFRTjK8k0A24Hf7iDGQMBfPSQcSF0mrE/O+yg280G+jGwosBsgNuAdngzLQbjnWawouxFNGnVAsuBbjC4uyaL6MVgeo67cYKwAjrutf+CmfocYBm6gFmcWfj5n2XJh+nk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683609; c=relaxed/simple; bh=guLRGRpcYPMXw+wCnbNxeBgH85MFtkvd0KIWUXR5jPM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=T8IVGvaDWbo2It/WP0KNyVakLKWFT7aH2pxiAvJ+axsI2niaUCp7sIbPGqx3/xEtB6Umm6WSchCYfjrFurKQUxewFlDRwJmFk+nsupfxvvLq9jeU1+ChuxI+OwqwPeXm6+64pBF85xVMXx+m1xdtlJk+wYbHfQJJrL0dpImfy5U= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.115 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=iQb8mPnTFLqb00+lD2OsGJHu8yV4MV3pjTqCQiuxvBu7bfSex4P20D3JrJ+JmYqDQ6KJFKheFt3Eyy/yIDsgVVMsoY3Ab6E8VimTUK+GM8t+jEvdE8ds54b2p/YymU6vPhqsh3xLKxxrYhjbhN9yS+AgxcPH8LH48uCE+VwWTQeUYYKZStbWfaq/Z/pT2XCZ6vBqY9gG7xmPOhJPIahR1q8UOpy7/uy8F8pHWPoyTHn46hMDqKRjPkM45sTkeIoddSofNdFMzkg3bAE2+wJ0ZrjTtHqMopv7O+4RoHCx+zkKm+aBHO8s981RW9oe1Ts4fNoMXz7Tkn4hLXx9Ez52Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=5w5uf12V33B4BT629MQ03nvD+WvztsB8y3Q+6+jREAM=; b=UX7q3SYdvve/GzTeYiEmOt5y3VCuG9P82GQNnc4EeDZXi5v7YblLujSOV7BtON3D20JatoEyC6ru1D5oHGVimQP0XKNRuhc4ra7J9G07S294uEQ7xfpebga7i81JqHswM/yWjT0pMrTFxm2wo2UX/4Qc2UL4RTf3+j59k8/l1G5seu/P/qCRC5vi3yoItVOqBEVRuLUJPOVo0M9kJbVnQohdNcKfETDBtcDZwAYK3POPnA53HYmCYTZEI2YCN6CQAACYtZGNmKroPY+Na5GHxjxDflO14MLSYeWY3EWSvyBwI6ML9d54ddlBfg+z8YSDYEwXjBsMBlX2dsj4ejJz7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) by LO7P123MB7501.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:40c::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.8; Tue, 25 Aug 2026 18:46:41 +0000 Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383]) by LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383%4]) with mapi id 15.21.0360.005; Tue, 25 Aug 2026 18:46:41 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 1/6] sched: Annotate rq->rd with __rcu and update lockless readers Date: Tue, 25 Aug 2026 14:46:32 -0400 Message-ID: <20260825184637.888364-2-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825184637.888364-1-atomlin@atomlin.com> References: <20260825184637.888364-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0091.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:191::6) To LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO6P123MB6616:EE_|LO7P123MB7501:EE_ X-MS-Office365-Filtering-Correlation-Id: 52d5f4c2-cab2-4a4f-4d85-08df02d93402 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|376014|7416014|1800799024|366016|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: VCKYWovPEomLldAfq+zGQFCuomy4IgseiYR975thVZvjgSccEeJVwmrNuYAjBda7eBBPEQ3pKErs3lRpwIZUCKoUXTOoWQSHtjKIA8fN0Y+jS+kNsyfYwJive/mmDgrIpOq/TPfv2zOAKyj2B28tahJkjgAV71iFAJ6c8EpOEbcL55u9zTBIG6j3Bm34g+E2pHHSbaipo/i73AIWVhzB6jFa2/XeCBQORs5rMgdtPMRmMbl3sxItcCaLd0KgvsBKgU3spmKJDdHo140yxT62eATbuDpYV4YcIiRlkaBk2qQt0VGGVNM2ZcFCpqBNpjJPRcZMD7GhTnW5zj4Md5DEEp3kFGy0qDG93AhTOUOC+I7IehD8i7MLX/AG4Odev/cAnyu4ts2dDCylOOJQxUOO6OxoqEFC3crUK0jd2xhnuAm+85ZTqHKZN2zH70vvg+AOBLshUA2M+0JfP3BGT7ZBPyDbfoTpS6l8v2a5/IICJlac5bNW45THDlUzD6ctYzkq49CniZL9CW/9PTnuN+ANN1H4tb2ruwablktHZMWUkTatb8nSzzbAAz42cGRoORnP57NWRZtriqWkUqRaj8WFms4iv+pSSUEgOk7AMQN9AKjYPMKsKIxH9IF8cs2Anfs2zocu4W5jYmCoFIASTmi/E1x3QJ4FeELQkWqnae8tl7Y= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(376014)(7416014)(1800799024)(366016)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?nbRv4NAvR9d9UEIiY25JG/HkaZFCsRerjPUUlpZrurpiZnTC3gbtj2y6qvBi?= =?us-ascii?Q?6t/ZbpqJcv6g1PfHtKx9QLIzcWM/scy7DcaDGRan9A/6LpmX/EXHykQ8OSB1?= =?us-ascii?Q?4ocw+WL0ZUfFTqJOqr420k7TVOzoiLQHmBxYrVmHBnpWQpK2xT9Ajt0n1TNS?= =?us-ascii?Q?U9/X6OITl5Ef9G+A4VnYIzBRaN5MtRTKZaM4JqMfv+/iuSe5vDZsNN/0gKvZ?= =?us-ascii?Q?BL6fo5Tt0cQ0J+Q7MjFrjaFBZ1GITMrif0WMj7ubXIING5aiuXvkwJkjum7I?= =?us-ascii?Q?PSb/kKbgL7godx5gZdPPpo/lLDdcJiX6RsbMHUU9R4MoF31SuX3J6Aib7U7V?= =?us-ascii?Q?pHCUaKcoflfNZgYbypUVg2kE8kLmXOW1l9scjjSxbCrpCdH4EK4qx3obqkiY?= =?us-ascii?Q?1y8RRYbYSYgmNw0SEqp1CgBKz6tKEBtn5xO1+lctoXK6z93wI7W++qRmHn9y?= =?us-ascii?Q?D82zOhu9Ha11EfkOFnvzIZyvsY6awbEfFGxVh7eC9LCAkF4+aPPemHv6EHQs?= =?us-ascii?Q?sT+cfQ3rOL3cmuCT9cBGMmRS/GzyN2yvgNpjLiSbNzMY4dhgYiT5+uWx+Imk?= =?us-ascii?Q?O4Gc9HAnXH4EsMJ89cu7mzNTgXP0aWR2yFF6b3WpitDMGFjC6zcFNC7RnDOD?= =?us-ascii?Q?ySUBQbcVFsUdijytNb6ZRDxt+ZRQTSZ6vid7248i1OQbhC1QaZRHuteNH7Qe?= =?us-ascii?Q?EC4zSUrYYV+PCwlnP9/KHevjgbCNQF1y09Ewio0G4e0RpWI9J0BxIEGlKI1J?= =?us-ascii?Q?kvezsw99fw0fgvaxzMQBsRrUmqgZaRfhdclfp6Q5qMQlRF0g+mP7RqSyE+mR?= =?us-ascii?Q?vj+D7/uV95WoIs0AaNO1qb/N8XWsVLmzp4VqkldsJCHnfzwSW386Olxz7h5b?= =?us-ascii?Q?sywPFUMFc211A8CAl16Q2mJkY31gSb/3qqe5r0Et0g3u8Nue5k7dbfpsufvd?= =?us-ascii?Q?3Tz61P9sCiZ2ySIglwrNk1sU6mgv1ivEwNMvM0rb2K73Z3lzs+5Ac83Qnv3H?= =?us-ascii?Q?gAtvviREv/S1LvRSvNpI/kSd9df3Q8ZeMrH+sDLBGYaAYHf4N0+YjNuwkBcO?= =?us-ascii?Q?znREWFTn2HAdmjCsIfBtuQ/NExjrxB4YmXUoDE2SwvvZTl2zXl1KK+oxIwuG?= =?us-ascii?Q?+9u8sxbRWfwrZ4r4VKUhapKlTv8A6P5Ia9heh4/ZIAqPgSzmRtTI2OaPXbk1?= =?us-ascii?Q?sRm8UzsIxajdTR8QNzIdxMfJfrnAxzhovxX9s3LLtall0BoGnbpEhWU+c94D?= =?us-ascii?Q?P7cMmtdphhc4sXSlyV3lMnEwYClR6y5EAyVERuHYab4nuAfYgGk+IWHnHWlS?= =?us-ascii?Q?8juWdJHYHBRMowm2/BJIC6slTRDnOXivH5lqs7eKKuvM4aAh4BiC7G14kIip?= =?us-ascii?Q?hD6lkhf2uXcj/NJtLVqHiyyiSC7SQLAU5OVH383CuZ8/8yv5Oy/ij6FlPK2c?= =?us-ascii?Q?Uk97s6JIjJaSclUM4hQS54hOSfuU5TJz4DIj72pHdf/tKeQEKg2++g0BfT+E?= =?us-ascii?Q?cEOBJjtQ0MhQUfbVMAKhNUVI8BsBY/tM0gZgxun/nhn6gkhvZ9CSLCcAniwR?= =?us-ascii?Q?q7nKUn5eUBnRiFOPD3PMTMVe9vnHjwCCbsxQObjse+lOF7iNkhBtDq9SzZyb?= =?us-ascii?Q?dmsZ2x7mQWPtLVFoVApeJgihgSNXBjE/dDUbJEG1ibrg74FPiVNnKptrUWsn?= =?us-ascii?Q?/KQCZjS6QemGuACi/nLFc445Qy3NFtE/Gbl673JSQPTfGHhPFX0vlW+rEg9R?= =?us-ascii?Q?0UoKx7Kn2A=3D=3D?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: 52d5f4c2-cab2-4a4f-4d85-08df02d93402 X-MS-Exchange-CrossTenant-AuthSource: LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 18:46:41.4558 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: EIU1VKPwS0qoEcOVbu3K/lPYWDG50aCG8En3SJwkPWv7G2VOa65oXgpRun4gCNbAAfKTL7HtaHkn5WVHPSb/nw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO7P123MB7501 Content-Type: text/plain; charset="utf-8" The root_domain pointer rd field in struct rq is updated dynamically using RCU, and its memory reclamation is deferred via call_rcu() in rq_attach_root(). However, struct rq's rd field was missing the __rcu compiler annotation, and several lockless readers across the scheduler subsystem accessed rq->rd directly without using RCU dereference primitives. Add the __rcu annotation to struct rq's rd field in kernel/sched/sched.h. Update lockless readers across kernel/sched/ to use rcu_dereference(), rcu_dereference_sched() or rcu_access_pointer() appropriately. This ensures proper data-dependency barriers on all architectures, enables Sparse static analysis validation, and documents RCU read-side ownership contracts. Signed-off-by: Aaron Tomlin --- kernel/sched/core.c | 16 ++++++++++------ kernel/sched/deadline.c | 8 ++++---- kernel/sched/fair.c | 29 +++++++++++++++-------------- kernel/sched/sched.h | 2 +- 4 files changed, 30 insertions(+), 25 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2e7cde033a31..8c81f7c0cea0 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8577,10 +8577,12 @@ void set_rq_offline(struct rq *rq) static inline void sched_set_rq_online(struct rq *rq, int cpu) { struct rq_flags rf; + struct root_domain *rd; =20 rq_lock_irqsave(rq, &rf); - if (rq->rd) { - BUG_ON(!cpumask_test_cpu(cpu, rq->rd->span)); + rd =3D rcu_dereference_protected(rq->rd, lockdep_is_held(&rq->__lock)); + if (rd) { + BUG_ON(!cpumask_test_cpu(cpu, rd->span)); set_rq_online(rq); } rq_unlock_irqrestore(rq, &rf); @@ -8589,10 +8591,12 @@ static inline void sched_set_rq_online(struct rq *r= q, int cpu) static inline void sched_set_rq_offline(struct rq *rq, int cpu) { struct rq_flags rf; + struct root_domain *rd; =20 rq_lock_irqsave(rq, &rf); - if (rq->rd) { - BUG_ON(!cpumask_test_cpu(cpu, rq->rd->span)); + rd =3D rcu_dereference_protected(rq->rd, lockdep_is_held(&rq->__lock)); + if (rd) { + BUG_ON(!cpumask_test_cpu(cpu, rd->span)); set_rq_offline(rq); } rq_unlock_irqrestore(rq, &rf); @@ -9009,8 +9013,8 @@ void __init sched_init(void) #endif rq->next_class =3D &idle_sched_class; =20 - rq->sd =3D NULL; - rq->rd =3D NULL; + RCU_INIT_POINTER(rq->sd, NULL); + RCU_INIT_POINTER(rq->rd, NULL); rq->cpu_capacity =3D SCHED_CAPACITY_SCALE; rq->balance_callback =3D &balance_push_callback; rq->active_balance =3D 0; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 857dbe3519a8..4137f8bbcef5 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -122,12 +122,12 @@ static inline struct dl_bw *dl_bw_of(int i) { RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held(), "sched RCU must be held"); - return &cpu_rq(i)->rd->dl_bw; + return &rcu_dereference_sched(cpu_rq(i)->rd)->dl_bw; } =20 static inline int dl_bw_cpus(int i) { - struct root_domain *rd =3D cpu_rq(i)->rd; + struct root_domain *rd =3D rcu_dereference_sched(cpu_rq(i)->rd); =20 RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held(), "sched RCU must be held"); @@ -159,13 +159,13 @@ static inline unsigned long dl_bw_capacity(int i) RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held(), "sched RCU must be held"); =20 - return __dl_bw_capacity(cpu_rq(i)->rd->span); + return __dl_bw_capacity(rcu_dereference_sched(cpu_rq(i)->rd)->span); } } =20 bool dl_bw_visited(int cpu, u64 cookie) { - struct root_domain *rd =3D cpu_rq(cpu)->rd; + struct root_domain *rd =3D rcu_dereference_sched(cpu_rq(cpu)->rd); =20 if (rd->visit_cookie =3D=3D cookie) return true; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index f79fcba4afec..d55c21811306 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7865,13 +7865,10 @@ static inline void set_rd_overutilized(struct root_= domain *rd, bool flag) =20 static inline void check_update_overutilized_status(struct rq *rq) { - /* - * overutilized field is used for load balancing decisions only - * if energy aware scheduler is being used - */ + struct root_domain *rd =3D rcu_dereference(rq->rd); =20 - if (!is_rd_overutilized(rq->rd) && cpu_overutilized(rq->cpu)) - set_rd_overutilized(rq->rd, 1); + if (rd && !is_rd_overutilized(rd) && cpu_overutilized(rq->cpu)) + set_rd_overutilized(rd, 1); } =20 /* Runqueue only has SCHED_IDLE tasks enqueued */ @@ -9500,7 +9497,7 @@ static int find_energy_efficient_cpu(struct task_stru= ct *p, int prev_cpu) unsigned long prev_delta =3D ULONG_MAX, best_delta =3D ULONG_MAX; unsigned long p_util_min =3D uclamp_is_used() ? uclamp_eff_value(p, UCLAM= P_MIN) : 0; unsigned long p_util_max =3D uclamp_is_used() ? uclamp_eff_value(p, UCLAM= P_MAX) : 1024; - struct root_domain *rd =3D this_rq()->rd; + struct root_domain *rd =3D rcu_dereference(this_rq()->rd); int cpu, best_energy_cpu, target =3D -1; int prev_fits =3D -1, best_fits =3D -1; unsigned long best_actual_cap =3D 0; @@ -9704,7 +9701,7 @@ select_task_rq_fair(struct task_struct *p, int prev_c= pu, int wake_flags) cpumask_test_cpu(cpu, p->cpus_ptr)) return cpu; =20 - if (!is_rd_overutilized(this_rq()->rd)) { + if (!is_rd_overutilized(rcu_dereference(this_rq()->rd))) { new_cpu =3D find_energy_efficient_cpu(p, prev_cpu); if (new_cpu >=3D 0) return new_cpu; @@ -12690,13 +12687,15 @@ static inline void update_sd_lb_stats(struct lb_e= nv *env, struct sd_lb_stats *sd env->fbq_type =3D fbq_classify_group(&sds->busiest_stat); =20 if (!env->sd->parent) { + struct root_domain *rd =3D rcu_dereference(env->dst_rq->rd); + /* update overload indicator if we are at root domain */ - set_rd_overloaded(env->dst_rq->rd, sg_overloaded); + set_rd_overloaded(rd, sg_overloaded); =20 /* Update over-utilization (tipping point, U >=3D 0) indicator */ - set_rd_overutilized(env->dst_rq->rd, sg_overutilized); + set_rd_overutilized(rd, sg_overutilized); } else if (sg_overutilized) { - set_rd_overutilized(env->dst_rq->rd, sg_overutilized); + set_rd_overutilized(rcu_dereference(env->dst_rq->rd), sg_overutilized); } =20 update_idle_cpu_scan(env, sum_util); @@ -12942,8 +12941,10 @@ static struct sched_group *sched_balance_find_src_= group(struct lb_env *env) if (busiest->group_type =3D=3D group_misfit_task) goto force_balance; =20 - if (!is_rd_overutilized(env->dst_rq->rd) && - rcu_dereference_all(env->dst_rq->rd->pd)) + struct root_domain *rd =3D rcu_dereference(env->dst_rq->rd); + + if (rd && !is_rd_overutilized(rd) && + rcu_dereference_all(rd->pd)) goto out_balanced; =20 /* ASYM feature bypasses nice load balance check */ @@ -14573,7 +14574,7 @@ static int sched_balance_newidle(struct rq *this_rq= , struct rq_flags *rf) if (!sd) goto out; =20 - if (!get_rd_overloaded(this_rq->rd) || + if (!get_rd_overloaded(rcu_dereference(this_rq->rd)) || this_rq->avg_idle < sd->max_newidle_lb_cost) { =20 update_next_balance(sd, &next_balance); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 26ae13c86b69..13a437032855 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1256,7 +1256,7 @@ struct rq { int membarrier_state; #endif =20 - struct root_domain *rd; + struct root_domain __rcu *rd; struct sched_domain __rcu *sd; =20 struct balance_callback *balance_callback; --=20 2.55.0 From nobody Mon Sep 28 05:42:23 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020115.outbound.protection.outlook.com [52.101.195.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A0753AF65B for ; Tue, 25 Aug 2026 18:46:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.115 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683611; cv=fail; b=TKzMmnVANX9PIcMF+aCoI7yZ2E9dW/syQmS55ejetlWywbN/3qLGhTQB2rmjdG0KYHajODpDvRI0o0hQRTgznceMkL4sRuBugOcdvzfDccbkTYO2dJSXwxJKJSdz8dAfeO+quXy2jKjvkoU4YxILVaZ4VL9BixbVR94We8Y8TQg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683611; c=relaxed/simple; bh=5ggOwnXG6FwY39eWmKBifexBY3w5FV3S+eN5Ith01pk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=M4tG5fbw0ckMPRqHIm+GLnyOqvl+r2P5hSt4zTTs0P8ZVQYFqChgiJ7i2p2AbisGJPWCMcckhyWHUzQ3SGJGAkS2Z4Y/S0Av3KPmRPX0EQcevSYwGD5kHIo1Q/BNslDWBi7Qn1jlujrkpHh7ejFvKuXX7KM5UhFIeYAJXwIDuFg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.115 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qoSMb+0kMEcrkPnRX6JLpPR3m/mg4svev/srRE9Kw2r4JUkRCb61iZBbSs4YjX4I3jq50eKxH9uZ5lBqKxVlfAoh12k1aRwo+Myqhk3iMMaPB5hg7k8Ze7RRbahIhUx05Kq6P/lu7ZZ2yGPp2ZPQc7gUNwgdSWvqNhtfQi6UM7u5t/6xiwpDmH4jkoCjx7fSInc4yhZ/jQ7mdPYfDl6Y5+rX2fYHm4whAXnuHfgVF6ftpeH1zaI36jMxpTJ0BQaNQy/Yu0qoFWzyn7ekHue4fZRXoviDF/q5rONGEsx7aysnLmHqfqDRxxv4LjXQT/CeqI6wxwN+7SJXKka3Resfqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=uu/34oQjqA7QmlhwVpJPpy5Fsip8pSdNq5egJ3VBJUg=; b=ypUVOuLphj+vGFrtZMI44uadSidjVsVWNGbAPFvfgNjdcB5W0CF9SmmTsQ3UPaD+RUDlWFmB+IAU4zlMgsZYecHZVpR2j5ydmNCC7dumKnGWsfVGng8dFOBjfGTeqI2EKoi8Y7Pq12/0A4k1LPIJomMMH/qpNWKF4uiRf/mJIFHQ9VvZvUz+/B7L/ZmQu7NMwBZl5LJ/iVuUIED9feUZwdhP7a/85P2q5kVV/6zE0ptDqfA5HCQCh4uVzl7zbdokmcamSLBm4ZeoE58lsEtf9zadzTw4CQ2FpNcgtVkH993rueRGNsXF+nvi06z6n3CvYkBE8wcKca8ItMgL3sumIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) by LO7P123MB7501.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:40c::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.8; Tue, 25 Aug 2026 18:46:43 +0000 Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383]) by LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383%4]) with mapi id 15.21.0360.005; Tue, 25 Aug 2026 18:46:43 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 2/6] sched/debug: Protect lockless rq->rd access in print_dl_rq() Date: Tue, 25 Aug 2026 14:46:33 -0400 Message-ID: <20260825184637.888364-3-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825184637.888364-1-atomlin@atomlin.com> References: <20260825184637.888364-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0099.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:191::14) To LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO6P123MB6616:EE_|LO7P123MB7501:EE_ X-MS-Office365-Filtering-Correlation-Id: 7f0079da-b12c-4d9c-0b38-08df02d9353e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|376014|7416014|1800799024|366016|6133799003|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: Js1Whz3arIjsntZ1toeYrmTry+DYoqHEdXjpKz7BnYcWq/7occs+k4TdkUaHi9VU5EemoVVYWdV9YFEk1fEGcgLMfEiao5tBo+ux8MYk34LJM2E8X++sVv4z20mBltf6VgWhpacAXJR63V7OZXYn8bWSDpHo/qB9E1kE1uKTBi19QQmrpmnvm/rOlK0/N6w5rqWKl1FlpijnHi7yzm9X5TAGUt4xzUlPPhvpgVxLfYPLBWtj++pZHZrGVyGstf5fukdC32/fL+kNTQwv0xv2wBOJ+P5Pj15J+pZIAL37oSCaKhMfDAwKDh0OjwFUnqTi/ToVy2nfo9D1SNSGkeTOH1MzdWGQ5p/68IKEevpHie7IpUYIjlS7GGAFWD0fjkG+o4WKuI7QbTfVozMA+KJT8TjBy2YRu69ElGBYVhsmn9tzPoZND6p9edonV13AiCuKDnqWLO0eq8RPEW/W2DQpJ4PLgmO5Q8bvOj744l0fC+Nq/fHFcI7bbY3yUKs4h+x5Gp8fMjQJV8GyNGNrfc57jbcocp4T1DCLpMuBbuDaY0EgJY0mP/6M5sGUJBKetxW7lwZjb5jx63G3N0JDctDxz0GdTwCejBac9HsuzJSC3CsmbjhlritRjgIJu2oefyZPolQ3MrZhpMRzhQGa7+d8X811bO8mMV4HDeu5O/wWk7w= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(376014)(7416014)(1800799024)(366016)(6133799003)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sgS0S+9wjuX33TKwFlDzJbmOtnxm1RU4o2wgHBTLSM9d/YmK2setjzV0L/7g?= =?us-ascii?Q?23KCPIpPkijdIyXMAI7h527ZBriDG8sA49FsFGu5xr3pIEw3nX9uVZIgw1tr?= =?us-ascii?Q?XJDSQ4TcLJYeosdIy7lqh8KBV1r+XWhFwcriC8bnCndWJqjmzIhonNGPfMQb?= =?us-ascii?Q?wI4SQe/Nv1RfzIr9ii/71gPB0bS3obQvuZHQwqPhWtmqXmkrK6aPpepAYZ1H?= =?us-ascii?Q?L2qGqeK85PBsEsnPT4KD2ZWrHpq6VpX5VBtsaJV0u0MOREkTPibKksY+7Ysk?= =?us-ascii?Q?QBkBn/0uwUpoSn6RYDA5A7/lq5bunExCBqlAfBx63vRdfu7UXCkppuv4dNSa?= =?us-ascii?Q?9snY1fsVJ7KpRqDx3SrJE2ZDgFBadPEvT7lqOnhFHmoILQc2X1WoAdVu5QUZ?= =?us-ascii?Q?28Y9Ijdg79xvOqWuHUY1saa2ojzndDqyyChjJYxd/c3iSpdIE6BaNHIQ6TO5?= =?us-ascii?Q?oSy6tfUBlMmCPkPUN/n7jPuMsd/ofRYWLO0a1UFI/rQkTiezupMZ1euL+4xq?= =?us-ascii?Q?JaSFKRax2l9jDR5NY7fkhFNRzynaJWqv977jSEfq/gJCbXt8+QUiaT4/u0BS?= =?us-ascii?Q?9ZDa6sYR+uqrsOyD7jLVHN4JFqhpM/vVQv+z7x8P39JVF72ZmQmT/SnjcUXX?= =?us-ascii?Q?rwl6loBeQDidYSZe1p5OSYI0VEQ1LZnQOhqsUrv70nfpQPYuPKC+2ZsX2LV+?= =?us-ascii?Q?Y2tL/2z4OmE9rXTKKfEvp2P7rPi9JV9YApwdBGPBIBM0WuS1pjESCQv1W1sb?= =?us-ascii?Q?noPwLz3MCyoo3YObnql/yywUTxL+qY1GT61Bo+98fgY4CzM5PeDr/tAwVl+M?= =?us-ascii?Q?48ARViNNjCIHnObqNYgC9Ic9JVT//VzzzbuIiRkOQbe0wdT8xkIXj+5nWhN+?= =?us-ascii?Q?nscMxswHYcoOFZXe4y/nGHcgWpxJxSfM72VN+7YO/hVPcviYY+5y4NXxnc4m?= =?us-ascii?Q?xrcHkMVBWb6JKaXyN+2k8bOqhpqFMkz1U4Q4jX9zHpZf5Z7zyHoYpPhNUYHP?= =?us-ascii?Q?dJ8NOeoeTqHsRhQ08GlUlpKIJMWltEy3v08kmVJfcjA2kYkQSdeog+djCT3B?= =?us-ascii?Q?xw9DhI6r/NouQgdsZdDpfMBEUQyq3IVC1YTXrzjM7enYIvU/Y4gR7hQs02dO?= =?us-ascii?Q?F+ukU03Gu0a+pILJLau1zsg0Wg1n+W6GBLEIm8P/crTq5SQNWUUkcTB8Kilk?= =?us-ascii?Q?VIxmOykTpLzXGAFcC+UJ8DE+qiIGp99XR90nUy+AKnJpo1B9LUoSAh2O0U8K?= =?us-ascii?Q?Ab8XxpWFJuMLFd7ostMTBwwh1fR+ZSXtQPWJZW0pK8qkUWScbQ2gcQp4LnhH?= =?us-ascii?Q?L+dMbXIbRiKF8NDTfTG04nayuE1YhAvP+gNASQeVRjW57hJUnS37Wvu/KWrY?= =?us-ascii?Q?+Y4RN7+TT8W0JAadBzwk7t+/C2qlsFjYjcVGX+DRiHQ0hl7akdzAMHMIOFxn?= =?us-ascii?Q?gip6rXXqDGefFZUj91pnad4v60waYCFVc21QtDu/Vge71OESYySvlak7lRbz?= =?us-ascii?Q?P04pDOt7snR8WFLQgh3Vj4p6kEWStb+QiAukwpS5pQ2nSjk8ZD3aghl/7cES?= =?us-ascii?Q?zplyBBYy2ESnVBjiQ8GXqBubx+MNrsfVTRfBNgVMtJwpgUDtyHTRXbuoiIYO?= =?us-ascii?Q?3MTbPfFz29XfH/P95wpUhaXpkY91iWOB5PLDukQu5O7ZkOPYPlDwPKgfW2tC?= =?us-ascii?Q?P2SEyLcy/ynN8vpdpAGk93RztPAF8MQWlb5MrNs4nJMW7I64wjXM1jNs8BnE?= =?us-ascii?Q?Ohe+SxkOgg=3D=3D?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7f0079da-b12c-4d9c-0b38-08df02d9353e X-MS-Exchange-CrossTenant-AuthSource: LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 18:46:43.4947 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tnCv/jjUjSwOw15yZPUDstdUpM+XtvOTrOsYFOVBuBXrCQG4W0aUmQCQiNSAg7r/Ks+uPivpDOiXTtERFrJtYg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO7P123MB7501 Content-Type: text/plain; charset="utf-8" In print_dl_rq(), cpu_rq(cpu)->rd is dereferenced locklessly to display deadline bandwidth statistics. During CPU hot-unplug or cgroup cpuset repartitioning events, partition_sched_domains() calls cpu_attach_domain(), which executes rq_attach_root() to detach the CPU from its root_domain. When the reference count of the detached root_domain drops to zero, rq_attach_root() calls call_rcu(&old_rd->rcu, free_rootdomain) to schedule memory teardown after an RCU grace period. However, rq_attach_root() previously updated rq->rd using a plain C store without an RCU publication barrier (i.e., rcu_assign_pointer()). Without a release memory barrier on the writer side, CPU or compiler reordering could allow the new rq->rd pointer store to become visible to other CPUs before the initialization writes to rd->dl_bw are committed. Furthermore, because print_dl_rq() did not hold an RCU read lock while dereferencing cpu_rq(cpu)->rd, an RCU grace period could elapse concurrently while debugfs is reading the file, allowing free_rootdomain() to execute kfree(old_rd) and causing a use-after-free race condition when print_dl_rq() reads dl_bw->bw. Resolve this by using rcu_assign_pointer(rq->rd, rd) in rq_attach_root() to guarantee a release memory barrier when publishing a root_domain. Finally, fetch rq->rd using guard(rcu)() and rcu_dereference() in print_dl_= rq(). Fixes: 02968ccf7b80 ("sched: add /proc/sched_debug file") Reported-by: sashiko-bot Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 3 ++- kernel/sched/topology.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 72236db67983..b8be86bf8849 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -1172,7 +1172,8 @@ void print_dl_rq(struct seq_file *m, int cpu, struct = dl_rq *dl_rq) SEQ_printf(m, " .%-30s: %lu\n", #x, (unsigned long)(dl_rq->x)) =20 PU(dl_nr_running); - dl_bw =3D &cpu_rq(cpu)->rd->dl_bw; + guard(rcu)(); + dl_bw =3D &rcu_dereference(cpu_rq(cpu)->rd)->dl_bw; SEQ_printf(m, " .%-30s: %lld\n", "dl_bw->bw", dl_bw->bw); SEQ_printf(m, " .%-30s: %lld\n", "dl_bw->total_bw", dl_bw->total_bw); =20 diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 21e816ad23ee..40c3bcf82a68 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -496,7 +496,7 @@ void rq_attach_root(struct rq *rq, struct root_domain *= rd) } =20 atomic_inc(&rd->refcount); - rq->rd =3D rd; + rcu_assign_pointer(rq->rd, rd); =20 cpumask_set_cpu(rq->cpu, rd->span); if (cpumask_test_cpu(rq->cpu, cpu_active_mask)) --=20 2.55.0 From nobody Mon Sep 28 05:42:23 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020115.outbound.protection.outlook.com [52.101.195.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44C373AE1BD for ; Tue, 25 Aug 2026 18:46:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.115 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683612; cv=fail; b=CHRrqiSKr33/NMHL8ArapFx0jomTiguQmrqi3Edh1RD2iu56aqnJA3MIJ2kDoYgLZ582X4CkkzEgcsVgKRsBbX23UrMbbLyBmTmb2LDclK7a2R83UdiIWeOYBoN5Fs04532O7kyCbifGiye8Lv72GsFn3tCQLE7VuAY5+S1OuMI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683612; c=relaxed/simple; bh=oCpfRzeW3vnLiZYXl9KF4nlo4FqZ8G5BMJbONYFGHqk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=cnEwBmsjwx0XZqIMxx9tL+guiyU/aSHYR+S0BkDjCVnsvGTl0euYyBgAmXeKBSQ9hE2N2Mu+bAyEThlm8YQFTW1WSIXMXnr+xIQEvoglM0y8WQysjhnxNp19sP00IfTtWAsN+wenJM9LrYnEhRkcEzNWPOXxmSMh15w0CCncWFI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.115 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=zPE6ifKzOqbq9nOX3OprpfqLYDkLGPnblFvq9RjJQI/Ov3bmZB4g+XQ/f9LpplH2FKeduGiD3BQSJbJDsa9PGfZREK3u7OM0RQqNLwEZR/8yLiF7rDTr6F9RcffzmKjq2YRVZyikj4mZM0H+nRET4g1gHc0V6xHMPF4kORly+84hnw316eLzDaTK8NqjmIKjnxX2x5Nr1YJp/4ndyBhUTy/B0cI736j1e1PEJjOnkNkzjAILaCiET7M+sdRk+WsYCsSeGRonvfV+RlnwHndxVjw+Glxbj/2+yBP5ad194yeXEFm7A3o1dyPQLVc7b1AzSLnFOvmgVDrEl25c9xYJ4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=dB/5EqRnB7B53vImUMVvge0SKpUE/VZxSsGjvVV0jHg=; b=voFfOTmzah09Ix28pSKQAO+8b2OWnQRb1F/Kcc60hM4B2M/Tzr/s7KGyVMiiPQclwL+gRZuKnLkNHPmqKmxmowSolCl2SDXUa0NveCySt+qDSdgUtFmokvkpqwIN8+N6+fDBB/+9IbQSSXb0g9UtSkmCp5mE1ubNqLMz6Dlg3ecW1QFIfoRhscil/Pif8Jqr3jTqYDznST35Kc8QP8zWJqnxOhTVG+T3pFftcWkNheC62JuzasXbt4pUBVozOM4qYSdEH9A8f0ZLX3Os6NhYwlAtqkfJsN11vcGp9gbgbvwtPdaYP+TAJdp07IkGPSj8XolOtDRYd6M8OE3Ocu/VQA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) by LO7P123MB7501.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:40c::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.8; Tue, 25 Aug 2026 18:46:45 +0000 Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383]) by LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383%4]) with mapi id 15.21.0360.005; Tue, 25 Aug 2026 18:46:45 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 3/6] sched/debug: Protect lockless rq->curr access in print_cpu() Date: Tue, 25 Aug 2026 14:46:34 -0400 Message-ID: <20260825184637.888364-4-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825184637.888364-1-atomlin@atomlin.com> References: <20260825184637.888364-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0107.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:191::22) To LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO6P123MB6616:EE_|LO7P123MB7501:EE_ X-MS-Office365-Filtering-Correlation-Id: 094bb048-a1ca-4224-d5c8-08df02d93665 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|376014|7416014|1800799024|366016|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: XRRlpz/lEqTQhcBmPzI1rQ9d7WqN9D0qzGBF7ISAqsNYSPLE7z/0MH+9GjagwmoUY+JrlKdZ+anVtLaAlb/3aMcow7Oghdeax8FtPSCfsEunFA7dsgHSPcuJpcQbegSukIv5AcxCRBUHCXEMy++AkHoB0WUjy4Ha1ZrlxYx+zh1OfJFY7Y3jlLmtSnDC0GSXkz4ssL+CB8M/aMZjFK1KZ/Xmj+nNgsEF+lKgfMOW8UYZ9o5hLqOjZi0S1nV2JoCIOuPBloWpZdtDXq1iDEgtDAn4mec1enpOJqS6dz8ShK4ELEqeR/PBndHnFRkuJxo5tf1lQdVqWnL55xa+pwCvRRaVp3BHsHWWV1roGdxAOF5U+8b3NSk++jS76MtFGh5MmXUNlemjLFnu/DioCnKeqnV11FiFapUiJoxTFe8rkJrswwwfthxxqZBfzg0Ox1AELGp6eCzxPeTI6mlXHivBYbznK4DNcJdFYNRBovH0FRXvmuN5h4qE2nELi1oEZPaFzIbGbSwUX9Au9jMjclg9TbTJE+Xs8Q9xSOu5lzqa+NJdpz1OmGE0M6xykk2+006MNb57Joj3miOmCS0LjiLRIVhXSr10f4a3kUYLnYyLppey/aoJ3X/sj/RE5L8iWwjOyHcvy1IkyfQL1vW1nW5hKB5l0RYqXpN43NsGtL8vbJk= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(376014)(7416014)(1800799024)(366016)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?k1NsgB41akniUqME/Ld70VgV8u+EtnJiBseKMvxVYIauKVsWOBC7sCgH31SY?= =?us-ascii?Q?hlzUz5BibhmCx4CbOP39uX+FuiUeFQgbzsCsB/XCEyXel7fS7pi278H/1vGJ?= =?us-ascii?Q?O+Of+7JtD4AprTbc+Mp1GPDGtRgoQxPVgpK0l7VTY+AColfT77EgYjRyDvMy?= =?us-ascii?Q?YILfGiXxQCz10G7l9FGhIiWKNmSeT/enBK2etMtwTWSwcgvq+W21MKyUZYoT?= =?us-ascii?Q?bMLiT55djd1x96+i9GY4iPbnwcz8vnXRP6WP7EUVTQirLFPEzxuDox/4ROBp?= =?us-ascii?Q?hckpf01uKyqe9Lupqr02sZ1gSt2huoJB6DbIIxvGoA2EwULkwUYfqz7i8+9g?= =?us-ascii?Q?xT/sGSJxP7VWyCEHRudkLSpyLrCHfQHsi0DTowMShP/ac/UQ4KVLQT402GqK?= =?us-ascii?Q?QgDPnPYnBZxQsYfTkvigO204yxGAwm9uIEt6wD1HN4fwIFVP8iyzw92UcnEr?= =?us-ascii?Q?ZEhxmE2/A/XVuoy2SjLPFwNAhPtJ2OLqG57KOesqjUEQjHORrH08miygTVe7?= =?us-ascii?Q?L/F09okMf7Zi8klLCWBWg7z0MDDLWOINM8J6NXpjTSrSoEmVaadOZfobbCoc?= =?us-ascii?Q?HRtTeDMFEWxNba/ZkYq1R4pDSCyCNtqnSMkVBtQp7v3GWG0a7cq0/2s27Ekj?= =?us-ascii?Q?NZ1xfv6KetjT6Xc7BJpj6FWIw2O4c6ynhIJuCYDXynp9+UhuPjH+aSd8jY5P?= =?us-ascii?Q?rJyyUG1FTjRp+Y85c8ybh5xiLRmgNBeTwV2zVkCHHHpMkjPKJ6IKr0f7dzvA?= =?us-ascii?Q?NCMWCOi4uNR/kfDht0B/L+T5WmFSTCg8ZG1N/QJPO6jxi8PNhki/EZKP03Td?= =?us-ascii?Q?Fz6YiZQoxSkdsjT1y9stErMKc/52OdtDnsvJpkDvFRRDzuqLdi0nEgHHfFWG?= =?us-ascii?Q?g8Z09ubDQUWpezW2ryDGem3HSnNDYNzVYZDcOJEOgk9To4u9BMyXX4ma0Lfx?= =?us-ascii?Q?B6IhOP2250oonIq284F3rycnHoQKntXlBsdBgZs10rzhNrX3DaisgaGB1yuK?= =?us-ascii?Q?SdXY7XlBaQuwfbSRU1vsbISBvlG+k1kKMhqHYBlWvijCe8gNMmTQ8ZMQN54k?= =?us-ascii?Q?F/j8a01vMuGZwWpw2Vp0DW2AMcaZIj/O4bOvooffBUYLgxvpd4tIrGgzUuac?= =?us-ascii?Q?ibn459js+YnwU6qcK15q11l6STzJLjgDrxf4vnjgfBuD+fFx8n68YCHH1sor?= =?us-ascii?Q?EInTfTvhBunCm2T+pKQveRRJGFUCXMvAwV7OialtqTukdtAQTD+4fF98ZI3E?= =?us-ascii?Q?PcOuXLEcOvvaNYIxmgPrPIC9KJdxQ1cEVozMy4eoVJ3lw7bX6a8APZwgJOMk?= =?us-ascii?Q?Qx1jiDbkKtGuu/Ewl7yHry+JotBBFR2QJ8NOOtii4Sc6OHnKS5mVUPngYKkk?= =?us-ascii?Q?+HBulug/YoCKDLi9Pq6Q/cvIkbT+CU5KZiVK4NbUAyQhTntPvKwyhofrLITJ?= =?us-ascii?Q?tiDI8nPOz3kG8GMxvIg2z/NB6ANv9KGkC5gtJKRgSSMXS0XE96J54xcBvHyr?= =?us-ascii?Q?LAXIOttcnHV/AJDj2eXv8Jmj2/MBCjGp31TCkzbFy8RG+uFtY0YfbwBAExKa?= =?us-ascii?Q?wpJ937LUXWm+JQNj2hIuoqzrE8cc4m1+oElNPyZjrDwjmYZLoyq4Fsc+HO48?= =?us-ascii?Q?l5GBICNsKbvkt6mpDzWqyMCCFlSyYbQUBf/k0KeoFZ6P6LMAtkkiKYPt7ElI?= =?us-ascii?Q?pQHhKnA6vuBVbQq3r6bQLYmtnxWimipC3H1q05T5vqEiJoyq?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: 094bb048-a1ca-4224-d5c8-08df02d93665 X-MS-Exchange-CrossTenant-AuthSource: LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 18:46:45.4348 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JDsTkbD8MiHMr+KtxXdWzyS9sbn1qRWK43m7/jLqPvnnu1Nh5EnxDNrAIaF25fzapmhSVbJRuRTaHa8RmtmAGQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO7P123MB7501 Content-Type: text/plain; charset="utf-8" In print_cpu(), rq->curr is dereferenced locklessly to print the current task's PID via task_pid_nr(rq->curr). While accessing /sys/kernel/debug/sched/debug is inherently best-effort only; rq->curr is indeed expected to change dynamically while print_cpu() is executing. However, if the task currently running on the CPU exits concurrently and its reference count drops to zero, put_task_struct() calls call_rcu() to schedule __put_task_struct_rcu_cb(). Because print_cpu() does not hold the RCU read lock while dereferencing rq->curr, an RCU grace period can complete concurrently and free the task structure via free_task(), creating a potential use-after-free race condition. Resolve this by reading rq->curr using rcu_dereference(rq->curr) inside an RCU read-side critical section. Holding the RCU read lock delays the invocation of __put_task_struct_rcu_cb() until after rcu_read_unlock(), ensuring that the struct task_struct memory remains valid while being accessed. Fixes: 02968ccf7b80 ("sched: add /proc/sched_debug file") Reported-by: sashiko-bot Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index b8be86bf8849..2949e6ce753d 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -1210,7 +1210,10 @@ do { \ P(nr_switches); P(nr_uninterruptible); PN(next_balance); - SEQ_printf(m, " .%-30s: %ld\n", "curr->pid", (long)(task_pid_nr(rq->curr= ))); + rcu_read_lock(); + SEQ_printf(m, " .%-30s: %ld\n", "curr->pid", + (long)(task_pid_nr(rcu_dereference(rq->curr)))); + rcu_read_unlock(); PN(clock); PN(clock_task); #undef P --=20 2.55.0 From nobody Mon Sep 28 05:42:23 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020115.outbound.protection.outlook.com [52.101.195.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC59B3B0AFB for ; Tue, 25 Aug 2026 18:46:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.115 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683614; cv=fail; b=c/CERBeXWBIUCwX9YHTaja5vUgdUf8WVT9FEC8uNWD4eJW/yPy8Y9ZSm7g2n204mUbtgn5rM22sbgJ8QE8iAgPg0Hp3xmx4rvbavZzaPbIKVJL3X39RTTqW4iDsw3e9tjUCu+qpI+Oi5qYXVQ4EFDcTg/7ll4/cbvVkxXJ+Tk80= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683614; c=relaxed/simple; bh=AOvXL1FiZbh7jBBHxWHb8ru8JIYgeWp5gL48/7CwlNU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=DXrlbzSkGUlZG82TDUcKFXa8oKpakO3eIPH9RONKImldyb3mVAaKnO2gdADzlTRBdn0eSGt1gls1QvaXuOsnoU/ef4aKgYDZBFCIy+wuUZLp81uMnHqzf9As184RyN7LFxEcQNnIaqSLw/XD7z6ge5R8wXN77OpHfYt9dyotHzk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.115 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RW5qp1b9XMTUCOKcRWd/toBeBJAptLvXMvrd2S8XZNkEwOcB5rJwhXMfgmwS/xFTFdctVPE9qsi3hUl8A4ON2pWZUGavKLl4VvzP8gFjcHK4lhbgJIqHmBm5nI99X/zWakRGij4Lg4L2vq1xkdlF0yjpunnfK42TEam7sR9hA8uO+R7y4wj617zjKAMoPXwgQi18YgVdl4G/Ij2vVLpDgcudAsmDU+83wgxdFY6U7lLshUvgYtvahYnf0S/iyk9GAjVMdttD0wnqahc+7hHxxww4UHmeOBmrLlA5Kq+Kmpcyo18wayfDm5nwnYUKyifQ8lI0KLMFY48BfBNkDCK9eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=gZodYWiNrUAcsvo6eVOxBWVXHILnkwJuR9fjL6RKnGg=; b=s8+33gGH+Skg7WSkNprvxkG1R76BpGzC1eHy21q6TJMXpBUOOdXyMIVaXYcICzqCX89d0USFMJKsVWfAffe7E1YM4IFsdRVxNW6imQ4qe+7+xPoeH3XyChf13X6NrffNhI3K8jD2LCIuqJC+aDgyAId1hk7MFepPUbVzKx//DABLMJn8KAQVyxxFhGLa7lSh5z4DDooe7vItgI55mFAIaA/g99Fe8eNgJSlCGReYM+xjZ6S2M0ybW+M3PThtYAOYaEJ0EWTxh0LJegDlkY3aChjgPFqPnRSDo9dhqd18B/UrMCjQ22PW6XqDKW24i7zSubJGECqq/grxWOgcc7LGQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) by LO7P123MB7501.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:40c::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.8; Tue, 25 Aug 2026 18:46:47 +0000 Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383]) by LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383%4]) with mapi id 15.21.0360.005; Tue, 25 Aug 2026 18:46:47 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 4/6] sched/debug: Protect p->mm access in sched_show_numa() Date: Tue, 25 Aug 2026 14:46:35 -0400 Message-ID: <20260825184637.888364-5-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825184637.888364-1-atomlin@atomlin.com> References: <20260825184637.888364-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P123CA0088.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:138::21) To LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO6P123MB6616:EE_|LO7P123MB7501:EE_ X-MS-Office365-Filtering-Correlation-Id: 120193a4-8649-4889-5e58-08df02d9379e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|376014|7416014|1800799024|366016|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: /PocrnE/b8JQtz4trJ+9OAeSwIof5JgSu+XMAhmx0iM8/rtxnH2jmaJ9KrpgKOWgNpkBL+VBxBMnOyDd0Bz2Z+2yv62rELWY7czs2FkKVnmS3jaQ8589inXIjb/et/Jp9Xx9hfQzopZH52BfEFXUFN0XB6p+hA7fEpAUa+168YA0r4Z323HvL7EPdMqswT0ThBPaWVCBWdoeLsiDIGh0k9md/2Q33kPxk1xEDFLa3ZG3ob6KrdpmY5XICNUnqNwPkVlnmZm0tnSGqHACCGQm7gthq2uflJbpbI9bOVIRiTh7I88tNt3XTfVCPJGTJOZn+8/PaHpnffdcEcTTUP7qbfQIGLY+WMwcrG9JCDQquJGDAGmrDANo51m9PZT5G4v3Opz85HQD8GEvfc4kPWmtD3GYacNwVezMQlpPpPibZ5ss1vuQEOYBtpaJuxkqL1dilJIAOAwuivbrPpL8tJsQTMDag1ukXm7GGvhEjRShKvaPyr4SPCTR2aEaq4dbkYW4lgOPaA3pvi+z7IpjbSiW37vBOE/dps3BjLSEk9rFLKnEZdxayu/urkOMiCFm8ub/3AE/cJR1Uv/tpTCvZbAeXxcIa074P3NCOXMMb2YOnBTLDIsq/TTf03GX/+Y7hrOzNb15HePBEONcJxsKEO/RKi11bTqAgqYB6xS4HuLaDPE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(376014)(7416014)(1800799024)(366016)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?97oEb/JWWapU1/ivD0qWYf9aAdcPDodlqTtefUMT9QOpU7ub0u7aobHCgd2P?= =?us-ascii?Q?aluEwcC5RLayOocfsqvmbYRCQ8o4p5uX5TBCHVpKrNujRejrThIHfwQFKmZh?= =?us-ascii?Q?ZZt57j67YkBrbQ/BK3BmL5+38CL+3dWdpxFs/zb7Uw1wf+BYW6FlE/tGSa1w?= =?us-ascii?Q?vxGl1qWh3ANoYLcn6w5PlVnbJsh/3++sCMlftL6CWoMP3VBPos7X3LMj+m2P?= =?us-ascii?Q?pSLn2CHO58/WB52nRI3L5/+SjYwQgQLB+3JJNAyqsvmVsuejHPFaPi2xYl2T?= =?us-ascii?Q?JJAMAENJbOjyi7ZfFm84OtUsUrbtJR1I5eLCOEqpMZ2atoennf1fYLwY9eka?= =?us-ascii?Q?uUuWQxqVeouS8fhbeylwtRBmYsBfIx5wRSlTQNB0UCTrEcNsWidEZAN5Kikj?= =?us-ascii?Q?sUGsO4slRkm/o93192UFC6y3yAEfBFuWfC/stLs4kFsCFlWuojqZ2YEVKlrk?= =?us-ascii?Q?MshfzSvOJ6yoLXOIDitki5rSZJ9+MYofVMXhm34SwAPTs5dEdMg1GdxTvXIL?= =?us-ascii?Q?NRDK3PZXo9wMvwGLJnUbRwchDiMgYM2APbCT5s20dcJwuavfG9Wq7ByGfQQn?= =?us-ascii?Q?/nefUE3PcVDOT0dQkjfhgKYLTFxS2eSR+D3gVeixljgFQcla2f2Lyklix7pr?= =?us-ascii?Q?ZNc3PDaiFee3LNVnW18M4Yqih0iSB0WoLPgAhQYpXrnTaFBImpWDGU49QUde?= =?us-ascii?Q?XXfL0v/2YKx2CT0Ky5TS8sUpk6mqW/LjTu3wTPS+/AFWEcpN5KSDGg4Ka/mT?= =?us-ascii?Q?IKeF+okU38sQi8Ms+j3mAh/8CYqMBB7kv+DVJp5zmfB/ns52tIf+R/UiVRQp?= =?us-ascii?Q?5ZLQ8OJgXjV2dSkEAUfCRlv2N9wekif6BTC8FRfgdf8iypLcqERAothvkCyG?= =?us-ascii?Q?2HMS6QHZXFFAnjJVvBDU2cCnUhGtl1KeKyQM+oA8lwADcc/LtExM2FCd9Gem?= =?us-ascii?Q?0thDltqjBY7DnJgYQ7wHU2xMGfFRjXM9DBZpKK2oidCKkMVMsYl3S3z6baUw?= =?us-ascii?Q?PEvXTQGafOrI+TjAqRvI3ljufkQ7nmMvsCzEOr36/6wIlfTspf49CF+Boutr?= =?us-ascii?Q?p0TWifg8TzcZfoj88+Lr9ndpudDHS9dVlypkVn4G3AmnUKxjFt0b4njXOnqL?= =?us-ascii?Q?/ZnnJ/AKvCWgk7y+wPGUEKGD/vLhZE3KnLFLtkM7NDtNNi49CG1ernE64ZoL?= =?us-ascii?Q?aSOTiqQ3v04VRE8qwh+evr6kePhCOJOHQe2oZwmDKCTRXVYV+VdADYDUymZr?= =?us-ascii?Q?KnVwD9HXNZ80lSlJ4wRZCyHMOUkqHT1N8Q1TH/jmO+qe8gW6P7c01AQiSV5P?= =?us-ascii?Q?ZDrKnwb3SA3xL3yrG0fkDx8Xpy7xZBSR4erVIQzxwZv2RbzVbVIamE66D2Xj?= =?us-ascii?Q?x8fHUZjHp7T1aED++unmhL+E5ly5RQcEyOACmEqRHSiD0hLw2WL+RvJKignq?= =?us-ascii?Q?dLoYjwiFWyRnIt8mRF5wTCACPETOjmdoTam7bUdGLOhZ79rEfNhVRyCQjKZY?= =?us-ascii?Q?50lpQSrtHb0/LqjparsbH6jK61UUuOpxHMNDFb8j67J5a2/KgR7GqnzhjIP+?= =?us-ascii?Q?2xYjvD4U66OiJSzZdX1geXJiFnh0tvlFyX/GIhmvDo2Tap1ReVGaImeIU6t1?= =?us-ascii?Q?UaeL3kcjHOD83fiU7y9v1BHDwogTs11fnpdLFotNKjg828njBvdOyOTh04Qy?= =?us-ascii?Q?3M7SHhwk7KlkQVpNXmqbWsdwSd53IsJvnutDGXZUtyLwdwDQ?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: 120193a4-8649-4889-5e58-08df02d9379e X-MS-Exchange-CrossTenant-AuthSource: LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 18:46:47.4997 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9VdBWmJxoXOXEETgs0VkSEDdXS5hFxVt6M74NNEaaUNg8wvAEEheQjToyE4HagJwKUIdhQnkjEiWfmKzu8TgXw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO7P123MB7501 Content-Type: text/plain; charset="utf-8" In sched_show_numa(), p->mm is checked locklessly and then passed to the P(mm->numa_scan_seq) macro. This presents both a time-of-change to time-of-use race and a potential use-after-free vulnerability. If a task exits concurrently via exit_mm(p), another CPU can set p->mm to NULL and call mmput(mm) to free the struct mm_struct. Dereferencing mm->numa_scan_seq without holding task_lock(p) can access freed memory if mmput() runs immediately after the check. Fix this by wrapping the p->mm check and macro dereference in task_lock(p) and task_unlock(p). In exit_mm(), current->mm is set to NULL under task_lock(p) before mmput() is called, guaranteeing that p->mm cannot be set to NULL or freed while task_lock(p) is held. Fixes: b32e86b4301e ("sched/numa: Add debugging") Reported-by: sashiko-bot Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 2949e6ce753d..2a01854f0f1c 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -1395,8 +1395,10 @@ void print_numa_stats(struct seq_file *m, int node, = unsigned long tsf, static void sched_show_numa(struct task_struct *p, struct seq_file *m) { #ifdef CONFIG_NUMA_BALANCING + task_lock(p); if (p->mm) P(mm->numa_scan_seq); + task_unlock(p); =20 P(numa_pages_migrated); P(numa_preferred_nid); --=20 2.55.0 From nobody Mon Sep 28 05:42:23 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020115.outbound.protection.outlook.com [52.101.195.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDED43B0ADC for ; Tue, 25 Aug 2026 18:46:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.115 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683617; cv=fail; b=nj/Dxooyp17hlmLXS5F7CK+nILAfJ/huYYwOwEk4yPOxqK+dMDHN7vAHXb1JwDgfHXRJcX3gyQluQ1HMfFxY3DHqC7msb86KeBSTWUrt0nSDiJ/wKMDY9tBEkRco1BTxBGbpsuGeNEAgmLGCoQWgdUIXW506TIiEZ+BMvDdMoac= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683617; c=relaxed/simple; bh=4VSgZFDUgRPDe99dqjzcFRpxGhKtKbIdQqTIJkjy32s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=tWg3X+H5s+a2yYaSLj6mYMGyiD5P2Jm81FUGwKGRLYDqOenZjFJXTzoa7s1agw355ikrte4eVo161yZVYPH8JrMlAV3KtGzoJxXzqdWRqBzu36x25PFQZj0cU0wis7Yo91tg/w0o/v5cxptV5HbnaDG9c7bEWzqx3+42NSgAUKo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.115 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=whriObpCk4lZPQdvFsJChcx+G9L35MAr75LxDGL5nW/nhKjh2YuQDZ2Td4jGyUgt63hcQ1Z1zWEGXVJTu4NMeihhzIZLBnZfTXraZ9IJ9NFnmzmGBlgqa319CGAkeBFx+M5KuVEY4EI0bQfvCjej3LQtQsxxiXDgdwYohqfEWtNJxD8BGpfxTk8UQojC3SH+vgzsW9nPQ4W2Bk/YdnixMMULcWLt13bkH2BHwoCKlNZAkIeFPZs4S5vvgUyaOY9cMe7RMwjULX60E18pBNJXc8oZvYi1jAvZMkV7ttdBgGc00g6aPxYkRx5X2Y6quPss0i2tddHme8OqQj7W14A0rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=dh1C6Mxxl1Fpm77Xcb9fC7AS7tLWN/bdQRddm5yid1k=; b=N2UzzmaJXpPVLpTnnNP9f5SguPmM/KoHFKsePsr1/wz193PbIlTOrdTgn56zrKnLdTDDQrwvzBFnKNuXZCKqLZ20U89/94QpmzM7jt0XKZbchVhG8bzMqdIkS/SaEmRLPk8rEKpAPHQwdkNeOY2qEuEx5TH9hLtDRZhGHoL+k7Zc+MOPtIO8xt8dhIVYIyzne4jIdPiuG0h8Igba7lkaZ3RZAqASf8jIzy+F9YkMAUL89Qi9l/AlU8/my8y6PSfIKHd28ilPmmsIFX4I1MPJ3EcUjWkeABKfwQp7VMRwDPi1f28NG1AEBsroc1QTOZkUhn1B+MLCaXtcU7m7cZKlog== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) by LO7P123MB7501.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:40c::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.8; Tue, 25 Aug 2026 18:46:49 +0000 Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383]) by LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383%4]) with mapi id 15.21.0360.005; Tue, 25 Aug 2026 18:46:49 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 5/6] sched/fair: Use list_for_each_entry_rcu() in print_cfs_stats() Date: Tue, 25 Aug 2026 14:46:36 -0400 Message-ID: <20260825184637.888364-6-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825184637.888364-1-atomlin@atomlin.com> References: <20260825184637.888364-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0071.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:153::22) To LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO6P123MB6616:EE_|LO7P123MB7501:EE_ X-MS-Office365-Filtering-Correlation-Id: f0c94a6e-f622-4b1b-5184-08df02d938dc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|376014|7416014|1800799024|366016|3023799007|6133799003|10067099003|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: T6bc2h2KOhvp2aEAoUxsuL8NnH9I8OO0wiANByn8PTF5b3u+7WL3yFn1piimUTtZNKznkQTBH1I4+UUmeLCX+WOqxHL4HXzF+hp84LLvYzywtKYYJgw69/NboQXl+Ac/6FVpkcE07LnzKH1tlNS/sgMB5sQb+0a+f2ozWc/B4ZDvB7KFPc47IrpEH/lwdhfhBEbvMjV6E+UcxUDFIFTvQM6JSANSyq/gO1i75FA4otP2LJTcwD1zC7EZZ7v6DyNw8QqJyFmz3GwBQzoqMsFv9/eeiYsClrJwKWXqugen5fSmJkLmwKs/TEzPTRWHplsh9ybWXr1x/LreAfuNnISY9TubVL7Njezflpj9JB1hOY07+0nQiWnoeg27nI0AERuuf3OASJklauVKK5v98jXvJD6d1brRFYNgCxvz4Md12S0/A8G7nH105QH9cMNmujYBXQmlWNfwNarsyiwcgSDKJdjF2zw79GpDb7GSiEvUTEDIcxz2ccVnJyJX1UFeSCqkYynLygi4xuOF+lVU33v5as66ohUsL85eJipZeCeLkAkj/RaMzkkWjeib+vNcsn87kdI+iPfR6EVs5y7OABAhdIefuTW5nD0tJl/Re9dyZuSp/9Vh/aWwIxBRVPdyXNALxZlHuBsCgSXbsNK1nhgmo4/oCpzLYh07dBisQn5khYw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(376014)(7416014)(1800799024)(366016)(3023799007)(6133799003)(10067099003)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?NvHVYOq3JwyNuHPXmz6I+xS+gS9N8uOKAWUNT44OsQMNbJgLFS5SXth/Xeyy?= =?us-ascii?Q?0gHxh+Kz/16tdhakuYxDQItfzTxr4C774yBaPyvICfZvd1ofdjM4XHstKWHD?= =?us-ascii?Q?QFY328F8a5dAAb2wKjKSeKLh6YFnscrR1TmmHiqZzcIGiZ+qZrf0rIf8kwKI?= =?us-ascii?Q?tbE8CH9jWIm4NxX7cmlVDv1Ecu19ZMZj7txR3b0QScDPUmqpi7+cIwa0AI2D?= =?us-ascii?Q?eDw90IFeixzG508ymwexh52PWsMRrHBoQRahHNkxNI95+kmuN+vMiZMBb16c?= =?us-ascii?Q?3lDrjn6QNu1cTo2SG8cdWfvs69ZuJmwrPFEmkZ5wAH0HraDdM8PZ6UsyOv61?= =?us-ascii?Q?mna+FROqKYnHe7THTtHhxmEX1h/y2S1Uj/Qmo73PVXsrioLfz7WqXZ/ClObo?= =?us-ascii?Q?u8ZOIbhJehIMIBVY/MDk2lWzSlGqOreSWzuAivP3cm6lp2y3nya2UoIjkv4X?= =?us-ascii?Q?krp4uBkqpBIcaYFsEd8msXruCY25jkSAf+K0DENVbNWTzUH9gTfjlvaE5HBo?= =?us-ascii?Q?0mk5Y6rkFEB04yqbtdnbZdbsj/o7w26NgBSNeuraYBshLTDRlMFG7iE1gMG4?= =?us-ascii?Q?a3OHfyvUcJFR6b7ZnCXNbF1z00uyyNIaH6kYxn0dq9a9aw4r5+zP+/ujEMpO?= =?us-ascii?Q?TX4BoobjLK0XzjGjnK2Vu1B0fzZtVjkqvgN5T9TOkkEPwMAvpz3AguwME+Vw?= =?us-ascii?Q?Wi+GllAWTM1LsghMh80welcaNfOBKMpdZYGhC/Ox9EBbBGtwo66O/QHK96aw?= =?us-ascii?Q?/+GfV22EXwdCRgTJL0NkYwdEqAitIEDABqv23t8b0gxqY8suVQTvJC0hu0hb?= =?us-ascii?Q?bo41T2XqQpTxIejahn86AwTLCsUbX2EEAtqjLfB/8muzMdQUJAH3xkNAZONF?= =?us-ascii?Q?NcBps5T3h9YowOqqfvWFvQmt1XsEtF5K6BeoE3KAmt8lWdd0/9KodQHVZ2EG?= =?us-ascii?Q?DxGKobog3kDB/2SUM2fjzhZzRXlkH3//YO2Af8tfmyGR7fNki+iHPzrNDLAk?= =?us-ascii?Q?sCR1R5ATJURgjuNZ6ZJtmwQJHshrtfEUBcMYGtJPyvtE3O1Nn27r6pITpft3?= =?us-ascii?Q?gmufDINF72GpCQxQTUPGKo2q2pcytJwVXQOQOjmpmEggY44hf9GNoKfyKE8a?= =?us-ascii?Q?3AhgiKgFpE7c2Pkbn+japLvU3nV8T6pwnwwkioaRPby72LZlL6DFl36ay347?= =?us-ascii?Q?DuFIwzQtY5h9YT7YRvBldGPGdgTlB4Fjd6WyB+m/uQuMmn3gYUPCnH5ZkerX?= =?us-ascii?Q?xYb0EBxUmtSCzE79w2zgbwMmls28IafXGO+dSytZUptR+V3EMzX1pxN3YS/w?= =?us-ascii?Q?2xH+65orRZDzOTYxyMVWPKscYksFboc/E1J7dtpZn8l1SmnCU4oeaUJ+Q5b4?= =?us-ascii?Q?GCoSRPtsTVI3NvbqlcUhiA0ijXxluSs0/2KRH/mIpY0wGWbnzzoMbFNpS+gW?= =?us-ascii?Q?DfNJN5VkF8s8OViabcSX6bhng0xOBSjuwgfoNK3LL0KuuWU/19xeUk1jMWmm?= =?us-ascii?Q?gk31s8HZWQI4PmlxGt/JXlda2pam4rYoh4C7qIp9AsKJOr5HUtuYJBRNAMOQ?= =?us-ascii?Q?A/HtcqWwpzropVOpqyFpbSV16sbnyPFEHJ2tdNcto8dfF8FO5otxr9Coi6Fq?= =?us-ascii?Q?VwprawW95VZH2Ufpevl/lqYA78bhTS/APzYnReoblCwpMrwVVl5n4NPLx5sh?= =?us-ascii?Q?TRkLrP1UM7npxcUKJruHMuMujHgpY5rKIA3T3gHXe49R97n7?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: f0c94a6e-f622-4b1b-5184-08df02d938dc X-MS-Exchange-CrossTenant-AuthSource: LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 18:46:49.5949 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ayLu5tC/vj4BLlN8k5CFHvCPnojrqHzS7mKZGxSPdlo5De2FiaE1ih2bEMVs7fFhcBJ1bA2kAMRB2hZ6G7Wwiw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO7P123MB7501 Content-Type: text/plain; charset="utf-8" In print_cfs_stats(), rq->leaf_cfs_rq_list is traversed using for_each_leaf_cfs_rq_safe(), which expands to list_for_each_entry_safe(). Although rq->leaf_cfs_rq_list is RCU-protected, list_for_each_entry_safe() is a non-RCU iteration macro. It dereferences pointer links without READ_ONCE() and pre-fetches the next pointer. When a writer concurrently adds a new cfs_rq to the list using list_add_rcu(), a reader traversing with list_for_each_entry_safe() lacks READ_ONCE() protection. Without READ_ONCE(), the compiler is free to re-fetch pointers or reorder instructions. As a result, the reader can observe a newly inserted cfs_rq's pointer before its internal fields are fully visible, leading to reading uninitialised data or dereferencing invalid pointers. Additionally, because print_cfs_rq() drops rq->lock during seq_file I/O, concurrent cfs_rq list removals and re-insertions can modify cfs_rq->next. If cfs_rq is re-inserted near the head of the list while rq->lock is dropped, lockless readers can jump backward in the list. Under sufficient load it could lead to unbounded list traversal inside the RCU read-side critical section and trigger an RCU stall. Fix this by introducing for_each_leaf_cfs_rq_rcu(), which expands to list_for_each_entry_rcu(). This uses READ_ONCE() during list traversal. Finally, capping print_cfs_stats() lockless list traversal with a hard iteration ceiling to guarantee loop termination and prevent RCU stalls under continuous list churn. Fixes: 039ae8bcf7a5 ("sched/fair: Fix O(nr_cgroups) in the load balancing p= ath") Reported-by: sashiko-bot Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 39 +++------------------------------ kernel/sched/fair.c | 33 ++++++++++++++++++++++++---- kernel/sched/sched.h | 51 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 83 insertions(+), 40 deletions(-) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 2a01854f0f1c..de453342481c 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -11,17 +11,6 @@ #include #include "sched.h" =20 -/* - * This allows printing both to /sys/kernel/debug/sched/debug and - * to the console - */ -#define SEQ_printf(m, x...) \ - do { \ - if (m) \ - seq_printf(m, x); \ - else \ - pr_cont(x); \ - } while (0) =20 /* * Ease the printing of nsec fields: @@ -933,38 +922,16 @@ static void print_cfs_group_stats(struct seq_file *m,= int cpu, struct task_group #endif /* CONFIG_FAIR_GROUP_SCHED */ =20 #ifdef CONFIG_CGROUP_SCHED -static DEFINE_SPINLOCK(sched_debug_lock); -static char group_path[PATH_MAX]; +DEFINE_SPINLOCK(sched_debug_lock); +char sched_debug_group_path[PATH_MAX]; =20 -static void task_group_path(struct task_group *tg, char *path, int plen) +void task_group_path(struct task_group *tg, char *path, int plen) { if (autogroup_path(tg, path, plen)) return; =20 cgroup_path(tg->css.cgroup, path, plen); } - -/* - * Only 1 SEQ_printf_task_group_path() caller can use the full length - * group_path[] for cgroup path. Other simultaneous callers will have - * to use a shorter stack buffer. A "..." suffix is appended at the end - * of the stack buffer so that it will show up in case the output length - * matches the given buffer size to indicate possible path name truncation. - */ -#define SEQ_printf_task_group_path(m, tg, fmt...) \ -{ \ - if (spin_trylock(&sched_debug_lock)) { \ - task_group_path(tg, group_path, sizeof(group_path)); \ - SEQ_printf(m, fmt, group_path); \ - spin_unlock(&sched_debug_lock); \ - } else { \ - char buf[128]; \ - char *bufend =3D buf + sizeof(buf) - 3; \ - task_group_path(tg, buf, bufend - buf); \ - strcpy(bufend - 1, "..."); \ - SEQ_printf(m, fmt, buf); \ - } \ -} #endif =20 static void diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d55c21811306..81c5e557fdce 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -416,6 +416,10 @@ static inline void assert_list_leaf_cfs_rq(struct rq *= rq) list_for_each_entry_safe(cfs_rq, pos, &rq->leaf_cfs_rq_list, \ leaf_cfs_rq_list) =20 +#define for_each_leaf_cfs_rq_rcu(rq, cfs_rq) \ + list_for_each_entry_rcu(cfs_rq, &(rq)->leaf_cfs_rq_list, \ + leaf_cfs_rq_list) + /* Do the two (enqueued) entities belong to the same group ? */ static inline struct cfs_rq * is_same_group(struct sched_entity *se, struct sched_entity *pse) @@ -469,6 +473,9 @@ static inline void assert_list_leaf_cfs_rq(struct rq *r= q) #define for_each_leaf_cfs_rq_safe(rq, cfs_rq, pos) \ for (cfs_rq =3D &rq->cfs, pos =3D NULL; cfs_rq; cfs_rq =3D pos) =20 +#define for_each_leaf_cfs_rq_rcu(rq, cfs_rq) \ + for (cfs_rq =3D &rq->cfs; cfs_rq; cfs_rq =3D NULL) + static inline struct sched_entity *parent_entity(struct sched_entity *se) { return NULL; @@ -15576,14 +15583,32 @@ DEFINE_SCHED_CLASS(fair) =3D { #endif }; =20 +#define SCHED_DEBUG_MAX_ITER 4096 +#define SCHED_DEBUG_TRUNCATED_MSG \ + "stats truncated at " __stringify(SCHED_DEBUG_MAX_ITER) " iterations\n" + void print_cfs_stats(struct seq_file *m, int cpu) { - struct cfs_rq *cfs_rq, *pos; + struct cfs_rq *cfs_rq; + int max_iter =3D SCHED_DEBUG_MAX_ITER; =20 - rcu_read_lock(); - for_each_leaf_cfs_rq_safe(cpu_rq(cpu), cfs_rq, pos) + guard(rcu)(); + for_each_leaf_cfs_rq_rcu(cpu_rq(cpu), cfs_rq) { + if (--max_iter < 0) { + SEQ_printf(m, "\n"); + if (IS_ENABLED(CONFIG_FAIR_GROUP_SCHED)) { + SEQ_printf_task_group_path(m, cfs_rq_tg(cfs_rq), + "cfs_rq[%d]:%s ... " + SCHED_DEBUG_TRUNCATED_MSG, + cpu); + } else { + SEQ_printf(m, "cfs_rq[%d]: " + SCHED_DEBUG_TRUNCATED_MSG, cpu); + } + break; + } print_cfs_rq(m, cpu, cfs_rq); - rcu_read_unlock(); + } } =20 #ifdef CONFIG_NUMA_BALANCING diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 13a437032855..ec99be60f807 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -777,6 +777,18 @@ struct cfs_rq { #endif /* CONFIG_FAIR_GROUP_SCHED */ }; =20 +#ifdef CONFIG_FAIR_GROUP_SCHED +static inline struct task_group *cfs_rq_tg(struct cfs_rq *cfs_rq) +{ + return cfs_rq->tg; +} +#else +static inline struct task_group *cfs_rq_tg(struct cfs_rq *cfs_rq) +{ + return NULL; +} +#endif + #ifdef CONFIG_SCHED_CLASS_EXT /* scx_rq->flags, protected by the rq lock */ enum scx_rq_flags { @@ -3410,6 +3422,45 @@ extern struct sched_entity *__pick_root_entity(struc= t cfs_rq *cfs_rq); extern struct sched_entity *__pick_first_entity(struct cfs_rq *cfs_rq); extern struct sched_entity *__pick_last_entity(struct cfs_rq *cfs_rq); =20 +/* + * This allows printing both to /sys/kernel/debug/sched/debug and + * to the console + */ +#define SEQ_printf(m, x...) \ +do { \ + if (m) \ + seq_printf(m, x); \ + else \ + pr_cont(x); \ +} while (0) + +#ifdef CONFIG_CGROUP_SCHED +extern spinlock_t sched_debug_lock; +extern char sched_debug_group_path[PATH_MAX]; +extern void task_group_path(struct task_group *tg, char *path, int plen); + +#define SEQ_printf_task_group_path(m, tg, fmt...) \ +{ \ + if (spin_trylock(&sched_debug_lock)) { \ + task_group_path(tg, sched_debug_group_path, sizeof(sched_debug_group_pat= h)); \ + SEQ_printf(m, fmt, sched_debug_group_path); \ + spin_unlock(&sched_debug_lock); \ + } else { \ + char buf[128]; \ + char *bufend =3D buf + sizeof(buf) - 3; \ + task_group_path(tg, buf, bufend - buf); \ + strscpy(bufend - 1, "...", sizeof("...")); \ + SEQ_printf(m, fmt, buf); \ + } \ +} +#else +static inline void __printf(3, 4) +SEQ_printf_task_group_path(struct seq_file *m, struct task_group *tg, + const char *fmt, ...) +{ +} +#endif + extern bool sched_debug_verbose; =20 extern void print_cfs_stats(struct seq_file *m, int cpu); --=20 2.55.0 From nobody Mon Sep 28 05:42:23 2026 Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020115.outbound.protection.outlook.com [52.101.195.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C180B3B05AF for ; Tue, 25 Aug 2026 18:46:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.195.115 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683620; cv=fail; b=XQYu5TbJ0LT2VSB1vQIYCAQrv49Gn1R4zjs/jWdpWYjUR0J4m4MwMxCnxb4v1aO3+3gPTJMFn/QO8waJKwu0WAU8l4uSXO/cGfGZ5tCbvMcNUIVj6EL76QCmQxpMy+GJBSsqF+hwWsk36o+G9IYd69XRA9h8IJekco+DwFYolRU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787683620; c=relaxed/simple; bh=bYChyL1EEXS8z4NdNcWvzhbQbozIYqmPSC3FYsbWxhQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=N24b/Iqsy0p7LG6iwrcBJ1GngaW4dDn1CxOUPa18PwQf5KH4FTKJ6nQc2cjviCWgEn3ASObETJNP9souZJ0ZvG54CM6fnZxuw8xhm9xImnKzfWvXE+pq8IOk4ulZvl0x0zpKgkDcD9YYwWlixYQoXZ/fMc6WSHViiwHDnivx1hI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.195.115 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dPmpU6AWKgLk4YOh5r3ex0WQg/hRYGwGgW7TdZZ0hb1HTYPt1E+OSjcyEe7CaIoC7OF8RpNLXfleEe7298JzgjYTe8F8aBhjbc0SWK5h4g2CxOzd3ZJN0lONajQxHjw4SLwTPPeWF9/v7XXnHArY5vFdaiDA9ERa+zVXjCeR7pNtOSqsLkKsYCzDIY2mMMSsRE9QUnE9IfQUr78xedQTVtFrLTtGhGkobLsl7DyQrr5PFfcq5tP/vqOl6zrmKD2gAfGkinJdlSE1GD1GaR8UJcWy0XJAtRL8mUwI6gMvgCteQyXmMDkghAp1GoQmChUpggKpcHA34W6Y2AIk8vFSVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=8YO/K+uCFDFm+dQzpvNntelCS2/7Y787NdbN+4/KaOw=; b=yIcP35EkKENhKUtO+lTgTgrFE7j/7d5DUOl/uGhQFkIgUVgdtE4s61jT9O3EJbCsuH1JQFkrcph4Gli2DVWo59EK1TaysT0zcb5Wsz8xEuNVBO6RdHgkFzoF9l5aFD5yZGHv/KzkECvQckNsAO+LGAl67L6swCxEOUW4x3BlTZUwmfescrKh/QxC8fv1F+OFfSW9H4hnXtdZLFrCEaS6lG5jJ44GoxgN0SEMu5N55Q7lC6bxLyH5+SCWeaB1rHOXIvsWceOaN9oQkMNbJysAnSMPrJlFJaBCc+2aBUFr1ckfi4HY5OrRSz/6FnKvJuIu8N7wInmtu3eQ74c9MiC75Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) by LO7P123MB7501.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:40c::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.360.8; Tue, 25 Aug 2026 18:46:51 +0000 Received: from LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383]) by LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM ([fe80::3975:c5e7:bc1a:f383%4]) with mapi id 15.21.0360.005; Tue, 25 Aug 2026 18:46:51 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, atomlin@atomlin.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, rishil1999@outlook.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 6/6] sched/debug: Introduce per-CPU debugfs files Date: Tue, 25 Aug 2026 14:46:37 -0400 Message-ID: <20260825184637.888364-7-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260825184637.888364-1-atomlin@atomlin.com> References: <20260825184637.888364-1-atomlin@atomlin.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0288.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:195::23) To LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2b2::11) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LO6P123MB6616:EE_|LO7P123MB7501:EE_ X-MS-Office365-Filtering-Correlation-Id: c2e97e25-c81f-4e6f-b41d-08df02d93a0c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|376014|7416014|1800799024|366016|3023799007|10067099003|56012099006|5023799004|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: 6xH38yBXr3MsHiKXUmyOkZTDOJ4Tejf7GCyjfN9xx27BKzDdVSIJOvoyU/e0aQciSjM/Ip5td7ldRi/zCoz9I/JIUznU5Gzz0sq5/N9szzgwfA4sY/ojykTzaiB5tmaCdnjWkv40gjeyCnrmysuCehf7L4L6n3YEbiWqcs2SpYQGNbSI/+NdO3pTYQzGEBmuES+FuwkBGMFlch8X/2hW2daxnmZsUW+zKGV1iz62UFRQzu9kiyEPrdq3sKtKdZBaQCQ6oawx/k8O29JqbYzn3ZsC2dK7zl5cAe5vy4sZmOF51MczL/dImbjkaVLDcuae/e/WGDJBIP2G+OzI/77IzAhJtGB2pI3Xt0tfpm6SuMiPgu1Bi6ct/oI0VYSMLnhigXbi4BBN9uQrM8QUwqutJkw4DXKu4BcYlio0JpsWtP1wn7hDpouw26Tte4Kvk2z9Q+bgFr49PsXpRsb3JQQ2rD5fh0bYrFApED7Lgzf+69jSv2PWppOOKI3+AxgMTr7x8EBP1AALr6fy3kdAZc4idO0Tam29FD9uNgAP5/MTLoG8YguH4ZmrEbfB3zHm/EBsiC+wrfkMioESEubbjBUcoEZn3cMtz5rBePg357YoBupJuy/li+Iuq39W7x5VUIvstuVyAdBBsadJXesstexEHm2QqaE2U9AZxQZX6KhNBpU= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(376014)(7416014)(1800799024)(366016)(3023799007)(10067099003)(56012099006)(5023799004)(18002099003)(22082099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/Q9WEOZXT6eD4aYHSf7jw6efh4+FRc/+WKY7zJaazYcZVLx3HSwkLL7XaHCI?= =?us-ascii?Q?9vPHyECNjLPxvYhgAnZZKeFOpbojTnykSLYUFoov++umEl+MrrhvupWjHpOe?= =?us-ascii?Q?OG/MMmrd5eZ2g5hyy1dnqEI8v3v0nP81niyHmVK++AEL8C+RvFm1gW15Ndrl?= =?us-ascii?Q?7OX3jGWNlYCHqkPoCpEhIoZh+yi2XkRp3ijRv6asmSpxLTI1ddrNXECpMWfA?= =?us-ascii?Q?MUSFTYP555vzouF0PFWus/lB4tDnJz7pub971XAphrDZOMnY3xyT5m8oBN6X?= =?us-ascii?Q?eACSBp4j5mpgc33thmALLRX6et2xQvy4TB15Q8wOQdQFdiaLg58LWtwAMR/X?= =?us-ascii?Q?getaQjfiE1iEih1SfZq+ZnYaUPhtVZ8hwrGpUTG3IaIkyn/XNhoo75fQIsGf?= =?us-ascii?Q?EWoPvjbN7X3+u8tNdWv6oFS2iYBaK6XNVo7bUuIypI/ybBdEQXb+1Vip+EWT?= =?us-ascii?Q?G18JI3WLKw/V3jBfiwm7JQ4ZukJwvp8c36XKNwdZ3B7VopaZarIAa5aVscid?= =?us-ascii?Q?kDqFMUbJvh+5udVpb95cMBmmBB0Nqm3tH7Pq4ri0kKum9MZQqABZgCcNvWbn?= =?us-ascii?Q?FIMDPLnvKQoc/tRHxedKfbiRVzZn13R3qoop6796xLccjoElXl8c3w1jdm4N?= =?us-ascii?Q?DBf92tllA4LKW6rGqKpVeFZg4gKTfleumC/wIhdP3RJPJPmejpG8wIZeeeAM?= =?us-ascii?Q?c/a9Yct6yVNUN5V53h9blQOECDkOt1N81gvy/T91N5kynunBCRxJkrj4bMve?= =?us-ascii?Q?MnS1mt0H4+nkfUW0+chveY7qSePODedKizkITKlIJ25L5yFZGfEfHWvCY8C/?= =?us-ascii?Q?psZcr8SU4l2mLeMLJsTZUEpEWHV33d5GlaVynYkprcMFPyS5tiarq7UuYWtd?= =?us-ascii?Q?cvadGaxvMQxB9TMd8Xg+wSwk8CO1LbcGQgEuY0xSc0h5qyML5tmV9MnlaMVY?= =?us-ascii?Q?5uLCCmg41Xhwacdc7xJKmTH2XsOi6AXFoIiyEq+dSW1h3sOomkCthsl7flES?= =?us-ascii?Q?P/cD9GuiWmrONISnFSnvDwHm4OaI2NDL5DnnMZ8UpXDMyVUHacVFURwOwudl?= =?us-ascii?Q?i/qYjwPqMtQT754DcGBO46O42O9LJgs6IFfkvIqQYutyhKA/WCKJRPzIqAHS?= =?us-ascii?Q?gcQKsBtxwQWyUHaGf55PZtOSGg+W16lEfV2ZF0jjMbYAthGelKHLsFf0zvep?= =?us-ascii?Q?1jELWF1mXQDySje0hQd898SnJrZyBaCxNOatremX/NaRwCiIWHg5+gXLy3eB?= =?us-ascii?Q?6gmFaJUm0gUmYIS9JXpaHq3wDiQTUiOhT14qT1X8Pmbd8swtLO5Q7PtrwtH7?= =?us-ascii?Q?TU6+KqRYTXM6GPkMrPHSIjPTHOlLb0QCFxW6stOC9xu1HD8Vf6wLNtm0PkNB?= =?us-ascii?Q?Ht7mCb1tW2KWadP+mAFbZfxlUcM99L69VNX84+VAHZaK3p43ehINGgpFrhqz?= =?us-ascii?Q?ngVeQM80514irY2n9WjHbAlxgbVZeXioHlI0QGZREuR8opYOx2RaDE4QV+Po?= =?us-ascii?Q?fZ2ljO7yg4c69ObaIJLl1O9cBMx5LIK/G8WHv9zenaDXAR/r/U9q9K61aWn9?= =?us-ascii?Q?vwb1Yxkw9Cf3RYAL+OCeamdZi2C+prU5A3h6kIU4Z46r2BEf7qKDYywRbhnH?= =?us-ascii?Q?iaYky1QHVxJCwgvGGVY163vHwPK4+1wR2xXBqZG5N1NYyYprJq0SgFUAtwfu?= =?us-ascii?Q?vTXVZdWEiCYE+GDB/rLjppFv6/lizjM2/KUhqAAErqD4qjY9?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: c2e97e25-c81f-4e6f-b41d-08df02d93a0c X-MS-Exchange-CrossTenant-AuthSource: LO6P123MB6616.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2026 18:46:51.5868 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 99AH9knLCIvz9JMBa6bjceySW/0CIxwjmzXItmQM8hS/VT5P/DNmbE80pKRDb1JNVaGBC4DBO1Fgjz87NmtMeQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO7P123MB7501 Content-Type: text/plain; charset="utf-8" Currently, accessing scheduler debugging details for a specific CPU requires reading /sys/kernel/debug/sched/debug, which outputs information for all online CPUs. When investigating a latency anomaly or scheduling issue isolated to a specific CPU, accessing /sys/kernel/debug/sched/cpu/cpu/debug provides an immediate, targeted view of that runqueue. Add support for per-CPU debug files under: /sys/kernel/debug/sched/cpu/cpu/debug. Reading /sys/kernel/debug/sched/cpu/cpu/debug calls print_cpu() specifically for CPU , exposing CPU-specific runqueue details on demand. If the target CPU is currently offline, reading its file returns -ENODEV. Signed-off-by: Aaron Tomlin --- kernel/sched/debug.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index de453342481c..d70387659ae8 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -345,6 +345,7 @@ static const struct file_operations sched_verbose_fops = =3D { }; =20 static const struct seq_operations sched_debug_sops; +static void print_cpu(struct seq_file *m, int cpu); =20 static int sched_debug_open(struct inode *inode, struct file *filp) { @@ -698,6 +699,47 @@ static const struct file_operations sched_cgroup_fops = =3D { }; #endif =20 +static int sched_debug_cpu_show(struct seq_file *m, void *v) +{ + unsigned long cpu =3D (unsigned long) m->private; + + if (!cpu_online(cpu)) + return -ENODEV; + + print_cpu(m, cpu); + return 0; +} + +static int sched_debug_cpu_open(struct inode *inode, struct file *filp) +{ + return single_open(filp, sched_debug_cpu_show, inode->i_private); +} + +static const struct file_operations sched_debug_cpu_fops =3D { + .open =3D sched_debug_cpu_open, + .read =3D seq_read, + .llseek =3D seq_lseek, + .release =3D single_release, +}; + +static __init void debugfs_cpu_init(void) +{ + struct dentry *d_cpu_dir; + unsigned long cpu; + char buf[16]; + + d_cpu_dir =3D debugfs_create_dir("cpu", debugfs_sched); + + for_each_possible_cpu(cpu) { + struct dentry *d_cpu; + + snprintf(buf, sizeof(buf), "cpu%lu", cpu); + d_cpu =3D debugfs_create_dir(buf, d_cpu_dir); + + debugfs_create_file("debug", 0444, d_cpu, (void *) cpu, &sched_debug_cpu= _fops); + } +} + static __init int sched_init_debug(void) { struct dentry __maybe_unused *numa, *llc; @@ -759,6 +801,7 @@ static __init int sched_init_debug(void) #ifdef CONFIG_SCHED_CLASS_EXT debugfs_ext_server_init(); #endif + debugfs_cpu_init(); =20 return 0; } --=20 2.55.0