From nobody Thu Jul 16 04:04:03 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E830B1C33 for ; Fri, 1 Mar 2024 15:18:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709306295; cv=none; b=fiTV8RdgX1SMsFl2Rn1gS6ESXu8wYsG0jN3AvwTYQKMTSN20vu2ULbhl0tHHhF/b336kaTl9XJcI66l+ANZ2IhX/gOp8QCyGQBchYhMUcOPewOTm89VVk2WR1/UvcP02uDResjZ5O9/zxtgXzKNzz5vJbavz2w9PwephIRpXTm8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709306295; c=relaxed/simple; bh=YSVqtMO+wKDDoKeJ5jfBn/EZv1n4za+zGw2Co/dOsvc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tHmK7Ej9y19iLHrC0k285436Z00YeKz+2gr1FrzUQmNoDLoxgW0yYpDA7ba8LFentvcrGnHrWj+WxQIP9/7t87d2GpAjptchXWeqT4NaoaKc1kx/yCLcNSTqzOAyQKdNDcGeJA2vb2w/kwSwBE7cvgRnggmNdy2gGeSAhj4LJYI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ag5jtLnq; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ag5jtLnq" Received: from pps.filterd (m0353723.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 421E0YWd027980; Fri, 1 Mar 2024 15:18:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=t9FnlzNERVOGw06pCKSKBO2k0r/g6mojuUzF5SI7rX4=; b=ag5jtLnqXyBTQcW9EGHj6bl6cXDA88El0j5CHkA27dlTMrwqLoi8uzWsqAuHv98JIe45 J7+sMccZeS/RpcPIDunpzBhgDL2JbNx4N1iTHiMgxnuPw6kz+A2WZk1US53uEmqKVQUN ryJq+V6MtTPF4WXNvk8f714YhrYBgw+85dBUfT0K2AJL4QP8wRnts0BpgRdAltQBi1Wn q9GyaPOtGJrpxwOs/ebvQwvB0HD3jh7sx9WE7f6XDNAOkD3o/8UbUAlf9RSdL0lCJpuU +CtLmzfnKo/2o3kwtfNFuP8yp4YBSp/MFH7tSyspaJj0yUU8o8CpBMlqb6qP7WKSHhHU XQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3wkgdd21e2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 01 Mar 2024 15:18:02 +0000 Received: from m0353723.ppops.net (m0353723.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 421EtTI9019806; Fri, 1 Mar 2024 15:18:01 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3wkgdd21dh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 01 Mar 2024 15:18:01 +0000 Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 421E9IRQ008189; Fri, 1 Mar 2024 15:18:00 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3wfv9mw0tu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 01 Mar 2024 15:18:00 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 421FHsBI47907146 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 1 Mar 2024 15:17:56 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6ED852004B; Fri, 1 Mar 2024 15:17:54 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 524D820043; Fri, 1 Mar 2024 15:17:51 +0000 (GMT) Received: from li-c1fdab4c-355a-11b2-a85c-ef242fe9efb4.ibm.com.com (unknown [9.43.108.184]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 1 Mar 2024 15:17:51 +0000 (GMT) From: Shrikanth Hegde To: mingo@kernel.org, peterz@infradead.org, vincent.guittot@linaro.org Cc: sshegde@linux.ibm.com, yu.c.chen@intel.com, dietmar.eggemann@arm.com, linux-kernel@vger.kernel.org, nysal@linux.ibm.com, aboorvad@linux.ibm.com, srikar@linux.ibm.com, vschneid@redhat.com, pierre.gondois@arm.com, morten.rasmussen@arm.com, qyousef@layalina.io Subject: [PATCH v4 1/2] sched/fair: Add EAS checks before updating overutilized Date: Fri, 1 Mar 2024 20:47:24 +0530 Message-Id: <20240301151725.874604-2-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240301151725.874604-1-sshegde@linux.ibm.com> References: <20240301151725.874604-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Sw86H_SlUa7znm7JbZvkSK28nLZKH0Uw X-Proofpoint-ORIG-GUID: yNlbNyxPqk4W5e6Nuo5_hYvltdF-2KPN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-03-01_14,2024-03-01_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 mlxscore=0 lowpriorityscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 adultscore=0 priorityscore=1501 malwarescore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2403010127 Content-Type: text/plain; charset="utf-8" Overutilized field of root domain is only used for EAS(energy aware schedul= er) to decide whether to do load balance or not. It is not used if EAS not possible. Currently enqueue_task_fair and task_tick_fair accesses, sometime updates this field. In update_sd_lb_stats it is updated often. This causes cache contention due to true sharing and burns a lot of cycles. overload and overutilized are part of the same cacheline. Updating it often invalidates the cacheline. That causes access to overload to slow down due to false sharing. Hence add EAS check before accessing/updating this field. EAS check is optimized at compile time or it is a static branch. Hence it shouldn't cost much. With the patch, both enqueue_task_fair and newidle_balance don't show up as hot routines in perf profile. 6.8-rc4: 7.18% swapper [kernel.vmlinux] [k] enqueue_task_fair 6.78% s [kernel.vmlinux] [k] newidle_balance +patch: 0.14% swapper [kernel.vmlinux] [k] enqueue_task_fair 0.00% swapper [kernel.vmlinux] [k] newidle_balance Minor change: trace_sched_overutilized_tp expect that second argument to be bool. So do a int to bool conversion for that. Fixes: 2802bf3cd936 ("sched/fair: Add over-utilization/tipping point indica= tor") Reviewed-by: Srikar Dronamraju Signed-off-by: Shrikanth Hegde Reviewed-by: Qais Yousef --- kernel/sched/fair.c | 49 +++++++++++++++++++++++++++------------------ 1 file changed, 30 insertions(+), 19 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6a16129f9a5c..a71f8a1506e4 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6670,15 +6670,29 @@ static inline bool cpu_overutilized(int cpu) return !util_fits_cpu(cpu_util_cfs(cpu), rq_util_min, rq_util_max, cpu); } -static inline void update_overutilized_status(struct rq *rq) +static inline void set_rd_overutilized_status(struct root_domain *rd, + unsigned int status) { - if (!READ_ONCE(rq->rd->overutilized) && cpu_overutilized(rq->cpu)) { - WRITE_ONCE(rq->rd->overutilized, SG_OVERUTILIZED); - trace_sched_overutilized_tp(rq->rd, SG_OVERUTILIZED); - } + WRITE_ONCE(rd->overutilized, status); + trace_sched_overutilized_tp(rd, !!status); +} + +static inline void check_update_overutilized_status(struct rq *rq) +{ + /* + * overutilized field is used for load balancing decisions only + * if energy aware scheduler is being used + */ + if (!sched_energy_enabled()) + return; + + if (!READ_ONCE(rq->rd->overutilized) && cpu_overutilized(rq->cpu)) + set_rd_overutilized_status(rq->rd, SG_OVERUTILIZED); } #else -static inline void update_overutilized_status(struct rq *rq) { } +static inline void check_update_overutilized_status(struct rq *rq) { } +static inline void set_rd_overutilized_status(struct root_domain *rd, + unsigned int status) { } #endif /* Runqueue only has SCHED_IDLE tasks enqueued */ @@ -6779,7 +6793,7 @@ enqueue_task_fair(struct rq *rq, struct task_struct *= p, int flags) * and the following generally works well enough in practice. */ if (!task_new) - update_overutilized_status(rq); + check_update_overutilized_status(rq); enqueue_throttle: assert_list_leaf_cfs_rq(rq); @@ -9902,7 +9916,7 @@ static inline void update_sg_lb_stats(struct lb_env *= env, if (nr_running > 1) *sg_status |=3D SG_OVERLOAD; - if (cpu_overutilized(i)) + if (sched_energy_enabled() && cpu_overutilized(i)) *sg_status |=3D SG_OVERUTILIZED; #ifdef CONFIG_NUMA_BALANCING @@ -10596,19 +10610,16 @@ static inline void update_sd_lb_stats(struct lb_e= nv *env, struct sd_lb_stats *sd env->fbq_type =3D fbq_classify_group(&sds->busiest_stat); if (!env->sd->parent) { - struct root_domain *rd =3D env->dst_rq->rd; - /* update overload indicator if we are at root domain */ - WRITE_ONCE(rd->overload, sg_status & SG_OVERLOAD); + WRITE_ONCE(env->dst_rq->rd->overload, sg_status & SG_OVERLOAD); /* Update over-utilization (tipping point, U >=3D 0) indicator */ - WRITE_ONCE(rd->overutilized, sg_status & SG_OVERUTILIZED); - trace_sched_overutilized_tp(rd, sg_status & SG_OVERUTILIZED); - } else if (sg_status & SG_OVERUTILIZED) { - struct root_domain *rd =3D env->dst_rq->rd; - - WRITE_ONCE(rd->overutilized, SG_OVERUTILIZED); - trace_sched_overutilized_tp(rd, SG_OVERUTILIZED); + if (sched_energy_enabled()) { + set_rd_overutilized_status(env->dst_rq->rd, + sg_status & SG_OVERUTILIZED); + } + } else if (sched_energy_enabled() && (sg_status & SG_OVERUTILIZED)) { + set_rd_overutilized_status(env->dst_rq->rd, SG_OVERUTILIZED); } update_idle_cpu_scan(env, sum_util); @@ -12609,7 +12620,7 @@ static void task_tick_fair(struct rq *rq, struct ta= sk_struct *curr, int queued) task_tick_numa(rq, curr); update_misfit_status(curr, rq); - update_overutilized_status(task_rq(curr)); + check_update_overutilized_status(task_rq(curr)); task_tick_core(rq, curr); } -- 2.39.3 From nobody Thu Jul 16 04:04:03 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16C6D67E8E for ; Fri, 1 Mar 2024 15:18:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709306306; cv=none; b=a/RTxbC3yVFIWydBZ5djodM1L42ERahGDUTm7Jai1MZbE7NURd0QExhKQT2n2pzd/Dcv8bP1NiTVBbVbUb6GrvnhsQ/Llh1wCK/4Qg93cJfxtZDSVFK4+BT3VIgGjarJbOiMI7/fdOLMdGMJWMSA7ly6237yRTbaGXzmg6ugxyU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709306306; c=relaxed/simple; bh=jt+JFfSHfHUQOM9ggg4OpOmchKW/BPOrXWMwdjqv1To=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TvwcNRZLO2g4VOD/o6DKNDfTT1zAsrgEZxyOCTFp37tu0GOoaGMP3hf3xCbZ6nhSEhYiN/alhSDUt7a9eu7qcTHrP4dx7wxtORf4fM0xT0oPByGpZTFaOU45o7Y0PnVADf6b4gvkyKWLyVRD2nn+Fgh0UU3HdOQOOhk3rMtKRAg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=dN3f/U8v; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="dN3f/U8v" Received: from pps.filterd (m0353727.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 421F2ich009667; Fri, 1 Mar 2024 15:18:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=ZKvArzhNe7ZFn1OPm5+2YBGXQFLX2h9iVzOhDC9zsG0=; b=dN3f/U8v0R+CKLTf6MoxM6LHi3q9ugmwWs/MbFryYhPiUimSkUxVjfxHl0ID54B9lKHr sCn5H+oee8zD4DT228lmiOEk8vxcbwtVktNiJQIBQSi0EamaWkAt8sehLacQFEuHWJlr FqVJyHqb3udVYr/QCdJAzhJP4VM/SeGAmnqSLKFjS5/olYYsUQ9twZJ5pRUIgvdl6xIC LhDxDlZB/0xa2hpd/uvdLT24F132US9Uwc7t9KmtBEhC29aVfcXMBLooqTsM4ma8xfmB PRMDY4ggoWO+B348OsODDQ7Wg+cZcc2VUr7mKd/kQli4YjSXxC/SnJZqmOvnbkB7Zqgp rw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3wkhaf8d63-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 01 Mar 2024 15:18:08 +0000 Received: from m0353727.ppops.net (m0353727.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 421F4QtJ016839; Fri, 1 Mar 2024 15:18:07 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3wkhaf8d5g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 01 Mar 2024 15:18:07 +0000 Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 421E2T6k012348; Fri, 1 Mar 2024 15:18:06 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 3wfwg2vpw7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 01 Mar 2024 15:18:06 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 421FI0Yt63832476 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 1 Mar 2024 15:18:02 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BBEB720043; Fri, 1 Mar 2024 15:18:00 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C9EDE20040; Fri, 1 Mar 2024 15:17:56 +0000 (GMT) Received: from li-c1fdab4c-355a-11b2-a85c-ef242fe9efb4.ibm.com.com (unknown [9.43.108.184]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 1 Mar 2024 15:17:56 +0000 (GMT) From: Shrikanth Hegde To: mingo@kernel.org, peterz@infradead.org, vincent.guittot@linaro.org Cc: sshegde@linux.ibm.com, yu.c.chen@intel.com, dietmar.eggemann@arm.com, linux-kernel@vger.kernel.org, nysal@linux.ibm.com, aboorvad@linux.ibm.com, srikar@linux.ibm.com, vschneid@redhat.com, pierre.gondois@arm.com, morten.rasmussen@arm.com, qyousef@layalina.io Subject: [PATCH v4 2/2] sched/fair: Use helper function to access rd->overutilized Date: Fri, 1 Mar 2024 20:47:25 +0530 Message-Id: <20240301151725.874604-3-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240301151725.874604-1-sshegde@linux.ibm.com> References: <20240301151725.874604-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: mFto1n2-DVLWy95vQsBBr-WooEgaK65G X-Proofpoint-GUID: nlWlmMTpAo4Wwm-XrCQnolwgm3u9XPgJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-03-01_14,2024-03-01_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0 adultscore=0 mlxlogscore=826 lowpriorityscore=0 impostorscore=0 clxscore=1015 suspectscore=0 mlxscore=0 priorityscore=1501 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2403010127 Content-Type: text/plain; charset="utf-8" Overutilized field is accessed directly in multiple places. So it could use a helper function. That way one might be more informed that it needs to be used only in case of EAS. No change in functionality intended. Signed-off-by: Shrikanth Hegde Reviewed-by: Qais Yousef --- kernel/sched/fair.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a71f8a1506e4..650909a648d0 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6670,6 +6670,15 @@ static inline bool cpu_overutilized(int cpu) return !util_fits_cpu(cpu_util_cfs(cpu), rq_util_min, rq_util_max, cpu); } +/* + * Ensure that caller can do EAS. overutilized value + * make sense only if EAS is enabled + */ +static inline int is_rd_overutilized(struct root_domain *rd) +{ + return READ_ONCE(rd->overutilized); +} + static inline void set_rd_overutilized_status(struct root_domain *rd, unsigned int status) { @@ -6686,13 +6695,14 @@ static inline void check_update_overutilized_status= (struct rq *rq) if (!sched_energy_enabled()) return; - if (!READ_ONCE(rq->rd->overutilized) && cpu_overutilized(rq->cpu)) + if (!is_rd_overutilized(rq->rd) && cpu_overutilized(rq->cpu)) set_rd_overutilized_status(rq->rd, SG_OVERUTILIZED); } #else static inline void check_update_overutilized_status(struct rq *rq) { } static inline void set_rd_overutilized_status(struct root_domain *rd, unsigned int status) { } +static inline int is_rd_overutilized(struct root_domain *rd) { } #endif /* Runqueue only has SCHED_IDLE tasks enqueued */ @@ -7974,7 +7984,7 @@ static int find_energy_efficient_cpu(struct task_stru= ct *p, int prev_cpu) rcu_read_lock(); pd =3D rcu_dereference(rd->pd); - if (!pd || READ_ONCE(rd->overutilized)) + if (!pd || is_rd_overutilized(rd)) goto unlock; /* @@ -10859,7 +10869,7 @@ static struct sched_group *find_busiest_group(struc= t lb_env *env) if (sched_energy_enabled()) { struct root_domain *rd =3D env->dst_rq->rd; - if (rcu_dereference(rd->pd) && !READ_ONCE(rd->overutilized)) + if (rcu_dereference(rd->pd) && !is_rd_overutilized(rd)) goto out_balanced; } -- 2.39.3