From nobody Mon Dec 1 20:56:36 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB23C2206AC for ; Mon, 1 Dec 2025 18:32:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764613958; cv=none; b=iC3Tv7wDPHrc09HqetyujYuCXMpg7zqmZyHch5Lqspu3MFw6ScRdtSmAMRUqtE0RkthANsns1a+UKkMg3ohKCrngslD0o7KZBelg500MiR7Ah5gdoxB5lhuBrV3WnZf5Qpqd4DtFxgb03HD6CN5FSdiU5KUIrWqPEoTbCtx/pqI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764613958; c=relaxed/simple; bh=DEZvwjAZ+zpGv0kF1fL/peNfRZQ1HQAXsewsHos63WU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NVuT7nR4WrVSc9BD6xanS1h0YjTQgCJc4kuVWZ/DKtOPbiTjDgtMRgF9jY0R4Qwa5w3ZG1KQgo1L7HUU9v8BoYmAlAcrUeszG+LM2ZUdx4Uy2vy5HtcsrhVse2syaAzPdpjwO0or5aRIkW8lk89ESnUqx7VMCbAZ68MIhqQhEP0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=FkPWWiI3; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="FkPWWiI3" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5B1BSJwL000885; Mon, 1 Dec 2025 18:32:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=3AziXFYeeXrphX8x4 yU45+pV+B8PoMtr1aVfZ0JST1I=; b=FkPWWiI3tz4xK+ifrOg7Ec6Tsr8W6YjL4 sUKTN672YnBJjui6diIITd/b6nKrXVkewKTvOSOurfe9rFFC7/w/qc5nVFyr2x5d fg7y1WrMpUI3I5TChETRMf6TYVtA8Xv/sZrWbd3Lv46DKfgCQ4LcYfeuLMf5Q+Dy 68luLrtPwsjKvSBHSjZP8tYnNyo9Z2iz4IMpx6fmMecN+vvfZl8c8y9NqjNs+pyC d1j4atpKx1WByW0mM3otFHL+rWJZ0Kkgq7uXRkEBxtYPVLae4ND+h/K9zcY7ZjVw nrzwMj+Y0eF5YgZyHx3MDLnSlLN0NqBwarxazYSCjq6jTaDWXYZAQ== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4aqrj9gve5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 01 Dec 2025 18:32:10 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 5B1GQhmB029343; Mon, 1 Dec 2025 18:32:09 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4ardv17t8w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 01 Dec 2025 18:32:09 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 5B1IW7Ys15532388 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 1 Dec 2025 18:32:07 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5B1BB20043; Mon, 1 Dec 2025 18:32:07 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7AF6B20040; Mon, 1 Dec 2025 18:32:04 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.221.183]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 1 Dec 2025 18:32:04 +0000 (GMT) From: Shrikanth Hegde To: mingo@kernel.org, peterz@infradead.org, vincent.guittot@linaro.org, linux-kernel@vger.kernel.org, kprateek.nayak@amd.com Cc: sshegde@linux.ibm.com, dietmar.eggemann@arm.com, vschneid@redhat.com, rostedt@goodmis.org, tglx@linutronix.de, tim.c.chen@linux.intel.com Subject: [PATCH 1/4] sched/fair: Move checking for nohz cpus after time check Date: Tue, 2 Dec 2025 00:01:43 +0530 Message-ID: <20251201183146.74443-2-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201183146.74443-1-sshegde@linux.ibm.com> References: <20251201183146.74443-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: -H7BarcusuyQ7M7qOS3ryFtYeP5Y4mEv X-Proofpoint-ORIG-GUID: -H7BarcusuyQ7M7qOS3ryFtYeP5Y4mEv X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTI5MDAyMCBTYWx0ZWRfX6bV5ocSY2qJs kNRaDo96teP6Oj6rSFC31YbBh73t+2+BL8ybVKjNK0cjnXImSIXl2GzC4N2zaFEWq8P5vlFBPJy Iel4hhZr1BPrNiCkw8CTGDz0/y+r8RxWWuln+7TVOnNQMqLD8+In+3d94xWotm36FUGuoCh7/b7 +o1T+4UgUDwQ1ANDdD9ZBGHA3orXzP7ML9o/VgtkBlNfCvTZJwiFu767VWSaunKvkSDXXyQk9tJ odLV7mSjxus2Gu286G3felssECqJy/E7TXgXijl5eqzLG+F+7iBUQnpuGLwZq/tIfPpMf1qymuL jvQ3vwjJs76iLY5FbXKzN863H9stYLkw1h7vZMvcQI3ZMmbmZZYvIHlcNWunka3D1OkGypeGzm6 5X8n2uaY4d7hv5qlSwbnWsMPuJMzDA== X-Authority-Analysis: v=2.4 cv=dYGNHHXe c=1 sm=1 tr=0 ts=692ddf2a cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=wP3pNCr1ah4A:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=OUFfOgFxQ-4zh-9IuIYA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-28_08,2025-11-27_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 suspectscore=0 clxscore=1015 adultscore=0 lowpriorityscore=0 bulkscore=0 phishscore=0 priorityscore=1501 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2510240000 definitions=main-2511290020 Content-Type: text/plain; charset="utf-8" Idle load balancer is kicked off only after time check. So move the atomic read after the time check. If there are no nohz CPUs there likely has_blocked should be reset too. So stats load balancing shouldn't much difference.=20 Signed-off-by: Shrikanth Hegde =20 --- kernel/sched/fair.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 769d7b7990df..61cc3fdfa45b 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12440,19 +12440,18 @@ static void nohz_balancer_kick(struct rq *rq) */ nohz_balance_exit_idle(rq); =20 - /* - * None are in tickless mode and hence no need for NOHZ idle load - * balancing: - */ - if (likely(!atomic_read(&nohz.nr_cpus))) - return; - if (READ_ONCE(nohz.has_blocked) && time_after(now, READ_ONCE(nohz.next_blocked))) flags =3D NOHZ_STATS_KICK; =20 if (time_before(now, nohz.next_balance)) goto out; + /* + * None are in tickless mode and hence no need for NOHZ idle load + * balancing: + */ + if (likely(!atomic_read(&nohz.nr_cpus))) + return; =20 if (rq->nr_running >=3D 2) { flags =3D NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; --=20 2.43.0 From nobody Mon Dec 1 20:56:36 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32F2ED27E for ; Mon, 1 Dec 2025 18:32:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764613955; cv=none; b=CNBbP0YjtWEmMeS9NFU0QnjwALv7Jga0Dx8CPElvQpmI0OAcTIHSCRwWODogpqhoIov8mU6O2y4H4TPmvEKjE+8QPY7MUynGHk03VQMOBVLxSe/tqNna8f1KO2ntL81r1cy6B2QUitaRgBgySIeQkIpfV35rzAjFjQbLbsMLDqk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764613955; c=relaxed/simple; bh=EGx1VX80j5w6Ct22InNO0TB6nscQ0V+nI7DYJZf7wjk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ByZZXmjUarLpMFbGM2yxmDh3EngBNUuDLewbdga/SL2gezgFcT3yi4OYnmxirD862iR4/xzs1taydvGgLAMlxL2Gp9JvYF8L4GElwwIu/+ZzvjpN0xg1SpgliKaE25eYExVI9/bCKoOuCup/rb3/xoYR924Ysbwt5Cvk57q7TQY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ap+prXhD; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ap+prXhD" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5B1CId24001931; Mon, 1 Dec 2025 18:32:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Mj7uRn/mcSfdfeWT4 jg0l6KH6OXeIq2vYAUbL6z58vM=; b=ap+prXhD0a8P2Me+zQaCvUuXFCmBkyHin WP4MaeY5LXZdst1Nbgd9fUfiml8QPCNIagosiL3lx+fLWsgFQ33fjMu5c8YrO9zN boHQSQAoNJgEa29Dk1vvvBCtL4E1fBWxaJb2R6LZyM4QdnQGx4utgSzlmuIFHyq5 kLpTmXTcppe6/wmhV2Y91EDCx/hsmlRJKK9S0tkYMnsIgamrzGFTgWiI56G9wLP3 ClTovfAHQirZQJwfohTDjqg5sr9ucbkfoH3FbGcUYcnwi9VH1Ccs8ji/OjhmRIgV oA+NEP5zfbCtH2z9BGinWpT/Dttn7JmKCi82/nh20NZLHVhBraKUg== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4aqp8prw6u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 01 Dec 2025 18:32:13 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 5B1IPojY019042; Mon, 1 Dec 2025 18:32:13 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4arbhxr5j7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 01 Dec 2025 18:32:12 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 5B1IWAJ645744546 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 1 Dec 2025 18:32:11 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D3FE72004D; Mon, 1 Dec 2025 18:32:10 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BE1EF20040; Mon, 1 Dec 2025 18:32:07 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.221.183]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 1 Dec 2025 18:32:07 +0000 (GMT) From: Shrikanth Hegde To: mingo@kernel.org, peterz@infradead.org, vincent.guittot@linaro.org, linux-kernel@vger.kernel.org, kprateek.nayak@amd.com Cc: sshegde@linux.ibm.com, dietmar.eggemann@arm.com, vschneid@redhat.com, rostedt@goodmis.org, tglx@linutronix.de, tim.c.chen@linux.intel.com Subject: [PATCH 2/4] sched/fair: Change likelyhood of nohz nr_cpus check Date: Tue, 2 Dec 2025 00:01:44 +0530 Message-ID: <20251201183146.74443-3-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201183146.74443-1-sshegde@linux.ibm.com> References: <20251201183146.74443-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTI5MDAwMCBTYWx0ZWRfX8SsTJDvUbncm Opc1DOjqjMJXiqrE4NBT1YStlbPylkXHzyALwR0p0YYvZUJKxBizGXl9fcoiqBFR3bb+rfPVaEB 8yMQH/qoUVG0kTW/rtSMThBM+1TLAtwQt8M+fdol7vyQMN/+yVsN8hhZoyuKRBuL/haJb2dYr3N TuuN8l4Sb93LlCW37YSOrHU6MaHgaWdnlZRx37Z1okPvIxL6adV/JsgvATezw5RIlbc4ZV3PoJQ g4htXluGk+wk/OLSAd31t3qjRTKVIyB50wyT4pkHOZYMYx6uYWQLli1ded2J6fQhEtMgakMzL+6 /HN+XP7yiBV1shro+OymdKnPkORXfVy16KPlvy+rp3KEfXcE57ujpuNOLQHcHBwHjhAF+8nCVJi FD+hMi0lwLzTU8/Wgq0IHnJFsZ+bdA== X-Authority-Analysis: v=2.4 cv=dIerWeZb c=1 sm=1 tr=0 ts=692ddf2d cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=wP3pNCr1ah4A:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=L2DUigS4eapgOTKWHN4A:9 X-Proofpoint-ORIG-GUID: HgD4PhUwfn7ec9YrsPVcelZ3hYtxdLRR X-Proofpoint-GUID: HgD4PhUwfn7ec9YrsPVcelZ3hYtxdLRR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-28_08,2025-11-27_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 clxscore=1015 spamscore=0 malwarescore=0 suspectscore=0 adultscore=0 bulkscore=0 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2510240000 definitions=main-2511290000 Content-Type: text/plain; charset="utf-8" These days most of the system have multi cores. The likelyhood of at least one or more CPUs in nohz (idle state) is higher. So move likely to unlikely. Allow stats balancing to complete when there are no nr_cpus as the check happens later. Current code might race and may not set nohz.has_blocked =3D 0 even when nohz.nr_cpus =3D=3D 0 Signed-off-by: Shrikanth Hegde --- kernel/sched/fair.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 61cc3fdfa45b..55746274af06 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12448,10 +12448,10 @@ static void nohz_balancer_kick(struct rq *rq) goto out; /* * None are in tickless mode and hence no need for NOHZ idle load - * balancing: + * balancing, do stats update if its due */ - if (likely(!atomic_read(&nohz.nr_cpus))) - return; + if (unlikely(!atomic_read(&nohz.nr_cpus))) + goto out; =20 if (rq->nr_running >=3D 2) { flags =3D NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; --=20 2.43.0 From nobody Mon Dec 1 20:56:36 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4932221FD0 for ; Mon, 1 Dec 2025 18:32:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764613959; cv=none; b=YdWG1aro/OX6eRbJNrFlj+ipDM6WcCcqqjaLpzNiFwVJ5Eyznttfzz1uvPWAUqYbTAJwjDCG3Bvu2bFLNAkdXxsPhpPK3cs/t1FzyiNDwNHf4oT/UN7Fz2I/oAWjapC+vrBP3wKMij30n0xmRrnrUxMbnSqN4PxScLJL7W8g9I8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764613959; c=relaxed/simple; bh=zxiZz4fJgBqxmgSFZakdwy9glip8p0ly1LnXVVrjKNI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mClhU2Q5jTZufnHPJQGPVpevK3N8+WWPf66XwwPPXSU0gVr7bmedQIwJ+wkgQNvvENdx5Z5ELxJFspfJqv5p1DofAphykeVdbhQcRNTxxdybJYn0F6ig1VvGTInPPffO/K41HxLcdTHYESK52j7NE49tVkvHG1lrG/FcrligCtA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=lWtjHR4s; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="lWtjHR4s" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5B1HRbJ6010434; Mon, 1 Dec 2025 18:32:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=oOmLwKPmmvMR5FLKE 2sayPCVrfZ3wzSiKp1JPSbdIps=; b=lWtjHR4saY7upDGcVwf+Ul4vvLeUdYS2B WzB+3anbrbDYMWoVjlwgqxk+QmgTZWPgEbvesfQ4mgFHSqmAwHr2XRb48JNJK08o AIBkAL7YKyzbjJNQoPRMKR8WVg8vj9GZnRdLvMHT8gsI77I9irLMe6hTsqjN+5F0 xqQyqNeb5E0C3RqOSfe2u/VluLjkYaQKdDsR8H4QzhvGonkVIi75/1SBeR/HTFCJ LEofk7HXoHkMTh+N3vhgua3lf2GCW3FqbWE626s8XH0cw0D7yoTIz3Vkk3YnHpLM kGLuk3zrWmBs+SyVjp+eI+Rddp9w9poW4iNPGhv1Z98r7aG7PDJhA== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4aqrh6rve6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 01 Dec 2025 18:32:17 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 5B1FO1gI019120; Mon, 1 Dec 2025 18:32:16 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4arbhxr5ja-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 01 Dec 2025 18:32:16 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 5B1IWExK14483778 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 1 Dec 2025 18:32:14 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2049F20043; Mon, 1 Dec 2025 18:32:14 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3A0D020040; Mon, 1 Dec 2025 18:32:11 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.221.183]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 1 Dec 2025 18:32:11 +0000 (GMT) From: Shrikanth Hegde To: mingo@kernel.org, peterz@infradead.org, vincent.guittot@linaro.org, linux-kernel@vger.kernel.org, kprateek.nayak@amd.com Cc: sshegde@linux.ibm.com, dietmar.eggemann@arm.com, vschneid@redhat.com, rostedt@goodmis.org, tglx@linutronix.de, tim.c.chen@linux.intel.com Subject: [PATCH 3/4] sched/fair: Check for blocked task after time check Date: Tue, 2 Dec 2025 00:01:45 +0530 Message-ID: <20251201183146.74443-4-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201183146.74443-1-sshegde@linux.ibm.com> References: <20251201183146.74443-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=dK+rWeZb c=1 sm=1 tr=0 ts=692ddf31 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=wP3pNCr1ah4A:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=gfQPMJu-hSpRr0ZdjbEA:9 a=zZCYzV9kfG8A:10 X-Proofpoint-GUID: bQj7SX8pRlbw0qgQcDwC13oo7L8kb8WT X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTI5MDAyMCBTYWx0ZWRfXwbD5eyabvJLh jgC9Q5GNP+I3BwynKg6zu1bYf1gb0yTvEv5MyyDrr+Aala2rY4HZGc6/CfHuw/ll7maMWYxfvn8 Ip8TFsybKKzT2JDzcsOd9yuVOtFt8pFOBNh3h9CW6LjlZ0f8A0gvfvh6ESTJBOefb6d9/dvbjdl F0ZqSoW65rAlIURA6FXH4buUoMMakv9US5LuizxAl+IB/gXx+dFBDpLHc2nwDwdgihSC9JajXsP 95sE/xCrkkI1hv7gQ5o2zPsA5NXKm/FTSPV+pScT9+nfwMmI18dT5vGb9JzJod3whsR5rMaXupe M5Zs8Y41bmxP6/wzZQjH84iMzbiYoAE2IC9iS0UWLFYTOXwkh3w0krgmlQdyNIXAwRpUheW5ZSj bX/r4GzZv98R3C2FcdEihvYj4qeREA== X-Proofpoint-ORIG-GUID: bQj7SX8pRlbw0qgQcDwC13oo7L8kb8WT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-28_08,2025-11-27_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 lowpriorityscore=0 clxscore=1015 priorityscore=1501 bulkscore=0 adultscore=0 phishscore=0 impostorscore=0 spamscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2510240000 definitions=main-2511290020 Content-Type: text/plain; charset="utf-8" nohz.has_blocked can be updated often as and when CPUs enter idle state. But stats are updated only at regular intervals. Usually fixed to LOAD_AVG_PERIOD=3D32.=20 Read the value only after time check is successful to avoid cache references to it. Signed-off-by: Shrikanth Hegde --- kernel/sched/fair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 55746274af06..5534822fd754 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12440,8 +12440,8 @@ static void nohz_balancer_kick(struct rq *rq) */ nohz_balance_exit_idle(rq); =20 - if (READ_ONCE(nohz.has_blocked) && - time_after(now, READ_ONCE(nohz.next_blocked))) + if (time_after(now, READ_ONCE(nohz.next_blocked)) && + READ_ONCE(nohz.has_blocked)) flags =3D NOHZ_STATS_KICK; =20 if (time_before(now, nohz.next_balance)) --=20 2.43.0 From nobody Mon Dec 1 20:56:36 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08473221542 for ; Mon, 1 Dec 2025 18:32:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764613958; cv=none; b=YLFHNGXHBs/8Jm0cfIZlnWuNB3oyYb6yJaeloTzPrMnxxR2PMy0jlCSzW03N+ByCWaP/OlUO+AapnKpxIEep0NGih/lKd1qioRD5EQXokfWxzwE/4lL6za0USPK/kGEFFD6BQyUGoaYT7ccDzLsaXXPvkFVALw/IHFf0NpQEJTM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764613958; c=relaxed/simple; bh=khE+E815ZzIpP56fuexZI2Xhl+QWYWGvU0ffzB4Z0EM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Km3wEmD6p2avVhk426FHyUzw7c2CxJbI2XX0VFq6SzH8uKhw1fh2WUS4jzWIuDpASEx7/AR4q/SsYC+VzEA9QxnRrdCgXJKQYzUruAhSQkAvLqGyC452Leaa6b8x5aBSED+u6ZZK+1aLghijlendN6Jl4oDndOOyNfb7aFUOins= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Yk6R4EvI; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Yk6R4EvI" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5B1BS3Ug024923; Mon, 1 Dec 2025 18:32:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ekHqwJV195NOLHtTq KYDkpq2K74QToanGg7EflUJTE4=; b=Yk6R4EvI4KZOaxidkERaGc6+RmBeeJooi B2GkExVtzScCYwyqNhHE9xSmKDrDACB/ri7z79VJFyrVHkI2CIrVXRRCfCGw/xil TeXkt/5EN6ddrI/8JCXsVNlFn4UVNh/M2ll3xpn6gmw29uMN4G5mzpIoMz6JWDje Lm+7cXR8WHgh4jjLIKciEQkHTHgZvCt0eLWu3TUWW3X4EiVXUPOiyHmqpcBi0kgp rHB8JCqvZ8lyYtSefKYLrxBMaKSA2eFUpCGYSzJnqYqS6Hw8D4Vx1ZybcDh7XIce PCeEkLs7SHRG/HthRmUw6MuTtLK4Xicp45tMoYbDwDC9hXqLuQF4A== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4aqrbg0y54-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 01 Dec 2025 18:32:19 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 5B1GcIvN003847; Mon, 1 Dec 2025 18:32:19 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4ardcjfv5q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 01 Dec 2025 18:32:18 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 5B1IWHIk43647380 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 1 Dec 2025 18:32:17 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3707B20043; Mon, 1 Dec 2025 18:32:17 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7D6AD20040; Mon, 1 Dec 2025 18:32:14 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.221.183]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 1 Dec 2025 18:32:14 +0000 (GMT) From: Shrikanth Hegde To: mingo@kernel.org, peterz@infradead.org, vincent.guittot@linaro.org, linux-kernel@vger.kernel.org, kprateek.nayak@amd.com Cc: sshegde@linux.ibm.com, dietmar.eggemann@arm.com, vschneid@redhat.com, rostedt@goodmis.org, tglx@linutronix.de, tim.c.chen@linux.intel.com Subject: [PATCH 4/4] sched/fair: Remove atomic nr_cpus and use cpumask instead Date: Tue, 2 Dec 2025 00:01:46 +0530 Message-ID: <20251201183146.74443-5-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251201183146.74443-1-sshegde@linux.ibm.com> References: <20251201183146.74443-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 6xYfk93DVir5BycqVQc3Q56KhyfbgK9x X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTI5MDAxNiBTYWx0ZWRfX3FIrG8j+MNcX 5PMV6F692Rv0a5R57Qpy9HHebOoNCGJf4bn3mTb3yFtlCaClqwzQVfW9071FykfTgdKZMEBLUZa S9TkU0/lQz7Xn2/Q1yA7J1cycUmLnCmm23k0/hBlgJtBbeGSRKTlAEMGN0O2PFrAlVljDf29oPp bZCQHioc6z7+M1yXxgiovg7Yl6+9PGW6kqZZ/rQ+v1z824/BiC/qivvbU1TaZoyzGTRQEK0Ebjo td1zeRakKOhejjKXlOV7WMy50xjdqijDm3X6fqOSTpvyjbuwvd4osvuyL5BhGSCzY+7t4Mr9RpX wFGTzUEWGtVy9E7diBXUpJDJeIbeyJrLhgGAfsMIkhALN73rfvCF/zGq9g7/4IHgu+NPhUMpMwE V7W1DmWdDS9++Q28I0GxbiMm7XfNiA== X-Authority-Analysis: v=2.4 cv=UO7Q3Sfy c=1 sm=1 tr=0 ts=692ddf34 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=wP3pNCr1ah4A:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=kHv2m6OS6Jo2Esq1wlcA:9 X-Proofpoint-GUID: 6xYfk93DVir5BycqVQc3Q56KhyfbgK9x X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-28_08,2025-11-27_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 malwarescore=0 bulkscore=0 adultscore=0 priorityscore=1501 impostorscore=0 spamscore=0 phishscore=0 clxscore=1015 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2510240000 definitions=main-2511290016 Content-Type: text/plain; charset="utf-8" nohz_balance_enter_idle: cpumask_set_cpu(cpu, nohz.idle_cpus_mask) atomic_inc(&nohz.nr_cpus) nohz_balance_exit_idle: cpumask_clear_cpu(rq->cpu, nohz.idle_cpus_mask) atomic_dec(&nohz.nr_cpus) kick_ilb: if (likely(!atomic_read(&nohz.nr_cpus))) return; So, idle_cpus_mask contains the same information. Instead of doing costly atomic in large systems, its better to check if cpumask is empty or not to make the same decision to trigger idle load balance. There might be race between cpumask_empty check and set of cpumask in the remote CPUs. In such case at next tick idle load balance will be triggered. Race of clearing the bit is not a concern, since _nohz_idle_bala= nce checks if CPU is idle or not before doing the balance. cpumask_empty uses ffs. So should not be very costly. Signed-off-by: Shrikanth Hegde --- kernel/sched/fair.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 5534822fd754..3afa79d733dd 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7144,7 +7144,6 @@ static DEFINE_PER_CPU(cpumask_var_t, should_we_balanc= e_tmpmask); =20 static struct { cpumask_var_t idle_cpus_mask; - atomic_t nr_cpus; int has_blocked; /* Idle CPUS has blocked load */ int needs_update; /* Newly idle CPUs need their next_balance collated */ unsigned long next_balance; /* in jiffy units */ @@ -12450,7 +12449,7 @@ static void nohz_balancer_kick(struct rq *rq) * None are in tickless mode and hence no need for NOHZ idle load * balancing, do stats update if its due */ - if (unlikely(!atomic_read(&nohz.nr_cpus))) + if (unlikely(cpumask_empty(nohz.idle_cpus_mask))) goto out; =20 if (rq->nr_running >=3D 2) { @@ -12563,7 +12562,6 @@ void nohz_balance_exit_idle(struct rq *rq) =20 rq->nohz_tick_stopped =3D 0; cpumask_clear_cpu(rq->cpu, nohz.idle_cpus_mask); - atomic_dec(&nohz.nr_cpus); =20 set_cpu_sd_state_busy(rq->cpu); } @@ -12621,7 +12619,6 @@ void nohz_balance_enter_idle(int cpu) rq->nohz_tick_stopped =3D 1; =20 cpumask_set_cpu(cpu, nohz.idle_cpus_mask); - atomic_inc(&nohz.nr_cpus); =20 /* * Ensures that if nohz_idle_balance() fails to observe our --=20 2.43.0