From nobody Sat Oct 4 01:39:26 2025 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2134.outbound.protection.outlook.com [40.107.93.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 045B036CDE0 for ; Thu, 21 Aug 2025 04:27:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.134 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755750480; cv=fail; b=IDhut6psHsqs3hcHGT/nnczyz/7rnOTozAwUMsazQwhlP/59zVN2G2FE/wxpQmlxCYHp37V0DAtzUYd1kdM3pmnmFeV3eAfG8iW9L71NRmjgKNj6+u+gyPuXfFJQS8rKwZepGp5vv1vycQIMPlSPjO5ygESMU+geApkaQgbiRaQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755750480; c=relaxed/simple; bh=+FEEHa1Uhz9P2jW/y8toxxyZ+dJWgHnxk1UbeNTM6ho=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=Mw2w5b8xMo8O89fFTXMIyafGDywRwS6Ezxslj7c0Rpy+cYEQdh4DAAN2jGWAAZBdfP52xDqUXuo/EUj6O2Uf1TG9CM1Ob64yWK7sGli1Plj4wOnIv2izE+r8G4Zm0KY4L2UYARlIjsrg0J6AsRcY+cJCjVxKn72RaTOg4sfbdo8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=os.amperecomputing.com; spf=pass smtp.mailfrom=os.amperecomputing.com; dkim=pass (1024-bit key) header.d=os.amperecomputing.com header.i=@os.amperecomputing.com header.b=JF/6Vn7L; arc=fail smtp.client-ip=40.107.93.134 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=os.amperecomputing.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=os.amperecomputing.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=os.amperecomputing.com header.i=@os.amperecomputing.com header.b="JF/6Vn7L" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qRhJEe+ZOi/8AP7kpYuAiYzQMtPnCkNXvaBUNODFylxeBYiHRqGDlEqM7ISICgdj15jH/TEbush5HTt41BJQzuflsxXEf41pNwE6V+pTkp7RWCbAuLwIyqjFcqppAbF3ljoBU2uSm6FEJeIDvHMRC3FLKP1i8aZuS8YRr5aPPhH5p/RkS4Rz9vxNiPd4lO8TiCKP6TeZeuG2wK6o+qyfkCEo9EQm6UoqDW8q4WCd9MLKwRTKjV/GwZ3P0kJxKEIAdJoa3tuQUQKYPy0Sv2TlpApMpePKGqS2j2p3K07Hswsq+H7xLRTIoztK6D1SEXtcWCOFqqkT9UeDyzf2Jv0MdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bvItEv/6X84XuwxJhznN8kW4mrOxJNzbxryF9jSKiZs=; b=hipiJR9QyXeRZz6So/3+GRGOTQVmqyNp1ccmh6irAKEYcyY3xPQD4UJ5maj82VuOTtRiqhZ8I7P2zR6sbBK+ijyFTHBsgHiI/m4iqlM9Ar8fZ789LdrYpURoLRSYMiz+1qzp5qRP1SLRbOhTk63uTSUVudqnPm/hmP0LgVmCwWteLBzPxKN42AH+skKODs2z09YQyRAqPLLoq3HyUGcWxQinbZVrM6k6pPlvO4vXDxElo/KKfam/dIYAjOmv6qtIhj+0zRLiIs43GThWMcXRw1kEK3bTZHO918ushe9e4GotBWkBOC2R25ePtbimXuURgaSwJyeByXpkh9svymhQaw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bvItEv/6X84XuwxJhznN8kW4mrOxJNzbxryF9jSKiZs=; b=JF/6Vn7LoYHmGrsp2sJclx121FgkF399FNc+QvbrzreJdMxphuuceLkgfnHobmTvAYLMO7KPRKYvz0zXVnMWBQE6fhen7im14yGZciDITAXtBiDGzCrqWxQ7WQTKPss1WFceG2n5jtE7GBKy24wE7920Q9zYIW5Igd+JggBpdt0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from PH0PR01MB6761.prod.exchangelabs.com (2603:10b6:510:76::23) by SA6PR01MB8829.prod.exchangelabs.com (2603:10b6:806:42f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9031.24; Thu, 21 Aug 2025 04:27:56 +0000 Received: from PH0PR01MB6761.prod.exchangelabs.com ([fe80::bf98:19e8:71ee:cf2]) by PH0PR01MB6761.prod.exchangelabs.com ([fe80::bf98:19e8:71ee:cf2%7]) with mapi id 15.20.9052.013; Thu, 21 Aug 2025 04:27:56 +0000 From: Adam Li To: anna-maria@linutronix.de, frederic@kernel.org, tglx@linutronix.de, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, vschneid@redhat.com Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, cl@linux.com, linux-kernel@vger.kernel.org, patches@amperecomputing.com, Adam Li , Christoph Lameter Subject: [PATCH RESEND 1/2] tick/nohz: Fix wrong NOHZ idle CPU state Date: Thu, 21 Aug 2025 04:27:06 +0000 Message-Id: <20250821042707.62993-2-adamli@os.amperecomputing.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250821042707.62993-1-adamli@os.amperecomputing.com> References: <20250821042707.62993-1-adamli@os.amperecomputing.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SI2PR02CA0003.apcprd02.prod.outlook.com (2603:1096:4:194::13) To PH0PR01MB6761.prod.exchangelabs.com (2603:10b6:510:76::23) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR01MB6761:EE_|SA6PR01MB8829:EE_ X-MS-Office365-Filtering-Correlation-Id: 81e051b1-cf3f-4796-1357-08dde06b1a0a X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|52116014|7416014|38350700014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?p+x/ubPtTurCysyn/oIhLwJ9QEI1W5JZqHky92IoAP8EFoBYMhmEhRb8W/qG?= =?us-ascii?Q?TIAdxRCnkMY6STw+kg9OM+iowIM5q6AMKTXP71xLhFMljNIaKccddQ0bJJ9/?= =?us-ascii?Q?U4E47LsySP3JtUNHvaMc8KFcEAVaULPg9vExQpTHlpYwut67LY92e5sy1kE3?= =?us-ascii?Q?5neE3zHoz1GGKqhRdA9S+NgMOEq5Eyywb8Dv7h7rKrMWfH717bwfDhEMJiyK?= =?us-ascii?Q?HB2lrP+Pp7qJLVUQWddBUT1iYoKPQNmnn8vQP43dIQ1QNtXqyzUe9gYHnGnm?= =?us-ascii?Q?wnSuu6Xw02pFeQaTQG1Or+00Q+LXVl9dcd8Nq9/IgRuBHZhLjR5KjmJ2vOx1?= =?us-ascii?Q?kwAgDiivMrr/NPYkHOkWLL2EawZ9JXd/3nybrZD3CXk3WpIVVffUrtjq2H8R?= =?us-ascii?Q?wIn6dYVI7kU7HS7LGyNU6Mj1qd9st+NknJdTt3LOs9FdpK1QQSMSlm04iA53?= =?us-ascii?Q?hk4kuHneV/z7eHxLbkubIcWsmgRIE/+TIY4FvtFfos9vhbBXBipIs5+VKsMD?= =?us-ascii?Q?nteh7EtbvABbaCProUGvIECilmG+YgosSyZI2QrgRo8HI8T50qyvQJZgSPCA?= =?us-ascii?Q?nHq/DU03fEBCUEf1xpeopvk2cc1WFlf3YLxSkRnn4M2IOd8BhKLlT7Zv7woQ?= =?us-ascii?Q?DzDrXMe08XFa18yeR/76K9Y2IWzObErqj4XgbDhX4QZLmiCtW4mc9DVEfyyA?= =?us-ascii?Q?WJDZwrvZrL2ZJMLQbSPIRSH7m59lGvyvl3JOdW5AarU02xjPaiOLVKEGHU7M?= =?us-ascii?Q?Xj6k6TFWz/s/MtjBJmmUSLogmLluy6cHNwscoEDbVtPoH7MCJbWm/1HD7jbV?= =?us-ascii?Q?uyz7Gh5ChiwMUlOLV1WT+gpt1InjjyXdLiCsrWzIa++tMTSWfwCWaA6P/jGQ?= =?us-ascii?Q?OGlma8YJkNavoiDAautByGKO+goIdgOrhnqnLgqi03dKJLyQS46Oe9uLEdbN?= =?us-ascii?Q?bTGm5VtIYHpFqzSlbs8nl7FS/FIHGdewmFH0P34SdIuRYCNJgoPy3/PiCiXj?= =?us-ascii?Q?xFa8RxJo6A8+5l6hSpHXMUVtaBMVxefNfFMbg5BCFmnrgtWKP91w1X+ctTPv?= =?us-ascii?Q?JMZ7PYS2dc5irvk/JXumvOf3dizWM+qGAmEvAvvnMveTtsltQABLtdV+OJvO?= =?us-ascii?Q?Rd1YQ3nPBOiSEWz/7t1UBL65xVYYrEIu29MFZgeSvhfQFO4LKEal6n/cZG3W?= =?us-ascii?Q?cxpdLXT5BY+6dcdfOohzH7oyoBdt3Cqo3rHasqVwBoYyrlQ/7hkYagNKgrwM?= =?us-ascii?Q?1rmaYmD0Koc1pDJPaCuBaNWkSqWGtJ3BnUpQA88Jc51YVeCOwKs0nzOtkoXg?= =?us-ascii?Q?rX3hUJRrs9z0S8EVVokARodkmRWpGDru1PqEeMDmv1d3c6Nep44Lzj1Vqueb?= =?us-ascii?Q?IieF1hwHOn0xE4e/w9iJI0NOhHcye7zVjsLSL77ilBtYrKUXYmccgtS9pLV0?= =?us-ascii?Q?LwlE/KD5e6Ce6jyt2imkJGop+BArrtGNFiGmoyEI9k/MlXldqXhBUA=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR01MB6761.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(52116014)(7416014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?+2CTsiDHEjjxFMGHygSibKRYRz7WCD334nvbN1aNn1d+3J7cweV9A0zPP2ti?= =?us-ascii?Q?MKJdJ4dREJuFLASCndXDUP5IHnSm1Ax8mgRL3va9MSoQHwMV1lf6hJpQ5l4N?= =?us-ascii?Q?LrgDBNzTVfXNh25PRWvZYM+Ej+Pmry6ggLAH3EzGB0GnqU/at3BlBU6iggyt?= =?us-ascii?Q?t1x/6d56shnAf8aDRpEKjbIZjT92nv8cSzlGOI1yst2fZKfXzKtnwKwQpajY?= =?us-ascii?Q?gLZSp5z645Ddurt83hwxV521Hx+wQ6BOmOPYnRrj2c6pNRg5DjU0vbpZ3eKq?= =?us-ascii?Q?haTT2RuTMBGfAyCiQGltpUB8Y6fa8HeLtz+fcwVGrTAY3BioJ6QYJ28lKpDW?= =?us-ascii?Q?2AE0XTgrvJF/3hS6AjEy94Q/N5ajgEKg2xrwN7CChcL0d4VbPQ0bILfFGx/R?= =?us-ascii?Q?NkjTBadGdz/Re07wyqQaperBkdSZa9RE1yRUAwSYxet76zxjEA5/7tWVVKLm?= =?us-ascii?Q?t4HgHs6RJQ6pwmFRNfKGihRK6JWiUAJ0A9kC6a1KUeOBc12jka9Tmy38QLwg?= =?us-ascii?Q?ECaH3ErwXVxc/Zu+a3Ht935wvDNAeinVcgl06G3/hQb4+ay69YHPpYu2Jo74?= =?us-ascii?Q?+smPKawPSDLdP+mFb5nbdHjIHzoL9frioLRYj7oM2ScpUnDZLBCWYjTZbKsh?= =?us-ascii?Q?n+glBp1D37T8wCkAqY5Rp9yrhcxWcuvPcLHOkScVRWGAGUWvqx9LGjxz5nkt?= =?us-ascii?Q?38nV1tefoi5Wm+7KiA/5Vt+j3exi0b+bF+H/GZtbRYyxqAEt2WVTPLlew/JG?= =?us-ascii?Q?ShLoHthKIgktDBbdjg5+aLtd4WlnyWg7vrkhud7HFPdKkPf9mAo3dJMkGmeo?= =?us-ascii?Q?sXirEqVY5hkTGoieyjX5WANwzbhxCaAyhtUzBHtc7eV9rOYO9EODwBkN9mOa?= =?us-ascii?Q?L+BmAbyCrLyORI5V1tYXzTlrpuXoB1g1MttvJNdd7+rd14TLTMdBrYwDuMQ8?= =?us-ascii?Q?ctKYpQsx1pouCzBq3CEnMw/jEwAhP/5tbqbfmW8Z3b7bSbb3x5rtAffB3Xq4?= =?us-ascii?Q?YmiXUys9HZmaotponTRcnWeUcY/RXSiJ+iX21YETT4jh4vsHNPVmZivUpFyx?= =?us-ascii?Q?obLF+7SSqXGd6Cq7d8o9CqtlNSQNzqSROCD/zOT+8UnCyF90NB+pMRskG6Y1?= =?us-ascii?Q?xyUJIYawSTgvWwQ1ObGu2uca5IZqxnIsmVFZh7kMWvoMpNhvdyWXnc7zGUX5?= =?us-ascii?Q?TXd6uJUX8T2Cu+p6qWpYctfe72FHgPYpJMdI8bm0DNGlNoPhoEEI6s2Z1hUd?= =?us-ascii?Q?PCM1Csw5m93xFRWkHj+P4pcJTK1ayaSchYaCbM8V570fAUxevVPayofm9Yk7?= =?us-ascii?Q?8ix2lWB7rgmsgGcOdtp2Tnf3SG1NaixyV2TcwGiEzCZm2cEeywyxEPsS1B19?= =?us-ascii?Q?wPMP2k80tHkALaZzeWW+3xufGIWQXrk5aQ1elX5TcWg/uVqCepSpRP/RRkoX?= =?us-ascii?Q?v50etTXooolMJ/NXd0HKyfFbFGkpfKqRBvv9v987+0e6NFsA8jtH85WzsQjY?= =?us-ascii?Q?X/RgUQ33LRSYiJdCTESUCtAA2m3mLBoTgTuFa5TfQsHslg7SFQj4muEJRscC?= =?us-ascii?Q?FjFBwodGnZ13ZVRfzYQTcj3iES9zmb+PXxEiuanH65InEkk04PZ5SWHOdO47?= =?us-ascii?Q?FpM7NOI20pJrVdPUGQokYVg=3D?= X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 81e051b1-cf3f-4796-1357-08dde06b1a0a X-MS-Exchange-CrossTenant-AuthSource: PH0PR01MB6761.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2025 04:27:56.1174 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wCj23+B0byr6m+CJVMY+jEWeKDcthXy78fl0ZqUJ1iMzHciFg/nw7xn0PNEcJMKAgNTEPsnAkAY/pQwKFSn5+I7Y1J6xXldkuuuBL1PguDnTpxY4NjfL0tTfT29Bl2mP X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA6PR01MB8829 Content-Type: text/plain; charset="utf-8" NOHZ idle load balance is done among CPUs in nohz.idle_cpus_mask. A CPU is added to nohz.idle_cpus_mask in: do_idle() -> tick_nohz_idle_stop_tick() -> nohz_balance_enter_idle() nohz_balance_enter_idle() is called if: 1) tick is stopped (TS_FLAG_STOPPED is set) 2) and tick was not already stopped before tick_nohz_idle_stop_tick() stops the tick (!was_stopped) When CONFIG_NO_HZ_FULL is set and the CPU is in the nohz_full list then 'was_stopped' may always be true. The flag 'TS_FLAG_STOPPED' may be already set in tick_nohz_full_stop_tick(). So nohz_balance_enter_idle() has no chance to be called. As a result, CPU will stay in a 'wrong' state: 1) tick is stopped (TS_FLAG_STOPPED is set) 2) and CPU is not in nohz.idle_cpus_mask 3) and CPU stays idle Neither the periodic nor the NOHZ idle load balancing can move task to this CPU. Some CPUs keep idle while others busy. In nohz_balance_enter_idle(), 'rq->nohz_tick_stopped' is checked to avoid duplicated nohz.idle_cpus_mask setting. So for nohz_balance_enter_idle() there is no need to check the '!was_stopped' condition. This patch will add the CPU to nohz.idle_cpus_mask as expected. Signed-off-by: Adam Li Reviewed-by: Christoph Lameter (Ampere) --- kernel/time/tick-sched.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index c527b421c865..b900a120ab54 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1229,8 +1229,9 @@ void tick_nohz_idle_stop_tick(void) ts->idle_sleeps++; ts->idle_expires =3D expires; =20 - if (!was_stopped && tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { - ts->idle_jiffies =3D ts->last_jiffies; + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { + if (!was_stopped) + ts->idle_jiffies =3D ts->last_jiffies; nohz_balance_enter_idle(cpu); } } else { --=20 2.34.1 From nobody Sat Oct 4 01:39:26 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2124.outbound.protection.outlook.com [40.107.94.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2CB32505AF for ; Thu, 21 Aug 2025 04:28:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.124 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755750502; cv=fail; b=Dx/IAiFjwQtJ7VG42refBi1XDqQ/mQOddobZkZx2GTFgHD54VitKZNTmxnh3Q2F/uupgedK5xqANEnm198Y9fU+csYgbv3Yxl4lYsLBK8NoS6ukEXXwLic3s7aWI7qfBeyzx0iJjtBbVtBRX3F0LNVt/Fn8FimnAG8ZkNP3lXWs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755750502; c=relaxed/simple; bh=12r47nHopHGHbbr8xRGiN3TedMTSSuNY8Z7u9KK4qVE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type:MIME-Version; b=U/A3cPpCvWL1e2SYMX5uLyltAGMdOoF4dhH7AEmpU5xyaZvM6hrJQhOdzWPjNFaU31jxPSh+CG2gAC6GHBXF4oqbWC/4q2rAfNefk2iU/bTFmCHjNJdt0ju78nnzg5blnaV1TSarEmlCf3Vf7VloTtY3jjxcfdWZrpDveWUgMTg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=os.amperecomputing.com; spf=pass smtp.mailfrom=os.amperecomputing.com; dkim=pass (1024-bit key) header.d=os.amperecomputing.com header.i=@os.amperecomputing.com header.b=lE3HVFM2; arc=fail smtp.client-ip=40.107.94.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=os.amperecomputing.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=os.amperecomputing.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=os.amperecomputing.com header.i=@os.amperecomputing.com header.b="lE3HVFM2" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OYSH5XVqs5LCfplvSBySm0ZHmxhqWjlmpY6jhTuU6UNkKDknFbY5LyH5F2ZmsD/E/ODL5hEMw6Ud6O804004gxsDBUWGJpoTdKUeQHu7zilipX3LiHLjr97MlnwusBrUhD/OuZ7ngBDT3u27a97o5zCWZGeA4tRvVXTzlrtF4sN2eQZ91bSIICtMNR1hu5jnVy40Jnci3iQDkfE+9HixLUnmKBLA41Bg7d9h4YVyCEbNGF+50eEpk3xnLTvqg8a20AvzSETv7cdoJppxL6x7QRYI+e1ZFVyS8w0fccN5Yiyv/XgKixibzU8WoMrs54KIGeUipU1SGIRn48JvkCkHdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2CKVqqUJQyAEM53ljMgb255zDalrevfUMEIDh/at2ao=; b=FQ2bAOvOpPCMzhcUjeIIV7ZFBlvGggHnBqfpJDaTuQOjGVvSnHY1rUBnZuAma2XRa/lXhmeM8elicCiW220+9TLsVJQIWEdcdgiHQG5TzcwlfbMjiaoDP77vsg3gGTPthydnjLFnvakxxMMjMh1ypG1d44hSlRtLfWqI934yOrR3h0kZloxpqW+GnMojslszqyAj93jcrroKVWB5Tok+PbakmDk7QyBPTAcEY906Kx/vOe1sT3uQU2CodG7kinH8KYh5RUCKdFjJl+n3dnmYRnXUHQe0swGYFdAJ4dgXQ05LaiikRXPj6//DMLXlSVELX5h7xWWG2YtpZwiEiOrelQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2CKVqqUJQyAEM53ljMgb255zDalrevfUMEIDh/at2ao=; b=lE3HVFM24wK57AD6kECPO1/E2k25gngN0ZOv7IKCYVcwu70NcWGA8dydxkNgfjv5k/msxEh6pjEvMJ5x3EscrKXZuNTDWjlvfXGmaAdAUZJDC9s7YuItIRoKmdNome9B8GRsq56Mp7J1Qy6R42WoSu16QLH/y0qBMV55bkdZ3T8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from PH0PR01MB6761.prod.exchangelabs.com (2603:10b6:510:76::23) by SA6PR01MB8829.prod.exchangelabs.com (2603:10b6:806:42f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9031.24; Thu, 21 Aug 2025 04:28:18 +0000 Received: from PH0PR01MB6761.prod.exchangelabs.com ([fe80::bf98:19e8:71ee:cf2]) by PH0PR01MB6761.prod.exchangelabs.com ([fe80::bf98:19e8:71ee:cf2%7]) with mapi id 15.20.9052.013; Thu, 21 Aug 2025 04:28:18 +0000 From: Adam Li To: anna-maria@linutronix.de, frederic@kernel.org, tglx@linutronix.de, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, vschneid@redhat.com Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, cl@linux.com, linux-kernel@vger.kernel.org, patches@amperecomputing.com, Adam Li Subject: [PATCH RESEND 2/2] tick/nohz: Trigger warning when CPU in wrong NOHZ idle state Date: Thu, 21 Aug 2025 04:27:07 +0000 Message-Id: <20250821042707.62993-3-adamli@os.amperecomputing.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250821042707.62993-1-adamli@os.amperecomputing.com> References: <20250821042707.62993-1-adamli@os.amperecomputing.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: JH0PR03CA0029.apcprd03.prod.outlook.com (2603:1096:990:74::9) To PH0PR01MB6761.prod.exchangelabs.com (2603:10b6:510:76::23) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR01MB6761:EE_|SA6PR01MB8829:EE_ X-MS-Office365-Filtering-Correlation-Id: 1c52272f-a5c1-4fb4-8854-08dde06b2778 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|52116014|7416014|38350700014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?snYbbVgNO4YMT5KY+hsil33c7BoYxnfCO9dvUTkDZr4y2QzJpR5SUMe6p88+?= =?us-ascii?Q?42p1xpyUuuWFEPe27wnNHsthYUw7syQ5fK5dhIOxIE7j991zGwawJ20F7FjR?= =?us-ascii?Q?c8ZaSaEklQKjr8/z9BjZHURxuqceic7IOkJtqmfARtWXaZOiB0/U/Jo0OT95?= =?us-ascii?Q?72F0IgSqhknHp2uFDmzHPETSM1SUX8LCmyDT0rI7n8CUyyTnhwABt37b5JN+?= =?us-ascii?Q?BDqTmN66LfnS4U7dx89uQlbhcqYD9Uac5+M+QF0rrl+4qfIgpSpeJycI9m3N?= =?us-ascii?Q?SASL6yNTram90kscx/FOIryVkgVlSzC4FRfjQfXNGaZ08/9ZN5R6WIO/A5Fb?= =?us-ascii?Q?qnidRAyfNhefWTRmHBsyyGeWZ9STGhBryta08FS2JYy3op2DtvFXLvLV7Svr?= =?us-ascii?Q?JV+nRuU8F53w4wEu5MCoDUOfIWnAiILh1PhaO795XVLKmy9ZRTSaR/v4pSKM?= =?us-ascii?Q?pDkX+1IVxXVv2trNeuv8V4CN16s6wsLXbPnl06+fs89nIsVsBBcQliPY/vlB?= =?us-ascii?Q?XVD5lyYndFR9DJMSfVRcLSAPzOrcgawxCYt4uxJMzHBbvtHSqzP5DL7KeZhS?= =?us-ascii?Q?bh/Nv33SMP4BxC/ZpZkg3TsCkpTwaK+HE9LFO22nsBICPcH6+3NDH1cbhdCv?= =?us-ascii?Q?kxfrcXvNFZj6RKdO0UrH2c3d64R67fIuwJ8yukgv+sYbwSXweSP7vX0rvJRK?= =?us-ascii?Q?n/eI2941/Kwv4PxGT0/AAJHlGeAVBIc6juDH3G+OheBn4YIRxRs6I00iag5P?= =?us-ascii?Q?tBKaOKdMU1tanoWB3t+PaJQkE4ccpVaAdCmUxbWW6gAMj63C/n7iKiEC3bam?= =?us-ascii?Q?eAJpIqnLr6g/7pB52S5YFQDkQe9Xp4SdYVinSMfY8PWqJNOEVTQkSXVhgEoU?= =?us-ascii?Q?sh3LM0rIV/L6ggxE6tH8uwIHge/x9mWycQ2Cte2sA5sOdEveb+xQYemHT1/4?= =?us-ascii?Q?jB2D7pyD7rnh5wAb/3YInbmUluL3WIjhf8peuh/lRuBxNi05pXXsiW0bnuWN?= =?us-ascii?Q?WF6jOKXn1oC/gR5KUDrsJ7sk1V0iGYS29n+uEBmcPAoBpWlaDIbBHNfXnXh4?= =?us-ascii?Q?GmUuvR2JTUTIzarv7MXKdVl1kvLZBVvMHCjkmjwgtR2SgBbby6TReMQo4S+5?= =?us-ascii?Q?uOaD+ubK3g5ZOSGwcis913GgiaJzY6qA66ZPQz5oaE7B9btbCxtvuJt2fJZk?= =?us-ascii?Q?5j9q5J+DOclI1OnRQ2vcoSM1IYPFQvYBu4Sa/1i8TUx+bgHaMz5l0Q8tcJJv?= =?us-ascii?Q?W2XeKhLvL5+0C/m9wvoLpbCxJXllDzZaaoCm+VGapfiyz89wXpv2YgVyDLwc?= =?us-ascii?Q?CnpDsl7RuMV95AU6IP1vHxE2D//O76ci9SL6XGhzByv28bI01Zhs+avy99Ax?= =?us-ascii?Q?h0I3tLOUDsnqcBix6m9ol2IPAM/aNP8SBTQiZ3duS9b/XD86WOfaknyjETDI?= =?us-ascii?Q?EBzPOqBs3lCXlcm8Pm01ESMd17jcqCg05AwzSar/WFIGZrWQorFyIg=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR01MB6761.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(52116014)(7416014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?V3IarDguh9zbbGxTBLtSFeLcz6TjwVa3yOc6gIduYH8Bb3jiofcQKSnnESXC?= =?us-ascii?Q?gqfsmVLSaohZ5a4EXUEhyRQue6Vq3yqK4eI2egZo6W1xvqvCQnrZuIqyRsf+?= =?us-ascii?Q?zWar7pu5aX4frJhRJqTxWzfeRBqoLZ4AOvuwQXiY7Dp1Mx8MSbWnyl1a/foa?= =?us-ascii?Q?GjbNCItoPbkfSJ8sL0O7RjbaErRan2RWZDLse5uTZxGnOI3GfVfAAwfeeyy/?= =?us-ascii?Q?q8wP2yXe3jr8s7zY4mTIFt3DEqB3yX3empRFz8HFcvxHEX/pNc8PqnOtn/un?= =?us-ascii?Q?4x5CCCc3hoqcZiO5p63RLIRuRTvPjg/8YfFT59S0tfN8ybtkDGdXqDKGvhsm?= =?us-ascii?Q?at8WYGZxmvZQ+6PdaE4su+lLRmmdTjESccdPsa6g2q7q1HIhYXwaoTV4ALOn?= =?us-ascii?Q?bmkJdZ4zGZNgyEsoVxYdwEdl59Is4WwYMvRrQAOhXfGniK21TqHYQCK0Xw2v?= =?us-ascii?Q?oKhxvLGxAdnMOp4kSVeOoBu+aC9R+ILdP+Awx9BhSOK9kX5COeKCRMAyKFAE?= =?us-ascii?Q?9AxWqXxZ6RsZ+HboScNqWzNsAauBH19qpkDq4NLWrNY/rHDBPASHHmBjMO+7?= =?us-ascii?Q?e7VI8uzdYdRqxwgvjKoLMJy+vriD1FMFgD6c+nebKbJdCDriGdLTxSLh3l0h?= =?us-ascii?Q?4lmSJ8vJci5S4+W1HkM3sKb+Oz7/KDjXJpy4XeVD0lrB3ruKxELH8Y5C6fuy?= =?us-ascii?Q?TTYxdIanVoiV8BiwSyQuhoWJzxHVvQVKowRMGcelynPWyQfI5vKOUw5eGiGL?= =?us-ascii?Q?uJtDLxwvXMWGqCXfAqgC3gkp5ArFrCarnCVRG1jjH+q+ges1FlQSYwko8mmv?= =?us-ascii?Q?SfmghwjsqBykvJbClISu404+jlWxZGYYd5db9QkH/pYQQJCYoxQFCRPR6teA?= =?us-ascii?Q?0ZSP1HJZAFGJVKmw56RgOt1WOPCEhKLxvSDpnCbcv9rCml70hDjvPmzvlzeB?= =?us-ascii?Q?+szAraUNPcL3IsuzNoUXtjb9WhENLCAkLW+bvdp3rHDtLfvLMv4SWQtO7PbR?= =?us-ascii?Q?aKCakRLjgUg1eKBMv01+oZrsAYQEb56jLesK/2vPjkusDn0mtF21ShtP+nPd?= =?us-ascii?Q?jtXklyWg/4us411y8zeNqGNKcKqUWCq/1EMaZqltW3kljhsZqLW80WSt0CYc?= =?us-ascii?Q?ORZN7YliPWlf7APUw47Lt2+oL9kxW4Vn3ifZA54U7PHP6fRramB9lowhQA/O?= =?us-ascii?Q?al1Xj8OJVP937tsQYnas/FsEVNRVgUqQpywgQoUeYHU40zGnS7W6up5Vc0r7?= =?us-ascii?Q?e2nSzAd+5Ui9rh8Q6+KsVZ+gbkPAbw0IRiA5oGJNqAdnNk93UkZ9W0oT0Vz7?= =?us-ascii?Q?aXvrUtNXFeoJsdYc6LWrKe56vZtelkmtR/ytxfmhzMCkwx0UhLo53vViXOKI?= =?us-ascii?Q?kwt6miDisgaMmm3ecwEl/PSTtxUucmvEIs5BpCDca2CazwagRqScM89eU0Z8?= =?us-ascii?Q?ZBuiHLe0V0020QV5J89jlHDl0imS7wH2Dt39JHq6ttjmrunhNEkYxLIJMUDY?= =?us-ascii?Q?esrGMGv0u2yoA5brVEf2aax5XZHtDs2/7RFARjp1wJLzUsyMJc+6qCs004mH?= =?us-ascii?Q?o445+qzwBPsaP59OEGs/5K2h6E/Vg6ssm/pkA/QWbGUdRSNb432lNFEn9N1V?= =?us-ascii?Q?TtKa7RW7R+xZ7HbpoD+x67k=3D?= X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1c52272f-a5c1-4fb4-8854-08dde06b2778 X-MS-Exchange-CrossTenant-AuthSource: PH0PR01MB6761.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2025 04:28:18.8395 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bozDZzIn9DK2Z3LFcrm8PpVr5SxRF2xj3gEy++wssduSKfScazLwlCXj00emf0RA28tygHg57hcnsNLomWG+xb7uJw3CQogKpS5if1sbt4PvVOX258NdAOwGAc7iO3Dg X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA6PR01MB8829 Content-Type: text/plain; charset="utf-8" This patch is for debug only. Warning is triggerred when CPU is in this state: 1) tick was already stopped before tick_nohz_idle_stop_tick() stops the tick 2) and CPU is not in nohz.idle_cpus_mask 3) and CPU is idle 4) and tick is stopped CPU will stay idle in this state, since neither the periodic nor the NOHZ idle load balancing can move task to this CPU. Signed-off-by: Adam Li --- include/linux/sched/nohz.h | 2 ++ kernel/sched/fair.c | 5 +++++ kernel/time/tick-sched.c | 3 ++- 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/linux/sched/nohz.h b/include/linux/sched/nohz.h index 0db7f67935fe..ea6e07777395 100644 --- a/include/linux/sched/nohz.h +++ b/include/linux/sched/nohz.h @@ -9,8 +9,10 @@ #ifdef CONFIG_NO_HZ_COMMON extern void nohz_balance_enter_idle(int cpu); extern int get_nohz_timer_target(void); +extern bool nohz_balance_idle_cpu(int cpu); #else static inline void nohz_balance_enter_idle(int cpu) { } +static inline bool nohz_balance_idle_cpu(int cpu) { return false; } #endif =20 #ifdef CONFIG_NO_HZ_COMMON diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b173a059315c..cd1c17368e05 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7109,6 +7109,11 @@ static struct { unsigned long next_blocked; /* Next update of blocked load in jiffies */ } nohz ____cacheline_aligned; =20 +inline bool nohz_balance_idle_cpu(int cpu) +{ + return cpumask_test_cpu(cpu, nohz.idle_cpus_mask); +} + #endif /* CONFIG_NO_HZ_COMMON */ =20 static unsigned long cpu_load(struct rq *rq) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index b900a120ab54..8241b14842f3 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1228,7 +1228,8 @@ void tick_nohz_idle_stop_tick(void) =20 ts->idle_sleeps++; ts->idle_expires =3D expires; - + WARN_ON_ONCE(was_stopped && !nohz_balance_idle_cpu(cpu) && + idle_cpu(cpu) && tick_nohz_tick_stopped_cpu(cpu)); if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { if (!was_stopped) ts->idle_jiffies =3D ts->last_jiffies; --=20 2.34.1