From nobody Tue Feb 10 20:06:06 2026 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2067.outbound.protection.outlook.com [40.107.100.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9768D3209 for ; Wed, 1 Jan 2025 18:25:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735755905; cv=fail; b=M2NBsXaDAgAdMzeiE/Vtiqm4s5x1LnouW4GanNtR4U8YD5CwpZda9a0jRebD95B4IgXkfsEq7yTypbk8yyPU/tCi4b/+V98aJjGQHERDSMmud01rDJbT69TTLq6bBzj0IQfELmrrRHsVzjlTZLSZwHkpRa4bksjIMWObgLr96U0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735755905; c=relaxed/simple; bh=jwL3CYY4UuIULZhlOJGJ7kFcVyHkpYw8Le7KuFFdulQ=; h=From:To:Cc:Subject:Date:Message-ID:Content-Type:MIME-Version; b=KBnGqx56rQDxX4bnfbVSEbT6TUn0JNLZz/rR0RJFmW67WSz38tLl/dytcUa5qFU0Ly+J1CWfBYV8k+cBx5bjXEbzyhx/kejlerkXz6DMyNzjzm4z9FqL7feAC2O/qp61fqXJgTv+YD4Qwk59mKrO4WctnXC7a0YQ09JA93s/VTs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=C6YZIhZ0; arc=fail smtp.client-ip=40.107.100.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="C6YZIhZ0" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QV5GGU1yLBDvlOAdnycKT53jkpJXuxYrOxV9xZdT4zeSclyyz+QqtFmTzDCXRN9Tcc76vf2awEsCsdFgQKBb3fELsysaEYmAAHH1zJQXt8Eu7/AA0QU7mM9nit2YTIi32koXngf6O1M/9bT7Qx8Susiqnbbwe/U5mC7g2yh8YIsE53ZJ2C0GijDMrdVDMrR6GOn28CoGEHAOvDqQmdJ1CqYRHK1OBqvPl3vxBQpMVLe0Jfq4FM23dLXY508aeN+osCITmLx8nHAHXJDWaIhvo79DUV29+g1dWGJH03NU88GTcuPk1uTjwrfsRYSXVQxhrxZvDO5N7rBHQEKLTNaUtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/qeYU43t1oLkqmMcAPGFcJxllwTnVfJs2Ygx9xDhbjA=; b=Bxo+5qV0hHetYBUY02RzufiWAMIn30BD2MaTk/iB3j++SWKRQ92l8+3g1yk4BhUS3127tRqz7H6YTBA7vUC5MFENRDq33KONn6USWKk5HkhuQ6XDGtEZETN+2J1mDavGsV/rpxQ7pgJK5snP4ks3pH9Q9lcP7lwn9xgrzH7i7Bg4lyCeNSBlax9RGhcGSzYj7AGD7puo1446h0Y2aqYqpHLtcEV9Yl6rSWXOtTtIr87XmEUGUGaS+bH9uWAe4hWOSEtODViC6mopdSnKy6m71XILqVGxXNTIzFX4RdzGWKDDu8JTysAufbcp7sVmyu7OKcrbrtSdxCbzMLOU5muarA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/qeYU43t1oLkqmMcAPGFcJxllwTnVfJs2Ygx9xDhbjA=; b=C6YZIhZ0E5nvrGm6QmKflLTBNNXOJgG2QLipgyy+ZECFaNBfraoD7fi9w4vN1d4hti9eCxaQa9kuccHvrCbI1zwLTmoWaOnBcRln1b9EuVoYx2+ohasjLkq58rU8sTY4rTEmIGHAAdT0hy/qTcIDXRAAFszy4Vg/01w8M87Lwb1Qz0Y4PjsgQlhjL5+3G6j9uRfl5RDtY1TJSWlI7XEMCS9tw1SsBLiA1X7bTHES3xQG5zgLM+yvu3FBAdvn/Kcs1kHyGKwg4u9aLpNEt0M5ZAVxIzzuZzHHwf7B0WDUgBAvdgvy3pi+yT4mCRQynsFxdYOJH3DgvnLkx9R/stKbYQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CY5PR12MB6405.namprd12.prod.outlook.com (2603:10b6:930:3e::17) by PH7PR12MB5855.namprd12.prod.outlook.com (2603:10b6:510:1d6::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8314.13; Wed, 1 Jan 2025 18:24:55 +0000 Received: from CY5PR12MB6405.namprd12.prod.outlook.com ([fe80::2119:c96c:b455:53b5]) by CY5PR12MB6405.namprd12.prod.outlook.com ([fe80::2119:c96c:b455:53b5%3]) with mapi id 15.20.8314.013; Wed, 1 Jan 2025 18:24:54 +0000 From: Andrea Righi To: Tejun Heo , David Vernet , Changwoo Min Cc: linux-kernel@vger.kernel.org Subject: [PATCH] sched_ext: Refresh idle state when kicking CPUs Date: Wed, 1 Jan 2025 19:24:49 +0100 Message-ID: <20250101182449.21517-1-arighi@nvidia.com> X-Mailer: git-send-email 2.47.1 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: FR3P281CA0208.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a5::18) To CY5PR12MB6405.namprd12.prod.outlook.com (2603:10b6:930:3e::17) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY5PR12MB6405:EE_|PH7PR12MB5855:EE_ X-MS-Office365-Filtering-Correlation-Id: a2288bca-edab-455d-634d-08dd2a9196b7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?pyzTk/3pF1DT/6QJJDbLhCW8OaSPnU3k9Zbiay3+fBKZbF/s4vv7z36sewoJ?= =?us-ascii?Q?vhp6NoZfkP6KMYO/hGV4tKqTQKY488cSYwUari99H8oCqTJQpwLC7TR8+Wez?= =?us-ascii?Q?rVHTUlNqY8ZnVGMmEReCM7Gp4o3WntC30YkX9RSlaszSZTUwV0SWf8hxqikR?= =?us-ascii?Q?IRUVCqgjMQAqgiRaIxogARndC1KntELgx+2ZpI1XZvzp4jVNDC3jXXZPPGS9?= =?us-ascii?Q?Yyr1XQ5fzI9gFoiGaEbqDKBQtPpLeYaEA3GoPCwckIrf0n7br6f92H5wJWt3?= =?us-ascii?Q?/qmy1QGYT41usMHBnOOa4tS1Esak2eteMRvzK0PHJlVrDs4Pe07ugpbZ7B+j?= =?us-ascii?Q?iZ4zJAfZdP+z8hcMbxgGTIMGlQAi7J+uviDcdbs2FgzhSq3+ucVF1P618JbQ?= =?us-ascii?Q?q5P+Id3K3MlUYLiD8lw2zfOYZqCUgCMuA0zrpaiic8NRL4HALuIKrOeUQ4+5?= =?us-ascii?Q?jIcQ2PcDKik0JYn00eZY7dPK1W3L5sJQWiicZvBwNlgY3vMUsmve11ejwLv8?= =?us-ascii?Q?d0/UpTsarBgYifT1qxCz1EVxL6FnlyVNceE47VbQ2E1J/1hc/IoSKMTsKsMg?= =?us-ascii?Q?NBLJqtyjcYfb8+K+lhOYbXTg0qjgctFHvvFk4DR+5e5CmohUHTWHi6wNHIQ4?= =?us-ascii?Q?cOTc44lN7KkVKlZglfuRQBayfZpuO6PeKaBXkkmgwNju4LzC1h6iMM4DXtjM?= =?us-ascii?Q?abHxngzVUve1nruvmUhDbNcHWXr2B+ECOoW4gidk65VDV86iIdnPDPMS4+tW?= =?us-ascii?Q?9Er1/3U3QLI0on0XDZw7a+aBQNDsz3Ah6xfjr92h/d9NoOotw9di9BkTegIg?= =?us-ascii?Q?7FZ/7LjFHAr8m8i7dDmMy58s0rdOMp1Y3J5YAsXR6lp0QnV7Cn8Ud7ku3i5j?= =?us-ascii?Q?V7gntYwUWex/e1AaFJYyLq52x4FHTN7kUsN2niZJLr841N8OD7Gh0bzzftZz?= =?us-ascii?Q?1dXHlLWMRhhRFt5y+BKRCGCXno9CURrWPoVV5LhWzA4dcNp09XIjVPqVwJdg?= =?us-ascii?Q?k4JYcKjshLY1HgQ5Zok24ejUU3T5cXXEmRVoEtFr1VlT1oYgH3HuHHU9kUbu?= =?us-ascii?Q?XVR91cCkfBpU4TyZl40dx7iq/IQ78DYZr+UJsEL/Y3ieUrcwewMSeDfXgCP/?= =?us-ascii?Q?mIV9cpAni54hF9hTFKBKaVJy7FV4rA2QclQ184wd9NtzBnsPrH2/RFR5aonC?= =?us-ascii?Q?G9zkSgsiB1kFW3Mhnk6mqpw1bWzLyku/s0HMRaC8fPwgfh3O+1SnSt4tHzrg?= =?us-ascii?Q?k4khiMQYdy8IVidewLTbgvjaMfjaVNkO92Xp3d+VKVZEVdMTtLCtUBMmZb0M?= =?us-ascii?Q?8Kw36IdyED1W9GftAwbj5odHvV54a/LNZKtGKmZpNA0jhp/hHcGhkOgqGIRX?= =?us-ascii?Q?0w6mS9wsX9tVP0EiQwTEyvJ7frK/?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY5PR12MB6405.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Bakiqjg8HNEcuU34oKMfoe8eNScp557UsJs/C5GG9jrO08UFKjAT5dixq6i7?= =?us-ascii?Q?Fe5lsxIebbqTXJih618j2qmgJIdKRioj+mpe+ck+EZsDvRKOup3lwF+UKg8k?= =?us-ascii?Q?x8D98rgfpf6awT/O/fVNlXbq1r+IkVRwiRGBtb0YevLn4Ns9+ROZn9iaGVKW?= =?us-ascii?Q?ytqhDf4FE5IMyg79jD9I15WPF1yyfboyyaNed/1U8l5vfvC7cwcNSoBuOd1J?= =?us-ascii?Q?ufrOyrv1fYyCkK6prseMebSfITpc6WGZ8p3s+9IvgYr1qmLlG9fczI15aiFE?= =?us-ascii?Q?EowrQ1jBlkyEgfLtnDhuX9+62ZoNoC9QVavuug+PCxBI/DYdq0yGGC1krsF8?= =?us-ascii?Q?AwDjG1UhH06UtERYBBGTaA5fdcgusc8tRYcsU8tWn50n0/Ad07X3WLKa7ZuF?= =?us-ascii?Q?An5Av8cNhVCT1oGUVGYUJxEDVyQA7TgthBOm6990cisTe339uZ71Iq1SvLe0?= =?us-ascii?Q?/1qE07xCSpGwZpjZT5RD3nowJzroMi5zOlBnCFv9TSsWqFKpekRgFtyDkOSF?= =?us-ascii?Q?ECf8btKiTrJZPKvWar5uVXxCtCWKAk6tgAve5dqtFjoO/ABgMT45da44oawG?= =?us-ascii?Q?Ku8egiCDTXEkWnIP0L2WlJiMBM+xsHkVuxrp/b1kFE2sOowFhw87btBsaaKM?= =?us-ascii?Q?VoTlZqzBnTB3j9wJvCNdmdlrPOTwEsAF0gUX6tqqIx8cbmduaP2nLVFBB2/n?= =?us-ascii?Q?VMPMUgvPQ+U3W2x1dEzmD2yN+4mDSQU4EVoW2+B5wgYrclMiYrOD9GGNpyeG?= =?us-ascii?Q?hnEPWTSDW5z8b4TzxOfAQFoNMEHiqSbtWRbc7qZxmzGEE2o5RdoVoRTfOZLe?= =?us-ascii?Q?ikTNBeiSq8eWM2PAsjGARY47naVG7Z1qBzSBtG3M4L0HX1PKjlvjLDdkjQb2?= =?us-ascii?Q?6aBe8+TBiITha20t3FN0KnIGGOq0kuhJGV734xxHMZiNhjTIcAjZ822vRhAY?= =?us-ascii?Q?sRQRbHPyoWWI5VWC1FaFz4dOO6Kfz5+UW2WeqMVVVSQsndpeYxaLKp6GY339?= =?us-ascii?Q?dU6Qa3qoFVGOKxZSJ5l2ZhQqMCEzVEX21zw3fJHo+wbQ+jx45EN11IjWiRUL?= =?us-ascii?Q?KU6G0lAmSkb0ZHU+E6+QXfyn9X05lO6gvU0i7nPqe8lE47H4zgVGmunx9CDu?= =?us-ascii?Q?X++I37sQ41Fj+iZaYCI6QJyaMiH2umZh5I8AmsWJnYS3UFiMa4s2XhJIqnDq?= =?us-ascii?Q?p3XLqM2BpBHYD3oYeuvBrQlRTZD97JVtCSfsOQG+e5tZL+Mo1Dy3Sae1x4uO?= =?us-ascii?Q?C/UiHdjkb5iDuNDC2EiXVl+5a1Kgydy1ZUCazlV/yiHSlEH+fFvEa51gL+5Z?= =?us-ascii?Q?qlmUq1wHZalfgjnyCz7MHBVJF2jjcwcR9TKukGHgAOgXr18ExXzfmtWjYTIj?= =?us-ascii?Q?19pAN/Erh3+8YB+X+LUvRCKvk8vACjLFkS+jLYAU3cGa/esRM7b1j7HQfKs1?= =?us-ascii?Q?sqlanDBGJ/Yv4JrSmGJDBSgYdQHk3RkNskeZ5V+CSIaP51GNV9CTE/OcA3+j?= =?us-ascii?Q?JAYKGPq2pV2hvhGZlYxIN74GCEPAomCbVpfaQ8bF5mQ4BPR+hrS+vKMxrszK?= =?us-ascii?Q?cKyIPnL/BIhs7sSDxKv6W7/m74jEPd3X4nrXZKtV?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a2288bca-edab-455d-634d-08dd2a9196b7 X-MS-Exchange-CrossTenant-AuthSource: CY5PR12MB6405.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jan 2025 18:24:54.4753 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Z7Dw4J+jY+SGFSQloR0ADAQBGGbd/a+v4y7liPt+aTVPrpH9VYSC/hECcGUMhmvyIAP7thze7lSdGbcc/zFOjw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5855 Content-Type: text/plain; charset="utf-8" Selecting an idle CPU and marking it as busy without directly dispatching a task can lead to scheduling inefficiencies, as the CPU remains incorrectly marked as busy (even if it returns back to an idle state), making it ineligible for selection in ops.select_cpu() and similar operations. This results in suboptimal core utilization, see for example [1]. This issue was introduced by the pick_next_task() refactoring, where the in-kernel CPU idle state is now updated only during transitions between non-idle and idle tasks. Previously, it was refreshed during every idle task cycle as part of the put_prev_task() call. To address this, update the idle state when a CPU is kicked from idle, provided no task is queued to the local DSQ. This ensures the CPU is correctly marked as idle when not running tasks, avoiding scheduling bubbles and maintaining efficient core utilization. [1] https://github.com/sched-ext/scx/pull/1139 Fixes: 7c65ae81ea86 ("sched_ext: Don't call put_prev_task_scx() before pick= ing the next task") Signed-off-by: Andrea Righi --- kernel/sched/ext.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 926579624c41..bdee66e7b353 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -6079,6 +6079,19 @@ static bool can_skip_idle_kick(struct rq *rq) return !is_idle_task(rq->curr) && !(rq->scx.flags & SCX_RQ_IN_BALANCE); } =20 +static void refresh_idle_state_on_kick(struct rq *rq) +{ + lockdep_assert_rq_held(rq); + + /* + * If the CPU is idle and the local DSQ has no queued tasks, update + * its idle state to prevent the CPU from staying busy even if it + * returns back to idle without executing any task. + */ + if (is_idle_task(rq->curr) && !rq->scx.local_dsq.nr) + __scx_update_idle(rq, true); +} + static bool kick_one_cpu(s32 cpu, struct rq *this_rq, unsigned long *pseqs) { struct rq *rq =3D cpu_rq(cpu); @@ -6104,6 +6117,7 @@ static bool kick_one_cpu(s32 cpu, struct rq *this_rq,= unsigned long *pseqs) should_wait =3D true; } =20 + refresh_idle_state_on_kick(rq); resched_curr(rq); } else { cpumask_clear_cpu(cpu, this_scx->cpus_to_preempt); @@ -6123,8 +6137,10 @@ static void kick_one_cpu_if_idle(s32 cpu, struct rq = *this_rq) raw_spin_rq_lock_irqsave(rq, flags); =20 if (!can_skip_idle_kick(rq) && - (cpu_online(cpu) || cpu =3D=3D cpu_of(this_rq))) + (cpu_online(cpu) || cpu =3D=3D cpu_of(this_rq))) { + refresh_idle_state_on_kick(rq); resched_curr(rq); + } =20 raw_spin_rq_unlock_irqrestore(rq, flags); } --=20 2.47.1