From nobody Mon Apr 6 10:31:52 2026 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010032.outbound.protection.outlook.com [52.101.56.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB64A2E92BA for ; Thu, 19 Mar 2026 17:26:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773941179; cv=fail; b=hkqwvcu4fW4D4wrG+LWE0ZUmR/8MK6o0oUgXuAIJbkGUPKzTSMob1ubuB3htrUAl6lpnr3PuVvEnvhS36b6o1GGdKB95ohO+fBJj49v93KYxzWJIjWrUs3t5kklJMRGkdmtl4bQFEWQ/uLuRjdP12jhhMFHHGSqm5Nfq0y+TCtw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773941179; c=relaxed/simple; bh=3zBr7AqqpPDQsIyqM9afNUzNHrK8A7i04pwToU5bLnw=; h=From:To:Cc:Subject:Date:Message-ID:Content-Type:MIME-Version; b=uqBbzoBRSjrTJE72x4h7iFyytGSVGLfDyNGFZZRJsxSUw+fJe2xOcl6c4V+CjE4EYpjK4Gfhj/mDNZWgPRM3n0XfTK52Wrm3Ze0jemPMrnQkElKQbGQ9BMY/2QwkB9e5PwsTVQzhvsETed0fRFdu3CATT4bk151NEB61t4YpCTs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=UC62TaD3; arc=fail smtp.client-ip=52.101.56.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="UC62TaD3" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fj7uEkAdGWzl5Da8kiwVK4PSH/VaxArMLCW+kdugzk9l3XsGPtUhyqQSn/JwruuXzzV/5Sh8hKYVsdfxiYA/+4wB3Y/xFtq93+h+J+jBHXG4M54Nh6nUbuATTDoKpZrWs/CF/vztrEqw20Ti7Ym0wqJWet6F+SkMwAlRXTmAukrWWVds5cuzSU36Xiwx9y+XMjBq6X3i2KzMKeN9mgWFsdkllk/I6MM/SrYY0FRK14ln5Qb39Yfb7LpbvzWXI/fygXypGff1vZ3Een1c4KJEpuG34DrK7siqk0lmrexKPeHCCxUm1j5EkZy2VdxIJ4DGZwCIuhQlNnYXfCfxwARdXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CjymvdCEQrkUOQHvXrTMuNlq7TkA97QgRVYe4YDUuXg=; b=IAOnyAXIiaX2XBXWWiWh5ikr+dj97Eb5E9PhBq0QmD7D8OxRaRXgJ/P5VIJRGsdHUuBzRz5kNnGOh6ePVR9Ppvqe2KDjpDkjqgKoJCcE+ZrZFQv8T+zGllR6RPKRbhVW8TElT/hxATw9wrrSETVgvbtuE5FdVEg9qJyevAO608QVNl2jDQqrjO0mQ8FTKl9jJ49aho99vvP9DeCidvg4uBFLDThj8NADFE+YHDsRrPgy8QYZtjS2QApP9GOneRLhCkym2IUB1TFgumOoqRBEmQSniLOpyesmmJnaxQMAOeYcxXE+9ANljaLumrMsY1WlFDZjYU1ce3tAbchnTjgtFQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CjymvdCEQrkUOQHvXrTMuNlq7TkA97QgRVYe4YDUuXg=; b=UC62TaD3R2DEPyE08KIydW8Lp2p1haPMEoYgiCb+c+z1pSXveWhZJbuOYU/BbTUfjlCLSTlVR74STyCX5ztV7xJGDSaudVXfTTCVBzKZ1K2A90JQE/mmT5XltfX8GhMso7UN4K1bjps8rRY604KGTq9w5n0VEMLszizdsYkwULeKGUqTdtsSzLG1ArNCx+AuO89HrYBWXz0NFtJ+50OTlOEfnelRUBNijbHAWxf1KmjQLrlSbG9XLEX2FVQprZp9Am8S22a3vOEyLKgt13NQY5XCEcnmvha/g8c1j4Nv7mRrERG3pQkirGE5XIAPqA9taBPlDYNs49KJMvvlle1ghg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from PH0PR12MB8800.namprd12.prod.outlook.com (2603:10b6:510:26f::12) by CH2PR12MB9496.namprd12.prod.outlook.com (2603:10b6:610:27e::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.16; Thu, 19 Mar 2026 17:26:11 +0000 Received: from PH0PR12MB8800.namprd12.prod.outlook.com ([fe80::f79d:ddc5:2ad7:762d]) by PH0PR12MB8800.namprd12.prod.outlook.com ([fe80::f79d:ddc5:2ad7:762d%4]) with mapi id 15.20.9745.007; Thu, 19 Mar 2026 17:26:10 +0000 From: Yury Norov To: Peter Zijlstra , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , linux-kernel@vger.kernel.org Cc: Yury Norov , Yury Norov Subject: [PATCH] sched/topology: optimize sched_numa_find_nth_cpu() Date: Thu, 19 Mar 2026 13:26:06 -0400 Message-ID: <20260319172607.926280-1-ynorov@nvidia.com> X-Mailer: git-send-email 2.43.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BN9PR03CA0420.namprd03.prod.outlook.com (2603:10b6:408:111::35) To PH0PR12MB8800.namprd12.prod.outlook.com (2603:10b6:510:26f::12) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR12MB8800:EE_|CH2PR12MB9496:EE_ X-MS-Office365-Filtering-Correlation-Id: 8ae9cd6e-30a4-4aaf-ec76-08de85dc9cef X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|10070799003|366016|7416014|376014|1800799024|56012099003|18002099003|921020; X-Microsoft-Antispam-Message-Info: ouj9XvURK0G3lbdbFY44Kvhgj7okWpaU9dZYMYQbAso1n0ROATM/wnHQ5K9DuyLIvCY3wilaW7b8fkuiLUuWiehW6smSABeWNl8Vl+xm/J5eHmIOeNGfqz8vZUwqYdxYFrd0uxNl9nT48bBmzg9StOs+GFQE1t7/u7Fo0XJDFPHL+kYkK41cEt3roj63kqJ3NQUuta0MPM7jW00RfvKIRGBCupFy3zR1D5xvVe5c+ZjFZW/U62rRtoEl80xuGQICGErX3/86SsFX9/eQulvZ/3vRH5bV3I+9AmNSpxp+q6g7T0CuUoiNolsu4AUj63a2FPaHfBmrUsYTG79571RR1vONmsCQ9G/yKovgGgcJ4jJRKZVCudsxyE0pcT1vjJ5cCwVSy+I+UGClogpa7sC9pFyhiOLcaHar9f8bugJXPP9mRC/IdIaA83LSm7ozoXWATAq76rPx7/EsbQ1siHZ9Bsfvp+vZ/kCOnl8NP5TkmsSO9ZO7cE+ZYWhWvXrWHp5yd/9aPodIDSA1/SqUSg0O4johlXeAY78ZE4umQ+Hy71WY+mXdpKY7CcdgYPMoy21i3UEASs09No7WDBBmUTLZ57NauKl3U4Ao/OBOpaHRuaZ0/++A/SSZQBL2SDapu+7/k3uzrVqkIRq9XIDgZu10tMeSJUpqd1eqSH8Rl6S3nCZMwz3YBPvv4XD9IaSk0TeBPauEt0W+HgC72w9VMdzK1hznRSYK9Phue4x6rH4xla+yqCXa8AvFi0Df0o1nhPwxLN1PLdP3q05LpGhhSWQ9JA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR12MB8800.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(10070799003)(366016)(7416014)(376014)(1800799024)(56012099003)(18002099003)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?39DPUruPgm37x5cjJq4M/vDyYac0lo946RQggBUhIdL5fdCVRAKzhs0gferQ?= =?us-ascii?Q?P8P3QXWOf5FeCzdpZHLpl/mbMS+JNe+jj87cvtc757FvXB1iVaBYgafabd69?= =?us-ascii?Q?iLXqDAmpJY05HWm2MAeJI+cxd7+IUtG84T3ANbEX8vYACTVMfmkbWqp6VXcD?= =?us-ascii?Q?/gteQGlY2QrO9xMx4iR7wnh0oMjZMh2dJZurHRVB/Uh9RspmHRepIXd7z3RK?= =?us-ascii?Q?RHP3N7f8MB5RRDsJ5WOWz5shj4thE9h6G3VY81DZshZSISbWyllBURAt+JxD?= =?us-ascii?Q?Rgw7vO7U8ZBESg7FWQsO0T2tEYcVq2DTGlJZ9a2Omcy2L9HZViB5fwsJuOHv?= =?us-ascii?Q?ECAP/NvsimktJqn3FWQMOa9Cbb1sATlb6JL+r3JFB7xWANe/ZapUXkUCttdH?= =?us-ascii?Q?t/zkQO45Ex4Wp9Hi3mXKecw5RsSTEpWVT20MhmKJZGJe9crFrTu+H/fbunri?= =?us-ascii?Q?thbtC25mNLGQ5caNRxr1vfVoSBbZCKg0XYj+YTgKeI4JcqW1GE9WSxiRX34J?= =?us-ascii?Q?xfSvVT+3r+nhyFfxtGIjku64COH5HsJYf/BURLjErQuWAZ4PMlkbOjRwZ++q?= =?us-ascii?Q?DblVE/sSEIJJwtFjAni6gkvEfHVl+h31x30yumHMGhFfPTE5j7C6WMzgMhPL?= =?us-ascii?Q?DY5ga7BXxv0Arf4WXiJA89IWCamgJ8wD2P6CYwSx5q8X6cWoqmjvaybdPhX3?= =?us-ascii?Q?n7zr8dXPexsqBYymsN727+lQ+A7gOMgbowMwc3JLL4L+R09osXwnpKguHu8p?= =?us-ascii?Q?lzLyvjHAflmdC6ZC98TGOdmaHdElMXLrOkqTxjQw6kgowr1VpoKu+9I5isOz?= =?us-ascii?Q?ISNIM3x2phnRhLaUfuiwlkNNnjbNNMEBo/tWJMAPQDyPPIjRqKtYau8OgPvu?= =?us-ascii?Q?AXUUChjeceKQ97d4Bir4kWi1Z8SfduccQE6bbcarGWkDpo1N8YyH0Dyy7gn4?= =?us-ascii?Q?NxtqUMJvqOAIPo9zCE72pHZDnOfOsEX4UT8IvEjKM1U5sE3jFaEopUIEMZ7T?= =?us-ascii?Q?zM7dej/2zeKfwkX5KchoXYFlJ+ZH4VDRoamnWK+xbyY/nN87OeOheN5U+oAg?= =?us-ascii?Q?ESt5mufnGDF+TigjeEplulVsB2NLg/SsMC6miwj5zlcHo59c0/ZLYu9aQ/W5?= =?us-ascii?Q?QvwYRtOI0RUCBgh4JfkGTzKtkmzM6e14BdS7BA6+rVHa8ARcdNi2TsgB+kM8?= =?us-ascii?Q?Mdi57Z4coBPhIzusi14kbUR6Wtb4BhntV4Qx+xOCmOYaqwuSI06m+cko1bZY?= =?us-ascii?Q?9aVKCROSm8iq+3+YJyCcI3mTwEOJBePFjYxPXkkPrycB4/fo/jpQ3B4yp5Om?= =?us-ascii?Q?QXxsAxYZyZJlsXVjgaDzp0epO7fEQY0Gfk5ME7G4iNGjiiCQnJLNjnWqGISy?= =?us-ascii?Q?QhEaa6Y4v8qNYAPhOVlQn3ySUGQD27Cqa2TaXxbGpnyAOKXXYwiWrvwIVBnx?= =?us-ascii?Q?zxllqKr1Ts6WqXRE9dAIWlRN60J6DoHKtFtS6iJtSG94ZVRZLGlCg4CnusRy?= =?us-ascii?Q?VIAlFa/XGkvm5QRqaWg8b6+OSseywlnNNC+HUMo/K5IgdK5YlPw5il5kDQ4M?= =?us-ascii?Q?oS3EslVTx+K5akCwmtFvgsP3beLw2LCJ/cTJmVAeoSx+EcR9LgwHeut0R6wM?= =?us-ascii?Q?iMrf2WT34kDGKvmC5HCqeJWAQI0Hl+NNl9iFsAzOYvl3TaPEqtATjjwdTM8Z?= =?us-ascii?Q?akKh4KsgOD5RapXMqv9x044yRMZIvBQ7KFNFRsiDZZkLnznHnvpioVX32UDB?= =?us-ascii?Q?lVbt5t0pCTHFLSwCW9nP234v7Va8HtFNNhPFsMDOUJRbHpNIC3iQ?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8ae9cd6e-30a4-4aaf-ec76-08de85dc9cef X-MS-Exchange-CrossTenant-AuthSource: PH0PR12MB8800.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2026 17:26:10.6870 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BmTdChPFKBp7IxDEFsN+sSZhtbk/+IinEHqQcirHkbgZs65ul2K57nv5bCwgt4b8SETfrt+VmDCDR+2yKLv3ig== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB9496 Content-Type: text/plain; charset="utf-8" The binary search callback hop_cmp() uses cpumask_weight_and() on each iteration. Switch it to cpumask_nth_and() as it returns earlier, as soon as the required number of CPUs is found. Signed-off-by: Yury Norov Reviewed-by: Valentin Schneider --- kernel/sched/topology.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 32dcddaead82..0437f5ded37b 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -2267,7 +2267,7 @@ static int hop_cmp(const void *a, const void *b) struct cpumask **prev_hop, **cur_hop =3D *(struct cpumask ***)b; struct __cmp_key *k =3D (struct __cmp_key *)a; =20 - if (cpumask_weight_and(k->cpus, cur_hop[k->node]) <=3D k->cpu) + if (cpumask_nth_and(k->cpu, k->cpus, cur_hop[k->node]) >=3D nr_cpu_ids) return 1; =20 if (b =3D=3D k->masks) { --=20 2.43.0