From nobody Sat Feb 7 11:52:16 2026 Received: from mx0b-002e3701.pphosted.com (mx0b-002e3701.pphosted.com [148.163.143.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38EE53242B1 for ; Tue, 27 Jan 2026 17:38:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.143.35 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769535511; cv=none; b=f3DLKwW6VDvvhm3FkLXbYnwOcU7np4zRc4PETPWI14BFFGe0HuOBPNVErjcpkj0fqfNjnEqr1vHNmzyQK2SzwmGj8h7fJ3GDREY0CbzMLf4jnu3zQvheN/HzQGmGfW1PziwjKNkTqpzEMqZeuu6q0+EpC/bGQFBeL6WGqu5Mhm8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769535511; c=relaxed/simple; bh=Q5BMRurgWmYTDYFbBTZmglCpbelZR5+6jvKSIK3yvzA=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=Gi3WTvTy6F5fDa5jd+FYZDtnt9kwkSDp0JVwfk5ZNKGODFBE5r+CQey80YD9TuyXSGUILTzM6GZ/G9Coyn8WpsTVoz4ch16FM8j1nvrc0M313ny8dDa4eeJmYxs/v8luzw+Zp6wJvUTfim+vGDRdZ121D95lp6C2MsL1j7ieetQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=hpe.com; spf=pass smtp.mailfrom=hpe.com; dkim=pass (2048-bit key) header.d=hpe.com header.i=@hpe.com header.b=JSyol1PE; arc=none smtp.client-ip=148.163.143.35 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=hpe.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=hpe.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hpe.com header.i=@hpe.com header.b="JSyol1PE" Received: from pps.filterd (m0150245.ppops.net [127.0.0.1]) by mx0b-002e3701.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 60RF1lXK4011206; Tue, 27 Jan 2026 17:02:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hpe.com; h=cc :content-type:date:from:message-id:mime-version:subject:to; s= pps0720; bh=h6w3ePL7ghodixWRD7l4aNJAbevXUVUzU4KX+KjZmBA=; b=JSyo l1PEjkWLOPeio5yq6XLZXNQoPlif0c2YDYKvkuymXcv0QBP641eCnvUhtpmbz6ql 29hVf9MEj89rxxJgBHJsuEARGQz1r8OScw0sG1xmXbBZjtDpPPL9DYmO3kXg+QMB Q36zg+c4pMiV2hGR1P5Udf4w4ywRfhaImmAyShe+EoZBkwWvTFah1a6GMPVWNGjf QnPNSo3K293tVCJnwIc+U93P2tEZ6j1foQ1Rk9EsB1oxkaHKUygiH5VrD6CMG5RQ VhbpE7V0rN9VjOlBYl6AUn4gIkUN/eK9t+3PvURTprE4Ps0gUji3jeCK68GoHQ1o iRnieu5fECdAvZDtDw== Received: from p1lg14879.it.hpe.com (p1lg14879.it.hpe.com [16.230.97.200]) by mx0b-002e3701.pphosted.com (PPS) with ESMTPS id 4bxymw9tmd-1 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Tue, 27 Jan 2026 17:02:58 +0000 (GMT) Received: from p1lg14885.dc01.its.hpecorp.net (unknown [10.119.18.236]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by p1lg14879.it.hpe.com (Postfix) with ESMTPS id 62E0814790; Tue, 27 Jan 2026 17:02:57 +0000 (UTC) Received: from HPE-5CG20646DK.localdomain (unknown [16.231.227.36]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (Client did not present a certificate) by p1lg14885.dc01.its.hpecorp.net (Postfix) with ESMTPS id E1E83802805; Tue, 27 Jan 2026 17:02:55 +0000 (UTC) Date: Tue, 27 Jan 2026 11:02:54 -0600 From: Kyle Meyer To: bp@alien8.de, dave.hansen@linux.intel.com, mingo@redhat.com, peterz@infradead.org, tglx@kernel.org, tim.c.chen@linux.intel.com, vinicius.gomes@intel.com Cc: brgerst@gmail.com, hpa@zytor.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org, patryk.wlazlyn@linux.intel.com, rafael.j.wysocki@intel.com, russ.anderson@hpe.com, x86@kernel.org, yu.c.chen@intel.com, zhao1.liu@intel.com Subject: [PATCH] sched/topology: Check average distances to remote packages Message-ID: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline X-Authority-Analysis: v=2.4 cv=OvJCCi/t c=1 sm=1 tr=0 ts=6978efc2 cx=c_pps a=5jkVtQsCUlC8zk5UhkBgHg==:117 a=5jkVtQsCUlC8zk5UhkBgHg==:17 a=kj9zAlcOel0A:10 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=MvuuwTCpAAAA:8 a=6ySUtH4C5LIAE7FatAYA:9 a=CjuIK1q_8ugA:10 X-Proofpoint-GUID: vbC1shvkFWrLtVIKiTWXiDyDVFc-lYeQ X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI3MDEzOCBTYWx0ZWRfX11CC40u7fMv5 a67+ldR2K61n/Ie7mRMth1FgVsWEmnE/mCBbHSAwdbU0M41CU0X/tX6Eyte/S4lB9EuFWIvMvrU P2K3nqaaLZ/GPNKmu0D+QFF7b84oqwAJQjzgl1DxmfLIBwKOHwFf2+XDbaHVaAyYRD1xgKaTWPh bnfmsM1ZEQ+zScfRlzvTikPaoL5UvtUlmasv5AphTCPn8CVKr/nGeZNoRyqUHwFBeRK1KACjbOV Gn1RZBIbX94a0ktiiO55xg1sq6YqQWVtnDpcN6dIelE2O5+3ct1I6GftjAI/cLk9aLS3wQJDeeZ WTLRXbpaidjWMSXm5GjSX1kOw0CILglmyfeLJfIElG2Kf7IvhCP1G17Ow4SpZMN6FgoJT2qpzqJ hR9J+l2Cb0GFwcdizJ0CHJMygsUbHmpPnUuVqweQGW2POs3PFGDuVO3th1lPD4Qj7fYn8hOkt5v xKPB5IfJSITVoJSJm8w== X-Proofpoint-ORIG-GUID: vbC1shvkFWrLtVIKiTWXiDyDVFc-lYeQ X-HPE-SCL: -1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-01-27_03,2026-01-27_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 priorityscore=1501 suspectscore=0 phishscore=0 lowpriorityscore=0 bulkscore=0 malwarescore=0 clxscore=1011 spamscore=0 adultscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2601150000 definitions=main-2601270138 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Granite Rapids (GNR) and Clearwater Forest (CWF) average distances to remote packages to fix scheduler domains, see [1] for more information. A warning and backtrace are printed when sub-NUMA clustering (SNC) is enabled and there are more than 2 packages because the average distances to remote packages could be different, skewing the single average remote distance. This is unnecessary when the average distances to remote packages are the same. Support single average remote distance on systems with more than 2 packages, preventing unnecessary warnings and backtraces by checking if average distances to remote packages are the same. [1] commit 4d6dd05d07d0 ("sched/topology: Fix sched domain build error for = GNR, CWF in SNC-3 mode"). Signed-off-by: Kyle Meyer Reviewed-by: Tim Chen --- The warning and backtrace were noticed on a 16 socket GNR system with SNC-2= enabled. --- arch/x86/kernel/smpboot.c | 70 ++++++++++++++++++++++++++++----------- 1 file changed, 51 insertions(+), 19 deletions(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 5cd6950ab672..4467716f4054 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -518,27 +518,70 @@ static int avg_remote_numa_distance(void) { int i, j; int distance, nr_remote, total_distance; + int max_pkgs =3D topology_max_packages(); + int cpu, pkg, pkg_avg_distance; + int *pkg_total_distance; + int *pkg_nr_remote; =20 if (sched_avg_remote_distance > 0) return sched_avg_remote_distance; =20 + sched_avg_remote_distance =3D REMOTE_DISTANCE; + nr_remote =3D 0; total_distance =3D 0; + + pkg_total_distance =3D kcalloc(max_pkgs, sizeof(int), GFP_KERNEL); + if (!pkg_total_distance) + goto cleanup; + + pkg_nr_remote =3D kcalloc(max_pkgs, sizeof(int), GFP_KERNEL); + if (!pkg_nr_remote) + goto cleanup; + for_each_node_state(i, N_CPU) { for_each_node_state(j, N_CPU) { distance =3D node_distance(i, j); =20 - if (distance >=3D REMOTE_DISTANCE) { - nr_remote++; - total_distance +=3D distance; - } + if (distance < REMOTE_DISTANCE) + continue; + + nr_remote++; + total_distance +=3D distance; + + cpu =3D cpumask_first(cpumask_of_node(j)); + if (cpu >=3D nr_cpu_ids) + continue; + + pkg =3D topology_physical_package_id(cpu); + pkg_total_distance[pkg] +=3D distance; + pkg_nr_remote[pkg]++; } } - if (nr_remote) - sched_avg_remote_distance =3D total_distance / nr_remote; - else - sched_avg_remote_distance =3D REMOTE_DISTANCE; =20 + if (!nr_remote) + goto cleanup; + + sched_avg_remote_distance =3D total_distance / nr_remote; + + /* + * Single average remote distance won't be appropriate if different + * packages have different distances to remote packages. + */ + for (i =3D 0; i < max_pkgs; i++) { + if (!pkg_nr_remote[i]) + continue; + + pkg_avg_distance =3D pkg_total_distance[i] / pkg_nr_remote[i]; + + pr_debug("sched: Avg. distance to remote package %d: %d\n", i, pkg_avg_d= istance); + + if (pkg_avg_distance !=3D sched_avg_remote_distance) + WARN_ONCE(1, "sched: Avg. distances to remote packages are different\n"= ); + } +cleanup: + kfree(pkg_nr_remote); + kfree(pkg_total_distance); return sched_avg_remote_distance; } =20 @@ -564,18 +607,7 @@ int arch_sched_node_distance(int from, int to) * in the remote package in the same sched group. * Simplify NUMA domains and avoid extra NUMA levels including * different remote NUMA nodes and local nodes. - * - * GNR and CWF don't expect systems with more than 2 packages - * and more than 2 hops between packages. Single average remote - * distance won't be appropriate if there are more than 2 - * packages as average distance to different remote packages - * could be different. */ - WARN_ONCE(topology_max_packages() > 2, - "sched: Expect only up to 2 packages for GNR or CWF, " - "but saw %d packages when building sched domains.", - topology_max_packages()); - d =3D avg_remote_numa_distance(); } return d; --=20 2.52.0