From nobody Sun Feb 8 07:21:53 2026 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3CB78163 for ; Wed, 7 Aug 2024 19:05:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723057534; cv=none; b=nHYnzDNtQ/VCKCNAAiZypqFkHJdb6Rw6WYqeMdJrfks6y78Q+Y7bVBOm+4Kk5S21WaqJziuweLBWxcCTwfyT4+Gq0QZf+Nqpx8ZdDEySifJFM+O4wDfiZX6mQdn2y4xYXs0eLPgnjKmmH2BpPGMIjqSgRefR1QN9KW/MXOjc/lU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723057534; c=relaxed/simple; bh=kGlgsQLMgDQ9FvUpMspEvNg1mFjz4e1RWbCwZIxjSWg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WZg+OhIVajSjyX7wmOEThKXs/dqpDyFU/bwIuNOLhOjipGhLHjS5VemVcRTnKdeE7RDEhlBkNvkPBtz5s4S5oVJQIa2GDiX0+52KS4rLS/0UPHpTFTirZhA62uFrbMsnmIsnop/e9HJWr9TLQwZleCtxc5PTFWnauFMB6C73ph8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dXbLbjLA; arc=none smtp.client-ip=209.85.215.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dXbLbjLA" Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-656d8b346d2so121719a12.2 for ; Wed, 07 Aug 2024 12:05:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723057530; x=1723662330; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pO/uXHwTaQnApHHXM2dwVTEOkF4Fz3bQoqmBiOkUQD4=; b=dXbLbjLA4jGvbptzRtJsXNgJf+lYhsTjw678LyYXjZybSbuc5ARtneovkxIXAhdmZx hRTOy8/EF+cHbroGjmEWwcGIFyjl5N0aWaKh0WHo+Egk1a+vf/9CgD6DY812qSNMc7qX Mqo1mvz4AwAZo44jHNMmr3Z0j6+vEsV0h/Ei3Ke1ZXu2WGnJ147TBjBOQoqrMVKJPlKs aZbuy+GINKE411BMwqi3z4obPmi57LtsiJ7bn71jpd3/gNREoqIOeJP+Xtdzh22YlTtn 3CnE2vwUMDG30IzYgbOU7dZ8zaKG/BO2ZGT4bETxKyBWiJV18slHWyzUH3GaOmmKBfZA 54qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723057530; x=1723662330; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pO/uXHwTaQnApHHXM2dwVTEOkF4Fz3bQoqmBiOkUQD4=; b=sBGBKAdAjYKQ1P/4deksQXSqTtD5G2S7tsKt1LhJjwJ5m8o70gTM6XPJGxVMcEWGyY mTpWRiG5GSx3vvyhn8u8jUJAQj5YS4SvgoryundX/s0zIEc5CYIbO/BjQC0GcryTwSod 40Ms9wAKNJKmB/SBHWEOfsMPSi/ax94qDM5FJc51BklT6lwm+W20GpMjBrNhMJT+zBcR nY7rOR8OzqNqoZFa9/RMFOzq6xo/2gRHz1W8Tcl/CrjLYiyAH7Te7tGmUAA5akphF5vT jiXi8JcDXs5lIbHIVoMPxA7Bn38TII1zLzoI00EbOm0TUXaUEOBrATF+vE1ZnKzBtKuO JUhw== X-Gm-Message-State: AOJu0YxG3KkY8fCaqg+fp3MGCSaoUU3gKX4aovNzIfdjvDy0En7/7bAs i8vsrmntJsWh91MPQ73ssVwIFXUa4/k3cwZnNzKZw4Xd3UNTEWxl2P+xJaxq X-Google-Smtp-Source: AGHT+IF1JocH9w109P6FFoz3xJeeU71EDXOYvkkIRlFHm0qDNYis3Y8iRTbjy3lVIsSEBhBdDDjs+w== X-Received: by 2002:a05:6a20:6a07:b0:1c2:8d33:af69 with SMTP id adf61e73a8af0-1c69961443amr22164571637.41.1723057530381; Wed, 07 Aug 2024 12:05:30 -0700 (PDT) Received: from localhost ([216.228.127.130]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7106ece0e02sm8727152b3a.116.2024.08.07.12.05.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Aug 2024 12:05:30 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org Cc: Yury Norov , Chen Yu , Christophe JAILLET , Leonardo Bras , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Subject: [PATCH 1/2] sched/topology: pre-compute topology_span_sane() loop params Date: Wed, 7 Aug 2024 12:05:20 -0700 Message-ID: <20240807190522.432388-2-yury.norov@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240807190522.432388-1-yury.norov@gmail.com> References: <20240807190522.432388-1-yury.norov@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" tl->mask() is called inside the loop with the same parameters more than once. We can pre-calculate it. Reviewed-by: Chen Yu Signed-off-by: Yury Norov --- kernel/sched/topology.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 76504b776d03..8af3b48da458 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -2353,12 +2353,15 @@ static struct sched_domain *build_sched_domain(stru= ct sched_domain_topology_leve static bool topology_span_sane(struct sched_domain_topology_level *tl, const struct cpumask *cpu_map, int cpu) { + const struct cpumask *mi, *mc; int i =3D cpu + 1; =20 /* NUMA levels are allowed to overlap */ if (tl->flags & SDTL_OVERLAP) return true; =20 + mc =3D tl->mask(cpu); + /* * Non-NUMA levels cannot partially overlap - they must be either * completely equal or completely disjoint. Otherwise we can end up @@ -2366,14 +2369,15 @@ static bool topology_span_sane(struct sched_domain_= topology_level *tl, * breaks the linking done for an earlier span. */ for_each_cpu_from(i, cpu_map) { + mi =3D tl->mask(i); + /* * We should 'and' all those masks with 'cpu_map' to exactly * match the topology we're about to build, but that can only * remove CPUs, which only lessens our ability to detect * overlaps */ - if (!cpumask_equal(tl->mask(cpu), tl->mask(i)) && - cpumask_intersects(tl->mask(cpu), tl->mask(i))) + if (!cpumask_equal(mc, mi) && cpumask_intersects(mc, mi)) return false; } =20 --=20 2.43.0 From nobody Sun Feb 8 07:21:53 2026 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73989143C45 for ; Wed, 7 Aug 2024 19:05:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723057535; cv=none; b=dtiUB748k5JwNinXpu4S4k8OoldDhNIA7idsUSJu2JttCDeRfX/+RaDurGCG+P8Q7CRsybT05nWMerPjpwtANNkbLLP3MciVqJHPaaxurhI8n3ya5g8PM2vn8iPym9Mb4hnXBBFdS1rSf2yj2WbQ72PSTirvW9JLs86nc+gdLzw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723057535; c=relaxed/simple; bh=eA1AI6V87YPxIg7o9zfBiyAPonmNWgXhOCclHObnY4w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hBRHb93PHPSnN9wiwEtYvNy8V9/se5a3ccGXo9SADDZ5Txj0GBCAz/vMf2LuVGAv6V5GgNByHqlIRCJpTFxgfUPoSyZ7QrB2DuGfQvfRa98gUxgGJgYtJfRTas+TUHrVEuVESAEVPaQOXOzDJv2bxkPFxbgXaJPhRfUhyMzN+QY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Q64LQVSD; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q64LQVSD" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1fd90c2fc68so2499995ad.1 for ; Wed, 07 Aug 2024 12:05:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723057534; x=1723662334; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=36H26RD5uR3ue5WSyAgLPKZiveEwsWRYqoMnPtQYeHs=; b=Q64LQVSDm+iZR0rywaggHALjQgXDDfHh5RwFR0QWeRph7CrhXASuZUx/DplUOnsMD1 dYG5k8NCWRLFFQS40zLHL1jIFLNBrJyKqzwfv6Af6jy3pGWBR/KUaNYfanYXboizuxew eGGAQ1hzAbc5Hb8VSz4oPk8X4VEqROOgiOs07Yy55GDB/IjXWTHXmM0zzcBuHzv1apgq OFiG3RuZ/iGVdZRKHXLfiYeAQnO1bc9+I95VBacfLjaPg0nrrzbrMZfco3+Vv77Ffg4C 4aTptJJFY9E4EcyFqMNRls/OfwJcktaRmS2fiaGm7ElSBd8urKKWMCDKqQAkpLwFejaB +Z8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723057534; x=1723662334; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=36H26RD5uR3ue5WSyAgLPKZiveEwsWRYqoMnPtQYeHs=; b=GTpuugNKzXPJn5HDgGcp3QTXPzvpXtTICOMp9e77mt3dJWYKaZTfN07xp9Z4+Xm0/x 7CDaUt7zM7T/MWM3CLXDRxKpMqzgRBretH8t5urhB5sMI6Uy0/v+1VCMkaeTZ5CaHTkW hkPlsAMCbqyuQq6fSizq4IocODJgPmKAqL2LHu3Y499UKV3yWKwdXetFt49vilfIrX7V A3OF3GB97SSqm575NtFnvje1qmSdbGu1N7IHlsPEeYfVYWjD55xQwqj75FTIel8Nf+FP GKMp3o9g5SzouyDz+vfhrZvTerdxuG38fNhgpd87HtO+bRS4CRewnUqT9/6eVifxN1mr rNGg== X-Gm-Message-State: AOJu0Yy7UrLE90SALwGwkZ4UWdVWCmnyO7iHBxE/Z7y1eiapuZM5SNr9 9KecxuCD2Z/SXzVJZ5thhSNfdQYpXf6iH/8q0iKoOG+R4z6evj95W42UM5SF X-Google-Smtp-Source: AGHT+IEkroaE82bVicHuBDOO2gbeleqKhRZWNfaT0squj8kHhv2ABdAL3dDmJD+gewBpF1kMQXg7Ag== X-Received: by 2002:a17:902:ce87:b0:1fd:7293:3d70 with SMTP id d9443c01a7336-1ff5723e5f5mr216745615ad.8.1723057533569; Wed, 07 Aug 2024 12:05:33 -0700 (PDT) Received: from localhost ([216.228.127.130]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1ff58f6ad65sm110024525ad.116.2024.08.07.12.05.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Aug 2024 12:05:33 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org Cc: Yury Norov , Chen Yu , Christophe JAILLET , Leonardo Bras , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider Subject: [PATCH 2/2] sched/topology: optimize topology_span_sane() Date: Wed, 7 Aug 2024 12:05:21 -0700 Message-ID: <20240807190522.432388-3-yury.norov@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240807190522.432388-1-yury.norov@gmail.com> References: <20240807190522.432388-1-yury.norov@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The function may call cpumask_equal with tl->mask(cpu) =3D=3D tl->mask(i), even though cpu !=3D i. In such case, cpumask_equal() would always return true, and we can proceed to the next iteration immediately. Comment is provided by Valentin Schneider. Reviewed-by: Valentin Schneider Signed-off-by: Yury Norov --- kernel/sched/topology.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 8af3b48da458..3661d4173d1f 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -2370,6 +2370,18 @@ static bool topology_span_sane(struct sched_domain_t= opology_level *tl, */ for_each_cpu_from(i, cpu_map) { mi =3D tl->mask(i); + /* + * Some topology levels (e.g. PKG in default_topology[]) + * have a sched_domain_mask_f implementation that reuses + * the same mask for several CPUs (in PKG's case, one mask + * for all CPUs in the same NUMA node). + * + * For such topology levels, repeating cpumask_equal() + * checks is wasteful. Instead, we first check that the + * tl->mask(i) pointers aren't the same. + */ + if (mi =3D=3D mc) + continue; =20 /* * We should 'and' all those masks with 'cpu_map' to exactly --=20 2.43.0