From nobody Wed Dec 17 06:44:52 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01CF6C4167B for ; Thu, 7 Dec 2023 20:39:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1443918AbjLGUjC (ORCPT ); Thu, 7 Dec 2023 15:39:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231301AbjLGUi6 (ORCPT ); Thu, 7 Dec 2023 15:38:58 -0500 Received: from mail-yw1-x112c.google.com (mail-yw1-x112c.google.com [IPv6:2607:f8b0:4864:20::112c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5EB11713 for ; Thu, 7 Dec 2023 12:39:04 -0800 (PST) Received: by mail-yw1-x112c.google.com with SMTP id 00721157ae682-5d226f51f71so12440627b3.3 for ; Thu, 07 Dec 2023 12:39:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701981544; x=1702586344; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t+ncE/Ke8W/9eSicUFGfjAMk7XfHt6TzmaC+XB67O8c=; b=Mn6cD47IEGNhYiJtgTi0Q/J42HOq51Rzpy9lsf1AA5BwEhob3wSOLyCoocML3DAXIq l7zLIztB116RuBHtxHVYsM5P+TtgFrb2Qxubla34kugzXgWtajUVoR7qXgPcA+o4yvcW eIkT2TMoZdFeUDW+HFKM4JIotiZ+3jWmmNMjGhj1pxLCfw1pOrbgfT3Wx9WuWOPkZopJ HWiAP09C2IJRrDA+KBWlEmDDWUYr2qIy383rF1/S2/jVBtUc9fDg63AHKK7lC5UsWU6E AdbIdXv9lpMiht8RLk4pmUN6f0kQVXqsPOyXWepQrRTw3S4m68LWHErWN2m6uM19v2lZ 5/Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701981544; x=1702586344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t+ncE/Ke8W/9eSicUFGfjAMk7XfHt6TzmaC+XB67O8c=; b=KN6Gxw7kIJeZAd49Q3LYMlmjydxPk72CVpOk/jkEgAhQjX8I7cYX6S66LfLwctejU9 jjEfkweBE8k7jfIlHC4gVFsnr31eLCBRxF31yA6NBAW9N2VnvybqLf1krfVJo/H/W2Rk B4pHe0SNonp1gK8zDc0wlwXlbzpPvmP7imq2DPW+ho892vW8X7TDhLiJ0tKIhQi0Xff8 MzmaHXBP/qhRlTYJ20R1aZQNZ0ekJ9puF4EG4TSu/J6nLTjMFoB8IfQSou/lFUH86Wtj oA/sGIQi/A8G8mrqYOOHyQX0bUvZdjA7TAexBx94mopaC1ZSv1uFa4znQYUkC8R+3lek 1gpg== X-Gm-Message-State: AOJu0Yysj9lsNjiu0NlEogAn4/cJEFZKEvgKafN/Vwb+A129aE9p0lDu lQEHqsdDoBMm1WLRMd6AZsY= X-Google-Smtp-Source: AGHT+IEjxfx4/yblkVJIQxF90BhwSszmQgB/4/z8ZfVuRkbqovYnHwqDHbbqqMtSKVytphbLBzIWOA== X-Received: by 2002:a81:574f:0:b0:5d7:545e:3bda with SMTP id l76-20020a81574f000000b005d7545e3bdamr2499373ywb.20.1701981543874; Thu, 07 Dec 2023 12:39:03 -0800 (PST) Received: from localhost ([2601:344:8301:57f0:8f7f:efcd:5388:af4f]) by smtp.gmail.com with ESMTPSA id m132-20020a81718a000000b005d4071e3adasm144853ywc.57.2023.12.07.12.39.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Dec 2023 12:39:03 -0800 (PST) From: Yury Norov To: Andrew Morton , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Yury Norov , Ming Lei , Andy Shevchenko , Rasmus Villemoes Subject: [PATCH v2 1/6] cpumask: introduce for_each_cpu_and_from() Date: Thu, 7 Dec 2023 12:38:55 -0800 Message-Id: <20231207203900.859776-2-yury.norov@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231207203900.859776-1-yury.norov@gmail.com> References: <20231207203900.859776-1-yury.norov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Similarly to for_each_cpu_and(), introduce a for_each_cpu_and_from(), which is handy when it's needed to traverse 2 cpumasks or bitmaps, starting from a given position. Signed-off-by: Yury Norov --- include/linux/cpumask.h | 11 +++++++++++ include/linux/find.h | 3 +++ 2 files changed, 14 insertions(+) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index cfb545841a2c..73ff2e0ef090 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -332,6 +332,17 @@ unsigned int __pure cpumask_next_wrap(int n, const str= uct cpumask *mask, int sta #define for_each_cpu_and(cpu, mask1, mask2) \ for_each_and_bit(cpu, cpumask_bits(mask1), cpumask_bits(mask2), small_cpu= mask_bits) =20 +/** + * for_each_cpu_and_from - iterate over every cpu in both masks starting f= rom a given cpu + * @cpu: the (optionally unsigned) integer iterator + * @mask1: the first cpumask pointer + * @mask2: the second cpumask pointer + * + * After the loop, cpu is >=3D nr_cpu_ids. + */ +#define for_each_cpu_and_from(cpu, mask1, mask2) \ + for_each_and_bit_from(cpu, cpumask_bits(mask1), cpumask_bits(mask2), smal= l_cpumask_bits) + /** * for_each_cpu_andnot - iterate over every cpu present in one mask, exclu= ding * those present in another. diff --git a/include/linux/find.h b/include/linux/find.h index 5e4f39ef2e72..dfd3d51ff590 100644 --- a/include/linux/find.h +++ b/include/linux/find.h @@ -563,6 +563,9 @@ unsigned long find_next_bit_le(const void *addr, unsign= ed (bit) =3D find_next_and_bit((addr1), (addr2), (size), (bit)), (bit) = < (size);\ (bit)++) =20 +#define for_each_and_bit_from(bit, addr1, addr2, size) \ + for (; (bit) =3D find_next_and_bit((addr1), (addr2), (size), (bit)), (bit= ) < (size); (bit)++) + #define for_each_andnot_bit(bit, addr1, addr2, size) \ for ((bit) =3D 0; \ (bit) =3D find_next_andnot_bit((addr1), (addr2), (size), (bit)), (bi= t) < (size);\ --=20 2.40.1 From nobody Wed Dec 17 06:44:52 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE6D8C4167B for ; Thu, 7 Dec 2023 20:39:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231345AbjLGUjL (ORCPT ); Thu, 7 Dec 2023 15:39:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1443910AbjLGUi7 (ORCPT ); Thu, 7 Dec 2023 15:38:59 -0500 Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58D211713 for ; Thu, 7 Dec 2023 12:39:06 -0800 (PST) Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-5d4f71f7e9fso12608867b3.0 for ; Thu, 07 Dec 2023 12:39:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701981545; x=1702586345; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FOAgpJU0JgXYQxB1eWGGjpttQzJCZxojyeFpRMgz7us=; b=Xqep590r941VZkOUVFs8NAvGcJyrybGQ+D4OK0fCdAIsOFEllsPu9MZxp6fCQyp8W8 255bYY/qu202GuFgq/qxuasMGiC3pt90EczWSvSCqXW7yEpHLbaBe1MUhYAccGXzvCnm 5iTP3+8rZsKmj5ZGsIzK++BGFc5cbb0d9hMfUlB4XW4egAaRqPgWehlX0yA+77mo1NCB W0w4II17E9M93yAnsBvi/biJHskcBg9wXNWla8tf+Qn7sGHwhz1pivRBg88bu17y0PJE Qr9COoYz0s5x5jFkEIJ86yQQRwhO4h9kBxvAA7uzhX5ONDlBmuFARoXrTNJkXLL54WbV mh/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701981545; x=1702586345; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FOAgpJU0JgXYQxB1eWGGjpttQzJCZxojyeFpRMgz7us=; b=Vd92i7AK2DbH2uzMKx+lwRyb7wSQCZyb+mlEXrgHkOiaBbrGELwDeXbC3K4uC5SrkB 6v8XU4axNELi1c+84hZR5TbkydInQNoM0vgB2YWqZ47WMTzokjFcjJdbXZxnbpO/4gUm QPqhks5orE8MHDElS9p5uMBl8gkGOtG0vAa/obTuYWjPLCJN69EuYPaiaqo+YpyxtwjQ +ifSyxgCueMrrt+b6M8XAuIks+vOZplShyNsWcbLL1yV63wKYwkOVW+29oHtVpdv+V7w 8Ihi+5ZTl9vJZAwnk72UTVi/LtL3DiREzmFXb4BXdwKvKMcC8AdSePG+dbYFm03AugbS 35kg== X-Gm-Message-State: AOJu0Yw042IcdAt32cPWgJYyvdIwuUekbLW6wkf9GpgZAmuX1YVhhjqt GhzJOkAAtROwU97VMDe0PJ0= X-Google-Smtp-Source: AGHT+IHXPZzbcjXplvZRnh8mEoikxeqtF6BBIDzDme3gYh4PuTEDeCy6cGD0/5b0VYJznxPv52UDhA== X-Received: by 2002:a05:690c:c0a:b0:5d7:3f4d:2e2f with SMTP id cl10-20020a05690c0c0a00b005d73f4d2e2fmr2972497ywb.14.1701981545421; Thu, 07 Dec 2023 12:39:05 -0800 (PST) Received: from localhost ([2601:344:8301:57f0:8f7f:efcd:5388:af4f]) by smtp.gmail.com with ESMTPSA id g68-20020a0dc447000000b005cb7fccffe2sm139604ywd.126.2023.12.07.12.39.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Dec 2023 12:39:04 -0800 (PST) From: Yury Norov To: Andrew Morton , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Yury Norov , Ming Lei , Andy Shevchenko , Rasmus Villemoes Subject: [PATCH v2 2/6] lib/group_cpus: relax atomicity requirement in grp_spread_init_one() Date: Thu, 7 Dec 2023 12:38:56 -0800 Message-Id: <20231207203900.859776-3-yury.norov@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231207203900.859776-1-yury.norov@gmail.com> References: <20231207203900.859776-1-yury.norov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Because nmsk and irqmsk are stable, extra atomicity is not required. Signed-off-by: Yury Norov --- lib/group_cpus.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/lib/group_cpus.c b/lib/group_cpus.c index ee272c4cefcc..8eb18c6bbf3b 100644 --- a/lib/group_cpus.c +++ b/lib/group_cpus.c @@ -24,8 +24,8 @@ static void grp_spread_init_one(struct cpumask *irqmsk, s= truct cpumask *nmsk, if (cpu >=3D nr_cpu_ids) return; =20 - cpumask_clear_cpu(cpu, nmsk); - cpumask_set_cpu(cpu, irqmsk); + __cpumask_clear_cpu(cpu, nmsk); + __cpumask_set_cpu(cpu, irqmsk); cpus_per_grp--; =20 /* If the cpu has siblings, use them first */ @@ -34,9 +34,8 @@ static void grp_spread_init_one(struct cpumask *irqmsk, s= truct cpumask *nmsk, sibl =3D cpumask_next(sibl, siblmsk); if (sibl >=3D nr_cpu_ids) break; - if (!cpumask_test_and_clear_cpu(sibl, nmsk)) - continue; - cpumask_set_cpu(sibl, irqmsk); + __cpumask_clear_cpu(sibl, nmsk); + __cpumask_set_cpu(sibl, irqmsk); cpus_per_grp--; } } --=20 2.40.1 From nobody Wed Dec 17 06:44:52 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC9B3C4167B for ; Thu, 7 Dec 2023 20:39:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1443939AbjLGUjO (ORCPT ); Thu, 7 Dec 2023 15:39:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1443913AbjLGUjB (ORCPT ); Thu, 7 Dec 2023 15:39:01 -0500 Received: from mail-yw1-x1129.google.com (mail-yw1-x1129.google.com [IPv6:2607:f8b0:4864:20::1129]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDDA41713 for ; Thu, 7 Dec 2023 12:39:07 -0800 (PST) Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-5d7a47d06eeso12681877b3.1 for ; Thu, 07 Dec 2023 12:39:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701981547; x=1702586347; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U6INlgZjI/rk3Hb4b9fsbV8+DsR4OZwnMfo8QGcQs8s=; b=DWbn/axiW8UwkzOroASm/q4/ycZikPr7mm/pCEFCPA+2NL6KsNH8EXmNrNBptxsBOO KCVbG8AqKBYzPvkdRo6I4jWBzauwVSy9oT+BVTelVbrx0qV5uXMtS4Nm5OxqJit2jZzF BWaWW1kcf8acUPRVskKQt7CoMaVlRC0oNKklYnTLOqFlbN2EacMELWepaAHwtOvyUvYC PIEJg/82hROW7B+uVV39u6GOdXxe07IrfiShuMYYRGCiOXUEqQ1TaKW3PAw8rv/5n/kB BKtGbP8xX8owPAyv/3KtFok3su+3TEjeJcZQvf/AEfbY9B0xM6f6O2lxbHE+1Tv9mR2U ivZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701981547; x=1702586347; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U6INlgZjI/rk3Hb4b9fsbV8+DsR4OZwnMfo8QGcQs8s=; b=CdApIFlW/csk+l9S1xSCsd7rT59uvkrAhicByuObD00ycW7XWKuE3tE4/4N9afFrYi O1p4Ro3rnQhKU92wubNCgzmLGYd6AU31bQQEECO5XngLaG6VrLpPQW48s2qsH5kVH58B HE+pt2GwyJxFE3CLGTCIrOUKKtyJChE0RE8Dz41zJGz7YUhw2Z8hLff6cySxrxz7KNkh j+TNfPEuzLq6kk/7t5v0OhYSvK20cUBWTOqq4hvdCWwnCMkFwVPWgQsUuZjQLQ4PjaJY LQmrLPNaUl7Yd9RDjZF3LWVKjJK8K0jY96vLA+sAmRetUOodV0o4j8puQ9YaA8ggao2S 1bGQ== X-Gm-Message-State: AOJu0YxGanI8IHSdsisxhUr2pXdFsDza49mRLx/ykw4kg3k9t3KbPG9F pH6xnfTOutaf5XacWETr9pk= X-Google-Smtp-Source: AGHT+IHIu9fJEqi+QBLPf7t0UdCT9e7RcHhX5A/4Hs+G96VMhRczxATgVX981Y4SyAQLJau2Pc/1pA== X-Received: by 2002:a81:bd42:0:b0:5d6:c177:4a31 with SMTP id n2-20020a81bd42000000b005d6c1774a31mr2916183ywk.50.1701981547007; Thu, 07 Dec 2023 12:39:07 -0800 (PST) Received: from localhost ([2601:344:8301:57f0:8f7f:efcd:5388:af4f]) by smtp.gmail.com with ESMTPSA id z20-20020a81c214000000b005d900a949ccsm141308ywc.97.2023.12.07.12.39.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Dec 2023 12:39:06 -0800 (PST) From: Yury Norov To: Andrew Morton , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Yury Norov , Ming Lei , Andy Shevchenko , Rasmus Villemoes Subject: [PATCH v2 3/6] lib/group_cpus: optimize inner loop in grp_spread_init_one() Date: Thu, 7 Dec 2023 12:38:57 -0800 Message-Id: <20231207203900.859776-4-yury.norov@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231207203900.859776-1-yury.norov@gmail.com> References: <20231207203900.859776-1-yury.norov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The loop starts from the beginning every time we switch to the next sibling mask. This is the Schlemiel the Painter's style of coding because we know for sure that nmsk is clear up to current CPU, and we can just continue from the next CPU. Also, we can do it nicer if leverage the dedicated for_each() iterator. Signed-off-by: Yury Norov --- lib/group_cpus.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/lib/group_cpus.c b/lib/group_cpus.c index 8eb18c6bbf3b..7ac94664230f 100644 --- a/lib/group_cpus.c +++ b/lib/group_cpus.c @@ -30,13 +30,13 @@ static void grp_spread_init_one(struct cpumask *irqmsk,= struct cpumask *nmsk, =20 /* If the cpu has siblings, use them first */ siblmsk =3D topology_sibling_cpumask(cpu); - for (sibl =3D -1; cpus_per_grp > 0; ) { - sibl =3D cpumask_next(sibl, siblmsk); - if (sibl >=3D nr_cpu_ids) - break; + sibl =3D cpu + 1; + + for_each_cpu_and_from(sibl, siblmsk, nmsk) { __cpumask_clear_cpu(sibl, nmsk); __cpumask_set_cpu(sibl, irqmsk); - cpus_per_grp--; + if (cpus_per_grp-- =3D=3D 0) + return; } } } --=20 2.40.1 From nobody Wed Dec 17 06:44:52 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 833CFC4167B for ; Thu, 7 Dec 2023 20:39:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235234AbjLGUjS (ORCPT ); Thu, 7 Dec 2023 15:39:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1443919AbjLGUjC (ORCPT ); Thu, 7 Dec 2023 15:39:02 -0500 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [IPv6:2607:f8b0:4864:20::1133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 196B51716 for ; Thu, 7 Dec 2023 12:39:09 -0800 (PST) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-5d74186170fso11258817b3.3 for ; Thu, 07 Dec 2023 12:39:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701981548; x=1702586348; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UfvFAKot0bp3HfeoPJ3dMZk7rFg3lsuy8X5EAYhpV4g=; b=MIScm2VvEVU4s1sy5ocmZstsmGKOmkh/6AYKwCPbDHLecICfLkOth+fXDtpsmimxc3 GB0dSRmWxMlNtoAiDh/v8cMEnCI5bVfF/OF1NWWaDKMoa3Kej2VZi6eSZX/U9u9D+SJf KIvcwuJwPbHBJVQG2/2mx8ouxmv7NQ+v9lkgm+wlLnauwIBy1TLUbtzn1QIFu22L5luD qwFREiw3asBYiOG695ZWNxPPnkW5r4yN3eS4UM4L49HfjzJ0ad5UFbEELIB2HXbCYQkA JOiNb3nrnfl6Md7TIeJch+GFpcZTcBOLI3ssVx2EXsTMbt8uD3V6vO0nRVoGEotW0zdL k6kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701981548; x=1702586348; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UfvFAKot0bp3HfeoPJ3dMZk7rFg3lsuy8X5EAYhpV4g=; b=Mm2pfLlFOGT0+6Sr/G+laWXve1YCehSkup6yaCIUv6Xg2F6/Pj1DeS7grch+yIcCG3 YStk6z/wsjvUvKuT1yrQ+QUmlnhZQYRLHGBRmMvCGsL3KO7+aQHYvCJhNcLJcCResYCx 22qsW3jZ6ngHEdrFxO7Z18eaIB1hUJl7KZOyZ5DGNS7UIPHsJdfxVDxJg2GcHiBgr6Wg RccAJTad38oWFpcyY8jht78xfsej8wxl3fsJS7hreJSRRk1JaYqLmT4uAQraAfWMZv61 c+k05nO6Vd2/5h9L/yin1ohxOXwaQBOBkoDM2gPlqaxL7LfkO3fGqht5jnZdGhXnOQRZ tnjQ== X-Gm-Message-State: AOJu0Yy/ie5n6pixmUBJlg+c03IX3x2X4sOuR2hc6iSnv71kgDzh71N7 1JkyROd6sgzck7wx/o0L12g= X-Google-Smtp-Source: AGHT+IGmFQHx3KF7M+23G862JFjaqYcOuv0Ygu+9uk90EFQyWZoU+XJkeUEZ7rjuysLaaldDheenKw== X-Received: by 2002:a0d:d455:0:b0:5d8:21f9:4efc with SMTP id w82-20020a0dd455000000b005d821f94efcmr2883577ywd.14.1701981548260; Thu, 07 Dec 2023 12:39:08 -0800 (PST) Received: from localhost ([2601:344:8301:57f0:8f7f:efcd:5388:af4f]) by smtp.gmail.com with ESMTPSA id i9-20020a81d509000000b005d774946228sm141804ywj.68.2023.12.07.12.39.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Dec 2023 12:39:07 -0800 (PST) From: Yury Norov To: Andrew Morton , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Yury Norov , Ming Lei , Andy Shevchenko , Rasmus Villemoes Subject: [PATCH v2 4/6] lib/group_cpus: optimize outer loop in grp_spread_init_one() Date: Thu, 7 Dec 2023 12:38:58 -0800 Message-Id: <20231207203900.859776-5-yury.norov@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231207203900.859776-1-yury.norov@gmail.com> References: <20231207203900.859776-1-yury.norov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Similarly to the inner loop, in the outer loop we can use for_each_cpu() macro, and skip CPUs that have been copied. With this patch, the function becomes O(1), despite that it's a double-loop. Signed-off-by: Yury Norov --- lib/group_cpus.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/lib/group_cpus.c b/lib/group_cpus.c index 7ac94664230f..cded3c8ea63b 100644 --- a/lib/group_cpus.c +++ b/lib/group_cpus.c @@ -17,16 +17,11 @@ static void grp_spread_init_one(struct cpumask *irqmsk,= struct cpumask *nmsk, const struct cpumask *siblmsk; int cpu, sibl; =20 - for ( ; cpus_per_grp > 0; ) { - cpu =3D cpumask_first(nmsk); - - /* Should not happen, but I'm too lazy to think about it */ - if (cpu >=3D nr_cpu_ids) - return; - + for_each_cpu(cpu, nmsk) { __cpumask_clear_cpu(cpu, nmsk); __cpumask_set_cpu(cpu, irqmsk); - cpus_per_grp--; + if (cpus_per_grp-- =3D=3D 0) + return; =20 /* If the cpu has siblings, use them first */ siblmsk =3D topology_sibling_cpumask(cpu); @@ -37,6 +32,7 @@ static void grp_spread_init_one(struct cpumask *irqmsk, s= truct cpumask *nmsk, __cpumask_set_cpu(sibl, irqmsk); if (cpus_per_grp-- =3D=3D 0) return; + cpu =3D sibl + 1; } } } --=20 2.40.1 From nobody Wed Dec 17 06:44:52 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14A15C4167B for ; Thu, 7 Dec 2023 20:39:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1443951AbjLGUjW (ORCPT ); Thu, 7 Dec 2023 15:39:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1443923AbjLGUjE (ORCPT ); Thu, 7 Dec 2023 15:39:04 -0500 Received: from mail-yb1-xb2a.google.com (mail-yb1-xb2a.google.com [IPv6:2607:f8b0:4864:20::b2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7038E171C for ; Thu, 7 Dec 2023 12:39:10 -0800 (PST) Received: by mail-yb1-xb2a.google.com with SMTP id 3f1490d57ef6-d9b9adaf291so1451425276.1 for ; Thu, 07 Dec 2023 12:39:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701981549; x=1702586349; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pqzdM1JWQlsu7k/N7xBtV+SLaOrDoqGHT+SByeHgWqU=; b=eY5ZlxwE1hSRnhfKQrV24Asbw4s4UPiuLDhy/aKiQj49IPXodCdeunRlPOLo8JMaNS 0sOZDG2WIyim06uB+hi66gsxpyUeTvD54SE4bmbRuXmd+NoY5u6nhCQ8fDp3MzVscJeo yeYN47pH4Ps72xWd99Ati6TWtoqQdXRumx7E87bOvaicaZN3zHH57z6vntgd7OH1mYCN wu/zTKXwAsesaT7cWruCDIBBQAUO7KIWg0gQqYlnc/QfXP3NZUFQz7bc/QuYC9ORBOtI +hcQOYNQawUsG4ritMFzpa4vahqVbji+eZGlYGrpNzJsdvZIerXZ1UI8t6hbnA7TUntL JbDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701981549; x=1702586349; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pqzdM1JWQlsu7k/N7xBtV+SLaOrDoqGHT+SByeHgWqU=; b=vpcs1v0z0/FZygKiJxF0wvIUoOjlqRwy0AlWhZ+qZ1TFHU3Hoh90+YSMX1eSubTzQm Au5oMIFx9yhyRmmfIXs3t2XrccelUV+a3CID56aGPxrXPx4udYKTyyMAqeNeYGu702s8 Zj3+tYvlb+BjLI3OSxOKeM484WZnEA7WSGJbMiiU/lrncljZ4tuiimQArVUHZqXC7ezQ CgH11Q3gGob3VDqQQxBeLjeBm2OwWceMuDIH4hnD78FFp18qslDH/BQLEqMCX3p3+jp4 lzRu5p+0EXBoor3e3HIehese4W8G+O1LbubLxCKM+kg8rICR6mv+ICiZUYEr8DCc/OOj UPFQ== X-Gm-Message-State: AOJu0YwW44dbQIJO67jJEUoQW9NKROrAv5NiP7h+BT+jv//EMcqmlskZ lcfKUTvXeb6hSxzwKVCK3XM= X-Google-Smtp-Source: AGHT+IH+tKfirfTfmD2dg4eYtMxnJm2/WGGr9CaK2O7JxG9U/sPfP1dhWUdUy23PTlCq/IILz+U1Tw== X-Received: by 2002:a25:c2c5:0:b0:db5:4cc0:68dd with SMTP id s188-20020a25c2c5000000b00db54cc068ddmr2565491ybf.21.1701981549533; Thu, 07 Dec 2023 12:39:09 -0800 (PST) Received: from localhost ([2601:344:8301:57f0:8f7f:efcd:5388:af4f]) by smtp.gmail.com with ESMTPSA id 186-20020a2505c3000000b00da0c49a588asm146925ybf.8.2023.12.07.12.39.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Dec 2023 12:39:09 -0800 (PST) From: Yury Norov To: Andrew Morton , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Yury Norov , Ming Lei , Andy Shevchenko , Rasmus Villemoes Subject: [PATCH 5/6] lib/cgroup_cpus.c: don't zero cpumasks in group_cpus_evenly() on allocation Date: Thu, 7 Dec 2023 12:38:59 -0800 Message-Id: <20231207203900.859776-6-yury.norov@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231207203900.859776-1-yury.norov@gmail.com> References: <20231207203900.859776-1-yury.norov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" nmsk and npresmsk are both allocated with zalloc_cpumask_var(), but they are initialized by copying later in the code, and so may be allocated uninitialized. Signed-off-by: Yury Norov --- lib/group_cpus.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/group_cpus.c b/lib/group_cpus.c index cded3c8ea63b..c7fcd04c87bf 100644 --- a/lib/group_cpus.c +++ b/lib/group_cpus.c @@ -347,10 +347,10 @@ struct cpumask *group_cpus_evenly(unsigned int numgrp= s) int ret =3D -ENOMEM; struct cpumask *masks =3D NULL; =20 - if (!zalloc_cpumask_var(&nmsk, GFP_KERNEL)) + if (!alloc_cpumask_var(&nmsk, GFP_KERNEL)) return NULL; =20 - if (!zalloc_cpumask_var(&npresmsk, GFP_KERNEL)) + if (!alloc_cpumask_var(&npresmsk, GFP_KERNEL)) goto fail_nmsk; =20 node_to_cpumask =3D alloc_node_to_cpumask(); --=20 2.40.1 From nobody Wed Dec 17 06:44:52 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B797FC4167B for ; Thu, 7 Dec 2023 20:39:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1443960AbjLGUj1 (ORCPT ); Thu, 7 Dec 2023 15:39:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1443936AbjLGUjG (ORCPT ); Thu, 7 Dec 2023 15:39:06 -0500 Received: from mail-yw1-x112b.google.com (mail-yw1-x112b.google.com [IPv6:2607:f8b0:4864:20::112b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E647E1725 for ; Thu, 7 Dec 2023 12:39:11 -0800 (PST) Received: by mail-yw1-x112b.google.com with SMTP id 00721157ae682-5d34f8f211fso12694467b3.0 for ; Thu, 07 Dec 2023 12:39:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701981551; x=1702586351; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AdhzPeT3AA9HptxSHcMzOSBTny61/xtguzzqnxzNv6M=; b=fbxVQMCBh6aSDW4fc5TmMHCIc+n8Se2z7D5ZprxNWPEcFxIM9uPVNQ+JS6oQjxJ31P 3t0k1sbxPjMGmgqqqn+KKnPwvTPrKAjGtYcEay8RvFEgFVr3dHb9QIOMbE5y/0xPJ5JW JR58N29bl4LAPZkajCme4vs5Ls8SCRwgWOQQzZCnxJPY8epMUXknvbOxEV9jaz9oxSNr kamv2vu/+Qnd80aYuBYkSbBjpZ1wpJQrNK3jfw/33SLFRnvMjuAeoeNb0KwpnrPxSmgl ww1BEZ/N85cewpFMR/VFwgGi/1pVsTf4DFtEEhiHfdWhn6j/7B9g99Uj7wRzDj4J/su8 m3fA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701981551; x=1702586351; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AdhzPeT3AA9HptxSHcMzOSBTny61/xtguzzqnxzNv6M=; b=B2GJHTknxypwbdcXGNr+He1vTNiFJarPjFL9smNMZBIDDhmEzFXbtgUG+p0BYrX9BZ L3rfAq9yGbBxkkH4Y85TdlZwIZtyGPEVvP8BQTDCy4//hA5O7tEEnpGDfrtKqgI6y4lJ sDAV2S6Hgv4zt1t5p/xvnEa/Ztp0qoRa9UP2YsKUjeLMVxCClMJFEAA3FqsBSZ9XHIF8 1v7ZtGogh8w5bjfjBT/H8QqOg9aPrcpjqwDpGNPwlD/vHSeKkjtH2pQEYDzETaOvCjbS nUJ21WIwJSARtpkcP5qvrJesXzYZUZLnvt5CXqXIkXIQZwAbPK69hvBqcnSPyv3wtrUl MWNg== X-Gm-Message-State: AOJu0YwinTWVzjdoLp2JIQhoo1jLbDyFL1eyZSh7hmK1daA0xscd1bIb q5q/e+h8xcCrtU+bn7bWyJQ= X-Google-Smtp-Source: AGHT+IFoRR+fMAQqHguc39v63CeE4a68qBPyp6WkyZssDcAd/eM9gVm70j8RBrJTh88u+YpFV7wLkQ== X-Received: by 2002:a81:6cc3:0:b0:5d3:464d:18d9 with SMTP id h186-20020a816cc3000000b005d3464d18d9mr3060800ywc.21.1701981551006; Thu, 07 Dec 2023 12:39:11 -0800 (PST) Received: from localhost ([2601:344:8301:57f0:8f7f:efcd:5388:af4f]) by smtp.gmail.com with ESMTPSA id n186-20020a0dcbc3000000b005ca4acaf181sm145636ywd.32.2023.12.07.12.39.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Dec 2023 12:39:10 -0800 (PST) From: Yury Norov To: Andrew Morton , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Yury Norov , Ming Lei , Andy Shevchenko , Rasmus Villemoes Subject: [PATCH 6/6] lib/group_cpus.c: drop unneeded cpumask_empty() call in __group_cpus_evenly() Date: Thu, 7 Dec 2023 12:39:00 -0800 Message-Id: <20231207203900.859776-7-yury.norov@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231207203900.859776-1-yury.norov@gmail.com> References: <20231207203900.859776-1-yury.norov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The function is called twice. First time it's called with cpumask_present as a parameter, which can't be empty. Second time it's called with a mask created with cpumask_andnot(), which returns false if the result is an empty mask. We can safely drop redundand cpumask_empty() call from the __group_cpus_evenly() and save few cycles. Signed-off-by: Yury Norov --- lib/group_cpus.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/lib/group_cpus.c b/lib/group_cpus.c index c7fcd04c87bf..664a56171a1b 100644 --- a/lib/group_cpus.c +++ b/lib/group_cpus.c @@ -252,9 +252,6 @@ static int __group_cpus_evenly(unsigned int startgrp, u= nsigned int numgrps, nodemask_t nodemsk =3D NODE_MASK_NONE; struct node_groups *node_groups; =20 - if (cpumask_empty(cpu_mask)) - return 0; - nodes =3D get_nodes_in_cpumask(node_to_cpumask, cpu_mask, &nodemsk); =20 /* @@ -394,9 +391,14 @@ struct cpumask *group_cpus_evenly(unsigned int numgrps) curgrp =3D 0; else curgrp =3D nr_present; - cpumask_andnot(npresmsk, cpu_possible_mask, npresmsk); - ret =3D __group_cpus_evenly(curgrp, numgrps, node_to_cpumask, - npresmsk, nmsk, masks); + + if (cpumask_andnot(npresmsk, cpu_possible_mask, npresmsk)) + /* If npresmsk is not empty */ + ret =3D __group_cpus_evenly(curgrp, numgrps, node_to_cpumask, + npresmsk, nmsk, masks); + else + ret =3D 0; + if (ret >=3D 0) nr_others =3D ret; =20 --=20 2.40.1