From nobody Sun Apr 19 10:41:16 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACF0EC433EF for ; Sat, 2 Jul 2022 16:08:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232419AbiGBQIp (ORCPT ); Sat, 2 Jul 2022 12:08:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232372AbiGBQIm (ORCPT ); Sat, 2 Jul 2022 12:08:42 -0400 Received: from polaris.svanheule.net (polaris.svanheule.net [84.16.241.116]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49104E0B0 for ; Sat, 2 Jul 2022 09:08:38 -0700 (PDT) Received: from terra.. (unknown [IPv6:2a02:a03f:eaf9:8401:aa9f:5d01:1b2a:e3cd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sander@svanheule.net) by polaris.svanheule.net (Postfix) with ESMTPSA id 894C92F29D9; Sat, 2 Jul 2022 18:08:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svanheule.net; s=mail1707; t=1656778114; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5skgPrJ9x2u3xnCYuEk8bAF9fVBu0gldzNJqqSQ5PUM=; b=X4bILSnWGfijScSJsNRz7j0Ze47qwJ5AkI4aIVTo8gYzwcBWTk/SwFzD6qhhWgA7pKr42F bAHlYOVA/Hah+/r9GaekI101LGDDQrvIWXjZAy/zLg2160eglDNBAGP2ZmUGn/94RDvxgc MFnhIO6g96tMzfQPrYfoAFdZhXPWwciQr3WeMFRTDFlHy66IWWIDSKBcEvclHEBWB24lT9 phhjUudrmdLs+i1im2bEJH0jKhgxTfCz9MVFx8627Fj9bgc1GDX4o5wu6VFzZ9QzFSaJBv Ad+k12HUyRxZRXCFZMjJkQCpFzqgA7zja/UvDqAEFW1mhNFGm6KW/bQqR3RFqw== From: Sander Vanheule To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Andrew Morton , Andy Shevchenko , elver@google.com, gregkh@linuxfoundation.org, Peter Zijlstra , Thomas Gleixner , vschneid@redhat.com, Yury Norov , Ingo Molnar , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Sander Vanheule Subject: [PATCH v4 1/5] x86/cacheinfo: move shared cache map definitions Date: Sat, 2 Jul 2022 18:08:24 +0200 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The maps to keep track of shared caches between CPUs on SMP systems are declared in asm/smp.h, among them specifically cpu_llc_shared_map. These maps are externally defined in cpu/smpboot.c. The latter is only compiled on CONFIG_SMP=3Dy, which means the declared extern symbols from asm/smp.h do not have a corresponding definition on uniprocessor builds. The inline cpu_llc_shared_mask() function from asm/smp.h refers to the map declaration mentioned above. This function is referenced in cacheinfo.c inside for_each_cpu() loop macros, to provide cpumask for the loop. On uniprocessor builds, the symbol for the cpu_llc_shared_map does not exist. However, the current implementation of for_each_cpu() also (wrongly) ignores the provided mask. By sheer luck, the compiler thus optimises out this unused reference to cpu_llc_shared_map, and the linker therefore does not require the cpu_llc_shared_mask to actually exist on uniprocessor builds. Only on SMP bulids does smpboot.o exist to provide the required symbols. To no longer rely on compiler optimisations for successful uniprocessor builds, move the definitions of cpu_llc_shared_map and cpu_l2c_shared_map from smpboot.c to cacheinfo.c. Signed-off-by: Sander Vanheule --- arch/x86/kernel/cpu/cacheinfo.c | 6 ++++++ arch/x86/kernel/smpboot.c | 4 ---- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinf= o.c index fe98a1465be6..66556833d7af 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -29,6 +29,12 @@ #define LVL_3 4 #define LVL_TRACE 5 =20 +/* Shared last level cache maps */ +DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_map); + +/* Shared L2 cache maps */ +DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_l2c_shared_map); + struct _cache_table { unsigned char descriptor; char cache_type; diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 5e7f9532a10d..f24227bc3220 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -95,10 +95,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_core_map); DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_die_map); EXPORT_PER_CPU_SYMBOL(cpu_die_map); =20 -DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_map); - -DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_l2c_shared_map); - /* Per CPU bogomips and other parameters */ DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info); EXPORT_PER_CPU_SYMBOL(cpu_info); --=20 2.36.1 From nobody Sun Apr 19 10:41:16 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1E9FC43334 for ; Sat, 2 Jul 2022 16:08:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232466AbiGBQIw (ORCPT ); Sat, 2 Jul 2022 12:08:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232395AbiGBQIn (ORCPT ); Sat, 2 Jul 2022 12:08:43 -0400 Received: from polaris.svanheule.net (polaris.svanheule.net [84.16.241.116]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED0BBE09F for ; Sat, 2 Jul 2022 09:08:38 -0700 (PDT) Received: from terra.. (unknown [IPv6:2a02:a03f:eaf9:8401:aa9f:5d01:1b2a:e3cd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sander@svanheule.net) by polaris.svanheule.net (Postfix) with ESMTPSA id 4A21D2F29DA; Sat, 2 Jul 2022 18:08:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svanheule.net; s=mail1707; t=1656778114; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HgR7Wjj7jo2qbr8hVcKZouv7Jy3+ITFYDiBA1XO9ego=; b=m1vSefP+aWdwrzMtbwfzU1VLn6AgOgi9cNAOkIUer7THOIGN+9681UNlxSh7rgmXenUsQk 41XYZG8Lo+1Vyb4ey7pqozvZsIzKn1U4jv+7damxplyZHIkIkE4XZY8RwNv1dotGVaUmFa 0g5+VUqjq+D+VgxkLETmcpBg4UQLkb584jZ3owcmCYKY39Lh+pkGkqTRYIxvbcN65UYQT9 SZ12QMtGYvr5oWGS7myR2kjOvCAvMmXF1hATnNc9wz2+67jAKnR2cZXkf1QgY0gQyc7kxV /yTnThJN8JKbx1ktpG8juYMcz6NHHabQA6JIY/s9A8ow1w6Vt/Ry0xaP3HKJHA== From: Sander Vanheule To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Andrew Morton , Andy Shevchenko , elver@google.com, gregkh@linuxfoundation.org, Peter Zijlstra , Thomas Gleixner , vschneid@redhat.com, Yury Norov , Ingo Molnar , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Sander Vanheule Subject: [PATCH v4 2/5] cpumask: Fix invalid uniprocessor mask assumption Date: Sat, 2 Jul 2022 18:08:25 +0200 Message-Id: <86bf3f005abba2d92120ddd0809235cab4f759a6.1656777646.git.sander@svanheule.net> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" On uniprocessor builds, any CPU mask is assumed to contain exactly one CPU (cpu0). This assumption ignores the existence of empty masks, resulting in incorrect behaviour. cpumask_first_zero(), cpumask_next_zero(), and for_each_cpu_not() don't provide behaviour matching the assumption that a UP mask is always "1", and instead provide behaviour matching the empty mask. Drop the incorrectly optimised code and use the generic implementations in all cases. Signed-off-by: Sander Vanheule Suggested-by: Yury Norov --- Notes: Changes since v3: - Add back UP-optimised cpumask_local_spread, cpumask_any_distribute, cpumask_any_and_distribute =20 Changes since v1: - Drop UP implementations instead of trying to fix them include/linux/cpumask.h | 99 ++++++++--------------------------------- lib/Makefile | 3 +- lib/cpumask.c | 2 + 3 files changed, 22 insertions(+), 82 deletions(-) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index fe29ac7cc469..7fbef41b3093 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -116,85 +116,6 @@ static __always_inline unsigned int cpumask_check(unsi= gned int cpu) return cpu; } =20 -#if NR_CPUS =3D=3D 1 -/* Uniprocessor. Assume all masks are "1". */ -static inline unsigned int cpumask_first(const struct cpumask *srcp) -{ - return 0; -} - -static inline unsigned int cpumask_first_zero(const struct cpumask *srcp) -{ - return 0; -} - -static inline unsigned int cpumask_first_and(const struct cpumask *srcp1, - const struct cpumask *srcp2) -{ - return 0; -} - -static inline unsigned int cpumask_last(const struct cpumask *srcp) -{ - return 0; -} - -/* Valid inputs for n are -1 and 0. */ -static inline unsigned int cpumask_next(int n, const struct cpumask *srcp) -{ - return n+1; -} - -static inline unsigned int cpumask_next_zero(int n, const struct cpumask *= srcp) -{ - return n+1; -} - -static inline unsigned int cpumask_next_and(int n, - const struct cpumask *srcp, - const struct cpumask *andp) -{ - return n+1; -} - -static inline unsigned int cpumask_next_wrap(int n, const struct cpumask *= mask, - int start, bool wrap) -{ - /* cpu0 unless stop condition, wrap and at cpu0, then nr_cpumask_bits */ - return (wrap && n =3D=3D 0); -} - -/* cpu must be a valid cpu, ie 0, so there's no other choice. */ -static inline unsigned int cpumask_any_but(const struct cpumask *mask, - unsigned int cpu) -{ - return 1; -} - -static inline unsigned int cpumask_local_spread(unsigned int i, int node) -{ - return 0; -} - -static inline int cpumask_any_and_distribute(const struct cpumask *src1p, - const struct cpumask *src2p) { - return cpumask_first_and(src1p, src2p); -} - -static inline int cpumask_any_distribute(const struct cpumask *srcp) -{ - return cpumask_first(srcp); -} - -#define for_each_cpu(cpu, mask) \ - for ((cpu) =3D 0; (cpu) < 1; (cpu)++, (void)mask) -#define for_each_cpu_not(cpu, mask) \ - for ((cpu) =3D 0; (cpu) < 1; (cpu)++, (void)mask) -#define for_each_cpu_wrap(cpu, mask, start) \ - for ((cpu) =3D 0; (cpu) < 1; (cpu)++, (void)mask, (void)(start)) -#define for_each_cpu_and(cpu, mask1, mask2) \ - for ((cpu) =3D 0; (cpu) < 1; (cpu)++, (void)mask1, (void)mask2) -#else /** * cpumask_first - get the first cpu in a cpumask * @srcp: the cpumask pointer @@ -260,10 +181,29 @@ static inline unsigned int cpumask_next_zero(int n, c= onst struct cpumask *srcp) =20 int __pure cpumask_next_and(int n, const struct cpumask *, const struct cp= umask *); int __pure cpumask_any_but(const struct cpumask *mask, unsigned int cpu); + +#if NR_CPUS =3D=3D 1 +/* Uniprocessor: there is only one valid CPU */ +static inline unsigned int cpumask_local_spread(unsigned int i, int node) +{ + return 0; +} + +static inline int cpumask_any_and_distribute(const struct cpumask *src1p, + const struct cpumask *src2p) { + return cpumask_first_and(src1p, src2p); +} + +static inline int cpumask_any_distribute(const struct cpumask *srcp) +{ + return cpumask_first(srcp); +} +#else unsigned int cpumask_local_spread(unsigned int i, int node); int cpumask_any_and_distribute(const struct cpumask *src1p, const struct cpumask *src2p); int cpumask_any_distribute(const struct cpumask *srcp); +#endif /* NR_CPUS */ =20 /** * for_each_cpu - iterate over every cpu in a mask @@ -324,7 +264,6 @@ extern int cpumask_next_wrap(int n, const struct cpumas= k *mask, int start, bool for ((cpu) =3D -1; \ (cpu) =3D cpumask_next_and((cpu), (mask1), (mask2)), \ (cpu) < nr_cpu_ids;) -#endif /* SMP */ =20 #define CPU_BITS_NONE \ { \ diff --git a/lib/Makefile b/lib/Makefile index f99bf61f8bbc..bcc7e8ea0cde 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -34,10 +34,9 @@ lib-y :=3D ctype.o string.o vsprintf.o cmdline.o \ is_single_threaded.o plist.o decompress.o kobject_uevent.o \ earlycpio.o seq_buf.o siphash.o dec_and_lock.o \ nmi_backtrace.o nodemask.o win_minmax.o memcat_p.o \ - buildid.o + buildid.o cpumask.o =20 lib-$(CONFIG_PRINTK) +=3D dump_stack.o -lib-$(CONFIG_SMP) +=3D cpumask.o =20 lib-y +=3D kobject.o klist.o obj-y +=3D lockref.o diff --git a/lib/cpumask.c b/lib/cpumask.c index a971a82d2f43..b9728513a4d4 100644 --- a/lib/cpumask.c +++ b/lib/cpumask.c @@ -192,6 +192,7 @@ void __init free_bootmem_cpumask_var(cpumask_var_t mask) } #endif =20 +#if NR_CPUS > 1 /** * cpumask_local_spread - select the i'th cpu with local numa cpu's first * @i: index number @@ -279,3 +280,4 @@ int cpumask_any_distribute(const struct cpumask *srcp) return next; } EXPORT_SYMBOL(cpumask_any_distribute); +#endif /* NR_CPUS */ --=20 2.36.1 From nobody Sun Apr 19 10:41:16 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5A7BC433EF for ; Sat, 2 Jul 2022 16:08:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232484AbiGBQIx (ORCPT ); Sat, 2 Jul 2022 12:08:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232400AbiGBQIn (ORCPT ); Sat, 2 Jul 2022 12:08:43 -0400 Received: from polaris.svanheule.net (polaris.svanheule.net [84.16.241.116]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 335FAE0B8 for ; Sat, 2 Jul 2022 09:08:40 -0700 (PDT) Received: from terra.. (unknown [IPv6:2a02:a03f:eaf9:8401:aa9f:5d01:1b2a:e3cd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sander@svanheule.net) by polaris.svanheule.net (Postfix) with ESMTPSA id 0037D2F29DB; Sat, 2 Jul 2022 18:08:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svanheule.net; s=mail1707; t=1656778115; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YIfcv98TK79lXB1Xs2fPU/OqBYaVs4O/uwnTJaPycVY=; b=PL7ZSkg1br50+AbV6Lvrm+pET2SO3PARypCXNyEaQcSoEABFbFmrTp5mEWmQJlinwasXRd j3J4tyMoisD0oBKspw7sVg34hldwTfC9K91PfEostphXXaUkhEq4XbmrXMjTGTj/xnE6YY MKPpI/blQ458YsG1DezvIjzT5kg24chhGs7pJtGM2TaTQYgwSPSFPmqZUXXZ3oVWjonuia ZiwbSeM/WNj0GS1sFqAkkMFCAm1r4DHyhoOOnX2/0JoKbUG0VZXsfL7NFClRXEmLUJtkX5 7uDCaRAjtrttrTOzt+Nm2bqya3oGv8kdAjxky0LC6RK5TpmJ1cuk8hl5C8LoVg== From: Sander Vanheule To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Andrew Morton , Andy Shevchenko , elver@google.com, gregkh@linuxfoundation.org, Peter Zijlstra , Thomas Gleixner , vschneid@redhat.com, Yury Norov , Ingo Molnar , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Sander Vanheule Subject: [PATCH v4 3/5] lib/test: Introduce cpumask KUnit test suite Date: Sat, 2 Jul 2022 18:08:26 +0200 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a basic suite of tests for cpumask, providing some tests for empty and completely filled cpumasks. Signed-off-by: Sander Vanheule Reviewed-by: Andy Shevchenko Suggested-by: Yury Norov --- Notes: Changes since v3: - Test for_each_cpu*() over empty mask and cpu_possible_mask - Add Andy's Reviewed-by - Use num_{online,present,possible}_cpus() for builtin masks - Guard against CPU hotplugging during test for dynamic builtin CPU mas= ks =20 Changes since v2: - Rework for_each_* test macros, as suggested by Yury =20 Changes since v1: - New patch lib/Kconfig.debug | 9 +++ lib/Makefile | 1 + lib/test_cpumask.c | 138 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 148 insertions(+) create mode 100644 lib/test_cpumask.c diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 2e24db4bff19..04aaa20d50f9 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2021,6 +2021,15 @@ config LKDTM Documentation on how to use the module can be found in Documentation/fault-injection/provoke-crashes.rst =20 +config TEST_CPUMASK + tristate "cpumask tests" if !KUNIT_ALL_TESTS + depends on KUNIT + default KUNIT_ALL_TESTS + help + Enable to turn on cpumask tests, running at boot or module load time. + + If unsure, say N. + config TEST_LIST_SORT tristate "Linked list sorting test" if !KUNIT_ALL_TESTS depends on KUNIT diff --git a/lib/Makefile b/lib/Makefile index bcc7e8ea0cde..de3e47453fe8 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -99,6 +99,7 @@ obj-$(CONFIG_TEST_HMM) +=3D test_hmm.o obj-$(CONFIG_TEST_FREE_PAGES) +=3D test_free_pages.o obj-$(CONFIG_KPROBES_SANITY_TEST) +=3D test_kprobes.o obj-$(CONFIG_TEST_REF_TRACKER) +=3D test_ref_tracker.o +obj-$(CONFIG_TEST_CPUMASK) +=3D test_cpumask.o CFLAGS_test_fprobe.o +=3D $(CC_FLAGS_FTRACE) obj-$(CONFIG_FPROBE_SANITY_TEST) +=3D test_fprobe.o # diff --git a/lib/test_cpumask.c b/lib/test_cpumask.c new file mode 100644 index 000000000000..a31a1622f1f6 --- /dev/null +++ b/lib/test_cpumask.c @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * KUnit tests for cpumask. + * + * Author: Sander Vanheule + */ + +#include +#include +#include + +#define EXPECT_FOR_EACH_CPU_EQ(test, mask) \ + do { \ + const cpumask_t *m =3D (mask); \ + int mask_weight =3D cpumask_weight(m); \ + int cpu, iter =3D 0; \ + for_each_cpu(cpu, m) \ + iter++; \ + KUNIT_EXPECT_EQ((test), mask_weight, iter); \ + } while (0) + +#define EXPECT_FOR_EACH_CPU_NOT_EQ(test, mask) \ + do { \ + const cpumask_t *m =3D (mask); \ + int mask_weight =3D cpumask_weight(m); \ + int cpu, iter =3D 0; \ + for_each_cpu_not(cpu, m) \ + iter++; \ + KUNIT_EXPECT_EQ((test), nr_cpu_ids - mask_weight, iter); \ + } while (0) + +#define EXPECT_FOR_EACH_CPU_WRAP_EQ(test, mask) \ + do { \ + const cpumask_t *m =3D (mask); \ + int mask_weight =3D cpumask_weight(m); \ + int cpu, iter =3D 0; \ + for_each_cpu_wrap(cpu, m, nr_cpu_ids / 2) \ + iter++; \ + KUNIT_EXPECT_EQ((test), mask_weight, iter); \ + } while (0) + +#define EXPECT_FOR_EACH_CPU_BUILTIN_EQ(test, name) \ + do { \ + int mask_weight =3D num_##name##_cpus(); \ + int cpu, iter =3D 0; \ + for_each_##name##_cpu(cpu) \ + iter++; \ + KUNIT_EXPECT_EQ((test), mask_weight, iter); \ + } while (0) + +static cpumask_t mask_empty; +static cpumask_t mask_all; + +static void test_cpumask_weight(struct kunit *test) +{ + KUNIT_EXPECT_TRUE(test, cpumask_empty(&mask_empty)); + KUNIT_EXPECT_TRUE(test, cpumask_full(cpu_possible_mask)); + KUNIT_EXPECT_TRUE(test, cpumask_full(&mask_all)); + + KUNIT_EXPECT_EQ(test, 0, cpumask_weight(&mask_empty)); + KUNIT_EXPECT_EQ(test, nr_cpu_ids, cpumask_weight(cpu_possible_mask)); + KUNIT_EXPECT_EQ(test, nr_cpumask_bits, cpumask_weight(&mask_all)); +} + +static void test_cpumask_first(struct kunit *test) +{ + KUNIT_EXPECT_LE(test, nr_cpu_ids, cpumask_first(&mask_empty)); + KUNIT_EXPECT_EQ(test, 0, cpumask_first(cpu_possible_mask)); + + KUNIT_EXPECT_EQ(test, 0, cpumask_first_zero(&mask_empty)); + KUNIT_EXPECT_LE(test, nr_cpu_ids, cpumask_first_zero(cpu_possible_mask)); +} + +static void test_cpumask_last(struct kunit *test) +{ + KUNIT_EXPECT_LE(test, nr_cpumask_bits, cpumask_last(&mask_empty)); + KUNIT_EXPECT_EQ(test, nr_cpumask_bits - 1, cpumask_last(cpu_possible_mask= )); +} + +static void test_cpumask_next(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, 0, cpumask_next_zero(-1, &mask_empty)); + KUNIT_EXPECT_LE(test, nr_cpu_ids, cpumask_next_zero(-1, cpu_possible_mask= )); + + KUNIT_EXPECT_LE(test, nr_cpu_ids, cpumask_next(-1, &mask_empty)); + KUNIT_EXPECT_EQ(test, 0, cpumask_next(-1, cpu_possible_mask)); +} + +static void test_cpumask_iterators(struct kunit *test) +{ + EXPECT_FOR_EACH_CPU_EQ(test, &mask_empty); + EXPECT_FOR_EACH_CPU_NOT_EQ(test, &mask_empty); + EXPECT_FOR_EACH_CPU_WRAP_EQ(test, &mask_empty); + + EXPECT_FOR_EACH_CPU_EQ(test, cpu_possible_mask); + EXPECT_FOR_EACH_CPU_NOT_EQ(test, cpu_possible_mask); + EXPECT_FOR_EACH_CPU_WRAP_EQ(test, cpu_possible_mask); +} + +static void test_cpumask_iterators_builtin(struct kunit *test) +{ + EXPECT_FOR_EACH_CPU_BUILTIN_EQ(test, possible); + + /* Ensure the dynamic masks are stable while running the tests */ + cpu_hotplug_disable(); + + EXPECT_FOR_EACH_CPU_BUILTIN_EQ(test, online); + EXPECT_FOR_EACH_CPU_BUILTIN_EQ(test, present); + + cpu_hotplug_enable(); +} + +static int test_cpumask_init(struct kunit *test) +{ + cpumask_clear(&mask_empty); + cpumask_setall(&mask_all); + + return 0; +} + +static struct kunit_case test_cpumask_cases[] =3D { + KUNIT_CASE(test_cpumask_weight), + KUNIT_CASE(test_cpumask_first), + KUNIT_CASE(test_cpumask_last), + KUNIT_CASE(test_cpumask_next), + KUNIT_CASE(test_cpumask_iterators), + KUNIT_CASE(test_cpumask_iterators_builtin), + {} +}; + +static struct kunit_suite test_cpumask_suite =3D { + .name =3D "cpumask", + .init =3D test_cpumask_init, + .test_cases =3D test_cpumask_cases, +}; +kunit_test_suite(test_cpumask_suite); + +MODULE_LICENSE("GPL"); --=20 2.36.1 From nobody Sun Apr 19 10:41:16 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D937DC433EF for ; Sat, 2 Jul 2022 16:08:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232500AbiGBQI5 (ORCPT ); Sat, 2 Jul 2022 12:08:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232404AbiGBQIn (ORCPT ); Sat, 2 Jul 2022 12:08:43 -0400 Received: from polaris.svanheule.net (polaris.svanheule.net [84.16.241.116]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AAD8E0BC for ; Sat, 2 Jul 2022 09:08:41 -0700 (PDT) Received: from terra.. (unknown [IPv6:2a02:a03f:eaf9:8401:aa9f:5d01:1b2a:e3cd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sander@svanheule.net) by polaris.svanheule.net (Postfix) with ESMTPSA id 9066C2F29DD; Sat, 2 Jul 2022 18:08:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svanheule.net; s=mail1707; t=1656778115; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y0kiPMKA4ub6TnlZjmYnLie9lce3w7s+m9C3Z6jYGD0=; b=uDrZT02ukAUmPf33UOIX2YLgEXI2PXpReD2oh0xAHi0zQAy9q+i6Xa+no0+56IHpTyIObe iif0HMFGU/gOi8lXZ6At1sEk2AUZ2vjZRps2vFJXGOUnDiYBW/RCjcqeQetyAvV3btVN/V cimdA6JfsnUZT7zpDzVtA4BMRxW1V2mJ+gNBLDOJMX/NHavKYn1ukCJ6W5Kd5QrDMhoTAN lT4QtQnWdBy3aFIlx+uDtx5EGrPgSjUwyQ+76FBpKV4UFaqpZs2XTQtldfiwZM61Bgi8Lz /J8M9IqT/pKGRTQPW2CEWz5m0jQXDrPkbUd0HH3JBFMbNGTUM8h/Kje27DlheQ== From: Sander Vanheule To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Andrew Morton , Andy Shevchenko , elver@google.com, gregkh@linuxfoundation.org, Peter Zijlstra , Thomas Gleixner , vschneid@redhat.com, Yury Norov , Ingo Molnar , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Sander Vanheule Subject: [PATCH v4 4/5] cpumask: Add UP optimised for_each_*_cpu versions Date: Sat, 2 Jul 2022 18:08:27 +0200 Message-Id: <3a92869b902a075b97be5d1452c9c6badbbff0df.1656777646.git.sander@svanheule.net> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" On uniprocessor builds, the following loops will always run over a mask that contains one enabled CPU (cpu0): - for_each_possible_cpu - for_each_online_cpu - for_each_present_cpu Provide uniprocessor-specific macros for these loops, that always run exactly once. Signed-off-by: Sander Vanheule Acked-by: Yury Norov --- include/linux/cpumask.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index 7fbef41b3093..6c5b4ee000f2 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -750,9 +750,16 @@ extern const DECLARE_BITMAP(cpu_all_bits, NR_CPUS); /* First bits of cpu_bit_bitmap are in fact unset. */ #define cpu_none_mask to_cpumask(cpu_bit_bitmap[0]) =20 +#if NR_CPUS =3D=3D 1 +/* Uniprocessor: the possible/online/present masks are always "1" */ +#define for_each_possible_cpu(cpu) for ((cpu) =3D 0; (cpu) < 1; (cpu)++) +#define for_each_online_cpu(cpu) for ((cpu) =3D 0; (cpu) < 1; (cpu)++) +#define for_each_present_cpu(cpu) for ((cpu) =3D 0; (cpu) < 1; (cpu)++) +#else #define for_each_possible_cpu(cpu) for_each_cpu((cpu), cpu_possible_mask) #define for_each_online_cpu(cpu) for_each_cpu((cpu), cpu_online_mask) #define for_each_present_cpu(cpu) for_each_cpu((cpu), cpu_present_mask) +#endif =20 /* Wrappers for arch boot code to manipulate normally-constant masks */ void init_cpu_present(const struct cpumask *src); --=20 2.36.1 From nobody Sun Apr 19 10:41:16 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBDF0C433EF for ; Sat, 2 Jul 2022 16:08:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232428AbiGBQIs (ORCPT ); Sat, 2 Jul 2022 12:08:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232394AbiGBQIn (ORCPT ); Sat, 2 Jul 2022 12:08:43 -0400 Received: from polaris.svanheule.net (polaris.svanheule.net [IPv6:2a00:c98:2060:a004:1::200]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4058BE0C6 for ; Sat, 2 Jul 2022 09:08:42 -0700 (PDT) Received: from terra.. (unknown [IPv6:2a02:a03f:eaf9:8401:aa9f:5d01:1b2a:e3cd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sander@svanheule.net) by polaris.svanheule.net (Postfix) with ESMTPSA id 2B6492F29DE; Sat, 2 Jul 2022 18:08:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svanheule.net; s=mail1707; t=1656778116; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9/sd1UNeYBxDZ/PxaUTdlpYf5mSD1eCp0bTuKG3MEE4=; b=7xNwjdXJBs8aSZWdsKNKz6mIkDpg9yXrCQURvoiDC0XvxZMnQMnzSjmJZRlBHWpPCyez7q L+j4tphaOO5xsdyGGfUnsnPvzRviiCe+WuxJV5S3+4piJH4RXDNsBsukAJyishTs7VVhsO 2CG4DK/nCDhmPLEVEcOtTTWjkdALaVKjeVAG9uhIUR/rs6+UkAF/oUZws5H2ZM65G+WkYZ H60PmnD39Jx7lKfiOyZmTpCk7foS/W3cos6viYk2cDVvFOyBuToPuiaT2z/8UTZbL7C6+c YQ94DXIVjlNu6vM/YmF7aMMTp+XrEKj5/fdfFBl9Lm4F12jTvGE5cXQ1Ltr5PQ== From: Sander Vanheule To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Andrew Morton , Andy Shevchenko , elver@google.com, gregkh@linuxfoundation.org, Peter Zijlstra , Thomas Gleixner , vschneid@redhat.com, Yury Norov , Ingo Molnar , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Sander Vanheule Subject: [PATCH v4 5/5] cpumask: Update cpumask_next_wrap() signature Date: Sat, 2 Jul 2022 18:08:28 +0200 Message-Id: <72ab755695b74bb5fbaa756ae4c0edd708d172f1.1656777646.git.sander@svanheule.net> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The extern specifier is not needed for this declaration, so drop it. The function also depends only on the input parameters, and has no side effects, so it can be marked __pure like other functions in cpumask.h. Signed-off-by: Sander Vanheule Reviewed-by: Andy Shevchenko --- Notes: Changes since v3: - Add Andy's Reviewed-by =20 Changes since v1: - Split off patch from other changes include/linux/cpumask.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index 6c5b4ee000f2..523857884ae4 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -229,7 +229,7 @@ int cpumask_any_distribute(const struct cpumask *srcp); (cpu) =3D cpumask_next_zero((cpu), (mask)), \ (cpu) < nr_cpu_ids;) =20 -extern int cpumask_next_wrap(int n, const struct cpumask *mask, int start,= bool wrap); +int __pure cpumask_next_wrap(int n, const struct cpumask *mask, int start,= bool wrap); =20 /** * for_each_cpu_wrap - iterate over every cpu in a mask, starting at a spe= cified location --=20 2.36.1