From nobody Tue Apr 7 14:38:07 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFBCD1A840A; Thu, 12 Mar 2026 16:18:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332312; cv=none; b=BiKiUIO+WSHIw/SXXVhje/BiaZrBNNPYHYj5O7bJzv3BGPBKWJXgbv8T6JOf0/seWOYZZ2Fp0gJQKfstOZIaYWiaDZ2vTYO851u/iEAAL6EImO4678D12SvnO9vJhVENoVXHtMBTIg2EX2CPiHlEePEpptznKgSUwmS/HJF3wAI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332312; c=relaxed/simple; bh=V3jtJrPXZl14ViLFqwIBk+7KYJV4qJ1TdSb5KiASj14=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JPY7ioV5VBbsAqjdanJZJ4upojA1AuuIshU3YwQctB7UzkTlvu1rYPD+sQg3rkoxwZXnhekHgDAuvK28Uy2pOM6+AUx2CMKfN/upU8CPvv/K6l62lLHHCCfmjz+dvVw2YNknxxYy/vSfWCZInUwTTodGMUmTVmPk2Nj3Swp0FYc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org; spf=none smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=kuogqekX; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="kuogqekX" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=+zvab4KFQlKv1qZs2xhxlP0HAajHsKybqOSHyuJZ5VU=; b=kuogqekXPh4sawAZLMoF3tpuOr orBMpfsDjko7kLjFsIhaXBVXy3OpU0j4QPEM6CGdZ1EIz0jiHysdr0lCDJb4hk9cZin7866h6Lkhq oWAC5quEs6i0+9LvjKxs9CxztTVlIcCMiTlwWkgaiXjUSCtlZuedpTbB6aQPTB4p4cv+j4XLyOsFg jkoIdobmysPHzGnBrzCkUWqQdHyO4YM0CCox4eP77tmR7TvpdwpFvqr1M5uPKRK6D/LpnDcxPLs4E Owu9+Y/B1IguLSYBcY92qmYeO6iRKEbbgsANdnVzq5Lc4EqdJMEXODXR+YCAHA/yjj0XQjZh8pI4K fD+4t83g==; Received: from authenticated user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.94.2) (envelope-from ) id 1w0ik0-004fNo-V3; Thu, 12 Mar 2026 16:18:29 +0000 From: Breno Leitao Date: Thu, 12 Mar 2026 09:12:02 -0700 Subject: [PATCH RFC 1/5] workqueue: fix parse_affn_scope() prefix matching bug Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260312-workqueue_sharded-v1-1-2c43a7b861d0@debian.org> References: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> In-Reply-To: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> To: Tejun Heo , Lai Jiangshan , Andrew Morton Cc: linux-kernel@vger.kernel.org, puranjay@kernel.org, linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, Michael van der Westhuizen , kernel-team@meta.com, Chuck Lever , Breno Leitao X-Mailer: b4 0.15-dev-363b9 X-Developer-Signature: v=1; a=openpgp-sha256; l=1927; i=leitao@debian.org; h=from:subject:message-id; bh=V3jtJrPXZl14ViLFqwIBk+7KYJV4qJ1TdSb5KiASj14=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBpsudLxsyud32dXZ5G3MiHEhCZKnipiEracsPWJ fvgnjaKRaCJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCabLnSwAKCRA1o5Of/Hh3 bXWOEACTU3VY+YMm03hIJ4LY0+9A+Jd6aYWkEc1EXPuwmYUW/V8hts/ypRFfVzm99iydnETghJZ ceaShHA92e4JmO+JUbNmhma261zvl1nFiJLw/3+pvEUzGJNwntWhnuzY/xzv8SN6QvmA4CMeWJc Hhm6eINC+adFiYY3AMKKUinCQNQlAlh1ZNe6bLAAdfJPLAQYiKuqt6nBHASsD76/TPynCnT4JfJ IT29cisfIOsnUYRY9dzbJrOaVFv7maZF1lK6EXStNgq8imuQ8BozEU3+zpkKaqHjVhUOlOAaz2E tr0oTPhodo4w9kXQ1RO3IeG4ODr6Fcw+0M1XBNLprwCHpzmz7Ohk/wBErOea1DvUhUaVYczWA7Z u6Rb+0EobJ935+5V2xoLWiALTv4lUyZfIQxYRPsIZQF129HiR6DJT7+gTWobAqqSygPl3aTAhkp 5a19aTVtFKnAIiI5zPHgpP9mLuVQn8ifcW5YvlELSiafi7nKcyo+qTLp8WapucuhgFcZ1dS2mGC qDEgu5393LOufMploSnTrz4ZHIO722l3m131XUDUMXUrJ5PDQp3h5rpPq/ka5K8kAvQh+sn/jPv Gq+9S3WAoT5RroxlJg1DcOCQfCbpFkqbp7co++7LbK4OEy2uiC184m7Q4Lzpe88XfuoZmzx7G/p K6bBV/yvoorFXnQ== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao parse_affn_scope() uses strncasecmp() with the length of the candidate name, which means it only checks if the input *starts with* a known scope name. Given that the upcoming diff will create "cache_shard" affinity scope, writing "cache_shard" to a workqueue's affinity_scope sysfs attribute always matches "cache" first, making it impossible to select "cache_shard" via sysfs, so, this fix enable it to distinguish "cache" and "cache_shard" Fix by replacing the hand-rolled prefix matching loop with sysfs_match_string(), which uses sysfs_streq() for exact matching (modulo trailing newlines). Also add the missing const qualifier to the wq_affn_names[] array declaration. Note that sysfs_streq() is case-sensitive, unlike the previous strncasecmp() approach. This is intentional and consistent with how other sysfs attributes handle string matching in the kernel. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index aeaec79bc09c4..028afc3d14e59 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -404,7 +404,7 @@ struct work_offq_data { u32 flags; }; =20 -static const char *wq_affn_names[WQ_AFFN_NR_TYPES] =3D { +static const char * const wq_affn_names[WQ_AFFN_NR_TYPES] =3D { [WQ_AFFN_DFL] =3D "default", [WQ_AFFN_CPU] =3D "cpu", [WQ_AFFN_SMT] =3D "smt", @@ -7063,13 +7063,7 @@ int workqueue_unbound_housekeeping_update(const stru= ct cpumask *hk) =20 static int parse_affn_scope(const char *val) { - int i; - - for (i =3D 0; i < ARRAY_SIZE(wq_affn_names); i++) { - if (!strncasecmp(val, wq_affn_names[i], strlen(wq_affn_names[i]))) - return i; - } - return -EINVAL; + return sysfs_match_string(wq_affn_names, val); } =20 static int wq_affn_dfl_set(const char *val, const struct kernel_param *kp) --=20 2.52.0 From nobody Tue Apr 7 14:38:07 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A7FD3B7762; Thu, 12 Mar 2026 16:18:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332316; cv=none; b=l4O0iCk7X4kOX5Qw5CdHzAW9jxSa8tCLHw5Go0TI8EXSyJhFU6iVY1mtwYRs5VcvZyYspAYDtjDYtlg5KpT5+duZG23siLcpKiWpuoLOviZerJUWMIOPhfZlB/OtKw8GdASgd1g4cMcCZGh5QzKTNYH/fbTLk5QFo0oc9ZAxEjk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332316; c=relaxed/simple; bh=nVjnReqOAl5uJl0frRqio6D4HaRCOrSArHGhyfJOQ3Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Rww/dhag99oyvTaJ3h2v6LWrd/pophVljRRpa6Xc6RVtJRZLhOyuBxVEBZt6YVTJzxKdX3jOFQ+RQlDRuWTyImiqsy4flPnUB4jMF5MEbWhf7AGH1744Ms2hwsFJ3UY+5Z9Mw5ziSMmGWwEwvyETRWGB36MfkUW1gla7G2oKCl4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org; spf=none smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=SbdrXXov; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="SbdrXXov" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=8Ss7mEOdT1RpJHhhABP0Jood3v2pFXXVan/EUTluaWU=; b=SbdrXXovGgIo5ZmmuF7Cn04cAU TuatgM4LSZCILGoHrbxz9pMYnIqDhCGQZZkHbuP/OBiphYSOLm+uAPMvRsGmv2lADlurWIbLRiJFN I0myjuvynAqzN4rgNJKh6khkPQ5A3eIS5IxS5rW+mexP664QS3i6VL78Zfm2BYMiOGr/VdfAGdd6R gx2jhsW62HbNTilSAER19bMaebF5QuQ0jUdzCJcIoiajWrpvnWKstYHrocYI7zcE4oL0GzdYF8RWV 3vFIXuf0foQ2uWvm3hCEIwoFB8UgGzX/EAZk+J/C2xPbI9jliS1GfNpuzVeT8okLkj9r3alIbSd1u bVmSyH6Q==; Received: from authenticated user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.94.2) (envelope-from ) id 1w0ik5-004fNu-3t; Thu, 12 Mar 2026 16:18:33 +0000 From: Breno Leitao Date: Thu, 12 Mar 2026 09:12:03 -0700 Subject: [PATCH RFC 2/5] workqueue: add WQ_AFFN_CACHE_SHARD affinity scope Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260312-workqueue_sharded-v1-2-2c43a7b861d0@debian.org> References: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> In-Reply-To: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> To: Tejun Heo , Lai Jiangshan , Andrew Morton Cc: linux-kernel@vger.kernel.org, puranjay@kernel.org, linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, Michael van der Westhuizen , kernel-team@meta.com, Chuck Lever , Breno Leitao X-Mailer: b4 0.15-dev-363b9 X-Developer-Signature: v=1; a=openpgp-sha256; l=5847; i=leitao@debian.org; h=from:subject:message-id; bh=nVjnReqOAl5uJl0frRqio6D4HaRCOrSArHGhyfJOQ3Q=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBpsudLwdUtxQFqQTctO8m367y6OICGWq/1bysG/ F+KM2ZDQbqJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCabLnSwAKCRA1o5Of/Hh3 bZHZEACxssmSChO53PROKFe45dY5yjvLvc3XjpM2V6Zdy1HX9dqtLw6sftpJ0qn4M8G9B6uTbJ3 HScEc7q0H5l90dk/8+/8TBt6eqdbm/FxJJcw6xsSklWQsLNreiafTIJ0p8pq3zkTlnY1QguZTWW 40lKdudb4xZw+fhGd9TZ2TWOmC0f4TXl8+l4xVRC//hBSahAcZrbUV90LoqIeDsAAAQVqGu4kYd UxVgwszYFY3kRPv4FG6PJKTbj4dRWu2oM/p+FZ7y1+i/hIcKNgHvG52m1U+6v/0kroWhvP01yqF RSl5oqMMZkfALqyQflMOq3GsbJrF71e+iq6ALO46UTsr2u9NAH7rNl/W+IdHT62MjShhEr2ixle 7CNEW5DvVYxxZn8HU7V+fpZ7PvUbxDBEf4uEFafqIHkyLm4QntfF1HnpqiSujViTO6S/Sdge/Uo lnWhlcd6DXPJib9rzNreh9RLroX36/Ej6FBFXlALJg9V4vpizzucEPGpzAGxSIVirCONN0ZvHxa PfNhc+ReKLGiKPcmQwmc/2PHk5mjE2A4C/NV/Ntx+zIAnYWdVhPKoG361yf2vNF7saLaWDbkag6 u4Z4lBlbEL+hd0cBcZhhL1L54AHHjT0mghoE5galkw0ZVUJ9edy2VjvH1Fcb7bdvFyctzmOznPO S6ssi1ULRBDAzNQ== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao On systems where many CPUs share one LLC, unbound workqueues using WQ_AFFN_CACHE collapse to a single worker pool, causing heavy spinlock contention on pool->lock. For example, Chuck Lever measured 39% of cycles lost to native_queued_spin_lock_slowpath on a 12-core shared-L3 NFS-over-RDMA system. The existing affinity hierarchy (cpu, smt, cache, numa, system) offers no intermediate option between per-LLC and per-SMT-core granularity. Add WQ_AFFN_CACHE_SHARD, which subdivides each LLC into groups of at most wq_cache_shard_size CPUs (default 8, tunable via boot parameter). CPUs are distributed across shards as evenly as possible -- for example, 72 CPUs with max shard size 8 produces 9 shards of 8 each. The implementation follows the same comparator pattern as other affinity scopes: cpu_cache_shard_id() computes a per-CPU shard index on the fly from the already-initialized WQ_AFFN_CACHE topology, and cpus_share_cache_shard() is passed to init_pod_type(). Benchmark on NVIDIA Grace (72 CPUs, single LLC, 50k items/thread): cpu 3433158 items/sec p50=3D16416 p90=3D17376 p95=3D17664 ns smt 3449709 items/sec p50=3D16576 p90=3D17504 p95=3D17792 ns cache_shard 2939917 items/sec p50=3D8192 p90=3D11488 p95=3D12512 ns cache 602096 items/sec p50=3D53056 p90=3D56320 p95=3D57248 ns numa 599090 items/sec p50=3D53152 p90=3D56448 p95=3D57376 ns system 598865 items/sec p50=3D53184 p90=3D56481 p95=3D57408 ns cache_shard delivers ~5x the throughput and ~6.5x lower p50 latency compared to cache scope on this 72-core single-LLC system. Suggested-by: Tejun Heo Signed-off-by: Breno Leitao --- include/linux/workqueue.h | 1 + kernel/workqueue.c | 60 +++++++++++++++++++++++++++++++++++++++++++= ++++ 2 files changed, 61 insertions(+) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index a4749f56398fd..41c946109c7d0 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -133,6 +133,7 @@ enum wq_affn_scope { WQ_AFFN_CPU, /* one pod per CPU */ WQ_AFFN_SMT, /* one pod poer SMT */ WQ_AFFN_CACHE, /* one pod per LLC */ + WQ_AFFN_CACHE_SHARD, /* synthetic sub-LLC shards */ WQ_AFFN_NUMA, /* one pod per NUMA node */ WQ_AFFN_SYSTEM, /* one pod across the whole system */ =20 diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 028afc3d14e59..6be884eb3450d 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -409,6 +409,7 @@ static const char * const wq_affn_names[WQ_AFFN_NR_TYPE= S] =3D { [WQ_AFFN_CPU] =3D "cpu", [WQ_AFFN_SMT] =3D "smt", [WQ_AFFN_CACHE] =3D "cache", + [WQ_AFFN_CACHE_SHARD] =3D "cache_shard", [WQ_AFFN_NUMA] =3D "numa", [WQ_AFFN_SYSTEM] =3D "system", }; @@ -431,6 +432,9 @@ module_param_named(cpu_intensive_warning_thresh, wq_cpu= _intensive_warning_thresh static bool wq_power_efficient =3D IS_ENABLED(CONFIG_WQ_POWER_EFFICIENT_DE= FAULT); module_param_named(power_efficient, wq_power_efficient, bool, 0444); =20 +static unsigned int wq_cache_shard_size =3D 8; +module_param_named(cache_shard_size, wq_cache_shard_size, uint, 0444); + static bool wq_online; /* can kworkers be created yet? */ static bool wq_topo_initialized __read_mostly =3D false; =20 @@ -8106,6 +8110,56 @@ static bool __init cpus_share_numa(int cpu0, int cpu= 1) return cpu_to_node(cpu0) =3D=3D cpu_to_node(cpu1); } =20 +/** + * cpu_cache_shard_id - compute the shard index for a CPU within its LLC p= od + * @cpu: the CPU to look up + * + * Returns a shard index that is unique within the CPU's LLC pod. CPUs in + * the same LLC are divided into shards no larger than wq_cache_shard_size, + * distributed as evenly as possible. E.g., 20 CPUs with max shard size 8 + * gives 3 shards of 7+7+6. + */ +static int __init cpu_cache_shard_id(int cpu) +{ + struct wq_pod_type *cache_pt =3D &wq_pod_types[WQ_AFFN_CACHE]; + const struct cpumask *pod_cpus; + int nr_cpus, nr_shards, shard_size, remainder, c; + int pos =3D 0; + + /* CPUs in the same LLC as @cpu */ + pod_cpus =3D cache_pt->pod_cpus[cache_pt->cpu_pod[cpu]]; + /* Total number of CPUs sharing this LLC */ + nr_cpus =3D cpumask_weight(pod_cpus); + /* Number of shards to split this LLC into */ + nr_shards =3D DIV_ROUND_UP(nr_cpus, wq_cache_shard_size); + /* Minimum number of CPUs per shard */ + shard_size =3D nr_cpus / nr_shards; + /* First @remainder shards get one extra CPU */ + remainder =3D nr_cpus % nr_shards; + + /* Find position of @cpu within its cache pod */ + for_each_cpu(c, pod_cpus) { + if (c =3D=3D cpu) + break; + pos++; + } + + /* + * Map position to shard index. The first @remainder shards have + * (shard_size + 1) CPUs, the rest have @shard_size CPUs. + */ + if (pos < remainder * (shard_size + 1)) + return pos / (shard_size + 1); + return remainder + (pos - remainder * (shard_size + 1)) / shard_size; +} + +static bool __init cpus_share_cache_shard(int cpu0, int cpu1) +{ + if (!cpus_share_cache(cpu0, cpu1)) + return false; + return cpu_cache_shard_id(cpu0) =3D=3D cpu_cache_shard_id(cpu1); +} + /** * workqueue_init_topology - initialize CPU pods for unbound workqueues * @@ -8118,9 +8172,15 @@ void __init workqueue_init_topology(void) struct workqueue_struct *wq; int cpu; =20 + if (!wq_cache_shard_size) { + pr_warn("workqueue: cache_shard_size must be > 0, setting to 1\n"); + wq_cache_shard_size =3D 1; + } + init_pod_type(&wq_pod_types[WQ_AFFN_CPU], cpus_dont_share); init_pod_type(&wq_pod_types[WQ_AFFN_SMT], cpus_share_smt); init_pod_type(&wq_pod_types[WQ_AFFN_CACHE], cpus_share_cache); + init_pod_type(&wq_pod_types[WQ_AFFN_CACHE_SHARD], cpus_share_cache_shard); init_pod_type(&wq_pod_types[WQ_AFFN_NUMA], cpus_share_numa); =20 wq_topo_initialized =3D true; --=20 2.52.0 From nobody Tue Apr 7 14:38:07 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 365703F7AA3; Thu, 12 Mar 2026 16:18:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332320; cv=none; b=Ry2N+HWyWlmgu5JQ3FB4dd51xMklHlJIsjGfXrSVQ2D+ieqfzuurGnWS4K2Oa1sLEscX5PmHrOWBNx7iA4kwQSnc5veoJmysCPTsk5H9IyTTW8pvv9Ml49Wmub9CbeeA3PbdGuyQfg9RO+AO+X4PIKD/wZPHMoNPEnP5DGIoUiA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332320; c=relaxed/simple; bh=xk51tRMJJDjV8HKN7WBOv83nOoyyjCCpQbhatYusSbw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oIdbgtEPTtfJJAXEZELgzuvGHi8MDgdUznnByWK2b6mOGmVdu6Aw1R+2+JabD+2eO3+axiNy08Dyo7jgR1z5WqPJHPKk7ytpFKMh5TOtY93kC9YhMY8C3iVYOYxyPWdupBelPu2X1PVoSjjxtTusMTtnsi0bNZunX6iS7t4ubqw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org; spf=none smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=aGtbaVfo; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="aGtbaVfo" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=0H6/PC9NLycqf/POlIHPnmw1gr1N1T/l6m4kpIw3Isw=; b=aGtbaVfotTNP7znBVkZ7vFHpGm aXxvouOdeAs14NEnmHtgtOAVhppgSSSniZPEbrS6CpVbl2bZa3qygduFUBEoSnrFbFV0FpGXZPmOa JDC8UNTurH2r2G/w4lv9C5KcuSEWD7Xs/M2ODFuO6UPGbePZxKKakvDOuw1sDsRwFV3OWnK3Mjmew Nhl/G/wZ07QpVhxk3vpT2eAr0F0nqeUj4GKlYmx9UIWfaQJt5ECBTfeMxEiFuo6Nh/0XTe1N/JF0F oT2aDuHMl8f9M1BKH/KwYqlEXF0Ayy/7fkDD7/Zfuo8VdVnDZwSFW+A2X0Yo5JuyR/kOYbETsMUKc +MHFTt+w==; Received: from authenticated user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.94.2) (envelope-from ) id 1w0ik9-004fO6-8a; Thu, 12 Mar 2026 16:18:37 +0000 From: Breno Leitao Date: Thu, 12 Mar 2026 09:12:04 -0700 Subject: [PATCH RFC 3/5] workqueue: set WQ_AFFN_CACHE_SHARD as the default affinity scope Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260312-workqueue_sharded-v1-3-2c43a7b861d0@debian.org> References: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> In-Reply-To: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> To: Tejun Heo , Lai Jiangshan , Andrew Morton Cc: linux-kernel@vger.kernel.org, puranjay@kernel.org, linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, Michael van der Westhuizen , kernel-team@meta.com, Chuck Lever , Breno Leitao X-Mailer: b4 0.15-dev-363b9 X-Developer-Signature: v=1; a=openpgp-sha256; l=1448; i=leitao@debian.org; h=from:subject:message-id; bh=xk51tRMJJDjV8HKN7WBOv83nOoyyjCCpQbhatYusSbw=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBpsudL6nyu3vy2n//387TV5//nEC+hUeDjV4qbW nJB3GeD0UuJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCabLnSwAKCRA1o5Of/Hh3 bSp5D/0fDX/gBCGV32XAVSFEIFj8a8n84+Md+6Dw97G5m97UW9sH/PVSgbl1F4bIonbQtKcGlhH Kd5ZBiKcRRFpVAvexFmBmQv8dY1Jp+WtkimsEyrjL73ch7A/XF8lrihNsqbwF/EZPWiS1dlOEE8 0w97YCaoQRrMYFe367rABdrpav5mmLUflcPmY2HiwzHGXxioWnBjVvK99250/IbXj2U5HYi3bDM p8G2DEAtDEaL5uNe9w1Ol0lxRNTVbmgcwI2vCvKw6MNGIMZaGv9yyJSYkwMZyZT3pn7AIHsB4d+ V0LgVoXoGl3VUyefJOz/pBPRVQF/XwIGv/atuV/lg0PWEokoBvivvDyiaSfmlC79NdmxPECwyVK 9D7h+lRajpwbevM4tu3cwuQX0CHTzUfktCD3qCplq2OOwcDkDqu05yXbxChjVfp3CCUL2ChlcyM 0tAJeFPiRGrtCVhtDmVBOoYoo5KATZa7BD59JM1CyXiMIw6ki9xFFYYu8Yn8WnXZ3znoxapTy83 HeMVhZfzna1aZo7zwtZI9TRpMF/Zm7lEFrSlHsLE0fB1yrsLvRhJe44As2ikgcvxUfGlHV0PgvW 6LI1f9R82OGXDZj66gQxNx+mYrRYLeGrMfsUbCNbfRCyAdH2ZmgXYitGKwzQv0hpyMFG8bOyWEW 0+64TdKtDiXaxWQ== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao Set WQ_AFFN_CACHE_SHARD as the default affinity scope for unbound workqueues. On systems where many CPUs share one LLC, the previous default (WQ_AFFN_CACHE) collapses all CPUs to a single worker pool, causing heavy spinlock contention on pool->lock. WQ_AFFN_CACHE_SHARD subdivides each LLC into smaller groups, providing a better balance between locality and contention. Users can revert to the previous behavior with workqueue.default_affinity_scope=3Dcache. On systems with 8 or fewer CPUs per LLC, CACHE_SHARD produces a single shard covering the entire LLC, making it functionally identical to the previous CACHE default. The sharding only activates when an LLC has more than 8 CPUs. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 6be884eb3450d..0d3bad2bfdaae 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -441,7 +441,7 @@ static bool wq_topo_initialized __read_mostly =3D false; static struct kmem_cache *pwq_cache; =20 static struct wq_pod_type wq_pod_types[WQ_AFFN_NR_TYPES]; -static enum wq_affn_scope wq_affn_dfl =3D WQ_AFFN_CACHE; +static enum wq_affn_scope wq_affn_dfl =3D WQ_AFFN_CACHE_SHARD; =20 /* buf for wq_update_unbound_pod_attrs(), protected by CPU hotplug exclusi= on */ static struct workqueue_attrs *unbound_wq_update_pwq_attrs_buf; --=20 2.52.0 From nobody Tue Apr 7 14:38:07 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B0CC3F20E8; Thu, 12 Mar 2026 16:18:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332324; cv=none; b=T8IX6N0jNCbbLG4yS2l1q6L9HbJH9zbK1FpxuJE8bPVlWiTDI2jG/Cl6wcIJboXkHKb8q3nMT5F36YNzsi2XpyTl5YKwzyQeuZw5j5y6/AtRVnlsR0rV3Mn0FvHmaW8ygIw/DrqRJh6i1dZR3/pNlecLcOXzVTtEKL16jBUtEAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332324; c=relaxed/simple; bh=Gj8IfKq47j96G0fKxsPudSwO7TRrM8S+UvuPgEmizNs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oVHsDwYkQ1CNGHsW3X4hvQ502j52JWzpoxCukeuxi0NtMv1rUvHIgMloj+/kzkpC1pSgCWnDN/SD3S8mjxgKy9RKkOLmj+KGY0pBTPRLfpW1AXlN9tcOoNaLmgxw0IZl9AIqEnJt9v/uhNUkWnKfCCPG7VP4dexE4oU11UMHtNM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org; spf=none smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=Y2Z4tnok; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="Y2Z4tnok" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=ZIHytvKe6v2JmVfhWdYItXKJ9NthmUM7xANrJTNvyAw=; b=Y2Z4tnok2izaOnmjLcLD+XStds tFNrXdcOXoBJFPfQyv3eRDDBqkAuf7KQsBdotVWPWwj6xM5NTSMG5N2eRkjT8fRFXZGiy30La5iZW NuRo/qtNnpzZTcq161ZIzOLwsPqiybw/NcyMFz5S2JfCjuKjHnjS6Kt5JAuA5aBXWMBEP7r+jDQwW 93m7XMIfRkt/xVk+mgF0jpu6oC1YJ+JJRYRulRLgm/JI/aUTmPcYQnKQ4R4TFrXQ3xG2kQOYpA7HV FUpIBGFhGvcgy2ukZS5/lMQdec6ZuDInYa1b1nwPMZpTvMo15yfahkewn33dkxqrmH8Vvwz82jmeh I5PZifJg==; Received: from authenticated user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.94.2) (envelope-from ) id 1w0ikD-004fOG-AP; Thu, 12 Mar 2026 16:18:41 +0000 From: Breno Leitao Date: Thu, 12 Mar 2026 09:12:05 -0700 Subject: [PATCH RFC 4/5] workqueue: add test_workqueue benchmark module Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260312-workqueue_sharded-v1-4-2c43a7b861d0@debian.org> References: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> In-Reply-To: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> To: Tejun Heo , Lai Jiangshan , Andrew Morton Cc: linux-kernel@vger.kernel.org, puranjay@kernel.org, linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, Michael van der Westhuizen , kernel-team@meta.com, Chuck Lever , Breno Leitao X-Mailer: b4 0.15-dev-363b9 X-Developer-Signature: v=1; a=openpgp-sha256; l=9313; i=leitao@debian.org; h=from:subject:message-id; bh=Gj8IfKq47j96G0fKxsPudSwO7TRrM8S+UvuPgEmizNs=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBpsudL+tfkq//0w425LRC/heBwiMlasFBT7f8cN jS21m7ItbGJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCabLnSwAKCRA1o5Of/Hh3 bSiHEACQp1tkO1ZaXom6sAAUHzGRAKZXHC3kdQ0XASKS7egkwXSmBf2rI//rtQkINdGpMhbL3c3 FRmywug4ymxsf5paetNu8lhki9loJ6ZQEGUxLz5BQyQX652uarRdA8JMhEV5iAC8ajaXLTN/TXz 0K5WU5/kLZFnUJ9/l3k0aqtgA0O6phdvFFjseUzO1dRVlQ0m1N+kO15E7NZs6zH+6V8AMCicDna EUYCsMf0IEOdVmRyeXjU1wUzCnXESMUxeS61Cbgqm0HeqtzTHO8e7iIARodZ1iK8Gyj6GSEUif+ aZXCVqCkviWwsX5nUX6HjtdE1fKpvp1ioiOMcTgSG6Guuoc1pC1VuDsUjQhtVAnSzTEXyLOlNgD 5JPhuvSR7ElMA0o9sFE7K7SXpFc8mL/J/KNa48OVNCIrjrSsUBB9CxUSQ++T/I8N+R7uA/GclUm iU2LvYWIzqkiinCKMIm+iPqlU8e+/79/Y0l9q+ITxz39qM5RLXtJXa+w32cw0g2pVg8Hihq4UGj vHnXSa30BqU8/DDupuSyKSwbXyhWt/yPJdbtYOQ9nskuR3bp0Gc10S5tQGItJoYMDlcQ+xTOKfR W9lBadVLh7UmTPvfCmRZS6RXAkgjQPfbsg4T3Gi5u8Dvo/FWBpQ/ypCmZDwcCjhLn6p0EA8m34a COmQpeyiR2uuKLw== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao Add a kernel module that benchmarks queue_work() throughput on an unbound workqueue to measure pool->lock contention under different affinity scope configurations (cache vs cache_shard). The module spawns N kthreads (default: num_online_cpus()), each bound to a different CPU. All threads start simultaneously and queue work items, measuring the latency of each queue_work() call. Results are reported as p50/p90/p95 latencies for each affinity scope. The affinity scope is switched between runs via the workqueue's sysfs affinity_scope attribute (WQ_SYSFS), avoiding the need for any new exported symbols. The module runs as __init-only, returning -EAGAIN to auto-unload, and can be re-run via insmod. Signed-off-by: Breno Leitao --- lib/Kconfig.debug | 10 ++ lib/Makefile | 1 + lib/test_workqueue.c | 275 +++++++++++++++++++++++++++++++++++++++++++++++= ++++ 3 files changed, 286 insertions(+) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 93f356d2b3d95..38bee649697f3 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2628,6 +2628,16 @@ config TEST_VMALLOC =20 If unsure, say N. =20 +config TEST_WORKQUEUE + tristate "Test module for stress/performance analysis of workqueue" + default n + help + This builds the "test_workqueue" module for benchmarking + workqueue throughput under contention. Useful for evaluating + affinity scope changes (e.g., cache_shard vs cache). + + If unsure, say N. + config TEST_BPF tristate "Test BPF filter functionality" depends on m && NET diff --git a/lib/Makefile b/lib/Makefile index 1b9ee167517f3..ea660cca04f40 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -79,6 +79,7 @@ UBSAN_SANITIZE_test_ubsan.o :=3D y obj-$(CONFIG_TEST_KSTRTOX) +=3D test-kstrtox.o obj-$(CONFIG_TEST_LKM) +=3D test_module.o obj-$(CONFIG_TEST_VMALLOC) +=3D test_vmalloc.o +obj-$(CONFIG_TEST_WORKQUEUE) +=3D test_workqueue.o obj-$(CONFIG_TEST_RHASHTABLE) +=3D test_rhashtable.o obj-$(CONFIG_TEST_STATIC_KEYS) +=3D test_static_keys.o obj-$(CONFIG_TEST_STATIC_KEYS) +=3D test_static_key_base.o diff --git a/lib/test_workqueue.c b/lib/test_workqueue.c new file mode 100644 index 0000000000000..82540e5536078 --- /dev/null +++ b/lib/test_workqueue.c @@ -0,0 +1,275 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Test module for stress and performance analysis of workqueue. + * + * Benchmarks queue_work() throughput on an unbound workqueue to measure + * pool->lock contention under different affinity scope configurations + * (e.g., cache vs cache_shard). + * + * The affinity scope is changed between runs via the workqueue's sysfs + * affinity_scope attribute (WQ_SYSFS). + * + * Copyright (c) 2026 Meta Platforms, Inc. and affiliates + * Copyright (c) 2026 Breno Leitao + * + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define WQ_NAME "bench_wq" +#define SCOPE_PATH "/sys/bus/workqueue/devices/" WQ_NAME "/affinity_scope" + +static int nr_threads; +module_param(nr_threads, int, 0444); +MODULE_PARM_DESC(nr_threads, + "Number of threads to spawn (default: 0 =3D num_online_cpus())"); + +static int wq_items =3D 50000; +module_param(wq_items, int, 0444); +MODULE_PARM_DESC(wq_items, + "Number of work items each thread queues (default: 50000)"); + +static struct workqueue_struct *bench_wq; +static atomic_t completed; +static atomic_t threads_done; +static DECLARE_COMPLETION(start_comp); +static DECLARE_COMPLETION(all_done_comp); + +struct thread_ctx { + struct work_struct work; + struct completion work_done; + u64 *latencies; + int cpu; + int items; +}; + +static void bench_work_fn(struct work_struct *work) +{ + struct thread_ctx *ctx =3D container_of(work, struct thread_ctx, work); + + atomic_inc(&completed); + complete(&ctx->work_done); +} + +static int bench_kthread_fn(void *data) +{ + struct thread_ctx *ctx =3D data; + ktime_t t_start, t_end; + int i; + + /* Wait for all threads to be ready */ + wait_for_completion(&start_comp); + + for (i =3D 0; i < ctx->items; i++) { + reinit_completion(&ctx->work_done); + INIT_WORK(&ctx->work, bench_work_fn); + + t_start =3D ktime_get(); + queue_work(bench_wq, &ctx->work); + t_end =3D ktime_get(); + + ctx->latencies[i] =3D ktime_to_ns(ktime_sub(t_end, t_start)); + wait_for_completion(&ctx->work_done); + } + + if (atomic_dec_and_test(&threads_done)) + complete(&all_done_comp); + + return 0; +} + +static int cmp_u64(const void *a, const void *b) +{ + u64 va =3D *(const u64 *)a; + u64 vb =3D *(const u64 *)b; + + if (va < vb) + return -1; + if (va > vb) + return 1; + return 0; +} + +static int __init set_affn_scope(const char *scope) +{ + struct file *f; + loff_t pos =3D 0; + ssize_t ret; + + f =3D filp_open(SCOPE_PATH, O_WRONLY, 0); + if (IS_ERR(f)) { + pr_err("test_workqueue: open %s failed: %ld\n", + SCOPE_PATH, PTR_ERR(f)); + return PTR_ERR(f); + } + + ret =3D kernel_write(f, scope, strlen(scope), &pos); + filp_close(f, NULL); + + if (ret < 0) { + pr_err("test_workqueue: write '%s' failed: %zd\n", scope, ret); + return ret; + } + + return 0; +} + +static int __init run_bench(int n_threads, const char *scope) +{ + struct thread_ctx *ctxs; + struct task_struct **tasks; + u64 *all_latencies; + unsigned long total_items; + ktime_t start, end; + s64 elapsed_us; + int cpu, i, j, ret; + + ret =3D set_affn_scope(scope); + if (ret) + return ret; + + ctxs =3D kcalloc(n_threads, sizeof(*ctxs), GFP_KERNEL); + if (!ctxs) + return -ENOMEM; + + tasks =3D kcalloc(n_threads, sizeof(*tasks), GFP_KERNEL); + if (!tasks) { + kfree(ctxs); + return -ENOMEM; + } + + total_items =3D (unsigned long)n_threads * wq_items; + all_latencies =3D kvmalloc_array(total_items, sizeof(u64), GFP_KERNEL); + if (!all_latencies) { + kfree(tasks); + kfree(ctxs); + return -ENOMEM; + } + + /* Allocate per-thread latency arrays */ + for (i =3D 0; i < n_threads; i++) { + ctxs[i].latencies =3D kvmalloc_array(wq_items, sizeof(u64), + GFP_KERNEL); + if (!ctxs[i].latencies) { + while (--i >=3D 0) + kvfree(ctxs[i].latencies); + kvfree(all_latencies); + kfree(tasks); + kfree(ctxs); + return -ENOMEM; + } + } + + atomic_set(&completed, 0); + atomic_set(&threads_done, n_threads); + reinit_completion(&all_done_comp); + reinit_completion(&start_comp); + + /* Create kthreads, each bound to a different online CPU */ + i =3D 0; + for_each_online_cpu(cpu) { + if (i >=3D n_threads) + break; + + ctxs[i].cpu =3D cpu; + ctxs[i].items =3D wq_items; + init_completion(&ctxs[i].work_done); + + tasks[i] =3D kthread_create(bench_kthread_fn, &ctxs[i], + "wq_bench/%d", cpu); + if (IS_ERR(tasks[i])) { + ret =3D PTR_ERR(tasks[i]); + pr_err("test_workqueue: failed to create kthread %d: %d\n", + i, ret); + while (--i >=3D 0) + kthread_stop(tasks[i]); + goto out_free; + } + + kthread_bind(tasks[i], cpu); + wake_up_process(tasks[i]); + i++; + } + + /* Start timing and release all threads */ + start =3D ktime_get(); + complete_all(&start_comp); + + /* Wait for all threads to finish */ + wait_for_completion(&all_done_comp); + + /* Drain any remaining work */ + flush_workqueue(bench_wq); + + end =3D ktime_get(); + elapsed_us =3D ktime_us_delta(end, start); + + /* Merge all per-thread latencies and sort for percentile calculation */ + j =3D 0; + for (i =3D 0; i < n_threads; i++) { + memcpy(&all_latencies[j], ctxs[i].latencies, + wq_items * sizeof(u64)); + j +=3D wq_items; + } + + sort(all_latencies, total_items, sizeof(u64), cmp_u64, NULL); + + pr_info("test_workqueue: %-12s %llu items/sec\tp50=3D%llu\tp90=3D%llu\t= p95=3D%llu ns\n", + scope, + elapsed_us ? total_items * 1000000ULL / elapsed_us : 0, + all_latencies[total_items * 50 / 100], + all_latencies[total_items * 90 / 100], + all_latencies[total_items * 95 / 100]); + + ret =3D 0; +out_free: + for (i =3D 0; i < n_threads; i++) + kvfree(ctxs[i].latencies); + kvfree(all_latencies); + kfree(tasks); + kfree(ctxs); + + return ret; +} + +static const char * const bench_scopes[] =3D { + "cpu", "smt", "cache_shard", "cache", "numa", "system", +}; + +static int __init test_workqueue_init(void) +{ + int n_threads =3D nr_threads ?: num_online_cpus(); + int i; + + bench_wq =3D alloc_workqueue(WQ_NAME, WQ_UNBOUND | WQ_SYSFS, 0); + if (!bench_wq) + return -ENOMEM; + + pr_info("test_workqueue: running %d threads, %d items/thread\n", + n_threads, wq_items); + + for (i =3D 0; i < ARRAY_SIZE(bench_scopes); i++) + run_bench(n_threads, bench_scopes[i]); + + destroy_workqueue(bench_wq); + + return -EAGAIN; +} + +module_init(test_workqueue_init); +MODULE_AUTHOR("Breno Leitao "); +MODULE_DESCRIPTION("Stress/performance benchmark for workqueue subsystem"); +MODULE_LICENSE("GPL"); --=20 2.52.0 From nobody Tue Apr 7 14:38:07 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B1AB3F7874; Thu, 12 Mar 2026 16:18:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332328; cv=none; b=e7ZF/9zEqDxR6n+LROcaEViRTZIlL9To+PfBLSl1mOyOCKciXyG0b+BR6UFA3C/w82+69BVy4dxZsAmvwlkfuQ/CCALwKUkxrUMY1snKKg0adkkNCGOos6c62vB9WOQqPwVqjmFrxrmsKx5ynwbyBzyO8S72MkrQYs7PLQVNQ0E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773332328; c=relaxed/simple; bh=r+H4nN8FEm5hqBTVEcbg+154s0B6+qEaV7hbQZF5BCU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ZoQYV6yvDdaMnNrLUAU/ahfsElvHQA8Y0l3dwa4QBKiAOSdPJ0q1iMAaknu0eyNZkw2y63BZ0S0j+xMI/9bwj32oEKvYBuEsxQjn0tJeyfFBdz9RkYLbA21VQr4a5Ltk/qV3YNhPiwL/HTtYmET59bSvwvh8UvzstRBVkc8jNfk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org; spf=none smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=FkQyeAB/; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="FkQyeAB/" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=VG0dZbZo9KDGej1wf7CUWCHE5nKIFztp1R/yW2wVRKY=; b=FkQyeAB/VV7RF0kXpLFUAlFwvL HXDl9sN9WFn2NMcnyOuatdnTn5etxr8r7l+ncxCQrJamiNBLbEnshGUzlpu0cFvPW/Ul3asIXF74A OQTYFDB17IMKzM0VXvOGPB7/kZMmIR+qVeS+NQGcpO9H+8c/wtTNw+AZ6JqhiXUvaeaLTTFSDY8nS 0V6yJLFqyAIQo8h3ZlyioJdOYOfTBWD11gOxS5/0sOjYA4x6kHCw3MJsDvq73q6fouRJq2EUa7vZZ uWN5UYEHY53aWAwHEjj5LhVDIHYQG7GRQzGqx5Gq1/Dahx/oqtHROrpokctpLyM5/L5zIBf14kvPQ WVKC1k8A==; Received: from authenticated user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.94.2) (envelope-from ) id 1w0ikH-004fOP-Fd; Thu, 12 Mar 2026 16:18:45 +0000 From: Breno Leitao Date: Thu, 12 Mar 2026 09:12:06 -0700 Subject: [PATCH RFC 5/5] tools/workqueue: add CACHE_SHARD support to wq_dump.py Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260312-workqueue_sharded-v1-5-2c43a7b861d0@debian.org> References: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> In-Reply-To: <20260312-workqueue_sharded-v1-0-2c43a7b861d0@debian.org> To: Tejun Heo , Lai Jiangshan , Andrew Morton Cc: linux-kernel@vger.kernel.org, puranjay@kernel.org, linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, Michael van der Westhuizen , kernel-team@meta.com, Chuck Lever , Breno Leitao X-Mailer: b4 0.15-dev-363b9 X-Developer-Signature: v=1; a=openpgp-sha256; l=1483; i=leitao@debian.org; h=from:subject:message-id; bh=r+H4nN8FEm5hqBTVEcbg+154s0B6+qEaV7hbQZF5BCU=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBpsudLdu6jztiCIa49vWfR67tssVeoNTyxtlM4e G2BSgAQvS2JAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCabLnSwAKCRA1o5Of/Hh3 bZiMD/9HmhdhcqFn2Ncw/zabyyHdLOliiElUKHcXEcS64MVyiyyML/7/jidN/oEELOSWa7qmcZq QorndoXg9UF0GhV9F0P7wZ3utzL8Z7kIymthUQ6IG3SM1OTMwUGYu3ooLLUQUfSSkFDwrU01rkj AIWpV4JZOigopP22XnRcrG4VY+6ewJog19NtlE/Geu0KfUzGDrSUsqjIPRKKY5B2bv1wRJ7rxKw SFmBXDWIJ7NuRZ/hL1vkJ/kGcePkyi04c+FOsnXk50Gt2ncEiBs8injNx+O+YBxN81mAbpcRmMC mmtXP2CJ+JyowH1LcXuiPMt0skOH0OQkaDNrqro07yTgaemEjek9rJwu8vGfY+g2zI/+gZCvGsY pfN/9Oq6wGuQvlN2sOQoMA3TEWZDCIxsK9s2C9FfaQpD1eE7ULjlzQs6jvY+K0cans6AItP5C42 7RawNU+sIkZGo7pa1TeZNFKiqIu+TyxvvqkKUDRiwGHa32+4F46tm4PYCcqFyd08gI3jZRs/ZzG KTnGE0eW7Ybwj9MU+Qr+G6PObV4U0GFThoDg78FqwLmqHMHlnJQY/TzjB54lwwZ0MzPwcDzO1AN l1OBNynFfFg3h5CNGLBFav8efnVzrCtofyNRyjr0XOOwCZSnXlF9hkV4k9GzScT2Sd+ZclxbSbn kqv+o3SIGuwqGvw== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao The WQ_AFFN_CACHE_SHARD affinity scope was added to the kernel but wq_dump.py was not updated to enumerate it. Add the missing constant lookup and include it in the affinity scopes iteration so that drgn output shows the CACHE_SHARD pod topology alongside the other scopes. Signed-off-by: Breno Leitao --- tools/workqueue/wq_dump.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/workqueue/wq_dump.py b/tools/workqueue/wq_dump.py index d29b918306b48..06948ffcfc4b6 100644 --- a/tools/workqueue/wq_dump.py +++ b/tools/workqueue/wq_dump.py @@ -107,6 +107,7 @@ WQ_MEM_RECLAIM =3D prog['WQ_MEM_RECLAIM'] WQ_AFFN_CPU =3D prog['WQ_AFFN_CPU'] WQ_AFFN_SMT =3D prog['WQ_AFFN_SMT'] WQ_AFFN_CACHE =3D prog['WQ_AFFN_CACHE'] +WQ_AFFN_CACHE_SHARD =3D prog['WQ_AFFN_CACHE_SHARD'] WQ_AFFN_NUMA =3D prog['WQ_AFFN_NUMA'] WQ_AFFN_SYSTEM =3D prog['WQ_AFFN_SYSTEM'] =20 @@ -138,7 +139,7 @@ def print_pod_type(pt): print(f' [{cpu}]=3D{pt.cpu_pod[cpu].value_()}', end=3D'') print('') =20 -for affn in [WQ_AFFN_CPU, WQ_AFFN_SMT, WQ_AFFN_CACHE, WQ_AFFN_NUMA, WQ_AFF= N_SYSTEM]: +for affn in [WQ_AFFN_CPU, WQ_AFFN_SMT, WQ_AFFN_CACHE, WQ_AFFN_CACHE_SHARD,= WQ_AFFN_NUMA, WQ_AFFN_SYSTEM]: print('') print(f'{wq_affn_names[affn].string_().decode().upper()}{" (default)" = if affn =3D=3D wq_affn_dfl else ""}') print_pod_type(wq_pod_types[affn]) --=20 2.52.0