From nobody Fri Oct 2 13:04:16 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D36023CB567 for ; Fri, 31 Jul 2026 11:57:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499078; cv=none; b=oMNdeuMi6iXkAMSy73ReGlueEmvXvqCXhsVI2Q9wYYHTuSzRx2gN0wCYaNG3sijm/aXtiwJBXCncwkuG4u+EpymVSoP7121fgZaN3++s3HFXrEWIxjfq1tpOueZP3ob/e8/fGniJdJXg3M7TgzRWkwRsswdTeSKVMaiOBYQnpIo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499078; c=relaxed/simple; bh=rdmdYMH2x++Ip0ojorw7l3QrxWNtPJvBUV4AjD3Yfr4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GFp4EF8urEZYbieF7NED4kMBNLcyuolFXndBYpCMUiPClWQlGeFWYT94d3k2EINIB8oM/zF4AlbaSp17+4BmMxdDo/sdjGdgoBUosd6i2h6fxNPdvLr5k7kQbW2B4HgAJNg4wjjdWSoELBFj0Av6WUiYa7cuWjxcpBwsQcvpUkY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=pc4Yywcm; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="pc4Yywcm" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=YzoQJ6AmlrH3orHeyfqBwZHpTshYUKv4cOhOLhYMmhw=; b=pc4Yywcm6NI7KbhOj7QNYbgNtv 8N44Ph/yRrDIfcb+dCtMiFXtxzbXpdEXNX7q5tg1S9bwBzIrp1v1mneeIGQb6vYOayuVbUwxRcJ95 ssldmQ9xJL2/I5KX0xpu4O1Glbid/A1TNM9O/LNJWFuBWyiPdRIFrCeJYjXqozfAtrxCW00A2cVak uiMeKEnjGDpqCGXybuZeYhSIWd4PCUsk8KZCDPcuI5FmiJBaO9ZUIEGV0YGLOREW5pqS7vDm/Dlif ra0MyaIPqBQNdqy5lSfYMO012sIWgxAZ+HsMF0VM7BRWFDRtO0JxeONWsqYZP+Udw4jTy98M5j/im 9Bg57lfA==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wplsA-009zcP-2u; Fri, 31 Jul 2026 11:57:55 +0000 From: Breno Leitao Date: Fri, 31 Jul 2026 04:57:33 -0700 Subject: [PATCH 1/6] workqueue: factor out get_percpu_pool() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260731-wq-pool-refactor-v1-1-8eaf71cdab5f@debian.org> References: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> In-Reply-To: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com, marco.crivellari@suse.com X-Mailer: b4 0.16-dev-d5d98 X-Developer-Signature: v=1; a=openpgp-sha256; l=1943; i=leitao@debian.org; h=from:subject:message-id; bh=rdmdYMH2x++Ip0ojorw7l3QrxWNtPJvBUV4AjD3Yfr4=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqbI286096eswp/d66bc7f2E7q4ABe+o/uOGnNn GsXLRXA9puJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCamyNvAAKCRA1o5Of/Hh3 bV2tD/9mU4kyDcRM9lpB2i5RcE4vQ57TNXFdwMKgdT7ol/AI1ZBlGay/V+ANlKvuBuVQGqGJniJ VYx4ZyvDpA9A0a0kQG2yx2zrOI4uV94xNgaG0vFtDy9XDUVq7kt0iHvI2Lb1t+tmdKLz5B5ohCD o47TMcCGLy3hIviq44dCRVATfUV2FQdGl8tZk54ziTRgrlPbFz0gBlCBt1Z+I0j89X3XZkxugC1 4uRfVLVuN8umqGKhyWiahmVK6lfAGXltEwZ7cNJFU9NXnAeT/j9lQRIk6bHFEdDT/KWb4K1JVbV +Ca8BZ/hq0648T1t7tPNwByIDYyCv0nwajXaWDedWK+hV9zOf2K32mCi6jIYlmVk7w/hTf9JC2h jxQw2VLUvGjdTHZfBhoyBCV1r0ksfACupe257HdWsm/EQHqm4FDQmDPbDkY1tvetyUCcIvICVy/ +XnThbxTxnDfM37ZtAoajrsb4EiaghZ9HS3Bn5wiDLkgZaLq2Yo5FTFfHP+SU342uTkL2SesuVK 9hrVyTh1He4F6e4cSumOHvPlJmDZ6Q0yQVR3vRXAtpWhMNxtIfp53UQ6X6n5a6rWydMrftaNCMh h8wXXUNMMje7IYZy7CF8KYtn8djhIDXZWKUIG4pruxqanOI/tLmcgH3WeMdGV6n4qpy3AyJMNWQ CHkxGZksYpKMFbQ== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao Move the static per-cpu worker_pool lookup in alloc_and_link_pwqs() into a helper, get_percpu_pool(), so the lookup can be shared by other pool-selection paths. No functional change. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 4ec3db31493d2..63a39bb3f5e47 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5350,6 +5350,20 @@ static void link_pwq(struct pool_workqueue *pwq) list_add_tail_rcu(&pwq->pwqs_node, &wq->pwqs); } =20 +/* Return the static per-cpu worker_pool that backs @wq on @cpu. */ +static struct worker_pool *get_percpu_pool(struct workqueue_struct *wq, in= t cpu) +{ + struct worker_pool __percpu *pools; + bool highpri =3D wq->flags & WQ_HIGHPRI; + + if (wq->flags & WQ_BH) + pools =3D bh_worker_pools; + else + pools =3D cpu_worker_pools; + + return &per_cpu_ptr(pools, cpu)[highpri]; +} + /* obtain a pool matching @attr and create a pwq associating the pool and = @wq */ static struct pool_workqueue *alloc_unbound_pwq(struct workqueue_struct *w= q, const struct workqueue_attrs *attrs) @@ -5664,19 +5678,9 @@ static int alloc_and_link_pwqs(struct workqueue_stru= ct *wq) goto enomem; =20 if (!(wq->flags & WQ_UNBOUND)) { - struct worker_pool __percpu *pools; - - if (wq->flags & WQ_BH) - pools =3D bh_worker_pools; - else - pools =3D cpu_worker_pools; - for_each_possible_cpu(cpu) { - struct pool_workqueue **pwq_p; - struct worker_pool *pool; - - pool =3D &(per_cpu_ptr(pools, cpu)[highpri]); - pwq_p =3D per_cpu_ptr(wq->cpu_pwq, cpu); + struct pool_workqueue **pwq_p =3D per_cpu_ptr(wq->cpu_pwq, cpu); + struct worker_pool *pool =3D get_percpu_pool(wq, cpu); =20 *pwq_p =3D kmem_cache_alloc_node(pwq_cache, GFP_KERNEL, pool->node); --=20 2.53.0-Meta From nobody Fri Oct 2 13:04:16 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F9883E4C9F for ; Fri, 31 Jul 2026 11:58:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499083; cv=none; b=pbp4rtZYowGBvNFuAsLm6GuFGFIyrb/Le+cKKvG7IxH1WZGCMl7a5hyFVPZvmKjwGNbx0zOOn1Gu/0zpKPwieP/nzDCeNOIbKhxfrPqvnt1btcjMtSYTa2GBI1JUysMWiJO6mBj+kkty41X+UcUWOLry523NNu7Q4TUF0/oVAd4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499083; c=relaxed/simple; bh=Qw4VZnmbt957FurFjdRpCtYcTdsvBE1VOkwxR+8VlJI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=C8i/bfy7evTrZkBS8VinyHI5b3ZcD3ItQschbVw98kPyw0QG0xNNIv6DMJrTNWkE0vNHvThLhYPIlOYnZZStu5s0sBj2tSUU/i0qow98uuVbF1C2krOaU+kg0SKU8e1vkrZ48Pvarl1zTZzqR6v0pgx6AQpSjJTyhi6z5+zliS4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=HmK9LsRE; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="HmK9LsRE" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=SFt5i19eclM1zEegOqGBo2n8sgiph5S6Qg+v1PM65yY=; b=HmK9LsREcu4gh5H+arKk6wQDJs M3rmgdTWkKXejZm2uwaP7wHxUyX+74QXdsdk3pGwCaLSMBLQLnHIuITFGltJX7/yF7NqiRejYdMX3 W6jD15SRSMKvgbT62UhJc8g07dsW+vfkEyhrKoNInWx992KF/1XxldzgIqPMEv7Nj99VdKWYyaog2 W3kxtP3bgd/69IX3TNduECmMaoZr9qVdtqrhr+/neeQmLpuVtqpKuwLHOmL8NMGFPNBgLhbgo3lD3 jGHYc3792NRGIZzo140cUJ8dTcWhQCLBq60sQ6nP5Xh7CDHkQQy/PYDQ2/u7wKGQcSVP1I66yQmoI JDZBv0VQ==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wplsE-009zcX-1O; Fri, 31 Jul 2026 11:57:58 +0000 From: Breno Leitao Date: Fri, 31 Jul 2026 04:57:34 -0700 Subject: [PATCH 2/6] workqueue: factor out alloc_and_link_percpu_pwqs() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260731-wq-pool-refactor-v1-2-8eaf71cdab5f@debian.org> References: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> In-Reply-To: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com, marco.crivellari@suse.com X-Mailer: b4 0.16-dev-d5d98 X-Developer-Signature: v=1; a=openpgp-sha256; l=2439; i=leitao@debian.org; h=from:subject:message-id; bh=Qw4VZnmbt957FurFjdRpCtYcTdsvBE1VOkwxR+8VlJI=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqbI28mH5+wUUKSqnPT1vNMt31voXGsjjtbzTjj xwaHygY2QSJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCamyNvAAKCRA1o5Of/Hh3 bbUHEACp+4mvxYED4n+oz0R/TC9lFdBrjjw2xSrYThQ62X88xIHbZn8hlGIgfHaMVrCBKx4P0lZ DTpCPEzTsXk0CUmqMpUcXPPjVP7AeCuKF6614ZW1X8tQNqdIaqtmrXNxVjNv6CkXq0s40nekgHK 1XuyzcqUM8cLt584rPOaaJI7rUYyZVdAX9vITGvYUXVsBdEMLJ2IlRL5Z7U0Y+HQaKBI5PUCQkm n7HRpnaYM/1xqHOdU5dHx0XtqSrosJh6h6G4XQE+zkqfErPXnOVsqGSrrNBob8/+1uJe1q0EnqU jWojHG7FvROv3mg3E67ff2ch+nOvrj98micXOvMWvpcILObvHZxPYymnYfDBj1fEjzF+IapMS2t WDXczkpPRt9d3LpkG3BoD/L9G7wMbp447d00AxlBKG6YqaljjOIXZZ++W56huLgqJ5MJUCl5gob 5Sxadiuyz1IE+h9x3PoDtzpyljzQPA30NEI0KVTVH/Pu2S5oAp69JmIIdgvYSTZuBdwsqHwQPJU SPR4eWI3ikt1am9yQN9LZquiFxhNnDoM+ILqmyAmBazxJYgCTpJBH/gy+eby2kTn0QfHuLj1LKY 6JHOwzjPsXrssEfFPAeWcSuhyxstThyjexrBPozsJCmRVA93nHwEX/6R4GRbIN3nsF+XcA7VZG4 PoIs5+NekoD2HQg== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao Move the per-cpu pwq allocation loop out of alloc_and_link_pwqs() into a helper. The inner allocation-failure path now returns -ENOMEM and the caller jumps to the existing enomem cleanup, equivalent to the previous goto. Now that the per-cpu branch returns a value through the helper, the per-cpu, ordered and unbound cases share a single error-handling tail. Turn the separate per-cpu block and the ordered/unbound if/else into one if/else-if/else chain, dropping the early return and the duplicated goto. No functional change. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 43 ++++++++++++++++++++++++------------------- 1 file changed, 24 insertions(+), 19 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 63a39bb3f5e47..b386a457c0381 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5666,6 +5666,28 @@ static void unbound_wq_update_pwq(struct workqueue_s= truct *wq, int cpu) put_pwq_unlocked(old_pwq); } =20 +static int alloc_and_link_percpu_pwqs(struct workqueue_struct *wq) +{ + int cpu; + + for_each_possible_cpu(cpu) { + struct pool_workqueue **pwq_p =3D per_cpu_ptr(wq->cpu_pwq, cpu); + struct worker_pool *pool =3D get_percpu_pool(wq, cpu); + + *pwq_p =3D kmem_cache_alloc_node(pwq_cache, GFP_KERNEL, pool->node); + if (!*pwq_p) + return -ENOMEM; + + init_pwq(*pwq_p, wq, pool); + + mutex_lock(&wq->mutex); + link_pwq(*pwq_p); + mutex_unlock(&wq->mutex); + } + + return 0; +} + static int alloc_and_link_pwqs(struct workqueue_struct *wq) { bool highpri =3D wq->flags & WQ_HIGHPRI; @@ -5678,25 +5700,8 @@ static int alloc_and_link_pwqs(struct workqueue_stru= ct *wq) goto enomem; =20 if (!(wq->flags & WQ_UNBOUND)) { - for_each_possible_cpu(cpu) { - struct pool_workqueue **pwq_p =3D per_cpu_ptr(wq->cpu_pwq, cpu); - struct worker_pool *pool =3D get_percpu_pool(wq, cpu); - - *pwq_p =3D kmem_cache_alloc_node(pwq_cache, GFP_KERNEL, - pool->node); - if (!*pwq_p) - goto enomem; - - init_pwq(*pwq_p, wq, pool); - - mutex_lock(&wq->mutex); - link_pwq(*pwq_p); - mutex_unlock(&wq->mutex); - } - return 0; - } - - if (wq->flags & __WQ_ORDERED) { + ret =3D alloc_and_link_percpu_pwqs(wq); + } else if (wq->flags & __WQ_ORDERED) { struct pool_workqueue *dfl_pwq; =20 ret =3D apply_workqueue_attrs_locked(wq, ordered_wq_attrs[highpri]); --=20 2.53.0-Meta From nobody Fri Oct 2 13:04:16 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E68340096D for ; Fri, 31 Jul 2026 11:58:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499085; cv=none; b=nD5fjFU1xhyeHqa0DiQyEWq7rOLMPR4x8dHDiSUKJAq23YQ5otot3kdgbf0s0YuMUaXtEROHociSfg2i0qDNc8vNirmCh/nhfZMhi1FK8ZdjxHng8+iDX7zP2oKRF37ry9tyWfzkyICCvJXSh0FPXuhsuzOesM4VaZwNvspuv9A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499085; c=relaxed/simple; bh=MiSspJt4lK7ECfV9N5IdfjY1SR2arU1yv9QRgEhDKNI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YPsb8MVhryuQdCCqzMEGPtoYkhRt4roEFukHzIn/jHnTJi2HmuGQ5HKCaGVOqiu31bZduGtLxsEteXnaPxmiPMtC9IVM4BnhgSFj81eatPF/o4sqhZXgrJHRZOta4uii1j7Hei+qCWFRzGk+qOpBmY4UVvLR8g403kAIVoXfl3s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=iBylOi/H; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="iBylOi/H" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=JrsPKIc8zNiNIdgz+OCZbFIxCbavGxYPyFlXhgD3gXc=; b=iBylOi/HtwqPCPkNhsMcaih8JD XdDrBueDlKBvqzmv2J+hMG5otH6q7lGnBOJCKz2sN1KlwTCiwaxQVnzZA/g+scRFkmcNnIFDDpgSr PKAHNnALKGNGtmvYtkRi2s8WSiS52yf5lXgKuPN9aBKROkuf2OA9XPvuR+d5/5VgkE0Mnr/otGcgZ OgqTPS/b5LNV4DOQKijYdNwWsuZILRTgfbUqno0Bm1BCC+UeVfJNe3LJJZRC0byFoMsHsOgK+9IAi dyUF6EXl9ygDgNGqv0/xi0JpsrPxHDqT0ZphWvbBBesPY10Ogh4rf+AVSgYHqSR25QcJ7lk5CG4nR txLU3hQQ==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wplsH-009zct-2X; Fri, 31 Jul 2026 11:58:02 +0000 From: Breno Leitao Date: Fri, 31 Jul 2026 04:57:35 -0700 Subject: [PATCH 3/6] workqueue: release pwq pools by pool type Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260731-wq-pool-refactor-v1-3-8eaf71cdab5f@debian.org> References: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> In-Reply-To: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com, marco.crivellari@suse.com X-Mailer: b4 0.16-dev-d5d98 X-Developer-Signature: v=1; a=openpgp-sha256; l=2663; i=leitao@debian.org; h=from:subject:message-id; bh=MiSspJt4lK7ECfV9N5IdfjY1SR2arU1yv9QRgEhDKNI=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqbI28K5oDicQy+nDwEkHvbZhhOIwJkJ0E1SVhH oDZDuAj/HCJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCamyNvAAKCRA1o5Of/Hh3 bciVD/0Q1e6CUslgKniSzpyOgnxkFiTb5C7ts7yxO2J+cYZ4gpb9SB7DgrQ7nWHuW9SOrMZeP6n qwgBqYnHMgzeSMGVGSxwKVOvGEWONHLYkw1bhu43e74ymlQy5V4Lr1iRaIwXIumWBTpUGsrjYsw 2wAy7kjrx52SVewKWEwyTsJzYpaA+mPycXOP668czJ7J3qkECPr+flyPRSeDTihCXYSj/+QMwkn 8NWy5MQffDiCLb4giEFSL2XYYWfLyDWk3a8aBVI1GmGyoRP8IGM3pln1OsT49gzY8/mJRLU9Nsf NbNwICKLue6ZX40L0ES7bnJqXN7arFg8KQGgHqWX91E0ypfIpcDQaEDdlF5Fb9dgvUR3usGMLXz JyAvHygw3ftj3PrhQzYoMbJbZgYyNtFnHE0AlHZyzFhXQoa+fr+yK6HXsFxR84DK5fhOwZs4v1K E3NzNE4yzr3lF7R7U/DBUYPwcb1KMHLkx2rJT6uwY0wpwSAXdwtu6F6MBJ5KRLZDHN4wtcAD6gI HsYvpOCsN+C11Ieqz0kiMsIAn4FH0giCKuXThL5GE0mGkaWK+a09KAOg02hWb+2FABTPomppOSw jBOCi3ZCk5oYleQeEL8RYM8xINxb87RGdFZsJOJCGOZ6lL11aLFkmJhaAaOXCAgV3LgBKYWQ7up j38dfFNeJ++MChA== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao pwq_release_workfn() calls put_unbound_pool() based on the WQ_UNBOUND flag. That works today because an unbound workqueue only ever points at unbound pools, but the flag is the wrong thing to test: what matters is whether the pool is a refcounted unbound pool or a permanent per-cpu one. Add is_pool_cpu_specific() and key the release on it instead of the flag. This is equivalent for every existing workqueue and stays correct if an unbound pwq is ever backed by a per-cpu pool. Convert the other open-coded pool->cpu type checks -- in put_unbound_pool(), pool_allowed_cpus() and the workqueue watchdog -- to the same helper. No functional change. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index b386a457c0381..513193be00a7d 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1604,6 +1604,12 @@ work_func_t wq_worker_last_func(struct task_struct *= task) return worker->last_func; } =20 +/* True if @pool is tied to a specific CPU, rather than an unbound pool. */ +static bool is_pool_cpu_specific(struct worker_pool *pool) +{ + return pool->cpu >=3D 0; +} + /** * wq_node_nr_active - Determine wq_node_nr_active to use * @wq: workqueue of interest @@ -2753,7 +2759,7 @@ static struct worker *alloc_worker(int node) =20 static cpumask_t *pool_allowed_cpus(struct worker_pool *pool) { - if (pool->cpu < 0 && pool->attrs->affn_strict) + if (!is_pool_cpu_specific(pool) && pool->attrs->affn_strict) return pool->attrs->__pod_cpumask; else return pool->attrs->cpumask; @@ -5121,7 +5127,7 @@ static void put_unbound_pool(struct worker_pool *pool) return; =20 /* sanity checks */ - if (WARN_ON(!(pool->cpu < 0)) || + if (WARN_ON(is_pool_cpu_specific(pool)) || WARN_ON(!list_empty(&pool->worklist))) return; =20 @@ -5273,7 +5279,7 @@ static void pwq_release_workfn(struct kthread_work *w= ork) mutex_unlock(&wq->mutex); } =20 - if (wq->flags & WQ_UNBOUND) { + if (!is_pool_cpu_specific(pool)) { mutex_lock(&wq_pool_mutex); put_unbound_pool(pool); mutex_unlock(&wq_pool_mutex); @@ -7949,7 +7955,7 @@ static void wq_watchdog_timer_fn(struct timer_list *u= nused) lockup_detected =3D true; stall_time =3D jiffies_to_msecs(now - pool_ts) / 1000; max_stall_time =3D max(max_stall_time, stall_time); - if (pool->cpu >=3D 0 && !(pool->flags & POOL_BH)) { + if (is_pool_cpu_specific(pool) && !(pool->flags & POOL_BH)) { pool->cpu_stall =3D true; cpu_pool_stall =3D true; } --=20 2.53.0-Meta From nobody Fri Oct 2 13:04:16 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 388E740683D for ; Fri, 31 Jul 2026 11:58:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499088; cv=none; b=Sjy3yR0oKSCD8IrOrD8MUNJbQ2Cnsas0X+yhEurNQlcEz6RoHRpwikw/dcjjPbiFAsDwM8ZcH0zNfl/g9IvWrZahQwmO5Qn7dFSrRjQi8D2WVjAGBKObdiLkqjAmBX09H0y/13YIc6JFbVkbOH26jz6xc0IPMdjUMncMIAVtQhw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499088; c=relaxed/simple; bh=+cmYs0OiKG42jBCyQx/PNot02QSRmI0WmfqsaiTeT20=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D3ulnloGpkaGVXuxINqIknvgEx7h/CK2IfKdA+7y8w0zXHfetQhUECN71ziLcCDoYP1wa++OvlTDAkvGjr5WTmarOge28sfTXuIUyRzH2mMn2fE6Qc79c7nyt0NVK3WRysRW5glNciY+bkhFkxdar9FCbHjLYz5V0nB7+3p19D8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=D7W6K2uf; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="D7W6K2uf" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=05lF33MZqn7vtw5qvL+tIYyGwExZlD48p9Gs4D2krxQ=; b=D7W6K2ufQFzrDLOcYC7TEbIrjx tpkm6zJBOwjherouoJBZOv/h8afDoXiEj6FH14/Jal7hv4LSquylkrFnxIM+YGz0qyGgRpesUwSer RV8oxLu87ctmehkxlwkAqdF/54uzbiYY3w0n7mLlN6ROBm3mkmPMcB9sDhaDxuTN1SStqANTOst/E V2FF2bgpj6tzCk0W70HnZbtblbBvQdrIwtPNvB9q7KWXUqAK7L+o6lM81/rt+7onjLmlhp94uEXjM Mip26kyxe7ATuW5nenf70Uzh5o+AC3AdEbXI/5duOqdvOEty6b7D1KnttxDlf5WhmzAUReRyihVae jsFIgfnw==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wplsL-009zd0-0c; Fri, 31 Jul 2026 11:58:05 +0000 From: Breno Leitao Date: Fri, 31 Jul 2026 04:57:36 -0700 Subject: [PATCH 4/6] workqueue: account nr_active by the backing pool Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260731-wq-pool-refactor-v1-4-8eaf71cdab5f@debian.org> References: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> In-Reply-To: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com, marco.crivellari@suse.com X-Mailer: b4 0.16-dev-d5d98 X-Developer-Signature: v=1; a=openpgp-sha256; l=2510; i=leitao@debian.org; h=from:subject:message-id; bh=+cmYs0OiKG42jBCyQx/PNot02QSRmI0WmfqsaiTeT20=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqbI28rb/kdfb+OUrIkA6Ynq+0aQ0t0F2XbX5AO gkY0hCGaryJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCamyNvAAKCRA1o5Of/Hh3 bbEYD/wJY9JXQV+VYXcQagQ8YNWYKakQ5BrCej2PKBe5P0O7BSTYJCK4AOZ3+truqLr1o0M42FZ tm3YM34zxXBFzgJvceozMSns1mXifqlVENOkea94mkS47abs+S2QbcBtGCLWP/cG5efDkEQkDJ2 Vcr9sTi9Z3CkBWw/8u/mxQa7kTK/GpHaIHt/dygQF9hl7SQysA7zAlnYFUZXPMMgJiE2N9w8gAs 8VoJWJ/XFhWGQljJYMuXXNawu6/2SyJ8/mpp5c96oCcmKOmU/yt/vPbG60XQtsA41OCS7rcIsZq z6RvP5Khh+rDPs7sQP9cusHIBMk6cUl8LFT93fY7w9xunqUo1Pbsxlh3iw2ZG517zZEsekeuz/z jjvYfEqT6gp6yQvyuw27c/PlFqo1vHSW7t1K7SunrRMR+ZmP8sp873DHbC9HAFrdTnpUJ/pGguQ 6/wKPVAojPq3GAeLk3ObV/Xa/qh9DL9pCYWW+6AIqRq4BdagEFw78GTl4ToGLYIxkqCi0GMhPnl 6sqv3VHAQerOgqcvocPVWWWrBqcPCfBVYFK/9XLMNW5FWWH4hRRCng/y0BJPIfGojyos7A2pc/O b/8oLUYjhdZIP3Xtb7/KoV+UIPjAbx2NIFV13bQ3bBcD5eiX5GPX+dEKMhCF7+uFEoxRLTWGp3W knf/WjhajhA45Ig== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao Use is_pool_cpu_specific() instead of wq_node_nr_active() to check for per workqueues. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 513193be00a7d..d17fdcaaf1685 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1782,13 +1782,16 @@ static bool pwq_tryinc_nr_active(struct pool_workqu= eue *pwq, bool fill) { struct workqueue_struct *wq =3D pwq->wq; struct worker_pool *pool =3D pwq->pool; - struct wq_node_nr_active *nna =3D wq_node_nr_active(wq, pool->node); + struct wq_node_nr_active *nna; bool obtained =3D false; =20 lockdep_assert_held(&pool->lock); =20 - if (!nna) { - /* BH or per-cpu workqueue, pwq->nr_active is sufficient */ + /* + * A concurrency-managed per-cpu pool accounts nr_active per pwq, so + * pwq->nr_active against wq->max_active is sufficient. + */ + if (is_pool_cpu_specific(pool)) { obtained =3D pwq->nr_active < READ_ONCE(wq->max_active); goto out; } @@ -1796,6 +1799,10 @@ static bool pwq_tryinc_nr_active(struct pool_workque= ue *pwq, bool fill) if (unlikely(pwq->plugged)) return false; =20 + nna =3D wq_node_nr_active(wq, pool->node); + if (WARN_ON_ONCE(!nna)) + return false; + /* * Unbound workqueue uses per-node shared nr_active $nna. If @pwq is * already waiting on $nna, pwq_dec_nr_active() will maintain the @@ -2013,7 +2020,7 @@ static void node_activate_pending_pwq(struct wq_node_= nr_active *nna, static void pwq_dec_nr_active(struct pool_workqueue *pwq) { struct worker_pool *pool =3D pwq->pool; - struct wq_node_nr_active *nna =3D wq_node_nr_active(pwq->wq, pool->node); + struct wq_node_nr_active *nna; =20 lockdep_assert_held(&pool->lock); =20 @@ -2024,14 +2031,18 @@ static void pwq_dec_nr_active(struct pool_workqueue= *pwq) pwq->nr_active--; =20 /* - * For a percpu workqueue, it's simple. Just need to kick the first + * A concurrency-managed per-cpu pool only needs to kick the first * inactive work item on @pwq itself. */ - if (!nna) { + if (is_pool_cpu_specific(pool)) { pwq_activate_first_inactive(pwq, false); return; } =20 + nna =3D wq_node_nr_active(pwq->wq, pool->node); + if (WARN_ON_ONCE(!nna)) + return; + /* * If @pwq is for an unbound workqueue, it's more complicated because * multiple pwqs and pools may be sharing the nr_active count. When a --=20 2.53.0-Meta From nobody Fri Oct 2 13:04:16 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 742BB412261 for ; Fri, 31 Jul 2026 11:58:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499091; cv=none; b=gkybr5DpCxB8bADlbP6fCHH9tvNMuY2DQROfAF93O1hkzJah1VTjKusH9TaKiIx5U9D01gNUZPfm3BLSp0GETbIXg74nLPMkGKGzm8aQG8crDY2v/OHGmT0bfNLZ9AnU4Bo4I0kLqKgdgZl067XjE31A1+3X/VNVEV3uC/92Jew= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499091; c=relaxed/simple; bh=Ysage/Ieve9S1NTuRRBk3vB5chjRRmeR1nUcMGMJy3E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=drpjuCO5422bHlpyv1cW++qxkZeeekk7hQQEeUeYtAhGkqs6yvbWp54X+InzpcLamjinowMWMv69a8Sm3yspFAPoaZTbNOSPl7g4UiR3SpdTOVG+j84gid5ZNgyhK5TEWvYItMCNqK5Fk47y3B/NjGv/bPWFQ54iN+bsFX/njAw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=sUcckFRW; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="sUcckFRW" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=2JV6uqXiLpEzoobLtRq2pI62muWcSY+g4Zhtc5RsR4E=; b=sUcckFRWGF4hXMNOsAt9fX7f4j 0S/UVvIDnlVtKY4WHh2iMrlvsFY+yXfa/gdSpD+OeVd5w6raFM4He1cNN77xxXC5BDW4OYLPFODfs RiNulWR/1B6hUtL/Za5b/l7G72axZTirO2BqGxvIerowGwwLeJ8lu7iH6s1eNK66Wki3iIWE4TA9I o8j5q6YEfn9assGOEnZXAqmF4dRsgaV5UsEAdLa4KxM+5cPKzAcny+fxLQULkIt3IDpNJz7b+A4sB tz9x1qWMO9kUHF6Pmoiu8wwyea5tclSD3w3x/kFLBMEB2fhKsidVCRjDRRYpGGT0xKeiY96du8D/O ABTWgSkQ==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wplsO-009zdB-1l; Fri, 31 Jul 2026 11:58:08 +0000 From: Breno Leitao Date: Fri, 31 Jul 2026 04:57:37 -0700 Subject: [PATCH 5/6] workqueue: add a per-cpu backend for unbound pwqs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260731-wq-pool-refactor-v1-5-8eaf71cdab5f@debian.org> References: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> In-Reply-To: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com, marco.crivellari@suse.com X-Mailer: b4 0.16-dev-d5d98 X-Developer-Signature: v=1; a=openpgp-sha256; l=3143; i=leitao@debian.org; h=from:subject:message-id; bh=Ysage/Ieve9S1NTuRRBk3vB5chjRRmeR1nUcMGMJy3E=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqbI28BAzyrP3KflRYnckzY4gVoAFHq2byG9Dqb V/mcONk9xyJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCamyNvAAKCRA1o5Of/Hh3 be7iD/9fJKiTIep33iXQqG+VmzfxT0BFxxhReU/hDWgL2+OEu463jZb7fmriSSorWfeYuUdHebA BvnIeK7oX/owuSqcIHGh/aSZ1Fl8I9T8PiQKsGme48MZaKt6OBjvaLSlRabsvTDhpEHXA8Cbhck Y14J+/ACNgSoFgJkU5f6c+JymVEF0F9eJ7Ox70DG4OEWmYA2SWghwxdH9F03KMu2sJzL3y8XZV2 Bgm2VvbujijtNv+ZE3J/oVm42fTBrfo5ilG0zbsRwdxW0wfZWnZz9Rlj01cCbPnXCUlX8FXBS/2 2cRjtLU3R7+cAKiBaB6C5whSnjeN4fIDMGHxWAijrWguQ65cxeC0KE7kYi9Q2cU6REQ+H9Nx5o5 hbPYqt0gT8kH3+RHlIoWidxsyF2HboMo1n/2qZeJjAJPmx6uhoEF+csQ2n1dDgIBQtKwINVuUiy yDtSj2hHkFs75RotKXO6Miyhum+AZpWb4O3PnUcXvs0Ap9KXJUopPBpBhPZ1EglQ5zqKTAF01iO bRkc2E0ITxrR2lPRvn5aQ9BV3ijbies+3la9FlNzKveV9fi+7BDXNDtSXdQUw24l0cew4FEopCY LmEw4iQptu8TB6eXloB++HpdnC2QkCAlkvd448LZFYX85Rt0VIkwyY8nzaIm6Jnw28V3g+geiO6 B/15s5cURwl4Grg== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao Add alloc_percpu_pwq(), which binds a pwq to get_percpu_pool(wq, cpu), and an internal __WQ_PERCPU_POOLS flag. unbound_wq_update_pwq() installs such a pwq per CPU when the flag is set, reusing the existing install/drain path. Pool release and nr_active are already keyed on the backing pool, so a per-cpu-backed pwq is torn down and throttled correctly. PS: We can do this using if/else for per cpu/unbound as well, instead of this labels:, would it be better? Signed-off-by: Breno Leitao --- include/linux/workqueue.h | 1 + kernel/workqueue.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index a283766a192aa..5bbbed94d2fa6 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -410,6 +410,7 @@ enum wq_flags { __WQ_ORDERED =3D 1 << 17, /* internal: workqueue is ordered */ __WQ_LEGACY =3D 1 << 18, /* internal: create*_workqueue() */ __WQ_DEPRECATED =3D 1 << 19, /* internal: workqueue is deprecated */ + __WQ_PERCPU_POOLS =3D 1 << 20, /* internal: back unbound pwqs with percpu= pools */ =20 /* BH wq only allows the following flags */ __WQ_BH_ALLOWS =3D WQ_BH | WQ_HIGHPRI | WQ_PERCPU, diff --git a/kernel/workqueue.c b/kernel/workqueue.c index d17fdcaaf1685..df4fc9ccb7b22 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5404,6 +5404,27 @@ static struct pool_workqueue *alloc_unbound_pwq(stru= ct workqueue_struct *wq, return pwq; } =20 +/* + * Create a pwq backing @wq on @cpu with the static per-cpu pool instead o= f a + * dedicated unbound pool. Used by the unbound pwq machinery for a workque= ue + * that requests the per-cpu backend. + */ +static struct pool_workqueue *alloc_percpu_pwq(struct workqueue_struct *wq, + int cpu) +{ + struct worker_pool *pool =3D get_percpu_pool(wq, cpu); + struct pool_workqueue *pwq; + + lockdep_assert_held(&wq_pool_mutex); + + pwq =3D kmem_cache_alloc_node(pwq_cache, GFP_KERNEL, pool->node); + if (!pwq) + return NULL; + + init_pwq(pwq, wq, pool); + return pwq; +} + /** * wq_calc_pod_cpumask - calculate a wq_attrs' cpumask for a pod * @attrs: the wq_attrs of the default pwq of the target workqueue @@ -5643,6 +5664,17 @@ static void unbound_wq_update_pwq(struct workqueue_s= truct *wq, int cpu) if (!(wq->flags & WQ_UNBOUND) || wq->unbound_attrs->ordered) return; =20 + if (wq->flags & __WQ_PERCPU_POOLS) { + /* nothing to do if @cpu is already backed by its per-cpu pool */ + if (is_pool_cpu_specific(unbound_pwq(wq, cpu)->pool)) + return; + + pwq =3D alloc_percpu_pwq(wq, cpu); + if (!pwq) + goto use_dfl_pwq; + goto install; + } + /* * We don't wanna alloc/free wq_attrs for each wq for each CPU. * Let's use a preallocated one. The following buf is protected by @@ -5666,6 +5698,7 @@ static void unbound_wq_update_pwq(struct workqueue_st= ruct *wq, int cpu) goto use_dfl_pwq; } =20 +install: /* Install the new pwq. */ mutex_lock(&wq->mutex); old_pwq =3D install_unbound_pwq(wq, cpu, pwq); --=20 2.53.0-Meta From nobody Fri Oct 2 13:04:16 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98E0341B8CA for ; Fri, 31 Jul 2026 11:58:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499094; cv=none; b=S5sDn+wv7IS+G935zWn+Vm223+qGTEFeKULXmlDbt7OxrGRgsXMB+cVhuu6FAWXiP2UG43/rC+iRf5dkBSelpZby6n3h89Nd7aRo7ds0BofRgyVZYvX1dnxmeu2iLD8IRpxJy1DgZrs/PC1a5ixwBNe2KYOJ76o7olaShBWxI24= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499094; c=relaxed/simple; bh=YUfbfimXVo3KDOhpG8Vj1nAnI5XwFIu21QDfZnPF4Lg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tV6mMl7M35wRDjQh76Mpikym0HF4XXBM0fVvT7E3JH6H+gU6jDwv2XXXP2hJElIpVQVW9reCCDrl1a/SwWl1njno8xkLP3GyfAwfUyrAFEysZXq7M1rPeBuPZ8S4Btl/WydrPEzJTwpkq5cWu5dnR1GJ2J6OVQG0llwX3KPBquw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=R5IY4UEW; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="R5IY4UEW" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=DhE0Cq7ezgPKterCbzngN7l9OBBsb9l1Zmy9ANn3EKs=; b=R5IY4UEWNffmlf61YGePT6okzg l/7xVojqBCYpdZecOu4OdWOp/TG3djjE2EuSS5b4C3b6JH87+uFhyLhQYTE3s0eA1HwLEtVTpw1B2 phg9UtU8/oSdqUe32NgLsQ8qIe53WDDkNx71UxXTrtwoX+pHpVbWGifsYP5B+r9IEQ70MmKW0LhfS EYfCFsnPwMwKhodWBsb+uk+R6v9bcfTc7Sn1NU7uu2Wqmn8K30bVXbn28POfPCKypb84CVc8DmycY Q8LxzSLorjpBguxJLG9bN2lhYN/QgefvzwvXNL8irZWIHdEKftUNeTpGSWrMjngnS8iN6ambWN0Dh PxiDt1UQ==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wplsR-009zdK-2K; Fri, 31 Jul 2026 11:58:11 +0000 From: Breno Leitao Date: Fri, 31 Jul 2026 04:57:38 -0700 Subject: [PATCH 6/6] workqueue: install per-cpu pwqs at creation for __WQ_PERCPU_POOLS Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260731-wq-pool-refactor-v1-6-8eaf71cdab5f@debian.org> References: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> In-Reply-To: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com, marco.crivellari@suse.com X-Mailer: b4 0.16-dev-d5d98 X-Developer-Signature: v=1; a=openpgp-sha256; l=974; i=leitao@debian.org; h=from:subject:message-id; bh=YUfbfimXVo3KDOhpG8Vj1nAnI5XwFIu21QDfZnPF4Lg=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqbI28+PonBHfBpq8QZLXIDXB4JTPfFYXBPfTcs XFxAHUJqROJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCamyNvAAKCRA1o5Of/Hh3 bc4QD/0ZI+3IU9soXazb31CETKciVd4Fbq2FLEm0Pp1lJDnSd11eyGIYmZiCUpNJNxnCr3R4zWe 6/kRdDqHJN7ly7Qelcrua8L4+qx6/iJd5KJbfft5a9iWcxlNzJ2n4jFLvAc87oPyffWFvC5V9LU y5ifnIsFKbYzKiuvMNhOx5ROw51AwWd6HwZ9R4HrahhLQA4THW7Gjv1FTST0x56pcey/SvytPwr f2kgk1fB2jyTjkgSMbG+3ZDcIOLuZ3thHvRk9V/6DeSo99JkvqAtsiK2qOkD0n43zQIwXwsg2FV AdGrY/95TMLVlwDRJ/kOsEzWc68mjAGOvPg3PlkyySiI/9QxehrBH0itl1B5mdjPv0qPP3N5Sca V6Zugylyshrq0lSqeY50RulSYF9Dgp62DXG0YhD1+mag589bQMPMfcm2YWjZlMeZA9QtVC/uP43 xqQHrB0fpEe3NuMtH0jIOTkDfxobw3eohYahqL6LPRjFJnVBPVx6MT4FJolKjOODHheKppmpzhi 5+IW3ZdtpP0hgC5yqTznYHxY3Om/eW3s3BGJVu4bPbQPvNFBajn7XRGgZ0ElS0IP1lwNMxNhLq5 q0wG16Bo/6+K1i9WhMQcoAATTnyymP6a+KL2YKzwadeLr80EAkPyVvdiqdbskzZXwJXUOrZnCXU jljC5hkEa/DgoLQ== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao alloc_and_link_pwqs() brings up an unbound workqueue with dedicated unbound pools. For a workqueue that requests the per-cpu backend, convert those pwqs to per-cpu-backed ones right after creation by running the per-CPU update, so the backend is in place before the workqueue is used rather than only after the first CPU hotplug event. No workqueue sets __WQ_PERCPU_POOLS yet, so there is no functional change. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index df4fc9ccb7b22..cd3d0d54dfddc 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5766,6 +5766,12 @@ static int alloc_and_link_pwqs(struct workqueue_stru= ct *wq) =20 if (ret) goto enomem; + + if (wq->flags & __WQ_PERCPU_POOLS) { + for_each_possible_cpu(cpu) + unbound_wq_update_pwq(wq, cpu); + } + return 0; =20 enomem: --=20 2.53.0-Meta