From nobody Thu Oct 2 09:21:02 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC80630216D; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190674; cv=none; b=itai4g0MUG8qUk3eX/lfxrmfOJzp+qc+9eEroSihuimimnpjRaSoFZR9d+e3VkE/cy4WqKB3dfUgqR37sGRGCjxrEZkBKDHeAp6RNVOU/ebvHB6fVrA2uP1MVYN2xOPTfE18gAdyf4WKmfghzE4ObXzd6r12DDt+FhG0wurgewE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190674; c=relaxed/simple; bh=tt2fnvxyKHuxMWduq2G5iWtwAxgOUA1tzVmzIKN+kzc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HfLVCandQuN4vIoufp8TWjyeS77+l26EATyGucuOHUTb1MVveqSF5u1eZ/VUkHGibQ13PW5AlUydSCv0gN3ZCWp7KYzNXV/x985JfOwpuvvuXhXn4dGX0vMZfD05MmJDkQuhomOTOWKWbEYMP/sPgA14mSnCwehi4Ud5/l0wS6U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UIPylFx9; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UIPylFx9" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8BCEBC4CEE7; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1758190674; bh=tt2fnvxyKHuxMWduq2G5iWtwAxgOUA1tzVmzIKN+kzc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UIPylFx9ChvcXM7wjTxfr4qUJQo/k9aqvgOr3xgTCMA4EYzaNxzibcmUyUz6oGiso Ltj5RIOX83KZpjI8IckALIVdER+Swnyzbtd3cXwFAMbIvaWAu8M8Mp33JILJKdqel5 Of3UD3LnODF3O2zLZu2V6hq7F39/+f7KHx835Yrr08G6kOTUf46UvVkKtayTB7C9HM WdLyraTGZr5/ku1efz2XxrmFjSj//O5lm233tT57feIsIM+ZkgKvTx/rP+Ubir++4H S1+eNngmWLx4oGogohgn9ImjJYD3A/wzbGcUNpSTkUhCqFVne2A8/Vx8ECmupkZKW+ f0MC9s+ZL8CaQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 116BFCE0B32; Thu, 18 Sep 2025 03:17:54 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH v2 1/5] rcu: Document that rcu_barrier() hurries lazy callbacks Date: Thu, 18 Sep 2025 03:17:48 -0700 Message-Id: <20250918101752.2592512-1-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> References: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds to the rcu_barrier() kerneldoc header stating that this function hurries lazy callbacks and that it does not normally result in additional RCU grace periods. Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 8eff357b0436be..1291e0761d70ab 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3800,6 +3800,11 @@ static void rcu_barrier_handler(void *cpu_in) * to complete. For example, if there are no RCU callbacks queued anywhere * in the system, then rcu_barrier() is within its rights to return * immediately, without waiting for anything, much less an RCU grace perio= d. + * In fact, rcu_barrier() will normally not result in any RCU grace periods + * beyond those that were already destined to be executed. + * + * In kernels built with CONFIG_RCU_LAZY=3Dy, this function also hurries a= ll + * pending lazy RCU callbacks. */ void rcu_barrier(void) { --=20 2.40.1 From nobody Thu Oct 2 09:21:02 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E371E30215C; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190675; cv=none; b=sS8Wo8Vpr389XZeIx4FevNC2FvbkhJiwUgpGZ4ntWB8tF3p6NtTtx0NRN6QeDW5oF4vbCNqNeuOVs3drWx0g8qMnZD4yAX0hv0KcPuMMpHzRdqXG32Jatp0IQKEUyUWzc10bUx+fw2MVczH2nyt7IkzW/a+/RWPsbm1HcM47k0M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190675; c=relaxed/simple; bh=rq+Ub4PUmYMUmVVmwp0dloL83Muh245hgWWFkowVhoc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=S0+jl+YGVW+gOmQY728qdxPGYZvsdoYkdTNwNkl/e7RLo0wbEATxLdOnX/Wl6xxSMd+PNNC+YsQSWRRZRuvqa7Jje4hoKe0Q2UTycudP00W6Kko0IFHDSDhwaOuAqy0Cr87b9uC1e+M7BgcOKjzBe2C6NDzu6ftT9DKjj0S+g/4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rJISAZut; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rJISAZut" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90283C4CEF7; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1758190674; bh=rq+Ub4PUmYMUmVVmwp0dloL83Muh245hgWWFkowVhoc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rJISAZutESJCuvWl0KKfu9OQ2JV04oaraSTyjAud3PkdnU7nqKzW/8kGxl7YmakUS dPEFe+Ajj+5ywh7ZSSzpNRupqEiy63T+9/5Tlu4EGVq6xHJ1PnfOy05bpBWtMBfTbP 4C5jvxT6rGBuDSz85m43OQ8MnGeCIxtfePR/Cszth/BYJP1utxfyYr737Yp5n51qEG p3Lnl/0kbC9ejhNa4E6vz2igf6rs9XTr3wUFspvqhaE3XygJpebVgNASshjV1ea0oJ WrAnKYKSsdgfqjzv96jcyq1PKhXfHS6287P0+hINnQQ79UDkXRwdHGGNkqsOX/dYwC Tj+IxYWz1YgLA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 14972CE0D66; Thu, 18 Sep 2025 03:17:54 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Zqiang , "Paul E . McKenney" Subject: [PATCH v2 2/5] rcu: Remove local_irq_save/restore() in rcu_preempt_deferred_qs_handler() Date: Thu, 18 Sep 2025 03:17:49 -0700 Message-Id: <20250918101752.2592512-2-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> References: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zqiang The per-CPU rcu_data structure's ->defer_qs_iw field is initialized by IRQ_WORK_INIT_HARD(), which means that the subsequent invocation of rcu_preempt_deferred_qs_handler() will always be executed with interrupts disabled. This commit therefore removes the local_irq_save/restore() operations from rcu_preempt_deferred_qs_handler() and adds a call to lockdep_assert_irqs_disabled() in order to enable lockdep to diagnose mistaken invocations of this function from interrupts-enabled code. Signed-off-by: Zqiang Signed-off-by: Paul E. McKenney --- kernel/rcu/tree_plugin.h | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 4cd170b2d6551d..d85763336b3c0f 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -626,11 +626,10 @@ notrace void rcu_preempt_deferred_qs(struct task_stru= ct *t) */ static void rcu_preempt_deferred_qs_handler(struct irq_work *iwp) { - unsigned long flags; struct rcu_data *rdp; =20 + lockdep_assert_irqs_disabled(); rdp =3D container_of(iwp, struct rcu_data, defer_qs_iw); - local_irq_save(flags); =20 /* * If the IRQ work handler happens to run in the middle of RCU read-side @@ -647,8 +646,6 @@ static void rcu_preempt_deferred_qs_handler(struct irq_= work *iwp) */ if (rcu_preempt_depth() > 0) WRITE_ONCE(rdp->defer_qs_iw_pending, DEFER_QS_IDLE); - - local_irq_restore(flags); } =20 /* --=20 2.40.1 From nobody Thu Oct 2 09:21:02 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E369B2F7AAD; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190675; cv=none; b=dBxB9ZO1bgKfusSVabsJOMkk2MTA1JFFzPyH0RbdZBZjTRVXFbbfwXSpFIldVW0x5+h0zkwUOHSr6OPXUyGjjxkD2vbmrHaTtJawa7e3MxHlp72JXptA/1k3aqLK77+RQrFIh11PtSmAHmkc2EDmdXocRj967BJ0+ywtjNtb4Mo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190675; c=relaxed/simple; bh=TClFAeQiiBqNMqlX5lL/MO+NhBy3SSx1O4T3SfYSpPg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OLDhGe26sdzrEfVOu3AftnGDKzakK/1EYvLasXpdDBDohsWJhPs02ipTLwLbCBY/5nsxpl+RC/xvx2FOH5TBIPrhgi5AN799vq2zyKljbWKZH6xBHIqGG7nR+Z75IAuBEELp5BFSa1sgQ2RH8lSGgrmrwQxncpj9DsaKINIhNMM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=q3rmf1DL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="q3rmf1DL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 97E6DC4CEFA; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1758190674; bh=TClFAeQiiBqNMqlX5lL/MO+NhBy3SSx1O4T3SfYSpPg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=q3rmf1DLlZycaJ92Qtkp86ioEzEb+1T/ca/6MReH8gkXFVJUdFici1LGxWuTUUhZz HKqSor9rTPDoL9JkVNZuTnWkIlwi5fOXMbZb1QMTmCFZXhTnnPq33C2EVWVHpbP8O9 z5XM+BHHDpAE9LyqZnBaITvxbICUI66lCtgvPNUPIJN8v+moZAMGxXQpKmWHm1+yk6 0V/O6VkGen2nKbhQjw0JOvmVk5iMTd6bxkQ5Dxng2MPcdhBbqQYXUPAzrGM0jSoWwq WlCFvuMrDDLf4UeYQbEB0hhkblr9PCSi/s/WdWUmIBdJK5ZJTKTpt86AfeG7NO9PGF Jgo2J+TmnFm3Q== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 176CACE0EFF; Thu, 18 Sep 2025 03:17:54 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Andy Shevchenko , Simona Vetter , "Paul E. McKenney" , Neeraj Upadhyay Subject: [PATCH v2 3/5] rculist: move list_for_each_rcu() to where it belongs Date: Thu, 18 Sep 2025 03:17:50 -0700 Message-Id: <20250918101752.2592512-3-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> References: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Andy Shevchenko The list_for_each_rcu() relies on the rcu_dereference() API which is not provided by the list.h. At the same time list.h is a low-level basic header that must not have dependencies like RCU, besides the fact of the potential circular dependencies in some cases. With all that said, move RCU related API to the rculist.h where it belongs. Signed-off-by: Andy Shevchenko Reviewed-by: Simona Vetter Reviewed-by: "Paul E. McKenney" Signed-off-by: Neeraj Upadhyay (AMD) Signed-off-by: "Paul E. McKenney" --- include/linux/list.h | 10 ---------- include/linux/rculist.h | 10 ++++++++++ kernel/cgroup/dmem.c | 1 + 3 files changed, 11 insertions(+), 10 deletions(-) diff --git a/include/linux/list.h b/include/linux/list.h index e7e28afd28f8ee..e7bdad9b861827 100644 --- a/include/linux/list.h +++ b/include/linux/list.h @@ -686,16 +686,6 @@ static inline void list_splice_tail_init(struct list_h= ead *list, #define list_for_each(pos, head) \ for (pos =3D (head)->next; !list_is_head(pos, (head)); pos =3D pos->next) =20 -/** - * list_for_each_rcu - Iterate over a list in an RCU-safe fashion - * @pos: the &struct list_head to use as a loop cursor. - * @head: the head for your list. - */ -#define list_for_each_rcu(pos, head) \ - for (pos =3D rcu_dereference((head)->next); \ - !list_is_head(pos, (head)); \ - pos =3D rcu_dereference(pos->next)) - /** * list_for_each_continue - continue iteration over a list * @pos: the &struct list_head to use as a loop cursor. diff --git a/include/linux/rculist.h b/include/linux/rculist.h index 1b11926ddd4710..2abba7552605c5 100644 --- a/include/linux/rculist.h +++ b/include/linux/rculist.h @@ -42,6 +42,16 @@ static inline void INIT_LIST_HEAD_RCU(struct list_head *= list) */ #define list_bidir_prev_rcu(list) (*((struct list_head __rcu **)(&(list)->= prev))) =20 +/** + * list_for_each_rcu - Iterate over a list in an RCU-safe fashion + * @pos: the &struct list_head to use as a loop cursor. + * @head: the head for your list. + */ +#define list_for_each_rcu(pos, head) \ + for (pos =3D rcu_dereference((head)->next); \ + !list_is_head(pos, (head)); \ + pos =3D rcu_dereference(pos->next)) + /** * list_tail_rcu - returns the prev pointer of the head of the list * @head: the head of the list diff --git a/kernel/cgroup/dmem.c b/kernel/cgroup/dmem.c index 10b63433f05737..e12b946278b6c6 100644 --- a/kernel/cgroup/dmem.c +++ b/kernel/cgroup/dmem.c @@ -14,6 +14,7 @@ #include #include #include +#include #include =20 struct dmem_cgroup_region { --=20 2.40.1 From nobody Thu Oct 2 09:21:02 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3612302174; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190675; cv=none; b=iIFzi/RzSDWbfbdcmCEVTQ3/EPuMD3zgXAN/T8wu0XjlQgWvQ5bMxNHSBqJHI+BY8109swcmWOIiq+H0tvSNcMASv2eTt2J3fpezUaFggjh+Jl9TWvZ+20cBInsIN//02PC6Gxo2psZ8RLb4LjrVGReXW+5UJ2IlxXxF1WZFwEQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190675; c=relaxed/simple; bh=OmlAZwLbOfV/U6YckvaKFi8EfINIzzau42SR+69WELM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sLbFY67eHlTGP+aRMs0kDnkbwRIgQSUslw4Dmg/awHxR24FLr+zMpvE/VN+hiiVF7+c0itUzcqHgt1qip9+LVbUAYHTodlJ4CGURf0wrq7CuHSfMzLwpAQOqIfgrPTlkHQVTaR2r+d5LF6n9fU4kJnvIhj+OaJmKAPmyHVsqG9c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QN5ra0Bn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QN5ra0Bn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A5485C4CEEB; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1758190674; bh=OmlAZwLbOfV/U6YckvaKFi8EfINIzzau42SR+69WELM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QN5ra0BnO82Cl55DDuat1rXGYCdh1ifE0O05l/3GeZyqxp58abTCS9wkzKJfcrTjm uh7DBao3ybasOXQYhCX6b6hGfiZ6X+3V3o0abCff+B8HvfYuDDbaj1aPj+GwMNGCEf 9gZ93KS/grhpzY1KPRFpDDUZ3I1StGBhXnB6o4IFl86o6qxMcZh0DSBnkakl0qgbXn RSB/LQ3XJYIO6eM6khfbj1s0fW5dj7e0PrRWbgP9lYcz2KYXC895XLfsREF1pBFgXl 7g9VtTcNoAvKj6W/L93RwI9g8R9N6fSZ6zg1yrn2JyTb9qTiGdyEIyXLxAwCYrht9X vF4VLpU/sBptw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 1A52CCE0F82; Thu, 18 Sep 2025 03:17:54 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Marco Crivellari , Tejun Heo , "Paul E . McKenney" Subject: [PATCH v2 4/5] rcu: replace use of system_wq with system_percpu_wq Date: Thu, 18 Sep 2025 03:17:51 -0700 Message-Id: <20250918101752.2592512-4-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> References: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Marco Crivellari Currently if a user enqueue a work item using schedule_delayed_work() the used wq is "system_wq" (per-cpu wq) while queue_delayed_work() use WORK_CPU_UNBOUND (used when a cpu is not specified). The same applies to schedule_work() that is using system_wq and queue_work(), that makes use again of WORK_CPU_UNBOUND. This lack of consistentcy cannot be addressed without refactoring the API. system_wq is a per-CPU worqueue, yet nothing in its name tells about that CPU affinity constraint, which is very often not required by users. Make it clear by adding a system_percpu_wq. queue_work() / queue_delayed_work() mod_delayed_work() will now use the new per-cpu wq: whether the user still stick on the old name a warn will be printed along a wq redirect to the new one. This patch add the new system_percpu_wq except for mm, fs and net subsystem, whom are handled in separated patches. The old wq will be kept for a few release cylces. Suggested-by: Tejun Heo Signed-off-by: Marco Crivellari Signed-off-by: Paul E. McKenney --- kernel/rcu/tasks.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index f92443561d365a..2dc044fd126eb0 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -553,13 +553,13 @@ static void rcu_tasks_invoke_cbs(struct rcu_tasks *rt= p, struct rcu_tasks_percpu rtpcp_next =3D rtp->rtpcp_array[index]; if (rtpcp_next->cpu < smp_load_acquire(&rtp->percpu_dequeue_lim)) { cpuwq =3D rcu_cpu_beenfullyonline(rtpcp_next->cpu) ? rtpcp_next->cpu : = WORK_CPU_UNBOUND; - queue_work_on(cpuwq, system_wq, &rtpcp_next->rtp_work); + queue_work_on(cpuwq, system_percpu_wq, &rtpcp_next->rtp_work); index++; if (index < num_possible_cpus()) { rtpcp_next =3D rtp->rtpcp_array[index]; if (rtpcp_next->cpu < smp_load_acquire(&rtp->percpu_dequeue_lim)) { cpuwq =3D rcu_cpu_beenfullyonline(rtpcp_next->cpu) ? rtpcp_next->cpu = : WORK_CPU_UNBOUND; - queue_work_on(cpuwq, system_wq, &rtpcp_next->rtp_work); + queue_work_on(cpuwq, system_percpu_wq, &rtpcp_next->rtp_work); } } } --=20 2.40.1 From nobody Thu Oct 2 09:21:02 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3599302152; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190675; cv=none; b=oKVuLdKViIBpyAaYN780fJg3zp5TecwX74yVDqq4EjbYzjmfnIfhTY7meV2xaz2VAasvvLyrBAPDvSGd7yAjdJfoVqFTUyLNw2mRZp1pyHlx+YQF0mSQ6EqeYVOVsyvebAhvRYOIETiu6Qa9AlZ2pPKLIjcmvNXbtbZKeLjyZNY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758190675; c=relaxed/simple; bh=6xDiGB8aFzb5XzIZSlcQKXA0X1480AudJy/Ai6h9dH4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=TW405nnqm7M88jR+kF1/dUavzFKqWikSWfyPHcyIsdwRKL6E1Xbwe0NZQqHGm0al1ucmRz0o4ernmJSFSLAxeYbeCDt5F/5ZMdOVAUno8mGF+wJu1SEwn6oRWp4/v3/b05TDJa8NWNQxtoyc1jU7NNy8UjL+DkGylTQqG9jGiBM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ntAvZCfS; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ntAvZCfS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id ABA14C4CEFE; Thu, 18 Sep 2025 10:17:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1758190674; bh=6xDiGB8aFzb5XzIZSlcQKXA0X1480AudJy/Ai6h9dH4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ntAvZCfSC1oWndqNEcqf8wbc3Q8Vft39a/xYjoJakKVz/t7G4Iaks5RCTrjP2o+IF GepSKHLkQVMSwP1MyQT+XZiy6TrJEmUa7i7apWiinbzByN82wTgjf7zvZeX86qHGxc SVeldUxfrKIA6C2x2masO2A04YSlURrJe2UVjv65JEAhLoWI3z8zMDMYrd0LPe2jgJ 2i2O41laAaX3W6+D8BV3f5TApmJ55sbSoOV57Ttc9xpFdNfJHJYC0EaVwlW+qPFgFe NrgjTXAu+Rkj7xbh5ng4w4dtSt6xNl+pRM5VAV/MKASm8u6bu3Q0JOVfOO59DxfooY EUcKeUDcR9Vig== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 1CD7DCE10B6; Thu, 18 Sep 2025 03:17:54 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Marco Crivellari , Tejun Heo , "Paul E . McKenney" Subject: [PATCH v2 5/5] rcu: WQ_PERCPU added to alloc_workqueue users Date: Thu, 18 Sep 2025 03:17:52 -0700 Message-Id: <20250918101752.2592512-5-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> References: <3773a6e3-8ec8-48c5-9277-264cd8ccbb10@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Marco Crivellari Currently if a user enqueue a work item using schedule_delayed_work() the used wq is "system_wq" (per-cpu wq) while queue_delayed_work() use WORK_CPU_UNBOUND (used when a cpu is not specified). The same applies to schedule_work() that is using system_wq and queue_work(), that makes use again of WORK_CPU_UNBOUND. This lack of consistentcy cannot be addressed without refactoring the API. alloc_workqueue() treats all queues as per-CPU by default, while unbound workqueues must opt-in via WQ_UNBOUND. This default is suboptimal: most workloads benefit from unbound queues, allowing the scheduler to place worker threads where they=E2=80=99re needed= and reducing noise when CPUs are isolated. This default is suboptimal: most workloads benefit from unbound queues, allowing the scheduler to place worker threads where they=E2=80=99re needed= and reducing noise when CPUs are isolated. This patch adds a new WQ_PERCPU flag to explicitly request the use of the per-CPU behavior. Both flags coexist for one release cycle to allow callers to transition their calls. Once migration is complete, WQ_UNBOUND can be removed and unbound will become the implicit default. With the introduction of the WQ_PERCPU flag (equivalent to !WQ_UNBOUND), any alloc_workqueue() caller that doesn=E2=80=99t explicitly specify WQ_UNB= OUND must now use WQ_PERCPU. All existing users have been updated accordingly. Suggested-by: Tejun Heo Signed-off-by: Marco Crivellari Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 1291e0761d70ab..c51c4a0af6aa5e 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4890,10 +4890,10 @@ void __init rcu_init(void) rcutree_online_cpu(cpu); =20 /* Create workqueue for Tree SRCU and for expedited GPs. */ - rcu_gp_wq =3D alloc_workqueue("rcu_gp", WQ_MEM_RECLAIM, 0); + rcu_gp_wq =3D alloc_workqueue("rcu_gp", WQ_MEM_RECLAIM | WQ_PERCPU, 0); WARN_ON(!rcu_gp_wq); =20 - sync_wq =3D alloc_workqueue("sync_wq", WQ_MEM_RECLAIM, 0); + sync_wq =3D alloc_workqueue("sync_wq", WQ_MEM_RECLAIM | WQ_PERCPU, 0); WARN_ON(!sync_wq); =20 /* Respect if explicitly disabled via a boot parameter. */ --=20 2.40.1