From nobody Sat Oct 4 12:46:55 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BE93482EB; Sat, 16 Aug 2025 00:01:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755302511; cv=none; b=h+OKqgdtCZb+sILslIHYwUmErcITSWoI54QKZLCwXnOFq5qP0tv7Ymy7SWUN5yijA6O7Wtks0SLeojmY8/zsNLMO8Mvl9FlYfHD6s255oN2MH9bdYu2wxXcUyKuRL0hK52CFgHC+0S+9SGl9LIE3sZZoH72KDNcM/oEvpnc+ejQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755302511; c=relaxed/simple; bh=tt2fnvxyKHuxMWduq2G5iWtwAxgOUA1tzVmzIKN+kzc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QYbYmhhxXWRqlMM7djLCcFm3sCB/I/yOUPGMHAjeRgfcw9pSw7rf4skk5BFnslBMBulO0TugS0uCk/DT33DVHMDuhe3KcQvEf4dPJs4uPaNoSGIwKsfzrXC4rfYuOdrviGPeXh9E3jE6g5T2lDguV/+ckCDQw7bDfPg54h3tPo8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RfVvMfOG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RfVvMfOG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 00C22C4CEEB; Sat, 16 Aug 2025 00:01:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755302511; bh=tt2fnvxyKHuxMWduq2G5iWtwAxgOUA1tzVmzIKN+kzc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RfVvMfOG0b1QlhMLnLmblX46MZolGzZjvIs3TBKOhafZopSS0PhOLo13QQXH+vwf6 LV76Q3gdo2WypaqEUB4777jVJV2WXpaFcdf2/e6DYEyx63kv35q7OKgxo1L2aJ8P2r Grz2OFYCTC7QMt6oxr4LPRTSP67Zme20HH0HmFF1fhO/81KwGndvGKuKtMNG6HHgGg eSQBIoNtQL14j+3+nPHjR9OqpnEIa+qR8a0uLdoYHmggQtkbboKywiDuH8WpznnWth f77u9gyEwkVISoZJDwK/eDJ82g12NBxsCob6Yzf9cHwQMWfbuQjRr1VfYr07UD3Yph vzJvhbLHS4Y6Q== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 4D80BCE0B2D; Fri, 15 Aug 2025 17:01:50 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH 1/3] rcu: Document that rcu_barrier() hurries lazy callbacks Date: Fri, 15 Aug 2025 17:01:47 -0700 Message-Id: <20250816000149.2622447-1-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds to the rcu_barrier() kerneldoc header stating that this function hurries lazy callbacks and that it does not normally result in additional RCU grace periods. Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 8eff357b0436be..1291e0761d70ab 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3800,6 +3800,11 @@ static void rcu_barrier_handler(void *cpu_in) * to complete. For example, if there are no RCU callbacks queued anywhere * in the system, then rcu_barrier() is within its rights to return * immediately, without waiting for anything, much less an RCU grace perio= d. + * In fact, rcu_barrier() will normally not result in any RCU grace periods + * beyond those that were already destined to be executed. + * + * In kernels built with CONFIG_RCU_LAZY=3Dy, this function also hurries a= ll + * pending lazy RCU callbacks. */ void rcu_barrier(void) { --=20 2.40.1 From nobody Sat Oct 4 12:46:55 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C958524F; Sat, 16 Aug 2025 00:01:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755302511; cv=none; b=OauXevomas49Ua5V3tsrHHL/TTNo3vp3BkDz7yQ+SWWKophNTAghaI4SanFkxSP8C9vOjSVmkhYBW4vsjmRLc3eLAtOCJDgYqWxiAtfh3lfyBXWknJvCkmX56yQlmwqz3Kiy1CbZxraW1dH8FTY2E98ATPMltFbIlhYeC04laoE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755302511; c=relaxed/simple; bh=rq+Ub4PUmYMUmVVmwp0dloL83Muh245hgWWFkowVhoc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ddNiplkY8JzkbppRKzypeJUzMc5eVGgW0DLmXP0AWcx5LaV6Nr9HvNxua83nBMP9PpiObNI57r67ZrpmeqxwKSxmIsmp0glacIwGJDhXMy9A7Bydv9oYT0MUPAhrM82zPGo52qIgjoq9tIz6mix+3pCKO/UW/9yLYYy9PCB+Tuo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jQ1BjZpu; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jQ1BjZpu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1EB61C4CEF4; Sat, 16 Aug 2025 00:01:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755302511; bh=rq+Ub4PUmYMUmVVmwp0dloL83Muh245hgWWFkowVhoc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jQ1BjZpur1oDlXcx+nf9J1Qt1ISdObHQ/YL6VAC3WGp56UtVLzLefibMBC6G+67uK SJxQGV8NtYOxAbGlhN7zROHkTRDV2m5O9N056JQRZrkhNkazAZmYB02vHYL1udz6Wu XF/X/aFpN/N559lymsW5RedGoOLqLwWJa2vD44TA9QhrPhQad31Hb1iirNuWdmG2PF uPZhgmkGW0lScxvc3T0C612hTHCS5z1E2oAQnS/6EPpHpPb/Su5N6RoLiZBp4LM5i6 odEUCyikyO8sqnSoYotzraFCJYFOV6akVHlOTaiyYareNlCaGpnxuAhoCzRwK3+AzS qnHbm/9BdcUuA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 5064BCE0B30; Fri, 15 Aug 2025 17:01:50 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Zqiang , "Paul E . McKenney" Subject: [PATCH 2/3] rcu: Remove local_irq_save/restore() in rcu_preempt_deferred_qs_handler() Date: Fri, 15 Aug 2025 17:01:48 -0700 Message-Id: <20250816000149.2622447-2-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zqiang The per-CPU rcu_data structure's ->defer_qs_iw field is initialized by IRQ_WORK_INIT_HARD(), which means that the subsequent invocation of rcu_preempt_deferred_qs_handler() will always be executed with interrupts disabled. This commit therefore removes the local_irq_save/restore() operations from rcu_preempt_deferred_qs_handler() and adds a call to lockdep_assert_irqs_disabled() in order to enable lockdep to diagnose mistaken invocations of this function from interrupts-enabled code. Signed-off-by: Zqiang Signed-off-by: Paul E. McKenney --- kernel/rcu/tree_plugin.h | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 4cd170b2d6551d..d85763336b3c0f 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -626,11 +626,10 @@ notrace void rcu_preempt_deferred_qs(struct task_stru= ct *t) */ static void rcu_preempt_deferred_qs_handler(struct irq_work *iwp) { - unsigned long flags; struct rcu_data *rdp; =20 + lockdep_assert_irqs_disabled(); rdp =3D container_of(iwp, struct rcu_data, defer_qs_iw); - local_irq_save(flags); =20 /* * If the IRQ work handler happens to run in the middle of RCU read-side @@ -647,8 +646,6 @@ static void rcu_preempt_deferred_qs_handler(struct irq_= work *iwp) */ if (rcu_preempt_depth() > 0) WRITE_ONCE(rdp->defer_qs_iw_pending, DEFER_QS_IDLE); - - local_irq_restore(flags); } =20 /* --=20 2.40.1 From nobody Sat Oct 4 12:46:55 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F2E0EED8; Sat, 16 Aug 2025 00:01:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755302511; cv=none; b=L0kL6NBrj/0WZDTjPl3wJoEMHkRjhcIUoDYGamgtOfdWDWXhyRY40Wxo9AOAFk5RCEpuF8YJGBPcBK+KTzCP3JqukcumlGNn/vnnsWjTuowg+XUGhXde9IiLBft92jiZReCt5w3Pm/YQ7Q955SvKM0cs+4g6sXO/gZ17Qy6rkdg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755302511; c=relaxed/simple; bh=FGAfKe2mUubPShjfHv2CMrtmsbPxnRWUzJcdffSD7pY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jyFtCkX8Q6ozpndncIHKCxT+EqQC7ly4UXUmooja90JIdpe+TelDdqKTJEL/5JgOL6SxdbJBW+XgLck1ATt9jxug0xhGwKYwhuH+u0Gl44pk3JCOBKpReqWONUIACN7jiEqagJXe3oReRSw+eqNBljoQ7Fq3fNoGNdoahkO7DtM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=q01lbT5S; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="q01lbT5S" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2649FC4CEF7; Sat, 16 Aug 2025 00:01:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755302511; bh=FGAfKe2mUubPShjfHv2CMrtmsbPxnRWUzJcdffSD7pY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=q01lbT5Sf7em/FB4qtZVWjiUNyBptsGA1+xRHaDUHJQSBHqi/A+9pxefcV9cUpLtA r/Huc/7OOLG1UgSyPwBmgllW+xQfRjelusSgyOiwiKp3tYopohsMyljCIHmh/YySzX n+wqJ5gKqRcyrI3a7w87kQI27af2JXmSzSMDMRBXbsjWx9uTOuTF8U6mUNfkfN9TEo v6TtEQWMwVRS60nnqKPhN0oHQoU9h5bgVAwjXnnFsvKF+9mR5+lNWzM1meNbNAItXt zvgqbwfEUcKv0Kphtd+EaVRoMB/vAEG+PqY0LvBKoIgYtmx1fwiZ+iPq8CBG+WFMGc Q9cTfXazjCcUA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 530F4CE0B31; Fri, 15 Aug 2025 17:01:50 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Andy Shevchenko , Simona Vetter , "Paul E. McKenney" , Neeraj Upadhyay Subject: [PATCH 3/3] rculist: move list_for_each_rcu() to where it belongs Date: Fri, 15 Aug 2025 17:01:49 -0700 Message-Id: <20250816000149.2622447-3-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Andy Shevchenko The list_for_each_rcu() relies on the rcu_dereference() API which is not provided by the list.h. At the same time list.h is a low-level basic header that must not have dependencies like RCU, besides the fact of the potential circular dependencies in some cases. With all that said, move RCU related API to the rculist.h where it belongs. Signed-off-by: Andy Shevchenko Reviewed-by: Simona Vetter Reviewed-by: "Paul E. McKenney" Signed-off-by: Neeraj Upadhyay (AMD) --- include/linux/list.h | 10 ---------- include/linux/rculist.h | 10 ++++++++++ kernel/cgroup/dmem.c | 1 + 3 files changed, 11 insertions(+), 10 deletions(-) diff --git a/include/linux/list.h b/include/linux/list.h index e7e28afd28f8ee..e7bdad9b861827 100644 --- a/include/linux/list.h +++ b/include/linux/list.h @@ -686,16 +686,6 @@ static inline void list_splice_tail_init(struct list_h= ead *list, #define list_for_each(pos, head) \ for (pos =3D (head)->next; !list_is_head(pos, (head)); pos =3D pos->next) =20 -/** - * list_for_each_rcu - Iterate over a list in an RCU-safe fashion - * @pos: the &struct list_head to use as a loop cursor. - * @head: the head for your list. - */ -#define list_for_each_rcu(pos, head) \ - for (pos =3D rcu_dereference((head)->next); \ - !list_is_head(pos, (head)); \ - pos =3D rcu_dereference(pos->next)) - /** * list_for_each_continue - continue iteration over a list * @pos: the &struct list_head to use as a loop cursor. diff --git a/include/linux/rculist.h b/include/linux/rculist.h index 1b11926ddd4710..2abba7552605c5 100644 --- a/include/linux/rculist.h +++ b/include/linux/rculist.h @@ -42,6 +42,16 @@ static inline void INIT_LIST_HEAD_RCU(struct list_head *= list) */ #define list_bidir_prev_rcu(list) (*((struct list_head __rcu **)(&(list)->= prev))) =20 +/** + * list_for_each_rcu - Iterate over a list in an RCU-safe fashion + * @pos: the &struct list_head to use as a loop cursor. + * @head: the head for your list. + */ +#define list_for_each_rcu(pos, head) \ + for (pos =3D rcu_dereference((head)->next); \ + !list_is_head(pos, (head)); \ + pos =3D rcu_dereference(pos->next)) + /** * list_tail_rcu - returns the prev pointer of the head of the list * @head: the head of the list diff --git a/kernel/cgroup/dmem.c b/kernel/cgroup/dmem.c index 10b63433f05737..e12b946278b6c6 100644 --- a/kernel/cgroup/dmem.c +++ b/kernel/cgroup/dmem.c @@ -14,6 +14,7 @@ #include #include #include +#include #include =20 struct dmem_cgroup_region { --=20 2.40.1