From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A847C433F5 for ; Mon, 18 Apr 2022 22:55:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233869AbiDRW5p (ORCPT ); Mon, 18 Apr 2022 18:57:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233279AbiDRW4s (ORCPT ); Mon, 18 Apr 2022 18:56:48 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4ED612CCB9; Mon, 18 Apr 2022 15:54:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D79756112D; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39A40C385A9; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=5pKw21ROdvL8mIx9oLf8vDqfS0PiGcNeehbP1/+zfCo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NVxz5Anhp90StSHZZp3f0jD9hYsyxxquGMbncxrx9grbgLZIJlgHkU5xfNHkCDR8h fYan2rmSX45Ccm3oz/iQ1GHXESMFgiply1ur1XqNcoIegvuZ0Oh2UqmUQ/u6iD0oA8 4qL3HrR9a/kaE/sJPjagHaOZdIfw86jevvgQRZZkz6NIbvTDUSDQ1Qh8LRw56fBnIw qST8By1VqngzONq3L21lOvSxeWU35vAjgQaPfNI7C04xDrVyZdbGoN/koC+J878aye n0pyhJuERn5z2a2sYiDpg3MKKTjl40Pa9BkLM0jeiZpaxHZhrGSwikme8xGCMUI95Z bhuXrGxsk4wXA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id DE20D5C04BD; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Boqun Feng Subject: [PATCH rcu 01/11] rcu: Clarify fill-the-gap comment in rcu_segcblist_advance() Date: Mon, 18 Apr 2022 15:53:49 -0700 Message-Id: <20220418225359.3945217-1-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Reported-by: Frederic Weisbecker Reported-by: Neeraj Upadhyay Reported-by: Boqun Feng Signed-off-by: Paul E. McKenney --- kernel/rcu/rcu_segcblist.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/rcu_segcblist.c b/kernel/rcu/rcu_segcblist.c index 81145c3ece25..c54ea2b6a36b 100644 --- a/kernel/rcu/rcu_segcblist.c +++ b/kernel/rcu/rcu_segcblist.c @@ -505,10 +505,10 @@ void rcu_segcblist_advance(struct rcu_segcblist *rscl= p, unsigned long seq) WRITE_ONCE(rsclp->tails[j], rsclp->tails[RCU_DONE_TAIL]); =20 /* - * Callbacks moved, so clean up the misordered ->tails[] pointers - * that now point into the middle of the list of ready-to-invoke - * callbacks. The overall effect is to copy down the later pointers - * into the gap that was created by the now-ready segments. + * Callbacks moved, so there might be an empty RCU_WAIT_TAIL + * and a non-empty RCU_NEXT_READY_TAIL. If so, copy the + * RCU_NEXT_READY_TAIL segment to fill the RCU_WAIT_TAIL gap + * created by the now-ready-to-invoke segments. */ for (j =3D RCU_WAIT_TAIL; i < RCU_NEXT_TAIL; i++, j++) { if (rsclp->tails[j] =3D=3D rsclp->tails[RCU_NEXT_TAIL]) --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCB2CC4332F for ; Mon, 18 Apr 2022 22:54:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233534AbiDRW4z (ORCPT ); Mon, 18 Apr 2022 18:56:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233489AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 629252CE28; Mon, 18 Apr 2022 15:54:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C595761155; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 356A0C385A7; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=T23Yz4zWKpA2oFr2jAlYdX5ehd6jgS26JMF7aI6PzXY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UHobq5IfsW7/XWjNQMzAzb0OWrrYP3MtawaQmCDA/ZLHQvpu5i2rQKrrq9kaJVPKV /FnfyduhPDzz1/jEFVXOHf9r6nIYDE7ULp2Xpznu2V+K+sxFoykqKiwVkLi2UthkYC TMmA3dWvFxa1Ts+g+h6XZMxeApB2oCHeOPEn5E8gSITJ5ccDcknKhGbp27kA0VuhXA KcsRiqKZQ1AHeHDQBmzD3gTzSKw0AL4K5VPAPvlFdhgvXsqOSMG2AeRHUP4nHkxm+j a0nRDMuVMVkp5X6s9SDE3JuczuvW8CEBYec5/xKohk4uGpw3J/YcLPR1XL0zLrZi4R F4DrSGdvl/5Ww== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id E16D95C04C6; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 02/11] rcu: Fix rcu_preempt_deferred_qs_irqrestore() strict QS reporting Date: Mon, 18 Apr 2022 15:53:50 -0700 Message-Id: <20220418225359.3945217-2-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Suppose we have a kernel built with both CONFIG_RCU_STRICT_GRACE_PERIOD=3Dy and CONFIG_PREEMPT=3Dy. Suppose further that an RCU reader from which RCU core needs a quiescent state ends in rcu_preempt_deferred_qs_irqrestore(). This function will then invoke rcu_report_qs_rdp() in order to immediately report that quiescent state. Unfortunately, it will not have cleared that reader's CPU's rcu_data structure's ->cpu_no_qs.b.norm field. As a result, rcu_report_qs_rdp() will take an early exit because it will believe that this CPU has not yet encountered a quiescent state, and there will be no reporting of the current quiescent state. This commit therefore causes rcu_preempt_deferred_qs_irqrestore() to clear the ->cpu_no_qs.b.norm field before invoking rcu_report_qs_rdp(). Kudos to Boqun Feng and Neeraj Upadhyay for helping with analysis of this issue! Signed-off-by: Paul E. McKenney --- kernel/rcu/tree_plugin.h | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 8360d86db1c0..176639c6215f 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -486,6 +486,7 @@ rcu_preempt_deferred_qs_irqrestore(struct task_struct *= t, unsigned long flags) t->rcu_read_unlock_special.s =3D 0; if (special.b.need_qs) { if (IS_ENABLED(CONFIG_RCU_STRICT_GRACE_PERIOD)) { + rdp->cpu_no_qs.b.norm =3D false; rcu_report_qs_rdp(rdp); udelay(rcu_unlock_delay); } else { --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81F23C433FE for ; Mon, 18 Apr 2022 22:54:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233680AbiDRW5G (ORCPT ); Mon, 18 Apr 2022 18:57:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233256AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C88A82CC9D; Mon, 18 Apr 2022 15:54:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 77511B81135; Mon, 18 Apr 2022 22:54:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3588FC385A8; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=3/4SJxZl7LRSsaEhih4xR7lzgf/M6pQIEs1EAdF+8Xg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t/MDb//Dx1aVAuwdSFZUrhtpeY5xRFXWDcqAFKG9dd54tI2v/+dDd+iUaG9JUUqLj gpUM8xB2YJpklamaW8j7k0VXsAFV3QePxlcNb9YMlZnWZb1jnuRXS3rVdv8TFL00zO KQaTFfBLM5xOOOtVUXVxSx/uWcgrv82TDyDTZKJ8GGjO7EPBxHIs7HCd3Ss5tw/Lcw KQ3kPD9x+7xMUG5ETaT/FVVVOEyLxJdV9Owk1LmBjyQ00cAmFHa4/N0siDX5fMtVYV yfsqpDZD+m+DJESF1aIRlwpQqQCtxx/rBYK2rNp6vQF5bfHKoxMDgKelyezv2VKCBI 045K4+Yqen8fA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id E36075C0848; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, "Paul E. McKenney" , Saravanan D Subject: [PATCH rcu 03/11] rcu: Check for jiffies going backwards Date: Mon, 18 Apr 2022 15:53:51 -0700 Message-Id: <20220418225359.3945217-3-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" A report of a 12-jiffy normal RCU CPU stall warning raises interesting questions about the nature of time on the offending system. This commit instruments rcu_sched_clock_irq(), which is RCU's hook into the scheduling-clock interrupt, checking for the jiffies counter going backwards. Reported-by: Saravanan D Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 10 ++++++++++ kernel/rcu/tree.h | 1 + 2 files changed, 11 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index a4b8189455d5..a5ea67454640 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -1679,6 +1679,8 @@ static bool __note_gp_changes(struct rcu_node *rnp, s= truct rcu_data *rdp) rdp->gp_seq =3D rnp->gp_seq; /* Remember new grace-period state. */ if (ULONG_CMP_LT(rdp->gp_seq_needed, rnp->gp_seq_needed) || rdp->gpwrap) WRITE_ONCE(rdp->gp_seq_needed, rnp->gp_seq_needed); + if (IS_ENABLED(CONFIG_PROVE_RCU) && READ_ONCE(rdp->gpwrap)) + WRITE_ONCE(rdp->last_sched_clock, jiffies); WRITE_ONCE(rdp->gpwrap, false); rcu_gpnum_ovf(rnp, rdp); return ret; @@ -2609,6 +2611,13 @@ static void rcu_do_batch(struct rcu_data *rdp) */ void rcu_sched_clock_irq(int user) { + unsigned long j; + + if (IS_ENABLED(CONFIG_PROVE_RCU)) { + j =3D jiffies; + WARN_ON_ONCE(time_before(j, __this_cpu_read(rcu_data.last_sched_clock))); + __this_cpu_write(rcu_data.last_sched_clock, j); + } trace_rcu_utilization(TPS("Start scheduler-tick")); lockdep_assert_irqs_disabled(); raw_cpu_inc(rcu_data.ticks_this_gp); @@ -4179,6 +4188,7 @@ rcu_boot_init_percpu_data(int cpu) rdp->rcu_ofl_gp_flags =3D RCU_GP_CLEANED; rdp->rcu_onl_gp_seq =3D rcu_state.gp_seq; rdp->rcu_onl_gp_flags =3D RCU_GP_CLEANED; + rdp->last_sched_clock =3D jiffies; rdp->cpu =3D cpu; rcu_boot_init_nocb_percpu_data(rdp); } diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h index 926673ebe355..94b55f669915 100644 --- a/kernel/rcu/tree.h +++ b/kernel/rcu/tree.h @@ -254,6 +254,7 @@ struct rcu_data { unsigned long rcu_onl_gp_seq; /* ->gp_seq at last online. */ short rcu_onl_gp_flags; /* ->gp_flags at last online. */ unsigned long last_fqs_resched; /* Time of last rcu_resched(). */ + unsigned long last_sched_clock; /* Jiffies of last rcu_sched_clock_irq().= */ =20 int cpu; }; --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1861CC433F5 for ; Mon, 18 Apr 2022 22:54:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233655AbiDRW5S (ORCPT ); Mon, 18 Apr 2022 18:57:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233305AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0C0F2CE2C; Mon, 18 Apr 2022 15:54:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 8A5F8B81136; Mon, 18 Apr 2022 22:54:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 42828C385A1; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=tPyCRwdFs1UacCDWo6gPWFP2J8tgNjkdVtTI/kMjO/s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TcUbXgjE7VF1cqmzO0nZknf3hCOB6m2pLrkKrPBeLeT9duKCb0DbzbydFSa50sG0k hcRX7uV/VAEGuvQEbnPMYNOHsXJCGFJCmzNXAl+2EWzwzQ54ak8oC8qeQCpinYhuAs CqtBVAiJGo8D2iuyaWcHhM4USdLCm898Bq+B2NFawfme1LKeRZQKO+IQodD93TI4vR v5VbPp/rAodcE/QRsmIGNmkOYd7xequfoHJKHDylT4mOfQBdsqmhh6t3ICVkAlYxJB rjgwNxfhHMDWRwSYlruRV7piO/azxnadspi1KG+660ywzzP9uLiZmM1VuOot1pvtlW ugsjYlo/5TWUg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id E554E5C0A0D; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, "Paul E. McKenney" , Rik van Riel , Peter Zijlstra , Ingo Molnar , Thomas Gleixner , Sebastian Andrzej Siewior , Juergen Gross Subject: [PATCH rcu 04/11] kernel/smp: Provide boot-time timeout for CSD lock diagnostics Date: Mon, 18 Apr 2022 15:53:52 -0700 Message-Id: <20220418225359.3945217-4-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Debugging of problems involving insanely long-running SMI handlers proceeds better if the CSD-lock timeout can be adjusted. This commit therefore provides a new smp.csd_lock_timeout kernel boot parameter that specifies the timeout in milliseconds. The default remains at the previously hard-coded value of five seconds. Cc: Rik van Riel Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Sebastian Andrzej Siewior Cc: Juergen Gross Signed-off-by: Paul E. McKenney Reviewed-by: Juergen Gross --- Documentation/admin-guide/kernel-parameters.txt | 11 +++++++++++ kernel/smp.c | 7 +++++-- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index 3f1cc5e317ed..645c4c001b16 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5377,6 +5377,17 @@ smart2=3D [HW] Format: [,[,...,]] =20 + smp.csd_lock_timeout=3D [KNL] + Specify the period of time in milliseconds + that smp_call_function() and friends will wait + for a CPU to release the CSD lock. This is + useful when diagnosing bugs involving CPUs + disabling interrupts for extended periods + of time. Defaults to 5,000 milliseconds, and + setting a value of zero disables this feature. + This feature may be more efficiently disabled + using the csdlock_debug- kernel parameter. + smsc-ircc2.nopnp [HW] Don't use PNP to discover SMC devices smsc-ircc2.ircc_cfg=3D [HW] Device configuration I/O port smsc-ircc2.ircc_sir=3D [HW] SIR base I/O port diff --git a/kernel/smp.c b/kernel/smp.c index 01a7c1706a58..d82439bac401 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -183,7 +183,9 @@ static DEFINE_PER_CPU(smp_call_func_t, cur_csd_func); static DEFINE_PER_CPU(void *, cur_csd_info); static DEFINE_PER_CPU(struct cfd_seq_local, cfd_seq_local); =20 -#define CSD_LOCK_TIMEOUT (5ULL * NSEC_PER_SEC) +static ulong csd_lock_timeout =3D 5000; /* CSD lock timeout in millisecon= ds. */ +module_param(csd_lock_timeout, ulong, 0444); + static atomic_t csd_bug_count =3D ATOMIC_INIT(0); static u64 cfd_seq; =20 @@ -329,6 +331,7 @@ static bool csd_lock_wait_toolong(struct __call_single_= data *csd, u64 ts0, u64 * u64 ts2, ts_delta; call_single_data_t *cpu_cur_csd; unsigned int flags =3D READ_ONCE(csd->node.u_flags); + unsigned long long csd_lock_timeout_ns =3D csd_lock_timeout * NSEC_PER_MS= EC; =20 if (!(flags & CSD_FLAG_LOCK)) { if (!unlikely(*bug_id)) @@ -341,7 +344,7 @@ static bool csd_lock_wait_toolong(struct __call_single_= data *csd, u64 ts0, u64 * =20 ts2 =3D sched_clock(); ts_delta =3D ts2 - *ts1; - if (likely(ts_delta <=3D CSD_LOCK_TIMEOUT)) + if (likely(ts_delta <=3D csd_lock_timeout_ns || csd_lock_timeout_ns <=3D = 0)) return false; =20 firsttime =3D !*bug_id; --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5FCFC433EF for ; Mon, 18 Apr 2022 22:55:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233906AbiDRW5X (ORCPT ); Mon, 18 Apr 2022 18:57:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44368 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233367AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 119EC2CE2E; Mon, 18 Apr 2022 15:54:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id AE5CFB81142; Mon, 18 Apr 2022 22:54:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4C0FBC385AB; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=2yulGwLtnjKwidHWmy2CXBtr+UjTlFlstTStV9RGdkk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K5prFdd1O9kNCDUHafYTSelzXQvBRmxsKjrM5bINh+/dbNrG5LkZcVG2y3hZsNTpI fUAQfCzM3FL9fM8zYUIkF0DDTtqgaG4Tc22gSIW6ygdVx1nUh8YDosQlHA5FNoGfPn Hph+Ix1YTlXHNfIQx1l4ENnVDS8wvJlvX/I4opIBzEkBX30Rpfp3xdYXfGGJIDKg8f /BJCjeYEajg2nXef0EgdCtI/pR77hh0agRbl9XGQxMSZunqjVVJ7RP9nKeH4JZ0jRp Qw5sm5RWCf+asGz5EVy0rrj+4WG21W9DCswCXyhMVKsSaJh31oKXgm/ALeIIaplRSj tv18NyrhzEC0Q== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id E726B5C0A23; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, "Paul E. McKenney" , Boqun Feng , Frederic Weisbecker , Neeraj Upadhyay , Uladzislau Rezki Subject: [PATCH rcu 05/11] rcu: Add comments to final rcu_gp_cleanup() "if" statement Date: Mon, 18 Apr 2022 15:53:53 -0700 Message-Id: <20220418225359.3945217-5-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The final "if" statement in rcu_gp_cleanup() has proven to be rather confusing, straightforward though it might have seemed when initially written. This commit therefore adds comments to its "then" and "else" clauses to at least provide a more elevated form of confusion. Reported-by: Boqun Feng Reported-by: Frederic Weisbecker Reported-by: Neeraj Upadhyay Reported-by: Uladzislau Rezki Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index a5ea67454640..29669070348e 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2098,14 +2098,29 @@ static noinline void rcu_gp_cleanup(void) /* Advance CBs to reduce false positives below. */ offloaded =3D rcu_rdp_is_offloaded(rdp); if ((offloaded || !rcu_accelerate_cbs(rnp, rdp)) && needgp) { + + // We get here if a grace period was needed (=E2=80=9Cneedgp=E2=80=9D) + // and the above call to rcu_accelerate_cbs() did not set + // the RCU_GP_FLAG_INIT bit in ->gp_state (which records + // the need for another grace period).=C2=A0 The purpose + // of the =E2=80=9Coffloaded=E2=80=9D check is to avoid invoking + // rcu_accelerate_cbs() on an offloaded CPU because we do not + // hold the ->nocb_lock needed to safely access an offloaded + // ->cblist.=C2=A0 We do not want to acquire that lock because + // it can be heavily contended during callback floods. + WRITE_ONCE(rcu_state.gp_flags, RCU_GP_FLAG_INIT); WRITE_ONCE(rcu_state.gp_req_activity, jiffies); - trace_rcu_grace_period(rcu_state.name, - rcu_state.gp_seq, - TPS("newreq")); + trace_rcu_grace_period(rcu_state.name, rcu_state.gp_seq, TPS("newreq")); } else { - WRITE_ONCE(rcu_state.gp_flags, - rcu_state.gp_flags & RCU_GP_FLAG_INIT); + + // We get here either if there is no need for an + // additional grace period or if rcu_accelerate_cbs() has + // already set the RCU_GP_FLAG_INIT bit in ->gp_flags.=C2=A0 + // So all we need to do is to clear all of the other + // ->gp_flags bits. + + WRITE_ONCE(rcu_state.gp_flags, rcu_state.gp_flags & RCU_GP_FLAG_INIT); } raw_spin_unlock_irq_rcu_node(rnp); =20 --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBCF6C4332F for ; Mon, 18 Apr 2022 22:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234410AbiDRW5e (ORCPT ); Mon, 18 Apr 2022 18:57:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233378AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8CB752CE38; Mon, 18 Apr 2022 15:54:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 23995B8114A; Mon, 18 Apr 2022 22:54:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89E87C385B4; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=uhiwOIrk44qfnhD7O0vrwgQ+fBsOSHKHX5yQ0StUtJA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=c0PVY37NJ/CMr5qudl02I2FfmOQw4uF6x8q94rBofcV3x8FmqaityrZORWq1kxu8+ jOGtBQMGqNuKeQLPOTY+cUwNKH4UHv5a8P9QOuXiELinf4ZJNyAg3GKrokIa1+gopJ 3kmkIFFwyWn2wfod+iV2ODQidTIc1PIUilJLAhFBPUERSGR1rgnLwDlbX4lurfhVMY dnrFI2VFnwHzhmtT4uXMJL7wM/IM7Z1jx59xsgLlcOMWpUM080ZWggf4xcujPlnyZZ imBYuI3fPMeVzxfZ+7Y4fuqZy+6OewxWqzDboxzhy8GvGnGmM/c8U3f2bGtwkQgOvb DknMQAaogHZ+A== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id E8F0A5C0B86; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 06/11] rcu: Print number of online CPUs in RCU CPU stall-warning messages Date: Mon, 18 Apr 2022 15:53:54 -0700 Message-Id: <20220418225359.3945217-6-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" RCU's synchronous grace periods act quite differently when there is only one online CPU, especially in the no-op case in kernels built with CONFIG_PREEMPTION=3Dn. This change in behavior can be important debugging information, so this commit adds the number of online CPUs to the RCU CPU stall warning messages. Signed-off-by: Paul E. McKenney --- kernel/rcu/tree_stall.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h index 0c5d8516516a..268dd79c58e7 100644 --- a/kernel/rcu/tree_stall.h +++ b/kernel/rcu/tree_stall.h @@ -565,9 +565,9 @@ static void print_other_cpu_stall(unsigned long gp_seq,= unsigned long gps) =20 for_each_possible_cpu(cpu) totqlen +=3D rcu_get_n_cbs_cpu(cpu); - pr_cont("\t(detected by %d, t=3D%ld jiffies, g=3D%ld, q=3D%lu)\n", + pr_cont("\t(detected by %d, t=3D%ld jiffies, g=3D%ld, q=3D%lu ncpus=3D%d)= \n", smp_processor_id(), (long)(jiffies - gps), - (long)rcu_seq_current(&rcu_state.gp_seq), totqlen); + (long)rcu_seq_current(&rcu_state.gp_seq), totqlen, rcu_state.n_onl= ine_cpus); if (ndetected) { rcu_dump_cpu_stacks(); =20 @@ -626,9 +626,9 @@ static void print_cpu_stall(unsigned long gps) raw_spin_unlock_irqrestore_rcu_node(rdp->mynode, flags); for_each_possible_cpu(cpu) totqlen +=3D rcu_get_n_cbs_cpu(cpu); - pr_cont("\t(t=3D%lu jiffies g=3D%ld q=3D%lu)\n", + pr_cont("\t(t=3D%lu jiffies g=3D%ld q=3D%lu ncpus=3D%d)\n", jiffies - gps, - (long)rcu_seq_current(&rcu_state.gp_seq), totqlen); + (long)rcu_seq_current(&rcu_state.gp_seq), totqlen, rcu_state.n_online_cp= us); =20 rcu_check_gp_kthread_expired_fqs_timer(); rcu_check_gp_kthread_starvation(); --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E3BEC433F5 for ; Mon, 18 Apr 2022 22:54:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233729AbiDRW5L (ORCPT ); Mon, 18 Apr 2022 18:57:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233512AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D745A2D1C5; Mon, 18 Apr 2022 15:54:05 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 24BEDB8114B; Mon, 18 Apr 2022 22:54:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89EBDC385B7; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=4F6D4WuqV0uCGvWLKe5pephkIMgIflKcXAmI/XqzuiM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gOM59wfhUB1kqFMcXsEmM5bKcvdKaiD3+q6Zib+RPbrKMoVMSsuOr7RBuFLCQzYl/ 5+cBSz2UNlUw+GQSAdi+bKckBf/ElV1qLqwk4vW08TqWfzLC8yZC9kTeTlDjfvyhi2 DZg0x/NSb3mgB9+i+MtZbwtFB92ZhtyEMoCam64HW3uVeiBGKvQNJKX592R/+JI+W9 1f9hnn36lb5tKosT29CjONUO/Awx+GE5csr2m0E+Npo7G5a8Ac4HevL/8gTRIMEbjT ZPw/VPOsG6OMbnwFTSEZ4bY2b232I5+4bYEA/33JKw1Wa5vdydGeaufkSIMb+QSU6F 0r8VixORs1C0w== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id EAF665C0DFD; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Frederic Weisbecker , "Paul E . McKenney" , Uladzislau Rezki , Joel Fernandes , Boqun Feng , Peter Zijlstra , Neeraj Upadhyay , Valentin Schneider Subject: [PATCH rcu 07/11] rcu: Fix preemption mode check on synchronize_rcu[_expedited]() Date: Mon, 18 Apr 2022 15:53:55 -0700 Message-Id: <20220418225359.3945217-7-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Frederic Weisbecker An early check on synchronize_rcu[_expedited]() tries to determine if the current CPU is in UP mode on an SMP no-preempt kernel, in which case there is no need to start a grace period since the current assumed quiescent state is all we need. However the preemption mode doesn't take into account the boot selected preemption mode under CONFIG_PREEMPT_DYNAMIC=3Dy, missing a possible early return if the running flavour is "none" or "voluntary". Use the shiny new preempt mode accessors to fix this. However, avoid invoking them during early boot because doing so triggers a WARN_ON_ONCE(). [ paulmck: Update for mainlined API. ] Reported-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker Cc: Uladzislau Rezki Cc: Joel Fernandes Cc: Boqun Feng Cc: Peter Zijlstra Cc: Neeraj Upadhyay Cc: Valentin Schneider Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 29669070348e..d3caa82b9954 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3741,7 +3741,9 @@ static int rcu_blocking_is_gp(void) { int ret; =20 - if (IS_ENABLED(CONFIG_PREEMPTION)) + // Invoking preempt_model_*() too early gets a splat. + if (rcu_scheduler_active =3D=3D RCU_SCHEDULER_INACTIVE || + preempt_model_full() || preempt_model_rt()) return rcu_scheduler_active =3D=3D RCU_SCHEDULER_INACTIVE; might_sleep(); /* Check for RCU read-side critical section. */ preempt_disable(); --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30529C433F5 for ; Mon, 18 Apr 2022 22:54:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233737AbiDRW5P (ORCPT ); Mon, 18 Apr 2022 18:57:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233497AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8CCDE2CE3B; Mon, 18 Apr 2022 15:54:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 17885B81148; Mon, 18 Apr 2022 22:54:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E70AC385B6; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=7YceAtQ4xCGuX9O3QcLmd4DKQm9RucoBz1BbPwEutSg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZmMN9xvPQderK9vVByZh5rBt/vyyHCPUQuUl6ftiTHVZ/2t43QFA+9TIRY5gm7jQX CPayMZBHu5nKUzvMsIwSiEwjHqMlCDjWX1u37LzOpm9cw8vAl57hZ81tFiTA01+ZrD Yo0fSZnkM+YE4fp04aoqH2qaY+PBW1Q3AMuxE+nu2cgIgikUJZwNgunlyqr0LnUUn9 rG2Mg8O7ChLolIdsuAwmhTjPXCm/4xwbytAmGXwmrdYhII1kadckulN//W1IQ/vwYm STPglXypdkND0e9CillyEdx7/yUbs2bkaGajISr/Hfs1JUe1QoTOBccNGqON6w507o XhsWqF3iozXMw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id ECC905C121E; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Lukas Bulwahn , "Paul E . McKenney" Subject: [PATCH rcu 08/11] srcu: Drop needless initialization of sdp in srcu_gp_start() Date: Mon, 18 Apr 2022 15:53:56 -0700 Message-Id: <20220418225359.3945217-8-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lukas Bulwahn Commit 9c7ef4c30f12 ("srcu: Make Tree SRCU able to operate without snp_node array") initializes the local variable sdp differently depending on the srcu's state in srcu_gp_start(). Either way, this initialization overwrites the value used when sdp is defined. This commit therefore drops this pointless definition-time initialization. Although there is no functional change, compiler code generation may be affected. Signed-off-by: Lukas Bulwahn Signed-off-by: Paul E. McKenney --- kernel/rcu/srcutree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 6833d8887181..e0ef018612a2 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -434,7 +434,7 @@ EXPORT_SYMBOL_GPL(__srcu_read_unlock); */ static void srcu_gp_start(struct srcu_struct *ssp) { - struct srcu_data *sdp =3D this_cpu_ptr(ssp->sda); + struct srcu_data *sdp; int state; =20 lockdep_assert_held(&ACCESS_PRIVATE(ssp, lock)); --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51172C43219 for ; Mon, 18 Apr 2022 22:54:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233424AbiDRW4w (ORCPT ); Mon, 18 Apr 2022 18:56:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233483AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 627B92CE20; Mon, 18 Apr 2022 15:54:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 779E161037; Mon, 18 Apr 2022 22:54:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 98C13C385B9; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=hK5x6BVv25qP8BKNXoIb9LDR8AFOXkO1p5pr+bQzLFU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Maic1LuAc80j57STgy7M0pyeJzX0a0C1VVbSKtTsh+079xE+aljXM6mk7Hqsx6ECA NZjLS4axQBYrFAY34jG7MWH3loaP6nFvMDbUP81auvSKrwA/59Y6Je0ng4u04YcAwa re/pU06wwSXkeFGMS1QlTJME5q340n2ntNCBkd/SZwEicb7O8Nn23jQiR8/ZY7l+2i Uv/f4U56agz3e70dhA9hzBooMgAQ2F7JIOP3ZGFTCeEyTWiRIqg0GMcF14UFTEk12a wdhWcQrFf5ZP+HQSuSrIkwcVqP2LAe0ubWTJ2RVWnPFQ4anWYl2zlqRca7dAotBCvR x+QPnAl/OvL9w== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id EE8B15C12AB; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Zqiang , "Paul E . McKenney" Subject: [PATCH rcu 09/11] rcu: Check for successful spawn of ->boost_kthread_task Date: Mon, 18 Apr 2022 15:53:57 -0700 Message-Id: <20220418225359.3945217-9-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zqiang For the spawning of the priority-boost kthreads can fail, improbable though this might seem. This commit therefore refrains from attemoting to initiate RCU priority boosting when The ->boost_kthread_task pointer is NULL. Signed-off-by: Zqiang Signed-off-by: Paul E. McKenney --- kernel/rcu/tree_plugin.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 176639c6215f..5c23aceecd62 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -1125,7 +1125,8 @@ static void rcu_initiate_boost(struct rcu_node *rnp, = unsigned long flags) __releases(rnp->lock) { raw_lockdep_assert_held_rcu_node(rnp); - if (!rcu_preempt_blocked_readers_cgp(rnp) && rnp->exp_tasks =3D=3D NULL) { + if (!rnp->boost_kthread_task || + (!rcu_preempt_blocked_readers_cgp(rnp) && !rnp->exp_tasks)) { raw_spin_unlock_irqrestore_rcu_node(rnp, flags); return; } --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 234F6C433F5 for ; Mon, 18 Apr 2022 22:54:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233483AbiDRW5C (ORCPT ); Mon, 18 Apr 2022 18:57:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233281AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 623EE2CE1E; Mon, 18 Apr 2022 15:54:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A5E41611B5; Mon, 18 Apr 2022 22:54:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 990FBC385BA; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=WEGp4fOqBZnoeWztmlY6PE4IqXgAbDfyysxBTX/6wbQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Fg2xfCuAmR1sRRz+Pj12/Rcv6uPeWFMc5UnRQJHmkOZTcwwJG/nv3YNbQDdr05tJ4 G4LJocC9/sACEapywezo9jleoZqAcWWPm1OV+iXeUZwyvVlf09wGsKl1RuTp8JdXZe fovK5sB8Fq23Zd8Akd2IpIdMVnZWCylr0Z3LNyEUjb3PjeXD7CgMzTfhYOJp1bTda+ 4nxIS+s58ykBRt1HIjS0PSpKk1Jlwpn8iJuuXWMobxT7JpG9QGgvFFEb88lNPQyaj6 e4c/6stCPBA5Pz8LG+RXLuhfCy5g57mcSwnlvLmDTFKNI3s7/8gwyC27cy6prfQjI9 TnSI9RARm5gmw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id F04905C1308; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, David Vernet , "Paul E . McKenney" Subject: [PATCH rcu 10/11] rcu_sync: Fix comment to properly reflect rcu_sync_exit() behavior Date: Mon, 18 Apr 2022 15:53:58 -0700 Message-Id: <20220418225359.3945217-10-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: David Vernet The rcu_sync_enter() function is used by updaters to force RCU readers (e.g. percpu-rwsem) to use their slow paths during an update. This is accomplished by setting the ->gp_state of the rcu_sync structure to GP_ENTER. In the case of percpu-rwsem, the readers' slow path waits on a semaphore instead of just incrementing a reader count. Each updater invokes the rcu_sync_exit() function to signal to readers that they may again take their fastpaths. The rcu_sync_exit() function sets the ->gp_state of the rcu_sync structure to GP_EXIT, and if all goes well, after a grace period the ->gp_state reverts back to GP_IDLE. Unfortunately, the rcu_sync_enter() function currently has a comment incorrectly stating that rcu_sync_exit() (by an updater) will re-enable reader "slowpaths". This patch changes the comment to state that this function re-enables reader fastpaths. Signed-off-by: David Vernet Signed-off-by: Paul E. McKenney --- kernel/rcu/sync.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/sync.c b/kernel/rcu/sync.c index 33d896d85902..5cefc702158f 100644 --- a/kernel/rcu/sync.c +++ b/kernel/rcu/sync.c @@ -111,7 +111,7 @@ static void rcu_sync_func(struct rcu_head *rhp) * a slowpath during the update. After this function returns, all * subsequent calls to rcu_sync_is_idle() will return false, which * tells readers to stay off their fastpaths. A later call to - * rcu_sync_exit() re-enables reader slowpaths. + * rcu_sync_exit() re-enables reader fastpaths. * * When called in isolation, rcu_sync_enter() must wait for a grace * period, however, closely spaced calls to rcu_sync_enter() can --=20 2.31.1.189.g2e36527f23 From nobody Mon May 11 02:06:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBAA1C433F5 for ; Mon, 18 Apr 2022 22:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234201AbiDRW53 (ORCPT ); Mon, 18 Apr 2022 18:57:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233519AbiDRW4t (ORCPT ); Mon, 18 Apr 2022 18:56:49 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D86562D1CA; Mon, 18 Apr 2022 15:54:05 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id CC07CB81147; Mon, 18 Apr 2022 22:54:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A1A7AC385B8; Mon, 18 Apr 2022 22:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650322441; bh=hgEd3cpLsRyZZOBZzaKl/DvFtkkxfVr47Q/gsMtP94Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fIwQHmfCQMiJX9gTWhnRiYrGeGRiAjSnmKfKNwTX3T8czHIPfXHr20hQj4H4RIV3Z 5+ikJZSPNMdE0p+FEq9BR/D+Dn8IsDp5na/wlQAWlR+R4StiOrff1O+nkjBVxTFqNl M42Ap8B38iWEo2feWC8XVIFDcliBrq/tMK2AuVYPwfTdhboYt0/on7ejp6W6yHg4NW YIzNXuPfYq/16AgqG5nCLTIhI3J+nOCMj7tcLwLLDMc1yVr0XnrEtSlERZlHTsQBhI 8EZHd7b0BDx7d6DrGd6HTNBpxtcsHdE2bKDYpFvObzSReN88mfq8excuSUkZ5DzHtc rxLMARwgOE+YQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id F20BD5C1355; Mon, 18 Apr 2022 15:54:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Zqiang , "Paul E . McKenney" Subject: [PATCH rcu 11/11] rcu: Use IRQ_WORK_INIT_HARD() to avoid rcu_read_unlock() hangs Date: Mon, 18 Apr 2022 15:53:59 -0700 Message-Id: <20220418225359.3945217-11-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> References: <20220418225345.GA3945110@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zqiang When booting kernels built with both CONFIG_RCU_STRICT_GRACE_PERIOD=3Dy and CONFIG_PREEMPT_RT=3Dy, the rcu_read_unlock_special() function's invocation of irq_work_queue_on() the init_irq_work() causes the rcu_preempt_deferred_qs_handler() function to work execute in SCHED_FIFO irq_work kthreads. Because rcu_read_unlock_special() is invoked on each rcu_read_unlock() in such kernels, the amount of work just keeps piling up, resulting in a boot-time hang. This commit therefore avoids this hang by using IRQ_WORK_INIT_HARD() instead of init_irq_work(), but only in kernels built with both CONFIG_PREEMPT_RT=3Dy and CONFIG_RCU_STRICT_GRACE_PERIOD=3Dy. Signed-off-by: Zqiang Signed-off-by: Paul E. McKenney --- kernel/rcu/tree_plugin.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 5c23aceecd62..2a3715419073 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -661,7 +661,13 @@ static void rcu_read_unlock_special(struct task_struct= *t) expboost && !rdp->defer_qs_iw_pending && cpu_online(rdp->cpu)) { // Get scheduler to re-evaluate and call hooks. // If !IRQ_WORK, FQS scan will eventually IPI. - init_irq_work(&rdp->defer_qs_iw, rcu_preempt_deferred_qs_handler); + if (IS_ENABLED(CONFIG_RCU_STRICT_GRACE_PERIOD) && + IS_ENABLED(CONFIG_PREEMPT_RT)) + rdp->defer_qs_iw =3D IRQ_WORK_INIT_HARD( + rcu_preempt_deferred_qs_handler); + else + init_irq_work(&rdp->defer_qs_iw, + rcu_preempt_deferred_qs_handler); rdp->defer_qs_iw_pending =3D true; irq_work_queue_on(&rdp->defer_qs_iw, rdp->cpu); } --=20 2.31.1.189.g2e36527f23