With PREEMPT_RCU=n, cond_resched() provides urgently needed quiescent
states for read-side critical sections via rcu_all_qs().
One reason why this was needed, was lacking preempt-count, the tick
handler has no way of knowing whether it is executing in a read-side
critical section or not.
With PREEMPT_LAZY=y, there can be configurations with PREEMPT_COUNT=y,
PREEMPT_RCU=n, where cond_resched() is a stub that does not provide
quiescent states via rcu_all_qs().
So, use the availability of preempt_count() to report quiescent states
in rcu_flavor_sched_clock_irq().
Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
kernel/rcu/tree_plugin.h | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 1c7cbd145d5e..da324d66034b 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -974,13 +974,16 @@ static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp)
*/
static void rcu_flavor_sched_clock_irq(int user)
{
- if (user || rcu_is_cpu_rrupt_from_idle()) {
+ if (user || rcu_is_cpu_rrupt_from_idle() ||
+ (IS_ENABLED(CONFIG_PREEMPT_COUNT) &&
+ !(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)))) {
/*
* Get here if this CPU took its interrupt from user
- * mode or from the idle loop, and if this is not a
- * nested interrupt. In this case, the CPU is in
- * a quiescent state, so note it.
+ * mode, from the idle loop without this being a nested
+ * interrupt, or while not holding a preempt count (but
+ * with PREEMPT_COUNT=y. In this case, the CPU is in a
+ * quiescent state, so note it.
*
* No memory barrier is required here because rcu_qs()
* references only CPU-local variables that other CPUs
--
2.43.5
On 2024-11-06 12:17:56 [-0800], Ankur Arora wrote: > With PREEMPT_RCU=n, cond_resched() provides urgently needed quiescent > states for read-side critical sections via rcu_all_qs(). > One reason why this was needed, was lacking preempt-count, the tick > handler has no way of knowing whether it is executing in a read-side > critical section or not. > > With PREEMPT_LAZY=y, there can be configurations with PREEMPT_COUNT=y, > PREEMPT_RCU=n, where cond_resched() is a stub that does not provide > quiescent states via rcu_all_qs(). With PREEMPT_LAZY=y && PREEMPT_DYNAMIC=n we get PREEMPT_COUNT=y and PREEMPT_RCU=n. In this configuration cond_resched() is an empty stub and does not provide quiescent states via rcu_all_qs(). PREEMPT_RCU=y provides this information via rcu_read_unlock() and its nesting counter. > So, use the availability of preempt_count() to report quiescent states > in rcu_flavor_sched_clock_irq(). Okay. You might also want to update the cond_resched() comment, s@In preemptible kernels, ->rcu_read_lock_nesting@ In PREEMPT_RCU kernels, ->rcu_read_lock_nesting@ Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> > Suggested-by: Paul E. McKenney <paulmck@kernel.org> > Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> Sebastian
Sebastian Andrzej Siewior <bigeasy@linutronix.de> writes: > On 2024-11-06 12:17:56 [-0800], Ankur Arora wrote: >> With PREEMPT_RCU=n, cond_resched() provides urgently needed quiescent >> states for read-side critical sections via rcu_all_qs(). >> One reason why this was needed, was lacking preempt-count, the tick >> handler has no way of knowing whether it is executing in a read-side >> critical section or not. >> >> With PREEMPT_LAZY=y, there can be configurations with PREEMPT_COUNT=y, >> PREEMPT_RCU=n, where cond_resched() is a stub that does not provide >> quiescent states via rcu_all_qs(). > > With PREEMPT_LAZY=y && PREEMPT_DYNAMIC=n we get PREEMPT_COUNT=y and > PREEMPT_RCU=n. In this configuration cond_resched() is an empty stub and > does not provide quiescent states via rcu_all_qs(). PREEMPT_RCU=y > provides this information via rcu_read_unlock() and its nesting counter. > >> So, use the availability of preempt_count() to report quiescent states >> in rcu_flavor_sched_clock_irq(). > > Okay. You might also want to update the cond_resched() comment, > s@In preemptible kernels, ->rcu_read_lock_nesting@ > In PREEMPT_RCU kernels, ->rcu_read_lock_nesting@ Good point. Will add. > Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Thanks! Ankur >> Suggested-by: Paul E. McKenney <paulmck@kernel.org> >> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com> > > Sebastian -- ankur
© 2016 - 2024 Red Hat, Inc.