When force_qs_rnp() forces quiescent states for idle or offline CPUs,
any tasks blocked on those CPUs' per-CPU blocked lists must first be
promoted to the rcu_node's blkd_tasks list.
Without this promotion, blocked tasks on per-CPU lists won't have
gp_tasks point to them, so the GP machinery won't wait for them. This
can cause "Wrong-GP reads" errors where a GP completes while readers
are still in their critical sections.
Therefore, call rcu_promote_blocked_tasks_rdp() before reporting QS.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
kernel/rcu/tree.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 2a20b1a8c5d3..19fd13c1e6be 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2790,6 +2790,11 @@ static void force_qs_rnp(int (*f)(struct rcu_data *rdp))
rdp = per_cpu_ptr(&rcu_data, cpu);
ret = f(rdp);
if (ret > 0) {
+ /*
+ * Promote blocked tasks before reporting QS.
+ * Otherwise tasks on per-CPU list aren't tracked.
+ */
+ rcu_promote_blocked_tasks_rdp(rdp, rnp);
mask |= rdp->grpmask;
rcu_disable_urgency_upon_qs(rdp);
}
--
2.34.1