[PATCH v4 0/4] workqueue: destroy_worker() vs isolated CPUs

Valentin Schneider posted 4 patches 3 years, 6 months ago
There is a newer version of this series
kernel/workqueue.c | 195 +++++++++++++++++++++++++++++++--------------
1 file changed, 136 insertions(+), 59 deletions(-)
[PATCH v4 0/4] workqueue: destroy_worker() vs isolated CPUs
Posted by Valentin Schneider 3 years, 6 months ago
Hi folks,

I haven't sent an update for this in a while, but the issue has risen again in
some other environment so I'm getting more reasons to push this out.

Revisions
=========

RFCv3 -> v4
+++++++++++

o Rebase onto v6.0
o Split into more patches for reviewability
o Take dying workers out of the pool->workers as suggested by Lai

RFCv2 -> RFCv3
++++++++++++++

o Rebase onto v5.19
o Add new patch (1/3) around accessing wq_unbound_cpumask

o Prevent WORKER_DIE workers for kfree()'ing themselves before the idle reaper
  gets to handle them (Tejun)

  Bit of an aside on that: I've been struggling to convince myself this can
  happen due to spurious wakeups and would like some help here.

  Idle workers are TASK_UNINTERRUPTIBLE, so they can't be woken up by
  signals. That state is set *under* pool->lock, and all wakeups (before this
  patch) are also done while holding pool->lock.
  
  wake_up_worker() is done under pool->lock AND only wakes a worker on the
  pool->idle_list. Thus the to-be-woken worker *cannot* have WORKER_DIE, though
  it could gain it *after* being woken but *before* it runs, e.g.:
                          
  LOCK pool->lock
  wake_up_worker(pool)
      wake_up_process(p)
  UNLOCK pool->lock
                          idle_reaper_fn()
                            LOCK pool->lock
                            destroy_worker(worker, list);
			    UNLOCK pool->lock
			                            worker_thread()
						      goto woke_up;
                                                      LOCK pool->lock
						      READ worker->flags & WORKER_DIE
                                                          UNLOCK pool->lock
                                                          ...
						          kfree(worker);
                            reap_worker(worker);
			        // Uh-oh
			  
  ... But IMO that's not a spurious wakeup, that's a concurrency issue. I don't
  see any spurious/unexpected worker wakeup happening once a worker is off the
  pool->idle_list.
  

RFCv1 -> RFCv2
++++++++++++++

o Change the pool->timer into a delayed_work to have a sleepable context for
  unbinding kworkers

Cheers,
Valentin

Lai Jiangshan (1):
  workqueue: Protects wq_unbound_cpumask with wq_pool_attach_mutex

Valentin Schneider (3):
  workqueue: Factorize unbind/rebind_workers() logic
  workqueue: Convert the idle_timer to a delayed_work
  workqueue: Unbind workers before sending them to exit()

 kernel/workqueue.c | 195 +++++++++++++++++++++++++++++++--------------
 1 file changed, 136 insertions(+), 59 deletions(-)

--
2.31.1
Re: [PATCH v4 0/4] workqueue: destroy_worker() vs isolated CPUs
Posted by Valentin Schneider 3 years, 5 months ago
On 04/10/22 16:05, Valentin Schneider wrote:
> Hi folks,
>
> I haven't sent an update for this in a while, but the issue has risen again in
> some other environment so I'm getting more reasons to push this out.

Gentle ping.
Re: [PATCH v4 0/4] workqueue: destroy_worker() vs isolated CPUs
Posted by Tejun Heo 3 years, 5 months ago
On Tue, Oct 25, 2022 at 10:42:46AM +0100, Valentin Schneider wrote:
> On 04/10/22 16:05, Valentin Schneider wrote:
> > Hi folks,
> >
> > I haven't sent an update for this in a while, but the issue has risen again in
> > some other environment so I'm getting more reasons to push this out.
> 
> Gentle ping.

The approach looks fine to me. I only have minor comments. I'll reply to
each patch.

Lai, you've been involved in this effort in earlier patch series, how does
this look to you now?

Thanks.

-- 
tejun