From nobody Tue Dec 16 19:40:42 2025 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E51F16C436 for ; Fri, 21 Jun 2024 07:29:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718954987; cv=none; b=RiggYQt0yXbfktlUlWTw7uWfYtLYEX/aYe3guuejHcSErMoldR0pFawx8Sx1+E9ftfZEdyQ4GLWlSOXf33JSEBiy6+UOcHdwYlQYlOLg7HW7HNlykcBgu3JrvX2TtzZSpp1iRuWz24YLZnrgcMAIRwJ+8K8YzCrUCqKB7+6RS40= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718954987; c=relaxed/simple; bh=jUbJuRYowq+lTNk/JeWk1wlSh7O1av6rtsq6MZo7vJQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dDHXNdYH4Ske9xuCezStuDSEUspaNAW4+M73cgLPylpvn5Gyq1Qf8lr/RUaTvwfJ9KXXr9Hqr/EEbUPIlKsWnZ2cLVJ2qiYf5WcqC52I+7OB281jGRaSgzY1HJ29lqnyvsHAW/XsXAGGJi8RVgN8YbCvTg0CUxSshsq8Z9otlEQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mEediqK5; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mEediqK5" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-7042a8ad9f5so1709427b3a.0 for ; Fri, 21 Jun 2024 00:29:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718954985; x=1719559785; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5RM/r+VPbqDnzVngpECuz1flcgt2geIdwHZBnUvHHTQ=; b=mEediqK5VjW0ZZ5yLPONKO/nSskzyU3uA+a9RqUEsXajQl4IKfnZPv1Ef2IkJfufBs rE4+NDVbTcY8eFln9Pqsw96UG90uFVlRtCHzDQLGLzMySZRvvcrGjtkVjqd2fgZSVOlP SBjr80jaXxoupNiWAoag1o7o/4LtBhYovq5FoCuvHOJXVPShjT3XIPwbftpsSMleGtgX QAifjv0GKbztIODq5kgmEDYwFkHblpulY+MrIIeFy5OEDGEd0xkH1b4FOkFvEzIf1bpJ SHKZjqI7mzB+6GoCS3SeA66rz1h84GzlnGspri8fpxPlo9jdjnckuTiTxcltzIA0ko8f gI3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718954985; x=1719559785; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5RM/r+VPbqDnzVngpECuz1flcgt2geIdwHZBnUvHHTQ=; b=LxFet+r9bOoG1Y6Ao25LvbmUT4GE4uDm3iCml09/0ay/1XNNMKVS8Or7kAZBDeVzHG E18qG5Li+hKbV1nrM+OcjMjJVcb/7mGJ+7U57+BkRhUTxmGk7JgBQrJ7BLcrewfK58im 3P8i4n4bqIz4St4sPWzeA6vBwBqbMz/0fhKGkxwtbMk/gAXgLtENEwh/+MFRNbTaJlHj n0xkYXOTT4L0j4rze0Valy/rh2owbhtyBaVWb5sxI0skkWNrL0l/C/4XipwNElX4FJno hDSyv2cwScIy5xf6ub8YyjqcC3MarmWcBS4JnvhE2tZ02NzFqZYiKcS3gKngzLYljb8H WDcQ== X-Gm-Message-State: AOJu0Yx88CrkBKngFwqhS/byAvNzo+P3bRbS2sJ4w5lRms3EWswuG0O0 vL8LX998yyHoVbGmGNnX5/qJyZezN0k9VJZKHJmSV4RhYSHExVm5DjbKZA== X-Google-Smtp-Source: AGHT+IFuwj1HMWnohHgc82cK/EZthf5aN4oaCsTbAnAxrkg0nHbdOgJcXrnVqnLZfv6QORdpNslrMQ== X-Received: by 2002:a17:90b:f82:b0:2c8:631:2f53 with SMTP id 98e67ed59e1d1-2c806313090mr4062787a91.7.1718954985202; Fri, 21 Jun 2024 00:29:45 -0700 (PDT) Received: from localhost ([47.88.5.130]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c7e55dcdffsm2934129a91.33.2024.06.21.00.29.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2024 00:29:44 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Valentin Schneider , Tejun Heo , Lai Jiangshan Subject: [PATCH 1/4] workqueue: Reap workers via kthread_stop() and remove detach_completion Date: Fri, 21 Jun 2024 15:32:22 +0800 Message-Id: <20240621073225.3600-2-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240621073225.3600-1-jiangshanlai@gmail.com> References: <20240621073225.3600-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan The code to kick off the destruction of workers is now in a process context (idle_cull_fn()), so kthread_stop() can be used in the process context to replace the work of pool->detach_completion. The wakeup in wake_dying_workers() is unneeded after this change, but it is harmless, jut keep it here until next patch renames wake_dying_workers() rather than renaming it again and again. Cc: Valentin Schneider Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 25a7d9a1a7ae..a0fb2f60e938 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -216,7 +216,6 @@ struct worker_pool { struct worker *manager; /* L: purely informational */ struct list_head workers; /* A: attached workers */ struct list_head dying_workers; /* A: workers about to die */ - struct completion *detach_completion; /* all workers detached */ =20 struct ida worker_ida; /* worker IDs for task name */ =20 @@ -2696,7 +2695,6 @@ static void worker_attach_to_pool(struct worker *work= er, static void worker_detach_from_pool(struct worker *worker) { struct worker_pool *pool =3D worker->pool; - struct completion *detach_completion =3D NULL; =20 /* there is one permanent BH worker per CPU which should never detach */ WARN_ON_ONCE(pool->flags & POOL_BH); @@ -2707,15 +2705,10 @@ static void worker_detach_from_pool(struct worker *= worker) list_del(&worker->node); worker->pool =3D NULL; =20 - if (list_empty(&pool->workers) && list_empty(&pool->dying_workers)) - detach_completion =3D pool->detach_completion; mutex_unlock(&wq_pool_attach_mutex); =20 /* clear leftover flags without pool->lock after it is detached */ worker->flags &=3D ~(WORKER_UNBOUND | WORKER_REBOUND); - - if (detach_completion) - complete(detach_completion); } =20 /** @@ -2816,10 +2809,9 @@ static void unbind_worker(struct worker *worker) =20 static void wake_dying_workers(struct list_head *cull_list) { - struct worker *worker, *tmp; + struct worker *worker; =20 - list_for_each_entry_safe(worker, tmp, cull_list, entry) { - list_del_init(&worker->entry); + list_for_each_entry(worker, cull_list, entry) { unbind_worker(worker); /* * If the worker was somehow already running, then it had to be @@ -2835,6 +2827,17 @@ static void wake_dying_workers(struct list_head *cul= l_list) } } =20 +static void reap_dying_workers(struct list_head *cull_list) +{ + struct worker *worker, *tmp; + + list_for_each_entry_safe(worker, tmp, cull_list, entry) { + list_del_init(&worker->entry); + kthread_stop_put(worker->task); + kfree(worker); + } +} + /** * set_worker_dying - Tag a worker for destruction * @worker: worker to be destroyed @@ -2866,6 +2869,9 @@ static void set_worker_dying(struct worker *worker, s= truct list_head *list) =20 list_move(&worker->entry, list); list_move(&worker->node, &pool->dying_workers); + + /* get an extra task struct reference for later kthread_stop_put() */ + get_task_struct(worker->task); } =20 /** @@ -2949,6 +2955,8 @@ static void idle_cull_fn(struct work_struct *work) raw_spin_unlock_irq(&pool->lock); wake_dying_workers(&cull_list); mutex_unlock(&wq_pool_attach_mutex); + + reap_dying_workers(&cull_list); } =20 static void send_mayday(struct work_struct *work) @@ -3330,7 +3338,6 @@ static int worker_thread(void *__worker) ida_free(&pool->worker_ida, worker->id); worker_detach_from_pool(worker); WARN_ON_ONCE(!list_empty(&worker->entry)); - kfree(worker); return 0; } =20 @@ -4863,7 +4870,6 @@ static void rcu_free_pool(struct rcu_head *rcu) */ static void put_unbound_pool(struct worker_pool *pool) { - DECLARE_COMPLETION_ONSTACK(detach_completion); struct worker *worker; LIST_HEAD(cull_list); =20 @@ -4917,12 +4923,9 @@ static void put_unbound_pool(struct worker_pool *poo= l) =20 wake_dying_workers(&cull_list); =20 - if (!list_empty(&pool->workers) || !list_empty(&pool->dying_workers)) - pool->detach_completion =3D &detach_completion; mutex_unlock(&wq_pool_attach_mutex); =20 - if (pool->detach_completion) - wait_for_completion(pool->detach_completion); + reap_dying_workers(&cull_list); =20 /* shut down the timers */ del_timer_sync(&pool->idle_timer); --=20 2.19.1.6.gb485710b From nobody Tue Dec 16 19:40:42 2025 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E3C016B3B4 for ; Fri, 21 Jun 2024 07:29:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718954991; cv=none; b=fwqcOFtxfmv8hkyQchEIOSz+zY2gI2yX9VVaW3Kz3FVPKHTBspgBlx3nLTVALTMjz/cmeVpxdZB+Qq6h22U1U13uM9PJWu2fC+ygze4v/f4ZxPWj9WXEB8j9NiTtE15Dl4xbk83oD/QaposVYjX2OVytmlSBPbO1FJRDKvHc4K0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718954991; c=relaxed/simple; bh=t+o9R0mNnCdZay49vfQTof+RAzBGQx5oc44gyH1HGnk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tjD89UmS0IJT3+/tTN4rufmh6WXHpGJDAO3RMN5PZ1UA0LQf3ELCGVlbZmi23ceiEuca4CurXg+91Qh17xeugYpgYGuoUFEgPI7P/Z6RHDg1NxLAnl0oQfBVS24Q/RNDjnTTy2qV4xiMfD7R2vc7hzsK5+9e5kx7/Yju2vcdtxM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Qxtwxufi; arc=none smtp.client-ip=209.85.216.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Qxtwxufi" Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-2c5362c7c0bso1336476a91.1 for ; Fri, 21 Jun 2024 00:29:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718954989; x=1719559789; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v6HUCjy2fc0wQf7MHhi0pPhuuSm7QAtjhyc2K8rqy84=; b=QxtwxufiPln94JjFTJgYRN1m0i6FL7MPn+kbyB+6gDg0s0g0aNBJqckmGvFgNYG3Bz Pk3zKBnvYsoZu3qCbThbEGHa+yiD7Gm1A7auymkMfOzJi7HpM2FAVh3PJ3pg/nGKr2ej ExlcfL/+2AJAr0y4I2TBrogTNF9cy0FLBIpjTtVa/7HEzymKnCs2woKwQ1wPYzixG7KO 6OEAY18sABpKZfSZnzP8zpCTDFZPdCKoaMUqP8PcDNbjdnsiV0uWF0VgNOkcAEKOk0f0 D0UOoZ8eG/WDVhDeDc2oHuB6Z8TRuaMgzbSB8PZcOezmi8lEijtJzy72RSbWPio0M6Dr e7oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718954989; x=1719559789; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v6HUCjy2fc0wQf7MHhi0pPhuuSm7QAtjhyc2K8rqy84=; b=I66q8JCTtWAysnRjK3vKeod8xnsfqGZGpz6KPmaChP+WcILa8GGgP7TZFVlppFwmog XB8/9w+pPt9bYR0ejGMxpi8mJzRi5bUYhEb8lRJ6O6SyGUalZXITYnoYA2ZwPgMkre9R ffgcT49UCuT0AOZGpGXs5yf1D+55Tt89/voMeYsvUq8ICZJ+XIL7iiknHV48hTLFnfwR IK0EUAeBNwXkXZwtmVnLI5EXgo7wrEcUSXDEQXx/wrB1Uv/1MpQpYQKamYiSaxwZaJkA yrFtHhjB5BhF3Tc4ZeeWiaeYAdrcr9g7TYZ0EK9uKl/GGaxa3xygBkQfZNwmbie8VQ+M wAtg== X-Gm-Message-State: AOJu0Yy0Ko+MxgcY2LwWxxij2PE4X9C71fyf9uzEvc7++OASykE0pEOS 8kBt8x6OICUTShU237FvYHS0rS1KvRJkdzjK8SowJZnWSD/LBmmlG6ImlQ== X-Google-Smtp-Source: AGHT+IHnpvVlCKjRyfRNqPE89ocyzJAtwCyhvKjGEPiIvBrRi8Ab+hy3P0d0WM7BrX5UcnSuBgY6YQ== X-Received: by 2002:a17:90b:378f:b0:2c2:d0cc:3b9e with SMTP id 98e67ed59e1d1-2c7b5d95a9bmr6929812a91.46.1718954988905; Fri, 21 Jun 2024 00:29:48 -0700 (PDT) Received: from localhost ([47.88.5.130]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c819db9506sm885063a91.44.2024.06.21.00.29.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2024 00:29:48 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Valentin Schneider , Tejun Heo , Lai Jiangshan Subject: [PATCH 2/4] workqueue: Don't bind the rescuer in the last working cpu Date: Fri, 21 Jun 2024 15:32:23 +0800 Message-Id: <20240621073225.3600-3-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240621073225.3600-1-jiangshanlai@gmail.com> References: <20240621073225.3600-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan So that when the rescuer is woken up next time, it will not interrupt the last working cpu which might be busy on other crucial works but have nothing to do with the rescuer's incoming works. Cc: Valentin Schneider Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index a0fb2f60e938..93b87ca63a7e 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2684,6 +2684,17 @@ static void worker_attach_to_pool(struct worker *wor= ker, mutex_unlock(&wq_pool_attach_mutex); } =20 +static void unbind_worker(struct worker *worker) +{ + lockdep_assert_held(&wq_pool_attach_mutex); + + kthread_set_per_cpu(worker->task, -1); + if (cpumask_intersects(wq_unbound_cpumask, cpu_active_mask)) + WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, wq_unbound_cpumask) < 0); + else + WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0); +} + /** * worker_detach_from_pool() - detach a worker from its pool * @worker: worker which is attached to its pool @@ -2701,7 +2712,7 @@ static void worker_detach_from_pool(struct worker *wo= rker) =20 mutex_lock(&wq_pool_attach_mutex); =20 - kthread_set_per_cpu(worker->task, -1); + unbind_worker(worker); list_del(&worker->node); worker->pool =3D NULL; =20 @@ -2796,17 +2807,6 @@ static struct worker *create_worker(struct worker_po= ol *pool) return NULL; } =20 -static void unbind_worker(struct worker *worker) -{ - lockdep_assert_held(&wq_pool_attach_mutex); - - kthread_set_per_cpu(worker->task, -1); - if (cpumask_intersects(wq_unbound_cpumask, cpu_active_mask)) - WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, wq_unbound_cpumask) < 0); - else - WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0); -} - static void wake_dying_workers(struct list_head *cull_list) { struct worker *worker; --=20 2.19.1.6.gb485710b From nobody Tue Dec 16 19:40:42 2025 Received: from mail-oi1-f181.google.com (mail-oi1-f181.google.com [209.85.167.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C255616B748 for ; Fri, 21 Jun 2024 07:29:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718954996; cv=none; b=IDRAX8pzo2e9u2ymyKcoVHfcuMLLGtfm6mxh5hwpGjO8TOVf8EYz/uuhGtDWZ4j6dKZmGbTYUeU0ddsNvSIP2B9HHB5f939fyY6FWATmk7Q/VoJF/dOuOPf23jIKxBcphsUHQwBcgBTGtTYycqJ0WeAc9d4hNzxisME56PiCh3o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718954996; c=relaxed/simple; bh=FZc60W1D4prLgx9i7Jc3Zhf9Nh67kWwv4Lplh2hE0Vk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=d4YazM80S7m0zYmEajlcfQ6LgrlmMdlDYCE9Hi9u588cp6LdfoPrdvbWNY4bEen/T2fW+9naH9ARyH0P6PzGG8D0G5Gj3BZ0BpIgu2qWu9ZgnI+Acpk8SCOUFSlU+J9iIFI6ZcyCZ2ponWnlQO0hpaCX6m6vkqpXp8hOIyPpHc8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kC84yBZi; arc=none smtp.client-ip=209.85.167.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kC84yBZi" Received: by mail-oi1-f181.google.com with SMTP id 5614622812f47-3d4430b8591so892664b6e.2 for ; Fri, 21 Jun 2024 00:29:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718954993; x=1719559793; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3DA4WJliprSRDwpu0SuE5J5N4pXsY0CSpLvln4uW1pg=; b=kC84yBZi3BL5SmayDPIADrSROJs3tTshvaRfVTTfLHZZPblWNrPPVSxhpSt47Gp94P MksOQo7FP4vMOtAfFTKs1Np5k/5re3Tc0vSLlH7Z7aC2H0g1ec5v8YGe8n5KZGenqtz+ UDCmJBmHtUPpum6dJuaZOPHRUcqmimX13np4dOH/hvE7PIPS54mRP2SCoaWZqHTeC/IQ xtkaVW65V0rj/3R6TakKJO2mJWs4a8DxdC7U9kC1IJ4Hue+tmfhvk6Lvwk9oL5NW+Cmv nhJ/utnKXxbiGGGsxUtuQ8ip/ffs+GbQQvZVWwmzYt03gobz0VmXs/Jm9pAv2C6MRTlb RNJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718954993; x=1719559793; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3DA4WJliprSRDwpu0SuE5J5N4pXsY0CSpLvln4uW1pg=; b=plFtzYASsQftJqa6cu1pjT//Hl82eE0FamGDMnJzURPonONnrio0Hpfl/2XkUU0WPm hp+p9Y+AEwry/Kh2Bi6cgeZBqrTz7RdkNsTzi2R99iP3PpUPzXZYIdXJS7PFGPppi5Pf 65IEPQy8AH0dX3kACvg9G3LM1cG7OoVTVzrvXfJZzXpLVYbUpeP9JTHf/wj9+yYAZJWR GSkigwzPtxrBeAI3q0D0ebMTyZtLGOZ/u4qNpMIoH3/c67vY21RIlT+4gH5DPM38rAZ1 7z0wrNzkG3ORTecJNz3AiY0OwLg+k0EZIdv7m8C2TPqUezedTg2nJGQVVqEN6emazhjY 1N4g== X-Gm-Message-State: AOJu0YyPA8OXgbZjsK36ZeVv+mHKTas6+/KAqporaiVWfReaw3510K8m gxaVMYfO9hsXSHKgGLcFOQ4Q7kHKruGvyjFWjB9/lS3qNSKdoynMws8dhA== X-Google-Smtp-Source: AGHT+IEc+rVG2JNPVSkQ2x7EL2wuL8Atz66faM4+WF1ldSAzcZ5e2TaCJ1vindP1viHUywzsKT4AJw== X-Received: by 2002:a05:6808:13d6:b0:3d2:1c91:86e8 with SMTP id 5614622812f47-3d51baea0c7mr8575872b6e.45.1718954993173; Fri, 21 Jun 2024 00:29:53 -0700 (PDT) Received: from localhost ([198.11.176.14]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-706511945f8sm765540b3a.69.2024.06.21.00.29.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2024 00:29:52 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Valentin Schneider , Tejun Heo , Lai Jiangshan Subject: [PATCH 3/4] workqueue: Detach workers directly in idle_cull_fn() Date: Fri, 21 Jun 2024 15:32:24 +0800 Message-Id: <20240621073225.3600-4-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240621073225.3600-1-jiangshanlai@gmail.com> References: <20240621073225.3600-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan The code to kick off the destruction of workers is now in a process context (idle_cull_fn()), and the detaching of a worker is not required to be inside the worker thread now, so just do the detaching directly in idle_cull_fn(). wake_dying_workers() is renamed to detach_dying_workers() and the unneeded wakeup in wake_dying_workers() is also removed. Cc: Valentin Schneider Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 45 +++++++++++++++++++-------------------------- 1 file changed, 19 insertions(+), 26 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 93b87ca63a7e..e9ca42ef425a 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2695,6 +2695,16 @@ static void unbind_worker(struct worker *worker) WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0); } =20 + +static void detach_worker(struct worker *worker) +{ + lockdep_assert_held(&wq_pool_attach_mutex); + + unbind_worker(worker); + list_del(&worker->node); + worker->pool =3D NULL; +} + /** * worker_detach_from_pool() - detach a worker from its pool * @worker: worker which is attached to its pool @@ -2711,11 +2721,7 @@ static void worker_detach_from_pool(struct worker *w= orker) WARN_ON_ONCE(pool->flags & POOL_BH); =20 mutex_lock(&wq_pool_attach_mutex); - - unbind_worker(worker); - list_del(&worker->node); - worker->pool =3D NULL; - + detach_worker(worker); mutex_unlock(&wq_pool_attach_mutex); =20 /* clear leftover flags without pool->lock after it is detached */ @@ -2807,24 +2813,12 @@ static struct worker *create_worker(struct worker_p= ool *pool) return NULL; } =20 -static void wake_dying_workers(struct list_head *cull_list) +static void detach_dying_workers(struct list_head *cull_list) { struct worker *worker; =20 - list_for_each_entry(worker, cull_list, entry) { - unbind_worker(worker); - /* - * If the worker was somehow already running, then it had to be - * in pool->idle_list when set_worker_dying() happened or we - * wouldn't have gotten here. - * - * Thus, the worker must either have observed the WORKER_DIE - * flag, or have set its state to TASK_IDLE. Either way, the - * below will be observed by the worker and is safe to do - * outside of pool->lock. - */ - wake_up_process(worker->task); - } + list_for_each_entry(worker, cull_list, entry) + detach_worker(worker); } =20 static void reap_dying_workers(struct list_head *cull_list) @@ -2930,9 +2924,9 @@ static void idle_cull_fn(struct work_struct *work) =20 /* * Grabbing wq_pool_attach_mutex here ensures an already-running worker - * cannot proceed beyong worker_detach_from_pool() in its self-destruct - * path. This is required as a previously-preempted worker could run after - * set_worker_dying() has happened but before wake_dying_workers() did. + * cannot proceed beyong set_pf_worker() in its self-destruct path. + * This is required as a previously-preempted worker could run after + * set_worker_dying() has happened but before detach_dying_workers() did. */ mutex_lock(&wq_pool_attach_mutex); raw_spin_lock_irq(&pool->lock); @@ -2953,7 +2947,7 @@ static void idle_cull_fn(struct work_struct *work) } =20 raw_spin_unlock_irq(&pool->lock); - wake_dying_workers(&cull_list); + detach_dying_workers(&cull_list); mutex_unlock(&wq_pool_attach_mutex); =20 reap_dying_workers(&cull_list); @@ -3336,7 +3330,6 @@ static int worker_thread(void *__worker) =20 set_task_comm(worker->task, "kworker/dying"); ida_free(&pool->worker_ida, worker->id); - worker_detach_from_pool(worker); WARN_ON_ONCE(!list_empty(&worker->entry)); return 0; } @@ -4921,7 +4914,7 @@ static void put_unbound_pool(struct worker_pool *pool) WARN_ON(pool->nr_workers || pool->nr_idle); raw_spin_unlock_irq(&pool->lock); =20 - wake_dying_workers(&cull_list); + detach_dying_workers(&cull_list); =20 mutex_unlock(&wq_pool_attach_mutex); =20 --=20 2.19.1.6.gb485710b From nobody Tue Dec 16 19:40:42 2025 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E91316B75B for ; Fri, 21 Jun 2024 07:29:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718954999; cv=none; b=VHYfqiOgrslxJqtmOSDMr9AxJ+EL6GTI0+8LbMNp4b7Wk3eIY4WLRMT825RmCWI5atX4JLzhpI32zdSQB4dy/y+e490DbxdtqxxcTeVuz8jZcUraCJvxIubj7p4S+Y2cVx8bhnBBa6DUmw3n/F9EhlSAhsnhDic4FkAflnDBnfM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718954999; c=relaxed/simple; bh=YjU2g1+wutM3QV7zQCcyIGcj/Ah5HqR145W3+dp7nGE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=O1/2Ea3DrPJ6iqtasiXwekOo9JTTL5AXR8u1b1daS9iZxn/IB/sl3VwD2265330kq+ZSS6rQxHr15qSIEJoZqhgdNRLKXpopVcnfFzAUyopW312v3lYzK45D4Ce3PlU7IiC/mrTPW9kVKOt4LhwYrMqr4viqzvNYINQa6yTanE8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=edU3hlkR; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="edU3hlkR" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-705fff50de2so1200316b3a.1 for ; Fri, 21 Jun 2024 00:29:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718954997; x=1719559797; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NeZC5A5/V+xzjKS1B6H8sOnZGAk45J9pzGB5ap2eybo=; b=edU3hlkRNJ7EquW52ZmX92KnrRKIVc8UwJXloElbUi8sh0vu1eC1NeWy2T8pp+BtkQ 3kPZXmjnp+EL/er4m1xtlGS3VKNN6aL80LzAtDYXkJK7vUT63jyKC5Asa/FHAe8BWK2j YdBUjr+Xp0Dl+IXKHbfvsS0izeNNZy/HCBfdys9QN2xI3YT+rmC2jiu+sxrftbyCVuIN MN1c7UAMDs+ryaK9HzaPtr82JLWONIkXKRb4PQf7wXU2GT8jzjnOUjqQM47ykbzjCJzO tF9LZS7W48gAcEn/h96DQBRSK8vEaVOON9RYZIlJg8TImuCzFChU7nrBP+cDd5zRxrQs atSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718954997; x=1719559797; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NeZC5A5/V+xzjKS1B6H8sOnZGAk45J9pzGB5ap2eybo=; b=cnFOc21tVz6MzrVVsX6uEO0/Rc8iS2Py+WO9cwY7xPL0DEdaLQ1qC5ZhAIL9VtjqhU HzF284MffZibC87ug/uDhlPrANrk854FiVMtjEwgiOVWhSu/T4xuxq0Zk7URudMw5hpa t/9MmZB6sj7susHMA2LxXKy4wPgGLN7AtDEFXOmJttUcnE/BTzISWpNbwi1YS9+tC064 qmXmHOuuUPm7Sja9OQze1hH2FhKjoTtTd0OhSjUSvSfmSqd3hITO+944zoyS0byqkIp3 rGsxqMdfTDCeDGe8biBil6fNfwU6Wkov47PGetFXeu0YFlpRdCve2VsohbOshahHh331 aS6Q== X-Gm-Message-State: AOJu0YyjSJrxdhb67dzZQdAhzIcC7k2c/4bWU0QIqSYeoooAPToGv3NP DkEegvxboBs+zmyAW7jY38JQJt0xCdk/9/fgeHgJbtIgOyCbSrv3exX7ew== X-Google-Smtp-Source: AGHT+IEEm0KzKTm3QDcstJ8dzqqFHta5cOn9wM2z2Po/BXblNDT78pefyekYmDwGwgapjoYIbGUoQg== X-Received: by 2002:a62:e309:0:b0:706:1ff1:c5ff with SMTP id d2e1a72fcca58-70628f770f3mr10844624b3a.1.1718954996685; Fri, 21 Jun 2024 00:29:56 -0700 (PDT) Received: from localhost ([198.11.178.15]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-706512cb0d8sm757616b3a.167.2024.06.21.00.29.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2024 00:29:56 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Valentin Schneider , Tejun Heo , Lai Jiangshan Subject: [PATCH 4/4] workqueue: Remove useless pool->dying_workers Date: Fri, 21 Jun 2024 15:32:25 +0800 Message-Id: <20240621073225.3600-5-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240621073225.3600-1-jiangshanlai@gmail.com> References: <20240621073225.3600-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan A dying worker is first moved from pool->workers to pool->dying_workers in set_worker_dying() and removed from pool->dying_workers in detach_dying_workers(). The whole procedure is in the some lock context of wq_pool_attach_mutex. So pool->dying_workers is useless, just remove it and keep the dying worker in pool->workers after set_worker_dying() and remove it in detach_dying_workers() with wq_pool_attach_mutex held. Cc: Valentin Schneider Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index e9ca42ef425a..dc9acf8ecd0c 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -215,7 +215,6 @@ struct worker_pool { =20 struct worker *manager; /* L: purely informational */ struct list_head workers; /* A: attached workers */ - struct list_head dying_workers; /* A: workers about to die */ =20 struct ida worker_ida; /* worker IDs for task name */ =20 @@ -2862,7 +2861,6 @@ static void set_worker_dying(struct worker *worker, s= truct list_head *list) worker->flags |=3D WORKER_DIE; =20 list_move(&worker->entry, list); - list_move(&worker->node, &pool->dying_workers); =20 /* get an extra task struct reference for later kthread_stop_put() */ get_task_struct(worker->task); @@ -4721,7 +4719,6 @@ static int init_worker_pool(struct worker_pool *pool) timer_setup(&pool->mayday_timer, pool_mayday_timeout, 0); =20 INIT_LIST_HEAD(&pool->workers); - INIT_LIST_HEAD(&pool->dying_workers); =20 ida_init(&pool->worker_ida); INIT_HLIST_NODE(&pool->hash_node); --=20 2.19.1.6.gb485710b