From nobody Sat Apr 11 22:45:42 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E99AC19F2B for ; Thu, 4 Aug 2022 08:40:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238735AbiHDIk5 (ORCPT ); Thu, 4 Aug 2022 04:40:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46274 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239292AbiHDIkr (ORCPT ); Thu, 4 Aug 2022 04:40:47 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A016A25C72 for ; Thu, 4 Aug 2022 01:40:46 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id s5-20020a17090a13c500b001f4da9ffe5fso4566032pjf.5 for ; Thu, 04 Aug 2022 01:40:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=GWE4Fsz2e2PmZ/o8Txibq+ksrvgnJCJQRSWGkgTYSfc=; b=jRQtZTZHdvZ1yokGaY/qgydPXnr+tpDIgguOusl+c7o6l95fYptk60YNC5t5kffWBN 0ra0lZxDxOzwD+sSWe0VSbTylxL+KC276B+J9OOsUxKjHEMUSm5q5jp/Rp9IIy6QUPqb OIrlJ9EI7W4in7kUk0cSqW8H24Kl6W+H3A+SN6c0K7JXvCT/+vyXQZIL7uTmGG2/vadY ilkNEJ3LNwjJPzCFI+qt2OlDqPBiP95+2vEFgqNBd9+TWNVMUlsk2g4OOPUCZTc85g5k XXB5dlyC2Y6zrp6E6zXiij1ropgl2q8HnWut3PaPpt9w+8jAapY1d51WAMip8bGr0Qn4 W0UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=GWE4Fsz2e2PmZ/o8Txibq+ksrvgnJCJQRSWGkgTYSfc=; b=c9EOBmcGGjS9USBglBo20pn19QwyHtvn+MpH3tWrvKhUQot4djxOStK14vYuUiTcnP Co4/Zera1+pJn5kP/wQLIwrpJRxb/oepeggECkIMYxDBIgFAW8W7pPyUzZMgTxb2Ve2w IJN2hHq5gq8xs2D0ln7MldrDdtDyY/tDDcPv5hJ7njZgp6MrkHkLXxuR+qiEHt+U0GlO +xgyqqX86Hw9E1r3R8czb0dYiCEl4ORiP7cH8zTC4bv4tSe0L2MB5DqQMEy90FEbrmZt d8SU3+WQU7Nb5uhXudI3zTDVsRPodRWJhasGeXH2vvuh8bgI2h9sqtWH5jCPFK3XD/b+ nAqg== X-Gm-Message-State: ACgBeo3Xem/bZUMrB5Y/g4hdB8QwAM0P4dYU6SsU9ppbscZvQdrhuDFb KBIcYe9zohma2yApzgQ3BZDVcS5yDXs= X-Google-Smtp-Source: AA6agR582QSlqLWTdGHZ+HcgtOlZaPGNAw8Z0fIzfT0l0JmeESTdTeQ8i8g1DsAs+Nd6YILAZ6b0TQ== X-Received: by 2002:a17:90b:3944:b0:1f5:4cc8:d46 with SMTP id oe4-20020a17090b394400b001f54cc80d46mr914418pjb.95.1659602445865; Thu, 04 Aug 2022 01:40:45 -0700 (PDT) Received: from localhost ([47.89.225.180]) by smtp.gmail.com with ESMTPSA id c78-20020a624e51000000b0052ce1737ee1sm377492pfb.37.2022.08.04.01.40.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Aug 2022 01:40:45 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Linus Torvalds , "Eric W. Biederman" , Tejun Heo , Petr Mladek , Michal Hocko , Peter Zijlstra , Wedson Almeida Filho , Lai Jiangshan , Valentin Schneider Subject: [RFC PATCH 1/8] workqueue: Unconditionally set cpumask in worker_attach_to_pool() Date: Thu, 4 Aug 2022 16:41:28 +0800 Message-Id: <20220804084135.92425-2-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20220804084135.92425-1-jiangshanlai@gmail.com> References: <20220804084135.92425-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan If a worker is spuriously woken up after kthread_bind_mask() but before worker_attach_to_pool(), and there are some cpu-hot-[un]plug happening during the same interval, the worker task might be pushed away from its bound CPU with its affinity changed by the scheduler and worker_attach_to_p= ool() doesn't rebind it properly. Do unconditionally affinity binding in worker_attach_to_pool() to fix the problem. Prepare for moving worker_attach_to_pool() from create_worker() to the starting of worker_thread() which will really cause the said interval even without spurious wakeup. Cc: Linus Torvalds Cc: "Eric W. Biederman" Cc: Tejun Heo , Cc: Petr Mladek Cc: Michal Hocko , Cc: Peter Zijlstra , Cc: Wedson Almeida Filho Fixes: 640f17c82460 ("workqueue: Restrict affinity change to rescuer") Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 1ea50f6be843..928aad7d6123 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1872,8 +1872,11 @@ static void worker_attach_to_pool(struct worker *wor= ker, else kthread_set_per_cpu(worker->task, pool->cpu); =20 - if (worker->rescue_wq) - set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask); + /* + * set_cpus_allowed_ptr() will fail if the cpumask doesn't have any + * online CPUs. It'll be re-applied when any of the CPUs come up. + */ + set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask); =20 list_add_tail(&worker->node, &pool->workers); worker->pool =3D pool; --=20 2.19.1.6.gb485710b From nobody Sat Apr 11 22:45:42 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C46CC19F2B for ; Thu, 4 Aug 2022 08:41:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239485AbiHDIlR (ORCPT ); Thu, 4 Aug 2022 04:41:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239472AbiHDIky (ORCPT ); Thu, 4 Aug 2022 04:40:54 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E95196555B for ; Thu, 4 Aug 2022 01:40:50 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id iw1so18716256plb.6 for ; Thu, 04 Aug 2022 01:40:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=vpxp3Y0BUl7k1Sv2yf06jUzgIsGTyq0WB4+2EoOmdIE=; b=L6sYZu3JbPkBfj4VwzAkVoJzbP+Il6yh8lyxMDDPavhRCJdZqNaas1+erN7EIW9tL7 wcC3/hcgL1gV8s6qmpNCOv92QKGK6ePa81LQsLp8zg28yxKR2bLoPd4vRcLDn8FUOjvW 8zMxRZAW5tVcV0vLRYsmqNbfcXhcYwj7Ql224xGEQpXuuYBYwplxwHY7bw4LYzaJHmae NEIj4sD2ePPtyBuA4E4iNuyj6C5GPtdsghjkgOxPIibZUl5xq5CQ4NfEoy7lUADPkdKA gnZ1+0XjhbK3aetpPOAu5dBqrYf2F8K4S9fcqrf3RyRRCWmjztfKQ5NReYcJqGR06BkR UkfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=vpxp3Y0BUl7k1Sv2yf06jUzgIsGTyq0WB4+2EoOmdIE=; b=yMNVbI+UGnz6KbOUPJGAdh9RnGTCvhilvC5ygmTGTuJRQ1v5V0RQ5Pn9TOFC5LDzDG 7z+mQcDYChW3pIsKK3YoR+zLCeRxsmzG4+r7Y8lYnF3JeG6HUnnL4pD0EXQqNarDCOBv MuMARrqEWTChqLq0Zcpf/gGaWXs60yFR70Xl6Dcf2PNX/W+/yKPUoq43G63o6M7hc4Q+ OOQ2CaSRsvIWM2UHq5ivKj3eLw1AC1xOIMsTe9dBmT4ajnUpr0RxUBFvwwozPIu2sVrO pIJZrhybJ7cnMO4Gig7WknVtevoqfMDJxZq4yJ+HM+xWS7NBLQIokNutuuPHNUN0EeNs 1Dzg== X-Gm-Message-State: ACgBeo2fAnTooHXRtDIXXZl5H3gyzCS4JvB9gOtsrNIKDGxl4sG46gcu CRVXj1TglY2MygvKieP9YTUufeldq+U= X-Google-Smtp-Source: AA6agR4d2+wq7SSECl5UjW0pU3ZA1/+lrjNacIkvDNV6O+q/fXMcpPTrg1JQC/x+5U1uRBj61wJElQ== X-Received: by 2002:a17:90b:3ec9:b0:1f5:62d5:4157 with SMTP id rm9-20020a17090b3ec900b001f562d54157mr2607732pjb.183.1659602450058; Thu, 04 Aug 2022 01:40:50 -0700 (PDT) Received: from localhost ([47.254.32.37]) by smtp.gmail.com with ESMTPSA id f187-20020a62dbc4000000b0052d98fbf8f3sm361442pfg.56.2022.08.04.01.40.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Aug 2022 01:40:49 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Linus Torvalds , "Eric W. Biederman" , Tejun Heo , Petr Mladek , Michal Hocko , Peter Zijlstra , Wedson Almeida Filho , Lai Jiangshan Subject: [RFC PATCH 2/8] workqueue: Make create_worker() safe against prematurely wakeups Date: Thu, 4 Aug 2022 16:41:29 +0800 Message-Id: <20220804084135.92425-3-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20220804084135.92425-1-jiangshanlai@gmail.com> References: <20220804084135.92425-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan A system crashed with the following BUG() report: [115147.050484] BUG: kernel NULL pointer dereference, address: 0000000000= 000000 [115147.050488] #PF: supervisor write access in kernel mode [115147.050489] #PF: error_code(0x0002) - not-present page [115147.050490] PGD 0 P4D 0 [115147.050494] Oops: 0002 [#1] PREEMPT_RT SMP NOPTI [115147.050498] CPU: 1 PID: 16213 Comm: kthreadd Kdump: loaded Tainted: G= O X 5.3.18-2-rt #1 SLE15-SP2 (unreleased) [115147.050510] RIP: 0010:_raw_spin_lock_irq+0x14/0x30 [115147.050513] Code: 89 c6 e8 5f 7a 9b ff 66 90 c3 66 66 2e 0f 1f 84 00 = 00 00 00 00 90 0f 1f 44 00 00 fa 65 ff 05 fb 53 6c 55 31 c0 ba 01 00 00 00 = 0f b1 17 75 01 c3 89 c6 e8 2e 7a 9b ff 66 90 c3 90 90 90 90 90 [115147.050514] RSP: 0018:ffffb0f68822fed8 EFLAGS: 00010046 [115147.050515] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00000000= 00000000 [115147.050516] RDX: 0000000000000001 RSI: 0000000000000002 RDI: 00000000= 00000000 [115147.050517] RBP: ffff9ca73af40a40 R08: 0000000000000001 R09: 00000000= 00027340 [115147.050519] R10: ffffb0f68822fe70 R11: 00000000000000a9 R12: ffffb0f6= 88067dc0 [115147.050520] R13: ffff9ca77e9a8000 R14: ffff9ca7634ca780 R15: ffff9ca7= 634ca780 [115147.050521] FS: 0000000000000000(0000) GS:ffff9ca77fb00000(0000) knl= GS:0000000000000000 [115147.050523] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [115147.050524] CR2: 00000000000000b8 CR3: 000000004472e000 CR4: 00000000= 003406e0 [115147.050524] Call Trace: [115147.050533] worker_thread+0xb4/0x3c0 [115147.050538] ? process_one_work+0x4a0/0x4a0 [115147.050540] kthread+0x152/0x170 [115147.050542] ? kthread_park+0xa0/0xa0 [115147.050544] ret_from_fork+0x35/0x40 Further debugging shown that the worker thread was woken before worker_attach_to_pool() finished in create_worker() though the reason why it was woken up is unknown yet which might be some real-time kernel activities. Any kthread is supposed to stay in TASK_UNINTERRUPTIBLE sleep until it is explicitly woken. But a spurious wakeup might break this expectation. As a result, worker_thread() might read worker->pool before it was set in worker create_worker() by worker_attach_to_pool(). Or it might leave idle before entering idle or process work items before attached for the pool. Also manage_workers() might want to create yet another worker before worker->pool->nr_workers is updated. It is a kind off a chicken & egg problem. Synchronize these operations using a completion API. There are two ways for the synchronization: either the manager does the worker initialization and the newly created worker waits for the completion of the initialization or the newly created worker does the worker initialization for itself and the manager waits for the completion. In current code, the manager does the worker initialization with the dependence that kthread API ensure the worker to be TASK_UNINTERRUPTIBLE. The ensuring is fragile, so one way of the synchronizations should be chosen and the dependence should be avoided. The newly created worker doing the worker initialization can simplify the code further, so the second way is chosen. Note that worker->pool might be then read without wq_pool_attach_mutex. Normal worker always belongs to the same pool and the locking rules for it is updated a bit. Also note that rescuer_thread() does not need this because all needed values are set before the kthreads is created. It is connected with a particular workqueue. It is attached to different pools when needed. Cc: Linus Torvalds Cc: "Eric W. Biederman" Cc: Tejun Heo , Cc: Petr Mladek Cc: Michal Hocko , Cc: Peter Zijlstra , Cc: Wedson Almeida Filho Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 22 ++++++++++++++-------- kernel/workqueue_internal.h | 11 +++++++++-- 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 928aad7d6123..f5b12c6778cc 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -176,6 +176,7 @@ struct worker_pool { /* L: hash of busy workers */ =20 struct worker *manager; /* L: purely informational */ + struct completion created; /* create_worker(): worker created */ struct list_head workers; /* A: attached workers */ struct completion *detach_completion; /* all workers detached */ =20 @@ -1942,6 +1943,7 @@ static struct worker *create_worker(struct worker_poo= l *pool) goto fail; =20 worker->id =3D id; + worker->pool =3D pool; =20 if (pool->cpu >=3D 0) snprintf(id_buf, sizeof(id_buf), "%d:%d%s", pool->cpu, id, @@ -1949,6 +1951,7 @@ static struct worker *create_worker(struct worker_poo= l *pool) else snprintf(id_buf, sizeof(id_buf), "u%d:%d", pool->id, id); =20 + reinit_completion(&pool->created); worker->task =3D kthread_create_on_node(worker_thread, worker, pool->node, "kworker/%s", id_buf); if (IS_ERR(worker->task)) @@ -1957,15 +1960,9 @@ static struct worker *create_worker(struct worker_po= ol *pool) set_user_nice(worker->task, pool->attrs->nice); kthread_bind_mask(worker->task, pool->attrs->cpumask); =20 - /* successful, attach the worker to the pool */ - worker_attach_to_pool(worker, pool); - /* start the newly created worker */ - raw_spin_lock_irq(&pool->lock); - worker->pool->nr_workers++; - worker_enter_idle(worker); wake_up_process(worker->task); - raw_spin_unlock_irq(&pool->lock); + wait_for_completion(&pool->created); =20 return worker; =20 @@ -2383,10 +2380,17 @@ static int worker_thread(void *__worker) struct worker *worker =3D __worker; struct worker_pool *pool =3D worker->pool; =20 + /* attach the worker to the pool */ + worker_attach_to_pool(worker, pool); + /* tell the scheduler that this is a workqueue worker */ set_pf_worker(true); -woke_up: + raw_spin_lock_irq(&pool->lock); + worker->pool->nr_workers++; + worker_enter_idle(worker); + complete(&pool->created); +woke_up: =20 /* am I supposed to die? */ if (unlikely(worker->flags & WORKER_DIE)) { @@ -2458,6 +2462,7 @@ static int worker_thread(void *__worker) __set_current_state(TASK_IDLE); raw_spin_unlock_irq(&pool->lock); schedule(); + raw_spin_lock_irq(&pool->lock); goto woke_up; } =20 @@ -3461,6 +3466,7 @@ static int init_worker_pool(struct worker_pool *pool) =20 timer_setup(&pool->mayday_timer, pool_mayday_timeout, 0); =20 + init_completion(&pool->created); INIT_LIST_HEAD(&pool->workers); =20 ida_init(&pool->worker_ida); diff --git a/kernel/workqueue_internal.h b/kernel/workqueue_internal.h index e00b1204a8e9..025861c4d1f6 100644 --- a/kernel/workqueue_internal.h +++ b/kernel/workqueue_internal.h @@ -37,8 +37,15 @@ struct worker { /* 64 bytes boundary on 64bit, 32 on 32bit */ =20 struct task_struct *task; /* I: worker task */ - struct worker_pool *pool; /* A: the associated pool */ - /* L: for rescuers */ + + /* + * The associated pool, locking rules: + * PF_WQ_WORKER: from the current worker + * PF_WQ_WORKER && wq_pool_attach_mutex: from remote tasks + * None: from the current worker when the worker is coming up + */ + struct worker_pool *pool; + struct list_head node; /* A: anchored at pool->workers */ /* A: runs through worker->node */ =20 --=20 2.19.1.6.gb485710b From nobody Sat Apr 11 22:45:42 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 616C7C19F2B for ; Thu, 4 Aug 2022 08:41:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238992AbiHDIlU (ORCPT ); Thu, 4 Aug 2022 04:41:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239254AbiHDIkz (ORCPT ); Thu, 4 Aug 2022 04:40:55 -0400 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B3B15E301 for ; Thu, 4 Aug 2022 01:40:54 -0700 (PDT) Received: by mail-pf1-x42f.google.com with SMTP id h28so12403109pfq.11 for ; Thu, 04 Aug 2022 01:40:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=qPFA3XwXyTSeEohLFMsskHgwupaXA0ENjQ4ZfIF3Cr8=; b=hJXN3MMDut33BAQgcXLu2y9Mz/rSpCX9vFcTVp67KTJbdjyIy2CJdr8C1xL8GF0pFN 7vndv5w/DHjEoPt3GhZlINZjFlnf/W4+tctA36dq3U5rb2B/VsKslCjMupHSsAyB1dUV 54qiuiz32UrXpAQIP9+I0QHPosdUE1DPlVGTtMqFH5zEpMQuDMbKTwt4sPEg0IAcaqSU tb5Gf7ZbRi8dhVhlJPHhewlhi5CORNOBx5XOjH5+tlTuAaYpsfDDpUriaai0sMPMzapP 0llzMQzV7efxSqchv03rZb87P9/YrTM/P8KAMjASuUM4ZHdVNuU8pVofR05W45L9gcw9 7pJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=qPFA3XwXyTSeEohLFMsskHgwupaXA0ENjQ4ZfIF3Cr8=; b=BQOeE6mnqgieeF55FudCjcvKPhRbPD5ywSqQVti+RXHMnlMner9FQeDJyBKnwMa15i /3qv1QxhG9lzPcNcQRMGCQFh5B6lrixKW3nj0CUSinss2VA3wvZ8yCBiZr2FtK1m5sZQ qD5DY6yXLcWUGGnmzzi+iRX+iOedjRKqt0+IugIDThmvvWwZRWValJxIAG99yf91kY+Q T56bqSrsJjoiYy+eVJlg8dNRO3ofn46iSpm2gssQ6GhI6cu5LIjkUXOXanneUEAT601c bGiFmmPQOnsgRQHo0mEHFggDgB6tNdZvXyFV6CTEVpODNbVadUC9IcvefYvHVEnl/Z// NlBA== X-Gm-Message-State: ACgBeo3sAa8yO5BDc06mff9qfRU7PEJHTG1Yfv79l61Gq20dVaoGh4eZ Hez7M3L/pOQdaPQQb+VaA3VwF8KP0FE= X-Google-Smtp-Source: AA6agR6UlT1jWktB4/zR56Vdl9FXyMKUzofgTnilPDLtuq1tyE5lWsnWBdzUpRgiqHkNukRSevNEUw== X-Received: by 2002:a63:6888:0:b0:3fe:49fc:3be3 with SMTP id d130-20020a636888000000b003fe49fc3be3mr781177pgc.182.1659602453267; Thu, 04 Aug 2022 01:40:53 -0700 (PDT) Received: from localhost ([47.88.5.130]) by smtp.gmail.com with ESMTPSA id a7-20020a1709027e4700b0016cd74e5f87sm242487pln.240.2022.08.04.01.40.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Aug 2022 01:40:52 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Linus Torvalds , "Eric W. Biederman" , Tejun Heo , Petr Mladek , Michal Hocko , Peter Zijlstra , Wedson Almeida Filho , Lai Jiangshan Subject: [RFC PATCH 3/8] workqueue: Set PF_NO_SETAFFINITY instead of kthread_bind_mask() Date: Thu, 4 Aug 2022 16:41:30 +0800 Message-Id: <20220804084135.92425-4-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20220804084135.92425-1-jiangshanlai@gmail.com> References: <20220804084135.92425-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan kthread_bind_mask() can't work correctly if spurious wakeup happens before kthread_bind_mask(). And a spuriously wakeup worker's cpumask can be possibly changed by a userspace if worker_attach_to_pool() is called earlier than kthread_bind_mask(). To avoid the problem caused by spurious wokeup, set PF_NO_SETAFFINITY at the starting of workers where kthread_bind_mask() can't be used and luckily workqueue code binds cpumask by itself, all it needs is only PF_NO_SETAFFINITY. Cc: Linus Torvalds Cc: "Eric W. Biederman" Cc: Tejun Heo , Cc: Petr Mladek Cc: Michal Hocko , Cc: Peter Zijlstra , Cc: Wedson Almeida Filho Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index f5b12c6778cc..82937c0fb21f 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1958,7 +1958,6 @@ static struct worker *create_worker(struct worker_poo= l *pool) goto fail; =20 set_user_nice(worker->task, pool->attrs->nice); - kthread_bind_mask(worker->task, pool->attrs->cpumask); =20 /* start the newly created worker */ wake_up_process(worker->task); @@ -2380,6 +2379,8 @@ static int worker_thread(void *__worker) struct worker *worker =3D __worker; struct worker_pool *pool =3D worker->pool; =20 + current->flags |=3D PF_NO_SETAFFINITY; + /* attach the worker to the pool */ worker_attach_to_pool(worker, pool); =20 @@ -2494,6 +2495,7 @@ static int rescuer_thread(void *__rescuer) struct list_head *scheduled =3D &rescuer->scheduled; bool should_stop; =20 + current->flags |=3D PF_NO_SETAFFINITY; set_user_nice(current, RESCUER_NICE_LEVEL); =20 /* @@ -4279,7 +4281,6 @@ static int init_rescuer(struct workqueue_struct *wq) } =20 wq->rescuer =3D rescuer; - kthread_bind_mask(rescuer->task, cpu_possible_mask); wake_up_process(rescuer->task); =20 return 0; --=20 2.19.1.6.gb485710b From nobody Sat Apr 11 22:45:42 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A013C19F2B for ; Thu, 4 Aug 2022 08:41:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239509AbiHDIlr (ORCPT ); Thu, 4 Aug 2022 04:41:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239310AbiHDIlO (ORCPT ); Thu, 4 Aug 2022 04:41:14 -0400 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3139C65839 for ; Thu, 4 Aug 2022 01:40:57 -0700 (PDT) Received: by mail-pg1-x531.google.com with SMTP id bh13so17355652pgb.4 for ; Thu, 04 Aug 2022 01:40:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=a8SD/ZTcIF1U9sLR5IbPm2YAiCCIs0oujdhG7j3kthQ=; b=TSARr/wBgWs1BYKva0hT7u5aUB+RzkIk2TDdNOXLBpKSGsARD/1R8u4rHx6ukA8Kwk ed9yP0FwBDWiV6aS2uUh68aSdsP9l7zVNxVVPa3UlW1WpBpbUTbonFKXvW38AMHRFtuA 6i+L6cTdBfCvEs3JLMoi83g2okH89EU8hmPl8WNpKp4RiHQxzaMtDgzj+HrxyPVXW+1g 21qOM+IzI3c0l3mLPU+1rGMUz5TUVH8NCvZ6SBhDz3NcaTlP68HXD4udRCCkOA7PIM4F 1RH3cK8Sh5NaUv6G+GWnL6YG2R0B4DKjWED+x9UDTLmdOWbXkp/uOvOkXKLzILaxBL/d //TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=a8SD/ZTcIF1U9sLR5IbPm2YAiCCIs0oujdhG7j3kthQ=; b=R5npVG8+ZVbitC6Yhag72LmUIRj/N0upTR12A+lsYaBDG9eYlPq4bTAX/ISgWCTkpg x6dd3aRMHcezwvCOeez43CqPBGokgcvNvn56Z7nIagvlumG2IxqF2beoLp9ekFiTEFhg W5G/j2mq2JERe0TJDWSC3bdd6LkLqzPQeoAn4N1OL8YbOU++Cqwf38WJcp6Aga4n5/M5 565f+kSMqU4fHMnVFR4CCb5LQO0vkm4gziGMy2MIdgA6phm339ONqr+/pzEo+IrQNXgH Y+8ZQ43XQcUbbGFQPx+c7b1dci5Nm87z6OcZqTpDTzDtSR6noKRvd3ylQSJZ0bsgkwMr 9FSA== X-Gm-Message-State: ACgBeo0zRzK/sssQ3SOSNzoY+/9xVwioYRRVrLotIUPVrW5DOuivKT1S oRg7qMmshq93hVa1Sv+GqCqESGNi04o= X-Google-Smtp-Source: AA6agR7jp2Wnbdp+IZ/ad1CRu9OgCfyZomNnK7lsjcwmsKk3v4ZmdPoJg9wBUlIK8/z7x/hVMlY5Og== X-Received: by 2002:a05:6a00:10ca:b0:4f7:5af4:47b6 with SMTP id d10-20020a056a0010ca00b004f75af447b6mr747251pfu.6.1659602456264; Thu, 04 Aug 2022 01:40:56 -0700 (PDT) Received: from localhost ([47.88.5.130]) by smtp.gmail.com with ESMTPSA id i6-20020a170902c94600b0016f1d578f71sm315347pla.139.2022.08.04.01.40.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Aug 2022 01:40:55 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan Subject: [RFC PATCH 4/8] workqueue: Set/Clear PF_WQ_WORKER while attaching/detaching Date: Thu, 4 Aug 2022 16:41:31 +0800 Message-Id: <20220804084135.92425-5-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20220804084135.92425-1-jiangshanlai@gmail.com> References: <20220804084135.92425-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan PF_WQ_WORKER is only needed when the worker is attached. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 82937c0fb21f..7fc4c2fa21d6 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1861,6 +1861,8 @@ static struct worker *alloc_worker(int node) static void worker_attach_to_pool(struct worker *worker, struct worker_pool *pool) { + WARN_ON_ONCE(worker->task !=3D current); + mutex_lock(&wq_pool_attach_mutex); =20 /* @@ -1882,6 +1884,9 @@ static void worker_attach_to_pool(struct worker *work= er, list_add_tail(&worker->node, &pool->workers); worker->pool =3D pool; =20 + /* tell the scheduler that this is a workqueue worker */ + current->flags |=3D PF_WQ_WORKER; + mutex_unlock(&wq_pool_attach_mutex); } =20 @@ -1898,8 +1903,11 @@ static void worker_detach_from_pool(struct worker *w= orker) struct worker_pool *pool =3D worker->pool; struct completion *detach_completion =3D NULL; =20 + WARN_ON_ONCE(worker->task !=3D current); + mutex_lock(&wq_pool_attach_mutex); =20 + current->flags &=3D ~PF_WQ_WORKER; kthread_set_per_cpu(worker->task, -1); list_del(&worker->node); worker->pool =3D NULL; @@ -2352,16 +2360,6 @@ static void process_scheduled_works(struct worker *w= orker) } } =20 -static void set_pf_worker(bool val) -{ - mutex_lock(&wq_pool_attach_mutex); - if (val) - current->flags |=3D PF_WQ_WORKER; - else - current->flags &=3D ~PF_WQ_WORKER; - mutex_unlock(&wq_pool_attach_mutex); -} - /** * worker_thread - the worker thread function * @__worker: self @@ -2384,9 +2382,6 @@ static int worker_thread(void *__worker) /* attach the worker to the pool */ worker_attach_to_pool(worker, pool); =20 - /* tell the scheduler that this is a workqueue worker */ - set_pf_worker(true); - raw_spin_lock_irq(&pool->lock); worker->pool->nr_workers++; worker_enter_idle(worker); @@ -2397,7 +2392,6 @@ static int worker_thread(void *__worker) if (unlikely(worker->flags & WORKER_DIE)) { raw_spin_unlock_irq(&pool->lock); WARN_ON_ONCE(!list_empty(&worker->entry)); - set_pf_worker(false); =20 set_task_comm(worker->task, "kworker/dying"); ida_free(&pool->worker_ida, worker->id); @@ -2498,11 +2492,6 @@ static int rescuer_thread(void *__rescuer) current->flags |=3D PF_NO_SETAFFINITY; set_user_nice(current, RESCUER_NICE_LEVEL); =20 - /* - * Mark rescuer as worker too. As WORKER_PREP is never cleared, it - * doesn't participate in concurrency management. - */ - set_pf_worker(true); repeat: set_current_state(TASK_IDLE); =20 @@ -2531,6 +2520,10 @@ static int rescuer_thread(void *__rescuer) =20 raw_spin_unlock_irq(&wq_mayday_lock); =20 + /* + * Attach the rescuer. As WORKER_PREP is never cleared, it + * doesn't participate in concurrency management. + */ worker_attach_to_pool(rescuer, pool); =20 raw_spin_lock_irq(&pool->lock); @@ -2600,7 +2593,6 @@ static int rescuer_thread(void *__rescuer) =20 if (should_stop) { __set_current_state(TASK_RUNNING); - set_pf_worker(false); return 0; } =20 --=20 2.19.1.6.gb485710b From nobody Sat Apr 11 22:45:42 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B39A2C19F2B for ; Thu, 4 Aug 2022 08:41:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239522AbiHDIlx (ORCPT ); Thu, 4 Aug 2022 04:41:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239343AbiHDIla (ORCPT ); Thu, 4 Aug 2022 04:41:30 -0400 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F3146610E for ; Thu, 4 Aug 2022 01:41:00 -0700 (PDT) Received: by mail-pj1-x1036.google.com with SMTP id 15-20020a17090a098f00b001f305b453feso4597951pjo.1 for ; Thu, 04 Aug 2022 01:41:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=708QVR8ULFWukUiEUmJH7O0JEBFAFzLzIg7WVeUtPxg=; b=OxTUh2nhcrkfShZ1PhvoX3EBXpK17kyGw1s016rcFqCXGbQWqHZrcVnWGjb3IFbA9v sxw7RqFxZxuD7tnHhhlhTHFutOskYMTyLTYuwstEhFGA0/sSySkluEG/vpKs+M67ce5C 2a/mkgV3Dpz58UczXONCpW+m/dOXdjpRRSPnHbluTW4/qJo0sWkGH9FUbgZnZcMbdMtV Fif2Tg0ps1kn5PnXk4+IpmittQqYK+iNYeNnk/NJFNB39PrpU7vNpyoYm0BXCNGhvLiJ gyQrxc2VorbuLGXOXyBMmSuzIc7fGZNDXyyav92uiwoDUMjnDt8ygGZxl95CaIeIQL3T XVHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=708QVR8ULFWukUiEUmJH7O0JEBFAFzLzIg7WVeUtPxg=; b=DTlmCSihwA7Y5MDZSceB5lC4f08WWfbNG1Krh3ie1JMxhKLJfePR6aQHt3JrBnEraA HZ35QjkOD4cYUrjAAVa84uPcpyb0Guk6D3KhyNXPlbBjCNd3U0J39RZJE+VTRxeLLJAc YNq+wlCRAQs9i6ewOCm8fWWGvchjg0SKbt2b/fIsfP58fo/StfWhQ+bTYTCxgWLkR0vl VMSjHwIT98+NM8QUybSo5e0A76EGjQYDhoQjcwpKBFdt+Kh6Wu/QVldSHZrQDEdYjpDH 8UqklYFYGowhPZ6NSOWcNCltqn5iPfALMOYTXV/aC6PhEm4gW23TMa7hbgzXHlItK5hI 3x+A== X-Gm-Message-State: ACgBeo14q7J3Ry6wQunYcSZQAZj739AnXhExMAtrHlBhEEqqO3CI18is jpIQtURddEeGTfk3uQ3EHnyGmyJ3Ed8= X-Google-Smtp-Source: AA6agR5NtqWTdH7rLWKdq8erbGeQV37dwwisMIYnS/HBxLBXEHfbClNi40x9bRlu0CxNPhE8jc2wTg== X-Received: by 2002:a17:90b:3586:b0:1f4:d507:783e with SMTP id mm6-20020a17090b358600b001f4d507783emr9180173pjb.171.1659602459295; Thu, 04 Aug 2022 01:40:59 -0700 (PDT) Received: from localhost ([47.254.32.37]) by smtp.gmail.com with ESMTPSA id s18-20020a170903215200b0016c3affe60esm329613ple.46.2022.08.04.01.40.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Aug 2022 01:40:58 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan Subject: [RFC PATCH 5/8] workqueue: Use worker_set_flags() in worker_enter_idle() Date: Thu, 4 Aug 2022 16:41:32 +0800 Message-Id: <20220804084135.92425-6-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20220804084135.92425-1-jiangshanlai@gmail.com> References: <20220804084135.92425-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan worker_enter_idle() is only called in worker_thread() now. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 7fc4c2fa21d6..afe62649fb3a 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1799,8 +1799,7 @@ static void worker_enter_idle(struct worker *worker) (worker->hentry.next || worker->hentry.pprev))) return; =20 - /* can't use worker_set_flags(), also called from create_worker() */ - worker->flags |=3D WORKER_IDLE; + worker_set_flags(worker, WORKER_IDLE); pool->nr_idle++; worker->last_active =3D jiffies; =20 --=20 2.19.1.6.gb485710b From nobody Sat Apr 11 22:45:42 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36E81C19F2B for ; Thu, 4 Aug 2022 08:42:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239539AbiHDImJ (ORCPT ); Thu, 4 Aug 2022 04:42:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239386AbiHDIlp (ORCPT ); Thu, 4 Aug 2022 04:41:45 -0400 Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0623A66125 for ; Thu, 4 Aug 2022 01:41:03 -0700 (PDT) Received: by mail-pl1-x635.google.com with SMTP id w14so5668983plp.9 for ; Thu, 04 Aug 2022 01:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=JYtx0gSEzUyP9tNNW1dyx5vOU7phI7IefN54Dq4Qn2I=; b=nPH69jq36PE5OSDqSsRcVNcq3kxLmc0zvtGfIflRk9S3woqyXqCymAOCoLVA80yOAv RRxjrJpwbytrD3aB12lEra/NWYQ/GSZGYdbm6tbMl3hS5cQS8mPJX5YK56zjt7hZhUTD 3HmDvXLtBv9/QWCxGv4CNz9rwUlPE/t7wxXUY9a7muEOO1/ysTqB9PETUCQjn9QjaxDS 7VIU8hYOz4sx6TceGzZxffzuOw7I1cw4CwrXG71pEo1EgrVMtldTuyFKfQpf86BCoxBn 6zlXds1v8kBDQGQ5RB+VCF1VzdpbkK3N75b7hgnH+FeNZjDWm8FJrE4C5jDie80AA1Nc h5UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=JYtx0gSEzUyP9tNNW1dyx5vOU7phI7IefN54Dq4Qn2I=; b=l2+1HmGqSJSkrRocXX8yE3lue0x0s7QGt2Hk85sBIvZ0Oy5xDhvzqgpITFhoa0uQ/K 0xJ6gV6WppCANKDxz005ewtikOGma4H/o7PFDKHmn95s+nqLLgZ35GUGFiT4+WpCgOp2 Wm/9a9GC7C5HFgVZCyFXKz75tcRqxwlwShYwEW7++ywYhrE8d6rBeBPaISYC/8zp16xZ vupUVDhqXZMr9LSka6K2wyn1RiagMaMZyNiDEAsiS4+m747/RLKQMyGqkhLM0jz9TQxX e2U4BCsgYAw1dOlIoMJCCYtq7I/dSwSaGGY1SZO0TGSbqnD09kphWEUYbOESaA+idHjh Mk3Q== X-Gm-Message-State: ACgBeo0Wl5J5OPyApcpvKdwhq9nTm34T1yRAeN0uopDffA6m2MqF4p6R PSeNrZc2gx3dKkY9aDUQ1qnMenIutYI= X-Google-Smtp-Source: AA6agR5VwKqCPDcRPZSLIdxQMKZkjNmhgeyswq844jixfkWUVrC/JiSV5Mt6SMfvDmSeXyKUnLfCaA== X-Received: by 2002:a17:902:ac97:b0:16e:eb08:fb15 with SMTP id h23-20020a170902ac9700b0016eeb08fb15mr941051plr.7.1659602462293; Thu, 04 Aug 2022 01:41:02 -0700 (PDT) Received: from localhost ([47.89.225.180]) by smtp.gmail.com with ESMTPSA id b27-20020aa78edb000000b0052b6277df9esm354103pfr.43.2022.08.04.01.41.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Aug 2022 01:41:01 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan Subject: [RFC PATCH 6/8] workqueue: Simplify the starting of the newly created worker Date: Thu, 4 Aug 2022 16:41:33 +0800 Message-Id: <20220804084135.92425-7-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20220804084135.92425-1-jiangshanlai@gmail.com> References: <20220804084135.92425-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan There is no point for the newly created worker entering and leaving idle in the same pool->lock held region and checking WORKER_DIE because it has not added to idle_list before the lock. Avoid the code in the starting of the newly created worker and move those code to the end of the worker_thread(). Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 35 +++++++++++++++++------------------ 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index afe62649fb3a..64dc1833d11a 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2383,31 +2383,16 @@ static int worker_thread(void *__worker) =20 raw_spin_lock_irq(&pool->lock); worker->pool->nr_workers++; - worker_enter_idle(worker); complete(&pool->created); -woke_up: - - /* am I supposed to die? */ - if (unlikely(worker->flags & WORKER_DIE)) { - raw_spin_unlock_irq(&pool->lock); - WARN_ON_ONCE(!list_empty(&worker->entry)); - - set_task_comm(worker->task, "kworker/dying"); - ida_free(&pool->worker_ida, worker->id); - worker_detach_from_pool(worker); - kfree(worker); - return 0; - } =20 - worker_leave_idle(worker); -recheck: +loop: /* no more worker necessary? */ if (!need_more_worker(pool)) goto sleep; =20 /* do we need to manage? */ if (unlikely(!may_start_working(pool)) && manage_workers(worker)) - goto recheck; + goto loop; =20 /* * ->scheduled list can only be filled while a worker is @@ -2457,7 +2442,21 @@ static int worker_thread(void *__worker) raw_spin_unlock_irq(&pool->lock); schedule(); raw_spin_lock_irq(&pool->lock); - goto woke_up; + + /* am I supposed to die? */ + if (unlikely(worker->flags & WORKER_DIE)) { + raw_spin_unlock_irq(&pool->lock); + WARN_ON_ONCE(!list_empty(&worker->entry)); + + set_task_comm(worker->task, "kworker/dying"); + ida_free(&pool->worker_ida, worker->id); + worker_detach_from_pool(worker); + kfree(worker); + return 0; + } + + worker_leave_idle(worker); + goto loop; } =20 /** --=20 2.19.1.6.gb485710b From nobody Sat Apr 11 22:45:42 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8591BC19F2B for ; Thu, 4 Aug 2022 08:42:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239502AbiHDImS (ORCPT ); Thu, 4 Aug 2022 04:42:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239000AbiHDIlr (ORCPT ); Thu, 4 Aug 2022 04:41:47 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54D346613A for ; Thu, 4 Aug 2022 01:41:06 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id pm17so13299011pjb.3 for ; Thu, 04 Aug 2022 01:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=UBybgKMsxDd15bCY1F3MJ8sBpGdDmhHWbWdFhTLioM4=; b=JyMGyYRPHz0w2mAM6iWbFWwyEfIO8ohR7V3+ADWVkFlblY4mCwDLRoTYMyOp6ERCq9 4CfqLcUyaVaoxRC5Kh5optSSga1giV+TNyMG0Sl/aE3ZWF3GFtSevZRitiYNdIPpQyAb JGZXsypHS4aK6aJT0gf1c6WiL3wGy0mdG+796IF/bMFqJKtYoCM7a5KgFa70UsK9DEr0 c+8Llqd6y4lqwjGoxiP4e6xyvb1MMTuqMoM/aUxp+lmhll/qo9f1yIfooXWR38EtVO0u 1cfGdUklQGF6lDuO5r73kC6/iYN8wPzUPvjee89JjLvw5LmV4QUloYS1Ilm7w8IbI33y FDzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=UBybgKMsxDd15bCY1F3MJ8sBpGdDmhHWbWdFhTLioM4=; b=WYyMmLV9o835/W4G2iWuiIHfz/r+FKEAjCljf8r/JKMp5+Bm7tP8WcNAW7ov3WJ7ZF YTK4H9nwbSXTOvHw8r6yuExNl/LAcjtKhL2o2n8OKuUAPBnkXmoDGxsrPqfiVCpIUq/f AJp82Bo/oiQVryx3YF3rZqQzAOV7QVnOH7BEv/pde5I8ADfZJc3EFosiUhJq3/CQmrYS 6rloW90yC78TUuXb/vYSkT3iVTiu99mwgvaWxL+C2BiQSUgbughxoIQyZWRGAglMW8th ylmMancwOoQxZd7gjuUo+8l265l88NceI15Bc+U7lg+G7ktpE3+CVWZ+5kBuKObysWtl BmFA== X-Gm-Message-State: ACgBeo3BflJU5pIg6Z1CCbI8W26Xq1zmGQ3umG2y8cBescBNAbfV+zzG yTPQif0Vo63pkZVUj506hfZIsZ1dkSA= X-Google-Smtp-Source: AA6agR7Hddhegxzh+YOOhSjAICNxy7uHcQ+QSzFRfb31XvIr1J7CMsivhFHMlkkgXm4CdcZyYnr1CQ== X-Received: by 2002:a17:903:189:b0:16f:24e4:1614 with SMTP id z9-20020a170903018900b0016f24e41614mr908929plg.103.1659602465204; Thu, 04 Aug 2022 01:41:05 -0700 (PDT) Received: from localhost ([47.88.5.130]) by smtp.gmail.com with ESMTPSA id f10-20020a170902860a00b0016d33b8a231sm232594plo.270.2022.08.04.01.41.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Aug 2022 01:41:04 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan Subject: [RFC PATCH 7/8] workqueue: Remove the outer loop in maybe_create_worker() Date: Thu, 4 Aug 2022 16:41:34 +0800 Message-Id: <20220804084135.92425-8-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20220804084135.92425-1-jiangshanlai@gmail.com> References: <20220804084135.92425-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan worker_thread() always does the recheck after getting the manager role, so the recheck in the maybe_create_worker() is unneeded and is removed. A piece of comment for maybe_create_worker() is removed because it is not true anymore after the recheck is removed. A piece of comment for manage_workers() is removed because there is already another piece of comment explaining the same thing which is more accurate. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 64dc1833d11a..0d9844b81482 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2091,9 +2091,6 @@ static void pool_mayday_timeout(struct timer_list *t) * sent to all rescuers with works scheduled on @pool to resolve * possible allocation deadlock. * - * On return, need_to_create_worker() is guaranteed to be %false and - * may_start_working() %true. - * * LOCKING: * raw_spin_lock_irq(pool->lock) which may be released and regrabbed * multiple times. Does GFP_KERNEL allocations. Called only from @@ -2103,7 +2100,6 @@ static void maybe_create_worker(struct worker_pool *p= ool) __releases(&pool->lock) __acquires(&pool->lock) { -restart: raw_spin_unlock_irq(&pool->lock); =20 /* if we don't make progress in MAYDAY_INITIAL_TIMEOUT, call for help */ @@ -2121,13 +2117,6 @@ __acquires(&pool->lock) =20 del_timer_sync(&pool->mayday_timer); raw_spin_lock_irq(&pool->lock); - /* - * This is necessary even after a new worker was just successfully - * created as @pool->lock was dropped and the new worker might have - * already become busy. - */ - if (need_to_create_worker(pool)) - goto restart; } =20 /** @@ -2138,10 +2127,6 @@ __acquires(&pool->lock) * to. At any given time, there can be only zero or one manager per * pool. The exclusion is handled automatically by this function. * - * The caller can safely start processing works on false return. On - * true return, it's guaranteed that need_to_create_worker() is false - * and may_start_working() is true. - * * CONTEXT: * raw_spin_lock_irq(pool->lock) which may be released and regrabbed * multiple times. Does GFP_KERNEL allocations. --=20 2.19.1.6.gb485710b From nobody Sat Apr 11 22:45:42 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75C91C19F2B for ; Thu, 4 Aug 2022 08:42:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239514AbiHDImf (ORCPT ); Thu, 4 Aug 2022 04:42:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239515AbiHDIlv (ORCPT ); Thu, 4 Aug 2022 04:41:51 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3DAB6714B for ; Thu, 4 Aug 2022 01:41:08 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id s5-20020a17090a13c500b001f4da9ffe5fso4566932pjf.5 for ; Thu, 04 Aug 2022 01:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=2QaqyHfbHLtpliP6PzpyY5kh+sI+NpRD+MBrExwxasE=; b=XQdV9/wIxqZlDtL98Dm1oldeJ5OqgRxntlIF+kEYjxekgPEU6xcw40fS56pvhAhIjh ung5k6CcRLk5vV+nmVOHPIQT/s+/tf/g3TFMEd/uEXuZ3Ilks9ug5T4YzDFnNmbBqSOs IkJERwsDObI0dO2mToe+Qi97mdrmo3/wPZtYAiwUwQvauYBJHLvsf4vEQGvoGfR2vw+C sGYpiqtrKgb4MwApdla51Uwc4QpwcSR1SF8baVZLWK/AItCdmGtYtJS9MJcPKnrWfRsM j8oUD57/QfFFQFbJgzm0XV6s6VBigXpu13yL399G31RQY/3aT7pwS+xUHh3HkUO08XKZ RPCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=2QaqyHfbHLtpliP6PzpyY5kh+sI+NpRD+MBrExwxasE=; b=2rIdHVecp/KfiNP5/aElYcohVZiMc2mo7nBy+KyTf7KDPkqyopjcAiHefT7D5xpPvb 6CVMhVPq3ywLNf9xwZ+RnU4SAWVH3RwHOkHJRUfcic+jqB65szmC9zwV23nQ77ltOJ5l rjD7ECTfH9j1ogg4E/8sFjLDoQSE3PvgZD6Aqe0uBa3H4Utc5p6N7IsS/6bpGStdTFuA xCQ0+u3CVqGs3SRHKh7eQzW3fBm+xbhq0UomPKY10MMqEg4DfPI21rOfg/7TJr4uyg6L Y4Xgq4CnxIBRES38xmAffWacMyLTRtivIH9kYEWYnIHfXv6qg3OwNpwJ9zD8+oaCYy+r vk5w== X-Gm-Message-State: ACgBeo1xtremPGRStHVQ17KV6rMW64E7ShUPigwCufRNZECx+DcND4R4 RIJj/2zQPoEsY3GDgFgXADs7lRjKZsU= X-Google-Smtp-Source: AA6agR5bGM+NH+ZIxjElVHsO+022e9ef8Utw5P5HvnbJPQ2hBJUz79XMxY70/3ZvyKTWQPvmwUvsow== X-Received: by 2002:a17:90b:33d1:b0:1f5:4fc5:3d72 with SMTP id lk17-20020a17090b33d100b001f54fc53d72mr953449pjb.60.1659602468271; Thu, 04 Aug 2022 01:41:08 -0700 (PDT) Received: from localhost ([198.11.178.15]) by smtp.gmail.com with ESMTPSA id l21-20020a17090a599500b001f4d4a1b494sm550103pji.7.2022.08.04.01.41.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Aug 2022 01:41:07 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan Subject: [RFC PATCH 8/8] workqueue: Move the locking out of maybe_create_worker() Date: Thu, 4 Aug 2022 16:41:35 +0800 Message-Id: <20220804084135.92425-9-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20220804084135.92425-1-jiangshanlai@gmail.com> References: <20220804084135.92425-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan The code is cleaner if the reversed pair of unlock() and lock() is moved into the caller. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 0d9844b81482..013ad61e67b9 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2091,17 +2091,10 @@ static void pool_mayday_timeout(struct timer_list *= t) * sent to all rescuers with works scheduled on @pool to resolve * possible allocation deadlock. * - * LOCKING: - * raw_spin_lock_irq(pool->lock) which may be released and regrabbed - * multiple times. Does GFP_KERNEL allocations. Called only from - * manager. + * Does GFP_KERNEL allocations. Called only from manager. */ static void maybe_create_worker(struct worker_pool *pool) -__releases(&pool->lock) -__acquires(&pool->lock) { - raw_spin_unlock_irq(&pool->lock); - /* if we don't make progress in MAYDAY_INITIAL_TIMEOUT, call for help */ mod_timer(&pool->mayday_timer, jiffies + MAYDAY_INITIAL_TIMEOUT); =20 @@ -2116,7 +2109,6 @@ __acquires(&pool->lock) } =20 del_timer_sync(&pool->mayday_timer); - raw_spin_lock_irq(&pool->lock); } =20 /** @@ -2147,7 +2139,9 @@ static bool manage_workers(struct worker *worker) pool->flags |=3D POOL_MANAGER_ACTIVE; pool->manager =3D worker; =20 + raw_spin_unlock_irq(&pool->lock); maybe_create_worker(pool); + raw_spin_lock_irq(&pool->lock); =20 pool->manager =3D NULL; pool->flags &=3D ~POOL_MANAGER_ACTIVE; --=20 2.19.1.6.gb485710b