From nobody Fri Dec 19 10:56:26 2025 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FCB717BCE for ; Wed, 3 Jul 2024 03:36:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977774; cv=none; b=XLapxzhZTukWAOGrSU1XVDE4skZYrrnjmS7MgY/nUh096XVSUfllCZ/PG6GejcFBlle+/3U7dIvJlQNMeB+e4qSd+GFUn3xfc3hI9qwqcb5HkSN3J8Tv3fGP1jEQ6JNhbwkxkVUXWK6q4P8s7EBBSyNZC34Hnb5eXLXq8m/wYNQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977774; c=relaxed/simple; bh=kfVfIUpaxcr171PH8DCDoG+zD7xsW9GTLz4zvQJLWMI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=poO3aQDMYMHi2TxSZ541ehJe0keRUQ0if/zssUcB4vkYRAcalM/mMJyQ7+piHHmXsOSQn56G/kwiaRXK6x95ejaU9qf6YbfxARdzeFFFqmgcSRSk4IhOW9050tIVc98kDqHJ5Ql5b89u3wT0y5zWeKwJSPRJprhNDqtrpUikOY8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CXmKqFI/; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CXmKqFI/" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1fab03d2f23so38079555ad.0 for ; Tue, 02 Jul 2024 20:36:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719977772; x=1720582572; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4GJ1xuGIg/34+KoYPJFdTKabqBtfsZIj48w+G481zs0=; b=CXmKqFI/wjz+Yy3dO4pEu/r4vKYHfNmxIS7xtN/RHG19AaboGU4ZDUrST6dTi3IKR8 VqDLmN4hLWtWSBfWrAzdImi21TKG1+3Rgg0Stg1yjaV9U/pDPUnpbqyceobAcGCyv7O+ TzJu3MtIs4MQZprU0bhXRUv+5UbnNz/APFzb2tl+cfLAQu8RNi8QtMjoUB+AKZ5w8Yb9 n68TANI8P+nhhCcJXBiBrBocrhOH+ZxOC5eGw3oDO4eflF+rp/dRvXe3XyIK6CvfzmjA gpp8WXXeY35X/lFOnJ+fa7BF3nvXVDrbEno4ePMDG5XTdDo2Z8fx/JQ/A8BZ2gadNBhM q4ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719977772; x=1720582572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4GJ1xuGIg/34+KoYPJFdTKabqBtfsZIj48w+G481zs0=; b=G2sPakNEl94DPJ2kyb9E+HsH68fXu0m/snICB+oltchGf4G6VmHSJG3YBUAUwJOoMm fLc4NR3iBMUA3ZfRkJVWIvV9iXS/Xvpibb6QrghHC6ihngTzbUEeclh/jrrGfwn6Mf6V dfiPAAzopz1BN2VVVMPqUz+38siwjDaEf5UI2D9nyV2N9EbPQ2N9UX2Rv7yMHfpOzfM9 WvHK4fwDG7HBJcEv/WHsRPhBQF/0nIC5Fjb7qkM+Ck4E2cc88woMX6dhXqV79iP+MQC4 D8QKS1onBj/xOTNZCvF5AcTvR192HPL+SIqOIwHUPY0is8r00OGqfduNIos9jl/kF++M +7EQ== X-Gm-Message-State: AOJu0Yy89r7OZ4KCoxowG1twF3pxZhBjzx3h2fwbgyfL5vLlQyOzXD0H O1o2XDdo36pV7NEz3MP7YdAzea8iB0Xqt40Qix7eMUVzJVgO+RRQfi2HHg== X-Google-Smtp-Source: AGHT+IGWu9ztojNDm+/rzblVADUU5I3e7uzJjkc7CF3Q9H/fd/D62poomL6IIZKCcyzRNxj+qpYgHg== X-Received: by 2002:a17:902:e88f:b0:1f7:2479:a50b with SMTP id d9443c01a7336-1fadbce9d4emr72308405ad.54.1719977772012; Tue, 02 Jul 2024 20:36:12 -0700 (PDT) Received: from localhost ([198.11.178.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1faf92d84cbsm22478025ad.127.2024.07.02.20.36.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Jul 2024 20:36:11 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan Subject: [PATCH 1/6] workqueue: Register sysfs after the whole creation of the new wq Date: Wed, 3 Jul 2024 11:38:50 +0800 Message-Id: <20240703033855.3373-2-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240703033855.3373-1-jiangshanlai@gmail.com> References: <20240703033855.3373-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan workqueue creation includes adding it to the workqueue list. Prepare for moving the whole workqueue initializing procedure into wq_pool_mutex and cpu hotplug locks. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 7a33f958dcb2..c738b3024cc2 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5689,9 +5689,6 @@ struct workqueue_struct *alloc_workqueue(const char *= fmt, if (wq_online && init_rescuer(wq) < 0) goto err_destroy; =20 - if ((wq->flags & WQ_SYSFS) && workqueue_sysfs_register(wq)) - goto err_destroy; - /* * wq_pool_mutex protects global freeze state and workqueues list. * Grab it, adjust max_active and add the new @wq to workqueues @@ -5707,6 +5704,9 @@ struct workqueue_struct *alloc_workqueue(const char *= fmt, =20 mutex_unlock(&wq_pool_mutex); =20 + if ((wq->flags & WQ_SYSFS) && workqueue_sysfs_register(wq)) + goto err_destroy; + return wq; =20 err_free_node_nr_active: --=20 2.19.1.6.gb485710b From nobody Fri Dec 19 10:56:26 2025 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E9C82233B for ; Wed, 3 Jul 2024 03:36:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977777; cv=none; b=sLTQ/6mwMkz36NUgeYnFtP5AFSht6P2Or/jOijQxPDssjHPFjyyfDCNjHtIkFOAg328MgCsk43n7SQ8j9kH7Y2ECoSn5toIedOgFi9loPQP9O9kBPu+GeORWrdCSWVEdaXkhoNkm+8I6CKEijXoRFXA1GQOmpjuxZuIjICEH2Gs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977777; c=relaxed/simple; bh=hDB7ErbKyrBBqUbkW2fkYqhmVxL7yx6cpxPwDaA56RE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cNwwhKgNJLv+BRu2cMVBTtJCnhDj6cybahy7eltPHFm7lWwZao7fcdHTqSzb7DFs/K0zXBm4/FblGO913/Ur+BjFauwH2Hpt37U97YvDuiPn4OsSkV0t5gZNSUBHC//d/jInG3/thXBe9CiofiX+NCWTFBQltASCgrXhPwZXe24= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ePkE+j+i; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ePkE+j+i" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1fb0d88fd25so1500735ad.0 for ; Tue, 02 Jul 2024 20:36:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719977775; x=1720582575; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EmfbtIC8gkMyEyGpTQo1qvjL1gkxM76DMe3jF1C+RTI=; b=ePkE+j+iXYoBYvsx5gYk1aHr9iqk0c4t9S4Zs8EYd0Cw7MzZC7iueY5lDSEWI3TXhc g+qS0eevmZI12liLxMdl8jVYmbHt1KKHQFYizQRImEBtBTK4oFuZxmlhuZe5Fiph8/pu C9uwCraDBIMgxMwb+BMgw2a/wW1Y0QE8+H4ApGzXw7KA7V6iO+JsNZRLcmmYW4MEnOyC kxLpNw3KquvUO19h4/6UinjDrm8KYgBmkfiDhxIGmAtBUQy3dAq/OTgPsw1gw/BmWrm9 d/mJexV3MhOt+Ya2UfOhmOYda8+zYu4QGNiDjiC0HYH1IDJSzyiL1SDJjFF0KoJwZk/W ACWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719977775; x=1720582575; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EmfbtIC8gkMyEyGpTQo1qvjL1gkxM76DMe3jF1C+RTI=; b=CLbmBThHEfrvnPKuhyYod9bJgsnwALYOUqHLlt3GTj1NaiJDwqJiNYjNjV51KoERnr eRra1rg2wDvOHZEDEZDQaQPmsWCyhmZcip7wbd7FQXWzyny7qGOAmIt025S0fqHj706j U5v5c/XxrBoOnd0ZsUF2RSUXN7m1ql9EjeNUEDxNrTI5FZ+4WHrNkNtcvEa4BFdDLRBL FAyiYYmuySFS7KE6EjT5f4LNODqhqA64iLCSBtgS/ITZQ1EXuFYJRc8O6ZSggKK+YR7l o76ZCeFuC6XbXP/d7EZNaU5H8hr8t4T48MIlx2eNzhMeF8rhoihViCgn590nepCSzcga wcwQ== X-Gm-Message-State: AOJu0Yxcg/776xe9TFwGVm0oOlFsJRGLLlqFRrNlcQDcDASnuGOqDZGC Sc+9wrB5Zf6U+YPYKmrhRN6SXwug8po+2DTM3hs0AbD5CejPfJIWkbZbVQ== X-Google-Smtp-Source: AGHT+IEG4fucmaycNaPyVyLnYq9fuwy0hOO0gizsJVsEVzuilbHFrUVX4pMFgO5ui6l7OfmFwG9yjQ== X-Received: by 2002:a17:902:dad1:b0:1f9:c52f:d9a6 with SMTP id d9443c01a7336-1fb1a02400fmr7829775ad.2.1719977775405; Tue, 02 Jul 2024 20:36:15 -0700 (PDT) Received: from localhost ([47.89.225.180]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fac15388d3sm92046925ad.181.2024.07.02.20.36.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Jul 2024 20:36:15 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan , Waiman Long Subject: [PATCH 2/6] workqueue: Protect wq_unbound_cpumask with wq_pool_attach_mutex in init_rescuer() Date: Wed, 3 Jul 2024 11:38:51 +0800 Message-Id: <20240703033855.3373-3-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240703033855.3373-1-jiangshanlai@gmail.com> References: <20240703033855.3373-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan wq_unbound_cpumask can be possibly changed without wq_pool_attach_mutex or wq_pool_mutex held in init_rescuer(). Use wq_pool_attach_mutex to protect it. Fixes: 49584bb8ddbe("workqueue: Bind unbound workqueue rescuer to wq_unboun= d_cpumask") Signed-off-by: Lai Jiangshan Reviewed-by: Waiman Long --- kernel/workqueue.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index c738b3024cc2..cf1a129eb547 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5533,6 +5533,9 @@ static int init_rescuer(struct workqueue_struct *wq) return ret; } =20 + /* lock wq_pool_attach_mutex for wq_unbound_cpumask */ + mutex_lock(&wq_pool_attach_mutex); + wq->rescuer =3D rescuer; if (wq->flags & WQ_UNBOUND) kthread_bind_mask(rescuer->task, wq_unbound_cpumask); @@ -5540,6 +5543,8 @@ static int init_rescuer(struct workqueue_struct *wq) kthread_bind_mask(rescuer->task, cpu_possible_mask); wake_up_process(rescuer->task); =20 + mutex_unlock(&wq_pool_attach_mutex); + return 0; } =20 --=20 2.19.1.6.gb485710b From nobody Fri Dec 19 10:56:26 2025 Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B87733D969 for ; Wed, 3 Jul 2024 03:36:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977782; cv=none; b=HCf5XEzEy/qUcoDUBLCeTMfWmNWjHPALCqglJEuOIbctqyAhj7dcZqt8+6fPK4ILHJ+FbPLfab+qNf523UOOSJL3gUuyv8LmctQXEqlVZw0NV08hWBAo0EGiT7muJPNBM8gchFstdQCP6TVY0ewedFDLkTLfWoWlFjlmcpcOLQU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977782; c=relaxed/simple; bh=4eMOivaNyGq3AAE/jp7yke6+6PAXIf8vuuNWSJKJq1A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JFlJbVX3R/XhEbjybeQr83onoHuKtGQRe++35DZBKdaRJ2JmlwH8YF6X1jpnaH7Sz6tHBVe+P7UNCtDYUwe5eP0ZP2N29p6TIC+CBah1LwmsqGfzw/6rR5lU721hQN/IC8I8hlgzHxYbhpToD1dn9xX85qPTiFYXuhR5i5AvVrw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NPzzBHJa; arc=none smtp.client-ip=209.85.215.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NPzzBHJa" Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-71884eda768so2820127a12.1 for ; Tue, 02 Jul 2024 20:36:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719977779; x=1720582579; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Lws+ZLig0oV0HzzX4lmzZbMJvRrN++IbAE2lJR9pEa4=; b=NPzzBHJaYuvklHTSLFevUUE2H1REwKCbQsW4kgl3WQDF3sYWdt0e+NbTGk4ZV/0Vz/ ceClXZrrAwcaj8pQ8LMjs5QDhusXzrxMODTQHM2Z9ipMCDv3zE/YFkZjwzgbHXn5qrfW Cp0piMoGsNTua2grVFMvmq+uWk3LjkgyTDfFa5y1j5nMPVD0eapwb1lQd4L5ratM1atV X1xdc8IpaMx1NYPYDdgMMNbNRzBAUyEzsGEZOXkb0d+cpD2CpImbhh0oBgwvG0XmuSIC CzuV6bs2oMf1yfaGoxTPqcS2a9PxX2gguVLFJtM9B7gHA7wViALFnNG6oHXjn50wnDXF le/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719977779; x=1720582579; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Lws+ZLig0oV0HzzX4lmzZbMJvRrN++IbAE2lJR9pEa4=; b=k4Bi90pU8MfNNzYWA0n8LsNEaIGwSv0VIKiVZinbM5DKnvC/v26G3M2EmQrm3kMO2x ghqTMweaKMH9jnhR/YFVWjihP8mR65OLHr0/A3lIlxLSjUdnSo+SVHB3R2TvxPsBVlYr se0C+6JXu0EnzSOro0cJmOgN4so7V9sE3hF3PnkKCRO7SW/B6+A/ucZYQRwMlVw6nhsj fHEBLl7ioiSY8I12xXNPtXWREbEXYuOh94KIANzLtPJuIBjm0KmEvfevnJeV4+GUkv1d TUKYaU5BURUBqewtJawgX4bXeJFJEULlhfGlK1vItBWEtHWJFMD8DcK2NoIxmJCnZYnM /U5g== X-Gm-Message-State: AOJu0YztRzZy8UTLDuzpzSN/h1M+30LFwMPkqEELtEfXLjerdVm8KcCL c4hj5bKdF+0eQZBI2eZCxBU7ZQ+Be9F8O9r5WsbGnIM8XurKNtK53CJCTw== X-Google-Smtp-Source: AGHT+IF0l8eugTgllvLJTnwYtOC4OQsSPG3BE9H1d3jWcGDuYL2+VncEdV4DmPcltO0syFLtFPW3Zw== X-Received: by 2002:a05:6a20:3d86:b0:1c0:bf35:ef42 with SMTP id adf61e73a8af0-1c0bf35f31cmr852347637.3.1719977779562; Tue, 02 Jul 2024 20:36:19 -0700 (PDT) Received: from localhost ([47.89.225.180]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c97cf8fc68sm160047a91.36.2024.07.02.20.36.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Jul 2024 20:36:19 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan Subject: [PATCH 3/6] workqueue: Separate out destroy_rescuer() Date: Wed, 3 Jul 2024 11:38:52 +0800 Message-Id: <20240703033855.3373-4-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240703033855.3373-1-jiangshanlai@gmail.com> References: <20240703033855.3373-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan Separate out destroy_rescuer() to simplify destroy_workqueue() and prepare of reusing it in alloc_workqueue(). Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index cf1a129eb547..0dd9a12befb5 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5548,6 +5548,22 @@ static int init_rescuer(struct workqueue_struct *wq) return 0; } =20 +static void destroy_rescuer(struct workqueue_struct *wq) +{ + struct worker *rescuer =3D wq->rescuer; + + if (rescuer) { + /* this prevents new queueing */ + raw_spin_lock_irq(&wq_mayday_lock); + wq->rescuer =3D NULL; + raw_spin_unlock_irq(&wq_mayday_lock); + + /* rescuer will empty maydays list before exiting */ + kthread_stop(rescuer->task); + kfree(rescuer); + } +} + /** * wq_adjust_max_active - update a wq's max_active to the current setting * @wq: target workqueue @@ -5772,18 +5788,7 @@ void destroy_workqueue(struct workqueue_struct *wq) drain_workqueue(wq); =20 /* kill rescuer, if sanity checks fail, leave it w/o rescuer */ - if (wq->rescuer) { - struct worker *rescuer =3D wq->rescuer; - - /* this prevents new queueing */ - raw_spin_lock_irq(&wq_mayday_lock); - wq->rescuer =3D NULL; - raw_spin_unlock_irq(&wq_mayday_lock); - - /* rescuer will empty maydays list before exiting */ - kthread_stop(rescuer->task); - kfree(rescuer); - } + destroy_rescuer(wq); =20 /* * Sanity checks - grab all the locks so that we wait for all --=20 2.19.1.6.gb485710b From nobody Fri Dec 19 10:56:26 2025 Received: from mail-yb1-f170.google.com (mail-yb1-f170.google.com [209.85.219.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF13C39FEF for ; Wed, 3 Jul 2024 03:36:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977785; cv=none; b=lyQtjsWETAVZJxKDh0fXav2bAOL9l46KfMztQE/YtHydE5+QWvFF2uPg7Fof8STVhceNd8/kFkOsYTdpvyTcJocC0YLv9/9FWk+BNzspwq3iTfg9sMWCkmzcyBMUEMZo0zYTNRFCheHaJseciInYtluL9QiePOK/KP1jrXKiScs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977785; c=relaxed/simple; bh=QeCToHbPzWcpfWXXN0pXnJv0lCpyhXKVWAK1z/fEb7U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KI0XuhAWC5NpMLN5aA9zBcnuCyZYiuYKl4Ai+kcxmP4XVl6+3OYE7r/VymNslYPNC2Uf4UCSrfzSHLvL7QutCAMgna6ie01y6aDn9N3TEsbmEWQ84JO8Jef37MvT2LBSMO/RSUHiw1pVyEg+w2vzKI3JarOpQEe3jP56OxdazQ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Nxbk4nCG; arc=none smtp.client-ip=209.85.219.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Nxbk4nCG" Received: by mail-yb1-f170.google.com with SMTP id 3f1490d57ef6-e03b0e73f14so7253276.1 for ; Tue, 02 Jul 2024 20:36:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719977782; x=1720582582; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TvgJeZKGGm17GDbCpMSw1b9wVEFwhXIl35XREnILA4o=; b=Nxbk4nCGinfAYrtWFYIkFCZWs5ZAyaavpVl4zt6PAUl0KFRjRbKhFUugD0o3y77yhi C+p51xVN+ce62IXMHueiCGDpbA4qw3EB3W8jPhmQkyIqXrYuvYCCVYfPARd9MQIAkKBR JpzbOZAsZtzeXI8o2fzuo1uthykYdzum+nAx2q8m7s9zw6xdbfV/Lx2RJFuIPiXkquoK 8KPCgTqNywYgMhiN0NzzXPi/y7B4SeZKN+I557Vz2wUg5L5ekQxgmH5USwWU6X0wCpVN y5pws+k7FBDq7MZlPyYb1us0ZaeXOQSb+09dilhzWexK/RxH3+X26aCqojiNfun0pHOA 9LIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719977782; x=1720582582; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TvgJeZKGGm17GDbCpMSw1b9wVEFwhXIl35XREnILA4o=; b=HvSVE7wg5OERhMMWsNdLyIwA/v4XTsJo4cuMXrzbXwQdqp7uiCmzYJ7mLKPF2wuVQJ oLyuCyAQir5AvxmjEOmO1mAMkRtXOLNp31xkbP+DbzChLE/pXFpoo8jWA2gP9IDJz/wv FxEQ7qivgagk8hZ7fnqKLXin11V2UY+K5jUcMn5yzuQHBtKvqgqPWjh27QQ7cEYi9DFC sRQ4aJZy0eW+yq6V+a+WSb1Z/OHgKq2+NG8332cbI62oQch4dCj4SmFvwRnIMQTvhWej oVm9S+SDRfZDaQcSsN0yiWeYzUkMI9tIJQESXdr/cywgAuNQziB1AZQvH98AAhJ+woej Aq9Q== X-Gm-Message-State: AOJu0Yy/0JlyoNp5j1h1D+oPJIYuQ8ANXlfwEG4BydoFcNuaw1J0b8f1 tgHhYpbQroxXjJ4rfIFyGgPqRylONxIkxtxCSis91oPFFO+uwopEM/5kxw== X-Google-Smtp-Source: AGHT+IEMxrB5TU29/Sg7Vd7BZZpIeNwKEjqmuJYkr2a2JsIy1hWgorAseX3NUmjTy2OHvdVREG+ddQ== X-Received: by 2002:a25:ab32:0:b0:e03:a4ba:856f with SMTP id 3f1490d57ef6-e03a4ba8622mr3241381276.53.1719977782470; Tue, 02 Jul 2024 20:36:22 -0700 (PDT) Received: from localhost ([47.89.225.180]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-72c6c9e0ee3sm7250544a12.60.2024.07.02.20.36.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Jul 2024 20:36:22 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan Subject: [PATCH 4/6] workqueue: Init rescuer before alloc and link pwqs Date: Wed, 3 Jul 2024 11:38:53 +0800 Message-Id: <20240703033855.3373-5-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240703033855.3373-1-jiangshanlai@gmail.com> References: <20240703033855.3373-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan Swap the order of the allocations for rescuer and pwqs to prepare for making alloc_and_link_pwqs() and the wq enlistment into the same wq_pool_mutex and cpu hotplug locks protection. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 0dd9a12befb5..810ea55c0ac9 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5704,11 +5704,11 @@ struct workqueue_struct *alloc_workqueue(const char= *fmt, goto err_unreg_lockdep; } =20 - if (alloc_and_link_pwqs(wq) < 0) + if (wq_online && init_rescuer(wq) < 0) goto err_free_node_nr_active; =20 - if (wq_online && init_rescuer(wq) < 0) - goto err_destroy; + if (alloc_and_link_pwqs(wq) < 0) + goto err_free_rescuer; =20 /* * wq_pool_mutex protects global freeze state and workqueues list. @@ -5730,6 +5730,8 @@ struct workqueue_struct *alloc_workqueue(const char *= fmt, =20 return wq; =20 +err_free_rescuer: + destroy_rescuer(wq); err_free_node_nr_active: if (wq->flags & WQ_UNBOUND) free_node_nr_active(wq->node_nr_active); --=20 2.19.1.6.gb485710b From nobody Fri Dec 19 10:56:26 2025 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1E464778E for ; Wed, 3 Jul 2024 03:36:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977788; cv=none; b=r2sot/X6Plx0ZX5m8aaenNvpz4ASbNDU2xbh5tDKRQB47U6SzPQk4MgZKXFbf+83SecmFkqPUoMA6FPdDPUqMXsfMZCGFeAd2i8HV9k2PNFcu4Zq+Dj9d3Ov4cBsDinVyy/x4+NlYgTOY6DC0Z8qwULnQHzJbehZ8f8gpvU7nlo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977788; c=relaxed/simple; bh=0XFSVIlzk9KeflJQ/V8WREO6DICk2IQJGTOsOZswXeo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ol286qlPZXrogIQ7+yYWtI3tDPyZGjBW+yX0qTQUGFtsjvWNoY8kzgXGnoBxTLQvTZXfCQ8adUMotGxti9PCak3HR19UdBRZa8VdQ7yiY4b37GURb0E9V+5js6LKu6d5QkBVir94LE9ZY6EAde8GsDL9qfi69qu8DEsLa6qeRXE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=btFD0uGp; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="btFD0uGp" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1fb0d88fd25so1501415ad.0 for ; Tue, 02 Jul 2024 20:36:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719977786; x=1720582586; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4xK30j/brdYQ+0YiuZHRD9U+3lQnGwcSFkyDSPgUQ3s=; b=btFD0uGp12pcHsZ4hiJThEcHWbtu/p0wB6Ak5dlhfK2ErFMSP+RUuC7j1zINEKC8u/ P/dovoE3+hTzR1MsppKo7Xce7ze3K91XsYdXHRAytEFqXXGd/JHPAZaCJTUjxRvgnOqX 1kJ7Vs7gcRJ4KUKUVPQK0JaQtex0R7c5N1GSGuEhyb6AjWfY4d+sTMwPwi6Zqwyg1tDa Tn5mXP0cM+mLPiE0UHNR/a3biHZtKN6nwhuq84dRkOF3KS30jk8J+wjDt3YW9zo3GaOK p2riXOUau69PGiAjOZRQ89tySCRXMmf2Ig9jITHz9u2OOkfjUeQiI8wmPXrqa9SX+R9y cKHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719977786; x=1720582586; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4xK30j/brdYQ+0YiuZHRD9U+3lQnGwcSFkyDSPgUQ3s=; b=jVP6PtdGxEinNjpBB2Qs9LjqlpyFrtdkvtIOWgweG4ukpYYLiK2myF/Q+7pf1nSnNT oJJ/mLGKpSe4U5wwxILExbuYvfC+9AGWoLzwsge18AT3EviLZ9YsbfbW1IdZB5X6ZtmG L5xWK8/ik0KTZAJob2f7go9+wzLtf33R9kStfzAgdkfFc/g7iz737nDmNXmnnubufUIE +LSJXxdYegEHzOg474CHgMB9D6aKP7hUxQSYyano0j6MlzlnCa+dgSN/X3r2TISR4aM8 JTBScqQHunfeO02t/nfz4aSivR4X1Jr6gpDypJnq9sK0SnYWLIGFrXk98SA34Ys8mogR +b6A== X-Gm-Message-State: AOJu0Yyhs5jY5jD+/CdrINOqj65Aix0Z3MDzOYJA2cZRX4YkAeKSYMIJ A4QJueE4GNAukQz1O/ANcyFB6JvIUhTHnZck69vUlW/Qvew9ppnkZoUL/Q== X-Google-Smtp-Source: AGHT+IHrB4cPTnryLg4bjh1sDUAI8jWYCt17Cc918NeYncRR3Sw1Ny1uK5+mLk7X+FLJiUTdpy1gFg== X-Received: by 2002:a17:902:d04b:b0:1f9:e2c0:d962 with SMTP id d9443c01a7336-1fb1a124fa7mr4906345ad.31.1719977785802; Tue, 02 Jul 2024 20:36:25 -0700 (PDT) Received: from localhost ([47.89.225.180]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fac10d1b61sm92222165ad.57.2024.07.02.20.36.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Jul 2024 20:36:25 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Zqiang , Tejun Heo , Lai Jiangshan Subject: [PATCH 5/6] workqueue: Move kthread_flush_worker() out of alloc_and_link_pwqs() Date: Wed, 3 Jul 2024 11:38:54 +0800 Message-Id: <20240703033855.3373-6-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240703033855.3373-1-jiangshanlai@gmail.com> References: <20240703033855.3373-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan kthread_flush_worker() can't be called with wq_pool_mutex held. Prepare for moving wq_pool_mutex and cpu hotplug lock out of alloc_and_link_pwqs(). Cc: Zqiang Link: https://lore.kernel.org/lkml/20230920060704.24981-1-qiang.zhang1211@g= mail.com/ Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 810ea55c0ac9..3203c67ec4cb 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5472,12 +5472,6 @@ static int alloc_and_link_pwqs(struct workqueue_stru= ct *wq) } cpus_read_unlock(); =20 - /* for unbound pwq, flush the pwq_release_worker ensures that the - * pwq_release_workfn() completes before calling kfree(wq). - */ - if (ret) - kthread_flush_worker(pwq_release_worker); - return ret; =20 enomem: @@ -5731,6 +5725,14 @@ struct workqueue_struct *alloc_workqueue(const char = *fmt, return wq; =20 err_free_rescuer: + /* + * Failed alloc_and_link_pwqs() may leave pending pwq->release_work, + * flushing the pwq_release_worker ensures that the pwq_release_workfn() + * completes before calling kfree(wq). + */ + if (wq->flags & WQ_UNBOUND) + kthread_flush_worker(pwq_release_worker); + destroy_rescuer(wq); err_free_node_nr_active: if (wq->flags & WQ_UNBOUND) --=20 2.19.1.6.gb485710b From nobody Fri Dec 19 10:56:26 2025 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09EBC49634 for ; Wed, 3 Jul 2024 03:36:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977791; cv=none; b=JK8ACXRMoEnwDvM9M1vTDH8rn7+lurwy2h2/xgsHmJwFbx+cxmaBocL5j+2wuK0eF2/JrCGBG10a/T/RmM6Dz4c5K5J007Ym0EP+IYruom8mgDzEitMUVTI9CPAnbssqL8hKR7c2J+c6GzSLvAn3MB+q0JghKIb9Cu5eTELZHoU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719977791; c=relaxed/simple; bh=BnF5+73szihYvqj0zHvkWllxCXsDLQRml8Xqbpqd6HM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WnA/oIGDEtppJgPnMGvZW3EWAuVwMhQowHmT/EPrrsy15VBNlq1wB9+NQPAHky03gpVjSRMhrYWhqbv8MoBE9LtwFOY+jT/ns1pppZpyDseYFbUY6K7lT+jWFvIoBNIiyVvh+E4ycvs2OI4OtaPSyooc41fQpk3R5Buiphx7yYI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=K8/18Mb3; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="K8/18Mb3" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-7066a4a611dso3172532b3a.3 for ; Tue, 02 Jul 2024 20:36:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719977789; x=1720582589; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Za6+QBdr6NNfCmuy1fTZl6euOgrMMXbb01qR6g9M+Jc=; b=K8/18Mb3jEwjEYwug3+uB1JTy7/7PLaLX4PoUsQkFesXKfELvV6aaVDtvYVM3g74AT oVbLC5WUKItEuXFoROO8Ydveg91kE+RejjbWq/l9WclqjNbDm4nlcSvUA2EIkeLCqQUR ahsnlg3sEpF3QSywGtPE56K9g5ocf+H6gtr3n/UqRwd9ptum0waQ/Me9wJSog3Y5WNYB 2HYHUjJIQF8HvC6Kq6qGYGS/YlMR8AlopA1XDNzYBN23hHM2alA0UOik+CEJToE7/y+6 jSGEtiLkta7fT5wS6FPMUoJPgN5SrrG/WuuMD3Nt9Nc0SOpXkUZ7gGERczIZzkb5IQB9 tISg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719977789; x=1720582589; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Za6+QBdr6NNfCmuy1fTZl6euOgrMMXbb01qR6g9M+Jc=; b=ZESmcdy3q+kj7jfWCmDZrRv6q/bqnVklE3YRRqPny3N601gXoNpE1Tcp+q4SIyiRgU p7RQKG6uQNVQMHaRegy9mjOYodMerihuzF2VYQeyxXJsjSjVFMWPnOswT2fA10DyOZzw uPe0D3ff1WOl6/AkEoTkWGtGEVBtXcM8KMgBdJen4ek0ysZAfNgZdqYenSNeCp1STy1L oPGXpZYei2Sjud4785UBhqb7FfJGWvnOZJ3s1mehJdZZQFyFHtCu+GO2uHvFbdB7blvu Ez2eZQ3U8mZhNsJfbUOkW9m4f4HGXJPAHvefgpBhoolsbcYYQJ50hAIxa4nx2NQ/UWq5 k3gQ== X-Gm-Message-State: AOJu0YwrZyRXVdur8e7ZvDAsUvTm7ZLSbQWXzXfItmxVkehUb/vTLLnE J6TWHdDfghnQpKN/mnubxBblr6NWpesKbpzGKLP3NW/YtaGGBCxpF4Atlw== X-Google-Smtp-Source: AGHT+IFjLx53PQoflrPWAjSDQDEOMNQXgVRKDKckBapjadOSjXPcX6m3NX4U62RjFW31ufOx+sUbsA== X-Received: by 2002:a05:6a20:3948:b0:1bd:1d6e:d444 with SMTP id adf61e73a8af0-1bef60f3db9mr10995950637.2.1719977788757; Tue, 02 Jul 2024 20:36:28 -0700 (PDT) Received: from localhost ([198.11.176.14]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fb19fabe98sm2311925ad.86.2024.07.02.20.36.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Jul 2024 20:36:28 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Lai Jiangshan , Tejun Heo , Lai Jiangshan Subject: [PATCH 6/6] workqueue: Put PWQ allocation and WQ enlistment in the same lock C.S. Date: Wed, 3 Jul 2024 11:38:55 +0800 Message-Id: <20240703033855.3373-7-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20240703033855.3373-1-jiangshanlai@gmail.com> References: <20240703033855.3373-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan The PWQ allocation and WQ enlistment are not within the same lock-held critical section; therefore, their states can become out of sync when the user modifies the unbound mask or if CPU hotplug events occur in the interim since those operations only update the WQs that are already in the list. Make the PWQ allocation and WQ enlistment atomic. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 54 ++++++++++++++++++++++++---------------------- 1 file changed, 28 insertions(+), 26 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 3203c67ec4cb..c910f3c28664 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5114,6 +5114,19 @@ static struct pool_workqueue *alloc_unbound_pwq(stru= ct workqueue_struct *wq, return pwq; } =20 +static void apply_wqattrs_lock(void) +{ + /* CPUs should stay stable across pwq creations and installations */ + cpus_read_lock(); + mutex_lock(&wq_pool_mutex); +} + +static void apply_wqattrs_unlock(void) +{ + mutex_unlock(&wq_pool_mutex); + cpus_read_unlock(); +} + /** * wq_calc_pod_cpumask - calculate a wq_attrs' cpumask for a pod * @attrs: the wq_attrs of the default pwq of the target workqueue @@ -5425,6 +5438,9 @@ static int alloc_and_link_pwqs(struct workqueue_struc= t *wq) bool highpri =3D wq->flags & WQ_HIGHPRI; int cpu, ret; =20 + lockdep_assert_cpus_held(); + lockdep_assert_held(&wq_pool_mutex); + wq->cpu_pwq =3D alloc_percpu(struct pool_workqueue *); if (!wq->cpu_pwq) goto enomem; @@ -5457,20 +5473,18 @@ static int alloc_and_link_pwqs(struct workqueue_str= uct *wq) return 0; } =20 - cpus_read_lock(); if (wq->flags & __WQ_ORDERED) { struct pool_workqueue *dfl_pwq; =20 - ret =3D apply_workqueue_attrs(wq, ordered_wq_attrs[highpri]); + ret =3D apply_workqueue_attrs_locked(wq, ordered_wq_attrs[highpri]); /* there should only be single pwq for ordering guarantee */ dfl_pwq =3D rcu_access_pointer(wq->dfl_pwq); WARN(!ret && (wq->pwqs.next !=3D &dfl_pwq->pwqs_node || wq->pwqs.prev !=3D &dfl_pwq->pwqs_node), "ordering guarantee broken for workqueue %s\n", wq->name); } else { - ret =3D apply_workqueue_attrs(wq, unbound_std_wq_attrs[highpri]); + ret =3D apply_workqueue_attrs_locked(wq, unbound_std_wq_attrs[highpri]); } - cpus_read_unlock(); =20 return ret; =20 @@ -5701,15 +5715,15 @@ struct workqueue_struct *alloc_workqueue(const char= *fmt, if (wq_online && init_rescuer(wq) < 0) goto err_free_node_nr_active; =20 - if (alloc_and_link_pwqs(wq) < 0) - goto err_free_rescuer; - /* - * wq_pool_mutex protects global freeze state and workqueues list. - * Grab it, adjust max_active and add the new @wq to workqueues - * list. + * wq_pool_mutex protects the workqueues list, allocations of PWQs, + * and the global freeze state. alloc_and_link_pwqs() also requires + * cpus_read_lock() for PWQs' affinities. */ - mutex_lock(&wq_pool_mutex); + apply_wqattrs_lock(); + + if (alloc_and_link_pwqs(wq) < 0) + goto err_unlock_free_rescuer; =20 mutex_lock(&wq->mutex); wq_adjust_max_active(wq); @@ -5717,14 +5731,15 @@ struct workqueue_struct *alloc_workqueue(const char= *fmt, =20 list_add_tail_rcu(&wq->list, &workqueues); =20 - mutex_unlock(&wq_pool_mutex); + apply_wqattrs_unlock(); =20 if ((wq->flags & WQ_SYSFS) && workqueue_sysfs_register(wq)) goto err_destroy; =20 return wq; =20 -err_free_rescuer: +err_unlock_free_rescuer: + apply_wqattrs_unlock(); /* * Failed alloc_and_link_pwqs() may leave pending pwq->release_work, * flushing the pwq_release_worker ensures that the pwq_release_workfn() @@ -7005,19 +7020,6 @@ static struct attribute *wq_sysfs_attrs[] =3D { }; ATTRIBUTE_GROUPS(wq_sysfs); =20 -static void apply_wqattrs_lock(void) -{ - /* CPUs should stay stable across pwq creations and installations */ - cpus_read_lock(); - mutex_lock(&wq_pool_mutex); -} - -static void apply_wqattrs_unlock(void) -{ - mutex_unlock(&wq_pool_mutex); - cpus_read_unlock(); -} - static ssize_t wq_nice_show(struct device *dev, struct device_attribute *a= ttr, char *buf) { --=20 2.19.1.6.gb485710b