From nobody Tue Dec 2 01:28:17 2025 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE6E12C08DF for ; Fri, 21 Nov 2025 14:53:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736818; cv=none; b=jwGhOlPOJBmHLtm9UVFOnYOYO6mwxnZ6CQTfMLk2na6WxAnKlb1y5OJ1u9hj0a4TBlkl4ZEeFia84b19SGSrUWWEXfffR99mOcqHFDg6t018ZpQEmnLNJsje09F6CPmpRc3aWctxIP41Nyi39+N5vl35qLTIU/bEOdgbVZnSRTE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736818; c=relaxed/simple; bh=Tl+vYFYrEF3vFgi4Z0sqB1CoRFc95UCaLAAb0/k1ZbM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=i0rpZDTdJxI2AfjoT1Y9ycy5Aj9CdPt2Fhkg9sOoEzfVmXWobjI6dFoG0dy4NysyPr71pAp+PPqpOH9CUDRvqmKfnZ5dy7Qbb0UwUv0lHYGO/90N/2LGKnP91CPuzgMxSMSKP/LWpoveLuyQxxbS0LDpVA9NFUSuqGauaLjehTw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=b8onUAm0; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="b8onUAm0" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2953ad5517dso27186405ad.0 for ; Fri, 21 Nov 2025 06:53:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763736811; x=1764341611; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=peo9T+HChxcH8m7niIk4iKeYpL4OnJbw1rxueSVVfHY=; b=b8onUAm025IaW9Cbg3AKX9hOBSQLP/DXIZtKALWyIetJkSFE/G/lkEheR+aIIZ/sOL PTprsrJ3VTk60eWIMiVje0whSHgeHzWctCHQM53SrpQcXRs+Pv3U2a5T0qep6uqraLrl O6vnnXYSoCnU0yiT1n+uXUbGnjnBZD3cZ5OseUKzooSbuE4xNhsC8OvVX0qDEEstvL5H s/j7lvuK+LjO4jFuJFR2wgyktAKKolgAmnkQat1yhlMA7uiDtui45MH/SBpcwh6gbp2T GhVnMUAqvbOERm6LKOrb1lcasIQMhCSNo2bIm97/1XCVG6y5uTgwFrMZQdznGKcYXW97 5DJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763736811; x=1764341611; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=peo9T+HChxcH8m7niIk4iKeYpL4OnJbw1rxueSVVfHY=; b=niWLfPaK5bcjaZKqGK/zA1FfKfwSN3I0yQSjfXOjT8qghYDf3/rJu0GcmZrNY1WsR/ DjuyAf9rzR4lYa8QZ/HDfGie0pElUcofe0ZIH41xNa4DHen2MFLV10gZ1zexgySe2K5q QVWagsmHhQSLp8k6Pc4SVwyDtHmv536745UBvBGrOh6NcJrCJ3OYVbwfB+LxKOwnjvqT yoDKMM2uuow6+rOu7ugw67eHTg2sJc2BWYZyRFhjYx8VAdpoeualP1loZ7PE9T0jWcpG 7iXlg/KyAAVQ5YcNNAFMA5OvjuTAL52xX5mFslcKe8KB1PsIT/x0RXhivO1se2hihy88 2HGQ== X-Gm-Message-State: AOJu0YwK7pagq9l3bYdjsFEJcVvuursX/+9Spqi6vSMvYd4xKUnj6NpN ggfZinzV5Y0uqxjnrPZTmnVmFgNRO97LkGNDPWerqkqJZc1RTOZn92gkWU7O4HF8 X-Gm-Gg: ASbGncuuaGbsq2oS/8HU2U1jrSynsrfOIAqBTxyZRUqcxl6etwb2Jso+de7IYpeF5Vr sPfdMFgQl0pgsXjyQuHhAN7Al9nGSKHptAJwiaFExVgp0rZYKimKEBVFpPy6XNoX5aGXLtILTX5 7nI/3VIyQIaAoUOE6n7jP/Y7xXQG8smFHFz0VOT90XKGKu2/rCT4cJpz92is+gU4AJiX8fhPxI8 jgC8Hkc2awSPkQ0qgpHQzFkQ6Wio1DpC+ysAoajTSWW8Kd57U7l66RvZTwvTe+Z9JFyJo7/fY/z f+ZltFa01czi8c1B2B9JHlFiaM+9pY9qMG92YqN50+0MQLUoRYjSxQKLZGySgiUUzxIQ19i9RsV PjUZE2VqGPMJxzPR1svuO6vPdyBcK/XlH3336pxrlt1bmB+LLFjwQGoRrJc5LlyS/uZjw0ZEjEH dPAqxSvennhNw= X-Google-Smtp-Source: AGHT+IF/BMIuRT+ea/pbIu7n1/+c+YEFoKjxloUbKtmryJ5wmQeEYVMysbcrZQ9QrWvZp7qQ8bbEUQ== X-Received: by 2002:a17:902:d489:b0:298:970:b24a with SMTP id d9443c01a7336-29b6bfa1944mr31867685ad.58.1763736810398; Fri, 21 Nov 2025 06:53:30 -0800 (PST) Received: from localhost ([240b:4000:bb:1700:7b36:494d:5625:5a1]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29b5b25defasm58866415ad.49.2025.11.21.06.53.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Nov 2025 06:53:30 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Tejun Heo , ying chen , Lai Jiangshan , Lai Jiangshan Subject: [PATCH V3 1/7] workqueue: Factor out assign_rescuer_work() Date: Fri, 21 Nov 2025 22:57:14 +0800 Message-Id: <20251121145720.342467-2-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20251121145720.342467-1-jiangshanlai@gmail.com> References: <20251121145720.342467-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan Move the code to assign work to rescuer and assign_rescuer_work(). Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 19bc7ea931d8..c60584a39fc9 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3439,6 +3439,23 @@ static int worker_thread(void *__worker) goto woke_up; } =20 +static bool assign_rescuer_work(struct pool_workqueue *pwq, struct worker = *rescuer) +{ + struct worker_pool *pool =3D pwq->pool; + struct work_struct *work, *n; + + /* + * Slurp in all works issued via this workqueue and + * process'em. + */ + list_for_each_entry_safe(work, n, &pool->worklist, entry) { + if (get_work_pwq(work) =3D=3D pwq && assign_work(work, rescuer, &n)) + pwq->stats[PWQ_STAT_RESCUED]++; + } + + return !list_empty(&rescuer->scheduled); +} + /** * rescuer_thread - the rescuer thread function * @__rescuer: self @@ -3493,7 +3510,6 @@ static int rescuer_thread(void *__rescuer) struct pool_workqueue *pwq =3D list_first_entry(&wq->maydays, struct pool_workqueue, mayday_node); struct worker_pool *pool =3D pwq->pool; - struct work_struct *work, *n; =20 __set_current_state(TASK_RUNNING); list_del_init(&pwq->mayday_node); @@ -3504,18 +3520,9 @@ static int rescuer_thread(void *__rescuer) =20 raw_spin_lock_irq(&pool->lock); =20 - /* - * Slurp in all works issued via this workqueue and - * process'em. - */ WARN_ON_ONCE(!list_empty(&rescuer->scheduled)); - list_for_each_entry_safe(work, n, &pool->worklist, entry) { - if (get_work_pwq(work) =3D=3D pwq && - assign_work(work, rescuer, &n)) - pwq->stats[PWQ_STAT_RESCUED]++; - } =20 - if (!list_empty(&rescuer->scheduled)) { + if (assign_rescuer_work(pwq, rescuer)) { process_scheduled_works(rescuer); =20 /* --=20 2.19.1.6.gb485710b From nobody Tue Dec 2 01:28:17 2025 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D76BA2BFC73 for ; Fri, 21 Nov 2025 14:53:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736819; cv=none; b=uqnSrEwPCqPQj1inp++0GgwvWXegRgGl0hHc6JejLkbMti+EKlVE/DRXeqEMESuev2cjKB0z3IP/YA2ppZz7RgZCrEefUy3RmW7h6/Ft0biKWsjYQRpA6IP6Gu2MMewXbMKK/hTkJiqSmzuSNlshsfdCv5REhcfE676UmegTsLE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736819; c=relaxed/simple; bh=ce3gPsKvy0mX0LtEhRY1aHsxfcRX9tj04hL9siQgck0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=X3Jgeduip7LXUvY59B67WErBCSqjRwxuyJuIu5rL/rXmKIewauQyIJca7yEhuX75wKsQ5yIduHr31YEZAzmhWfnAgMgYdMUvlebk1bhZG3xCyqXbUnOyblN5RKpHJeK4O0N+aBQ/l9XkxxdFMxAWYbL10kRTDBMb2UrE2niy6kE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=m23mOoGc; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="m23mOoGc" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-3436a97f092so2579333a91.3 for ; Fri, 21 Nov 2025 06:53:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763736815; x=1764341615; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2DcFk8ItUkceG01mzjMMcYxL/epZi+IR73cveIeK4b0=; b=m23mOoGcduz7CpNa57CkS750yN7DNO0S1s1Ks4RG1WZOVhJxfokI1QDXvPp6M7btmV UU0/H2KP7A4+psajY5kRnyVAYU+8NNV5r/HITqYLum+h03pXtdWHg7QTEXfkwKiVQipf 5045CcsVvDNVEo4grqEYgcZMUVEyc+RvHrIvBJuMLlznJ/Va3nIM8AXFJYQbARav3672 ViUm4E1W+99leJnaK+IBwvHka/SHAktlg/XtGoZpw6FDtdqkerMPpYX4+AUJrMv0Q0sg mERI8V1ZFVEELKbY9a3K0hZkwshZ4FolibZ6udLleiBrBh1i2UrTfxh0RSybjYdRYmYB 7k/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763736815; x=1764341615; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=2DcFk8ItUkceG01mzjMMcYxL/epZi+IR73cveIeK4b0=; b=S3rAsNlP/q1/5W7Q+JGowLc/O73g25jvapEWrJSQ65rKW/tpGI3mY6HeaTr0Lm2IJY 8auQZTOE2/5nAbkdS5jD3ROL23UTI1BMGBCK3S5MhEGK8pAVo48zO9FH96+TkCfFvInR YgPxhLnnmCw0gCVyf8C49qO32NEc+TqWeli97aTz4AUf6rlMP+yrO36Xc2/SmZLd0G+x zvk5Kr4fVr6Ywfaczt4WJBwZ2ph+mQ2KoOj+2uc+DOtk3MZt1pfhKu86HSnRmlvexfXA 5Ejy2k3ieIy66PH+OkZ4VLWrJoytgKHEVUZPowZiOyIddAdaZeLWoW/cYqvkEYT8dsfk LB+g== X-Gm-Message-State: AOJu0Yzy1CWto17hlouL0aJgzDyzg1AoQg2PdhxmAO67ziOxrbowtRws gl9KfJDrNwBIHjCldb0KRkmIFrkYldP87tO/JWeh0mVK9kT4ZVnmK03lV0AZ96mp X-Gm-Gg: ASbGncttFVB9zhm2dWCZzIc8T0MLqCaXyObCGOYZBKiFgbf2Yfe5xn6TK5eA7+Wnv74 AYyvq2w0zpbcck//dr+X3Dv5pYzox1Gb2RGZ99O9ktWDQfoB31BS7XDka5H4P5Ai/8yDv3bHEi0 +2avamMkkYoqtFKlfCsqov9H4B++jf/hRlM/lBfMRnv3FmmCY+SF2duWztYgXHYHJ4Ve4J1Kpc4 +gSa2CPFyx1+GqzPzwYgP1GbJyMjwzCGYuINkx12MaetNr+CCHhjeonLxnkV3yZYNxyCL2RlZd+ ifV7eUZODwWx/kKs8VspY15DgJqGg/Ll7fbfj78sTxNOb3PIeB+SWZCXynAmtZwITxIveHuy7M4 W69RvjhJYGOvAOEG5F+EggEpcHCr+y4Lv2MpaxmoSbgvHMsitEm6YlfiDjs5gUAQMsccBrp340k 5R0i9log5Tx9zYwZ9HGaOlGA== X-Google-Smtp-Source: AGHT+IFEP0pAxCEZ1nmiYT+OREY1eaH0pgAJTj+Ugh+GQ5qaK04ydI+Jf5KBSn3nLZUhL/ICJ+MVQg== X-Received: by 2002:a17:90b:4a:b0:340:dd2c:a3f5 with SMTP id 98e67ed59e1d1-34733e55f59mr2727432a91.3.1763736814650; Fri, 21 Nov 2025 06:53:34 -0800 (PST) Received: from localhost ([240b:4000:bb:1700:7b36:494d:5625:5a1]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-bd75e8b37f1sm5895864a12.14.2025.11.21.06.53.33 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Nov 2025 06:53:34 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Tejun Heo , ying chen , Lai Jiangshan , Lai Jiangshan Subject: [PATCH V3 2/7] workqueue: Only assign rescuer work when really needed Date: Fri, 21 Nov 2025 22:57:15 +0800 Message-Id: <20251121145720.342467-3-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20251121145720.342467-1-jiangshanlai@gmail.com> References: <20251121145720.342467-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan If the pwq does not need rescue (normal workers have been created or become available), the rescuer can immediately move on to other stalled pwqs. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index c60584a39fc9..932581d93edb 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3444,6 +3444,10 @@ static bool assign_rescuer_work(struct pool_workqueu= e *pwq, struct worker *rescu struct worker_pool *pool =3D pwq->pool; struct work_struct *work, *n; =20 + /* need rescue? */ + if (!pwq->nr_active || !need_to_create_worker(pool)) + return false; + /* * Slurp in all works issued via this workqueue and * process'em. --=20 2.19.1.6.gb485710b From nobody Tue Dec 2 01:28:17 2025 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFCEA248898 for ; Fri, 21 Nov 2025 14:53:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736821; cv=none; b=OgQSvF9UsGq2yVqmLlZtjlOfLo1qNKatc97fEJD4ZUq9JvuxXXh2H9a5Umg5vE0spU2NB8+uR6f9qH7x2W5guW2o6uAiWpthWX2m8vuF4kLCHw5xkn6LnhT3Vzz6DTtNcJ9CJrWottC2w5xK8DWjONPobW/cfJZ4L5xLm7B4tN0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736821; c=relaxed/simple; bh=k9iZ2DgdKU/18GNdd2pJu0GViMRNc3S/1SMZlUWB/eI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=s6tpYbUrGCMU416lCkTxY9BFaTb9pLdBX23od8cl9cOzVNmmKDe3MVZ87ZWlu5vcRoGfEOyfO85RglJq1SuGT1VicSPjnw+hUnjFjhOKBdGQfDngLjAgifcq5EgJiVraw70YA5Y8iN46dV2t2j7c6/DK3sCvuLBcTBwRnkwNTOA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eGLt6m/6; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eGLt6m/6" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-298456bb53aso25993655ad.0 for ; Fri, 21 Nov 2025 06:53:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763736819; x=1764341619; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rQ7Sd+AEjcf6Y3lyv8l7R72b7jJab1Ykn2VcIHqlJ4g=; b=eGLt6m/65BYa5SbQ/xB5umwK+q2artLH5VBmSbmm0xoAxd0PbQkGki6UUzpwRQeksQ 6g+tnLox8yS2PclDkTPO1WezDCcpeC8Okd3qES8WY0uokGekfA5TJuljQqw5uMI1+2iq FNhPLMw3WhJQ7fkTktHkaiEjEOoMZkNi83/npT0dPFuKpA/KBoHEut0AzMor8t8h6yKL vc2MPB4jhoVTM9RWLecaNunYMgNg0XQ/h0wR46DFJ7NP6RuQQZoJZlHqvINhp3rhSiNG jxX8lKDAWFpIEaEersvaT6pFGibBHhYDCq9iOeg1b+D0J/gNaAkFH/rYeAYxVQE5mf+0 XUig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763736819; x=1764341619; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=rQ7Sd+AEjcf6Y3lyv8l7R72b7jJab1Ykn2VcIHqlJ4g=; b=ZCTPm8f97NvJ5mw2fcJhYjGO0+eaU8o2i2RBVvnc8+IEAtoxCc5qF+zga8SagHedfr /58f4N4igIcX140wosBjQpk4knyfwOZcd2fWZjiSPWaENvZiyjTFrMmJbgzm18HxDon3 CposAIqmypZryD3F2BOIGeAzAgVYjKgzhNpcw7khbcE4C/VfC0yWcRZxprkP6MewEju5 WUCnDAqbSBvztGOprH29UaEjpeENQo3YFFOLjEjJIF2VIBJMfOUcmEuLMUKgH+HiEYi/ rzZ2yLOBENLev8Aj2cWldVLNnEMKxKRxxPgDXRLQw6QiVR3gIgD1sbOB+ResrTMXOVbp uDrg== X-Gm-Message-State: AOJu0YzrcBhudUafVOJzUJt8+XLZueuTv8mBHMifGX/8a1YFI4+P+vzY HSL8HrIXZW3VlI+itHKKMA5Gu8K2InD/rxpuf3MbKfzUiyaqqSEUhBFdzUYMaEkk X-Gm-Gg: ASbGncshUGfXwtR/y0J6bkKl9ez6n/p3j8ClSasaYdlheKD6+hSI5q3sDG456xF5uwr ER5IEpkKjDVMMU42Lal00eMi94Xi0Rmxu14N9ILdllx65XtP02lSSrD1p2xWqk6jcyiAoO/OIXn Oal9jKLcNCZ5Oi82uVTI+TSihG0IYF9KCnbxNWnXUm/TI0hm2I7+CdNAmYa5QZ7VP5dEyySOsoK JHKPs0teyKKXA98Nm1Vf4HZ4yClP6zSgSv4UffPG2Gf5xH80upMw4A/PXUjvhjRBbpvOPfyxP8r DJ5RhHFkfxtC9qV3vExAgFw/3Hk2MN4s1yPcco4C1RtDc7XKfdBOG6uQEe9dn/jaNeKKTyV9NUE k7B11qWbNyRsSqeXLnA85n/dbmDs7Qs2SkMvNJ49CvHrG9ULJd4HJI3+2Xx9NblOdhvHTUndPW0 wwwnNCcCAh3qG5 X-Google-Smtp-Source: AGHT+IH1hZqJxv9HyD/4d0WjqaAItNa2SouO+A/T3wt/60WciJKs4wVY01X97Y72dBzkQFNMGkqtWw== X-Received: by 2002:a17:902:ce05:b0:26c:2e56:ec27 with SMTP id d9443c01a7336-29b6c404f19mr29979155ad.19.1763736818585; Fri, 21 Nov 2025 06:53:38 -0800 (PST) Received: from localhost ([240b:4000:bb:1701:388a:b68c:56c7:adeb]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29b5b107cc2sm60541535ad.16.2025.11.21.06.53.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Nov 2025 06:53:38 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Tejun Heo , ying chen , Lai Jiangshan , Lai Jiangshan Subject: [PATCH V3 3/7] workqueue: Don't rely on wq->rescuer to stop rescuer Date: Fri, 21 Nov 2025 22:57:16 +0800 Message-Id: <20251121145720.342467-4-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20251121145720.342467-1-jiangshanlai@gmail.com> References: <20251121145720.342467-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan The commit1 def98c84b6cd ("workqueue: Fix spurious sanity check failures in destroy_workqueue()") tries to fix spurious sanity check failures by stopping send_mayday() via setting wq->rescuer to NULL. But it fails to stop the pwq->mayday_node requeuing in the rescuer, and the commit2 e66b39af00f4 ("workqueue: Fix pwq ref leak in rescuer_thread()") fixes it by checking wq->rescuer which is the result of commit1. Both commits together really fix spurious sanity check failures caused by the rescuer, but they both use a convoluted method by relying on wq->rescuer state rather than the real count of work items. Actually __WQ_DESTROYING and drain_workqueue() together already stop send_mayday() by draining all the work items and ensuring no new work item requeuing. And the more proper fix to stop the pwq->mayday_node requeuing in the rescuer is from commit3 4f3f4cf388f8 ("workqueue: avoid unneeded requeuing the pwq in rescuer thread") and renders the checking of wq->rescuer in commit2 unnecessary. So __WQ_DESTROYING, drain_workqueue() and commit3 together fix spurious sanity check failures introduced by the rescuer. Just remove the convoluted code of using wq->rescuer. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 932581d93edb..943fa27e272b 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3541,10 +3541,9 @@ static int rescuer_thread(void *__rescuer) if (pwq->nr_active && need_to_create_worker(pool)) { raw_spin_lock(&wq_mayday_lock); /* - * Queue iff we aren't racing destruction - * and somebody else hasn't queued it already. + * Queue iff somebody else hasn't queued it already. */ - if (wq->rescuer && list_empty(&pwq->mayday_node)) { + if (list_empty(&pwq->mayday_node)) { get_pwq(pwq); list_add_tail(&pwq->mayday_node, &wq->maydays); } @@ -5903,16 +5902,10 @@ void destroy_workqueue(struct workqueue_struct *wq) =20 /* kill rescuer, if sanity checks fail, leave it w/o rescuer */ if (wq->rescuer) { - struct worker *rescuer =3D wq->rescuer; - - /* this prevents new queueing */ - raw_spin_lock_irq(&wq_mayday_lock); - wq->rescuer =3D NULL; - raw_spin_unlock_irq(&wq_mayday_lock); - /* rescuer will empty maydays list before exiting */ - kthread_stop(rescuer->task); - kfree(rescuer); + kthread_stop(wq->rescuer->task); + kfree(wq->rescuer); + wq->rescuer =3D NULL; } =20 /* --=20 2.19.1.6.gb485710b From nobody Tue Dec 2 01:28:17 2025 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 603A42E6CDE for ; Fri, 21 Nov 2025 14:53:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736825; cv=none; b=cjw8ICpz5dYy7Cyl1QMauqw/i9QPdne0htQzCOQSjpI6bGLv2UW5R5c34NIM8/Zpexvl201ubw88L5oQVok/9mcz0kipESIAtk7DWjD1rf/skCsywe0qe+A+oTHR4pLyUj5xozYP4MlUwYTsITNnOlww2nEtKM6hhOQgWN7ISOM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736825; c=relaxed/simple; bh=4AluHisqWo8poypxlftnMHX4uSJuo2VGr086g1FiszM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Y1Xjj2LVv5lFjfyOzrtKozZciAVhv4+UdYRhCj2XeymwbB1uV8rSykLfO0NHqcOm9FBJP2U4vCTjCUqgVqy248zzrXi03LiOshDYnWadaPqgdPcsDJoUbDliERGtZDHub50gUfl9ooglGiNHhqinMqAomsAMLq9vHVe7IaqJdLU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JgFCpBpI; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JgFCpBpI" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-29844c68068so22782545ad.2 for ; Fri, 21 Nov 2025 06:53:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763736822; x=1764341622; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CtR9meOzOWHKnJZaBZzQ9II1g+OHQE4AnIqOa3Hn1OY=; b=JgFCpBpIdeFC6SbiP6P4r2lxRwUL0pCWbZE2peQkkZ+TeThF1+v+asFuKb2WXq6R/8 06pIqfWpUdT9aPw0CWD/59kPMIkf3oAElxKuZnDZDEj/xpuO0HWJbWj60uovA5UYGDgi Clxdc/EWWPjb1Z9SWxqUIj1ko3H+h78DanNF06kHCYlHEU7hj0zj2ddh/E30h2Es5Ko8 7Bues/kf40WXw9mQhcHv18vO4dQahteTczNy8erSXex9kNA2AX9x/uYHRzpUXjvxlx6t TUzplnWRopINwI4KxaRDA68RnJUA1khq4bQiRWFfAnG47UvZAvZ8uEmQWTLUIWqM2qKS 9oXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763736822; x=1764341622; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CtR9meOzOWHKnJZaBZzQ9II1g+OHQE4AnIqOa3Hn1OY=; b=i9I42Xrf9GzKYYJVolBpGjbCoo9850aCu8iMqYR92hEEK82h0RaBNHikPaSgZZUo9v +IvIxvNe8LJxUVhH9cD/OnbT+yxoXPrFWD0QcNQfpqLBxQ/NVOu7iBDn0s5nyQVqM+9R NpZIYd5oeuZ6hReajpktLeroMC46B9A8OTDoNDKaTpkY2V1CmylSZ5Cg5+EIwSUOefSO izl9jW1+2L4OGdmhbYRJrYCkqpYsjYy4LtRZniSJhz9k+lKIwFfpSgVDtGSqOKAcF0HN 6wEIgwpc9CyWIQYYLZdDVSFJIQ9RPLm8507PPBlS2QP9bhkrd27LOjMEjimgX0QbK6iB 0ZRA== X-Gm-Message-State: AOJu0Yxz8ou6FoBEP84jFu4abXwY/PX42Fw3GBeZf3wMZTT2lCd/DpLc f2149KyerhyFGw1fYymyHDfXdqQLDcfrlMgPfCz4MKQNx+QLerivOmdLf1CcXr5z X-Gm-Gg: ASbGnctu4CedOamdQ80dZVEy9DadQpfTDTUNuDoVzvd+3mi3XxtTIPxdlL0Iag/Km7w J+DRnT9YLK8RPEhyP4D+E2jybgrT5gQCtBdg/1LpYpW/V1KdoKiaMLXmVrfz5Ov0r5/ceilKSQv Uu0Lv3XGqKFw1yLtSEEI0KNV+S24J9X1aun5Y8mKPvPDdb7yxHlVkMoAK5U4hXoHm5oHYgJbq5Q jndEVkzblYDZeoV026CpebvoBkpJdjc9sT1mHq2A//LMw5Es9wgdU3JY01gFGeum6pTxDjhtlZm k80P2tDWzOU4UIRaBRgfg7Cair0g9c70ZTnd6YtB1HMqB3L7culPDxKzsvt+eGM6SAAGx6Q9ASG M2XkAN3h2NpqHZwhdGncOGXyf9lOodUBVhzej5AP74clhsJdtr5YY8LVpHpN7J+tdiP9F60veO8 7klXwAuLdnApQ= X-Google-Smtp-Source: AGHT+IF7tBHo/VRGa1ZvSvdUHpsnCMd+LWcZytsFK3yUE3RX1Jny/JlTPb2Iz2BHSDLlPO+qIvbt2Q== X-Received: by 2002:a17:90b:5588:b0:341:8ac6:2244 with SMTP id 98e67ed59e1d1-34733e7677amr2755582a91.9.1763736822414; Fri, 21 Nov 2025 06:53:42 -0800 (PST) Received: from localhost ([240b:4000:bb:1700:7b36:494d:5625:5a1]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34727be2fa7sm5809443a91.6.2025.11.21.06.53.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Nov 2025 06:53:42 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Tejun Heo , ying chen , Lai Jiangshan , Lai Jiangshan Subject: [PATCH V3 4/7] workqueue: Loop over in rescuer until all its work is done Date: Fri, 21 Nov 2025 22:57:17 +0800 Message-Id: <20251121145720.342467-5-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20251121145720.342467-1-jiangshanlai@gmail.com> References: <20251121145720.342467-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan Simplify the rescuer work by looping directly in the rescuer rather than adding the pwq back to the maydays list. This also helps when max_requests is 1 or small but pwq->inactive_works has a large number of pending work items. This might hurt fairness among PWQs and the rescuer could end up being stuck on one PWQ indefinitely, but the rescuer's objective is to make forward progress rather than ensure fairness. Fairness can be further improved in future by assigning work items to the rescuer one by one. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 24 +----------------------- 1 file changed, 1 insertion(+), 23 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 943fa27e272b..3032235a131e 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3526,31 +3526,9 @@ static int rescuer_thread(void *__rescuer) =20 WARN_ON_ONCE(!list_empty(&rescuer->scheduled)); =20 - if (assign_rescuer_work(pwq, rescuer)) { + while (assign_rescuer_work(pwq, rescuer)) process_scheduled_works(rescuer); =20 - /* - * The above execution of rescued work items could - * have created more to rescue through - * pwq_activate_first_inactive() or chained - * queueing. Let's put @pwq back on mayday list so - * that such back-to-back work items, which may be - * being used to relieve memory pressure, don't - * incur MAYDAY_INTERVAL delay inbetween. - */ - if (pwq->nr_active && need_to_create_worker(pool)) { - raw_spin_lock(&wq_mayday_lock); - /* - * Queue iff somebody else hasn't queued it already. - */ - if (list_empty(&pwq->mayday_node)) { - get_pwq(pwq); - list_add_tail(&pwq->mayday_node, &wq->maydays); - } - raw_spin_unlock(&wq_mayday_lock); - } - } - /* * Leave this pool. Notify regular workers; otherwise, we end up * with 0 concurrency and stalling the execution. --=20 2.19.1.6.gb485710b From nobody Tue Dec 2 01:28:17 2025 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 555A634676F for ; Fri, 21 Nov 2025 14:53:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736831; cv=none; b=Qz7OasoSRy5VSwAPQgxz5i0z9Da5h/VeXakEGr3Xj468zpTWQ3pNPQ4uEh+Drln7SYnewDfwlDS4QGh6+YYICZ4HkdSgrdzRQJTohvqD75eJR1fjHCbo0FwBfZxZ0MtKXu7lgi6l2zLxHr06pwtWk7sNZHQ/BhdF47IteqRzc40= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736831; c=relaxed/simple; bh=14RzNSw0iMTNH8xOoc2TPH06WRB4giCpnLlrDiKeCFU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KD1ZTlRMrIoc4DCbA/PGMjuPdrWThnaryfSGo045TwJ6i6TfykdgSXU31QYHv8OcjtKqvC3OEw5y7i0a5nsOku6HvKYTQnBUdwx6MFaBBOZPpjBfuBddKb3/MHIIHQqMK171pJR544HSP5JOWVKueN6zU6wAXy/jIYgJMF8HaC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SvUaW5Aw; arc=none smtp.client-ip=209.85.215.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SvUaW5Aw" Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-bbf2c3eccc9so1757088a12.0 for ; Fri, 21 Nov 2025 06:53:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763736827; x=1764341627; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0Pi7iXAYcJk1Q+rYpdW9NsZWiIUVlt0px9qmxFiyQGM=; b=SvUaW5AwNsn7iKjwenq/iHLSJkKqTgR6mFDH5nLboMsPAUuJlVZSRvA11wIZ+Tvdxc zw2XOgVY8/2tAQjoHwIaHpiIItgonnO/KSg8uyJbYkkb5uKm974suNLvqBv3poaXlXer oAGyrUqH+oEqDmOWFlRuZm4QywiLKvN4uZJl7pNjWF+e1lkdmZGR4Tz2olq1t1Yc2Cst Jvn5+ZZ4StnKW9PHICAjxrj9GWVs9DvbHSsiob5QrtEUk/CsDLL9U+EcjFwL9zHy8YqO 2FezuJD7+kAL33s5QysfjmD4oAOMZLQa11SPBHOjiTT4ASILjLMfnA64m+e4b6VEXmAJ Y2Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763736827; x=1764341627; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=0Pi7iXAYcJk1Q+rYpdW9NsZWiIUVlt0px9qmxFiyQGM=; b=HjPGu6QkAnk1meYjIvZysQCMkV4EwtNaV6Y9CR0Daa+swrUhT0Y9EA+BD9TmbE7IkS BY/ttXj9GHYQIWw0+7RdXSHhOXOT/LH4UTMksPj6MfDD4YH3KK5SFa4QQ/OP0SRq/GMu xqU9y/1DwobI7Xpj8ntBpi95tr7tISKStSKN5rrahvCmdEkLkCF9Pa0suBXNu1RVwEou ZojZkBvViMKCzQI1/WeZj84O7v+nqsEI8OUYcZM2Kfxugu3YSY7RgVM0jWeHtoikaR8q SZ5XpXMURuvmL/58ww0Kwb4buKfF5QV3KVTuGK0aeFM6WIT7ywQ5G+C1p9sMhTMYkTmX 1CXg== X-Gm-Message-State: AOJu0YxbGt0vbfvcKquEH1RUA9juzLn6WYmFz9crKnpR87t64y24+bGr do/+NDxnuMySRs1guHFQ8VE93uHmosRNQ3jSHD4rdK/qVfPVt5Itayzobk2OVpU0 X-Gm-Gg: ASbGncvDSbwyhRGOMVmNYXquYLPxz48Ttc5G6kpP3Nnk0EteofgHG9pLKIIL9k3X1j9 wrrke+FAgD05mj39M25nhuJx8vm1U7IheNb/Li0k/J9Oe2ytYqIHmKf1goV6BuCHsHUGXGKr7zJ z384d4TzHQAoGzodvKxjqPqbkXaFAQQEQZ0LhlzioNIFL+LmaOwkQu5N659KN71pDRzveqpSvfj hBHY2hFNV9yqYLMYZ9G/S37XHTZ9l85ZG81ySx1eNjYKWMKfNl7ZXIuavb8qL/t0LHVmCv+7AKU ccnHqXNerHH9GH5NSH/lRRMlb0is3FRrbuZo+rKIeYgaQ6lj0b30cW+4lGOGcPVb+G1t5ZInEOt 3nHb7i1ORerekG9SjCkZ2WZD4VuuHXRmjewB8fafdJqUIyNpuXMtNkTZrL5IQqmYaxA9bAC1eDk nCbbs9fcpmKes= X-Google-Smtp-Source: AGHT+IEiOAApG0Ko/1NI5Mc/BgBZCrtEGVbCO5vDTVBLn05aqwQUzaMg9NZfBGRmEzUXqe9jNXlQ3g== X-Received: by 2002:a17:90b:4f8a:b0:343:78ed:8d19 with SMTP id 98e67ed59e1d1-3472985089cmr8156616a91.7.1763736826713; Fri, 21 Nov 2025 06:53:46 -0800 (PST) Received: from localhost ([240b:4004:a2:7900:ecf3:dec8:8c1e:4f5]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34726696ba5sm5912877a91.2.2025.11.21.06.53.45 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Nov 2025 06:53:46 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Tejun Heo , ying chen , Lai Jiangshan , Lai Jiangshan Subject: [PATCH V3 5/7] workqueue: Process rescuer work items one-by-one using a cursor Date: Fri, 21 Nov 2025 22:57:18 +0800 Message-Id: <20251121145720.342467-6-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20251121145720.342467-1-jiangshanlai@gmail.com> References: <20251121145720.342467-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan Previously, the rescuer scanned for all matching work items at once and processed them within a single rescuer thread, which could cause one blocking work item to stall all others. Make the rescuer process work items one-by-one instead of slurping all matches in a single pass. Break the rescuer loop after finding and processing the first matching work item, then restart the search to pick up the next. This gives normal worker threads a chance to process other items which gives them the opportinity to be processed instead of waiting on the rescuer's queue and prevents a blocking work item from stalling the rest once memory pressure is relieved. Introduce a dummy cursor work item to avoid potentially O(N^2) rescans of the work list. The marker records the resume position for the next scan, eliminating redundant traversals. Cc: ying chen Reported-by: ying chen Fixes: e22bee782b3b ("workqueue: implement concurrency managed dynamic work= er pool") Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 56 ++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 49 insertions(+), 7 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 3032235a131e..49dce50ff647 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -286,6 +286,7 @@ struct pool_workqueue { struct list_head pending_node; /* LN: node on wq_node_nr_active->pending_= pwqs */ struct list_head pwqs_node; /* WR: node on wq->pwqs */ struct list_head mayday_node; /* MD: node on wq->maydays */ + struct work_struct mayday_cursor; /* L: cursor on pool->worklist */ =20 u64 stats[PWQ_NR_STATS]; =20 @@ -1126,6 +1127,12 @@ static struct worker *find_worker_executing_work(str= uct worker_pool *pool, return NULL; } =20 +static void mayday_cursor_func(struct work_struct *work) +{ + /* should not be processed, only for marking position */ + BUG(); +} + /** * move_linked_works - move linked works to a list * @work: start of series of works to be scheduled @@ -1188,6 +1195,16 @@ static bool assign_work(struct work_struct *work, st= ruct worker *worker, =20 lockdep_assert_held(&pool->lock); =20 + /* The cursor work should not be processed */ + if (unlikely(work->func =3D=3D mayday_cursor_func)) { + /* only worker_thread() can possibly take this branch */ + WARN_ON_ONCE(worker->rescue_wq); + if (nextp) + *nextp =3D list_next_entry(work, entry); + list_del_init(&work->entry); + return false; + } + /* * A single work shouldn't be executed concurrently by multiple workers. * __queue_work() ensures that @work doesn't jump to a different pool @@ -3442,22 +3459,33 @@ static int worker_thread(void *__worker) static bool assign_rescuer_work(struct pool_workqueue *pwq, struct worker = *rescuer) { struct worker_pool *pool =3D pwq->pool; + struct work_struct *cursor =3D &pwq->mayday_cursor; struct work_struct *work, *n; =20 + /* from where to search */ + if (list_empty(&cursor->entry)) { + work =3D list_first_entry(&pool->worklist, struct work_struct, entry); + } else { + work =3D list_next_entry(cursor, entry); + /* It will be at a new position or not need cursor anymore */ + list_del_init(&cursor->entry); + } + /* need rescue? */ if (!pwq->nr_active || !need_to_create_worker(pool)) return false; =20 - /* - * Slurp in all works issued via this workqueue and - * process'em. - */ - list_for_each_entry_safe(work, n, &pool->worklist, entry) { - if (get_work_pwq(work) =3D=3D pwq && assign_work(work, rescuer, &n)) + /* try to assign a work to rescue */ + list_for_each_entry_safe_from(work, n, &pool->worklist, entry) { + if (get_work_pwq(work) =3D=3D pwq && assign_work(work, rescuer, &n)) { pwq->stats[PWQ_STAT_RESCUED]++; + /* put the cursor for next search */ + list_add_tail(&cursor->entry, &n->entry); + return true; + } } =20 - return !list_empty(&rescuer->scheduled); + return false; } =20 /** @@ -5141,6 +5169,20 @@ static void init_pwq(struct pool_workqueue *pwq, str= uct workqueue_struct *wq, INIT_LIST_HEAD(&pwq->pwqs_node); INIT_LIST_HEAD(&pwq->mayday_node); kthread_init_work(&pwq->release_work, pwq_release_workfn); + + /* + * Set the dumpy cursor work with valid function and get_work_pwq(). + * + * The cursor work should only be in the pwq->pool->worklist, and + * should never be queued, processed, flushed, cancelled or even examed + * as a work item. + * + * WORK_STRUCT_PENDING and WORK_STRUCT_INACTIVE just make it less + * surprise for kernel debuging tools and reviewers. + */ + INIT_WORK(&pwq->mayday_cursor, mayday_cursor_func); + atomic_long_set(&pwq->mayday_cursor.data, (unsigned long)pwq | + WORK_STRUCT_PENDING | WORK_STRUCT_PWQ | WORK_STRUCT_INACTIVE); } =20 /* sync @pwq with the current state of its associated wq and link it */ --=20 2.19.1.6.gb485710b From nobody Tue Dec 2 01:28:17 2025 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43F7E345CB9 for ; Fri, 21 Nov 2025 14:53:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736834; cv=none; b=c13KeEqtwkOh18vbDEYTtlx3BMxGj6d8OVg7MBqjpSjLPuKk4vc0s2irsSqr0zOvWS8U3UWXFrr2pAlx4oajczWDFpxef9n87tQjDINULeqUQ8HhpvXfiyw2GuJJ1NY5KDe4Eq2Zuz+DOskM7/9Xj77hAQu/x59XOVc9nBxeghM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736834; c=relaxed/simple; bh=/RWFACrE7ta8w5uDCG4CIvTyekhq3MpL4bkiyTJ3FwA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=p/mEG21rd/2ZFK2OJifSyGlca9oKA3qJF8Lyd7sXgB6h8Bs5a+MrIXlERc7pzx3Ug1oXbFZFIOrdLAyLtu4AunUhLSQuFNP0t4qQQVdBLrH4T/mfjsWyPc0o1reAXAP8ZyzHBmC+QPidomGY1I8vsOJyPXBrsQW8PvNPtOMLBwI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PWqwueig; arc=none smtp.client-ip=209.85.216.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PWqwueig" Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-3436d6ca17bso2249817a91.3 for ; Fri, 21 Nov 2025 06:53:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763736831; x=1764341631; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+YqoquxtoxaKApcpDx2mUm88KYst5CvhVDV5Vwut5xw=; b=PWqwueiglqZFIX3/T9FqYNoKRlA79/KoG8Vce3BBCBCgtBtMVOZoDbYgCpxgcpm6NZ 6Qkcj6hcT/T6S5wicP84PFdR1klAxMxUJW2kay521l1LhxBhMJA+DEsAurmO3VxftBwV 1zCrvJ7NxEHQioaKiilCgfqGXqZyTlC/ZG2g5mIlr/FRFbhPRHMkoYX3EZZQFHIerN9U poK5rZ69wAElTUOyyvhUCdf9BeNHorPan/BBCT/0l+EOTMb9Keo8mgnbXLHTw5iZE46L 3rOpLDOQz8feg815r6WyB76UPJKcd5SbkqHl1HSMTXEHxGxsE+uD8nYq3Cl9M2OJTdnU /tZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763736831; x=1764341631; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=+YqoquxtoxaKApcpDx2mUm88KYst5CvhVDV5Vwut5xw=; b=VrI/HdrwmN9K/4pMrc+TQlGaZ2KACGoWmsDfwoLT//BZyrwaG+2BM+Olm9K3XFIRaj +7KpYvy2DHhkfVVgAxluukpTJxj0KwqOxgEX6O0gqAUh0ZSvghVTjdnr9Bqv+gDYfL46 ZZNI0KJ89RYBp0A1ToQz1PLV36m3Zc/43DPx2KmPpRKpGcii4MVEYo6S4ZIRGCov+E0m FxYxrBBWtS51y9JEumUuD8DsHExHQ52sh+pNJfx5OiUid90iABq9QzQWR8ye4ZabrG/9 /zfwySwh9d9hg3Ay3YO5STHIpmziL5Dqu3afUuvK6v8wms4qGWcuMQKSRdqDOZUytLe6 RfZA== X-Gm-Message-State: AOJu0YyT2oJOIBX88aclh1r8BAe8db/QN4AzPM6yIez+DI2A8Oo96zRl frwHQ5G7E6KCdPsJtLWB9d26RCTHLAxUO5VXX8iOYxQlgCv+FPGcoGvvqE7Cp+Z+ X-Gm-Gg: ASbGncs5TVSyTNyRsrjtoDLTGFOgvTNVObl4FVonZZoyynRDYJNb88A5Bo5aJ/S3ioH /mi+n2jBcIDWE5RwaIiC0CTw2SF5yDCjy1W/wVsqwjdHYnkjVHpILkWBvcWTj90e4d60m8Rb4II ffWQ0yu5SP0riBMi6iE2fQbAKbs/JOKZinSKhylje27Bhi1SGhrmPiv8cXdn2Om7I7xf9tE0TUL KizrbWw0I3pGNYjovENgbfyMJFIlzoU3nAufSGx2LPm1JT1nz1idTsWiUhJhWGoYdRcqkhbD/vk RN8iIkdDdCSPPcBtCALnxm2lMDKfoT6xcs2/9FR1p4wQlQ7Em+gYTQaKVafxBGM49rljAgZFuVo D2q2tIfs5MnzZz/RaOA+L7ylBusoM+499+ntcoBmTRO6+276PktUngQ0Ta64INVBklObCSbxdoT ndUGVBmb1mAq8= X-Google-Smtp-Source: AGHT+IHlnTvompo3KDRTFF+ohjGY2TAYG+XGSUoETjvNJR/0cvYo1pGz64CRX0roFJqKh+pt3GgHIA== X-Received: by 2002:a17:90b:4d0f:b0:339:d1f0:c740 with SMTP id 98e67ed59e1d1-34733e45877mr2698974a91.1.1763736830918; Fri, 21 Nov 2025 06:53:50 -0800 (PST) Received: from localhost ([240b:4004:a2:7900:ecf3:dec8:8c1e:4f6]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34726696b52sm5968449a91.1.2025.11.21.06.53.50 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Nov 2025 06:53:50 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Tejun Heo , ying chen , Lai Jiangshan , Lai Jiangshan Subject: [PATCH V3 6/7] workqueue: Limit number of processed works in rescuer per turn Date: Fri, 21 Nov 2025 22:57:19 +0800 Message-Id: <20251121145720.342467-7-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20251121145720.342467-1-jiangshanlai@gmail.com> References: <20251121145720.342467-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan Currently the rescuer keeps looping until all work on a PWQ is done, and this may hurt fairness among PWQs, as the rescuer could remain stuck on one PWQ indefinitely. Introduce RESCUER_BATCH to control the maximum number of work items the rescuer processes in each turn, and move on to other PWQs when the limit is reached. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 49dce50ff647..9bc155545492 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -117,6 +117,8 @@ enum wq_internal_consts { MAYDAY_INTERVAL =3D HZ / 10, /* and then every 100ms */ CREATE_COOLDOWN =3D HZ, /* time to breath after fail */ =20 + RESCUER_BATCH =3D 16, /* process items per turn */ + /* * Rescue workers are used only on emergencies and shared by * all cpus. Give MIN_NICE. @@ -3456,7 +3458,7 @@ static int worker_thread(void *__worker) goto woke_up; } =20 -static bool assign_rescuer_work(struct pool_workqueue *pwq, struct worker = *rescuer) +static bool assign_rescuer_work(struct pool_workqueue *pwq, struct worker = *rescuer, bool limited) { struct worker_pool *pool =3D pwq->pool; struct work_struct *cursor =3D &pwq->mayday_cursor; @@ -3477,7 +3479,20 @@ static bool assign_rescuer_work(struct pool_workqueu= e *pwq, struct worker *rescu =20 /* try to assign a work to rescue */ list_for_each_entry_safe_from(work, n, &pool->worklist, entry) { - if (get_work_pwq(work) =3D=3D pwq && assign_work(work, rescuer, &n)) { + if (get_work_pwq(work) !=3D pwq) + continue; + /* + * put the cursor, resend mayday for itself and move on to other + * PWQs when the limit is reached. + */ + if (limited && !list_empty(&pwq->wq->maydays)) { + list_add_tail(&cursor->entry, &work->entry); + raw_spin_lock(&wq_mayday_lock); /* for wq->maydays */ + send_mayday(work); + raw_spin_unlock(&wq_mayday_lock); + return false; + } + if (assign_work(work, rescuer, &n)) { pwq->stats[PWQ_STAT_RESCUED]++; /* put the cursor for next search */ list_add_tail(&cursor->entry, &n->entry); @@ -3542,6 +3557,7 @@ static int rescuer_thread(void *__rescuer) struct pool_workqueue *pwq =3D list_first_entry(&wq->maydays, struct pool_workqueue, mayday_node); struct worker_pool *pool =3D pwq->pool; + unsigned int count =3D 0; =20 __set_current_state(TASK_RUNNING); list_del_init(&pwq->mayday_node); @@ -3554,7 +3570,7 @@ static int rescuer_thread(void *__rescuer) =20 WARN_ON_ONCE(!list_empty(&rescuer->scheduled)); =20 - while (assign_rescuer_work(pwq, rescuer)) + while (assign_rescuer_work(pwq, rescuer, ++count > RESCUER_BATCH)) process_scheduled_works(rescuer); =20 /* --=20 2.19.1.6.gb485710b From nobody Tue Dec 2 01:28:17 2025 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A37C231A578 for ; Fri, 21 Nov 2025 14:53:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736841; cv=none; b=ciDuIrJ9u3Wt2RjUQnFPYqmxPVXvmGwmuMCMp1EChtttCgKSxHrwuCFhFLPqPwGCmHm6io9nROt91l8gkQhKW2LU2YBqX84QbQ7KqMtg6Aj30v87xZXXCORnTn3p++OEQeW0/ljoNzlkhfMMD4hWNGI3rGQ6c19jwPC7N37acR8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763736841; c=relaxed/simple; bh=ytmMm38EiE85qgzZR8avey2qDbohVdLk9mQE7f85GDI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=flhQlsZH2ZQk7XTlLKLrlf3SiTZetugy+lSBeCEsD7oWf/q4GTOBcudjeKruXUwhE2bc9gK6JWj29IRXiCyI0bx75ywoT2SmaBqY6t2+sOAvp1xFhO3Nq17d7d9eTULllzSByXsapq8HT6mVrtfEJ/+ykKk9/LVO8PnUirGzDXA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OY0PHOEQ; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OY0PHOEQ" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-7acd9a03ba9so2369928b3a.1 for ; Fri, 21 Nov 2025 06:53:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763736836; x=1764341636; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P1QGhBuRCqIVwoTMitPXGLd+RJLkpfV0sHccooqbqpc=; b=OY0PHOEQDjA0D8VDlN3aqUNCgZweH7esf6gFySs3s+MT0ajRQZh2F8pFl1XIxa2hsj 8suzskMCmAbKT/Tcf5RfWhZoXDWFcsU4ZpJ9Boura7mLzT4F2kAbFb6vmS1FT/Kk9asp jFo9/qUKQ9av2h9kZ93PgXkyC/mxe/sD9uUCO8eUF6vo8qt5zwp1T8xjabJOB2oqUTB4 NrfKF2wXlGu2DNxAIQbaJ6nbVwENixiYMSPXCA1myV//ejKftEa/8wfGGZz0bGpEW7la z3hQIRI4V58c6aSPgYX3+rgTwx1AxkZEapaEj3UhVhcpkYtxLO69DO9eRBDd+YDYY7ds +0xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763736836; x=1764341636; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=P1QGhBuRCqIVwoTMitPXGLd+RJLkpfV0sHccooqbqpc=; b=lulXmGzqim72Kw+//VH3HyXBgtcRA6orPxj+ZMjDjPArt2gYNoLacb0Bee8ra3opo0 3CuTMjFoEg6BbEMZYqDA/m9/G1w1+l9jtFawhay++ZnUvEu+PCI2Gyw2+ka6E6w0LY9n Hc2S+reQWAqIRl0FOc/5OBwkOW+SNuaQEoLJn3XtOMRoba9qq7KC8VPoGhbQYBReti89 2TF80eUIqo0xf6WK4P9nbbbIv2qANwnS/CAFfnkgBLj00hh9ZkHp+JsbpMiPe1uh30CP OAzvgTWa6QDEUppx3hqhaFaKRkp5atNvGQFo74AxxEYpB/NAkQU048xDwUUfUyoM12RV BSLg== X-Gm-Message-State: AOJu0Yx3uc/PUtCzg6TqLBOqawp7DOUCy/w+vo1PYtJNineB8qrTzcM4 TT2TTrxlFNd7+G4f/qoK/dAn/gyz3TxCa5Upl0362aaiB+qqdUk0X+kmuBguEiAI X-Gm-Gg: ASbGncvC66Y9ZMfTTAKSxjeBJuQ6ZWv44QUr02C+eeVzuvuMe64OogHtot3mGpk+qmg UkyjsNvsiapPsv4xQj91Oz+KQ92Ny9hzCZfxgn2XI6SpXvVHPzB6dmY0I1DcbbAADC3TYHzS6tp 2g4oKLsZaVh95qX1PGRvtdAyi95hvHU2vSuHGSBdftPS/moHYkgsvpYfLH4wD6x/EgUY7roFt2j 1odb8vCKJNDuQ7chNFBHSz0PNmMyDbErfgIpl4IC5sM9tosZaNFkqgHOlp91Ufzz0YjZFeSyJ00 K7MLVBYtFPjk/31aJBwscIUqaAph1EbNvUKtWPNW74rIN38BUtBkZ44o8Fia1JxobODtNyQ9sCH 6lVTpPYnUkqOhIaq566hnjxsgPf25+H4qpJ8qAMdJO63t+58dImy5yqMLLSTkXu9E+7USkpM2v8 saPxonEmfvZvQ07x6s264+8A== X-Google-Smtp-Source: AGHT+IGk5d52c0FU06wlPaGcL7cijPH3sfsVkPqTVfzjw+NrTNKj3nG1FAE4PaH3bpFHFvVMnZRdSQ== X-Received: by 2002:a05:6a00:94fa:b0:7a2:1b8a:ca22 with SMTP id d2e1a72fcca58-7c58eaff9ebmr2618491b3a.25.1763736835659; Fri, 21 Nov 2025 06:53:55 -0800 (PST) Received: from localhost ([240b:4000:bb:1700:7b36:494d:5625:5a1]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7c3f0243b3csm6331930b3a.36.2025.11.21.06.53.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Nov 2025 06:53:55 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: Tejun Heo , ying chen , Lai Jiangshan , Lai Jiangshan Subject: [PATCH V3 7/7] workqueue: Process extra works in rescuer when there are no more to rescue Date: Fri, 21 Nov 2025 22:57:20 +0800 Message-Id: <20251121145720.342467-8-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20251121145720.342467-1-jiangshanlai@gmail.com> References: <20251121145720.342467-1-jiangshanlai@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan Make the rescuer process more work on the last pwq when there are no more to rescue for the whole workqueue to help the regular workers in case it is a temporary relief and to reduce relapse. Also limit the number of extra works. Signed-off-by: Lai Jiangshan --- kernel/workqueue.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 9bc155545492..86acf2a9cd41 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3473,10 +3473,30 @@ static bool assign_rescuer_work(struct pool_workque= ue *pwq, struct worker *rescu list_del_init(&cursor->entry); } =20 - /* need rescue? */ - if (!pwq->nr_active || !need_to_create_worker(pool)) + /* have work items to rescue? */ + if (!pwq->nr_active) return false; =20 + /* need rescue? */ + if (!need_to_create_worker(pool)) { + /* + * The pool has idle workers and doesn't need the rescuer, so it + * could simply return false here. However, the memory pressure + * might not be fully relieved, and if the regular workers later + * go to sleep and the pool runs out of idle workers, mayday + * would be triggered again. + * + * Do more work even if it doesn't need rescue, to help the + * regular workers in case it is a temporary relief and to + * reduce relapse. + * + * Unless the limit is reached or there are other PWQs needing + * help. + */ + if (limited || !list_empty(&pwq->wq->maydays)) + return false; + } + /* try to assign a work to rescue */ list_for_each_entry_safe_from(work, n, &pool->worklist, entry) { if (get_work_pwq(work) !=3D pwq) --=20 2.19.1.6.gb485710b