From nobody Wed Apr 1 09:43:45 2026 Received: from mail-lj1-f176.google.com (mail-lj1-f176.google.com [209.85.208.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F185377ECD for ; Mon, 30 Mar 2026 17:58:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774893509; cv=none; b=MsJRDvZYdyxKcg5t6VBswRAOiFWaS4iHa3ngqjyOBo8PLLfsZ58Ok6SVOPcnbKdd2HaR8m9KnrezYGbywa8lu0cwurgA6fnv2gHk+91gjoyDXpcoCrKoF7VoPJ1m/F02TOMuiqo7o2+TzbelwEF340cv6qmhCC9n2EpunJAuAXk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774893509; c=relaxed/simple; bh=yPvS2J/ZCbjRmieP2dNL7twgf/uz6gRsLwnFtNneEIE=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=nKWQ67Xrau5GWknChS+FyS/8WK0AgMCt8UlZ83mXwYC/t0Qp62y7AabFcdoIj+pVaPI04toWrXt6giICbYT9jrv9kiD4jc8+zWAlLVcX15+sm1EjcaNbmd7X+aRdHKUdI0E2DXO2ywKvW6UAJG0lMnptdk2xnZrP0dGHTlo4nTs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Waj2dr53; arc=none smtp.client-ip=209.85.208.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Waj2dr53" Received: by mail-lj1-f176.google.com with SMTP id 38308e7fff4ca-38be5d7c27cso46128511fa.0 for ; Mon, 30 Mar 2026 10:58:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774893506; x=1775498306; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=IebezMsHMybLTK7aOLHXktLPT4Jkv4NWF4iYsG8JrgM=; b=Waj2dr53MeDNWD+/hHyoTgxajjNZa23feL88cHip6BxuGGCqa7sS5q3SD3MhnORwxB pGx2vPvEV7zur44WGWi2W8SALfBU/kpuj2HYDHzmQUGk0AVuB8Xr5YqZKTcZ6vWxpaST zCsaq12DZhoZ5GUk34xswWDDEGcZeHugiBgXGrrfvobRRtuWDYQalTW1JStFs0+xP3FC 75UY4msFpkok7rQ6l3wy4FL6nApXg8SX/W5rnYoNnSrGpc/XS8XC08LdIrj8srbBl7f2 TMmX3bdrbyFqUIB3sIvl96maKSg/HxZZzJ2k7cen+bFuRlfKb41qUof3XHk+kOMSd1oq 7NzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774893506; x=1775498306; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=IebezMsHMybLTK7aOLHXktLPT4Jkv4NWF4iYsG8JrgM=; b=lrTqhwhpUkA80wTqZPTlpr/oniDLS3NZjzEC4Cgir3ISuArjQdEdkpkMIWJX83D6SP xt7upGI3nnUgudHGtJqkqyagFV0LAI7FhjfVNhOSaKQbkmjEWYq/47yYZzzjAIjY4OnC S/c9Co/QlSa9ZXMV5Nx6atHxONArLb5vN4U6lhX9xiiSYUf9pz68qGvS0JopuqJE7CMX 4l6bwYriR18JUnIPF9Gaum0ECAMipDv+pHNYAidoIVpnHoUAbjXuQ7xtI+xeOGmHHlde O4AlLKWdeP3YQTe4ok8+CMO0Jdv3mdHQHbxrjx1EQZUD2D6a8StVB+2Ls1LEI0l4YynE cAnQ== X-Forwarded-Encrypted: i=1; AJvYcCUO9jzmDIydKZhMysyMDlgOuzuCeiAlOK+f+xmU6mOvOTmbEfpCe6z7P0e0gDAIxhlVBcdSYH9V4pqL2NI=@vger.kernel.org X-Gm-Message-State: AOJu0Yxfmij2p1WitMgxjr09dxcm4jLOgG3X5A1GZt3byJQwSThlpK3J zLX9w5DTkBZTdz5viY6YANOmsTd3kSe7wbjiSyeGGvt/CKjtBkklG6ee X-Gm-Gg: ATEYQzw5DKCzbYoazvBQ5zRU72BxstzWnNNlbZWTgo6FaH+iNL93n4ll2Ecrsn+uwCm yACZ3MqyDovI9d/3QwbXZIaX4Oy1qHLfS9z7R7kSHGPzdsCYYildJn2k9TO7QWjqLhZBKRAvt/5 TLeSOdqrhkjvg41Hca6Vex/V0gm8LMIpDS0/Ij9dcCCzNZ7lEKG+5q1llp23msCL3PBU/LjLDjV P7wlAsyk0jJPvIlZ72OEMRWj5agkXv9E78dkDP3lZfNrGN3cw0boNLpk841eboGRtWyQgpFDuEj Rvx2jguXU+wVBTjOV1KfdFj5d0IN/l0tONpgCWIJI8d+k6fAa1vuLHry6BnK9Dijjxq+ILuHdZj 3WVHLARbEf7lynxSqrCdQg5/tS6tMD/xWBJNSZthL6XNWUMrbOlEFVcGlfVmyCz5GKZ69nG6Rc6 z4SBCnoPa09HpWH5s= X-Received: by 2002:a05:6512:3088:b0:5a2:b379:22d6 with SMTP id 2adb3069b0e04-5a2b379244cmr2628747e87.17.1774893505968; Mon, 30 Mar 2026 10:58:25 -0700 (PDT) Received: from localhost.localdomain ([2001:9b1:d5a0:a500::24b]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-38c838db4b3sm15752631fa.33.2026.03.30.10.58.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Mar 2026 10:58:25 -0700 (PDT) From: "Uladzislau Rezki (Sony)" To: linux-mm@kvack.org, Andrew Morton Cc: Baoquan He , LKML , Uladzislau Rezki , lirongqing Subject: [PATCH v2] mm/vmalloc: Use dedicated unbound workqueue for vmap purge/drain Date: Mon, 30 Mar 2026 19:58:24 +0200 Message-ID: <20260330175824.2777270-1-urezki@gmail.com> X-Mailer: git-send-email 2.47.3 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The drain_vmap_area_work() function can take >10ms to complete when there are many accumulated vmap areas in a system with a high CPU count, causing workqueue watchdog warnings when run via schedule_work(): [ 2069.796205] workqueue: drain_vmap_area_work hogged CPU for >10000us 4 ti= mes, consider switching to WQ_UNBOUND [ 2192.823225] workqueue: drain_vmap_area_work hogged CPU for >10000us 5 ti= mes, consider switching to WQ_UNBOUND Switch to a dedicated WQ_UNBOUND workqueue to allow the scheduler to run this background task on any available CPU, improving responsiveness. Use WQ_MEM_RECLAIM to ensure forward progress under memory pressure. If queuing work to the dedicated workqueue is not possible(during early boot), fall back to processing locally to avoid losing progress. Also simplify purge helper scheduling by removing cpumask-based iteration in favour to iterating directly over vmap nodes with pending work. Cc: lirongqing Link: https://lore.kernel.org/all/20260319074307.2325-1-lirongqing@baidu.co= m/ Signed-off-by: Uladzislau Rezki (Sony) --- mm/vmalloc.c | 74 +++++++++++++++++++++++++++++++++------------------- 1 file changed, 47 insertions(+), 27 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 61caa55a4402..6bc2523bf75b 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -949,6 +949,7 @@ static struct vmap_node { struct list_head purge_list; struct work_struct purge_work; unsigned long nr_purged; + bool work_queued; } single; =20 /* @@ -1067,6 +1068,7 @@ static void reclaim_and_purge_vmap_areas(void); static BLOCKING_NOTIFIER_HEAD(vmap_notify_list); static void drain_vmap_area_work(struct work_struct *work); static DECLARE_WORK(drain_vmap_work, drain_vmap_area_work); +static struct workqueue_struct *drain_vmap_wq; =20 static __cacheline_aligned_in_smp atomic_long_t nr_vmalloc_pages; static __cacheline_aligned_in_smp atomic_long_t vmap_lazy_nr; @@ -2335,6 +2337,19 @@ static void purge_vmap_node(struct work_struct *work) reclaim_list_global(&local_list); } =20 +static bool +schedule_drain_vmap_work(struct work_struct *work) +{ + struct workqueue_struct *wq =3D READ_ONCE(drain_vmap_wq); + + if (wq) { + queue_work(wq, work); + return true; + } + + return false; +} + /* * Purges all lazily-freed vmap areas. */ @@ -2342,19 +2357,12 @@ static bool __purge_vmap_area_lazy(unsigned long st= art, unsigned long end, bool full_pool_decay) { unsigned long nr_purged_areas =3D 0; + unsigned int nr_purge_nodes =3D 0; unsigned int nr_purge_helpers; - static cpumask_t purge_nodes; - unsigned int nr_purge_nodes; struct vmap_node *vn; - int i; =20 lockdep_assert_held(&vmap_purge_lock); =20 - /* - * Use cpumask to mark which node has to be processed. - */ - purge_nodes =3D CPU_MASK_NONE; - for_each_vmap_node(vn) { INIT_LIST_HEAD(&vn->purge_list); vn->skip_populate =3D full_pool_decay; @@ -2374,10 +2382,9 @@ static bool __purge_vmap_area_lazy(unsigned long sta= rt, unsigned long end, end =3D max(end, list_last_entry(&vn->purge_list, struct vmap_area, list)->va_end); =20 - cpumask_set_cpu(node_to_id(vn), &purge_nodes); + nr_purge_nodes++; } =20 - nr_purge_nodes =3D cpumask_weight(&purge_nodes); if (nr_purge_nodes > 0) { flush_tlb_kernel_range(start, end); =20 @@ -2385,29 +2392,30 @@ static bool __purge_vmap_area_lazy(unsigned long st= art, unsigned long end, nr_purge_helpers =3D atomic_long_read(&vmap_lazy_nr) / lazy_max_pages(); nr_purge_helpers =3D clamp(nr_purge_helpers, 1U, nr_purge_nodes) - 1; =20 - for_each_cpu(i, &purge_nodes) { - vn =3D &vmap_nodes[i]; + for_each_vmap_node(vn) { + vn->work_queued =3D false; + + if (list_empty(&vn->purge_list)) + continue; =20 if (nr_purge_helpers > 0) { INIT_WORK(&vn->purge_work, purge_vmap_node); + vn->work_queued =3D schedule_drain_vmap_work(&vn->purge_work); =20 - if (cpumask_test_cpu(i, cpu_online_mask)) - schedule_work_on(i, &vn->purge_work); - else - schedule_work(&vn->purge_work); - - nr_purge_helpers--; - } else { - vn->purge_work.func =3D NULL; - purge_vmap_node(&vn->purge_work); - nr_purged_areas +=3D vn->nr_purged; + if (vn->work_queued) { + nr_purge_helpers--; + continue; + } } - } =20 - for_each_cpu(i, &purge_nodes) { - vn =3D &vmap_nodes[i]; + /* Sync path. Process locally. */ + purge_vmap_node(&vn->purge_work); + nr_purged_areas +=3D vn->nr_purged; + } =20 - if (vn->purge_work.func) { + /* Wait for completion if queued any. */ + for_each_vmap_node(vn) { + if (vn->work_queued) { flush_work(&vn->purge_work); nr_purged_areas +=3D vn->nr_purged; } @@ -2471,7 +2479,7 @@ static void free_vmap_area_noflush(struct vmap_area *= va) =20 /* After this point, we may free va at any time */ if (unlikely(nr_lazy > nr_lazy_max)) - schedule_work(&drain_vmap_work); + schedule_drain_vmap_work(&drain_vmap_work); } =20 /* @@ -5483,3 +5491,15 @@ void __init vmalloc_init(void) vmap_node_shrinker->scan_objects =3D vmap_node_shrink_scan; shrinker_register(vmap_node_shrinker); } + +static int __init vmalloc_init_workqueue(void) +{ + struct workqueue_struct *wq; + + wq =3D alloc_workqueue("vmap_drain", WQ_UNBOUND | WQ_MEM_RECLAIM, 0); + WARN_ON(wq =3D=3D NULL); + WRITE_ONCE(drain_vmap_wq, wq); + + return 0; +} +early_initcall(vmalloc_init_workqueue); --=20 2.47.3