From nobody Thu Jun 11 22:56:56 2026 Received: from out-174.mta1.migadu.com (out-174.mta1.migadu.com [95.215.58.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 464FF3AE712 for ; Thu, 4 Jun 2026 06:18:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780553892; cv=none; b=rwAF/qmq82vs9NVx8EQ3vuLbOxh5g4sxOHHut77aQLcXn6uo9SiGn31mNsAobQgcydsnrlacMMWQ/zWoMQ1zB5j5lfOOBxMr5A0x2pbiCZVXuPH3SaV7S6gL9FI3u5cH9dUi5XZQXnol2yXNF8Yfwj7Ri9/eGG/DuludXHnf620= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780553892; c=relaxed/simple; bh=gmbsR9vDbidsqEsTawu4X+ZaNPNMdzmm6IKLkxtaZVw=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=ILjh+Qx8mluPCShE2g75Js9I7a+T0y7Iiy2xBzn70lJPdN4R8HFiRyYNC/kWu8txdC5GUhU8NkiKopTkugCABxnzWTIUvQzzFAHmNukPRfDWaLBVIiOpczUuO5pcPJsOBucWKzPXk6Ib40uQrMA6SXl4/P3/rheI7Gphe3nCsKE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=MbFhU2LC; arc=none smtp.client-ip=95.215.58.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="MbFhU2LC" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1780553886; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=uhNb1e5oAaAySIL2wfE6fLLmLDDye9t6TqH0x1C5HS0=; b=MbFhU2LCuUY+aSj/13pvSUxdoBFZf2JPQ/rLfTIkJhA7bVQnwsZGbYfROKpc41BvSZWXaQ SPbYbfspX/dw9CMZqrnaNbbkk1zL7vA4yzG2s88ko6WqEnyI0TnLLH9HTja6XmALJL+iZR sukH8LwIbMOtrnFUAce9B+WLeZBlgqs= From: JP Kobryn To: vbabka@kernel.org, akpm@linux-foundation.org, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com Cc: shakeel.butt@linux.dev, usama.arif@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2] mm/compaction: cap compact_gap() at COMPACT_CLUSTER_MAX Date: Wed, 3 Jun 2026 23:17:25 -0700 Message-ID: <20260604061725.13800-1-jp.kobryn@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" From: "JP Kobryn" compact_gap() returns 2 << order, which is used as watermark headroom in __compaction_suitable() and as a threshold in kswapd reclaim decisions. The computed value scales exponentially by order. For order-9 THP allocations this evaluates to 1024 pages, but the compaction free scanner's working set is bounded by COMPACT_CLUSTER_MAX (32 pages). The scanner stops isolating free pages once it matches the migration batch. The current gap over-reserves by 32x. On fragmented production hosts, kswapd will try to reclaim up to the gap, but it only reaches that threshold in 18% of attempts. As a result, reclaim continues in the majority of cases despite many lower-order free pages being available. The over-sized gap also causes 46% of order-9 compaction suitability checks to fail unnecessarily: the zone has sufficient free pages for the scanner to operate, but not enough to clear the inflated threshold. Cap compact_gap() at COMPACT_CLUSTER_MAX so the watermark headroom reflects the scanner's actual capacity. This function is used by two key heuristics. The first is when kswapd can stop high-order reclaim and downgrade to order-0 balancing, allowing kcompactd to be woken for the original higher allocation order. The second is zone suitability checking, where the smaller gap allows compaction to start sooner. Note that orders 0-4 are unaffected since their gap is already less than or equal to COMPACT_CLUSTER_MAX. A/B test on v6.13-based instagram production hosts (64GB, 60s measurement): Unpatched (43 hosts) pgscan_kswapd (mean/host): ~1.6M reclaim efficiency (steal/scan): 83.8% per-compaction success (success/stall): 2.1% THP success (alloc/alloc+fallback): 4.9% forced lru_add_drain (mean/host): ~107K Patched (59 hosts) pgscan_kswapd (mean/host): ~449K reclaim efficiency (steal/scan): 91.0% per-compaction success (success/stall): 28.3% THP success (alloc/alloc+fallback): 17.2% forced lru_add_drain (mean/host): ~64K Additional tests were also performed using a workload of similar shape and based on mm-new at the time of testing. Across three 60s runs, the patch showed improvements consistent with the previous test: reduced kswapd reclaim and fewer THP fault fallbacks. Unpatched kswapd_shrink_node downgrade to order-0 (mean): 0 thp_fault_fallback (mean): 1217 pgscan_kswapd (mean): 6328 pgsteal_kswapd (mean): 5657 Patched kswapd_shrink_node downgrade to order-0 (mean): 28 thp_fault_fallback (mean): 738 pgscan_kswapd (mean): 3773 pgsteal_kswapd (mean): 3243 Signed-off-by: JP Kobryn (Meta) Reviewed-by: Vlastimil Babka (SUSE) Cc: Brendan Jackman Cc: Johannes Weiner Cc: Michal Hocko Cc: Suren Baghdasaryan Cc: Zi Yan --- v2: - reword changelog and add mm-new validation data - update comment in kswapd_shrink_node to reflect the gap change - no functional changes v1: https://lore.kernel.org/linux-mm/20260519200851.141955-1-jp.kobryn@linu= x.dev/ include/linux/compaction.h | 8 ++++---- mm/vmscan.c | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/include/linux/compaction.h b/include/linux/compaction.h index c829c48d1c71..f29ef0653546 100644 --- a/include/linux/compaction.h +++ b/include/linux/compaction.h @@ -2,6 +2,8 @@ #ifndef _LINUX_COMPACTION_H #define _LINUX_COMPACTION_H =20 +#include + /* * Determines how hard direct compaction should try to succeed. * Lower value means higher priority, analogically to reclaim priority. @@ -73,11 +75,9 @@ static inline unsigned long compact_gap(unsigned int ord= er) * effectively limited by COMPACT_CLUSTER_MAX, as that's the maximum * that the migrate scanner can have isolated on migrate list, and free * scanner is only invoked when the number of isolated free pages is - * lower than that. But it's not worth to complicate the formula here - * as a bigger gap for higher orders than strictly necessary can also - * improve chances of compaction success. + * lower than that. */ - return 2UL << order; + return min(2UL << order, COMPACT_CLUSTER_MAX); } =20 static inline int current_is_kcompactd(void) diff --git a/mm/vmscan.c b/mm/vmscan.c index e8a90911bf88..3f3ff25e561a 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -7014,7 +7014,7 @@ static bool kswapd_shrink_node(pg_data_t *pgdat, =20 /* * Fragmentation may mean that the system cannot be rebalanced for - * high-order allocations. If twice the allocation size has been + * high-order allocations. If at least the compaction gap has been * reclaimed then recheck watermarks only at order-0 to prevent * excessive reclaim. Assume that a process requested a high-order * can direct reclaim/compact. --=20 2.54.0