From nobody Thu Sep 24 19:59:50 2026 Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7723A4BA1E0; Mon, 21 Sep 2026 15:14:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790003661; cv=none; b=EuHtAxp5Xt7J2Ly6xqXVzHOKBJl/y8TLw9xnTrwdIOF7oYkSXCDQu4UIDaEf4Zs3gEEBd5eAWlgUtdWW43YBXFdbF483uN2OLdp3SJWZ8oubddoADxMTcTW0M2dY+Hft2yvf+45tuv8z5HmwRoOPRIkNhce/WP3wT9oRZ7MiKac= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790003661; c=relaxed/simple; bh=odZczmMOiuod/U8U6Qq754J9RoujPsTLa9emtGfnw8Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ebcadx4B7XTYiqsNUv/PodeUfDUKdMYv3VIDHsnktHH29qMrGBlGg81e93cJESlgNcK8MrSL04vAcYQ2Xk6McxE+QguqWgRguCNK2BOk/mmlgC9auMu0Gxt1LlErQMjTEf+ObkQUXXyd4VnM0vwdFYHDk8ysFx5VWKpXN1eHzTU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr; spf=pass smtp.mailfrom=ghiti.fr; arc=none smtp.client-ip=217.70.183.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ghiti.fr Received: by mail.gandi.net (Postfix) with ESMTPSA id AC5263EC61; Mon, 21 Sep 2026 15:14:12 +0000 (UTC) From: Alexandre Ghiti To: Johannes Weiner , Yosry Ahmed , Nhat Pham , Andrew Morton , Chris Li , Kairui Song Cc: Kairui Song , Chengming Zhou , "Matthew Wilcox (Oracle)" , Jan Kara , Kemeng Shi , Baoquan He , Barry Song , Youngjun Park , Alexander Viro , Christian Brauner , David Hildenbrand , Lorenzo Stoakes , Michal Hocko , Axel Rasmussen , Qi Zheng , Shakeel Butt , Wei Xu , Yuanchu Xie , Kunwu Chan , Tal Zussman , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Alexandre Ghiti , Usama Arif Subject: [PATCH v6 1/3] mm: swap: move LRU insertion out of the swap cache allocator Date: Mon, 21 Sep 2026 17:13:02 +0200 Message-ID: <20260921151306.625134-2-alex@ghiti.fr> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260921151306.625134-1-alex@ghiti.fr> References: <20260921151306.625134-1-alex@ghiti.fr> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-GND-Sasl: alex@ghiti.fr X-GND-Cause: dmFkZTErTfvtT3pejKA2c3+atWyTlcQY/2uOK/WC4ZxB6ECJeRWaowlY444Pz8MwijVZahcejcCp//J/4zmYyFaV9n5+fTE2W/zk+j08buYeKqj57bfKmR/1aCQIHlePtPRPO7BFkHK0YG5Nc1HJwubtzFLSUafD39eqhbCa1WaoWlky0QU7/s0p+KOQ9WQHVlH/Z4GSeEq8dey3bbFn2mfiX3yy4C/x/sPvRb/5ajFS224lU/TDUiRGF84sCwEmKZTfv8RTO++xBjYCxbdesAxiH6ogpHphBvCw2671svHud/lqwjl9g1Z9dZdvEeBo4gqi7f4j2kALMQ4m3/C5K84EELuYnQSo0Fzqc8ZTTA0+9nAquWP+qQ4kXODBrGakDS5ipyR6CoFemQ2DOLDBdM1qT4ZrsvclOD4iQJ7U+hAbhxi/6FKTLlHA5wME7G8Il4neAhsMjSrWPfTZu8MmI7v8uk0m8wtkPm5jIfuahWyJII0jsK7SxTFI0q3mYvJhAhh2YGnkqcJ18ig/8xECLwA/IKZIRMp8oz3jXYKUV1vWkKe8OZjFCOCWxn4h/6ghpWs2bsPbzOaAGkYMKAr+H0vbErjMz6eXvlZhUKwFYR/RKgxDwiDs6qrCSZAOA+XKbsP791bK5WkKBLjHxwgcHYV/TnhJ2/K8JlECJzTmaHzTtOIbhA X-GND-State: clean X-GND-Score: -100 Content-Type: text/plain; charset="utf-8" This is a preparatory patch. swap_cache_alloc_folio() adds the new folio to the LRU itself, which leaves its callers no way to act on the folio before it becomes visible to reclaim. Two users need exactly that: - moving the refault evaluation out of the swap cache folio allocation requires it to happen before folio_add_lru(): that consumes PG_active to file the folio on the inactive or the active list, and under MGLRU it also reads PG_workingset to pick the generation. Setting either flag afterwards does not move the folio; - zswap writeback dropbehind needs the buffer folio to stay off the LRU entirely, as the per-CPU LRU batch would hold a reference on it and keep remove_mapping() from freeing it once writeback completes. Defer the LRU insertion to the callers and rename the helper to __swap_cache_alloc_folio(): each caller adds the folio right after the allocation, so there is no functional change intended. Suggested-by: Kairui Song Reviewed-by: Kairui Song Reviewed-by: Nhat Pham Reviewed-by: Kunwu Chan Acked-by: Usama Arif Reviewed-by: Barry Song Signed-off-by: Alexandre Ghiti --- mm/swap.h | 6 +++--- mm/swap_state.c | 20 ++++++++++++-------- mm/swapfile.c | 2 +- mm/zswap.c | 5 +++-- 4 files changed, 19 insertions(+), 14 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index 90a551a88df6..8679cb61268e 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -312,9 +312,9 @@ bool swap_cache_has_folio(swp_entry_t entry); struct folio *swap_cache_get_folio(swp_entry_t entry); void *swap_cache_get_shadow(swp_entry_t entry); void swap_cache_del_folio(struct folio *folio); -struct folio *swap_cache_alloc_folio(swp_entry_t target_entry, gfp_t gfp_m= ask, - unsigned long orders, struct vm_fault *vmf, - struct mempolicy *mpol, pgoff_t ilx); +struct folio *__swap_cache_alloc_folio(swp_entry_t target_entry, gfp_t gfp= _mask, + unsigned long orders, struct vm_fault *vmf, + struct mempolicy *mpol, pgoff_t ilx); /* Below helpers require the caller to lock and pass in the swap cluster. = */ void __swap_cache_add_folio(struct swap_cluster_info *ci, struct folio *folio, swp_entry_t entry); diff --git a/mm/swap_state.c b/mm/swap_state.c index f3961fdd857d..e5b7fa468ade 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -489,13 +489,11 @@ static struct folio *__swap_cache_alloc(struct swap_c= luster_info *ci, node_stat_mod_folio(folio, NR_FILE_PAGES, nr_pages); lruvec_stat_mod_folio(folio, NR_SWAPCACHE, nr_pages); =20 - /* Caller will initiate read into locked new_folio */ - folio_add_lru(folio); return folio; } =20 /** - * swap_cache_alloc_folio - Allocate folio for swapped out slot in swap ca= che. + * __swap_cache_alloc_folio - Allocate folio for swapped out slot in swap = cache. * @targ_entry: swap entry indicating the target slot * @gfp: memory allocation flags * @orders: allocation orders, must be non zero @@ -507,13 +505,17 @@ static struct folio *__swap_cache_alloc(struct swap_c= luster_info *ci, * doing IO (e.g. swap in or zswap writeback). The swap slot indicated by * @targ_entry must have a non-zero swap count (swapped out). * + * The returned folio is locked and is NOT on the LRU. The caller must eit= her + * add it to the LRU with folio_add_lru() so page reclaim can find it, or = free + * it directly once done; a folio left off the LRU is unreclaimable and le= aks. + * * Context: Caller must protect the swap device with reference count or lo= cks. * Return: Returns the folio if allocation succeeded and folio is in the s= wap * cache. Returns error code if failed due to race, OOM or invalid argumen= ts. */ -struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp, - unsigned long orders, struct vm_fault *vmf, - struct mempolicy *mpol, pgoff_t ilx) +struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp, + unsigned long orders, struct vm_fault *vmf, + struct mempolicy *mpol, pgoff_t ilx) { int order, err; struct folio *ret; @@ -649,12 +651,13 @@ static struct folio *swap_cache_read_folio(struct swa= p_io_ctx *ctx, folio =3D swap_cache_get_folio(entry); if (folio) return folio; - folio =3D swap_cache_alloc_folio(entry, gfp, BIT(0), NULL, mpol, ilx); + folio =3D __swap_cache_alloc_folio(entry, gfp, BIT(0), NULL, mpol, ilx); } while (PTR_ERR(folio) =3D=3D -EEXIST); =20 if (IS_ERR_OR_NULL(folio)) return NULL; =20 + folio_add_lru(folio); swap_read_folio(ctx, folio); if (readahead) { folio_set_readahead(folio); @@ -690,12 +693,13 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gf= p, unsigned long orders, folio =3D swap_cache_get_folio(entry); if (folio) return folio; - folio =3D swap_cache_alloc_folio(entry, gfp, orders, vmf, mpol, ilx); + folio =3D __swap_cache_alloc_folio(entry, gfp, orders, vmf, mpol, ilx); } while (PTR_ERR(folio) =3D=3D -EEXIST); =20 if (IS_ERR(folio)) return folio; =20 + folio_add_lru(folio); swap_read_folio(&ctx, folio); swap_read_submit(&ctx); return folio; diff --git a/mm/swapfile.c b/mm/swapfile.c index 53bf01d5f7f1..d678a40fcaac 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1870,7 +1870,7 @@ void folio_put_swap(struct folio *folio, struct page = *page) * CPU1 CPU2 * do_swap_page() * ... swapoff+swapon - * swap_cache_alloc_folio() + * __swap_cache_alloc_folio() * // check swap_map * // verify PTE not changed * diff --git a/mm/zswap.c b/mm/zswap.c index 37f34e406c8e..0d2efe21f18a 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1001,8 +1001,8 @@ static int zswap_writeback_entry(struct zswap_entry *= entry, return -EEXIST; =20 mpol =3D get_task_policy(current); - folio =3D swap_cache_alloc_folio(swpentry, GFP_KERNEL, BIT(0), NULL, mpol, - NO_INTERLEAVE_INDEX); + folio =3D __swap_cache_alloc_folio(swpentry, GFP_KERNEL, BIT(0), NULL, mp= ol, + NO_INTERLEAVE_INDEX); put_swap_device(si); =20 /* @@ -1014,6 +1014,7 @@ static int zswap_writeback_entry(struct zswap_entry *= entry, */ if (IS_ERR(folio)) return PTR_ERR(folio); + folio_add_lru(folio); =20 /* * folio is locked, and the swapcache is now secured against --=20 2.53.0-Meta From nobody Thu Sep 24 19:59:50 2026 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECBB54A5EAF; Mon, 21 Sep 2026 15:15:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790003727; cv=none; b=nORwsNIencV6OzjidKVZPR514FOH3H7rs4KE4sg0hqLaiSuYduJWTEVlvPTB6q+KPJ5cfsJvExn29qBBRJiReyFfqvy4QRN6kXv8W6J+OI9EfEZfI1B/Et/D0YMarp4SMpK12ZIeMStfeqwPtg1IYHMRjcoqUQHgGyxrixo034c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790003727; c=relaxed/simple; bh=wBCx1z+Y3RWTTlot8qNLW7QCznFkivKvHBbE2QFsbIM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H42CF83IDmv0NCzPMlRqO/sHpXFXUwaPdEJ6EXQbfA/6X8F2BluY+0iil8tLFMcIF+8JbtXDWQtCDLKn8e2ozzPrTWjpAo7VKBAbRjpGKt6cXOV6ify1pqL2Z/oOiJBIikNykxxREEMRdxZiwS0GXpfEWWoj/0Qg/fT4yrSxD/A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr; spf=pass smtp.mailfrom=ghiti.fr; arc=none smtp.client-ip=217.70.183.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ghiti.fr Received: by mail.gandi.net (Postfix) with ESMTPSA id B251C3F08D; Mon, 21 Sep 2026 15:15:16 +0000 (UTC) From: Alexandre Ghiti To: Johannes Weiner , Yosry Ahmed , Nhat Pham , Andrew Morton , Chris Li , Kairui Song Cc: Kairui Song , Chengming Zhou , "Matthew Wilcox (Oracle)" , Jan Kara , Kemeng Shi , Baoquan He , Barry Song , Youngjun Park , Alexander Viro , Christian Brauner , David Hildenbrand , Lorenzo Stoakes , Michal Hocko , Axel Rasmussen , Qi Zheng , Shakeel Butt , Wei Xu , Yuanchu Xie , Kunwu Chan , Tal Zussman , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Alexandre Ghiti Subject: [PATCH v6 2/3] mm: swap: drop dropbehind swap cache folios on writeback completion Date: Mon, 21 Sep 2026 17:13:03 +0200 Message-ID: <20260921151306.625134-3-alex@ghiti.fr> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260921151306.625134-1-alex@ghiti.fr> References: <20260921151306.625134-1-alex@ghiti.fr> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-GND-Sasl: alex@ghiti.fr X-GND-State: clean X-GND-Score: -100 X-GND-Cause: dmFkZTErTfvtT3pejKA2c3+atWyTlcQY/2uOK/WC4ZxB6ECJeRWaowlY444Pz8MwijVZahcejcCp//J/4zmYyFaV9n5+fTE2W/zk+j08buYeKqj57bfKmR/1aCQIHlePtPRPO7BFkHK0YG5Nc1HJwubtzFLSUafD39eqhbCa1WaoWlky0QU7/s0p+KOQ9WQHVlH/Z4GSeEq8dey3bbFn2mfiX3yy4C/x/sPvRb/5ajFS224lU/TDUiRGF84sCwEmKZTfv8RTO++xBjYCxbdesAxiH6ogpHphBvCw2671svHud/lqwjl9g1Z9dZdvEeBo4gqi7f4j2kALMQ4m3/C5K84EELuYK+ju9CO5tMjR++bNyjSsG34zbPjZN5buQHRpcSy3us9PXqNjqBevp1Cm+mijAV8c+qV/NU5jXq4RH867rh805hwx0502zVy33bVuha3XUqm8NT/42nWatRmS8F19Wi7YskUn/igDxF0gXphrSs1TT4K3T0wdEHExhNUyeG2iJC5fzJNX4l2Q29O0ZTq2G2bAh/AR/Sf66I4Cki7uDRd/V+4slfP5gzJQO4zyj6lBK3zheTSrQacTgOrHiWrynQUcz8XrLyQmTPMe6mNfEy1VGdln5OSVu1R73Cjv+nFUqKwsslBYSKryLLo9Jako1HQvVV0JcbkFJJyefd81JzIipw Content-Type: text/plain; charset="utf-8" A PG_dropbehind folio is dropped from its cache once writeback completes rather than left for reclaim to find later; this is implemented for file folios in folio_end_dropbehind(). Extend it to swap cache folios. The drop blocks on the folio lock, so it cannot run in interrupt context. Set BIO_COMPLETE_IN_TASK on the write, as the file dropbehind paths do, and drop the folio directly from folio_end_writeback(). It has to block rather than trylock: the folio is off the LRU, so skipping it would leave it in the swap cache with nothing able to reclaim it, and it cannot be put back while another thread holds its lock. Suggested-by: Yosry Ahmed Suggested-by: Johannes Weiner Suggested-by: Nhat Pham Reviewed-by: Nhat Pham Reviewed-by: Kunwu Chan Reviewed-by: Barry Song Signed-off-by: Alexandre Ghiti --- include/linux/swap.h | 6 ++++++ mm/filemap.c | 19 +++++++++++++++++ mm/page_io.c | 9 ++++++++ mm/swap_state.c | 42 +++++++++++++++++++++++++++++++++++++ mm/vmscan.c | 49 +++++++++++++++++++++++++++++++++++--------- 5 files changed, 115 insertions(+), 10 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 5658a1634b85..538b723a276f 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -318,6 +318,9 @@ static inline bool lru_cache_disabled(void) =20 extern unsigned long shrink_all_memory(unsigned long nr_pages); long remove_mapping(struct address_space *mapping, struct folio *folio); +long remove_mapping_set_shadow(struct address_space *mapping, + struct folio *folio, + struct mem_cgroup *target_memcg); =20 #if defined(CONFIG_SYSFS) && defined(CONFIG_NUMA) extern int reclaim_register_node(struct node *node); @@ -402,6 +405,8 @@ void swap_put_entries_direct(swp_entry_t entry, int nr); */ bool folio_free_swap(struct folio *folio); =20 +void swap_writeback_dropbehind_folio(struct folio *folio); + /* Allocate / free (hibernation) exclusive entries */ swp_entry_t swap_alloc_hibernation_slot(int type); void swap_free_hibernation_slot(swp_entry_t entry); @@ -412,6 +417,7 @@ static inline void put_swap_device(struct swap_info_str= uct *si) } =20 #else /* CONFIG_SWAP */ +static inline void swap_writeback_dropbehind_folio(struct folio *folio) {} static inline struct swap_info_struct *get_swap_device(swp_entry_t entry) { return NULL; diff --git a/mm/filemap.c b/mm/filemap.c index 6afec636881f..e1f1bbe943ce 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1686,6 +1686,8 @@ EXPORT_SYMBOL_GPL(folio_end_writeback_no_dropbehind); */ void folio_end_writeback(struct folio *folio) { + bool swap_dropbehind; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); =20 /* @@ -1695,7 +1697,24 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + + /* + * Sample this before folio_end_writeback_no_dropbehind() clears + * PG_writeback: until then a racing swapin cannot remove the folio from + * the swap cache. Afterwards it can, and the drop below then finds a + * non-swapcache folio and puts it back on the LRU instead. The + * reference taken above keeps the folio alive across that window. + */ + swap_dropbehind =3D folio_test_swapcache(folio) && + folio_test_dropbehind(folio); + folio_end_writeback_no_dropbehind(folio); + + if (swap_dropbehind) { + swap_writeback_dropbehind_folio(folio); + return; + } + folio_end_dropbehind(folio); folio_put(folio); } diff --git a/mm/page_io.c b/mm/page_io.c index 88962571cb93..52eae99de6e3 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -602,6 +602,15 @@ static void swap_bdev_submit_write(struct swap_io_ctx = *ctx) submit_bio_wait(bio); end_swap_bio_write(bio); } else { + int p; + + for (p =3D 0; p < sio->nr_bvecs; p++) { + if (folio_test_dropbehind(bvec_folio(&sio->bvecs[p]))) { + bio_set_flag(bio, BIO_COMPLETE_IN_TASK); + break; + } + } + bio->bi_end_io =3D end_swap_bio_write; submit_bio(bio); } diff --git a/mm/swap_state.c b/mm/swap_state.c index e5b7fa468ade..b1656e2d5288 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -543,6 +543,48 @@ struct folio *__swap_cache_alloc_folio(swp_entry_t tar= g_entry, gfp_t gfp, return ret; } =20 +/** + * swap_writeback_dropbehind_folio - drop a dropbehind swap cache folio + * @folio: the off-LRU folio whose writeback has completed + * + * Context: task context, with the reference taken by folio_end_writeback() + * donated to us. + */ +void swap_writeback_dropbehind_folio(struct folio *folio) +{ + struct mem_cgroup *memcg; + + folio_lock(folio); + + /* The folio was allocated off the LRU and nothing re-adds it here. */ + VM_WARN_ON_ONCE_FOLIO(folio_test_lru(folio), folio); + + rcu_read_lock(); + memcg =3D folio_memcg(folio); + if (!mem_cgroup_tryget(memcg)) + memcg =3D NULL; + rcu_read_unlock(); + + /* + * Gate remove_mapping_set_shadow() on folio_test_swapcache(): a racing + * swapin may have freed the swap slot (folio_free_swap()) and dropped the + * folio from the cache, and it must not run on a non-swapcache folio (it + * would trip __remove_mapping()'s mapping =3D=3D folio_mapping() check). + */ + if (!folio_test_swapcache(folio) || folio_test_writeback(folio) || + !remove_mapping_set_shadow(swap_address_space(folio->swap), folio, + memcg)) { + /* Raced: the folio is now owned by the swapin; put it back. */ + folio_clear_dropbehind(folio); + folio_add_lru(folio); + } + + mem_cgroup_put(memcg); + + folio_unlock(folio); + folio_put(folio); +} + /* * If we are the only user, then try to free up the swap cache. * diff --git a/mm/vmscan.c b/mm/vmscan.c index f11491ee9ed5..a02f942418d3 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -783,6 +783,22 @@ static int __remove_mapping(struct address_space *mapp= ing, struct folio *folio, return 0; } =20 +static long __remove_mapping_unfreeze(struct address_space *mapping, + struct folio *folio, bool reclaimed, + struct mem_cgroup *target_memcg) +{ + if (__remove_mapping(mapping, folio, reclaimed, target_memcg)) { + /* + * Unfreezing the refcount with 1 effectively + * drops the pagecache ref for us without requiring another + * atomic operation. + */ + folio_ref_unfreeze(folio, 1); + return folio_nr_pages(folio); + } + return 0; +} + /** * remove_mapping() - Attempt to remove a folio from its mapping. * @mapping: The address space. @@ -797,16 +813,29 @@ static int __remove_mapping(struct address_space *map= ping, struct folio *folio, */ long remove_mapping(struct address_space *mapping, struct folio *folio) { - if (__remove_mapping(mapping, folio, false, NULL)) { - /* - * Unfreezing the refcount with 1 effectively - * drops the pagecache ref for us without requiring another - * atomic operation. - */ - folio_ref_unfreeze(folio, 1); - return folio_nr_pages(folio); - } - return 0; + return __remove_mapping_unfreeze(mapping, folio, false, NULL); +} + +/** + * remove_mapping_set_shadow() - Remove a folio and record an eviction sha= dow. + * @mapping: The address space. + * @folio: The folio to remove. + * @target_memcg: The memcg to charge the eviction shadow to; the caller m= ust + * keep it alive across the call. + * + * Like remove_mapping(), but stores a workingset eviction shadow the way = page + * reclaim does, so that a later refault can be detected and the folio + * re-activated. + * Return: The number of pages removed from the mapping. 0 if the folio + * could not be removed. + * Context: The caller should have a single refcount on the folio and + * hold its lock. + */ +long remove_mapping_set_shadow(struct address_space *mapping, + struct folio *folio, + struct mem_cgroup *target_memcg) +{ + return __remove_mapping_unfreeze(mapping, folio, true, target_memcg); } =20 /** --=20 2.53.0-Meta From nobody Thu Sep 24 19:59:50 2026 Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 062ED3C10A7; Mon, 21 Sep 2026 15:16:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.195 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790003799; cv=none; b=mphJYNegKLJ+b8wYWzO2GCCo2FE8j1NsUWVZKbrl9x5+jepweL/WmKNu/YOKDKzkoMMaeXWJCg0UarKrKv3bfb5BtRwEaeq64gXcBmRRIR3uJqdIYoI/hjNfmfy3H7fxZtqWjlVhDZ/5f874j0A2xrxZuvJnmOUjIlCm1ZbNYbk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790003799; c=relaxed/simple; bh=6UkOOgSYQ2RsZFW63ooaHnq8belFDQLyzUab5uagDTY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RiX/078tDuZNO5aKAcrQpr5rtwXHQyMBPr31KzhDttemFHsqLrsg5F5hBMNX/5HaGgVJ9wvo2jc8bfLDqf4bmjMQkPy+xmoa4spearPy8RAxMZh7slYminxvIdhv6lfPBNTVacFP3q8FQf/dnweihUTbB2mk751bdj2ry6njarA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr; spf=pass smtp.mailfrom=ghiti.fr; arc=none smtp.client-ip=217.70.183.195 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ghiti.fr Received: by mail.gandi.net (Postfix) with ESMTPSA id 68AEC20CF6; Mon, 21 Sep 2026 15:16:22 +0000 (UTC) From: Alexandre Ghiti To: Johannes Weiner , Yosry Ahmed , Nhat Pham , Andrew Morton , Chris Li , Kairui Song Cc: Kairui Song , Chengming Zhou , "Matthew Wilcox (Oracle)" , Jan Kara , Kemeng Shi , Baoquan He , Barry Song , Youngjun Park , Alexander Viro , Christian Brauner , David Hildenbrand , Lorenzo Stoakes , Michal Hocko , Axel Rasmussen , Qi Zheng , Shakeel Butt , Wei Xu , Yuanchu Xie , Kunwu Chan , Tal Zussman , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Alexandre Ghiti Subject: [PATCH v6 3/3] mm: zswap: drop cold writeback folios via swap dropbehind Date: Mon, 21 Sep 2026 17:13:04 +0200 Message-ID: <20260921151306.625134-4-alex@ghiti.fr> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260921151306.625134-1-alex@ghiti.fr> References: <20260921151306.625134-1-alex@ghiti.fr> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-GND-Sasl: alex@ghiti.fr X-GND-Score: -100 X-GND-Cause: dmFkZTE0SYM0WXNjdwubovITJrTYXyTSCwr/PlRPkkCz6KhxaAixi/QamYFRTWojpv77hYY7HjUcktzM93E0S7aP74KErDRnXAJ+8qBUZxDaDQ5JnexSaYr1F6K/qzS9RP6Q9NI0CX3RagQv2c/zqGxQWHxXflkOyoED1Sdwit9/i76dx7VxOUiBK8Q/gl4KmGjDzdQItBSNGWeNMb/7oLm/w9+WdgNkyz1o/4STbT4WL3/jko4pyhjinYBBZBp9KOF1J5FSuZ+4olXkmIuTnf3/D2n3GGfFEmJXsZU+tWWpUdrPtJniqkZUNoy4ZSEspEQMmwoGgTXrRtbYeOlg8rcIdNUZWXA9n5/PlSFk5VE9dwj5B2btg/LeyHrb4ABOY49XQIHak0YNNcycGU778Jq/RcxfzfS9/OkijGlqH6xM8TJRPm34TO8QzBEEplujliQBBl4g1MrFlGeuIL/HWzCO+aXF9QWrFXo3wwSNPvN70PwS4JpAAM8WVlDhfhDJMRKJ9RH2d4/ZKSW2PYTT5Yj+vqvmHHgVMLlTov0N1kJn9YynFmvhzKCargpgIRdwwp9rEjQ/9He1XjXC4a4bWw9fxO3R09ZKij8kswkeSR3egkE8ZzzZEU6Fzzhr//lwcRaGtZZAsXf/Q/YSr4s+b8Ncxxi3GEt0Y/7qxKV+jvAvbAw07w X-GND-State: clean Content-Type: text/plain; charset="utf-8" zswap writeback decompresses an entry into a fresh swap cache folio and writes it back. The folio is cold by construction, yet it is left on the LRU for reclaim to find and free later, wasting a reclaim scan and keeping cold memory resident longer than necessary. Allocate the folio off the LRU and mark it PG_dropbehind so the swap dropbehind path frees it from the swap cache once writeback completes. __swap_cache_alloc_folio() evaluates a refault on the new folio, and workingset_refault() sets PG_active when it looks recent. Until now folio_add_lru() consumed that flag and __page_cache_release() cleared it once the folio left the LRU. This folio never reaches the LRU, so nothing would clear PG_active and the folio would be freed with a PAGE_FLAGS_CHECK_AT_FREE flag set, tripping bad_page() under CONFIG_DEBUG_VM. Clear it after allocation. That is a workaround: the refault should not be evaluated on a writeback buffer at all. A fix for that is on the mailing list [1]. Link: https://lore.kernel.org/linux-mm/20260911092012.92399-1-alex@ghiti.fr= / [1] Suggested-by: Johannes Weiner Suggested-by: Nhat Pham Reviewed-by: Nhat Pham Reviewed-by: Kunwu Chan Signed-off-by: Alexandre Ghiti --- mm/zswap.c | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 0d2efe21f18a..dc8425d6b21e 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1014,7 +1014,8 @@ static int zswap_writeback_entry(struct zswap_entry *= entry, */ if (IS_ERR(folio)) return PTR_ERR(folio); - folio_add_lru(folio); + + folio_clear_active(folio); =20 /* * folio is locked, and the swapcache is now secured against @@ -1028,12 +1029,12 @@ static int zswap_writeback_entry(struct zswap_entry= *entry, tree =3D swap_zswap_tree(swpentry); if (entry !=3D xa_load(tree, offset)) { ret =3D -ENOMEM; - goto out; + goto err; } =20 if (!zswap_decompress(entry, folio)) { ret =3D -EIO; - goto out; + goto err; } =20 xa_erase(tree, offset); @@ -1047,18 +1048,30 @@ static int zswap_writeback_entry(struct zswap_entry= *entry, /* folio is up to date */ folio_mark_uptodate(folio); =20 - /* move it to the tail of the inactive list after end_writeback */ - folio_set_reclaim(folio); + folio_set_dropbehind(folio); + + /* + * Drop our reference before starting writeback so the swap cache holds + * the only one: the drop in folio_end_writeback() needs that for + * remove_mapping_set_shadow() to succeed, otherwise the folio is + * handed back to reclaim instead. + * + * Nothing can free the folio in the meantime: we hold the folio lock + * until writeback starts, PG_writeback then blocks swap cache removal, + * and folio_end_writeback() takes its own reference before clearing + * PG_writeback and donates it to the drop. + */ + folio_put(folio); =20 /* start writeback */ __swap_writepage(&ctx, folio); swap_write_submit(&ctx); =20 -out: - if (ret) { - swap_cache_del_folio(folio); - folio_unlock(folio); - } + return 0; + +err: + swap_cache_del_folio(folio); + folio_unlock(folio); folio_put(folio); return ret; } --=20 2.53.0-Meta