From nobody Fri Sep 25 13:55:50 2026 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 450343515C2; Fri, 11 Sep 2026 12:15:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789128908; cv=none; b=LdbtP1rXHz8eEMP8vnOqkgigJ3U258pj2m1RQW3JIMywOckDHbTGSibGlIGxS3jQrp3Ux+7gvP9q7I40UweVTw5G7hIbo1EJScI3+frCRCbqawM6lBHl13DxUwavVgbz2Vg5+C65jDiYWbms8sY6s8I2gTaDl7+5LYkpTsbqoJ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789128908; c=relaxed/simple; bh=xe7RTnGJ07oHf8k2Dj/sAgJUe+EDTT6QFFlh85x6Crg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YS0rRQDC6bvi6LwCGzFOk4Za2W3iXqgfyioPVHIhgvncuFZ+h3VtRD6A1nPCSN3jUC30MYsCaGLK8i1y5qeUHo+7vPWKthr3co34HxqGmGiZyQBH+1bbMBo4jeGpT28acPBeeCgX9ugHdjTDG1MLXRBW5NNSyDcQWuP+qPfYEbU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr; spf=pass smtp.mailfrom=ghiti.fr; arc=none smtp.client-ip=217.70.183.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ghiti.fr Received: by mail.gandi.net (Postfix) with ESMTPSA id DBC853EE31; Fri, 11 Sep 2026 12:14:51 +0000 (UTC) From: Alexandre Ghiti To: Johannes Weiner , Yosry Ahmed , Nhat Pham , Andrew Morton , Chris Li , Kairui Song Cc: Kairui Song , Chengming Zhou , "Matthew Wilcox (Oracle)" , Jan Kara , Kemeng Shi , Baoquan He , Barry Song , Youngjun Park , Alexander Viro , Christian Brauner , David Hildenbrand , Lorenzo Stoakes , Michal Hocko , Axel Rasmussen , Qi Zheng , Shakeel Butt , Wei Xu , Yuanchu Xie , Kunwu Chan , Tal Zussman , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Alexandre Ghiti , Usama Arif Subject: [PATCH v5 1/3] mm: swap: move LRU insertion out of the swap cache allocator Date: Fri, 11 Sep 2026 14:13:28 +0200 Message-ID: <20260911121341.178028-2-alex@ghiti.fr> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260911121341.178028-1-alex@ghiti.fr> References: <20260911121341.178028-1-alex@ghiti.fr> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-GND-Sasl: alex@ghiti.fr X-GND-State: clean X-GND-Score: -100 X-GND-Cause: dmFkZTEHRPri608QdqX6Uj/BbR+hiAEqvAJNom1pMUaPCDF92wYu3ptg6D8gDZW2m7AvuQwIR2KkJm7RAJk3qeVC81VqEBriyq2Xdjp0AkofstRRVIHILe7cZr6l7t4f+ZKKWwj7745VQLOUqBJD9XxbkV5akW9m/vZnNgm4vNpb9ydmOxvLDanCIkyPQkljOkGS/d6zP/hgl5t79Smmd9JlhTybu2Diwh+eRj0dOJKXMe8t0uExHKdrFvDVypESTLCitwfT3LQsgEesG/8CTTaCUNDOth3Vv6FT5jDyJo9k/tttrmUE3LC7BxoOrohdZXA7bWgJIy8/L4PoQV6JuGOeZNVrAydr/M/Unk1WMzvuPcjb4Hd4uk4M8iOuPTUmHhjU58cXIEjZeyJQlaRrUBY4zkTuVoehOKQUGCsV/ONEYVrKv39Wks4nHPDUS0ZmY3jXR5P9iw2byjlaTIr8qSHXgVNi5xOmkBeXJIaZiWXzFL2loFPw3GIhhYeAfYEAmy33dbUQ6iFfIo2XbUY2HE3oNuYwAZAODPAkv5Bhv/RITaIy5Y+VlE1InwPCj2LkbRYK3uzszINY0nDeIfQcwUcEpxtypI2L+HqLeS5xdm/9y3UY7NELZObB8MtI7hIjPwJZaDK6DL2EH80xTRXH1V0rF+eyEV92wcx2Y57t8IV5lQJk5g Content-Type: text/plain; charset="utf-8" This is a preparatory patch. __swap_cache_alloc_folio() adds the new folio to the LRU itself, which leaves its callers no way to act on the folio before it becomes visible to reclaim. Two users need exactly that: - moving the refault evaluation out of the swap cache folio allocation requires it to happen before folio_add_lru(): that consumes PG_active to file the folio on the inactive or the active list, and under MGLRU it also reads PG_workingset to pick the generation. Setting either flag afterwards does not move the folio; - zswap writeback dropbehind needs the buffer folio to stay off the LRU entirely, as the per-CPU LRU batch would hold a reference on it and keep remove_mapping() from freeing it once writeback completes. Defer the LRU insertion to the callers of __swap_cache_alloc_folio(): each of them adds the folio right after the allocation, so there is no functional change intended. Suggested-by: Kairui Song Reviewed-by: Nhat Pham Reviewed-by: Kunwu Chan Acked-by: Usama Arif Signed-off-by: Alexandre Ghiti Reviewed-by: Barry Song --- mm/swap.h | 6 +++--- mm/swap_state.c | 20 ++++++++++++-------- mm/swapfile.c | 2 +- mm/zswap.c | 5 +++-- 4 files changed, 19 insertions(+), 14 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index 90a551a88df6..8679cb61268e 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -312,9 +312,9 @@ bool swap_cache_has_folio(swp_entry_t entry); struct folio *swap_cache_get_folio(swp_entry_t entry); void *swap_cache_get_shadow(swp_entry_t entry); void swap_cache_del_folio(struct folio *folio); -struct folio *swap_cache_alloc_folio(swp_entry_t target_entry, gfp_t gfp_m= ask, - unsigned long orders, struct vm_fault *vmf, - struct mempolicy *mpol, pgoff_t ilx); +struct folio *__swap_cache_alloc_folio(swp_entry_t target_entry, gfp_t gfp= _mask, + unsigned long orders, struct vm_fault *vmf, + struct mempolicy *mpol, pgoff_t ilx); /* Below helpers require the caller to lock and pass in the swap cluster. = */ void __swap_cache_add_folio(struct swap_cluster_info *ci, struct folio *folio, swp_entry_t entry); diff --git a/mm/swap_state.c b/mm/swap_state.c index f3961fdd857d..e5b7fa468ade 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -489,13 +489,11 @@ static struct folio *__swap_cache_alloc(struct swap_c= luster_info *ci, node_stat_mod_folio(folio, NR_FILE_PAGES, nr_pages); lruvec_stat_mod_folio(folio, NR_SWAPCACHE, nr_pages); =20 - /* Caller will initiate read into locked new_folio */ - folio_add_lru(folio); return folio; } =20 /** - * swap_cache_alloc_folio - Allocate folio for swapped out slot in swap ca= che. + * __swap_cache_alloc_folio - Allocate folio for swapped out slot in swap = cache. * @targ_entry: swap entry indicating the target slot * @gfp: memory allocation flags * @orders: allocation orders, must be non zero @@ -507,13 +505,17 @@ static struct folio *__swap_cache_alloc(struct swap_c= luster_info *ci, * doing IO (e.g. swap in or zswap writeback). The swap slot indicated by * @targ_entry must have a non-zero swap count (swapped out). * + * The returned folio is locked and is NOT on the LRU. The caller must eit= her + * add it to the LRU with folio_add_lru() so page reclaim can find it, or = free + * it directly once done; a folio left off the LRU is unreclaimable and le= aks. + * * Context: Caller must protect the swap device with reference count or lo= cks. * Return: Returns the folio if allocation succeeded and folio is in the s= wap * cache. Returns error code if failed due to race, OOM or invalid argumen= ts. */ -struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp, - unsigned long orders, struct vm_fault *vmf, - struct mempolicy *mpol, pgoff_t ilx) +struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp, + unsigned long orders, struct vm_fault *vmf, + struct mempolicy *mpol, pgoff_t ilx) { int order, err; struct folio *ret; @@ -649,12 +651,13 @@ static struct folio *swap_cache_read_folio(struct swa= p_io_ctx *ctx, folio =3D swap_cache_get_folio(entry); if (folio) return folio; - folio =3D swap_cache_alloc_folio(entry, gfp, BIT(0), NULL, mpol, ilx); + folio =3D __swap_cache_alloc_folio(entry, gfp, BIT(0), NULL, mpol, ilx); } while (PTR_ERR(folio) =3D=3D -EEXIST); =20 if (IS_ERR_OR_NULL(folio)) return NULL; =20 + folio_add_lru(folio); swap_read_folio(ctx, folio); if (readahead) { folio_set_readahead(folio); @@ -690,12 +693,13 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gf= p, unsigned long orders, folio =3D swap_cache_get_folio(entry); if (folio) return folio; - folio =3D swap_cache_alloc_folio(entry, gfp, orders, vmf, mpol, ilx); + folio =3D __swap_cache_alloc_folio(entry, gfp, orders, vmf, mpol, ilx); } while (PTR_ERR(folio) =3D=3D -EEXIST); =20 if (IS_ERR(folio)) return folio; =20 + folio_add_lru(folio); swap_read_folio(&ctx, folio); swap_read_submit(&ctx); return folio; diff --git a/mm/swapfile.c b/mm/swapfile.c index 53bf01d5f7f1..d678a40fcaac 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1870,7 +1870,7 @@ void folio_put_swap(struct folio *folio, struct page = *page) * CPU1 CPU2 * do_swap_page() * ... swapoff+swapon - * swap_cache_alloc_folio() + * __swap_cache_alloc_folio() * // check swap_map * // verify PTE not changed * diff --git a/mm/zswap.c b/mm/zswap.c index 37f34e406c8e..0d2efe21f18a 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1001,8 +1001,8 @@ static int zswap_writeback_entry(struct zswap_entry *= entry, return -EEXIST; =20 mpol =3D get_task_policy(current); - folio =3D swap_cache_alloc_folio(swpentry, GFP_KERNEL, BIT(0), NULL, mpol, - NO_INTERLEAVE_INDEX); + folio =3D __swap_cache_alloc_folio(swpentry, GFP_KERNEL, BIT(0), NULL, mp= ol, + NO_INTERLEAVE_INDEX); put_swap_device(si); =20 /* @@ -1014,6 +1014,7 @@ static int zswap_writeback_entry(struct zswap_entry *= entry, */ if (IS_ERR(folio)) return PTR_ERR(folio); + folio_add_lru(folio); =20 /* * folio is locked, and the swapcache is now secured against --=20 2.53.0-Meta From nobody Fri Sep 25 13:55:50 2026 Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D89E364059; Fri, 11 Sep 2026 12:16:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789128966; cv=none; b=ankrrb1E71J9PDkix/nKpVO0OJ0DFoM5FKqmF1sLanF0L7v12NO6J7EYTvjbjkd9CUhqPeq0e1UMI3SpdLUdbGyIUdHq62ALHEGezp0yLO6BtmK4GerNRGNL34hoJdv/r1WzmPrNQjcK3c07xPm5bQTe/juQhUvz48jUpiF24kg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789128966; c=relaxed/simple; bh=490GEyhkTwzzmJ0XnkWlaB6Ze1NRBFNHe3rMrKdQjOc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LE0JDDTe0s//Of4UXEUGRJcOGoTndm6LKxkINUqiXbpHdldT/YCEJ8/H/Vn6RfK2kHBra9ApVYVNF9Ely7umfFj5vrN0bdZTkT8HDJe42OfiAuSEvWEJvtCSsgh5xlcFlrUS5uQuB6FLR7sEXifFlp7TjcPAvj3gSOd1J/VS2IA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr; spf=pass smtp.mailfrom=ghiti.fr; arc=none smtp.client-ip=217.70.183.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ghiti.fr Received: by mail.gandi.net (Postfix) with ESMTPSA id 07E7E3EBAD; Fri, 11 Sep 2026 12:15:56 +0000 (UTC) From: Alexandre Ghiti To: Johannes Weiner , Yosry Ahmed , Nhat Pham , Andrew Morton , Chris Li , Kairui Song Cc: Kairui Song , Chengming Zhou , "Matthew Wilcox (Oracle)" , Jan Kara , Kemeng Shi , Baoquan He , Barry Song , Youngjun Park , Alexander Viro , Christian Brauner , David Hildenbrand , Lorenzo Stoakes , Michal Hocko , Axel Rasmussen , Qi Zheng , Shakeel Butt , Wei Xu , Yuanchu Xie , Kunwu Chan , Tal Zussman , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Alexandre Ghiti Subject: [PATCH v5 2/3] mm: swap: drop dropbehind swap cache folios on writeback completion Date: Fri, 11 Sep 2026 14:13:29 +0200 Message-ID: <20260911121341.178028-3-alex@ghiti.fr> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260911121341.178028-1-alex@ghiti.fr> References: <20260911121341.178028-1-alex@ghiti.fr> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-GND-Sasl: alex@ghiti.fr X-GND-Cause: dmFkZTGfAlh9c+pEh1Y00g1ucsY9TXoPvlSpq+kLO/TyS5eegFUhSZk1HWGBEKI2EIGpMjwYV84dXepg2QrFzjdjhJpD6uJ7UOjwxR4tmFH3HJlDtAdZIv/b82CTk0cvtKYnnlQSesrrWi8eaFeV5GPBNEW00Fw38Sew7azh5zrRlSzfWnGj/zBzOo8f54f1hMdr3s25eShLr9Q2E+gJ5nIKtpdbn7VmBtfAShkJZR/VKWYVh9m0eENOkd+/YMjkMt4MJdqVG1j0L+yDgSQOi9ZZkE0CUfSq92WrHbhe1A00cDoD08RJOchvE15pjmEcvjeF9JzmN2OHb/AlwyGI3s7wzPg8PnXvoD9pAIOfCi0mwSxodaEGLbHW1jDUkqzt745Bp0HYIoaIc5WW0VCvRtpYrA4qU1pkGmrhM1XY02MEj7deCOandoFqRVbSSfKnu1LbHA4/N1uBCV7MQfzlK5PgHmRL/Xzs4QHkkBgiSHLOvqwc9WNOvlb7q0vFUIDCLQXFWsHH6Mj7zSfqhFrNWh2p3tj+EKWRiBWRPb1X+UPF3eUJsAb0MWMJPPAJzAkIwKm/tv/Fwea4FXSeGShkyeiWvvj3hM+2On/na6Wor0wKce3QPDICoRIeudagdRsbeNUCjdaCIDHV30bApqUl5mmAfuH50egqlXbeb+64eAeQL/VMQQ X-GND-State: clean X-GND-Score: -100 Content-Type: text/plain; charset="utf-8" A PG_dropbehind folio is dropped from its cache once writeback completes rather than left for reclaim to find later; this is implemented for file folios in folio_end_dropbehind(). Extend it to swap cache folios. The drop takes the folio and swap cluster locks and may sleep, so it cannot run in interrupt context. Set BIO_COMPLETE_IN_TASK on the write, as the file dropbehind paths do, and drop the folio directly from folio_end_writeback(). Suggested-by: Yosry Ahmed Suggested-by: Johannes Weiner Suggested-by: Nhat Pham Reviewed-by: Nhat Pham Reviewed-by: Kunwu Chan Signed-off-by: Alexandre Ghiti Reviewed-by: Barry Song --- include/linux/swap.h | 6 ++++++ mm/filemap.c | 19 +++++++++++++++++ mm/page_io.c | 9 ++++++++ mm/swap_state.c | 42 +++++++++++++++++++++++++++++++++++++ mm/vmscan.c | 49 +++++++++++++++++++++++++++++++++++--------- 5 files changed, 115 insertions(+), 10 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 5658a1634b85..538b723a276f 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -318,6 +318,9 @@ static inline bool lru_cache_disabled(void) =20 extern unsigned long shrink_all_memory(unsigned long nr_pages); long remove_mapping(struct address_space *mapping, struct folio *folio); +long remove_mapping_set_shadow(struct address_space *mapping, + struct folio *folio, + struct mem_cgroup *target_memcg); =20 #if defined(CONFIG_SYSFS) && defined(CONFIG_NUMA) extern int reclaim_register_node(struct node *node); @@ -402,6 +405,8 @@ void swap_put_entries_direct(swp_entry_t entry, int nr); */ bool folio_free_swap(struct folio *folio); =20 +void swap_writeback_dropbehind_folio(struct folio *folio); + /* Allocate / free (hibernation) exclusive entries */ swp_entry_t swap_alloc_hibernation_slot(int type); void swap_free_hibernation_slot(swp_entry_t entry); @@ -412,6 +417,7 @@ static inline void put_swap_device(struct swap_info_str= uct *si) } =20 #else /* CONFIG_SWAP */ +static inline void swap_writeback_dropbehind_folio(struct folio *folio) {} static inline struct swap_info_struct *get_swap_device(swp_entry_t entry) { return NULL; diff --git a/mm/filemap.c b/mm/filemap.c index 6afec636881f..e1f1bbe943ce 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1686,6 +1686,8 @@ EXPORT_SYMBOL_GPL(folio_end_writeback_no_dropbehind); */ void folio_end_writeback(struct folio *folio) { + bool swap_dropbehind; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); =20 /* @@ -1695,7 +1697,24 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + + /* + * Sample this before folio_end_writeback_no_dropbehind() clears + * PG_writeback: until then a racing swapin cannot remove the folio from + * the swap cache. Afterwards it can, and the drop below then finds a + * non-swapcache folio and puts it back on the LRU instead. The + * reference taken above keeps the folio alive across that window. + */ + swap_dropbehind =3D folio_test_swapcache(folio) && + folio_test_dropbehind(folio); + folio_end_writeback_no_dropbehind(folio); + + if (swap_dropbehind) { + swap_writeback_dropbehind_folio(folio); + return; + } + folio_end_dropbehind(folio); folio_put(folio); } diff --git a/mm/page_io.c b/mm/page_io.c index 88962571cb93..52eae99de6e3 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -602,6 +602,15 @@ static void swap_bdev_submit_write(struct swap_io_ctx = *ctx) submit_bio_wait(bio); end_swap_bio_write(bio); } else { + int p; + + for (p =3D 0; p < sio->nr_bvecs; p++) { + if (folio_test_dropbehind(bvec_folio(&sio->bvecs[p]))) { + bio_set_flag(bio, BIO_COMPLETE_IN_TASK); + break; + } + } + bio->bi_end_io =3D end_swap_bio_write; submit_bio(bio); } diff --git a/mm/swap_state.c b/mm/swap_state.c index e5b7fa468ade..b1656e2d5288 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -543,6 +543,48 @@ struct folio *__swap_cache_alloc_folio(swp_entry_t tar= g_entry, gfp_t gfp, return ret; } =20 +/** + * swap_writeback_dropbehind_folio - drop a dropbehind swap cache folio + * @folio: the off-LRU folio whose writeback has completed + * + * Context: task context, with the reference taken by folio_end_writeback() + * donated to us. + */ +void swap_writeback_dropbehind_folio(struct folio *folio) +{ + struct mem_cgroup *memcg; + + folio_lock(folio); + + /* The folio was allocated off the LRU and nothing re-adds it here. */ + VM_WARN_ON_ONCE_FOLIO(folio_test_lru(folio), folio); + + rcu_read_lock(); + memcg =3D folio_memcg(folio); + if (!mem_cgroup_tryget(memcg)) + memcg =3D NULL; + rcu_read_unlock(); + + /* + * Gate remove_mapping_set_shadow() on folio_test_swapcache(): a racing + * swapin may have freed the swap slot (folio_free_swap()) and dropped the + * folio from the cache, and it must not run on a non-swapcache folio (it + * would trip __remove_mapping()'s mapping =3D=3D folio_mapping() check). + */ + if (!folio_test_swapcache(folio) || folio_test_writeback(folio) || + !remove_mapping_set_shadow(swap_address_space(folio->swap), folio, + memcg)) { + /* Raced: the folio is now owned by the swapin; put it back. */ + folio_clear_dropbehind(folio); + folio_add_lru(folio); + } + + mem_cgroup_put(memcg); + + folio_unlock(folio); + folio_put(folio); +} + /* * If we are the only user, then try to free up the swap cache. * diff --git a/mm/vmscan.c b/mm/vmscan.c index f11491ee9ed5..a02f942418d3 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -783,6 +783,22 @@ static int __remove_mapping(struct address_space *mapp= ing, struct folio *folio, return 0; } =20 +static long __remove_mapping_unfreeze(struct address_space *mapping, + struct folio *folio, bool reclaimed, + struct mem_cgroup *target_memcg) +{ + if (__remove_mapping(mapping, folio, reclaimed, target_memcg)) { + /* + * Unfreezing the refcount with 1 effectively + * drops the pagecache ref for us without requiring another + * atomic operation. + */ + folio_ref_unfreeze(folio, 1); + return folio_nr_pages(folio); + } + return 0; +} + /** * remove_mapping() - Attempt to remove a folio from its mapping. * @mapping: The address space. @@ -797,16 +813,29 @@ static int __remove_mapping(struct address_space *map= ping, struct folio *folio, */ long remove_mapping(struct address_space *mapping, struct folio *folio) { - if (__remove_mapping(mapping, folio, false, NULL)) { - /* - * Unfreezing the refcount with 1 effectively - * drops the pagecache ref for us without requiring another - * atomic operation. - */ - folio_ref_unfreeze(folio, 1); - return folio_nr_pages(folio); - } - return 0; + return __remove_mapping_unfreeze(mapping, folio, false, NULL); +} + +/** + * remove_mapping_set_shadow() - Remove a folio and record an eviction sha= dow. + * @mapping: The address space. + * @folio: The folio to remove. + * @target_memcg: The memcg to charge the eviction shadow to; the caller m= ust + * keep it alive across the call. + * + * Like remove_mapping(), but stores a workingset eviction shadow the way = page + * reclaim does, so that a later refault can be detected and the folio + * re-activated. + * Return: The number of pages removed from the mapping. 0 if the folio + * could not be removed. + * Context: The caller should have a single refcount on the folio and + * hold its lock. + */ +long remove_mapping_set_shadow(struct address_space *mapping, + struct folio *folio, + struct mem_cgroup *target_memcg) +{ + return __remove_mapping_unfreeze(mapping, folio, true, target_memcg); } =20 /** --=20 2.53.0-Meta From nobody Fri Sep 25 13:55:50 2026 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5AFF046D548; Fri, 11 Sep 2026 12:17:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789129036; cv=none; b=TrVD8DgRID5tUJ2aTz9XLaP/BzS1/SolUYF3R0e2xrauP7NMA65Ia/SHIMjWbVsdwPm7zsz77RymMvvv/n0hmvgih54KrDprskq/tHNp4uMwIOjQeujUuuhJvoaIcfZqVPpzDzpMOqv6dFjkZJUfgKLZw6OkLqOhl5/1OVR4IcU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789129036; c=relaxed/simple; bh=hgG4ex11E74cUjRpihO9PAt9otkIlr4zzhbWOYoUoiY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=idaZjo0ucuV2m9hOjO7Ezj8aI0/iapRYKV6EOk4v2cxrIIvShfUraP/IqetudE1E8kV7VRsV8Vcf6dH4jfXcQ72hsTczwcjONkC9k8iNYHQ7ZGizft1f61TogTQWrERb1wnwWJb9cYVzCu4XIXEwb9QLT5CnC8c8ufJYac3q8lM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr; spf=pass smtp.mailfrom=ghiti.fr; arc=none smtp.client-ip=217.70.183.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ghiti.fr Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ghiti.fr Received: by mail.gandi.net (Postfix) with ESMTPSA id F311F3EE2F; Fri, 11 Sep 2026 12:17:02 +0000 (UTC) From: Alexandre Ghiti To: Johannes Weiner , Yosry Ahmed , Nhat Pham , Andrew Morton , Chris Li , Kairui Song Cc: Kairui Song , Chengming Zhou , "Matthew Wilcox (Oracle)" , Jan Kara , Kemeng Shi , Baoquan He , Barry Song , Youngjun Park , Alexander Viro , Christian Brauner , David Hildenbrand , Lorenzo Stoakes , Michal Hocko , Axel Rasmussen , Qi Zheng , Shakeel Butt , Wei Xu , Yuanchu Xie , Kunwu Chan , Tal Zussman , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Alexandre Ghiti Subject: [PATCH v5 3/3] mm: zswap: drop cold writeback folios via swap dropbehind Date: Fri, 11 Sep 2026 14:13:30 +0200 Message-ID: <20260911121341.178028-4-alex@ghiti.fr> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260911121341.178028-1-alex@ghiti.fr> References: <20260911121341.178028-1-alex@ghiti.fr> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-GND-Sasl: alex@ghiti.fr X-GND-State: clean X-GND-Score: -100 X-GND-Cause: dmFkZTGjfH5CE7EP1PUuoeu/l2HFd6FdF++hpOgW9bG6M/BTM9F3kapSlk9U7ADtn6kKTISwel0dj9XMJHazyTDJbuCGf1+V6dtQFmMVDsz/j6U8zJS5UWFxlLhlncEBXEX9YRxi/C/PDeW/pyF/LEG1lC35hb8/T6rd2drmQXNcsrOJKh3DWo1SZ4dO8uSsp8GEYnEzEJX5QWflCLS80+j+zqwa+6xD+CuZg9bY1SF9yIU/qkfoyyWDW3JGDJ+aEoJjjp0rdWhKxA6l0J4myGoeFOAENCEQex0bOWMMxHXz5RPKc2sXHCvtKxjhZn7rZZwnhvVGcaI9kZTf2hXpEZopWDbcmhuI+lkgO1VZkGtngqreNfPmyf09XQyJBcekrXeysSX6w+3lqnF8NI/Sss5Y9CLV3j4zFIVtAvlLtHyzjI01JS4lDYoffcBQM7HP0VdS8IKursJsgKwnxu844P8/eGsX7PaeaSHiuGSsUeJrNPKAHUlC5OIcapVTLWGKEdCVdnt4Poaya2cUHRSJJfNVtdy8HPTF8ZTFiplDr9GK5+OWkhuRKiGPKtsY5bCnmQ65H5rYfs2ZF82vh/9baR+n6C7HASFm0sLmnUyyJsJbrU2U6AtoFd2H8ZiO43+6SsmE/N8qdn+EU909sSEeMIJHFlxk/nhu4blmsYlbO56LUzZtDw Content-Type: text/plain; charset="utf-8" zswap writeback decompresses an entry into a fresh swap cache folio and writes it back. The folio is cold by construction, yet it is left on the LRU for reclaim to find and free later, wasting a reclaim scan and keeping cold memory resident longer than necessary. Allocate the folio off the LRU and mark it PG_dropbehind so the swap dropbehind path frees it from the swap cache once writeback completes. __swap_cache_alloc_folio() evaluates a refault on the new folio, and workingset_refault() sets PG_active when it looks recent. Until now folio_add_lru() consumed that flag and __page_cache_release() cleared it once the folio left the LRU. This folio never reaches the LRU, so nothing would clear PG_active and the folio would be freed with a PAGE_FLAGS_CHECK_AT_FREE flag set, tripping bad_page() under CONFIG_DEBUG_VM. Clear it after allocation. That is a workaround: the refault should not be evaluated on a writeback buffer at all. A fix for that is on the mailing list [1]. Link: https://lore.kernel.org/linux-mm/20260911092012.92399-1-alex@ghiti.fr= / [1] Suggested-by: Johannes Weiner Suggested-by: Nhat Pham Reviewed-by: Kunwu Chan Signed-off-by: Alexandre Ghiti Reviewed-by: Nhat Pham --- mm/zswap.c | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 0d2efe21f18a..dc8425d6b21e 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1014,7 +1014,8 @@ static int zswap_writeback_entry(struct zswap_entry *= entry, */ if (IS_ERR(folio)) return PTR_ERR(folio); - folio_add_lru(folio); + + folio_clear_active(folio); =20 /* * folio is locked, and the swapcache is now secured against @@ -1028,12 +1029,12 @@ static int zswap_writeback_entry(struct zswap_entry= *entry, tree =3D swap_zswap_tree(swpentry); if (entry !=3D xa_load(tree, offset)) { ret =3D -ENOMEM; - goto out; + goto err; } =20 if (!zswap_decompress(entry, folio)) { ret =3D -EIO; - goto out; + goto err; } =20 xa_erase(tree, offset); @@ -1047,18 +1048,30 @@ static int zswap_writeback_entry(struct zswap_entry= *entry, /* folio is up to date */ folio_mark_uptodate(folio); =20 - /* move it to the tail of the inactive list after end_writeback */ - folio_set_reclaim(folio); + folio_set_dropbehind(folio); + + /* + * Drop our reference before starting writeback so the swap cache holds + * the only one: the drop in folio_end_writeback() needs that for + * remove_mapping_set_shadow() to succeed, otherwise the folio is + * handed back to reclaim instead. + * + * Nothing can free the folio in the meantime: we hold the folio lock + * until writeback starts, PG_writeback then blocks swap cache removal, + * and folio_end_writeback() takes its own reference before clearing + * PG_writeback and donates it to the drop. + */ + folio_put(folio); =20 /* start writeback */ __swap_writepage(&ctx, folio); swap_write_submit(&ctx); =20 -out: - if (ret) { - swap_cache_del_folio(folio); - folio_unlock(folio); - } + return 0; + +err: + swap_cache_del_folio(folio); + folio_unlock(folio); folio_put(folio); return ret; } --=20 2.53.0-Meta