From nobody Thu Oct 2 14:24:05 2025 Received: from mail-oo1-f49.google.com (mail-oo1-f49.google.com [209.85.161.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C67D5288C26 for ; Tue, 16 Sep 2025 16:02:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758038527; cv=none; b=bdJNQJUjZDBGSS+PMZo/yRK+buJnoOJU20nlNh6hm8uOyjQsoGYgWIH908/e33VeMaCuOpvJ3nTI1DZhTFpMLUeHcbY27lnp71WWZMj2DFXBLtBAx4E0vNr5PupBMdeBKqoGehjVK1Jf5gMh8orf3sdxr4BH9ZS6mvKgI3D7ahI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758038527; c=relaxed/simple; bh=RkZBErRXaxG7+svdbz8Q7l9R8uRrzR3VYE5pQ9kOhqo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=s91C+ua57sOrx5ElhxckszcS5bR4O5mM/1GR21+2Bt3XC65NSC0s2NLn2cM0d8TG8cai3EC6Bogx539v5W5R0J2JIAr6crLeQ7XAMnWnVzp3rgE88SGCegD/INB3cF+Q2q2xCijgq+7kxcSJyqFlWgQVwjx7SDl5Pi1eG1/OtWE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AnxL8q14; arc=none smtp.client-ip=209.85.161.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AnxL8q14" Received: by mail-oo1-f49.google.com with SMTP id 006d021491bc7-6234b7bdf3dso759882eaf.0 for ; Tue, 16 Sep 2025 09:02:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1758038525; x=1758643325; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=qcEphP+ICQn2G6iNahIPkKAJNc9HmOiqOSMT0ntzzdE=; b=AnxL8q14oTP39kWthZ+olKP5bjcm7TvrXT6dZquB/WvAWKlsUujjk+p4OZdzovFnof p1reojFC6oWltgiXlG+aM4GV+suLb449hueDnypoVWKyYHOTKHGWLnULShghI1RK9Biz 56NQWF4nT5KiPAg5Ow8S7JBgd7YhrWThyAFR93JrE35BJ9xHsjp7RztiEgdD72rNmz6u ys8vCP000xYMjXG1/HFFZCN1MhA76WyAo6gRi2wLBzSvXFG1oYYeDfB7Zd7XYzMOZofr cYskWTQrWvaSvZxoW1vjimyUndYL8GcDtdxKawhQLME35TK81aqy56AQSY9vfIxx0ahV HwNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758038525; x=1758643325; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=qcEphP+ICQn2G6iNahIPkKAJNc9HmOiqOSMT0ntzzdE=; b=FB9PrIFFym13tkbrM4e16of9xJaCghsmHFqhJ6qQ9AwPyvq4jKAL4V2ndWIIG6z7Tt sI8oSI6C+zY4tNAebmxQ7CmabNnoWTdCFQ7ZhBLYSHtV84mwdwAXEZ13W4vt4NoKix1k W2m1flPPdFvnmOhhnNqJXLXIqEzmM9GK82xfdTtnrICuum1u28ixDrYVy60nxXE86+ly ICcPs1f6B0tkfEFoX6KntghFJDaCgS4R66PSBUrrW5JivFXKIXNra9QGTbq6UEGLyNjx QZYSlRvdRZgXw1XLy+7HvzF2mQpTbFtIzygmhSDNeM63WA4xcP8JucojCzDvZarAe4Ez g83g== X-Forwarded-Encrypted: i=1; AJvYcCXGPostIHxfRKodvQFlq7RLrbgcA8MVe1eG2gHKlIx45WggZe1h7LRIrE/KMWktUpoFm6/TyglXmwr4Vrg=@vger.kernel.org X-Gm-Message-State: AOJu0YxVSsVSLRpVoA4AyMzU7Pp7urdJ3xZF2ZRh4Xp9CfH8+w1HT6mh zXRJi7DKaUhCZ/c8/+BnmgBDehBWuMvQLCnrNunEzvO/2yqlMajDVtUm X-Gm-Gg: ASbGnct6FSGYPD0lYFfbzBDZsmEg9+BOcx2nDHPgZDdQaX+oguPa/3GfVvb4yUzTVUl r2a9wnwlyvuQFfKgqT7rpJ6fPRYjSRKtFqwKBYClDbaK1jG3Ef5EzJhF+oQgdkppTH56bQRJw5Y 78ymjt7JYamYT/YvuWEciB2XTsxHV0Hd23WlytLC1/g7ojgRviD0KVGRKsNS6dFCIgFMQ7rtQio 7RnksQ3iwrbliZw7DVC1UXg+a8zem5mfsVbNZf6DaD2iSLn+Ka/pW2Epocxc0nAb176Dq9WNWfP j0JFtnwtrH39GXTiqjh4r1Ve9Jvyc1+VpBptXu87kZy629PhrXYBn/Fn2UVYMCYu1b+jGuk2MIR Z X-Google-Smtp-Source: AGHT+IELVtFRC6Nki4vFxd93AiETiEyoU50dddSTirN6BjBYWWFVae4So0375D/Ao7WW2cQMIoMTcQ== X-Received: by 2002:a05:6808:1507:b0:438:27af:3ff8 with SMTP id 5614622812f47-43b8d86ef79mr8425509b6e.7.1758038524551; Tue, 16 Sep 2025 09:02:04 -0700 (PDT) Received: from KASONG-MC4.tencent.com ([101.32.222.185]) by smtp.gmail.com with ESMTPSA id af79cd13be357-820cd703f54sm969765485a.37.2025.09.16.09.01.58 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 16 Sep 2025 09:02:03 -0700 (PDT) From: Kairui Song To: linux-mm@kvack.org Cc: Kairui Song , Andrew Morton , Matthew Wilcox , Hugh Dickins , Chris Li , Barry Song , Baoquan He , Nhat Pham , Kemeng Shi , Baolin Wang , Ying Huang , Johannes Weiner , David Hildenbrand , Yosry Ahmed , Lorenzo Stoakes , Zi Yan , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v4 08/15] mm, swap: cleanup swap cache API and add kerneldoc Date: Wed, 17 Sep 2025 00:00:53 +0800 Message-ID: <20250916160100.31545-9-ryncsn@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250916160100.31545-1-ryncsn@gmail.com> References: <20250916160100.31545-1-ryncsn@gmail.com> Reply-To: Kairui Song Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kairui Song In preparation for replacing the swap cache backend with the swap table, clean up and add proper kernel doc for all swap cache APIs. Now all swap cache APIs are well-defined with consistent names. No feature change, only renaming and documenting. Signed-off-by: Kairui Song Acked-by: Chris Li Reviewed-by: Barry Song Reviewed-by: Baolin Wang Acked-by: David Hildenbrand --- mm/filemap.c | 2 +- mm/memory-failure.c | 2 +- mm/memory.c | 2 +- mm/shmem.c | 10 +++--- mm/swap.h | 48 ++++++++++++++----------- mm/swap_state.c | 86 ++++++++++++++++++++++++++++++++------------- mm/swapfile.c | 8 ++--- mm/vmscan.c | 2 +- mm/zswap.c | 2 +- 9 files changed, 103 insertions(+), 59 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 8d078aa2738a..2a05b1fdd445 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -4525,7 +4525,7 @@ static void filemap_cachestat(struct address_space *m= apping, * invalidation, so there might not be * a shadow in the swapcache (yet). */ - shadow =3D get_shadow_from_swap_cache(swp); + shadow =3D swap_cache_get_shadow(swp); if (!shadow) goto resched; } diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 6d9134e3d115..3edebb0cda30 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1127,7 +1127,7 @@ static int me_swapcache_clean(struct page_state *ps, = struct page *p) struct folio *folio =3D page_folio(p); int ret; =20 - delete_from_swap_cache(folio); + swap_cache_del_folio(folio); =20 ret =3D delete_from_lru_cache(folio) ? MF_FAILED : MF_RECOVERED; folio_unlock(folio); diff --git a/mm/memory.c b/mm/memory.c index 5808c4ef21b3..41e641823558 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4699,7 +4699,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) =20 memcg1_swapin(entry, nr_pages); =20 - shadow =3D get_shadow_from_swap_cache(entry); + shadow =3D swap_cache_get_shadow(entry); if (shadow) workingset_refault(folio, shadow); =20 diff --git a/mm/shmem.c b/mm/shmem.c index 410f27bc4752..077744a9e9da 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1661,13 +1661,13 @@ int shmem_writeout(struct folio *folio, struct swap= _iocb **plug, } =20 /* - * The delete_from_swap_cache() below could be left for + * The swap_cache_del_folio() below could be left for * shrink_folio_list()'s folio_free_swap() to dispose of; * but I'm a little nervous about letting this folio out of * shmem_writeout() in a hybrid half-tmpfs-half-swap state * e.g. folio_mapping(folio) might give an unexpected answer. */ - delete_from_swap_cache(folio); + swap_cache_del_folio(folio); goto redirty; } if (nr_pages > 1) @@ -2045,7 +2045,7 @@ static struct folio *shmem_swap_alloc_folio(struct in= ode *inode, new->swap =3D entry; =20 memcg1_swapin(entry, nr_pages); - shadow =3D get_shadow_from_swap_cache(entry); + shadow =3D swap_cache_get_shadow(entry); if (shadow) workingset_refault(new, shadow); folio_add_lru(new); @@ -2183,7 +2183,7 @@ static void shmem_set_folio_swapin_error(struct inode= *inode, pgoff_t index, nr_pages =3D folio_nr_pages(folio); folio_wait_writeback(folio); if (!skip_swapcache) - delete_from_swap_cache(folio); + swap_cache_del_folio(folio); /* * Don't treat swapin error folio as alloced. Otherwise inode->i_blocks * won't be 0 when inode is released and thus trigger WARN_ON(i_blocks) @@ -2422,7 +2422,7 @@ static int shmem_swapin_folio(struct inode *inode, pg= off_t index, folio->swap.val =3D 0; swapcache_clear(si, swap, nr_pages); } else { - delete_from_swap_cache(folio); + swap_cache_del_folio(folio); } folio_mark_dirty(folio); swap_free_nr(swap, nr_pages); diff --git a/mm/swap.h b/mm/swap.h index 30b1039c27fe..6c4acb549bec 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -167,17 +167,29 @@ static inline bool folio_matches_swap_entry(const str= uct folio *folio, return folio_entry.val =3D=3D round_down(entry.val, nr_pages); } =20 +/* + * All swap cache helpers below require the caller to ensure the swap entr= ies + * used are valid and stablize the device by any of the following ways: + * - Hold a reference by get_swap_device(): this ensures a single entry is + * valid and increases the swap device's refcount. + * - Locking a folio in the swap cache: this ensures the folio's swap entr= ies + * are valid and pinned, also implies reference to the device. + * - Locking anything referencing the swap entry: e.g. PTL that protects + * swap entries in the page table, similar to locking swap cache folio. + * - See the comment of get_swap_device() for more complex usage. + */ +struct folio *swap_cache_get_folio(swp_entry_t entry); +void *swap_cache_get_shadow(swp_entry_t entry); +int swap_cache_add_folio(struct folio *folio, swp_entry_t entry, + gfp_t gfp, void **shadow); +void swap_cache_del_folio(struct folio *folio); +void __swap_cache_del_folio(struct folio *folio, + swp_entry_t entry, void *shadow); +void swap_cache_clear_shadow(int type, unsigned long begin, + unsigned long end); + void show_swap_cache_info(void); -void *get_shadow_from_swap_cache(swp_entry_t entry); -int add_to_swap_cache(struct folio *folio, swp_entry_t entry, - gfp_t gfp, void **shadowp); -void __delete_from_swap_cache(struct folio *folio, - swp_entry_t entry, void *shadow); -void delete_from_swap_cache(struct folio *folio); -void clear_shadow_from_swap_cache(int type, unsigned long begin, - unsigned long end); void swapcache_clear(struct swap_info_struct *si, swp_entry_t entry, int n= r); -struct folio *swap_cache_get_folio(swp_entry_t entry); struct folio *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct vm_area_struct *vma, unsigned long addr, struct swap_iocb **plug); @@ -305,28 +317,22 @@ static inline struct folio *swap_cache_get_folio(swp_= entry_t entry) return NULL; } =20 -static inline void *get_shadow_from_swap_cache(swp_entry_t entry) +static inline void *swap_cache_get_shadow(swp_entry_t entry) { return NULL; } =20 -static inline int add_to_swap_cache(struct folio *folio, swp_entry_t entry, - gfp_t gfp_mask, void **shadowp) -{ - return -1; -} - -static inline void __delete_from_swap_cache(struct folio *folio, - swp_entry_t entry, void *shadow) +static inline int swap_cache_add_folio(swp_entry_t entry, struct folio *fo= lio, + gfp_t gfp, void **shadow) { + return -EINVAL; } =20 -static inline void delete_from_swap_cache(struct folio *folio) +static inline void swap_cache_del_folio(struct folio *folio) { } =20 -static inline void clear_shadow_from_swap_cache(int type, unsigned long be= gin, - unsigned long end) +static inline void __swap_cache_del_folio(struct folio *folio, swp_entry_t= entry, void *shadow) { } =20 diff --git a/mm/swap_state.c b/mm/swap_state.c index 0ad4f3b41f1b..f3a32a06a950 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -78,8 +78,8 @@ void show_swap_cache_info(void) * Context: Caller must ensure @entry is valid and protect the swap device * with reference count or locks. * Return: Returns the found folio on success, NULL otherwise. The caller - * must lock and check if the folio still matches the swap entry before - * use (e.g. with folio_matches_swap_entry). + * must lock nd check if the folio still matches the swap entry before + * use (e.g., folio_matches_swap_entry). */ struct folio *swap_cache_get_folio(swp_entry_t entry) { @@ -90,7 +90,15 @@ struct folio *swap_cache_get_folio(swp_entry_t entry) return folio; } =20 -void *get_shadow_from_swap_cache(swp_entry_t entry) +/** + * swap_cache_get_shadow - Looks up a shadow in the swap cache. + * @entry: swap entry used for the lookup. + * + * Context: Caller must ensure @entry is valid and protect the swap device + * with reference count or locks. + * Return: Returns either NULL or an XA_VALUE (shadow). + */ +void *swap_cache_get_shadow(swp_entry_t entry) { struct address_space *address_space =3D swap_address_space(entry); pgoff_t idx =3D swap_cache_index(entry); @@ -102,12 +110,21 @@ void *get_shadow_from_swap_cache(swp_entry_t entry) return NULL; } =20 -/* - * add_to_swap_cache resembles filemap_add_folio on swapper_space, - * but sets SwapCache flag and 'swap' instead of mapping and index. +/** + * swap_cache_add_folio - Add a folio into the swap cache. + * @folio: The folio to be added. + * @entry: The swap entry corresponding to the folio. + * @gfp: gfp_mask for XArray node allocation. + * @shadowp: If a shadow is found, return the shadow. + * + * Context: Caller must ensure @entry is valid and protect the swap device + * with reference count or locks. + * The caller also needs to mark the corresponding swap_map slots with + * SWAP_HAS_CACHE to avoid race or conflict. + * Return: Returns 0 on success, error code otherwise. */ -int add_to_swap_cache(struct folio *folio, swp_entry_t entry, - gfp_t gfp, void **shadowp) +int swap_cache_add_folio(struct folio *folio, swp_entry_t entry, + gfp_t gfp, void **shadowp) { struct address_space *address_space =3D swap_address_space(entry); pgoff_t idx =3D swap_cache_index(entry); @@ -155,12 +172,20 @@ int add_to_swap_cache(struct folio *folio, swp_entry_= t entry, return xas_error(&xas); } =20 -/* - * This must be called only on folios that have - * been verified to be in the swap cache. +/** + * __swap_cache_del_folio - Removes a folio from the swap cache. + * @folio: The folio. + * @entry: The first swap entry that the folio corresponds to. + * @shadow: shadow value to be filled in the swap cache. + * + * Removes a folio from the swap cache and fills a shadow in place. + * This won't put the folio's refcount. The caller has to do that. + * + * Context: Caller must hold the xa_lock, ensure the folio is + * locked and in the swap cache, using the index of @entry. */ -void __delete_from_swap_cache(struct folio *folio, - swp_entry_t entry, void *shadow) +void __swap_cache_del_folio(struct folio *folio, + swp_entry_t entry, void *shadow) { struct address_space *address_space =3D swap_address_space(entry); int i; @@ -186,27 +211,40 @@ void __delete_from_swap_cache(struct folio *folio, __lruvec_stat_mod_folio(folio, NR_SWAPCACHE, -nr); } =20 -/* - * This must be called only on folios that have - * been verified to be in the swap cache and locked. - * It will never put the folio into the free list, - * the caller has a reference on the folio. +/** + * swap_cache_del_folio - Removes a folio from the swap cache. + * @folio: The folio. + * + * Same as __swap_cache_del_folio, but handles lock and refcount. The + * caller must ensure the folio is either clean or has a swap count + * equal to zero, or it may cause data loss. + * + * Context: Caller must ensure the folio is locked and in the swap cache. */ -void delete_from_swap_cache(struct folio *folio) +void swap_cache_del_folio(struct folio *folio) { swp_entry_t entry =3D folio->swap; struct address_space *address_space =3D swap_address_space(entry); =20 xa_lock_irq(&address_space->i_pages); - __delete_from_swap_cache(folio, entry, NULL); + __swap_cache_del_folio(folio, entry, NULL); xa_unlock_irq(&address_space->i_pages); =20 put_swap_folio(folio, entry); folio_ref_sub(folio, folio_nr_pages(folio)); } =20 -void clear_shadow_from_swap_cache(int type, unsigned long begin, - unsigned long end) +/** + * swap_cache_clear_shadow - Clears a set of shadows in the swap cache. + * @type: Indicates the swap device. + * @begin: Beginning offset of the range. + * @end: Ending offset of the range. + * + * Context: Caller must ensure the range is valid and hold a reference to + * the swap device. + */ +void swap_cache_clear_shadow(int type, unsigned long begin, + unsigned long end) { unsigned long curr =3D begin; void *old; @@ -393,7 +431,7 @@ struct folio *__read_swap_cache_async(swp_entry_t entry= , gfp_t gfp_mask, goto put_and_return; =20 /* - * We might race against __delete_from_swap_cache(), and + * We might race against __swap_cache_del_folio(), and * stumble across a swap_map entry whose SWAP_HAS_CACHE * has not yet been cleared. Or race against another * __read_swap_cache_async(), which has set SWAP_HAS_CACHE @@ -412,7 +450,7 @@ struct folio *__read_swap_cache_async(swp_entry_t entry= , gfp_t gfp_mask, goto fail_unlock; =20 /* May fail (-ENOMEM) if XArray node allocation failed. */ - if (add_to_swap_cache(new_folio, entry, gfp_mask & GFP_RECLAIM_MASK, &sha= dow)) + if (swap_cache_add_folio(new_folio, entry, gfp_mask & GFP_RECLAIM_MASK, &= shadow)) goto fail_unlock; =20 memcg1_swapin(entry, 1); diff --git a/mm/swapfile.c b/mm/swapfile.c index 6f7a8c98d14d..51f781c43537 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -267,7 +267,7 @@ static int __try_to_reclaim_swap(struct swap_info_struc= t *si, if (!need_reclaim) goto out_unlock; =20 - delete_from_swap_cache(folio); + swap_cache_del_folio(folio); folio_set_dirty(folio); ret =3D nr_pages; out_unlock: @@ -1124,7 +1124,7 @@ static void swap_range_free(struct swap_info_struct *= si, unsigned long offset, swap_slot_free_notify(si->bdev, offset); offset++; } - clear_shadow_from_swap_cache(si->type, begin, end); + swap_cache_clear_shadow(si->type, begin, end); =20 /* * Make sure that try_to_unuse() observes si->inuse_pages reaching 0 @@ -1289,7 +1289,7 @@ int folio_alloc_swap(struct folio *folio, gfp_t gfp) * TODO: this could cause a theoretical memory reclaim * deadlock in the swap out path. */ - if (add_to_swap_cache(folio, entry, gfp | __GFP_NOMEMALLOC, NULL)) + if (swap_cache_add_folio(folio, entry, gfp | __GFP_NOMEMALLOC, NULL)) goto out_free; =20 return 0; @@ -1759,7 +1759,7 @@ bool folio_free_swap(struct folio *folio) if (folio_swapped(folio)) return false; =20 - delete_from_swap_cache(folio); + swap_cache_del_folio(folio); folio_set_dirty(folio); return true; } diff --git a/mm/vmscan.c b/mm/vmscan.c index ca9e1cd3cd68..c79c6806560b 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -776,7 +776,7 @@ static int __remove_mapping(struct address_space *mappi= ng, struct folio *folio, =20 if (reclaimed && !mapping_exiting(mapping)) shadow =3D workingset_eviction(folio, target_memcg); - __delete_from_swap_cache(folio, swap, shadow); + __swap_cache_del_folio(folio, swap, shadow); memcg1_swapout(folio, swap); xa_unlock_irq(&mapping->i_pages); put_swap_folio(folio, swap); diff --git a/mm/zswap.c b/mm/zswap.c index 63045e3fb1f5..1b1edecde6a7 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1069,7 +1069,7 @@ static int zswap_writeback_entry(struct zswap_entry *= entry, =20 out: if (ret && ret !=3D -EEXIST) { - delete_from_swap_cache(folio); + swap_cache_del_folio(folio); folio_unlock(folio); } folio_put(folio); --=20 2.51.0