From nobody Mon Feb 9 01:45:39 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE11B1C9EC8 for ; Wed, 7 Aug 2024 08:25:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723019137; cv=none; b=sH4yUFxwJoyoZ7DVmO1hwFeHLdKussYNwSta89f0mJx6ByFPF1mffsVbt+4bqMWYJHuqZtu4PeIcxleS4/wjuzaiCg7yYhGSLPt3ZGt6xK/VfdzDpg57maMhelOlFonG0XQ4DOS3W9+JeQMOWzlgVi+uJxCqZ24XUm6NAvco5N4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723019137; c=relaxed/simple; bh=vFHUEqQ1BbRbYvdkss7zznFtXcpaAU4aiHUJIfCTLw0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=t1aHaLktz7E4yOusD7hW2/NBlbewwjeBkMBdWVkHSUZ4V+PYixm48ArCP0WXIjYvmc+dBCklxtjGBN+gL6pUvNxnCk+//EdwroOR2mGv68ibMOoEGR3s5QLXDth+8mg+Dw3rvvML5/IqPXIylIqSAUnpY7+8MNUPwo3OhpKgGkg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=g7l3h44M; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="g7l3h44M" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-70eb0ae23e4so1222194b3a.0 for ; Wed, 07 Aug 2024 01:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723019134; x=1723623934; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UdPKEnDMiO3Rq0OJoiXWd7L+WrOyBVW1Yp9rdNsv9MY=; b=g7l3h44Mk/3nWFs49xbzCv+fZ31gKdW0BnYSgruxsGZWr21q3QbTXNO+3RnRzSJe/L f4kpqKn4zzlUAY+NfJFM4EnEEccNmq5wAIwIJL0W4ZR5De2jniWLuBxtMez7FM0ZjqnF CkOQK0RvV+FoWVkQCWWlhTVQR1Po3OJTJu7i9KvcQkRUCOh+zVRHzhPbK23vaQ48G8Vz 8Fh5xR5SLsaznewdHXxfnYH2nbhRP+C7CEPszRjNZfOfwFc7vqVxtVjmwwdpmJ1naXYH 2am3sX994ef2RzBAh1f/uIXBRhDH3mge/UZMdyliU5DBoeh2NvydV5OLS0qdn3PuShrc wHpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723019134; x=1723623934; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UdPKEnDMiO3Rq0OJoiXWd7L+WrOyBVW1Yp9rdNsv9MY=; b=vEH3HlaaVyIO6BZ9xW6snhlDQVYSovxZcZgJoQkZslE4C+sXlmj2sdZGGQbsqN0QPS SoO/DsAPx1tsFlUIL7eD6/J//cpgebxEFXfiR7UZUyqiW8P6LpckiqeFlXn1siJes/x6 XoBRVeLrLDB5sLuSM0++cjZJzYiv78KEIjop6X9IG8iBnIZ9OAPMHDl63eVQ80F84t5g 83IhG+vbvs1o06Gq+k176Dg0c3fkA0r/P0J7VaqhoUE5z8e4XmpAha/fg2Kzfj66RzMl cdO0OQCnpSg9h1HQOUrDYDwRUEl5bO2S4R+AJyWor/4uUzQy7y0ikkXHFIkmgPHAZa0m 2x1Q== X-Forwarded-Encrypted: i=1; AJvYcCWBgFOJf4d42NBi2WGSVNrqJ1aSzzmbtn/gBWzCPhnzLDXoEreTCAeLKWzOTUOD5kJUbVNEz7kcbYc79pEIZrrivfm/4XTUb/mklH6W X-Gm-Message-State: AOJu0YwtCKm470qf7I4oFlUPfF7YZT/5amOD/lEHospV7Dqr3XApZpIz UTFazPbhwaMG9VHyXPq7CB/4+YLXu6/mgGhaBLoqRiHCy5tr1qLb X-Google-Smtp-Source: AGHT+IEvoNTIDvwaehPfHv3nbCH9Vy94+618VYT5kTxzOYUtRrps/4mu4t31x/NmGjn8hlWfndnoXA== X-Received: by 2002:a05:6a00:cc4:b0:70b:1b51:b8af with SMTP id d2e1a72fcca58-7106d02fb2bmr18987093b3a.19.1723019133756; Wed, 07 Aug 2024 01:25:33 -0700 (PDT) Received: from barry-desktop.hub ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7106ed0d457sm7911763b3a.156.2024.08.07.01.25.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Aug 2024 01:25:33 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: justinjiang@vivo.com, chrisl@kernel.org, david@redhat.com, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, v-songbaohua@oppo.com, ying.huang@intel.com Subject: [PATCH v2 1/2] mm: rename instances of swap_info_struct to meaningful 'si' Date: Wed, 7 Aug 2024 20:25:07 +1200 Message-Id: <20240807082508.358322-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240807082508.358322-1-21cnbao@gmail.com> References: <20240807082508.358322-1-21cnbao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Barry Song "P" means "pointer to something", rename it to a more meaningful identifier - "si". We also have a case with the name "sis", rename it to "si" as well. Signed-off-by: Barry Song Acked-by: David Hildenbrand --- mm/swapfile.c | 334 +++++++++++++++++++++++++------------------------- 1 file changed, 167 insertions(+), 167 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index ea023fc25d08..35cb58373493 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -532,7 +532,7 @@ static void free_cluster(struct swap_info_struct *si, s= truct swap_cluster_info * * added to free cluster list and its usage counter will be increased by 1. * Only used for initialization. */ -static void inc_cluster_info_page(struct swap_info_struct *p, +static void inc_cluster_info_page(struct swap_info_struct *si, struct swap_cluster_info *cluster_info, unsigned long page_nr) { unsigned long idx =3D page_nr / SWAPFILE_CLUSTER; @@ -553,28 +553,28 @@ static void inc_cluster_info_page(struct swap_info_st= ruct *p, * which means no page in the cluster is in use, we can optionally discard * the cluster and add it to free cluster list. */ -static void dec_cluster_info_page(struct swap_info_struct *p, +static void dec_cluster_info_page(struct swap_info_struct *si, struct swap_cluster_info *ci, int nr_pages) { - if (!p->cluster_info) + if (!si->cluster_info) return; =20 VM_BUG_ON(ci->count < nr_pages); VM_BUG_ON(cluster_is_free(ci)); - lockdep_assert_held(&p->lock); + lockdep_assert_held(&si->lock); lockdep_assert_held(&ci->lock); ci->count -=3D nr_pages; =20 if (!ci->count) { - free_cluster(p, ci); + free_cluster(si, ci); return; } =20 if (!(ci->flags & CLUSTER_FLAG_NONFULL)) { VM_BUG_ON(ci->flags & CLUSTER_FLAG_FREE); if (ci->flags & CLUSTER_FLAG_FRAG) - p->frag_cluster_nr[ci->order]--; - list_move_tail(&ci->list, &p->nonfull_clusters[ci->order]); + si->frag_cluster_nr[ci->order]--; + list_move_tail(&ci->list, &si->nonfull_clusters[ci->order]); ci->flags =3D CLUSTER_FLAG_NONFULL; } } @@ -872,19 +872,19 @@ static unsigned long cluster_alloc_swap_entry(struct = swap_info_struct *si, int o return found; } =20 -static void __del_from_avail_list(struct swap_info_struct *p) +static void __del_from_avail_list(struct swap_info_struct *si) { int nid; =20 - assert_spin_locked(&p->lock); + assert_spin_locked(&si->lock); for_each_node(nid) - plist_del(&p->avail_lists[nid], &swap_avail_heads[nid]); + plist_del(&si->avail_lists[nid], &swap_avail_heads[nid]); } =20 -static void del_from_avail_list(struct swap_info_struct *p) +static void del_from_avail_list(struct swap_info_struct *si) { spin_lock(&swap_avail_lock); - __del_from_avail_list(p); + __del_from_avail_list(si); spin_unlock(&swap_avail_lock); } =20 @@ -905,13 +905,13 @@ static void swap_range_alloc(struct swap_info_struct = *si, unsigned long offset, } } =20 -static void add_to_avail_list(struct swap_info_struct *p) +static void add_to_avail_list(struct swap_info_struct *si) { int nid; =20 spin_lock(&swap_avail_lock); for_each_node(nid) - plist_add(&p->avail_lists[nid], &swap_avail_heads[nid]); + plist_add(&si->avail_lists[nid], &swap_avail_heads[nid]); spin_unlock(&swap_avail_lock); } =20 @@ -1291,22 +1291,22 @@ int get_swap_pages(int n_goal, swp_entry_t swp_entr= ies[], int entry_order) =20 static struct swap_info_struct *_swap_info_get(swp_entry_t entry) { - struct swap_info_struct *p; + struct swap_info_struct *si; unsigned long offset; =20 if (!entry.val) goto out; - p =3D swp_swap_info(entry); - if (!p) + si =3D swp_swap_info(entry); + if (!si) goto bad_nofile; - if (data_race(!(p->flags & SWP_USED))) + if (data_race(!(si->flags & SWP_USED))) goto bad_device; offset =3D swp_offset(entry); - if (offset >=3D p->max) + if (offset >=3D si->max) goto bad_offset; - if (data_race(!p->swap_map[swp_offset(entry)])) + if (data_race(!si->swap_map[swp_offset(entry)])) goto bad_free; - return p; + return si; =20 bad_free: pr_err("%s: %s%08lx\n", __func__, Unused_offset, entry.val); @@ -1339,14 +1339,14 @@ static struct swap_info_struct *swap_info_get_cont(= swp_entry_t entry, return p; } =20 -static unsigned char __swap_entry_free_locked(struct swap_info_struct *p, +static unsigned char __swap_entry_free_locked(struct swap_info_struct *si, unsigned long offset, unsigned char usage) { unsigned char count; unsigned char has_cache; =20 - count =3D p->swap_map[offset]; + count =3D si->swap_map[offset]; =20 has_cache =3D count & SWAP_HAS_CACHE; count &=3D ~SWAP_HAS_CACHE; @@ -1362,7 +1362,7 @@ static unsigned char __swap_entry_free_locked(struct = swap_info_struct *p, count =3D 0; } else if ((count & ~COUNT_CONTINUED) <=3D SWAP_MAP_MAX) { if (count =3D=3D COUNT_CONTINUED) { - if (swap_count_continued(p, offset, count)) + if (swap_count_continued(si, offset, count)) count =3D SWAP_MAP_MAX | COUNT_CONTINUED; else count =3D SWAP_MAP_MAX; @@ -1372,9 +1372,9 @@ static unsigned char __swap_entry_free_locked(struct = swap_info_struct *p, =20 usage =3D count | has_cache; if (usage) - WRITE_ONCE(p->swap_map[offset], usage); + WRITE_ONCE(si->swap_map[offset], usage); else - WRITE_ONCE(p->swap_map[offset], SWAP_HAS_CACHE); + WRITE_ONCE(si->swap_map[offset], SWAP_HAS_CACHE); =20 return usage; } @@ -1453,16 +1453,16 @@ struct swap_info_struct *get_swap_device(swp_entry_= t entry) return NULL; } =20 -static unsigned char __swap_entry_free(struct swap_info_struct *p, +static unsigned char __swap_entry_free(struct swap_info_struct *si, swp_entry_t entry) { struct swap_cluster_info *ci; unsigned long offset =3D swp_offset(entry); unsigned char usage; =20 - ci =3D lock_cluster_or_swap_info(p, offset); - usage =3D __swap_entry_free_locked(p, offset, 1); - unlock_cluster_or_swap_info(p, ci); + ci =3D lock_cluster_or_swap_info(si, offset); + usage =3D __swap_entry_free_locked(si, offset, 1); + unlock_cluster_or_swap_info(si, ci); if (!usage) free_swap_slot(entry); =20 @@ -1473,27 +1473,27 @@ static unsigned char __swap_entry_free(struct swap_= info_struct *p, * Drop the last HAS_CACHE flag of swap entries, caller have to * ensure all entries belong to the same cgroup. */ -static void swap_entry_range_free(struct swap_info_struct *p, swp_entry_t = entry, +static void swap_entry_range_free(struct swap_info_struct *si, swp_entry_t= entry, unsigned int nr_pages) { unsigned long offset =3D swp_offset(entry); - unsigned char *map =3D p->swap_map + offset; + unsigned char *map =3D si->swap_map + offset; unsigned char *map_end =3D map + nr_pages; struct swap_cluster_info *ci; =20 - ci =3D lock_cluster(p, offset); + ci =3D lock_cluster(si, offset); do { VM_BUG_ON(*map !=3D SWAP_HAS_CACHE); *map =3D 0; } while (++map < map_end); - dec_cluster_info_page(p, ci, nr_pages); + dec_cluster_info_page(si, ci, nr_pages); unlock_cluster(ci); =20 mem_cgroup_uncharge_swap(entry, nr_pages); - swap_range_free(p, offset, nr_pages); + swap_range_free(si, offset, nr_pages); } =20 -static void cluster_swap_free_nr(struct swap_info_struct *sis, +static void cluster_swap_free_nr(struct swap_info_struct *si, unsigned long offset, int nr_pages, unsigned char usage) { @@ -1501,26 +1501,26 @@ static void cluster_swap_free_nr(struct swap_info_s= truct *sis, DECLARE_BITMAP(to_free, BITS_PER_LONG) =3D { 0 }; int i, nr; =20 - ci =3D lock_cluster_or_swap_info(sis, offset); + ci =3D lock_cluster_or_swap_info(si, offset); while (nr_pages) { nr =3D min(BITS_PER_LONG, nr_pages); for (i =3D 0; i < nr; i++) { - if (!__swap_entry_free_locked(sis, offset + i, usage)) + if (!__swap_entry_free_locked(si, offset + i, usage)) bitmap_set(to_free, i, 1); } if (!bitmap_empty(to_free, BITS_PER_LONG)) { - unlock_cluster_or_swap_info(sis, ci); + unlock_cluster_or_swap_info(si, ci); for_each_set_bit(i, to_free, BITS_PER_LONG) - free_swap_slot(swp_entry(sis->type, offset + i)); + free_swap_slot(swp_entry(si->type, offset + i)); if (nr =3D=3D nr_pages) return; bitmap_clear(to_free, 0, BITS_PER_LONG); - ci =3D lock_cluster_or_swap_info(sis, offset); + ci =3D lock_cluster_or_swap_info(si, offset); } offset +=3D nr; nr_pages -=3D nr; } - unlock_cluster_or_swap_info(sis, ci); + unlock_cluster_or_swap_info(si, ci); } =20 /* @@ -1646,28 +1646,28 @@ int swap_swapcount(struct swap_info_struct *si, swp= _entry_t entry) int swp_swapcount(swp_entry_t entry) { int count, tmp_count, n; - struct swap_info_struct *p; + struct swap_info_struct *si; struct swap_cluster_info *ci; struct page *page; pgoff_t offset; unsigned char *map; =20 - p =3D _swap_info_get(entry); - if (!p) + si =3D _swap_info_get(entry); + if (!si) return 0; =20 offset =3D swp_offset(entry); =20 - ci =3D lock_cluster_or_swap_info(p, offset); + ci =3D lock_cluster_or_swap_info(si, offset); =20 - count =3D swap_count(p->swap_map[offset]); + count =3D swap_count(si->swap_map[offset]); if (!(count & COUNT_CONTINUED)) goto out; =20 count &=3D ~COUNT_CONTINUED; n =3D SWAP_MAP_MAX + 1; =20 - page =3D vmalloc_to_page(p->swap_map + offset); + page =3D vmalloc_to_page(si->swap_map + offset); offset &=3D ~PAGE_MASK; VM_BUG_ON(page_private(page) !=3D SWP_CONTINUED); =20 @@ -1681,7 +1681,7 @@ int swp_swapcount(swp_entry_t entry) n *=3D (SWAP_CONT_MAX + 1); } while (tmp_count & COUNT_CONTINUED); out: - unlock_cluster_or_swap_info(p, ci); + unlock_cluster_or_swap_info(si, ci); return count; } =20 @@ -2542,52 +2542,52 @@ static int setup_swap_extents(struct swap_info_stru= ct *sis, sector_t *span) return generic_swapfile_activate(sis, swap_file, span); } =20 -static int swap_node(struct swap_info_struct *p) +static int swap_node(struct swap_info_struct *si) { struct block_device *bdev; =20 - if (p->bdev) - bdev =3D p->bdev; + if (si->bdev) + bdev =3D si->bdev; else - bdev =3D p->swap_file->f_inode->i_sb->s_bdev; + bdev =3D si->swap_file->f_inode->i_sb->s_bdev; =20 return bdev ? bdev->bd_disk->node_id : NUMA_NO_NODE; } =20 -static void setup_swap_info(struct swap_info_struct *p, int prio, +static void setup_swap_info(struct swap_info_struct *si, int prio, unsigned char *swap_map, struct swap_cluster_info *cluster_info) { int i; =20 if (prio >=3D 0) - p->prio =3D prio; + si->prio =3D prio; else - p->prio =3D --least_priority; + si->prio =3D --least_priority; /* * the plist prio is negated because plist ordering is * low-to-high, while swap ordering is high-to-low */ - p->list.prio =3D -p->prio; + si->list.prio =3D -si->prio; for_each_node(i) { - if (p->prio >=3D 0) - p->avail_lists[i].prio =3D -p->prio; + if (si->prio >=3D 0) + si->avail_lists[i].prio =3D -si->prio; else { - if (swap_node(p) =3D=3D i) - p->avail_lists[i].prio =3D 1; + if (swap_node(si) =3D=3D i) + si->avail_lists[i].prio =3D 1; else - p->avail_lists[i].prio =3D -p->prio; + si->avail_lists[i].prio =3D -si->prio; } } - p->swap_map =3D swap_map; - p->cluster_info =3D cluster_info; + si->swap_map =3D swap_map; + si->cluster_info =3D cluster_info; } =20 -static void _enable_swap_info(struct swap_info_struct *p) +static void _enable_swap_info(struct swap_info_struct *si) { - p->flags |=3D SWP_WRITEOK; - atomic_long_add(p->pages, &nr_swap_pages); - total_swap_pages +=3D p->pages; + si->flags |=3D SWP_WRITEOK; + atomic_long_add(si->pages, &nr_swap_pages); + total_swap_pages +=3D si->pages; =20 assert_spin_locked(&swap_lock); /* @@ -2600,40 +2600,40 @@ static void _enable_swap_info(struct swap_info_stru= ct *p) * which allocates swap pages from the highest available priority * swap_info_struct. */ - plist_add(&p->list, &swap_active_head); + plist_add(&si->list, &swap_active_head); =20 /* add to available list iff swap device is not full */ - if (p->highest_bit) - add_to_avail_list(p); + if (si->highest_bit) + add_to_avail_list(si); } =20 -static void enable_swap_info(struct swap_info_struct *p, int prio, +static void enable_swap_info(struct swap_info_struct *si, int prio, unsigned char *swap_map, struct swap_cluster_info *cluster_info) { spin_lock(&swap_lock); - spin_lock(&p->lock); - setup_swap_info(p, prio, swap_map, cluster_info); - spin_unlock(&p->lock); + spin_lock(&si->lock); + setup_swap_info(si, prio, swap_map, cluster_info); + spin_unlock(&si->lock); spin_unlock(&swap_lock); /* * Finished initializing swap device, now it's safe to reference it. */ - percpu_ref_resurrect(&p->users); + percpu_ref_resurrect(&si->users); spin_lock(&swap_lock); - spin_lock(&p->lock); - _enable_swap_info(p); - spin_unlock(&p->lock); + spin_lock(&si->lock); + _enable_swap_info(si); + spin_unlock(&si->lock); spin_unlock(&swap_lock); } =20 -static void reinsert_swap_info(struct swap_info_struct *p) +static void reinsert_swap_info(struct swap_info_struct *si) { spin_lock(&swap_lock); - spin_lock(&p->lock); - setup_swap_info(p, p->prio, p->swap_map, p->cluster_info); - _enable_swap_info(p); - spin_unlock(&p->lock); + spin_lock(&si->lock); + setup_swap_info(si, si->prio, si->swap_map, si->cluster_info); + _enable_swap_info(si); + spin_unlock(&si->lock); spin_unlock(&swap_lock); } =20 @@ -3019,20 +3019,20 @@ static struct swap_info_struct *alloc_swap_info(voi= d) return p; } =20 -static int claim_swapfile(struct swap_info_struct *p, struct inode *inode) +static int claim_swapfile(struct swap_info_struct *si, struct inode *inode) { if (S_ISBLK(inode->i_mode)) { - p->bdev =3D I_BDEV(inode); + si->bdev =3D I_BDEV(inode); /* * Zoned block devices contain zones that have a sequential * write only restriction. Hence zoned block devices are not * suitable for swapping. Disallow them here. */ - if (bdev_is_zoned(p->bdev)) + if (bdev_is_zoned(si->bdev)) return -EINVAL; - p->flags |=3D SWP_BLKDEV; + si->flags |=3D SWP_BLKDEV; } else if (S_ISREG(inode->i_mode)) { - p->bdev =3D inode->i_sb->s_bdev; + si->bdev =3D inode->i_sb->s_bdev; } =20 return 0; @@ -3067,7 +3067,7 @@ __weak unsigned long arch_max_swapfile_size(void) return generic_max_swapfile_size(); } =20 -static unsigned long read_swap_header(struct swap_info_struct *p, +static unsigned long read_swap_header(struct swap_info_struct *si, union swap_header *swap_header, struct inode *inode) { @@ -3098,9 +3098,9 @@ static unsigned long read_swap_header(struct swap_inf= o_struct *p, return 0; } =20 - p->lowest_bit =3D 1; - p->cluster_next =3D 1; - p->cluster_nr =3D 0; + si->lowest_bit =3D 1; + si->cluster_next =3D 1; + si->cluster_nr =3D 0; =20 maxpages =3D swapfile_maximum_size; last_page =3D swap_header->info.last_page; @@ -3118,7 +3118,7 @@ static unsigned long read_swap_header(struct swap_inf= o_struct *p, if ((unsigned int)maxpages =3D=3D 0) maxpages =3D UINT_MAX; } - p->highest_bit =3D maxpages - 1; + si->highest_bit =3D maxpages - 1; =20 if (!maxpages) return 0; @@ -3142,7 +3142,7 @@ static unsigned long read_swap_header(struct swap_inf= o_struct *p, #define SWAP_CLUSTER_COLS \ max_t(unsigned int, SWAP_CLUSTER_INFO_COLS, SWAP_CLUSTER_SPACE_COLS) =20 -static int setup_swap_map_and_extents(struct swap_info_struct *p, +static int setup_swap_map_and_extents(struct swap_info_struct *si, union swap_header *swap_header, unsigned char *swap_map, struct swap_cluster_info *cluster_info, @@ -3153,19 +3153,19 @@ static int setup_swap_map_and_extents(struct swap_i= nfo_struct *p, unsigned int nr_good_pages; int nr_extents; unsigned long nr_clusters =3D DIV_ROUND_UP(maxpages, SWAPFILE_CLUSTER); - unsigned long col =3D p->cluster_next / SWAPFILE_CLUSTER % SWAP_CLUSTER_C= OLS; + unsigned long col =3D si->cluster_next / SWAPFILE_CLUSTER % SWAP_CLUSTER_= COLS; unsigned long i, idx; =20 nr_good_pages =3D maxpages - 1; /* omit header page */ =20 - INIT_LIST_HEAD(&p->free_clusters); - INIT_LIST_HEAD(&p->full_clusters); - INIT_LIST_HEAD(&p->discard_clusters); + INIT_LIST_HEAD(&si->free_clusters); + INIT_LIST_HEAD(&si->full_clusters); + INIT_LIST_HEAD(&si->discard_clusters); =20 for (i =3D 0; i < SWAP_NR_ORDERS; i++) { - INIT_LIST_HEAD(&p->nonfull_clusters[i]); - INIT_LIST_HEAD(&p->frag_clusters[i]); - p->frag_cluster_nr[i] =3D 0; + INIT_LIST_HEAD(&si->nonfull_clusters[i]); + INIT_LIST_HEAD(&si->frag_clusters[i]); + si->frag_cluster_nr[i] =3D 0; } =20 for (i =3D 0; i < swap_header->info.nr_badpages; i++) { @@ -3179,13 +3179,13 @@ static int setup_swap_map_and_extents(struct swap_i= nfo_struct *p, * Haven't marked the cluster free yet, no list * operation involved */ - inc_cluster_info_page(p, cluster_info, page_nr); + inc_cluster_info_page(si, cluster_info, page_nr); } } =20 /* Haven't marked the cluster free yet, no list operation involved */ for (i =3D maxpages; i < round_up(maxpages, SWAPFILE_CLUSTER); i++) - inc_cluster_info_page(p, cluster_info, i); + inc_cluster_info_page(si, cluster_info, i); =20 if (nr_good_pages) { swap_map[0] =3D SWAP_MAP_BAD; @@ -3193,13 +3193,13 @@ static int setup_swap_map_and_extents(struct swap_i= nfo_struct *p, * Not mark the cluster free yet, no list * operation involved */ - inc_cluster_info_page(p, cluster_info, 0); - p->max =3D maxpages; - p->pages =3D nr_good_pages; - nr_extents =3D setup_swap_extents(p, span); + inc_cluster_info_page(si, cluster_info, 0); + si->max =3D maxpages; + si->pages =3D nr_good_pages; + nr_extents =3D setup_swap_extents(si, span); if (nr_extents < 0) return nr_extents; - nr_good_pages =3D p->pages; + nr_good_pages =3D si->pages; } if (!nr_good_pages) { pr_warn("Empty swap-file\n"); @@ -3223,11 +3223,11 @@ static int setup_swap_map_and_extents(struct swap_i= nfo_struct *p, continue; if (ci->count) { ci->flags =3D CLUSTER_FLAG_NONFULL; - list_add_tail(&ci->list, &p->nonfull_clusters[0]); + list_add_tail(&ci->list, &si->nonfull_clusters[0]); continue; } ci->flags =3D CLUSTER_FLAG_FREE; - list_add_tail(&ci->list, &p->free_clusters); + list_add_tail(&ci->list, &si->free_clusters); } } return nr_extents; @@ -3235,7 +3235,7 @@ static int setup_swap_map_and_extents(struct swap_inf= o_struct *p, =20 SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags) { - struct swap_info_struct *p; + struct swap_info_struct *si; struct filename *name; struct file *swap_file =3D NULL; struct address_space *mapping; @@ -3261,11 +3261,11 @@ SYSCALL_DEFINE2(swapon, const char __user *, specia= lfile, int, swap_flags) if (!swap_avail_heads) return -ENOMEM; =20 - p =3D alloc_swap_info(); - if (IS_ERR(p)) - return PTR_ERR(p); + si =3D alloc_swap_info(); + if (IS_ERR(si)) + return PTR_ERR(si); =20 - INIT_WORK(&p->discard_work, swap_discard_work); + INIT_WORK(&si->discard_work, swap_discard_work); =20 name =3D getname(specialfile); if (IS_ERR(name)) { @@ -3280,12 +3280,12 @@ SYSCALL_DEFINE2(swapon, const char __user *, specia= lfile, int, swap_flags) goto bad_swap; } =20 - p->swap_file =3D swap_file; + si->swap_file =3D swap_file; mapping =3D swap_file->f_mapping; dentry =3D swap_file->f_path.dentry; inode =3D mapping->host; =20 - error =3D claim_swapfile(p, inode); + error =3D claim_swapfile(si, inode); if (unlikely(error)) goto bad_swap; =20 @@ -3313,7 +3313,7 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialf= ile, int, swap_flags) } swap_header =3D kmap(page); =20 - maxpages =3D read_swap_header(p, swap_header, inode); + maxpages =3D read_swap_header(si, swap_header, inode); if (unlikely(!maxpages)) { error =3D -EINVAL; goto bad_swap_unlock_inode; @@ -3326,19 +3326,19 @@ SYSCALL_DEFINE2(swapon, const char __user *, specia= lfile, int, swap_flags) goto bad_swap_unlock_inode; } =20 - if (p->bdev && bdev_stable_writes(p->bdev)) - p->flags |=3D SWP_STABLE_WRITES; + if (si->bdev && bdev_stable_writes(si->bdev)) + si->flags |=3D SWP_STABLE_WRITES; =20 - if (p->bdev && bdev_synchronous(p->bdev)) - p->flags |=3D SWP_SYNCHRONOUS_IO; + if (si->bdev && bdev_synchronous(si->bdev)) + si->flags |=3D SWP_SYNCHRONOUS_IO; =20 - if (p->bdev && bdev_nonrot(p->bdev)) { + if (si->bdev && bdev_nonrot(si->bdev)) { int cpu, i; unsigned long ci, nr_cluster; =20 - p->flags |=3D SWP_SOLIDSTATE; - p->cluster_next_cpu =3D alloc_percpu(unsigned int); - if (!p->cluster_next_cpu) { + si->flags |=3D SWP_SOLIDSTATE; + si->cluster_next_cpu =3D alloc_percpu(unsigned int); + if (!si->cluster_next_cpu) { error =3D -ENOMEM; goto bad_swap_unlock_inode; } @@ -3347,8 +3347,8 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialf= ile, int, swap_flags) * SSD */ for_each_possible_cpu(cpu) { - per_cpu(*p->cluster_next_cpu, cpu) =3D - get_random_u32_inclusive(1, p->highest_bit); + per_cpu(*si->cluster_next_cpu, cpu) =3D + get_random_u32_inclusive(1, si->highest_bit); } nr_cluster =3D DIV_ROUND_UP(maxpages, SWAPFILE_CLUSTER); =20 @@ -3362,15 +3362,15 @@ SYSCALL_DEFINE2(swapon, const char __user *, specia= lfile, int, swap_flags) for (ci =3D 0; ci < nr_cluster; ci++) spin_lock_init(&((cluster_info + ci)->lock)); =20 - p->percpu_cluster =3D alloc_percpu(struct percpu_cluster); - if (!p->percpu_cluster) { + si->percpu_cluster =3D alloc_percpu(struct percpu_cluster); + if (!si->percpu_cluster) { error =3D -ENOMEM; goto bad_swap_unlock_inode; } for_each_possible_cpu(cpu) { struct percpu_cluster *cluster; =20 - cluster =3D per_cpu_ptr(p->percpu_cluster, cpu); + cluster =3D per_cpu_ptr(si->percpu_cluster, cpu); for (i =3D 0; i < SWAP_NR_ORDERS; i++) cluster->next[i] =3D SWAP_NEXT_INVALID; } @@ -3379,11 +3379,11 @@ SYSCALL_DEFINE2(swapon, const char __user *, specia= lfile, int, swap_flags) inced_nr_rotate_swap =3D true; } =20 - error =3D swap_cgroup_swapon(p->type, maxpages); + error =3D swap_cgroup_swapon(si->type, maxpages); if (error) goto bad_swap_unlock_inode; =20 - nr_extents =3D setup_swap_map_and_extents(p, swap_header, swap_map, + nr_extents =3D setup_swap_map_and_extents(si, swap_header, swap_map, cluster_info, maxpages, &span); if (unlikely(nr_extents < 0)) { error =3D nr_extents; @@ -3391,14 +3391,14 @@ SYSCALL_DEFINE2(swapon, const char __user *, specia= lfile, int, swap_flags) } =20 if ((swap_flags & SWAP_FLAG_DISCARD) && - p->bdev && bdev_max_discard_sectors(p->bdev)) { + si->bdev && bdev_max_discard_sectors(si->bdev)) { /* * When discard is enabled for swap with no particular * policy flagged, we set all swap discard flags here in * order to sustain backward compatibility with older * swapon(8) releases. */ - p->flags |=3D (SWP_DISCARDABLE | SWP_AREA_DISCARD | + si->flags |=3D (SWP_DISCARDABLE | SWP_AREA_DISCARD | SWP_PAGE_DISCARD); =20 /* @@ -3408,24 +3408,24 @@ SYSCALL_DEFINE2(swapon, const char __user *, specia= lfile, int, swap_flags) * Now it's time to adjust the p->flags accordingly. */ if (swap_flags & SWAP_FLAG_DISCARD_ONCE) - p->flags &=3D ~SWP_PAGE_DISCARD; + si->flags &=3D ~SWP_PAGE_DISCARD; else if (swap_flags & SWAP_FLAG_DISCARD_PAGES) - p->flags &=3D ~SWP_AREA_DISCARD; + si->flags &=3D ~SWP_AREA_DISCARD; =20 /* issue a swapon-time discard if it's still required */ - if (p->flags & SWP_AREA_DISCARD) { - int err =3D discard_swap(p); + if (si->flags & SWP_AREA_DISCARD) { + int err =3D discard_swap(si); if (unlikely(err)) pr_err("swapon: discard_swap(%p): %d\n", - p, err); + si, err); } } =20 - error =3D init_swap_address_space(p->type, maxpages); + error =3D init_swap_address_space(si->type, maxpages); if (error) goto bad_swap_unlock_inode; =20 - error =3D zswap_swapon(p->type, maxpages); + error =3D zswap_swapon(si->type, maxpages); if (error) goto free_swap_address_space; =20 @@ -3445,15 +3445,15 @@ SYSCALL_DEFINE2(swapon, const char __user *, specia= lfile, int, swap_flags) if (swap_flags & SWAP_FLAG_PREFER) prio =3D (swap_flags & SWAP_FLAG_PRIO_MASK) >> SWAP_FLAG_PRIO_SHIFT; - enable_swap_info(p, prio, swap_map, cluster_info); + enable_swap_info(si, prio, swap_map, cluster_info); =20 pr_info("Adding %uk swap on %s. Priority:%d extents:%d across:%lluk %s%s= %s%s\n", - K(p->pages), name->name, p->prio, nr_extents, + K(si->pages), name->name, si->prio, nr_extents, K((unsigned long long)span), - (p->flags & SWP_SOLIDSTATE) ? "SS" : "", - (p->flags & SWP_DISCARDABLE) ? "D" : "", - (p->flags & SWP_AREA_DISCARD) ? "s" : "", - (p->flags & SWP_PAGE_DISCARD) ? "c" : ""); + (si->flags & SWP_SOLIDSTATE) ? "SS" : "", + (si->flags & SWP_DISCARDABLE) ? "D" : "", + (si->flags & SWP_AREA_DISCARD) ? "s" : "", + (si->flags & SWP_PAGE_DISCARD) ? "c" : ""); =20 mutex_unlock(&swapon_mutex); atomic_inc(&proc_poll_event); @@ -3462,22 +3462,22 @@ SYSCALL_DEFINE2(swapon, const char __user *, specia= lfile, int, swap_flags) error =3D 0; goto out; free_swap_zswap: - zswap_swapoff(p->type); + zswap_swapoff(si->type); free_swap_address_space: - exit_swap_address_space(p->type); + exit_swap_address_space(si->type); bad_swap_unlock_inode: inode_unlock(inode); bad_swap: - free_percpu(p->percpu_cluster); - p->percpu_cluster =3D NULL; - free_percpu(p->cluster_next_cpu); - p->cluster_next_cpu =3D NULL; + free_percpu(si->percpu_cluster); + si->percpu_cluster =3D NULL; + free_percpu(si->cluster_next_cpu); + si->cluster_next_cpu =3D NULL; inode =3D NULL; - destroy_swap_extents(p); - swap_cgroup_swapoff(p->type); + destroy_swap_extents(si); + swap_cgroup_swapoff(si->type); spin_lock(&swap_lock); - p->swap_file =3D NULL; - p->flags =3D 0; + si->swap_file =3D NULL; + si->flags =3D 0; spin_unlock(&swap_lock); vfree(swap_map); kvfree(cluster_info); @@ -3529,23 +3529,23 @@ void si_swapinfo(struct sysinfo *val) */ static int __swap_duplicate(swp_entry_t entry, unsigned char usage, int nr) { - struct swap_info_struct *p; + struct swap_info_struct *si; struct swap_cluster_info *ci; unsigned long offset; unsigned char count; unsigned char has_cache; int err, i; =20 - p =3D swp_swap_info(entry); + si =3D swp_swap_info(entry); =20 offset =3D swp_offset(entry); VM_WARN_ON(nr > SWAPFILE_CLUSTER - offset % SWAPFILE_CLUSTER); VM_WARN_ON(usage =3D=3D 1 && nr > 1); - ci =3D lock_cluster_or_swap_info(p, offset); + ci =3D lock_cluster_or_swap_info(si, offset); =20 err =3D 0; for (i =3D 0; i < nr; i++) { - count =3D p->swap_map[offset + i]; + count =3D si->swap_map[offset + i]; =20 /* * swapin_readahead() doesn't check if a swap entry is valid, so the @@ -3573,7 +3573,7 @@ static int __swap_duplicate(swp_entry_t entry, unsign= ed char usage, int nr) } =20 for (i =3D 0; i < nr; i++) { - count =3D p->swap_map[offset + i]; + count =3D si->swap_map[offset + i]; has_cache =3D count & SWAP_HAS_CACHE; count &=3D ~SWAP_HAS_CACHE; =20 @@ -3581,7 +3581,7 @@ static int __swap_duplicate(swp_entry_t entry, unsign= ed char usage, int nr) has_cache =3D SWAP_HAS_CACHE; else if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) count +=3D usage; - else if (swap_count_continued(p, offset + i, count)) + else if (swap_count_continued(si, offset + i, count)) count =3D COUNT_CONTINUED; else { /* @@ -3592,11 +3592,11 @@ static int __swap_duplicate(swp_entry_t entry, unsi= gned char usage, int nr) goto unlock_out; } =20 - WRITE_ONCE(p->swap_map[offset + i], count | has_cache); + WRITE_ONCE(si->swap_map[offset + i], count | has_cache); } =20 unlock_out: - unlock_cluster_or_swap_info(p, ci); + unlock_cluster_or_swap_info(si, ci); return err; } =20 --=20 2.34.1 From nobody Mon Feb 9 01:45:39 2026 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9366C1CB30E for ; Wed, 7 Aug 2024 08:25:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723019141; cv=none; b=bWGL6bPe4ERzkHa4QRpoQDBM30pg04pc+bJ5WUevoYKmnZ25mJR31CmIUO7HYQZMetcyXrfVuwrJVaaVu5r+zPpmVBWk86rZBJKYXbfip53g2q3SYmAKqROxKeJjDCmbrODJOFWydpYaADReWYluYy4HrQfCh/vvA3FpMsurFqQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723019141; c=relaxed/simple; bh=TE0dMbcGCouZl9eTik1kK4YJtAM4y+zty2arx2p8O8k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DUF0yLAknkaBBczpIXND5xMfd+en4n8SQZQwBnj/tawpBeTYf7hhmE8nyI+iBgS2guiN11HcNjX20/F4FeRcJZE5XuL5BooA6dnJJW89xmvPwCm3gZWME4DHlZtDabjO1rzs7Kl2t1Pawp0jjIfP83d0jAVdYR+MEa0bwDfUD/o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=C3WN5WbT; arc=none smtp.client-ip=209.85.215.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="C3WN5WbT" Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-71871d5e087so918967a12.1 for ; Wed, 07 Aug 2024 01:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723019139; x=1723623939; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BlFQkb2cMvN1c1QVMf0JMKTkCt490ubd3m2R1WYvp/E=; b=C3WN5WbTXyB0YRM1ZeBstPG+BA5bQ8YWOUmRxWxteylUEMaEUVxJ0a0+XmyBhead9k Yj6/08i3N+lkSdXa9q0ybsLhgwFxIyM9oUqbargZ6E4TGTlE7QTALGM6k0zU2rpvlE/a Fp1X3ySjMEx21ErdQB7nXowEmvbuM+7vxfW2dNYIfZ1q/v3Ft6+Bj1brTjjXwRQ5BHh7 tVHN7Q26T5+rQc8Z/cDQtB+mCnX7Xmnv8eX/aGoWOpr/ISC78TTofwVB/0iAPKuGPzdW OOYO/oSHXH4Hmj1LQO+CnIm8AyeDW0ojnAc/+8/55EB/KhVuQAQofWZkWJ1y2dRu1uDB 7OOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723019139; x=1723623939; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BlFQkb2cMvN1c1QVMf0JMKTkCt490ubd3m2R1WYvp/E=; b=Lybhw9Bg/UYLrmtkWHEmmjyQZHdcKfm98xgX2KlnxyTGZ+4FINlUwz1+K3voMhgJ8y HkaGxWCSWLwNW0aFG1h3lntiOc7AdsiACN/rxHYw5niqtzspknC6CKOFSUKHjk636ruI XH6dH3hQ8bGUGkwrqBkU4pK06ro5H6Gbn5f4dLOftguKI/BlJtatiFsBNsZTQLOZ4fbY sLh/0tWhNjm6cd68Ljs32DuliHsf/6NS1e1CtOzTbICmScxiw80UjveO9uN7SfNNiBUK pYrHXLFgmuIzO5pr8FLcZuo+YpBti3DsWx6UwkhfciRn4Dfrn+uaUFu2VfiaX23K0jsk XaEg== X-Forwarded-Encrypted: i=1; AJvYcCW4jiaH4HOpGO/I+putamRYC1Tlxh9Z1wJrtfxKADELBYNh/RkMp8wN/e+JYn+s6Qel/Bzc3nv5dDBMAr1jl2MqYUiVuphHsBVeD0xd X-Gm-Message-State: AOJu0YyGa/MDHtA9ajoHp3hwpBEuXcwy6rNd8R1aJeAW/gQLpH3EBM7T LwAKZuWgJXZUW+EL9fGsVVgN2hCjdSRpM95i0KMh2Z9FSdhEIrPA X-Google-Smtp-Source: AGHT+IG8xn5/66z7PTJPV4BCOi2Y/r1TSf0uignh/HpGSQRadyx5jGGO3Tw+w7ebVq7amQ4F6zh+BA== X-Received: by 2002:a05:6a20:de16:b0:1c6:ba8b:1e41 with SMTP id adf61e73a8af0-1c6ba8b1ebfmr12002331637.1.1723019138731; Wed, 07 Aug 2024 01:25:38 -0700 (PDT) Received: from barry-desktop.hub ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7106ed0d457sm7911763b3a.156.2024.08.07.01.25.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Aug 2024 01:25:38 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: justinjiang@vivo.com, chrisl@kernel.org, david@redhat.com, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, v-songbaohua@oppo.com, ying.huang@intel.com Subject: [PATCH v2 2/2] mm: attempt to batch free swap entries for zap_pte_range() Date: Wed, 7 Aug 2024 20:25:08 +1200 Message-Id: <20240807082508.358322-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240807082508.358322-1-21cnbao@gmail.com> References: <20240807082508.358322-1-21cnbao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Barry Song Zhiguo reported that swap release could be a serious bottleneck during process exits[1]. With mTHP, we have the opportunity to batch free swaps. Thanks to the work of Chris and Kairui[2], I was able to achieve this optimization with minimal code changes by building on their efforts. If swap_count is 1, which is likely true as most anon memory are private, we can free all contiguous swap slots all together. Ran the below test program for measuring the bandwidth of munmap using zRAM and 64KiB mTHP: #include #include #include unsigned long long tv_to_ms(struct timeval tv) { return tv.tv_sec * 1000 + tv.tv_usec / 1000; } main() { struct timeval tv_b, tv_e; int i; #define SIZE 1024*1024*1024 void *p =3D mmap(NULL, SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (!p) { perror("fail to get memory"); exit(-1); } madvise(p, SIZE, MADV_HUGEPAGE); memset(p, 0x11, SIZE); /* write to get mem */ madvise(p, SIZE, MADV_PAGEOUT); gettimeofday(&tv_b, NULL); munmap(p, SIZE); gettimeofday(&tv_e, NULL); printf("munmap in bandwidth: %ld bytes/ms\n", SIZE/(tv_to_ms(tv_e) - tv_to_ms(tv_b))); } The result is as below (munmap bandwidth): mm-unstable mm-unstable-with-patch round1 21053761 63161283 round2 21053761 63161283 round3 21053761 63161283 round4 20648881 67108864 round5 20648881 67108864 munmap bandwidth becomes 3X faster. [1] https://lore.kernel.org/linux-mm/20240731133318.527-1-justinjiang@vivo.= com/ [2] https://lore.kernel.org/linux-mm/20240730-swap-allocator-v5-0-cb9c148b9= 297@kernel.org/ Cc: Kairui Song Cc: Chris Li Cc: "Huang, Ying" Cc: Hugh Dickins Cc: Kalesh Singh Cc: Ryan Roberts Cc: David Hildenbrand Signed-off-by: Barry Song --- mm/swapfile.c | 78 +++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 67 insertions(+), 11 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index 35cb58373493..25c3f98fa8d5 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -156,6 +156,25 @@ static bool swap_is_has_cache(struct swap_info_struct = *si, return true; } =20 +static bool swap_is_last_map(struct swap_info_struct *si, + unsigned long offset, int nr_pages, + bool *has_cache) +{ + unsigned char *map =3D si->swap_map + offset; + unsigned char *map_end =3D map + nr_pages; + bool cached =3D false; + + do { + if ((*map & ~SWAP_HAS_CACHE) !=3D 1) + return false; + if (*map & SWAP_HAS_CACHE) + cached =3D true; + } while (++map < map_end); + + *has_cache =3D cached; + return true; +} + /* * returns number of pages in the folio that backs the swap entry. If posi= tive, * the folio was reclaimed. If negative, the folio was not reclaimed. If 0= , no @@ -1469,6 +1488,53 @@ static unsigned char __swap_entry_free(struct swap_i= nfo_struct *si, return usage; } =20 +static bool __swap_entries_free(struct swap_info_struct *si, + swp_entry_t entry, int nr) +{ + unsigned long offset =3D swp_offset(entry); + unsigned int type =3D swp_type(entry); + struct swap_cluster_info *ci; + bool has_cache =3D false; + unsigned char count; + bool can_batch; + int i; + + if (nr <=3D 1 || swap_count(data_race(si->swap_map[offset])) !=3D 1) + goto fallback; + /* cross into another cluster */ + if (nr > SWAPFILE_CLUSTER - offset % SWAPFILE_CLUSTER) + goto fallback; + + ci =3D lock_cluster_or_swap_info(si, offset); + can_batch =3D swap_is_last_map(si, offset, nr, &has_cache); + if (can_batch) { + for (i =3D 0; i < nr; i++) + WRITE_ONCE(si->swap_map[offset + i], SWAP_HAS_CACHE); + } + unlock_cluster_or_swap_info(si, ci); + + if (!can_batch) + goto fallback; + if (!has_cache) { + spin_lock(&si->lock); + swap_entry_range_free(si, entry, nr); + spin_unlock(&si->lock); + } + return has_cache; + +fallback: + for (i =3D 0; i < nr; i++) { + if (data_race(si->swap_map[offset + i])) { + count =3D __swap_entry_free(si, swp_entry(type, offset + i)); + if (count =3D=3D SWAP_HAS_CACHE) + has_cache =3D true; + } else { + WARN_ON_ONCE(1); + } + } + return has_cache; +} + /* * Drop the last HAS_CACHE flag of swap entries, caller have to * ensure all entries belong to the same cgroup. @@ -1792,11 +1858,9 @@ void free_swap_and_cache_nr(swp_entry_t entry, int n= r) { const unsigned long start_offset =3D swp_offset(entry); const unsigned long end_offset =3D start_offset + nr; - unsigned int type =3D swp_type(entry); struct swap_info_struct *si; bool any_only_cache =3D false; unsigned long offset; - unsigned char count; =20 if (non_swap_entry(entry)) return; @@ -1811,15 +1875,7 @@ void free_swap_and_cache_nr(swp_entry_t entry, int n= r) /* * First free all entries in the range. */ - for (offset =3D start_offset; offset < end_offset; offset++) { - if (data_race(si->swap_map[offset])) { - count =3D __swap_entry_free(si, swp_entry(type, offset)); - if (count =3D=3D SWAP_HAS_CACHE) - any_only_cache =3D true; - } else { - WARN_ON_ONCE(1); - } - } + any_only_cache =3D __swap_entries_free(si, entry, nr); =20 /* * Short-circuit the below loop if none of the entries had their --=20 2.34.1