From nobody Wed Dec 17 03:03:45 2025 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4156778676 for ; Tue, 18 Jun 2024 06:54:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718693680; cv=none; b=bPnSeHvZ1Df0Wtbwej/H+Jp7rsa2QyTqRBYo2kljbw2D8pIGdXT5zgZ91ZheBvk9WXjfLTfosk4TciJOpERyaf71FTrzek+pBZG+E7/FZf6LEs0laIYSw2b3LsXgniwx1l9dF1POXQpFa0YIayaITiZn3BQFOqGeIw+LNPFBR9w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718693680; c=relaxed/simple; bh=b8LuyynHN3NsClPyf8jZS4sH1SoZUwxuViazv8I7w1Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=A3KS+eobtbhuw7HGZ0J/nDzoXyYpP9ysDekn2k20TxvUvZD/e5HHftZN6kZ6ub27/+nPgKlFyr5s0Zy99N2DVDFwhKEW0xZt/jxYUTD4gHfWCt1hJ8py+tdD9QxXLYFb9Sugxs0WWarCblMv0M8LDg2w64WVTebop9QtcJJuank= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=a2KWH0XI; arc=none smtp.client-ip=115.124.30.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="a2KWH0XI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1718693676; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=koLTHCsWXBAa3N27f7lrGDpaxVfvYWdW4ehMIBy6BWA=; b=a2KWH0XIyyGXX0IJR7KJRMXwHPjSTQyPFdqJvj180MWe561lG62uXZ7FRST7HhVxf44mnaYwJineiUzGU/oUGkK1deKfhunH4GrlMm2vZowgDqG2RkMYQ7m/v/ijRGm3N9RfjISejOvYrtB0RTHaFrHjDe5B2VrFW2NTYPTc4Y8= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033045046011;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0W8jKUZG_1718693674; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W8jKUZG_1718693674) by smtp.aliyun-inc.com; Tue, 18 Jun 2024 14:54:34 +0800 From: Baolin Wang To: akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 7/9] mm: shmem: support large folio allocation for shmem_replace_folio() Date: Tue, 18 Jun 2024 14:54:19 +0800 Message-Id: <75d1c1f2fbc88dce476a53fa90f085b3bebb83c5.1718690645.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To support large folio swapin for shmem in the following patches, add large folio allocation for the new replacement folio in shmem_replace_folio= (). Moreover large folios occupy N consecutive entries in the swap cache instead of using multi-index entries like the page cache, therefore we should replace each consecutive entries in the swap cache instead of using the shmem_replace_entry(). As well as updating statistics and folio reference count using the number of pages in the folio. Signed-off-by: Baolin Wang --- mm/shmem.c | 53 ++++++++++++++++++++++++++++++----------------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 4d7996962388..c0a9253f3a99 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1878,28 +1878,24 @@ static bool shmem_should_replace_folio(struct folio= *folio, gfp_t gfp) static int shmem_replace_folio(struct folio **foliop, gfp_t gfp, struct shmem_inode_info *info, pgoff_t index) { - struct folio *old, *new; - struct address_space *swap_mapping; - swp_entry_t entry; - pgoff_t swap_index; - int error; - - old =3D *foliop; - entry =3D old->swap; - swap_index =3D swap_cache_index(entry); - swap_mapping =3D swap_address_space(entry); + struct folio *new, *old =3D *foliop; + swp_entry_t entry =3D old->swap; + struct address_space *swap_mapping =3D swap_address_space(entry); + pgoff_t swap_index =3D swap_cache_index(entry); + XA_STATE(xas, &swap_mapping->i_pages, swap_index); + int nr_pages =3D folio_nr_pages(old); + int error =3D 0, i; =20 /* * We have arrived here because our zones are constrained, so don't * limit chance of success by further cpuset and node constraints. */ gfp &=3D ~GFP_CONSTRAINT_MASK; - VM_BUG_ON_FOLIO(folio_test_large(old), old); - new =3D shmem_alloc_folio(gfp, 0, info, index); + new =3D shmem_alloc_folio(gfp, folio_order(old), info, index); if (!new) return -ENOMEM; =20 - folio_get(new); + folio_ref_add(new, nr_pages); folio_copy(new, old); flush_dcache_folio(new); =20 @@ -1909,18 +1905,24 @@ static int shmem_replace_folio(struct folio **folio= p, gfp_t gfp, new->swap =3D entry; folio_set_swapcache(new); =20 - /* - * Our caller will very soon move newpage out of swapcache, but it's - * a nice clean interface for us to replace oldpage by newpage there. - */ + /* Swap cache still stores N entries instead of a high-order entry */ xa_lock_irq(&swap_mapping->i_pages); - error =3D shmem_replace_entry(swap_mapping, swap_index, old, new); + for (i =3D 0; i < nr_pages; i++) { + void *item =3D xas_store(&xas, new); + + if (item !=3D old) { + error =3D -ENOENT; + break; + } + + xas_next(&xas); + } if (!error) { mem_cgroup_replace_folio(old, new); - __lruvec_stat_mod_folio(new, NR_FILE_PAGES, 1); - __lruvec_stat_mod_folio(new, NR_SHMEM, 1); - __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -1); - __lruvec_stat_mod_folio(old, NR_SHMEM, -1); + __lruvec_stat_mod_folio(new, NR_FILE_PAGES, nr_pages); + __lruvec_stat_mod_folio(new, NR_SHMEM, nr_pages); + __lruvec_stat_mod_folio(old, NR_FILE_PAGES, -nr_pages); + __lruvec_stat_mod_folio(old, NR_SHMEM, -nr_pages); } xa_unlock_irq(&swap_mapping->i_pages); =20 @@ -1940,7 +1942,12 @@ static int shmem_replace_folio(struct folio **foliop= , gfp_t gfp, old->private =3D NULL; =20 folio_unlock(old); - folio_put_refs(old, 2); + /* + * The old folio are removed from swap cache, drop the 'nr_pages' + * reference, as well as one temporary reference getting from swap + * cache. + */ + folio_put_refs(old, nr_pages + 1); return error; } =20 --=20 2.39.3