From nobody Mon May 25 00:08:42 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 590B23D6CDA; Wed, 20 May 2026 10:19:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272401; cv=none; b=qaREBkPFPXMse9o8RuZCpdxOQPDcbVOXqRGLocaKVOoyh6RNcrddTaiXl4w+I4rgGhk0Gf94rtzEO4r93k9qx74rYYVpgZSGTzt6hisupCQlbH+wfmQxk2n9RLzJdLMzkpE6w0GBwtJ7hY22iIATH4YtcMrPojvDfPWUn52UCu0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272401; c=relaxed/simple; bh=f9J+IhisTdLnLgXR7JSjhUN2PwnfX+gYzer2vCIsUE0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=amCHhSHHi8Oggegs3fFZ6cngsHB8qSm2eg3pTbyUqwvQA1sfJblF0KApzeLnL65r3CkSE3MdFHv3wiRGQ1S0bWFOev53Mpo4DmcDT5SrjFtdmtRnshoXYsAA6tHbvBkPMpwm9R3T6ahXLqUWKM05xDzN8GWmeIA3MWebDAP9f2I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=OBC51bwB; arc=none smtp.client-ip=117.135.210.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="OBC51bwB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=9B VjTk8nQGGmUxI8PCEpxJWmL7Kaf4Qp03cf8IYb8qA=; b=OBC51bwB+qfYf08FyI e7P8qz+PHuCddXwTbn7uKjr4zqkZU0fNQ3zSbi6dn1aUAPqtA7YCBEow+TVMt7p9 lgTaW14EdA7BkCwq3S7lUnZykKo4W1Um4BYx0mKHKXPTXseMEG9nn+PZrYEez25h qfZ4rCsR5z/5lXO9nRS5Toi2w= Received: from czl-ubuntu-pc.. (unknown []) by gzsmtp5 (Coremail) with SMTP id QCgvCgDXbbprig1qQK31EQ--.641S3; Wed, 20 May 2026 18:18:25 +0800 (CST) From: Chi Zhiling To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: Hugh Dickins , Baolin Wang , "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Chi Zhiling Subject: [PATCH v1 1/5] mm/filemap: reduce unnecessary xarray lookups when read cached pages Date: Wed, 20 May 2026 18:15:34 +0800 Message-ID: <20260520101538.58745-2-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260520101538.58745-1-chizhiling@163.com> References: <20260520101538.58745-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: QCgvCgDXbbprig1qQK31EQ--.641S3 X-Coremail-Antispam: 1Uf129KBjvJXoWxCw4Dtw48ZFyDCF4fur4kJFb_yoW5CFW3pr 1rG3WkGrsrJw1YkrnxJay2ga4rW3s7XF45JFySg3WSvFn8Arn0kayxKF15KFyDZFy3ZF1f XF18Ja4vgF1Yq3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jso7tUUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC3BGcOmoNinFzrgAA3q Content-Type: text/plain; charset="utf-8" From: Chi Zhiling When reading small amounts of data from the page cache, only a single folio is typically returned from filemap_read_get_batch(). In this case, calling xas_advance() or xas_next() after adding the folio to the batch is unnecessary and only introduces extra branches. The same issue exists for large reads, where one additional xarray walk is always performed before termination. Move the boundary check to after the folio is added to the batch so the final redundant xarray advancement can be avoided. This significantly reduces the branch count in the read path. xas_next() does not update xa_index when xas->xa_node is set to XAS_RESTART, so checking the boundary before updating xa_index is sufficient to keep the folio within range. The warning should therefore never trigger. The branch count: 654.198 M/sec -> 646.444 M/sec Performance counter stats for 'fio --ioengine=3Dsync --rw=3Dread --bs=3D4k = --size=3D1G --runtime=3D300 --time_based --group_reporting --name=3Dseq_read_test --fil= ename=3Dfile': before: READ: bw=3D2697MiB/s (2828MB/s), 2697MiB/s-2697MiB/s (2828MB/s-2828MB/s), i= o=3D790GiB (848GB), run=3D300001-300001msec 245602051556 task-clock # 0.821 CPUs ut= ilized 78467 context-switches # 319.488 /sec 40 cpu-migrations # 0.163 /sec 3388 page-faults # 13.795 /sec 758312319204 instructions # 0.74 insn pe= r cycle 1025881497502 cycles # 4.177 GHz 160672383734 branches # 654.198 M/sec 361904512 branch-misses # 0.23% of all = branches after: READ: bw=3D2709MiB/s (2841MB/s), 2709MiB/s-2709MiB/s (2841MB/s-2841MB/s), i= o=3D794GiB (852GB), run=3D300000-300000msec 243985503670 task-clock # 0.812 CPUs ut= ilized 79004 context-switches # 323.806 /sec 30 cpu-migrations # 0.123 /sec 3355 page-faults # 13.751 /sec 747830935069 instructions # 0.73 insn pe= r cycle 1019609333322 cycles # 4.179 GHz 157722976668 branches # 646.444 M/sec 348984893 branch-misses # 0.22% of all = branches Signed-off-by: Chi Zhiling --- mm/filemap.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 4e636647100c..d54450e529bd 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2458,12 +2458,16 @@ static void filemap_get_read_batch(struct address_s= pace *mapping, { XA_STATE(xas, &mapping->i_pages, index); struct folio *folio; + pgoff_t next; + + if (unlikely(index > max)) + return; =20 rcu_read_lock(); for (folio =3D xas_load(&xas); folio; folio =3D xas_next(&xas)) { if (xas_retry(&xas, folio)) continue; - if (xas.xa_index > max || xa_is_value(folio)) + if (xa_is_value(folio) || WARN_ON(xas.xa_index > max)) break; if (xa_is_sibling(folio)) break; @@ -2479,7 +2483,11 @@ static void filemap_get_read_batch(struct address_sp= ace *mapping, break; if (folio_test_readahead(folio)) break; - xas_advance(&xas, folio_next_index(folio) - 1); + + next =3D folio_next_index(folio); + if (next > max) + break; + xas_advance(&xas, next - 1); continue; put_folio: folio_put(folio); --=20 2.43.0 From nobody Mon May 25 00:08:42 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EBB8371076; Wed, 20 May 2026 10:19:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272386; cv=none; b=E+vWukdEgOjSNf5zU0tc+KPMX6ec4HHwc/1+sZhN2J0Og4V+Yi9AIrYXp+0TUsXzYzxtXoHIA2DOsH+tvxAd8LZZoDErI6JblknhZ5C6jp2CGo6VLgx49MX333zHSHyS30N36vd7iO35sCbzn2lmuIgrglX5csKt/XIgsAai7cI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272386; c=relaxed/simple; bh=+cmwFjakZiPSRkrOfY05sf/K/tS/w+gImnoUnzNZxAc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tLj6IceAQokglemVu0f4ZM0e50nOzW3TRpPwz2XfsYr8H2xCfogQ/izjlXHi3XFTqnH/JipRilH2GkGpF1hjKeD6eua5zkn50mi2Ic4ZPPdekd+N488nwtkvURqPjlruOVdZjJyfhNmfIikeJVzbiDxtPr61sT97EpC00OQdRwc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=pD55Ilf4; arc=none smtp.client-ip=117.135.210.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="pD55Ilf4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=JJ QSb989Ft1cng8RSYH6fugEX9Hh0WZTKoGw3OGruD8=; b=pD55Ilf4nur0tnOU3G mx8swE2IrmRVcgZOcH5xJ6wXEP5i2Iww7dZ2JUazo2UZsnqUCQd0SsmwQWanpMxp Cg8d62lthDrQK0WnvTLCAazsnzvaCs6WZDkHMYyxMQzSRVtOOUcXSK5BpmqhevWy Okwi+hkZict3J5KTkTuy/EkfE= Received: from czl-ubuntu-pc.. (unknown []) by gzsmtp5 (Coremail) with SMTP id QCgvCgDXbbprig1qQK31EQ--.641S4; Wed, 20 May 2026 18:18:25 +0800 (CST) From: Chi Zhiling To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: Hugh Dickins , Baolin Wang , "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Chi Zhiling Subject: [PATCH v1 2/5] mm/filemap: reduce xarray lookups in filemap_get_folios_contig() Date: Wed, 20 May 2026 18:15:35 +0800 Message-ID: <20260520101538.58745-3-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260520101538.58745-1-chizhiling@163.com> References: <20260520101538.58745-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: QCgvCgDXbbprig1qQK31EQ--.641S4 X-Coremail-Antispam: 1Uf129KBjvJXoW7ur1ktw17Cw4kAry5Zw4kCrg_yoW8uF4DpF 43Ka4kWrWxJr13urnxA3Z5C3WYg34kWa15CFW7G3WavFnIyFnIkrnrK3W5Gr98ZrykJF1x XF48A348WF1Ut37anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jSsjbUUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC3BGcOmoNinFztgAA3y Content-Type: text/plain; charset="utf-8" From: Chi Zhiling Apply the same optimization used in filemap_get_read_batch() by moving the boundary check from the loop condition to before xas_advance(), avoiding an unnecessary xarray lookup and reducing branches in the fast path. Signed-off-by: Chi Zhiling --- mm/filemap.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index d54450e529bd..5a5268a7c802 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2258,11 +2258,14 @@ unsigned filemap_get_folios_contig(struct address_s= pace *mapping, XA_STATE(xas, &mapping->i_pages, *start); unsigned long nr; struct folio *folio; + pgoff_t next; + + if (unlikely(*start > end)) + return 0; =20 rcu_read_lock(); =20 - for (folio =3D xas_load(&xas); folio && xas.xa_index <=3D end; - folio =3D xas_next(&xas)) { + for (folio =3D xas_load(&xas); folio; folio =3D xas_next(&xas)) { if (xas_retry(&xas, folio)) continue; /* @@ -2270,11 +2273,11 @@ unsigned filemap_get_folios_contig(struct address_s= pace *mapping, * No current caller is looking for DAX entries. */ if (xa_is_value(folio)) - goto update_start; + break; =20 /* If we landed in the middle of a THP, continue at its end. */ if (xa_is_sibling(folio)) - goto update_start; + break; =20 if (!folio_try_get(folio)) goto retry; @@ -2282,12 +2285,15 @@ unsigned filemap_get_folios_contig(struct address_s= pace *mapping, if (unlikely(folio !=3D xas_reload(&xas))) goto put_folio; =20 + next =3D folio_next_index(folio); if (!folio_batch_add(fbatch, folio)) { - nr =3D folio_nr_pages(folio); - *start =3D folio->index + nr; + *start =3D next; goto out; } - xas_advance(&xas, folio_next_index(folio) - 1); + + if (next > end) + break; + xas_advance(&xas, next - 1); continue; put_folio: folio_put(folio); @@ -2296,9 +2302,7 @@ unsigned filemap_get_folios_contig(struct address_spa= ce *mapping, xas_reset(&xas); } =20 -update_start: nr =3D folio_batch_count(fbatch); - if (nr) { folio =3D fbatch->folios[nr - 1]; *start =3D folio_next_index(folio); --=20 2.43.0 From nobody Mon May 25 00:08:42 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D28B6369D68; Wed, 20 May 2026 10:19:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.2 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272391; cv=none; b=Wwe86pRpftOsQLj5fYI06yMb67Y+3QuDO3uwjEPGa6BOzcw+pDsRJiCPb4qopjsmgwDmRKQmu1YSqmTGs7r020+NO6nQ7WwwinMtUjGthD5TgcY18iw7hl9jhf/fUnXsqmXGYf6MTxwVkhvhmikPZanSUd9ji5eQ43Mjxuod9bY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272391; c=relaxed/simple; bh=p8Ki2orvcNd7P/MQkjn4RdNRgzdigcU4LQDNFRtpKFg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H7D64tiLaJdnCfkldnNg914YjtKThSaW1r/nq9dfSxsOa/lduayByyGL/0x20/BSj8FnW/ydAJeg/LLRQ1J1LIHJocJIyKOLOe0bU5L8yCU8BfwL/IhQVllpWi28gvk2ID9V9ypuuCRFhKGd3RqiodfhoL7JhzLkNSIkjt6CTKU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=V/5qubSD; arc=none smtp.client-ip=220.197.31.2 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="V/5qubSD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=hK /M4mH8Af7RP/R8Z/X9FqFJJChjz35upJb+IeYgpMA=; b=V/5qubSDLft5TwdJtV ji/UPnm6o8HUMhqxwbmT67iJuxFHpfcZP5qyyF6H40TTnmtfUxWlLAwYaMvVlUYX KvCkiSUNznp6I8G0qB2lrlumzeg7mkGMy90uKxfybGcrJEagKkxViXGlyj9EuiEK qE5pyR6+iAk31H6TCbfM6L3bM= Received: from czl-ubuntu-pc.. (unknown []) by gzsmtp5 (Coremail) with SMTP id QCgvCgDXbbprig1qQK31EQ--.641S5; Wed, 20 May 2026 18:18:25 +0800 (CST) From: Chi Zhiling To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: Hugh Dickins , Baolin Wang , "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Chi Zhiling Subject: [PATCH v1 3/5] mm/shmem: make SGP_NOALLOC succeed on hole like SGP_READ Date: Wed, 20 May 2026 18:15:36 +0800 Message-ID: <20260520101538.58745-4-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260520101538.58745-1-chizhiling@163.com> References: <20260520101538.58745-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: QCgvCgDXbbprig1qQK31EQ--.641S5 X-Coremail-Antispam: 1Uf129KBjvJXoWxXFW5Xr1rAry5Kr1UGry8Zrb_yoW5CrW3pr 9rG3s7GrWYg348Cr1DZws7Zr1UXan3tryUKryIg34qvF98AF10k347Aw1j9F18ArW8Ca1F qFWjkws8uF1UX3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jpApnUUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC2xKcOmoNinJltQAA3g Content-Type: text/plain; charset="utf-8" From: Chi Zhiling Change SGP_NOALLOC to return 0 with NULL folio on hole, matching SGP_READ behavior. This simplifies the sgp_type handling by unifying hole semantics across these types. Previously, SGP_NOALLOC returned -ENOENT on hole, while SGP_READ returned 0. This inconsistency required special handling in callers like khugepaged and userfaultfd. After this change: - khugepaged: behavior unchanged (checks both error and NULL folio) - userfaultfd: behavior unchanged (both -ENOENT and NULL are converted to -EFAULT before returning to userspace) Signed-off-by: Chi Zhiling --- include/linux/shmem_fs.h | 2 +- mm/khugepaged.c | 2 +- mm/shmem.c | 9 +++------ 3 files changed, 5 insertions(+), 8 deletions(-) diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index 93a0ba872ebe..d461713c095b 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -165,7 +165,7 @@ extern unsigned long shmem_partial_swap_usage(struct ad= dress_space *mapping, /* Flag allocation requirements to shmem_get_folio */ enum sgp_type { SGP_READ, /* don't exceed i_size, don't allocate page */ - SGP_NOALLOC, /* similar, but fail on hole or use fallocated page */ + SGP_NOALLOC, /* like SGP_READ, but accept fallocated page */ SGP_CACHE, /* don't exceed i_size, may allocate page */ SGP_WRITE, /* may exceed i_size, may allocate !Uptodate page */ SGP_FALLOC, /* like SGP_WRITE, but make existing page Uptodate */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index b8452dbdb043..3309d1c094df 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1950,7 +1950,7 @@ static enum scan_result collapse_file(struct mm_struc= t *mm, unsigned long addr, xas_unlock_irq(&xas); /* swap in or instantiate fallocated page */ if (shmem_get_folio(mapping->host, index, 0, - &folio, SGP_NOALLOC)) { + &folio, SGP_NOALLOC) || !folio) { result =3D SCAN_FAIL; goto xa_unlocked; } diff --git a/mm/shmem.c b/mm/shmem.c index 3b5dc21b323c..458853c506ea 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2524,14 +2524,11 @@ static int shmem_get_folio_gfp(struct inode *inode,= pgoff_t index, } =20 /* - * SGP_READ: succeed on hole, with NULL folio, letting caller zero. - * SGP_NOALLOC: fail on hole, with NULL folio, letting caller fail. + * SGP_READ/SGP_NOALLOC: succeed on hole, with NULL folio. */ *foliop =3D NULL; - if (sgp =3D=3D SGP_READ) + if (sgp <=3D SGP_NOALLOC) return 0; - if (sgp =3D=3D SGP_NOALLOC) - return -ENOENT; =20 /* * Fast cache lookup and swap lookup did not find it: allocate. @@ -2657,7 +2654,7 @@ static int shmem_get_folio_gfp(struct inode *inode, p= goff_t index, * * When no folio is found, the behavior depends on @sgp: * - for SGP_READ, *@foliop is %NULL and 0 is returned - * - for SGP_NOALLOC, *@foliop is %NULL and -ENOENT is returned + * - for SGP_NOALLOC, *@foliop is %NULL and 0 is returned * - for all other flags a new folio is allocated, inserted into the * page cache and returned locked in @foliop. * --=20 2.43.0 From nobody Mon May 25 00:08:42 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 001D63D8914; Wed, 20 May 2026 10:20:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272407; cv=none; b=Jg9Am5ftQo2y8tJvhOV5xTEA5DuhB2Z9v+rgownaZeyzAvScpMFzEHbVtPjfLosGT8C/c93zv3wkU5Cnu+smAly55zyehQu/rhTcFSXlCp/d3MggLkATyLqCbAfpGSxzv7HXaAgOJWOuaNl4jR//Os/8Ca7ZWemTyJZabe2NaEs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272407; c=relaxed/simple; bh=nlOvajUKQD2BdgpVjY1kpIMhzz4fKhzvO5HvrkmpJ+g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ts93Jqvey7R2NMXHg25jJwrxF0nOVmAYB/mQXh7lNczQnTpvdMP0nTt1Q15AYCQvJxSoEi3gaiOzjIsE4u58TEVJpesxK8+xmbSm2Fa0uyMtKIRzfARGW4pAL6S6me6o6OK/e/Nq2U2sI5eY+IsxbEM6oW4LmiNwD5nB9puCxCw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=AuLw0WFb; arc=none smtp.client-ip=220.197.31.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="AuLw0WFb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=th 5U1L5vmkhODOQbkCC1ck4kSZkOj2dVnpqhVaNU8Qs=; b=AuLw0WFbsvQEJCokO1 Yp3PrR43mnNKiQroLSUV6P0BDgy0E1XKXhERkECG0XxbAuXsuiH8pwsyEVyDP/ub xKkUmmJRSPbRJl6EWXabBLOkqpdR4LEhpERQ0p3TbmRrawwbo8AH+6Zv8lUOivZZ qt2H6lhmk3kJ4Ha6m2pWwH10k= Received: from czl-ubuntu-pc.. (unknown []) by gzsmtp5 (Coremail) with SMTP id QCgvCgDXbbprig1qQK31EQ--.641S6; Wed, 20 May 2026 18:18:26 +0800 (CST) From: Chi Zhiling To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: Hugh Dickins , Baolin Wang , "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Chi Zhiling Subject: [PATCH v1 4/5] mm/shmem: introduce copy_zero_to_iter() for large zeroing Date: Wed, 20 May 2026 18:15:37 +0800 Message-ID: <20260520101538.58745-5-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260520101538.58745-1-chizhiling@163.com> References: <20260520101538.58745-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: QCgvCgDXbbprig1qQK31EQ--.641S6 X-Coremail-Antispam: 1Uf129KBjvJXoW7CF4kZr13Kw18Kry8CryxuFg_yoW8WFyxpF y3G3Z8Ar4kJayfWr4fXF4rur1Fvr93J3yUGFWxGF4YvFn0qr98KFyrKFyYkFWfAryUCw1I qa10yryYka10yrJanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07j1-erUUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC9xKcOmoNinIMnAAA3M Content-Type: text/plain; charset="utf-8" From: Chi Zhiling This is a prep patch for shmem folio batching in the read path, where non-uptodate folios need to be handled in the main iteration loop. A large non-uptodate folio should be treated as a hole. Currently, holes larger than PAGE_SIZE cannot be handled because ZERO_PAGE is limited to a single page. Add copy_zero_to_iter() as a wrapper to support copying larger zero ranges to the iterator. Signed-off-by: Chi Zhiling --- mm/shmem.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index 458853c506ea..96ea5a4c0aff 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -3336,6 +3336,20 @@ shmem_write_end(const struct kiocb *iocb, struct add= ress_space *mapping, return copied; } =20 +static size_t copy_zero_to_iter(size_t bytes, struct iov_iter *i) +{ + struct folio *zero =3D largest_zero_folio(); + unsigned long nr, ret, written =3D 0; + + do { + nr =3D min(bytes - written, folio_size(zero)); + ret =3D copy_folio_to_iter(zero, 0, nr, i); + written +=3D ret; + } while (written < bytes && ret =3D=3D nr); + + return written; +} + static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *t= o) { struct file *file =3D iocb->ki_filp; @@ -3430,7 +3444,7 @@ static ssize_t shmem_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) * clear_user() not so much, that it is noticeably * faster to copy the zero page instead of clearing. */ - ret =3D copy_page_to_iter(ZERO_PAGE(0), offset, nr, to); + ret =3D copy_zero_to_iter(nr, to); } else { /* * But submitting the same page twice in a row to --=20 2.43.0 From nobody Mon May 25 00:08:42 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5DAFF3D75CF; Wed, 20 May 2026 10:19:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.3 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272395; cv=none; b=VOhel/qj0qXRPHH6fYrKwFnHoHsxImHiOLaKgW8eeGcTC5ynOC+3Fhl1CkPOMkRW5H6X2yV8/R254KhBo+DGr1k9QMWt2Kc3PzKdb6maWi42MvrhrOpW2rUdTN20zuDUmovregJH8lDXWKsMkKGg3PptbBkJy3PoTYcasfwsoQ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779272395; c=relaxed/simple; bh=j1SC3CWyTpLRoSsCegCksqPOhFsaSmPh/3cUQzXMzDI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OF2Ttj98jzNlS/T8MWZB2wFga58kVAChUQPztBQnfel7vinqgHCIS50GvGAsGPstANpAuDBfLWz14KFoTjuOZ+Z1khAfy0DNaYNLCTiM1kurBayKQRHAeHpgWdBXecEQKUUrta0i+KKcV2TwrCMqZIKHZmjMLDLuaa9S0804wfI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=bjPI6Jhy; arc=none smtp.client-ip=117.135.210.3 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="bjPI6Jhy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=PO Y07p4j3GPNthuCu4k23r6LNTlq73DDrD9MKLgmiwk=; b=bjPI6JhyYxNG7cJPhz OGjxgBVi3zkiNRH/3i+lshSHVTH24MxKIYauZD+5SBvgzQWsezV4h2+64cYie76f uv28g+XqDwsRzDUgD8bim6RipQNFx138Z1I96Ei5JAfgWRtda+9IeqorWlpieuP3 7n9R9gbRlPYJVkReKTkh5exwg= Received: from czl-ubuntu-pc.. (unknown []) by gzsmtp5 (Coremail) with SMTP id QCgvCgDXbbprig1qQK31EQ--.641S7; Wed, 20 May 2026 18:18:26 +0800 (CST) From: Chi Zhiling To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: Hugh Dickins , Baolin Wang , "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Chi Zhiling Subject: [PATCH v1 5/5] mm/shmem: optimize file read with folio batching Date: Wed, 20 May 2026 18:15:38 +0800 Message-ID: <20260520101538.58745-6-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260520101538.58745-1-chizhiling@163.com> References: <20260520101538.58745-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: QCgvCgDXbbprig1qQK31EQ--.641S7 X-Coremail-Antispam: 1Uf129KBjvJXoWxuF13ur1kJF4UXr4fKryfZwb_yoWrXrWxpF W3KwnxWrWxGw43Wr4fJF48Za4rK3s3tFW5JFyxG3W3Z3W5J3s8KF18t34Y9FW5Jry7C3WI gw1vkr18X3Wjvr7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jISoJUUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC9xKcOmoNinIMpwAA33 Content-Type: text/plain; charset="utf-8" From: Chi Zhiling Optimize shmem file read by using filemap_get_folios_contig() to batch fetch contiguous folios from the page cache, reducing the overhead of repeated shmem_get_folio() calls. When the folio batch is exhausted, attempt to refill it with filemap_get_folios_contig(). If no folios are found (hole or swapped out pages), fall back to shmem_get_folio() to handle these cases individually. Additionally: - Defer folio_put() until the batch is exhausted or on exit - Add folio_test_uptodate() check before copying to ensure data validity Signed-off-by: Chi Zhiling --- mm/shmem.c | 50 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 36 insertions(+), 14 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 96ea5a4c0aff..e0eacc23cccd 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -3355,11 +3355,14 @@ static ssize_t shmem_file_read_iter(struct kiocb *i= ocb, struct iov_iter *to) struct file *file =3D iocb->ki_filp; struct inode *inode =3D file_inode(file); struct address_space *mapping =3D inode->i_mapping; + struct folio_batch fbatch; pgoff_t index; unsigned long offset; int error =3D 0; ssize_t retval =3D 0; =20 + folio_batch_init(&fbatch); + for (;;) { struct folio *folio =3D NULL; struct page *page =3D NULL; @@ -3372,18 +3375,38 @@ static ssize_t shmem_file_read_iter(struct kiocb *i= ocb, struct iov_iter *to) break; =20 index =3D iocb->ki_pos >> PAGE_SHIFT; - error =3D shmem_get_folio(inode, index, 0, &folio, SGP_READ); - if (error) { - if (error =3D=3D -EINVAL) - error =3D 0; - break; +fetch: + folio =3D folio_batch_next(&fbatch); + if (!folio) { + pgoff_t start =3D index; + pgoff_t end =3D (iocb->ki_pos + to->count - 1) >> PAGE_SHIFT; + + if (folio_batch_count(&fbatch)) { + for (int i =3D 0; i < folio_batch_count(&fbatch); i++) + folio_put(fbatch.folios[i]); + folio_batch_reinit(&fbatch); + } + + filemap_get_folios_contig(inode->i_mapping, &start, end, &fbatch); + if (folio_batch_count(&fbatch)) + goto fetch; + + error =3D shmem_get_folio(inode, index, 0, &folio, SGP_READ); + if (unlikely(error)) { + if (error =3D=3D -EINVAL) + error =3D 0; + break; + } + if (folio) { + folio_unlock(folio); + folio_batch_add(&fbatch, folio); + fbatch.i++; + } } - if (folio) { - folio_unlock(folio); =20 + if (folio) { page =3D folio_file_page(folio, index); if (PageHWPoison(page)) { - folio_put(folio); error =3D -EIO; break; } @@ -3398,11 +3421,9 @@ static ssize_t shmem_file_read_iter(struct kiocb *io= cb, struct iov_iter *to) * are called without i_rwsem protection against truncate */ i_size =3D i_size_read(inode); - if (unlikely(iocb->ki_pos >=3D i_size)) { - if (folio) - folio_put(folio); + if (unlikely(iocb->ki_pos >=3D i_size)) break; - } + end_offset =3D min_t(loff_t, i_size, iocb->ki_pos + to->count); if (folio && likely(!fallback_page_copy)) fsize =3D folio_size(folio); @@ -3411,7 +3432,7 @@ static ssize_t shmem_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) offset =3D iocb->ki_pos & (fsize - 1); nr =3D min_t(loff_t, end_offset - iocb->ki_pos, fsize - offset); =20 - if (folio) { + if (folio && folio_test_uptodate(folio)) { /* * If users can be writing to this page using arbitrary * virtual addresses, take care about potential aliasing @@ -3437,7 +3458,6 @@ static ssize_t shmem_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) ret =3D copy_folio_to_iter(folio, offset, nr, to); else ret =3D copy_page_to_iter(page, offset, nr, to); - folio_put(folio); } else if (user_backed_iter(to)) { /* * Copy to user tends to be so well optimized, but @@ -3466,6 +3486,8 @@ static ssize_t shmem_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) cond_resched(); } =20 + for (int i =3D 0; i < folio_batch_count(&fbatch); i++) + folio_put(fbatch.folios[i]); file_accessed(file); return retval ? retval : error; } --=20 2.43.0