From nobody Mon Jun 8 07:23:00 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B47E273D6D; Mon, 1 Jun 2026 05:58:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293490; cv=none; b=d0txyWAc7Zp/vcvvxjv7l7V8Z7Fi56y718bYEvCClmDaq8htMwQ3AuwxxzJwcW8zpaAVEJQ4zuRyFB2jbLwDh8flFFnZat5E1zwWKjAAVepdM5xPX95wyHMLuz38rgtui8YUDN8c98Ar2PxvmuNa/usT4iCe6QCrMxfraQpZVFo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293490; c=relaxed/simple; bh=q0on3xmktCoWaC80cHJbstOkLYMejQm7YbRQnLLX2tA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NFBJL9ZqYHBYmOPCG73UfyJf1COno5WbZvtMEn3MCwg+LxRty/qboa3SBJf0XQvBq13R0xJJWy3lGZqGu3KuN/vwvzeqvwOFB4YO5LfxeAQ1KgVl5STpvYc0p0pT/v9XadnWdPzJcvP99iMKmrJxKRgAPfrocVWMI1edCyszngA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=dQkui9ew; arc=none smtp.client-ip=117.135.210.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="dQkui9ew" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=y8 RQrqromftC/nCPgQZhg6QK2Ox4EieThtfTf6AdAQo=; b=dQkui9ew9GNqXbNcE6 xWnzBmmRZQ25fnJ9RyI8uabGx6AnJkV84lvibaoa6svGxSn/AT6BDIhsk+QIK/9+ lWV5UmeJTHkArpo0ghvPMKylnRj5RUe15kNf8/UUF4F1/kwpXdR8vGjAcobHfgfd 8oyBC2hV+UbLEc0R/leHRdMEs= Received: from czl-ubuntu-pc.. (unknown []) by gzga-smtp-mtada-g0-3 (Coremail) with SMTP id _____wD3_7Q6Hx1qsb0MAw--.47504S3; Mon, 01 Jun 2026 13:57:19 +0800 (CST) From: Chi Zhiling To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , Hugh Dickins , Baolin Wang , Chi Zhiling Subject: [PATCH v2 1/5] mm/filemap: reduce unnecessary xarray lookups when read cached pages Date: Mon, 1 Jun 2026 13:57:00 +0800 Message-ID: <20260601055704.167436-2-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260601055704.167436-1-chizhiling@163.com> References: <20260601055704.167436-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wD3_7Q6Hx1qsb0MAw--.47504S3 X-Coremail-Antispam: 1Uf129KBjvJXoW7ur4ruFW7JF1rZr1DCFyrXrb_yoW8Zr13pr n5Ka4ktr4DJFWYkrsxJ3WxZa4rK3sYqay5JFyxKw1fZFn8JFn8Kr92ga45G3Z8uFyrXF1I qF18Xa40gF45t3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07j7a93UUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC9x+4VmodHz9QMQAA3w Content-Type: text/plain; charset="utf-8" From: Chi Zhiling When reading small amounts of data from the page cache, only a single folio is typically returned from filemap_read_get_batch(). In this case, calling xas_advance() or xas_next() after adding the folio to the batch is unnecessary and only introduces extra branches. The same issue exists for large reads, where one additional xarray walk is always performed before termination. Quit the loop once we get the last folio in the range, so the final redundant xarray advancement can be avoided. The xas_next() does not update xa_index when xas->xa_node is set to XAS_RESTART, so the put and retry path would not update xa_index, hence the warning should therefore never trigger. During the 4k reads test, the overhead of this function dropped from 2.91% to 2.53%. Signed-off-by: Chi Zhiling --- mm/filemap.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 4e636647100c..d54450e529bd 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2458,12 +2458,16 @@ static void filemap_get_read_batch(struct address_s= pace *mapping, { XA_STATE(xas, &mapping->i_pages, index); struct folio *folio; + pgoff_t next; + + if (unlikely(index > max)) + return; =20 rcu_read_lock(); for (folio =3D xas_load(&xas); folio; folio =3D xas_next(&xas)) { if (xas_retry(&xas, folio)) continue; - if (xas.xa_index > max || xa_is_value(folio)) + if (xa_is_value(folio) || WARN_ON(xas.xa_index > max)) break; if (xa_is_sibling(folio)) break; @@ -2479,7 +2483,11 @@ static void filemap_get_read_batch(struct address_sp= ace *mapping, break; if (folio_test_readahead(folio)) break; - xas_advance(&xas, folio_next_index(folio) - 1); + + next =3D folio_next_index(folio); + if (next > max) + break; + xas_advance(&xas, next - 1); continue; put_folio: folio_put(folio); --=20 2.43.0 From nobody Mon Jun 8 07:23:00 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B431346A10; Mon, 1 Jun 2026 05:58:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.3 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293495; cv=none; b=jjlDDFs/tBaT0HBHmsZBrIC61Eb7lOUsjComr4Y1q8+IUkxcGC9P/AZGN6GHK/ajy2SwP590oyc5bV+wx/Wvz8vrNSp9AsYjIgneFsG1SP3hJ2Lp0lF1g1wPIL+598ipteA9oayrqB2A5EycQwjIyrQBYik/Vnmcz5cn+ySjTjo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293495; c=relaxed/simple; bh=UMxeq9LPuPXgUjRNwxWespCzYt4u0qUiwMiR3PxLwQw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NJQhqdSj61IksVRSMc+NlXaR7z4/M/VN0plSjwq/vMqo5vRZsAA4/5QlqfhPcwEFzJJEKQOv/LL2ey0OHwws0dAOfDVoN9xWAB3W6ZPf8g+77m7YFa3I4Q4T3GTXJbHiC2Eg3KPp85sh0x9fk7L3Ow/1HRZMnZqmkkQx+YQuhAY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=HQh1rnhi; arc=none smtp.client-ip=117.135.210.3 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="HQh1rnhi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=hS FfhvzoChQOQgIIJG5v3NjnH7WU8ceJmhHARcctEEs=; b=HQh1rnhiRosYsEScko GkClTFquAuKP+Vh79FH9s2EGthCwz9V6MaOhQB2tX/5wFhrFQjJ565KBQYkPMy3a +1/ckkRU5cUMXC45gK942ZQ4HdnVnAi6hpF719Fa67drlNBfIfnF/4te+WYqDEoZ bY6j/Z8TL74u0/qa58UchjLI0= Received: from czl-ubuntu-pc.. (unknown []) by gzga-smtp-mtada-g0-3 (Coremail) with SMTP id _____wD3_7Q6Hx1qsb0MAw--.47504S4; Mon, 01 Jun 2026 13:57:20 +0800 (CST) From: Chi Zhiling To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , Hugh Dickins , Baolin Wang , Chi Zhiling Subject: [PATCH v2 2/5] mm/filemap: reduce xarray lookups in filemap_get_folios_contig() Date: Mon, 1 Jun 2026 13:57:01 +0800 Message-ID: <20260601055704.167436-3-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260601055704.167436-1-chizhiling@163.com> References: <20260601055704.167436-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wD3_7Q6Hx1qsb0MAw--.47504S4 X-Coremail-Antispam: 1Uf129KBjvJXoW7ur1ktw17Cw4kAF17WFy7trb_yoW8tFWfpF 4agas7WrWxJw13Cr1fA3WrC3WYk34vqa15GFWxGw1fZFn0yFnIkr17K3W5Jas8ZrykJF1x XF48J34kWF1UJa7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07j24E_UUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC9wC5V2odH0BQSAAA3p Content-Type: text/plain; charset="utf-8" From: Chi Zhiling Apply the same optimization used in filemap_get_read_batch() by moving the boundary check from the loop condition to before xas_advance(), avoiding an unnecessary xarray lookup and reducing branches in the fast path. Signed-off-by: Chi Zhiling --- mm/filemap.c | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index d54450e529bd..6bc717f205a7 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2258,11 +2258,13 @@ unsigned filemap_get_folios_contig(struct address_s= pace *mapping, XA_STATE(xas, &mapping->i_pages, *start); unsigned long nr; struct folio *folio; + pgoff_t next; =20 - rcu_read_lock(); + if (unlikely(*start > end)) + return 0; =20 - for (folio =3D xas_load(&xas); folio && xas.xa_index <=3D end; - folio =3D xas_next(&xas)) { + rcu_read_lock(); + for (folio =3D xas_load(&xas); folio; folio =3D xas_next(&xas)) { if (xas_retry(&xas, folio)) continue; /* @@ -2270,11 +2272,11 @@ unsigned filemap_get_folios_contig(struct address_s= pace *mapping, * No current caller is looking for DAX entries. */ if (xa_is_value(folio)) - goto update_start; + break; =20 /* If we landed in the middle of a THP, continue at its end. */ if (xa_is_sibling(folio)) - goto update_start; + break; =20 if (!folio_try_get(folio)) goto retry; @@ -2282,30 +2284,28 @@ unsigned filemap_get_folios_contig(struct address_s= pace *mapping, if (unlikely(folio !=3D xas_reload(&xas))) goto put_folio; =20 - if (!folio_batch_add(fbatch, folio)) { - nr =3D folio_nr_pages(folio); - *start =3D folio->index + nr; - goto out; - } - xas_advance(&xas, folio_next_index(folio) - 1); + if (!folio_batch_add(fbatch, folio)) + break; + + next =3D folio_next_index(folio); + if (next > end) + break; + xas_advance(&xas, next - 1); continue; + put_folio: folio_put(folio); - retry: xas_reset(&xas); } + rcu_read_unlock(); =20 -update_start: nr =3D folio_batch_count(fbatch); - if (nr) { folio =3D fbatch->folios[nr - 1]; *start =3D folio_next_index(folio); } -out: - rcu_read_unlock(); - return folio_batch_count(fbatch); + return nr; } EXPORT_SYMBOL(filemap_get_folios_contig); =20 --=20 2.43.0 From nobody Mon Jun 8 07:23:00 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E4113358C4; Mon, 1 Jun 2026 05:58:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293497; cv=none; b=tMK8jg5pGu1hcCHmHVKei17Bi5WLYoO3ofvq47x1bSmhoSKXmg6fl49RBdvbCn3yVDGAdZEDdkrmpKjp1segh5Kke87K1318w663jnYjf345KsigV6/7nOQ6fCP1Qib+tf3KZL1feIgqA+xX/iKSrSnm5yQIqaE1MLOsAobvFnY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293497; c=relaxed/simple; bh=L5BeWN0U7lKv4EdCz4K3mK8Rpw62dharsFq3h0wdT1c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZIPeMe6n3uj8S2kLnRtF8JwKliPBSzVzZpqm5lx7JdqhP5WdbboTc9DTvtuV9ebYJfeKFtgKm0jAZcQE67/YS0OnpL4QZHR/gesCTI8BKUhZHVDIfIsQhvMtBWyBqN6lmEs1I4SaFFN4MaveBKIkXb9kihT8N2C0dKqIv+smlag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=CAX058sM; arc=none smtp.client-ip=220.197.31.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="CAX058sM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=Gw M4pkP8SJrbgKz+ttAiFjnJ2HZxTx+PZU1H2nT4urU=; b=CAX058sMffpNy193aq v83uUOayxBBAYsZi1oA9/Il+UOCBIyuaJ/tp5HCEYtUKcpGyAsIHqg2IdM+NzByK AU3bAlF4tkHoGIB6qszRG9esP2NcI6zlmsVDlR63pgHbPRq0SKe8iTBTdT4tO7Fv XYfQCCgLVv44ZxHWLx2aw+F6w= Received: from czl-ubuntu-pc.. (unknown []) by gzga-smtp-mtada-g0-3 (Coremail) with SMTP id _____wD3_7Q6Hx1qsb0MAw--.47504S5; Mon, 01 Jun 2026 13:57:21 +0800 (CST) From: Chi Zhiling To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , Hugh Dickins , Baolin Wang , Chi Zhiling Subject: [PATCH v2 3/5] mm/shmem: introduce copy_zero_to_iter() for large zeroing Date: Mon, 1 Jun 2026 13:57:02 +0800 Message-ID: <20260601055704.167436-4-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260601055704.167436-1-chizhiling@163.com> References: <20260601055704.167436-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wD3_7Q6Hx1qsb0MAw--.47504S5 X-Coremail-Antispam: 1Uf129KBjvJXoW7CF4kZr13Kw18Kry8CryxuFg_yoW8WFyxpF y3G3Z8ArWkJayxWw4fXF4rur1Fvr93JrWUGFWxGF4YvFn0qr90qFyrKFyYkFWfA34UGw1I qa10yrWYka10yrJanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07j70eQUUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC9wG5V2odH0FQZAAA3F Content-Type: text/plain; charset="utf-8" From: Chi Zhiling This is a prep patch for shmem folio batching in the read path, where non-uptodate folios need to be handled in the main iteration loop. A large non-uptodate folio should be treated as a hole. Currently, holes larger than PAGE_SIZE cannot be handled because ZERO_PAGE is limited to a single page. Add copy_zero_to_iter() as a wrapper to support copying larger zero ranges to the iterator. Signed-off-by: Chi Zhiling --- mm/shmem.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index 3b5dc21b323c..b08118fbd275 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -3339,6 +3339,20 @@ shmem_write_end(const struct kiocb *iocb, struct add= ress_space *mapping, return copied; } =20 +static size_t copy_zero_to_iter(size_t bytes, struct iov_iter *i) +{ + struct folio *zero =3D largest_zero_folio(); + unsigned long nr, ret, written =3D 0; + + do { + nr =3D min(bytes - written, folio_size(zero)); + ret =3D copy_folio_to_iter(zero, 0, nr, i); + written +=3D ret; + } while (written < bytes && ret =3D=3D nr); + + return written; +} + static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *t= o) { struct file *file =3D iocb->ki_filp; @@ -3433,7 +3447,7 @@ static ssize_t shmem_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) * clear_user() not so much, that it is noticeably * faster to copy the zero page instead of clearing. */ - ret =3D copy_page_to_iter(ZERO_PAGE(0), offset, nr, to); + ret =3D copy_zero_to_iter(nr, to); } else { /* * But submitting the same page twice in a row to --=20 2.43.0 From nobody Mon Jun 8 07:23:00 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4603732ED27; Mon, 1 Jun 2026 05:58:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.3 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293491; cv=none; b=LvjLe6vbK9QxiwVH1Pr9wngnNk5qR1wKYYTpOMYRqx34FpBkz3gpDJNW5i1Wq2NudyPGemlP/jBXT7yOCstk/1H06iIbxP+sUyrdZqOtuZpfdvi4+JbyZ/T+spOLqe3Y0Tpz9WQco1hxEP2iky9Ad+bK24N3vU6Nd2CphEQkTq8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293491; c=relaxed/simple; bh=j43uHXa6eXfEW8A4IsZeegZRPNbSqERL6PtFpiBJVBo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LFWFlQwg8vAL588t1Sr/pRfHVTnodEnmK+5B7H5JZ5SxtoU1j8d/nnMY7Y747ZENSt6i7gXnhLPa2KkKKQePPvZvOd9acSSdzECR5t9Jj5wO5snSVzSoCoMikcM73QTXtj8CC7/tJgcRRPIvKKGfa4itFVQ3xAudgOuitZDZg6k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=ND7AD0lX; arc=none smtp.client-ip=220.197.31.3 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="ND7AD0lX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=9+ 6KLnho3oezstBwK6DjSQ5GUBc1NeDmONzGfJ6sfTU=; b=ND7AD0lX3Pw6CblwZR ne7nKzAjfBi6+VNGaDazXYvQ2e/OV8wq2YI/gr5wnKvO6Deoxmyw3RcL9CDvKEAY 984cD5HEAgPcOEsRcoNUG8NGtOJuupYBnu2HaWK4sNYfs4QWdZfkn+5SZra5iJFa QicBpYy42Xf5N6kzCLl4Rhsa4= Received: from czl-ubuntu-pc.. (unknown []) by gzga-smtp-mtada-g0-3 (Coremail) with SMTP id _____wD3_7Q6Hx1qsb0MAw--.47504S6; Mon, 01 Jun 2026 13:57:22 +0800 (CST) From: Chi Zhiling To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , Hugh Dickins , Baolin Wang , Chi Zhiling Subject: [PATCH v2 4/5] mm/shmem: remove page-copy fallback in shmem read path Date: Mon, 1 Jun 2026 13:57:03 +0800 Message-ID: <20260601055704.167436-5-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260601055704.167436-1-chizhiling@163.com> References: <20260601055704.167436-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wD3_7Q6Hx1qsb0MAw--.47504S6 X-Coremail-Antispam: 1Uf129KBjvJXoWxAF1DWF1xCFW7Kr17Wr17ZFb_yoWrCw4xpF W3KasIyrWxJw43Gr17JFs8Za4rK3s3JFWUJFyfG3WfA3Z3tryrKFyUta4F9FWfJ34UC3WI gw4qyr1UWa1UAr7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jeE_NUUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC+AK5V2odH0JEmQAA3j Content-Type: text/plain; charset="utf-8" From: Chi Zhiling This is perp patch for folio batch support which removing the fallback to page-copy mode from shmem reads. The existing hwpoison fallback would require fetching the same folio multiple times to read its each pages, which complicates the batching model. Instead, move hwpoison handling into copy_each_pages_to_iter(), allowing the error path to be centralized. This simplifies the shmem fast read path and avoids special-case handling that conflicts with folio batching support. Signed-off-by: Chi Zhiling --- mm/shmem.c | 63 +++++++++++++++++++++++++++++------------------------- 1 file changed, 34 insertions(+), 29 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index b08118fbd275..cac355685e49 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -3353,6 +3353,30 @@ static size_t copy_zero_to_iter(size_t bytes, struct= iov_iter *i) return written; } =20 +static size_t copy_pages_to_iter(struct folio *folio, size_t offset, + size_t bytes, struct iov_iter *i, int *error) +{ + struct page *page; + unsigned long off, nr, ret, written =3D 0; + + do { + page =3D folio_page(folio, offset >> PAGE_SHIFT); + if (PageHWPoison(page)) { + *error =3D -EIO; + break; + } + + off =3D offset_in_page(offset); + nr =3D min(PAGE_SIZE - off, bytes - written); + + ret =3D copy_page_to_iter(page, off, nr, i); + offset +=3D ret; + written +=3D ret; + } while (written < bytes && ret =3D=3D nr); + + return written; +} + static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *t= o) { struct file *file =3D iocb->ki_filp; @@ -3365,10 +3389,8 @@ static ssize_t shmem_file_read_iter(struct kiocb *io= cb, struct iov_iter *to) =20 for (;;) { struct folio *folio =3D NULL; - struct page *page =3D NULL; unsigned long nr, ret; loff_t end_offset, i_size =3D i_size_read(inode); - bool fallback_page_copy =3D false; size_t fsize; =20 if (unlikely(iocb->ki_pos >=3D i_size)) @@ -3376,25 +3398,13 @@ static ssize_t shmem_file_read_iter(struct kiocb *i= ocb, struct iov_iter *to) =20 index =3D iocb->ki_pos >> PAGE_SHIFT; error =3D shmem_get_folio(inode, index, 0, &folio, SGP_READ); + if (folio) + folio_unlock(folio); if (error) { if (error =3D=3D -EINVAL) error =3D 0; break; } - if (folio) { - folio_unlock(folio); - - page =3D folio_file_page(folio, index); - if (PageHWPoison(page)) { - folio_put(folio); - error =3D -EIO; - break; - } - - if (folio_test_large(folio) && - folio_test_has_hwpoisoned(folio)) - fallback_page_copy =3D true; - } =20 /* * We must evaluate after, since reads (unlike writes) @@ -3406,12 +3416,9 @@ static ssize_t shmem_file_read_iter(struct kiocb *io= cb, struct iov_iter *to) folio_put(folio); break; } - end_offset =3D min_t(loff_t, i_size, iocb->ki_pos + to->count); - if (folio && likely(!fallback_page_copy)) - fsize =3D folio_size(folio); - else - fsize =3D PAGE_SIZE; + fsize =3D folio ? folio_size(folio) : PAGE_SIZE; offset =3D iocb->ki_pos & (fsize - 1); + end_offset =3D min_t(loff_t, i_size, iocb->ki_pos + to->count); nr =3D min_t(loff_t, end_offset - iocb->ki_pos, fsize - offset); =20 if (folio) { @@ -3420,12 +3427,8 @@ static ssize_t shmem_file_read_iter(struct kiocb *io= cb, struct iov_iter *to) * virtual addresses, take care about potential aliasing * before reading the page on the kernel side. */ - if (mapping_writably_mapped(mapping)) { - if (likely(!fallback_page_copy)) - flush_dcache_folio(folio); - else - flush_dcache_page(page); - } + if (mapping_writably_mapped(mapping)) + flush_dcache_folio(folio); =20 /* * Mark the folio accessed if we read the beginning. @@ -3436,10 +3439,10 @@ static ssize_t shmem_file_read_iter(struct kiocb *i= ocb, struct iov_iter *to) * Ok, we have the page, and it's up-to-date, so * now we can copy it to user space... */ - if (likely(!fallback_page_copy)) + if (likely(!folio_contain_hwpoisoned_page(folio))) ret =3D copy_folio_to_iter(folio, offset, nr, to); else - ret =3D copy_page_to_iter(page, offset, nr, to); + ret =3D copy_pages_to_iter(folio, offset, nr, to, &error); folio_put(folio); } else if (user_backed_iter(to)) { /* @@ -3462,6 +3465,8 @@ static ssize_t shmem_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) =20 if (!iov_iter_count(to)) break; + if (unlikely(error)) + break; if (ret < nr) { error =3D -EFAULT; break; --=20 2.43.0 From nobody Mon Jun 8 07:23:00 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0D73345CA5; Mon, 1 Jun 2026 05:58:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.3 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293494; cv=none; b=tHS3xVgMFimyTt+1+RGxGNg7hdOYM+IiRqPwTobXiP5GQYAtkTYCf2xL0kNozpYT5dWlyOE4vqisIWSBJCOwSEDwYJh3/PCINRUF8AK1PsqDsVds0HebzD3lOO5yl5oNDPkZfYsyMWcoBJtNPZZT8ywW7IiZGmCXQ3qffcMOrBs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780293494; c=relaxed/simple; bh=Posza5aSe4A/ZDQYZQwTpCrA94Vi1sSARixQYgqr0RQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=d3VdPtAdxluMxECAGeyq84j1HU9KFmJyupVOiS8GpI27Uo79GXe16HuUmEAnWc5B0/+6w7FQr3LZOsJb0x83C6OrewAOiZJ/X+wJ2Ipvp+Ha2R6i/yzSo/HJPjYTBZysz6slUcmUVnB0HH05H6gRzS4ybVE2crOXYhO3MGjsfM0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=BvOaqovQ; arc=none smtp.client-ip=117.135.210.3 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="BvOaqovQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=uD FqU2w9pHRcm8QTKIbB6nvNnwHJfSBH479J/vtIl7g=; b=BvOaqovQ3mE7YeTvEm gYj756jset545UWabWC0ifzfxwm6IYv7JKAQM/28XgmBs/VEo4LD0uJXLrTp/UUp mKnKyZeI49ZYbFSsOKunpFrwGO5dGqrJa1tNPg6/NcS38JtNUPA/7uq74CzckZrS U53BRo2AJFTM9E26kwN0iPxEg= Received: from czl-ubuntu-pc.. (unknown []) by gzga-smtp-mtada-g0-3 (Coremail) with SMTP id _____wD3_7Q6Hx1qsb0MAw--.47504S7; Mon, 01 Jun 2026 13:57:23 +0800 (CST) From: Chi Zhiling To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , Hugh Dickins , Baolin Wang , Chi Zhiling Subject: [PATCH v2 5/5] mm/shmem: optimize file read with folio batching Date: Mon, 1 Jun 2026 13:57:04 +0800 Message-ID: <20260601055704.167436-6-chizhiling@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260601055704.167436-1-chizhiling@163.com> References: <20260601055704.167436-1-chizhiling@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wD3_7Q6Hx1qsb0MAw--.47504S7 X-Coremail-Antispam: 1Uf129KBjvJXoWxuF1xKF18Gr1rAFWUuFy3Jwb_yoWrZr17pF W3KasxKrZ7G3y3Wr1fJF18Z3WrK3sayFW5Ja4fG3W3A3W5Jas8KF1xt34YgFW5JrZrC3WI gw1vkr1UW3WjvFJanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jeuWdUUUUU= X-CM-SenderInfo: hfkl6xxlol0wi6rwjhhfrp/xtbC2wO5V2odH0Nu4gAA3R Content-Type: text/plain; charset="utf-8" From: Chi Zhiling Optimize shmem file read by using filemap_get_folios_contig() to batch fetch contiguous folios from the page cache, reducing the overhead of repeated shmem_get_folio() calls. This patch checks the uptodate flag without holding the folio lock, so it may observe a non-uptodate state on a locked folio that is still being initialized. This is safe because only zero-filled data can be copied to the user buffer in that scenario. A non-uptodate folio in the swap cache cannot be added to the shmem page cache. This creates a semantic conflict, as shmem zeroes the folio out, but the swap cache would fill it by reading from the swap backing store. Signed-off-by: Chi Zhiling --- mm/shmem.c | 57 ++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index cac355685e49..61937582f08c 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -891,6 +891,14 @@ int shmem_add_to_page_cache(struct folio *folio, VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(!folio_test_swapbacked(folio), folio); =20 + /* + * Don't add a non-uptodate folio that is in swap cache to page + * cache, since shmem will zero it instead of reading from swap + * backing. + */ + VM_BUG_ON_FOLIO(folio_test_swapcache(folio) && + !folio_test_uptodate(folio), folio); + folio_ref_add(folio, nr); folio->mapping =3D mapping; folio->index =3D index; @@ -3382,11 +3390,13 @@ static ssize_t shmem_file_read_iter(struct kiocb *i= ocb, struct iov_iter *to) struct file *file =3D iocb->ki_filp; struct inode *inode =3D file_inode(file); struct address_space *mapping =3D inode->i_mapping; - pgoff_t index; + struct folio_batch fbatch; unsigned long offset; int error =3D 0; ssize_t retval =3D 0; =20 + folio_batch_init(&fbatch); + for (;;) { struct folio *folio =3D NULL; unsigned long nr, ret; @@ -3395,15 +3405,33 @@ static ssize_t shmem_file_read_iter(struct kiocb *i= ocb, struct iov_iter *to) =20 if (unlikely(iocb->ki_pos >=3D i_size)) break; +fetch: + folio =3D folio_batch_next(&fbatch); + if (!folio) { + pgoff_t start =3D iocb->ki_pos >> PAGE_SHIFT; + pgoff_t end =3D (iocb->ki_pos + to->count - 1) >> PAGE_SHIFT; + + if (folio_batch_count(&fbatch)) { + for (int i =3D 0; i < folio_batch_count(&fbatch); i++) + folio_put(fbatch.folios[i]); + folio_batch_reinit(&fbatch); + } =20 - index =3D iocb->ki_pos >> PAGE_SHIFT; - error =3D shmem_get_folio(inode, index, 0, &folio, SGP_READ); - if (folio) - folio_unlock(folio); - if (error) { - if (error =3D=3D -EINVAL) - error =3D 0; - break; + filemap_get_folios_contig(inode->i_mapping, &start, end, &fbatch); + if (folio_batch_count(&fbatch)) + goto fetch; + + error =3D shmem_get_folio(inode, start, 0, &folio, SGP_READ); + if (unlikely(error)) { + if (error =3D=3D -EINVAL) + error =3D 0; + break; + } + if (folio) { + folio_unlock(folio); + folio_batch_add(&fbatch, folio); + fbatch.i++; + } } =20 /* @@ -3411,17 +3439,15 @@ static ssize_t shmem_file_read_iter(struct kiocb *i= ocb, struct iov_iter *to) * are called without i_rwsem protection against truncate */ i_size =3D i_size_read(inode); - if (unlikely(iocb->ki_pos >=3D i_size)) { - if (folio) - folio_put(folio); + if (unlikely(iocb->ki_pos >=3D i_size)) break; - } + fsize =3D folio ? folio_size(folio) : PAGE_SIZE; offset =3D iocb->ki_pos & (fsize - 1); end_offset =3D min_t(loff_t, i_size, iocb->ki_pos + to->count); nr =3D min_t(loff_t, end_offset - iocb->ki_pos, fsize - offset); =20 - if (folio) { + if (folio && folio_test_uptodate(folio)) { /* * If users can be writing to this page using arbitrary * virtual addresses, take care about potential aliasing @@ -3443,7 +3469,6 @@ static ssize_t shmem_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) ret =3D copy_folio_to_iter(folio, offset, nr, to); else ret =3D copy_pages_to_iter(folio, offset, nr, to, &error); - folio_put(folio); } else if (user_backed_iter(to)) { /* * Copy to user tends to be so well optimized, but @@ -3474,6 +3499,8 @@ static ssize_t shmem_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) cond_resched(); } =20 + for (int i =3D 0; i < folio_batch_count(&fbatch); i++) + folio_put(fbatch.folios[i]); file_accessed(file); return retval ? retval : error; } --=20 2.43.0