From nobody Sat Jul 25 03:20:30 2026 Received: from outbound.qs.icloud.com (qs-2001l-snip4-2.eps.apple.com [57.103.87.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A5142BE7BE for ; Mon, 20 Jul 2026 05:08:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=57.103.87.115 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524081; cv=none; b=KJO98LhP1/EFYY2gYBytcsz8Gg8EmKq3DKNYVClY7Azqyd0pGTXgvcYdg/qQZ3rwt1nfpRPPA0E6Pz1iShAlhWjGNyQlTXDbbeexI5+dq3ShUqXtgFARKEJtDpdydUKl1pMZGkGvVEq2nFIWmehMMBWZL8Ahl6NswjWybmv/bxw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524081; c=relaxed/simple; bh=/o7xQp7LYCOHZ22PnOkQZVfltgn9zHZheEocHYd6IiQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=UcjB4ua/oJnD4EgGZqb2U7bR84CPLr0QU9Nvv0jZxUq8ev1UCRHgJ3abAiX1p/a+DovsFVHH7620xVRUVrofYXW6PggYplpMl0V96YE2P/OcT+YNsL1G2XP2OBom0m/fQ2I64d+zC1lC6Q5LMTUtcjukB3cnFpWnKqYe2LENibY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com; spf=pass smtp.mailfrom=icloud.com; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b=nElY1mpx; arc=none smtp.client-ip=57.103.87.115 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=icloud.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b="nElY1mpx" Received: from outbound.qs.icloud.com (unknown [127.0.0.2]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPS id 8CAC7180014C; Mon, 20 Jul 2026 05:07:57 +0000 (UTC) X-ICL-RepId: 019f7dec-79d8-73c7-91f6-a38505208ad8 X-ICL-Out-Info: HUtFAUMEWwJACUgBTUQeDx5WFlZNRAJCTQhMHVwGXRxCCkEdXgBLVxQEAlodRw5AHVYWWAhOK1sTVRdGCRkIXR0ZHldQXgheH0wcHQ5YBhICWkUBXRcDVxxWRVwYQwldBVccHRxERVsTVRdGCRkIXR0ZCEcfCjADQg5WA0MHRQAtGRxXUF4IXh9MHB0OWAYSHVAcDlEFWwBGCU8BXRoJUwRaEB4ZWwkfFlUNQAUaHQddCFVXDw5fAREJHAMJAQlyGVoUXBhTRVEfVEYTGU4bV01QG18CQg8= Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1784524079; x=1787116079; bh=10j7WY6pZYOXAZEZpweX4crZCY4OfUrYCS2F0Vpwmv8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:x-icloud-hme; b=nElY1mpxXvsXBRr9n2BLhYUsrVAAyAUFZDjRYKE4tBuUC86UqF/ub2746q6jG/Gbt3YUkR3OcTQ1MVojpEkXA6X2uOc3U5AUD5LI3QffBj2DnV20Od761mWaBccX3sVzSA+bv6Vj/m+QITqn/o+bQp3uHR8DgIQJp8/meSBprIiVDykwFgv7IwxOj7vScaz1e08J+v/xB4l7LFCqrNgIBCmOCHWXF14V2PQftjiT7MJND/XPaC0DwbCMnrpgG3kqgbO8tENM6qYTFpkXf+IALyisXCPYM773gAYJDhpDZ9oAsTbx0O/T7ub0BdO3Ilo8oVOwJEOzKlRmDQbzB2fqLw== Received: from [21.6.122.162] (unknown [17.57.155.37]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPSA id 16CE0180010D; Mon, 20 Jul 2026 05:07:50 +0000 (UTC) From: Zhang Peng Date: Mon, 20 Jul 2026 13:07:38 +0800 Subject: [PATCH v5 1/5] mm/vmscan: introduce folio_activate_locked() helper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260720-batch-tlb-flush-v5-1-db943a0d0d6b@icloud.com> References: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> In-Reply-To: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Johannes Weiner , Shakeel Butt , Axel Rasmussen , Yuanchu Xie , Wei Xu , Michal Hocko , Qi Zheng , "Liam R. Howlett" , Qi Zheng Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Barry Song , Kairui Song , Zhang Peng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784524064; l=2344; i=zippermonkey@icloud.com; s=20260309; h=from:subject:message-id; bh=/o7xQp7LYCOHZ22PnOkQZVfltgn9zHZheEocHYd6IiQ=; b=rhn1vXifhzIq4xrjZIPPTD8K48FziRZqJrzYtujzYoZ7k1ac5K55GLoD6NWFWzwaHpknPJXeC wB2/2OcCH3RBJLZ5LdN3ftGi21PhW2czvgmaUeTa/RP8/Z+OSFdFa2e X-Developer-Key: i=zippermonkey@icloud.com; a=ed25519; pk=tPCLpFnBfIyHsp0k7eaUTUREEa36bQNW/69X+NS8wBU= X-Proofpoint-GUID: hV3P6CG3dzB9ygQveRXF6_8kY3ngsOKF X-Proofpoint-ORIG-GUID: hV3P6CG3dzB9ygQveRXF6_8kY3ngsOKF X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNzIwMDA1NCBTYWx0ZWRfX9TfIUtfr4Y/6 0W0xMzbYYVkmZ4Z/NcTqTvdfq3seAKfGJDpyXgWJ2EdH8QJnCUn0SJPdvL9jTafxgzAowXNF5+Z JrX5b9buGldQ38R767BQRAAQraU8qmn7Nz9GfsVSOsZQXOxB635t5kYV5IKYxJvM6CX9QbtopML cuuETdFy9BtmPU1OfTaWkbJY/yam8vbDI+hyxe21BHMS0Y/f+EcFe0qFAM9aOLevQqeo0Daea2S tU34N1urIidECC8pGfm4yzaLTJaUWUV6Cm8ETiHQFCQPBz52OqUZb0saZcJpPOXvS4Zgl4r0dRK blNl9y+tPDXd/SSW30S The activate_locked label in shrink_folio_list() reclaims swap cache when needed, marks the folio active, and updates activation statistics. Extract this block into folio_activate_locked() so it can be reused. No functional change. Signed-off-by: Zhang Peng Suggested-by: Kairui Song --- mm/vmscan.c | 38 +++++++++++++++++++++++++++----------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index ca4533eba701..5ba880dce21e 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1050,6 +1050,32 @@ static bool may_enter_fs(struct folio *folio, gfp_t = gfp_mask) return !data_race(folio_swap_flags(folio) & SWP_FS_OPS); } =20 +/* + * Prepare a locked folio to be kept active rather than reclaimed. + * Reclaims its swap slot if it will not be swapped, then marks it + * active and updates activation statistics. + */ +static void folio_activate_locked(struct folio *folio, + struct reclaim_stat *stat) +{ + unsigned int nr_pages =3D folio_nr_pages(folio); + + VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio); + VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), folio); + + /* Not a candidate for swapping, so reclaim swap space. */ + if (folio_test_swapcache(folio) && + (mem_cgroup_swap_full(folio) || folio_test_mlocked(folio))) + folio_free_swap(folio); + if (!folio_test_mlocked(folio)) { + int type =3D folio_is_file_lru(folio); + + folio_set_active(folio); + stat->nr_activate[type] +=3D nr_pages; + count_memcg_folio_events(folio, PGACTIVATE, nr_pages); + } +} + /* * shrink_folio_list() returns the number of reclaimed pages */ @@ -1525,17 +1551,7 @@ static unsigned int shrink_folio_list(struct list_he= ad *folio_list, nr_pages =3D 1; } activate_locked: - /* Not a candidate for swapping, so reclaim swap space. */ - if (folio_test_swapcache(folio) && - (mem_cgroup_swap_full(folio) || folio_test_mlocked(folio))) - folio_free_swap(folio); - VM_BUG_ON_FOLIO(folio_test_active(folio), folio); - if (!folio_test_mlocked(folio)) { - int type =3D folio_is_file_lru(folio); - folio_set_active(folio); - stat->nr_activate[type] +=3D nr_pages; - count_memcg_folio_events(folio, PGACTIVATE, nr_pages); - } + folio_activate_locked(folio, stat); keep_locked: folio_unlock(folio); keep: --=20 2.43.7 From nobody Sat Jul 25 03:20:30 2026 Received: from outbound.qs.icloud.com (qs-2001g-snip4-6.eps.apple.com [57.103.87.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F84E396588 for ; Mon, 20 Jul 2026 05:08:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=57.103.87.69 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524087; cv=none; b=MsPYYxxLhI/tE36/s7IJxBEJMOKLarBr7btOPFmf/gr0dLgddpIIIj9ZHzl5lcd7atvcmMJN+/dMEoQm+Qew1LzPq6RdOICWAqZnSXj6e0FkjNWAGSq+eI6n/GKswhJFt8XJH6S//B0r7YjgFpkR52NHk8t7amyR15W/uyhjvco= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524087; c=relaxed/simple; bh=3E2ywKByDNomV6rtuYwrWIJ7KWxsBoMRkdyyYPY6J20=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QfxXQLy+rKmGDC4RvrZvIeqaI9l8J03bYLxNUlsbP+GNsgGyLgpi/CcAbliV/m0HW7XfUJXibZWiltHko7L5Q3BumJ+ussryeOJaAlqIq5VSIdrDNLD62WMP1v7UMih/40pr9HsqqrAVBrARW6BS/LuwhuY/N36ebLTBIsfEbYs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com; spf=pass smtp.mailfrom=icloud.com; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b=1LuCG8cT; arc=none smtp.client-ip=57.103.87.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=icloud.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b="1LuCG8cT" Received: from outbound.qs.icloud.com (unknown [127.0.0.2]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPS id 2B5771800142; Mon, 20 Jul 2026 05:08:02 +0000 (UTC) X-ICL-RepId: 019f7dec-8f5b-7282-86be-ea88f13742cf X-ICL-Out-Info: HUtFAUMEWwJACUgBTUQeDx5WFlZNRAJCTQhMHVwGXRxCCkEdXgBLVxQEAlodRw5AHVYWWAhOK1sTVRdGCRkIXR0ZHldQXgheH0wcHQ5YBhICWkUBXRcDVxxWRVwYQwldBVccHRxERVsTVRdGCRkIXR0ZCEcfCjADQg5WA0MHRQAtGRxXUF4IXh9MHB0OWAYSHVAcDlEFWwBGCU8BXRoJUwRaEB4ZWwkfFlUNQAUaHQddC1VXDw5fAREJHAMJAQlyGVoUXBhTRVEfVEYTGU4bV01QG18CQg8= Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1784524085; x=1787116085; bh=r8aZQYm37v3YWegSE0f/522iUhXwfG0dV8XiPNn7/zA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:x-icloud-hme; b=1LuCG8cTW02fWgPcouVYMSt/nzspcCwjz8cBq9KFVAXIBqTng9tCvRTn2dCAnZzYQFsu8X2ti+c3ZnV9NtPM+N7T25BeOEs2Tudu8un+4Zjos0vfjGlyzgLiYAk2rG1tCHBraDZzKpWBv8on5BivMhdm66hbpF67YFCZb8vDd8gR5eykWimuxJ/DJrI2iQ+xnuWbvzeNPJpat6UbVTHrkAX+8vviAtWQ54Y4KA6BiJTJYaiIGmENrWF0TQyFT5jvdYuHY6FhF0xDU4MAWXrHKYSKOlJyLDh06clYHN/LMDblWVjCdarKt4ARPPn5vGCti3y/e641H3iLHoukXxm8pw== Received: from [21.6.122.162] (unknown [17.57.155.37]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPSA id A8A6A180014F; Mon, 20 Jul 2026 05:07:57 +0000 (UTC) From: Zhang Peng Date: Mon, 20 Jul 2026 13:07:39 +0800 Subject: [PATCH v5 2/5] mm/vmscan: extract folio_free() from shrink_folio_list() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260720-batch-tlb-flush-v5-2-db943a0d0d6b@icloud.com> References: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> In-Reply-To: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Johannes Weiner , Shakeel Butt , Axel Rasmussen , Yuanchu Xie , Wei Xu , Michal Hocko , Qi Zheng , "Liam R. Howlett" , Qi Zheng Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Barry Song , Kairui Song , Zhang Peng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784524064; l=6784; i=zippermonkey@icloud.com; s=20260309; h=from:subject:message-id; bh=3E2ywKByDNomV6rtuYwrWIJ7KWxsBoMRkdyyYPY6J20=; b=xwLl6qIZQKNcuhsChppd92HF64KnsIhpGCL3RSjuWjDbfuuywcAeINyOZGLuqw21yHt4lJDb3 pmdrUVoASFpB7EEMrB8fI9pnl2tna6V9T/qRQxbTy923iTqsnu9AQp5 X-Developer-Key: i=zippermonkey@icloud.com; a=ed25519; pk=tPCLpFnBfIyHsp0k7eaUTUREEa36bQNW/69X+NS8wBU= X-Proofpoint-GUID: S0J1xPK0SItHssd-vov55hLNs4bFa7L1 X-Proofpoint-ORIG-GUID: S0J1xPK0SItHssd-vov55hLNs4bFa7L1 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNzIwMDA1NCBTYWx0ZWRfX4w7MkmROKZPX ShHbKgcSO4k4EoMq9RR4TnRAan8uFln4NGYmXZUbrCC8edrwzkW7A+e6EjmB7r/nsllO0aMVBmt BGM85RqenWX1Ak0Ugjd8tH57hIGC/Q7bsa9ahUuRG+leIagFrP3YxLl5NZICMtjKZARYLqJqqAn NdwubE7x2XEVovL4nvUvomWRLsKyknQ1KEaddr/z2LVup4uts9L5GCl4UUg3rqd2RJuqMO/GIcP zbQDNG/J/Rb5DNO9At2fBazqvPAlQ/v96i+oES0BqJkPMvqLCjOnfcn5LMuODdxu7t+Q0weuyUO Hyozsts2ySvEPNUhM5V shrink_folio_list() contains a self-contained folio-freeing section: buffer release, lazyfree, __remove_mapping, and folio_batch drain. Extract it into folio_free() to reduce the size of shrink_folio_list() and make the freeing step independently readable. No functional change. Signed-off-by: Zhang Peng Suggested-by: Kairui Song --- mm/vmscan.c | 164 +++++++++++++++++++++++++++++++++-----------------------= ---- 1 file changed, 89 insertions(+), 75 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 5ba880dce21e..a0807dd01c5a 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1076,6 +1076,93 @@ static void folio_activate_locked(struct folio *foli= o, } } =20 +static bool folio_try_reclaim_free(struct folio *folio, + struct folio_batch *free_folios, + struct scan_control *sc, struct reclaim_stat *stat, + unsigned int *nr_reclaimed) +{ + const unsigned int nr_pages =3D folio_nr_pages(folio); + struct address_space *mapping =3D folio_mapping(folio); + + /* + * If the folio has buffers, try to free the buffer mappings + * associated with this folio. If we succeed we try to free + * the folio as well. + * + * We do this even if the folio is dirty. + * filemap_release_folio() does not perform I/O, but it is + * possible for a folio to have the dirty flag set, but it + * is actually clean (all its buffers are clean). This + * happens if the buffers were written out directly, with + * submit_bh(). ext3 will do this, as well as the blockdev + * mapping. filemap_release_folio() will discover that + * cleanness and will drop the buffers and mark the folio + * clean - it can be freed. + * + * Rarely, folios can have buffers and no ->mapping. These + * are the folios which were not successfully invalidated in + * truncate_cleanup_folio(). We try to drop those buffers + * here and if that worked, and the folio is no longer + * mapped into process address space (refcount =3D=3D 1) it can + * be freed. Otherwise, leave the folio on the LRU so it is + * swappable. + */ + if (folio_needs_release(folio)) { + if (!filemap_release_folio(folio, sc->gfp_mask)) { + folio_activate_locked(folio, stat); + return false; + } + + if (!mapping && folio_ref_count(folio) =3D=3D 1) { + folio_unlock(folio); + if (folio_put_testzero(folio)) + goto free_it; + else { + /* + * rare race with speculative reference. + * the speculative reference will free + * this folio shortly, so we may + * increment nr_reclaimed here (and + * leave it off the LRU). + */ + *nr_reclaimed +=3D nr_pages; + return true; + } + } + } + + if (folio_test_lazyfree(folio)) { + /* follow __remove_mapping for reference */ + if (!folio_ref_freeze(folio, 1)) + return false; + /* + * The folio has only one reference left, which is + * from the isolation. After the caller puts the + * folio back on the lru and drops the reference, the + * folio will be freed anyway. It doesn't matter + * which lru it goes on. So we don't bother checking + * the dirty flag here. + */ + count_vm_events(PGLAZYFREED, nr_pages); + count_memcg_folio_events(folio, PGLAZYFREED, nr_pages); + } else if (!mapping || !__remove_mapping(mapping, folio, true, + sc->target_mem_cgroup)) + return false; + + folio_unlock(folio); +free_it: + VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio), folio); + *nr_reclaimed +=3D nr_pages; + + folio_unqueue_deferred_split(folio); + if (folio_batch_add(free_folios, folio) =3D=3D 0) { + mem_cgroup_uncharge_folios(free_folios); + try_to_unmap_flush(); + free_unref_folios(free_folios); + } + return true; +} + /* * shrink_folio_list() returns the number of reclaimed pages */ @@ -1463,82 +1550,9 @@ static unsigned int shrink_folio_list(struct list_he= ad *folio_list, } } =20 - /* - * If the folio has buffers, try to free the buffer - * mappings associated with this folio. If we succeed - * we try to free the folio as well. - * - * We do this even if the folio is dirty. - * filemap_release_folio() does not perform I/O, but it - * is possible for a folio to have the dirty flag set, - * but it is actually clean (all its buffers are clean). - * This happens if the buffers were written out directly, - * with submit_bh(). ext3 will do this, as well as - * the blockdev mapping. filemap_release_folio() will - * discover that cleanness and will drop the buffers - * and mark the folio clean - it can be freed. - * - * Rarely, folios can have buffers and no ->mapping. - * These are the folios which were not successfully - * invalidated in truncate_cleanup_folio(). We try to - * drop those buffers here and if that worked, and the - * folio is no longer mapped into process address space - * (refcount =3D=3D 1) it can be freed. Otherwise, leave - * the folio on the LRU so it is swappable. - */ - if (folio_needs_release(folio)) { - if (!filemap_release_folio(folio, sc->gfp_mask)) - goto activate_locked; - if (!mapping && folio_ref_count(folio) =3D=3D 1) { - folio_unlock(folio); - if (folio_put_testzero(folio)) - goto free_it; - else { - /* - * rare race with speculative reference. - * the speculative reference will free - * this folio shortly, so we may - * increment nr_reclaimed here (and - * leave it off the LRU). - */ - nr_reclaimed +=3D nr_pages; - continue; - } - } - } - - if (folio_test_lazyfree(folio)) { - /* follow __remove_mapping for reference */ - if (!folio_ref_freeze(folio, 1)) - goto keep_locked; - /* - * The folio has only one reference left, which is - * from the isolation. After the caller puts the - * folio back on the lru and drops the reference, the - * folio will be freed anyway. It doesn't matter - * which lru it goes on. So we don't bother checking - * the dirty flag here. - */ - count_vm_events(PGLAZYFREED, nr_pages); - count_memcg_folio_events(folio, PGLAZYFREED, nr_pages); - } else if (!mapping || !__remove_mapping(mapping, folio, true, - sc->target_mem_cgroup)) + if (!folio_try_reclaim_free(folio, &free_folios, sc, stat, + &nr_reclaimed)) goto keep_locked; - - folio_unlock(folio); -free_it: - /* - * Folio may get swapped out as a whole, need to account - * all pages in it. - */ - nr_reclaimed +=3D nr_pages; - - folio_unqueue_deferred_split(folio); - if (folio_batch_add(&free_folios, folio) =3D=3D 0) { - mem_cgroup_uncharge_folios(&free_folios); - try_to_unmap_flush(); - free_unref_folios(&free_folios); - } continue; =20 activate_locked_split: --=20 2.43.7 From nobody Sat Jul 25 03:20:30 2026 Received: from outbound.qs.icloud.com (qs-2006h-snip4-11.eps.apple.com [57.103.85.211]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1F872D97BB for ; Mon, 20 Jul 2026 05:08:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=57.103.85.211 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524095; cv=none; b=IMmOIJqA9iTN5OXnJmnrcVxs/BNqxA+IBtty3emb3CaPeaLH2TJZp7ae7FViQQDEpQtQaLhju9kFJJXr5AhUebCzTKobNVuo7p+ybAZTmvlKiY2aa4MU1+MmhlLqDoLZlbp5vl8To3A0gXRyOqvvsHHJsb7BixjT7nAKC0T0vrk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524095; c=relaxed/simple; bh=l+BMBCV4bE1SQ4xqKdkiAu5ODtsRbK+KeBkmi8ZvYtM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Nc2KhKytxOXVHWQ67G8EHYrNRC5MLiHFgj1mDw3qBS3b/gIA6wm/4dDYZaPJuakFmm7I3pRy8s01+sK6I8w8X2H9Lxq0ElE1JzFvZU0KSbfoJ4L1Xo0eTrbpjJWMgEaLeFg3i4eM1VjBpnYpir3upHfR1CdtJ06BXnyFE7YuZrE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com; spf=pass smtp.mailfrom=icloud.com; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b=ZH6PEq/n; arc=none smtp.client-ip=57.103.85.211 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=icloud.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b="ZH6PEq/n" Received: from outbound.qs.icloud.com (unknown [127.0.0.2]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPS id BCFAA1800094; Mon, 20 Jul 2026 05:08:09 +0000 (UTC) X-ICL-RepId: 019f7dec-a959-7740-851f-8b0f8f71ecbf X-ICL-Out-Info: HUtFAUMEWwJACUgBTUQeDx5WFlZNRAJCTQhMHVwGXRxCCkEdXgBLVxQEAlodRw5AHVYWWAhOK1sTVRdGCRkIXR0ZHldQXgheH0wcHQ5YBhICWkUBXRcDVxxWRVwYQwldBVccHRxERVsTVRdGCRkIXR0ZCEcfCjADQg5WA0MHRQAtGRxXUF4IXh9MHB0OWAYSHVAcDlEFWwBGCU8BXRoJUwRaEB4ZWwkfFlUNQAUaHQddClVXDw5fAREJHAMJAQlyGVoUXBhTRVEfVEYTGU4bV01QG18CQg8= Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1784524092; x=1787116092; bh=EsyP5irdsQmvLw/lSwMp6yzpUiFqRmMgjYO2e0T/6PY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:x-icloud-hme; b=ZH6PEq/n7EF6ttWD+sQmQGhBz67cZwgnvFO7VlmkFe57n+8Zwmlr5EphsZODxmOl8hYKo/ykWEKipDaqDchQFniG0jB5oHmHDuVH8035kurVuSXwMLJeWrhrVHV6IpOe0GBLB1zu5Yf4E3Z5YC8BuFWkhbvvbu1Qvn7m8jGxW9X7eo8pn07GkBCDPtDIkV5jhnLzKMBK5BJnlJsZGW/38bj5lrVMiO/r/PO6sYlDF81Riw4XhzOZ6HP7cPHAbANaYLBAeJeCCYCHQoppdGsFaxYJGb2Zd0AUSuzo5BaX1beIYuC4XZZGmP8Th0cd4cwIXo5jc0ntSa9OlyvwUUTIXA== Received: from [21.6.122.162] (unknown [17.57.155.37]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPSA id 30CDB1800153; Mon, 20 Jul 2026 05:08:02 +0000 (UTC) From: Zhang Peng Date: Mon, 20 Jul 2026 13:07:40 +0800 Subject: [PATCH v5 3/5] mm/vmscan: extract pageout_one() from shrink_folio_list() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260720-batch-tlb-flush-v5-3-db943a0d0d6b@icloud.com> References: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> In-Reply-To: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Johannes Weiner , Shakeel Butt , Axel Rasmussen , Yuanchu Xie , Wei Xu , Michal Hocko , Qi Zheng , "Liam R. Howlett" , Qi Zheng Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Barry Song , Kairui Song , Zhang Peng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784524064; l=4293; i=zippermonkey@icloud.com; s=20260309; h=from:subject:message-id; bh=l+BMBCV4bE1SQ4xqKdkiAu5ODtsRbK+KeBkmi8ZvYtM=; b=s9/X+XJmXWZJHCGfES1UArzwc9PYINiT2yqXEOrpNsPIR8ow12vpoftGTs9JFz4tFXW8/BwWh Rdf4e32+7epDNexK7cWPfQxaFy8jTAF1BtJ+voGRpU+DXH6ZElLoy+i X-Developer-Key: i=zippermonkey@icloud.com; a=ed25519; pk=tPCLpFnBfIyHsp0k7eaUTUREEa36bQNW/69X+NS8wBU= X-Proofpoint-GUID: N5T4eMakfuls9gzg2ocoGR5sco53UcYa X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNzIwMDA1NCBTYWx0ZWRfXzyk1OZOlY0jB PXGCZLeXDDRdTfmZZb89tH6u6ihldkVUk3XA+9PUHITim49G6I5DisTizDnJt1XQwxfE00KxJQQ NjdVL+1c1HNTLNj+SybrR9M96ixdk+/iKxcyUWKNK5B8ViNSoXmENTqXbjUN3c3eukZ42BQaqw0 R7GOI4ds7bNWKPSoNPFN5jpKCW5AIWAPG55crnVrbs43W6CuBnhKu99QQXtZtit3cFBSZzAEjgB dQGmfGSo41Oena+wmZMQOwGAcygvjBNAL67xefp8/HO2bQEk5/olPI8h6LiPkhoVUTo1VzFX+5B uIktOytflSG50jhLTJB X-Proofpoint-ORIG-GUID: N5T4eMakfuls9gzg2ocoGR5sco53UcYa shrink_folio_list() contains a self-contained pageout() dispatch state machine. Extract it into pageout_one() to reduce the size of shrink_folio_list() and make the pageout step independently readable. No functional change. Signed-off-by: Zhang Peng Suggested-by: Kairui Song --- mm/vmscan.c | 104 ++++++++++++++++++++++++++++++++++++--------------------= ---- 1 file changed, 62 insertions(+), 42 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index a0807dd01c5a..3e18948e90d1 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1163,8 +1163,65 @@ static bool folio_try_reclaim_free(struct folio *fol= io, return true; } =20 +static bool folio_try_pageout(struct folio *folio, + struct folio_batch *free_folios, + struct scan_control *sc, struct reclaim_stat *stat, + struct swap_iocb **plug, struct list_head *folio_list, + unsigned int *nr_reclaimed) +{ + struct address_space *mapping =3D folio_mapping(folio); + unsigned int nr_pages =3D folio_nr_pages(folio); + + switch (pageout(folio, mapping, plug, folio_list)) { + case PAGE_ACTIVATE: + /* + * If shmem folio is split when writeback to swap, the + * tail pages will make their own pass through this + * function and be accounted then. + */ + if (nr_pages > 1 && !folio_test_large(folio)) + sc->nr_scanned -=3D (nr_pages - 1); + folio_activate_locked(folio, stat); + folio_unlock(folio); + return false; + case PAGE_KEEP: + folio_unlock(folio); + return false; + case PAGE_SUCCESS: + if (nr_pages > 1 && !folio_test_large(folio)) { + sc->nr_scanned -=3D (nr_pages - 1); + nr_pages =3D 1; + } + stat->nr_pageout +=3D nr_pages; + + if (folio_test_writeback(folio)) + return false; + if (folio_test_dirty(folio)) + return false; + + /* + * A synchronous write - probably a ramdisk. Go ahead + * and try to reclaim the folio. + */ + if (!folio_trylock(folio)) + return false; + if (folio_test_dirty(folio) || + folio_test_writeback(folio)) { + folio_unlock(folio); + return false; + } + fallthrough; + case PAGE_CLEAN: + ; /* try to free the folio below */ + } + if (folio_try_reclaim_free(folio, free_folios, sc, stat, nr_reclaimed)) + return true; + folio_unlock(folio); + return false; +} + /* - * shrink_folio_list() returns the number of reclaimed pages + * Reclaimed folios are counted in the return value. */ static unsigned int shrink_folio_list(struct list_head *folio_list, struct pglist_data *pgdat, struct scan_control *sc, @@ -1501,53 +1558,16 @@ static unsigned int shrink_folio_list(struct list_h= ead *folio_list, goto keep_locked; if (!sc->may_writepage) goto keep_locked; - /* * Folio is dirty. Flush the TLB if a writable entry * potentially exists to avoid CPU writes after I/O * starts and then write it out here. */ try_to_unmap_flush_dirty(); - switch (pageout(folio, mapping, &plug, folio_list)) { - case PAGE_KEEP: - goto keep_locked; - case PAGE_ACTIVATE: - /* - * If shmem folio is split when writeback to swap, - * the tail pages will make their own pass through - * this function and be accounted then. - */ - if (nr_pages > 1 && !folio_test_large(folio)) { - sc->nr_scanned -=3D (nr_pages - 1); - nr_pages =3D 1; - } - goto activate_locked; - case PAGE_SUCCESS: - if (nr_pages > 1 && !folio_test_large(folio)) { - sc->nr_scanned -=3D (nr_pages - 1); - nr_pages =3D 1; - } - stat->nr_pageout +=3D nr_pages; - - if (folio_test_writeback(folio)) - goto keep; - if (folio_test_dirty(folio)) - goto keep; - - /* - * A synchronous write - probably a ramdisk. Go - * ahead and try to reclaim the folio. - */ - if (!folio_trylock(folio)) - goto keep; - if (folio_test_dirty(folio) || - folio_test_writeback(folio)) - goto keep_locked; - mapping =3D folio_mapping(folio); - fallthrough; - case PAGE_CLEAN: - ; /* try to free the folio below */ - } + if (!folio_try_pageout(folio, &free_folios, sc, stat, + &plug, folio_list, &nr_reclaimed)) + goto keep; + continue; } =20 if (!folio_try_reclaim_free(folio, &free_folios, sc, stat, --=20 2.43.7 From nobody Sat Jul 25 03:20:30 2026 Received: from outbound.qs.icloud.com (qs-2006g-snip4-4.eps.apple.com [57.103.85.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 64C3A2D97BB for ; Mon, 20 Jul 2026 05:08:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=57.103.85.195 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524101; cv=none; b=q2rjWX/S9V+EvZCq3chJfczN2Qui24dmAQ7lGuodOb0dSgl1HbDlE8slhK0kCTQN0J/5VaFt7Soyg6huonU8LbvrOXWDVELg0YCLdAmdsVRKd3QrF6ojAyIfHUcASUSc48t2p9E90RYneS07IrlUM/z/gvEAs5pW0UxmPF7uflw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524101; c=relaxed/simple; bh=1wbXcA3EwOgkLmlnWmmIkzMkHOobeEr8JUOUrutpQDg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ftu3qKjwBkWn0XPTOoxEDV/UvDPZnhN7icNefhoeNXwaxoag9JClbmqEkZHV7Ua5SiKueP+BXokHLkWZcoF++FuMGkpf70gEJhyql/wCF13/Nu3UGeaJl5BaOKK1JWGqWOWwD5Ic9O5GY3FAhHzAtgUXhQdfAM4UgIqjVX5JM4k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com; spf=pass smtp.mailfrom=icloud.com; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b=HdTIZPLJ; arc=none smtp.client-ip=57.103.85.195 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=icloud.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b="HdTIZPLJ" Received: from outbound.qs.icloud.com (unknown [127.0.0.2]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPS id 89436180013A; Mon, 20 Jul 2026 05:08:16 +0000 (UTC) X-ICL-RepId: 019f7dec-c3da-751e-9eb6-011f49e09339 X-ICL-Out-Info: HUtFAUMEWwJACUgBTUQeDx5WFlZNRAJCTQhMHVwGXRxCCkEdXgBLVxQEAlodRw5AHVYWWAhOK1sTVRdGCRkIXR0ZHldQXgheH0wcHQ5YBhICWkUBXRcDVxxWRVwYQwldBVccHRxERVsTVRdGCRkIXR0ZCEcfCjADQg5WA0MHRQAtGRxXUF4IXh9MHB0OWAYSHVAcDlEFWwBGCU8BXRoJUwRaEB4ZWwkfFlUNQAUaHQddDVVXDw5fAREJHAMJAQlyGVoUXBhTRVEfVEYTGU4bV01QG18CQg8= Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1784524099; x=1787116099; bh=oxorjIprI4mpAO6TiEZUTmw1D9wu+4GV0+L2FBilTss=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:x-icloud-hme; b=HdTIZPLJfpgSLxqnWXwavTqKBbqneLmo5rYK9L4QUqrDsCOwqDFpGMKBM8By1mBAlgHm2uGgChRFQI6Cecf+5sd+K+HSMq+cEi76FuLZmJqOwHTAY3VZLVjykJl40gkgDsdwyrcnJxWLLk2OblSUWJC+dt1mdkxD8nZZ7kfza5rPAqRQAxWenSspL/d0Mwm+SZNOfvD9CPHjV1BerWWegs0B+UXWGzy7AzoRACohH850NPYFeBZlyj/8bO15C2WUWpZKDFlbNW0hFvqhHYOiWkMPwyj+unnY2IcLaA4Pi63Dibq7juxpKVrKZwC8mGI0eF2vb3twRnz2pCHjQNtHRw== Received: from [21.6.122.162] (unknown [17.57.155.37]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPSA id 04C041800142; Mon, 20 Jul 2026 05:08:09 +0000 (UTC) From: Zhang Peng Date: Mon, 20 Jul 2026 13:07:41 +0800 Subject: [PATCH v5 4/5] mm/vmscan: extract folio unmap logic into folio_try_unmap() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260720-batch-tlb-flush-v5-4-db943a0d0d6b@icloud.com> References: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> In-Reply-To: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Johannes Weiner , Shakeel Butt , Axel Rasmussen , Yuanchu Xie , Wei Xu , Michal Hocko , Qi Zheng , "Liam R. Howlett" , Qi Zheng Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Barry Song , Kairui Song , Zhang Peng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784524064; l=3675; i=zippermonkey@icloud.com; s=20260309; h=from:subject:message-id; bh=1wbXcA3EwOgkLmlnWmmIkzMkHOobeEr8JUOUrutpQDg=; b=A0Ybc/7n8H9VBZRr6lMHF+lordRq5fKgCvcJzGPaTIFmdjV7Kt2djrb+R+w34345x0tJId+O5 eDmYa8PgD6BClRepn7xUiQO66idzQllacf+nVS/AVw6vzDFHQqcY6EG X-Developer-Key: i=zippermonkey@icloud.com; a=ed25519; pk=tPCLpFnBfIyHsp0k7eaUTUREEa36bQNW/69X+NS8wBU= X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNzIwMDA1NCBTYWx0ZWRfXwY1/FyH0WiBA k74iNQLescyWi2rEr4XWd/WubxdgqWEAtqPeXf9mLvJ1yT+6zVSHltWjIKLTa7iZSlcC14ZVoOY 0REYlT5ADNfuDHM9YeoHw08cSyWKT4FCSZvtdnb3cClwYjle5EquaQPSNYPvxvPlCK/ie1F0RGW 486IL/k682lPvsZlnMa60/FqUoLCK8ZLRAN0Ne8ZHpjGooo0EUtYKhHAkCXiWSuJQnZtds/M+QY JkURkqkl6zea6miskrgrFQC6vGjt2rc9gYbsDeUJZWJYq2O3cjNNTZGsKAgvkUx5sPRq7vGd7Ta KgA06dJZpNwiGxjSK4O X-Proofpoint-GUID: GG3EJULa8lsx5m573CbJU4TMSUiB9zCl X-Proofpoint-ORIG-GUID: GG3EJULa8lsx5m573CbJU4TMSUiB9zCl shrink_folio_list() contains a self-contained block that sets up TTU flags and calls try_to_unmap(), accounting for failures via reclaim_stat. Extract it into folio_try_unmap() to reduce the size of shrink_folio_list() and make the unmap step independently readable. folio_try_unmap() is only called when the folio is actually mapped; the !folio_mapped() check stays in the caller, keeping the function's semantics clear: it tries to unmap a mapped folio and returns whether the unmap succeeded. No functional change. Signed-off-by: Zhang Peng Suggested-by: Kairui Song --- mm/vmscan.c | 65 +++++++++++++++++++++++++++++++++------------------------= ---- 1 file changed, 35 insertions(+), 30 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 3e18948e90d1..bb479ead1ee0 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1220,6 +1220,38 @@ static bool folio_try_pageout(struct folio *folio, return false; } =20 +static bool folio_try_unmap(struct folio *folio, struct reclaim_stat *stat, + unsigned int nr_pages) +{ + enum ttu_flags flags =3D TTU_BATCH_FLUSH; + const bool was_swapbacked =3D folio_test_swapbacked(folio); + + if (folio_test_pmd_mappable(folio)) + flags |=3D TTU_SPLIT_HUGE_PMD; + /* + * Without TTU_SYNC, try_to_unmap will only begin to hold PTL + * from the first present PTE within a large folio. Some + * initial PTEs might be skipped due to races with parallel + * PTE writes in which PTEs can be cleared temporarily before + * being written new present values. This will lead to a large + * folio is still mapped while some subpages have been + * partially unmapped after try_to_unmap; TTU_SYNC helps + * try_to_unmap acquire PTL from the first PTE, eliminating the + * influence of temporary PTE values. + */ + if (folio_test_large(folio)) + flags |=3D TTU_SYNC; + + try_to_unmap(folio, flags); + if (folio_mapped(folio)) { + stat->nr_unmap_fail +=3D nr_pages; + if (!was_swapbacked && folio_test_swapbacked(folio)) + stat->nr_lazyfree_fail +=3D nr_pages; + return false; + } + return true; +} + /* * Reclaimed folios are counted in the return value. */ @@ -1494,36 +1526,9 @@ static unsigned int shrink_folio_list(struct list_he= ad *folio_list, * The folio is mapped into the page tables of one or more * processes. Try to unmap it here. */ - if (folio_mapped(folio)) { - enum ttu_flags flags =3D TTU_BATCH_FLUSH; - bool was_swapbacked =3D folio_test_swapbacked(folio); - - if (folio_test_pmd_mappable(folio)) - flags |=3D TTU_SPLIT_HUGE_PMD; - /* - * Without TTU_SYNC, try_to_unmap will only begin to - * hold PTL from the first present PTE within a large - * folio. Some initial PTEs might be skipped due to - * races with parallel PTE writes in which PTEs can be - * cleared temporarily before being written new present - * values. This will lead to a large folio is still - * mapped while some subpages have been partially - * unmapped after try_to_unmap; TTU_SYNC helps - * try_to_unmap acquire PTL from the first PTE, - * eliminating the influence of temporary PTE values. - */ - if (folio_test_large(folio)) - flags |=3D TTU_SYNC; - - try_to_unmap(folio, flags); - if (folio_mapped(folio)) { - stat->nr_unmap_fail +=3D nr_pages; - if (!was_swapbacked && - folio_test_swapbacked(folio)) - stat->nr_lazyfree_fail +=3D nr_pages; - goto activate_locked; - } - } + if (folio_mapped(folio) && + !folio_try_unmap(folio, stat, nr_pages)) + goto activate_locked; =20 /* * Folio is unmapped now so it cannot be newly pinned anymore. --=20 2.43.7 From nobody Sat Jul 25 03:20:30 2026 Received: from outbound.qs.icloud.com (qs-2006f-snip4-3.eps.apple.com [57.103.85.184]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A24E02BE7BE for ; Mon, 20 Jul 2026 05:08:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=57.103.85.184 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524109; cv=none; b=loqc7a92xMICSJ+IXLv/e7x6CJF8oZ9asZ2E8rDV1YEGW8GFsDGglcJByjDVhWZDOMxAE3UPATQ1Ts6KUQ0d7NEYX1G65orhmGt8c626XdgxUiQsYThN1S2EJOekvElZGOca111+e+O6zGTyk/SpoQfCI58wOdTT4X86XdnpdVw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784524109; c=relaxed/simple; bh=vRfWGSnXj1NRnsdshI2zRjQUulhC/QjmbpiKDROQvIQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Swlv+hvZeXo/2+SzJsWPWIa2gKFwhX9LYT4QMGB8MVEjqKrBjxzNzxnn0CMwsDRNMZTBxHBVaey6FWbzV3vuwf3yZSz5kX7qx7I+8dC78a1HOwK0Og8gC23Cu71KaBcm1jFuBv4cAbslXsLo4czTi8zN3b2hIOXUHJxsqriKaL0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com; spf=pass smtp.mailfrom=icloud.com; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b=pUdKd/7d; arc=none smtp.client-ip=57.103.85.184 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=icloud.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b="pUdKd/7d" Received: from outbound.qs.icloud.com (unknown [127.0.0.2]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPS id CA171180013D; Mon, 20 Jul 2026 05:08:23 +0000 (UTC) X-ICL-RepId: 019f7dec-e043-74fb-9e93-c65bb5ca77ab X-ICL-Out-Info: HUtFAUMEWwJACUgBTUQeDx5WFlZNRAJCTQhMHVwGXRxCCkEdXgBLVxQEAlodRw5AHVYWWAhOK1sTVRdGCRkIXR0ZHldQXgheH0wcHQ5YBhICWkUBXRcDVxxWRVwYQwldBVccHRxERVsTVRdGCRkIXR0ZCEcfCjADQg5WA0MHRQAtGRxXUF4IXh9MHB0OWAYSHVAcDlEFWwBGCU8BXRoJUwRaEB4ZWwkfFlUNQAUaHQddDFVXDw5fAREJHAMJAQlyGVoUXBhTRVEfVEYTGU4bV01QG18CQg8= Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1784524106; x=1787116106; bh=p5HyYFMk+G/YKnBn8V0qp4YenxBWpgT+6vv4nWfMrPQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:x-icloud-hme; b=pUdKd/7dRm/6yP4H6XvLXbgOKm9ud6pR7M/zuwXdYHFmheJkAtI80umHQAJFeiQWRoG7yOjoK/Dbfv/amK/oJz3IJWANDVkDjUD4GC2w6RDu5rY57wKKBtUE0cagorvn/PX+ls+T5U9P1zGqnK8UF0y7CYSNb8+gAKj30gT1upYolOgdmLvItQqDoRRo956YDAOSHn9WQe/0lO5kHQYHZKnsVUrJ3u7YX1yW6Z8k+xN+NA+nd+I26wCcougb7AZMaTIcoxS4aS7j2HPsCBHfQUNvuIheMuongNEiwyBfGUFlPvt5IC3QkIyrJomIR+ZeqeoWCFn1n7Bmi84SgZXVNA== Received: from [21.6.122.162] (unknown [17.57.155.37]) by p00-icloudmta-asmtp-us-east-2d-60-percent-7 (Postfix) with ESMTPSA id 5E84E180012E; Mon, 20 Jul 2026 05:08:16 +0000 (UTC) From: Zhang Peng Date: Mon, 20 Jul 2026 13:07:42 +0800 Subject: [PATCH v5 5/5] mm/vmscan: flush TLB for every 31 folios evictions Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260720-batch-tlb-flush-v5-5-db943a0d0d6b@icloud.com> References: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> In-Reply-To: <20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com> To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Johannes Weiner , Shakeel Butt , Axel Rasmussen , Yuanchu Xie , Wei Xu , Michal Hocko , Qi Zheng , "Liam R. Howlett" , Qi Zheng Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Barry Song , Kairui Song , Zhang Peng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784524064; l=5688; i=zippermonkey@icloud.com; s=20260309; h=from:subject:message-id; bh=vRfWGSnXj1NRnsdshI2zRjQUulhC/QjmbpiKDROQvIQ=; b=EvGUlVN4y70BkfYRM2zX0HE1C0Yr+zYubCcxUThrRzZh/56vO9L1FCC5cpqFHfSBKXSks/K6x ab87sJ8slkXC8FC71/vMjQO1RI+QwEvN7iw8I9vdFs7xRGYx+2jvU/A X-Developer-Key: i=zippermonkey@icloud.com; a=ed25519; pk=tPCLpFnBfIyHsp0k7eaUTUREEa36bQNW/69X+NS8wBU= X-Proofpoint-ORIG-GUID: NROdlRa12C_BqvsJFnYbjXxtk8lK5x-b X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNzIwMDA1NCBTYWx0ZWRfX9Qxm5rjC/Y+/ Nyqv2bSDk/Ad1hz66wVCz5lEFOdq1ruTvxVh8zXnAitzw8Lioa4ihJjdbfn9toUKidIR8+jXm1T PYfDTm1JbV7+Bt7WNkH9dFnpHYsXTmjiV8Oc/ZQrpnVI6IqdHFN+MBbMi5LZRiEsSp2nzVLbGRF 5M7za2BW9HVDq/ecRgMVspYC/kvC2wRn67QlDRz3cUced4gnl6mqPOja5Xck3rXtAaZINeFpdFP C86P+xbvl3aaVLJ8GQL812k2UPb6Sx5DA79KlMWXKCqYiqYZWpaioaajzr5eOin97XWGzet02KK 9tcZL8xbjj1Mu0wWWRi X-Proofpoint-GUID: NROdlRa12C_BqvsJFnYbjXxtk8lK5x-b Currently we flush TLB for every dirty folio, which is a bottleneck for systems with many cores as this causes heavy IPI usage. So instead, batch the folios, and flush once for every 31 folios (one folio_batch). These folios will be held in a folio_batch with their lock released, then when the folio_batch is full, do the following steps: - For each folio: trylock - recheck still evictable (writeback, mapped, dma_pinned). If no longer evictable, put back via ret_folios. - Flush TLB once for the whole batch. - Pageout each survivor via folio_try_pageout(). The recheck step is required because dropping the folio lock between shrink_folio_list() and pageout_batch() opens a window in which a parallel swapin (do_swap_page) can fully complete and install a new PTE; once mapped, a parallel GUP can pin the folio without taking the folio lock. Folios caught by any of these checks are put back via ret_folios. Suggested-by: Kairui Song Signed-off-by: Zhang Peng --- mm/vmscan.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++++--= ---- 1 file changed, 84 insertions(+), 8 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index bb479ead1ee0..251535613336 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1220,6 +1220,71 @@ static bool folio_try_pageout(struct folio *folio, return false; } =20 +static void pageout_batch(struct folio_batch *fbatch, + struct list_head *ret_folios, + struct folio_batch *free_folios, + struct scan_control *sc, struct reclaim_stat *stat, + struct swap_iocb **plug, struct list_head *folio_list, + unsigned int *nr_reclaimed) +{ + int i, nr =3D folio_batch_count(fbatch); + int count =3D 0; + struct folio *folios[FOLIO_BATCH_SIZE]; + + /* + * Collect survivors into a local array: this avoids walking and + * mutating @fbatch in the same loop, so its invariant (only + * folios[0..nr) are valid) is preserved throughout. + */ + for (i =3D 0; i < nr; i++) { + struct folio *folio =3D fbatch->folios[i]; + + if (!folio_trylock(folio)) { + list_add(&folio->lru, ret_folios); + continue; + } + + VM_WARN_ON_FOLIO(folio_test_lru(folio), folio); + + /* + * Recheck what shrink_folio_list() verified before dropping + * the folio lock. Between that folio_unlock() and our + * folio_trylock() here, a parallel swapin (do_swap_page) can + * fully complete -- taking the lock, installing a new PTE, + * and releasing the lock -- after which a parallel GUP + * (O_DIRECT, vmsplice, RDMA, ...) can pin the folio through + * that PTE without taking the folio lock. Each flag catches + * one case: + * folio_test_writeback -- defensive guard. + * folio_mapped -- swapin installed a PTE. + * folio_maybe_dma_pinned -- GUP pinned through such a PTE. + */ + if (folio_test_writeback(folio) || folio_mapped(folio) || + folio_maybe_dma_pinned(folio)) { + folio_unlock(folio); + list_add(&folio->lru, ret_folios); + continue; + } + + folios[count++] =3D folio; /* keep lock held */ + } + folio_batch_reinit(fbatch); + + if (!count) + return; + + /* One TLB flush for the whole batch */ + try_to_unmap_flush_dirty(); + + for (i =3D 0; i < count; i++) { + struct folio *folio =3D folios[i]; + + if (!folio_try_pageout(folio, free_folios, sc, stat, plug, + folio_list, nr_reclaimed)) + list_add(&folio->lru, ret_folios); + } +} + static bool folio_try_unmap(struct folio *folio, struct reclaim_stat *stat, unsigned int nr_pages) { @@ -1261,6 +1326,7 @@ static unsigned int shrink_folio_list(struct list_hea= d *folio_list, struct mem_cgroup *memcg) { struct folio_batch free_folios; + struct folio_batch flush_folios; LIST_HEAD(ret_folios); LIST_HEAD(demote_folios); unsigned int nr_reclaimed =3D 0, nr_demoted =3D 0; @@ -1269,6 +1335,7 @@ static unsigned int shrink_folio_list(struct list_hea= d *folio_list, struct swap_iocb *plug =3D NULL; =20 folio_batch_init(&free_folios); + folio_batch_init(&flush_folios); memset(stat, 0, sizeof(*stat)); cond_resched(); do_demote_pass =3D can_demote(pgdat->node_id, sc, memcg); @@ -1564,15 +1631,18 @@ static unsigned int shrink_folio_list(struct list_h= ead *folio_list, if (!sc->may_writepage) goto keep_locked; /* - * Folio is dirty. Flush the TLB if a writable entry - * potentially exists to avoid CPU writes after I/O - * starts and then write it out here. + * Drop the lock so swap faults finding this folio + * via swap cache lookup can make progress; the + * recheck that this necessitates is documented in + * pageout_batch(). */ - try_to_unmap_flush_dirty(); - if (!folio_try_pageout(folio, &free_folios, sc, stat, - &plug, folio_list, &nr_reclaimed)) - goto keep; - continue; + folio_unlock(folio); + if (!folio_batch_add(&flush_folios, folio)) + pageout_batch(&flush_folios, + &ret_folios, &free_folios, + sc, stat, &plug, + folio_list, &nr_reclaimed); + goto next; } =20 if (!folio_try_reclaim_free(folio, &free_folios, sc, stat, @@ -1597,6 +1667,12 @@ static unsigned int shrink_folio_list(struct list_he= ad *folio_list, list_add(&folio->lru, &ret_folios); VM_BUG_ON_FOLIO(folio_test_lru(folio) || folio_test_unevictable(folio), folio); +next: + continue; + } + if (folio_batch_count(&flush_folios)) { + pageout_batch(&flush_folios, &ret_folios, &free_folios, sc, + stat, &plug, folio_list, &nr_reclaimed); } /* 'folio_list' is always empty here */ =20 --=20 2.43.7