From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97462C4167B for ; Mon, 6 Nov 2023 15:50:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232650AbjKFPuL (ORCPT ); Mon, 6 Nov 2023 10:50:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232331AbjKFPuE (ORCPT ); Mon, 6 Nov 2023 10:50:04 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50F7AD42; Mon, 6 Nov 2023 07:50:01 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG4s51RDzVlYW; Mon, 6 Nov 2023 23:49:49 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:49:59 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 01/10] mm: swap: introduce pfn_swap_entry_to_folio() Date: Mon, 6 Nov 2023 23:49:41 +0800 Message-ID: <20231106154950.3399469-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Introduce a new pfn_swap_entry_to_folio(), it is similar to pfn_swap_entry_to_page(), but return a folio, which allow us to completely replace the struct page variables with struct folio variables. Signed-off-by: Kefeng Wang --- include/linux/swapops.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index bff1e8d97de0..85cb84e4be95 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -468,6 +468,19 @@ static inline struct page *pfn_swap_entry_to_page(swp_= entry_t entry) return p; } =20 +static inline struct folio *pfn_swap_entry_to_folio(swp_entry_t entry) +{ + struct folio *folio =3D pfn_folio(swp_offset_pfn(entry)); + + /* + * Any use of migration entries may only occur while the + * corresponding folio is locked + */ + BUG_ON(is_migration_entry(entry) && !folio_test_locked(folio)); + + return folio; +} + /* * A pfn swap entry is a special type of swap entry that always has a pfn = stored * in the swap offset. They are used to represent unaddressable device mem= ory --=20 2.27.0 From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0358EC0018A for ; Mon, 6 Nov 2023 15:50:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232588AbjKFPuJ (ORCPT ); Mon, 6 Nov 2023 10:50:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232326AbjKFPuE (ORCPT ); Mon, 6 Nov 2023 10:50:04 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51558D45; Mon, 6 Nov 2023 07:50:01 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG4t11wVzVlmQ; Mon, 6 Nov 2023 23:49:50 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:49:59 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 02/10] s390: pgtable: use a folio in ptep_zap_swap_entry() Date: Mon, 6 Nov 2023 23:49:42 +0800 Message-ID: <20231106154950.3399469-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in ptep_zap_swap_entry(), which is preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- arch/s390/mm/pgtable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 3bd2ab2a9a34..2f946b493fff 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -730,9 +730,9 @@ static void ptep_zap_swap_entry(struct mm_struct *mm, s= wp_entry_t entry) if (!non_swap_entry(entry)) dec_mm_counter(mm, MM_SWAPENTS); else if (is_migration_entry(entry)) { - struct page *page =3D pfn_swap_entry_to_page(entry); + struct folio *folio =3D pfn_swap_entry_to_folio(entry); =20 - dec_mm_counter(mm, mm_counter(page)); + dec_mm_counter(mm, mm_counter(&folio->page)); } free_swap_and_cache(entry); } --=20 2.27.0 From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E791AC4167D for ; Mon, 6 Nov 2023 15:50:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232847AbjKFPuW (ORCPT ); Mon, 6 Nov 2023 10:50:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232350AbjKFPuF (ORCPT ); Mon, 6 Nov 2023 10:50:05 -0500 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D9CFD51; Mon, 6 Nov 2023 07:50:02 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4SPG1S1t8Mz1P7vj; Mon, 6 Nov 2023 23:46:52 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:49:59 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 03/10] mm: huge_memory: use a folio in __split_huge_pmd_locked() Date: Mon, 6 Nov 2023 23:49:43 +0800 Message-ID: <20231106154950.3399469-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in __split_huge_pmd_locked() which replaces five compound_head() call with two page_folio() calls. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f31f02472396..34dd01970927 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2117,6 +2117,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, count_vm_event(THP_SPLIT_PMD); =20 if (!vma_is_anonymous(vma)) { + struct folio *folio; old_pmd =3D pmdp_huge_clear_flush(vma, haddr, pmd); /* * We are going to unmap this huge page. So @@ -2130,17 +2131,17 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, swp_entry_t entry; =20 entry =3D pmd_to_swp_entry(old_pmd); - page =3D pfn_swap_entry_to_page(entry); + folio =3D pfn_swap_entry_to_folio(entry); } else { - page =3D pmd_page(old_pmd); - if (!PageDirty(page) && pmd_dirty(old_pmd)) - set_page_dirty(page); - if (!PageReferenced(page) && pmd_young(old_pmd)) - SetPageReferenced(page); - page_remove_rmap(page, vma, true); - put_page(page); + folio =3D page_folio(pmd_page(old_pmd)); + if (!folio_test_dirty(folio) && pmd_dirty(old_pmd)) + folio_set_dirty(folio); + if (!folio_test_referenced(folio) && pmd_young(old_pmd)) + folio_set_referenced(folio); + page_remove_rmap(&folio->page, vma, true); + folio_put(folio); } - add_mm_counter(mm, mm_counter_file(page), -HPAGE_PMD_NR); + add_mm_counter(mm, mm_counter_file(&folio->page), -HPAGE_PMD_NR); return; } =20 --=20 2.27.0 From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B4FAC4167B for ; Mon, 6 Nov 2023 15:50:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232704AbjKFPuO (ORCPT ); Mon, 6 Nov 2023 10:50:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229922AbjKFPuF (ORCPT ); Mon, 6 Nov 2023 10:50:05 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BDCB6D61; Mon, 6 Nov 2023 07:50:02 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SPG4x5P3BzvQTB; Mon, 6 Nov 2023 23:49:53 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:00 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 04/10] mm: huge_memory: use a folio in zap_huge_pmd() Date: Mon, 6 Nov 2023 23:49:44 +0800 Message-ID: <20231106154950.3399469-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in zap_huge_pmd(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 34dd01970927..78a00fe22c2d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1716,11 +1716,13 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_= area_struct *vma, zap_deposited_table(tlb->mm, pmd); spin_unlock(ptl); } else { - struct page *page =3D NULL; + struct folio *folio =3D NULL; int flush_needed =3D 1; =20 if (pmd_present(orig_pmd)) { - page =3D pmd_page(orig_pmd); + struct page *page =3D pmd_page(orig_pmd); + + folio =3D page_folio(page); page_remove_rmap(page, vma, true); VM_BUG_ON_PAGE(page_mapcount(page) < 0, page); VM_BUG_ON_PAGE(!PageHead(page), page); @@ -1729,12 +1731,12 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_= area_struct *vma, =20 VM_BUG_ON(!is_pmd_migration_entry(orig_pmd)); entry =3D pmd_to_swp_entry(orig_pmd); - page =3D pfn_swap_entry_to_page(entry); + folio =3D pfn_swap_entry_to_folio(entry); flush_needed =3D 0; } else WARN_ONCE(1, "Non present huge pmd without pmd migration enabled!"); =20 - if (PageAnon(page)) { + if (folio_test_anon(folio)) { zap_deposited_table(tlb->mm, pmd); add_mm_counter(tlb->mm, MM_ANONPAGES, -HPAGE_PMD_NR); } else { @@ -1745,7 +1747,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_ar= ea_struct *vma, =20 spin_unlock(ptl); if (flush_needed) - tlb_remove_page_size(tlb, page, HPAGE_PMD_SIZE); + tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE); } return 1; } --=20 2.27.0 From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2516AC0018A for ; Mon, 6 Nov 2023 15:50:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232782AbjKFPuS (ORCPT ); Mon, 6 Nov 2023 10:50:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232348AbjKFPuF (ORCPT ); Mon, 6 Nov 2023 10:50:05 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ACBBB191; Mon, 6 Nov 2023 07:50:02 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG4v34Z0zVlpd; Mon, 6 Nov 2023 23:49:51 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:00 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 05/10] mm: memory: use a folio in copy_nonpresent_pte() Date: Mon, 6 Nov 2023 23:49:45 +0800 Message-ID: <20231106154950.3399469-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in copy_nonpresent_pte(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 1f18ed4a5497..914353d1c7f1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -779,7 +779,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, unsigned long vm_flags =3D dst_vma->vm_flags; pte_t orig_pte =3D ptep_get(src_pte); pte_t pte =3D orig_pte; - struct page *page; + struct folio *folio; swp_entry_t entry =3D pte_to_swp_entry(orig_pte); =20 if (likely(!non_swap_entry(entry))) { @@ -801,9 +801,9 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, } rss[MM_SWAPENTS]++; } else if (is_migration_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + folio =3D pfn_swap_entry_to_folio(entry); =20 - rss[mm_counter(page)]++; + rss[mm_counter(&folio->page)]++; =20 if (!is_readable_migration_entry(entry) && is_cow_mapping(vm_flags)) { @@ -822,8 +822,9 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, set_pte_at(src_mm, addr, src_pte, pte); } } else if (is_device_private_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + struct page *page =3D pfn_swap_entry_to_page(entry); =20 + folio =3D page_folio(page); /* * Update rss count even for unaddressable pages, as * they should treated just like normal pages in this @@ -833,7 +834,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, * for unaddressable pages, at some point. But for now * keep things as they are. */ - get_page(page); + folio_get(folio); rss[mm_counter(page)]++; /* Cannot fail as these pages cannot get pinned. */ BUG_ON(page_try_dup_anon_rmap(page, false, src_vma)); --=20 2.27.0 From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A13BC4332F for ; Mon, 6 Nov 2023 15:50:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232888AbjKFPud (ORCPT ); Mon, 6 Nov 2023 10:50:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232569AbjKFPuH (ORCPT ); Mon, 6 Nov 2023 10:50:07 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58B85191; Mon, 6 Nov 2023 07:50:04 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG0K1YGGzPnbL; Mon, 6 Nov 2023 23:45:53 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:01 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 06/10] mm: memory: use a folio in zap_pte_range() Date: Mon, 6 Nov 2023 23:49:46 +0800 Message-ID: <20231106154950.3399469-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Make should_zap_page() to take a folio and use a folio in zap_pte_range(), which save several compound_head() calls. Signed-off-by: Kefeng Wang --- mm/memory.c | 44 ++++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 18 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 914353d1c7f1..669f167f45cd 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1359,19 +1359,20 @@ static inline bool should_zap_cows(struct zap_detai= ls *details) return details->even_cows; } =20 -/* Decides whether we should zap this page with the page pointer specified= */ -static inline bool should_zap_page(struct zap_details *details, struct pag= e *page) +/* Decides whether we should zap this folio with the folio pointer specifi= ed */ +static inline bool should_zap_folio(struct zap_details *details, + struct folio *folio) { - /* If we can make a decision without *page.. */ + /* If we can make a decision without *folio.. */ if (should_zap_cows(details)) return true; =20 - /* E.g. the caller passes NULL for the case of a zero page */ - if (!page) + /* E.g. the caller passes NULL for the case of a zero folio */ + if (!folio) return true; =20 - /* Otherwise we should only zap non-anon pages */ - return !PageAnon(page); + /* Otherwise we should only zap non-anon folios */ + return !folio_test_anon(folio); } =20 static inline bool zap_drop_file_uffd_wp(struct zap_details *details) @@ -1424,7 +1425,7 @@ static unsigned long zap_pte_range(struct mmu_gather = *tlb, arch_enter_lazy_mmu_mode(); do { pte_t ptent =3D ptep_get(pte); - struct page *page; + struct folio *folio =3D NULL; =20 if (pte_none(ptent)) continue; @@ -1434,9 +1435,13 @@ static unsigned long zap_pte_range(struct mmu_gather= *tlb, =20 if (pte_present(ptent)) { unsigned int delay_rmap; + struct page *page; =20 page =3D vm_normal_page(vma, addr, ptent); - if (unlikely(!should_zap_page(details, page))) + if (page) + folio =3D page_folio(page); + + if (unlikely(!should_zap_folio(details, folio))) continue; ptent =3D ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); @@ -1450,16 +1455,16 @@ static unsigned long zap_pte_range(struct mmu_gathe= r *tlb, } =20 delay_rmap =3D 0; - if (!PageAnon(page)) { + if (!folio_test_anon(folio)) { if (pte_dirty(ptent)) { - set_page_dirty(page); + folio_set_dirty(folio); if (tlb_delay_rmap(tlb)) { delay_rmap =3D 1; force_flush =3D 1; } } if (pte_young(ptent) && likely(vma_has_recency(vma))) - mark_page_accessed(page); + folio_mark_accessed(folio); } rss[mm_counter(page)]--; if (!delay_rmap) { @@ -1478,9 +1483,12 @@ static unsigned long zap_pte_range(struct mmu_gather= *tlb, entry =3D pte_to_swp_entry(ptent); if (is_device_private_entry(entry) || is_device_exclusive_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); - if (unlikely(!should_zap_page(details, page))) + struct page *page =3D pfn_swap_entry_to_page(entry); + + folio =3D page_folio(page); + if (unlikely(!should_zap_folio(details, folio))) continue; + /* * Both device private/exclusive mappings should only * work with anonymous page so far, so we don't need to @@ -1491,7 +1499,7 @@ static unsigned long zap_pte_range(struct mmu_gather = *tlb, rss[mm_counter(page)]--; if (is_device_private_entry(entry)) page_remove_rmap(page, vma, false); - put_page(page); + folio_put(folio); } else if (!non_swap_entry(entry)) { /* Genuine swap entry, hence a private anon page */ if (!should_zap_cows(details)) @@ -1500,10 +1508,10 @@ static unsigned long zap_pte_range(struct mmu_gathe= r *tlb, if (unlikely(!free_swap_and_cache(entry))) print_bad_pte(vma, addr, ptent, NULL); } else if (is_migration_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); - if (!should_zap_page(details, page)) + folio =3D pfn_swap_entry_to_folio(entry); + if (!should_zap_folio(details, folio)) continue; - rss[mm_counter(page)]--; + rss[mm_counter(&folio->page)]--; } else if (pte_marker_entry_uffd_wp(entry)) { /* * For anon: always drop the marker; for file: only --=20 2.27.0 From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F15C2C4167D for ; Mon, 6 Nov 2023 15:50:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232706AbjKFPuZ (ORCPT ); Mon, 6 Nov 2023 10:50:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232346AbjKFPuH (ORCPT ); Mon, 6 Nov 2023 10:50:07 -0500 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56BF7118; Mon, 6 Nov 2023 07:50:04 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4SPFzz4mWdzMmWr; Mon, 6 Nov 2023 23:45:35 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:01 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 07/10] mm: memory: use a folio in do_set_pmd() Date: Mon, 6 Nov 2023 23:49:47 +0800 Message-ID: <20231106154950.3399469-8-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in do_set_pmd(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 669f167f45cd..fbed32a09fec 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4322,12 +4322,13 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct = page *page) unsigned long haddr =3D vmf->address & HPAGE_PMD_MASK; pmd_t entry; vm_fault_t ret =3D VM_FAULT_FALLBACK; + struct folio *folio; =20 if (!transhuge_vma_suitable(vma, haddr)) return ret; =20 - page =3D compound_head(page); - if (compound_order(page) !=3D HPAGE_PMD_ORDER) + folio =3D page_folio(page); + if (folio_order(folio) !=3D HPAGE_PMD_ORDER) return ret; =20 /* --=20 2.27.0 From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6726CC4332F for ; Mon, 6 Nov 2023 15:50:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232726AbjKFPu3 (ORCPT ); Mon, 6 Nov 2023 10:50:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232564AbjKFPuH (ORCPT ); Mon, 6 Nov 2023 10:50:07 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EF38D45; Mon, 6 Nov 2023 07:50:04 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG0L0dT2zPntQ; Mon, 6 Nov 2023 23:45:54 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:02 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 08/10] mm: memory: use a folio in insert_page_into_pte_locked() Date: Mon, 6 Nov 2023 23:49:48 +0800 Message-ID: <20231106154950.3399469-9-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in insert_page_into_pte_locked(), which is preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index fbed32a09fec..dd3760988e02 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1854,10 +1854,13 @@ static int validate_page_before_insert(struct page = *page) static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *= pte, unsigned long addr, struct page *page, pgprot_t prot) { + struct folio *folio; + if (!pte_none(ptep_get(pte))) return -EBUSY; + folio =3D page_folio(page); /* Ok, finally just insert the thing.. */ - get_page(page); + folio_get(folio); inc_mm_counter(vma->vm_mm, mm_counter_file(page)); page_add_file_rmap(page, vma, false); set_pte_at(vma->vm_mm, addr, pte, mk_pte(page, prot)); --=20 2.27.0 From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81131C4332F for ; Mon, 6 Nov 2023 15:50:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232842AbjKFPug (ORCPT ); Mon, 6 Nov 2023 10:50:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42922 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232791AbjKFPuQ (ORCPT ); Mon, 6 Nov 2023 10:50:16 -0500 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A7B1D69; Mon, 6 Nov 2023 07:50:09 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4SPG1X25YHz1P7x5; Mon, 6 Nov 2023 23:46:56 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:02 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 09/10] mm: convert mm_counter() to take a folio Date: Mon, 6 Nov 2023 23:49:49 +0800 Message-ID: <20231106154950.3399469-10-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since all mm_counter() callers with a folio, let's convert mm_counter() to take a folio. Signed-off-by: Kefeng Wang --- arch/s390/mm/pgtable.c | 2 +- include/linux/mm.h | 6 +++--- mm/memory.c | 10 +++++----- mm/rmap.c | 8 ++++---- mm/userfaultfd.c | 2 +- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 2f946b493fff..54b184648db6 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -732,7 +732,7 @@ static void ptep_zap_swap_entry(struct mm_struct *mm, s= wp_entry_t entry) else if (is_migration_entry(entry)) { struct folio *folio =3D pfn_swap_entry_to_folio(entry); =20 - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } free_swap_and_cache(entry); } diff --git a/include/linux/mm.h b/include/linux/mm.h index 418d26608ece..fea78900bf84 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2591,11 +2591,11 @@ static inline int mm_counter_file(struct page *page) return MM_FILEPAGES; } =20 -static inline int mm_counter(struct page *page) +static inline int mm_counter(struct folio *folio) { - if (PageAnon(page)) + if (folio_test_anon(folio)) return MM_ANONPAGES; - return mm_counter_file(page); + return mm_counter_file(&folio->page); } =20 static inline unsigned long get_mm_rss(struct mm_struct *mm) diff --git a/mm/memory.c b/mm/memory.c index dd3760988e02..3ffef84dd7bb 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -803,7 +803,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, } else if (is_migration_entry(entry)) { folio =3D pfn_swap_entry_to_folio(entry); =20 - rss[mm_counter(&folio->page)]++; + rss[mm_counter(folio)]++; =20 if (!is_readable_migration_entry(entry) && is_cow_mapping(vm_flags)) { @@ -835,7 +835,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, * keep things as they are. */ folio_get(folio); - rss[mm_counter(page)]++; + rss[mm_counter(folio)]++; /* Cannot fail as these pages cannot get pinned. */ BUG_ON(page_try_dup_anon_rmap(page, false, src_vma)); =20 @@ -1466,7 +1466,7 @@ static unsigned long zap_pte_range(struct mmu_gather = *tlb, if (pte_young(ptent) && likely(vma_has_recency(vma))) folio_mark_accessed(folio); } - rss[mm_counter(page)]--; + rss[mm_counter(folio)]--; if (!delay_rmap) { page_remove_rmap(page, vma, false); if (unlikely(page_mapcount(page) < 0)) @@ -1496,7 +1496,7 @@ static unsigned long zap_pte_range(struct mmu_gather = *tlb, * see zap_install_uffd_wp_if_needed(). */ WARN_ON_ONCE(!vma_is_anonymous(vma)); - rss[mm_counter(page)]--; + rss[mm_counter(folio)]--; if (is_device_private_entry(entry)) page_remove_rmap(page, vma, false); folio_put(folio); @@ -1511,7 +1511,7 @@ static unsigned long zap_pte_range(struct mmu_gather = *tlb, folio =3D pfn_swap_entry_to_folio(entry); if (!should_zap_folio(details, folio)) continue; - rss[mm_counter(&folio->page)]--; + rss[mm_counter(folio)]--; } else if (pte_marker_entry_uffd_wp(entry)) { /* * For anon: always drop the marker; for file: only diff --git a/mm/rmap.c b/mm/rmap.c index 7a27a2b41802..7a563490ce08 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1678,7 +1678,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } =20 @@ -1693,7 +1693,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else if (folio_test_anon(folio)) { swp_entry_t entry =3D page_swap_entry(subpage); pte_t swp_pte; @@ -2075,7 +2075,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } =20 @@ -2090,7 +2090,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else { swp_entry_t entry; pte_t swp_pte; diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 96d9eae5c7cc..9a6759fa9b06 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -124,7 +124,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, * Must happen after rmap, as mm_counter() checks mapping (via * PageAnon()), which is set by __page_set_anon_rmap(). */ - inc_mm_counter(dst_mm, mm_counter(page)); + inc_mm_counter(dst_mm, mm_counter(folio)); =20 set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); =20 --=20 2.27.0 From nobody Wed Dec 31 07:14:40 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FAE7C4332F for ; Mon, 6 Nov 2023 15:50:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232910AbjKFPuk (ORCPT ); Mon, 6 Nov 2023 10:50:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232766AbjKFPuR (ORCPT ); Mon, 6 Nov 2023 10:50:17 -0500 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54C8CD76; Mon, 6 Nov 2023 07:50:10 -0800 (PST) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4SPG1b3kWqz1P7xD; Mon, 6 Nov 2023 23:46:59 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:02 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 10/10] mm: convert mm_counter_file() to take a folio Date: Mon, 6 Nov 2023 23:49:50 +0800 Message-ID: <20231106154950.3399469-11-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since all mm_counter_file() callers with a folio, let's convert mm_counter_file() to take a folio. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 8 ++++---- kernel/events/uprobes.c | 2 +- mm/huge_memory.c | 5 +++-- mm/khugepaged.c | 4 ++-- mm/memory.c | 10 +++++----- mm/rmap.c | 2 +- 6 files changed, 16 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index fea78900bf84..95573065a46b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2583,10 +2583,10 @@ static inline void dec_mm_counter(struct mm_struct = *mm, int member) mm_trace_rss_stat(mm, member); } =20 -/* Optimized variant when page is already known not to be PageAnon */ -static inline int mm_counter_file(struct page *page) +/* Optimized variant when folio is already known not to be anon */ +static inline int mm_counter_file(struct folio *folio) { - if (PageSwapBacked(page)) + if (folio_test_swapbacked(folio)) return MM_SHMEMPAGES; return MM_FILEPAGES; } @@ -2595,7 +2595,7 @@ static inline int mm_counter(struct folio *folio) { if (folio_test_anon(folio)) return MM_ANONPAGES; - return mm_counter_file(&folio->page); + return mm_counter_file(folio); } =20 static inline unsigned long get_mm_rss(struct mm_struct *mm) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 435aac1d8c27..ce251e3a4ae6 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -188,7 +188,7 @@ static int __replace_page(struct vm_area_struct *vma, u= nsigned long addr, dec_mm_counter(mm, MM_ANONPAGES); =20 if (!folio_test_anon(old_folio)) { - dec_mm_counter(mm, mm_counter_file(old_page)); + dec_mm_counter(mm, mm_counter_file(old_folio)); inc_mm_counter(mm, MM_ANONPAGES); } =20 diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 78a00fe22c2d..88420d067477 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1742,7 +1742,8 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_ar= ea_struct *vma, } else { if (arch_needs_pgtable_deposit()) zap_deposited_table(tlb->mm, pmd); - add_mm_counter(tlb->mm, mm_counter_file(page), -HPAGE_PMD_NR); + add_mm_counter(tlb->mm, mm_counter_file(folio), + -HPAGE_PMD_NR); } =20 spin_unlock(ptl); @@ -2143,7 +2144,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, page_remove_rmap(&folio->page, vma, true); folio_put(folio); } - add_mm_counter(mm, mm_counter_file(&folio->page), -HPAGE_PMD_NR); + add_mm_counter(mm, mm_counter_file(folio), -HPAGE_PMD_NR); return; } =20 diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 064654717843..39393f4262b2 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1630,7 +1630,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, uns= igned long addr, /* step 3: set proper refcount and mm_counters. */ if (nr_ptes) { folio_ref_sub(folio, nr_ptes); - add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); + add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } =20 /* step 4: remove empty page table */ @@ -1661,7 +1661,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, uns= igned long addr, if (nr_ptes) { flush_tlb_mm(mm); folio_ref_sub(folio, nr_ptes); - add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); + add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } if (start_pte) pte_unmap_unlock(start_pte, ptl); diff --git a/mm/memory.c b/mm/memory.c index 3ffef84dd7bb..bf4ea31150a9 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -961,7 +961,7 @@ copy_present_pte(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, } else if (page) { folio_get(folio); page_dup_file_rmap(page, false); - rss[mm_counter_file(page)]++; + rss[mm_counter_file(folio)]++; } =20 /* @@ -1861,7 +1861,7 @@ static int insert_page_into_pte_locked(struct vm_area= _struct *vma, pte_t *pte, folio =3D page_folio(page); /* Ok, finally just insert the thing.. */ folio_get(folio); - inc_mm_counter(vma->vm_mm, mm_counter_file(page)); + inc_mm_counter(vma->vm_mm, mm_counter_file(folio)); page_add_file_rmap(page, vma, false); set_pte_at(vma->vm_mm, addr, pte, mk_pte(page, prot)); return 0; @@ -3170,7 +3170,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) if (likely(vmf->pte && pte_same(ptep_get(vmf->pte), vmf->orig_pte))) { if (old_folio) { if (!folio_test_anon(old_folio)) { - dec_mm_counter(mm, mm_counter_file(&old_folio->page)); + dec_mm_counter(mm, mm_counter_file(old_folio)); inc_mm_counter(mm, MM_ANONPAGES); } } else { @@ -4363,7 +4363,7 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct pa= ge *page) if (write) entry =3D maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); =20 - add_mm_counter(vma->vm_mm, mm_counter_file(page), HPAGE_PMD_NR); + add_mm_counter(vma->vm_mm, mm_counter_file(folio), HPAGE_PMD_NR); page_add_file_rmap(page, vma, true); =20 /* @@ -4426,7 +4426,7 @@ void set_pte_range(struct vm_fault *vmf, struct folio= *folio, folio_add_new_anon_rmap(folio, vma, addr); folio_add_lru_vma(folio, vma); } else { - add_mm_counter(vma->vm_mm, mm_counter_file(page), nr); + add_mm_counter(vma->vm_mm, mm_counter_file(folio), nr); folio_add_file_rmap_range(folio, page, nr, vma, false); } set_ptes(vma->vm_mm, addr, vmf->pte, entry, nr); diff --git a/mm/rmap.c b/mm/rmap.c index 7a563490ce08..9e3d0eff8b05 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1801,7 +1801,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, * * See Documentation/mm/mmu_notifier.rst */ - dec_mm_counter(mm, mm_counter_file(&folio->page)); + dec_mm_counter(mm, mm_counter_file(folio)); } discard: page_remove_rmap(subpage, vma, folio_test_hugetlb(folio)); --=20 2.27.0