From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66035C0018A for ; Sat, 4 Nov 2023 03:56:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346238AbjKDD4B (ORCPT ); Fri, 3 Nov 2023 23:56:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59648 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234316AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B5141AA; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkKh6BgFzvQ5W; Sat, 4 Nov 2023 11:55:36 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:40 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 01/10] mm: swap: introduce pfn_swap_entry_to_folio() Date: Sat, 4 Nov 2023 11:55:13 +0800 Message-ID: <20231104035522.2418660-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Introduce a new pfn_swap_entry_to_folio(), it is similar to pfn_swap_entry_to_page(), but return a folio, which allow us to completely replace the struct page variables with struct folio variables. Signed-off-by: Kefeng Wang --- include/linux/swapops.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index bff1e8d97de0..85cb84e4be95 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -468,6 +468,19 @@ static inline struct page *pfn_swap_entry_to_page(swp_= entry_t entry) return p; } =20 +static inline struct folio *pfn_swap_entry_to_folio(swp_entry_t entry) +{ + struct folio *folio =3D pfn_folio(swp_offset_pfn(entry)); + + /* + * Any use of migration entries may only occur while the + * corresponding folio is locked + */ + BUG_ON(is_migration_entry(entry) && !folio_test_locked(folio)); + + return folio; +} + /* * A pfn swap entry is a special type of swap entry that always has a pfn = stored * in the swap offset. They are used to represent unaddressable device mem= ory --=20 2.27.0 From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C09B1C4332F for ; Sat, 4 Nov 2023 03:56:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376863AbjKDD4J (ORCPT ); Fri, 3 Nov 2023 23:56:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234297AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B5CDD45; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkGD0WzxzrTQX; Sat, 4 Nov 2023 11:52:36 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:41 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 02/10] s390: pgtable: use a folio in ptep_zap_swap_entry() Date: Sat, 4 Nov 2023 11:55:14 +0800 Message-ID: <20231104035522.2418660-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in ptep_zap_swap_entry(), which is preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- arch/s390/mm/pgtable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 3bd2ab2a9a34..2f946b493fff 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -730,9 +730,9 @@ static void ptep_zap_swap_entry(struct mm_struct *mm, s= wp_entry_t entry) if (!non_swap_entry(entry)) dec_mm_counter(mm, MM_SWAPENTS); else if (is_migration_entry(entry)) { - struct page *page =3D pfn_swap_entry_to_page(entry); + struct folio *folio =3D pfn_swap_entry_to_folio(entry); =20 - dec_mm_counter(mm, mm_counter(page)); + dec_mm_counter(mm, mm_counter(&folio->page)); } free_swap_and_cache(entry); } --=20 2.27.0 From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 822C1C4167B for ; Sat, 4 Nov 2023 03:56:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1377972AbjKDD4G (ORCPT ); Fri, 3 Nov 2023 23:56:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234308AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B3CD1A8; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkGD3sw9zrTX9; Sat, 4 Nov 2023 11:52:36 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:41 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 03/10] mm: huge_memory: use a folio in __split_huge_pmd_locked() Date: Sat, 4 Nov 2023 11:55:15 +0800 Message-ID: <20231104035522.2418660-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in __split_huge_pmd_locked() which replaces five compound_head() call with two page_folio() calls. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f31f02472396..34dd01970927 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2117,6 +2117,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, count_vm_event(THP_SPLIT_PMD); =20 if (!vma_is_anonymous(vma)) { + struct folio *folio; old_pmd =3D pmdp_huge_clear_flush(vma, haddr, pmd); /* * We are going to unmap this huge page. So @@ -2130,17 +2131,17 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, swp_entry_t entry; =20 entry =3D pmd_to_swp_entry(old_pmd); - page =3D pfn_swap_entry_to_page(entry); + folio =3D pfn_swap_entry_to_folio(entry); } else { - page =3D pmd_page(old_pmd); - if (!PageDirty(page) && pmd_dirty(old_pmd)) - set_page_dirty(page); - if (!PageReferenced(page) && pmd_young(old_pmd)) - SetPageReferenced(page); - page_remove_rmap(page, vma, true); - put_page(page); + folio =3D page_folio(pmd_page(old_pmd)); + if (!folio_test_dirty(folio) && pmd_dirty(old_pmd)) + folio_set_dirty(folio); + if (!folio_test_referenced(folio) && pmd_young(old_pmd)) + folio_set_referenced(folio); + page_remove_rmap(&folio->page, vma, true); + folio_put(folio); } - add_mm_counter(mm, mm_counter_file(page), -HPAGE_PMD_NR); + add_mm_counter(mm, mm_counter_file(&folio->page), -HPAGE_PMD_NR); return; } =20 --=20 2.27.0 From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45DA1C41535 for ; Sat, 4 Nov 2023 03:56:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345612AbjKDDzu (ORCPT ); Fri, 3 Nov 2023 23:55:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233666AbjKDDzr (ORCPT ); Fri, 3 Nov 2023 23:55:47 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A5EB194; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SMkF31wjJzPnp7; Sat, 4 Nov 2023 11:51:35 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:42 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 04/10] mm: huge_memory: use a folio in zap_huge_pmd() Date: Sat, 4 Nov 2023 11:55:16 +0800 Message-ID: <20231104035522.2418660-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in zap_huge_pmd(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 34dd01970927..78a00fe22c2d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1716,11 +1716,13 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_= area_struct *vma, zap_deposited_table(tlb->mm, pmd); spin_unlock(ptl); } else { - struct page *page =3D NULL; + struct folio *folio =3D NULL; int flush_needed =3D 1; =20 if (pmd_present(orig_pmd)) { - page =3D pmd_page(orig_pmd); + struct page *page =3D pmd_page(orig_pmd); + + folio =3D page_folio(page); page_remove_rmap(page, vma, true); VM_BUG_ON_PAGE(page_mapcount(page) < 0, page); VM_BUG_ON_PAGE(!PageHead(page), page); @@ -1729,12 +1731,12 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_= area_struct *vma, =20 VM_BUG_ON(!is_pmd_migration_entry(orig_pmd)); entry =3D pmd_to_swp_entry(orig_pmd); - page =3D pfn_swap_entry_to_page(entry); + folio =3D pfn_swap_entry_to_folio(entry); flush_needed =3D 0; } else WARN_ONCE(1, "Non present huge pmd without pmd migration enabled!"); =20 - if (PageAnon(page)) { + if (folio_test_anon(folio)) { zap_deposited_table(tlb->mm, pmd); add_mm_counter(tlb->mm, MM_ANONPAGES, -HPAGE_PMD_NR); } else { @@ -1745,7 +1747,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_ar= ea_struct *vma, =20 spin_unlock(ptl); if (flush_needed) - tlb_remove_page_size(tlb, page, HPAGE_PMD_SIZE); + tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE); } return 1; } --=20 2.27.0 From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78096C4332F for ; Sat, 4 Nov 2023 03:56:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234297AbjKDD4P (ORCPT ); Fri, 3 Nov 2023 23:56:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234331AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C57E6D50; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4SMkGG17Kyz1P7rh; Sat, 4 Nov 2023 11:52:38 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:42 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 05/10] mm: memory: use a folio in copy_nonpresent_pte() Date: Sat, 4 Nov 2023 11:55:17 +0800 Message-ID: <20231104035522.2418660-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in copy_nonpresent_pte() to save one compound_head() call. Signed-off-by: Kefeng Wang --- mm/memory.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 1f18ed4a5497..d9314dee355e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -779,7 +779,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, unsigned long vm_flags =3D dst_vma->vm_flags; pte_t orig_pte =3D ptep_get(src_pte); pte_t pte =3D orig_pte; - struct page *page; + struct folio *folio; swp_entry_t entry =3D pte_to_swp_entry(orig_pte); =20 if (likely(!non_swap_entry(entry))) { @@ -801,9 +801,9 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, } rss[MM_SWAPENTS]++; } else if (is_migration_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + folio =3D pfn_swap_entry_to_folio(entry); =20 - rss[mm_counter(page)]++; + rss[mm_counter(&folio->page)]++; =20 if (!is_readable_migration_entry(entry) && is_cow_mapping(vm_flags)) { @@ -822,7 +822,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, set_pte_at(src_mm, addr, src_pte, pte); } } else if (is_device_private_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + folio =3D pfn_swap_entry_to_folio(entry); =20 /* * Update rss count even for unaddressable pages, as @@ -833,10 +833,10 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, * for unaddressable pages, at some point. But for now * keep things as they are. */ - get_page(page); - rss[mm_counter(page)]++; + folio_get(folio); + rss[mm_counter(&folio->page)]++; /* Cannot fail as these pages cannot get pinned. */ - BUG_ON(page_try_dup_anon_rmap(page, false, src_vma)); + BUG_ON(page_try_dup_anon_rmap(&folio->page, false, src_vma)); =20 /* * We do not preserve soft-dirty information, because so --=20 2.27.0 From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77ED3C4332F for ; Sat, 4 Nov 2023 03:56:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1377977AbjKDD4S (ORCPT ); Fri, 3 Nov 2023 23:56:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343547AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6FB15D51; Fri, 3 Nov 2023 20:55:45 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4SMkDl5J28zMlhj; Sat, 4 Nov 2023 11:51:19 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:43 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 06/10] mm: memory: use a folio in zap_pte_range() Date: Sat, 4 Nov 2023 11:55:18 +0800 Message-ID: <20231104035522.2418660-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Make should_zap_page() to take a folio and use a folio in zap_pte_range(), which save several compound_head() calls. Signed-off-by: Kefeng Wang --- mm/memory.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index d9314dee355e..806568f9605b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1358,19 +1358,19 @@ static inline bool should_zap_cows(struct zap_detai= ls *details) return details->even_cows; } =20 -/* Decides whether we should zap this page with the page pointer specified= */ -static inline bool should_zap_page(struct zap_details *details, struct pag= e *page) +/* Decides whether we should zap this folio with the folio pointer specifi= ed */ +static inline bool should_zap_page(struct zap_details *details, struct fol= io *folio) { - /* If we can make a decision without *page.. */ + /* If we can make a decision without *folio.. */ if (should_zap_cows(details)) return true; =20 - /* E.g. the caller passes NULL for the case of a zero page */ - if (!page) + /* E.g. the caller passes NULL for the case of a zero folio */ + if (!folio) return true; =20 - /* Otherwise we should only zap non-anon pages */ - return !PageAnon(page); + /* Otherwise we should only zap non-anon folios */ + return !folio_test_anon(folio); } =20 static inline bool zap_drop_file_uffd_wp(struct zap_details *details) @@ -1423,7 +1423,7 @@ static unsigned long zap_pte_range(struct mmu_gather = *tlb, arch_enter_lazy_mmu_mode(); do { pte_t ptent =3D ptep_get(pte); - struct page *page; + struct folio *folio =3D NULL; =20 if (pte_none(ptent)) continue; @@ -1433,9 +1433,13 @@ static unsigned long zap_pte_range(struct mmu_gather= *tlb, =20 if (pte_present(ptent)) { unsigned int delay_rmap; + struct page *page; =20 page =3D vm_normal_page(vma, addr, ptent); - if (unlikely(!should_zap_page(details, page))) + if (page) + folio =3D page_folio(page); + + if (unlikely(!should_zap_page(details, folio))) continue; ptent =3D ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); @@ -1449,16 +1453,16 @@ static unsigned long zap_pte_range(struct mmu_gathe= r *tlb, } =20 delay_rmap =3D 0; - if (!PageAnon(page)) { + if (!folio_test_anon(folio)) { if (pte_dirty(ptent)) { - set_page_dirty(page); + folio_set_dirty(folio); if (tlb_delay_rmap(tlb)) { delay_rmap =3D 1; force_flush =3D 1; } } if (pte_young(ptent) && likely(vma_has_recency(vma))) - mark_page_accessed(page); + folio_mark_accessed(folio); } rss[mm_counter(page)]--; if (!delay_rmap) { @@ -1477,9 +1481,10 @@ static unsigned long zap_pte_range(struct mmu_gather= *tlb, entry =3D pte_to_swp_entry(ptent); if (is_device_private_entry(entry) || is_device_exclusive_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); - if (unlikely(!should_zap_page(details, page))) + folio =3D pfn_swap_entry_to_folio(entry); + if (unlikely(!should_zap_page(details, folio))) continue; + /* * Both device private/exclusive mappings should only * work with anonymous page so far, so we don't need to @@ -1487,10 +1492,10 @@ static unsigned long zap_pte_range(struct mmu_gathe= r *tlb, * see zap_install_uffd_wp_if_needed(). */ WARN_ON_ONCE(!vma_is_anonymous(vma)); - rss[mm_counter(page)]--; + rss[mm_counter(&folio->page)]--; if (is_device_private_entry(entry)) - page_remove_rmap(page, vma, false); - put_page(page); + page_remove_rmap(&folio->page, vma, false); + folio_put(folio); } else if (!non_swap_entry(entry)) { /* Genuine swap entry, hence a private anon page */ if (!should_zap_cows(details)) @@ -1499,10 +1504,10 @@ static unsigned long zap_pte_range(struct mmu_gathe= r *tlb, if (unlikely(!free_swap_and_cache(entry))) print_bad_pte(vma, addr, ptent, NULL); } else if (is_migration_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); - if (!should_zap_page(details, page)) + folio =3D pfn_swap_entry_to_folio(entry); + if (!should_zap_page(details, folio)) continue; - rss[mm_counter(page)]--; + rss[mm_counter(&folio->page)]--; } else if (pte_marker_entry_uffd_wp(entry)) { /* * For anon: always drop the marker; for file: only --=20 2.27.0 From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B456C4167D for ; Sat, 4 Nov 2023 03:56:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378043AbjKDD4M (ORCPT ); Fri, 3 Nov 2023 23:56:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234193AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81CFBD53; Fri, 3 Nov 2023 20:55:45 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkKl453TzvQCR; Sat, 4 Nov 2023 11:55:39 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:43 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 07/10] mm: memory: use a folio in do_set_pmd() Date: Sat, 4 Nov 2023 11:55:19 +0800 Message-ID: <20231104035522.2418660-8-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in do_set_pmd(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 806568f9605b..ac247850919a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4318,12 +4318,13 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct = page *page) unsigned long haddr =3D vmf->address & HPAGE_PMD_MASK; pmd_t entry; vm_fault_t ret =3D VM_FAULT_FALLBACK; + struct folio *folio; =20 if (!transhuge_vma_suitable(vma, haddr)) return ret; =20 - page =3D compound_head(page); - if (compound_order(page) !=3D HPAGE_PMD_ORDER) + folio =3D page_folio(page); + if (folio_order(folio) !=3D HPAGE_PMD_ORDER) return ret; =20 /* --=20 2.27.0 From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C60CAC4332F for ; Sat, 4 Nov 2023 03:56:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346053AbjKDD4U (ORCPT ); Fri, 3 Nov 2023 23:56:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343779AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08EAFD54; Fri, 3 Nov 2023 20:55:46 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4SMkDm4Z5TzMmQq; Sat, 4 Nov 2023 11:51:20 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:43 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 08/10] mm: memory: use a folio in insert_page_into_pte_locked() Date: Sat, 4 Nov 2023 11:55:20 +0800 Message-ID: <20231104035522.2418660-9-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use a folio in insert_page_into_pte_locked(), which is preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index ac247850919a..a2cf240b1975 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1850,10 +1850,13 @@ static int validate_page_before_insert(struct page = *page) static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *= pte, unsigned long addr, struct page *page, pgprot_t prot) { + struct folio *folio; + if (!pte_none(ptep_get(pte))) return -EBUSY; + folio =3D page_folio(page); /* Ok, finally just insert the thing.. */ - get_page(page); + folio_get(folio); inc_mm_counter(vma->vm_mm, mm_counter_file(page)); page_add_file_rmap(page, vma, false); set_pte_at(vma->vm_mm, addr, pte, mk_pte(page, prot)); --=20 2.27.0 From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78DD6C4167B for ; Sat, 4 Nov 2023 03:56:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378157AbjKDD41 (ORCPT ); Fri, 3 Nov 2023 23:56:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345321AbjKDDzu (ORCPT ); Fri, 3 Nov 2023 23:55:50 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7408F194; Fri, 3 Nov 2023 20:55:46 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4SMkDn0qXmzMmSn; Sat, 4 Nov 2023 11:51:21 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:44 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 09/10] mm: convert mm_counter() to take a folio Date: Sat, 4 Nov 2023 11:55:21 +0800 Message-ID: <20231104035522.2418660-10-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since all mm_counter() callers with a folio, let's convert mm_counter() to take a folio. Signed-off-by: Kefeng Wang --- arch/s390/mm/pgtable.c | 2 +- include/linux/mm.h | 6 +++--- mm/memory.c | 10 +++++----- mm/rmap.c | 8 ++++---- mm/userfaultfd.c | 2 +- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 2f946b493fff..54b184648db6 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -732,7 +732,7 @@ static void ptep_zap_swap_entry(struct mm_struct *mm, s= wp_entry_t entry) else if (is_migration_entry(entry)) { struct folio *folio =3D pfn_swap_entry_to_folio(entry); =20 - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } free_swap_and_cache(entry); } diff --git a/include/linux/mm.h b/include/linux/mm.h index 418d26608ece..fea78900bf84 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2591,11 +2591,11 @@ static inline int mm_counter_file(struct page *page) return MM_FILEPAGES; } =20 -static inline int mm_counter(struct page *page) +static inline int mm_counter(struct folio *folio) { - if (PageAnon(page)) + if (folio_test_anon(folio)) return MM_ANONPAGES; - return mm_counter_file(page); + return mm_counter_file(&folio->page); } =20 static inline unsigned long get_mm_rss(struct mm_struct *mm) diff --git a/mm/memory.c b/mm/memory.c index a2cf240b1975..ad30d4ad2223 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -803,7 +803,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, } else if (is_migration_entry(entry)) { folio =3D pfn_swap_entry_to_folio(entry); =20 - rss[mm_counter(&folio->page)]++; + rss[mm_counter(folio)]++; =20 if (!is_readable_migration_entry(entry) && is_cow_mapping(vm_flags)) { @@ -834,7 +834,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, * keep things as they are. */ folio_get(folio); - rss[mm_counter(&folio->page)]++; + rss[mm_counter(folio)]++; /* Cannot fail as these pages cannot get pinned. */ BUG_ON(page_try_dup_anon_rmap(&folio->page, false, src_vma)); =20 @@ -1464,7 +1464,7 @@ static unsigned long zap_pte_range(struct mmu_gather = *tlb, if (pte_young(ptent) && likely(vma_has_recency(vma))) folio_mark_accessed(folio); } - rss[mm_counter(page)]--; + rss[mm_counter(folio)]--; if (!delay_rmap) { page_remove_rmap(page, vma, false); if (unlikely(page_mapcount(page) < 0)) @@ -1492,7 +1492,7 @@ static unsigned long zap_pte_range(struct mmu_gather = *tlb, * see zap_install_uffd_wp_if_needed(). */ WARN_ON_ONCE(!vma_is_anonymous(vma)); - rss[mm_counter(&folio->page)]--; + rss[mm_counter(folio)]--; if (is_device_private_entry(entry)) page_remove_rmap(&folio->page, vma, false); folio_put(folio); @@ -1507,7 +1507,7 @@ static unsigned long zap_pte_range(struct mmu_gather = *tlb, folio =3D pfn_swap_entry_to_folio(entry); if (!should_zap_page(details, folio)) continue; - rss[mm_counter(&folio->page)]--; + rss[mm_counter(folio)]--; } else if (pte_marker_entry_uffd_wp(entry)) { /* * For anon: always drop the marker; for file: only diff --git a/mm/rmap.c b/mm/rmap.c index 7a27a2b41802..7a563490ce08 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1678,7 +1678,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } =20 @@ -1693,7 +1693,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else if (folio_test_anon(folio)) { swp_entry_t entry =3D page_swap_entry(subpage); pte_t swp_pte; @@ -2075,7 +2075,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } =20 @@ -2090,7 +2090,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else { swp_entry_t entry; pte_t swp_pte; diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 96d9eae5c7cc..9a6759fa9b06 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -124,7 +124,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, * Must happen after rmap, as mm_counter() checks mapping (via * PageAnon()), which is set by __page_set_anon_rmap(). */ - inc_mm_counter(dst_mm, mm_counter(page)); + inc_mm_counter(dst_mm, mm_counter(folio)); =20 set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); =20 --=20 2.27.0 From nobody Wed Dec 31 10:51:33 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD17CC4332F for ; Sat, 4 Nov 2023 03:56:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346210AbjKDD4e (ORCPT ); Fri, 3 Nov 2023 23:56:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345405AbjKDDzu (ORCPT ); Fri, 3 Nov 2023 23:55:50 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D72B5D47; Fri, 3 Nov 2023 20:55:46 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkGH51DZzrTXs; Sat, 4 Nov 2023 11:52:39 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:44 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 10/10] mm: convert mm_counter_file() to take a folio Date: Sat, 4 Nov 2023 11:55:22 +0800 Message-ID: <20231104035522.2418660-11-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since all mm_counter_file() callers with a folio, let's convert mm_counter_file() to take a folio. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 8 ++++---- kernel/events/uprobes.c | 2 +- mm/huge_memory.c | 5 +++-- mm/khugepaged.c | 4 ++-- mm/memory.c | 10 +++++----- mm/rmap.c | 2 +- 6 files changed, 16 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index fea78900bf84..95573065a46b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2583,10 +2583,10 @@ static inline void dec_mm_counter(struct mm_struct = *mm, int member) mm_trace_rss_stat(mm, member); } =20 -/* Optimized variant when page is already known not to be PageAnon */ -static inline int mm_counter_file(struct page *page) +/* Optimized variant when folio is already known not to be anon */ +static inline int mm_counter_file(struct folio *folio) { - if (PageSwapBacked(page)) + if (folio_test_swapbacked(folio)) return MM_SHMEMPAGES; return MM_FILEPAGES; } @@ -2595,7 +2595,7 @@ static inline int mm_counter(struct folio *folio) { if (folio_test_anon(folio)) return MM_ANONPAGES; - return mm_counter_file(&folio->page); + return mm_counter_file(folio); } =20 static inline unsigned long get_mm_rss(struct mm_struct *mm) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 435aac1d8c27..ce251e3a4ae6 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -188,7 +188,7 @@ static int __replace_page(struct vm_area_struct *vma, u= nsigned long addr, dec_mm_counter(mm, MM_ANONPAGES); =20 if (!folio_test_anon(old_folio)) { - dec_mm_counter(mm, mm_counter_file(old_page)); + dec_mm_counter(mm, mm_counter_file(old_folio)); inc_mm_counter(mm, MM_ANONPAGES); } =20 diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 78a00fe22c2d..88420d067477 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1742,7 +1742,8 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_ar= ea_struct *vma, } else { if (arch_needs_pgtable_deposit()) zap_deposited_table(tlb->mm, pmd); - add_mm_counter(tlb->mm, mm_counter_file(page), -HPAGE_PMD_NR); + add_mm_counter(tlb->mm, mm_counter_file(folio), + -HPAGE_PMD_NR); } =20 spin_unlock(ptl); @@ -2143,7 +2144,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, page_remove_rmap(&folio->page, vma, true); folio_put(folio); } - add_mm_counter(mm, mm_counter_file(&folio->page), -HPAGE_PMD_NR); + add_mm_counter(mm, mm_counter_file(folio), -HPAGE_PMD_NR); return; } =20 diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 064654717843..39393f4262b2 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1630,7 +1630,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, uns= igned long addr, /* step 3: set proper refcount and mm_counters. */ if (nr_ptes) { folio_ref_sub(folio, nr_ptes); - add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); + add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } =20 /* step 4: remove empty page table */ @@ -1661,7 +1661,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, uns= igned long addr, if (nr_ptes) { flush_tlb_mm(mm); folio_ref_sub(folio, nr_ptes); - add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); + add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } if (start_pte) pte_unmap_unlock(start_pte, ptl); diff --git a/mm/memory.c b/mm/memory.c index ad30d4ad2223..3418ace5e0ad 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -960,7 +960,7 @@ copy_present_pte(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, } else if (page) { folio_get(folio); page_dup_file_rmap(page, false); - rss[mm_counter_file(page)]++; + rss[mm_counter_file(folio)]++; } =20 /* @@ -1857,7 +1857,7 @@ static int insert_page_into_pte_locked(struct vm_area= _struct *vma, pte_t *pte, folio =3D page_folio(page); /* Ok, finally just insert the thing.. */ folio_get(folio); - inc_mm_counter(vma->vm_mm, mm_counter_file(page)); + inc_mm_counter(vma->vm_mm, mm_counter_file(folio)); page_add_file_rmap(page, vma, false); set_pte_at(vma->vm_mm, addr, pte, mk_pte(page, prot)); return 0; @@ -3166,7 +3166,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) if (likely(vmf->pte && pte_same(ptep_get(vmf->pte), vmf->orig_pte))) { if (old_folio) { if (!folio_test_anon(old_folio)) { - dec_mm_counter(mm, mm_counter_file(&old_folio->page)); + dec_mm_counter(mm, mm_counter_file(old_folio)); inc_mm_counter(mm, MM_ANONPAGES); } } else { @@ -4359,7 +4359,7 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct pa= ge *page) if (write) entry =3D maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); =20 - add_mm_counter(vma->vm_mm, mm_counter_file(page), HPAGE_PMD_NR); + add_mm_counter(vma->vm_mm, mm_counter_file(folio), HPAGE_PMD_NR); page_add_file_rmap(page, vma, true); =20 /* @@ -4422,7 +4422,7 @@ void set_pte_range(struct vm_fault *vmf, struct folio= *folio, folio_add_new_anon_rmap(folio, vma, addr); folio_add_lru_vma(folio, vma); } else { - add_mm_counter(vma->vm_mm, mm_counter_file(page), nr); + add_mm_counter(vma->vm_mm, mm_counter_file(folio), nr); folio_add_file_rmap_range(folio, page, nr, vma, false); } set_ptes(vma->vm_mm, addr, vmf->pte, entry, nr); diff --git a/mm/rmap.c b/mm/rmap.c index 7a563490ce08..9e3d0eff8b05 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1801,7 +1801,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, * * See Documentation/mm/mmu_notifier.rst */ - dec_mm_counter(mm, mm_counter_file(&folio->page)); + dec_mm_counter(mm, mm_counter_file(folio)); } discard: page_remove_rmap(subpage, vma, folio_test_hugetlb(folio)); --=20 2.27.0