From nobody Sun Apr 5 21:16:29 2026 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9412C1DFF0 for ; Tue, 24 Feb 2026 01:56:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.112 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771898194; cv=none; b=JvPgauhcZIChWwBnLyAa+ILybbIWe0lDZyLofP+AvRsg3j51792AdiEDfcA7JTPG4r4xO447U2qjxAzhZ7sKKKiYbC/K073rytGoJrd43+y8gXgjgwF9H0tY7/lGen+3u1YaCgP3jgPIPmiwANK1ybZFc7jS4Zg/6u8DXgpnxoU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771898194; c=relaxed/simple; bh=bNKn2v+xNPVNp5DfmTLHihfbDnwak5cNlJ8ViVUZW2Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fjmRkj4oC2SoQfjAAoYi4vsU8lYs9An9o1kSJ00j1VL5LD6qEPTkG1fwUE5EG/LFGv9IU3fTlhxyvJfr27kWBnpCug4yuvlHeOgzPsLMeiYraRmFwXJwt0ionYYn/04wiz8wJTNn0I8N1viZ0YPcy8nVmLqBivyX4TgfF6xjyRc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=NhGBWGGY; arc=none smtp.client-ip=115.124.30.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="NhGBWGGY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1771898189; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=4d1KkxuixEJPZL7S77r0HMbd94GuLo9p14SF+rl+Gpc=; b=NhGBWGGY2o92HSp67eKJSHCacJr800EoJn+Me3UNzRSLcuJDpbCPeB/4iaQX7D6975W2KDCCwS8lLlnBZRGSKJi9tOc3pPCcDUBNn8kB7JUzo3qYk/nsrz6RaZ5+xC7iZP3lxCwU9SmYQs/vWWOgS8+yzOlqvyn7mhN4MLAG5ig= Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WzhqZSC_1771898186 cluster:ay36) by smtp.aliyun-inc.com; Tue, 24 Feb 2026 09:56:27 +0800 From: Baolin Wang To: akpm@linux-foundation.org, david@kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, lorenzo.stoakes@oracle.com, ryan.roberts@arm.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, riel@surriel.com, harry.yoo@oracle.com, jannh@google.com, willy@infradead.org, baohua@kernel.org, dev.jain@arm.com, axelrasmussen@google.com, yuanchu@google.com, weixugc@google.com, hannes@cmpxchg.org, zhengqi.arch@bytedance.com, shakeel.butt@linux.dev, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/5] mm: add a batched helper to clear the young flag for large folios Date: Tue, 24 Feb 2026 09:56:06 +0800 Message-ID: X-Mailer: git-send-email 2.47.3 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, MGLRU will call ptep_clear_young_notify() to check and clear the young flag for each PTE sequentially, which is inefficient for large folios reclamation. Moreover, on Arm64 architecture, which supports contiguous PTEs, the Arm64- specific ptep_test_and_clear_young() already implements an optimization to clear the young flags for PTEs within a contiguous range. However, this is = not sufficient. Similar to the Arm64 specific clear_flush_young_ptes(), we can extend this to perform batched operations for the entire large folio (which might exceed the contiguous range: CONT_PTE_SIZE). Thus, we can introduce a new batched helper: test_and_clear_young_ptes() and its wrapper clear_young_ptes_notify(), to perform batched checking of the y= oung flags for large folios, which can help improve performance during large fol= io reclamation when MGLRU is enabled. And it will be overridden by the archite= cture that implements a more efficient batch operation in the following patches. Signed-off-by: Baolin Wang --- include/linux/pgtable.h | 36 ++++++++++++++++++++++++++++++++++++ mm/internal.h | 23 ++++++++++++++++++----- 2 files changed, 54 insertions(+), 5 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 776993d4567b..0bcd3be524d3 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1103,6 +1103,42 @@ static inline int clear_flush_young_ptes(struct vm_a= rea_struct *vma, } #endif =20 +#ifndef test_and_clear_young_ptes +/** + * test_and_clear_young_ptes - Mark PTEs that map consecutive pages of the= same + * folio as old + * @vma: The virtual memory area the pages are mapped into. + * @addr: Address the first page is mapped at. + * @ptep: Page table pointer for the first entry. + * @nr: Number of entries to clear access bit. + * + * May be overridden by the architecture; otherwise, implemented as a simp= le + * loop over ptep_test_and_clear_young(). + * + * Note that PTE bits in the PTE range besides the PFN can differ. For exa= mple, + * some PTEs might be write-protected. + * + * Context: The caller holds the page table lock. The PTEs map consecutive + * pages that belong to the same folio. The PTEs are all in the same PMD. + */ +static inline int test_and_clear_young_ptes(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + unsigned int nr) +{ + int young =3D 0; + + for (;;) { + young |=3D ptep_test_and_clear_young(vma, addr, ptep); + if (--nr =3D=3D 0) + break; + ptep++; + addr +=3D PAGE_SIZE; + } + + return young; +} +#endif + /* * On some architectures hardware does not set page access bit when access= ing * memory page, it is responsibility of software setting this bit. It brin= gs diff --git a/mm/internal.h b/mm/internal.h index 1ba175b8d4f1..1b59be99dc3f 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1813,16 +1813,23 @@ static inline int pmdp_clear_flush_young_notify(str= uct vm_area_struct *vma, return young; } =20 -static inline int ptep_clear_young_notify(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) +static inline int clear_young_ptes_notify(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + unsigned int nr) { int young; =20 - young =3D ptep_test_and_clear_young(vma, addr, ptep); - young |=3D mmu_notifier_clear_young(vma->vm_mm, addr, addr + PAGE_SIZE); + young =3D test_and_clear_young_ptes(vma, addr, ptep, nr); + young |=3D mmu_notifier_clear_young(vma->vm_mm, addr, addr + nr * PAGE_SI= ZE); return young; } =20 +static inline int ptep_clear_young_notify(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ + return clear_young_ptes_notify(vma, addr, ptep, 1); +} + static inline int pmdp_clear_young_notify(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmdp) { @@ -1837,9 +1844,15 @@ static inline int pmdp_clear_young_notify(struct vm_= area_struct *vma, =20 #define clear_flush_young_ptes_notify clear_flush_young_ptes #define pmdp_clear_flush_young_notify pmdp_clear_flush_young -#define ptep_clear_young_notify ptep_test_and_clear_young +#define clear_young_ptes_notify test_and_clear_young_ptes #define pmdp_clear_young_notify pmdp_test_and_clear_young =20 +static inline int ptep_clear_young_notify(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ + return test_and_clear_young_ptes(vma, addr, ptep, 1); +} + #endif /* CONFIG_MMU_NOTIFIER */ =20 #endif /* __MM_INTERNAL_H */ --=20 2.47.3