From nobody Sat Feb 7 08:44:03 2026 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79FB733C198 for ; Fri, 23 Jan 2026 08:23:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156582; cv=none; b=I/goaQun1xD7RqQgRPC40SXxqVr24DpeppGbYJSmqdAfdaKSAJgTwSAcyD6Rti4EfZ8IgjyLEWeLoxtBle1pUE33JaarKjGpLqD2JvTNlcygXeG3tVUYcwK7K5yxZu2pB8twAFNC291Atm8Cazd5CfUlcqO5tLqcBaqoJXn4BTw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156582; c=relaxed/simple; bh=NQsQPSRmrBOkCJtFGLiY85wqwbfW7VPqhPFAOPL68QY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Txe6Qo2uN1EM9tOM3dJ0YNoEgsAViJeiXsRAhke5wcQwaMb8OaEq0VercigU0C1HRd39o/BFUj+atFc5D/iaum3pSg64hjyxnOPlZzGZeHNXTjYrzptRepicuu/JtlqDFXU5UpAdQ9XLvUQMeZZA4WSZTaTeXwVC0geJTd8a+Q0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OcZiwx+r; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OcZiwx+r" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-81f47610542so1072332b3a.0 for ; Fri, 23 Jan 2026 00:23:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769156581; x=1769761381; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NPVrcdkZH2YRU1Jo6XFkYQwOXRIVFhmIZ9Yfhc/+QHk=; b=OcZiwx+rBJFf5N4pdHBAD7CUoyqlp1ouHBbPMz5pXAkZr0O9hKOGftjj41aav8HG5z HhYgjf6CCrakZY9oovRxToJkjjiRj6AqZkO8XjCyEV+Alhh+lI1B1qxQLRfcLE55bRlT qyhuar7U/n46D4C68NiyJRxCfpY6w4H7PK92deHKBYKyL3B0BjK5q9lUcw7M5qILVfR2 z3FuIwYjTovzpubUsPTdRIsnl5R+C3QBQX4Ok3mrsZomNr7a8NhzhRO74cpPJpaDKFpC awzy1GlNBsb/1fyFZrwmlTYz7rjUkibAookNHOOsqOBfqmuWt5fP1LQKvSiykEGkVn6c F3/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769156581; x=1769761381; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=NPVrcdkZH2YRU1Jo6XFkYQwOXRIVFhmIZ9Yfhc/+QHk=; b=ek88zhi391fHDpjDcDcqIxWU6EyjA4AIeTgnQOuXxf2QSdCzLBluif1cqy6kho4wja 41r3HOP/7GrT0iYZoPDQbLf94sRuXqQcaNy1ywykN3JOm14RXMBrUHTwifA+ITXrkZHz pzEj88JA1ApHoe94vDJzEA/DpgHwDRLPDrBefEdIxHuvpWPfEBEejo2HN73Kdx9pchbF Ht6WLMqO7M1r6AfLUazcTziyxHQ5/GJldgmfC6Mm0USt+iwemCtrPvKtbYGEcBRHAz4U q0aByEfw+OecQzXxKsTb1OAVv/td7JZxUEMHvd4mwMknVpOgRxduT7pu3nanwrValT1W 2T0w== X-Forwarded-Encrypted: i=1; AJvYcCV2gRxg1OnXX4o0Y/qQ0cQs+Vo+/7Wer5vymVbjfWgpd+w/cpyU5hgHjF+Ekl/LhkIvlv96tHl9FBeRHIc=@vger.kernel.org X-Gm-Message-State: AOJu0YyGsKOoG63UuBZ48Gw8olzoDZtR5+luq5CJLF86b/VcshIJHgIP NorNMGyZsavq6h5/WMAeLJNRGfoE98QHXjQ7j2nWTl9Ooi/VNGtK+50C X-Gm-Gg: AZuq6aK3lMWd6m1gUEDJSO5sSNue0CKOfYxzpHHbvRqx9jdqgktMvO/0kSFZ1Xd7pab h54hJJLAsvC2P2ksa5yuzaJt5tB5xKEAAUiulyXMgDWmQYtcZMAI/ikEQ1f4KSw/TtFMf2Keyux VYfQeUwDYoDMdbvGgMEHzDkJav0hs6ZENqIDCfJUc1F4Ik0wNBsPETcIj4DCcBRAI+WU8bHmZyO Ekaz7ZWT6/EqrrhIUH3Nrnl9WkJQAhi397J4ZnlsRxZgSBio/DvVbfky+/Q5LpblX/HYXJ+UbY2 YXDLnZABvfnIQklk4f7AbY7xFPrpmdNS450Ns1WBEmv+I1TsPF/AewherrGGQDVTC7Bj5pc2iho 3c5iuVp2KOntfEIRTTcQF/t+UYcatwMp0eRK4TLc9kU0P6RNakzNN1v5HDtTUf3TUGWWhPw7VJE i80MAYaiVOLgTxXYBgDqyla9E53xfACmcm8F4= X-Received: by 2002:a05:6a20:7490:b0:366:14ac:e20a with SMTP id adf61e73a8af0-38e77340a5dmr644811637.72.1769156580724; Fri, 23 Jan 2026 00:23:00 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c635a43f11csm1348363a12.35.2026.01.23.00.22.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Jan 2026 00:23:00 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v5 1/5] mm: khugepaged: add trace_mm_khugepaged_scan event Date: Fri, 23 Jan 2026 16:22:28 +0800 Message-ID: <20260123082232.16413-2-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260123082232.16413-1-vernon2gm@gmail.com> References: <20260123082232.16413-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang Add mm_khugepaged_scan event to track the total time for full scan and the total number of pages scanned of khugepaged. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Barry Song Reviewed-by: Lance Yang Reviewed-by: Dev Jain --- include/trace/events/huge_memory.h | 25 +++++++++++++++++++++++++ mm/khugepaged.c | 2 ++ 2 files changed, 27 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 4e41bff31888..384e29f6bef0 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -237,5 +237,30 @@ TRACE_EVENT(mm_khugepaged_collapse_file, __print_symbolic(__entry->result, SCAN_STATUS)) ); =20 +TRACE_EVENT(mm_khugepaged_scan, + + TP_PROTO(struct mm_struct *mm, unsigned int progress, + bool full_scan_finished), + + TP_ARGS(mm, progress, full_scan_finished), + + TP_STRUCT__entry( + __field(struct mm_struct *, mm) + __field(unsigned int, progress) + __field(bool, full_scan_finished) + ), + + TP_fast_assign( + __entry->mm =3D mm; + __entry->progress =3D progress; + __entry->full_scan_finished =3D full_scan_finished; + ), + + TP_printk("mm=3D%p, progress=3D%u, full_scan_finished=3D%d", + __entry->mm, + __entry->progress, + __entry->full_scan_finished) +); + #endif /* __HUGE_MEMORY_H */ #include diff --git a/mm/khugepaged.c b/mm/khugepaged.c index e76f42243534..6f0f05148765 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2535,6 +2535,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result collect_mm_slot(slot); } =20 + trace_mm_khugepaged_scan(mm, progress, khugepaged_scan.mm_slot =3D=3D NUL= L); + return progress; } =20 --=20 2.51.0 From nobody Sat Feb 7 08:44:03 2026 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3BF333E367 for ; Fri, 23 Jan 2026 08:23:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156586; cv=none; b=Ka5XFd+yQDi8W0vRumEf1gIDFuxxKMCCcr/hb27OdcF+Sn2plvKaAAvPSedmP1Dau7ljXdzJQpIEir/H9K4bgo/3HeCHHucWp3LXd1JpHE1hOdRZ1qLo0sBr+LxRzjxaJvGXZknz4uaFRVWMON3UQV8pFN2OBT0tA7HbZRijy+I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156586; c=relaxed/simple; bh=IkHfdbJmzL0qf+WIlOJlSWUnMkFUcaiF6Vhs50DkoNA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mT0I1OorV1Z8iHLr0woIThs+L5K+zRL9KDbPXuSpuL4P3mVEdSw1y6NkCcLqY6ZBgUfgZCWNJJy4sjF5koSARo6XwN8t9jWpS5shdCJm/swZZbOij2+nColT7QhiW9BaurZw1QNJrZiHvh0PjlhMpl74DakgoTIfA85F1hv3AJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kF8HVcXG; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kF8HVcXG" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-7f89d0b37f0so1526069b3a.0 for ; Fri, 23 Jan 2026 00:23:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769156584; x=1769761384; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gNFjSEtLjmo3zavrqqxSndVZ5Mb9BZ08qMwOVvVVYnk=; b=kF8HVcXGhPaI665OGA7qDNTc537IXj/kjhaidOtq12Xe5nBKskkEYCun9XB4Fn7vRZ Tn6tKm5cTdnWURbHYJ0Kk1Ibh06Cp0lrmz3UOTm/AsRt+/IrzFqn78S6WAy61aaSwI2T DfTYEyJI2j9mXzCVnsGT/rSIDj0+gREglrY8l25zGwsL5s398tsHZ1LtFbcdzGZEZEnu Jpy6LuprB6+ipY9woDYXM6iKXtJPUiD/QLqbjzOA0+YijnnYXF3vVwJyzoY6xlF3yDBU 9S1sULN/XSgTQ0dFiv9DBQPA71JpEVpC+vHkeev5ICC81zFHhFqubUPtDCa+cWPnny6g aPJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769156584; x=1769761384; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=gNFjSEtLjmo3zavrqqxSndVZ5Mb9BZ08qMwOVvVVYnk=; b=hqXRGQwOED4IPrc6DiR8TWO+nL+HxEpdahrV9R/KuYsHhVLc3azjt0Jour3qayEj0P ynwdVr7Fj/AfFvppafs7F+S+oN+ThhZm9Z+eKibhWFPEXU3ssG5ymJ/McsYgVRx9GA6Q BBC0biu3g1YUnsQakrQE9qgFeYzpSO6ovg1ZEWqX2sYDzfNmzhXoE0oHc3jmsTJIi4Vt reaV5j1GlsMfms0jOIxFKE4TNQe0lhCAfpA9HPGk8D96Pmp7Er2d0aXMvqohusebpTTs jqJvBzH2l3ANu+6TEYqOsrN8rnhcnL86QoLTVMcnrZ+db1n+pink5basn4lXKaDgakHW xMHQ== X-Forwarded-Encrypted: i=1; AJvYcCWax2HhBBUXPm7YeBtsjSuHS6tK39lhsl/vBQbmKFzWqUTiq16iOwYeMh5q/jx3Wxln3xGxEMaTlpCoD2s=@vger.kernel.org X-Gm-Message-State: AOJu0YxWfRZiXofKiFhQYR77LVVKaqg37Um4k/6ynrLQlqWkuU05CRHm 85NRzgcxM1sfA5zKMV4f4vkc+QsXKdf0+e8l2I0MdluuQPeZZ/jABmW1 X-Gm-Gg: AZuq6aJTFThCPNBnyVl/wcGtLxFveCbFhUaBicWfeboI4EgM1pDFoS4KTr5Pty7QD1t b00TziHZLAplDf7uzCsR87kcrxAnJGGuj67aEKqj9pHzIxDQuwZzd4TSFOgqwLqK8imxICfEcL+ T6d9+wvYYwGkabEwYug8mkDhkFb/maU8NCHN6AozdPhcjUh0Tm5OaZWrvbqqsni1Hxqkdj34R7o fue6qIBy+fZiOwEoiBtl9VkSnnhiZ+Fr5ZZlLwTZksRmADslPV5BKGBSbx7iIROcl0r95UJmxgX bG7QbXsdU6NdoFQJi+fFM6Cf2mpZ6fX9QS5JQ1fOm12KROCBze2Iehpl7OWIuD28lczjUEOHw1S BZQGgasaUuZzzHaU3IJnqBql/EBccDagHP1ajZEUR3NnaVUXIcfUlDbDwNK+xqEy/tzYIzJCykX HinvIMV7FgfY8Byjn9dPMqVDBT X-Received: by 2002:a05:6a21:116:b0:35f:84c7:4012 with SMTP id adf61e73a8af0-38e7008e9e2mr1886524637.29.1769156583989; Fri, 23 Jan 2026 00:23:03 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c635a43f11csm1348363a12.35.2026.01.23.00.23.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Jan 2026 00:23:03 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v5 2/5] mm: khugepaged: refine scan progress number Date: Fri, 23 Jan 2026 16:22:29 +0800 Message-ID: <20260123082232.16413-3-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260123082232.16413-1-vernon2gm@gmail.com> References: <20260123082232.16413-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang Currently, each scan always increases "progress" by HPAGE_PMD_NR, even if only scanning a single PTE/PMD entry. - When only scanning a sigle PTE entry, let me provide a detailed example: static int hpage_collapse_scan_pmd() { for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { pte_t pteval =3D ptep_get(_pte); ... if (pte_uffd_wp(pteval)) { <-- first scan hit result =3D SCAN_PTE_UFFD_WP; goto out_unmap; } } } During the first scan, if pte_uffd_wp(pteval) is true, the loop exits directly. In practice, only one PTE is scanned before termination. Here, "progress +=3D 1" reflects the actual number of PTEs scanned, but previously "progress +=3D HPAGE_PMD_NR" always. - When the memory has been collapsed to PMD, let me provide a detailed example: The following data is traced by bpftrace on a desktop system. After the system has been left idle for 10 minutes upon booting, a lot of SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE are observed during a full scan by khugepaged. @scan_pmd_status[1]: 1 ## SCAN_SUCCEED @scan_pmd_status[6]: 2 ## SCAN_EXCEED_SHARED_PTE @scan_pmd_status[3]: 142 ## SCAN_PMD_MAPPED @scan_pmd_status[2]: 178 ## SCAN_NO_PTE_TABLE total progress size: 674 MB Total time : 419 seconds ## include khugepaged_scan_sleep_millisecs The khugepaged_scan list save all task that support collapse into hugepage, as long as the task is not destroyed, khugepaged will not remove it from the khugepaged_scan list. This exist a phenomenon where task has already collapsed all memory regions into hugepage, but khugepaged continues to scan it, which wastes CPU time and invalid, and due to khugepaged_scan_sleep_millisecs (default 10s) causes a long wait for scanning a large number of invalid task, so scanning really valid task is later. After applying this patch, when the memory is either SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE, just skip it, as follow: @scan_pmd_status[6]: 2 @scan_pmd_status[3]: 147 @scan_pmd_status[2]: 173 total progress size: 45 MB Total time : 20 seconds Signed-off-by: Vernon Yang --- include/linux/xarray.h | 9 ++++++++ mm/khugepaged.c | 47 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 47 insertions(+), 9 deletions(-) diff --git a/include/linux/xarray.h b/include/linux/xarray.h index be850174e802..f77d97d7b957 100644 --- a/include/linux/xarray.h +++ b/include/linux/xarray.h @@ -1646,6 +1646,15 @@ static inline void xas_set(struct xa_state *xas, uns= igned long index) xas->xa_node =3D XAS_RESTART; } =20 +/** + * xas_get_index() - Get XArray operation state for a different index. + * @xas: XArray operation state. + */ +static inline unsigned long xas_get_index(struct xa_state *xas) +{ + return xas->xa_index; +} + /** * xas_advance() - Skip over sibling entries. * @xas: XArray operation state. diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 6f0f05148765..de95029e3763 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -68,7 +68,10 @@ enum scan_result { static struct task_struct *khugepaged_thread __read_mostly; static DEFINE_MUTEX(khugepaged_mutex); =20 -/* default scan 8*HPAGE_PMD_NR ptes (or vmas) every 10 second */ +/* + * default scan 8*HPAGE_PMD_NR ptes, pmd_mapped, no_pte_table or vmas + * every 10 second. + */ static unsigned int khugepaged_pages_to_scan __read_mostly; static unsigned int khugepaged_pages_collapsed; static unsigned int khugepaged_full_scans; @@ -1240,7 +1243,8 @@ static enum scan_result collapse_huge_page(struct mm_= struct *mm, unsigned long a } =20 static enum scan_result hpage_collapse_scan_pmd(struct mm_struct *mm, - struct vm_area_struct *vma, unsigned long start_addr, bool *mmap_locked, + struct vm_area_struct *vma, unsigned long start_addr, + bool *mmap_locked, unsigned int *cur_progress, struct collapse_control *cc) { pmd_t *pmd; @@ -1255,6 +1259,9 @@ static enum scan_result hpage_collapse_scan_pmd(struc= t mm_struct *mm, =20 VM_BUG_ON(start_addr & ~HPAGE_PMD_MASK); =20 + if (cur_progress) + *cur_progress +=3D 1; + result =3D find_pmd_or_thp_or_none(mm, start_addr, &pmd); if (result !=3D SCAN_SUCCEED) goto out; @@ -1396,6 +1403,12 @@ static enum scan_result hpage_collapse_scan_pmd(stru= ct mm_struct *mm, result =3D SCAN_SUCCEED; } out_unmap: + if (cur_progress) { + if (_pte >=3D pte + HPAGE_PMD_NR) + *cur_progress +=3D HPAGE_PMD_NR - 1; + else + *cur_progress +=3D _pte - pte; + } pte_unmap_unlock(pte, ptl); if (result =3D=3D SCAN_SUCCEED) { result =3D collapse_huge_page(mm, start_addr, referenced, @@ -2286,8 +2299,9 @@ static enum scan_result collapse_file(struct mm_struc= t *mm, unsigned long addr, return result; } =20 -static enum scan_result hpage_collapse_scan_file(struct mm_struct *mm, uns= igned long addr, - struct file *file, pgoff_t start, struct collapse_control *cc) +static enum scan_result hpage_collapse_scan_file(struct mm_struct *mm, + unsigned long addr, struct file *file, pgoff_t start, + unsigned int *cur_progress, struct collapse_control *cc) { struct folio *folio =3D NULL; struct address_space *mapping =3D file->f_mapping; @@ -2376,6 +2390,18 @@ static enum scan_result hpage_collapse_scan_file(str= uct mm_struct *mm, unsigned cond_resched_rcu(); } } + if (cur_progress) { + unsigned long idx =3D xas_get_index(&xas) - start; + + if (folio =3D=3D NULL) + *cur_progress +=3D HPAGE_PMD_NR; + else if (xa_is_value(folio)) + *cur_progress +=3D idx + (1 << xas_get_order(&xas)); + else if (folio_order(folio) =3D=3D HPAGE_PMD_ORDER) + *cur_progress +=3D idx + 1; + else + *cur_progress +=3D idx + folio_nr_pages(folio); + } rcu_read_unlock(); =20 if (result =3D=3D SCAN_SUCCEED) { @@ -2456,6 +2482,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result =20 while (khugepaged_scan.address < hend) { bool mmap_locked =3D true; + unsigned int cur_progress =3D 0; =20 cond_resched(); if (unlikely(hpage_collapse_test_exit_or_disable(mm))) @@ -2472,7 +2499,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result mmap_read_unlock(mm); mmap_locked =3D false; *result =3D hpage_collapse_scan_file(mm, - khugepaged_scan.address, file, pgoff, cc); + khugepaged_scan.address, file, pgoff, + &cur_progress, cc); fput(file); if (*result =3D=3D SCAN_PTE_MAPPED_HUGEPAGE) { mmap_read_lock(mm); @@ -2486,7 +2514,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result } } else { *result =3D hpage_collapse_scan_pmd(mm, vma, - khugepaged_scan.address, &mmap_locked, cc); + khugepaged_scan.address, &mmap_locked, + &cur_progress, cc); } =20 if (*result =3D=3D SCAN_SUCCEED) @@ -2494,7 +2523,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result =20 /* move to next address */ khugepaged_scan.address +=3D HPAGE_PMD_SIZE; - progress +=3D HPAGE_PMD_NR; + progress +=3D cur_progress; if (!mmap_locked) /* * We released mmap_lock so break loop. Note @@ -2817,7 +2846,7 @@ int madvise_collapse(struct vm_area_struct *vma, unsi= gned long start, mmap_locked =3D false; *lock_dropped =3D true; result =3D hpage_collapse_scan_file(mm, addr, file, pgoff, - cc); + NULL, cc); =20 if (result =3D=3D SCAN_PAGE_DIRTY_OR_WRITEBACK && !triggered_wb && mapping_can_writeback(file->f_mapping)) { @@ -2832,7 +2861,7 @@ int madvise_collapse(struct vm_area_struct *vma, unsi= gned long start, fput(file); } else { result =3D hpage_collapse_scan_pmd(mm, vma, addr, - &mmap_locked, cc); + &mmap_locked, NULL, cc); } if (!mmap_locked) *lock_dropped =3D true; --=20 2.51.0 From nobody Sat Feb 7 08:44:03 2026 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04F5C341AAA for ; Fri, 23 Jan 2026 08:23:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156589; cv=none; b=E29gZ+XyHdxc+dwmMUp78fTxipvAuVmOo8PJlL60Pd+cKJvISrdj9yOzZCr6mljZtRir4r2wq21xlLwfgdfnEkCLJhi3cHIffW3IFykdpnBgKfxFTTOIQFwWEtaQWh1udp3KB26mmK7jwyHyyfkXbwShJ3k8AgTUxcSFhk/4m88= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156589; c=relaxed/simple; bh=i/yWbjfb0CNW46x6vwyUE68O1+z5guc/+TQPcN8hoJA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rRVm9wVZyGglWXk1QrCPLX+Uf6vKvIqEP4Utwl1AFa5ibFC76MhC1Lr69zT/BWWo7/JcPk0/U9SIV3MK64UZx3cG5b2wzf6YhwuIDa6bBUW5RRDyxesdlC6nE37YP35aO5AiRDpRdZ5AjO97A7u7Tu5xFzaBpOtIYC23DkE7HCU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OtVn2HRV; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OtVn2HRV" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2a7b47a5460so12037075ad.1 for ; Fri, 23 Jan 2026 00:23:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769156587; x=1769761387; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UjszNGqHfA22eaPPL9vpz70+uk7Z7KgIqX3QyklITGY=; b=OtVn2HRVQrsmMXFtzOovVD6JrE7Kd5VZOTn60qZchRNxgk0V/b/V5vJwOmRyhzdjha yKTiCGRBz1QquiBDbvwV0qpITYN3+N6WEOGXTNXGXH64q1xTJvQzdrxPkn+Ye8ObsTdY YRpwTgezP58yxO+Vur6M1BBdo3SMHJN/1eum7fVg1A+N7mvLktQRes9b3Zo11ayunh+7 4OTt0SzRogfK4Dm8WjYBp8CAJKIkHFbSqUWMLni+EZxQNZKGgB+ozniw4Rw2i0QM9Qep CyikfzTHh6VuC7rKe7x4Rx+jaAJMQyFwDZ5SLZUBugLD1qVWO4uAuXu1BmKfy7PKWMWa JPTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769156587; x=1769761387; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=UjszNGqHfA22eaPPL9vpz70+uk7Z7KgIqX3QyklITGY=; b=xDRKe8adlmrvSZg3mLF24n+ay5v7LWl8TY2Wj1ERkorKAnxw+vT8cR7c1DxdR/pPfL K2coDhiI6UKiBuggsAc1RvBBgBwtc6+yk4yj8KBd0HbsV4YwJIyskBuPJ4F9DYQrbhwj lIcYjaPhwFkL6YLJVSQlOxe6Q62A7AwAtvCADAuUSEwpDyXek0m4PoExi1MXiZ3jJNZo XmAuHosVG0ZQgbgKiqoo6QCPFyW3ylf0iB1yezd2JDtTi6/K/XWs4vlNBqvJN7XehBXZ Kus7wzQSfozEfJ/n/KBfvByO7xxxTPLn7UCfQuv2SznkLb53ntphe4YJ+TKbRRyPDA5B WAOg== X-Forwarded-Encrypted: i=1; AJvYcCVl8cnX20RPqXBVBtPRGYSqaMFrJeK5AkcjlJCKO8a2aex5p3XdJtZ7wPigp5RRtCT8j0o0a6dE/goloVQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzaVBPlSHNg7KC999XeRENy8TH4oY2RHAkJ/8OyBZ4ifYSGyWWO c/RWDsPn1M1Jocgj/NjDYOhnJXWYfKfuB3QFMqhWh5PS0v8c4bC4bPXv X-Gm-Gg: AZuq6aKlkNIqFhlXVWOCy6YUIszxqWuIxXDsW9YJw8mrlzee7nYltBgJdpI4nC4QCuv cpCLr4ToT+F2TxN0LaeB7yi/gL0ZK6a1E3j9D39bkBO3sObGy+PwBlGEV6EOamMdCBcegBhg2M2 QI1ABvz4Y0T2CYtwH20nvmE/cKYiY7fuj981uRWpNDRK0YJYRBJkhTf1arlLkyaNmwya77/eVFH yiHi6dTiGpSfzAhrEQP0jADbzeoaVO08QwvURCe+nezKgfCzahD6vO1xRkV2gHo4MESOur8FO7r Yj9/UQYxH72MLY39W1XSnjFyY7eWVpUHt3XaC82pC6pNf6m72eO5rv/LQ3ZCDgWxsE1J1Lch/0z yndMsbkgLgiLp0AM1VXOMadp5fYameakmlJg2beEZ8qno40bbu+BJmE0tRCkXa3GAFzXLGYFFXf iwus/U9AZbkTR/h/lS6F7wTbW5B3uGmlzMKOA= X-Received: by 2002:a17:902:e546:b0:2a7:6289:faa with SMTP id d9443c01a7336-2a7d2f05e2emr56904475ad.6.1769156587238; Fri, 23 Jan 2026 00:23:07 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c635a43f11csm1348363a12.35.2026.01.23.00.23.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Jan 2026 00:23:06 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v5 3/5] mm: add folio_test_lazyfree helper Date: Fri, 23 Jan 2026 16:22:30 +0800 Message-ID: <20260123082232.16413-4-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260123082232.16413-1-vernon2gm@gmail.com> References: <20260123082232.16413-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang Add folio_test_lazyfree() function to identify lazy-free folios to improve code readability. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Lance Yang Reviewed-by: Barry Song Reviewed-by: Dev Jain --- include/linux/page-flags.h | 5 +++++ mm/rmap.c | 2 +- mm/vmscan.c | 5 ++--- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index f7a0e4af0c73..415e9f2ef616 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -724,6 +724,11 @@ static __always_inline bool folio_test_anon(const stru= ct folio *folio) return ((unsigned long)folio->mapping & FOLIO_MAPPING_ANON) !=3D 0; } =20 +static __always_inline bool folio_test_lazyfree(const struct folio *folio) +{ + return folio_test_anon(folio) && !folio_test_swapbacked(folio); +} + static __always_inline bool PageAnonNotKsm(const struct page *page) { unsigned long flags =3D (unsigned long)page_folio(page)->mapping; diff --git a/mm/rmap.c b/mm/rmap.c index 088723f909e4..6b09be06c707 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -2049,7 +2049,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, } =20 if (!pvmw.pte) { - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_test_lazyfree(folio)) { if (unmap_huge_pmd_locked(vma, pvmw.address, pvmw.pmd, folio)) goto walk_done; /* diff --git a/mm/vmscan.c b/mm/vmscan.c index 8643c0fb9162..1f32c74ec738 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -944,8 +944,7 @@ static void folio_check_dirty_writeback(struct folio *f= olio, * They could be mistakenly treated as file lru. So further anon * test is needed. */ - if (!folio_is_file_lru(folio) || - (folio_test_anon(folio) && !folio_test_swapbacked(folio))) { + if (!folio_is_file_lru(folio) || folio_test_lazyfree(folio)) { *dirty =3D false; *writeback =3D false; return; @@ -1489,7 +1488,7 @@ static unsigned int shrink_folio_list(struct list_hea= d *folio_list, } } =20 - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_test_lazyfree(folio)) { /* follow __remove_mapping for reference */ if (!folio_ref_freeze(folio, 1)) goto keep_locked; --=20 2.51.0 From nobody Sat Feb 7 08:44:03 2026 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6100333B97D for ; Fri, 23 Jan 2026 08:23:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156593; cv=none; b=U7pQbsycMJ+mb/feusW4OqNxKxWUvSnqBSXPGG3kSg5fDPbU7jmwdRKvEqYhfM0J4Co+lCdZ2J9w8LYtjXWRcbdls6CwIs1xP1BKwQ2nCG9tlhhDUmAPLFWpNNjEPiTIrAZaSGah9LJN3tPzRPce0OAK/Y3qwIKf1NhyZAA/ZbE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156593; c=relaxed/simple; bh=exL1bWSKIR70ViPzmj/Xtp4Z05YHqWI7lOre6cPxfLs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YJXrt9/xQX5M7AIPnNHcktv0e1AM8fv1vg4U5YW7iodhjj+eIATMSjf9UCdFX436PqBR3vzZo/D3+rrhtYtCGz4UM/X7O8z3FhL+RTCWDOHrwgZg9wSIP2SwmnHF3OVCHgJaI0iM4fVGWhmRFw7m8SIOUI4aFm6EnJHx1cWK8K4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dXjgG4O6; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dXjgG4O6" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-8230c839409so1413546b3a.3 for ; Fri, 23 Jan 2026 00:23:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769156590; x=1769761390; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0NYRDoQ/3A47f1BvErhk2cyyDCVpwgJBfQUk6OyEvgE=; b=dXjgG4O6SZ3QhkempR2o3+PwJ+DjEiXwCsZhyS+tRNzQ8RqOMvz//Lmr2OJLmICv4p jzZlYuchgGLghFwOHzeuvOpCNBy9bsXon48e8jFRghmnl9ULKYuBRXGJFulZvoqJQJtq 689A+x6iF7v+/hiI57VXJD1A5I9thC2U9wBR505LPMvqlwXmuc6VRIJcdZdv5rkCrQpT 1f+R8rHjZ/E+9lZJ9Ghgvd48aI2xHCCL+CqIL8h8NQ9FmKC/lUGKQRKM79SCxCgoqd5Q pPxUTaD5CU4CvzfjyTA0f88N56XOCMklvTrksIIcl38PMcajmN7An7hcJXRg7ohb8GEz 2lVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769156590; x=1769761390; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=0NYRDoQ/3A47f1BvErhk2cyyDCVpwgJBfQUk6OyEvgE=; b=KTmdgoAw59Vga54QnOuQWD8zg9zGNXR87l0lp0MPfgdyS/h8kTReXL4VEo+JybDggP oH0te5ZOKirMQBwtjBaw/p3nIi2L/4boQYktGdAJKdPFligXzu+W9RB0HKXPHPAaYfqG jCdsCarT4gmElEShVfCnpQAkC3M0Sz45VvwFP+KDEcD3GedNLbXo1NuLgR7PIkDYxRmA vBPyq0SR6R/P6T9gDlVx6J9Hsy0kzd8CLw+MTkx3cIr62x2p3VfyzVZxegBEO5hgc7io mr1O+B/y8CkACdL/pmf5RhfNinpALcUUA7/g3hfW3+mFThici92mKQ2+Lw6Kct/4V20y IaRQ== X-Forwarded-Encrypted: i=1; AJvYcCV25RKsbx9t5dL95BrjG5UcOZtHSFhDvClO8Hn/aJDqQsjjRtaZAJIuF6ox8Zeypu8NZfaYN8RoHY+myWA=@vger.kernel.org X-Gm-Message-State: AOJu0YwjrqSdWNcsVwi1+9IQV04XpYk+oem2e9kRWdNxpMVjstMjlhRL rDumkPRjHhju82DV1mBsY+6Vj/EUmBzVPtS3SWQUrrEDkZO0erKIrEnj X-Gm-Gg: AZuq6aIMCXHabu50eMIgbJAvdQFJUzpJbiqFNM02DyW6XrtrA9MSBW2Noo4OWa8bRTr Y+I4vgCjg6UEoXV1RfvqZZivlScOomqQy9k50qpXLENzs8awcC54FR2MbS6DvxfHE1Lo5Tmf6QO e69yOLo8b0FGpSsU1RlmlP5jiuMpzNjEjY7FapA0F9E0q+52DiEmGeZDZi+5Pfuml8BUuhcFo8k xhdXgFl3K+jNLwSo16FyEOxeIWa6iAql+mzxq/OlescyZqAwxGOH2TS6J5If3cdYGR8ZGwnER4K NLbhLvd0BmPBYQPhYmRo13ElZ1FSCK9sa6i27TUPb4kGVruBdUawomc2VAHnw0yF1aBFdNkxoeU No9ORHX6CFW9v+ojEm78NQN7dVLmUcQUlCETSRgWONhh91/dnsGeLye1pankCzfEGTmRtFOfG3Q kt0AtmsNMoxXdazps00yoUK/4Cz22LxO3uH7Y= X-Received: by 2002:a05:6a21:33aa:b0:38b:eb25:7741 with SMTP id adf61e73a8af0-38e6f6a6badmr2287356637.1.1769156590493; Fri, 23 Jan 2026 00:23:10 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c635a43f11csm1348363a12.35.2026.01.23.00.23.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Jan 2026 00:23:10 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v5 4/5] mm: khugepaged: skip lazy-free folios Date: Fri, 23 Jan 2026 16:22:31 +0800 Message-ID: <20260123082232.16413-5-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260123082232.16413-1-vernon2gm@gmail.com> References: <20260123082232.16413-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang For example, create three task: hot1 -> cold -> hot2. After all three task are created, each allocate memory 128MB. the hot1/hot2 task continuously access 128 MB memory, while the cold task only accesses its memory briefly andthen call madvise(MADV_FREE). However, khugepaged still prioritizes scanning the cold task and only scans the hot2 task after completing the scan of the cold task. And if we collapse with a lazyfree page, that content will never be none and the deferred shrinker cannot reclaim them. So if the user has explicitly informed us via MADV_FREE that this memory will be freed, it is appropriate for khugepaged to skip it only, thereby avoiding unnecessary scan and collapse operations to reducing CPU wastage. Here are the performance test results: (Throughput bigger is better, other smaller is better) Testing on x86_64 machine: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.14 sec | 2.93 sec | -6.69% | | cycles per access | 4.96 | 2.21 | -55.44% | | Throughput | 104.38 M/sec | 111.89 M/sec | +7.19% | | dTLB-load-misses | 284814532 | 69597236 | -75.56% | Testing on qemu-system-x86_64 -enable-kvm: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.35 sec | 2.96 sec | -11.64% | | cycles per access | 7.29 | 2.07 | -71.60% | | Throughput | 97.67 M/sec | 110.77 M/sec | +13.41% | | dTLB-load-misses | 241600871 | 3216108 | -98.67% | Signed-off-by: Vernon Yang --- include/trace/events/huge_memory.h | 1 + mm/khugepaged.c | 11 +++++++++++ 2 files changed, 12 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 384e29f6bef0..bcdc57eea270 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -25,6 +25,7 @@ EM( SCAN_PAGE_LRU, "page_not_in_lru") \ EM( SCAN_PAGE_LOCK, "page_locked") \ EM( SCAN_PAGE_ANON, "page_not_anon") \ + EM( SCAN_PAGE_LAZYFREE, "page_lazyfree") \ EM( SCAN_PAGE_COMPOUND, "page_compound") \ EM( SCAN_ANY_PROCESS, "no_process_for_page") \ EM( SCAN_VMA_NULL, "vma_null") \ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index de95029e3763..be1c09842ea2 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -46,6 +46,7 @@ enum scan_result { SCAN_PAGE_LRU, SCAN_PAGE_LOCK, SCAN_PAGE_ANON, + SCAN_PAGE_LAZYFREE, SCAN_PAGE_COMPOUND, SCAN_ANY_PROCESS, SCAN_VMA_NULL, @@ -583,6 +584,11 @@ static enum scan_result __collapse_huge_page_isolate(s= truct vm_area_struct *vma, folio =3D page_folio(page); VM_BUG_ON_FOLIO(!folio_test_anon(folio), folio); =20 + if (!pte_dirty(pteval) && folio_test_lazyfree(folio)) { + result =3D SCAN_PAGE_LAZYFREE; + goto out; + } + /* See hpage_collapse_scan_pmd(). */ if (folio_maybe_mapped_shared(folio)) { ++shared; @@ -1330,6 +1336,11 @@ static enum scan_result hpage_collapse_scan_pmd(stru= ct mm_struct *mm, } folio =3D page_folio(page); =20 + if (!pte_dirty(pteval) && folio_test_lazyfree(folio)) { + result =3D SCAN_PAGE_LAZYFREE; + goto out_unmap; + } + if (!folio_test_anon(folio)) { result =3D SCAN_PAGE_ANON; goto out_unmap; --=20 2.51.0 From nobody Sat Feb 7 08:44:03 2026 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20612346FD2 for ; Fri, 23 Jan 2026 08:23:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156597; cv=none; b=rjRBPBPQnpXO9VqPYIDDqlBMWSl4LOy4bLowqf92RXdzCRnSgQuzzmMxRgLihESh3Is5Yl4idfbT23AQuvDdmd77aMDLoFegvxq+Y0WE+iW40UhYjIoeQHiiBmAyRYex9FVF/erTOxnnmrbxKFRsh5dUZoC25gQ3sOzU60Wt13w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769156597; c=relaxed/simple; bh=JAvYor1XNSyxvcCnfU0wEyiHKfk2NaIbLJu0SO3ilwY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pb3WWUHeC/hsiLZ4GO9FT/3NkTSFDCxeEc220Q+3EI6ItdS9FjyzcVu31op7tJrKsSA7mHDD6g9+p2F7Ksjwm+C0+CUcru28v1TeEXvVbdznm/0+CLKA9fMWYs/Nv9JqzF/xXJk/yi9SX5y4In/6vcnTbMDNSnOCcUr2lzLecfI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Y/i+TEzM; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Y/i+TEzM" Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-81f4e136481so1043733b3a.3 for ; Fri, 23 Jan 2026 00:23:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769156593; x=1769761393; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1s3RUysc9inCnOBypifiKMBUtySmMfCZj5RN3VGE8+E=; b=Y/i+TEzM1nbe4Ep7rkExVVIIiMm3pH92WJ5pWz2Xv0cCajxkfoi830ivn5jiVGbXAm UOMbnp8Taxmv/3XIIvZSFifeW9avpnoMgy+/nOjP/uZLNAWPCQkxNNUopxXExS4G4ZV2 +g6yZTwGfIu5VWEYyGCM2xYIwB7WH8Bz9zp43EXaf+RcWeJH/+COE/QRcMhO/mjMK4qm QV42OGMZCfq/myKy72hJPqtdFNAz2ePsMHMs0gxxfy1ovjXHL5Hf7vaMbq63PBidVizx /3TDj5Dv+U7Lx/ALaN8Uc+ZChfOOsCMjZcpSTQEXIecX/if0AeK8encYm9Uabh2cQJr1 2CEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769156593; x=1769761393; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=1s3RUysc9inCnOBypifiKMBUtySmMfCZj5RN3VGE8+E=; b=rQeWFFTd1sJrdPLAFL3KbjkOYtIKf48AnyGMwvKHwvVc3bNFdhoBipPMRwWcZlRerQ MVRB1uYMdURAxsT26ytc+teSwm+3MxT2gI9l9Rdf8nNpGoiXf0w2e7uIFscN/CDQH+Ie mJd00i5SM6ze7IwPb/N44ForeJhwnRn7Db5o84KUIlT1uIAUsos19pYvq9zB6vmb1LGW vhVmKoqW9hNns7sls8GWYUHNes7bJ6oddiMHvaSwQBARqPmrxV2QgmA86Oox/KXXzvGp Ermnm3mSeLv9OcPFuLK06ujkvEerD54fKzfKM2rw0gTb8vkHuW0bL8n2CyBqldYGC9Qx hUMA== X-Forwarded-Encrypted: i=1; AJvYcCXSE/fUysMyj73jLcReTUexMTI5ief4gPibpbfRu2E/1hB4iMdZcynjpUzlfZI0ViOlg2O9Y/jpG3Ho27Y=@vger.kernel.org X-Gm-Message-State: AOJu0YzL8Xo2T590QFpSRKBcf9dksqhA6L3k1wOgrAZk3OlWMw90KHh+ 2zQVJfDOtX9QyqEVM+5x0yXbMKVX+Le/dFCVjrYiwAHckvu7mql7gJXm X-Gm-Gg: AZuq6aKEiori1NttgzEVeTQ7YZQBWo1M5X4PjDoLMu4keavWQXY+T7rduG7aS5tfPAr uQoE2iIjsoB2uuq/BKhR4EfxEk4LsMCnlhbchvVEPtNHTARUsD46Bp9/S2FookQ3ZjR8CrW5VOk cmwZQEVY98gLzNF6Oi0OLuwZjBvnqJ5UFuTdh8Rxt/ki0B4CejAounG2lwSVGT4GH8boO7sJQ2W 395XV9hr9x4Tm1n2gBO1cYANn/Dzu5pSAzMLUToPn3sEdAkU7JhDqU5gJndlf0nMfWX2QUNTjQ6 jaZuFXFEtAvknttat1drat+4p3BkEFdkRXCuoV8hsJjrYx2Fys5tNoTt004TJ9SZK5lLt3/xQiT rqQOaXART29rFCcJRARS2UujS+CJKQCkEyPiURKkZzG5Tkci1wKCKWRBVzbAIjN/w0veyrWJe6Q vvqfZ2Sdrzh6SrtkgddTCtNn9S X-Received: by 2002:a05:6a20:2585:b0:350:55e0:5522 with SMTP id adf61e73a8af0-38e6f7ce9f1mr2253928637.37.1769156593579; Fri, 23 Jan 2026 00:23:13 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c635a43f11csm1348363a12.35.2026.01.23.00.23.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Jan 2026 00:23:13 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v5 5/5] mm: khugepaged: set to next mm direct when mm has MMF_DISABLE_THP_COMPLETELY Date: Fri, 23 Jan 2026 16:22:32 +0800 Message-ID: <20260123082232.16413-6-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260123082232.16413-1-vernon2gm@gmail.com> References: <20260123082232.16413-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang When an mm with the MMF_DISABLE_THP_COMPLETELY flag is detected during scanning, directly set khugepaged_scan.mm_slot to the next mm_slot, reduce redundant operation. Without this patch, entering khugepaged_scan_mm_slot() next time, we will set khugepaged_scan.mm_slot to the next mm_slot. With this patch, we will directly set khugepaged_scan.mm_slot to the next mm_slot. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Lance Yang Reviewed-by: Barry Song Reviewed-by: Dev Jain --- mm/khugepaged.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index be1c09842ea2..1dc19c1b1f97 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2558,7 +2558,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result * Release the current mm_slot if this mm is about to die, or * if we scanned all vmas of this mm. */ - if (hpage_collapse_test_exit(mm) || !vma) { + if (hpage_collapse_test_exit_or_disable(mm) || !vma) { /* * Make sure that if mm_users is reaching zero while * khugepaged runs here, khugepaged_exit will find --=20 2.51.0