From nobody Sun Feb 8 04:30:23 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7882B329C79 for ; Sun, 1 Feb 2026 12:26:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948805; cv=none; b=S1ZUO/YKAb/6UMnnZw6K1OUebqVLHogMWeVLame+aoKrGRGor/KRPtnnAnzVTvUOduMUjKsh5+FbzCcVNkBPg9b3Bdl6RoOsjEDwq1cxLmq4oGLcnmCu6+BxWC5yiQO+zwitu1LMK+yWdxqEz5SXVBpuBP5ZoHQo7V338LRvSN8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948805; c=relaxed/simple; bh=QVI7vVBByQFaSey6seObYjguitPIU0LCLdie+ypXgfA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PkyuKKAsJMihW952e95l/w7II+wNXS4PEnW/PjwnBnSqloPy64V9hSupxxGnWQvzj6l+24DyF1mi12NIHwWbyO3byX4+hIjvL7CCtgMdMggxbSSjexyuwsoFAXL6O8n+TDpby+RWbHIIJLlS+U6P4PPwnXYbo6rhX5bl89gw4mw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cmG/rdaN; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cmG/rdaN" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2a8fc56a9b2so7265985ad.0 for ; Sun, 01 Feb 2026 04:26:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769948804; x=1770553604; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S2QkQq6f6BhxvJCwuPAb4algJqA9XG0jgqYj1vlwjKY=; b=cmG/rdaNWSdfhqik68XimfYOME8zmRlx+oA59C8BLcqiwtlr+wYUvwtNcfhgOQcJZu KIZYV1lMkfVm0YKrZDimCS3aMcuffX+eGIkKB0gboDyPNBbmMG+ZNb4+vSraIaTFsH3K +TZ6H3ivM/mpBkChgtgfofXtW0JG0yRboFk16A2huMKw31xT9Yro3qZiLtptVoL+bvrX gwGUHBVmOHg6INx1sEk2xCiDzD4UE3Ppp3kcg/DKNKzSnL7bs/urZtEAr0Dwyuqqflj6 f1SItKCa5hrvjgvVwnHfvN9WX9iq6smiGxIpDczDdqPCegb07FjM00AYQpqsqXbF7TAy MVnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769948804; x=1770553604; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=S2QkQq6f6BhxvJCwuPAb4algJqA9XG0jgqYj1vlwjKY=; b=Yn0aNkgSrxo31H6+30DavAYlaibTssdV/YLsIUZsL3lZPuTQYEaLtdCVldYyAoPJyT 86N/qXwWmmVkqGVwx90AtX2Jxz8GDrX3dkUNEF6vgyBUAnyP1sSwra6QJKduyYM0mYWa eWyWXAnj4j2dePxKs05zpwrRBflKamahRCgdlMqGm/HS36A+bF2TtbAwpW2jf1YK4W5i Wv3n1Bxkhp/0FWJejfeqWGr2oyGe7S6tUaktH0i1alRo2F9Msee+Gp5aN0t8g/hyXKf+ okiL+IqNwlekbyGUPJwuRPpPu5ZFyglIdguJMfpaKVt6x/hs3Czoa2L0YaOStqVgPqru CAbg== X-Forwarded-Encrypted: i=1; AJvYcCWyIh4MYHJSIA1h+9sG5QADM+0/j4O919KbRHaAjOPr2JkYnUIQwqQi8vKqfP3eI4Po1l7GolHP8nDuYd4=@vger.kernel.org X-Gm-Message-State: AOJu0YwtWxqVtB9pf2sUHcbYvOCIpP0+Oq4vWxJ77D0uxHfbelujecSX ck3EKp+9yvRZt7oXI2ePUwu0XuqhaLvonOdJM7QrgeBlfp70absAVD6G X-Gm-Gg: AZuq6aLON5K2oYMetgrHai2smKtvlwZkN2WPPkZbffmeeFcFuTU9bYbyskipJfAR1py TGBcS0rgIknci04uGtPQXUE7OKmXX6ucbiqL77n5Ypv/E7y+EoqErehbsnpDrPTJhKWLSaJCY/u 89mCiV3bscM9B6tX3IyMe2F/hkeolX6HjnGf/RpLSWlhCAuRPQ8mFr2HrcwfNokgqWoWaML6/GF tjuoIymafJM52j8KGL/E5zWpKK/IMbSOl7+IRDpxfUJRRWhhy/qABVU7NuCiE71tnulTfzbAcEJ 1sGf49Ny8KfSEy36KZD6movWp0KnhkggR+AZ7kTsNUPD3BbppF1k8i2DMyEOUlBLCQ135MpAQWS +Gf34qnREYHKrvxSEqP07tsZu26P3Ezd3g5OW3PZxIerwhYg7gAckvWRHfZ2AfXrGB3uyGOmDRE hPhPRBUM6LP49IpJUM4Ir1MiSl X-Received: by 2002:a17:903:388d:b0:2a7:5171:9222 with SMTP id d9443c01a7336-2a8d9a62833mr92541075ad.49.1769948803766; Sun, 01 Feb 2026 04:26:43 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a88b6e5ac4sm116941185ad.86.2026.02.01.04.26.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Feb 2026 04:26:43 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v6 1/5] mm: khugepaged: add trace_mm_khugepaged_scan event Date: Sun, 1 Feb 2026 20:25:50 +0800 Message-ID: <20260201122554.1470071-2-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260201122554.1470071-1-vernon2gm@gmail.com> References: <20260201122554.1470071-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang Add mm_khugepaged_scan event to track the total time for full scan and the total number of pages scanned of khugepaged. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Barry Song Reviewed-by: Lance Yang Reviewed-by: Dev Jain --- include/trace/events/huge_memory.h | 25 +++++++++++++++++++++++++ mm/khugepaged.c | 2 ++ 2 files changed, 27 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 4e41bff31888..384e29f6bef0 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -237,5 +237,30 @@ TRACE_EVENT(mm_khugepaged_collapse_file, __print_symbolic(__entry->result, SCAN_STATUS)) ); =20 +TRACE_EVENT(mm_khugepaged_scan, + + TP_PROTO(struct mm_struct *mm, unsigned int progress, + bool full_scan_finished), + + TP_ARGS(mm, progress, full_scan_finished), + + TP_STRUCT__entry( + __field(struct mm_struct *, mm) + __field(unsigned int, progress) + __field(bool, full_scan_finished) + ), + + TP_fast_assign( + __entry->mm =3D mm; + __entry->progress =3D progress; + __entry->full_scan_finished =3D full_scan_finished; + ), + + TP_printk("mm=3D%p, progress=3D%u, full_scan_finished=3D%d", + __entry->mm, + __entry->progress, + __entry->full_scan_finished) +); + #endif /* __HUGE_MEMORY_H */ #include diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 1d994b6c58c6..d94b34e10bdf 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2534,6 +2534,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result collect_mm_slot(slot); } =20 + trace_mm_khugepaged_scan(mm, progress, khugepaged_scan.mm_slot =3D=3D NUL= L); + return progress; } =20 --=20 2.51.0 From nobody Sun Feb 8 04:30:23 2026 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4266233A6ED for ; Sun, 1 Feb 2026 12:26:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948809; cv=none; b=RcMX33/RLcsSEk3QMN0Cnk520eJ8BW3m1xYiElRKj6BBrEH/yDACXWYaVJ66JBJfm0zLpZMdWOHir0s4uVWz8QL+iXdS0R1Or0AxBkRBvMVovEbh4htjv5kfo9oz3orMkhJ2L2Byt0z/dfcGOD61YRp0jxmlij/faQxXN1+uCoI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948809; c=relaxed/simple; bh=2pRh8GPmZij6N9LO0SmM0Xcev1FEQIJIC1v7G/HRITU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lkTEW0RSbw0ZdY9Ub90bZnEGj8mYQ0XdBWAGY9AKfIp6Yy0OsoCLgQ33wFX1tlTkU8gE9y5rJL1D57EOwEaDTXzlhqdw7TkHApLmDw6/xGaSxesZ3zfmCzeiwlIcZm8xr19XTepuyIfpkdxOuH+0xcUIViUZwRusgtiDVBTSMl0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eT8FbD8I; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eT8FbD8I" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2a09d981507so25679885ad.1 for ; Sun, 01 Feb 2026 04:26:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769948808; x=1770553608; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=te4Qk5Zt2UCvHwxxzNavYvXAnoEwtEAAn0AB0X/qh6c=; b=eT8FbD8I8a8RlQcOeWLbrKSQZ6FjCrr1Tm31CqmM8aphiCwZdSJuNiwKV6hd5nkmJn Pw8steiw2MLSMKy1TAEfOrbKb0FlKJnRSlwhDqTd2cZg1sTbIKc2Sw43MaVeqB9ZdcOi SMgJbPPvlHjSGv/hEsLiRYBE7h9MOOCkjTHKe4IpVyom4LxMDNMV4SQ9pR15N4ksiCjQ CQQFVPXKfxwNBqy1YpSataRFeYR0uX5Ou1Pv/C46O0eJfFCiWqisgefixvraMdIpPGBz PsTVofBVAMK7dLp8730xH49IQMglzVk/yzEA3d+DSkdvmBf7XeXU9UrRBOTbLWryyO57 PoOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769948808; x=1770553608; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=te4Qk5Zt2UCvHwxxzNavYvXAnoEwtEAAn0AB0X/qh6c=; b=Py2tvLlWoh+TiP331TiNC+9r7cjDJUKFhF2TMgUUjfjGpN7KIThLuDT5xAuv6tI7Cc ssTNDC9BoSt5wRCSE4y7kkKsCFYrTMhuOm6e0cwCPUS4KFnv40p/5A8wa7ErpMOM44zU ThahUefZfRL84jtN8+Jy1r52bTGeUlLxxvOPXDuKKhOvSIW1j5x/ZMsK48nuZ5BI5VEY XveSlt4I9wuy+VJD5+PILQmuSF26fe9eXMym2CIfLtcnaGuy+qqbhIaMGykSe27Wqs1H vy7nwjoQRgnOGb3fmXQp/ZIcE+vd2+wmLLI06R8j+rNduk3jQDSj1iW2FKwcSW/kyhVi 6Qgw== X-Forwarded-Encrypted: i=1; AJvYcCU861icjqt/ToB8GSSyWNG4+kqYZWDavhAnFm7Vw1GkbYcr15wKUemfhARt6CQFGbwYSRqIu0vvtF4+Iq4=@vger.kernel.org X-Gm-Message-State: AOJu0YzD2VRLNuFdloMg+fVZsnP/xDJykMOX9FHhEa6kRs6ocCwwVCFN 0Yr4dmseGnFmTZQFfbnUjtM1oqJyDwnttxuicexKzjVDNI1CAEJoA6eG X-Gm-Gg: AZuq6aIVPkSsdXyPzYIO+HAhGQVsyLoO+7eUIv9N+U9t2qCY9X7j166uJ5cZvxcsbDY CDxpzyUrxCZNSGPyGuYPAFDKjVvqrPqtYNQfTTl2H7ofGSa367ubXjMOujMAr5Fy2L75B3TW3p+ l9bkvQP8Uiara+9nLyzFOQ2N8y5CnwTWvY2FPBbW2J5ZBSi7Rja/PWi/iTs4K5waE1wDhRTlAxd umZF6oowqePZ+zG0KGeDKFSPYKojZpAaX1mEdz87QFyEMwCTKpWDyoTsRrh+S76PGLoqbvPOkrL VvqsLiLhmJb5iT9cFFVoUYUra/zbujeSxZcrVcsLqUeTp1i5XCwlPkgnL467HnHcilIdFMc3lrZ T/eX61KPBbWAu6d3Q1wBVgesFoMnpJVZ3p+x/9bvvt7LDltkA1o92yrYTQV2y6ZOEIj9HQ+TrYH xue1tUSSr+fTBs4Ga9mjL7gp2F X-Received: by 2002:a17:903:fa3:b0:2a7:a9b8:eb9f with SMTP id d9443c01a7336-2a8d9d7a101mr91618605ad.30.1769948807602; Sun, 01 Feb 2026 04:26:47 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a88b6e5ac4sm116941185ad.86.2026.02.01.04.26.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Feb 2026 04:26:47 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v6 2/5] mm: khugepaged: refine scan progress number Date: Sun, 1 Feb 2026 20:25:51 +0800 Message-ID: <20260201122554.1470071-3-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260201122554.1470071-1-vernon2gm@gmail.com> References: <20260201122554.1470071-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang Currently, each scan always increases "progress" by HPAGE_PMD_NR, even if only scanning a single PTE/PMD entry. - When only scanning a sigle PTE entry, let me provide a detailed example: static int hpage_collapse_scan_pmd() { for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { pte_t pteval =3D ptep_get(_pte); ... if (pte_uffd_wp(pteval)) { <-- first scan hit result =3D SCAN_PTE_UFFD_WP; goto out_unmap; } } } During the first scan, if pte_uffd_wp(pteval) is true, the loop exits directly. In practice, only one PTE is scanned before termination. Here, "progress +=3D 1" reflects the actual number of PTEs scanned, but previously "progress +=3D HPAGE_PMD_NR" always. - When the memory has been collapsed to PMD, let me provide a detailed example: The following data is traced by bpftrace on a desktop system. After the system has been left idle for 10 minutes upon booting, a lot of SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE are observed during a full scan by khugepaged. From trace_mm_khugepaged_scan_pmd and trace_mm_khugepaged_scan_file, the following statuses were observed, with frequency mentioned next to them: SCAN_SUCCEED : 1 SCAN_EXCEED_SHARED_PTE: 2 SCAN_PMD_MAPPED : 142 SCAN_NO_PTE_TABLE : 178 total progress size : 674 MB Total time : 419 seconds, include khugepaged_scan_sleep_millisecs The khugepaged_scan list save all task that support collapse into hugepage, as long as the task is not destroyed, khugepaged will not remove it from the khugepaged_scan list. This exist a phenomenon where task has already collapsed all memory regions into hugepage, but khugepaged continues to scan it, which wastes CPU time and invalid, and due to khugepaged_scan_sleep_millisecs (default 10s) causes a long wait for scanning a large number of invalid task, so scanning really valid task is later. After applying this patch, when the memory is either SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE, just skip it, as follow: SCAN_EXCEED_SHARED_PTE: 2 SCAN_PMD_MAPPED : 147 SCAN_NO_PTE_TABLE : 173 total progress size : 45 MB Total time : 20 seconds Signed-off-by: Vernon Yang --- mm/khugepaged.c | 41 +++++++++++++++++++++++++++++++---------- 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index d94b34e10bdf..df22b2274d92 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -68,7 +68,10 @@ enum scan_result { static struct task_struct *khugepaged_thread __read_mostly; static DEFINE_MUTEX(khugepaged_mutex); =20 -/* default scan 8*HPAGE_PMD_NR ptes (or vmas) every 10 second */ +/* + * default scan 8*HPAGE_PMD_NR ptes, pmd_mapped, no_pte_table or vmas + * every 10 second. + */ static unsigned int khugepaged_pages_to_scan __read_mostly; static unsigned int khugepaged_pages_collapsed; static unsigned int khugepaged_full_scans; @@ -1240,7 +1243,8 @@ static enum scan_result collapse_huge_page(struct mm_= struct *mm, unsigned long a } =20 static enum scan_result hpage_collapse_scan_pmd(struct mm_struct *mm, - struct vm_area_struct *vma, unsigned long start_addr, bool *mmap_locked, + struct vm_area_struct *vma, unsigned long start_addr, + bool *mmap_locked, unsigned int *cur_progress, struct collapse_control *cc) { pmd_t *pmd; @@ -1256,13 +1260,18 @@ static enum scan_result hpage_collapse_scan_pmd(str= uct mm_struct *mm, VM_BUG_ON(start_addr & ~HPAGE_PMD_MASK); =20 result =3D find_pmd_or_thp_or_none(mm, start_addr, &pmd); - if (result !=3D SCAN_SUCCEED) + if (result !=3D SCAN_SUCCEED) { + if (cur_progress) + *cur_progress =3D 1; goto out; + } =20 memset(cc->node_load, 0, sizeof(cc->node_load)); nodes_clear(cc->alloc_nmask); pte =3D pte_offset_map_lock(mm, pmd, start_addr, &ptl); if (!pte) { + if (cur_progress) + *cur_progress =3D 1; result =3D SCAN_NO_PTE_TABLE; goto out; } @@ -1396,6 +1405,12 @@ static enum scan_result hpage_collapse_scan_pmd(stru= ct mm_struct *mm, result =3D SCAN_SUCCEED; } out_unmap: + if (cur_progress) { + if (_pte >=3D pte + HPAGE_PMD_NR) + *cur_progress =3D HPAGE_PMD_NR; + else + *cur_progress =3D _pte - pte + 1; + } pte_unmap_unlock(pte, ptl); if (result =3D=3D SCAN_SUCCEED) { result =3D collapse_huge_page(mm, start_addr, referenced, @@ -2286,8 +2301,9 @@ static enum scan_result collapse_file(struct mm_struc= t *mm, unsigned long addr, return result; } =20 -static enum scan_result hpage_collapse_scan_file(struct mm_struct *mm, uns= igned long addr, - struct file *file, pgoff_t start, struct collapse_control *cc) +static enum scan_result hpage_collapse_scan_file(struct mm_struct *mm, + unsigned long addr, struct file *file, pgoff_t start, + unsigned int *cur_progress, struct collapse_control *cc) { struct folio *folio =3D NULL; struct address_space *mapping =3D file->f_mapping; @@ -2376,6 +2392,8 @@ static enum scan_result hpage_collapse_scan_file(stru= ct mm_struct *mm, unsigned cond_resched_rcu(); } } + if (cur_progress) + *cur_progress =3D max(xas.xa_index - start, 1UL); rcu_read_unlock(); =20 if (result =3D=3D SCAN_SUCCEED) { @@ -2455,6 +2473,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result =20 while (khugepaged_scan.address < hend) { bool mmap_locked =3D true; + unsigned int cur_progress =3D 0; =20 cond_resched(); if (unlikely(hpage_collapse_test_exit_or_disable(mm))) @@ -2471,7 +2490,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result mmap_read_unlock(mm); mmap_locked =3D false; *result =3D hpage_collapse_scan_file(mm, - khugepaged_scan.address, file, pgoff, cc); + khugepaged_scan.address, file, pgoff, + &cur_progress, cc); fput(file); if (*result =3D=3D SCAN_PTE_MAPPED_HUGEPAGE) { mmap_read_lock(mm); @@ -2485,7 +2505,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result } } else { *result =3D hpage_collapse_scan_pmd(mm, vma, - khugepaged_scan.address, &mmap_locked, cc); + khugepaged_scan.address, &mmap_locked, + &cur_progress, cc); } =20 if (*result =3D=3D SCAN_SUCCEED) @@ -2493,7 +2514,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result =20 /* move to next address */ khugepaged_scan.address +=3D HPAGE_PMD_SIZE; - progress +=3D HPAGE_PMD_NR; + progress +=3D cur_progress; if (!mmap_locked) /* * We released mmap_lock so break loop. Note @@ -2816,7 +2837,7 @@ int madvise_collapse(struct vm_area_struct *vma, unsi= gned long start, mmap_locked =3D false; *lock_dropped =3D true; result =3D hpage_collapse_scan_file(mm, addr, file, pgoff, - cc); + NULL, cc); =20 if (result =3D=3D SCAN_PAGE_DIRTY_OR_WRITEBACK && !triggered_wb && mapping_can_writeback(file->f_mapping)) { @@ -2831,7 +2852,7 @@ int madvise_collapse(struct vm_area_struct *vma, unsi= gned long start, fput(file); } else { result =3D hpage_collapse_scan_pmd(mm, vma, addr, - &mmap_locked, cc); + &mmap_locked, NULL, cc); } if (!mmap_locked) *lock_dropped =3D true; --=20 2.51.0 From nobody Sun Feb 8 04:30:23 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9290432936D for ; Sun, 1 Feb 2026 12:26:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948813; cv=none; b=WgctdjQ2duzcGcC6Ys7fAI2cHhftiE+estdv6buipcbm9EKSRjVW5gJwKSBTMF0tREouvCWW1CC5eiNpxftZ1MYYo913KfUqYxgPq5Qkj7AZ1d4hdy8iT+koPYh2qz0DLruCcpR/kbwdArTtooYujuf0mTKq4PkKvggpUtD1nvA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948813; c=relaxed/simple; bh=GbT8X6Pj7xImUDLCBcrpiNsFx0cEAlktXtDWFbTKTtU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Dv4SI7grqgfkaA7ELJwfA2x2b+Had5FZQJ1GywdFFt67yH5oWB13CBpp0LhHChwkBHT8uI2/AP6PdMVw7mOCPWB2wQKvINaB2E5frQKm8p57mqdoKzkNmAP46sMxFTx0kZs1z/vWACIajhiBEjOLnTOSaVJXzv8DrXPOsUTTUS8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Y65yI55C; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Y65yI55C" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-81e8b1bdf0cso2138246b3a.3 for ; Sun, 01 Feb 2026 04:26:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769948812; x=1770553612; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sZwZ8lWYZ1YuosjYTvabOBDHoXCdZD0VZNoRi/PLmWs=; b=Y65yI55CTYZnf4hccG/FoLdOFBzKtHOjnTjVGiQZeHllDopm/sHMdAoHjNk6KadzoI mtcYP2xVeGha1aXsKQewtPez5yMqrx+B2JJUs5BwAgpntAmhRpILV0/gUobIZm6yuxbt kP4MzMbASv4RMzutuwmwcivTb2EgysXnrSRraYcGELMNxHGF8H8wVD05SWlBh+vsES+0 yNoVtYOXetKMlLz2ydAza5lnEXrTlMYF5s0aASwWiPR6D+CcdbrhD50rCfEA7+7+Ng8K Ejb8NTZOw5WbTFshshnUxpuz6SJs6BlxQVNgyqpPj9arKCB1f8+a+xxyE2SWA5xfqPGm dlow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769948812; x=1770553612; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=sZwZ8lWYZ1YuosjYTvabOBDHoXCdZD0VZNoRi/PLmWs=; b=HadItjHxOUw27mX1c1pJbBsmrFD7dda1CYrckAMAv7E8b2oE3/zgWFTa7AeRt5DdjX +Kx87MHoIOV+fpeEV2RFAsZzrwxK52lA9pacPARlDYjHW449O2nQMbX/KUQuH0osTEJ9 5P+eHVCxhxYbJ0w1VURIije4Adric0t+kkDHys3BMbXIOayEIaKASssmsqWpMWSD0uj0 SPyAQYsyhE0G/VKfECUbR1Xuloq1vPbWWJv11ab9JV3NsNsGXEWmRaK3x3+tgFmbVxeT snpV4Y3tMsbdjO6y1eo5pA+JlNJ25JpC1JouLi6QFwiWJZrclPetpqiLSrgi/so4qD23 fDtQ== X-Forwarded-Encrypted: i=1; AJvYcCVkS2SwaykEHeiIUngfG9k1T3Aih2rMzt3ojdKs94hh45XE4EvMeWDs1F1AC6R2i1n+TWLZwod0f4N14BM=@vger.kernel.org X-Gm-Message-State: AOJu0YwzHqF2kW5yQ+N39oeP8Yn4mxzPNnH7LzcDBI/e86DWdJCbjI4N g0+rwzFV6fegWlEGVPe0N5NdqHZd9xTnf0P5X4HyRkr9uS/4vtlZFv9u X-Gm-Gg: AZuq6aJhtPyElvw5o8IkX2Jpe+QYs2KAH8Ko83m50qsMLg7unfgro72Dg0ISBnPaU2V T0BHGY8sSJBDGbjkC/I8mhW7/hVTqd6UZK/wTChL67cdFccyoL8JMJRmFOlcFGqumxfUOJwy8ek srFPFqyozExyuZjDjrQRFMzMPMVwJBvamx6vrESyg+fv4MeOdfDmy6u4g1CtuFyqfx7jDxKEMBu fbYHFI2dnSZ9PDASGDKa4zIndIJ/t4gGH6v6g7j56h0WJx2RF0Su3muVRa4YKnD8xt6W1pQ801I fzwdaO3OEsQs5J3zcDhsLS9GYOmyscvOajySVrZmLIrqWCEVhesIatwYolckNc5gzyyGVMIYv4y Rc1QGHUaH6lelcZaheCgxOlx6WVhKAvyqgAjoF2l7ts6neyMlVt4qzXsjah74JdR9kySsR9u0l5 b22cX/UqFKwolXCkNWVl2aSiiA X-Received: by 2002:a05:6a20:7f98:b0:38e:8842:6683 with SMTP id adf61e73a8af0-392dffda013mr9289237637.5.1769948811817; Sun, 01 Feb 2026 04:26:51 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a88b6e5ac4sm116941185ad.86.2026.02.01.04.26.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Feb 2026 04:26:51 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v6 3/5] mm: add folio_test_lazyfree helper Date: Sun, 1 Feb 2026 20:25:52 +0800 Message-ID: <20260201122554.1470071-4-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260201122554.1470071-1-vernon2gm@gmail.com> References: <20260201122554.1470071-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang Add folio_test_lazyfree() function to identify lazy-free folios to improve code readability. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Lance Yang Reviewed-by: Dev Jain Reviewed-by: Barry Song --- include/linux/page-flags.h | 5 +++++ mm/rmap.c | 2 +- mm/vmscan.c | 5 ++--- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index f7a0e4af0c73..415e9f2ef616 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -724,6 +724,11 @@ static __always_inline bool folio_test_anon(const stru= ct folio *folio) return ((unsigned long)folio->mapping & FOLIO_MAPPING_ANON) !=3D 0; } =20 +static __always_inline bool folio_test_lazyfree(const struct folio *folio) +{ + return folio_test_anon(folio) && !folio_test_swapbacked(folio); +} + static __always_inline bool PageAnonNotKsm(const struct page *page) { unsigned long flags =3D (unsigned long)page_folio(page)->mapping; diff --git a/mm/rmap.c b/mm/rmap.c index 618df3385c8b..ea55e12b3e87 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -2049,7 +2049,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, } =20 if (!pvmw.pte) { - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_test_lazyfree(folio)) { if (unmap_huge_pmd_locked(vma, pvmw.address, pvmw.pmd, folio)) goto walk_done; /* diff --git a/mm/vmscan.c b/mm/vmscan.c index 01d3364fe506..17d039bdbb53 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -963,8 +963,7 @@ static void folio_check_dirty_writeback(struct folio *f= olio, * They could be mistakenly treated as file lru. So further anon * test is needed. */ - if (!folio_is_file_lru(folio) || - (folio_test_anon(folio) && !folio_test_swapbacked(folio))) { + if (!folio_is_file_lru(folio) || folio_test_lazyfree(folio)) { *dirty =3D false; *writeback =3D false; return; @@ -1508,7 +1507,7 @@ static unsigned int shrink_folio_list(struct list_hea= d *folio_list, } } =20 - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_test_lazyfree(folio)) { /* follow __remove_mapping for reference */ if (!folio_ref_freeze(folio, 1)) goto keep_locked; --=20 2.51.0 From nobody Sun Feb 8 04:30:23 2026 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 177D333986D for ; Sun, 1 Feb 2026 12:26:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948818; cv=none; b=lUYBxxCSK38xx9b9fzX1nSsAjSdJS12TRFIju8Btj5B4Ebr781X8vhaNIQ8FZb/9v/yfGHBK7EG858ZM3+GOrJCrtK4iVyiUU3eHL26tyTK5NFB9dP5fCYdDPwFAJhipriEuPZC/r48heDBcPvLkOhKOY1/SeeKVxsPUB6aTDTk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948818; c=relaxed/simple; bh=mcYHQF7V8hw0U8tFlKOZdDI846s1K/YXmZZhLqUi7TQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=r0LcxU9xflgCeEJBtiQoEVhf4ZuB4Lt4uvYxIQiS5LIsCsqMjjWqISuw47PSx6XA/rzqlW6Yy1LD4cwSfGv7nSegoBsueALW9hMJLwWzXbNsnSGcgOOuglYrlRKRC7oMrXafb37eoa+7pFcyWW3FotHjVkd/QATY8OQUSpLa9YA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SSMVz2+0; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SSMVz2+0" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2a7a9b8ed69so37403145ad.2 for ; Sun, 01 Feb 2026 04:26:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769948816; x=1770553616; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g1KIjvX7rp1rAqsSo7NpZu+Xr3OmRRK8GikF7hwaL00=; b=SSMVz2+07rAdn3h8HLPK5qfJfKMoGsQs4n1EQpvDgOFcNjacmWWeMYGYJ5XHAqv7F5 92YArhQyH5kPOc/iKEatQBU5mrmXBNcBZiJCuToevGqU++dwp5V76V3Bk8TeH8RWxGDu gC/6Xv9pFbJnLFp3OyD3LWvTdyIeSu7N9/3rIgfIp2Y6ugL7BGbbmks7h9tuygTaPHYG sgUhdVc+tyY14r4GYZBhb06dDzfXgedap2oXDw+S8gKHWvqvhc8iqHw3fLmSIHSwQ9wP 5aGyf6G/tu8mlhYYMT5pptpe4d7GASkWwOkHdRJFACkxNN8fexiFZkt+0pHpzE1sy7M3 qzMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769948816; x=1770553616; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=g1KIjvX7rp1rAqsSo7NpZu+Xr3OmRRK8GikF7hwaL00=; b=wEQc35DK6KbU3MOmWESsE/XauEl/fHubMGZhNNuRP+V3eWMFwVAaGfG6SZIhC3GVuX rNLml2qxxJKHCrgFsqdy8AVPFHBU8PbwxRPDOOamnjQnTZlp9SGhwP/GlrSv5MG8J6ZH As/sX4zlLdDGoJ90zuQcYm4AFDd2jZBdrgkD1+vakHsTKqyWQeWmmKAgLzAsV3VNtuCo piWmt+FX8Fx0mvgr0JE9jgyqoYG3PiCNCr4dZ6oF4JL7blyVguG1e8gGusQ9LoZC1Sqn nabIEWsxelV26Chgkq3WKoEL2sXHnmo6B+aDe3jGByDyZnM75Oft+DgzA86T1YqfWQCZ 4fGg== X-Forwarded-Encrypted: i=1; AJvYcCVXuv/VUlcw23YWBN5MOYBNofVofthpbM5Kov26lsPndWvQH85O0zO5/19voxR0Y2tsMP7BV0e7YXFcTDs=@vger.kernel.org X-Gm-Message-State: AOJu0YwgH5P1EsARUI/FdzBPFBFmk7Vct5P23ACUza1ioyTMknFFWxR/ 8aSiyDPIOkdc3NZtZT4jd8d//ag7FzFncwvsxKeEYH6oqDyU5PFhOWhh X-Gm-Gg: AZuq6aKBT5I8eNuF4JV/KJQPbnFoTWjw5bO503HzRi5m/i5xbFXZmZm8vYFT/I4FUgq AUnDf9I5lNah8xukZtkitLLxaGZnB4DxODpoS65evOL0XmcuAxMWwLUwNUsUkOIZ0iRUzYR1btl riE/dNhfDTwwgf9rce2u7rWgC5yNfBIG6CnegoFfSsrJdfK1+AoXctYRfTiy0yGqp+vjRq1z/GF v22wr85J3I+V4b86QWmd+Lu1VU3K4mE1SyyOjnFoC2IaufQf5N+TrFU5TpDTghwAU+stGReuofJ gJJ1NxBBVUnN7gk4JDRwxhl+7hgTrf43e8/WOEyIT6gWipgEqkf4/u1OsQP3fkuItRBEL3KTxW4 l/oJazJ5RKDrTRj9XaQh1fHn+NZTpUzMvDNZJjuIaZm2D18vTq4aZ6E1MhSjD9hDl+ZggAoDg3r wbKO62u+LEYaFHTHN7Sta2LbJV X-Received: by 2002:a17:903:1b10:b0:2a0:b02b:210c with SMTP id d9443c01a7336-2a8d96a6e47mr88899225ad.1.1769948816454; Sun, 01 Feb 2026 04:26:56 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a88b6e5ac4sm116941185ad.86.2026.02.01.04.26.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Feb 2026 04:26:56 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v6 4/5] mm: khugepaged: skip lazy-free folios Date: Sun, 1 Feb 2026 20:25:53 +0800 Message-ID: <20260201122554.1470071-5-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260201122554.1470071-1-vernon2gm@gmail.com> References: <20260201122554.1470071-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang For example, create three task: hot1 -> cold -> hot2. After all three task are created, each allocate memory 128MB. the hot1/hot2 task continuously access 128 MB memory, while the cold task only accesses its memory briefly andthen call madvise(MADV_FREE). However, khugepaged still prioritizes scanning the cold task and only scans the hot2 task after completing the scan of the cold task. And if we collapse with a lazyfree page, that content will never be none and the deferred shrinker cannot reclaim them. So if the user has explicitly informed us via MADV_FREE that this memory will be freed, it is appropriate for khugepaged to skip it only, thereby avoiding unnecessary scan and collapse operations to reducing CPU wastage. Here are the performance test results: (Throughput bigger is better, other smaller is better) Testing on x86_64 machine: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.14 sec | 2.93 sec | -6.69% | | cycles per access | 4.96 | 2.21 | -55.44% | | Throughput | 104.38 M/sec | 111.89 M/sec | +7.19% | | dTLB-load-misses | 284814532 | 69597236 | -75.56% | Testing on qemu-system-x86_64 -enable-kvm: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.35 sec | 2.96 sec | -11.64% | | cycles per access | 7.29 | 2.07 | -71.60% | | Throughput | 97.67 M/sec | 110.77 M/sec | +13.41% | | dTLB-load-misses | 241600871 | 3216108 | -98.67% | Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (arm) Reviewed-by: Lance Yang --- include/trace/events/huge_memory.h | 1 + mm/khugepaged.c | 13 +++++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 384e29f6bef0..bcdc57eea270 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -25,6 +25,7 @@ EM( SCAN_PAGE_LRU, "page_not_in_lru") \ EM( SCAN_PAGE_LOCK, "page_locked") \ EM( SCAN_PAGE_ANON, "page_not_anon") \ + EM( SCAN_PAGE_LAZYFREE, "page_lazyfree") \ EM( SCAN_PAGE_COMPOUND, "page_compound") \ EM( SCAN_ANY_PROCESS, "no_process_for_page") \ EM( SCAN_VMA_NULL, "vma_null") \ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index df22b2274d92..b4def001ccd0 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -46,6 +46,7 @@ enum scan_result { SCAN_PAGE_LRU, SCAN_PAGE_LOCK, SCAN_PAGE_ANON, + SCAN_PAGE_LAZYFREE, SCAN_PAGE_COMPOUND, SCAN_ANY_PROCESS, SCAN_VMA_NULL, @@ -583,6 +584,12 @@ static enum scan_result __collapse_huge_page_isolate(s= truct vm_area_struct *vma, folio =3D page_folio(page); VM_BUG_ON_FOLIO(!folio_test_anon(folio), folio); =20 + if (cc->is_khugepaged && !pte_dirty(pteval) && + folio_test_lazyfree(folio)) { + result =3D SCAN_PAGE_LAZYFREE; + goto out; + } + /* See hpage_collapse_scan_pmd(). */ if (folio_maybe_mapped_shared(folio)) { ++shared; @@ -1332,6 +1339,12 @@ static enum scan_result hpage_collapse_scan_pmd(stru= ct mm_struct *mm, } folio =3D page_folio(page); =20 + if (cc->is_khugepaged && !pte_dirty(pteval) && + folio_test_lazyfree(folio)) { + result =3D SCAN_PAGE_LAZYFREE; + goto out_unmap; + } + if (!folio_test_anon(folio)) { result =3D SCAN_PAGE_ANON; goto out_unmap; --=20 2.51.0 From nobody Sun Feb 8 04:30:23 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E59BE33A6ED for ; Sun, 1 Feb 2026 12:27:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948822; cv=none; b=V1hpOPFDTBgU5g2FFWyLkPZat76YHpRDqfWdYWDT1Xyxj+aZAM036vIVAzKVYqgBY9xHRhzfCdzmIi/rvKAsMnop/0x7jftunAClQncvlrlUyMq+RIhvPVYUvs3KTvw5IDV063I3Ls9ww02y8MLCsU9NwFr9Kn+PeC/F9TM9U74= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769948822; c=relaxed/simple; bh=VDCM9nEfnOLdCXbFbKW641ZkGAhWz7aIBdyjkBkWUFA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c+WflnIK/YYkpUkMXx7ydYpyqrCc9/UmmN46GQeRAo+LmY+llN13GEYU4CoHOlp/0XZ8T+S0jCm4GX44Ar9KToHLJCaYQ1r2ozaOpZbqwu6W05L+28PyYAaHDYWguXFxi/Jmv0GpS/x7K9BRWwcAoaZ4CozBe6tuj4gYdv5X76c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jxm4hBul; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jxm4hBul" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2a9004e4239so2437625ad.0 for ; Sun, 01 Feb 2026 04:27:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769948820; x=1770553620; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nLkYdcrr/V75vjPKPAtmOhxaUd8xlxxQ6DkPLWqUmvU=; b=jxm4hBulWmeeKTVSupYagVDcFKXPjNL588NJgi87Avbf9KJCCVZWrgwlQa/HBlWCte jnOpAV+0kBpXqlCp5YKUoa80tNIjcZE0T7WIK/Q8QX1U7LorFjPHCvknMtu+WF9NKxRB iPlXgRzw7nPWSIwWnTIqQ62vUP+TRFJlY8EPOizMupDKgLmIymbtxqdAzC+kaBa40OWx 1J0chVAQUws+7hivlT7gZqm7WUfL7cSkYuFDzIKZc/ki5ZYydl0uGzXj1tYUdaRTsz+K eUhVGlpfClEeEOo6fUt2en9Flp8PWtLOPQa+j9/EJ7oBrXLaJgeygamPGsaxfGONO67J +MEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769948820; x=1770553620; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=nLkYdcrr/V75vjPKPAtmOhxaUd8xlxxQ6DkPLWqUmvU=; b=WQIgtfOejfEkNkRYEeshUV3Sdr3eRCA7deCBZGJex+yj1uy3CwoPzyIIqEkmBVEPb+ fuYmPG7vhmFV/vZldGuxbNSKck5lysxQ5CRe5ZLH3UNLDJ864/FojguBEvnWVgJwQ+o9 0aWwJlLKBcDbKK3zIufmFGQvqmnp1C4yEKEzlVXQswOUWrgVmqmL1psBXOCpYhePIY7U C8nmU1BQRaPs4TCQOUC+m74uh6OrdB2tR/Bis017WEwtRoJzTt63mKmOmEgKTChlDiH1 fGXySOAQzQwjnF/n7YSle/RwfwPkQ9VkZRh0tCSCpK3yuddBwgvSP+NLzPKgvBks77CZ 3fxA== X-Forwarded-Encrypted: i=1; AJvYcCUILKHyEQxaEYKwvY3HcOqy3AEFW+7eCNKUl9uhaX1kMOqgf5Uubf5JEDhZh40rn8lBfTbmbJcloLMpQug=@vger.kernel.org X-Gm-Message-State: AOJu0YxQlrHAmbz+lSXH3q1jOMtPzO8AXKI9xQddP+kp8ciwbYEJkqb5 6v+5Lv+/77sSUZaeAbVm4jV8Pqx2HohSJtssTR16qO0ofwsTK5p/y+At X-Gm-Gg: AZuq6aKchuWZRQ16LHT+Y+EA8Wd25Fzy2cFnr9nlGzXafqAr0CLxWmnMlfAr+JEM1gd g4UHpAfPnIqrrhnSSiHpRa+vHP0QS6f39jtsLHVIt35EKxJtCDkOsZI5ToHn8d923TLEd+T4Qjj b2UulhxsVGuw0EIMm1dm9WzomvMMe+Xgeua/KaGAMXxgKS/wpbMN21k5PmJbMPO9dBDJWJEQ3AL f8N282P9cOMtFRFKRT7l5aOuHxw/WkDmUY3jyJwyVO4ZRAMR3rUVFG9lO+ySJpOeRnULzR5c2KG U8OKtE2vb9JSdZ1iiUG/nn97/EG3Ha8yV4kibPvkNcKaMrSKwxHHuAsC+kgOXiIV214t3PzXn+r mKAH8CJ7WTcfogD4w3cD7YSw17PuoIyzZl/e9QjH9+ep7+jy4zqS6bodT+Fy/i3E+cT6/qXABhC 8yAYWa6ZVrKLbpeRS3qRDm/Rit X-Received: by 2002:a17:903:2442:b0:2a7:9da7:d21d with SMTP id d9443c01a7336-2a8d9a75759mr89932075ad.56.1769948820242; Sun, 01 Feb 2026 04:27:00 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a88b6e5ac4sm116941185ad.86.2026.02.01.04.26.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Feb 2026 04:26:59 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v6 5/5] mm: khugepaged: set to next mm direct when mm has MMF_DISABLE_THP_COMPLETELY Date: Sun, 1 Feb 2026 20:25:54 +0800 Message-ID: <20260201122554.1470071-6-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260201122554.1470071-1-vernon2gm@gmail.com> References: <20260201122554.1470071-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang When an mm with the MMF_DISABLE_THP_COMPLETELY flag is detected during scanning, directly set khugepaged_scan.mm_slot to the next mm_slot, reduce redundant operation. Without this patch, entering khugepaged_scan_mm_slot() next time, we will set khugepaged_scan.mm_slot to the next mm_slot. With this patch, we will directly set khugepaged_scan.mm_slot to the next mm_slot. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Lance Yang Reviewed-by: Dev Jain Reviewed-by: Barry Song --- mm/khugepaged.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index b4def001ccd0..94cd064f79a5 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2549,9 +2549,9 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result VM_BUG_ON(khugepaged_scan.mm_slot !=3D slot); /* * Release the current mm_slot if this mm is about to die, or - * if we scanned all vmas of this mm. + * if we scanned all vmas of this mm, or THP got disabled. */ - if (hpage_collapse_test_exit(mm) || !vma) { + if (hpage_collapse_test_exit_or_disable(mm) || !vma) { /* * Make sure that if mm_users is reaching zero while * khugepaged runs here, khugepaged_exit will find --=20 2.51.0