From nobody Sat Feb 7 17:42:18 2026 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 140242E5B09 for ; Sat, 7 Feb 2026 08:16:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452198; cv=none; b=DjG39rLyX03A+rUIth7gLMUZK0ffKjcNqoU9p61B1yVyi8ViHI28k+TkUpRgbppyAgBnZHychBW61vJMeoYhUBwycIcfhwb5aRFbO+xa3FGHquVPvqi/TGcVhNq/27g/Rqw2ncYAIZARBX4JG7Bg9AiR3sk1Rq7QY7vEdXIi6EI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452198; c=relaxed/simple; bh=1p6SZCQrdbboucL0Q2oNL3jrZX5kqL7bBTlXarhNvAY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RaXV66ltlo4P1D0Qv9k1XV8QgDl+n7bzHJkIkYbN5oXShVxEEefYK3YoqmZoO9Zz38tZ8I4sD3PBiDCmRSj7VQBR3EMqxZ52A0eqQC0jF1RfOtPsRIlVOdb2qCtRJJ87j99sNW7ZKVvvjlHbuUkrnNa/hdXkL9KqKIwBJ8KpXPg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IETuN7uW; arc=none smtp.client-ip=209.85.215.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IETuN7uW" Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-c56188aef06so1146411a12.2 for ; Sat, 07 Feb 2026 00:16:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1770452197; x=1771056997; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vfAfOKttZZZtCqQp8K1Dj0tvri2/xt7pdNxOOLU+9Pg=; b=IETuN7uWb3sXnBcSqk5o+ZUKPYk4fx1h/Oy4kn9Iofta0XnxB9Qv/GiLwSgbjFQfNU lXKCGOJN9hjLSOXtH/LO/cerFnsVlspSU58D+Ca8zMiDapTY+8OphIfagK78X+SuoC1A tB/pk/FpuoOC2pafC9yHEIDv5FXHoi9GPibaJkfi6k40sIkmUR/+PNCbqhxXjwBA0gZk K/KteqsDM9tLnkrYjw9TcldbTEvChui37OaIv/6wsGQhlHreuWKevfvKd31eFJEUGzcV dBy9u6lIW/fQWrCVXp4rV9t8NIjY78NXauZPBSvPZkTeuF2sklSZnuumQTGAeJd4wePD 0/BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770452197; x=1771056997; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=vfAfOKttZZZtCqQp8K1Dj0tvri2/xt7pdNxOOLU+9Pg=; b=NOAl0JQc/pnIdCDsrxIm9PUpS4ceuqjK2543b5kHqy0LPt23xgcFizDeF8lkyTVnRO V3vaY73Z7zxzaHRpZh2sx+gjlMsABUrNNr6Pn9zYSSwtO5GMq7Jg5iAyI5D99cZY1Yx8 FehEUT04qtQ9PK/2O3AANuoVQV/BjgmcqmfTZ1uZkiGO7ato2lI4eeQ1Zipmf6iV3U/D ejH2qZzQgp2hHEj9T4jFmRfVoGUgznIb/QrcBV0h5hUogcK+7vCv9119tuvV+DxySTGm RZGGpIXtHdQXlBok97QMWOjaIrf6iLrc11FPNdR6ANoXvuHYuxDn9nt1HDz5o6KGutGq eiaA== X-Forwarded-Encrypted: i=1; AJvYcCVSJ2t1iRNdlug5wm5PI7UgPavOYSdKHlfWEQWgUOtT3Ez1Mg/iXTe0WqI3dXtXgc81Ag/Vayw1NSVKX9o=@vger.kernel.org X-Gm-Message-State: AOJu0Ywis/EoczIaUTmfSp5HLi7Y47zlCEYdvB7pgqdvPukugaBG5ORl O/5bi7x9pGx/OseThWO1bHjQcdWkiNC4xp09IqqQGyq9MqF1qeMnp7HZ X-Gm-Gg: AZuq6aJ3FU1m87mQGBrSlfX0HTzdtCYOi9GVjkPIdNCwxd9zAk0gkJRD7GNOFuSGzlj IyDGFuWM06TSBQFRsbP6pRIAfdDXhsIVlh0HJXMANnWMbaeFVK+a/VoDdvL9C3u4YfKxReLu11c hswDa56fHfYz5Iz3oxdRbJ/suKb77nFc4ZEboTAp+hOjCV7rWTBvq8un0aHdwot4xiOFYpd7UYL kUaeyW23gZ7jsOvjeGbMdMwPXlDFT/p9AB1Wjf5IbXLWyyIIElJQvh9HGfncxq7iPRrRDzjAotz NdHlCOxMSgbvFcJfJv7IJaU6Sk05uuo4k40A0tbncgcKD8OoqlrYmNoUv2KywAUY1m5Xr17HfYz /hPqCj2gX9V2x+LfvAtdWAy+wf4A9OzsFbfmTs+NUekBzpzAowCRA2jZaOjlwFN7oTDKfYhtrDM sCXctMQc/ljgqmP8ZXuR915mU= X-Received: by 2002:a05:6a20:e20d:b0:38b:d93f:b45a with SMTP id adf61e73a8af0-393ad376b08mr5111954637.58.1770452197241; Sat, 07 Feb 2026 00:16:37 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3549c5955a7sm8028189a91.17.2026.02.07.00.16.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 07 Feb 2026 00:16:36 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v7 1/5] mm: khugepaged: add trace_mm_khugepaged_scan event Date: Sat, 7 Feb 2026 16:16:09 +0800 Message-ID: <20260207081613.588598-2-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260207081613.588598-1-vernon2gm@gmail.com> References: <20260207081613.588598-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang Add mm_khugepaged_scan event to track the total time for full scan and the total number of pages scanned of khugepaged. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Barry Song Reviewed-by: Lance Yang Reviewed-by: Dev Jain --- include/trace/events/huge_memory.h | 25 +++++++++++++++++++++++++ mm/khugepaged.c | 2 ++ 2 files changed, 27 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 4e41bff31888..384e29f6bef0 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -237,5 +237,30 @@ TRACE_EVENT(mm_khugepaged_collapse_file, __print_symbolic(__entry->result, SCAN_STATUS)) ); =20 +TRACE_EVENT(mm_khugepaged_scan, + + TP_PROTO(struct mm_struct *mm, unsigned int progress, + bool full_scan_finished), + + TP_ARGS(mm, progress, full_scan_finished), + + TP_STRUCT__entry( + __field(struct mm_struct *, mm) + __field(unsigned int, progress) + __field(bool, full_scan_finished) + ), + + TP_fast_assign( + __entry->mm =3D mm; + __entry->progress =3D progress; + __entry->full_scan_finished =3D full_scan_finished; + ), + + TP_printk("mm=3D%p, progress=3D%u, full_scan_finished=3D%d", + __entry->mm, + __entry->progress, + __entry->full_scan_finished) +); + #endif /* __HUGE_MEMORY_H */ #include diff --git a/mm/khugepaged.c b/mm/khugepaged.c index fa1e57fd2c46..4049234e1c8b 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2536,6 +2536,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result collect_mm_slot(slot); } =20 + trace_mm_khugepaged_scan(mm, progress, khugepaged_scan.mm_slot =3D=3D NUL= L); + return progress; } =20 --=20 2.51.0 From nobody Sat Feb 7 17:42:18 2026 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 66DEE2E889C for ; Sat, 7 Feb 2026 08:16:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452203; cv=none; b=knOg8U3eKrm/qC3tN+uaEhPjGkL4SWypn9KJQkHGq385xo+Rd3ptIgmwc3CgTDWsqlgHS9Htm40PSNabxUFTy2O7cnbUv6uKIJEnUCWaEzyofWMWkCVmCa0ImrKsQ4pnP9lLH7AX0DHzosG1LTryVl+X2WZUlV6l25fDKptekds= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452203; c=relaxed/simple; bh=R95W9Hwhrpb9t/80p3jLPzvvpaiuV/mmSq7XJn55mGA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qRyRYC36LJowhcAveKobFu7Bdd8l5nWPg25RdUGQh7e1NHSIzHBgIAuA8PwzH+MRcsPyPI4r2tV0iBTNLwEleXymZTgqzMFZIdlIKr2NqjH5fb+Qxp7MUU7nYp4cPlPaE4y/sVhSCxlxPlLV9mWs2w25h0Oyrl+ADNicg8u5XGQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QiasCdal; arc=none smtp.client-ip=209.85.216.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QiasCdal" Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-3543b9f60e3so1306943a91.3 for ; Sat, 07 Feb 2026 00:16:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1770452203; x=1771057003; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aSufcolqrQVMe5y8EVPFg87ZCQCerJ/wUIqeIeJwkF0=; b=QiasCdal3wWzs0XISbf1pElsSi+vbt4qfdhS9Cs2sytdnl2DshZNbp/YK0DPqT0iVY tJ9XHxRDe/yj+pd8jfjnXi2OJdXloOlxI/gBunJCKd0XKsplI5ZoLlCB3gIFgjb9Ub4v k8h6mm4hqSHced3kT/LrAgngm6xIfJvri0gwca6JPNN2ElNJ1D2CSjPjKH9VmuIIIu2M qkVUJx4HnImuSvA3UR4I/PQ5JIAM4EfK73yBAv7qLSY+QmoghNkBRUMINiIoN1TVSP0Z T1/7AYdLt14ILTRr6gAA52VetyMrIRbY7hWnCW55m2AMl/4iV6XQcTpr+Q7WL9w7GJKn U1yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770452203; x=1771057003; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=aSufcolqrQVMe5y8EVPFg87ZCQCerJ/wUIqeIeJwkF0=; b=FABDfxItcuLlJl4jxXYIx6bUY51yTg4D2w5+lQOwcD6QSPjcvnrzHaC6rMLNgmcTkH tyTWjNZtT2CpC2AmIlL2Y7EGTtVw0YmJue5Id7ppfp8xVv1QlRRa5mVGoGGuswQvOlwL CiyDZAhv6Jj65m4LKd9/2Sz2ymgkudhjdkWnLysThVgHShDTbuZUzuoPoStpDkHdgD58 3Jy/9q/5C/jnVvXS+3mmEB3HXjDgIivmITeHVWs31Jrb3nUN8ekGPIAvEGtLNet3uRIH 4MkdspOQUdJQ88pJyWhwNnXcbhv3i0LgIKdGXkpDeT6FKGNh4HKAnVa0fJBrKsMH75Sw GXbQ== X-Forwarded-Encrypted: i=1; AJvYcCW1QOay0rG0t0y5f0sqjOvKkWpRNV8Mqnso2EtI13qOXV2joe+swMfovEiiXT5HjE/EN23GPyFL4hHUiv4=@vger.kernel.org X-Gm-Message-State: AOJu0YwRIXUzji+syDEhBXsmZ9CEmdWaeikfHdFkR80oqqRXHEq3rvpN tGXkYjwRPfZC312LxoXtuILVdiUyoa9fMmsNRTKDCNTziow1k7F3F7amRI0AJ9RN X-Gm-Gg: AZuq6aLWrQ1J+OIW3vvJlGOITfWleL7I3RqukaWpmG5C6myO6HBgdiPgGJsVSgonYtI Oa3Vqh7LeJiAqZsr8GjG6IaxiKmxAxWPin1B+GXXLzC9+bn2q61AiWV/SUkUcSO68WVdOaVJQVJ UL6R/9SNfGGiiYlhrRuWiPPds0J1pxrRzcHyuZDhyjZsVAJ/Xz3PgBQlT2Jn2MmHeh9Dbi6Wtkq BLoQNixiP5WHAoX3FKtIg+K8wJ/2O2WOwcgf+nto3FioyCGUrKsqGcTHTWUjzkMyJx/UzjwzVpr HUOj12JR7wyvWRD9vKBSeMrX8eSC+jn25E229QsP4f/pfPAmJL3a6YbQJjhoECTjCO5PKDKljxj 0u2Sx/9HmGoY8DJIHBQu/X1teGfaVTjcWBiiBrytZiXyigiuYTeaPot3Y58wpHZ68ITcTMX80s7 GainoGpC3KyqCuFYdVLQqMR20= X-Received: by 2002:a17:90b:3c12:b0:354:be2e:c056 with SMTP id 98e67ed59e1d1-354be2ed154mr2936251a91.18.1770452202653; Sat, 07 Feb 2026 00:16:42 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3549c5955a7sm8028189a91.17.2026.02.07.00.16.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 07 Feb 2026 00:16:42 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v7 2/5] mm: khugepaged: refine scan progress number Date: Sat, 7 Feb 2026 16:16:10 +0800 Message-ID: <20260207081613.588598-3-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260207081613.588598-1-vernon2gm@gmail.com> References: <20260207081613.588598-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang Currently, each scan always increases "progress" by HPAGE_PMD_NR, even if only scanning a single PTE/PMD entry. - When only scanning a sigle PTE entry, let me provide a detailed example: static int hpage_collapse_scan_pmd() { for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { pte_t pteval =3D ptep_get(_pte); ... if (pte_uffd_wp(pteval)) { <-- first scan hit result =3D SCAN_PTE_UFFD_WP; goto out_unmap; } } } During the first scan, if pte_uffd_wp(pteval) is true, the loop exits directly. In practice, only one PTE is scanned before termination. Here, "progress +=3D 1" reflects the actual number of PTEs scanned, but previously "progress +=3D HPAGE_PMD_NR" always. - When the memory has been collapsed to PMD, let me provide a detailed example: The following data is traced by bpftrace on a desktop system. After the system has been left idle for 10 minutes upon booting, a lot of SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE are observed during a full scan by khugepaged. From trace_mm_khugepaged_scan_pmd and trace_mm_khugepaged_scan_file, the following statuses were observed, with frequency mentioned next to them: SCAN_SUCCEED : 1 SCAN_EXCEED_SHARED_PTE: 2 SCAN_PMD_MAPPED : 142 SCAN_NO_PTE_TABLE : 178 total progress size : 674 MB Total time : 419 seconds, include khugepaged_scan_sleep_millisecs The khugepaged_scan list save all task that support collapse into hugepage, as long as the task is not destroyed, khugepaged will not remove it from the khugepaged_scan list. This exist a phenomenon where task has already collapsed all memory regions into hugepage, but khugepaged continues to scan it, which wastes CPU time and invalid, and due to khugepaged_scan_sleep_millisecs (default 10s) causes a long wait for scanning a large number of invalid task, so scanning really valid task is later. After applying this patch, when the memory is either SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE, just skip it, as follow: SCAN_EXCEED_SHARED_PTE: 2 SCAN_PMD_MAPPED : 147 SCAN_NO_PTE_TABLE : 173 total progress size : 45 MB Total time : 20 seconds Signed-off-by: Vernon Yang --- mm/khugepaged.c | 38 ++++++++++++++++++++++++++++---------- 1 file changed, 28 insertions(+), 10 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 4049234e1c8b..8b68ae3bc2c5 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -68,7 +68,10 @@ enum scan_result { static struct task_struct *khugepaged_thread __read_mostly; static DEFINE_MUTEX(khugepaged_mutex); =20 -/* default scan 8*HPAGE_PMD_NR ptes (or vmas) every 10 second */ +/* + * default scan 8*HPAGE_PMD_NR ptes, pmd_mapped, no_pte_table or vmas + * every 10 second. + */ static unsigned int khugepaged_pages_to_scan __read_mostly; static unsigned int khugepaged_pages_collapsed; static unsigned int khugepaged_full_scans; @@ -1240,7 +1243,8 @@ static enum scan_result collapse_huge_page(struct mm_= struct *mm, unsigned long a } =20 static enum scan_result hpage_collapse_scan_pmd(struct mm_struct *mm, - struct vm_area_struct *vma, unsigned long start_addr, bool *mmap_locked, + struct vm_area_struct *vma, unsigned long start_addr, + bool *mmap_locked, unsigned int *cur_progress, struct collapse_control *cc) { pmd_t *pmd; @@ -1256,19 +1260,27 @@ static enum scan_result hpage_collapse_scan_pmd(str= uct mm_struct *mm, VM_BUG_ON(start_addr & ~HPAGE_PMD_MASK); =20 result =3D find_pmd_or_thp_or_none(mm, start_addr, &pmd); - if (result !=3D SCAN_SUCCEED) + if (result !=3D SCAN_SUCCEED) { + if (cur_progress) + *cur_progress =3D 1; goto out; + } =20 memset(cc->node_load, 0, sizeof(cc->node_load)); nodes_clear(cc->alloc_nmask); pte =3D pte_offset_map_lock(mm, pmd, start_addr, &ptl); if (!pte) { + if (cur_progress) + *cur_progress =3D 1; result =3D SCAN_NO_PTE_TABLE; goto out; } =20 for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { + if (cur_progress) + *cur_progress +=3D 1; + pte_t pteval =3D ptep_get(_pte); if (pte_none_or_zero(pteval)) { ++none_or_zero; @@ -2288,8 +2300,9 @@ static enum scan_result collapse_file(struct mm_struc= t *mm, unsigned long addr, return result; } =20 -static enum scan_result hpage_collapse_scan_file(struct mm_struct *mm, uns= igned long addr, - struct file *file, pgoff_t start, struct collapse_control *cc) +static enum scan_result hpage_collapse_scan_file(struct mm_struct *mm, + unsigned long addr, struct file *file, pgoff_t start, + unsigned int *cur_progress, struct collapse_control *cc) { struct folio *folio =3D NULL; struct address_space *mapping =3D file->f_mapping; @@ -2378,6 +2391,8 @@ static enum scan_result hpage_collapse_scan_file(stru= ct mm_struct *mm, unsigned cond_resched_rcu(); } } + if (cur_progress) + *cur_progress =3D HPAGE_PMD_NR; rcu_read_unlock(); =20 if (result =3D=3D SCAN_SUCCEED) { @@ -2457,6 +2472,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result =20 while (khugepaged_scan.address < hend) { bool mmap_locked =3D true; + unsigned int cur_progress =3D 0; =20 cond_resched(); if (unlikely(hpage_collapse_test_exit_or_disable(mm))) @@ -2473,7 +2489,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result mmap_read_unlock(mm); mmap_locked =3D false; *result =3D hpage_collapse_scan_file(mm, - khugepaged_scan.address, file, pgoff, cc); + khugepaged_scan.address, file, pgoff, + &cur_progress, cc); fput(file); if (*result =3D=3D SCAN_PTE_MAPPED_HUGEPAGE) { mmap_read_lock(mm); @@ -2487,7 +2504,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result } } else { *result =3D hpage_collapse_scan_pmd(mm, vma, - khugepaged_scan.address, &mmap_locked, cc); + khugepaged_scan.address, &mmap_locked, + &cur_progress, cc); } =20 if (*result =3D=3D SCAN_SUCCEED) @@ -2495,7 +2513,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result =20 /* move to next address */ khugepaged_scan.address +=3D HPAGE_PMD_SIZE; - progress +=3D HPAGE_PMD_NR; + progress +=3D cur_progress; if (!mmap_locked) /* * We released mmap_lock so break loop. Note @@ -2818,7 +2836,7 @@ int madvise_collapse(struct vm_area_struct *vma, unsi= gned long start, mmap_locked =3D false; *lock_dropped =3D true; result =3D hpage_collapse_scan_file(mm, addr, file, pgoff, - cc); + NULL, cc); =20 if (result =3D=3D SCAN_PAGE_DIRTY_OR_WRITEBACK && !triggered_wb && mapping_can_writeback(file->f_mapping)) { @@ -2833,7 +2851,7 @@ int madvise_collapse(struct vm_area_struct *vma, unsi= gned long start, fput(file); } else { result =3D hpage_collapse_scan_pmd(mm, vma, addr, - &mmap_locked, cc); + &mmap_locked, NULL, cc); } if (!mmap_locked) *lock_dropped =3D true; --=20 2.51.0 From nobody Sat Feb 7 17:42:18 2026 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98F3921D599 for ; Sat, 7 Feb 2026 08:16:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452208; cv=none; b=sBCHyKkE96vd2IOEAHSNM90shhl6xITo/fBZzRc6gQHWEpbDUwytOQwTXjg1IfLYGquccmkbxIjWz6u/DMG8hA1kRCiix6lxo72iNoCJEiS5ANxoZwXWcLFyWm32bYiabvryZCRkmFzaiZ8tiDPbT/dgyYOH97nMH1jaj8vjgKk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452208; c=relaxed/simple; bh=Z6d7rqqZyoR5g+yVG0uq1rhLun/EFdgXG0Q3Hb9WDXM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HpIIooozHWmC+qk3YNACZGMvkuiFF12Itp5ZGmSIOalOza5GCcT8Y8DWeOgKukrhPq5h256CIgFsWfXkL4VqcLtcKLck6+2LiLmEV8nJm3KfiO6EvPEAuvRovgul0gHvTxEc6vSdtztVYBqQLOJ50goP4RQqYn1HoQUGQtqJfS8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dgJ+BeKx; arc=none smtp.client-ip=209.85.216.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dgJ+BeKx" Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-354a18c48b5so2590788a91.1 for ; Sat, 07 Feb 2026 00:16:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1770452208; x=1771057008; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3c08OVs7S+JipvsoiksDQinUdwR5RLYJ0JjNcJIizyM=; b=dgJ+BeKxzQDlty+fHQwLoep3hpWfrdiOHya+mnTJrROdvHi8a4Ey0IyaEcxWMhJkK0 DPfMPFbVOhvRcvNOyBPPJAkbTiPz+W2UltYT/mo2a41JV7HF+r1XiExb3sHO1INe+4Ml c+h/7qAm1NNlwVf4tJeA7ieAglOWTW+iuN8BC5QyeAQLmVzvtB5PFMZs/lrCk46/LohY 8d7HZdVJlUHPyeOJdR0t/jAxqcDu934JncQr/bV8E6briT8fNTxVPR27DcCD6XufIcg1 byia3O48tYEnS88Q+xJ4n7web5R/C5VSQrr2HOIQ3OM1BGtQfSAM8bLRFKTR17+IJ5sl vJZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770452208; x=1771057008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=3c08OVs7S+JipvsoiksDQinUdwR5RLYJ0JjNcJIizyM=; b=oVCGAWKp0l9BpsK3sTxSBbHby3HWlI8RXtlsAPBEAmmpX9o2PWK3fRr83oXHpsEQYf faWwMiOpvXz6pimGV7PZcL622pNqsSTg2OP8bwmqOdWsoY2iA3RfNGBDO8Aw0ENQKW3t ENg/xgP+5TTCHC6VSH+heI4vJCnLmLz2GA4lH+XBZxtdcokLU+INBe+iGMfDwsClDWuy yW8cffsmjw7Gzwh2ZlVib2LGoAlJM4MnEXU/dxdPMLS1KDdzuSpRqgmiTvCBult7QL3N BL/JvTik/zWbY88+FS7ied6VupZ5u4aB6t1JrQ+RIF4YKPeP496qy6A5x/OyKUx3MNNW /jAg== X-Forwarded-Encrypted: i=1; AJvYcCX6aj+u2rkczw2V443i99Q2qLo75WW3ypKXEqa27Rv2XLGozaLHDXm5zAlU9gvQgtnYTZ+lorr1htsiaGI=@vger.kernel.org X-Gm-Message-State: AOJu0Yz+skJInUSvIyOmJqhvDxRQ8d3IrsFK0clYMDMPGVLloYbL1WbE b1PVtvEZOjuo1ahKRCTej8ssQnjG83zchKhPn0yWecJB+z6Fhq4UEFUd X-Gm-Gg: AZuq6aKpSzghdPPHQO8qUayhWMg0i+68xt1qT2AvOTKgPNuPeedbH7OFXp1lQUx5Fk9 rid1KHb7Yhohhfmciy4woqIig5z614qpzFBh74uK+U7tqXzrSndYpv8BckGKxQx8Sc1a7/XZT8D 0CJiITM06rIwTg53RPhnfq2sak0U1MhqC+lEmPRLnGHwusII7LeLsub8b5yJcyAsSRi8hes/Mec r7dgWjWJJ1uT6Eybslk6x2UTPuepwddzoInLNNgWl2MZLhwOHczOHsO4AX9rrVTaF0nnZFpuiTU 4bZSiKjqTeYVulghs3UcwsfgFhapy1p1Tij1tZ/51i1O6VKpKYCYah2Hoew606sfzqUkwGzkTXt 4ZhirRTieb8+eEztwdFjoS6cwoNKio5r2iCJzrhuxHI8EvWmtZRKzDy+RVq/otNa99HXaBBDmnW OBeCM37Okj4HPHqIOD7Hu/aPg= X-Received: by 2002:a17:90b:50d0:b0:354:a05d:9dc2 with SMTP id 98e67ed59e1d1-354b3bc1a64mr4635025a91.9.1770452207842; Sat, 07 Feb 2026 00:16:47 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3549c5955a7sm8028189a91.17.2026.02.07.00.16.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 07 Feb 2026 00:16:47 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v7 3/5] mm: add folio_test_lazyfree helper Date: Sat, 7 Feb 2026 16:16:11 +0800 Message-ID: <20260207081613.588598-4-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260207081613.588598-1-vernon2gm@gmail.com> References: <20260207081613.588598-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang Add folio_test_lazyfree() function to identify lazy-free folios to improve code readability. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Lance Yang Reviewed-by: Dev Jain Reviewed-by: Barry Song --- include/linux/page-flags.h | 5 +++++ mm/rmap.c | 2 +- mm/vmscan.c | 5 ++--- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index f7a0e4af0c73..415e9f2ef616 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -724,6 +724,11 @@ static __always_inline bool folio_test_anon(const stru= ct folio *folio) return ((unsigned long)folio->mapping & FOLIO_MAPPING_ANON) !=3D 0; } =20 +static __always_inline bool folio_test_lazyfree(const struct folio *folio) +{ + return folio_test_anon(folio) && !folio_test_swapbacked(folio); +} + static __always_inline bool PageAnonNotKsm(const struct page *page) { unsigned long flags =3D (unsigned long)page_folio(page)->mapping; diff --git a/mm/rmap.c b/mm/rmap.c index c67a374bb1a3..e3f799c99057 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -2049,7 +2049,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, } =20 if (!pvmw.pte) { - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_test_lazyfree(folio)) { if (unmap_huge_pmd_locked(vma, pvmw.address, pvmw.pmd, folio)) goto walk_done; /* diff --git a/mm/vmscan.c b/mm/vmscan.c index 44e4fcd6463c..02b522b8c66c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -963,8 +963,7 @@ static void folio_check_dirty_writeback(struct folio *f= olio, * They could be mistakenly treated as file lru. So further anon * test is needed. */ - if (!folio_is_file_lru(folio) || - (folio_test_anon(folio) && !folio_test_swapbacked(folio))) { + if (!folio_is_file_lru(folio) || folio_test_lazyfree(folio)) { *dirty =3D false; *writeback =3D false; return; @@ -1508,7 +1507,7 @@ static unsigned int shrink_folio_list(struct list_hea= d *folio_list, } } =20 - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_test_lazyfree(folio)) { /* follow __remove_mapping for reference */ if (!folio_ref_freeze(folio, 1)) goto keep_locked; --=20 2.51.0 From nobody Sat Feb 7 17:42:18 2026 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99ED021D599 for ; Sat, 7 Feb 2026 08:16:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452215; cv=none; b=DdOl19dkFAYldYHkS5WkljVI2/NMq70cqXMXiY1n+cMf0n1K6ASuv5ftVRwFF4vmeO8tlgRHQjcDxF/gm+HivNTxgCRdMXwIZaLIKNo/Z1YWbkRW2TiqC8g0fTuNEXrOsMcWT52yxnx4m+hWdT5L/vXb2ca1KTpJcdh7KqlLeHE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452215; c=relaxed/simple; bh=vt4tY3XJ3Z6GHRXm44uljElM26c4m3xCmnKDN05g95s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ah6HYZJquRyfvUWRMqta9hpFRz4DB98Y8NnjkjZnVv6iiwusTnq3HX6NObHS2wUK1sFpzEUJIHnwUUPcOc4Q21Ix7XDP/fxvoRTrfdDuhVD/crgGjAQYzlQ1AbdmyE4hGZaXwaQ95n7hi+UmLzRNT29p9Cay8f2fELV0F5zzWQY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nLqjqsYs; arc=none smtp.client-ip=209.85.216.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nLqjqsYs" Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-354b79a9ad5so628732a91.1 for ; Sat, 07 Feb 2026 00:16:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1770452215; x=1771057015; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sRPloOPmIvdTkQu3q4j2KYKMkvQYHLzGzqGMt4kDAVw=; b=nLqjqsYs1zOdZOnlFwCgjghOH5iI1c+SeOOTfEB+dIRpw1+V4D/LW3gWmD3HMiiyth w0XPuy6EiZvMsT6Kd19d2tbiiwNv52gVxKuuvBTv1/H9IMvVIvM3Hw7NpbiYOZFv8zSz x84OmjnzOFvtlMAU8wId0e2C6wlcU+coycKsBW1hdy5nNkvktotf2Dqo6LQ3sTkveo31 TSvyE9kfp/I+UhU3N+kTRT/cTeETFY1FAlpQaYpdRZjzsCcqO3oQGWkETUu+nDNQiQQZ yHz9THwprAl1MSWpWLKVtRurRdCPxC5rSIgAqD3257GIxgYUw8nOAp9l4X4I5XMMniPr sC3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770452215; x=1771057015; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=sRPloOPmIvdTkQu3q4j2KYKMkvQYHLzGzqGMt4kDAVw=; b=JOFmotrUu9eMSjeQWdQoKgeqbFhs9BqAFc/teJ3liz6GpImbYqvf/C2/b4J8NxG2Qi jv0FXI0qaNpgIPqaW9TdxVxY38dxlCVwrfNqwoyjsDHDs0/fSHyr5z3v2KcnSKJP0o59 EMxwSKJvCvttqCCEm1mFGijzb+NvFb8OpsBa+QHqmhn45RLXP0pSRfhzB832pGW1U/zA b6LTRsY5d+BTq1T4DdknfgaT4e6nxkUpEybeSx8AQHhfmavtv18PWG/J7lJWRwjAc9Up 1qI/2hH30tKwhe6lq4OiLiTca8U3vdQi5f6HHP5poTmdjaY1JsjTk/Gr4wRKsm+KoeF7 F12Q== X-Forwarded-Encrypted: i=1; AJvYcCWs90If6fB7oAbegUg4LXMNfOpoO/YPET2at44mY5/1qvUvWPIAMAb3bK6w9SynLVR6djQY0GJ3ONB7XXE=@vger.kernel.org X-Gm-Message-State: AOJu0Yy9xw9WE2Jjxd/QeqqFghlbm3+J7aCYRSNShl5AiTvbpDGVOM6x aTxy2P5+15qcQyFmpDX9OcN4AMM8P6v7QoeFn5Pnyu/EnfeLzur/4HeF X-Gm-Gg: AZuq6aKWsfjQjGAqPpy2UQFBOh6qOT3xygTPSUJjSXLemgh0Dvz/ojsEel50ONtnicW ECR36ahtHW9KAhkwBNqy+QEzVeuXDJTjigUeK4Pz378iv6gG+5u2Rg4epHCDksVNsB4VoqaDglp 9t+iVO/EbkxjJAo6K7h8k7Ibo2U9yRS/0oBMapfU2aGV0p9bHS2txbOLdboXS5udz6jaRmXjFYF giPshRfE+d15LRpN9iSMgEH2lTF+RSfgwQSbNUAhsFc7w6HpW4PGk/mEakUTRmeh90M51k0Ca9z iPpZSL20EWoE5yAF8M6DXl6+hP5OLQvzuXgx8PedmhF7C+MPdGMRbZUsTPd2VukOWA+9oCN5ngh YnWb8ckHvtWaEiUveV1rAKOVcj/UsVZB6cnUD/4dLMyVy00rrAhMb9D31sAnJJE7lgJk3NfI6jN FA15Yua8z/l9xeG3cmu27VAk0= X-Received: by 2002:a17:90b:1b51:b0:341:2141:df76 with SMTP id 98e67ed59e1d1-354b3c81322mr4971509a91.13.1770452214869; Sat, 07 Feb 2026 00:16:54 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3549c5955a7sm8028189a91.17.2026.02.07.00.16.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 07 Feb 2026 00:16:54 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v7 4/5] mm: khugepaged: skip lazy-free folios Date: Sat, 7 Feb 2026 16:16:12 +0800 Message-ID: <20260207081613.588598-5-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260207081613.588598-1-vernon2gm@gmail.com> References: <20260207081613.588598-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang For example, create three task: hot1 -> cold -> hot2. After all three task are created, each allocate memory 128MB. the hot1/hot2 task continuously access 128 MB memory, while the cold task only accesses its memory briefly and then call madvise(MADV_FREE). However, khugepaged still prioritizes scanning the cold task and only scans the hot2 task after completing the scan of the cold task. And if we collapse with a lazyfree page, that content will never be none and the deferred shrinker cannot reclaim them. So if the user has explicitly informed us via MADV_FREE that this memory will be freed, it is appropriate for khugepaged to skip it only, thereby avoiding unnecessary scan and collapse operations to reducing CPU wastage. Here are the performance test results: (Throughput bigger is better, other smaller is better) Testing on x86_64 machine: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.14 sec | 2.93 sec | -6.69% | | cycles per access | 4.96 | 2.21 | -55.44% | | Throughput | 104.38 M/sec | 111.89 M/sec | +7.19% | | dTLB-load-misses | 284814532 | 69597236 | -75.56% | Testing on qemu-system-x86_64 -enable-kvm: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.35 sec | 2.96 sec | -11.64% | | cycles per access | 7.29 | 2.07 | -71.60% | | Throughput | 97.67 M/sec | 110.77 M/sec | +13.41% | | dTLB-load-misses | 241600871 | 3216108 | -98.67% | Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (arm) Reviewed-by: Lance Yang --- include/trace/events/huge_memory.h | 1 + mm/khugepaged.c | 13 +++++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 384e29f6bef0..bcdc57eea270 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -25,6 +25,7 @@ EM( SCAN_PAGE_LRU, "page_not_in_lru") \ EM( SCAN_PAGE_LOCK, "page_locked") \ EM( SCAN_PAGE_ANON, "page_not_anon") \ + EM( SCAN_PAGE_LAZYFREE, "page_lazyfree") \ EM( SCAN_PAGE_COMPOUND, "page_compound") \ EM( SCAN_ANY_PROCESS, "no_process_for_page") \ EM( SCAN_VMA_NULL, "vma_null") \ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 8b68ae3bc2c5..0d160e612e16 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -46,6 +46,7 @@ enum scan_result { SCAN_PAGE_LRU, SCAN_PAGE_LOCK, SCAN_PAGE_ANON, + SCAN_PAGE_LAZYFREE, SCAN_PAGE_COMPOUND, SCAN_ANY_PROCESS, SCAN_VMA_NULL, @@ -583,6 +584,12 @@ static enum scan_result __collapse_huge_page_isolate(s= truct vm_area_struct *vma, folio =3D page_folio(page); VM_BUG_ON_FOLIO(!folio_test_anon(folio), folio); =20 + if (cc->is_khugepaged && !pte_dirty(pteval) && + folio_test_lazyfree(folio)) { + result =3D SCAN_PAGE_LAZYFREE; + goto out; + } + /* See hpage_collapse_scan_pmd(). */ if (folio_maybe_mapped_shared(folio)) { ++shared; @@ -1335,6 +1342,12 @@ static enum scan_result hpage_collapse_scan_pmd(stru= ct mm_struct *mm, } folio =3D page_folio(page); =20 + if (cc->is_khugepaged && !pte_dirty(pteval) && + folio_test_lazyfree(folio)) { + result =3D SCAN_PAGE_LAZYFREE; + goto out_unmap; + } + if (!folio_test_anon(folio)) { result =3D SCAN_PAGE_ANON; goto out_unmap; --=20 2.51.0 From nobody Sat Feb 7 17:42:18 2026 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 929D72ECE9B for ; Sat, 7 Feb 2026 08:17:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452221; cv=none; b=j+FELX71XBv5bSQ50m4XwrGUByZFnk237E64r63Q9/3jVHT+r56OrcfkJlTLE6V1PRhph9xu1Odq4vG9jG05damO4BbH5rE/QYRWjCQJWIak4LzWPlYmPUjoV2PsFxr3RhGmUrQ8dEmQDBwl5rNL0f/5RzB8oDcnQg8dZE7ZaGQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770452221; c=relaxed/simple; bh=Rcg+tXwacEl249McOhXGg7HBkn5NuR3nohA1vEaW5U4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j6zz8MKLoQlp2yZJ8UXEYu1nAT4icb90Aw8ppXcVgs5PVs4jpqJna5lrc0magWeX87KrdwKM+97kZSD9Ad8w170E3ZUZPCNkKcbVVSshWWBzx4yrgH51ntLKX5w0Wxm89B1mvyTXLDaLNT5JC6NziA75rI9NnVcpa/Ytfl4Hihc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QhcWKeKG; arc=none smtp.client-ip=209.85.216.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QhcWKeKG" Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-354bc4bb75dso566121a91.2 for ; Sat, 07 Feb 2026 00:17:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1770452221; x=1771057021; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xJyWfDVg0o3SBi33JofJIZ8UeyAfWxaA6T2+Wopx5fg=; b=QhcWKeKG9TrEjA5/GNSnzo9N2vYAu01hjsQK46AjV/V0D3TRkAuMUNSzmvAzOkVO7N P0Vdlyjq5lIAis53KJ+ICmCOViEz8Rt/D+Ac1aOHQ/BnkJWWooWbDXBT10fCjRbzLLDW uMizS5aAetQFsfrbuOGhFBzkbYIxnpzvNvb5WKVMn3LR2PgmajxOOowGd2JKAVfon+ah xGFpoMmwXX90/iwcYTH8pGMpwsWi8F+G5H//bI/tni3H7/or/sMxLLKiUp8GxMWZUunY +PQwUIjbEIiwLbTNQmrmRrOoPSsymYW8hA/3tdmOb1YutQujB3vTRvDlDpIyDjOZN/0z vlAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770452221; x=1771057021; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=xJyWfDVg0o3SBi33JofJIZ8UeyAfWxaA6T2+Wopx5fg=; b=YtH9M2xIUg6O/3267YeTjIphcQSdW+kzTTckxS+AAkCEde2uFqBOgu4CGuOPfy/+on eHkQ6z4MfAlKhayXql1ixJOaPmhfQIBU1KNo2s2sGnmM0UsW1hlcAQGBuswHLmzlsgqB I6o1v9AQFl+obP5PGpFWbTsvj0+7dJFl1NQGYEpooExP8wjDyMU/MGAME1TCVJYDpwFy w/sRHniuote+nHMHcqm4Jh5tf0FXVoTaEEYzL7TFO9gx6099xaxegvUko1EWnkjtWscQ fobthVpTPT4kue6IIH+WU9Xy+OHhkLgTpW/w950UubiR4gwtgP4QZxFUINID/VzffxER WaKQ== X-Forwarded-Encrypted: i=1; AJvYcCU03+JCljjoaz0asSjhTWOclturyjSOqfwAJYxwCydMypTKZre8j91fYnW8TvwQ3S4pvokl3XrFXcr6nXI=@vger.kernel.org X-Gm-Message-State: AOJu0YyILG3yTOFcntwiXYItHwCOYq0ColFUKffscDGP8BHr9L/MStSl A4K7u5L3TgE/boid9wGXpPJ48LYIWS0JdZZyHM28e5ZuPr3B3scGTaNF X-Gm-Gg: AZuq6aL8MQ4ZLvh4O0poOLcXilOkeQNIz0RZiq9KGRLAJOr5PAqrRyrS8PwM82gtpSM 4WZoT+0l9g+VkTBwHTxsnF71tz5xYjqk4OnZ0J0K9DrwzdmpnxweUmiX9zpKr0BvzJZUV8wlrcy SukQMViqCw/otAAxRW5YeSscCMCrYmDVltEtUy7W007nZMYCGESBXu90nmxSCESicShf3ZAkCTq m+Ydf0OGPuAC3TImz9VSSylGRPAMteVAnNso/yn/TLOrkJ5RCv43qM6olaCBBDhw6IouPB/Kta+ hw5U5jWwnEexCHl1ZEVBZpM6x8jSLt8HCn6cvc8nDCqtIb/By5AQUljnzol5TlBYx2FS06ETGCc j32Wq3oos3V3Xzw9sWuUan+FVMsjLtVTtRulIzeKY511IG4l6Q5Xn2nzs9VlRJXdkaj2VftMast yJaj5jpCkfZMJuykzHWja6+dg= X-Received: by 2002:a17:90b:3d90:b0:354:bd08:4812 with SMTP id 98e67ed59e1d1-354bd084d2bmr3021238a91.31.1770452220884; Sat, 07 Feb 2026 00:17:00 -0800 (PST) Received: from localhost.localdomain ([114.231.118.96]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3549c5955a7sm8028189a91.17.2026.02.07.00.16.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 07 Feb 2026 00:17:00 -0800 (PST) From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v7 5/5] mm: khugepaged: set to next mm direct when mm has MMF_DISABLE_THP_COMPLETELY Date: Sat, 7 Feb 2026 16:16:13 +0800 Message-ID: <20260207081613.588598-6-vernon2gm@gmail.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260207081613.588598-1-vernon2gm@gmail.com> References: <20260207081613.588598-1-vernon2gm@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Vernon Yang When an mm with the MMF_DISABLE_THP_COMPLETELY flag is detected during scanning, directly set khugepaged_scan.mm_slot to the next mm_slot, reduce redundant operation. Without this patch, entering khugepaged_scan_mm_slot() next time, we will set khugepaged_scan.mm_slot to the next mm_slot. With this patch, we will directly set khugepaged_scan.mm_slot to the next mm_slot. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Lance Yang Reviewed-by: Dev Jain Reviewed-by: Barry Song --- mm/khugepaged.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 0d160e612e16..b3854d990fd9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2548,9 +2548,9 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result VM_BUG_ON(khugepaged_scan.mm_slot !=3D slot); /* * Release the current mm_slot if this mm is about to die, or - * if we scanned all vmas of this mm. + * if we scanned all vmas of this mm, or THP got disabled. */ - if (hpage_collapse_test_exit(mm) || !vma) { + if (hpage_collapse_test_exit_or_disable(mm) || !vma) { /* * Make sure that if mm_users is reaching zero while * khugepaged runs here, khugepaged_exit will find --=20 2.51.0