From nobody Mon Feb 9 03:47:46 2026 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 823B02BDC17 for ; Sun, 4 Jan 2026 05:41:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505295; cv=none; b=GQeu6CsSQGavX5KAxBg+9ovEiMNiew7pl8IdCdUaTTnmhf3oHyeAJT0mE/4GihuD4GYP8sujAUUFFEapsnAW3xuRjH2zr5ES25MW6U67t+Gi0i6Pjql+j1AtHU+rLEd95haWazXS6clR3ZRaxZqN6cHU6sNXpWxybu46Wrzaju8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505295; c=relaxed/simple; bh=QQ0QO/8xstp0D10MRrLCeqlQVvDF6qM4JXq6mrSbg8o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=An/HWUl/bM+QlP6vAToCYq/38QlSLFhuvV6l45JPGXFfYAq37b/rkfR0FHzHhIOwoaxqgcXVwcu0nAD5w5XYCEWf7cLORJjnLf3w+uYgj9YvIeBudamALec0uhlZpu6fiPQZ6PLeN+1toKTp3ZthBKVI6IDB6n/0PNq1XXZ0/I4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fJDXWQgN; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fJDXWQgN" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-7aab061e7cbso17428734b3a.1 for ; Sat, 03 Jan 2026 21:41:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1767505292; x=1768110092; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Rpl0DJg2v/KakAKdFhQZ5XDWhxnaAUoonNHKQfZhl3w=; b=fJDXWQgNiCTL02ZGLZdQdx7G6HdSJYx6Fr64mNaUPeXUVtwGL1yrHgFZ2UCff3MXBn vAzPW2x7v0EXpPhAICY9EogpeRnCB+QbeucvrLbPJRdtdjxswMgutf7YNt92ZIzcn6v8 IKRZ+lcHrwlrr4yNeTuRMeGf1GLitWrIEpiDZ5yg+QWzqRewAl9eqwFsbsYiAykO7uCS +U7V4hTgEjFxWPrhS96pLmpnYXLm4j3YlWmr/UGal0Daif8rRKELFkNH0OmI+042xr/a /P3lSN9ZTJGet21WBiLqxHRsz/xdgCa9CZ1IXRjMWe8BiE3XJfyAUfPc0dRpUy73WUsy nCJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767505292; x=1768110092; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Rpl0DJg2v/KakAKdFhQZ5XDWhxnaAUoonNHKQfZhl3w=; b=nFEByTIznYDnCp2DgM+5WNT8H6xR6Bl0Aw7o1lVa0ZCOT0sIWcqI829N6iFPE3zAcr EfVc3A6MnWekr8HE6cQquPgjzQQ27t5oZsZk1bEOlnF915OXlimMpk8XI5v6pL7JqN9X UL9ug9Az20ZO71gkgrxLDfmXb3oqarpfLlKJ9aoLP8Ib7RfeRDZVmoKaYlgsV4vTyQxs 73yB2734GBRiHELFSJXwathTq/uw8QyGixhQL+wANhoHN3qK0AhRqz9OUx41v028g+L8 NeP2nPBQbl84ANG/TvlodC9eWj7+DblssnvpRevskqRDEDOEUGwyF8KPFMc9DGBpZUR1 g5Eg== X-Forwarded-Encrypted: i=1; AJvYcCXX1xW/XVrISfdhUiDSqqyDsykzPKpDZ25tQYJub0xpjE8E2IU1+5tzYlx5TeJOymHlPab6O7Y2xtknEdQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yxrz0/AgxFD98pg3Xld29yhWJkZsYmmN9lAhH0qYiELInjebKXK GNZZtKSz2uy4rc6R7pUdwwPxpW6QxQmvvlM048FvWfwC3eQsgQYCFw4V X-Gm-Gg: AY/fxX5v/rYVgU+aCp8xS/jw4bMTrhGIUMN2HaPR9xqA1QXDW/UdM8KxWGHRdoaANsC 9J+zW242eog0i1dWjyUO4aghvlozOtacuiYSIvDXRxvbWijc2MmGCXztpJA7pjHtABcvfsRGCFk vrG5TrEuZbgY7ADhQlHiugUT4FF/PNUUh2lnRtCnNU6kmijQhkwY+KHlOckEIzg/OIhkd4Eskka vWQ2f6/JqpOHZNXP/u4KIiHbDbVpj2BKm6OZjn7bVI0ElYAJJ8ewnz55T+ni4SlqiZB7DkB3JoW a87oIEreELglTrZ4sDKsDfLueiT8aScmg/ApiHcM9Sq0NOBgjdKP761mTvtkaJWxk3ILL5RewvR O87dQZbTcZaYx7hLDaVp0ZSGpA8UpaFFIqe7PObMXji+VGbtn07RxYy597XQFs50AQ9Srmw8QK3 oMYbH5T67g3EvketbDprQIlJf/lw7+ X-Google-Smtp-Source: AGHT+IFdgXlhA+f0858RuZwqzqH7rMm8CggQEup+GBlChYcdnewuKcV3RbKYDUtgdiXOZ/Cbz/FXfA== X-Received: by 2002:a05:6a00:4217:b0:7e2:c0d5:372d with SMTP id d2e1a72fcca58-7ff64404001mr41125060b3a.11.1767505291778; Sat, 03 Jan 2026 21:41:31 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7ff7b31c479sm44378305b3a.24.2026.01.03.21.41.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Jan 2026 21:41:31 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v3 1/6] mm: khugepaged: add trace_mm_khugepaged_scan event Date: Sun, 4 Jan 2026 13:41:07 +0800 Message-ID: <20260104054112.4541-2-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260104054112.4541-1-yanglincheng@kylinos.cn> References: <20260104054112.4541-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add mm_khugepaged_scan event to track the total time for full scan and the total number of pages scanned of khugepaged. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Barry Song --- include/trace/events/huge_memory.h | 24 ++++++++++++++++++++++++ mm/khugepaged.c | 2 ++ 2 files changed, 26 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 4cde53b45a85..01225dd27ad5 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -236,5 +236,29 @@ TRACE_EVENT(mm_khugepaged_collapse_file, __print_symbolic(__entry->result, SCAN_STATUS)) ); =20 +TRACE_EVENT(mm_khugepaged_scan, + + TP_PROTO(struct mm_struct *mm, int progress, bool full_scan_finished), + + TP_ARGS(mm, progress, full_scan_finished), + + TP_STRUCT__entry( + __field(struct mm_struct *, mm) + __field(int, progress) + __field(bool, full_scan_finished) + ), + + TP_fast_assign( + __entry->mm =3D mm; + __entry->progress =3D progress; + __entry->full_scan_finished =3D full_scan_finished; + ), + + TP_printk("mm=3D%p, progress=3D%d, full_scan_finished=3D%d", + __entry->mm, + __entry->progress, + __entry->full_scan_finished) +); + #endif /* __HUGE_MEMORY_H */ #include diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 97d1b2824386..9f99f61689f8 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2533,6 +2533,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, int *result, collect_mm_slot(slot); } =20 + trace_mm_khugepaged_scan(mm, progress, khugepaged_scan.mm_slot =3D=3D NUL= L); + return progress; } =20 --=20 2.51.0 From nobody Mon Feb 9 03:47:46 2026 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F3732BFC73 for ; Sun, 4 Jan 2026 05:41:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505310; cv=none; b=YG7PYpG3cB0nLJH4lTXT4pV3BJ1gRq48BLPfg/wnb/htLrDxYsB/fhzwDdTdR2BECQibqgTfsccoMYMTNw2cufiI4IdoWg0buM51C4H/uUXJ850pZEvRPGCo0MHG1xb31VbuyTq6iwvmlnr/V2alHxSudHdhstXAbgjhHNU3Ie0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505310; c=relaxed/simple; bh=shcEAb+9YNdzsOxwpOc4hx66paPDoqZDRqqII1oMoPM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hS+F/6kCYChg8w8L7yQptXNilX6rIA/OYkt/pxgHIoF5o/NlxnRWHvZJeRBaXqxuDEz9EHRYih7/8yhjbOhYPCLVl/T5ce909OXUI0a2rPuaKqRin4/dHH1nTnwbh2xFiL5HYj/qqlLpr/nUANAFAYRTYJtnN8r1ExlOdSUAyS8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Jd8uJ6qv; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Jd8uJ6qv" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-7b8eff36e3bso20445321b3a.2 for ; Sat, 03 Jan 2026 21:41:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1767505308; x=1768110108; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YE9pc/4tAvhZsZXrRzmjeNPdx2fm0i699jC8cyE+mak=; b=Jd8uJ6qvzuT05culh3UdZDSuQpBpUwbORb5/5qL3ejWYJg7BxIXPM/lxQYCPXGXg2E fm6iGdGuloTuc3zNFycp92x/78fGB63wr9Rt1k8I/XVlEH7dRfq3wwr5SqkKXCNITay0 GzXQ1MO7knWCvFv9lvrGaqwidIX/kYsYkjiTfDRGOVeGmPDjlS1qnI2Ypxq1LsQcguGd MQ+XZfGSqoC5TsP39Wp8BMKONNRqsc7d7zPkB47TlYBNb5CALhypUEZI0uGXMhR33qC4 uEMhSopRnzbsKjSVb/AHFVdFePOluZ+HW+JQKAHJYDdAacYZVE+lNQliVypGI3abq0FX Wm9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767505308; x=1768110108; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=YE9pc/4tAvhZsZXrRzmjeNPdx2fm0i699jC8cyE+mak=; b=g3QYB14DbQ+URvt1Llv+C1h0OhvG5q1lfSY8rBhE3DW6fL+WGYJr8QfWHF6skle/DZ Zk7KcbpLDxRV6nDl23JdGV+Pmg4ev//d42Q5eLt2KfgpIXhhmkXn6deuSvntr5tQ5ztY oqsTanpCrCVc+uLHoqwh3XGD7GzsrMhXhvN/QYmi/EEljIz20Ip0iADw1L2tgAFEEURN m6wHLGzCYKZ52ttVQnTtcrnb7bPQpNeOt7t/t54T99+Kp//a228/S6mTXTfv792LfpLs oF1ZdsJa8v1wfYVhd815fsCwsblevK0M3jvEEVs319qQWosZc2xqBc3Hy/cswUidWG39 EzRw== X-Forwarded-Encrypted: i=1; AJvYcCUhHEAvFXq9sYnmN1Xmwh1yT44Ol8aD5sNlVtYOTlWlGVwQeVNwypxWPihtnveLICmnrMXUrPCMrRLhMKw=@vger.kernel.org X-Gm-Message-State: AOJu0Yx2rJrtat8+gdgoqnINrZhOoM5v27yRe/6Vg6VO5yALeoL0tUh0 iHpPmekhA2CtOGp6zXsIUZjwdrhfs/4Hjf8gXugLIew+eyODKFHr1eIC X-Gm-Gg: AY/fxX6mAzj5z9k9+zxP0iLOfr+0qbrHc0kgS0u591nEdR2xf7wB+ccS9rQMFRq/Psz KpSY8W91Sq5uwt/PdHU6xHyVcWSvWs8coRSMlFYL4nHSakYRWVO0+mAg9k9C9Oi9Y/l+/JGgyTO tNH9s9efBSSi0twRvjVsotFHADmhw+lGI1hliqTPe7ynfF+wPaTJ7DFhEEianrniPQMvdCwqQPY L/SOayixEsUER3Wb4GEVSDm03Uzw6LWy3/wFmX/4URVlj48gYF4GaHL2IRkmObcE5L5sN2f3nTb 0Lx7B5chpAgeb16br7RUMyMT1VkPW5+uNJXv07rQ9DrEr+0e4gVntjZ+7K7G61I2jcBWXLRYbDR w2/0XojrqyL9t5o5MWttC6NbtBzR+Tgony6C/SaR3nU3De1RoA4444PMJbaLDb/biOdk8nHJwgB BR0PTHNmFkK8iEZSYBK/jSyIEkW9kR X-Google-Smtp-Source: AGHT+IF7QgKeDTmtbweC8O75d3jf2oErQKa2BZ42N85Bzgsk9GFBi54QbRbc4L6KqI2F4q+/nYahLA== X-Received: by 2002:aa7:9906:0:b0:7ff:da9b:bd82 with SMTP id d2e1a72fcca58-7ffda9bbf79mr28733868b3a.40.1767505308352; Sat, 03 Jan 2026 21:41:48 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7ff7b31c479sm44378305b3a.24.2026.01.03.21.41.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Jan 2026 21:41:48 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v3 2/6] mm: khugepaged: refine scan progress number Date: Sun, 4 Jan 2026 13:41:08 +0800 Message-ID: <20260104054112.4541-3-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260104054112.4541-1-yanglincheng@kylinos.cn> References: <20260104054112.4541-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, each PMD scan always increases `progress` by HPAGE_PMD_NR, even if only scanning a single page. By counting the actual number of pages scanned, the `progress` is tracked accurately. Signed-off-by: Vernon Yang --- mm/khugepaged.c | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 9f99f61689f8..4b124e854e2e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1247,7 +1247,7 @@ static int collapse_huge_page(struct mm_struct *mm, u= nsigned long address, static int hpage_collapse_scan_pmd(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long start_addr, bool *mmap_locked, - struct collapse_control *cc) + int *progress, struct collapse_control *cc) { pmd_t *pmd; pte_t *pte, *_pte; @@ -1258,23 +1258,28 @@ static int hpage_collapse_scan_pmd(struct mm_struct= *mm, unsigned long addr; spinlock_t *ptl; int node =3D NUMA_NO_NODE, unmapped =3D 0; + int _progress =3D 0; =20 VM_BUG_ON(start_addr & ~HPAGE_PMD_MASK); =20 result =3D find_pmd_or_thp_or_none(mm, start_addr, &pmd); - if (result !=3D SCAN_SUCCEED) + if (result !=3D SCAN_SUCCEED) { + _progress =3D HPAGE_PMD_NR; goto out; + } =20 memset(cc->node_load, 0, sizeof(cc->node_load)); nodes_clear(cc->alloc_nmask); pte =3D pte_offset_map_lock(mm, pmd, start_addr, &ptl); if (!pte) { + _progress =3D HPAGE_PMD_NR; result =3D SCAN_NO_PTE_TABLE; goto out; } =20 for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { + _progress++; pte_t pteval =3D ptep_get(_pte); if (pte_none_or_zero(pteval)) { ++none_or_zero; @@ -1410,6 +1415,9 @@ static int hpage_collapse_scan_pmd(struct mm_struct *= mm, *mmap_locked =3D false; } out: + if (progress) + *progress +=3D _progress; + trace_mm_khugepaged_scan_pmd(mm, folio, referenced, none_or_zero, result, unmapped); return result; @@ -2287,7 +2295,7 @@ static int collapse_file(struct mm_struct *mm, unsign= ed long addr, =20 static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long ad= dr, struct file *file, pgoff_t start, - struct collapse_control *cc) + int *progress, struct collapse_control *cc) { struct folio *folio =3D NULL; struct address_space *mapping =3D file->f_mapping; @@ -2295,6 +2303,7 @@ static int hpage_collapse_scan_file(struct mm_struct = *mm, unsigned long addr, int present, swap; int node =3D NUMA_NO_NODE; int result =3D SCAN_SUCCEED; + int _progress =3D 0; =20 present =3D 0; swap =3D 0; @@ -2327,6 +2336,8 @@ static int hpage_collapse_scan_file(struct mm_struct = *mm, unsigned long addr, continue; } =20 + _progress +=3D folio_nr_pages(folio); + if (folio_order(folio) =3D=3D HPAGE_PMD_ORDER && folio->index =3D=3D start) { /* Maybe PMD-mapped */ @@ -2388,6 +2399,9 @@ static int hpage_collapse_scan_file(struct mm_struct = *mm, unsigned long addr, } } =20 + if (progress) + *progress +=3D _progress; + trace_mm_khugepaged_scan_file(mm, folio, file, present, swap, result); return result; } @@ -2470,7 +2484,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, int *result, mmap_read_unlock(mm); mmap_locked =3D false; *result =3D hpage_collapse_scan_file(mm, - khugepaged_scan.address, file, pgoff, cc); + khugepaged_scan.address, file, pgoff, + &progress, cc); fput(file); if (*result =3D=3D SCAN_PTE_MAPPED_HUGEPAGE) { mmap_read_lock(mm); @@ -2484,7 +2499,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, int *result, } } else { *result =3D hpage_collapse_scan_pmd(mm, vma, - khugepaged_scan.address, &mmap_locked, cc); + khugepaged_scan.address, &mmap_locked, + &progress, cc); } =20 if (*result =3D=3D SCAN_SUCCEED) @@ -2492,7 +2508,6 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, int *result, =20 /* move to next address */ khugepaged_scan.address +=3D HPAGE_PMD_SIZE; - progress +=3D HPAGE_PMD_NR; if (!mmap_locked) /* * We released mmap_lock so break loop. Note @@ -2810,11 +2825,11 @@ int madvise_collapse(struct vm_area_struct *vma, un= signed long start, mmap_read_unlock(mm); mmap_locked =3D false; result =3D hpage_collapse_scan_file(mm, addr, file, pgoff, - cc); + NULL, cc); fput(file); } else { result =3D hpage_collapse_scan_pmd(mm, vma, addr, - &mmap_locked, cc); + &mmap_locked, NULL, cc); } if (!mmap_locked) *lock_dropped =3D true; --=20 2.51.0 From nobody Mon Feb 9 03:47:46 2026 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4A642D7DF6 for ; Sun, 4 Jan 2026 05:41:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505317; cv=none; b=AYXgMSs9h05sMPOOXTQBamc9q3hQOjH/fytf2mngMLbM4svp0hl5pKNqAnzd4dT7ss5zHFdEh5EeIwjTF6WoWNpxrlNr/RakymWVtd+BIQzQUOXLir7CIm243Os8XPhExehxKFrYcPXKsC8kuQn2b+tty/peXB8s+YpRRnmpxrw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505317; c=relaxed/simple; bh=9yOpQtH2zcc67JQBBuQEcbumnyamBcAx/g5ECxUw7J0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ACYgfpNU3aGA6riPt1OdyQz5z9JJW/aDrY1COzKFlmBl1wo22Swzhw9X0mmYARn6tky4/02x+nxDugRzJTFPMseP8L1VxYG7xS9m3GYDxy49z9CP+UINN9a4g44wocWefIf2fGmaigq9GR0K86+wYsttFWPCOdSo4Ib+EtBSq4g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=B8EQ0A1m; arc=none smtp.client-ip=209.85.210.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B8EQ0A1m" Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-8035e31d834so10701031b3a.2 for ; Sat, 03 Jan 2026 21:41:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1767505315; x=1768110115; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QUwkpMJcxEqSkkbLVyH1UfcFj9WYjjkK0bK86vvl/Zk=; b=B8EQ0A1mK3QkIR5vFL+H9z4skEKLnroOxkfYada6JU5JQDUghs/DLc9m68H54hyaiT qLQbAbCCcJdHACT+CGpOb3hdY17VQHHdj6hRUSKLWdD7RvStHRxXtjWtdkwk7pEHCYIl fcmdPrB3nj+WzBSWbLWJuhx0wHppmaLaIXH2ppGwt4apc06MFi5tOrcZUo2+va0GLFZu 4bQ3L9xC9Dq15PlX5SpUo0vT42YhHRId7jLFFawxIQcvxLtPLyiAsF1tDQZKZ3oUjZ2b Z+zKN1EuSa68vftpD77lPJGTAPCHrPp+VlCKNe6/+qvR1L+5pdywDaOWKCd9p5/MQN7N XumQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767505315; x=1768110115; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=QUwkpMJcxEqSkkbLVyH1UfcFj9WYjjkK0bK86vvl/Zk=; b=lG5oVlbwKKET+3aNSM9hXKScz54VauYd8mbpRb43zBwyXsPh3VE9punY1WwzXYVqxe 49aLfhRA+JUh9wMfyplcpFSuJeVlx7YuBP3TJADQcI+rABS1sBVbLExjtkjTKQCtjGkc eSUxuopWM7+FzEAHoze7+L0WTJr8iA6WY4BZ1eWRJOWnSwguostH+NyeDzbo5yZ1e8ZO /hP+Wi7skZuj1yBO9SkPg1JUEzaHAvJ6md8nXBtGBjQqMsOKW+La0FKuqqU5wUahWV2B 1afpSVLykT4D1Zkm1ogLnn1+q/bRvtYnDykE++Rff0kct5W3RENG4P27gQSTMEBoyVuq M7WQ== X-Forwarded-Encrypted: i=1; AJvYcCX4iuwPLEGzJK5kRM8JRrPreAJ3mFEuZ9IFIrqGv/VmIKHYtX8fgeDnM+Dn7y1UaBG3pxOJXR02WIeAA0Y=@vger.kernel.org X-Gm-Message-State: AOJu0YwlTVa4GrF54nMN3S/I1XN4nYjne/k1A6SludVAJMZfrXiOFXIG JiO9eDLxw3JGVNJzaOqWwn3A+UsjeH0/BhbWfJ79Sy0fD5aMRWB8g1+P X-Gm-Gg: AY/fxX5V8JLukM8d3VlEyntfTs/fjoh/b3/N43ZARJTp3xd45mrctRpuCKf6aWxxONv ZRPPIUrl1cLEtVDg83xH6YSZYVbLc+o5+QLvyKLHt1abaNkx3y8nu0U6F1Z1rav/r1O7DPy9C6h 8MHvjOrfMQftPT7JFJiLSRyFK2KOhBZ9ehoAc/SS3RcQ2GeHqv0u6btchE42MQL1mRNVBm+PEiG 2KT+WZ5s0Fjmhy0bLMHFrll3Rqqj7iIktObq0ilFLoWwa6doWFzOywYSskJT9VKA4816wapba0o 9bsYRRLQGFJ7Ks46Al+Puw6mkRtN+FtuXn/UMFxGjxnmr/UlUicG2gHuXSnhGmWxURhar962Jp8 xLfNEWEY9VwPCaCN+acsmJ4yrHdJu8X4I2BJShOYGpSxLups2aCi1kEmTSQam8ca3m/+yRA5MEL Bz18JePrULdKsuKQVx4rWjnry51NL6 X-Google-Smtp-Source: AGHT+IGwHwdxXTQyKxKpW2n1/+t3fWr60BwAf/OcttAH9+pfbFW7cQq89AcUb3EhW+mxdXl3KrSRHA== X-Received: by 2002:a05:6a00:3385:b0:7e8:43f5:bd1a with SMTP id d2e1a72fcca58-7ff6735e923mr38179784b3a.47.1767505315075; Sat, 03 Jan 2026 21:41:55 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7ff7b31c479sm44378305b3a.24.2026.01.03.21.41.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Jan 2026 21:41:54 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v3 3/6] mm: khugepaged: just skip when the memory has been collapsed Date: Sun, 4 Jan 2026 13:41:09 +0800 Message-ID: <20260104054112.4541-4-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260104054112.4541-1-yanglincheng@kylinos.cn> References: <20260104054112.4541-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The following data is traced by bpftrace on a desktop system. After the system has been left idle for 10 minutes upon booting, a lot of SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE are observed during a full scan by khugepaged. @scan_pmd_status[1]: 1 ## SCAN_SUCCEED @scan_pmd_status[6]: 2 ## SCAN_EXCEED_SHARED_PTE @scan_pmd_status[3]: 142 ## SCAN_PMD_MAPPED @scan_pmd_status[2]: 178 ## SCAN_NO_PTE_TABLE total progress size: 674 MB Total time : 419 seconds ## include khugepaged_scan_sleep_millisecs The khugepaged_scan list save all task that support collapse into hugepage, as long as the task is not destroyed, khugepaged will not remove it from the khugepaged_scan list. This exist a phenomenon where task has already collapsed all memory regions into hugepage, but khugepaged continues to scan it, which wastes CPU time and invalid, and due to khugepaged_scan_sleep_millisecs (default 10s) causes a long wait for scanning a large number of invalid task, so scanning really valid task is later. After applying this patch, when the memory is either SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE, just skip it, as follow: @scan_pmd_status[6]: 2 @scan_pmd_status[3]: 147 @scan_pmd_status[2]: 173 total progress size: 45 MB Total time : 20 seconds Signed-off-by: Vernon Yang --- mm/khugepaged.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 4b124e854e2e..30786c706c4a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -66,7 +66,10 @@ enum scan_result { static struct task_struct *khugepaged_thread __read_mostly; static DEFINE_MUTEX(khugepaged_mutex); =20 -/* default scan 8*HPAGE_PMD_NR ptes (or vmas) every 10 second */ +/* + * default scan 8*HPAGE_PMD_NR ptes, pmd_mapped, no_pte_table or vmas + * every 10 second. + */ static unsigned int khugepaged_pages_to_scan __read_mostly; static unsigned int khugepaged_pages_collapsed; static unsigned int khugepaged_full_scans; @@ -1264,7 +1267,7 @@ static int hpage_collapse_scan_pmd(struct mm_struct *= mm, =20 result =3D find_pmd_or_thp_or_none(mm, start_addr, &pmd); if (result !=3D SCAN_SUCCEED) { - _progress =3D HPAGE_PMD_NR; + _progress =3D 1; goto out; } =20 @@ -1272,7 +1275,7 @@ static int hpage_collapse_scan_pmd(struct mm_struct *= mm, nodes_clear(cc->alloc_nmask); pte =3D pte_offset_map_lock(mm, pmd, start_addr, &ptl); if (!pte) { - _progress =3D HPAGE_PMD_NR; + _progress =3D 1; result =3D SCAN_NO_PTE_TABLE; goto out; } @@ -2336,8 +2339,6 @@ static int hpage_collapse_scan_file(struct mm_struct = *mm, unsigned long addr, continue; } =20 - _progress +=3D folio_nr_pages(folio); - if (folio_order(folio) =3D=3D HPAGE_PMD_ORDER && folio->index =3D=3D start) { /* Maybe PMD-mapped */ @@ -2349,9 +2350,12 @@ static int hpage_collapse_scan_file(struct mm_struct= *mm, unsigned long addr, * returning. */ folio_put(folio); + _progress++; break; } =20 + _progress +=3D folio_nr_pages(folio); + node =3D folio_nid(folio); if (hpage_collapse_scan_abort(node, cc)) { result =3D SCAN_SCAN_ABORT; --=20 2.51.0 From nobody Mon Feb 9 03:47:46 2026 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36A122C0270 for ; Sun, 4 Jan 2026 05:42:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505324; cv=none; b=bhhbkiQkCKb036c2+mdDCINC3OtXesOpwDl7CqYZacFkhfR8z+ucZW+jDB6R6lt3Geq7//umGrNQajtKwXtOclnuFvGeY4iANNyh8+jj189AcLdTMSYmsFV1cC8ewMQryc0o+IwhZgzi76FzImLUSOH4MvJ36E8c97m8O5uegEU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505324; c=relaxed/simple; bh=2gLwmmyo9Nd+x57pM8mh0o0Nhyq+PXJVQ5Nuw1iQFkE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TepezVlkyroo1ig1OoL8q4pLtCjyVFGeqXO55fZctY6+3SoiZI38Q3Wn7usZuw/pfdEdCD+4B7MfK9uiTUVSRvnTXMlV0DuJRrPGWStPF0BjGjDzVXJ0oe+RIx2gYt9zlfqMXjYmYBN9mDBwLrLi4iUdEOqZNlXpizjqvykQD8A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BXEkNz63; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BXEkNz63" Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-7b9215e55e6so7984475b3a.2 for ; Sat, 03 Jan 2026 21:42:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1767505322; x=1768110122; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a0Qdf4zMNC4D7WMZX9lJ+qvZLPu8CeBhQKpS69vVnOI=; b=BXEkNz63bg1EzRIxFgn3S7U4rimMsJgIKInbjweLnThx4JVx2KaWb/ln6vH74+7PeJ h6Zg+QRf8u+0B4VLtmTdhar9Ev5nHzF4/oo1a2sBHjuo258Ql1PYSOtjqc/gESDXovjB h5JFHWcdZ8Iad6KiZJSaZVqzP+kyTb6By3VPSRPz6c0g0X00TYa5insGVwj0f12+zjbP URm2Sy4Cn4Ro4m/AUhIiuc0VqNYVFz751jwPhxi3FjMZzZgpT4e5CQ/ISo18fdlZMMD6 I7bHcYo+CPFbgGLXo9vgTZmNw7+VGtEjCI8EnQ484FdWd7IKIaG7Z83pSE/nOe3N3A8z idiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767505322; x=1768110122; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=a0Qdf4zMNC4D7WMZX9lJ+qvZLPu8CeBhQKpS69vVnOI=; b=Q64ExeB5DSD+Zu8tGSCyD98r/gw910Fs5s5U+YqE1UdSyXOEGrIqF4kIT/DbzEc7Ux t2S4wnJm+h6VVFZYN7jATpg2zJHwBpHL2PbbGTjtSZWYL/eVOS6OrVKM/erl/HBOMDFt FzFqUMIPEbzOuVHpejJxvE6fcX7wgNe8NvpNPc/9guuldOZQ2IIphTxbme4SGH6GO/AW odatTT3FjBmEQAe+4mFnF1usvTX40+on0LuwLX+jUiNrp7GMZVy4rROIPuaxotdPKnKQ LR2bRZ8BbMVPiOrlVyvjM7EjzUSeFThZgs0GNgBNcUynYjWVA8N3e3mVysoxLBFK6OVR qwLg== X-Forwarded-Encrypted: i=1; AJvYcCVwzsVfFhAsNyJza2X3r/gLHjQIQIsW0uVb1AdXp4TtrNzTurpUu+cjRFLzIlGSrr3lse8nyP1zGlnkZuU=@vger.kernel.org X-Gm-Message-State: AOJu0Yzv+JthAEJ8akxq4AYwNxFvXLXMvglykU3Dbj6agsBhH5LvwcRs UisKSxVCAHXLpdXSu2DduZyiobJx1hQrRnY775P/JJyFmbppAkIfmXDk X-Gm-Gg: AY/fxX7Ehy7MigFsSMguBhVwXwjYTem8JwPfynzuwFbtappXR7WYSnWUu5fRwS6l7U/ OVBxOyobAgoOZJhHWec6ZUmybwFE+nzTGNSEQC2SPyro8+WSEd7h2Ovu1WnxTx2twC4fgrN/Asb UrgGN/DfRBv9VvDJYqCx/CxOUtuyBrxRqLOT6lSra74ULh5/zJudcJB65qcDEUzBRV4CQY2Kzz+ SRI7eRo6+8iJGUb/aFyAPoIrWasIyR6spneht79BltAtUol5pr4IWsIzDmqMbwRAIf58hvMD/pR vtEvG0w67f8bvf63ce0rGppmH7eId+QaYixW/Nt1kHjjW0TvDkuRKLGuwuGBqh3dxWsXt3NH8nA WeIUFgM9G+0v5LP3tkdoTSG7xFZyjwFkTry9Z0LzIIx9UXYkPUYb16050jkoqsuKp3jgyrzBAkx AMW1TfvDJCbxtNgHNZwt8n3qc4pQCD X-Google-Smtp-Source: AGHT+IFirQNJJjZbxgLMgoG5jja4E2rvCFQkOAek5C12Sly8D8X+LIOspm5GGHAV1I80LaxS1ZUgvQ== X-Received: by 2002:a05:6a00:1c81:b0:7e8:4587:e8c5 with SMTP id d2e1a72fcca58-7ff664807c4mr40316534b3a.56.1767505322454; Sat, 03 Jan 2026 21:42:02 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7ff7b31c479sm44378305b3a.24.2026.01.03.21.41.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Jan 2026 21:42:01 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v3 4/6] mm: add folio_is_lazyfree helper Date: Sun, 4 Jan 2026 13:41:10 +0800 Message-ID: <20260104054112.4541-5-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260104054112.4541-1-yanglincheng@kylinos.cn> References: <20260104054112.4541-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add folio_is_lazyfree() function to identify lazy-free folios to improve code readability. Signed-off-by: Vernon Yang --- include/linux/mm_inline.h | 5 +++++ mm/rmap.c | 4 ++-- mm/vmscan.c | 5 ++--- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index fa2d6ba811b5..65a4ae52d915 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -35,6 +35,11 @@ static inline int page_is_file_lru(struct page *page) return folio_is_file_lru(page_folio(page)); } =20 +static inline int folio_is_lazyfree(const struct folio *folio) +{ + return folio_test_anon(folio) && !folio_test_swapbacked(folio); +} + static __always_inline void __update_lru_size(struct lruvec *lruvec, enum lru_list lru, enum zone_type zid, long nr_pages) diff --git a/mm/rmap.c b/mm/rmap.c index f955f02d570e..7241a3fa8574 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1838,7 +1838,7 @@ static inline unsigned int folio_unmap_pte_batch(stru= ct folio *folio, max_nr =3D (end_addr - addr) >> PAGE_SHIFT; =20 /* We only support lazyfree batching for now ... */ - if (!folio_test_anon(folio) || folio_test_swapbacked(folio)) + if (!folio_is_lazyfree(folio)) return 1; if (pte_unused(pte)) return 1; @@ -1934,7 +1934,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, } =20 if (!pvmw.pte) { - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_is_lazyfree(folio)) { if (unmap_huge_pmd_locked(vma, pvmw.address, pvmw.pmd, folio)) goto walk_done; /* diff --git a/mm/vmscan.c b/mm/vmscan.c index 670fe9fae5ba..f357f74b5a35 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -963,8 +963,7 @@ static void folio_check_dirty_writeback(struct folio *f= olio, * They could be mistakenly treated as file lru. So further anon * test is needed. */ - if (!folio_is_file_lru(folio) || - (folio_test_anon(folio) && !folio_test_swapbacked(folio))) { + if (!folio_is_file_lru(folio) || folio_is_lazyfree(folio)) { *dirty =3D false; *writeback =3D false; return; @@ -1501,7 +1500,7 @@ static unsigned int shrink_folio_list(struct list_hea= d *folio_list, } } =20 - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_is_lazyfree(folio)) { /* follow __remove_mapping for reference */ if (!folio_ref_freeze(folio, 1)) goto keep_locked; --=20 2.51.0 From nobody Mon Feb 9 03:47:46 2026 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82EFE2D8362 for ; Sun, 4 Jan 2026 05:42:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505333; cv=none; b=m25AbH7i3XRfapgGfB4PeSjX9HzD1yChihYgsa5ymTnwWqPz7DZEUv+9+NxNCKzIga+80bgB0w4bLfW5nJZNFb5jDEP117W4EWn+8/wqigs3vXe0hrvWW1Ej602VBhAAAoNEKU/L9sg+TYmb2Gl1V70q/BmqQ+8X/O0sPXYzCQw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505333; c=relaxed/simple; bh=FOysYN8ltHVU1MuqeLKq7oUvq9wUK5NU10bKPOJTS1c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Exx6hdZncujujQkiVqkABpe/AEEAqfZIAlHk4xvOFV3Wz9tv7X1yMWlNWid+U2x4VcK/1vkEiRv7yax6KQdZv/q2WtOLxzPnjH9fFSOxfL/OXiMMA/YDstDUaGjCdPRGfDhFjTqIWdN7EApvKK3T6K1YBzQ7/xGTurPvsCFampQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=V0ryY+DZ; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="V0ryY+DZ" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-7f121c00dedso17443790b3a.0 for ; Sat, 03 Jan 2026 21:42:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1767505329; x=1768110129; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F44pzaXhOPAiorm8Jky5wxgt9AYKY8ytZVulzCsdHj0=; b=V0ryY+DZtl3uyhGB8jUQcbUWgmcF/68QOK64KTVagWfw4CN7/Nd1FYDed/59wxVVRN +Rkzas6acqkO2tEoU9ztnVZoVmMeIoUAE9pTdED9iIRHvOenw4cSoVPa+bvxnNhn7XLG k2qubou6a3bEAeIciJc+xdX9FAUdRl3q8+P+2sgqzktn2gpdHAaTfAPcEb+DbQ6CpSmq inKVneETBgX5gtjDkqw/93vtjElDDHv84uWLy/iy8FXmk/9lf9l0i3lEN3DOZhMCPVAv bFIWxkB/hI0vSWyig+kne59AKG9FbqapYRWk5lUccrZzcNFyb2eSH1rBVyD83aS5aCM/ ooQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767505329; x=1768110129; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=F44pzaXhOPAiorm8Jky5wxgt9AYKY8ytZVulzCsdHj0=; b=iB3WOB66cGTJMqL9nlIAdTqHYozY+7IXkrP77xnS8Y7BFkVJdQLp7t2CChWuKQuIdT DEQsgPWM+NKkWH4HFQqDGzLZp0QkBXfO5X23VgGQXH/jZrTTRenocdmq2RAaqy7Mf9u5 NKjB6LgJUFuqbVAOK+8d7W4ST1R4lerjgX9fosQEKBwuoX913I0YNXloFjBED/IIlCpS 7LeUX1gFsTKJLDuotxMTwE3WjU8NAT1BtiVJ2er+v+A6/unlup21UBbLoFGJTygjw5GL urxWvdVrrql3hKixFPRT2oh15fYfxPiv+R70Jb69hgKRo/7XefklWYh+BHDNqZguBwnu UrxA== X-Forwarded-Encrypted: i=1; AJvYcCV+DD4iL3d1RswEPb9HH5gxNxPXBn02xOklOOelCi6OWfw6thZz69dTqxrISGFmyhRhDuqON2IBUDBVEJQ=@vger.kernel.org X-Gm-Message-State: AOJu0YyB+/nGsvPYjHooRODxib6edoeKA9wdSSRfhh5nym6jzcliFoO8 aiqhAJcSz8ryQtjNtiLk+8qFFbYm2aVRI/pai4eXq6WKBtBFyh9qN3c0 X-Gm-Gg: AY/fxX5mTIrXVblWdd31ZulfVBlnapQ/WgdMfM3D1Gz7bGqACIub+cclQNSvvHF6WVf RnPRTqaXnogfZkX8o07FpalL3z39QLi7ohmt1t5u18yqV0ElEMNER3XrGLrOhIDDhP0wC80aPZ+ wLdKQINpvXdABhQZejKxXuo8Tx/UagI/bGl66NJGvlCHNYMN1uLyIsVbhfgEdfTX9MacMBsw5pz ZOI7o+upfIT0e1U1KOjEx81oaZB5h+omu0UhUaATRoLgiuBGXGvEOUv9AVeKyUtyBTA9t5Od4E1 288LR53MgGk5UF9b9sOcizWtetfYtVyCyKK1FA8FTHAU/zoNCo25Dngrg9OZt6l6YuQAavmymWa jWqGnmHzVXb/7NiQDejuGmQ+hnUhItJGj7kyqM2NxK0pagiMFonH7x7dzxB7rXVuRw0Tn5BGH9j T4MtjBHxkCNblOTcTvoda3Sul8e8St X-Google-Smtp-Source: AGHT+IHQdG3bTbRK9bIIj2RXdTLOpt0yuYf8P9QBrnIfSlCobBa6x5vT+EOIMVWV1mueCxVPAE89wQ== X-Received: by 2002:a05:6a00:e8b:b0:7e8:450c:61a9 with SMTP id d2e1a72fcca58-7ff6705d2damr42086059b3a.64.1767505328876; Sat, 03 Jan 2026 21:42:08 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7ff7b31c479sm44378305b3a.24.2026.01.03.21.42.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Jan 2026 21:42:08 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v3 5/6] mm: khugepaged: skip lazy-free folios at scanning Date: Sun, 4 Jan 2026 13:41:11 +0800 Message-ID: <20260104054112.4541-6-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260104054112.4541-1-yanglincheng@kylinos.cn> References: <20260104054112.4541-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For example, create three task: hot1 -> cold -> hot2. After all three task are created, each allocate memory 128MB. the hot1/hot2 task continuously access 128 MB memory, while the cold task only accesses its memory briefly andthen call madvise(MADV_FREE). However, khugepaged still prioritizes scanning the cold task and only scans the hot2 task after completing the scan of the cold task. So if the user has explicitly informed us via MADV_FREE that this memory will be freed, it is appropriate for khugepaged to skip it only, thereby avoiding unnecessary scan and collapse operations to reducing CPU wastage. Here are the performance test results: (Throughput bigger is better, other smaller is better) Testing on x86_64 machine: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.14 sec | 2.93 sec | -6.69% | | cycles per access | 4.96 | 2.21 | -55.44% | | Throughput | 104.38 M/sec | 111.89 M/sec | +7.19% | | dTLB-load-misses | 284814532 | 69597236 | -75.56% | Testing on qemu-system-x86_64 -enable-kvm: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.35 sec | 2.96 sec | -11.64% | | cycles per access | 7.29 | 2.07 | -71.60% | | Throughput | 97.67 M/sec | 110.77 M/sec | +13.41% | | dTLB-load-misses | 241600871 | 3216108 | -98.67% | Signed-off-by: Vernon Yang --- include/trace/events/huge_memory.h | 1 + mm/khugepaged.c | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 01225dd27ad5..e99d5f71f2a4 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -25,6 +25,7 @@ EM( SCAN_PAGE_LRU, "page_not_in_lru") \ EM( SCAN_PAGE_LOCK, "page_locked") \ EM( SCAN_PAGE_ANON, "page_not_anon") \ + EM( SCAN_PAGE_LAZYFREE, "page_lazyfree") \ EM( SCAN_PAGE_COMPOUND, "page_compound") \ EM( SCAN_ANY_PROCESS, "no_process_for_page") \ EM( SCAN_VMA_NULL, "vma_null") \ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 30786c706c4a..1ca034a5f653 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -45,6 +45,7 @@ enum scan_result { SCAN_PAGE_LRU, SCAN_PAGE_LOCK, SCAN_PAGE_ANON, + SCAN_PAGE_LAZYFREE, SCAN_PAGE_COMPOUND, SCAN_ANY_PROCESS, SCAN_VMA_NULL, @@ -1337,6 +1338,11 @@ static int hpage_collapse_scan_pmd(struct mm_struct = *mm, } folio =3D page_folio(page); =20 + if (folio_is_lazyfree(folio)) { + result =3D SCAN_PAGE_LAZYFREE; + goto out_unmap; + } + if (!folio_test_anon(folio)) { result =3D SCAN_PAGE_ANON; goto out_unmap; --=20 2.51.0 From nobody Mon Feb 9 03:47:46 2026 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF5002D839D for ; Sun, 4 Jan 2026 05:42:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505336; cv=none; b=IoXMUdb57kOVa/bDDsvmMK3H9ipP0bKjiVyA4vUJFZguZQrkc/PQgLJDLxvq4L/+10Q8WWCPAbnFsWdB8U1CJ++q+81R41cNM2Ohr8GouxwxdGNLzSu2br9meeMXadN65Z+aCDOQbv0dkv2Kz1Q8eNET2Vz96oK1C5CfwkBAiNI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767505336; c=relaxed/simple; bh=5Kz6Pr1aeiH0Q/BtXKHq2S3RLj6B5okwKqMkpOwrvmM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O3vlPvetC/UqjPXJ9Qaag5DUJ1aUCz3nsZbEV8vdrih4FHOT+osv0AdA5VEXwsvqSCKHM+xDI/r40FH8X2Y6FZBzJHxr01ZoauIPNjC+RAppYefqdBez7qp56ta83pVTmoHjJau1A90ZbpDtCCuY2LJr8qI3ab+ZzVYCqePjJRw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MJuC39rD; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MJuC39rD" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-7f651586be1so422659b3a.1 for ; Sat, 03 Jan 2026 21:42:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1767505334; x=1768110134; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mVZUTb4Qer8ajklVk6CG0+GjbQO/cX8wCCqLg6sFq40=; b=MJuC39rD1VgKVh7D3h+KV0U74WRBO+5UYrQEEHTQqsi+D7iD9trBO/eA6WnXbKrYzM 1437IZ/JIs1k0TXkm/fEhuvyTwvqxez4vnjGcEdayDCAJPrzhgtuqS4dbWHWUGKMdg68 Ues+TkDn8v94wSW35oln6xHJ+GhIruD5r1F88O7o2N6Ji7MaaAikBFUXn6T/0KDcCV64 WBhkB89Ge0ez3kQqexVmVUIkgPmG6wvGs8Nm7WNG/0II0XRyWCfm2KkkltmM09M0b4a6 pJ5q0HeAAKVpSNKOAxf2eibJ1nPJhszVNkua5VN4GQEyAoApgdDX4xFgVLSzqclFr/eZ Y88A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767505334; x=1768110134; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=mVZUTb4Qer8ajklVk6CG0+GjbQO/cX8wCCqLg6sFq40=; b=aPaabh/zs74HTf/Fma/tXZxA/qgQ7OxPT7+5cME5E9voJ072I4heSiAUZWqJqJlnh/ eVpDaxOCV8n1duFpD+dEDD8XREexoUJ4np83xUDmIhBShhOGAjL7CsgbsaqmOvLlfg9y kn0YKQPIA7aAqV+RC/5J8N9ZaQAH0HLXPNG8blEK0Jvnr848KdVW5ql3MofRURJBXv+K 1LVFntRN3xGuv221ohCg+WffB5IT5gX9WLfyMSujNT7MFAaF8JSo0pt8uXEKaxYG4iYt 5yCpDdf4+4m/D/79lH4D7fj62yIhzzHzcYn7DSD/2m3TJ8r7kvgYricLBo8TCI1KHgOi /u2g== X-Forwarded-Encrypted: i=1; AJvYcCUZ58PtpS1jlQc9M4BCE2qml0ugAc+qOzlyOADji07GIyt8TIYKNHvVgORfmxpRkCFdPtA01jdfdn06hS8=@vger.kernel.org X-Gm-Message-State: AOJu0Yx0au+IOeaIEZ+dlEC/ZVqBNDD21FlfngUZCYRikuGvVElJ7t/p 0GkjqH8bTK1MIQfywdbc9bzqcBSuHJ4qY08XqZuX4m+g6YdAWGi7WKON X-Gm-Gg: AY/fxX6TqI5EgPdrSzB5rGbTv5ge8yUe9KtsaQ6LHFIhkfNjtbx8W4wt6DC02/sYYyt jx4vkq4nmN+VseT9S+wu+MIPmevFiHRJsCIg75TLBBMEwxmResMRWk0yoUYAOhyFIAuxpz3TexD wyFAcWPKBjhBxTZ+pE4dWa+oUOCa0Na9kMh1xMvoxK1QHCwVCz7FHrRnZMVEtri5DkhGcjHlZER SbN2q90ZvCSCbeyQnp+S9c47ZomHwD6o4++mTTHYXn+BsyDoDPCJnqt2db+8EBQIcuIkyGC/Wzb OiIHZW+gP1JxRG82HyZ8ppPyXhzsJYGS9TyFc+m/rwOYIN0oWqhWydpgr5VToq3Pe6fBvko0GLO 3IFeR6yRJtUcWe8+HSVIEg9JDmv6/l88S5E/1hMJYCNF5LemKcH3d9lEbLq3x32JtKGB2JYCoSe F2S6TnLv1vnH9+U0CNF66LHmKkZWpx X-Google-Smtp-Source: AGHT+IF9unx0Mow5O8mwt5WnoBPeKUfe93MefKFW8YHG9g4srn7KZftNBnfeDA9v9eopBz9E5oLcWQ== X-Received: by 2002:a05:6a00:a386:b0:7e8:3fcb:bc3c with SMTP id d2e1a72fcca58-8148ac395cdmr3725987b3a.17.1767505334198; Sat, 03 Jan 2026 21:42:14 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7ff7b31c479sm44378305b3a.24.2026.01.03.21.42.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Jan 2026 21:42:13 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v3 6/6] mm: khugepaged: set to next mm direct when mm has MMF_DISABLE_THP_COMPLETELY Date: Sun, 4 Jan 2026 13:41:12 +0800 Message-ID: <20260104054112.4541-7-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260104054112.4541-1-yanglincheng@kylinos.cn> References: <20260104054112.4541-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When an mm with the MMF_DISABLE_THP_COMPLETELY flag is detected during scanning, directly set khugepaged_scan.mm_slot to the next mm_slot, reduce redundant operation. Signed-off-by: Vernon Yang --- mm/khugepaged.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 1ca034a5f653..d4ed0f397335 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2541,7 +2541,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, int *result, * Release the current mm_slot if this mm is about to die, or * if we scanned all vmas of this mm. */ - if (hpage_collapse_test_exit(mm) || !vma) { + if (hpage_collapse_test_exit_or_disable(mm) || !vma) { /* * Make sure that if mm_users is reaching zero while * khugepaged runs here, khugepaged_exit will find --=20 2.51.0