From nobody Mon Feb 9 11:07:27 2026 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7631B318BB1 for ; Sun, 11 Jan 2026 12:19:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768133983; cv=none; b=FxI+hn+NBl+GQbJ18EXt0hj/BNxk09NjvyIAhGh0pFA8LcxMFnVpnfqF3j9XtYNpZUua92GafzyZXrhbqFZf/qeDynvC9GOy7YZ7ZQCr/DaGQA2NbFtPsnz91woAIj3H8Wm7Tf5KDhcIwH8Aampo3/U0FozEC/h5GpitE8lKqLo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768133983; c=relaxed/simple; bh=0+fBA1Gt6ekqQD4rd04KAvEGgb+zmNONNonULFDkgVE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jHVEwJphlcGLs1FOFORzGVjl2ITuXZmZXFtraLs1SD6eTsqeCgSytcRwbZ2auy02qEPhQIRYTSBjGaEV55y+S+NezGlie79bKeco0bQtSkYxTsYtsxCu9gVWyTwNWzjHwWGxtK23nAFKPBOXxe/aMnJO9iO8kXW5iGqQJkQZiMk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Gil3MM+B; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Gil3MM+B" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-34c93e0269cso2475080a91.1 for ; Sun, 11 Jan 2026 04:19:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768133979; x=1768738779; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ci4bwqZz1BI8ky/Etxr2jWaIj/tqNbukbL1aId5SFks=; b=Gil3MM+BnSnUXoJFl3UQwA9bPS+ttlWjP6qq60GvEk/PF6jg8NGwXAhXceRQCzEeO7 OfnU6vOozS08d6eQKIZK3JhLlvgQGXgWdVChaauRFQOcNTENYH+Ri8I/1wQB+0tJCqub lAQeYQ4dEUobPDdasZEeJZw6iZsbb6+VkQ2mHDlEtw6TKn3vbH3SWj7YtyDlX6z37tSl dA9cSdX2ALXnMj6wgT/0Nb0NnOMsxs21vKrWXLykyojb8eB8FWCeteYeZYCeZo1Wt9gm gXu5C31PHExb1bWOVlHIwS62L01BrHovg69ZkfWGDMA1kmMK5vTdwOjmclefzWkYSS0+ wnzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768133979; x=1768738779; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Ci4bwqZz1BI8ky/Etxr2jWaIj/tqNbukbL1aId5SFks=; b=pfJ9s7J9jrIgaVBlgKdgLG0yQPsX0oV8DG4VgDc4db1dAH0CMmIb4DFtpSvlsaoVDs iughZtnFGxHneCPK+CNQ/irTRCUIqOfqC8o0CtdYumbCKwCj3Ei59TvgSvGTNv6mzTKY 5vSrmmIYKVMoUqawlMUQslv0OuL8fb17DHregEphsa5IDJ/pRGkSzVkWJD4hIffVxai0 PwZY7f5fcIbbqF/O4tr8Oo0GVMRGNBVG49iYtse4rHosFl+zs53OBF4r4NZaO4fz0KQk h4tsES8io/Q8PcT1ehudwBYdm2JMNKsW0p/A9bVrjox7fO3fROIn1cfvE+72/XcIxX3t OxCw== X-Forwarded-Encrypted: i=1; AJvYcCWKMXbrKKvXSi3P+/qQ3djohO/V94kBVTJpUub3NM9GxmkehS2MZppXFqWXjtoiNCjiO/+uDODPpHr74RE=@vger.kernel.org X-Gm-Message-State: AOJu0YwQbLn94vghCFRFtxG1Upxcw/x4r2D3aW2VOaP8HDv1pB0HZ/+k 5gU7EBlbsQV4INUX0gCHZZQh8Wf3fpIo8lJmR/soMAli4aU43uOGMoWP X-Gm-Gg: AY/fxX5KzrhaH8hcCCt+jMhH3mdxu1jMrEp4JR45M++Xkwa/4OnS2w54yYRtOboiUIB +HqqCax27cEf6IB0b3ZtWMR71Elj4qC4Sw4batuKV2GzYXIJfDA2Tce4P/TefnTFtDqju9P5zsr e+vuiTfTIMR5Rq4x9F5HwsPkSofktTLyDWwGpVcM6AKiJzy3qFqgD5eoafk5ICfqQqEQKgSKrMA HNvHToEviNkewqzhJtkkb2oM6ifoLYVsfsmlLlFTv2yxV5gZBQRWQBfGWKqlnJf6F6lfIxNyy+c TwRoGs62e1AkUNCsqfFlkmIDbB8wkpMQ3cCvDObCSLPBZKxWi7D/pC7Kvc/h6LLIcmEfMTUR/h6 oElRX/3taMN/TjTPawPmJXyh1YQY0L2znCg7MqyoswGZDptaiFn2dOJ/qYnpwrgnuPclYU3JPDv AhqysgICSitbIuVfJCGuDoWyQVztrjru9t+IRc0erV X-Google-Smtp-Source: AGHT+IFH2xsCT3CG7f7DYogW77RlqSSEClZMKIQyo43ry1EWvc24Y940VBQIwPn8aGgvf8ovWZ9/lA== X-Received: by 2002:a17:90b:57f0:b0:343:6a63:85d5 with SMTP id 98e67ed59e1d1-34f68c2799emr12946073a91.16.1768133979341; Sun, 11 Jan 2026 04:19:39 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34f6b07029fsm4877954a91.2.2026.01.11.04.19.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Jan 2026 04:19:38 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v4 1/6] mm: khugepaged: add trace_mm_khugepaged_scan event Date: Sun, 11 Jan 2026 20:19:04 +0800 Message-ID: <20260111121909.8410-2-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260111121909.8410-1-yanglincheng@kylinos.cn> References: <20260111121909.8410-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add mm_khugepaged_scan event to track the total time for full scan and the total number of pages scanned of khugepaged. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Barry Song Reviewed-by: Lance Yang --- include/trace/events/huge_memory.h | 24 ++++++++++++++++++++++++ mm/khugepaged.c | 2 ++ 2 files changed, 26 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 4e41bff31888..3d1069c3f0c5 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -237,5 +237,29 @@ TRACE_EVENT(mm_khugepaged_collapse_file, __print_symbolic(__entry->result, SCAN_STATUS)) ); =20 +TRACE_EVENT(mm_khugepaged_scan, + + TP_PROTO(struct mm_struct *mm, int progress, bool full_scan_finished), + + TP_ARGS(mm, progress, full_scan_finished), + + TP_STRUCT__entry( + __field(struct mm_struct *, mm) + __field(int, progress) + __field(bool, full_scan_finished) + ), + + TP_fast_assign( + __entry->mm =3D mm; + __entry->progress =3D progress; + __entry->full_scan_finished =3D full_scan_finished; + ), + + TP_printk("mm=3D%p, progress=3D%d, full_scan_finished=3D%d", + __entry->mm, + __entry->progress, + __entry->full_scan_finished) +); + #endif /* __HUGE_MEMORY_H */ #include diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 9f790ec34400..2e570f83778c 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2545,6 +2545,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result collect_mm_slot(slot); } =20 + trace_mm_khugepaged_scan(mm, progress, khugepaged_scan.mm_slot =3D=3D NUL= L); + return progress; } =20 --=20 2.51.0 From nobody Mon Feb 9 11:07:27 2026 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C98953314A4 for ; Sun, 11 Jan 2026 12:19:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768133989; cv=none; b=MJtQQh8WHZlfKr4ZfCuactFiY0cRD8I/CXOT06D6FfxTRVi+6FFmJbRxwzyV/SJCPmEJhOiJd+JoimEL2COPkG/4XJR8ZvjxyHhZKs5tIWPwU69txBRZ9e8vh0mrqof+mZmOWLOJomIJjUP20qz0/icVpkcGXNRYY4lKNNbJbJw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768133989; c=relaxed/simple; bh=ATYelhh3sGOVmusl0h84GHhs4HP/IrSSeIe8nFQ99bg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XLZ4tRLiZvWuuyzvVSLE60AVCv8PlVNPfkCPcRr/IjcsEiHqvsd0+Gdluf7Mfp/42jPhcajqG874a5OzJh5ywedAQaModBQ/rhcFH6gsJIUfSEXgUQ4BWTrdkJr+CpSGzFUs0juBe4HswMjIDSBl3HSZnJqtAY4yp+sdeteplmI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=e4GbwDis; arc=none smtp.client-ip=209.85.216.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="e4GbwDis" Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-34c1d98ba11so4407771a91.3 for ; Sun, 11 Jan 2026 04:19:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768133987; x=1768738787; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TXcEVyCz2mdgQ9fw8ki3ifTfwIPNt8/MvwZFlR641cE=; b=e4GbwDisEjAuNAE1xUSG36Or7EEh9xk8HwxdsmKrRfWmfqfnsGNs593jmm3Lxu1RXY MktmQ401nGIEUJ3uSkhO4NhemcsnsqMRgImUIox3/iDuWi4xVNqzLwEN+kAEjMnLaFkc uAK0UecM5b5CpYK6lkF0t6wQUo0qqlZoKDLnzcaOjMQfBdq7sL57THqQd6LN1QJfqdLY 84kx0bL6dWQEzIee3TLreacRtyhwowTkOqCsoy4aJM6ajiMxM3ffov2inQHih7ybiqYz pM1oisGVeR3kOadEYV2vUWj1fSNfIII6J/9mwIhdpRs/C9tmoqyQAnqRBaAUjrQViS50 qbuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768133987; x=1768738787; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=TXcEVyCz2mdgQ9fw8ki3ifTfwIPNt8/MvwZFlR641cE=; b=bi1WIW8ymx7CkfS/wWhXRBu+3k7dKPLE2x73lrslGk8sIxcF4OTkBad+PLJQXdqTqL FiTHQvVtfwxgE9URJwh/IZRgIjG0uRiPrhTiP3sHPBxyZf4xm5uDi97srBwhoric4TV5 h89L5yTJJIK+N5R5GcqaGoaayZ1hNu6f1ZVJhnqiZdWbzDu0zo3PPWGt9d6W6xx/VDCN iTSxl3IamWr1YK9yeroVfxKThhySmz5R1HJ2DzcVkyeQNkPBuqhWUh9sDzzEcxjQqYSq sOiiT3pIef9iyDVBQceL5QqPCtiwSZ3Qtw3uCr9AZy8XSezktVrsRFRWMd+rXOTRASZK PYeg== X-Forwarded-Encrypted: i=1; AJvYcCXpgHH0rLBXoi/6WF5km/rEj9Y3EVy73rjW9zSs0OoDSOJZYMgeP2QTGipdBKNB66UlPHdMaKMyAvhcZdQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzFV+654F9LJAoIrFDPTn6rWHeajwshz+aHpuaE0JJjnQe2lqsD 4p9VbiFX+tgicNaZih03iss2sbpiiUVpMY7MJwJrZCAUdYgOixmK53th X-Gm-Gg: AY/fxX6nhTjTXzfDcql7NSS5GKYKSoGfxSzwFGj6sDF1IoATjVbRQHeaVqrVcUzJxcd qNtmxH8WWnw/aaqhZ+wkgfHbk93Yw/bL0b3RnHzdavFCe47X/+8z53uypnLEts4DVWkkuLk9fQR GHR1XU6IhYANY2f/SuU/bnsFJG+PjuInPjzYxu611t5i8KmhHUzopyko+5a+kxAV2zJn3TWx40c NRIogmMl/hjIX3Syd7QbQZOrI/9Ms4t1Upn78xwzuAxiUR+3IQ3QOfp59Ocm8++nE78DyEd1FIt IOkNxY/iakcQMFtssXwgM3ekplX4P0i0kx+25HErIkLbpwoeiJcYIZWZJxKnS+OCQboX1tcBr9D ojoS6rWIexwK+luCGoGesa4UOsSZhW5AMtoN/N77t13ibn3npvDmFzikf4rOt9jtfwLRGrQfZee tVUF/tB/SlrKZqsMKVimxqMB5QnBOlyg== X-Google-Smtp-Source: AGHT+IGPJhW2ZDBzHgiDmQNFPVyqEA2J/ZIwwfVz+XH/ksrWs4Qx3T634999igMQa6zqSxteeK/3Ug== X-Received: by 2002:a17:90b:2711:b0:349:8184:af8b with SMTP id 98e67ed59e1d1-34f68c7a652mr15922912a91.7.1768133987030; Sun, 11 Jan 2026 04:19:47 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34f6b07029fsm4877954a91.2.2026.01.11.04.19.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Jan 2026 04:19:46 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v4 2/6] mm: khugepaged: refine scan progress number Date: Sun, 11 Jan 2026 20:19:05 +0800 Message-ID: <20260111121909.8410-3-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260111121909.8410-1-yanglincheng@kylinos.cn> References: <20260111121909.8410-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, each scan always increases "progress" by HPAGE_PMD_NR, even if only scanning a single pte. This patch does not change the original semantics of "progress", it simply uses the exact number of PTEs counted to replace HPAGE_PMD_NR. Let me provide a detailed example: static int hpage_collapse_scan_pmd() { for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { pte_t pteval =3D ptep_get(_pte); ... if (pte_uffd_wp(pteval)) { <-- first scan hit result =3D SCAN_PTE_UFFD_WP; goto out_unmap; } } } During the first scan, if pte_uffd_wp(pteval) is true, the loop exits directly. In practice, only one PTE is scanned before termination. Here, "progress +=3D 1" reflects the actual number of PTEs scanned, but previously "progress +=3D HPAGE_PMD_NR" always. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) --- mm/khugepaged.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 2e570f83778c..5c6015ac7b5e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1249,6 +1249,7 @@ static enum scan_result collapse_huge_page(struct mm_= struct *mm, unsigned long a static enum scan_result hpage_collapse_scan_pmd(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long start_addr, bool *mmap_locked, + int *cur_progress, struct collapse_control *cc) { pmd_t *pmd; @@ -1264,19 +1265,27 @@ static enum scan_result hpage_collapse_scan_pmd(str= uct mm_struct *mm, VM_BUG_ON(start_addr & ~HPAGE_PMD_MASK); =20 result =3D find_pmd_or_thp_or_none(mm, start_addr, &pmd); - if (result !=3D SCAN_SUCCEED) + if (result !=3D SCAN_SUCCEED) { + if (cur_progress) + *cur_progress =3D HPAGE_PMD_NR; goto out; + } =20 memset(cc->node_load, 0, sizeof(cc->node_load)); nodes_clear(cc->alloc_nmask); pte =3D pte_offset_map_lock(mm, pmd, start_addr, &ptl); if (!pte) { + if (cur_progress) + *cur_progress =3D HPAGE_PMD_NR; result =3D SCAN_NO_PTE_TABLE; goto out; } =20 for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { + if (cur_progress) + *cur_progress +=3D 1; + pte_t pteval =3D ptep_get(_pte); if (pte_none_or_zero(pteval)) { ++none_or_zero; @@ -2297,6 +2306,7 @@ static enum scan_result collapse_file(struct mm_struc= t *mm, unsigned long addr, =20 static enum scan_result hpage_collapse_scan_file(struct mm_struct *mm, uns= igned long addr, struct file *file, pgoff_t start, + int *cur_progress, struct collapse_control *cc) { struct folio *folio =3D NULL; @@ -2337,6 +2347,9 @@ static enum scan_result hpage_collapse_scan_file(stru= ct mm_struct *mm, unsigned continue; } =20 + if (cur_progress) + *cur_progress +=3D folio_nr_pages(folio); + if (folio_order(folio) =3D=3D HPAGE_PMD_ORDER && folio->index =3D=3D start) { /* Maybe PMD-mapped */ @@ -2466,6 +2479,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result =20 while (khugepaged_scan.address < hend) { bool mmap_locked =3D true; + int cur_progress =3D 0; =20 cond_resched(); if (unlikely(hpage_collapse_test_exit_or_disable(mm))) @@ -2482,7 +2496,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result mmap_read_unlock(mm); mmap_locked =3D false; *result =3D hpage_collapse_scan_file(mm, - khugepaged_scan.address, file, pgoff, cc); + khugepaged_scan.address, file, pgoff, + &cur_progress, cc); fput(file); if (*result =3D=3D SCAN_PTE_MAPPED_HUGEPAGE) { mmap_read_lock(mm); @@ -2496,7 +2511,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result } } else { *result =3D hpage_collapse_scan_pmd(mm, vma, - khugepaged_scan.address, &mmap_locked, cc); + khugepaged_scan.address, &mmap_locked, + &cur_progress, cc); } =20 if (*result =3D=3D SCAN_SUCCEED) @@ -2504,7 +2520,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result =20 /* move to next address */ khugepaged_scan.address +=3D HPAGE_PMD_SIZE; - progress +=3D HPAGE_PMD_NR; + progress +=3D cur_progress; if (!mmap_locked) /* * We released mmap_lock so break loop. Note @@ -2826,11 +2842,11 @@ int madvise_collapse(struct vm_area_struct *vma, un= signed long start, mmap_read_unlock(mm); mmap_locked =3D false; result =3D hpage_collapse_scan_file(mm, addr, file, pgoff, - cc); + NULL, cc); fput(file); } else { result =3D hpage_collapse_scan_pmd(mm, vma, addr, - &mmap_locked, cc); + &mmap_locked, NULL, cc); } if (!mmap_locked) *lock_dropped =3D true; --=20 2.51.0 From nobody Mon Feb 9 11:07:27 2026 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC1C9330670 for ; Sun, 11 Jan 2026 12:19:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768133993; cv=none; b=d5w3/Mi9TdWjAaKgQW5CKrsvprz39RPAF6loNN3L9m9DLMdaLQX/Ua7qhZNpe1WmPlovU2I/SF1qHHHUzHYsmHPt1FZ1Q4pEw2iZl/lyfj6UffN/Bnq9T2S7ncf9F4lMC8d6dowP7T/E58LJivobxWdT3AoIzQJteZvKkUhWwf0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768133993; c=relaxed/simple; bh=BwOmIVHUpQGCoutNoM03UUVYggDibsP0l5ejwo0bs+0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jFd0Bfb+Z2AN80qDuPzcLA1jAwnYRh3GPbdRc8IvRNkoUjs1mplTHyMfomQc/U8MVwHPMQ62a/4AoI5jz7ApU2eon/kJJ6trcFVG2ZMu4Dk4OjT/tLT/iE0zALGMDYIYiEUHS3D6mOfDe3z1IHHUq+CPOxZjWUxytlY43E4vW2g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YxlvtFfD; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YxlvtFfD" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2a0fe77d141so42051565ad.1 for ; Sun, 11 Jan 2026 04:19:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768133991; x=1768738791; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zzjqHFHsTu/NzGPB+PE+U2CGfVjIazwd87Qrshcmnl0=; b=YxlvtFfDjXDnfFVjseBKrzhUSbc/BGFyWnkNwC0a41/hPHdg8hJKROfC9LbAlWITq8 1XdD9mA8Sqi/piUCtGXYyKWhYNIB8/WRV+d8hoP/7iN9isFkfXQdwfjrt0EDSJre8uUh Wo7p0/7Dp91hjiIOKwxkoLbqyfCLUUgKfMcTKoloPJArjZxsqbJa+LY/IJt2HsepJDnx Aq2DHNzNDal5DCSn0wGpdaSkPuAjJUYXVdpSJn/nlPDpNCJ3XLB9CherQe7itSRCbgP4 kLItmcJDvjCMzaykAjg3yvnlvN9rundEFV5O+331vZoLfYrVCDt8mR3E9nyYllTZ6JvQ R3iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768133991; x=1768738791; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=zzjqHFHsTu/NzGPB+PE+U2CGfVjIazwd87Qrshcmnl0=; b=egCZywv+HTXicW0Lnkr0eFOLve80HxkzWGtCrDHb+Q1Nq5EaWk+57By0/ZYCih+K8Y Zf1O3pZWFEsjXtgWs3cXxfhe64pPZsCX0J24WVziW5LngXoPLhw4V1qfZ4QYVZLPW/qU WBUn3i3cZTdSyTj3tuFdE/0Tr6hcJYFNfHp0+bZkl78I9AXXXBOhwYJU/dkJVk0L/GtF wRo/5OwKMfSLxX4ljO4u201G/sbTx3k39LXHyrVyCKJAzPegb9qhbiXMISHB2mQjsns6 H3/2gYUTs7LCIectZDZaUsDbzEuVTykL86Iby8sGHx44AhswNs5Aui7ZHwonXlIJZUx4 wzZg== X-Forwarded-Encrypted: i=1; AJvYcCUjVFPYvOy38q0PTomnQ1p0R9GhJjxnstzg+gqP4qE/nqGwtL9WL9xtPqcFWhsefaYNIp7XnDHIYJZcoK4=@vger.kernel.org X-Gm-Message-State: AOJu0Yw9nYax40nx3Kz5xb399Q4YNlDAjLH22SZKacMNLvqjL3hJZ3EC C59Usn5i8gLop/KW9oKlrY37Lo3eh+8EHyAAdWb9qxHhWiZW9SbksKys X-Gm-Gg: AY/fxX6wE4l8C/fmnnnRNLk/YJEY1i+sOyA3kT/0xBj/CY2qTCy/tNCRNUoDylHQgMH jW2J1WTpFz22j7LDk829GBmlr/MCVSvFLuj/nCNUktkLBHrF3RcRVuEflytU5ZmbybUi/W4q2nZ Go4z/qY0ygcTSKyRuQCQxI6Qrq9m+ViH8vsoLJxuq+UO99XHY4ADw13bdECTMyQRO0hMjIVQ0+a TD1ZhEchkjnNf//3t8e2PNbRBXEnAHL/CQ8MvdlEPOOb4t0vz2NKVogvDrPBDMChtCXfSRuUGrx l3Riw4wBwvFvG0doSNx6XmVOlXyuGuWloVJrLHRs5v5IshwBUzF1uJjJMKkSFjXfmO5ZuU1g3wx ZzyrzTFyUcdNCBr75dP1ub103FPvy2kVXfflRJQFS9HbkBfdT6qJDYYrhoniS/il2fvUGIGnMmH eh1R1qhrxx9cB2tjbbLpY6GHiGz12+qA== X-Google-Smtp-Source: AGHT+IEoqWK3ugDB7/VD7wzPQM66g18mZY1tdzLq1qScr45zyMjWbfKXpi/m32xd7/USrBdm6hou+g== X-Received: by 2002:a17:902:c402:b0:2a0:97d2:a265 with SMTP id d9443c01a7336-2a3ee452277mr138508085ad.14.1768133991082; Sun, 11 Jan 2026 04:19:51 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34f6b07029fsm4877954a91.2.2026.01.11.04.19.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Jan 2026 04:19:50 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v4 3/6] mm: khugepaged: just skip when the memory has been collapsed Date: Sun, 11 Jan 2026 20:19:06 +0800 Message-ID: <20260111121909.8410-4-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260111121909.8410-1-yanglincheng@kylinos.cn> References: <20260111121909.8410-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The following data is traced by bpftrace on a desktop system. After the system has been left idle for 10 minutes upon booting, a lot of SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE are observed during a full scan by khugepaged. @scan_pmd_status[1]: 1 ## SCAN_SUCCEED @scan_pmd_status[6]: 2 ## SCAN_EXCEED_SHARED_PTE @scan_pmd_status[3]: 142 ## SCAN_PMD_MAPPED @scan_pmd_status[2]: 178 ## SCAN_NO_PTE_TABLE total progress size: 674 MB Total time : 419 seconds ## include khugepaged_scan_sleep_millisecs The khugepaged_scan list save all task that support collapse into hugepage, as long as the task is not destroyed, khugepaged will not remove it from the khugepaged_scan list. This exist a phenomenon where task has already collapsed all memory regions into hugepage, but khugepaged continues to scan it, which wastes CPU time and invalid, and due to khugepaged_scan_sleep_millisecs (default 10s) causes a long wait for scanning a large number of invalid task, so scanning really valid task is later. After applying this patch, when the memory is either SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE, just skip it, as follow: @scan_pmd_status[6]: 2 @scan_pmd_status[3]: 147 @scan_pmd_status[2]: 173 total progress size: 45 MB Total time : 20 seconds Signed-off-by: Vernon Yang --- mm/khugepaged.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 5c6015ac7b5e..6df2857d94c6 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -68,7 +68,10 @@ enum scan_result { static struct task_struct *khugepaged_thread __read_mostly; static DEFINE_MUTEX(khugepaged_mutex); =20 -/* default scan 8*HPAGE_PMD_NR ptes (or vmas) every 10 second */ +/* + * default scan 8*HPAGE_PMD_NR ptes, pmd_mapped, no_pte_table or vmas + * every 10 second. + */ static unsigned int khugepaged_pages_to_scan __read_mostly; static unsigned int khugepaged_pages_collapsed; static unsigned int khugepaged_full_scans; @@ -1267,7 +1270,7 @@ static enum scan_result hpage_collapse_scan_pmd(struc= t mm_struct *mm, result =3D find_pmd_or_thp_or_none(mm, start_addr, &pmd); if (result !=3D SCAN_SUCCEED) { if (cur_progress) - *cur_progress =3D HPAGE_PMD_NR; + *cur_progress =3D 1; goto out; } =20 @@ -1276,7 +1279,7 @@ static enum scan_result hpage_collapse_scan_pmd(struc= t mm_struct *mm, pte =3D pte_offset_map_lock(mm, pmd, start_addr, &ptl); if (!pte) { if (cur_progress) - *cur_progress =3D HPAGE_PMD_NR; + *cur_progress =3D 1; result =3D SCAN_NO_PTE_TABLE; goto out; } @@ -2347,9 +2350,6 @@ static enum scan_result hpage_collapse_scan_file(stru= ct mm_struct *mm, unsigned continue; } =20 - if (cur_progress) - *cur_progress +=3D folio_nr_pages(folio); - if (folio_order(folio) =3D=3D HPAGE_PMD_ORDER && folio->index =3D=3D start) { /* Maybe PMD-mapped */ @@ -2361,9 +2361,14 @@ static enum scan_result hpage_collapse_scan_file(str= uct mm_struct *mm, unsigned * returning. */ folio_put(folio); + if (cur_progress) + *cur_progress +=3D 1; break; } =20 + if (cur_progress) + *cur_progress +=3D folio_nr_pages(folio); + node =3D folio_nid(folio); if (hpage_collapse_scan_abort(node, cc)) { result =3D SCAN_SCAN_ABORT; --=20 2.51.0 From nobody Mon Feb 9 11:07:27 2026 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 437E12C0F7F for ; Sun, 11 Jan 2026 12:19:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768133999; cv=none; b=kynWkDrG7s8RbdxHHEVJOX3HUq77o3hk9/5UkUW8W8IavprW6NP02eS7TQk2LZOpJP+taKa80rH3NE0A0nVGZseoyJfdfLZsI/fJ5/sBLfBcmKpXIPrcbODEJnQbx0UHJq7MZlblJKztpOL+lpcreAJ2hCCwjXqOhJUGhAoFXA0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768133999; c=relaxed/simple; bh=0sKw2WlJhHW9MizmMQjnT6BFPyREgPjFwExJQG8I1+M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mVNDATuL4Oe95V0pZpUF3x4X7SJvu8UqK8zo4pk29rqaYJfTVGfdeFFXqrSeB6a1ZsOydsedPF825y22odyP7HDOtnNWnA7gQ8vTTBS52RmYl32V8Ay7kqayDhlfVLS+BZXAvqw0SrGZ0DBmDj5bL1Fp9MW7RDFGOO8VxLFGqaw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HICPGM1C; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HICPGM1C" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-2a0d67f1877so43820415ad.2 for ; Sun, 11 Jan 2026 04:19:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768133995; x=1768738795; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EIuAQwNEq1ZTksmpQsTcj8MLTiZO5nZ88SkZX9qosx4=; b=HICPGM1CYKD/J51+nKkT2ePuHOsEM51k+SRirPZuZdmo24hC/lT6oA5JIoLBQQEZ6B 3iFwSegXtIb57K/NZc7mm4xoEskcS4hbisYPF7I098RZYkuqR2GurkghZBfLY7cYbz7P PoO8FW2F8SyZ95BO0LHEvUIswSut28hGMAdeRS9EVTAALU/NlwAkxzPv6/HrA7Gc52EP Orwa147hqgGne8l+JjeB6JU61qguhhhbe/PzZow3UcrR9Eo4Vf0op8/9i3B4+bWsrTvC 7QktEmfkJzYf2M5G4FOlcFiPC/xNcApR/n/n0yRG5SYRdi8LBvfIbI/XJxw5RXxBB7sm pOXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768133995; x=1768738795; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=EIuAQwNEq1ZTksmpQsTcj8MLTiZO5nZ88SkZX9qosx4=; b=AgTAG07jxLnj5r+5NC1SnEV0xdnC1zzE+AwqbXs+aBztUq1Yklp42hWYWdu317vhss QKKoecw0EQv5UAgUAyr3dCSv9me0r3WvjcyeOkb9v7MRC+fKk9U12V8zSLwFtz7PyC9s OexoyFYFIyIpFLbs8jSjnQFywV+6adxYZwwtIjhCgqO+GjDjl+9yu5GrL6GbWBJC8e3W 9DKKIJj4DBH9LOjL4mAv0lzhgTMLG/osKtZ53DTBVyF3FW+DsI7neh/VtwyqWbzKggDW icAPjCHJeOdte++3v2efVK8ftRReQFOZWJXHmxP3YTNA0seZDs7xDtY/RoX4nSrDIYa/ B9yw== X-Forwarded-Encrypted: i=1; AJvYcCW9MGvKkTeWS0dGNDQ422HQm/EMtZCVgmgkevIYsyFNFi/1SG1WZIE9GYed5HPszA9VqWlHYU5aVxD35HE=@vger.kernel.org X-Gm-Message-State: AOJu0YwwY2mzLd9AcYoDc3Vc4WW7QSrVz+bCw9NIUpw+ksmEgwFJWJOO Bcw5N/31KJMUFrrj1Pl+gWDv4E/td6dTSpY/Mg5E79A8MZbEPPqIaLmPK2gO2fEIHZA= X-Gm-Gg: AY/fxX4pjpgF8/Y99KC+Q6U4LW7R2gl8+o3bgwspW/hVJmI6AeGA1oZEdGbERv+ITrN kfb09Z6XnFFZ197nyorT73uq2ac7zxq6geyDn+SdoAZRD8IEnADwPrLjQdY5KJngJsEciSLyRML EXkogFT7gjV2VDJORaW+JrzB39L2GA25Itn7a3ZtEMHIrbZR2UH+BBiBnO4fJ8YJPq4j4fguLAb 4QvbHZ4jXrVnFA5KU2zktXDOsUL94AZNaX/7c5vSTKTzTS7yN0NbxsWmfSvkkkGYU6VsvDFqLDL MgCaVbZObfopChMaPo3+4eS7I7ZOVJPv5ZRJiRWVTV2h/w/TXXpF3Yxzh7y8SF+fz5UNEe7eufs 8gpTYzblbHtFlDcSDd16ANxnrZpCaYRUK8m55t/3BgDjCc/ujkKc9d9dXtF50JlPP2WcTY80C+a 6jzNpVLVgQAx0jGRHeXUwSVrva4zGshA== X-Google-Smtp-Source: AGHT+IG7hx5em0fMnXH4KgBvC6u6j9pXcTSYDThlQSxU918X6KNR+BDgXGkT0wU8NhOuk8XcsmBDlw== X-Received: by 2002:a17:902:fc43:b0:2a0:ccdb:218d with SMTP id d9443c01a7336-2a3ee442824mr159043145ad.17.1768133995527; Sun, 11 Jan 2026 04:19:55 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34f6b07029fsm4877954a91.2.2026.01.11.04.19.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Jan 2026 04:19:55 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v4 4/6] mm: add folio_is_lazyfree helper Date: Sun, 11 Jan 2026 20:19:07 +0800 Message-ID: <20260111121909.8410-5-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260111121909.8410-1-yanglincheng@kylinos.cn> References: <20260111121909.8410-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add folio_is_lazyfree() function to identify lazy-free folios to improve code readability. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Lance Yang --- include/linux/mm_inline.h | 5 +++++ mm/rmap.c | 2 +- mm/vmscan.c | 5 ++--- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index fa2d6ba811b5..65a4ae52d915 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -35,6 +35,11 @@ static inline int page_is_file_lru(struct page *page) return folio_is_file_lru(page_folio(page)); } =20 +static inline int folio_is_lazyfree(const struct folio *folio) +{ + return folio_test_anon(folio) && !folio_test_swapbacked(folio); +} + static __always_inline void __update_lru_size(struct lruvec *lruvec, enum lru_list lru, enum zone_type zid, long nr_pages) diff --git a/mm/rmap.c b/mm/rmap.c index 336b27e00238..fd335b171ea7 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -2042,7 +2042,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, } =20 if (!pvmw.pte) { - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_is_lazyfree(folio)) { if (unmap_huge_pmd_locked(vma, pvmw.address, pvmw.pmd, folio)) goto walk_done; /* diff --git a/mm/vmscan.c b/mm/vmscan.c index 81828fa625ed..ad3516ff1381 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -964,8 +964,7 @@ static void folio_check_dirty_writeback(struct folio *f= olio, * They could be mistakenly treated as file lru. So further anon * test is needed. */ - if (!folio_is_file_lru(folio) || - (folio_test_anon(folio) && !folio_test_swapbacked(folio))) { + if (!folio_is_file_lru(folio) || folio_is_lazyfree(folio)) { *dirty =3D false; *writeback =3D false; return; @@ -1506,7 +1505,7 @@ static unsigned int shrink_folio_list(struct list_hea= d *folio_list, } } =20 - if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) { + if (folio_is_lazyfree(folio)) { /* follow __remove_mapping for reference */ if (!folio_ref_freeze(folio, 1)) goto keep_locked; --=20 2.51.0 From nobody Mon Feb 9 11:07:27 2026 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 66F67322B6C for ; Sun, 11 Jan 2026 12:20:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768134007; cv=none; b=EcoNU3BGcYH2+w67Tn/KOHW0gjaW2OVRjCSMB2h9BVYJk3WfRscwdQdtN8S+igPGv7hJtaXYg3WDgU6Bhwy9qhuqF7ekeKYNZ9g4l2AaWxFfeJQjo2kCt2MV8qrC2mXrN2uvcBFiw5F/T00+x+rHXNClj4nyrfnQxRSGU6zbYNw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768134007; c=relaxed/simple; bh=Dd6a77pU7u2o1FsMS5TQAH3aWnT4zay+bnWq2AxeVXE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Tja+rdi6okxC9aLb3txmN5j9EjyXlpNUtmr95khOGKqEs02PWlHvZlofSFS5S36L3x4VrFPiY7q9kPgDtqidtDUGtvsjgqvhEK3ZOtW9XCjzRJnJNKjP9TxwV0MVvimMt6s34FyUEiH9QaKS585KWAqFZJT9ZyQHBJq6Q0X1wyw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=No9fC9fl; arc=none smtp.client-ip=209.85.216.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="No9fC9fl" Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-34be2be4b7cso2821599a91.3 for ; Sun, 11 Jan 2026 04:20:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768134006; x=1768738806; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0G/uIiDDNOHZeA/w76cf8T95IZ/86Sc506HuzqiPhO4=; b=No9fC9flmcz05WsYR2XnlEwCEbxlWCxxvi/L/i3G11yxNvOl9jQ/wlec3ZTewqK4jq PscurEa+5hVuj+iwyvs/YV1JRih/Qmb0x1b2mFGKraMyG3PG4J8s1mAkaD2siAEc8sdz GiG6+6SlUK9OZykIvBJE2zBSnHSZNdwMmWm74x3eUJIEZa33mFe+RzfYcm8RaXF6LVGu pXIVizjMVjjBBdyn7rhkZG1bF1eDI6sp1xZOpTtj3kloCqdf1HJtsHqR6B+PiFdTHqRX YYxcfzl4EuFiY1Va5UvTgUl7e74hGTf5fxLX08C3C35rEOIZXU5AuWbqMF9cxmavXZRu NUDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768134006; x=1768738806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=0G/uIiDDNOHZeA/w76cf8T95IZ/86Sc506HuzqiPhO4=; b=FGNxb+maYReVxqjuCjydRFmA8b5rJ/88PyPaZzkcRAzsafEr/IVsdwrk+F2YZsIM2i ss2vG4yCDZBAoYYABEI81mPPMMjJwiULAZqYdJ9XgJI1cDrvyUqX+5/zteIjXrgY3JBz z+kRf7SWen1sFCOPePL5n2rAyXWkRkl6BzYT6rl6/nwNa4sOBjKpvdCVtw8wHMGx7xEM uIjfIDwautbeyc1cYncDQIYpF4FzWLneQoc5LYinhXSGxSFjur1Y7IDX14xGpqL5VTsd DI8tuQPA6riva1+u/wFf3au7HwNQaQG0JMv6qfsybtOl8xABmwIDfSodCOoWZSDZEeYt 58Ew== X-Forwarded-Encrypted: i=1; AJvYcCX/zxAMj1eNsgw2mYzbBli0kOSGK7EC9ywrRlLx0kOAsvhcB1TH9CmMCh91shaLxFFag7Jw2EspxsIYCVI=@vger.kernel.org X-Gm-Message-State: AOJu0YwaLCVSjh2iCwIZOld1SDqSj8Vek5g0LwsticA1P0TlCc0AV0Hn 9LdZCicNJ9yAY6bHL4J4Dr8C9g4N1dA3YqMM30EK/dUkgeZwK4po3Hmc X-Gm-Gg: AY/fxX4PFF1mxLtwsGlN+/5MJEHf22ik8droqSV0POHQsFOuPcYJmKkx/+Tc7OSXPhF qLytnvqlH/XVrT9p7y4+CzGUWteGSPC5JrY1aVwNzF2CR8x99aMW3Da0giNOhnMvjX6I1MKs+v/ k3QRD6SQ+kNVCU87fjmCyPMPT32I1LsNz6Gtb3ReZGxS0qI738lMPVdOUJTklssuEBiL/6cl8UM ykRtB7N4Y6TYxvqJp8WkVPBsUrEr1D2ldfLAlHIk9mJnxmfREr5R1BGEm17lJtcd0jBjs2lfcdD 8HSkxwPPC2DUR0Hwo5dy6cxNUGwnsjgYlHerez+vzRhUrW7/xPOBAB4eoN5JJBBKU7CyIBtnA7+ u69yfXNnc8EyZS+JdtH41avbD+nCeWMJidqqNjCg/8SUd3MHBsBODavJgE7oipZ3IkxzjX2/PF2 8upmNijEhcRFOfCfACSPMg9YGDeWJPEw== X-Google-Smtp-Source: AGHT+IFS7T+REkqaxeUsDPamccnMWWVG2Bbbk1hI/lGfgiUMKIziKTqaxzze5hdv2cumkVYboUFq3g== X-Received: by 2002:a17:90b:4b88:b0:340:6f07:fefa with SMTP id 98e67ed59e1d1-34f68c03251mr14942182a91.20.1768134005658; Sun, 11 Jan 2026 04:20:05 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34f6b07029fsm4877954a91.2.2026.01.11.04.19.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Jan 2026 04:20:04 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v4 5/6] mm: khugepaged: skip lazy-free folios at scanning Date: Sun, 11 Jan 2026 20:19:08 +0800 Message-ID: <20260111121909.8410-6-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260111121909.8410-1-yanglincheng@kylinos.cn> References: <20260111121909.8410-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For example, create three task: hot1 -> cold -> hot2. After all three task are created, each allocate memory 128MB. the hot1/hot2 task continuously access 128 MB memory, while the cold task only accesses its memory briefly andthen call madvise(MADV_FREE). However, khugepaged still prioritizes scanning the cold task and only scans the hot2 task after completing the scan of the cold task. So if the user has explicitly informed us via MADV_FREE that this memory will be freed, it is appropriate for khugepaged to skip it only, thereby avoiding unnecessary scan and collapse operations to reducing CPU wastage. Here are the performance test results: (Throughput bigger is better, other smaller is better) Testing on x86_64 machine: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.14 sec | 2.93 sec | -6.69% | | cycles per access | 4.96 | 2.21 | -55.44% | | Throughput | 104.38 M/sec | 111.89 M/sec | +7.19% | | dTLB-load-misses | 284814532 | 69597236 | -75.56% | Testing on qemu-system-x86_64 -enable-kvm: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.35 sec | 2.96 sec | -11.64% | | cycles per access | 7.29 | 2.07 | -71.60% | | Throughput | 97.67 M/sec | 110.77 M/sec | +13.41% | | dTLB-load-misses | 241600871 | 3216108 | -98.67% | Signed-off-by: Vernon Yang --- include/trace/events/huge_memory.h | 1 + mm/khugepaged.c | 17 +++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 3d1069c3f0c5..e3856f8ab9eb 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -25,6 +25,7 @@ EM( SCAN_PAGE_LRU, "page_not_in_lru") \ EM( SCAN_PAGE_LOCK, "page_locked") \ EM( SCAN_PAGE_ANON, "page_not_anon") \ + EM( SCAN_PAGE_LAZYFREE, "page_lazyfree") \ EM( SCAN_PAGE_COMPOUND, "page_compound") \ EM( SCAN_ANY_PROCESS, "no_process_for_page") \ EM( SCAN_VMA_NULL, "vma_null") \ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 6df2857d94c6..8a7008760566 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -46,6 +46,7 @@ enum scan_result { SCAN_PAGE_LRU, SCAN_PAGE_LOCK, SCAN_PAGE_ANON, + SCAN_PAGE_LAZYFREE, SCAN_PAGE_COMPOUND, SCAN_ANY_PROCESS, SCAN_VMA_NULL, @@ -1258,6 +1259,7 @@ static enum scan_result hpage_collapse_scan_pmd(struc= t mm_struct *mm, pmd_t *pmd; pte_t *pte, *_pte; int none_or_zero =3D 0, shared =3D 0, referenced =3D 0; + int lazyfree =3D 0; enum scan_result result =3D SCAN_FAIL; struct page *page =3D NULL; struct folio *folio =3D NULL; @@ -1343,6 +1345,21 @@ static enum scan_result hpage_collapse_scan_pmd(stru= ct mm_struct *mm, } folio =3D page_folio(page); =20 + if (cc->is_khugepaged && !pte_dirty(pteval) && + folio_is_lazyfree(folio)) { + ++lazyfree; + + /* + * The lazyfree folios are reclaimed and become pte_none. + * Ensure they do not continue to be collapsed when + * skipped ahead. + */ + if ((lazyfree + none_or_zero) > khugepaged_max_ptes_none) { + result =3D SCAN_PAGE_LAZYFREE; + goto out_unmap; + } + } + if (!folio_test_anon(folio)) { result =3D SCAN_PAGE_ANON; goto out_unmap; --=20 2.51.0 From nobody Mon Feb 9 11:07:27 2026 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8ABF1330675 for ; Sun, 11 Jan 2026 12:20:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768134013; cv=none; b=Yhs966fQgpg9s41YrMpU/Wi3MhJW7oqTb0e6nlXWtJ+de4s96v9S1GhZAxRwbrpM5vdQHTuJgaLw5sq84PXLd2tEtMDqrZtCjZk5NzTGbvY1XqYnkW4ie5A/wlgbouTIUgWq5DKdperHHlet9KwnS5Td15on+mmuxSpCwq5I6/k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768134013; c=relaxed/simple; bh=fUt6RAX724hkZ+6GTRB0r6cfhW3X2/VBPyf2ywplpCc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RV1yDtbRDAZjPlCb7VuNWNgYtLoDye6K3cJdF3VxdanH/jvmLkcyPxF/yKXKMqoBS2Fucx4T2N0d24kvGxJ4b/bghSaqvE3CyyNppQZn6Gna2hgyTTbFH8lHzgg1eXLMRXcRQHfMsx5AIgTNKKOq6RwPrOz6BSlNumufo1rp2Dw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iBwL4kn/; arc=none smtp.client-ip=209.85.216.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iBwL4kn/" Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-34c84ec3b6eso5330130a91.3 for ; Sun, 11 Jan 2026 04:20:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768134012; x=1768738812; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WId1gR6GClOcsxtsBz4LovM65cz0Q2hsHXawAMGxngw=; b=iBwL4kn/84EFNZmvhZehin1chvcjamtC1+E2nfvV+UTDkIxTynn5XA6YXjtRYc1jQu XyAO5xEBcpbz2YvwQCzjsYZWlfnwdIaE/K6xLcQRgE5m96i+SiUPJp5R4jdmrv2ENsUN 454WS5O3C1k0Y0Q1vtnWsrPfI3I4RMTGHf7DCk6BLctncMX2I3NNj68PJLpHIubc+CC5 hE+WkUj31S3a7Lqa6T70Gw7vZ1FSnLCJSqOXwFEqsBhkP9fdnvuc94e7efDT63yc3Oox +SN5vyImuVkzIMUBhaa9G/Dfu9NGCDChZi6QUPNlviGBT/MGnjcijxGA5NUd6Yu7HByk taUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768134012; x=1768738812; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=WId1gR6GClOcsxtsBz4LovM65cz0Q2hsHXawAMGxngw=; b=BXCv/HCyJqHAf4pjqbmHx5fsueITJj1+axarJmq43tSJpyOxNj2YkSHQbHvOIBd9Cy 8USCBHdjNBH+keegJsHkxtPLzCreXCVGgGrdLYfsu+B8DAq6IIB5Jd2UPAJz9qYWiZ6S QXPLE8VJ/0VcqaJHb3XSaR/88yZ+C8+S2h3kCRZO5yHzIHMGrKh9w+KzHWV7o2juWk40 p3+w0Ie1OkbXrXBJmjRNLRmVpcmlE5ZXX0swmCfHTYJCPw9ixO/1ykmsJFFjoax4ZUME 0pOXeA3fVyPtPB/OgWX9MivhXtGS24bYY4UeFnE8pINiHGuZDKURVTlDgmfUAIMpdoMo Gd0Q== X-Forwarded-Encrypted: i=1; AJvYcCUSjFjADxrFweUlzV3ilfUBC51jOblIjDHULXlC0nFI8b/NhPDGUAAimoFL143fwA/f+NE4UobK8X8isfk=@vger.kernel.org X-Gm-Message-State: AOJu0YxtDarJe3g1N3QzlKjeO7clOzmZwpplNogEAHW2zC+qRqPUpakq NFxLKqO/1DBinMgnBEICFW31P5sgkvme6SWZpP9RusRbH7cRs6piyajlKzaHovU91Wg= X-Gm-Gg: AY/fxX51ZvWKo/4XrnSns0bMCksZYFY/EycuR0zu00Ub0mgKqlL3LbD8CYt43omxC42 IlLGVIyzFLubiiEZ5bRfCFcDAFNKVo/Kv38YBAjYWOl6wlcjGItufyg5pTtPNJvemZXlRJftMnV G2henxOtHs3DFrchOCkuSY60+hZdUnDaaxy0J2c2qoQEH7Tv9ik7KLbL2p8m2EzSjPSqqAgMNT5 dmA7bzf8XnjGqqfrtT9rzOiQZX76W6wzRtx6Z9Lpx6pvFWiqYeHlqJ9CEjfXDowkVhFapIoRg6z 5sE7y6MFne+38p4r3P0YX2+j4G7Zg9msNaQwfYEgtFD4mtkOI/mpbKWzvvsX1ykai6rduuFoDkG rAareOlCVBRmG/u2PMwyWhIf5aY2NGVB1Sx9x6xCDp9p0kh8WAk98/e31he0DM7DnA0Xm64LMm5 j5zuJWrVUPYP97Gtiopm+X/p/s023bgA== X-Google-Smtp-Source: AGHT+IGZPr3xARA+SSffhvwMckbZFHDMqc3wMCnmhn1Y/ESS4DoaMHMGv640C4e4rdbes7Vs3G1Cdw== X-Received: by 2002:a17:90b:3d08:b0:33b:be31:8194 with SMTP id 98e67ed59e1d1-34f68cea72fmr14573218a91.34.1768134011656; Sun, 11 Jan 2026 04:20:11 -0800 (PST) Received: from localhost.localdomain ([221.227.246.159]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34f6b07029fsm4877954a91.2.2026.01.11.04.20.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Jan 2026 04:20:11 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org Cc: lorenzo.stoakes@oracle.com, ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH mm-new v4 6/6] mm: khugepaged: set to next mm direct when mm has MMF_DISABLE_THP_COMPLETELY Date: Sun, 11 Jan 2026 20:19:09 +0800 Message-ID: <20260111121909.8410-7-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260111121909.8410-1-yanglincheng@kylinos.cn> References: <20260111121909.8410-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When an mm with the MMF_DISABLE_THP_COMPLETELY flag is detected during scanning, directly set khugepaged_scan.mm_slot to the next mm_slot, reduce redundant operation. Without this patch, entering khugepaged_scan_mm_slot() next time, we will set khugepaged_scan.mm_slot to the next mm_slot. With this patch, we will directly set khugepaged_scan.mm_slot to the next mm_slot. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Lance Yang --- mm/khugepaged.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 8a7008760566..4c055d6c2717 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2566,7 +2566,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, enum scan_result * Release the current mm_slot if this mm is about to die, or * if we scanned all vmas of this mm. */ - if (hpage_collapse_test_exit(mm) || !vma) { + if (hpage_collapse_test_exit_or_disable(mm) || !vma) { /* * Make sure that if mm_users is reaching zero while * khugepaged runs here, khugepaged_exit will find --=20 2.51.0