From nobody Mon Feb 9 00:46:10 2026 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C28BE274B26 for ; Mon, 29 Dec 2025 05:52:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766987537; cv=none; b=R0H0LRe/0jWnYjuuPfm7aVeAKUH7m2Du4wHDH/VZvib9eO5zXlvXfxMrKKDLK/EdhnYzYAnjhfGxP8/nTxtzMnNS8rC+Xj3nFNd8iRPFeIcx7SuJMP3IIVBgHM/+2KmocZ9BN7OpE4ics2/LGJO/TbPALkHjk2RgGIXe/0ob5xI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766987537; c=relaxed/simple; bh=PP7Bf5a7La35U+3KgFHMszgzWxuNMsNrbSra3216AMQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AkXZAULu71PNMITAr8sAxPSZsZwUEAxuxLb6CY9/nCYfBQF3iaADvx6nqLHqhmBb9g/5x0oqijUAsPsxmT9s4f5gzRnhVFPssYl5fsDQRfkRd4cV8aAEFxegAA9fvCh5FoIaSIslVFc/zhvwRK5qOOt1++g3i9GsldMW1Mqgjpo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=LxXbrn9M; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LxXbrn9M" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-34c565b888dso9790435a91.0 for ; Sun, 28 Dec 2025 21:52:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1766987534; x=1767592334; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FIJuJHeLF/rAuxCLr8FDuBbtKBKNJj3d8hTlo90/eSY=; b=LxXbrn9Mj9vCRPW3ngKRtJ9I5/0J00LvKjhi1wxnmPN//e/LFEiY6VDElWHCwQPEdw gDtMJKOcUTEWocw/JSFJgnXsK16XBFEQ5sT0un3doKRao+7Cfx70hwjPUqVc/mQq6GMj GBoa+wqh5f6liTA8Sc9e4jxoc5ztodDBggiEkm4O7/sSeEgQBY0t7ybufeyNaDLskkq6 rmesA9mzfr0Bb1hP0pTY73uV+RoLVWEk0DxcjSjI6NCNhUUhHtaLaomP4EtHT8lzrHIY U+nKFrOk36VhUYVNeYB+XBPXrkCAs7+UIn80zpunzWMwdd8q2/jH+/HgmFLWJjaSO4bS yx1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766987534; x=1767592334; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=FIJuJHeLF/rAuxCLr8FDuBbtKBKNJj3d8hTlo90/eSY=; b=rxCz+PgfcfIWGpuWmZ1k9+RkepepDp9k+hk8AVzx6VCa5OWTLkZkFFp+vnVZvI5Rsx 8mcwU1gI0FcaO/0ecwBiHPknV3Yr3TLCEbSOAOMMt/Crt655Z8D5uqXJB2vAr+cAciGH 7/bLn2XlH3ZHN68Qqd/khTzAubc0vxo4sO/a0TFL4/IrIsYncplm3bwpcVkPB7a5dMOT Q1tCyUMnZmih3V87AWeIhX8Y8pHK6ORBiBcobbIBOznfCODuigZgxV5uGP/ngFRhTd07 Oc9+0UwaDzJCSxwnZ+ARor3tU7tbZ5cZgnkdfuI9/eQxfPT1FTBLUjOmBEyduB6OLaie eLRA== X-Forwarded-Encrypted: i=1; AJvYcCUGh3IXG4I5FHbdol5F44u2LnM0KTMPwBncBw3pZj6abNXKPf/xkYXjhi6ff7MvxlF0RJmouwtDLReZx+A=@vger.kernel.org X-Gm-Message-State: AOJu0YxEG07zxiIykJ52c5ca5H4MH1mS1vcM2IJyqc96Mo43LMZZE5U2 K5r2KVXuTpFlXExNWG7XwrS87/8Sh5FuhFM+n53iYVR7Sa/bPwFsbQ7C X-Gm-Gg: AY/fxX6dzXd5bLEqAs8zn5Q3iLorTnvEQrcZeXy6nOJZCbWXYCeBHcStiPw8k7RKwUc PlS1k2ojnDezOzabiUHCFcn59c2fntZmqrhD3FRrzy0TNaYDeKA/20uOo5OCzYhTI9xvhMxjQ+L fOEETqDNAKRU98xCidD1C1MIbkYb7qekk8TBF8ECMAR0w+d+dLFCXlGFGPONCUvWZDxn2kkFaSo 8+2rikphr9JZ6ypeh7PdAY4GC/Zwd6IbcHgVLc4c+nnsLq3Tx8kguc/Tk7HPNQs9Gkb/+znaYHH rpT1Xy4MhlST5yUl7dx81w9mBviBP1Tr3iLjL8yRQwhSrw9R43yG785d7V5bXn8sBJ2/RDieyw9 H1CtSv4VHAmW9cq6yZ6r6Qu0gODtQRsoZyw2T10fhl8QztPBnBLQOd/2UdLOCzfqL1PaLnLdDUt 2ktue4qHe/cEKlT6f168Nu9iO9vRxu X-Google-Smtp-Source: AGHT+IH63XzFCnyuKu0Wgj97o/tbzy2W8ufp0/6+lca0YfQu/VsLM3I8AlkxUXQCVC5mDh8izDSaoQ== X-Received: by 2002:a17:90b:54c4:b0:340:c261:f9db with SMTP id 98e67ed59e1d1-34e92143acfmr22203621a91.10.1766987533986; Sun, 28 Dec 2025 21:52:13 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34e920c9a7csm26164019a91.0.2025.12.28.21.52.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 Dec 2025 21:52:13 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org, lorenzo.stoakes@oracle.com Cc: ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v2 1/4] mm: khugepaged: add trace_mm_khugepaged_scan event Date: Mon, 29 Dec 2025 13:51:48 +0800 Message-ID: <20251229055151.54887-2-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251229055151.54887-1-yanglincheng@kylinos.cn> References: <20251229055151.54887-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add mm_khugepaged_scan event to track the total time for full scan and the total number of pages scanned of khugepaged. Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Barry Song --- include/trace/events/huge_memory.h | 24 ++++++++++++++++++++++++ mm/khugepaged.c | 2 ++ 2 files changed, 26 insertions(+) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge= _memory.h index 4cde53b45a85..01225dd27ad5 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -236,5 +236,29 @@ TRACE_EVENT(mm_khugepaged_collapse_file, __print_symbolic(__entry->result, SCAN_STATUS)) ); =20 +TRACE_EVENT(mm_khugepaged_scan, + + TP_PROTO(struct mm_struct *mm, int progress, bool full_scan_finished), + + TP_ARGS(mm, progress, full_scan_finished), + + TP_STRUCT__entry( + __field(struct mm_struct *, mm) + __field(int, progress) + __field(bool, full_scan_finished) + ), + + TP_fast_assign( + __entry->mm =3D mm; + __entry->progress =3D progress; + __entry->full_scan_finished =3D full_scan_finished; + ), + + TP_printk("mm=3D%p, progress=3D%d, full_scan_finished=3D%d", + __entry->mm, + __entry->progress, + __entry->full_scan_finished) +); + #endif /* __HUGE_MEMORY_H */ #include diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 97d1b2824386..9f99f61689f8 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2533,6 +2533,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, int *result, collect_mm_slot(slot); } =20 + trace_mm_khugepaged_scan(mm, progress, khugepaged_scan.mm_slot =3D=3D NUL= L); + return progress; } =20 --=20 2.51.0 From nobody Mon Feb 9 00:46:10 2026 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32850274FD3 for ; Mon, 29 Dec 2025 05:52:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766987541; cv=none; b=Xx/FJfCPzeIlif2a4cy4de+LByDWO7w65tzAkHhp8aCcK5WO45Hj4G/diLzMd7mCgQ/1MypSAV8Vs/RsVjZ8Rt0OHAEJVXGUjUmramEUHScCN9zR7A2lV9uuKaMPuA+0zleVo+I8alfpKNaPsX9idk97FMD5L+MIggnATWG9PFM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766987541; c=relaxed/simple; bh=1edMOW9sfIMZ2ZM4j2SyJBQZXvYzK5XyWIWeY1r1Jx0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=grMU9AtNowmBMR4txkPesV7QkUcUAMSINO+xiqCsM8sNAJWzUAfn9eriuqwOZlUkUv1faKEoPBsLLW5zUyEdO1IhLCdSLJrE5H2sxl7Cneb8LkdY+uvbzs7FB3jp6VpQMo0ZNFcMH0vRwmiXndaifqJUpGF8CT5yk58DvbsQasE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IqaPlkSG; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IqaPlkSG" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-7aa9be9f03aso7126654b3a.2 for ; Sun, 28 Dec 2025 21:52:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1766987539; x=1767592339; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EARtjTW69X1feehvRUMcsBqUnG2mKJGDqG3MBoLBks0=; b=IqaPlkSGnMLze9Uq26b3ImkYc4sDjC+wyS0SVt+ZfJIr2CrKAZDHl8gb3/YhUw3zSL kREexZ1A0dIlcksCuG6b4aqSXewpYwc7wpSQaDmu2HnZV77/5xHaqitEZxwX/D6Pq1lA QgQISzSKTb3mhAZ+6FUgg8DmM5+yWfgbh/QAnVihBTwrWfsp146YBkM7C6t3pTbNSoyB 7NUl37LJjMk8Ft7cod53sDVv7pB4LbfIbISAB3xF+6jdmUVmZR6FN3lW3uah9dARUzlS 1rl1fG0JNK70exHtaf/OUXf+X0dIwnjW6N9JQ0P9Ghdjw/HH2ulab9J/RBTf+RhsnVt2 MZGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766987539; x=1767592339; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=EARtjTW69X1feehvRUMcsBqUnG2mKJGDqG3MBoLBks0=; b=gSQr246mAS3jh9UPHyG91iUtu8hgysC2T5sOgfGPeQo+yy18xWf9x3wEghffBDwMeR jhySOHDtX/XsQME/NhR/gins0T/tR7RBYubBB2V86Tkdq3Hng7/3xI4sbruZtFOuSmbt zaDSBb68+uD7epcGvYY0dVc3O6u8VSrYTOHbXyC26xSITZqO7kcj3ICEPMall1vQUf55 grNEpvT7e2EkiYs3qkr6nCbOE5aVHQWXpa7I+xMoiM6ym//ZJ16cJ+AEEMX1gD8fb09v Hq17Uo66Q8JmBnC+eCQSADzXivKtb/eaMsI+SU4WIT2+L0DuNEQmhmYJwLsTuMyTp2U7 HWvA== X-Forwarded-Encrypted: i=1; AJvYcCV6D2BNj4i8goRZtLkzQOxb9eBlvlqWqWXtRoK8S6DVsMua5U0fl12ceX+2QKhCP+PwLkQXqEzphTqqtdQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yw9FgDVFFzQm66sRlT6xhKpHur4vMHFlzu/GgPN6f8tkZIvWoLW xS7doWGr1fOy1TVxxnXy1bId9HRMgnfRSoQ2oGznIPObUCxWhqjz3bNa X-Gm-Gg: AY/fxX5WtUTuMjZpk6YDF9/aIJNfe07v96TJ0pBB394Bf/R/JhhM2AJ4hsYJoBgD24t c9IEOXbQOLXb+uYJMp0MMqtx4IEUtpLUKgzplyhqSw1HGSYg3AF5b27tJKvUJNmObVNygtFZIJg AzC0h+5vO5JBBqUqaZ4zhyJNbFdICTkidWdEjCIaQby0ZbVBOUpqNmTwLj24gQHsLh78IXVQLxt a/K4eRpaX/qWVlQTOrwXIEmJ+cOCEepJvuFlTpsZe6h9hEAGiD1s5B26nld2fkqqr1VLvjZY9Ml AxBusTTGsxfs07GYKhyZd7cm48ZHFKhWBoWuIZM+6nceoeSjXpKO4ZkshHLSGcWliR+OADt2vTd XWl+HMEZ7DAiRGyE6yUFtXZ2k27Kg6tYTTy++vJVmmI0NCewbELFtVK8qnaKqjWf394Df44PkJ8 JYyTzs4hAmIgVJLtUdoed6pRAyLIuY X-Google-Smtp-Source: AGHT+IHVi3ccRtDHsngBuCENxg/OidCCTnf3ExwP3w2v4XiSKIa9g0fHlTh7h+pojj04ZQfQ+6JjDw== X-Received: by 2002:a05:6a20:6a08:b0:366:14b0:4b04 with SMTP id adf61e73a8af0-376aa5008c5mr23752758637.64.1766987539272; Sun, 28 Dec 2025 21:52:19 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34e920c9a7csm26164019a91.0.2025.12.28.21.52.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 Dec 2025 21:52:17 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org, lorenzo.stoakes@oracle.com Cc: ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v2 2/4] mm: khugepaged: just skip when the memory has been collapsed Date: Mon, 29 Dec 2025 13:51:49 +0800 Message-ID: <20251229055151.54887-3-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251229055151.54887-1-yanglincheng@kylinos.cn> References: <20251229055151.54887-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The following data is traced by bpftrace on a desktop system. After the system has been left idle for 10 minutes upon booting, a lot of SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE are observed during a full scan by khugepaged. @scan_pmd_status[1]: 1 ## SCAN_SUCCEED @scan_pmd_status[6]: 2 ## SCAN_EXCEED_SHARED_PTE @scan_pmd_status[3]: 142 ## SCAN_PMD_MAPPED @scan_pmd_status[2]: 178 ## SCAN_NO_PTE_TABLE total progress size: 674 MB Total time : 419 seconds ## include khugepaged_scan_sleep_millisecs The khugepaged_scan list save all task that support collapse into hugepage, as long as the task is not destroyed, khugepaged will not remove it from the khugepaged_scan list. This exist a phenomenon where task has already collapsed all memory regions into hugepage, but khugepaged continues to scan it, which wastes CPU time and invalid, and due to khugepaged_scan_sleep_millisecs (default 10s) causes a long wait for scanning a large number of invalid task, so scanning really valid task is later. After applying this patch, when the memory is either SCAN_PMD_MAPPED or SCAN_NO_PTE_TABLE, just skip it, as follow: @scan_pmd_status[6]: 2 @scan_pmd_status[3]: 147 @scan_pmd_status[2]: 173 total progress size: 45 MB Total time : 20 seconds Signed-off-by: Vernon Yang --- mm/khugepaged.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 9f99f61689f8..2b3685b195f5 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -66,7 +66,10 @@ enum scan_result { static struct task_struct *khugepaged_thread __read_mostly; static DEFINE_MUTEX(khugepaged_mutex); =20 -/* default scan 8*HPAGE_PMD_NR ptes (or vmas) every 10 second */ +/* + * default scan 8*HPAGE_PMD_NR ptes, pmd_mapped, no_pte_table or vmas + * every 10 second. + */ static unsigned int khugepaged_pages_to_scan __read_mostly; static unsigned int khugepaged_pages_collapsed; static unsigned int khugepaged_full_scans; @@ -2487,12 +2490,22 @@ static unsigned int khugepaged_scan_mm_slot(unsigne= d int pages, int *result, khugepaged_scan.address, &mmap_locked, cc); } =20 - if (*result =3D=3D SCAN_SUCCEED) - ++khugepaged_pages_collapsed; - /* move to next address */ khugepaged_scan.address +=3D HPAGE_PMD_SIZE; - progress +=3D HPAGE_PMD_NR; + + switch (*result) { + case SCAN_NO_PTE_TABLE: + case SCAN_PMD_MAPPED: + case SCAN_PTE_MAPPED_HUGEPAGE: + progress++; + break; + case SCAN_SUCCEED: + ++khugepaged_pages_collapsed; + fallthrough; + default: + progress +=3D HPAGE_PMD_NR; + } + if (!mmap_locked) /* * We released mmap_lock so break loop. Note --=20 2.51.0 From nobody Mon Feb 9 00:46:10 2026 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06A9026C385 for ; Mon, 29 Dec 2025 05:52:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766987545; cv=none; b=ErzXVbPK5e4ImNoivafFZV5kgcddjuY/WKV8MHlmqAytTuiD5Cqy1csUibBEo2y+WMG02niKkVPWjpVQVinZQA++P3WJQzmL6Y3LDv2ER+Rl6BmXxSdPRnWijui9ECm6XjtiBkw5thFbMhxLg6sRe5hQTgYnHJnqH1bdsZLeMks= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766987545; c=relaxed/simple; bh=s04s2hnGgwxDpsfL9iSgvkVcB8t+j5N/EEA9utdpMVk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qxtBUEyvbS+mYfM7QZ4akg/29OvPscEYg7dsT+r56VTfSC9NW6oK+ooaUN0GhOGmSD0Vm2o/qC7OyQ9eT4X1PG96bG3JB188Q97MN+Aq51TBgkbNF3JFfzcSg21e/ul+u6Pq2urs4RODcKfzg4rzuR0jyqxJ27CPIrWAHKRwIPU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AhY7Duqv; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AhY7Duqv" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-34c718c5481so8487723a91.3 for ; Sun, 28 Dec 2025 21:52:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1766987543; x=1767592343; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G+YV981o5ya8+p67zEySvWRoO1hxPee1KmoTUwsGb3A=; b=AhY7DuqvKWHOfP9ljKl1oZIi04vtVje29ZUQc3heNzo02isPoKMOUAH3m4mBkSmlnp v0529AxWSlhGQV8M+9S47G6L48sLCz94qxkjfl+28VUgPwD9jPC7jM+75Opmz+nGoBqx 8M4PUjPr96SbtJ3zgirF/cXk8jGRBVBdrMu8O2fJOrW5+PvR6BUOY4wv15I9P99QDvSv urAS1u14SZU9VZyVkZVGqarrd7NfsShUCXXRTgzWkWYA/904A6n1TpJ0+JfD8mimZTxw ESXiVHAOD+WRr84BaiaaDbS0IKzBSOVXCkDfoNA/gMyyftFdaKWmZi3eVkSFmuRwSyJR 0fuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766987543; x=1767592343; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=G+YV981o5ya8+p67zEySvWRoO1hxPee1KmoTUwsGb3A=; b=FZR2ZM0htkk9tcigx2qA3cYMqvEkq2N6odgjA1aOwx0AX4kW/TtgVLYAl3ypYnGZsO QJcNeSkr1XkvjtryfA5MBFK3dcp75N1dxQiZeUgy48tJTUka29HdoL2UweWdQ0w+jcCV BoHujgIFZjwikxQc1IM1SZYhR8H8+w8KEqBSeddS0AE0YF7gVwEKN+h2+1F3ugALlhVh sOI0HGSnbGIIk2zp2k4C8JtNvorjtGlDOlExAoVuPphYn1vcih98X1/0OXbbuhgCOqBV f4Qu8s1ACT2AfiMdAOAsdeAXvNVmTCVT9Jjm6C3Fi8OOH8iPvhO3VRlNo1U/cyO3MjLl ZHEA== X-Forwarded-Encrypted: i=1; AJvYcCXsyUQYtJROYDRikb5i3xBwdVuhe3FvV+tNULH59yiXcVR0G3qKGfiJp6EUxr7CHNJxacmE4msK9IW+uwU=@vger.kernel.org X-Gm-Message-State: AOJu0Yzw3HGTer+TZ5rRtUnhl6WuSvEww/uypSpvXOCDMQPjXpqAcG8l /5lXE3ui7b9RcH5YJ5a9s2TsEBuKradOAwjdpHSOPT5Eq5FdnKrW3Ih8 X-Gm-Gg: AY/fxX7C32L6hnMG4LLfPaWOwuxB+z8dIeQrYbJTsFolF1Gadh05hpUBHiL7rTXWh2x hDsPsYNKICR49ksgBW3szNSsaCyPnq4IKQTdV0U9Jh4waAP3lB06Cijkf0fEbyASHKE0NVgzZSV IoEhH7ZiSqF3FTIQflo/MxHhXjsV2bsE6DsnuyUCoMsNHdRa6P1oTjCdnHZLgOq5Xbggv3/ZzHw L6NHwuN60jfp/G71RebU4QvpGKiLBuQjkk6nho610cCaO7js91bq1S86l1rQs6DXayynDU29Msd F0v8nzgwTRfQTN6sSKXjKRW2bSH9n8NtgeaQB682Dg7MjM48WTGJJYh93bGqTvJLXU8YINgoldg CgunHrv6Yac1LmYkZrJLaWlZcx/50Lm+t6SLCduzVG0NMGyn/O6mopI9ajt+PtvMfHjuVgwNgH4 SJlMKfLsmScM3GzdGtHio28BZkXFaI X-Google-Smtp-Source: AGHT+IElx6QVmh8bkNAdqI996CICA7RGsTegEhzBCgArsJpY8uIeyJ+BEZslcnT19OkVzRSeb+gp8g== X-Received: by 2002:a17:90b:1c12:b0:340:2a3a:71b7 with SMTP id 98e67ed59e1d1-34e9214bebcmr31415353a91.12.1766987543189; Sun, 28 Dec 2025 21:52:23 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34e920c9a7csm26164019a91.0.2025.12.28.21.52.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 Dec 2025 21:52:22 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org, lorenzo.stoakes@oracle.com Cc: ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v2 3/4] mm: khugepaged: set VM_NOHUGEPAGE flag when MADV_COLD/MADV_FREE Date: Mon, 29 Dec 2025 13:51:50 +0800 Message-ID: <20251229055151.54887-4-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251229055151.54887-1-yanglincheng@kylinos.cn> References: <20251229055151.54887-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For example, create three task: hot1 -> cold -> hot2. After all three task are created, each allocate memory 128MB. the hot1/hot2 task continuously access 128 MB memory, while the cold task only accesses its memory briefly andthen call madvise(MADV_COLD). However, khugepaged still prioritizes scanning the cold task and only scans the hot2 task after completing the scan of the cold task. So if the user has explicitly informed us via MADV_COLD/FREE that this memory is cold or will be freed, it is appropriate for khugepaged to skip it only, thereby avoiding unnecessary scan and collapse operations to reducing CPU wastage. Here are the performance test results: (Throughput bigger is better, other smaller is better) Testing on x86_64 machine: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.14 sec | 2.93 sec | -6.69% | | cycles per access | 4.96 | 2.21 | -55.44% | | Throughput | 104.38 M/sec | 111.89 M/sec | +7.19% | | dTLB-load-misses | 284814532 | 69597236 | -75.56% | Testing on qemu-system-x86_64 -enable-kvm: | task hot2 | without patch | with patch | delta | |---------------------|---------------|---------------|---------| | total accesses time | 3.35 sec | 2.96 sec | -11.64% | | cycles per access | 7.29 | 2.07 | -71.60% | | Throughput | 97.67 M/sec | 110.77 M/sec | +13.41% | | dTLB-load-misses | 241600871 | 3216108 | -98.67% | Signed-off-by: Vernon Yang --- mm/madvise.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index b617b1be0f53..3a48d725a3fc 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1360,11 +1360,8 @@ static int madvise_vma_behavior(struct madvise_behav= ior *madv_behavior) return madvise_remove(madv_behavior); case MADV_WILLNEED: return madvise_willneed(madv_behavior); - case MADV_COLD: - return madvise_cold(madv_behavior); case MADV_PAGEOUT: return madvise_pageout(madv_behavior); - case MADV_FREE: case MADV_DONTNEED: case MADV_DONTNEED_LOCKED: return madvise_dontneed_free(madv_behavior); @@ -1378,6 +1375,18 @@ static int madvise_vma_behavior(struct madvise_behav= ior *madv_behavior) =20 /* The below behaviours update VMAs via madvise_update_vma(). */ =20 + case MADV_COLD: + error =3D madvise_cold(madv_behavior); + if (error) + goto out; + new_flags =3D (new_flags & ~VM_HUGEPAGE) | VM_NOHUGEPAGE; + break; + case MADV_FREE: + error =3D madvise_dontneed_free(madv_behavior); + if (error) + goto out; + new_flags =3D (new_flags & ~VM_HUGEPAGE) | VM_NOHUGEPAGE; + break; case MADV_NORMAL: new_flags =3D new_flags & ~VM_RAND_READ & ~VM_SEQ_READ; break; @@ -1756,7 +1765,6 @@ static enum madvise_lock_mode get_lock_mode(struct ma= dvise_behavior *madv_behavi switch (madv_behavior->behavior) { case MADV_REMOVE: case MADV_WILLNEED: - case MADV_COLD: case MADV_PAGEOUT: case MADV_POPULATE_READ: case MADV_POPULATE_WRITE: @@ -1766,7 +1774,6 @@ static enum madvise_lock_mode get_lock_mode(struct ma= dvise_behavior *madv_behavi case MADV_GUARD_REMOVE: case MADV_DONTNEED: case MADV_DONTNEED_LOCKED: - case MADV_FREE: return MADVISE_VMA_READ_LOCK; default: return MADVISE_MMAP_WRITE_LOCK; --=20 2.51.0 From nobody Mon Feb 9 00:46:10 2026 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13E932609E3 for ; Mon, 29 Dec 2025 05:52:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766987552; cv=none; b=r7Mw4RPBnlmOFKSKXIOh7VcJSFY9jgZEIdgizEHHPm60msmrGH8VURXkk4S8L7sntXZ2d/8KYHnC7GPzifjeyKozpWOno13zd+rPxGQ4NMHFSui1uX0Iz+VeUZUUMc48Tyjs6b0l+tbysFsvX7gCPPnCq40CsidvhHngvIHb2fI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766987552; c=relaxed/simple; bh=OWMjmZ/uSGwWXdkiYONTFrdrbUEJaOW12h0tMPezM9A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BEA074cf4GG6s8Z4npMZdJD3OKKbsrIvs5f+S6pLeMMp7o0S5BJ0NNYt/Iqcg5Zdq9MLn6v9eI1SQmVn7/8+nFoJxF6U/GJIXkhWLzlrcKOCoSnTwNqnl8z9QPc0n9BYaTJNUwOOPtStlej0tUpKURvf1nyj5fQDwA2H4Rs6Eqw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=LwzL61Lh; arc=none smtp.client-ip=209.85.216.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LwzL61Lh" Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-34c21417781so8948593a91.3 for ; Sun, 28 Dec 2025 21:52:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1766987550; x=1767592350; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p0TAcRTppXCDnIWmkxS69nQ13SWq+RnVhYEMaAViPXU=; b=LwzL61LhzplyF0QGsroe4C22234N6LtgfAqQkPiImdb5IjdX+ina3o8fLpJim3hGam 3CJqfPYH3Bpn5sBaoB7LE8i6TSMJ3YYDQ4wpPfP6y+GQegYLpIofEZH6VXbygG47H+hs Uwa3/HXJM10Z7lBSs6f+CybXkKuNH6hFXL+1q86S0DZDMov6HrTUUeotyNuLewTpHEpy mqm1zo68GRs9ylA1FWmoAIFlqt4xQ1aYiiw0MKMPBmJ/PUEHLDDDCVp7XG6XrGNIg5UK lI1jCT+5OlmS0Aj9Gn3l6OvZh3BwPjmBaO8bZ/B+rBot2nXZej+e+iyI8XewlZ2rgwmA pD6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766987550; x=1767592350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=p0TAcRTppXCDnIWmkxS69nQ13SWq+RnVhYEMaAViPXU=; b=fJxA8LdnmXNIQ2mUUOperHA4P0OT1SJ4O4jMXvJeO34HXn98RzCqro/F3QqrTxnFNi GVaxSvxVArtoqGirbfK9Zoa6KOgjXT3a4SrkMfudr2VySuoF+xPwsClV3cPZF0pFjJDS /7pZpTxeTHIyLGVY7bghUoGEbf9j+qE5bB8Mf8rw/OZVUE5MjsF+134uowWFwnoKerp8 RkSqTSjJ4jBRJPa0JKGjR+6UnXXivdfS6OhWd44j8jj1C2HnupZgzWb/sYQDCE9m8z53 P1SdQFmWlBsfo9It2y1qiZYUNiqNLDV02A+sjLWB5TPv59qWiyXKCGMwq2uwQf0B6JBl waRQ== X-Forwarded-Encrypted: i=1; AJvYcCVS6uEC+mTKA+mwNDn4BSpvJB7hGvk7DnWFz83DFXCbSVyOgnyP1up1uL2pNGmr2t0GMJ2qxPOMR1c5BVU=@vger.kernel.org X-Gm-Message-State: AOJu0YwnSbAuNPuQsNuKGtT9XRdLSnze96s5jxN5BoRhCqATRAScSDWa 4kGvgWCZguLDKhT5UZyxofHWuPnR8Naf/G4mJ63ByQe8b1HfgUfujksT X-Gm-Gg: AY/fxX6wqQ6hIju3vYdu4lY/1I8Qi1RFd1rPu8SN4WwNjvJxBic3GOfMN0Pg/7x8MmQ KymkwCzpA8Y4943oj9y9p9o/gnaU13VmifrhzK25O+X2AkO1+q5YTuzRunlOI1VWUcOMN816zdB yJ6Q1IZorwwoqn24T4RvQJFMvvy4dzcgX0UQLfTnDcJ0ErcHNiYuj59U9fFZOjMObhEcVvac5sQ 6fSG1XgF2lDKlwI/ihg8Xj8+bXybrX6TdXCUFpiBdskyeVZamXqyE7rgPmjThtD561mhEl2mBRi YWWx3oTQFFZF4b0JRYwInKQeFHKuFK94DM2OI48KFALUE+OfJy4NvY1IAV7zQ9yQSSKVR46FbKe CBLeD2LJgBfaEg6JLUrmTFAY2ePQG7Vrjx3/OLjYdMbJ/5mwWwYfVC29PEBBENzanGwftsIVRVs IonX6FKJTGCYqNi2vg10Y61GDatUok9gJMRsbX9Sw= X-Google-Smtp-Source: AGHT+IFN+5k0M6/l/01VR9YiTbqghhmhkxo3d/mqneg0X5+Q2LxFb2X5MQf8t67NEYAVtiiwDIIDfA== X-Received: by 2002:a17:90b:582c:b0:340:5c27:a096 with SMTP id 98e67ed59e1d1-34e92113697mr26001072a91.6.1766987550278; Sun, 28 Dec 2025 21:52:30 -0800 (PST) Received: from localhost.localdomain ([121.232.80.251]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-34e920c9a7csm26164019a91.0.2025.12.28.21.52.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 Dec 2025 21:52:28 -0800 (PST) From: Vernon Yang X-Google-Original-From: Vernon Yang To: akpm@linux-foundation.org, david@kernel.org, lorenzo.stoakes@oracle.com Cc: ziy@nvidia.com, dev.jain@arm.com, baohua@kernel.org, lance.yang@linux.dev, richard.weiyang@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vernon Yang Subject: [PATCH v2 4/4] mm: khugepaged: set to next mm direct when mm has MMF_DISABLE_THP_COMPLETELY Date: Mon, 29 Dec 2025 13:51:51 +0800 Message-ID: <20251229055151.54887-5-yanglincheng@kylinos.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251229055151.54887-1-yanglincheng@kylinos.cn> References: <20251229055151.54887-1-yanglincheng@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When an mm with the MMF_DISABLE_THP_COMPLETELY flag is detected during scanning, directly set khugepaged_scan.mm_slot to the next mm_slot, reduce redundant operation. Signed-off-by: Vernon Yang --- mm/khugepaged.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 2b3685b195f5..72be87ef384b 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2439,6 +2439,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned = int pages, int *result, =20 cond_resched(); if (unlikely(hpage_collapse_test_exit_or_disable(mm))) { + vma =3D NULL; progress++; break; } @@ -2459,8 +2460,10 @@ static unsigned int khugepaged_scan_mm_slot(unsigned= int pages, int *result, bool mmap_locked =3D true; =20 cond_resched(); - if (unlikely(hpage_collapse_test_exit_or_disable(mm))) + if (unlikely(hpage_collapse_test_exit_or_disable(mm))) { + vma =3D NULL; goto breakouterloop; + } =20 VM_BUG_ON(khugepaged_scan.address < hstart || khugepaged_scan.address + HPAGE_PMD_SIZE > @@ -2477,8 +2480,10 @@ static unsigned int khugepaged_scan_mm_slot(unsigned= int pages, int *result, fput(file); if (*result =3D=3D SCAN_PTE_MAPPED_HUGEPAGE) { mmap_read_lock(mm); - if (hpage_collapse_test_exit_or_disable(mm)) + if (hpage_collapse_test_exit_or_disable(mm)) { + vma =3D NULL; goto breakouterloop; + } *result =3D collapse_pte_mapped_thp(mm, khugepaged_scan.address, false); if (*result =3D=3D SCAN_PMD_MAPPED) --=20 2.51.0