From nobody Fri Sep 25 21:06:46 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 127E14963B3 for ; Mon, 21 Sep 2026 12:31:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789993902; cv=none; b=Xjxd8cmavV45Wc+vDKk+onVw0dyGZ5eQE2MUdFCA/ZmtDb5vdhvB4zu1QMOoKdvZo1eOjTbu9t5Upw+Ods9vJWAGw9q4Ch8iBdR1HyYtAnz6x37Gma33MhJYzU1cRP5103VkA0HQ5AElvT2tiGdaLnaGENeRJvyV9+qjPNe3nbo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789993902; c=relaxed/simple; bh=yc/Zs6xPgQccUGSBvUxKF4tBdDWNVRPWwVRuT71kyJY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PCbS1NTiwdHFOYAdwTxueRpqGWKg+kV7C38QLaiQJQXaEZ76swfPsnDAecnfa5/QIwGfJUmNMbx/oOd09A8OM0lyrw6Fh5zY9P9bfP9i5x79sPV3pi851QT2lSBfl6ibSKD51GWUwDtzjPxYXzE3GCjxTJit/ax4Xc2nIvN6++Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=ibPXFwr2; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="ibPXFwr2" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=RJyhK/H/cx9UDbk1fQ3qzOg9bJhZWQMaDj6VPrkWrC4=; b=ibPXFwr2uzqV+04gSCXcVxns3G 0N6SGj4ckD/GEGj+9xBvm5gc2lUWYpXaO9EGoQeAmTcgwnwUCfKPFP94fpIcoVjB5HywRUgzUF/hR P/iU7uXfEp0qRmAKS3eAQ6aObaPREGzcQo9tI0SmP6YEnP+iGJXDv7Q1hknLVeR9C6XLs6hfQgBLE +3mx/rYlCOm9+wYr2sw8HXCWia72DJTqwWByVSSY1xcnV/ApUoEcvbM7mhcs7kazjHAaJdOOw8Jp0 B//KozZRjdxVbA7qQVKd7fWlnpmim48rsTfkOoF+QAkRMVIRN8wB5pDFCjIJ3qV+bNrrolsTKfBUN MjyV2gsg==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1x8dBE-002CbX-27; Mon, 21 Sep 2026 12:31:32 +0000 From: Breno Leitao Date: Mon, 21 Sep 2026 05:30:26 -0700 Subject: [PATCH 1/2] mm: kmemleak: move the struct page scan into a helper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260921-b4-kmemleak-page-scan-v1-1-fb97d4801b3a@debian.org> References: <20260921-b4-kmemleak-page-scan-v1-0-fb97d4801b3a@debian.org> In-Reply-To: <20260921-b4-kmemleak-page-scan-v1-0-fb97d4801b3a@debian.org> To: Catalin Marinas , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com X-Mailer: b4 0.15-dev-47773 X-Developer-Signature: v=1; a=openpgp-sha256; l=2538; i=leitao@debian.org; h=from:subject:message-id; bh=yc/Zs6xPgQccUGSBvUxKF4tBdDWNVRPWwVRuT71kyJY=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqsSOdoIsG5nmdHaB1wUgSXy9i1JwH500pnNE3N rKnFnTXCkqJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCarEjnQAKCRA1o5Of/Hh3 bRmPD/43icZSLUGLZ74SsnNFAMz6NWdw+tbqTD/eGO2VlOk2wSFuDubLQq39MsAJzdVK0/DREfb LPlqtsuWSvFjBEriJP3QiK+lkfkvsFHaFzK0L6sVRBBN78wBHyBqmE5rznQR8Lqf2B7xlKY9Mul HYYi+fdPlKPdsT4A/U2lrniCNziKKwip0EKsnIaJsSSH/7sO4eb9UmDxzXUw1zdYV7ECKjbIlZK gSRaXh/kfXQvUFBTnNqNCgLPwOWHnsbQo+N3BYh2YMeAGC7EdsnFMj+xZ6Rcu+GYz7ABCX7kqnM yEehZDzKAn4WBLgvV0hFG3lGCpzSzcr8WL9LT1PdWyZGBZRXBrOVu8koVsJwkdmi2mCyWShnq8u hHi2LMO6717dMa0FZ9Gg7lYfgpdRowt33o1xgr9fnPko2ob900IojCDijifeCd6BB8To9KLL97J 1oAqFyDjTQy2kjjPvTkyRdHckfWiRqekTnWUa3lwtPFIjLAsbCGQZnZ+YWWiGMdkfrQUBGSYPJX CUNA3HZIQ2lzOaFi3jszMfEbxJMGTM5220SpiVDCvZ4GWFFWZF+Y/DHYFfpUCcwc+/RY8W5PLIF 8luOvbt0/zosAXojGm6bm0G9Ve5nYHnrmaL/z4ececCrm8AcFv09NDofT1MvH8U71UStfiuDHVF NL5ckeoawor7maQ== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao The struct page scanning loop sits inline in __kmemleak_scan(), nested three levels deep and sharing the caller's "stop" variable with the zone walk around it. Move it into scan_zone_pages(), which scans one zone and returns 1 if scanning should stop. No functional change; this only makes room for changing how the pages are handed to scan_block(). Signed-off-by: Breno Leitao Reviewed-by: Catalin Marinas --- mm/kmemleak.c | 57 ++++++++++++++++++++++++++++++++++---------------------= -- 1 file changed, 34 insertions(+), 23 deletions(-) diff --git a/mm/kmemleak.c b/mm/kmemleak.c index 8fa409a4f9fb2..4040547a0af84 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -1855,6 +1855,39 @@ static void dedup_flush(struct xarray *dedup) } } =20 +/* + * Scan the struct pages of a zone, skipping memory holes, pages that belo= ng to + * another zone and pages that are not in use. Returns 1 if the scan shoul= d be + * stopped. + */ +static int scan_zone_pages(struct zone *zone) +{ + unsigned long start_pfn =3D zone->zone_start_pfn; + unsigned long end_pfn =3D zone_end_pfn(zone); + unsigned long pfn; + + for (pfn =3D start_pfn; pfn < end_pfn; pfn++) { + struct page *page =3D pfn_to_online_page(pfn); + + if (!(pfn & 63)) + cond_resched_tasks_rcu_qs(); + + if (!page) + continue; + + /* only scan pages belonging to this zone */ + if (page_zone(page) !=3D zone) + continue; + /* only scan if page is in use */ + if (page_count(page) =3D=3D 0) + continue; + if (scan_block(page, page + 1, NULL)) + return 1; + } + + return 0; +} + /* * Scan data sections and all the referenced memory blocks allocated via t= he * kernel's standard allocators. This function must be called with the @@ -1928,29 +1961,7 @@ static int __kmemleak_scan(bool full) */ get_online_mems(); for_each_populated_zone(zone) { - unsigned long start_pfn =3D zone->zone_start_pfn; - unsigned long end_pfn =3D zone_end_pfn(zone); - unsigned long pfn; - - for (pfn =3D start_pfn; pfn < end_pfn; pfn++) { - struct page *page =3D pfn_to_online_page(pfn); - - if (!(pfn & 63)) - cond_resched_tasks_rcu_qs(); - - if (!page) - continue; - - /* only scan pages belonging to this zone */ - if (page_zone(page) !=3D zone) - continue; - /* only scan if page is in use */ - if (page_count(page) =3D=3D 0) - continue; - stop =3D scan_block(page, page + 1, NULL); - if (stop) - break; - } + stop =3D scan_zone_pages(zone); if (stop) break; } --=20 2.53.0-Meta From nobody Fri Sep 25 21:06:46 2026 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EB2146EC84 for ; Mon, 21 Sep 2026 12:31:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789993902; cv=none; b=rKYkiOHtZoXFazDAFIrOcVI6gQO3E2EJFslocrHaGYUxe5oMtyWgreXrRVMGrzimYTQqZUec/vnrWguAH3aAG7dIgl31arBqG8PaYMgroxR988JdmTln9rNMRqkmYsiCBgx2z+eoSf1781OdWkWE+rEPWuGI65NDPhO07hcHmvE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789993902; c=relaxed/simple; bh=HJ5nwvE4BGwjNEPPGJ48wKJt/yd7AFzI93B8qcnsppg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=SlovxdhRnDW9idvGw3cvvHW2AEIELaOwjF0DRsCvx2jsvwnJTTlh3C6ctro0qQhr3XWs8iiIHW7PqTcuomerIeG95b1lyP8KtuzujwEmpwCY7DvX9DKR+fjlbPToAFhxvDRLIqQ0aET9jRWY3Ql4MUSP7WOUqdQtP57rExBVwTA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=hC26Q4qe; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="hC26Q4qe" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=4hgW/FJkW3mwJCGKwE/mUcy/xjFFdmovY7N91WUWEFM=; b=hC26Q4qecCQvJ0tcjkj2v/tsuX qVcXeyp2OjNQ0RE6QFiHeSNjmbFHucskq0RJNSlO4a/tXIf4OIpDe2EOn5vKxmEFs0sbB4CKy5Z3Z R5FbZCYK7t6bfvrSzRG3lZkzqWdH2iV04tpFZ/gD/YScfwm74ZgMas5eXSOajgSJ95j81yjK7dAKY 2VUrv+Fh74RgpvQigb6EMxSNZ5FcA36+AVx3vWFX+QIg/9QBgOq20dPr44rK7bfqEdS+YvM22r9gY wH6GqGpg+suQbdImjSp9YEw20Y2A7gVjdK8G+mvV7UNJGGqmssbvSjEh2rqTB8QEdUz+hUIR7Asxu 6zOFJ2Tg==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1x8dBH-002CbZ-1W; Mon, 21 Sep 2026 12:31:35 +0000 From: Breno Leitao Date: Mon, 21 Sep 2026 05:30:27 -0700 Subject: [PATCH 2/2] mm: kmemleak: scan the struct page array in MAX_SCAN_SIZE batches Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260921-b4-kmemleak-page-scan-v1-2-fb97d4801b3a@debian.org> References: <20260921-b4-kmemleak-page-scan-v1-0-fb97d4801b3a@debian.org> In-Reply-To: <20260921-b4-kmemleak-page-scan-v1-0-fb97d4801b3a@debian.org> To: Catalin Marinas , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com X-Mailer: b4 0.15-dev-47773 X-Developer-Signature: v=1; a=openpgp-sha256; l=2932; i=leitao@debian.org; h=from:subject:message-id; bh=HJ5nwvE4BGwjNEPPGJ48wKJt/yd7AFzI93B8qcnsppg=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqsSOdriwWS76lw2mmVu50YCXUKoW/8idyboDiJ HUO7c2CQZOJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCarEjnQAKCRA1o5Of/Hh3 bfwSEACgjjlP64qqLp9HIM9XrwDJpwy21iR+rcS8/Zaa2PBasil6vpvB4qpzQ+0NoUGhUTtTW8L CAH/mHe5ZAXEEYWEO5rsrGQuQ907IrNckn0dO8rRpq/hQihiughnGexmWju2jv5QB8Viksw3ELT 1k5bPp6wQICRp67FYXFljlmVkpkA6dE13CMgFZ4zEfIK9r4C5h7zj6QEVrOrTOPUdpV8LUPwli9 0L+vEiEQyHJq9Z0qEwx4QT//hld93MdLWv72TFhhJXJp9wFKlOEn/PkqXhr7qBRMqu6CT3MmuEX wyR/WQXwPe+JW/oYYep6vCJKiHLQVIRg4ZeFem8acP44UKKaX7U8q0dgYbo/i1aok4s45TPysHT M2V99jOwUdnBZclMOg87JYjGkz3BgeUs7EX3Car4YeMC7Zq3P7gJ8DJ21fJxAalKxbu2ghFH8OA XtPLnJeA77+JqE/MrfzBphMAiPWrG2at/SGHKzwOcyU+F1I50FrxhSDSgMqpyGDKazKrQoAuWdq 0N2TBAPZKABDU4zeUEXt8WZ361DSbCZMkRYYbOmDeGT9zvAf+GcVH3xDz2byU7k4CclpPtab8p1 ypNLqooJr9TP+NYbopVSWlBP/hz/QIxvvE4GjHUHymeCW52WfDB0mfbQVITHExZG2docK57XUPD uTYKxgRns17Tm7w== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao scan_zone_pages() scans the struct page array one page per scan_block() call: if (scan_block(page, page + 1, NULL)) scan_block() takes kmemleak_lock with interrupts disabled for the duration of the call, so this acquires the lock once per online PFN to scan a single struct page. Gather runs of adjacent eligible struct pages and pass each run to scan_block() in one call, capped at MAX_SCAN_SIZE. The longest kmemleak_lock is now held is MAX_SCAN_SIZE worth of words, the same bound scan_large_block() already applies to the data sections and the per-CPU areas. This can make the scan faster. On an arm64 VM with 24 GiB and ~17 GiB in use, the struct page phase goes from 4390k scan_block() calls down to 69k for the same 4390k pages scanned, and the whole scan about 20% faster (on debug kernel). The win is in the per-acquisition cost, so on a kernel built without the lock debugging options the scan time is unchanged. I don't think it will be a problem doing more on scan_block(), given we have the scan_should_stop() protection. Coverage is unchanged: a temporary assertion comparing the number of eligible pages against the number actually passed to scan_block() matched on every zone of every scan, including while memory was being freed underneath the scan. Signed-off-by: Breno Leitao Reviewed-by: Catalin Marinas --- mm/kmemleak.c | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/mm/kmemleak.c b/mm/kmemleak.c index 4040547a0af84..cb953df3b414a 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -1862,8 +1862,11 @@ static void dedup_flush(struct xarray *dedup) */ static int scan_zone_pages(struct zone *zone) { + const unsigned int max_batch =3D MAX_SCAN_SIZE / sizeof(struct page); unsigned long start_pfn =3D zone->zone_start_pfn; unsigned long end_pfn =3D zone_end_pfn(zone); + struct page *first =3D NULL, *last =3D NULL; + unsigned int batch =3D 0; unsigned long pfn; =20 for (pfn =3D start_pfn; pfn < end_pfn; pfn++) { @@ -1872,19 +1875,29 @@ static int scan_zone_pages(struct zone *zone) if (!(pfn & 63)) cond_resched_tasks_rcu_qs(); =20 - if (!page) - continue; + /* only scan in-use pages belonging to this zone */ + if (page && (page_zone(page) !=3D zone || + page_count(page) =3D=3D 0)) + page =3D NULL; =20 - /* only scan pages belonging to this zone */ - if (page_zone(page) !=3D zone) - continue; - /* only scan if page is in use */ - if (page_count(page) =3D=3D 0) + if (page && first && page =3D=3D last + 1 && + batch < max_batch) { + last =3D page; + batch++; continue; - if (scan_block(page, page + 1, NULL)) + } + + if (first && scan_block(first, last + 1, NULL)) return 1; + + first =3D page; + last =3D page; + batch =3D page ? 1 : 0; } =20 + if (first && scan_block(first, last + 1, NULL)) + return 1; + return 0; } =20 --=20 2.53.0-Meta