From nobody Mon Jun 15 05:18:44 2026 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010051.outbound.protection.outlook.com [52.101.193.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D28AE25F988 for ; Tue, 21 Apr 2026 04:25:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776745523; cv=fail; b=pg0jtxJj/9vhNYcfx40fzhyLhdS2COAbhWIM/hGYocLFXprqTGs3Pbe7cSqLZsQ5JnUhsc6jAsJ8xYpjdIjmuBcoG6iomPKR/iqaQk/Hu8w/3WTYsUX0dCMrKghIZtJ1fMotjXm+lyJC982bbe11PvIaEp/oREBaaCF1cwCyPf0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776745523; c=relaxed/simple; bh=DjHVG+yBqLoacqDuwAzTf1HUO8ShvXi6d5nX7BA2si8=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=budz/GkfxfTFKLp1RKZovsJ2SWN42tfpk7tTBnkibRY6sey0WUJijMnsZyWw0IaIqTUbGafSS1BzpmS3ILpc/pc7X9+MUy7cSTotDsNRqW+cegzcUGTyyxznO56Yvo2+5uRsLho8BFXbSt7pEcMJ0EtsTe4f6g530QOhRgQz9Kg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=V6d/KUrI; arc=fail smtp.client-ip=52.101.193.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="V6d/KUrI" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xSIhlwQ4w814gVAnIZKMX27q1FRHMG0p53N71xjYfBXTBGflobN+BX89jhIc1AhrpxAundbwFuaeWLDgoO5zpqK63RTTKSUXSNdUkUWCk8r3O/XhJAljd2pDchCGuwBvsUCHVlA/FNFXT026bSGpDLK48e7Z/mKZj0t2gclOlv0nhJJ/NKtSQDAmKYcw22nBgWgyXOfIEE90Rp4OhugaSJks/HLsoSr4efRKrpYmv5JskgdHR4YzJdB+2dNbIe8qInrnlcfhExg7I7oJWYRL6alvFQAQhIeocZBe/I+4+subFfo84Lm9Wjqw6obPWVziwJf2ToWTw0yvhOJMBPjYJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kn9Nb6bGppO0y7ftKbz2SIRH0RqK/G5Rr21hXk07hyE=; b=DhxKga9ouEHg3XeZ48mF4ES/5uBHzqqyDLsc0WRax/VnHQfkIeancEvjMyhIorEy6YQTVWzgIXkJDnIdweJA+9N7TIt0UqB5EGlTO4990tvwukK3aqghLU5iYuQlzeeEPp3sgb0Sy9vrWVqmYinW5xco9iJtqkfMGMa5wGaVwr2eWD41wGMz70ZAS+9ZFZOh3MOZVyWTaP1mFdUao3kb2t/oBYmb6fyDeP/smDyrFBKe7ss1F8NJ4V9JjMiycwIXnrKtW56AQcEGA2q7oIPe9h6XfeXIGdMqSX/EY13+6SM0sj+vSzTiog3IqBkUGN3yQPLS/syzoi69lyLCoHALIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kn9Nb6bGppO0y7ftKbz2SIRH0RqK/G5Rr21hXk07hyE=; b=V6d/KUrIbfsqNWzoTSUxPDNbdHd1jAP83lYM5NKy8PJ17KtxYy2ujegKq14pb6mRPBRS+l1vJS/kBoNNkyg1lh77fCMDEJtEPul9R3pd4kS4yFqLfMo40Ei70uyLPiBZNAN46TJY2jmkKh3qjhtqyEbR2kbdFCsMCJH4CG1P9eA= Received: from DM6PR02CA0166.namprd02.prod.outlook.com (2603:10b6:5:332::33) by DS0PR12MB7898.namprd12.prod.outlook.com (2603:10b6:8:14c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.14; Tue, 21 Apr 2026 04:25:18 +0000 Received: from CY4PEPF0000EE39.namprd03.prod.outlook.com (2603:10b6:5:332:cafe::e3) by DM6PR02CA0166.outlook.office365.com (2603:10b6:5:332::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9791.48 via Frontend Transport; Tue, 21 Apr 2026 04:25:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by CY4PEPF0000EE39.mail.protection.outlook.com (10.167.242.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9791.48 via Frontend Transport; Tue, 21 Apr 2026 04:25:17 +0000 Received: from kenya-2193host.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Mon, 20 Apr 2026 23:25:10 -0500 From: Hrushikesh Salunke To: , , , , , , , , , , CC: , , , , , , Subject: [PATCH v2] mm/page_alloc: use batch page clearing in kernel_init_pages() Date: Tue, 21 Apr 2026 04:24:49 +0000 Message-ID: <20260421042451.76918-1-hsalunke@amd.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE39:EE_|DS0PR12MB7898:EE_ X-MS-Office365-Filtering-Correlation-Id: 89b3da7c-dc86-403a-e0ee-08de9f5dfde2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|376014|7416014|82310400026|921020|13003099007|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: lnWFfiAJIaH/bCATqxAFQLaNpz8EN4+OyCLrhXQRmmNjcek2aUSaYMtA13StzqTpXycq90V30Bnb93smoPF/6Aqy7gmcv1i+0CLxPB3GK0XCZ/qnpoE5n1Ek7fkYlnrI1Om3l8QJktH613Um22xPIU/eVR3EQGYJG9EEoVHK/K2TEz9lZ75rfoVz+cIXOiIkcAehytnHWRWmNALODAr0tBL8Ul13Ck6Eksm+Fs7cf2T2wGofTmfZuQC3sqiofYDvEgc2m/1sruy+qMxOaFDT8Vzk36lyCHQuk/jDND9WrInPXrMo5oVf2dwpYEP0iL8cBKuyKECF9Wzb8DvMEAV64HktxtURAxi2arMjW+AcJJjgdYx5Ad0Ny3cwot9EQktppuIJbYZZQi0ONXtLNf55tv1EmUcE3JR9dn1aY24DadBnIXwqhcnhfZhYtpO8O7F9EJfoXrBgKkAauJ/Dk/JkUxWQ3yUd2+6lzPymRekMFxdLzp/v+k4lQV7rj8yfTV38F8jgtFg8rXan+80dUK2QFl2inNGXqM+Hay0tK+S3iOc/eIcxx9lHVxM0Hcx6Kk0bI2Uplk0WI3+qWX/M8lAsjP+Oc92VQOn7yYOejC0JcvNoOniXlyQJ4cHuQnhnQgw6rikvhMo3AmZsGFzIIx3HaUs9bkupoKAulcQaekvusU77lNXRL59zJ09gD8IHmRmGjkYZsgiCpmewBF4mxNZwqGVo18budSpDhj460jI1KyB4OzV4MzDY/g8f0gi6FsR7Sc7gEwIUTw9EQolAj7uULJyRlNnjyLvfaR+yzp5zQLbtwmOBFYjjsBsFdCmTcOoi X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(376014)(7416014)(82310400026)(921020)(13003099007)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cXUB9LvueLd1Le9yxNkoPG/MNLf9LZNPUDXsxIFLRISXsp6iocP3RZFP5W/ao71udsWfb71gSntKb/zs+Tam+C/PS7ox0RGr0HFY5WbE3lVXaGtWVUz0Psc+Q3hoSRPUZpYg4Av0fDR3ba8JlmWrW7Yaa/7uASSC8gwxMEaYggTQ2grt6zt8Ovj6KLGDq+RggCHMbKP20BFJV4c1vDdIIc3YRkyO8fEyA7bLz2cbZRPGuv3kHcJ1PlDaXXBK+OVgbRPdUNWKZeLCBDb5hiYzqEwfXWPoSCDZ/D+OfUrCei+aonJLraHtSq/B8KAXgqo4LibBq64BdM5/Ch4gAHKgRksmCrWCJ3DlNLCAVzP+fSO/d29T822gFNEMRYGymhjKVfhrQsYq0E2nsd1CboaxFatgvaqh/TgcSwHWPxK/8rFNpLATOAyFRk/cWzvCA7OZ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Apr 2026 04:25:17.2500 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 89b3da7c-dc86-403a-e0ee-08de9f5dfde2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE39.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7898 Content-Type: text/plain; charset="utf-8" When init_on_alloc is enabled, kernel_init_pages() clears every page one at a time via clear_highpage_kasan_tagged(), which incurs per-page kmap_local_page()/kunmap_local() overhead and prevents the architecture clearing primitive from operating on contiguous ranges. Introduce clear_highpages_kasan_tagged() in highmem.h, a batch clearing helper that calls clear_pages() for the full contiguous range on !HIGHMEM systems, bypassing the per-page kmap overhead and allowing a single invocation of the arch clearing primitive across the entire allocation. The HIGHMEM path falls back to per-page clearing since those pages require kmap. Use it in kernel_init_pages() to replace the per-page loop. Allocating 8192 x 2MB HugeTLB pages (16GB) with init_on_alloc=3D1: Before: 0.445s After: 0.166s (-62.7%, 2.68x faster) Kernel time (sys) reduction per workload with init_on_alloc=3D1: Workload Before After Change Graph500 64C128T 30m 41.8s 15m 14.8s -50.3% Graph500 16C32T 15m 56.7s 9m 43.7s -39.0% Pagerank 32T 1m 58.5s 1m 12.8s -38.5% Pagerank 128T 2m 36.3s 1m 40.4s -35.7% Signed-off-by: Hrushikesh Salunke Acked-by: Vlastimil Babka (SUSE) Acked-by: Zi Yan --- base commit: f1541b40cd422d7e22273be9b7e9edfc9ea4f0d7 v1: https://lore.kernel.org/all/20260408092441.435133-1-hsalunke@amd.com/ Changes since v1: - Dropped cond_resched() and PROCESS_PAGES_NON_PREEMPT_BATCH as kernel_init_pages() runs inside the page allocator and can be called from atomic context, making cond_resched() unsafe. The=20 original code never had a cond_resched() here, and the=20 performance gain comes from batching, not rescheduling. - Moved the !HIGHMEM/HIGHMEM branching into a new clear_highpages_kasan_tagged() helper in highmem.h, per David's suggestion. include/linux/highmem.h | 12 ++++++++++++ mm/page_alloc.c | 5 +---- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index af03db851a1d..ad0f42d06ce6 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -345,6 +345,18 @@ static inline void clear_highpage_kasan_tagged(struct = page *page) kunmap_local(kaddr); } =20 +static inline void clear_highpages_kasan_tagged(struct page *page, int num= pages) +{ + if (!IS_ENABLED(CONFIG_HIGHMEM)) { + clear_pages(kasan_reset_tag(page_address(page)), numpages); + } else { + int i; + + for (i =3D 0; i < numpages; i++) + clear_highpage_kasan_tagged(page + i); + } +} + #ifndef __HAVE_ARCH_TAG_CLEAR_HIGHPAGES =20 /* Return false to let people know we did not initialize the pages */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b1c5430cad4e..1aaf7f839ff4 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1220,12 +1220,9 @@ static inline bool should_skip_kasan_poison(struct p= age *page) =20 static void kernel_init_pages(struct page *page, int numpages) { - int i; - /* s390's use of memset() could override KASAN redzones. */ kasan_disable_current(); - for (i =3D 0; i < numpages; i++) - clear_highpage_kasan_tagged(page + i); + clear_highpages_kasan_tagged(page, numpages); kasan_enable_current(); } =20 --=20 2.43.0