From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 867163D45CB for ; Mon, 20 Jul 2026 09:31:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539916; cv=none; b=IfjQCJ4XFkreXdPT+Dpejc92fESmy1dCjkNG7A0deETcC+WXcd6AMG9jGZggtORjU7j71EgDzLvU+NMpPOt5JPLY3Z5n5kkhjTtqhkYBLgZJNwlXVoE/mdKiT99bBtilUfLAa11l6mORYkQET49SeuK07RI6LnHts2XD878/EPc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539916; c=relaxed/simple; bh=rLofQWHjRYkso5/XmB2nkMHlMv/dzmF9AEWfCv08OcQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QSR1FLfQiFnqIQhO069YNcboVaM7j3fBA1uDzdWvZzwpoemyK1egXhPBfOMhd0oj2F39EsK7wNUmv8EXHst6UTAIbukgd0/JSszdFukVKhVcBUFLEiMaSoxKAxICbwsN/FauGUbtii2/1eSBVIOW61JVFoZ27fdIUxLwl4N2WLg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=R/eNVNt0; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="R/eNVNt0" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2ce7d2adef4so94146685ad.3 for ; Mon, 20 Jul 2026 02:31:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539914; x=1785144714; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=xuZ6BXpDFSgmVKQGs+ltcwR2zNvZZ9A7kJPisvEZrvg=; b=R/eNVNt0M0UTYurSq5OD0oCjOD2KAvlmAovL4+ERTGlpa0Jj8LUbzERl6wFKhAJQef 44CoNGJhCGePPG/OMJ5S/eY6ZyFbPD8mmVxLwIIfh7xKmmlaCfapphcPtZTTKC9URA+D DpkIcS1duhjH/1K2BLv393Mi8hWYkfM1EqmNHEPXVsuskS/hVfGFcxStmkjDjlibp1U6 J9+wmyyL71YQiNBZh0QxfRuE2sjXRk6suMg+NbNwl53xyK9ZChPYLZYSdrZXfBspWXJi HUohzbraZxBh5tzN/gdSFI8dxm9HIkZusJsHmaUDOJjwOWTUI9HacYHjbvYa8+qGCA3n cjDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539914; x=1785144714; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=xuZ6BXpDFSgmVKQGs+ltcwR2zNvZZ9A7kJPisvEZrvg=; b=qDLzCFLYEEfhMuSZ2H76Wu3zQHZepJQ4RI9k3v8DugXPxD3xYMNNRtaOssfH7dD9YA OakdGmAFRhIy8Hv/yhLbWyfS8j1vNedVhZ0Upgmgck33IdAnYYtujSJ0F7r11Ncis8Rq nMWcZeY0LogpD4t+SYbwvNb9sKu1A66i4RmL/SCsrtOQgU/3tokCj9gmdwzagOVOGT7r qSHWeLBcS46ROdmrcbbzw+biaWyvsM7PICcgbn+mpgzWu8xUuFA/4PL7xRvSQahqY39o c8Erl4coEu85yU5BXuL/tz08VHOIr7d/kSa4ZdIIdxoa0p1NW0yX5lPH0Z4777uC8GP3 Ittw== X-Forwarded-Encrypted: i=1; AHgh+RpZoNel75+7zUqByOwV3lZUSjf+UygLSOqBqs/qJiEiE1kc4rKzoIbT3riT1wCBBycqhyJsxjvP66Lcx8U=@vger.kernel.org X-Gm-Message-State: AOJu0Ywm1IV0zbjyIGr1s7PKXrf4HDRZEc8rwKd6C96a7qf3/rqRt5Uu c6XaXXrhwrotoyPK0AG3O+ZhveiVqQrQQda/b2UA19Ty3BuW1zME6wfEQ76Xd7vfyZs= X-Gm-Gg: AfdE7ckzl5mO0CPFIA67eW+4hDf9T84cUoYQQ22vsgGzferw1FSkxyFbXoXhSCZJq2Z jEwDKAevAZdvwpYn9nEdC/7FY1H/PjT6jcFT4rYfzfwC3WeG5mEHfHa1HE4ivRLRSF8n+zTjYWc WJyjnbYjIUGNcMgBrJSFmDb5gLN3HkiurheFFkAwT05p5MyetzEJBJG8V/7tAX+I7dop0jCYmmp Wrwr7mGaQIzzDrw8I3JhK120TVPTtZOFFCs+sZYMA7C0sLff0VDrQNx405o3qWBMS4/UB+hNOAx LS16VdKigEemoyK1Yn2g83vVEBJIOM2V0YcU/h5T0IQPT1ZxdpJbG76tYaL7sr9I1DNogGp2f8D I+bhKJmAxcfFgQfDNY03srnYrIKFZvuCi4QAldGXgF7N05Uo2oIlk+DNDGCwUyjpkPlRa72LFZe upOzJdrZ1fNNLn84+lsQfmdxk= X-Received: by 2002:a17:903:4405:b0:2ca:ca48:c380 with SMTP id d9443c01a7336-2cf34a07a97mr145241885ad.47.1784539913858; Mon, 20 Jul 2026 02:31:53 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.31.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:31:53 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 01/17] mm/sparse: relax struct mem_section size constraints Date: Mon, 20 Jul 2026 17:31:11 +0800 Message-ID: <20260720093127.540540-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" struct mem_section is currently forced to a power-of-2 size so the section-to-root lookup can use a mask instead of a modulo. That requirement makes future extensions harder than necessary: adding a small field can require configuration-dependent padding or layout checks just to preserve the lookup scheme. Keep the lookup correct for any struct mem_section size by using a plain modulo instead. Do not leave the layout entirely unconstrained, though. Keep struct mem_section double-word aligned so modest size changes, such as adding another word-sized field on 64-bit systems, still keep a compact and efficient layout. If future fields grow the structure beyond that sweet spot, the lookup remains correct; only the exact layout efficiency changes. Signed-off-by: Muchun Song --- v2: - Align struct mem_section to power-of-2 where possible, but support non-power-of-2 lookup (suggested by David Laight) --- include/linux/mmzone.h | 11 +++-------- mm/sparse.c | 2 -- scripts/gdb/linux/mm.py | 6 ++---- 3 files changed, 5 insertions(+), 14 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 0507193b3ae3..04f3409edacb 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2016,13 +2016,9 @@ struct mem_section { * section. (see page_ext.h about this.) */ struct page_ext *page_ext; - unsigned long pad; #endif - /* - * WARNING: mem_section must be a power-of-2 in size for the - * calculation and use of SECTION_ROOT_MASK to make sense. - */ -}; +/* Sacrifice minor padding space for efficient lookup. */ +} __aligned(2 * sizeof(unsigned long)); =20 #ifdef CONFIG_SPARSEMEM_EXTREME #define SECTIONS_PER_ROOT (PAGE_SIZE / sizeof (struct mem_section)) @@ -2032,7 +2028,6 @@ struct mem_section { =20 #define SECTION_NR_TO_ROOT(sec) ((sec) / SECTIONS_PER_ROOT) #define NR_SECTION_ROOTS DIV_ROUND_UP(NR_MEM_SECTIONS, SECTIONS_PER_ROOT) -#define SECTION_ROOT_MASK (SECTIONS_PER_ROOT - 1) =20 #ifdef CONFIG_SPARSEMEM_EXTREME extern struct mem_section **mem_section; @@ -2056,7 +2051,7 @@ static inline struct mem_section *__nr_to_section(uns= igned long nr) if (!mem_section || !mem_section[root]) return NULL; #endif - return &mem_section[root][nr & SECTION_ROOT_MASK]; + return &mem_section[root][nr % SECTIONS_PER_ROOT]; } extern size_t mem_section_usage_size(void); =20 diff --git a/mm/sparse.c b/mm/sparse.c index 704a9dec2b9a..ca9875f568d3 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -332,8 +332,6 @@ void __init sparse_init(void) unsigned long pnum_end, pnum_begin, map_count =3D 1; int nid_begin; =20 - /* see include/linux/mmzone.h 'struct mem_section' definition */ - BUILD_BUG_ON(!is_power_of_2(sizeof(struct mem_section))); memblocks_present(); =20 if (compound_info_has_mask()) { diff --git a/scripts/gdb/linux/mm.py b/scripts/gdb/linux/mm.py index dffadccbb01d..da4e8e9655a6 100644 --- a/scripts/gdb/linux/mm.py +++ b/scripts/gdb/linux/mm.py @@ -70,7 +70,6 @@ class x86_page_ops(): self.SECTIONS_PER_ROOT =3D 1 =20 self.NR_SECTION_ROOTS =3D DIV_ROUND_UP(self.NR_MEM_SECTIONS, self.= SECTIONS_PER_ROOT) - self.SECTION_ROOT_MASK =3D self.SECTIONS_PER_ROOT - 1 =20 try: self.SECTION_HAS_MEM_MAP =3D 1 << int(gdb.parse_and_eval('SECT= ION_HAS_MEM_MAP_BIT')) @@ -100,7 +99,7 @@ class x86_page_ops(): def __nr_to_section(self, nr): root =3D self.SECTION_NR_TO_ROOT(nr) mem_section =3D gdb.parse_and_eval("mem_section") - return mem_section[root][nr & self.SECTION_ROOT_MASK] + return mem_section[root][nr % self.SECTIONS_PER_ROOT] =20 def pfn_to_section_nr(self, pfn): return pfn >> self.PFN_SECTION_SHIFT @@ -249,7 +248,6 @@ class aarch64_page_ops(): self.SECTIONS_PER_ROOT =3D 1 =20 self.NR_SECTION_ROOTS =3D DIV_ROUND_UP(self.NR_MEM_SECTIONS, self.= SECTIONS_PER_ROOT) - self.SECTION_ROOT_MASK =3D self.SECTIONS_PER_ROOT - 1 self.SUBSECTION_SHIFT =3D 21 self.SEBSECTION_SIZE =3D 1 << self.SUBSECTION_SHIFT self.PFN_SUBSECTION_SHIFT =3D self.SUBSECTION_SHIFT - self.PAGE_SH= IFT @@ -304,7 +302,7 @@ class aarch64_page_ops(): def __nr_to_section(self, nr): root =3D self.SECTION_NR_TO_ROOT(nr) mem_section =3D gdb.parse_and_eval("mem_section") - return mem_section[root][nr & self.SECTION_ROOT_MASK] + return mem_section[root][nr % self.SECTIONS_PER_ROOT] =20 def pfn_to_section_nr(self, pfn): return pfn >> self.PFN_SECTION_SHIFT --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E19533D6682 for ; Mon, 20 Jul 2026 09:31:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539920; cv=none; b=LzMtHPJj3dEc3BGWlh9I9BdyWBT0zBhv7bMl9hpuJVVNJcJpIpk1UVPKibEw075HrCwdR0AF9Fbw0qvOyKmmd7PZp25N284mceuR3y5j0jH4/T86d2kb7MJfHufnNbxlXmPwlw+OzvKpl9QrXazfkK+owZI9kks3EEFjQwWDC1g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539920; c=relaxed/simple; bh=cNl+6VBOHIoFsvdJJSSS0tC18U6xUXLvtG1I6EfSYtU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sA9P0/6WV7SBwksG0qtXuexQWQqW0Ue9+eQFQMrwpmTZ8lIcP4Js8GeSYNyKqS7bF6vvLAPu5BgoWbsZ/pmOPL792nxyxz6cPfWuuFrNIUYHiwOjHkBuy+pJVYox5SngFZYYs5+C9R+GzmCgkpG9sgcOLHEQQ+6VmHlI9SoF86E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=cjYrc9mu; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="cjYrc9mu" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2cacf197759so107867315ad.2 for ; Mon, 20 Jul 2026 02:31:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539918; x=1785144718; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ZFSAOL0DZsiwYlqzPouS8XnC2DPIgfqzINxJ7kc1Vuo=; b=cjYrc9mu/jj3pFbe/trmWYmZ4xSs2Nyvzsk2GzZUfJ62aKOllOp/iN+5TlyUq4+aLA sInQqQP3Nv8KsV72gGq2sxOTUDpSUYIWXOegkfUQnI8U5Bv9+PJfXVG3SLEjPCTDWSBm yXl4EQvVvq3Qt5yUlXcKDLXRtJxdU1T65w9gvlnRxn7ed+IbZ8fXdxEMLgo9mVcCbOF1 RvHXiNkeq3OUJRF4qHcgoXx7OPGUdhAraWfDV5Q+ZQdPYhQL/LwhUiQxt383IgAPCBM0 LCEyTbupy6f6iV6MQTlmjvcNa1NRYH3ZsOOFhTd8Az3ASn6NsCvUXLB6Gx5QycKTKLN4 iBmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539918; x=1785144718; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ZFSAOL0DZsiwYlqzPouS8XnC2DPIgfqzINxJ7kc1Vuo=; b=Ryl/h+km4Kn46fkpScHMKLaIfcp6Pc0SjtHNqEZsxqz0/HEnPCWlYi3Xmm6NLpbBFt aLyPtuwLKB2u9QCE/Q57/aL23KE5m4cVuv4wW5uVUXLktc/XJDOFLBnSiQ2PrI+ASb26 0EvREd3bQLN6alTH1GKpfDd3xjDPuuqOV5IHBsPVJW9+m1MGT7a+FEZP3d8JkWUJHCb/ BeefSAvooNq9bIRJrSGEs7CSouYV21a4011EZL5raqLuN83QbBfRJPkqSSbeX1eNsB5r J6xQrr8UBPJCRf76+yCSPG7shMJRp14B12DdBuntlwISPHXrVxslQC6QTeSboULTv1Yu blbQ== X-Forwarded-Encrypted: i=1; AHgh+Ros7Sq7jSuxH+RaxxrczFbM/6sHLGBAZpkn+Lm5HhRpfPi/faXWcphFKFX/2QP8zygl49vGfCQWM1buE34=@vger.kernel.org X-Gm-Message-State: AOJu0Yziwd/wVWeHTmzPiU7d0uE8NWshn486qJ9oWT5FthEYBNx5P7Hk F0wC2GQtz3HCYXF6QDsCCFuTsRLrjrNYn+r5TyTDP3CoxAfHRb8MBxZ4nAi9VPKK0cY= X-Gm-Gg: AfdE7ckuv1jyI1rcU2HAU0LtSwyTvf+PSVnX3HnQk+J2kC8cm0jSnoqHHMCQKx7OYMy OiMoSUQjLROpIb1AfOTP/j62rg3VMHCYmVTzILVEe0r3XJR34spPeegrye7d1PJhTAD5+1TgnqQ 9RSDCWdPRIdUPU6DcKvuxxk8/W9DVPOQ/YEcZWLevufzG9DC3685VXB40ZI6l7g44yVgwZOVgGT g4l5Ko5d1YRS7hbsa/9eaE7pAQ0VYwYe8Hu0IMeIu2kqbpaZtazkFV1aI2flLcFKA8qc/bvz2F2 8XRfFlsaN/45Y1Plxsro9B21JOb74ggsfefMoHxYsjSBaa0iM9BKgS5dXsHuTaO+4BTH86XA6q8 tVu+qbaqDHiMvy/xoxlPHWAYS9+FQvgj1XMAnTxvr6jS5Nfh0PfZCi8sRUBKXHXC7KDr5eA0Goy Mwiz8Oy8Pjd2fdUWTEZQxRhmoKUjDCEBvJkg== X-Received: by 2002:a17:902:e748:b0:2c1:98b7:ecf3 with SMTP id d9443c01a7336-2cf3494aebdmr144468675ad.23.1784539918154; Mon, 20 Jul 2026 02:31:58 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.31.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:31:57 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 02/17] mm/sparse-vmemmap: rename HVO order macros Date: Mon, 20 Jul 2026 17:31:12 +0800 Message-ID: <20260720093127.540540-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The macros VMEMMAP_TAIL_MIN_ORDER and NR_VMEMMAP_TAILS describe the order range where HVO can be applied, but their names tie that range to the tail-page cache implementation. Rename them with a VMEMMAP_OPTIMIZATION prefix and use the new names in the HVO paths. This makes the code describe the optimization requirements rather than the tail-page cache implementation detail. No functional change intended. Signed-off-by: Muchun Song --- v2: - Rename the macros with a VMEMMAP_OPTIMIZATION prefix (suggested by Mike Rapoport) - Drop intermediate optimized-folio size macros (suggested by Mike Rapoport) --- include/linux/mmzone.h | 17 +++++++++-------- mm/hugetlb.c | 4 ++-- mm/hugetlb_vmemmap.c | 2 +- mm/sparse-vmemmap.c | 4 ++-- 4 files changed, 14 insertions(+), 13 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 04f3409edacb..82b0155d886f 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -107,13 +107,14 @@ is_power_of_2(sizeof(struct page)) ? \ MAX_FOLIO_NR_PAGES * sizeof(struct page) : 0) =20 -/* - * vmemmap optimization (like HVO) is only possible for page orders that f= ill - * two or more pages with struct pages. - */ -#define VMEMMAP_TAIL_MIN_ORDER (ilog2(2 * PAGE_SIZE / sizeof(struct page))) -#define __NR_VMEMMAP_TAILS (MAX_FOLIO_ORDER - VMEMMAP_TAIL_MIN_ORDER + 1) -#define NR_VMEMMAP_TAILS (__NR_VMEMMAP_TAILS > 0 ? __NR_VMEMMAP_TAILS : 0) +/* The number of struct pages covered by the retained vmemmap pages with H= VO enabled. */ +#define VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES (PAGE_SIZE / sizeof(struct pa= ge)) +#define VMEMMAP_OPTIMIZATION_MIN_ORDER (ilog2(VMEMMAP_OPTIMIZATION_NR_STR= UCT_PAGES) + 1) + +#define __VMEMMAP_OPTIMIZATION_NR_ORDERS \ + (MAX_FOLIO_ORDER - VMEMMAP_OPTIMIZATION_MIN_ORDER + 1) +#define VMEMMAP_OPTIMIZATION_NR_ORDERS \ + (__VMEMMAP_OPTIMIZATION_NR_ORDERS > 0 ? __VMEMMAP_OPTIMIZATION_NR_ORDERS = : 0) =20 enum migratetype { MIGRATE_UNMOVABLE, @@ -1147,7 +1148,7 @@ struct zone { atomic_long_t vm_stat[NR_VM_ZONE_STAT_ITEMS]; atomic_long_t vm_numa_event[NR_VM_NUMA_EVENT_ITEMS]; #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP - struct page *vmemmap_tails[NR_VMEMMAP_TAILS]; + struct page *vmemmap_tails[VMEMMAP_OPTIMIZATION_NR_ORDERS]; #endif } ____cacheline_internodealigned_in_smp; =20 diff --git a/mm/hugetlb.c b/mm/hugetlb.c index db09f0ab6d97..b8e1569bda15 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3358,7 +3358,7 @@ void __init hugetlb_bootmem_struct_page_init(void) struct zone *zone; =20 for_each_zone(zone) { - for (int i =3D 0; i < NR_VMEMMAP_TAILS; i++) { + for (int i =3D 0; i < VMEMMAP_OPTIMIZATION_NR_ORDERS; i++) { struct page *tail, *p; unsigned int order; =20 @@ -3366,7 +3366,7 @@ void __init hugetlb_bootmem_struct_page_init(void) if (!tail) continue; =20 - order =3D i + VMEMMAP_TAIL_MIN_ORDER; + order =3D i + VMEMMAP_OPTIMIZATION_MIN_ORDER; p =3D page_to_virt(tail); /* * prep_and_add_bootmem_folios() can access pageblock diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 917db0984143..ae8fdaa42118 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -494,7 +494,7 @@ static bool vmemmap_should_optimize_folio(const struct = hstate *h, struct folio * =20 static struct page *vmemmap_get_tail(unsigned int order, struct zone *zone) { - const unsigned int idx =3D order - VMEMMAP_TAIL_MIN_ORDER; + const unsigned int idx =3D order - VMEMMAP_OPTIMIZATION_MIN_ORDER; struct page *tail, *p; int node =3D zone_to_nid(zone); =20 diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 5a2469fb1838..aa6a4a2fae98 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -329,12 +329,12 @@ static __meminit struct page *vmemmap_get_tail(unsign= ed int order, struct zone * unsigned int idx; int node =3D zone_to_nid(zone); =20 - if (WARN_ON_ONCE(order < VMEMMAP_TAIL_MIN_ORDER)) + if (WARN_ON_ONCE(order < VMEMMAP_OPTIMIZATION_MIN_ORDER)) return NULL; if (WARN_ON_ONCE(order > MAX_FOLIO_ORDER)) return NULL; =20 - idx =3D order - VMEMMAP_TAIL_MIN_ORDER; + idx =3D order - VMEMMAP_OPTIMIZATION_MIN_ORDER; tail =3D zone->vmemmap_tails[idx]; if (tail) return tail; --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D3593D7A18 for ; Mon, 20 Jul 2026 09:32:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539925; cv=none; b=qnuvUnElEB62weQJWCmYMi0TcH7Gha6qAXRNJgElMk5wGu1PLs5wyvR42wS9+df6xhVzjiM/k9nnUtlTbAMQ+rQQMbnxCcZZhhcP9G/bAQlykEVcz6D+62lbFACKHjO9zJaabw9Ne9G6OFsmknXfaYIcNp8MhikkS3JAkIauqFg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539925; c=relaxed/simple; bh=UlVatc8sakduVZEP4OOQmq1leBTxdEvteeXD1rofskI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XIaeyvhZ6EteNHZnV5QDGP40gUj13Rm2j/jA4OiX34HBNtkxAzgHZd/GjQof3l2s/Fyz8165hPeC9+JtPOuZf7ii765NfXg/41vdf9vShz0E3KP209Yoy16j9EgabhK/A52Pn2/VEg5fjl1UVRqa6P0IFaL0WLBMIRKtE7SSSms= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=NhzUcQfk; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="NhzUcQfk" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2cf6d65d8a7so7781375ad.0 for ; Mon, 20 Jul 2026 02:32:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539923; x=1785144723; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=T/mMY7wFDptG7mxg7a2Zuzc5cMIttYTPNMUojd8S+fU=; b=NhzUcQfklXJlSf+NWPBID41peYOD9S12DCg6uxdhsYQ6RAfPtbC1cO2VWD/a+/DYYD EQeYbgVpyxka10hbJnl+2o2+VI+YsfGRwXjSQV37wy/KNSlqHxxwwkba4sLJ7Eat7+HL tXie9/bROW3MuUOrIymDAMalHF1SBmcHfBuv/rdctr52AdEcTQqqeT9jicrhj2EUbRBv 8YpZrsI9l3HEGM70HIFapgK6G7khbgnj0gXwWh/yC5L9ReouUMj3MNPYuiTAe7swp0J7 FfPCxGiz+eqOjbeGCEJYRux9RQzAf7m7TXojN+4l0kd/CpOSB54sC8k8Tv/gEIM9Awca k+kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539923; x=1785144723; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=T/mMY7wFDptG7mxg7a2Zuzc5cMIttYTPNMUojd8S+fU=; b=dlI6mNiLfTdNjqKaUo4Ra4xXURMimz010sq5Njdn53cC/GouW5bcg+b6ugZAK2sB6Z IQCX44LLVhydy4rzV5cbPmZjfS5vYSVebCOmZd7npjASzduWnhjTSFli2/KInjTnhT2l 0/2PDyy4nshwIN9EltHWXdvq4Ro8p+c1PKGE55jZrCrHlpIunTqblBVa+KVmsvou7J26 8vkMz0GgxOhQGKBvy2ttCIBkgdi+b8h0A//aTFH6GsLSV401lIbN6a2tOcdqK6tna/cs mW7svrjrGcGT92ooJZsG3FihjEddo1neAR4HfGSizI+1DVo0fIOxfT22Mlf6BumhL4JU ql7w== X-Forwarded-Encrypted: i=1; AHgh+RrnUQpYWCk6xih93z3Sz3eOBdPez17zoCcN2koyCR0JBKruOd2jUuGLk89g4FBBDjP++UjqzZVPLjqpbzg=@vger.kernel.org X-Gm-Message-State: AOJu0YweKPog4GvBmaLnC6z3ITCwrMfu70Bnh3E1p94RgEf1rIUeP49X jv7pkKJM2M2ruU6gwPhEEZjmbjWFWHC+rcXz9WZbzsvH+dj8nYlOb/P/o5vWXyrk/YE= X-Gm-Gg: AfdE7cnkr+DYXZDve7vdLXPr6cnaqgemX7cEYbA221HDj6+D2HjIw8hkbCRq+qmvUmj WTErn1v5w36oZ+dpMz5ha+QN+JMydIc5IsKPuDbQG9vFs8zAPCy9ajqivAmDHJhCFg9nTmPht6K UgjW+GCrvnClelM1RP2JTQV7aY7Oz0DjQJhyBLvrNIRfUY8BK30nCWsMdti0RY2NIogDNCBUFkS q3OPYWr8rn83EuxTpKzevN4mOSRZoYTlPtKi+Wv91Y7Lq69LKs5tCu3FKbJYATi/HF4GmRRZygC FYoIhOIgQheZ1ypvfCmLtFmspUinTFniPCKrDlBpZnEtMZC1Eug9O2mAeXlWBbcnQ/Nqp8pIagf UkjH7AvwqUcsZ/mARmZXfsu3UK3d18JoZaWjVinDJ6zU9F6S3G/uixLqcKzzXPe4PSUmiD4fqwn YkpvOUmB91B52rPmhWSDtyon0rr6DfQGeqbg== X-Received: by 2002:a17:903:41c3:b0:2c7:1200:bd7e with SMTP id d9443c01a7336-2cf3486b0b1mr144824815ad.18.1784539922678; Mon, 20 Jul 2026 02:32:02 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.31.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:01 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 03/17] mm/mm_init: skip initializing shared vmemmap tail pages Date: Mon, 20 Jul 2026 17:31:13 +0800 Message-ID: <20260720093127.540540-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" memmap_init_range() initializes every struct page in the target range. For compound pages with vmemmap optimization, the tail struct pages are backed by a shared vmemmap page. Initializing those tail struct pages would overwrite the shared vmemmap page contents, requiring users such as HugeTLB to restore the metadata afterwards. Track the compound order for HVO-backed sections and use that metadata to detect struct pages that fall into the shared tail vmemmap range. Skip those shared tail pages in memmap_init_range(), then initialize pageblock migratetypes for the processed range with a helper after the per-page initialization loop. The !SPARSEMEM __pfn_to_section() stub is needed only for the build: memmap_init_range() references __pfn_to_section() after checking pfn_vmemmap_optimizable(), and !SPARSEMEM builds still have to compile that code even though pfn_vmemmap_optimizable() folds to false. This is a preparatory change for consolidating handling across users of vmemmap optimization, and it also avoids redundant initialization of shared tail vmemmap pages during early boot. Signed-off-by: Muchun Song --- v2: - Fold section order tracking into the first user instead of keeping a standalone API-only patch (suggested by Mike Rapoport) - Rename page_vmemmap_optimizable() to pfn_vmemmap_optimizable() and pass a PFN directly (suggested by Mike Rapoport) - Initialize pageblock migratetypes from a helper after the per-page loop (suggested by Mike Rapoport) - Use a 1G PFN chunk for cond_resched() in the pageblock helper (suggested by Mike Rapoport) - Guard section_order() with CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP so it returns 0 when HVO is disabled and lets the compiler optimize the code as much as possible (suggested by Mike Rapoport) - Explain why the !SPARSEMEM __pfn_to_section() stub belongs here (suggested by Mike Rapoport) --- include/linux/mmzone.h | 14 ++++++++++++++ mm/mm_init.c | 33 +++++++++++++++++---------------- mm/sparse.h | 23 +++++++++++++++++++++++ 3 files changed, 54 insertions(+), 16 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 82b0155d886f..2a32101d55e6 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2011,6 +2011,14 @@ struct mem_section { unsigned long section_mem_map; =20 struct mem_section_usage *usage; +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP + /* + * Normally, sections hold regular (order-0) pages. However, for + * sections with HVO enabled, this tracks the compound page order + * to enable deduplication of redundant vmemmap pages. + */ + unsigned int order; +#endif #ifdef CONFIG_PAGE_EXTENSION /* * If SPARSEMEM, pgdat doesn't have page_ext pointer. We use @@ -2365,8 +2373,14 @@ static inline unsigned long next_present_section_nr(= unsigned long section_nr) #endif =20 #else +struct mem_section; + #define sparse_vmemmap_init_nid_early(_nid) do {} while (0) #define pfn_in_present_section pfn_valid +static inline struct mem_section *__pfn_to_section(unsigned long pfn) +{ + return NULL; +} #endif /* CONFIG_SPARSEMEM */ =20 /* diff --git a/mm/mm_init.c b/mm/mm_init.c index ddda9d6837f3..dcb757b36902 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -677,19 +678,17 @@ static inline void fixup_hashdist(void) static inline void fixup_hashdist(void) {} #endif /* CONFIG_NUMA */ =20 -#if defined(CONFIG_ZONE_DEVICE) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INI= T) static __meminit void pageblock_migratetype_init_range(unsigned long pfn, - unsigned long nr_pages, int migratetype, bool atomic) + unsigned long nr_pages, int migratetype, bool isolate, bool atomic) { const unsigned long end =3D pfn + nr_pages; =20 for (pfn =3D pageblock_align(pfn); pfn < end; pfn +=3D pageblock_nr_pages= ) { - init_pageblock_migratetype(pfn_to_page(pfn), migratetype, false); - if (!atomic && IS_ALIGNED(pfn, PAGES_PER_SECTION)) + init_pageblock_migratetype(pfn_to_page(pfn), migratetype, isolate); + if (!atomic && IS_ALIGNED(pfn, PFN_DOWN(SZ_1G))) cond_resched(); } } -#endif =20 #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT static inline void pgdat_set_deferred_range(pg_data_t *pgdat) @@ -884,6 +883,13 @@ void __meminit memmap_init_range(unsigned long size, i= nt nid, unsigned long zone } } =20 + if (pfn_vmemmap_optimizable(pfn)) { + unsigned int order =3D section_order(__pfn_to_section(pfn)); + + pfn =3D min(ALIGN(pfn, 1UL << order), end_pfn); + continue; + } + page =3D pfn_to_page(pfn); __init_single_page(page, pfn, zone, nid); if (context =3D=3D MEMINIT_HOTPLUG) { @@ -895,18 +901,13 @@ void __meminit memmap_init_range(unsigned long size, = int nid, unsigned long zone __SetPageOffline(page); } =20 - /* - * Usually, we want to mark the pageblock MIGRATE_MOVABLE, - * such that unmovable allocations won't be scattered all - * over the place during system boot. - */ - if (pageblock_aligned(pfn)) { - init_pageblock_migratetype(page, migratetype, - isolate_pageblock); + if (pageblock_aligned(pfn)) cond_resched(); - } pfn++; } + + pageblock_migratetype_init_range(start_pfn, pfn - start_pfn, migratetype, + isolate_pageblock, false); } =20 static void __init memmap_init_zone_range(struct zone *zone, @@ -1111,7 +1112,7 @@ void __ref memmap_init_zone_device(struct zone *zone, compound_nr_pages(pfn, altmap, pgmap)); } =20 - pageblock_migratetype_init_range(start_pfn, nr_pages, MIGRATE_MOVABLE, fa= lse); + pageblock_migratetype_init_range(start_pfn, nr_pages, MIGRATE_MOVABLE, fa= lse, false); =20 pr_debug("%s initialised %lu pages in %ums\n", __func__, nr_pages, jiffies_to_msecs(jiffies - start)); @@ -1923,7 +1924,7 @@ static void __init deferred_free_pages(unsigned long = pfn, if (!nr_pages) return; =20 - pageblock_migratetype_init_range(pfn, nr_pages, mt, true); + pageblock_migratetype_init_range(pfn, nr_pages, mt, false, true); =20 page =3D pfn_to_page(pfn); =20 diff --git a/mm/sparse.h b/mm/sparse.h index 95aa031213f2..030248030dc7 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -51,6 +51,29 @@ static inline void __section_mark_present(struct mem_sec= tion *ms, static inline void sparse_init(void) {} #endif /* CONFIG_SPARSEMEM */ =20 +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP +static inline unsigned int section_order(const struct mem_section *section) +{ + return section->order; +} +#else +static inline unsigned int section_order(const struct mem_section *section) +{ + return 0; +} +#endif + +static inline bool pfn_vmemmap_optimizable(unsigned long pfn) +{ + const unsigned int order =3D section_order(__pfn_to_section(pfn)); + const unsigned long nr_pages =3D 1UL << order; + + if (!is_power_of_2(sizeof(struct page))) + return false; + + return (pfn & (nr_pages - 1)) >=3D VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES; +} + /* * mm/sparse-vmemmap.c */ --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25B6E3D812C for ; Mon, 20 Jul 2026 09:32:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539929; cv=none; b=uZUzgfvEGY/WUQ+N1tvwi/i0jHs5ighIBpbMSn8yjhnuHllewBFoXfUx7eksMLCic5+iwWUqRWf1a9S5Ux/POWNIg2uwkvL1gQ5ExWvY5SGENgjm8+xquIKhJvlQD6OorEzbwUeIfAUHRjY7EOY4pKNZOWqvbMf9cD7nie+OL8U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539929; c=relaxed/simple; bh=owQ+Elu7FhHy0RmBaGNrJ1F4/GpIAVuOEqAZSW69+JA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pptPN5okqjYGRBWJhlXDG74QfSICSuNH+iDdcuJwduSl+VBRXSb3rUrPrndSqTijb+8jRc3Ls/F7NlzbzMtHqS7qGiDAoBYZQ7p4g43NTlKsDZbyBEu2JbJ5Um1ZbQN9185ANkqyZ7vBumBkHpbaUDKwyjaA6k8XKoNLryEGfck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=RKDYNREl; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="RKDYNREl" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2cc7e86e7aeso70230535ad.2 for ; Mon, 20 Jul 2026 02:32:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539927; x=1785144727; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=bOzdkIPPNoupexe8+AVOrQT3DBRmXPJ6Kybkle+RLi4=; b=RKDYNREla1uG37/wf44pFvvzb2W/jzJpro7wPYewaJDT8M6mgu0NTEKP159QkbVwKa NEzmwInMn88rWTE7pxHAAzPc+Aq3rphoJPBqqprir9kBgdjx7ezpcfYfUvs+hHSaMqJV e0HOOcKaIzPVHih93OZiIyeH95QDwZzfevYgrL7dlQdoA2FxRO3Eiff1WQ3zAAh/eofh 6AENa/5MwpAqHWZ+yCt6gYN+mQlmuTrUB9jCXczwllGHlv3CQFUmhdSlcy0MdIuxbyVm iqcg02IrnU3Rr0xeLYtn+QNfGM4c6Vyz/IaoexIVKyCgTO1nB8zHfx1pu0a0y1jLKqit WQfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539927; x=1785144727; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=bOzdkIPPNoupexe8+AVOrQT3DBRmXPJ6Kybkle+RLi4=; b=tJM3R36UU8C4wpAKA2HiC5aPoAEo/GOQiGMedd7zIp07n9uzG1UaQhg4Xjb3EuFpfb 2KNb9oCgCCFnzOS0b53qkIxoeapbNsOYaCgPUmK8owphR/10FfB4AkBH7KJ0FhHwd+sx kiqr7nyOYLNdp5wV91aW+U2LDJ7v5+Enio19ktGC4HrpQ6THiprhOL5fGE6DokDuCZtY ZHJeWbzbsxt/qDWSBbqaRP6Dz5wQ6KBlhym5O3sPsZYbKUtfdRuWB/PAQi4AYWwJp3UR a/hsjinvW2AogBgGpI+UOaAKdEBjwAllgsueLn7DkOzDmrnB6trH5GLzbKeCGexvpOo2 5/ew== X-Forwarded-Encrypted: i=1; AHgh+RpMLk/H2sBtyTJwETRIJRZ11VYPRpVQic9H49dkbwn6NvadLUIdkbanBJ8TNgGoXNIA+4sOzfVQz1tMhZs=@vger.kernel.org X-Gm-Message-State: AOJu0YzGzG6UxENgazz2tGZ53g92IHzowRBUJbxU8GE2fMnF0ty5YC9j AsN8soIsqDlToawsN5bh/gZzLG4rTO9ZjvxzJjYFoUNNOJ2rB6HChQqueWfcNBR2pH0= X-Gm-Gg: AfdE7cm0nWZL5UneXw6ManDQj3NIM2i91A7biO5mBhOoSs6d46Zqvs79sHCO9gylfwY qJRe6QMV4qsjrFWu/oP9tN6EOoFM3+fNqHYV/KxrxKNFOGPGAKoledZjmzocBrkEKoTlG/Y930k vEEOjzdeqRoTrUDBoP3lW5lkj+9/uCyS+JYjbnZ2wbceq7P/gdvx03vaux69mrfJ75DQcWuqp+m WjxqCSoLAzqq6Z+WYWWcaNkDAVYPYSveF44Z4Tf7gLwoYFoeQidRDNk3U1jczkURwl9BgVgoomS zTZqiLOLO/Jn7qKH7yrhcJKDGrOkWAy+KtmVruvznzpEaW+fV7NExQvBX13j9GuYXX33eambl3j 9QfMMvT+JGWeblRaiCh5SQarmapyiFQZXlmt5r9w0DNvsdUe1GXdHk9A/w94eZg8Ji1y3H4Tv0F 5/y4boP8tHNKNuRXWPc80yGxw= X-Received: by 2002:a17:903:2a8b:b0:2ca:ddbd:a19b with SMTP id d9443c01a7336-2cf34997827mr147416145ad.28.1784539927267; Mon, 20 Jul 2026 02:32:07 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:06 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 04/17] mm/sparse-vmemmap: initialize shared tail vmemmap pages on allocation Date: Mon, 20 Jul 2026 17:31:14 +0800 Message-ID: <20260720093127.540540-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The shared tail vmemmap page allocated in vmemmap_get_tail() used to be left uninitialized, because memmap_init_range() would later overwrite it. That forced users such as HugeTLB to defer the initialization to their own setup paths. Now that memmap_init_range() skips shared tail vmemmap pages, initialize them immediately in vmemmap_get_tail() with init_compound_tail() instead. This moves the initialization to the point where the shared tail page is allocated and avoids relying on deferred handling in individual users. The remaining deferred initialization in HugeTLB will be removed once it switches to the section-based vmemmap optimization mechanism. Signed-off-by: Muchun Song --- mm/sparse-vmemmap.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index aa6a4a2fae98..107215cf8488 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -338,19 +338,11 @@ static __meminit struct page *vmemmap_get_tail(unsign= ed int order, struct zone * tail =3D zone->vmemmap_tails[idx]; if (tail) return tail; - - /* - * Only allocate the page, but do not initialize it. - * - * Any initialization done here will be overwritten by memmap_init(). - * - * hugetlb_bootmem_struct_page_init() will take care of initialization - * after memmap_init(). - */ - p =3D vmemmap_alloc_block_zero(PAGE_SIZE, node); if (!p) return NULL; + for (int i =3D 0; i < PAGE_SIZE / sizeof(struct page); i++) + init_compound_tail(p + i, NULL, order, zone); =20 tail =3D virt_to_page(p); zone->vmemmap_tails[idx] =3D tail; --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E1C13D45F4 for ; Mon, 20 Jul 2026 09:32:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539933; cv=none; b=H28hI1CGt0qkNGG434Wwfa4fTkMGmm7ccpR9FoW/R9pBtk6puBMk1dEFMhB4wZeMovK2WOB4+vk/Xx4kyI7x1W2lMrFAJY6gWkDYk0GyzYTghTySAFHuesfqJHO8tFyTP164sCx5JPBcaLyG9BbwQiuwfyqpoRZF/mqDDaSkRFU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539933; c=relaxed/simple; bh=0FwcDnLTwW838J36dxBTO8HnjB/9Fk3bHuVdNDrHNOU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Inj04KCZr8GWpaKi2wG3oQqWE5AnMf2thV3XLEvRgvbYoQHtNaF0nDa1CTyIYdLAjaTAyBx/ru5VxTiiufr+Q3YPphz5ZTIs/Xnvee7rbMAND6hQFUUnSWAcnVHkLvBOoekIhmaonoYVFADc6zFAw0J+Gjp1GHsJ+IR1DOIPOdA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=W4UT3Fto; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="W4UT3Fto" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2ce7d2adef4so94149755ad.3 for ; Mon, 20 Jul 2026 02:32:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539931; x=1785144731; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=s6zsu4ImsI5QIEzL19H+gxsmGfblKWRGuAokU/tHqk0=; b=W4UT3Fto9AzzhRiFeQKezJ/F3fEb9+O5Y9LBfCmctgBs973/6oGy+YvjWNCkHf4CL6 PSHvFjFGWs51HAMMKnCJwPOuv6/7bMzlBCuv9TZLC9R6hJb5GxCc3XOYzB0f4C7v8UPZ DpdiLzIPURF0+BKFbQ2r6ysY5KyVidhOoV+sEJSgcutgh/CKnY5dHIWlIlz5EbWKrPcG hY4Wk7lbXFhRRrVLyyyzuZbn24h4cMlCreFYpK536Lg7HSEkgv0kWhRw7Wdc40b7q7Dg mlGVnZIWapte156hif9sZm3VEvxaK/hAeTsHPdCGL254yP61kmJJ2BXr4YmZE4J2D5of 3Qsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539931; x=1785144731; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=s6zsu4ImsI5QIEzL19H+gxsmGfblKWRGuAokU/tHqk0=; b=HsDlPdXEF6hExHM9m/W54XXLUpYwPzCtOAfZegIbZjg/79hIvH1lM6MeldhC+khTld Jnx+dPISZXV9MVq0biYvVsv4XlMsGTHqqOyMutigPgAkPi/etSz2Q8FbPZIp5PDxnPR/ BeZ7iVjLYu+6nv/iMPS6GTZZL1fs4YlA4PQYoU1SmjQ3uix9ng/H7/ftsb1p3SlAhZe5 tzgo6K49PUKoqfBnQ3y1DUICZzkZKxqxZOEnHoTJDFDdw53lbSXbajxSlWViiZUGMb8l eg4yiJ+b9zxB8XNlZl48vKk5Nq9IFw+EKe7BRbPZOVaCF3E2nhsXOwZJ2RtrXFgtbzkU W1tw== X-Forwarded-Encrypted: i=1; AHgh+RrStn+6390EnlNayoQEZLSjFmNgEMMP2AMk+Eo2aK2TTdY7ZVhzfO0H2ybXs+f35nyoaAOfyiEc3NsHve8=@vger.kernel.org X-Gm-Message-State: AOJu0Yw+yH6J88xYNQebS6NrqZHj050PhMidCmRBbrXumZHeeuENoLHt tCmYJb4L2MO58sCpf7Cat6UbsdwGhZT22ZWmWmG3swTNhShUkAJ2UCFq/bODvXnptoc= X-Gm-Gg: AfdE7ck5PhGGf9UYKVgZy9+tl/MRU9J/r9U02vvjaAApgliZ/x3F5JfFy8nBHeD/2X2 eCbJM0tkMuFR4fDNbEDApVWQIlQzRAo/MyE/ETAmmXp7KjN51rSKdsKJF/0kkY8QkwKmV3zbm6Q XCvVEQlGuyjdBcp9AcFaJ20GdpJ12AKnqGRqEdceSp+nZB5El2wvYB0mzGibA9hpOO4dAEPG9iC MmbsIfw7okCMtiSugXOIZ3l50x897+ACZcXTTPjDu57MLosMqFgQxFAFJpHRizHSjmfw6d8SvzU dG87k8OyO6/bnTw4VcjO+30VdRhADnIKRlOwdMARQYjamAiHDj///gb6g24P36nFI4CGMmG7Jf0 P3RexLmVs0DQjazoL0HaSMgcoqWtxX60lMptuKoAF5rexz6gf8S0xUBSC2yE8IHGs45138WOZEX Bs2/M7xPjQuFGVsc6qeoFTjRY= X-Received: by 2002:a17:902:e80a:b0:2c9:fc14:588f with SMTP id d9443c01a7336-2cf3494afc3mr147908865ad.29.1784539931402; Mon, 20 Jul 2026 02:32:11 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:10 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 05/17] mm/sparse-vmemmap: support section-based vmemmap accounting Date: Mon, 20 Jul 2026 17:31:15 +0800 Message-ID: <20260720093127.540540-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" section_nr_vmemmap_pages() can account ordinary sections and DAX sections, but section-based vmemmap optimization keeps its compound order in struct mem_section and retains a different number of vmemmap pages. Teach section_nr_vmemmap_pages() to recognize section-based optimized sections and calculate their vmemmap page count from the section order and the HVO retained page count. Signed-off-by: Muchun Song --- Hi Mike, I reintroduced VMEMMAP_OPTIMIZATION_PAGES (the intermediate macro), which we discussed removing in v1, so the section-based HVO path doesn't have to hard-code its retained vmemmap page count (i.e. 1) in section_nr_vmemmap_pa= ges(). To better unify the codebase in the future, the new macro introduced in thi= s patch will also be applied to the HugeTLB code (e.g., to calculate HUGETLB_VMEMMAP_RESERVE_SIZE from this new macro as well). v2: - Remove an unnecessary vmemmap_can_optimize() call to simplify the code (suggested by Mike Rapoport). - Rewrite the commit message for better understanding. --- include/linux/mmzone.h | 6 ++++-- mm/sparse-vmemmap.c | 10 ++++++---- mm/sparse.h | 8 ++++++++ 3 files changed, 18 insertions(+), 6 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 2a32101d55e6..025ea69f57d8 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -107,8 +107,10 @@ is_power_of_2(sizeof(struct page)) ? \ MAX_FOLIO_NR_PAGES * sizeof(struct page) : 0) =20 -/* The number of struct pages covered by the retained vmemmap pages with H= VO enabled. */ -#define VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES (PAGE_SIZE / sizeof(struct pa= ge)) +/* The number of retained vmemmap pages with HVO enabled. */ +#define VMEMMAP_OPTIMIZATION_PAGES 1 +#define VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES \ + (VMEMMAP_OPTIMIZATION_PAGES * PAGE_SIZE / sizeof(struct page)) #define VMEMMAP_OPTIMIZATION_MIN_ORDER (ilog2(VMEMMAP_OPTIMIZATION_NR_STR= UCT_PAGES) + 1) =20 #define __VMEMMAP_OPTIMIZATION_NR_ORDERS \ diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 107215cf8488..b7abc5494bb9 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -649,24 +649,26 @@ void offline_mem_sections(unsigned long start_pfn, un= signed long end_pfn) static int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned = long nr_pages, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { - const unsigned int order =3D pgmap ? pgmap->vmemmap_shift : 0; + const struct mem_section *ms =3D __pfn_to_section(pfn); + const int order =3D pgmap ? pgmap->vmemmap_shift : section_order(ms); + const int vmemmap_pages =3D pgmap ? VMEMMAP_RESERVE_NR : VMEMMAP_OPTIMIZA= TION_PAGES; const unsigned long pages_per_compound =3D 1UL << order; =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SUBSECTION)); VM_WARN_ON_ONCE(nr_pages > PAGES_PER_SECTION); =20 - if (!vmemmap_can_optimize(altmap, pgmap)) + if (!vmemmap_can_optimize(altmap, pgmap) && !section_vmemmap_optimizable(= ms)) return DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE); =20 if (order < PFN_SECTION_SHIFT) { VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, pages_per_compound)); - return VMEMMAP_RESERVE_NR * nr_pages / pages_per_compound; + return vmemmap_pages * nr_pages / pages_per_compound; } =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SECTION)); =20 if (IS_ALIGNED(pfn, pages_per_compound)) - return VMEMMAP_RESERVE_NR; + return vmemmap_pages; =20 return 0; } diff --git a/mm/sparse.h b/mm/sparse.h index 030248030dc7..1e9fcea10181 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -74,6 +74,14 @@ static inline bool pfn_vmemmap_optimizable(unsigned long= pfn) return (pfn & (nr_pages - 1)) >=3D VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES; } =20 +static inline bool section_vmemmap_optimizable(const struct mem_section *s= ection) +{ + if (!is_power_of_2(sizeof(struct page))) + return false; + + return section_order(section) >=3D VMEMMAP_OPTIMIZATION_MIN_ORDER; +} + /* * mm/sparse-vmemmap.c */ --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DFE9F3D45F4 for ; Mon, 20 Jul 2026 09:32:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539938; cv=none; b=kxCKMt6yUnNLVUVzwcRXm0gKkg+z0LfVQpL4L5h4h0/tXp11ws7teay4aUhFBw4xkySmDRLVbKQZ7F8ji8Vw9eDX12tPv7ix8c/yx+pY8bNEsj+JkNQ/Sm2+f5J07Qhr0su82Ljv9F8FHh+ypjVBuRpUMa+2j8v7pMG3Ce2HLC8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539938; c=relaxed/simple; bh=ynRjGf/9kUBaaReEwOGUp45Yp7rWP1SzjJfyEbOWZy4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BN7D5XqRVbO8MIxgtHLicK5A0+6CiHB74KxM/WPFhPnSeT72mn/x/OJ02OVjQHaoG1co50wveSgeQUaVEXlT3AfzZttt+uSWDTzyupKtESwJenkCL9asnREOW9C96ZgrAE6sdQX2/rsxfEnLMaNMLzuteB9xWBTJqoWrQeW0zx4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=NuImI7QK; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="NuImI7QK" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2cad4170e8eso134031015ad.3 for ; Mon, 20 Jul 2026 02:32:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539936; x=1785144736; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=MkBZ2UA32uh7FYNyMozkAkSKVt/rRvYA4e3apZ5ZvVc=; b=NuImI7QKG+nH8eQnZUm9QfPsrU5GjvstNRMF1TcsWM9znlddzaJY1TQRW8xBDRtihd bq6wGAusyiSnEbSPFczb+Lz8mebwlJGuDfoBb9QmNdkuenJ1VvW6xfJ6kNkS8Px+epeS NNFMO/RCoeoB3ntiBtwOP1SpvbwSx2jpN5upNvkREpq9xv8hz7ox/MQrVK/ETCx6MYZV WvMiRytergaDQ6LevNQbMEbhZhXhLb2pGOLKOlVNIwatZ+cTPPJ3nauWoau/oLmXXQ7N tQPd6bsqvGi0V0C51JjjUEhYFpgEMeSxGPZVzcwcG47mr7Lpy+Cpt11+MRpQBfgs/1lU hCug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539936; x=1785144736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=MkBZ2UA32uh7FYNyMozkAkSKVt/rRvYA4e3apZ5ZvVc=; b=U3sw2yEeH4zIrmPdtz7ZKlfeVCSWLAUpGozZLyNYd0wJHITnOvHa+VFckvTvRetl9d nxX7kcRtlJZAy0jtP1uq8i6rGcRBLUsGiYD9OnDpLLmCENO8IxUg32atdDaoPx+694NK oAQHoFSKZMU2o9pLsi5zYkDnSEBACTPIEo0vXeLhQUl+nc/N5YkuiqqvJoVQielouJXY Yekb2QaROtjmGf2TdJBSxzNhmjZee4ZtLRUBJ5xanztoft1O+o1rIkOOlB6ZGcU8dJTC XJEH0lN2ImHyBwfHzI1d4lrHdvzEX7ACppA4TCENJFgh4VjPHicnq518fpbKksOQkqZh c+Mg== X-Forwarded-Encrypted: i=1; AHgh+RreJchO8FuAaTz1h71H/YQr2YtsylLSSiI8dtJjQfkZc8E9DNVArskJYdVdsFoGSj4QMOg8Ca1OL9bb+jA=@vger.kernel.org X-Gm-Message-State: AOJu0Yyb0gEh8eTD7YUGXYPbETVWd8h+gh6xWwtkuM25+goqMTuj7jzj j6D9U5Cq4uhZe/92bU7uofexDT1Mt/O0lBf4yJXjefQSuIub1cxqdJyRi4ws5ZpgYFA= X-Gm-Gg: AfdE7cly3hiGRsSpQ1CVusizl/foLB9UH38z45W0l7kvo2VvkoiB0vixCfwu9YBhqrd +eB+w82sGXYwjLKjVScHOW1UHgzn/BiX5Yf7NZ43lEWhQHHHJtuVHXSA+Pc/kn5P9rnYf15ZOJO 1nhE0PUtS+6PnCYMreOOnI6+y6JSnWUhpeecN6SaDy8Kb5zcEz5TLudz4pB7NQ7SIjkmb3Z/Myx oSYkYmmix2yXuaoWdFNAP5iRCHxU3enCNX6zRwWfOATxq4HEw+TWiKva6lavT2TGpW99++WsdEx knazBei5YL+kf/sG3vlj2bqo6LIPPCnTSaKAwltYc2akcaHPCsBYCKLMLs1WnZrPMS7qu+oofma h/ENjbrDS8m/xLdNH+YANI+sPQPJ7zc1AZqjJcEvE70YYySKf8Nqx8Bcmk6658ZDioX4tboA0pU TNwgRfX+CQUZ77WpbGy/7pgJs= X-Received: by 2002:a17:902:fc85:b0:2c9:bd37:ae76 with SMTP id d9443c01a7336-2cf34a3ef59mr151673285ad.47.1784539936212; Mon, 20 Jul 2026 02:32:16 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:15 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 06/17] mm/mm_init: factor out pfn_to_zone() Date: Mon, 20 Jul 2026 17:31:16 +0800 Message-ID: <20260720093127.540540-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" pfn_to_zone() in hugetlb_vmemmap.c duplicates the zone lookup logic in __init_deferred_page(). Move it to mm_init.c, declare it in mm/internal.h, and reuse it from __init_deferred_page() and HugeTLB early vmemmap initialization instead of open-coding the zone walk there. Signed-off-by: Muchun Song --- v2: - Move this preparatory patch before the sparse-vmemmap optimization changes (suggested by Mike Rapoport) --- mm/hugetlb_vmemmap.c | 17 ++--------------- mm/mm_init.c | 28 ++++++++++++++++++---------- mm/mm_init.h | 1 + 3 files changed, 21 insertions(+), 25 deletions(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index ae8fdaa42118..c48fcea076a5 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -19,6 +19,7 @@ #include #include "hugetlb_vmemmap.h" #include "internal.h" +#include "mm_init.h" =20 /** * struct vmemmap_remap_walk - walk vmemmap page table @@ -744,20 +745,6 @@ static bool vmemmap_should_optimize_bootmem_page(struc= t huge_bootmem_page *m) return true; } =20 -static struct zone *pfn_to_zone(unsigned nid, unsigned long pfn) -{ - struct zone *zone; - enum zone_type zone_type; - - for (zone_type =3D 0; zone_type < MAX_NR_ZONES; zone_type++) { - zone =3D &NODE_DATA(nid)->node_zones[zone_type]; - if (zone_spans_pfn(zone, pfn)) - return zone; - } - - return NULL; -} - /* * Initialize memmap section for a gigantic page, HVO-style. */ @@ -787,7 +774,7 @@ void __init hugetlb_vmemmap_init_early(int nid) map =3D pfn_to_page(pfn); start =3D (unsigned long)map; end =3D start + hugetlb_vmemmap_size(m->hstate); - zone =3D pfn_to_zone(nid, pfn); + zone =3D pfn_to_zone(pfn, nid); =20 if (vmemmap_populate_hvo(start, end, huge_page_order(m->hstate), zone, HUGETLB_VMEMMAP_RESERVE_SIZE)) diff --git a/mm/mm_init.c b/mm/mm_init.c index dcb757b36902..191d4ebe9314 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -690,6 +690,20 @@ static __meminit void pageblock_migratetype_init_range= (unsigned long pfn, } } =20 +struct zone __meminit *pfn_to_zone(unsigned long pfn, int nid) +{ + pg_data_t *pgdat =3D NODE_DATA(nid); + + for (enum zone_type zone_type =3D 0; zone_type < MAX_NR_ZONES; zone_type+= +) { + struct zone *zone =3D &pgdat->node_zones[zone_type]; + + if (zone_spans_pfn(zone, pfn)) + return zone; + } + + return NULL; +} + #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT static inline void pgdat_set_deferred_range(pg_data_t *pgdat) { @@ -748,20 +762,14 @@ defer_init(int nid, unsigned long pfn, unsigned long = end_pfn) =20 static void __meminit __init_deferred_page(unsigned long pfn, int nid) { - pg_data_t *pgdat =3D NODE_DATA(nid); - int zid; + struct zone *zone; =20 if (early_page_initialised(pfn, nid)) return; =20 - for (zid =3D 0; zid < MAX_NR_ZONES; zid++) { - struct zone *zone =3D &pgdat->node_zones[zid]; - - if (zone_spans_pfn(zone, pfn)) - break; - } - __init_single_page(pfn_to_page(pfn), pfn, zid, nid); - + zone =3D pfn_to_zone(pfn, nid); + __init_single_page(pfn_to_page(pfn), pfn, + zone ? zone_idx(zone) : MAX_NR_ZONES, nid); if (pageblock_aligned(pfn)) { enum migratetype mt =3D kho_scratch_migratetype(pfn, MIGRATE_MOVABLE); diff --git a/mm/mm_init.h b/mm/mm_init.h index 39f75df9be1c..c9fc35e7e9f1 100644 --- a/mm/mm_init.h +++ b/mm/mm_init.h @@ -39,6 +39,7 @@ void memmap_init_range(unsigned long size, int nid, unsig= ned long zone, enum meminit_context context, struct vmem_altmap *altmap, int migratetype, bool isolate_pageblock); +struct zone *pfn_to_zone(unsigned long pfn, int nid); =20 #if defined CONFIG_COMPACTION || defined CONFIG_CMA /* Free whole pageblock and set its migration type to MIGRATE_CMA. */ --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A232E3D45F4 for ; Mon, 20 Jul 2026 09:32:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539943; cv=none; b=hxXkhsTWgY0GBI5plXiJBc0+D4dLfl5ERKfOk9ZDJ0BFY35LLNJ/oTWnkpNTSFU9WKFE41Ol/5P0Jm5c/FT5cee8HrQxaAKhP7dzErCO3ivXW8Rwpo4IghXJSkM80BZzFq6cZRxhXvpmlaadufv19tfdUCAoIqhYPvYWQrvNXWg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539943; c=relaxed/simple; bh=0ODrG2D+Miu/T3JxvV66aoHR/C/eKIjGGN+mPwwGD8Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=f90ZS5Ph6YnUKL+9h1Bh9S//4bPdWRn/GKoje0NAAQXIEF7S8lPeqZMUwQjQ97klCqvzZtBJGn/SYlhHlALwcHh/7HP1XB13VX3Ps9bp4elyD/fXqIwMn3sKg/FlDSXqTy99CBBMIPOTLlVfriXs0KUFHCuJx7m/ArSQC0bAlXY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=MjNDTTi6; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="MjNDTTi6" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-2cc7e86e7aeso70232525ad.2 for ; Mon, 20 Jul 2026 02:32:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539941; x=1785144741; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=j+FzYBHgL+IKE5/vY4Q2WTb8Xgc0rS8reYNZSakryMo=; b=MjNDTTi6IrbouQJBqdTfHdJCMJSWTVGcqjrokJgXM/6vw1o19MFOFgNb1aVC4QW2nC xHiC6ae9doTeu5GMdiRFNPuXKAimw/UdlFApGqYhTAYhYy/aCMyC3v9wvwTTfIuA6hTj WDuMimcqOgbe3C/4lLGqw2H8j1AIwPlhJe0F89l+5xCVBhdEEeERA52u+4qC+2fpmOMW P0ufxm33YyQcmFY0I4z0Oj9IPYeSXsM08zuhmqn3jQsB7dIqJqKU+BqnTVjz9459hL+h aIK2mTfJxATcbmS//XeaxgYz5PfWtXUz50Y1r332u2sWhKJZMtG10yrCB30re8WYa0kU Z7Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539941; x=1785144741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=j+FzYBHgL+IKE5/vY4Q2WTb8Xgc0rS8reYNZSakryMo=; b=DXRtAbRtuJWe+Kufr1WpNvt81OD0PXUdaC3CipdG20TtyL/rvxmErV6PSejBg2iRTD Li+6wTBc2ojvpYCRyZI/fFESDYBumhIn6vOEOPpAz1hWVbv58yWkpVKSpR9X7yjhbQWf hmFwVOA4H6IDLDhgGr5Ev0q2QNaFNOpzTHm9XQVziiqRAibc3tRMhjAjrk++S14deryR eDZnSfdvGpT7boXw8pZMmafqYi36JPGOpX0PeoVJfyjKxTogqpPAZg3kF86NnOf15xFX hEXGCPHOpzlfMAcIGEeVuYuSDvBUOu1hbAVcDYzOUmeVTxJtwwRQh8uyfd/0rFCxUyXN jBZg== X-Forwarded-Encrypted: i=1; AHgh+RpnY0VsEtgfHeLNNus6qFG1bUQHTZ6IkbtYGLvftfcMpi6xI/R+NVzYRGXnQ8trm/mYi4rtzT8ItX445mk=@vger.kernel.org X-Gm-Message-State: AOJu0YxHcgNZNKMJsqJhNY5c6GWJaWoQjoF7W8d4uSc2bHsNdju7j4GW uya7KgqIPRS7IFTwuxBXPzhn4PLCo+KqIJm0AZ0PQ5MZCHdVLDgQpZdxEAzzUlMfPy0= X-Gm-Gg: AfdE7cmaTvkNevp8FCuuJlNQz/rUe/O+iCcaSbkqFFblUtKc1pEN5URyQOr/+rVqbag 1bQRwAx8GTkZWH8DGbEls700ykqmfFoDWjQWvCSDPMQheB6QNM1wbaU7ZelTfv7cgm95cFsar6l MyIoJueovzYz4c13w+OikaCtKTROzS7hMn+yXg+baTbo0arPudl1BrGrmyc9tMmUA6EP9pG9uYW f/7sB9AcB+EyMWJrtgrku8TbWsKAmBo+gL2ZpYHcd4xeHoGhzTga7duYUn4zw1Iab+m4NIDionb 6Ph3oQC1j4RdGMglKK0Rje94UhVtbHOIUTOO3r3VuS08bNHQPiwcAEmIT2onateMOJ79KnoczId vUA+6vCaD2w9xS5IokZgyuYGDi7PKS+m/0ZMzwYAqvvLuK5yEq2NMK38ADqccS5OFr7YH5Ruv8S 4QxtkQpNpRMwbl7TBmyj7EW2UiV4ngIO0HOQ== X-Received: by 2002:a17:903:a8f:b0:2ce:a600:6022 with SMTP id d9443c01a7336-2cf34850c21mr145375985ad.7.1784539940962; Mon, 20 Jul 2026 02:32:20 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:20 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 07/17] mm/sparse-vmemmap: move vmemmap_get_tail() before PTE population Date: Mon, 20 Jul 2026 17:31:17 +0800 Message-ID: <20260720093127.540540-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A follow-up change will call vmemmap_get_tail() from vmemmap_pte_populate(). Move it before the PTE population helpers to avoid adding a forward declaration. Move vmemmap_alloc_block_zero() with it because vmemmap_get_tail() depends on that helper. No functional change is intended. Signed-off-by: Muchun Song --- mm/sparse-vmemmap.c | 78 +++++++++++++++++++++++---------------------- 1 file changed, 40 insertions(+), 38 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index b7abc5494bb9..b770fe2428fd 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -148,6 +148,46 @@ void __meminit vmemmap_verify(pte_t *pte, int node, start, end - 1); } =20 +static void * __meminit vmemmap_alloc_block_zero(unsigned long size, int n= ode) +{ + void *p =3D vmemmap_alloc_block(size, node); + + if (!p) + return NULL; + memset(p, 0, size); + + return p; +} + +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP +static __meminit struct page *vmemmap_get_tail(unsigned int order, struct = zone *zone) +{ + struct page *p, *tail; + unsigned int idx; + int node =3D zone_to_nid(zone); + + if (WARN_ON_ONCE(order < VMEMMAP_OPTIMIZATION_MIN_ORDER)) + return NULL; + if (WARN_ON_ONCE(order > MAX_FOLIO_ORDER)) + return NULL; + + idx =3D order - VMEMMAP_OPTIMIZATION_MIN_ORDER; + tail =3D zone->vmemmap_tails[idx]; + if (tail) + return tail; + p =3D vmemmap_alloc_block_zero(PAGE_SIZE, node); + if (!p) + return NULL; + for (int i =3D 0; i < PAGE_SIZE / sizeof(struct page); i++) + init_compound_tail(p + i, NULL, order, zone); + + tail =3D virt_to_page(p); + zone->vmemmap_tails[idx] =3D tail; + + return tail; +} +#endif + static pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long ad= dr, int node, struct vmem_altmap *altmap, unsigned long ptpfn, unsigned long flags) @@ -181,17 +221,6 @@ static pte_t * __meminit vmemmap_pte_populate(pmd_t *p= md, unsigned long addr, in return pte; } =20 -static void * __meminit vmemmap_alloc_block_zero(unsigned long size, int n= ode) -{ - void *p =3D vmemmap_alloc_block(size, node); - - if (!p) - return NULL; - memset(p, 0, size); - - return p; -} - static pmd_t * __meminit vmemmap_pmd_populate(pud_t *pud, unsigned long ad= dr, int node) { pmd_t *pmd =3D pmd_offset(pud, addr); @@ -323,33 +352,6 @@ void vmemmap_wrprotect_hvo(unsigned long addr, unsigne= d long end, } =20 #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP -static __meminit struct page *vmemmap_get_tail(unsigned int order, struct = zone *zone) -{ - struct page *p, *tail; - unsigned int idx; - int node =3D zone_to_nid(zone); - - if (WARN_ON_ONCE(order < VMEMMAP_OPTIMIZATION_MIN_ORDER)) - return NULL; - if (WARN_ON_ONCE(order > MAX_FOLIO_ORDER)) - return NULL; - - idx =3D order - VMEMMAP_OPTIMIZATION_MIN_ORDER; - tail =3D zone->vmemmap_tails[idx]; - if (tail) - return tail; - p =3D vmemmap_alloc_block_zero(PAGE_SIZE, node); - if (!p) - return NULL; - for (int i =3D 0; i < PAGE_SIZE / sizeof(struct page); i++) - init_compound_tail(p + i, NULL, order, zone); - - tail =3D virt_to_page(p); - zone->vmemmap_tails[idx] =3D tail; - - return tail; -} - int __meminit vmemmap_populate_hvo(unsigned long addr, unsigned long end, unsigned int order, struct zone *zone, unsigned long headsize) --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E22A33D5222 for ; Mon, 20 Jul 2026 09:32:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539948; cv=none; b=uXQFP36I26PK6rQeH3Aze8kZM4njn9xBSuCe3vRJDJRIIuyv2aA21ldMCqMoCs20YrBcz2oY00pE7gObP3N8xsF6Pi/h5HrSK6NZg3EzTxy85rp2DexFGRmD+74T6kZqenT0x59tlGxfAF6jA+w6Ks2n5VxeYsHGMnDW1YElufQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539948; c=relaxed/simple; bh=LzdMqlEotTeBoZebm0qCJLpqHk7PFGo/ia/2dpI8Xqo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Jm4vMF+nNvo3tMbX1feaTLjn9M1QufxvuWo3VmE0AkfKXIjYf9LOO5kCkMksc6utvqN+GK33IlTMpV501LREV62soatTuygKYLO1klkIUAVpdIVCslE7ix5IWXZ781gzBAZfu2D7DY1SE+SLRrNNlRm5g/4ZMRokKd9N1rAeci0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=bztreUtP; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="bztreUtP" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2cf452def93so12080015ad.1 for ; Mon, 20 Jul 2026 02:32:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539946; x=1785144746; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=XIl1erdJ2bKlvmuHeltJ/d9wztXmxG8GJiFqIKkszIE=; b=bztreUtPMDqSwEXvpOGrxwR4UjELESWoqgCb5rHOtrSl4lrtvlSrJsw3i0x0nLUU8Z h44BUG9B0BO2kCTaYFOaW5aodCwoHND/ucarvhOOYGoi2ppfe2c7QDMpnLHylbsJkivg lwNltSJNXeVbva7TSvMoH1K/wWTYOyN0GJz+05OwrUHFRAWJPZc7+EPjxRZDbLL1uzAi LjrDBVWfnnitlsBSVgUSBU+Y7ar70ztZKetB8UyQzlXzLG4hd0aewti97obzTEpSJI5z pEBHn+IHszfAiii6HeRc9SvKKgDfa9paE7SJVYIcYMQZ5AE7KaZX9iHJ6ADBQITMNcpe jD3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539946; x=1785144746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=XIl1erdJ2bKlvmuHeltJ/d9wztXmxG8GJiFqIKkszIE=; b=VMTTiBGDhUnUw/OwUWaHrbhG79W+YuFjfNlxLLOhal3nyXcJh+0sol5/hbA8koeFde BEMw03WAYSS6c8+vn5XmpBpw1A3uwzOiG+Hmu+ckBij1tw+ONQ9w4q8wRc+Qn05+1AiX QGTHVnClJ2G3HRgwyVs7Tq8zKIcfiyHo9mrDHfJRiML90F1cive2fccd2qBd0vxV1fsK g8m063JYjB2JswlVhiyHGTUFA8004ChCGXq6YyktIe/fF6z5navDPFZU6qy56+OJEnYq hjPHuE+Z6jJoddYFu28DlPDE07w24y6j9R4jJWMHd3ePUbpIdZDK3Sbmz+5RkQIjWRNp K0EA== X-Forwarded-Encrypted: i=1; AHgh+RrpEjluNB9RSLjFxWijSWGIt7xIH1jWAo2W1l1jgHMYHstfdjnaxW5meYl1wgOkjpmMh+1nwU4602D+3Cs=@vger.kernel.org X-Gm-Message-State: AOJu0YxeL936/swPFgxaf8jrVj38McMPfenqWh4grbNh9EaVsaJIJJAS ttR4YIsFTKk7i3bVqE0SCuF7zd3KM+pO/yBMjQ2gpFsHPceRvKIgWNg9eoZYETObPmY= X-Gm-Gg: AfdE7cm19IKA5fYy9CzlRBRX1Scw5XakM9J3LGmzVEcUNUdbSGOV3fcp8M1vYDUH6Ld qrSjYH6eGwuarHr2NErU7YCBCs4TrujqFu/ApjsQI9zDEBIQ+1aQlksR6a3vgnhtgJwzZ9ogZ8t pQ7HhhQFIo1Q5ufkoHqVagtkKUJ9LV8PgPPgXwFy9uxf0dv3B7X8P6whmBZwEWWgQe3CuhxxSqX RTUZqd1dUStJq9KKQWNcH2avlNlGvHYpBSOidQThPWVqXKBfCuFNr7n6u/T0MpC99ksQq2kpEn7 yh4pzd1wOQHaqC+4af/0f/Ueq/CaYeUyUTPXceYcoNZyi+Vd3VCxupw744gnDbga3qnGfebX03N +rtqXXPM3mnhn48bJykrJaSeeGpAdQsEq/PePet8XYGY5mJfzBhoq0xlsYDPiLn1txOe0wzRGkC ctTSJfbuUR7y2lpZUWxEHvWWo= X-Received: by 2002:a17:903:198c:b0:2c9:c517:d08b with SMTP id d9443c01a7336-2cf34a50497mr134993955ad.22.1784539946139; Mon, 20 Jul 2026 02:32:26 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:24 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 08/17] mm/sparse-vmemmap: support section-based vmemmap optimization Date: Mon, 20 Jul 2026 17:31:18 +0800 Message-ID: <20260720093127.540540-9-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Teach sparse-vmemmap population code to use the compound page order when deciding whether a vmemmap page can be optimized. With this information, the common sparse-vmemmap population path can allocate or reuse shared tail vmemmap pages directly instead of relying on HugeTLB-specific handling. This centralizes vmemmap optimization logic in the sparse-vmemmap code, based on section metadata, and prepares for sharing the same mechanism across different users of vmemmap optimization, including HugeTLB and DAX. Signed-off-by: Muchun Song --- v2: - Keep vmemmap accounting and population logic in sparse-vmemmap.c (suggested by Mike Rapoport) - Move vmemmap_get_tail() before its first use instead of adding only a forward declaration in the previous patch (suggested by Mike Rapoport) - Simplify the PMD path handling for HVO-covered sections --- mm/sparse-vmemmap.c | 36 ++++++++++++++++++++++++++++++------ mm/sparse.c | 4 ++-- mm/sparse.h | 7 +++++++ 3 files changed, 39 insertions(+), 8 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index b770fe2428fd..e818b4109777 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -186,6 +186,11 @@ static __meminit struct page *vmemmap_get_tail(unsigne= d int order, struct zone * =20 return tail; } +#else +static inline struct page *vmemmap_get_tail(unsigned int order, struct zon= e *zone) +{ + return NULL; +} #endif =20 static pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long ad= dr, int node, @@ -193,12 +198,24 @@ static pte_t * __meminit vmemmap_pte_populate(pmd_t *= pmd, unsigned long addr, in unsigned long ptpfn, unsigned long flags) { pte_t *pte =3D pte_offset_kernel(pmd, addr); + unsigned long pfn =3D page_to_pfn((struct page *)addr); + if (pte_none(ptep_get(pte))) { pte_t entry; - void *p; + + if (pfn_vmemmap_optimizable(pfn) && ptpfn =3D=3D (unsigned long)-1) { + unsigned int order =3D section_order(__pfn_to_section(pfn)); + struct zone *zone =3D pfn_to_zone(pfn, node); + struct page *page =3D vmemmap_get_tail(order, zone); + + if (!page) + return NULL; + ptpfn =3D page_to_pfn(page); + } =20 if (ptpfn =3D=3D (unsigned long)-1) { - p =3D vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap); + void *p =3D vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap); + if (!p) return NULL; ptpfn =3D PHYS_PFN(__pa(p)); @@ -217,7 +234,8 @@ static pte_t * __meminit vmemmap_pte_populate(pmd_t *pm= d, unsigned long addr, in } entry =3D pfn_pte(ptpfn, PAGE_KERNEL); set_pte_at(&init_mm, addr, pte, entry); - } + } else if (WARN_ON_ONCE(pfn_vmemmap_optimizable(pfn))) + return NULL; return pte; } =20 @@ -406,6 +424,9 @@ int __meminit vmemmap_populate_hugepages(unsigned long = start, unsigned long end, pmd_t *pmd; =20 for (addr =3D start; addr < end; addr =3D next) { + unsigned long pfn =3D page_to_pfn((struct page *)addr); + const struct mem_section *ms =3D __pfn_to_section(pfn); + next =3D pmd_addr_end(addr, end); =20 pgd =3D vmemmap_pgd_populate(addr, node); @@ -421,7 +442,7 @@ int __meminit vmemmap_populate_hugepages(unsigned long = start, unsigned long end, return -ENOMEM; =20 pmd =3D pmd_offset(pud, addr); - if (pmd_none(pmdp_get(pmd))) { + if (pmd_none(pmdp_get(pmd)) && !section_vmemmap_optimizable(ms)) { void *p; =20 p =3D vmemmap_alloc_block_buf(PMD_SIZE, node, altmap); @@ -439,8 +460,11 @@ int __meminit vmemmap_populate_hugepages(unsigned long= start, unsigned long end, */ return -ENOMEM; } - } else if (vmemmap_check_pmd(pmd, node, addr, next)) + } else if (vmemmap_check_pmd(pmd, node, addr, next)) { + if (WARN_ON_ONCE(section_vmemmap_optimizable(ms))) + return -ENOTSUPP; continue; + } if (vmemmap_populate_basepages(addr, next, node, altmap)) return -ENOMEM; } @@ -648,7 +672,7 @@ void offline_mem_sections(unsigned long start_pfn, unsi= gned long end_pfn) } } =20 -static int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned = long nr_pages, +int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr= _pages, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { const struct mem_section *ms =3D __pfn_to_section(pfn); diff --git a/mm/sparse.c b/mm/sparse.c index ca9875f568d3..24555a32a5d9 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -315,8 +315,8 @@ static void __init sparse_init_nid(int nid, unsigned lo= ng pnum_begin, nid, NULL, NULL); if (!map) panic("Failed to allocate memmap for section %lu\n", pnum); - memmap_boot_pages_add(DIV_ROUND_UP(PAGES_PER_SECTION * sizeof(struct pa= ge), - PAGE_SIZE)); + memmap_boot_pages_add(section_nr_vmemmap_pages(pfn, PAGES_PER_SECTION, + NULL, NULL)); sparse_init_early_section(nid, map, pnum, 0); } } diff --git a/mm/sparse.h b/mm/sparse.h index 1e9fcea10181..4377b097dc8d 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -87,8 +87,15 @@ static inline bool section_vmemmap_optimizable(const str= uct mem_section *section */ #ifdef CONFIG_SPARSEMEM_VMEMMAP void sparse_init_subsection_map(void); +int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr= _pages, + struct vmem_altmap *altmap, struct dev_pagemap *pgmap); #else static inline void sparse_init_subsection_map(void) {} +static inline int section_nr_vmemmap_pages(unsigned long pfn, unsigned lon= g nr_pages, + struct vmem_altmap *altmap, struct dev_pagemap *pgmap) +{ + return DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE); +} #endif /* CONFIG_SPARSEMEM_VMEMMAP */ =20 #endif /* __MM_SPARSE_H */ --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACD833D5676 for ; Mon, 20 Jul 2026 09:32:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539954; cv=none; b=gG8XDAxxsH9N+IAXrXS3c/agPeUYLbs86nzWhpKlWjivF/gxPa2Il8gUzBfqNtA9xK95XwZIfbImZ3JuADv+K6jGcGv+ML3ijlNbehFwuee9CsKt3kXW7NCfOpEypIaTeLYmpK7/xJjFsWtfyBLNGN3ZQ5GWCkVRZ4UGwFxULOY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539954; c=relaxed/simple; bh=Vi3mEBLnTt/SlYJioXlvIsYpN4MDDoAaP/pEGRtg6Q8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SykpG9wAFz5QQ6DllOkSDNLlsme1gy+dga1nrUcX+ISo52PZxDjnSXM5ecTnrk8j/Kc2q7EM2/XgAy7Z000smYm1BG5yqyrWuCr6VAsrPVinRVyi1ymaZTcNH87PsUQzHkmr9/vZtYhcgEtYOrhPGpVul5ncTEt4vbF7tdVnkwE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=frDN7Kke; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="frDN7Kke" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-2caea3f742bso118937925ad.0 for ; Mon, 20 Jul 2026 02:32:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539952; x=1785144752; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=5Xc+9wTjB/ZbPSuhtSmXNsjcgy94ONeB8wn4Fc53kDI=; b=frDN7KkesFJ5jAwjsKgLkMleaAIZG1PfFlMT5aeJg8BF0dsfs4HZSiv6QYrutpe9Og X1bL263MIbgZAPV1Aq29pEE2biJB7Zffej1omkrzj/lcJzFtlxncYgrLpuA31cjs6twE 3DglWnN4YuKMVLIdVVjyn+9pI1nZyv0xyrUamR8C4bEjo5vEGiG16Soa6qLYRnHHR2hd JYyPyL7txl9rDRoUN+fm3+JuLnXAQ+cJYqTKdK76aVuNzh/3bmkf0mMHLBdR9NFSqyP6 9mrjOKeOAg95tzoryiLsVbdWlYDthzlODAZ1NG1r1Xh73Yt17RMI9R/mLnkcn2Z6odYY CNvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539952; x=1785144752; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=5Xc+9wTjB/ZbPSuhtSmXNsjcgy94ONeB8wn4Fc53kDI=; b=p2wwArBFgucmjfrvyNprguaGhyN8VqIarPqvbbHTiMvHpJlg/wXwBupDLu/ox2uvWw ihRT43n+mKArRh66feertuNXCmwouJmj8x46Q3+lBH0I6bgDyXOXhA1JOhu4XSjlN+Mc JgUT7L/Juy2Tw/gYAVyiLIA4T+3ewrmYkTcibCuhbxMSMiDNDYZ0alJBTuKKEebgOfar 5ACI+P6Fzuk2QEdNbGdhy3M495V95dGyyH+YyhaeFJgYNx29RJ3Rq14nij6rCEOc9JMJ RBkbxxvSvb+3q/iF4nhCSrAr2ATit3qn92SzX92stYxv/SwNpxzBIRS4+nwyX7yiHZ2G aQfw== X-Forwarded-Encrypted: i=1; AHgh+Ro1IUGQTU99cFneok+uLgEKoD5Tnie7itObSftOQBMxezZch9kb2+LnQKkE9o0OtkUWJ0zpCdUx/SOLjV8=@vger.kernel.org X-Gm-Message-State: AOJu0YxrKb+ImWYdIMPlrSDBi1LA5hwbdChIi4BaxL0fof9SAMvX7Kq9 mE4RTTrB2lc7Gq3xxr9/HDOBPT+3yHCX7jUf04Xoz3yyByXlQpNxLcuRUxUF+QEZ3C2FN5rQoJc pUvfAQN9/8g== X-Gm-Gg: AR+sD13iQ2dXIf+krxfHU4ynvqxDIU3CZyH5ZJMGnOIuHMX9viExYRvzdlYG/qb85mJ 15OmlHoMb4/Pitgyls6AuvUTqA5jF2HR7VKNOn9DH5OqWxwLMFp8CqBfDDxFegCQc10cp2OnSfe igArLCL42re3lCS3buyWmoB4qIHqH/N99pTKtfFWRJsSQnp/z1lVyOzP31QqmHNubzeWLQzaY03 Q9IDYZzBe6M1JM3+SNX98VmAKWBn2kQi1czGiPdbMdJUYyMGQ7vEvM4H3HmGqPtxPIqA4eyC9TE nCvi4QzKVDOc3zQ2kWsij+yJhBDaDoqp5O6a/D0EBw9EZ4eSVRGmT8WT7jK0UiPEV6cwEP6YAv/ RP5qft0GNJyMJI08eMCXYmEEyV69oEw9YGHLkOjcBG92jn6f2fF+k/pamB4Tnz7dL95BRUeJEfB yMxSaRRRyul1cduOOpsKGAZ5ECz552CzCuUw== X-Received: by 2002:a17:903:2449:b0:2cf:477f:57c5 with SMTP id d9443c01a7336-2cf477f5bacmr91609105ad.1.1784539951960; Mon, 20 Jul 2026 02:32:31 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:31 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 09/17] mm/sparse: initialize memory sections earlier Date: Mon, 20 Jul 2026 17:31:19 +0800 Message-ID: <20260720093127.540540-10-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Upcoming HugeTLB bootmem changes need sparsemem section metadata before the HugeTLB bootmem allocation path runs. The memory sections are initialized from sparse_init(), which is called too late for that setup. Move the code that initializes sparsemem section metadata for memblock ranges into mm_core_init_early(), before free_area_init() and the HugeTLB bootmem setup. Rename the helper to sparse_sections_init() so the new caller describes the sparsemem-specific initialization step. This is a preparatory change. Signed-off-by: Muchun Song --- v2: - Rename the helper to sparse_sections_init() to describe the section metadata initialization (suggested by Mike Rapoport) - Fix the !SPARSEMEM stub name so SPARSEMEM=3Dn builds compile (reported by Sashiko) --- mm/mm_init.c | 1 + mm/sparse.c | 10 ++-------- mm/sparse.h | 2 ++ 3 files changed, 5 insertions(+), 8 deletions(-) diff --git a/mm/mm_init.c b/mm/mm_init.c index 191d4ebe9314..b3ebecf077a0 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -2636,6 +2636,7 @@ void __init __weak mem_init(void) =20 void __init mm_core_init_early(void) { + sparse_sections_init(); free_area_init(); =20 hugetlb_cma_reserve(); diff --git a/mm/sparse.c b/mm/sparse.c index 24555a32a5d9..d4641aa70497 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -191,12 +191,8 @@ static void __init memory_present(int nid, unsigned lo= ng start, unsigned long en } } =20 -/* - * Mark all memblocks as present using memory_present(). - * This is a convenience function that is useful to mark all of the systems - * memory as present during initialization. - */ -static void __init memblocks_present(void) +/* Initialize memory section metadata for all system memory. */ +void __init sparse_sections_init(void) { unsigned long start, end; int i, nid; @@ -332,8 +328,6 @@ void __init sparse_init(void) unsigned long pnum_end, pnum_begin, map_count =3D 1; int nid_begin; =20 - memblocks_present(); - if (compound_info_has_mask()) { VM_WARN_ON_ONCE(!IS_ALIGNED((unsigned long) pfn_to_page(0), MAX_FOLIO_VMEMMAP_ALIGN)); diff --git a/mm/sparse.h b/mm/sparse.h index 4377b097dc8d..b2dc7362af4f 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -14,6 +14,7 @@ * mm/sparse.c */ #ifdef CONFIG_SPARSEMEM +void sparse_sections_init(void); void sparse_init(void); int sparse_index_init(unsigned long section_nr, int nid); =20 @@ -48,6 +49,7 @@ static inline void __section_mark_present(struct mem_sect= ion *ms, ms->section_mem_map |=3D SECTION_MARKED_PRESENT; } #else +static inline void sparse_sections_init(void) {} static inline void sparse_init(void) {} #endif /* CONFIG_SPARSEMEM */ =20 --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D807A3D9DDB for ; Mon, 20 Jul 2026 09:32:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539958; cv=none; b=erEwGGZ58BmxMKN5hqEdi9AdWBrnoY4R6WdZA8SJtCXnmI7FMHjrY0DeRzsvcECPW0KV0fZW+WNru8lMU5VfHoWUW8E8IXXRM2gjBlLFVPzk0w6hFShTSDEr/HFwpG4gFoH8mFiSt7rJPJdIdwIK6dhsRjd+USolHaCv+PoTAXg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539958; c=relaxed/simple; bh=2EN4x5DmthyTNfsRPHSiJE1XEerzpCPOD0cgHpQk9qg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rsA9X3hBPxw8PumMtQS1eYX1tkkp7k57s55XThKql29VMzYL6/jGtzfQRgryr5BQRhstvRD+S78I7X6NuwcolqWKjvLYCu0kqH6bTBUxxZK/7JqC4CuKo5RML1ANO9coYzXBTLYSGGx8XVjCcDhvHhZ4CJwxtSVY/GPMM1onUz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=GD2fKq0G; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="GD2fKq0G" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2caea3f742bso118938625ad.0 for ; Mon, 20 Jul 2026 02:32:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539956; x=1785144756; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=SiNjPlndk1ck6Ov28MIeX8psN0Q1u/TAdFlhmNElnGE=; b=GD2fKq0GvDBwpPVTLz1I+zen+R5AN1KOLshMntUL0R2oyt4h9cSOKkH5VkZFGiLTLk t0Js/LkuF0W6z1LHT2ApZvH0JV1h3EbPNT/KnLkWfEKoGmB9/2B6hNxFSpqMdwLUn2dM f4bSSnlPiKXod0I6AWhhHKFsddcXKfO6rlvOyajCZnGX8442kIxEeZDUcVCCWgZc7+i4 qH5wXKNuektreM6JWrY8BUZ67dm4BsZEFDslQhN5pkKw8N/5Odgkeb0pnW4SKD77rxWj LCEU82Ur6zUQe8C8chFw/YVO6o4hn/BZlKHhf0n5ZMTurWCuSnA2IsHCP7MJ1Pl8qVVz GN0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539956; x=1785144756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=SiNjPlndk1ck6Ov28MIeX8psN0Q1u/TAdFlhmNElnGE=; b=VJgnmXblyHlQo2mSTOczFsv2sQg3eB13h8ODTmjAePi+mBlPvbBK+P5V7GMQblOYAj 1C87GVso8cN6avrA0wXAbENrENuf8Eg1OIz1OU907RIWNrT4tETCQMLG1bI67XluSlIj bh8NGqKDtKjbckYdqapAnD/Rar9FFl/03LOBm52XqPWE+xErvoUra9hTCN8z0+PAZHyk D1gv3iObjn4eNqQFCnWOUjq4WhxmldH51hlLV2c9Lt3Jx4X35ej0W28jq2Ta/DL4D9rq obXi3SS+7RXY1Y3Z0Hp+Atult+wH3P4oAuUNP5tOEtuj5xw0EonuaB7gmkjHUdd6by17 jW0Q== X-Forwarded-Encrypted: i=1; AHgh+RoICwBJeN5Smm0gV3vuzFJP8JrHcEqF8ovtbtCzTPCW7K3DfIk9Opi2JK+WtxIwqtKycfRpFQ+Y+npSNL8=@vger.kernel.org X-Gm-Message-State: AOJu0YxYBp3PKOdjezMqvlry0YrF0pLWoJJ+fpLPxG3qdXmTepBeIoJi rfHJO5ELTE6+FVLsrLQahSb1hKDatuW+SfU0nD4ORSFHIffTDDexfgmFyasXJgqPmno= X-Gm-Gg: AfdE7cmpCtts3IAO0kwrjWBneblbMt++01RGXKC1VexzE08W/OG2DhydyFWpj8qz9Kg 7b75PRtVOXOVSFH8EgGJApxjvC+ZMb7d7zG4wZyz/u4NRpIV24H+qgVkU3HQ6Gq+3PTJrnl2id3 GzJ2/hq3110s3VVoko7bk/Z7bPXAorjDIFIOr2dS+Mc2nnlDlFduG20Ux2OaU7U60uBLvt23SLw 2jciSgFckRiGPHtyYZUhB5IX4eKHszB0QjB3ouz9PNkUk79w3/w5pSIwmbPigF18Ne2NMjD/S5H P4m+Fu0Kq3/U56v5qJwoKPoUbnfZ826ekY4WiuJc4ouABm91kkq3CcsYSHeCHyPFMarSbgiWuaC 6KMsasiu1//waCkrcZ6t5CM3Y7KZSoBtAUYspN7/E6U15zLX0o146gz1Y6FsFB91g9q0N94Xm7D k+Ujd8Mw3W5NHVsAgshe0Ixdg= X-Received: by 2002:a17:902:ce87:b0:2c9:bf82:dd11 with SMTP id d9443c01a7336-2cf34895290mr137263395ad.7.1784539956087; Mon, 20 Jul 2026 02:32:36 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:35 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 10/17] mm/hugetlb: switch HugeTLB to section-based vmemmap optimization Date: Mon, 20 Jul 2026 17:31:20 +0800 Message-ID: <20260720093127.540540-11-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" HugeTLB bootmem vmemmap optimization still carries its own early setup path, including pre-populating optimized mappings before the generic sparse-vmemmap code runs. Now that the section-based vmemmap optimization can derive HugeTLB vmemmap deduplication from section metadata, HugeTLB only needs to mark the bootmem huge page range with the appropriate order. The generic sparse-vmemmap population path can then allocate and map the shared tail vmemmap pages without any HugeTLB-specific early population code. Do that by setting the section order when a bootmem huge page is allocated and dropping the dedicated pre-HVO helpers and related special-casing. This removes duplicate early setup logic and switches HugeTLB to the section-based vmemmap optimization path. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/hugetlb.h | 1 - include/linux/mm.h | 3 -- mm/hugetlb.c | 30 ++------------ mm/hugetlb_vmemmap.c | 90 +++-------------------------------------- mm/hugetlb_vmemmap.h | 14 +++---- mm/sparse-vmemmap.c | 31 -------------- mm/sparse.h | 27 +++++++++++++ 7 files changed, 42 insertions(+), 154 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 16c4c4caa126..fe28f98e1b22 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -171,7 +171,6 @@ struct address_space *hugetlb_folio_mapping_lock_write(= struct folio *folio); =20 extern int movable_gigantic_pages __read_mostly; extern int sysctl_hugetlb_shm_group __read_mostly; -extern struct list_head huge_boot_pages[MAX_NUMNODES]; =20 void hugetlb_bootmem_struct_page_init(void); void hugetlb_bootmem_alloc(void); diff --git a/include/linux/mm.h b/include/linux/mm.h index 550fb92957d1..1462669418b1 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -5041,9 +5041,6 @@ int vmemmap_populate_hugepages(unsigned long start, u= nsigned long end, int node, struct vmem_altmap *altmap); int vmemmap_populate(unsigned long start, unsigned long end, int node, struct vmem_altmap *altmap); -int vmemmap_populate_hvo(unsigned long start, unsigned long end, - unsigned int order, struct zone *zone, - unsigned long headsize); void vmemmap_wrprotect_hvo(unsigned long start, unsigned long end, int nod= e, unsigned long headsize); void vmemmap_populate_print_last(void); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index b8e1569bda15..efe0d639933e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -52,6 +52,7 @@ #include "hugetlb_cma.h" #include "hugetlb_internal.h" #include "mm_init.h" +#include "sparse.h" #include =20 int hugetlb_max_hstate __read_mostly; @@ -59,7 +60,7 @@ unsigned int default_hstate_idx; struct hstate hstates[HUGE_MAX_HSTATE]; =20 __initdata nodemask_t hugetlb_bootmem_nodes; -__initdata struct list_head huge_boot_pages[MAX_NUMNODES]; +static struct list_head huge_boot_pages[MAX_NUMNODES] __initdata; =20 /* * Due to ordering constraints across the init code for various @@ -3151,6 +3152,7 @@ static bool __init alloc_bootmem_huge_page(struct hst= ate *h, int nid) } else { list_add_tail(&m->list, &huge_boot_pages[nid]); m->flags |=3D HUGE_BOOTMEM_ZONES_VALID; + hugetlb_vmemmap_optimize_bootmem_page(m); /* * Only initialize the head struct page in memmap_init_reserved_pages, * rest of the struct pages will be initialized by the HugeTLB @@ -3311,6 +3313,7 @@ static void __init gather_bootmem_prealloc_node(unsig= ned long nid) * this folio. */ folio_set_hugetlb_vmemmap_optimized(folio); + section_set_order_range(folio_pfn(folio), folio_nr_pages(folio), 0); =20 if (hugetlb_bootmem_page_earlycma(m)) folio_set_hugetlb_cma(folio); @@ -3354,31 +3357,6 @@ void __init hugetlb_bootmem_struct_page_init(void) .max_threads =3D num_node_state(N_MEMORY), .numa_aware =3D true, }; -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP - struct zone *zone; - - for_each_zone(zone) { - for (int i =3D 0; i < VMEMMAP_OPTIMIZATION_NR_ORDERS; i++) { - struct page *tail, *p; - unsigned int order; - - tail =3D zone->vmemmap_tails[i]; - if (!tail) - continue; - - order =3D i + VMEMMAP_OPTIMIZATION_MIN_ORDER; - p =3D page_to_virt(tail); - /* - * prep_and_add_bootmem_folios() can access pageblock - * flags on bootmem HugeTLB pages, so initialize the - * shared tail struct pages here before bootmem folios - * start using them. - */ - for (int j =3D 0; j < PAGE_SIZE / sizeof(struct page); j++) - init_compound_tail(p + j, NULL, order, zone); - } - } -#endif =20 padata_do_multithreaded(&job); } diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index c48fcea076a5..e37ac4745add 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -18,8 +18,7 @@ =20 #include #include "hugetlb_vmemmap.h" -#include "internal.h" -#include "mm_init.h" +#include "sparse.h" =20 /** * struct vmemmap_remap_walk - walk vmemmap page table @@ -706,95 +705,18 @@ void hugetlb_vmemmap_optimize_bootmem_folios(struct h= state *h, struct list_head __hugetlb_vmemmap_optimize_folios(h, folio_list, true); } =20 -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT - -/* Return true of a bootmem allocated HugeTLB page should be pre-HVO-ed */ -static bool vmemmap_should_optimize_bootmem_page(struct huge_bootmem_page = *m) +void __init hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootmem_page= *m) { - unsigned long section_size, psize, pmd_vmemmap_size; - phys_addr_t paddr; - - if (!READ_ONCE(vmemmap_optimize_enabled)) - return false; - - if (!hugetlb_vmemmap_optimizable(m->hstate)) - return false; - - psize =3D huge_page_size(m->hstate); - paddr =3D virt_to_phys(m); - - /* - * Pre-HVO only works if the bootmem huge page - * is aligned to the section size. - */ - section_size =3D (1UL << PA_SECTION_SHIFT); - if (!IS_ALIGNED(paddr, section_size) || - !IS_ALIGNED(psize, section_size)) - return false; - - /* - * The pre-HVO code does not deal with splitting PMDS, - * so the bootmem page must be aligned to the number - * of base pages that can be mapped with one vmemmap PMD. - */ - pmd_vmemmap_size =3D (PMD_SIZE / (sizeof(struct page))) << PAGE_SHIFT; - if (!IS_ALIGNED(paddr, pmd_vmemmap_size) || - !IS_ALIGNED(psize, pmd_vmemmap_size)) - return false; - - return true; -} - -/* - * Initialize memmap section for a gigantic page, HVO-style. - */ -void __init hugetlb_vmemmap_init_early(int nid) -{ - unsigned long psize, paddr, section_size; - unsigned long ns, i, pnum, pfn, nr_pages; - unsigned long start, end; - struct huge_bootmem_page *m =3D NULL; - void *map; + struct hstate *h =3D m->hstate; + unsigned long pfn =3D PHYS_PFN(__pa(m)); =20 if (!READ_ONCE(vmemmap_optimize_enabled)) return; =20 - section_size =3D (1UL << PA_SECTION_SHIFT); - - list_for_each_entry(m, &huge_boot_pages[nid], list) { - struct zone *zone; - - if (!vmemmap_should_optimize_bootmem_page(m)) - continue; - - nr_pages =3D pages_per_huge_page(m->hstate); - psize =3D nr_pages << PAGE_SHIFT; - paddr =3D virt_to_phys(m); - pfn =3D PHYS_PFN(paddr); - map =3D pfn_to_page(pfn); - start =3D (unsigned long)map; - end =3D start + hugetlb_vmemmap_size(m->hstate); - zone =3D pfn_to_zone(pfn, nid); - - if (vmemmap_populate_hvo(start, end, huge_page_order(m->hstate), - zone, HUGETLB_VMEMMAP_RESERVE_SIZE)) - panic("Failed to allocate memmap for HugeTLB page\n"); - memmap_boot_pages_add(DIV_ROUND_UP(HUGETLB_VMEMMAP_RESERVE_SIZE, PAGE_SI= ZE)); - - pnum =3D pfn_to_section_nr(pfn); - ns =3D psize / section_size; - - for (i =3D 0; i < ns; i++) { - sparse_init_early_section(nid, map, pnum, - SECTION_IS_VMEMMAP_PREINIT); - map +=3D section_map_size(); - pnum++; - } - + section_set_order_range(pfn, pages_per_huge_page(h), huge_page_order(h)); + if (section_vmemmap_optimizable(__pfn_to_section(pfn))) m->flags |=3D HUGE_BOOTMEM_HVO; - } } -#endif =20 static const struct ctl_table hugetlb_vmemmap_sysctls[] =3D { { diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index 7ac49c52457d..20eb03df542a 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -9,8 +9,7 @@ #ifndef _LINUX_HUGETLB_VMEMMAP_H #define _LINUX_HUGETLB_VMEMMAP_H #include -#include -#include +#include "internal.h" =20 /* * Reserve one vmemmap page, all vmemmap addresses are mapped to it. See @@ -27,10 +26,7 @@ long hugetlb_vmemmap_restore_folios(const struct hstate = *h, void hugetlb_vmemmap_optimize_folio(const struct hstate *h, struct folio *= folio); void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *f= olio_list); void hugetlb_vmemmap_optimize_bootmem_folios(struct hstate *h, struct list= _head *folio_list); -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT -void hugetlb_vmemmap_init_early(int nid); -#endif - +void hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootmem_page *m); =20 static inline unsigned int hugetlb_vmemmap_size(const struct hstate *h) { @@ -76,13 +72,13 @@ static inline void hugetlb_vmemmap_optimize_bootmem_fol= ios(struct hstate *h, { } =20 -static inline void hugetlb_vmemmap_init_early(int nid) +static inline unsigned int hugetlb_vmemmap_optimizable_size(const struct h= state *h) { + return 0; } =20 -static inline unsigned int hugetlb_vmemmap_optimizable_size(const struct h= state *h) +static inline void hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootm= em_page *m) { - return 0; } #endif /* CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP */ =20 diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index e818b4109777..531927bc9380 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -32,8 +32,6 @@ #include #include =20 -#include "hugetlb_vmemmap.h" - /* * Flags for vmemmap_populate_range and friends. */ @@ -369,34 +367,6 @@ void vmemmap_wrprotect_hvo(unsigned long addr, unsigne= d long end, } } =20 -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP -int __meminit vmemmap_populate_hvo(unsigned long addr, unsigned long end, - unsigned int order, struct zone *zone, - unsigned long headsize) -{ - unsigned long maddr; - struct page *tail; - pte_t *pte; - int node =3D zone_to_nid(zone); - - tail =3D vmemmap_get_tail(order, zone); - if (!tail) - return -ENOMEM; - - for (maddr =3D addr; maddr < addr + headsize; maddr +=3D PAGE_SIZE) { - pte =3D vmemmap_populate_address(maddr, node, NULL, -1, 0); - if (!pte) - return -ENOMEM; - } - - /* - * Reuse the last page struct page mapped above for the rest. - */ - return vmemmap_populate_range(maddr, end, node, NULL, - page_to_pfn(tail), 0); -} -#endif - void __weak __meminit vmemmap_set_pmd(pmd_t *pmd, void *p, int node, unsigned long addr, unsigned long next) { @@ -599,7 +569,6 @@ struct page * __meminit __populate_section_memmap(unsig= ned long pfn, */ void __init sparse_vmemmap_init_nid_early(int nid) { - hugetlb_vmemmap_init_early(nid); } #endif =20 diff --git a/mm/sparse.h b/mm/sparse.h index b2dc7362af4f..b904cf3f0a2f 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -58,11 +58,38 @@ static inline unsigned int section_order(const struct m= em_section *section) { return section->order; } + +static inline void section_set_order(struct mem_section *section, unsigned= int order) +{ + VM_WARN_ON(section_order(section) && order && section_order(section) !=3D= order); + section->order =3D order; +} + +static inline void section_set_order_range(unsigned long pfn, unsigned lon= g nr_pages, + unsigned int order) +{ + unsigned long section_nr =3D pfn_to_section_nr(pfn); + + if (!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SECTION)) + return; + + for (unsigned long i =3D 0; i < nr_pages / PAGES_PER_SECTION; i++) + section_set_order(__nr_to_section(section_nr + i), order); +} #else static inline unsigned int section_order(const struct mem_section *section) { return 0; } + +static inline void section_set_order(struct mem_section *section, unsigned= int order) +{ +} + +static inline void section_set_order_range(unsigned long pfn, unsigned lon= g nr_pages, + unsigned int order) +{ +} #endif =20 static inline bool pfn_vmemmap_optimizable(unsigned long pfn) --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32CB73DA5AB for ; Mon, 20 Jul 2026 09:32:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539962; cv=none; b=ox5wZoFnSF7zKhHjc9QEML8u0aAW50UsHzskwt4Rx38p0gSfvuXd+1LaBKOcYWJmUgDZsRJmWq+Qe5KGEYfl2+AfOcGpuChNO7/242sPVpcsItJgAweOPjq9lPIw//agZ9Q/8U2YHkueUN4HufQbyBr3chs5rOYKC4cSkXj+OtQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539962; c=relaxed/simple; bh=AoqjN5Y4T/Rbx8qvY3Q/Zhal/T9D/bKOrqkOA8yAF1k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gj/HrobixVsVglA+sdA0T8gXrXhRVrr6AKLfWYbGiv6seEq3ub9rjnIkuxUAyaghxQ+v9Z8viFeUOVxuHYdAvcyuU8JtxexHiBi/5aAM6CHalrPq+w1cS7Oyt+HiEv1yvP3zbuolEtx3go+lII8KtNKY0WNt50v/iqd8zE01g6s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=lscmam1H; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="lscmam1H" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2cc891373e0so120748005ad.2 for ; Mon, 20 Jul 2026 02:32:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539960; x=1785144760; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=JZmQ/vHHln8qDG8kPEldYQhj1SA8zCGzuopSX4ZYgfs=; b=lscmam1HJ8whXkOHzSFYsGDbP6umdYF9M9DaXUsd4uNmPOs6DSHueUCQ4ct9vV1EH9 n+hBBv3mgAskT7XrA7x9mmNG1Ter8vFUzTxNbfg1HNKBqN07CmZfzSkwCJy6WP+AhBj6 7txBaCTnssRLACXXCVlDxZCR7j8QJuh3LG25rN1ZxUXkzAAf0o3veBLwIh9ThRIlYCEG urFbCpeR/wRd3y/NDRF2ulLXyrtVJarbXQ0SWX58sdlYou95/alCNx13sAZ71rIFHjwp V2oyqLNx/9RFpdgfqrBzCfWOVT2QenuvKGjoyHSbVYgqLyelwQ+LVOkWN8UfG2022t/8 KSOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539960; x=1785144760; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=JZmQ/vHHln8qDG8kPEldYQhj1SA8zCGzuopSX4ZYgfs=; b=fxseRVZ483xcEHgqlvd5bnil5geYlq6X7ouUIgKFL7cd5KU6lw0EsYdV2mmacKV6Xr e/qzp5vUZpgaS5soSrzY/HYziEEeP19bnY24/lGyZwoD/lLsEHxyGRm3wVdtiMwK1D1r 2uT1hHijWSXdWVNinzwLb4AaSJxj+bzciKQFTdNhHLS09WGtAMsOwln5vFlRcyNuHiXt d6NahfxMPGNTNW1qpO+2iU8vuYGbUiDgHf07+plUS/ZNPMGBjO0zU4/iPsuNnxqX2Ibq SMiPjPUKnzg0i+Haj2ZgnffOlHBYbS45DPCQyRIG94jZkemTb8AL8eytwHAd/SaQ+5Rh 1Tqg== X-Forwarded-Encrypted: i=1; AHgh+Rob7zSfCC8V/3nsjE1LCLU2XZdOAroadwq9SaQ0vowl6/yZHMFceTOncbdms1qQzIcGBvZMopu7auCLjVQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzVQ2EXf4rLkmAfnw5/9mohxBy/cG33+5Evb/B0ePuFvW1/J45p KR/lqvJFWHKD/uKNAeaVppc1O5Kmkz2Wu+tOmvxcbGzkcZhmfGBL8FeaxcwVD22AdQE= X-Gm-Gg: AfdE7clSPHoSFBz2lLgwC9YZN61WXEAFkkaPTIMpZ88/SOrO9X719UtENHDlVxoBVXD D+KihzwXaeRcYd8cNR68JNfmRFXo2TrAIRRXgxThOgbzTANcdrITW8I1nzjLUZdEgVIOV8JcH8O c6xxxouJFvImusloFltR7VxbqjE46aTe3dLs5TnhJLaYYNRjmw1budmB5bWT3jTbHXBNhS6EQlq r3uHLWJ0jdfgoIVho8DxCeMm9QQXyWDxopfCrqa/okb+jmTSwdI4yaxySeix4sbZmc1H61BKjUm daSLBE7+iUx27wCLqcNu+ozT6KQCTe7JXUqVLuZHN8aaqSXFKhcBdgDmiTfiVRLEPznro8WmfYk zn8gvf+sX2ed8h+Xm98c3M7tGT8RpctaZ8pWzAS+61oYBj7iInZmUjfAAIp8njZtxyFj7ndF9qh L6ewYYghl7NOSRdJpvP5jgAmE= X-Received: by 2002:a17:903:184b:b0:2c0:e5ee:f55e with SMTP id d9443c01a7336-2cf3481b67bmr141615205ad.7.1784539960299; Mon, 20 Jul 2026 02:32:40 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:39 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 11/17] mm/sparse-vmemmap: remove SPARSEMEM_VMEMMAP_PREINIT support Date: Mon, 20 Jul 2026 17:31:21 +0800 Message-ID: <20260720093127.540540-12-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" SPARSEMEM_VMEMMAP_PREINIT existed only to support HugeTLB's early vmemmap optimization setup. Now that HugeTLB bootmem vmemmap optimization uses the common section-based sparse-vmemmap path, the sparse initialization code no longer needs a separate pre-initialization mechanism for vmemmap population. Remove the related Kconfig symbols, section flag, and empty early hook, so present sections always go through the normal sparse setup path. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- arch/x86/Kconfig | 1 - fs/Kconfig | 1 - include/linux/mmzone.h | 25 ------------------------- mm/Kconfig | 5 ----- mm/sparse-vmemmap.c | 13 ------------- mm/sparse.c | 23 ++++++++--------------- 6 files changed, 8 insertions(+), 60 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 583b4cf4922a..7be1fab6f365 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -149,7 +149,6 @@ config X86 select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP if X86_64 select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP if X86_64 - select ARCH_WANT_HUGETLB_VMEMMAP_PREINIT if X86_64 select ARCH_WANTS_THP_SWAP if X86_64 select ARCH_HAS_PARANOID_L1D_FLUSH select ARCH_WANT_IRQS_OFF_ACTIVATE_MM diff --git a/fs/Kconfig b/fs/Kconfig index 64ff193fb42c..5af471024abc 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -278,7 +278,6 @@ config HUGETLB_PAGE_OPTIMIZE_VMEMMAP def_bool HUGETLB_PAGE depends on ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP depends on SPARSEMEM_VMEMMAP - select SPARSEMEM_VMEMMAP_PREINIT if ARCH_WANT_HUGETLB_VMEMMAP_PREINIT =20 config HUGETLB_PMD_PAGE_TABLE_SHARING def_bool HUGETLB_PAGE diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 025ea69f57d8..4a4310552976 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2085,9 +2085,6 @@ enum { SECTION_IS_EARLY_BIT, #ifdef CONFIG_ZONE_DEVICE SECTION_TAINT_ZONE_DEVICE_BIT, -#endif -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT - SECTION_IS_VMEMMAP_PREINIT_BIT, #endif SECTION_MAP_LAST_BIT, }; @@ -2099,9 +2096,6 @@ enum { #ifdef CONFIG_ZONE_DEVICE #define SECTION_TAINT_ZONE_DEVICE BIT(SECTION_TAINT_ZONE_DEVICE_BIT) #endif -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT -#define SECTION_IS_VMEMMAP_PREINIT BIT(SECTION_IS_VMEMMAP_PREINIT_BIT) -#endif #define SECTION_MAP_MASK (~(BIT(SECTION_MAP_LAST_BIT) - 1)) #define SECTION_NID_SHIFT SECTION_MAP_LAST_BIT =20 @@ -2156,24 +2150,6 @@ static inline int online_device_section(const struct= mem_section *section) } #endif =20 -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT -static inline int preinited_vmemmap_section(const struct mem_section *sect= ion) -{ - return (section && - (section->section_mem_map & SECTION_IS_VMEMMAP_PREINIT)); -} - -void sparse_vmemmap_init_nid_early(int nid); -#else -static inline int preinited_vmemmap_section(const struct mem_section *sect= ion) -{ - return 0; -} -static inline void sparse_vmemmap_init_nid_early(int nid) -{ -} -#endif - static inline int online_section_nr(unsigned long nr) { return online_section(__nr_to_section(nr)); @@ -2377,7 +2353,6 @@ static inline unsigned long next_present_section_nr(u= nsigned long section_nr) #else struct mem_section; =20 -#define sparse_vmemmap_init_nid_early(_nid) do {} while (0) #define pfn_in_present_section pfn_valid static inline struct mem_section *__pfn_to_section(unsigned long pfn) { diff --git a/mm/Kconfig b/mm/Kconfig index 3cec14092b28..2885fba13ebe 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -459,8 +459,6 @@ config SPARSEMEM_VMEMMAP pfn_to_page and page_to_pfn operations. This is the most efficient option when sufficient kernel resources are available. =20 -config SPARSEMEM_VMEMMAP_PREINIT - bool # # Select this config option from the architecture Kconfig, if it is prefer= red # to enable the feature of HugeTLB/dev_dax vmemmap optimization. @@ -471,9 +469,6 @@ config ARCH_WANT_OPTIMIZE_DAX_VMEMMAP config ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP bool =20 -config ARCH_WANT_HUGETLB_VMEMMAP_PREINIT - bool - config HAVE_MEMBLOCK_PHYS_MAP bool =20 diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 531927bc9380..3cfcd490dcba 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -559,19 +559,6 @@ struct page * __meminit __populate_section_memmap(unsi= gned long pfn, return pfn_to_page(pfn); } =20 -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT -/* - * This is called just before initializing sections for a NUMA node. - * Any special initialization that needs to be done before the - * generic initialization can be done from here. Sections that - * are initialized in hooks called from here will be skipped by - * the generic initialization. - */ -void __init sparse_vmemmap_init_nid_early(int nid) -{ -} -#endif - static void subsection_mask_set(unsigned long *map, unsigned long pfn, unsigned long nr_pages) { diff --git a/mm/sparse.c b/mm/sparse.c index d4641aa70497..f0d9b7263bb2 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -294,27 +294,20 @@ static void __init sparse_init_nid(int nid, unsigned = long pnum_begin, if (sparse_usage_init(nid, map_count)) panic("Failed to allocate usemap for node %d\n", nid); =20 - sparse_vmemmap_init_nid_early(nid); - for_each_present_section_nr(pnum_begin, pnum) { - struct mem_section *ms; unsigned long pfn =3D section_nr_to_pfn(pnum); + struct page *map; =20 if (pnum >=3D pnum_end) break; =20 - ms =3D __nr_to_section(pnum); - if (!preinited_vmemmap_section(ms)) { - struct page *map; - - map =3D __populate_section_memmap(pfn, PAGES_PER_SECTION, - nid, NULL, NULL); - if (!map) - panic("Failed to allocate memmap for section %lu\n", pnum); - memmap_boot_pages_add(section_nr_vmemmap_pages(pfn, PAGES_PER_SECTION, - NULL, NULL)); - sparse_init_early_section(nid, map, pnum, 0); - } + map =3D __populate_section_memmap(pfn, PAGES_PER_SECTION, + nid, NULL, NULL); + if (!map) + panic("Failed to allocate memmap for section %lu\n", pnum); + memmap_boot_pages_add(section_nr_vmemmap_pages(pfn, PAGES_PER_SECTION, + NULL, NULL)); + sparse_init_early_section(nid, map, pnum, 0); } sparse_usage_fini(); } --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B0E13DA7C3 for ; Mon, 20 Jul 2026 09:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539966; cv=none; b=K90mj8XF17tIj6wFo2jYv+ub1cqBpZV6heqoKY30kMRzWnIG+vJjQPzeO9ToPWqAypEgjt/Rv1ieqwYHGIO9Y/dhtlR8qHr8eYpvyp/3OMWKNZA+pMONzqOWsd04wCFIT2WBVSHo5S1/CC5Y5bc1kWoCEIsT0LIxtyQYo45KOCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539966; c=relaxed/simple; bh=jpFHEQEGD1796Th8d+ZFPTbYHpOBdHFjrbNWUyGmdKs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EIfpRRwvhypFQ31HQf4rQlHLgIBrY+eNU4S1TDRA3xiLhRmwEysDNv/Ct1gUmcxbLW2RGllybElrMc9b52NngfEJgzP2qFpTeEONNtDQzslAblFhJQwA8j1gf9O36Pzw3zit4UUb+89pms/eC+/SC830yGF+TPgvdy5JlU+hRbg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=RBaBvm4w; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="RBaBvm4w" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2ca64c3ce5fso111467885ad.3 for ; Mon, 20 Jul 2026 02:32:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539964; x=1785144764; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=tEa/ZPXIktSh9GylaagybywGK91Ia7MaQl8ZD0pcrE0=; b=RBaBvm4wTExi1qVjjV+kyKqlqbQfoFEkM4PHuoxNYTV0UW7Tr77xUEa3qI1d4d2BQF EmRoYLy2MChE48xj8Ni8bc04uATjKgGz+ITdQ0JnI2fK2j9/BpQwygx7+0dL+5QDB/zD QMuqJHYPFdytPKGusnrpXtEh1DQygcZPfZDKzBjB7G2vi0Q/W+ExG456h1Ls4ezgn26Y WMrdWMgZLb+VAdNfClxtxhJfq6U89HTlQTvZIb7PksXOS2eHMvIRZReVQfBVBAMGMCSU Ria+BPAGXGFzJrqJ5lDUJCyIFZea92ex2SjHtNFCDag0QWe8yOM7rIgRFJgnOuWDHrkl gNmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539964; x=1785144764; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=tEa/ZPXIktSh9GylaagybywGK91Ia7MaQl8ZD0pcrE0=; b=VaojM2Y1CBqiTIMRDAbcfC7YXwZZS2EjDFnTt8Jj1Vz4bEPn+bLWO4h/6sqm0d/RVD /vnLXwHAljiMG6uHpQVx4LwHYo6+/ay+gnc1L0quNogfmvwSHaPEBs57fmCDBEAuVW2d OQM+l4Q7g4xTbCUw4nsu1OyKGAN+utlUgTfRGXSwxgcd9XXSFzJQMZs4l/Xk45d3srfh Z4q9X4t0zzahdA+88NUS9z2mnCcljRBnC4xBePH9UqHVsVgg/18sfdgMWzWsSNxT+xKq 5m1gHlX8r+AEEnpryK62cQkJtJ1xZ1eWgOXgTd3cChXJau5l/qlDJ74641qk7I09sjGJ DyZA== X-Forwarded-Encrypted: i=1; AHgh+RqOirkdgI8jRq8+WXK56qDxnhsCajDPBnfM34F4vNEqIrJ5y5KcMagQHSeAEYJVLStoVa9Rmw8eunj51yU=@vger.kernel.org X-Gm-Message-State: AOJu0Yxdb5o4w3gRE0ryhNRmuQ/3CfrjLti6U6mOqO7cpPAaerdAefiu 9Gip/v0HsridKtSEb4F3FvOrYjXJAJID4ai09vtj97v1VYp06p+/ROYupP/y1zECdJg= X-Gm-Gg: AfdE7cmh4dgTJmlORzWZNMwbGYzjQpGcqwPbmWFH+PyNzSPeb7FH4UquVhcPpSz3lg9 f0OdeEIfyZFn6p+qlg1j/PMiHaBcB454JPsrfroJXVpFrxrbhu6Xo6XCPJU5fZWvfmk3tEOGbC8 jw0nzsmC26DhKU8ZaOwJ3wrc0hP9L9h/ls2TABhCftPufvAU5hz2eh5MJ9mrdagjhD+5YO5W/KU AzE9xFvyf33nG3OwPr3YrA+Eb81H30OaybD25ZkI5Vmr5C5m7GnEnjJGPbFvxvFBM2d8Zqkdg31 C+j4cL2AdBhfZ7CKI/Vg2OoECF1T7h6JlPVrtbvlU/ooUwcslkPEcs3tcgbcYsB4Jr5ONOzRT44 cpC/2OjYlO6jZmW32cZslw1ZwzIRrN8DdoNuYEHIppocAT4w0iThKsEe5yo5Y31gcv/rMij00Np 1lhdFEz0mH9CV2HNf5CZdjovI= X-Received: by 2002:a17:902:f549:b0:2ca:481:de45 with SMTP id d9443c01a7336-2cf348953ecmr149854895ad.6.1784539964372; Mon, 20 Jul 2026 02:32:44 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:43 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 12/17] mm/sparse: inline usemap allocation into sparse_init_nid() Date: Mon, 20 Jul 2026 17:31:22 +0800 Message-ID: <20260720093127.540540-13-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" After removing SPARSEMEM_VMEMMAP_PREINIT, sparse_init_nid() no longer needs the transient sparse_usagebuf state and its helper wrappers. Allocate the usemap buffer directly in sparse_init_nid(), pass it to sparse_init_one_section(), and drop sparse_usage_init(), sparse_usage_fini(), and sparse_init_early_section(). Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/mmzone.h | 3 --- mm/sparse.c | 46 +++++++----------------------------------- 2 files changed, 7 insertions(+), 42 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 4a4310552976..2e9e5369f599 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2213,9 +2213,6 @@ static inline bool pfn_section_first_valid(struct mem= _section *ms, unsigned long } #endif =20 -void sparse_init_early_section(int nid, struct page *map, unsigned long pn= um, - unsigned long flags); - #ifndef CONFIG_HAVE_ARCH_PFN_VALID /** * pfn_valid - check if there is a valid memory map entry for a PFN diff --git a/mm/sparse.c b/mm/sparse.c index f0d9b7263bb2..743112ac84b1 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -245,42 +245,6 @@ void __weak __meminit vmemmap_populate_print_last(void) { } =20 -static void *sparse_usagebuf __meminitdata; -static void *sparse_usagebuf_end __meminitdata; - -/* - * Helper function that is used for generic section initialization, and - * can also be used by any hooks added above. - */ -void __init sparse_init_early_section(int nid, struct page *map, - unsigned long pnum, unsigned long flags) -{ - BUG_ON(!sparse_usagebuf || sparse_usagebuf >=3D sparse_usagebuf_end); - sparse_init_one_section(__nr_to_section(pnum), pnum, map, - sparse_usagebuf, SECTION_IS_EARLY | flags); - sparse_usagebuf =3D (void *)sparse_usagebuf + mem_section_usage_size(); -} - -static int __init sparse_usage_init(int nid, unsigned long map_count) -{ - unsigned long size; - - size =3D mem_section_usage_size() * map_count; - sparse_usagebuf =3D memblock_alloc_node(size, SMP_CACHE_BYTES, nid); - if (!sparse_usagebuf) { - sparse_usagebuf_end =3D NULL; - return -ENOMEM; - } - - sparse_usagebuf_end =3D sparse_usagebuf + size; - return 0; -} - -static void __init sparse_usage_fini(void) -{ - sparse_usagebuf =3D sparse_usagebuf_end =3D NULL; -} - /* * Initialize sparse on a specific node. The node spans [pnum_begin, pnum_= end) * And number of present sections in this node is map_count. @@ -290,8 +254,11 @@ static void __init sparse_init_nid(int nid, unsigned l= ong pnum_begin, unsigned long map_count) { unsigned long pnum; + struct mem_section_usage *usage; =20 - if (sparse_usage_init(nid, map_count)) + usage =3D memblock_alloc_node(map_count * mem_section_usage_size(), + SMP_CACHE_BYTES, nid); + if (!usage) panic("Failed to allocate usemap for node %d\n", nid); =20 for_each_present_section_nr(pnum_begin, pnum) { @@ -307,9 +274,10 @@ static void __init sparse_init_nid(int nid, unsigned l= ong pnum_begin, panic("Failed to allocate memmap for section %lu\n", pnum); memmap_boot_pages_add(section_nr_vmemmap_pages(pfn, PAGES_PER_SECTION, NULL, NULL)); - sparse_init_early_section(nid, map, pnum, 0); + sparse_init_one_section(__nr_to_section(pnum), pnum, map, usage, + SECTION_IS_EARLY); + usage =3D (void *)usage + mem_section_usage_size(); } - sparse_usage_fini(); } =20 /* --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EDA93D666A for ; Mon, 20 Jul 2026 09:32:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539970; cv=none; b=c1R0a1ETAEI0AnbCInGtBqB08rkDcYHaUL58gk0KuZTFuNIiCtwoems/g0M8/dOrHaHVE4Bsmdn3yk3Tw9gzdpQKYSTMsvZjOvBzuxYpi/2EYEli+CMabNiz3ufdmS/q5f9ZBNcM+DL5WIhwq7ZJU05R9+axsz8GHCj0T25LEfw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539970; c=relaxed/simple; bh=RkxSLxVzMjNPk6HylTyW2aggjgjq95jth/O4oHCm7HU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sNAjC89rBOLBYTjEXXe5tPz1GdkvME8mXUGy2Ub1r8tPRVbOVP34kaCA285eh8p1LipDE9ezlnsgCcYdxF4jVaRiN+jYhXqmtP8KzGkOhbBAsAw4FitUInzq/ejFqXWPC5UQd1bnWDXAKzskbRo7k2FAfFlK1QQIYG5YFmvsTLg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=A0rz+oT7; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="A0rz+oT7" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2cc7e86e7aeso70236645ad.2 for ; Mon, 20 Jul 2026 02:32:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539968; x=1785144768; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=YUQR5bYOFscVYvmZ5A+sUAKgtlLdOTzx7v4nsieLPYo=; b=A0rz+oT7vMGn/sS6vtv0CdAwwvMcUxeRSbWRrqA1rUCCARcOeqYgCIK6L+/g2UFkWk cSZlxc8PPsDilialwQ1NsS5vF37whEwMDnra+f93mgXUZtqSBJ/ZZT/ModfON0mR3+7r hBqn9eof48dDeR0yh/qKveiM74K1ez35U0zO2iuWt1/rgSB6TyzWpapcu9ezbmalhyAf TFrtLTcHNIXv4kMburk2j8ny8ck/FA90Z1SBhEnawHgxJ+StdZ+aXFmpMKtDPsHvKlAU DN+y/7qpvNtpI/csw+C3ffQkyM6UNbaPnI1LwpxVR6LyPVWeyWFbT5ESbTNHOwr1apKf fDlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539968; x=1785144768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=YUQR5bYOFscVYvmZ5A+sUAKgtlLdOTzx7v4nsieLPYo=; b=Hu3e9qBkoImYuYjQ5xZCR5PdBYJBXCJotLF1+eWAf3oAES4Rrg9sbRdH+mLPodTwIX 53GjtAZs1VaXOK5KCH2fCygClfuyHyrjdaffyLFBsi2xktVaDQTn45qluPgwF44qpKS6 QJ+BJChvhFeB+WPGjc1Wa+xcZsja3UZhUSpnH8A2kXUtcpPuOcrkaET5f/uy2Eb8DbIr 7luT78X+a01kQhqRgXfHlzrHV//YfmcKeBshkUzHKxfYABj75EbJA9W8lueeJ+u7lSCs dE0Mjm2ZS6gzp8vk7Xncm3IBFuQ7Y4i7/TITc5eSYLtXgUkbdyO0KblliSQ8FnydU9gT d/rA== X-Forwarded-Encrypted: i=1; AHgh+Rqw5Co7rj8yGKg5RFkus3TM2DeYJW5PMd0mguzWs1uaJKWW8LLEOR0KVLriaz1DknCPzZCLLalkNpewMDo=@vger.kernel.org X-Gm-Message-State: AOJu0Yx83EoK48flXHSjiD8aRdmYqf1CaYmlZxU3UZuIbboZtGcPyETz rZi9VcScC0qu82mIOA/aJXCyH4ZlOjtB8Jr2uoonO78vkswfkHkedfmv7NGv/eE9QLA= X-Gm-Gg: AfdE7cmVAMZAw4jUcIoF5kyGvfbsXOkLbkHQWjX6tLtVG7/Kp+H+pL30ESgFFpnL6od 2HLRY9ctCCCj5+6pBUPcuYAoJ6M+uhy67EHxEwgr3up2XM3ajOI3wLefViiPgcNcA8+Phvy5rhw H2IzsZugIZID+oatJekta2lWGYUFnX4BuVxnOGiuiFz26kpv2kiTl1ztxzM4bxemzETJXPvso/Y qrUjxudtbwjCb1sHLnkLngdPc8Uj6ZTTo2MEkMcmR9Fn2QZK7EfMOGTwU/kem45s48cBEf6p5Ts 28Nqm9qEA1GXyr553pRRYEh/s61uMYUCutfC4krcYnypShigYn1kSCQFQEXyhhRI8KGSRdr8LE1 +5ANYY84IzYF65WwQjr9UmyHX80u1TYg8pFaF27C09DJKa6+ac0SNFOLJUfNDDyr4FKfdDZBOAQ BeVTa3dL1vcKG2jAmpy+/6Lbw= X-Received: by 2002:a17:903:2a8b:b0:2ca:ddbd:a19b with SMTP id d9443c01a7336-2cf34997827mr147433445ad.28.1784539968387; Mon, 20 Jul 2026 02:32:48 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:47 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 13/17] mm/sparse: remove section_map_size() Date: Mon, 20 Jul 2026 17:31:23 +0800 Message-ID: <20260720093127.540540-14-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" section_map_size() no longer provides any shared logic. After the sparse-vmemmap changes, its only remaining user is the !CONFIG_SPARSEMEM_VMEMMAP path in __populate_section_memmap(), which can compute the size inline with PAGE_ALIGN(sizeof(struct page) * PAGES_PER_SECTION). Remove section_map_size() and inline the remaining calculation. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/mm.h | 1 - mm/sparse.c | 15 ++------------- 2 files changed, 2 insertions(+), 14 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 1462669418b1..624c52da88c4 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -5022,7 +5022,6 @@ static inline void print_vma_addr(char *prefix, unsig= ned long rip) } #endif =20 -unsigned long section_map_size(void); struct page * __populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap, struct dev_pagemap *pgmap); diff --git a/mm/sparse.c b/mm/sparse.c index 743112ac84b1..785c0be80aed 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -219,23 +219,12 @@ size_t mem_section_usage_size(void) return sizeof(struct mem_section_usage) + usemap_size(); } =20 -#ifdef CONFIG_SPARSEMEM_VMEMMAP -unsigned long __init section_map_size(void) -{ - return ALIGN(sizeof(struct page) * PAGES_PER_SECTION, PMD_SIZE); -} - -#else -unsigned long __init section_map_size(void) -{ - return PAGE_ALIGN(sizeof(struct page) * PAGES_PER_SECTION); -} - +#ifndef CONFIG_SPARSEMEM_VMEMMAP struct page __init *__populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { - unsigned long size =3D section_map_size(); + unsigned long size =3D PAGE_ALIGN(sizeof(struct page) * PAGES_PER_SECTION= ); =20 return memmap_alloc(size, size, __pa(MAX_DMA_ADDRESS), nid, false); } --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A1A43D667F for ; Mon, 20 Jul 2026 09:32:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539976; cv=none; b=WgwNg03gOoE6hpRplsAcYQhcXusDgBfNOX8K0uBoDhioi81pZojQ3NicgbWSErAkfvA6lZhv64SY5N5sQty8nB2yVSyKiLYdaaYgiivn422wb9MZgSEn0eL3GljKGnXZuHWgy8xOqnil4vjurHel1ymrPGoGiNi73TZAjqVbMqQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539976; c=relaxed/simple; bh=DVQosjoO3IatXFLkryuTbwNd+5HzdJdOPMPCWdzzfKQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rxpcx4FXJpcoox2r75izVjXHQdf5/rvqacUL/CMxYiQrw4wp9nuNx2ncFB5jrzprZNft4eRjeW7tFix3uz9lCsalMXnsfSPpWWFzptkqTPIFzRGSphyt4MImRdW1iaGj/AT6qGYOimfyp+/k+xy94cjzt4yUtvCxqzXW4vAumoc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=dr1Ut3B3; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="dr1Ut3B3" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2cc73e322dbso116446905ad.1 for ; Mon, 20 Jul 2026 02:32:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539974; x=1785144774; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=O6r6YRMeV0eXgcqg3s9gtqIKBRl0ecB5VwywLRj0fuw=; b=dr1Ut3B3mhqtQqAhb2mePrTLGT85UVJAhTj9P1u2Xxq6vlSo+FL0HnlFFzgOn6m9Au uT6B2vdc3xW7YHhKhw+Igjuge765yO6HDde81x1MRx154Me5vhzq9GISsbCAkJFMIAKq 1cW9hDR7xhrfGNrlEYLXOLXlrolXygBGQdLPUQNm72UIo0Zxt/ZJBONZAIZqCl3JfuDL E0Sq5WAszkf4UYHC6kzquQA7CkFjDl8R1FVvkUGecPBgtztgM72R18j9xqztTFU/gyFx gQYRS3EbqBxnA1PjrrCvq+HMij+HOewT7TUa747cPm0LfxmF0Wx4IBXPXQl4ybgmUhB5 OkQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539974; x=1785144774; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=O6r6YRMeV0eXgcqg3s9gtqIKBRl0ecB5VwywLRj0fuw=; b=qjNQg6auNJYPPZqwPf6HaRlWmaF3xdBjJF1TZlOZNrLtUR+naEzl1e1hJr3s7w1OrC ypoYklOgE1lTsHVQ5bSHhKXR6kEiWi9RJ2hJAoBzp1ivWKXf57y97lPeWaZ5WSU8Rgju /ZxzDgPa2WfwMNquSJTNREqL1DAQ2G+Z74LfjvP1HCqaHSac7CSE1pfqLuAyQIbdgd+/ r5ZCK9dfTJFXRKpJf4JtrlEtkDt1XG+sS6/IxgC3Le3nG9B3WDhPRBP9Q6SISZebBIRc w4vrSCIPM68rOKJ8GBKWvM7IGTThfXwnqsfNwVmKZtii6LwW3KbcZBKfEErfFy5Ib+Kc fTKA== X-Forwarded-Encrypted: i=1; AHgh+RrHuMBFifmPqYWDZXVn87x+Hsghhnir7jBR1z1SVh3p6Bt+v19ctG/ZoAkytQ5NjrAv1Razz9kRXUF1hPw=@vger.kernel.org X-Gm-Message-State: AOJu0Yzumy4pGGbyUN4pltRXv8FnCgwBP0sIcoUM7vwZPlvWzJC/X9Wj nbsU3FGQoNNlQksLTqi3fLDyVRFNwdYBDa6TjdGbs98r2Y3TEEaRB5j55N+DMeW+qEM= X-Gm-Gg: AfdE7clzhxXYOBKnjkUNReT91GRQqqoYaX+6rdqsQ8q6bCNUVZ9VZUxnqYlMa5DSfb6 Q1YvdD5U+Hlaatgg6i6EoYo7bS5+byejJbPbUYQIoLGBzvkgoWop0hRuP1CYCcHJ/s964PVl3kI ETntQ+ywPwGXyNSbbqQgm7ORhRVQgxV5W63/1XDlmCbWB9FLZax9AOF7OQolNDeFu25Zxf0f83c MxiwrhOKszjJemSMydWnfkjD8eSUTAh6uNtNMGT/170Spa2vaTmh8b0nn40EmJo9GdMwcglRq7a yr4hAfJqyZ0AU9Gs7bcoMkXKVZ6GtCI7hKefKSyOcVZTmkCLJj4ngEr1jd8f/3ZmBWyLKGhwyBs RzliSjFxOGP/lAn1reA3gGJi0hzn0KYFpmAW6+IdeV/BjZblWS3Xm4PWgzahpziAUMt7s1g3AWp QuCWhZ/+0yWs5wn7GqomSesuw= X-Received: by 2002:a17:903:3c44:b0:2ca:ed41:d331 with SMTP id d9443c01a7336-2cf34a3f5c3mr145432835ad.45.1784539973596; Mon, 20 Jul 2026 02:32:53 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:52 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 14/17] mm/hugetlb: remove HUGE_BOOTMEM_HVO Date: Mon, 20 Jul 2026 17:31:24 +0800 Message-ID: <20260720093127.540540-15-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The HUGE_BOOTMEM_HVO flag tracked whether a bootmem huge page had already gone through the old early vmemmap optimization path. Now that HugeTLB uses section-based vmemmap optimization, that state is already reflected in the section order. Remove HUGE_BOOTMEM_HVO and its helper, and use the section state directly when deciding whether to mark a folio as vmemmap-optimized. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/hugetlb.h | 5 ++--- mm/hugetlb.c | 12 +----------- mm/hugetlb_vmemmap.c | 2 -- 3 files changed, 3 insertions(+), 16 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index fe28f98e1b22..3559041a5a57 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -675,9 +675,8 @@ struct hstate { char name[HSTATE_NAME_LEN]; }; =20 -#define HUGE_BOOTMEM_HVO 0x0001 -#define HUGE_BOOTMEM_ZONES_VALID 0x0002 -#define HUGE_BOOTMEM_CMA 0x0004 +#define HUGE_BOOTMEM_ZONES_VALID BIT(0) +#define HUGE_BOOTMEM_CMA BIT(1) =20 int isolate_or_dissolve_huge_folio(struct folio *folio, struct list_head *= list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index efe0d639933e..c03c5c805cd7 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3210,11 +3210,6 @@ static void __init hugetlb_folio_init_vmemmap(struct= folio *folio, prep_compound_head(&folio->page, huge_page_order(h)); } =20 -static bool __init hugetlb_bootmem_page_prehvo(struct huge_bootmem_page *m) -{ - return m->flags & HUGE_BOOTMEM_HVO; -} - static bool __init hugetlb_bootmem_page_earlycma(struct huge_bootmem_page = *m) { return m->flags & HUGE_BOOTMEM_CMA; @@ -3306,12 +3301,7 @@ static void __init gather_bootmem_prealloc_node(unsi= gned long nid) HUGETLB_VMEMMAP_RESERVE_PAGES); init_new_hugetlb_folio(folio); =20 - if (hugetlb_bootmem_page_prehvo(m)) - /* - * If pre-HVO was done, just set the - * flag, the HVO code will then skip - * this folio. - */ + if (section_vmemmap_optimizable(__pfn_to_section(folio_pfn(folio)))) folio_set_hugetlb_vmemmap_optimized(folio); section_set_order_range(folio_pfn(folio), folio_nr_pages(folio), 0); =20 diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index e37ac4745add..a25adc474351 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -714,8 +714,6 @@ void __init hugetlb_vmemmap_optimize_bootmem_page(struc= t huge_bootmem_page *m) return; =20 section_set_order_range(pfn, pages_per_huge_page(h), huge_page_order(h)); - if (section_vmemmap_optimizable(__pfn_to_section(pfn))) - m->flags |=3D HUGE_BOOTMEM_HVO; } =20 static const struct ctl_table hugetlb_vmemmap_sysctls[] =3D { --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 064CB3D6CA4 for ; Mon, 20 Jul 2026 09:32:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539981; cv=none; b=skWAgRVuMFyTM6OVGSEH2hicemQnAoB2lvZwMKYVFyP9nBwlWlLQAj0i21hHFMR5uxVpbEjhgdHUKd9dMG9Y9lM4a2NakKbAtAepO0OdFUwHmT0iyxSbThpGJELFWYPAnbtg3XppKZGblIIBOWxkWnkDbTwjnbw3Z41OHwjZroc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539981; c=relaxed/simple; bh=ew8ENqGqhsCjjcX+HItTQFVtUNOWnzokwik8WlK1MjM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iE5VtloD34r0zlw/xVZxabWr3qz8OGiIkeQYH+NZ1fhIZ/Fwn6RA+BkScy5fFKlVESshk6PcfIUcK6VGPlBII9Xmha5uJA2Alz4gbM29SwaacNtvR237/kDP2FFIYy/XaoMbY5L302vcXI26ImDLWgPgFroS+ceisRSMC5Gwcs4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=JXuFAh7E; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="JXuFAh7E" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-2cace91f112so100152245ad.0 for ; Mon, 20 Jul 2026 02:32:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539979; x=1785144779; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=230YrAdIWpow2s7v8XUtsXYdB2VvrVwtnmY6omTo098=; b=JXuFAh7EI5PKXekdOMTaNSm4p2d1GiEtBmDDpfE+UXDZV9b2d1P1bRmlJylWJCoJxp gq2E7Jt0VRzfdi3GzM+kZyh/OlFz3LN3eqGLAqc6hxSs7IiDvuPZSnFa/+6DnXmmeZ3O Ns6u9t49b5guPjn6VF3D/alhFWSYovtiG1eIM+4ttWEsu53snCi8Q5UgLZo1bJgMDTj2 MPQPgsrI2EI1u007WzG3koLWcdYPI+UI/qTyS+pnMRLWxXB3PeUf/SIkmM83s5snlqiz zPD5TN2UjBl7yl0GtnVrgN6QKlRbgzGXT+RvBdYym84yajOTXliLBcKmBMBf6nJ/WC77 M5rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539979; x=1785144779; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=230YrAdIWpow2s7v8XUtsXYdB2VvrVwtnmY6omTo098=; b=Uscvcq9LH+m8zkcVNkWBTCI3W82jNMi7LCWTX8KmId2p1cMsFTOKInXo8hqARVpSAb S4bNj9vIY8APbhfRQYHEXj1DviaB3YtVs+X1Ve/WQa3XIxhBpbk/f34ha6Hy5nuZ150S 7uOCcIeezWpBfmV/8Soqx1heuTwk0IyQBLxA3CETT/hYqg1xx9NsLTpcABSj5a9618xV n1PHpKT9Z7ZXkIwCC5v9cOqknLWdlHKb0mDwT8QcspzmgwxV+5eK4AtG9BNaazGF03nB 8P4FjDo18SYPDBdIumkR1/SvXuVf6nygMRJ2le4Blb4TgBBK79UE3L3b44tL46BHszzO qFJg== X-Forwarded-Encrypted: i=1; AHgh+RoEnE52jrE8oNGBcD8QBYGBPH9FasAJ+eJy/4P/7GCitVvP1awKNjxVEtlsgC/uauHyftt3xqhk5TWGFHs=@vger.kernel.org X-Gm-Message-State: AOJu0YzhRt5BlbKGd7LwGi2zyNE+SadZfBiF9g3hcMa9mF2RdMSUCCrn yh0dqDoBibHooWymi9w4zOhXNTcxmYJ+zvgwybo9aIGjyR+ePsq3PWDrGIIG2TMuSim5pXwLUOZ CtdZFTcxs6g== X-Gm-Gg: AR+sD12MnAxWGZQW3wGhuEflI7V1t/oW4PuBMMKSKxJNAsK48hey3WBE1nClla2juw9 tEei3hhlQ8qpTfYlj/zVrWLVhDpcuhmjuHI/9QctQWHT4/++zd93ch3splR23cSWtTqpa1wRzNZ nauIqkPTcbh91NFCkQqW4oNgg3wytYMs7ZKiaRhgF4HHZNR+/5I2t+y4xGMrLrSJepLu7i0fEFc VQQd7E9NZ+ux99NuRZ/MWckarABiM/4+nQN8MS1pWvbT2kN7feAFNmnZuIdGEexS73srhiIM4yZ 9fT0iQ+xtQ3iefkREkIXdFzPajTM6hRvwDyh/dZAe7c4rCriFdl8EqhEhSCgBXgfnKZAFsXXhom P29cQMG3GdrADfhqCvo3gZV6ndljNlE2dh6cAgbFxTF174rV4jlHkv5qwBkKRlMe27Dawip35DD GTRnizX0B0/rKr6ucDbnUmVO2Vpl2AIja62A== X-Received: by 2002:a17:902:f70d:b0:2c6:9f6a:9f6f with SMTP id d9443c01a7336-2cf349b8d41mr140942625ad.28.1784539979176; Mon, 20 Jul 2026 02:32:59 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:32:58 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 15/17] mm/hugetlb: remove HUGE_BOOTMEM_CMA Date: Mon, 20 Jul 2026 17:31:25 +0800 Message-ID: <20260720093127.540540-16-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Track early CMA hugetlb pages from the hstate instead of storing a redundant bootmem flag. This removes the unused helper and keeps the bootmem metadata minimal. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/hugetlb.h | 1 - mm/hugetlb.c | 14 ++++---------- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 3559041a5a57..255a258f11d1 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -676,7 +676,6 @@ struct hstate { }; =20 #define HUGE_BOOTMEM_ZONES_VALID BIT(0) -#define HUGE_BOOTMEM_CMA BIT(1) =20 int isolate_or_dissolve_huge_folio(struct folio *folio, struct list_head *= list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index c03c5c805cd7..d0c726845fb9 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3134,7 +3134,7 @@ static bool __init alloc_bootmem_huge_page(struct hst= ate *h, int nid) */ INIT_LIST_HEAD(&m->list); m->hstate =3D h; - m->flags =3D hugetlb_early_cma(h) ? HUGE_BOOTMEM_CMA : 0; + m->flags =3D 0; =20 /* CMA pages: zone-crossing is validated in hugetlb_cma_reserve(). */ if (!hugetlb_early_cma(h) && @@ -3210,11 +3210,6 @@ static void __init hugetlb_folio_init_vmemmap(struct= folio *folio, prep_compound_head(&folio->page, huge_page_order(h)); } =20 -static bool __init hugetlb_bootmem_page_earlycma(struct huge_bootmem_page = *m) -{ - return m->flags & HUGE_BOOTMEM_CMA; -} - /* * memblock-allocated pageblocks might not have the migrate type set * if marked with the 'noinit' flag. Set it to the default (MIGRATE_MOVABL= E) @@ -3305,9 +3300,6 @@ static void __init gather_bootmem_prealloc_node(unsig= ned long nid) folio_set_hugetlb_vmemmap_optimized(folio); section_set_order_range(folio_pfn(folio), folio_nr_pages(folio), 0); =20 - if (hugetlb_bootmem_page_earlycma(m)) - folio_set_hugetlb_cma(folio); - list_add(&folio->lru, &folio_list); =20 /* @@ -3318,7 +3310,9 @@ static void __init gather_bootmem_prealloc_node(unsig= ned long nid) * For CMA pages, this is done in init_cma_pageblock * (via hugetlb_bootmem_init_migratetype), so skip it here. */ - if (!folio_test_hugetlb_cma(folio)) + if (hugetlb_early_cma(h)) + folio_set_hugetlb_cma(folio); + else adjust_managed_page_count(page, pages_per_huge_page(h)); cond_resched(); } --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BEF203D6CAA for ; Mon, 20 Jul 2026 09:33:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539986; cv=none; b=Qt9GMNmLfnpwmCRsCzDM9tQxRH7xbG/Ezj5MOFejAJu+5NvrqZ3OGfd79RrsHVMeoOZ3MxY5J5QHhT0Zw3wBHdOGi88AO6od134pvCdmLPNrYm90f16rEaQ0RmQPRt7WsBYwFSbOM8B6alMWJhRx7qt3xfq91EiK6Pqlym+0GUc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539986; c=relaxed/simple; bh=u1V+4ZUlNV9cMHZPYL3/IxXNlkGMBGQGdNmABRsX1jY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KAvSrm2/0WYANsuyClWCKngH7jdgCg5slCSQmJzca8NS+XAGEcOjS1DxL190EgaIOeJCeHnWMzFn3PSVdWVIr19xvPd5kmxexEfusa+XVS7FG2gblK8yvwca0kIhTty7+6j/xxAcU0APywCEUQfaFLTm3w+QHJxyXG4zNhHRpp8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=kqzQImqf; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="kqzQImqf" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2cc73e322dbso116448765ad.1 for ; Mon, 20 Jul 2026 02:33:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539984; x=1785144784; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=tj81g8LuTQ8+sncASIkT/kKzX02ru341igJMpdvJrHk=; b=kqzQImqfd46v+S+4pKVX7Kxf6OUHEZG0EuhVc1Um6K8xUPp5qBDrboQMKpKuP4Pxof iuTh+BJssZyhxMXStG+YWq5oba6SoJryZrrWZHRPbpHC8EE1gejzexUx4aKelLwdoo8/ ux79YCcOBcHA/lCt1FIBdxslfSdB+tAkcZCIQFlJBeuRR4VJ4FX5n/k0k8T+JPI34ImC +DJjp3BpjCTYsgGwaQW1JJ/RnyeVLmUugKDhb2J3OC0b64/HKm3vYUb4FdKDla6ri9DY JgSItXmHW49sxxSIsHBaRB2U9/CwHiSR/y4mVQy/OEi5KvXmy6mYNcOemp1HVqgPRf+D 0JkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539984; x=1785144784; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=tj81g8LuTQ8+sncASIkT/kKzX02ru341igJMpdvJrHk=; b=UyDbthRIFTt9r6raOFfaaBD2rckFgKD7lZliIHH6TI/98f7Rk5xPk/D/zuIca0IbTB yJUaBucmQR7FeJ6zysQDsinzB6Zk7KQzhA74vl0QcqiNLGyjRInDrMjwoKuZCw2+X8Hl TI0LI020INjVUJ2iINjbVb53VHjxsT8t9lKOGLW488ipbug/CzcQJd15zgnzXroo/xZw moOZwnnHp3GXSma81ielCZ38pEukL6AxzbKDadNo+fJY1lRmZQjBSJZPsM2s6UR0c9YM RJl8kWpwp509BiIluxVZO2uyFb5yV3ok839mdGRtvNDF/hbXm0yAtVLlqbOzGfJwQA6s W/Lg== X-Forwarded-Encrypted: i=1; AHgh+RoU1NwogBETTKXDgWii6qNuRGV/CcbIb6QntM/RPB8FFhJGUo22mVOiuUKqNTgYvIVgle8W/nQLypZ175g=@vger.kernel.org X-Gm-Message-State: AOJu0YwVXcG/SMxFSql3zuHvIonTDrB4/udzDOrplgMSyRKA/vWFUbHm H/igkisrNaXYOwYQjIA/GW3VqdSecfq0NE4vVRoCywmx88TOZZa/92P66iOy81dBtOs= X-Gm-Gg: AfdE7ckH0gVGkDBNDJhyNs/Wvu08wI0u5J+gmJKcdsKUc5xme+idgZoIcn5iDyWqShW PxoqRfgGVYgYWrIfs8Zh8McMnf6gdmY6xJJsaQ6bdJPbclAB/gX+sawaqqyf1Cw3u1vBFPqr6VH UXfCG22Mbf8ObLPvkIbRhagUYHloxOnmR9QhvzH9sRIlXUOvQHdx/7Y3/F2SLjE/QjKGSZn5VpT iEuOQlki+lyuqPE8AiL3oVdpxz0vRgj82LoKytRzpqgzs/1WS11yMNY95tHeHTJeyfqkolsW4Dt 86oNNIq4q3/udZzLHksLJHI+BW3DcKORe+IKdJaEVt+LHS7ikbazdR9K6sE2Avz4nZrZEq5RknL 3wm7AktBSnhiAEP4YLLQPm7UpK6H6meIQvxFxqFNmQzuZFIwfY+YIU/7O0b6OHdgU2OWPpdmSPm x2XpJiuj0D+FHnKe8KDvioYdo= X-Received: by 2002:a17:903:2301:b0:2c9:aae1:a611 with SMTP id d9443c01a7336-2cf349bb7cbmr136649455ad.23.1784539983964; Mon, 20 Jul 2026 02:33:03 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.32.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:33:03 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 16/17] mm/hugetlb: localize struct huge_bootmem_page Date: Mon, 20 Jul 2026 17:31:26 +0800 Message-ID: <20260720093127.540540-17-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" struct huge_bootmem_page is only used by hugetlb boot-time allocation code, but its definition currently lives in mm/internal.h because hugetlb_vmemmap_optimize_bootmem_page() takes it as an argument. This exposes a hugetlb-specific internal type more broadly than needed. Change hugetlb_vmemmap_optimize_bootmem_page() to take the information it actually needs. With that interface, mm/hugetlb_vmemmap.h no longer needs to include mm/internal.h, and struct huge_bootmem_page can move into mm/hugetlb.c. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Pass PFN and order to hugetlb_vmemmap_optimize_bootmem_page() instead of struct hstate and bootmem metadata to simplify the code further - Collect Acked-by from Mike Rapoport --- mm/hugetlb.c | 8 +++++++- mm/hugetlb_vmemmap.c | 8 +++----- mm/hugetlb_vmemmap.h | 5 ++--- mm/internal.h | 7 ------- 4 files changed, 12 insertions(+), 16 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index d0c726845fb9..364fdf254e78 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -55,6 +55,12 @@ #include "sparse.h" #include =20 +struct huge_bootmem_page { + struct list_head list; + struct hstate *hstate; + unsigned long flags; +}; + int hugetlb_max_hstate __read_mostly; unsigned int default_hstate_idx; struct hstate hstates[HUGE_MAX_HSTATE]; @@ -3152,7 +3158,7 @@ static bool __init alloc_bootmem_huge_page(struct hst= ate *h, int nid) } else { list_add_tail(&m->list, &huge_boot_pages[nid]); m->flags |=3D HUGE_BOOTMEM_ZONES_VALID; - hugetlb_vmemmap_optimize_bootmem_page(m); + hugetlb_vmemmap_optimize_bootmem_page(pfn, huge_page_order(h)); /* * Only initialize the head struct page in memmap_init_reserved_pages, * rest of the struct pages will be initialized by the HugeTLB diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index a25adc474351..eb339c4a71f4 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -19,6 +19,7 @@ #include #include "hugetlb_vmemmap.h" #include "sparse.h" +#include "internal.h" =20 /** * struct vmemmap_remap_walk - walk vmemmap page table @@ -705,15 +706,12 @@ void hugetlb_vmemmap_optimize_bootmem_folios(struct h= state *h, struct list_head __hugetlb_vmemmap_optimize_folios(h, folio_list, true); } =20 -void __init hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootmem_page= *m) +void __init hugetlb_vmemmap_optimize_bootmem_page(unsigned long pfn, unsig= ned int order) { - struct hstate *h =3D m->hstate; - unsigned long pfn =3D PHYS_PFN(__pa(m)); - if (!READ_ONCE(vmemmap_optimize_enabled)) return; =20 - section_set_order_range(pfn, pages_per_huge_page(h), huge_page_order(h)); + section_set_order_range(pfn, 1UL << order, order); } =20 static const struct ctl_table hugetlb_vmemmap_sysctls[] =3D { diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index 20eb03df542a..464192e32dec 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -9,7 +9,6 @@ #ifndef _LINUX_HUGETLB_VMEMMAP_H #define _LINUX_HUGETLB_VMEMMAP_H #include -#include "internal.h" =20 /* * Reserve one vmemmap page, all vmemmap addresses are mapped to it. See @@ -26,7 +25,7 @@ long hugetlb_vmemmap_restore_folios(const struct hstate *= h, void hugetlb_vmemmap_optimize_folio(const struct hstate *h, struct folio *= folio); void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *f= olio_list); void hugetlb_vmemmap_optimize_bootmem_folios(struct hstate *h, struct list= _head *folio_list); -void hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootmem_page *m); +void hugetlb_vmemmap_optimize_bootmem_page(unsigned long pfn, unsigned int= order); =20 static inline unsigned int hugetlb_vmemmap_size(const struct hstate *h) { @@ -77,7 +76,7 @@ static inline unsigned int hugetlb_vmemmap_optimizable_si= ze(const struct hstate return 0; } =20 -static inline void hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootm= em_page *m) +static inline void hugetlb_vmemmap_optimize_bootmem_page(unsigned long pfn= , unsigned int order) { } #endif /* CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP */ diff --git a/mm/internal.h b/mm/internal.h index f26423de4ca2..9d3196f9d19e 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -23,13 +23,6 @@ #include "vma.h" =20 struct folio_batch; -struct hstate; - -struct huge_bootmem_page { - struct list_head list; - struct hstate *hstate; - unsigned long flags; -}; =20 /* mm/workingset.c */ bool workingset_test_recent(void *shadow, bool file, bool *workingset, --=20 2.54.0 From nobody Sat Jul 25 02:43:53 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30FAC3D75BE for ; Mon, 20 Jul 2026 09:33:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539990; cv=none; b=EFdE/bm0R15z5MPDsNibH1Nv/Xt5npYSXxbFfGfeslYEP6fs+2sYfFWshNbLefEJmn5M9t+s9WHT1JT4SdZMmo0zfhH/o287H3FSWtk1yKRGHwZefCHmVl5RUmkkrUZ8/lJ7JIonOmLYjAx3Hg3yo/LQAN+qXd4U4y5Hxn7TCPI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784539990; c=relaxed/simple; bh=dWkmKKbC9a4VMZAP8AVtWLJZSwM6xjF4UQcY5gXe+Gc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TQ63iCS5SW62P6LhUh01BbzKpFlc7IRxRDXUw/IC4upvcvX/avRgKVVe3ch8Rr21gnZLTyIEDUWM7HlA37pvLiGSI0lk/nPIykF8UsYbIdWykz/BqKkCsVgbng5JRPfg1Kc2hrk3BCrCLvz5Zmb4F2meB1tH2GTxXZ1En4YnhTo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=aLz77jaF; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="aLz77jaF" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2cad4170e8eso134044705ad.3 for ; Mon, 20 Jul 2026 02:33:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1784539988; x=1785144788; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=cX0YJdXVCNf3FoytxuzouNIakKfOBk60qEk143M0obg=; b=aLz77jaFGOUl+HnCyQcJrd80YEMj2+oVL7Kgax4bndEJiICsfjvYy5oL1qPvZMQhZg FcXm8pkpLOnfV3nd4CkMkutv9JTeYnYncly3Pa51zLHCvKgd7l3b1WhCrXejoSH15qML PnkqfYQ7IxM5rurju0IcFGxBf9gq05B9MKwiTUi97iZBWRKBptHtc8M97mSM+XLxtuBZ K/fa+TeNcH4jQiCpgZCsvUUnehp5AX8sVHWdx6Wotp7xLNroMmIfZymdPuSIHeDJJ63n 4yu/b+cPK5QfB8Aw8DCQsnk0ctXxQWDVUOXKiyuicTPnbnLbkDj5tNXNjlScSaPugtP1 /BWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784539988; x=1785144788; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=cX0YJdXVCNf3FoytxuzouNIakKfOBk60qEk143M0obg=; b=W1hGYHkDbqrt12+e8L0u10hezgVk69FnTTW0FRoQLN2ctUASSnhImeejSrXj97OdC2 NyuVaFg9xW4lWqwctqOzh/bWQhephIY0imeQZAiO3FybEkPfIdcHjb7raQXEexwc5oWt GaQhBfRl8jEmUp8iL2Nrrh5gYP1uSCXiWjVTRMG1F0/LocNg+rxIov1O7568i8qV2e81 u2LysKWYK/R57MoEyyYg5O426sSjZoOc5NHF5KE6BBx4n3QbDzmgQSNIciH2lHbRGbld WWS6MvMw85y4lw23KPJT/LwIq1EA58owROPFpfUbDAGD+/zQ29kFIj3FqTI5/TtMcQLb wu9A== X-Forwarded-Encrypted: i=1; AHgh+RpnYx5ym3ncAgg1lS437uX8r74gQy3q7pzspceIUgUKBZOtknN1UwO6TliTICRDdOZqQops7T11YKs4QeI=@vger.kernel.org X-Gm-Message-State: AOJu0Yy7R4Mo4vQTVIWumj/hbOx+op708w9zSipvwMZqHAXDQ+kdxgOY 1e8FyfnFLgkUGLUByCGhPEb7MUOAWCKzEUzchfyZzFqjXoW3Bxpsh4/t9uTfKAxTP0I= X-Gm-Gg: AfdE7cnUtLKRgYpmZpNHtmJ1LRa9v4zXTEb1uLRacd62YBT3eHhfL/m21Df95IxGM89 1VevEi6FGxZB20PDO5bnqqpv2IK9w1EMxU5Ok/Rut/ZBi5xujmopdW3uF5nxstmC3zoynuhuSgv bzNeYgozu4bxtc78ikPTm5KsngUIoNGJ3MVg6AuvfdzEUB4jMFLP/mtIRJPtRx9UA0EjHYLFY/8 H1Zk/feaKwPY7obinyTV6PE9+svDuFUamWFeiwXr20Fhbc51luyJTTNhFtNXtqomrMH7SOSurfN QS/BH3MSsKydixEL6FqaoIil0ARXsDFl7zGYv3t+jWw1/evU/xkxLXSuUsGH6meLq2vCB5EFxXp 36Gy8mcjVBfAzwiCvdgvWJq/PDoSaj4Lu4A4NUYUoa+QCYuZ5cyiPoJFGcsNGh97OQNCBTne/T3 8JrHLZRYEkeiLKT3dFD2vtIaA= X-Received: by 2002:a17:902:cec4:b0:2cc:6329:b7ba with SMTP id d9443c01a7336-2cf34857697mr136542375ad.16.1784539988367; Mon, 20 Jul 2026 02:33:08 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.101]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf347a7c9fsm53136455ad.84.2026.07.20.02.33.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 02:33:07 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v2 17/17] mm/hugetlb: localize HUGE_BOOTMEM_ZONES_VALID Date: Mon, 20 Jul 2026 17:31:27 +0800 Message-ID: <20260720093127.540540-18-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260720093127.540540-1-songmuchun@bytedance.com> References: <20260720093127.540540-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" HUGE_BOOTMEM_ZONES_VALID is only used by the huge_bootmem_page flag handling in mm/hugetlb.c. Keep the definition next to that private data structure instead of exposing it through the public hugetlb header. No functional change is intended. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/hugetlb.h | 2 -- mm/hugetlb.c | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 255a258f11d1..900c95e346b2 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -675,8 +675,6 @@ struct hstate { char name[HSTATE_NAME_LEN]; }; =20 -#define HUGE_BOOTMEM_ZONES_VALID BIT(0) - int isolate_or_dissolve_huge_folio(struct folio *folio, struct list_head *= list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); void wait_for_freed_hugetlb_folios(void); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 364fdf254e78..47e3147ddc12 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -55,6 +55,8 @@ #include "sparse.h" #include =20 +#define HUGE_BOOTMEM_ZONES_VALID BIT(0) + struct huge_bootmem_page { struct list_head list; struct hstate *hstate; --=20 2.54.0