From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E76F73F39F1 for ; Tue, 4 Aug 2026 03:55:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815750; cv=none; b=D9jMCnd8tskfyfx5FmGUcR735D48LLgGyABz6iNTAoJyYxwbaT+RNzbZgiCSamZjOrILi8IIrtcy7RT9obGkXXY27OCQOsjKsEB5Nqfnz7VB8hrEu5vihN+2qeYjzPvD11sCyL9jrK4k3YftwOzW1BfAl8FGfT9asliPj4U77kU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815750; c=relaxed/simple; bh=eBaZmxrrKNzPfQOLnBFfEL/leoSxI1cGXiOdwYMNA0g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NDCoITnNux124WbMzcs7KbpLxcRNDnCO3dl6rZZvOAZLA2+AcOpAxzfwz3x6mt++seby5r7BHOlVzF+/8U5mg8D7jFOe3UKSs8NPyDI4gg2VjhJrpr8vz8CXlIT6CT0Vn/Rfj4p+PRlMbiXxVXSm2hH9qurIyVSN8G1y2tg9hkc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=KFkwuroU; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="KFkwuroU" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2caced6038eso6997185ad.0 for ; Mon, 03 Aug 2026 20:55:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815748; x=1786420548; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=yICpqG88O94O0jXrkeJyzojk9mtxZcw6jMcQUVcEauY=; b=KFkwuroUpheUSxMsSYZa0NYf79qhwECLrp3mRJCnYt+F2yRxTWjPbSNccldoz9WkBt 0d5jqPgWczZ4WpNKG5PqKF5zIHIoRXkFNVzt2ViEE3yYEX6yKOIx6DcSWsbATpT6HEut I+a6sBV5Lbjl0YHVM/d/Yzb+7n9tywccYV8u5fLyPl606o4YqUgBnRaj/Foq5b27UmQx hUVnDAYOGstbLHXzijcwPw6gOQ1AbmyDRGS7EFtUlCcx20lkcG2MD5qB0zOdZD7+HbWQ TY5Dp9ifz2ABMiMUTaMVGeCzKm0P27IJg+L3LBTRbcbDkDWo/RTuepq8/+rUBAPIpUyX LRTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815748; x=1786420548; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=yICpqG88O94O0jXrkeJyzojk9mtxZcw6jMcQUVcEauY=; b=nSTjDQYBQQLf1fY1r2ffWPKMv9J4HKErAwUr35ehzgCdydfO9l9WbgMNjgoJ/PpfI4 UatUXc0HIDE2ij9cdIsovNa49/UEMrPnJFjl4j+Xj1eziXQXQfzDQXkKUjnsk+ehsCRF 0NRw9ezHTSflM1lTCWjZaBkm2kFFbRUYfNl/+uprUVfKIYJ1rI49lPJPkM3rYKovoaBm 5n2Gi+5/tN/EAoEMQWw/D4QxVOH0uVV7i5FXKmFXZoQysqWeINxZgF5QHyEvG8DEIg6e /MKfPq2HA+5yAAyqA3D7nnJ1yq8SHbzzZCLmpP4ix/7NswK+eCiaOeHGSb/y73VTcIJb mMtQ== X-Forwarded-Encrypted: i=1; AHgh+Rp6YgtFkLEr6CBZFC4pQKptK1TPqBPIFhsDrYzHKFBAL+WSPX11f726eYstj+k2cUsUHP61y7j4bnGOXa0=@vger.kernel.org X-Gm-Message-State: AOJu0YwG91phqYDV0eDaRoAMGFyIDHCFHO50nntjinT37v4CxiTw/TLd vhJVNuXWzPLCM6CAPzXbH+fmPyUZwYhJn+Ym6H1LeIsxxuhlr93KXi5FpsHHTiLbJ9s= X-Gm-Gg: AR+sD125cRNDA+4pMvXtSSW9Lz7Ox8jdMybV88AD5Hzo3Yo3i1bmWPg5P9mBVTum5bz Kkg5OIbfYuBpWWT/XeV5t0EFYp5uW86qscGZ9EHXjp7Jpeg/B/m6cEFRirW66mRYsL+QXs0ZbVt gVcaUhd/nYWbZxWcs9lH4M+zTkT/Nw1M0lTRC0H0B4mhPWKbs+pzhTHQQfFiq9v04Sn4hsG6we8 pbSzx1sHE+S0gBETVHbLG8dwChxDjZvcaLbVBaMvGCBZyvsS6xaqFfi/YKSjHRKJ8SoHAJwyR1e znhqcWjMYz+4JsQtqHKTG57sPyYy0gijzi83SDvfly/Dyf93B9l01aLPRs1HU1tpOqCi09iI0l4 j9kF26gDPMkHvGtKjAubQFf28HzrG0nLfl1Oftu6tPrq1g1gLC+x44olil29+GKKhD5WwhauiY5 x8yHhy/CJiwUQwgCjU5Fp7CPiO9us7Xti9UhbQ5N4zhqJ3AdkKOQoUliKTWcEpYlxbhgq/qhpNB 5xtp8nw X-Received: by 2002:a17:902:ec87:b0:2cf:bbdb:425d with SMTP id d9443c01a7336-2d08ab5dee1mr17986345ad.18.1785815748049; Mon, 03 Aug 2026 20:55:48 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.55.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:55:47 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 01/17] mm/sparse: relax struct mem_section size constraints Date: Tue, 4 Aug 2026 11:55:19 +0800 Message-ID: <20260804035535.2846016-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" struct mem_section is currently forced to a power-of-2 size so the section-to-root lookup can use a mask instead of a modulo. That requirement makes future extensions harder than necessary: adding a small field can require configuration-dependent padding or layout checks just to preserve the lookup scheme. Keep the lookup correct for any struct mem_section size by using a plain modulo instead. Do not leave the layout entirely unconstrained, though. Keep struct mem_section double-word aligned so modest size changes, such as adding another word-sized field on 64-bit systems, still keep a compact and efficient layout. If future fields grow the structure beyond that sweet spot, the lookup remains correct; only the exact layout efficiency changes. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Align struct mem_section to power-of-2 where possible, but support non-power-of-2 lookup (suggested by David Laight) - Collect Acked-by from Mike Rapoport --- include/linux/mmzone.h | 11 +++-------- mm/sparse.c | 2 -- scripts/gdb/linux/mm.py | 6 ++---- 3 files changed, 5 insertions(+), 14 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index a26c8b855222..acf02c87900b 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2028,13 +2028,9 @@ struct mem_section { * section. (see page_ext.h about this.) */ struct page_ext *page_ext; - unsigned long pad; #endif - /* - * WARNING: mem_section must be a power-of-2 in size for the - * calculation and use of SECTION_ROOT_MASK to make sense. - */ -}; +/* Sacrifice minor padding space for efficient lookup. */ +} __aligned(2 * sizeof(unsigned long)); =20 #ifdef CONFIG_SPARSEMEM_EXTREME #define SECTIONS_PER_ROOT (PAGE_SIZE / sizeof (struct mem_section)) @@ -2044,7 +2040,6 @@ struct mem_section { =20 #define SECTION_NR_TO_ROOT(sec) ((sec) / SECTIONS_PER_ROOT) #define NR_SECTION_ROOTS DIV_ROUND_UP(NR_MEM_SECTIONS, SECTIONS_PER_ROOT) -#define SECTION_ROOT_MASK (SECTIONS_PER_ROOT - 1) =20 #ifdef CONFIG_SPARSEMEM_EXTREME extern struct mem_section **mem_section; @@ -2068,7 +2063,7 @@ static inline struct mem_section *__nr_to_section(uns= igned long nr) if (!mem_section || !mem_section[root]) return NULL; #endif - return &mem_section[root][nr & SECTION_ROOT_MASK]; + return &mem_section[root][nr % SECTIONS_PER_ROOT]; } extern size_t mem_section_usage_size(void); =20 diff --git a/mm/sparse.c b/mm/sparse.c index 704a9dec2b9a..ca9875f568d3 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -332,8 +332,6 @@ void __init sparse_init(void) unsigned long pnum_end, pnum_begin, map_count =3D 1; int nid_begin; =20 - /* see include/linux/mmzone.h 'struct mem_section' definition */ - BUILD_BUG_ON(!is_power_of_2(sizeof(struct mem_section))); memblocks_present(); =20 if (compound_info_has_mask()) { diff --git a/scripts/gdb/linux/mm.py b/scripts/gdb/linux/mm.py index dffadccbb01d..da4e8e9655a6 100644 --- a/scripts/gdb/linux/mm.py +++ b/scripts/gdb/linux/mm.py @@ -70,7 +70,6 @@ class x86_page_ops(): self.SECTIONS_PER_ROOT =3D 1 =20 self.NR_SECTION_ROOTS =3D DIV_ROUND_UP(self.NR_MEM_SECTIONS, self.= SECTIONS_PER_ROOT) - self.SECTION_ROOT_MASK =3D self.SECTIONS_PER_ROOT - 1 =20 try: self.SECTION_HAS_MEM_MAP =3D 1 << int(gdb.parse_and_eval('SECT= ION_HAS_MEM_MAP_BIT')) @@ -100,7 +99,7 @@ class x86_page_ops(): def __nr_to_section(self, nr): root =3D self.SECTION_NR_TO_ROOT(nr) mem_section =3D gdb.parse_and_eval("mem_section") - return mem_section[root][nr & self.SECTION_ROOT_MASK] + return mem_section[root][nr % self.SECTIONS_PER_ROOT] =20 def pfn_to_section_nr(self, pfn): return pfn >> self.PFN_SECTION_SHIFT @@ -249,7 +248,6 @@ class aarch64_page_ops(): self.SECTIONS_PER_ROOT =3D 1 =20 self.NR_SECTION_ROOTS =3D DIV_ROUND_UP(self.NR_MEM_SECTIONS, self.= SECTIONS_PER_ROOT) - self.SECTION_ROOT_MASK =3D self.SECTIONS_PER_ROOT - 1 self.SUBSECTION_SHIFT =3D 21 self.SEBSECTION_SIZE =3D 1 << self.SUBSECTION_SHIFT self.PFN_SUBSECTION_SHIFT =3D self.SUBSECTION_SHIFT - self.PAGE_SH= IFT @@ -304,7 +302,7 @@ class aarch64_page_ops(): def __nr_to_section(self, nr): root =3D self.SECTION_NR_TO_ROOT(nr) mem_section =3D gdb.parse_and_eval("mem_section") - return mem_section[root][nr & self.SECTION_ROOT_MASK] + return mem_section[root][nr % self.SECTIONS_PER_ROOT] =20 def pfn_to_section_nr(self, pfn): return pfn >> self.PFN_SECTION_SHIFT --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B741833F582 for ; Tue, 4 Aug 2026 03:55:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815754; cv=none; b=slsS4I136VXzfjgf2mnIhpucuZpVSAgOrUPi+JdzESs9+VWILBXuoEnLB3IDeyEe2Kqy+TjgNhiUSmv2EdFk8+07FuJbP+5vb7GlvDlIjn3CjMGrMk+1VanrGgvvuoFz4TH7UeRm9zWaq8JnwtMlb1Gx3PYvjfqvpxArKxB/wgw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815754; c=relaxed/simple; bh=6DQ2DC5d0nbkF0z3SDMQ6pW/7nObJaWDgVOHt5A3hSA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=W0jZFx8UTMAOeq0lI7t4OGG+FYlHDCIhdon0j7SOCe5BlvPFO2f6GBM6zvMgruNZpBW7olBP3MlsLIDXWhfT4QZPRxjZFp5s2Mum8gENgeKtHqp/AKGmLZsRqn9Z3AtP++dRB2ju5G2sWARKMA4y6PYGrB6gOOpXOA+kpS2Z/tE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=h0OHFvcs; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="h0OHFvcs" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2cace91f112so46099515ad.0 for ; Mon, 03 Aug 2026 20:55:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815752; x=1786420552; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=gqHmNnl5B8M8iecx70lwawtohlkJTRxAzYxhsQadj3M=; b=h0OHFvcsskGxakLJp245rB131cZyDX20Oni9HwTt3MJ+fGMsP0VsR9XiW7m26d6GG9 REVptvcWeWagYWAok+v7N8xBbIu7cJ3LbUxQkrnGUaJBc0omWBHdDO3k/X6x/TOfwrhs fNuoEW02z62JGwYTuJpmjyFCh3ylrpC+7z0PshaDP68xhr4wQkDH9HlPqkbCft622zMM F9SjUAPlSz4Db9QlA2l7jXG9mZCTFjpefYGBfrBSJPZXus/cYHSWSdgYRekuI85usYnq N1YGWCs7y9UVic7Rck8RjaWY9ky1d8E690xbgEs07OKdZ0sr69hnOn/4ZIzS9TI9Hc1h wy4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815752; x=1786420552; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=gqHmNnl5B8M8iecx70lwawtohlkJTRxAzYxhsQadj3M=; b=AgdtQ7IKbHX4cxA0cgvGITiIMeKOsgRER+RQ35oqSQYalrEXHlaPMYDc41PxyBqt5B XefjwCdilbr8z71npn3HP4fau2lKYfwYysf9urav8hQ619izBUKVvZMnPhEV2bIY8E8H z29R2vscdooILecVTM6Fagxk0Iq9Zo1KcKkNt7josthzQuPZfIRBkJMhIPoy4qPB3M1E X7L8Cso0vblN0COWg72LkQKUB2dXrxYaJQHEktDYLNHtWcRUFNuJI3H53602hZ56RSM2 TsIeyj5ZoAmMy2SR9YTZ8sTboNVOpDxmTLCjosiuD7kn4MTMOzHkNaFTIcn7NU6IjAUZ byrg== X-Forwarded-Encrypted: i=1; AHgh+RovpRv3ouEwPSXEz9OWIFbXqGF7pgv/5SHWxtB+UIcE2s1NBPoJ8fHOlt560wok3x+PUU9mKGXrnxj7m0c=@vger.kernel.org X-Gm-Message-State: AOJu0YyG20Y06iHp/3WkkUQ5ZnXNg9Ve4AKf0H9a3qEnymNswwoZ2gOO 1YWyAWld4BHMlfojTVTNpI7sCHF6enFX3sWk5H4fpCBSbCx+mw4H/aDPRAsPucX9sYc= X-Gm-Gg: AR+sD10fOWTZp42MJ60dJTXkYW1nwykwHbxhx4hJ9PdctPUcZOY4xrJTbQO0yRGRNQX +Kwixdm5nZwJzcyO7j2mKZktRkzf+H725ypGLNviXFqd8wfZ+2QwQrgZXLdJ4p+M094no439x1e ny81VCYXibYXE8hy2nf/92QO/S2oRmP2V5rBp4NoS2qdPLjHDlTly+YmaFHkyJ16zmtAXRkE7Mi 3BnCc6owtw5pM/W/X/hJqLlsIkFgS96Bt7/Jw+7JQZu/OBEimrNnwOfUcTnVQ265QJMmNyt9eCN ctKy1dEY4icLziQWqvO0uh3SHpkj7ibq5ZBayKTOFRK4KZNpJCzV0JnTCked6n80owL1U2Rr+BS X7arDBUnhnNSkAQPru9WNjga7l8sCWBmkxrDyP/6EmyhkR7/JO7KPqn0TtOopre7lScp7hS2GO0 ZWWVHo6cFROc/RNqGGtCCLbr9Zw/5aRUHOHyJFutnQyLQiwcX+NU6RPC3XzUOvrizKnepFJzMeo gaW6izQ X-Received: by 2002:a17:903:2f4b:b0:2ca:ec6f:aeca with SMTP id d9443c01a7336-2d0521bcb22mr129595065ad.8.1785815752030; Mon, 03 Aug 2026 20:55:52 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.55.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:55:51 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 02/17] mm/sparse-vmemmap: rename HVO order macros Date: Tue, 4 Aug 2026 11:55:20 +0800 Message-ID: <20260804035535.2846016-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The macros VMEMMAP_TAIL_MIN_ORDER and NR_VMEMMAP_TAILS describe the order range where HVO can be applied, but their names tie that range to the tail-page cache implementation. Rename them with a VMEMMAP_OPTIMIZATION prefix and use the new names in the HVO paths. This makes the code describe the optimization requirements rather than the tail-page cache implementation detail. No functional change intended. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v3: - Collect Acked-by from Mike Rapoport v2: - Rename the macros with a VMEMMAP_OPTIMIZATION prefix (suggested by Mike Rapoport) - Drop intermediate optimized-folio size macros (suggested by Mike Rapoport) --- include/linux/mmzone.h | 17 +++++++++-------- mm/hugetlb.c | 4 ++-- mm/hugetlb_vmemmap.c | 2 +- mm/sparse-vmemmap.c | 4 ++-- 4 files changed, 14 insertions(+), 13 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index acf02c87900b..f96a45ac558f 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -107,13 +107,14 @@ is_power_of_2(sizeof(struct page)) ? \ MAX_FOLIO_NR_PAGES * sizeof(struct page) : 0) =20 -/* - * vmemmap optimization (like HVO) is only possible for page orders that f= ill - * two or more pages with struct pages. - */ -#define VMEMMAP_TAIL_MIN_ORDER (ilog2(2 * PAGE_SIZE / sizeof(struct page))) -#define __NR_VMEMMAP_TAILS (MAX_FOLIO_ORDER - VMEMMAP_TAIL_MIN_ORDER + 1) -#define NR_VMEMMAP_TAILS (__NR_VMEMMAP_TAILS > 0 ? __NR_VMEMMAP_TAILS : 0) +/* The number of struct pages covered by the retained vmemmap pages with H= VO enabled. */ +#define VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES (PAGE_SIZE / sizeof(struct pa= ge)) +#define VMEMMAP_OPTIMIZATION_MIN_ORDER (ilog2(VMEMMAP_OPTIMIZATION_NR_STR= UCT_PAGES) + 1) + +#define __VMEMMAP_OPTIMIZATION_NR_ORDERS \ + (MAX_FOLIO_ORDER - VMEMMAP_OPTIMIZATION_MIN_ORDER + 1) +#define VMEMMAP_OPTIMIZATION_NR_ORDERS \ + (__VMEMMAP_OPTIMIZATION_NR_ORDERS > 0 ? __VMEMMAP_OPTIMIZATION_NR_ORDERS = : 0) =20 enum migratetype { MIGRATE_UNMOVABLE, @@ -1159,7 +1160,7 @@ struct zone { atomic_long_t vm_stat[NR_VM_ZONE_STAT_ITEMS]; atomic_long_t vm_numa_event[NR_VM_NUMA_EVENT_ITEMS]; #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP - struct page *vmemmap_tails[NR_VMEMMAP_TAILS]; + struct page *vmemmap_tails[VMEMMAP_OPTIMIZATION_NR_ORDERS]; #endif } ____cacheline_internodealigned_in_smp; =20 diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 234218e14c3b..d212bbff4c83 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3344,7 +3344,7 @@ void __init hugetlb_bootmem_struct_page_init(void) struct zone *zone; =20 for_each_zone(zone) { - for (int i =3D 0; i < NR_VMEMMAP_TAILS; i++) { + for (int i =3D 0; i < VMEMMAP_OPTIMIZATION_NR_ORDERS; i++) { struct page *tail, *p; unsigned int order; =20 @@ -3352,7 +3352,7 @@ void __init hugetlb_bootmem_struct_page_init(void) if (!tail) continue; =20 - order =3D i + VMEMMAP_TAIL_MIN_ORDER; + order =3D i + VMEMMAP_OPTIMIZATION_MIN_ORDER; p =3D page_to_virt(tail); /* * prep_and_add_bootmem_folios() can access pageblock diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 917db0984143..ae8fdaa42118 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -494,7 +494,7 @@ static bool vmemmap_should_optimize_folio(const struct = hstate *h, struct folio * =20 static struct page *vmemmap_get_tail(unsigned int order, struct zone *zone) { - const unsigned int idx =3D order - VMEMMAP_TAIL_MIN_ORDER; + const unsigned int idx =3D order - VMEMMAP_OPTIMIZATION_MIN_ORDER; struct page *tail, *p; int node =3D zone_to_nid(zone); =20 diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 5a2469fb1838..aa6a4a2fae98 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -329,12 +329,12 @@ static __meminit struct page *vmemmap_get_tail(unsign= ed int order, struct zone * unsigned int idx; int node =3D zone_to_nid(zone); =20 - if (WARN_ON_ONCE(order < VMEMMAP_TAIL_MIN_ORDER)) + if (WARN_ON_ONCE(order < VMEMMAP_OPTIMIZATION_MIN_ORDER)) return NULL; if (WARN_ON_ONCE(order > MAX_FOLIO_ORDER)) return NULL; =20 - idx =3D order - VMEMMAP_TAIL_MIN_ORDER; + idx =3D order - VMEMMAP_OPTIMIZATION_MIN_ORDER; tail =3D zone->vmemmap_tails[idx]; if (tail) return tail; --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FA0733F582 for ; Tue, 4 Aug 2026 03:55:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815759; cv=none; b=rLkAn15Em1p78sJtLXwyNA47X8xAYBwd2rGFaZO6XbBKvkz/9+tpC1c/BnnSW/U/nZFwbzZx91zmYg+9j7AnMhYNWGQBzAMmK9vmEZ18sknMnjHm+lgSQGoV8SA50a9L+KuUqjAx1lItPgvDUfHOlhbNDv5iQqddAZChtakxa5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815759; c=relaxed/simple; bh=UWIqLaFuB5ADylvK38yUsqmlDhJ6djahCGnbGC2YYAo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MD08Tnj3C+4pMyKLSDXgPVJXEG6+OJzAcdX7b0988lok4nP5D1/qC2a+mbuQDkxPJ7VM/am1zazpoiJ3vspxnaTERRnlrfAwSkYeFehhXf7LBX0ErM0DBputykn2NRM7RNHjBqtf0uzdFsLrJ/4ejukExo5NRTBeLTs53NJR5LM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=bZgTcPy0; arc=none smtp.client-ip=209.85.215.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="bZgTcPy0" Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-cbb85186d43so1964537a12.3 for ; Mon, 03 Aug 2026 20:55:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815757; x=1786420557; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=c7rwKsso5Cym2RVCEFeMn64q7fu6QVT+bsIVBVphtqw=; b=bZgTcPy0ix4bTVYfkC80PS3rgXc6n3VWD0f9CciMaEprTPGFZJBvtjU+WPDjf2MGsu dpltGJZb92faDVMVcwCwRdu2KzTYktoXSUlowuJrtiP1W8JS1FjeQ9UC9pnDwq1xaOhG oRwmFGu2WSBQ08xhP0XS+EEVNTrwc5fHYxtQ74+DipTL1MbJnPoqyvWbKQ6bS4bL2asL Uc2eSzndh3SWL8s619poU3ihrb52FAHSbvWVAQyDQsLkoRH0yVcKwK0Xxc/BbfcjZTzq X4oO5bIBcFEyuUWGSQ8eXmd2DzpY4UcpKym2m1Gtx1r/l/YT5a5aCdcCfT1HOnprxCP+ Jt5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815757; x=1786420557; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=c7rwKsso5Cym2RVCEFeMn64q7fu6QVT+bsIVBVphtqw=; b=UNteug5fkZchrJ9oIz40ywvx1m2SLmxdtXr3ngWjdZA6XljCkVESq2KIdxCZIlN1hf hUQFmH3INo/oZ5ICeOKhUEh589F4WnCabXA10cv17hQW3RLwXFYB0oVxqIA3zjwVX8bA /7DMLWPxe0oqjy5cW2h5W+SdIcso4H43gHSMNTx9jtsdlIrALkTGExgvd3SWRFihUWdB ggiQlgKU5sm5p8S9j7rS50rtFTSrFjA+mbx9yM4Mvhv0ZwlquEpya0AMMALXVgd0HMBQ 7RqKrTETozcIheZMTVxUoOB31Jiv961IQo+dVLkoTN+ae3qs7J6EX5ddCVwzQg6cT1Pq Sb3w== X-Forwarded-Encrypted: i=1; AHgh+RouyQf6QoOg/VjCl+Zm6wwrt70z1crdPZ5X2F4x3TmU3hDFCrStcjFXubiOjlZdl3XzWVHuslOmX0M0Cms=@vger.kernel.org X-Gm-Message-State: AOJu0YzoQlaqT/9PfmcHCS6fgCD1gX9UvZirnkex1dmXdbBw6nrFMQkr P1kbE0sdTYczifjLw+fwohVl/CSodf1oLJFpkjNTZe7ruR0DEhgf5lufEPfCrDgHSVY= X-Gm-Gg: AR+sD13XMqmgqYuxufjQG/u63wiOVZDrgLdKDXb94LbU+hrubFuW2Eprz4bC//llw8+ 2TSKRxEVIYt3YODjz+Bv/hYrQWQ4BKONQrTySNZIbDbYXtjVm4eF19DJgAhHFdzqKi8LNibRQzU bEwNAOYYUpv1+o6jFQLBbedx3ckX1SccsCTX5abuXzjDPwV4sCUKCIl0GfScJ6Ne1N37WX8k2or Ko6aJ8ko7j+sdFId9W7cgbW83045OgRV/+AjybxWtXAmMK/7MmEhRf4LVM/hmEEfmvFeeQfh+j+ 3ME/FAvOOvmxXlqZo4apq1TJV5C+NZm0fqUGznTBTqEvPOiVYZcXqQZVikPzledZ6O1wh28poHk Hu/g/EDQleO6DodWmy8ROh3PWFSkMtmpkjJsiCIEohkZGSCO8GC0ideeg+C8Jlqh7Ifda3JAJnc fDu/vZJCAm0MQIf/2M05RcLA0mRVyAHQVd6TfjlX52RnL+cgGvqX23orIucfP++LoSYCH2CIa88 a9XN6yapA== X-Received: by 2002:a17:903:2f50:b0:2c9:b8b7:5d1c with SMTP id d9443c01a7336-2d0521b7f1emr128708045ad.16.1785815757210; Mon, 03 Aug 2026 20:55:57 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.55.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:55:55 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 03/17] mm/mm_init: skip initializing shared vmemmap tail pages Date: Tue, 4 Aug 2026 11:55:21 +0800 Message-ID: <20260804035535.2846016-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" memmap_init_range() initializes every struct page in the target range. For compound pages with vmemmap optimization, the tail struct pages are backed by a shared vmemmap page. Initializing those tail struct pages would overwrite the shared vmemmap page contents, requiring users such as HugeTLB to restore the metadata afterwards. Track the compound order for HVO-backed sections and use that metadata to detect struct pages that fall into the shared tail vmemmap range. Skip those shared tail pages in memmap_init_range(), then initialize pageblock migratetypes for the processed range with a helper after the per-page initialization loop. Keep direct mem_section access inside sparse helpers by exposing pfn_to_section_order() to users that only need the order associated with a PFN. This lets memmap_init_range() skip shared tail vmemmap pages without exposing __pfn_to_section() to !SPARSEMEM builds. This is a preparatory change for consolidating handling across users of vmemmap optimization, and it also avoids redundant initialization of shared tail vmemmap pages during early boot. Signed-off-by: Muchun Song --- v3: - Replace the !SPARSEMEM __pfn_to_section() stub with pfn_to_section_order() (suggested by Mike Rapoport) v2: - Fold section order tracking into the first user instead of keeping a standalone API-only patch (suggested by Mike Rapoport) - Rename page_vmemmap_optimizable() to pfn_vmemmap_optimizable() and pass a PFN directly (suggested by Mike Rapoport) - Initialize pageblock migratetypes from a helper after the per-page loop (suggested by Mike Rapoport) - Use a 1G PFN chunk for cond_resched() in the pageblock helper (suggested by Mike Rapoport) - Guard section_order() with CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP so it returns 0 when HVO is disabled and lets the compiler optimize the code as much as possible (suggested by Mike Rapoport) - Explain why the !SPARSEMEM __pfn_to_section() stub belongs here (suggested by Mike Rapoport) --- include/linux/mmzone.h | 8 ++++++++ mm/mm_init.c | 34 +++++++++++++++++----------------- mm/sparse.h | 33 +++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+), 17 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index f96a45ac558f..81e16d71e1f0 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2023,6 +2023,14 @@ struct mem_section { unsigned long section_mem_map; =20 struct mem_section_usage *usage; +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP + /* + * Normally, sections hold regular (order-0) pages. However, for + * sections with HVO enabled, this tracks the compound page order + * to enable deduplication of redundant vmemmap pages. + */ + unsigned int order; +#endif #ifdef CONFIG_PAGE_EXTENSION /* * If SPARSEMEM, pgdat doesn't have page_ext pointer. We use diff --git a/mm/mm_init.c b/mm/mm_init.c index e5aa20a9b898..1823381a69bb 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -677,21 +678,19 @@ static inline void fixup_hashdist(void) static inline void fixup_hashdist(void) {} #endif /* CONFIG_NUMA */ =20 -#if defined(CONFIG_ZONE_DEVICE) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INI= T) static __meminit void pageblock_migratetype_init_range(unsigned long pfn, - unsigned long nr_pages, int migratetype, bool atomic) + unsigned long nr_pages, int migratetype, bool isolate, bool atomic) { const unsigned long end =3D pfn + nr_pages; =20 for (pfn =3D pageblock_align(pfn); pfn < end; pfn +=3D pageblock_nr_pages= ) { enum migratetype mt =3D kho_scratch_migratetype(pfn, migratetype); =20 - init_pageblock_migratetype(pfn_to_page(pfn), mt, false); - if (!atomic && IS_ALIGNED(pfn, PAGES_PER_SECTION)) + init_pageblock_migratetype(pfn_to_page(pfn), mt, isolate); + if (!atomic && IS_ALIGNED(pfn, PFN_DOWN(SZ_1G))) cond_resched(); } } -#endif =20 #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT static inline void pgdat_set_deferred_range(pg_data_t *pgdat) @@ -886,6 +885,13 @@ void __meminit memmap_init_range(unsigned long size, i= nt nid, unsigned long zone } } =20 + if (pfn_vmemmap_optimizable(pfn)) { + unsigned int order =3D pfn_to_section_order(pfn); + + pfn =3D min(ALIGN(pfn, 1UL << order), end_pfn); + continue; + } + page =3D pfn_to_page(pfn); __init_single_page(page, pfn, zone, nid); if (context =3D=3D MEMINIT_HOTPLUG) { @@ -897,19 +903,13 @@ void __meminit memmap_init_range(unsigned long size, = int nid, unsigned long zone __SetPageOffline(page); } =20 - /* - * Usually, we want to mark the pageblock MIGRATE_MOVABLE, - * such that unmovable allocations won't be scattered all - * over the place during system boot. - */ - if (pageblock_aligned(pfn)) { - enum migratetype mt =3D kho_scratch_migratetype(pfn, migratetype); - - init_pageblock_migratetype(page, mt, isolate_pageblock); + if (pageblock_aligned(pfn)) cond_resched(); - } pfn++; } + + pageblock_migratetype_init_range(start_pfn, pfn - start_pfn, migratetype, + isolate_pageblock, false); } =20 static void __init memmap_init_zone_range(struct zone *zone, @@ -1112,7 +1112,7 @@ void __ref memmap_init_zone_device(struct zone *zone, compound_nr_pages(pfn, altmap, pgmap)); } =20 - pageblock_migratetype_init_range(start_pfn, nr_pages, MIGRATE_MOVABLE, fa= lse); + pageblock_migratetype_init_range(start_pfn, nr_pages, MIGRATE_MOVABLE, fa= lse, false); =20 pr_debug("%s initialised %lu pages in %ums\n", __func__, nr_pages, jiffies_to_msecs(jiffies - start)); @@ -1922,7 +1922,7 @@ static void __init deferred_free_pages(unsigned long = pfn, if (!nr_pages) return; =20 - pageblock_migratetype_init_range(pfn, nr_pages, MIGRATE_MOVABLE, true); + pageblock_migratetype_init_range(pfn, nr_pages, MIGRATE_MOVABLE, false, t= rue); =20 page =3D pfn_to_page(pfn); =20 diff --git a/mm/sparse.h b/mm/sparse.h index 95aa031213f2..b9b6b47e85ce 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -10,6 +10,39 @@ =20 #include =20 +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP +static inline unsigned int section_order(const struct mem_section *section) +{ + return section->order; +} + +static inline unsigned int pfn_to_section_order(unsigned long pfn) +{ + return section_order(__pfn_to_section(pfn)); +} +#else +static inline unsigned int section_order(const struct mem_section *section) +{ + return 0; +} + +static inline unsigned int pfn_to_section_order(unsigned long pfn) +{ + return 0; +} +#endif + +static inline bool pfn_vmemmap_optimizable(unsigned long pfn) +{ + const unsigned int order =3D pfn_to_section_order(pfn); + const unsigned long nr_pages =3D 1UL << order; + + if (!is_power_of_2(sizeof(struct page))) + return false; + + return (pfn & (nr_pages - 1)) >=3D VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES; +} + /* * mm/sparse.c */ --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 829BF3F99E3 for ; Tue, 4 Aug 2026 03:56:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815763; cv=none; b=dwZ+pyOHzGBqkHy6svTe7UFV+YQXEBxrpwVYSX6/rrXvvt4+2ikc6qyFaMj97YRRAligccbUUFmGVi2c864t522J9XItCkByGFANRE2Mz4TXvxr2cEcW909rz6zczdifn/IKq2tthgDytplYD85aQKJ965INUM0zW910rd4cID4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815763; c=relaxed/simple; bh=owQ+Elu7FhHy0RmBaGNrJ1F4/GpIAVuOEqAZSW69+JA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y16894Qgs//w8AM4qoHFDqLFHcyxLTTDao07OEGVpWSTZu+OFh3J6Amm2RiUb1Qg/93T8OnK4Jqo3ogQVW2fSmmZb29lbxVi+4KGQJg9jvq1jSiv+BozMTFRq/k6HyDlSkrq88g0IXb6cpKACG/HeqjxMWB9fSshnKxJl6pKMU8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=WMMSrGPh; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="WMMSrGPh" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2ce87c7e3bbso43751165ad.1 for ; Mon, 03 Aug 2026 20:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815762; x=1786420562; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=bOzdkIPPNoupexe8+AVOrQT3DBRmXPJ6Kybkle+RLi4=; b=WMMSrGPhzfQAKTE3Iutdp+1XKFfbim0oZtRODhIgDjDnI1N/w+v8qqVTTuvAwKFwV7 Jxo4ZZZ0VeD8EJdOxzVQcAX0Di5ua8ZZ8s0wFLwoETJObdM/oUfBMTZz5Kb8FWjP0YYr et42CbNcJSSXnjCBnWtBqpHY0aAyf4P50zHr6UWBrbDmCB2OMrkmIcvsFXwQE8VvF26F Cmlq1mUqDt42Fr+fmOdoYE9roIT+1IFHdpN4L26xKWBCCj6Bv+4aJ4lFyDfULdqzF3ly BWshfpGNas03Nn0aP9fk7bv5waTmzokXt40fzThoNbxTZG2qBc2y5oo8R0MOfNvy/Y7i 8Z5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815762; x=1786420562; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=bOzdkIPPNoupexe8+AVOrQT3DBRmXPJ6Kybkle+RLi4=; b=XdcwUIMUyOwIyZrKGCRgEVQ2mQwGEo7Y4lRdCB1zLIT+ckW8a2/pwWjWYXK+xW+MNh /axz0M7VrA41uaZ2xesZL+pWlDbZxIknWhIeGncR+u5dD5rYgH2N0FVE4nFGh+lBirqj p1NgOQs51IwMd6LjpcLDjF+jdrZPjNLtUrZil8TwmolvHhXSTPkw96iQ96qMYiS7VRKW FWbwCShaeXQablO1Xub84dptVW/Wlx2aDEMwMlpPX2Q+CCOp0FuNw5g6/yS0xlAk9qbq SBy8hfsGweJP6baQaJZkYFaPqzjirjlbwJu1BflrIqQ9/3HWsuEI8d21mjMPqAxeNeFY 7ufQ== X-Forwarded-Encrypted: i=1; AHgh+Rpap8boOzmiBuioHOrpmLq7IYArSKtRYLCsgBP7CZjzAOsKnXUtwdiJTQI0wbGflrW4car49UiM080rj/M=@vger.kernel.org X-Gm-Message-State: AOJu0YzgHXHGQ+++4ngCwYm5wubmKTjLBxBtWJYQm+HuYfftArWJVUtK hKKmD+rCM312z2kwIK+uJq2eve6cHSvnh2bKZDeIsVPpTXBEYWWTrfE+/1bR8IDIYPY= X-Gm-Gg: AR+sD11s2jOqYSGlyVx5HIlmauQYxbAtyT0F5+JG4p2kzhn9tZ5LRE7GMquFMGLWmQ/ 4pUlD3mKtwFG7916KetEl7uypMYtHa3VYl51G2uOOeDUzR3Gm0ekdhfhuzU/yu0aoHkdfgaYC0c 6Ri4jo45gVeo383iOBVZdci4gIb5b1rY1SJX4oWZchkxa7MUhXDAGmm/YS4W92tF26bevDWRhgr 3fX1etDCwyOKFa2JsP9EHcZxuaLzkgVgyS144Un7I7bZ3dfh4OmZSukEckpDkVF97Sfc+pfl6ZO 8ga5rB/6xsehzIdZOQYnpOsiZF9OBnIHb6h2g2n0g8In04NTECwg4R6kyVgyirfeO87wbhVDmus QYNW3CYgaAYvvV87/4HRBmK3NaDnxQ4giaaCgICQ0tXQezNMReKc0go5LJ4t0UaCPyONU2cvIyF a5ScwzJlNwfj3zKGyY3CSqziklI8PNF6okJoWrhWXKD4KZY245fdaIryzKau6/DYrYBu8inVXjK u5iWksI X-Received: by 2002:a17:903:2ac7:b0:2ca:e5c:7fcf with SMTP id d9443c01a7336-2d0521eac7amr124957695ad.3.1785815761731; Mon, 03 Aug 2026 20:56:01 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.55.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:00 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 04/17] mm/sparse-vmemmap: initialize shared tail vmemmap pages on allocation Date: Tue, 4 Aug 2026 11:55:22 +0800 Message-ID: <20260804035535.2846016-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The shared tail vmemmap page allocated in vmemmap_get_tail() used to be left uninitialized, because memmap_init_range() would later overwrite it. That forced users such as HugeTLB to defer the initialization to their own setup paths. Now that memmap_init_range() skips shared tail vmemmap pages, initialize them immediately in vmemmap_get_tail() with init_compound_tail() instead. This moves the initialization to the point where the shared tail page is allocated and avoids relying on deferred handling in individual users. The remaining deferred initialization in HugeTLB will be removed once it switches to the section-based vmemmap optimization mechanism. Signed-off-by: Muchun Song --- mm/sparse-vmemmap.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index aa6a4a2fae98..107215cf8488 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -338,19 +338,11 @@ static __meminit struct page *vmemmap_get_tail(unsign= ed int order, struct zone * tail =3D zone->vmemmap_tails[idx]; if (tail) return tail; - - /* - * Only allocate the page, but do not initialize it. - * - * Any initialization done here will be overwritten by memmap_init(). - * - * hugetlb_bootmem_struct_page_init() will take care of initialization - * after memmap_init(). - */ - p =3D vmemmap_alloc_block_zero(PAGE_SIZE, node); if (!p) return NULL; + for (int i =3D 0; i < PAGE_SIZE / sizeof(struct page); i++) + init_compound_tail(p + i, NULL, order, zone); =20 tail =3D virt_to_page(p); zone->vmemmap_tails[idx] =3D tail; --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A76F3F1658 for ; Tue, 4 Aug 2026 03:56:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815767; cv=none; b=PluNV5YdJAbEZC8Twlz+bKJzMoH1HuGJHHQP/l+OgP/bQdqcctJAIGfNecDNUl670dH+LoHgp+mNZZovw9ZkKXRFnqsBsc6lb7cCnyk+9iAWCx+U1l7ebNvKKw0MRmVMqUM/5+YSCQ1oviR1d/gc81fzVIFhgmC9oAquNxd1dHM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815767; c=relaxed/simple; bh=aGliExxAdCE1SWzccPm92kXWpTfztNQL50a4w//D6XI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=erOip0gmwQsmB4iRAxDTo3RcTkGS1b/XuMRBAQ0xbLX6yfXWgtwIdXURcyEuelPSvYu9njChIDHjcU8qoCvEtGHVZf+czLdT0UxSgZKFsEo+IOthUvIAe+DqgalI9idG3QUVHUlBDkbblIEy8MJf/qejoiEIvpZmAKZTKjg5xlU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=RRYOIRjt; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="RRYOIRjt" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2cee9b74ee1so32481255ad.3 for ; Mon, 03 Aug 2026 20:56:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815766; x=1786420566; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Yohn6K8Ptfa/JAUk45adVcXjDpZpli+rU2UEedJpBRU=; b=RRYOIRjtJDdcNYwH9Y6lJHDABzMcvKA96JbiZk3DbPNtG6P5QMqUNyjWRJUowkAvs1 3pKdXhByHhzqmGL8rAbCed9HmDOVmKRZnWKBOK7AcGOSYCjPi+ihkegJQfZNI/PRyyq5 GRaXZU7qmAw17OscWB3RN1XvROXKi7Cg6Lqp7mgNFi9wmjs0HHOYoFgVQv/G4O8ea5Qs CYuJI/nDeGnRF7bl9BOn5WbXbg9ImXLXPe8WriIcQ5gOcY1tlV9zEraU7HGnbpFE0evH D8qgrFgtxvXKWxljt5SE2s+CxvhZu1buXWmNZy6Y4kFDyVGkuXb1oeunSZTPKKbTqgob fpmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815766; x=1786420566; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Yohn6K8Ptfa/JAUk45adVcXjDpZpli+rU2UEedJpBRU=; b=MiltZSg+wyIven0vPfOsTNr6Yjdxerd1VFGc35+D7mkKhIlhmm2bBACHfOPsWZez0N Nk94z0wAwF9yiaZT2/cQVGxhGRips27zUSZN7bSU9x4dwuP2miLdmQCmHSHqS5wmZHlw mNPKtvRv104gQpCK3tqSY1xiMco3lpydW7q3M50JHlLjGlFZi/KXbJoNr5pGdnoBQp4I +JKT4+D+wqgMIMN/D4UvWtsNZUw4yapetJkJmlzNTPbdPQG3TO7XCZnqa1PAkdi01/wj iGDqv+8kB2FJAIbQZM8dUrAy4QEfbkCmTQfz7TQDfxEUl63V1PTpsT7AxaXlLRJwAYdk K6jg== X-Forwarded-Encrypted: i=1; AHgh+RoqzMce13K0nUw06vapLeEnywqanA9kru6O/jfSUIuI/PkseHw7yufx2eYmC3gaB6gs9Y6iA8hk50EritM=@vger.kernel.org X-Gm-Message-State: AOJu0Ywk7Wy9ouYq9jwLK3qtMwEmrIGZElzZ8h6zQT66vpIwIFYFrUhb rd73kVtlmWgfBCwVeRC1olB4UVUPAMyk1ogtCGAGeF/6HZFINaYl5EktpqqRpM7SYn0= X-Gm-Gg: AR+sD10IBbdCZUG6ErTEaaxyM14D9CQxeGp0GI9IMdMDmTK5+3rszqKOwQwf9dQ3Pyo 1utKSCmg6tJ9a5K4oFnO11YiCBlLBbF6NcNzqOw3Aeiy3JQmfjVWCrD/WHkHlv00Npl1e+nNvFg JDgHSGQMOaPCkkvItXOOwFHPU85Q6OvHsNa82zhP0tPxITfQhUUwdydk3d2vkw5JHC2Q/LUIIb6 QDVkv7A7VUIstXkp/w1WyNPFTJf5b3oudD52gRt4+Zz6ok7jyhJryGpnDuRcOuWsJVfyKJumO+Q 0DVbqre8rpWqg0o2iCOdJQzE1tOCPW1VTPJKqbRqh7zW3KdVNoOgc0cYtWvJfPwBsGe4RGZm9Lv lHAP2bebKpad0eyw2k+5whFvkBt1O3tV8t3rs+SJJsGrvSKTZROvg11teQs5lD8UPoz/xzg3qDX iMWEWEnyFDjjPKVrpkG7RJRVsSFjb4hEyTvVJShBVES61WG6k8hvw965k/DPz7APaifQHQSfMFM L7X7gpw X-Received: by 2002:a17:902:ce08:b0:2cf:9f0b:b560 with SMTP id d9443c01a7336-2d05246fd0fmr131647825ad.29.1785815765771; Mon, 03 Aug 2026 20:56:05 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:05 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 05/17] mm/sparse-vmemmap: support section-based vmemmap accounting Date: Tue, 4 Aug 2026 11:55:23 +0800 Message-ID: <20260804035535.2846016-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" section_nr_vmemmap_pages() can account ordinary sections and DAX sections, but section-based vmemmap optimization keeps its compound order in struct mem_section and retains a different number of vmemmap pages. Teach section_nr_vmemmap_pages() to recognize section-based optimized sections and calculate their vmemmap page count from the section order and the HVO retained page count. Signed-off-by: Muchun Song --- v3: - Add vmemmap_optimizable_order() for order-based optimization checks v2: - Remove an unnecessary vmemmap_can_optimize() call to simplify the code (suggested by Mike Rapoport). - Rewrite the commit message for better understanding. --- include/linux/mmzone.h | 6 ++++-- mm/sparse-vmemmap.c | 10 ++++++---- mm/sparse.h | 16 ++++++++++++++++ 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 81e16d71e1f0..663726dd6463 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -107,8 +107,10 @@ is_power_of_2(sizeof(struct page)) ? \ MAX_FOLIO_NR_PAGES * sizeof(struct page) : 0) =20 -/* The number of struct pages covered by the retained vmemmap pages with H= VO enabled. */ -#define VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES (PAGE_SIZE / sizeof(struct pa= ge)) +/* The number of retained vmemmap pages with HVO enabled. */ +#define VMEMMAP_OPTIMIZATION_PAGES 1 +#define VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES \ + (VMEMMAP_OPTIMIZATION_PAGES * PAGE_SIZE / sizeof(struct page)) #define VMEMMAP_OPTIMIZATION_MIN_ORDER (ilog2(VMEMMAP_OPTIMIZATION_NR_STR= UCT_PAGES) + 1) =20 #define __VMEMMAP_OPTIMIZATION_NR_ORDERS \ diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 107215cf8488..b7abc5494bb9 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -649,24 +649,26 @@ void offline_mem_sections(unsigned long start_pfn, un= signed long end_pfn) static int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned = long nr_pages, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { - const unsigned int order =3D pgmap ? pgmap->vmemmap_shift : 0; + const struct mem_section *ms =3D __pfn_to_section(pfn); + const int order =3D pgmap ? pgmap->vmemmap_shift : section_order(ms); + const int vmemmap_pages =3D pgmap ? VMEMMAP_RESERVE_NR : VMEMMAP_OPTIMIZA= TION_PAGES; const unsigned long pages_per_compound =3D 1UL << order; =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SUBSECTION)); VM_WARN_ON_ONCE(nr_pages > PAGES_PER_SECTION); =20 - if (!vmemmap_can_optimize(altmap, pgmap)) + if (!vmemmap_can_optimize(altmap, pgmap) && !section_vmemmap_optimizable(= ms)) return DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE); =20 if (order < PFN_SECTION_SHIFT) { VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, pages_per_compound)); - return VMEMMAP_RESERVE_NR * nr_pages / pages_per_compound; + return vmemmap_pages * nr_pages / pages_per_compound; } =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SECTION)); =20 if (IS_ALIGNED(pfn, pages_per_compound)) - return VMEMMAP_RESERVE_NR; + return vmemmap_pages; =20 return 0; } diff --git a/mm/sparse.h b/mm/sparse.h index b9b6b47e85ce..6ad190ec48cf 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -43,6 +43,17 @@ static inline bool pfn_vmemmap_optimizable(unsigned long= pfn) return (pfn & (nr_pages - 1)) >=3D VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES; } =20 +static inline bool vmemmap_optimizable_order(unsigned int order) +{ + if (!IS_ENABLED(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP)) + return false; + + if (!is_power_of_2(sizeof(struct page))) + return false; + + return order >=3D VMEMMAP_OPTIMIZATION_MIN_ORDER; +} + /* * mm/sparse.c */ @@ -80,6 +91,11 @@ static inline void __section_mark_present(struct mem_sec= tion *ms, =20 ms->section_mem_map |=3D SECTION_MARKED_PRESENT; } + +static inline bool section_vmemmap_optimizable(const struct mem_section *s= ection) +{ + return vmemmap_optimizable_order(section_order(section)); +} #else static inline void sparse_init(void) {} #endif /* CONFIG_SPARSEMEM */ --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A681E3D75D3 for ; Tue, 4 Aug 2026 03:56:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815772; cv=none; b=B2uUmIX4Ly1trrdJEwnfnQutMrLDl+jvs9DYFGAP/NTmMUD+4Wdmwox1TqNGLt5dEsQJ4Ipl8Mflr9d2nm3XnjtGTs2Qc2I7kqxOIZmSgcjEjw4vs4osOCH3UQ7LECwpmo+8obcC0AI26qVDW2y2uJWYv3q4qrWuGBBCdjUTYhg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815772; c=relaxed/simple; bh=e19bNRICAS6h6tkdqjgjcWmIKa+G4Mx+L4NkBPFthZ8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SPO066Hqu0c6596qHMIVbT0NWRBU0tBp2FvDJKhYLax/Pktbrce8wKVd06HN9PCREoFMjzSHbTylwQVzqIIVbOAfGxSl7CT9xO6rBg5nzUAJsyZBQTGFYcYmiNHkI3SM0H8J4gTKzlzuDZeUPxG3i+wDAzLhgMHz1gQNKRkI5Kc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=VmXfZCiX; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="VmXfZCiX" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2d01663d816so32168015ad.1 for ; Mon, 03 Aug 2026 20:56:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815770; x=1786420570; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=am0LZeSfTFW8Wc/d/bGYBseEk8tEkihNTPFhmZRA4WY=; b=VmXfZCiXhcm1yVTursVVBMHVwSFSWMhnbqFQYx9p9WR7rW/qqXZgkXaBw08L732h67 TneoXiDH8alans907z/sdAQq3+kfyx0CDO+mOhmZJSpoPDHefrGc+d/MYMLQkgKlyLaR /DSW5bCeTjwHM9XbEPzkUB0MFD1SjBBBMrnRwbuO2h0t5IZD4vTHELr5uX3RVWzY7q5Q bYxda3kdkF+Qo7CZAKZJF+z0V8EOHuvsXKQ46hVA3w61Tm6ZQQ+sB+QKxTs5qVz8UHPe +cluDtyQjiJxYtZJReYuBpwpVxvU2RG77KGKWCtMXGINOZe3vhDBu6nmoYL+wXQWsOCA mxtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815770; x=1786420570; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=am0LZeSfTFW8Wc/d/bGYBseEk8tEkihNTPFhmZRA4WY=; b=BqBg8Js/1Ltgl+GwP1bqYo2arg5Y+7OfO+I3SRRC5yHpc9qVVwVmKPT7aSjc3QIAau IlTTn09wIfaIornFrcdNBbLEt2Ix7BMb5O50/Tlq2gAuSCZmFhrOlJt/hoPhcGU1YQ+U S6R1wLpz9oEgzTahbcuTutNFNBaGcJ1j1kPA1J2MV1Zaym5nLDC+VQj/Caho4S386lk2 TVCYT9tCb8rB15Ivm0OFaO1iIKGTdO/roSbOA4KSD9H3dldEsgNQ/lkheYGeqBH7aUwG eYc2OmYnS7i/MZtNOoXtftx3ENDC49oNf6tqO31/kkeV95n0Gd+mbfuZifhSz9XuwswL qSdA== X-Forwarded-Encrypted: i=1; AHgh+Rpu2J+qxkhgeQ4nez2EIAnYXflP/VMewSXs6u/edM2o3Pbkj82civ+HeqODgQLlC+h9e435tHRR7EytjyU=@vger.kernel.org X-Gm-Message-State: AOJu0YzkbTWaY0wPOJCG7CEr0Px2/fvmvW3irD4QsP9E4QEXgw7Tl/l2 Z59yKddiCiyNgwKmWZ5EaihHNeI/zZEG82i9gCN0xKEXXpaHUX3c5KoHPL7yWfZKNKA= X-Gm-Gg: AR+sD10187gtsc297mNCrjnYE4qg39lbacXQ+ciwFqqm24s1FNH+fPh1oNG08gmsk52 7tKErLWk1FiZxJRqW2+1hzZXtrc7xr6L9KOQ/F81JfVPcKr73qFMAn4k/p36AyGH3PPiqaZ89SP lG2gDfGhy3ZNbFXykNTyuhCFw6LufmkF3T/6ix4C0lcizc8KFJj1ZkocwFY/BA+E51ubzWsm7iz 5KEElEGK03eiAeWVh1nh3ZQyFf7wSZCr85pDV6ZyG+yWGRQRmD+WOxfpIkBIYxN+o2DT+1SOfLw DfV60W8Ka2T+Nk+n65IQvoQNow7V8FR7TDmSQ83GFAySyjyV3qpXTtg4NLnX06qU1IxvUpH7kFd 9FuKjdmQvLB/FSNAtDNH0pRzjYEAvmld1rcF0Jfxa3o1spIMKfOsvXFZmwe7bENg8BmNySQO6wL Kl93P9tuU2gt9m/xPC9KDh2YYVmi/TdrgDD8OXEa6bQLl3U3bmoVSKkuhFveMeTVXtYHduSkCjF 2hpjBLIRRP1yuN6h3s= X-Received: by 2002:a17:902:ebc2:b0:2cf:bf32:b754 with SMTP id d9443c01a7336-2d05219f1cfmr130780515ad.8.1785815769791; Mon, 03 Aug 2026 20:56:09 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:09 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 06/17] mm/mm_init: factor out pfn_to_zone() Date: Tue, 4 Aug 2026 11:55:24 +0800 Message-ID: <20260804035535.2846016-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" pfn_to_zone() in hugetlb_vmemmap.c duplicates the zone lookup logic in __init_deferred_page(). Move it to mm_init.c, declare it in mm/mm_init.h, and reuse it from __init_deferred_page() and HugeTLB early vmemmap initialization instead of open-coding the zone walk there. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v3: - Fix the commit message to name mm/mm_init.h instead of mm/internal.h - Collect Acked-by from Mike Rapoport v2: - Move this preparatory patch before the sparse-vmemmap optimization changes (suggested by Mike Rapoport) --- mm/hugetlb_vmemmap.c | 17 ++--------------- mm/mm_init.c | 28 ++++++++++++++++++---------- mm/mm_init.h | 1 + 3 files changed, 21 insertions(+), 25 deletions(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index ae8fdaa42118..c48fcea076a5 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -19,6 +19,7 @@ #include #include "hugetlb_vmemmap.h" #include "internal.h" +#include "mm_init.h" =20 /** * struct vmemmap_remap_walk - walk vmemmap page table @@ -744,20 +745,6 @@ static bool vmemmap_should_optimize_bootmem_page(struc= t huge_bootmem_page *m) return true; } =20 -static struct zone *pfn_to_zone(unsigned nid, unsigned long pfn) -{ - struct zone *zone; - enum zone_type zone_type; - - for (zone_type =3D 0; zone_type < MAX_NR_ZONES; zone_type++) { - zone =3D &NODE_DATA(nid)->node_zones[zone_type]; - if (zone_spans_pfn(zone, pfn)) - return zone; - } - - return NULL; -} - /* * Initialize memmap section for a gigantic page, HVO-style. */ @@ -787,7 +774,7 @@ void __init hugetlb_vmemmap_init_early(int nid) map =3D pfn_to_page(pfn); start =3D (unsigned long)map; end =3D start + hugetlb_vmemmap_size(m->hstate); - zone =3D pfn_to_zone(nid, pfn); + zone =3D pfn_to_zone(pfn, nid); =20 if (vmemmap_populate_hvo(start, end, huge_page_order(m->hstate), zone, HUGETLB_VMEMMAP_RESERVE_SIZE)) diff --git a/mm/mm_init.c b/mm/mm_init.c index 1823381a69bb..c37fcbe9a319 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -692,6 +692,20 @@ static __meminit void pageblock_migratetype_init_range= (unsigned long pfn, } } =20 +struct zone __meminit *pfn_to_zone(unsigned long pfn, int nid) +{ + pg_data_t *pgdat =3D NODE_DATA(nid); + + for (enum zone_type zone_type =3D 0; zone_type < MAX_NR_ZONES; zone_type+= +) { + struct zone *zone =3D &pgdat->node_zones[zone_type]; + + if (zone_spans_pfn(zone, pfn)) + return zone; + } + + return NULL; +} + #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT static inline void pgdat_set_deferred_range(pg_data_t *pgdat) { @@ -750,20 +764,14 @@ defer_init(int nid, unsigned long pfn, unsigned long = end_pfn) =20 static void __meminit __init_deferred_page(unsigned long pfn, int nid) { - pg_data_t *pgdat =3D NODE_DATA(nid); - int zid; + struct zone *zone; =20 if (early_page_initialised(pfn, nid)) return; =20 - for (zid =3D 0; zid < MAX_NR_ZONES; zid++) { - struct zone *zone =3D &pgdat->node_zones[zid]; - - if (zone_spans_pfn(zone, pfn)) - break; - } - __init_single_page(pfn_to_page(pfn), pfn, zid, nid); - + zone =3D pfn_to_zone(pfn, nid); + __init_single_page(pfn_to_page(pfn), pfn, + zone ? zone_idx(zone) : MAX_NR_ZONES, nid); if (pageblock_aligned(pfn)) { enum migratetype mt =3D kho_scratch_migratetype(pfn, MIGRATE_MOVABLE); diff --git a/mm/mm_init.h b/mm/mm_init.h index 39f75df9be1c..c9fc35e7e9f1 100644 --- a/mm/mm_init.h +++ b/mm/mm_init.h @@ -39,6 +39,7 @@ void memmap_init_range(unsigned long size, int nid, unsig= ned long zone, enum meminit_context context, struct vmem_altmap *altmap, int migratetype, bool isolate_pageblock); +struct zone *pfn_to_zone(unsigned long pfn, int nid); =20 #if defined CONFIG_COMPACTION || defined CONFIG_CMA /* Free whole pageblock and set its migration type to MIGRATE_CMA. */ --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DBCEB3EDE7C for ; Tue, 4 Aug 2026 03:56:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815776; cv=none; b=h4dj2wq9p55pT8tBKMp55mPbje+RzryCviZfWyD5DjcTzOLBlyly8zAPHTb65m70GzPyKgxs7/PTD86kQfYpVyTvnh/qIxSVBoGu5nOn4S/f7l3sP9VbbHFYVwfnKE8WZyHTmyIfLesNzosqk4ltFvc9So45inTczQFo515bN9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815776; c=relaxed/simple; bh=cNXcXDZgPY6FKuMArv5HC7EpvCTKLUm1kC82mrQQeO4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Atu8NggFLsGW5eIEekNn6yizFZ+BCelAgt9y18nZ4QXYMuCvpq9Gl3qY68yz4DdD8GTwn+wIcm79Qx42qXtYBe13sp1AeRzskvXefIdKaqtWxIKKaG+YO0bveLa7C3q7t8TmN/UJmlC9vCZq+n9VxkpfpOQUNQAAf/tNS7p2oIA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=PyZ/Nexz; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="PyZ/Nexz" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2caea3f742bso42816995ad.0 for ; Mon, 03 Aug 2026 20:56:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815774; x=1786420574; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=qLDW1MKsRg3YxabDjaQCuEM3DeM9EuZ/ZJJJqQFbzN0=; b=PyZ/NexzI8UdnRgWnRPaqYmS7sC9/jbO1f1A8sn09CVfsUDfK0HFNW8gAGiiEVrtmS csEEhkL1Xk0dnq8ghJyP4iciWDyVlSBysGZlpguOdJ/GmcOydjf+dxzEO1a7S/c8FAAo ZiRU1brQGjQZBaI6tgO3xKMZpE+1e2S6E0NWJEvoSMoo15cvWgFHrp6I0lfBmMljgXsp kPjhUx8HYfUd5Z/h4Ou4TsEyG8bOO+yPUld6f/DoL4fOZGy3LwQG6yxcD9w5IK9QX7lp yTvMiDSDnDoIeGbv5IX9eqfQlG07Ro48aHAB/nSdqm0U2NrDuhjV+6MewfeA5ZqEEPuY 1WdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815774; x=1786420574; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=qLDW1MKsRg3YxabDjaQCuEM3DeM9EuZ/ZJJJqQFbzN0=; b=k4GOLahJjDpsGaDPqrT+2+dIbUMNCBll+5mEbxCZufxsTn6xHmmhNFjT2xyYL3cwf1 IfiKCbRnxjukj4MeRXFifMSW4eHG1UgsdeM5o44apv3/mJcN2+V/7l7O2V90TTUAXocb 9e7FwE3eOos4U71DMrAvg3dzkhNsiLdhPfC7S++PQ9Mi1gGUGWct25dXMuceieg/bJwK BPVk5IkzTKCi37vYGv0IjYhX2MziKJet0zOCyajxSPUD4HPt/WgoJ39wvpW3sKINLm0E GWavFKy+b4nx+0dqAgCt/cN6fhdW1LPLZ1XM38KYU7/WFvjAgokip9vUhrEvcYyKm0N7 Qm8w== X-Forwarded-Encrypted: i=1; AHgh+RrQDWbdIutkECmb3/j7sG5VCBxYN8FBQS5D591aTex899vKxZcohSWSXnFUe9nT0nEp3jZoa5XvLqtWqrU=@vger.kernel.org X-Gm-Message-State: AOJu0Yz3vRLtu49K/eiKwHl7a0rGs/ZrhrHreUFn5R7aMtS3+bg80Kmn sKGiVGMrBQ9Gj3/EzNdsXR7uNl98H5I1JlpHkBcPS0CX9LKTgZizCIfX5dWwToNlByzEPnTg7kQ jIvrU4DkBIw== X-Gm-Gg: AR+sD11qnFKS4oVhhHF/t5zySmLeZq7oc6G9REInn5hod0N1YbsnQIn+Da8SzNy5/N3 d2yo79kqQnu16e3AMbm2loHpPG6Mhmon6B4hG2p8YPsfeBJiXqjYPAuQNQ31nXWTPwsjC+yp5Rd dVowqzUTHNlBWVd/tGiHP7L8x3ZPJQvSVtwR/F1RmIf0kW9QTEHLkufFGLTf/JXqd1ae3ypdRjw sRlaNGYn+FzmDB6HUQnrkXcQbMM4rD+XH3ET1ZstknCelczr629bHuRZqHXDG9mY+qCylyBdO0e BP2Hj2sJv2OLCUyYI2qb2REDFIs/rLbOASCEZ4IlR+HoiPnWOny/eTbYFOKSD1BI3cWvtUgNG2C NqaNySpDBez8uuKmkhYwt/CtYVkwlAm1x2+yisPG4qnVqFcrsI6X2BtZy0for1MoOQdovi1taLY GmLHaKBhFpojPg3dy0gV2Q0AMKoPkqxL0cKwXAuuerbNrFi8b+K4DdfXSa40Rfd9FPxGr04QLAN TCoGGq1 X-Received: by 2002:a17:903:26cf:b0:2c9:a9c0:80b6 with SMTP id d9443c01a7336-2d052428130mr120815945ad.34.1785815774133; Mon, 03 Aug 2026 20:56:14 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:13 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 07/17] mm/sparse-vmemmap: move vmemmap_get_tail() before PTE population Date: Tue, 4 Aug 2026 11:55:25 +0800 Message-ID: <20260804035535.2846016-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A follow-up change will call vmemmap_get_tail() from vmemmap_pte_populate(). Move it before the PTE population helpers to avoid adding a forward declaration. Move vmemmap_alloc_block_zero() with it because vmemmap_get_tail() depends on that helper. No functional change is intended. Signed-off-by: Muchun Song --- v2: - Add this new patch to move vmemmap_get_tail() before PTE population (suggested by Mike Rapoport) --- mm/sparse-vmemmap.c | 78 +++++++++++++++++++++++---------------------- 1 file changed, 40 insertions(+), 38 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index b7abc5494bb9..b770fe2428fd 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -148,6 +148,46 @@ void __meminit vmemmap_verify(pte_t *pte, int node, start, end - 1); } =20 +static void * __meminit vmemmap_alloc_block_zero(unsigned long size, int n= ode) +{ + void *p =3D vmemmap_alloc_block(size, node); + + if (!p) + return NULL; + memset(p, 0, size); + + return p; +} + +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP +static __meminit struct page *vmemmap_get_tail(unsigned int order, struct = zone *zone) +{ + struct page *p, *tail; + unsigned int idx; + int node =3D zone_to_nid(zone); + + if (WARN_ON_ONCE(order < VMEMMAP_OPTIMIZATION_MIN_ORDER)) + return NULL; + if (WARN_ON_ONCE(order > MAX_FOLIO_ORDER)) + return NULL; + + idx =3D order - VMEMMAP_OPTIMIZATION_MIN_ORDER; + tail =3D zone->vmemmap_tails[idx]; + if (tail) + return tail; + p =3D vmemmap_alloc_block_zero(PAGE_SIZE, node); + if (!p) + return NULL; + for (int i =3D 0; i < PAGE_SIZE / sizeof(struct page); i++) + init_compound_tail(p + i, NULL, order, zone); + + tail =3D virt_to_page(p); + zone->vmemmap_tails[idx] =3D tail; + + return tail; +} +#endif + static pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long ad= dr, int node, struct vmem_altmap *altmap, unsigned long ptpfn, unsigned long flags) @@ -181,17 +221,6 @@ static pte_t * __meminit vmemmap_pte_populate(pmd_t *p= md, unsigned long addr, in return pte; } =20 -static void * __meminit vmemmap_alloc_block_zero(unsigned long size, int n= ode) -{ - void *p =3D vmemmap_alloc_block(size, node); - - if (!p) - return NULL; - memset(p, 0, size); - - return p; -} - static pmd_t * __meminit vmemmap_pmd_populate(pud_t *pud, unsigned long ad= dr, int node) { pmd_t *pmd =3D pmd_offset(pud, addr); @@ -323,33 +352,6 @@ void vmemmap_wrprotect_hvo(unsigned long addr, unsigne= d long end, } =20 #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP -static __meminit struct page *vmemmap_get_tail(unsigned int order, struct = zone *zone) -{ - struct page *p, *tail; - unsigned int idx; - int node =3D zone_to_nid(zone); - - if (WARN_ON_ONCE(order < VMEMMAP_OPTIMIZATION_MIN_ORDER)) - return NULL; - if (WARN_ON_ONCE(order > MAX_FOLIO_ORDER)) - return NULL; - - idx =3D order - VMEMMAP_OPTIMIZATION_MIN_ORDER; - tail =3D zone->vmemmap_tails[idx]; - if (tail) - return tail; - p =3D vmemmap_alloc_block_zero(PAGE_SIZE, node); - if (!p) - return NULL; - for (int i =3D 0; i < PAGE_SIZE / sizeof(struct page); i++) - init_compound_tail(p + i, NULL, order, zone); - - tail =3D virt_to_page(p); - zone->vmemmap_tails[idx] =3D tail; - - return tail; -} - int __meminit vmemmap_populate_hvo(unsigned long addr, unsigned long end, unsigned int order, struct zone *zone, unsigned long headsize) --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 265433D75D3 for ; Tue, 4 Aug 2026 03:56:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815781; cv=none; b=hDmC1aVk6GvecrP809BXTEg37qzd1uQEdlrfCCDxVrj7ZTkBPAdNQsBfFmKNUdupHqZ16cOabxDAwXzP3R/+e67uweDvDxdSXQx5dC5OaQb2fKH1F8DNd0XnWy5NGL/lS774m2ZldRYwCylfc7WEe7BI/IUlGgma1hiRumSBaLU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815781; c=relaxed/simple; bh=LlrwP37Q9dNw/uNASIwodbu8P2owhkU3Wtc0xAgHy54=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EvzE5ZpRQ6NLLl+Q/xLcAlR3Wnu8A6wbCq1k1RkqPNNSnjYlDT8q9PBbHaVc7ynWP2+aq2Y7aHIcK/2ykb54UCNb1vjK6rOEenka4rbpsDgDngRZZVM61hwkW4UwIeyIABbY32bc/ZZMf7WYJAEOz4c1Kb9tAQp1TG7GyOXsw+s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=bkryH+7C; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="bkryH+7C" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2ce98cb8165so7067405ad.1 for ; Mon, 03 Aug 2026 20:56:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815779; x=1786420579; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=uZCX0uRt3bq26YcZIyKc0yaDQHyGPM1xGVsxvUVCVRw=; b=bkryH+7ClhxRkymQz6ZIWtWI/GCGRBtvcxCk/rbWYdA57rxxasXPeuX499FwAuhn/2 oN/oCgikFmqu1JWB6Xw8L6GYn60ob34KqCVLn3JQZdC8FStPAgHatf12BCuMpoiqeUqt 1m49mdiDw0iNfm/GWdmPmpoO0X/uZK1yoxoliz8ISyYysgq+wnuBbD/iv3YGdf5acJJz k88QrlW66kB9p7OU/E0+3Gk4GSn/YIxf002HhJjxN2x+PJcUq2KH6lIHrsFUz/MC4gjR wrlROYuliq0CoP6t1nh64kLPdtM5z/9KgZ+1luwimlXl1TrryVnjWXG4eqT5/73pfVaQ Gn/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815779; x=1786420579; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=uZCX0uRt3bq26YcZIyKc0yaDQHyGPM1xGVsxvUVCVRw=; b=cOxBAzdBYo3hVsDbBPdd90KeiZrx80F1HZZykcyxTmEJ5r6R4LYnk2XlRcerwIJR2w 9BMARm6FvtDZZuWeVbtFad2l8M8jSNCHo952ME3hD8j4SVIZAKDSPmDcwsgghgCeQIVW P5q/sr31t6m2jjbfsjqF7AzORu5QUvZxgd09eJWW0n26YFfER0dnnby2UhiSqZoB++rA oR+4MEpbLicXHM2eL1gf60wAXCStB539tJmuP8f3sWdc4M/jubPdPIFNn7FI3cVfY8M0 FRndMpDz1F4b3N4Mji5LZvSWWtLlxshrBW0bloPIUfgpPOtnivCcgyPFZIeuWYSXJxG9 xaBg== X-Forwarded-Encrypted: i=1; AHgh+RqHuhBWKc7AVPM5von9No/9C8WURoFcmAZGwxQwOu+kt08U3ODZZTjGr086/B9/OQ0mHKcG2NSMUtIbjGU=@vger.kernel.org X-Gm-Message-State: AOJu0Yzx2sMtih4EOGEaIEVF4yJfZ4LfWqHKA7MvQC/mNwqz+W0vS6WA WvchysHB63TxLfmfp0eOCMUEL/7ohAHHFO6C6m7cAB23EvoiTwzdMSDOeddaN+YUMgQ= X-Gm-Gg: AR+sD12odOPoCwVlL/U0+pCp2vprx2rioUewQP7RMd+/Cdb4FgUlowXmM88Ydi+Vh1s av6nNSruE7WIv8+OizZLXEgEWQE7vUh1mQSC4wd5Ev+j33Bw9nkshuHof1VEL1AdSfsRCOb0AbU abMVAOuh/1SHTDPs8cYxu9m0CBlYwSrqZ0mmrd5DG64Me3F+xqm7AsqMl1cS7S0cY/4uoujJc+7 IW+0u6TQVL+wZ24Jo5Fvn/csIHysqGhYVLggZEzDBrZAie4mlJMHGDD4Iw1hgqL8pdItq1cszEP cDC1yRGUSqBCRv2oNnZepmPMsRPW8t/oyL9UN0sKdHNuc/UVsYNPj/QyihS3pWe5tn00rRpsJQc XpdMmVod78B3x6jLlFbk3/bGYndBmfVNxuM4KjGAeu7yHHvvYGcb6B3A2TkRzUgbCsnoV7luqFD Bwe3N4v9zoSvQPKoeN8EFyhnLLRTNFWcvQiEe/1oRvmEyKnRYnfLY9+NZChI9Pcku+wxGLRpIxz 7UzFGBI X-Received: by 2002:a17:903:98f:b0:2cf:461a:3863 with SMTP id d9443c01a7336-2d08ab8bec3mr18723735ad.22.1785815779325; Mon, 03 Aug 2026 20:56:19 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:18 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 08/17] mm/sparse-vmemmap: support section-based vmemmap optimization Date: Tue, 4 Aug 2026 11:55:26 +0800 Message-ID: <20260804035535.2846016-9-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Teach sparse-vmemmap population code to use the compound page order when deciding whether a vmemmap page can be optimized. With this information, the common sparse-vmemmap population path can allocate or reuse shared tail vmemmap pages directly instead of relying on HugeTLB-specific handling. This centralizes vmemmap optimization logic in the sparse-vmemmap code, based on section metadata, and prepares for sharing the same mechanism across different users of vmemmap optimization, including HugeTLB and DAX. Signed-off-by: Muchun Song --- v2: - Keep vmemmap accounting and population logic in sparse-vmemmap.c (suggested by Mike Rapoport) - Move vmemmap_get_tail() before its first use instead of adding only a forward declaration in the previous patch (suggested by Mike Rapoport) - Simplify the PMD path handling for HVO-covered sections --- mm/sparse-vmemmap.c | 36 ++++++++++++++++++++++++++++++------ mm/sparse.c | 4 ++-- mm/sparse.h | 7 +++++++ 3 files changed, 39 insertions(+), 8 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index b770fe2428fd..b69a7af76858 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -186,6 +186,11 @@ static __meminit struct page *vmemmap_get_tail(unsigne= d int order, struct zone * =20 return tail; } +#else +static inline struct page *vmemmap_get_tail(unsigned int order, struct zon= e *zone) +{ + return NULL; +} #endif =20 static pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long ad= dr, int node, @@ -193,12 +198,24 @@ static pte_t * __meminit vmemmap_pte_populate(pmd_t *= pmd, unsigned long addr, in unsigned long ptpfn, unsigned long flags) { pte_t *pte =3D pte_offset_kernel(pmd, addr); + unsigned long pfn =3D page_to_pfn((struct page *)addr); + if (pte_none(ptep_get(pte))) { pte_t entry; - void *p; + + if (pfn_vmemmap_optimizable(pfn) && ptpfn =3D=3D (unsigned long)-1) { + unsigned int order =3D pfn_to_section_order(pfn); + struct zone *zone =3D pfn_to_zone(pfn, node); + struct page *page =3D vmemmap_get_tail(order, zone); + + if (!page) + return NULL; + ptpfn =3D page_to_pfn(page); + } =20 if (ptpfn =3D=3D (unsigned long)-1) { - p =3D vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap); + void *p =3D vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap); + if (!p) return NULL; ptpfn =3D PHYS_PFN(__pa(p)); @@ -217,7 +234,8 @@ static pte_t * __meminit vmemmap_pte_populate(pmd_t *pm= d, unsigned long addr, in } entry =3D pfn_pte(ptpfn, PAGE_KERNEL); set_pte_at(&init_mm, addr, pte, entry); - } + } else if (WARN_ON_ONCE(pfn_vmemmap_optimizable(pfn))) + return NULL; return pte; } =20 @@ -406,6 +424,9 @@ int __meminit vmemmap_populate_hugepages(unsigned long = start, unsigned long end, pmd_t *pmd; =20 for (addr =3D start; addr < end; addr =3D next) { + unsigned long pfn =3D page_to_pfn((struct page *)addr); + const struct mem_section *ms =3D __pfn_to_section(pfn); + next =3D pmd_addr_end(addr, end); =20 pgd =3D vmemmap_pgd_populate(addr, node); @@ -421,7 +442,7 @@ int __meminit vmemmap_populate_hugepages(unsigned long = start, unsigned long end, return -ENOMEM; =20 pmd =3D pmd_offset(pud, addr); - if (pmd_none(pmdp_get(pmd))) { + if (pmd_none(pmdp_get(pmd)) && !section_vmemmap_optimizable(ms)) { void *p; =20 p =3D vmemmap_alloc_block_buf(PMD_SIZE, node, altmap); @@ -439,8 +460,11 @@ int __meminit vmemmap_populate_hugepages(unsigned long= start, unsigned long end, */ return -ENOMEM; } - } else if (vmemmap_check_pmd(pmd, node, addr, next)) + } else if (vmemmap_check_pmd(pmd, node, addr, next)) { + if (WARN_ON_ONCE(section_vmemmap_optimizable(ms))) + return -ENOTSUPP; continue; + } if (vmemmap_populate_basepages(addr, next, node, altmap)) return -ENOMEM; } @@ -648,7 +672,7 @@ void offline_mem_sections(unsigned long start_pfn, unsi= gned long end_pfn) } } =20 -static int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned = long nr_pages, +int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr= _pages, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { const struct mem_section *ms =3D __pfn_to_section(pfn); diff --git a/mm/sparse.c b/mm/sparse.c index ca9875f568d3..24555a32a5d9 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -315,8 +315,8 @@ static void __init sparse_init_nid(int nid, unsigned lo= ng pnum_begin, nid, NULL, NULL); if (!map) panic("Failed to allocate memmap for section %lu\n", pnum); - memmap_boot_pages_add(DIV_ROUND_UP(PAGES_PER_SECTION * sizeof(struct pa= ge), - PAGE_SIZE)); + memmap_boot_pages_add(section_nr_vmemmap_pages(pfn, PAGES_PER_SECTION, + NULL, NULL)); sparse_init_early_section(nid, map, pnum, 0); } } diff --git a/mm/sparse.h b/mm/sparse.h index 6ad190ec48cf..f8f852f9f8a2 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -105,8 +105,15 @@ static inline void sparse_init(void) {} */ #ifdef CONFIG_SPARSEMEM_VMEMMAP void sparse_init_subsection_map(void); +int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr= _pages, + struct vmem_altmap *altmap, struct dev_pagemap *pgmap); #else static inline void sparse_init_subsection_map(void) {} +static inline int section_nr_vmemmap_pages(unsigned long pfn, unsigned lon= g nr_pages, + struct vmem_altmap *altmap, struct dev_pagemap *pgmap) +{ + return DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE); +} #endif /* CONFIG_SPARSEMEM_VMEMMAP */ =20 #endif /* __MM_SPARSE_H */ --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A01523F58F0 for ; Tue, 4 Aug 2026 03:56:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815786; cv=none; b=YWSql0SR0WX59maqOitE5YRf0mjVBTVod6hRp7bdFg22jhrXXO8vsK21nyGMPVyEWJ0KBQ4rpBb6rUpSF20X/b6n1dTdz9gyQr0PICsxghvnHBVo6lmcjGTVXijv6Z0GipDw3vm+63BicJ7nY0dxR8XMnCA0TnmkYyjYCzSUp/c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815786; c=relaxed/simple; bh=BWIIjkI8gomiahBT+UzAKqYDEFeNWcYCGDnTuxNs0Dw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rX+/LwWkKBkWs2Ow4HJStm9GNzhG2JfAxa+auvsgkSWtn7ZHB8xavFHTafZPh6XF/mP1oZ5DEahpxp2FtFq25rIffgKZUoFFBuUCczp0/AiS8R39pK5m1X4IdgLPbEEVd0f2LvJp9z+3S29HjqJ8YCNInlvUJh/qJ0zK9HNl9Yw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=ItsaoIZV; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="ItsaoIZV" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2cc61541f8cso6443425ad.0 for ; Mon, 03 Aug 2026 20:56:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815784; x=1786420584; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=7mhQWWJoMKDun63XH5Agr7zXFay3hqxZE0Eint2vdW0=; b=ItsaoIZVSrSGzlrzSFggW4lkj40sb+4M/KWWJadFYPpD5MEOYTXyulH8tu7Yv7AP9r 3dnfeQYYrUakvGal006vxAdpnmFPPNxDvd2A/+EALZR+MX9CJUpPnio6X1bhm+C0FqAo n7jzEuWf6vTJYlveNd3+vzrgozkoIJ+nJSX6oS9Xty7QXv+kj5V8BfU20amao40Ofy+A N4SmZy2GgJNi9IRobX1w+Q9sEhiry+ndAhV7STrcARXQWSYS6PbZf6mrtjW70yH3rhUP wldLdsMIOgR3YCXoahuf9dZy2q/3EtWaCgnyxNiRBJRPRCRsCkx9X5mhtGpq6H9aFNfP Di5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815784; x=1786420584; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=7mhQWWJoMKDun63XH5Agr7zXFay3hqxZE0Eint2vdW0=; b=Tepv1JQ3DLDSTu0R8wQIqZAcEa/Rjry9DVss9uYqYlMjqExQuUmZpWhLY+/BLUhs9R vJ4zAOKbTCDNFdkasgFtOwQC5c/zdFfzY1l1KloOUT+A7649F5uKO/yPClU+ZEtnWnrV o9VpNs4pY09qFe7zpEgPLe6cPruLddhj6dxw+JeqYNPmvKe9108bI98UWcRshJoClMM9 VSp1yPVggvu5l4oJGMXeCBv4qcLnmHrGs8OEsQr2itD7L+mmDEibEccOSvcTXjBuU7Hx aQ83egQKyy0knRqt0Hv7muWoAeFEfwVpaFi8XaUkqSXRdBkEOYy8OVmv5w6vJ/absrbw K//g== X-Forwarded-Encrypted: i=1; AHgh+RqrWHYdkN8+MJp/WSADclJQf35EoSnKGiB/BpFIAXpkTq55MKcuCZ2zuIsTI9aC71/iaCeltzgv8mWvthw=@vger.kernel.org X-Gm-Message-State: AOJu0Yw6DuSHYJ2k32uqzD00Ggv6/NyWrxI0e+QNZ7I0xrdS6A5PDQ9q 98oCLTM12frzaNGTZF1FBpZbel9bq6PFA5QCLEkePs80U9cO4oJSAxwjolA7I0Wq8Yk= X-Gm-Gg: AR+sD13/gvqyjDG+slYfBbdM9tTbasYAVF54Ks5Cd4gOgYef2zxSo5qE1Pa28bldH4m EucxsDBtVdU9mUP9V0oqq1wSHtTLwpYdc7Vwa9aupXJANapfZ3R8kBIOSrlF85ja8SS6yQxX/DA Y3DUxE2AwM1TOQuVF08gRsztbC+QkD/2aIMfB/S0sIC/W1OkLhR++tAVQnzmoDea86E2Fs/Uxkv aWV0K6blO/PNMIoGQqyp0suioGK69IR1UwnmTvPm1l8UX24Nalv6XcF3kJ31hEU5nrH3yRkBgue oxkboSCbgv1/6+NWYjerTwuzytBvurywTTwW+kLvrk6wRWMz6E3wemnRReX5Gqlt4YLMKpDsF4C dE4zwrvio1CcxSjSCxmkqvR2D1cXq20mF1EIEsIAW0ZATfms0zQNSlGhEiKGw5qQd16fg7X2NaV zC3OCY713WC+6qPcbnPRcBdOL84Cf7wXt7ExeFwSxhkjcjw2KHXgWxkdUgyBrcumaYeiGqtobEb GYQIDnv X-Received: by 2002:a17:903:3d0d:b0:2c9:97a7:3283 with SMTP id d9443c01a7336-2d08ab6ce8fmr19535515ad.23.1785815783938; Mon, 03 Aug 2026 20:56:23 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:23 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 09/17] mm/sparse: initialize memory sections earlier Date: Tue, 4 Aug 2026 11:55:27 +0800 Message-ID: <20260804035535.2846016-10-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Upcoming HugeTLB bootmem changes need sparsemem section metadata before the HugeTLB bootmem allocation path runs. The memory sections are initialized from sparse_init(), which is called too late for that setup. Move the code that initializes sparsemem section metadata for memblock ranges into mm_core_init_early(), before free_area_init() and the HugeTLB bootmem setup. Rename the helper to sparse_sections_init() so the new caller describes the sparsemem-specific initialization step. This is a preparatory change. Signed-off-by: Muchun Song --- v2: - Rename the helper to sparse_sections_init() to describe the section metadata initialization (suggested by Mike Rapoport) - Fix the !SPARSEMEM stub name so SPARSEMEM=3Dn builds compile (reported by Sashiko) --- mm/mm_init.c | 1 + mm/sparse.c | 10 ++-------- mm/sparse.h | 2 ++ 3 files changed, 5 insertions(+), 8 deletions(-) diff --git a/mm/mm_init.c b/mm/mm_init.c index c37fcbe9a319..1855ad3373aa 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -2634,6 +2634,7 @@ void __init mm_core_init_early(void) { kho_memory_init_early(); =20 + sparse_sections_init(); free_area_init(); =20 hugetlb_cma_reserve(); diff --git a/mm/sparse.c b/mm/sparse.c index 24555a32a5d9..d4641aa70497 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -191,12 +191,8 @@ static void __init memory_present(int nid, unsigned lo= ng start, unsigned long en } } =20 -/* - * Mark all memblocks as present using memory_present(). - * This is a convenience function that is useful to mark all of the systems - * memory as present during initialization. - */ -static void __init memblocks_present(void) +/* Initialize memory section metadata for all system memory. */ +void __init sparse_sections_init(void) { unsigned long start, end; int i, nid; @@ -332,8 +328,6 @@ void __init sparse_init(void) unsigned long pnum_end, pnum_begin, map_count =3D 1; int nid_begin; =20 - memblocks_present(); - if (compound_info_has_mask()) { VM_WARN_ON_ONCE(!IS_ALIGNED((unsigned long) pfn_to_page(0), MAX_FOLIO_VMEMMAP_ALIGN)); diff --git a/mm/sparse.h b/mm/sparse.h index f8f852f9f8a2..16b9bd3070aa 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -59,6 +59,7 @@ static inline bool vmemmap_optimizable_order(unsigned int= order) */ #ifdef CONFIG_SPARSEMEM void sparse_init(void); +void sparse_sections_init(void); int sparse_index_init(unsigned long section_nr, int nid); =20 static inline void sparse_init_one_section(struct mem_section *ms, @@ -98,6 +99,7 @@ static inline bool section_vmemmap_optimizable(const stru= ct mem_section *section } #else static inline void sparse_init(void) {} +static inline void sparse_sections_init(void) {} #endif /* CONFIG_SPARSEMEM */ =20 /* --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13F733F39F1 for ; Tue, 4 Aug 2026 03:56:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815790; cv=none; b=AJk4QfeZAxx7HnVaq3aveQWhOlB/Jm3RKiEjZPXy1Hk/EhooA7Zmw1qPdEvzDPXDTV6MoHr3483kee1+MUAeExEmm9OKUiOx0TSS+ETg2cMgNL1JAPqttdlsBxqrjrwa9ilpfgVWxsW3q0jVkstzpF6YrG/fxsHaKtS94U4ktHc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815790; c=relaxed/simple; bh=Yb0Tvmn6iIDfQgyLB08pKjPTfKHUNurJpqJNlg02Lfg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n27oHBHVFzL1EMRy7beI5oCSLA8AAuByX+Jty7ZUpJBmKMWUzaWdCuwHQipgyvZv87tngOtB4nmGkZTF3h75Dkl8kIobIews14edivJ1jXyRYr/e5I5BIv4H0iXZTV93jG9V4wGBsYqKrFK4SLP58ggPiSFmZE5S9TlB7PL5OaU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=aZ06KquS; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="aZ06KquS" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2ced3386430so41274535ad.1 for ; Mon, 03 Aug 2026 20:56:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815788; x=1786420588; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Rx6ooyM+fYIa9IWfVQ2BTryW26cGLhdtaYHkTbeZ4Mw=; b=aZ06KquSQLp9EDNcgGe3370pT4faQBOwLMM7LYVQlIs4pZPr/ju4KNQxOO2IlG3n5k gQp2T6qJ++kYxCRWbk92y0b6bFdlfsy52Sg6zGCpYyYl/f9pShMcUSItvTRtf5JCLN6g MlsCwVjcYdO79ZMSRCvia+Z+G2Of0OUcv7FrtfaC3Det3nPlou7GNtV+psL+XNAE0pQo h+neDaRZX6evQzt0W6yeGVS4v9I7Rkyv7FbF7cUAcFkxQLWNS2RVsZ/3KEI0ZVJJDfac ZoM3ZpyIew45Xesvv6rNGNal7mzHeEUobW2t+rjFHkrUgxZvEO1XQ18PJtviNbNrG/UK AT5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815788; x=1786420588; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Rx6ooyM+fYIa9IWfVQ2BTryW26cGLhdtaYHkTbeZ4Mw=; b=S9OEbObzcY/mpB13A+G6LLBg3m2/VrligdWM83Fvk0ac7lbVRdYkXwH84P7C1nJEjs kRVBvsxVgMaTRTsdIKKiN4SDFLzcbxpbv6eXKruzKf8vLjz0JHGir37vC4bi/95hfAxk l6o3vW64WWFkgFn6u49hy7FxHJgsTBE5zXU+1cUashuqt4RJ6Ur6QB2WFqJKEjk6hmQj e1THUcx9lJZIh2rIq9k3CnrdZIzgXIGLk/SFNjqvrelCq2hZp76b3FQmZP/z9v8mIfOn xlLrWulnv0JK+Qub4UPD9vF3403AjcIVb7LB/fjg5C+Fj+tplzthFb9VivxKK348hIDF FszA== X-Forwarded-Encrypted: i=1; AHgh+Rr7HYiW6OkZqnHMAT43YHOJhkvnrjZ+g9iZthETu+s2rs3KPqruhIBaj3YQTaio6idcyngxl27TPcQhxV4=@vger.kernel.org X-Gm-Message-State: AOJu0YyTbTLJFJVhsNZLuih6Id6WAjgrKGMyrk0HVyOBxVKTwTNvaMne nM0N9/pIXxAt2/4N0Yi+RorQM8xsVfVFGTR9dM62SLmSgr7/TyzVZoTwNoYzHK5ONqU= X-Gm-Gg: AR+sD132erzXjjlh2o8TBYEimX1L+llnckXK3/rbWiPMJIeNHDV1njE3GQUI9UrgGFo pl+q0gzA4It8IkO+ry9GxxEzQ2evR0Pq9vMa2n4beINLMP4/T6vnqsGKQXEYUrJWUNvFTWEF3Ty CkP9gB4H68+PI6v2DZpaC3W44FBpA9cwKZAuPSRF37iM8E0ZZz5H3WMZI0sVx17D9G+rIg2cbKt n9P1f+gAVXyTSDnAJHm2xn9wmSjNqYhbtOJ7LK6YqD3Mr8qGetv2GdQhdFrN8bDzDGf+HwZ9gOX RgTSbexNJmxi7HRdBFBzh/ZFp78ONz9LEuTwZypENuWwLOdJhI1tMMbnmzkezvvNxgAiZ9VZXyJ m45KQzsaxKSMlTegr+M1SbWa61UOcKMLp0/rj7OHpXeh9GrpJi/oPE7TLfpwdkb7/3l2ig49YE0 BMxR+aFr19JRQGOgpgCEleBouuGxzLxmorfUkvldZ1qC7UxFhcswdm6BJUMuRN4QUK32v/70AQL 0I+mDnO X-Received: by 2002:a17:903:2347:b0:2ca:bb6:e7c6 with SMTP id d9443c01a7336-2d0523f9f88mr112432745ad.37.1785815788226; Mon, 03 Aug 2026 20:56:28 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:27 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 10/17] mm/hugetlb: switch HugeTLB to section-based vmemmap optimization Date: Tue, 4 Aug 2026 11:55:28 +0800 Message-ID: <20260804035535.2846016-11-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" HugeTLB bootmem vmemmap optimization still carries its own early setup path, including pre-populating optimized mappings before the generic sparse-vmemmap code runs. Now that the section-based vmemmap optimization can derive HugeTLB vmemmap deduplication from section metadata, HugeTLB only needs to mark the bootmem huge page range with the appropriate order. The generic sparse-vmemmap population path can then allocate and map the shared tail vmemmap pages without any HugeTLB-specific early population code. Do that by setting the section order when a bootmem huge page is allocated and dropping the dedicated pre-HVO helpers and related special-casing. This removes duplicate early setup logic and switches HugeTLB to the section-based vmemmap optimization path. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v3: - Use the order-based helper for the bootmem vmemmap-optimized check v2: - Collect Acked-by from Mike Rapoport --- include/linux/hugetlb.h | 1 - include/linux/mm.h | 3 -- mm/hugetlb.c | 30 ++------------ mm/hugetlb_vmemmap.c | 90 +++-------------------------------------- mm/hugetlb_vmemmap.h | 14 +++---- mm/sparse-vmemmap.c | 31 -------------- mm/sparse.h | 27 +++++++++++++ 7 files changed, 42 insertions(+), 154 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 16c4c4caa126..fe28f98e1b22 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -171,7 +171,6 @@ struct address_space *hugetlb_folio_mapping_lock_write(= struct folio *folio); =20 extern int movable_gigantic_pages __read_mostly; extern int sysctl_hugetlb_shm_group __read_mostly; -extern struct list_head huge_boot_pages[MAX_NUMNODES]; =20 void hugetlb_bootmem_struct_page_init(void); void hugetlb_bootmem_alloc(void); diff --git a/include/linux/mm.h b/include/linux/mm.h index 7fabe6c66b4b..fd7dc85f58f3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -5097,9 +5097,6 @@ int vmemmap_populate_hugepages(unsigned long start, u= nsigned long end, int node, struct vmem_altmap *altmap); int vmemmap_populate(unsigned long start, unsigned long end, int node, struct vmem_altmap *altmap); -int vmemmap_populate_hvo(unsigned long start, unsigned long end, - unsigned int order, struct zone *zone, - unsigned long headsize); void vmemmap_wrprotect_hvo(unsigned long start, unsigned long end, int nod= e, unsigned long headsize); void vmemmap_populate_print_last(void); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index d212bbff4c83..7d8507aa4c5b 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -52,6 +52,7 @@ #include "hugetlb_cma.h" #include "hugetlb_internal.h" #include "mm_init.h" +#include "sparse.h" #include =20 int hugetlb_max_hstate __read_mostly; @@ -59,7 +60,7 @@ unsigned int default_hstate_idx; struct hstate hstates[HUGE_MAX_HSTATE]; =20 __initdata nodemask_t hugetlb_bootmem_nodes; -__initdata struct list_head huge_boot_pages[MAX_NUMNODES]; +static struct list_head huge_boot_pages[MAX_NUMNODES] __initdata; =20 /* * Due to ordering constraints across the init code for various @@ -3137,6 +3138,7 @@ static bool __init alloc_bootmem_huge_page(struct hst= ate *h, int nid) } else { list_add_tail(&m->list, &huge_boot_pages[nid]); m->flags |=3D HUGE_BOOTMEM_ZONES_VALID; + hugetlb_vmemmap_optimize_bootmem_page(m); /* * Only initialize the head struct page in memmap_init_reserved_pages, * rest of the struct pages will be initialized by the HugeTLB @@ -3297,6 +3299,7 @@ static void __init gather_bootmem_prealloc_node(unsig= ned long nid) * this folio. */ folio_set_hugetlb_vmemmap_optimized(folio); + section_set_order_range(folio_pfn(folio), folio_nr_pages(folio), 0); =20 if (hugetlb_bootmem_page_earlycma(m)) folio_set_hugetlb_cma(folio); @@ -3340,31 +3343,6 @@ void __init hugetlb_bootmem_struct_page_init(void) .max_threads =3D num_node_state(N_MEMORY), .numa_aware =3D true, }; -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP - struct zone *zone; - - for_each_zone(zone) { - for (int i =3D 0; i < VMEMMAP_OPTIMIZATION_NR_ORDERS; i++) { - struct page *tail, *p; - unsigned int order; - - tail =3D zone->vmemmap_tails[i]; - if (!tail) - continue; - - order =3D i + VMEMMAP_OPTIMIZATION_MIN_ORDER; - p =3D page_to_virt(tail); - /* - * prep_and_add_bootmem_folios() can access pageblock - * flags on bootmem HugeTLB pages, so initialize the - * shared tail struct pages here before bootmem folios - * start using them. - */ - for (int j =3D 0; j < PAGE_SIZE / sizeof(struct page); j++) - init_compound_tail(p + j, NULL, order, zone); - } - } -#endif =20 padata_do_multithreaded(&job); } diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index c48fcea076a5..7293706b532f 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -18,8 +18,7 @@ =20 #include #include "hugetlb_vmemmap.h" -#include "internal.h" -#include "mm_init.h" +#include "sparse.h" =20 /** * struct vmemmap_remap_walk - walk vmemmap page table @@ -706,95 +705,18 @@ void hugetlb_vmemmap_optimize_bootmem_folios(struct h= state *h, struct list_head __hugetlb_vmemmap_optimize_folios(h, folio_list, true); } =20 -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT - -/* Return true of a bootmem allocated HugeTLB page should be pre-HVO-ed */ -static bool vmemmap_should_optimize_bootmem_page(struct huge_bootmem_page = *m) +void __init hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootmem_page= *m) { - unsigned long section_size, psize, pmd_vmemmap_size; - phys_addr_t paddr; - - if (!READ_ONCE(vmemmap_optimize_enabled)) - return false; - - if (!hugetlb_vmemmap_optimizable(m->hstate)) - return false; - - psize =3D huge_page_size(m->hstate); - paddr =3D virt_to_phys(m); - - /* - * Pre-HVO only works if the bootmem huge page - * is aligned to the section size. - */ - section_size =3D (1UL << PA_SECTION_SHIFT); - if (!IS_ALIGNED(paddr, section_size) || - !IS_ALIGNED(psize, section_size)) - return false; - - /* - * The pre-HVO code does not deal with splitting PMDS, - * so the bootmem page must be aligned to the number - * of base pages that can be mapped with one vmemmap PMD. - */ - pmd_vmemmap_size =3D (PMD_SIZE / (sizeof(struct page))) << PAGE_SHIFT; - if (!IS_ALIGNED(paddr, pmd_vmemmap_size) || - !IS_ALIGNED(psize, pmd_vmemmap_size)) - return false; - - return true; -} - -/* - * Initialize memmap section for a gigantic page, HVO-style. - */ -void __init hugetlb_vmemmap_init_early(int nid) -{ - unsigned long psize, paddr, section_size; - unsigned long ns, i, pnum, pfn, nr_pages; - unsigned long start, end; - struct huge_bootmem_page *m =3D NULL; - void *map; + struct hstate *h =3D m->hstate; + unsigned long pfn =3D PHYS_PFN(__pa(m)); =20 if (!READ_ONCE(vmemmap_optimize_enabled)) return; =20 - section_size =3D (1UL << PA_SECTION_SHIFT); - - list_for_each_entry(m, &huge_boot_pages[nid], list) { - struct zone *zone; - - if (!vmemmap_should_optimize_bootmem_page(m)) - continue; - - nr_pages =3D pages_per_huge_page(m->hstate); - psize =3D nr_pages << PAGE_SHIFT; - paddr =3D virt_to_phys(m); - pfn =3D PHYS_PFN(paddr); - map =3D pfn_to_page(pfn); - start =3D (unsigned long)map; - end =3D start + hugetlb_vmemmap_size(m->hstate); - zone =3D pfn_to_zone(pfn, nid); - - if (vmemmap_populate_hvo(start, end, huge_page_order(m->hstate), - zone, HUGETLB_VMEMMAP_RESERVE_SIZE)) - panic("Failed to allocate memmap for HugeTLB page\n"); - memmap_boot_pages_add(DIV_ROUND_UP(HUGETLB_VMEMMAP_RESERVE_SIZE, PAGE_SI= ZE)); - - pnum =3D pfn_to_section_nr(pfn); - ns =3D psize / section_size; - - for (i =3D 0; i < ns; i++) { - sparse_init_early_section(nid, map, pnum, - SECTION_IS_VMEMMAP_PREINIT); - map +=3D section_map_size(); - pnum++; - } - + section_set_order_range(pfn, pages_per_huge_page(h), huge_page_order(h)); + if (vmemmap_optimizable_order(pfn_to_section_order(pfn))) m->flags |=3D HUGE_BOOTMEM_HVO; - } } -#endif =20 static const struct ctl_table hugetlb_vmemmap_sysctls[] =3D { { diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index 7ac49c52457d..20eb03df542a 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -9,8 +9,7 @@ #ifndef _LINUX_HUGETLB_VMEMMAP_H #define _LINUX_HUGETLB_VMEMMAP_H #include -#include -#include +#include "internal.h" =20 /* * Reserve one vmemmap page, all vmemmap addresses are mapped to it. See @@ -27,10 +26,7 @@ long hugetlb_vmemmap_restore_folios(const struct hstate = *h, void hugetlb_vmemmap_optimize_folio(const struct hstate *h, struct folio *= folio); void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *f= olio_list); void hugetlb_vmemmap_optimize_bootmem_folios(struct hstate *h, struct list= _head *folio_list); -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT -void hugetlb_vmemmap_init_early(int nid); -#endif - +void hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootmem_page *m); =20 static inline unsigned int hugetlb_vmemmap_size(const struct hstate *h) { @@ -76,13 +72,13 @@ static inline void hugetlb_vmemmap_optimize_bootmem_fol= ios(struct hstate *h, { } =20 -static inline void hugetlb_vmemmap_init_early(int nid) +static inline unsigned int hugetlb_vmemmap_optimizable_size(const struct h= state *h) { + return 0; } =20 -static inline unsigned int hugetlb_vmemmap_optimizable_size(const struct h= state *h) +static inline void hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootm= em_page *m) { - return 0; } #endif /* CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP */ =20 diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index b69a7af76858..7759f9de748c 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -32,8 +32,6 @@ #include #include =20 -#include "hugetlb_vmemmap.h" - /* * Flags for vmemmap_populate_range and friends. */ @@ -369,34 +367,6 @@ void vmemmap_wrprotect_hvo(unsigned long addr, unsigne= d long end, } } =20 -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP -int __meminit vmemmap_populate_hvo(unsigned long addr, unsigned long end, - unsigned int order, struct zone *zone, - unsigned long headsize) -{ - unsigned long maddr; - struct page *tail; - pte_t *pte; - int node =3D zone_to_nid(zone); - - tail =3D vmemmap_get_tail(order, zone); - if (!tail) - return -ENOMEM; - - for (maddr =3D addr; maddr < addr + headsize; maddr +=3D PAGE_SIZE) { - pte =3D vmemmap_populate_address(maddr, node, NULL, -1, 0); - if (!pte) - return -ENOMEM; - } - - /* - * Reuse the last page struct page mapped above for the rest. - */ - return vmemmap_populate_range(maddr, end, node, NULL, - page_to_pfn(tail), 0); -} -#endif - void __weak __meminit vmemmap_set_pmd(pmd_t *pmd, void *p, int node, unsigned long addr, unsigned long next) { @@ -599,7 +569,6 @@ struct page * __meminit __populate_section_memmap(unsig= ned long pfn, */ void __init sparse_vmemmap_init_nid_early(int nid) { - hugetlb_vmemmap_init_early(nid); } #endif =20 diff --git a/mm/sparse.h b/mm/sparse.h index 16b9bd3070aa..bc4c58ef24a9 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -16,6 +16,24 @@ static inline unsigned int section_order(const struct me= m_section *section) return section->order; } =20 +static inline void section_set_order(struct mem_section *section, unsigned= int order) +{ + VM_WARN_ON(section_order(section) && order && section_order(section) !=3D= order); + section->order =3D order; +} + +static inline void section_set_order_range(unsigned long pfn, unsigned lon= g nr_pages, + unsigned int order) +{ + unsigned long section_nr =3D pfn_to_section_nr(pfn); + + if (!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SECTION)) + return; + + for (unsigned long i =3D 0; i < nr_pages / PAGES_PER_SECTION; i++) + section_set_order(__nr_to_section(section_nr + i), order); +} + static inline unsigned int pfn_to_section_order(unsigned long pfn) { return section_order(__pfn_to_section(pfn)); @@ -26,6 +44,15 @@ static inline unsigned int section_order(const struct me= m_section *section) return 0; } =20 +static inline void section_set_order(struct mem_section *section, unsigned= int order) +{ +} + +static inline void section_set_order_range(unsigned long pfn, unsigned lon= g nr_pages, + unsigned int order) +{ +} + static inline unsigned int pfn_to_section_order(unsigned long pfn) { return 0; --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D41C3EDE7C for ; Tue, 4 Aug 2026 03:56:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815795; cv=none; b=kWqE5KHOR75tEdEKPsGSSZOVyUNSH0do6tACEK8ZjuLD4czXkL0bmJWwfaCW3SZfiPcIm1sTPRzXq+wwLFx8LnOn698EAo48bcB8/oXLMb3mHSPMXVupTPIt6CWaHTeXmnLkQ0tHDEmgHhPYJW74X+mcXRywqbA/n1TeTM9E1qk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815795; c=relaxed/simple; bh=hG7gj5kWW/y00TWpg1aWybHmnHSkedm1FGuaHoq2Lrg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IaGVAIdfh0kigvEfYUsgnklvaryV5xhkE14vyyDpF/dsL5OlOmnWWMsQZsOPN7F5YwM2Qvo9/TSiqLon9Lciq+2DpyYQzICIvggdmPjP3aXDXReJ1MvGjgAvJi+wN4g7NoEacOOmWfRnDTHXtOu80InFSfNUKmbG2tVswke5oEk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=TDQXLANt; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="TDQXLANt" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-38dcbade417so3873364a91.1 for ; Mon, 03 Aug 2026 20:56:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815793; x=1786420593; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=obMgSt9JudTp6IA7cwyJwAbMqHf5ZvnKhdpNNbjote4=; b=TDQXLANtaw3lMM1izkYyO/194Kb9Y2wNfQR/O5t2IHnu6UYOVp8DybgtFHVc5ou0Od 9waFvRY3eOf/r6RysZfdqmg+T3J2loXXRjhauw+ZnCX19TP2FC9zaO1TacE5ISqviSmL nyf+8p6wN/ccOjYfbF2viAR/hQfiEODD5KxjSYcOU3zcPCc1vk/dLyLvB2ulH6ewR1JK t1H5BCjRx80egByYYnyM+q2SwwMNCA/NnBCOSk3Eli7PrJg3FV7LIZKiVadzd99o6F3T A5Is0BEAnpuhbrX30fxIP87hdwBnEhsrhxz0rT0H8KCFzOqIxbPcRB9VvUTR5c0RonFG ZzsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815793; x=1786420593; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=obMgSt9JudTp6IA7cwyJwAbMqHf5ZvnKhdpNNbjote4=; b=OI6SXtXqBv7RVxrUBisMiEu6vu8+mMniuZU/nfnWdBuRiJQ5WzRigakoFg5f9cbVn8 VFAE7PveLdxvtxYeiGiZpy0TuZbNE4EgmfVn8k5GUV6GIo9+QXuG//GxUomrFqwyUD/p ZUfOeIeORY/1rzzGcmukaTLQBcfsvt3vB3Df9iw6/Y7hPrMdk/eBQ2/0Y0aVqnkwA8uI 9ZpngHla9V0SzqvhsIWrDoMHsC3teFo91xTeSTL3KKk/JFMUb7CQJHVwiQirlR1F4KJ9 quNTuTf8LvnMKykatTvs5FlDt3fez6Eo0ThqhS+PvMgxqF/V0O1Nl2gMLohD0VHQCqZQ 8pgA== X-Forwarded-Encrypted: i=1; AHgh+RqjU0Ad2kEjrQc5ZxPUXupgH/5WFMnP1szfs2jrIjOfYoalfoxc8qAxmIbM2Guj6Oj9Ir+j4+CGOlK0lms=@vger.kernel.org X-Gm-Message-State: AOJu0YywlQUUG61y5bVH7drX1J3o4lsdYC8PhPlJJyrZi6YYbEJGuKz3 sAytATNflnqAlgO8s6COB5T8jjOdF5HJktwOSocUJ9mA5k560Aku+9qO+ICc8tOBi5k= X-Gm-Gg: AR+sD11fnbnysdAF2FLkCc9snhwKuD9i/OZtND+NcE75+WYPgKeSWC4wMP3K/J0AV/u KRW2hCyCXjD7DtNg9ds8vGshIheWAbOkOl/pxLuQeJE4D3cRizWs5mKHUKDCONA6PEI0rjSZYiG B1KKbY2C3TncQqPHBHMBMgckAk89/YvKoQPM6eE03XpogjCYHSyXQ0KH5lfQ2kL8rXeXBHoNiyH Gyd0nwqlRLSVhh5dul64vOUGTf+hAEXhI3RsVbGI59/UAkyffa4fVPq31yuXMd0SjsP9OWaWOBr W30E1eqGLCLm3IZtpDKRuDo8ee6hcjTFkjNP+CGNwC6ds93MOZT9SEoiW4Tyrha401YoI4XuxFL Pb0O0U1XkPz5FVfiCEm9CZ3zwWcbP7Ib0mdWVM+7seVZ1RLJumpN+c2ARTspvelbZhrBPrPDi4o zTJSgvSRXFsmHkGJcWJVCmeqrWH49aeieMexA26RutnW3eKDRduSR7FnLH9qWQlNkzrh6EmKa3d y3L3va3 X-Received: by 2002:a17:90b:5867:b0:381:96a2:14d7 with SMTP id 98e67ed59e1d1-38fbc4ae4e7mr11397300a91.22.1785815792669; Mon, 03 Aug 2026 20:56:32 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:31 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 11/17] mm/sparse-vmemmap: remove SPARSEMEM_VMEMMAP_PREINIT support Date: Tue, 4 Aug 2026 11:55:29 +0800 Message-ID: <20260804035535.2846016-12-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" SPARSEMEM_VMEMMAP_PREINIT existed only to support HugeTLB's early vmemmap optimization setup. Now that HugeTLB bootmem vmemmap optimization uses the common section-based sparse-vmemmap path, the sparse initialization code no longer needs a separate pre-initialization mechanism for vmemmap population. Remove the related Kconfig symbols, section flag, and empty early hook, so present sections always go through the normal sparse setup path. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- arch/x86/Kconfig | 1 - fs/Kconfig | 1 - include/linux/mmzone.h | 25 ------------------------- mm/Kconfig | 5 ----- mm/sparse-vmemmap.c | 13 ------------- mm/sparse.c | 23 ++++++++--------------- 6 files changed, 8 insertions(+), 60 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f7b6735db7ed..c851dc47ad9c 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -149,7 +149,6 @@ config X86 select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP if X86_64 select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP if X86_64 - select ARCH_WANT_HUGETLB_VMEMMAP_PREINIT if X86_64 select ARCH_WANTS_THP_SWAP if X86_64 select ARCH_HAS_PARANOID_L1D_FLUSH select ARCH_WANT_IRQS_OFF_ACTIVATE_MM diff --git a/fs/Kconfig b/fs/Kconfig index e05917adcd60..d1c210c6508f 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -278,7 +278,6 @@ config HUGETLB_PAGE_OPTIMIZE_VMEMMAP def_bool HUGETLB_PAGE depends on ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP depends on SPARSEMEM_VMEMMAP - select SPARSEMEM_VMEMMAP_PREINIT if ARCH_WANT_HUGETLB_VMEMMAP_PREINIT =20 config HUGETLB_PMD_PAGE_TABLE_SHARING def_bool HUGETLB_PAGE diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 663726dd6463..db019fea5152 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2097,9 +2097,6 @@ enum { SECTION_IS_EARLY_BIT, #ifdef CONFIG_ZONE_DEVICE SECTION_TAINT_ZONE_DEVICE_BIT, -#endif -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT - SECTION_IS_VMEMMAP_PREINIT_BIT, #endif SECTION_MAP_LAST_BIT, }; @@ -2111,9 +2108,6 @@ enum { #ifdef CONFIG_ZONE_DEVICE #define SECTION_TAINT_ZONE_DEVICE BIT(SECTION_TAINT_ZONE_DEVICE_BIT) #endif -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT -#define SECTION_IS_VMEMMAP_PREINIT BIT(SECTION_IS_VMEMMAP_PREINIT_BIT) -#endif #define SECTION_MAP_MASK (~(BIT(SECTION_MAP_LAST_BIT) - 1)) #define SECTION_NID_SHIFT SECTION_MAP_LAST_BIT =20 @@ -2168,24 +2162,6 @@ static inline int online_device_section(const struct= mem_section *section) } #endif =20 -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT -static inline int preinited_vmemmap_section(const struct mem_section *sect= ion) -{ - return (section && - (section->section_mem_map & SECTION_IS_VMEMMAP_PREINIT)); -} - -void sparse_vmemmap_init_nid_early(int nid); -#else -static inline int preinited_vmemmap_section(const struct mem_section *sect= ion) -{ - return 0; -} -static inline void sparse_vmemmap_init_nid_early(int nid) -{ -} -#endif - static inline int online_section_nr(unsigned long nr) { return online_section(__nr_to_section(nr)); @@ -2387,7 +2363,6 @@ static inline unsigned long next_present_section_nr(u= nsigned long section_nr) #endif =20 #else -#define sparse_vmemmap_init_nid_early(_nid) do {} while (0) #define pfn_in_present_section pfn_valid #endif /* CONFIG_SPARSEMEM */ =20 diff --git a/mm/Kconfig b/mm/Kconfig index 060190e12bce..adf5ab53dc25 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -459,8 +459,6 @@ config SPARSEMEM_VMEMMAP pfn_to_page and page_to_pfn operations. This is the most efficient option when sufficient kernel resources are available. =20 -config SPARSEMEM_VMEMMAP_PREINIT - bool # # Select this config option from the architecture Kconfig, if it is prefer= red # to enable the feature of HugeTLB/dev_dax vmemmap optimization. @@ -471,9 +469,6 @@ config ARCH_WANT_OPTIMIZE_DAX_VMEMMAP config ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP bool =20 -config ARCH_WANT_HUGETLB_VMEMMAP_PREINIT - bool - config HAVE_MEMBLOCK_PHYS_MAP bool =20 diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 7759f9de748c..d9d2f4d02420 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -559,19 +559,6 @@ struct page * __meminit __populate_section_memmap(unsi= gned long pfn, return pfn_to_page(pfn); } =20 -#ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT -/* - * This is called just before initializing sections for a NUMA node. - * Any special initialization that needs to be done before the - * generic initialization can be done from here. Sections that - * are initialized in hooks called from here will be skipped by - * the generic initialization. - */ -void __init sparse_vmemmap_init_nid_early(int nid) -{ -} -#endif - static void subsection_mask_set(unsigned long *map, unsigned long pfn, unsigned long nr_pages) { diff --git a/mm/sparse.c b/mm/sparse.c index d4641aa70497..f0d9b7263bb2 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -294,27 +294,20 @@ static void __init sparse_init_nid(int nid, unsigned = long pnum_begin, if (sparse_usage_init(nid, map_count)) panic("Failed to allocate usemap for node %d\n", nid); =20 - sparse_vmemmap_init_nid_early(nid); - for_each_present_section_nr(pnum_begin, pnum) { - struct mem_section *ms; unsigned long pfn =3D section_nr_to_pfn(pnum); + struct page *map; =20 if (pnum >=3D pnum_end) break; =20 - ms =3D __nr_to_section(pnum); - if (!preinited_vmemmap_section(ms)) { - struct page *map; - - map =3D __populate_section_memmap(pfn, PAGES_PER_SECTION, - nid, NULL, NULL); - if (!map) - panic("Failed to allocate memmap for section %lu\n", pnum); - memmap_boot_pages_add(section_nr_vmemmap_pages(pfn, PAGES_PER_SECTION, - NULL, NULL)); - sparse_init_early_section(nid, map, pnum, 0); - } + map =3D __populate_section_memmap(pfn, PAGES_PER_SECTION, + nid, NULL, NULL); + if (!map) + panic("Failed to allocate memmap for section %lu\n", pnum); + memmap_boot_pages_add(section_nr_vmemmap_pages(pfn, PAGES_PER_SECTION, + NULL, NULL)); + sparse_init_early_section(nid, map, pnum, 0); } sparse_usage_fini(); } --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BAA5C41DE12 for ; Tue, 4 Aug 2026 03:56:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815799; cv=none; b=OCGWO64H7QsyZ+C4ELEcK1XjjOMZBLhup/Pv3fq8iN9MVR6h5wP0hZ6CCMZXyuza6YIqUnZhL9Sta2yBGwuDe6f2u8ywGxnN5mpPZPGHd8Urww/5O7yUQzYb95QvfvHv093rd3jJyje+Nw6R8ntRHOCwvEUTu5Al+QzkhjbTbMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815799; c=relaxed/simple; bh=jkYVRIFpg6oRaM2zZp2VMF/zSa5+K+mhbwNa5GF+QgI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NheFKYQlfQyMvEPGPpxe5Zb6jAytq6FrgJfauxMOaG6n1gfiWI6VHmWikg3o9rGLqSgdP9soy2Aj/aU3GbdANM8GmcBcjBntdp5G80hOJ3JEC0ZlIpC6IttgvFmtK6gM62U0OBiy85Wwy5+czW2UA9HtuYR3yOXVI7deq12E6ms= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=aQ8qTXvJ; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="aQ8qTXvJ" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-2d02b4c3601so45516735ad.3 for ; Mon, 03 Aug 2026 20:56:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815797; x=1786420597; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=kX0jaK+msJLtgVaqNEWt9m21jyQQ3JvP88rAHTiy+Ko=; b=aQ8qTXvJ2dpZu/MmbbLV+IFJ4iczeKg3tMiR9NOtG3PYtd+Rn6bm5Ybt8SA8poMtMK JiIvQT0Az33sLUpVXzYO8Vok3PvI0gh6ltnEpdhOZoovOYIm7tmzzRG8FvwDOC2H2ICY G9ULd5ud1FdTv5HDFJp0J6wlFNc3KXMyRPvM+wS0Iue9Q9ZQ2S1q6hV2iZN++U+5gZzr gUI6iqxu67bu2y2n3H43+JcFaMZd5qq1V/mbp4quJaubzoZLxk56rubiOTzJ6crUUhio gfCQWrh5UqQj9s3OpwvCj8XQ5cEwcW3aSnNy6r0811dOkUQDFLwzjaHpS+KsaaUQB0wX XWxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815797; x=1786420597; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=kX0jaK+msJLtgVaqNEWt9m21jyQQ3JvP88rAHTiy+Ko=; b=HTIIoDaFnXIZbeUsYk9M38S5t3TYsWKK/EfIlVzTW05ZHIQZboouaHeLagaAJ14mRS +ZSOpGWhaeem/5Qk7/o3hDa+9e34VZWiukj/o6+HMME0BJ8sni3BM65QMIZV7rFJZdvk OWH8VNbxpvZ+mwiRsPucqvgkEzsdncj7F2RgkjJzJSVINSk0ckjOoi+Ap/NaHpNSQ5WG Hqhzb5Pj1lJojz2z9zbdaKo5T479NiUpI3Is2DMHyI07BjUGL9+Y0Renl2UUlTmO4knR FnhnpPoQsmrBu7pnaQaL5l+Fw8B8fap7TkDL6APxuCyCX2pSyX0paaCBJZlMntQYbPeA wzPg== X-Forwarded-Encrypted: i=1; AHgh+RqHL26i7JKeTKv3HFA26JaKvzpjDqMUfGz6hv+5XjGKokkJTMtjlJBMVm9+72QKgT2U0L6l4PSpRZ0YFJg=@vger.kernel.org X-Gm-Message-State: AOJu0YzC3eRSVrDrwymRx6LiEtQDsfBWG9Tx2/q9lRm+P/UUni/lERu/ A85Pckip7rw+lZTP2PKYAlfPPBXOsO+JUaVYXIfdIDNR/jxyYaxqK8o4sBu9ZTaNCRU= X-Gm-Gg: AR+sD10IR6tZVxmO032cPOrvuxSn0GOcMhCg1pY3An6nS16vzTHAvhqNW5jeyD6P3vS AG5O6MDjPE5i8pq+GeQ5jyNokVvSgjWFrqIAdJ2LcoG3sT+38a2UrM3qesz2zatkNUV893CyBCU N5ge+99z569w9LGKhEDR759s0HfZo/N4jVLyxf7nGB9FQAvlfMxn+gD31UowFLEkgZlVTTiuDP/ SuB3ruMJMyomVIHzLhhdiqlmLN1EzEQvkkt1YOWSfNgeV6dC7kuQ2o0Svx1OTAFjdMRyfWf/bvI O5Cgdb+GSpIV0fxjxzwiLkM1WwHzb7L3CQ6FytwbRWogc6Bv8hwsAdoYTr2dhEPBrb4MyFxgeLe CY/SDTYl46gf6cDQOUHsPayWC+dEECVJdtuqJtdKgc/1pXBQxmZoypjGuWXwpuxX+jRU44ReFbN bWJikSL9DrjI/F4nPzjcFOzVSkS+XqAhhTP5dlXSZDM3fFgOCPc9hC/FJMO0gwL3fnUjmeFgNYp uWNBDM3 X-Received: by 2002:a17:903:1108:b0:2cf:9f0b:b562 with SMTP id d9443c01a7336-2d05242bbb5mr121614555ad.22.1785815796949; Mon, 03 Aug 2026 20:56:36 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:36 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 12/17] mm/sparse: inline usemap allocation into sparse_init_nid() Date: Tue, 4 Aug 2026 11:55:30 +0800 Message-ID: <20260804035535.2846016-13-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" After removing SPARSEMEM_VMEMMAP_PREINIT, sparse_init_nid() no longer needs the transient sparse_usagebuf state and its helper wrappers. Allocate the usemap buffer directly in sparse_init_nid(), pass it to sparse_init_one_section(), and drop sparse_usage_init(), sparse_usage_fini(), and sparse_init_early_section(). Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/mmzone.h | 3 --- mm/sparse.c | 46 +++++++----------------------------------- 2 files changed, 7 insertions(+), 42 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index db019fea5152..bcf536ec46af 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2225,9 +2225,6 @@ static inline bool pfn_section_first_valid(struct mem= _section *ms, unsigned long } #endif =20 -void sparse_init_early_section(int nid, struct page *map, unsigned long pn= um, - unsigned long flags); - #ifndef CONFIG_HAVE_ARCH_PFN_VALID /** * pfn_valid - check if there is a valid memory map entry for a PFN diff --git a/mm/sparse.c b/mm/sparse.c index f0d9b7263bb2..743112ac84b1 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -245,42 +245,6 @@ void __weak __meminit vmemmap_populate_print_last(void) { } =20 -static void *sparse_usagebuf __meminitdata; -static void *sparse_usagebuf_end __meminitdata; - -/* - * Helper function that is used for generic section initialization, and - * can also be used by any hooks added above. - */ -void __init sparse_init_early_section(int nid, struct page *map, - unsigned long pnum, unsigned long flags) -{ - BUG_ON(!sparse_usagebuf || sparse_usagebuf >=3D sparse_usagebuf_end); - sparse_init_one_section(__nr_to_section(pnum), pnum, map, - sparse_usagebuf, SECTION_IS_EARLY | flags); - sparse_usagebuf =3D (void *)sparse_usagebuf + mem_section_usage_size(); -} - -static int __init sparse_usage_init(int nid, unsigned long map_count) -{ - unsigned long size; - - size =3D mem_section_usage_size() * map_count; - sparse_usagebuf =3D memblock_alloc_node(size, SMP_CACHE_BYTES, nid); - if (!sparse_usagebuf) { - sparse_usagebuf_end =3D NULL; - return -ENOMEM; - } - - sparse_usagebuf_end =3D sparse_usagebuf + size; - return 0; -} - -static void __init sparse_usage_fini(void) -{ - sparse_usagebuf =3D sparse_usagebuf_end =3D NULL; -} - /* * Initialize sparse on a specific node. The node spans [pnum_begin, pnum_= end) * And number of present sections in this node is map_count. @@ -290,8 +254,11 @@ static void __init sparse_init_nid(int nid, unsigned l= ong pnum_begin, unsigned long map_count) { unsigned long pnum; + struct mem_section_usage *usage; =20 - if (sparse_usage_init(nid, map_count)) + usage =3D memblock_alloc_node(map_count * mem_section_usage_size(), + SMP_CACHE_BYTES, nid); + if (!usage) panic("Failed to allocate usemap for node %d\n", nid); =20 for_each_present_section_nr(pnum_begin, pnum) { @@ -307,9 +274,10 @@ static void __init sparse_init_nid(int nid, unsigned l= ong pnum_begin, panic("Failed to allocate memmap for section %lu\n", pnum); memmap_boot_pages_add(section_nr_vmemmap_pages(pfn, PAGES_PER_SECTION, NULL, NULL)); - sparse_init_early_section(nid, map, pnum, 0); + sparse_init_one_section(__nr_to_section(pnum), pnum, map, usage, + SECTION_IS_EARLY); + usage =3D (void *)usage + mem_section_usage_size(); } - sparse_usage_fini(); } =20 /* --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8142C3F4DFE for ; Tue, 4 Aug 2026 03:56:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815803; cv=none; b=Yr2IqDYalLBjl4V0Lwvyh9PyLW6bVYHEly8YHC50qfW8PrR7D0k/Lg64y7g/xcjS6Bc7gwkycQnLxIRaKnxAJi2hj3vJ29NaIiOnevkz3Eb+D3CCiBPpSlqtOqROBXYEQuvqEB1HEQPn+kXsEY9vpv3CIkc44vAUQPNDUMUrY+s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815803; c=relaxed/simple; bh=8dpVOPn5e8B5FjRVSHB30ztoDWTSs3HTSirWlW7yEN4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ev1mc9bwhWWNOv3v4YN63Qa7BKn1Xqee1+hGyJyUMqrFBckmcz4T3+bWBeJbXSIVAcqbM8NBV4jmyHKVZvw2DM2jzwGR7Xo02XzRpAth91A6ZvAt4tXGSJoom/+5+Jq5pDmMRLAI7gztsBFO8G8XDoXSenQhpmiPGEAvNlXnfG8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=BgZ3lvUd; arc=none smtp.client-ip=209.85.216.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="BgZ3lvUd" Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-38e7109321dso2401455a91.3 for ; Mon, 03 Aug 2026 20:56:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815801; x=1786420601; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=AD8u4rMpjqzdKGZCDXb9eLWKbdh2DMC0CGOyfVIpFQ8=; b=BgZ3lvUd5pLHXu/rkLFS1gFLr1/7aN9mwi5ak2cacMs4JznLDtPc20tyXCTKZr32mx A8MN5mFxUVX82P2+7JwgcLbPgIYJ3Ym93V3A/rfdE0MhOimhF/IHavcyYm3fSrwuRhUC 8vhwHJsWh+p6TG+7RL42BsLwCwUR3CeyaI5sFwsWzKS5+NRcnZ5keO5fFKfKuNHQg4gh H2vwSzEdErgEJkJqYdtNDKTv5d7vlEKeT/Dse5NlYt71+DT7Mpki4BPKYcN/rEZRZVnL pUyR96Ydj8XxKLeqhMIHeZpPwITsQUoQrybCRABRWNtOuc/KRy1LEMSLhUo6nPJYeotE Xm6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815801; x=1786420601; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=AD8u4rMpjqzdKGZCDXb9eLWKbdh2DMC0CGOyfVIpFQ8=; b=HXq8q9XiySI1X8UovNQJWLvzv8eY4pn0eBvMgv7as0ecNaMqnIQXwjjvtsDDiS5Hd0 jq9fSbm2IHucsPfZ6XAY7X/Ky/h1lq8mLFCUXIJb3yPj3zLbmDpb4i4FXK9GdVoh1wq1 YgHElAFxRjqXHHc5Hi6Rq82AW6T54R4tBwqr+whfkq4n/QzjPduu+xhUocE8XG7e1VZP yBIJr/0w0RK0iyfutt0dV+o4seTWUz+B1ii5wq4iWPH2Ubv/zhEVgun1viCNLZAk02/E WPUVhy0zS1MqlFBwvwTvP3XCKDjTkqeCbp5sXt4eHbbQs+A3CtMw5HjpsrT1d6YsXDqC 0MhQ== X-Forwarded-Encrypted: i=1; AHgh+RqlYHp4lPv20qg+mxUh9m7OuKUi6dPga6uXodzOXhyDkkcUdeDtjp2zBUjjzGYubZ1hemce9cOPxFjal4M=@vger.kernel.org X-Gm-Message-State: AOJu0YwmM5qLt+jwj9UiywAPqaGb2cU8LRBRGLX/scrq1emQ4MIKxrax 5dm8iNVZKQ2MCmLajfVBI49GENtyqwICNQpURAJ52vVKX89CFabGdVzF3dOKgnp7zuiw1qsI4/t VSy59mgN2QQ== X-Gm-Gg: AR+sD1142twXTt+fqbbnwKgYQqyvCTrrIXKOsn404qhhQT2ptyZc4/m0srfS4ilN77A ygGYvtoI46FSksxQxDlCIdZaaSRlMWa5gRmo9ti3vh/eUUYvPa7ju+SXYnDl/rDxqdT7bKWUwi7 bYcBFJKSseEdWdlHkjp7tn+F6u2QbguON2cv1zYVjkxryHnCsAnI54Vgg+U7vDgx5zgi590/UmR moUQrjlGyoWoEplughN8ZSP3QEiSoJYcImcTs7coUTV9Oybbp2s9mNgo5wp3mOAr2YUKGZH4ePD 15GEq5miLlW9Ieg0p9A9G4htgVVIboouHHDXSHEMkGyoC7U8BzY6VaczWzvOJJ8cZsPngjWDque oVhVnRcG41aQNj+qIge+uttabYh5Kk3PYsr0eY/n27ZFtlmRn+npryXao06+e2ZQekR7SAOpRmT 8UQCr0V0LrEQ++NOHlXiztpDWpvFZB6U/3d7WgWCRljz/C/miwh/xarweHhO4gxgQfYAuCK5kCp Zy1/y+izmBRB/KkoO4= X-Received: by 2002:a17:902:e5cd:b0:2c9:d8c6:1db8 with SMTP id d9443c01a7336-2d052298492mr116429075ad.28.1785815801366; Mon, 03 Aug 2026 20:56:41 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:40 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 13/17] mm/sparse: remove section_map_size() Date: Tue, 4 Aug 2026 11:55:31 +0800 Message-ID: <20260804035535.2846016-14-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" section_map_size() no longer provides any shared logic. After the sparse-vmemmap changes, its only remaining user is the !CONFIG_SPARSEMEM_VMEMMAP path in __populate_section_memmap(), which can compute the size inline with PAGE_ALIGN(sizeof(struct page) * PAGES_PER_SECTION). Remove section_map_size() and inline the remaining calculation. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/mm.h | 1 - mm/sparse.c | 15 ++------------- 2 files changed, 2 insertions(+), 14 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index fd7dc85f58f3..1adea81e80c5 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -5078,7 +5078,6 @@ static inline void print_vma_addr(char *prefix, unsig= ned long rip) } #endif =20 -unsigned long section_map_size(void); struct page * __populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap, struct dev_pagemap *pgmap); diff --git a/mm/sparse.c b/mm/sparse.c index 743112ac84b1..785c0be80aed 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -219,23 +219,12 @@ size_t mem_section_usage_size(void) return sizeof(struct mem_section_usage) + usemap_size(); } =20 -#ifdef CONFIG_SPARSEMEM_VMEMMAP -unsigned long __init section_map_size(void) -{ - return ALIGN(sizeof(struct page) * PAGES_PER_SECTION, PMD_SIZE); -} - -#else -unsigned long __init section_map_size(void) -{ - return PAGE_ALIGN(sizeof(struct page) * PAGES_PER_SECTION); -} - +#ifndef CONFIG_SPARSEMEM_VMEMMAP struct page __init *__populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { - unsigned long size =3D section_map_size(); + unsigned long size =3D PAGE_ALIGN(sizeof(struct page) * PAGES_PER_SECTION= ); =20 return memmap_alloc(size, size, __pa(MAX_DMA_ADDRESS), nid, false); } --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ABB042586C for ; Tue, 4 Aug 2026 03:56:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815807; cv=none; b=uBfFVTVKCNKg59Z+QVAUTMaYRhpJc8nEDtQHEHBdpn7A6eisKV3VO8SMYJZbzUl+PSKXHok7vswXhwnqeUKLxwIsJCGhyh2x8FbuO31y+aGJPhoKjGyh6O0ow1Hn9Xeb/6oA+HsBtp3t9mH6hkZuVE7ntas9zqaw0QMkxrzKDEU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815807; c=relaxed/simple; bh=vRo268ihAPpowRo6iHCo+NEZfdbxjoNlQ1X5Pan3s2k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QCpK6PjiHpjBGcTNhz0jRAPwG9uv46cquSO77qohHsgbaI/d+dABzdM2mDIzMYCJAqMk7FK8tFlja+WP8VjRtpH+SLt+e7OCiP93FrBp62JUu1DwTSPaHCLoZMyQgLnYV6yrHDDIm+cogBT0pGDb+k/1VzyY67YRwMR4l7iTwvQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=g7FFr44/; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="g7FFr44/" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2caed617615so56033705ad.3 for ; Mon, 03 Aug 2026 20:56:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815806; x=1786420606; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=xv+zhz7QIxLw7sIiewiDXoEbk5KnAZjoH00yp8HlINA=; b=g7FFr44/2fhWQQl6t9cyzpiiKF9KtY0yVKmRnl4zTizwgLuOSr+A5h6Av5LGcY8oE3 4GYUacvMje25RSQAg5LuxWJutfjwgsaiNZnddNnSjYtBDTWlnJ21HYXIVoGV4A0v9ShU WNqJN1XmoG0irMacm2DObfR6f3aj6O2LM3kqwXNe+JJXF/Hj1VgoHxvux6mYQYsNuS1F eZX9YiJyidjA3IpMdoOeiN6GdMkdIZz5FOnAhveNPmWbDykATUMEUcJFIZYK+yvSFW9p 2CGEOjUx1fxLPB12lEY5R8eM5CC57YVpI0UxJh+XyQ3kBAb9XbIQP6bRALjJiQTO9xAL dAOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815806; x=1786420606; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=xv+zhz7QIxLw7sIiewiDXoEbk5KnAZjoH00yp8HlINA=; b=AImVQczPPCI6vy+FBj9ABpiyklAQA7TjYwUnTyQF+cEYvMvi+Iuv0pENSuk3va5bhw yKlW9mnPCCi6SfCCEruJTkT9jMy9+RYJU92JRO72h0jQ6fpAD/5EIHmOjb5hdsyIGQof LpT6f+b0OrWTkXfIClcgaUTwRhZM1zFXIfBMoS5rgCJzWsemIjFWZVob7rUOzvbJmhvL 2Gi1A/2NhGkCfx86KlZoPgRg9Coeea4uKF9YC2a6v/86vWxruB/NyvtbcwAo42514yPC auezkFy/n2u/o++EE+uJJcFdWEkS2aquW33MoP78o1eZDV/ZSa3ThByHBBYBtC2gPUyf 7Flg== X-Forwarded-Encrypted: i=1; AHgh+Rp3swICYOITTBm2FTX5q1IIfF6AF6ZewhOhOrN1i8LR2XOfd2QrV+M5gSz0p/RgM3J/yKvY4U10ieDZx2M=@vger.kernel.org X-Gm-Message-State: AOJu0Yy7DwkuUU1HWN/JFK2UGCzqpm1CjHpAMK12yJWTlfXPATpb8qIA x4pvK8hS3peuh0hvqVG2YMjGHi/1GXSI3R9BraliySiVrtecCLaOwQ9aD+WWbwpkHb0= X-Gm-Gg: AR+sD11n2FmlV5jSV04+g0tcRPv2NbUR7UJT8YHPJuounctDogUyaF5A/A4OcQcvIhN P1Fi+/cpmXDpRaZQJdOzyiMaRbAqv5hFq2pBPeUAUOkq1RcO8aw8Gr9Pr26fhan7eZV47pjqhp5 Ed51PkQzC5kc0XUxiDb8njCzV9Bt+Om6ZlDYWF7If+D60hxIRq/8GVdA5XsjcfQGIHbirekUE7h SbXV31Aajx54foxOYUQBwvnWcZn+Eb+P7HNwWmVvxAJ1KqNrsDQaV7QzN+KlDfHACUyOiMxLco8 cyjD5Q3+oU6abBXedwN18dRre5CJyj6gLNvoDOsoD/pKRw4pYox8LQ9bz4nvioNz4t+ptqjlcO7 CZ5Jxl5oK8l1KWCkBiVIQr3QNDp6qv6g/4SpsBRIfwMWKwKL6Ff9OsyHrV2So1MzhgCicvby918 1SxGxRetO/huMZ2Y5kbTiwGn/EjluHmmJIjq9bMlx8jWzoYemfYK0wPNoMZmcp55XqQzIzx7xvA bEbIveC X-Received: by 2002:a17:902:e543:b0:2ca:329:3da1 with SMTP id d9443c01a7336-2d052215bb2mr125290605ad.16.1785815805645; Mon, 03 Aug 2026 20:56:45 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:45 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 14/17] mm/hugetlb: remove HUGE_BOOTMEM_HVO Date: Tue, 4 Aug 2026 11:55:32 +0800 Message-ID: <20260804035535.2846016-15-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The HUGE_BOOTMEM_HVO flag tracked whether a bootmem huge page had already gone through the old early vmemmap optimization path. Now that HugeTLB uses section-based vmemmap optimization, that state is already reflected in the section order. Remove HUGE_BOOTMEM_HVO and its helper, and use the section state directly when deciding whether to mark a folio as vmemmap-optimized. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v3: - Use the order-based helper for the bootmem vmemmap-optimized check v2: - Collect Acked-by from Mike Rapoport --- include/linux/hugetlb.h | 5 ++--- mm/hugetlb.c | 12 +----------- mm/hugetlb_vmemmap.c | 2 -- 3 files changed, 3 insertions(+), 16 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index fe28f98e1b22..3559041a5a57 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -675,9 +675,8 @@ struct hstate { char name[HSTATE_NAME_LEN]; }; =20 -#define HUGE_BOOTMEM_HVO 0x0001 -#define HUGE_BOOTMEM_ZONES_VALID 0x0002 -#define HUGE_BOOTMEM_CMA 0x0004 +#define HUGE_BOOTMEM_ZONES_VALID BIT(0) +#define HUGE_BOOTMEM_CMA BIT(1) =20 int isolate_or_dissolve_huge_folio(struct folio *folio, struct list_head *= list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 7d8507aa4c5b..7e2f186fa224 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3196,11 +3196,6 @@ static void __init hugetlb_folio_init_vmemmap(struct= folio *folio, prep_compound_head(&folio->page, huge_page_order(h)); } =20 -static bool __init hugetlb_bootmem_page_prehvo(struct huge_bootmem_page *m) -{ - return m->flags & HUGE_BOOTMEM_HVO; -} - static bool __init hugetlb_bootmem_page_earlycma(struct huge_bootmem_page = *m) { return m->flags & HUGE_BOOTMEM_CMA; @@ -3292,12 +3287,7 @@ static void __init gather_bootmem_prealloc_node(unsi= gned long nid) HUGETLB_VMEMMAP_RESERVE_PAGES); init_new_hugetlb_folio(folio); =20 - if (hugetlb_bootmem_page_prehvo(m)) - /* - * If pre-HVO was done, just set the - * flag, the HVO code will then skip - * this folio. - */ + if (vmemmap_optimizable_order(pfn_to_section_order(folio_pfn(folio)))) folio_set_hugetlb_vmemmap_optimized(folio); section_set_order_range(folio_pfn(folio), folio_nr_pages(folio), 0); =20 diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 7293706b532f..a25adc474351 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -714,8 +714,6 @@ void __init hugetlb_vmemmap_optimize_bootmem_page(struc= t huge_bootmem_page *m) return; =20 section_set_order_range(pfn, pages_per_huge_page(h), huge_page_order(h)); - if (vmemmap_optimizable_order(pfn_to_section_order(pfn))) - m->flags |=3D HUGE_BOOTMEM_HVO; } =20 static const struct ctl_table hugetlb_vmemmap_sysctls[] =3D { --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCDD642586C for ; Tue, 4 Aug 2026 03:56:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815812; cv=none; b=i83XZFSw62kvqXVpARdSWM3sWjQSSI1eDOYN6F9FCzoOXCF2/nXg848jYbItL3QipLjEcHkTssgQT2hQuhdmlRMp/zepo8qRJwT9BHjMZ2Ajbj7FWIWMWKM3wUOx7jxMOW8IB+i+CIiqWOQI4c6qGRzuDLMovq6mMQSKt1E1W8U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815812; c=relaxed/simple; bh=xXbFxxbirBe0zkcb579nCjp/W9qfDrTX0RqcI3ul3Wc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RwQFqVJhtTCDSIuOi/vyWYuIOANFzqI66CFx9/4azoAN97QtFDHDC5zrYeDdBeEU0ER70YHwX+HhgZv4dIO/5x/byVU9Cw1mGqfpnz2J8p4YHGJ71mRZoiz4dHLpDDXpThVgsOB/mpQXKw4D3MzgpHp8iLQMhSWVHSslihuj8TI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=DYyliYxo; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="DYyliYxo" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2ceb096e675so48790075ad.0 for ; Mon, 03 Aug 2026 20:56:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815810; x=1786420610; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=p9bQlBBDcNhy6LJjhudbhYloxJ0liqCISFNIfGIIeqo=; b=DYyliYxoCTfPz4SnB/qPOC7MdpWAyc+pHQI5Zo042ygKA5YsJKcw0wdla+lfb5Av10 UYqj+OpR5Vyd47Y9cVJKVV3bFA1u5bkr06U4KJAOokV9hirYZyH5Frax6+AhixajtFjy ItYgkpI8sigPQIfIW9IhziFexNPSnD521FCFzB5XaQyM1PWzIWxvyfrTyIGbu8AiYQFi 9kBRgVICbA0eB9SMujWIcMY0XFicpbPAFSr34YCQ4BDajzreFR2lpUUUF2HYatsZE4OK QULkDS+j6E2aIT+hDNK9iFUpNIQvwJIpkEqh7nXyNVk1I1g4nRf+84vFCNlGm86pXOhC PEAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815810; x=1786420610; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=p9bQlBBDcNhy6LJjhudbhYloxJ0liqCISFNIfGIIeqo=; b=SY6w8ZXltB0HruljvtTSrEQN4erHNfwvAWgOu/ZVmNBubLGKAwT5wngwGbH59uU/jC XHrLBIVAbUmoX8q6oTtNlfdHtkuNwL3J4GDGzsBDKRLtprNOfyJh0RpZXJPL2EU59Gu0 Ub3w5yFrNo0mY8nf/NR+wk8ouX9c+C+XCSd1WiVQ19aj1vjLF8e5D4kfEUR6NLudsXUT eUU5GREsmB5dLMD3V2+1/hPEtMdbzBYqcehlqa2PCvnSm26jYS74z5DJQaPnEbEaFy+D Mr6xV6aCWuOFvASpTYkMweCEqhntOzEhHu+3mmHB4jwWqu/78wHfA67F+wjN9PqrpuKL zQqg== X-Forwarded-Encrypted: i=1; AHgh+RqBcev4WeO+FmGXCGfSbIFCOvxe8sSZtY4jEXfBc1WDKqLFEYrWXaCkq9pNBVfGC/3fH7DOJnaDncxMj5c=@vger.kernel.org X-Gm-Message-State: AOJu0Yw5liNY9K9K4AYfsHGkvBx124l1vT6w0w1z7pNJqo8WBlL8XkI2 W2U/ywLlvbaqrwtUQQ+8NUmtnAr6BnNvwfwww/LnV0JYCdrlSD3svOTxbH4tWUcgm3Q= X-Gm-Gg: AR+sD1398YaPOhE3n6jlyiiiZwUK96I5sHwxIa2ruAQTbD34/Szq7dGoJ5gYcnX9WJh GcmBAadgTpUaS+fwar1VvQhljANIfOf9qKkWLJ2F1JSVfMgGQwqbdnzsivw7dJaIl+fVM4Uiea3 AS7s18guXT/14MuoXj2mmH2Mm6EDBehuT8Dq64lX4Y4N6Lp4vSGC46DeqG+JK3f4p//sJRkekdK q1wCUPpj+ksk35P4ag8xdbL6tOrwTGCZLc6Kkkhkjz82Lq+3TYGqDdtpLqmKgbBf4opA/MwuxpR MT+Tx1+WOrOKmazSpw+PY+GDQmI0yRzBBkwwIYBScc0OZ4Wb+obTI93CpCQC+teRX1LR6wd0IPI 8Ok0Xwggi5sVbC3R1IO2gYP+D42c/XGYEZ63DkFzuEgPFd5YHif9EAzvMdwzCg5Qlq+twvLf2vd QMbrURz5/VhX+4saot7QoMMWoD9xACMg8S6UK5OvFDzISMXOU8wBdZj4jxXoZoiqEW76nag6iEP PBqo3iEwd/3+kMkodM= X-Received: by 2002:a17:903:983:b0:2cc:8ad3:423d with SMTP id d9443c01a7336-2d052299eb4mr125935575ad.29.1785815810135; Mon, 03 Aug 2026 20:56:50 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:49 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 15/17] mm/hugetlb: remove HUGE_BOOTMEM_CMA Date: Tue, 4 Aug 2026 11:55:33 +0800 Message-ID: <20260804035535.2846016-16-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Track early CMA hugetlb pages from the hstate instead of storing a redundant bootmem flag. This removes the unused helper and keeps the bootmem metadata minimal. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/hugetlb.h | 1 - mm/hugetlb.c | 14 ++++---------- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 3559041a5a57..255a258f11d1 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -676,7 +676,6 @@ struct hstate { }; =20 #define HUGE_BOOTMEM_ZONES_VALID BIT(0) -#define HUGE_BOOTMEM_CMA BIT(1) =20 int isolate_or_dissolve_huge_folio(struct folio *folio, struct list_head *= list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 7e2f186fa224..df5944651b9a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3120,7 +3120,7 @@ static bool __init alloc_bootmem_huge_page(struct hst= ate *h, int nid) */ INIT_LIST_HEAD(&m->list); m->hstate =3D h; - m->flags =3D hugetlb_early_cma(h) ? HUGE_BOOTMEM_CMA : 0; + m->flags =3D 0; =20 /* CMA pages: zone-crossing is validated in hugetlb_cma_reserve(). */ if (!hugetlb_early_cma(h) && @@ -3196,11 +3196,6 @@ static void __init hugetlb_folio_init_vmemmap(struct= folio *folio, prep_compound_head(&folio->page, huge_page_order(h)); } =20 -static bool __init hugetlb_bootmem_page_earlycma(struct huge_bootmem_page = *m) -{ - return m->flags & HUGE_BOOTMEM_CMA; -} - /* * memblock-allocated pageblocks might not have the migrate type set * if marked with the 'noinit' flag. Set it to the default (MIGRATE_MOVABL= E) @@ -3291,9 +3286,6 @@ static void __init gather_bootmem_prealloc_node(unsig= ned long nid) folio_set_hugetlb_vmemmap_optimized(folio); section_set_order_range(folio_pfn(folio), folio_nr_pages(folio), 0); =20 - if (hugetlb_bootmem_page_earlycma(m)) - folio_set_hugetlb_cma(folio); - list_add(&folio->lru, &folio_list); =20 /* @@ -3304,7 +3296,9 @@ static void __init gather_bootmem_prealloc_node(unsig= ned long nid) * For CMA pages, this is done in init_cma_pageblock * (via hugetlb_bootmem_init_migratetype), so skip it here. */ - if (!folio_test_hugetlb_cma(folio)) + if (hugetlb_early_cma(h)) + folio_set_hugetlb_cma(folio); + else adjust_managed_page_count(page, pages_per_huge_page(h)); cond_resched(); } --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A9F142586C for ; Tue, 4 Aug 2026 03:56:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815816; cv=none; b=S5+BqU/u1paK7fFlZG7RUP3oVwOrYYXh5TCT1eckm+Emv/ZhOrPLXCsQwItv+7xy1vTrjnC4sq6dmk3yzOv8R90oBPaNh5InNCpww6ZYESCKbHs+ZlQ2cZQgAtC+XoONAPhige4aneBmxPIG6Ck62nbxATT6D+BD3S3LGf16vaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815816; c=relaxed/simple; bh=OD8QJVB4aCHONKwfQz11L99i05mBCOntBpMb+YSZ3ZE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hrGIy+1nS49C6Vs4A0LxhRkzYr9B5QmK7+kLLn6z20blg9YUrCtpovx3F6ssem5rm631YIWapaagzAnVyjCL2ayvsqiWCD0PkLdxymkd1iZCUOuA5cKU73f7EkN3RsiSEuHwLRR5yiHdWIb2ezET7mLktqvwwzMbVtsCGFYXEO0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=Np3jU5W+; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="Np3jU5W+" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2cacf197759so60702705ad.2 for ; Mon, 03 Aug 2026 20:56:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815815; x=1786420615; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=HVMuForKZApPU3H4utqVM44IJGHsuiuqxhUD4Ue7wX4=; b=Np3jU5W+eznc96Jo9qv3vXhzVjFxLYB9+jy4FZTl9oF1wRU0KReB5nLKG2jc/2CiCR dfSS3qxcFf6Xf0ob/kkzDrx2FN3TJglLBrUxwqi/sCZ8GEpOFNvqcF1Z+cK46A522If/ b2HH/mvRhR4QkVr+nB1kTkYuA/xZNkdifdKg1fnmoc6K9QK+tL5m8a5L3Yevs1pT4hxm kJ+SHsvwOx3KBB+6mKRaJ/cAMH9ryx/sczKZuoPQA/kitPu6pWz8ZLuE90yZqq/Nvw8q Lt6+38tYq6LVQ4ao3GClzS2NIxcCdj1MsJgXpTP93fOmT8HkiUm1ErZ1aqbRXwhSlC4I F07Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815815; x=1786420615; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=HVMuForKZApPU3H4utqVM44IJGHsuiuqxhUD4Ue7wX4=; b=H/+REMWPsA2A3oYZw8Ha80JrG3EyhnBBMzQx0IG/8bnuFaPj3WQPRgjzuUmToPuABB kpjIsFPrWOkGn4uyEhaDtxVzX+GNRygy06jVY8HdJZNPbY+ieWHtsTxMUFe7tovTtDgs AJVSmD4hQJUs2ESabhMa7dYJML6YW5vOUt3/aAZMB79Ax3vchFSphyCKjnj6b5mKS2Nz k4gPZVYPs0XqeSofCZk3/njmf/H2AKdbNc+uzxA+Y64RyL2T1aa+VQ6Bjjig1TUfhLK0 i2abCu1rTJfiCR4MMAhqkxtzBGt9IHo7FSSZY5KO1LVaVpWDBx1OBOwcJ4OeaO2H/biS qw0g== X-Forwarded-Encrypted: i=1; AHgh+RovF59RTBQnVsCULAqTXS3SAO9kzYH/64Tt2TzJJpiEembZVHc6YC3DNO4Dy5USkDayYBFoVIlmgTIpu74=@vger.kernel.org X-Gm-Message-State: AOJu0YwbX8Xa7Re0yQH6Z+kyj+XbuOWI4roPXdxSGODG2NeckPLLMmVy fPvc0MY54oexnM3BM/gZRMhdcQzFmk+LnQyMr9M5lDCJxgIZ6ZR4R+8HOIO4hqo29oA= X-Gm-Gg: AR+sD13RMCGk77E605R03FIBgMUmuZwMLDYuYrd8d0o4F6jPM/zntReii6wEU8vcjIe V64hGey50tIIf7MFRpgpJ/CwmmSBz5Skf5Kj4NPV5F3/qIoNBYXwS7cLVAfaJWr6nNs1KefZSTJ R7MX/Rn1I4vVRN7mnV9EQ5mgj8AqvYV5YK+/iTZl+Z0e6Qdguhp2tMeqm1yzPG3h6qWdpWUfMX9 bczLaHDm4tzVcAXln96YaOD2OO83r0FQTHu14iECDBxYFZ5usjZEhwpL2Mqt88sJ0m+XY2JUT+2 rgNyUB1PL5Nbjk60zA5gsrYBbko1wE9qfMsOW34MPlm8ykKPjAGhg4/bUldNiJ4BCVPaR1d4vhi sdGL/oofZvxZzqy+GnsTEedgp4w/KFYB4NiYNbIPq5rbTv2Z3gQQxIr/zWrL/kwnexxCwL0N0Qe YRPBcXHYxDjV4uSglgM3+dQXkJCb2U7bhe5WWmIBuB65YIMwQPMrnKtUl0hcqHj5pCar/MKOG+U 0dNN6uo X-Received: by 2002:a17:902:ce8b:b0:2cc:ee78:3236 with SMTP id d9443c01a7336-2d0523e8aadmr122285575ad.33.1785815814649; Mon, 03 Aug 2026 20:56:54 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:54 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 16/17] mm/hugetlb: localize struct huge_bootmem_page Date: Tue, 4 Aug 2026 11:55:34 +0800 Message-ID: <20260804035535.2846016-17-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" struct huge_bootmem_page is only used by hugetlb boot-time allocation code, but its definition currently lives in mm/internal.h because hugetlb_vmemmap_optimize_bootmem_page() takes it as an argument. This exposes a hugetlb-specific internal type more broadly than needed. Change hugetlb_vmemmap_optimize_bootmem_page() to take the information it actually needs. With that interface, mm/hugetlb_vmemmap.h no longer needs to include mm/internal.h, and struct huge_bootmem_page can move into mm/hugetlb.c. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Pass PFN and order to hugetlb_vmemmap_optimize_bootmem_page() instead of struct hstate and bootmem metadata to simplify the code further - Collect Acked-by from Mike Rapoport --- mm/hugetlb.c | 8 +++++++- mm/hugetlb_vmemmap.c | 8 +++----- mm/hugetlb_vmemmap.h | 5 ++--- mm/internal.h | 7 ------- 4 files changed, 12 insertions(+), 16 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index df5944651b9a..d978992e2770 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -55,6 +55,12 @@ #include "sparse.h" #include =20 +struct huge_bootmem_page { + struct list_head list; + struct hstate *hstate; + unsigned long flags; +}; + int hugetlb_max_hstate __read_mostly; unsigned int default_hstate_idx; struct hstate hstates[HUGE_MAX_HSTATE]; @@ -3138,7 +3144,7 @@ static bool __init alloc_bootmem_huge_page(struct hst= ate *h, int nid) } else { list_add_tail(&m->list, &huge_boot_pages[nid]); m->flags |=3D HUGE_BOOTMEM_ZONES_VALID; - hugetlb_vmemmap_optimize_bootmem_page(m); + hugetlb_vmemmap_optimize_bootmem_page(pfn, huge_page_order(h)); /* * Only initialize the head struct page in memmap_init_reserved_pages, * rest of the struct pages will be initialized by the HugeTLB diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index a25adc474351..eb339c4a71f4 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -19,6 +19,7 @@ #include #include "hugetlb_vmemmap.h" #include "sparse.h" +#include "internal.h" =20 /** * struct vmemmap_remap_walk - walk vmemmap page table @@ -705,15 +706,12 @@ void hugetlb_vmemmap_optimize_bootmem_folios(struct h= state *h, struct list_head __hugetlb_vmemmap_optimize_folios(h, folio_list, true); } =20 -void __init hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootmem_page= *m) +void __init hugetlb_vmemmap_optimize_bootmem_page(unsigned long pfn, unsig= ned int order) { - struct hstate *h =3D m->hstate; - unsigned long pfn =3D PHYS_PFN(__pa(m)); - if (!READ_ONCE(vmemmap_optimize_enabled)) return; =20 - section_set_order_range(pfn, pages_per_huge_page(h), huge_page_order(h)); + section_set_order_range(pfn, 1UL << order, order); } =20 static const struct ctl_table hugetlb_vmemmap_sysctls[] =3D { diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index 20eb03df542a..464192e32dec 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -9,7 +9,6 @@ #ifndef _LINUX_HUGETLB_VMEMMAP_H #define _LINUX_HUGETLB_VMEMMAP_H #include -#include "internal.h" =20 /* * Reserve one vmemmap page, all vmemmap addresses are mapped to it. See @@ -26,7 +25,7 @@ long hugetlb_vmemmap_restore_folios(const struct hstate *= h, void hugetlb_vmemmap_optimize_folio(const struct hstate *h, struct folio *= folio); void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *f= olio_list); void hugetlb_vmemmap_optimize_bootmem_folios(struct hstate *h, struct list= _head *folio_list); -void hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootmem_page *m); +void hugetlb_vmemmap_optimize_bootmem_page(unsigned long pfn, unsigned int= order); =20 static inline unsigned int hugetlb_vmemmap_size(const struct hstate *h) { @@ -77,7 +76,7 @@ static inline unsigned int hugetlb_vmemmap_optimizable_si= ze(const struct hstate return 0; } =20 -static inline void hugetlb_vmemmap_optimize_bootmem_page(struct huge_bootm= em_page *m) +static inline void hugetlb_vmemmap_optimize_bootmem_page(unsigned long pfn= , unsigned int order) { } #endif /* CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP */ diff --git a/mm/internal.h b/mm/internal.h index f47f06c55548..8862d447ba8b 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -23,13 +23,6 @@ #include "vma.h" =20 struct folio_batch; -struct hstate; - -struct huge_bootmem_page { - struct list_head list; - struct hstate *hstate; - unsigned long flags; -}; =20 /* mm/workingset.c */ bool workingset_test_recent(void *shadow, bool file, bool *workingset, --=20 2.54.0 From nobody Fri Oct 2 07:46:34 2026 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24748432BF4 for ; Tue, 4 Aug 2026 03:56:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815820; cv=none; b=YwsZwq2u1hiv4gGEgZ+N+S6NubhWiAcW82wKTTrN8oxHxy9NaVh/T3VIgV9sCrghsnjOSOk+l9zzaQDAePJkMDEeEFZJ5gsfxNftN3ER0M4s4fvpim3HPqwIhmPgY2cGwvN/yrmG5k68/DDf+j0PdldDPjfSoKvtucCXJuoIXzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785815820; c=relaxed/simple; bh=jf7amfsptPrd3VY+wDIBL7whqZH6cfJQTniFP/6VU0s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JARUxNHgmspnLytj342rMp4Hhny7GGrcpYs8YidaV9QvnRj8I7XrMNJMC9cr2JFPnuAQj4eGGt2kwNq6x5MEgAcjgwjY0e+TQCefhCiVAZ6Uuby2JuVRmDHmJBF8RHuNsDVeakNM9K+ht4oCThUssPBCnPGsM2mQLjZizOoETYQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=g0CTlZ3Z; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="g0CTlZ3Z" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2ced3386430so41277655ad.1 for ; Mon, 03 Aug 2026 20:56:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1785815818; x=1786420618; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Mqxr9eZTBHl5pXezv3m2vidjAQ6kefgcXwuYmW18Fa0=; b=g0CTlZ3ZdT+PhVYVeUvtEA4bFcBHJGiimp7sVlVCieGm8q/+XO3xP7G2wD/dtg017F Noz/Ze8IfeeTFLB6WNRRgEapn8yxGMCnJaVDMGX9ytm3xOFP5vtXHMFBnW6wGF0V6W/O a8yXGtFl478kdI3DvSlXdJzrmB0hcW6et/axwqKi3SqgmS0t1QlQCTW0Nouj46RZmAFw gpDs6/rVcCKQ6MHIumC0Gv5sNn4jrP0tukE+WqmL99ngdUU8sBIDBniBp2ggSo+HpCL5 w3zxFVYDQpKDgnKqYzIkxwr9yJsFEeUM2p/1utdmm5zT83jzP9sHdRelyKIyet3rJ3ah 9kCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785815818; x=1786420618; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Mqxr9eZTBHl5pXezv3m2vidjAQ6kefgcXwuYmW18Fa0=; b=FbWnoeG4Ac3ocqQGpqQv4K9W79uwQmVwxna11o6UbPNVy2b8y+UCzcwDEHT36bLOVC Cp3VUnX20foIHSdKgpSUIUdsaXLW367HGigRRtWFcxz/tzqhPMA6ekTEM/4IKHrNFgb/ RxrPThBf+LFRE+9PNretHgXeruk0IVjMm2Hj63Pmab4tx0umgJco9ngKmccam10RyyJP MxPWfhftvNSAq4PRZPpbfi6em4UanWbC+K5kuKEx4l79T4q63++AQVSaYXtQem6oEOdn Qdc9s4pu2RKquNj///FOrX2GuyWRxm4A4Gcog6iMSYNGbuDjfHx+Pco91OQJ4Xba/mp+ 7VMA== X-Forwarded-Encrypted: i=1; AHgh+RpmPX5K3KS+Xf60xvwkhSAWrYxj2jwlsCX4dwpYjsrLPaS4chPuZE0xU813pInjwG/6I1/OZEhID42Dhzk=@vger.kernel.org X-Gm-Message-State: AOJu0YwclEeDHgAQspGSPOq8I4Thuhm4VfKx8CJYbUrYhleNepbHIP7q mQ2t2U2YOZ+wK+FwhjVO9ooip8+8iP+6+p1Dh1vSZe7QP/DOyQUaXqRl9clJxYXNHvY= X-Gm-Gg: AR+sD12QhGo2WV/9wH8wDu4HY7q+KgDrotynAiHOgYfjccukMgqR6KD0N2vVB6Rdqxi jCAt3XPlDxuJkJhLUZCCGJhUfVHhhA+tFnkooVcc3YNmmNgdEEMHfR62daduxYYUgUKIv/0pDN9 PrqAf1Lnk5GgdnzMD3z6Ad/ZC2i94/zAbC+xvE3QKwkADCrnzSkDgFi34X3SAomC/8iynFW3Ks2 QrycGjXXHsRGIzN/JP50ofioLLXsDTWhXwziOt9hwvnM/GPMaXoWk/c/u7aUq6wggy2xyfLKa2B BVCaNmgUTpzqqanLox6vcZBCbv3uIWsIzfolNiQ8aRYZma5A9wO/HC9GkIbcg5ka/U3pfrU+PdL Qxr6lpuTV8eqnbnq0Jq2xmFAZnAWNj7XzXoCjVpV+Js8BIUo3t46nv8jpSBY/djOSj8SvIDKm06 uBw8lE+LXHmguBOSRRyneNudNGkVS4l8wOhI2eUcirxGLG3+qoD0BxWgDtc9+LDaPDPjkzhr0pf 7TuC2u9 X-Received: by 2002:a17:902:d2cf:b0:2ce:93a3:c168 with SMTP id d9443c01a7336-2d0521a4f6emr120492725ad.7.1785815818485; Mon, 03 Aug 2026 20:56:58 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.99]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04ae99af1sm46313695ad.33.2026.08.03.20.56.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 20:56:57 -0700 (PDT) From: Muchun Song To: Andrew Morton , Oscar Salvador , David Hildenbrand Cc: Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Michal Hocko , David Laight , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Muchun Song Subject: [PATCH v3 17/17] mm/hugetlb: localize HUGE_BOOTMEM_ZONES_VALID Date: Tue, 4 Aug 2026 11:55:35 +0800 Message-ID: <20260804035535.2846016-18-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260804035535.2846016-1-songmuchun@bytedance.com> References: <20260804035535.2846016-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" HUGE_BOOTMEM_ZONES_VALID is only used by the huge_bootmem_page flag handling in mm/hugetlb.c. Keep the definition next to that private data structure instead of exposing it through the public hugetlb header. No functional change is intended. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- v2: - Collect Acked-by from Mike Rapoport --- include/linux/hugetlb.h | 2 -- mm/hugetlb.c | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 255a258f11d1..900c95e346b2 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -675,8 +675,6 @@ struct hstate { char name[HSTATE_NAME_LEN]; }; =20 -#define HUGE_BOOTMEM_ZONES_VALID BIT(0) - int isolate_or_dissolve_huge_folio(struct folio *folio, struct list_head *= list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); void wait_for_freed_hugetlb_folios(void); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index d978992e2770..cddcd752505f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -55,6 +55,8 @@ #include "sparse.h" #include =20 +#define HUGE_BOOTMEM_ZONES_VALID BIT(0) + struct huge_bootmem_page { struct list_head list; struct hstate *hstate; --=20 2.54.0