From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 463E73DE453 for ; Tue, 2 Jun 2026 10:10:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395056; cv=none; b=HyaHTnD1aGGEKkvtO9RgH8NG/VPlda4o6C8irJqvKyUjBzH1gpcSKGGRonrG2dTMJEIgnZ2h7eFqarHkC2iqZ3I7gQ4VAKnr4EAkAJek9nvdiEKf6j8olgEK2Cum4X7nHRzexG8aBiPbLZAEamRNJfR/SICVyZj87aSeI35TV1s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395056; c=relaxed/simple; bh=5Q1oHKwNBCRLgohR+kG/Ms9n24KMIUmkvYdRvoHO5J0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nPRnDTTaTr2rjfDM1NtSqRvihdAu+pGa5ZGWr58yc2HcIhT6J3e0zDs/McwF6OIx5MbJeafXtew0TC+txwD/kB5Yt6GurzqnzWMIi0A2bWgRazlHcm6bpLmRdE2ak61qZptEyUPb0q8XeexRAtwC0pwmiNs0utrmoMn3lka8+Zc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=c77rcPdF; arc=none smtp.client-ip=209.85.215.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="c77rcPdF" Received: by mail-pg1-f178.google.com with SMTP id 41be03b00d2f7-c85b73ffb52so816488a12.3 for ; Tue, 02 Jun 2026 03:10:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395054; x=1780999854; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BDasRSBzRzMBw4N3DGUFq9UV94c4KSJz0jxvz4ocu1Q=; b=c77rcPdFO1PfJJiOma0VTVlYiBdxECrTUngMvg+HqIy30Rh8ixO4RRBXgfkztA0ks7 huhfYAgtVv9Q86AY1AEoydU3AYam/qVCNw+I34gbT3Hz0/BOUtH3hNhyztaOfr8BYYgX cTmepXRpZRH5iTCIYXZx98KLsw+mj/Ril+MeXxAU+FNk0FSP2w2S/85O9d6peiUrHIRb f5PGK2Q2RfyjEQGaRzD8/DDdEhMOcEHhhkMXtExPrbOr9Z5k4lesQiVWkYFiIfxfihB/ idRnXzl3pp9KnhWGPziSkO+m95ISCBj2tuEujMuLWJNtDQ8amaDXh7dQKUoTN/ZCN+YD v7rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395054; x=1780999854; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=BDasRSBzRzMBw4N3DGUFq9UV94c4KSJz0jxvz4ocu1Q=; b=IOMnoIViAD0z188j0WiTv0AiNVQF+R7jJLZOB8/Fo3mYi/6VakCJ8iLc2aTEOUk4wJ 1GAiKDFH8IERg27Yugs+t7BuFd/mCiw5WicDUH6PacSkoPDcMf2OJgsXAts4bTpzMd6K AocvElnKKhgkvTqNlGiupuCdJdVtgoamfneBKsNrIpcfu9nc2MWDmdZXkUk2grGjlHDL DJzttCKKDR+GXUTj2jmmiAJNtwt3/1mhTTlG9jbwfLr85u3cwKqEMfsAfc3PXCoDgY2j hvqPWpp6Qra/Y9iZ28RKdEnU6CC5U8QH/QaGrs63Yu800Ibwbs5L15DjzQ1GxUsFjTUH MVJA== X-Forwarded-Encrypted: i=1; AFNElJ9l0ipNS3hpQ6/fyYkHdJd7vz+u5OP/RFUJ5SIFDhJ26dDRMi+XG5LLCKsZqr05hQU9N5studEAzRNrLxU=@vger.kernel.org X-Gm-Message-State: AOJu0YwDR3+ceAb/z8SMdqtlzfKralbfJC18lLvBq6C1RrC4bAt/hz65 DXsYtkoWTH+K7Dly6XiHL8A7w1gwd2iJw8Ax5nesHS5/EW10IDOIVf0uxBFa/5xhI+k= X-Gm-Gg: Acq92OHA4pTwsM88gADcxW16dWhakezCo1QPo0KMX5UKTlwOuFBq1q3hpFVAA1cVc1Z oG27/cbRjijuT3uUi9grGuVTnEs1Kc0JGvMON2x7hzVrIlqRsEV2LNEMeNLaCFQpcxWQxX413j/ gjQda62qUryFZIFeFd8vE/8BS0ALlD+j7RTxdgrScjvXvu3tRNidmDHSRpYtrIw+aPM4Vptmt8i HwLAoXCzs1bG8iTVWbWjC9c4sxlYemu8dJ38iQxZsXzKepLEee60PnwtTC0ccg0ZVy1aC/lUIEp 5MT74wWeITjEVhV8OiRj60ib4tWWnxiSSniB/l9yNY7wB3BuodrRCaYTVNuxVbw1Awdkpv1VO20 YpVr76f797UiVvD1RUyLla+NOxSTpCqIrPbOewTAzKumooURoZXr8y6wIJyg+KYx4027ZTId+AG P5vfBj1Tub3zj4HX82t9CrwzMLV0JI5cl092mcykP4kxtQDY+8tJviX8M= X-Received: by 2002:a05:6a00:4406:b0:842:4327:6cfb with SMTP id d2e1a72fcca58-8424327755cmr10725987b3a.46.1780395054175; Tue, 02 Jun 2026 03:10:54 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.10.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:10:53 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 01/19] mm/hugetlb: Fix boot panic with CONFIG_DEBUG_VM and HVO bootmem pages Date: Tue, 2 Jun 2026 18:10:21 +0800 Message-ID: <20260602101039.1867613-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Commit 622026e87c40 ("mm/hugetlb: remove fake head pages") switched HVO to reuse per-zone shared tail pages from zone->vmemmap_tails[]. Those shared tail pages were initialized in hugetlb_vmemmap_init(), but bootmem HugeTLB folios are prepared earlier from gather_bootmem_prealloc(). With hugetlb_free_vmemmap=3Don, prep_and_add_bootmem_folios() can access pageblock flags on bootmem HugeTLB pages whose mirrored tail struct pages already point to the shared tail page. On CONFIG_DEBUG_VM kernels, get_pfnblock_bitmap_bitidx() then dereferences the still-uninitialized shared tail page and can panic during boot. Initialize zone->vmemmap_tails[] from gather_bootmem_prealloc(), before bootmem HugeTLB folios are processed, and drop the later initialization from hugetlb_vmemmap_init(). This bug only affects CONFIG_DEBUG_VM kernels, where the relevant assertion is evaluated. Fixes: 622026e87c40 ("mm/hugetlb: remove fake head pages") Signed-off-by: Muchun Song Acked-by: Oscar Salvador --- v2->v3: - add a comment explaining why shared tail pages must be initialized from gather_bootmem_prealloc() before hugetlb_vmemmap_init() runs (per Oscar Salvador) - update the stale sparse-vmemmap comment to point to gather_bootmem_preall= oc() as the bootmem HugeTLB shared-tail initialization site (reported by Oscar Salvador) --- mm/hugetlb.c | 25 +++++++++++++++++++++++++ mm/hugetlb_vmemmap.c | 17 ----------------- mm/sparse-vmemmap.c | 2 +- 3 files changed, 26 insertions(+), 18 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 571212b80835..cd55524c7e30 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3365,6 +3365,31 @@ static void __init gather_bootmem_prealloc(void) .max_threads =3D num_node_state(N_MEMORY), .numa_aware =3D true, }; +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP + struct zone *zone; + + for_each_zone(zone) { + for (int i =3D 0; i < NR_VMEMMAP_TAILS; i++) { + struct page *tail, *p; + unsigned int order; + + tail =3D zone->vmemmap_tails[i]; + if (!tail) + continue; + + order =3D i + VMEMMAP_TAIL_MIN_ORDER; + p =3D page_to_virt(tail); + /* + * prep_and_add_bootmem_folios() can access pageblock + * flags on bootmem HugeTLB pages, so initialize the + * shared tail struct pages here before bootmem folios + * start using them. + */ + for (int j =3D 0; j < PAGE_SIZE / sizeof(struct page); j++) + init_compound_tail(p + j, NULL, order, zone); + } + } +#endif =20 padata_do_multithreaded(&job); } diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 133b46dfb09f..c713c0d2593a 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -870,27 +870,10 @@ static const struct ctl_table hugetlb_vmemmap_sysctls= [] =3D { static int __init hugetlb_vmemmap_init(void) { const struct hstate *h; - struct zone *zone; =20 /* HUGETLB_VMEMMAP_RESERVE_SIZE should cover all used struct pages */ BUILD_BUG_ON(__NR_USED_SUBPAGE > HUGETLB_VMEMMAP_RESERVE_PAGES); =20 - for_each_zone(zone) { - for (int i =3D 0; i < NR_VMEMMAP_TAILS; i++) { - struct page *tail, *p; - unsigned int order; - - tail =3D zone->vmemmap_tails[i]; - if (!tail) - continue; - - order =3D i + VMEMMAP_TAIL_MIN_ORDER; - p =3D page_to_virt(tail); - for (int j =3D 0; j < PAGE_SIZE / sizeof(struct page); j++) - init_compound_tail(p + j, NULL, order, zone); - } - } - for_each_hstate(h) { if (hugetlb_vmemmap_optimizable(h)) { register_sysctl_init("vm", hugetlb_vmemmap_sysctls); diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 112ccf9c71ca..8f41b73fb674 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -342,7 +342,7 @@ static __meminit struct page *vmemmap_get_tail(unsigned= int order, struct zone * * * Any initialization done here will be overwritten by memmap_init(). * - * hugetlb_vmemmap_init() will take care of initialization after + * gather_bootmem_prealloc() will take care of initialization after * memmap_init(). */ =20 --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B3623D75B4 for ; Tue, 2 Jun 2026 10:11:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395061; cv=none; b=R/gjWsViqEW6pAeZpNPsufWoX9Z3h8/zXG8SiII4ZxTQfBiIOUqVUIlUvAEi65HbuoJd7BXZs00zvVhmy6/rY5575ki8xqUCgV3B/o+RgKkTHW+bNKX9owRa3mt8xjcT6PlUP3mF6g5fpWCS9kI8hEseh+d0ONVbYOAA8wbybnA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395061; c=relaxed/simple; bh=tddPdcrngiq1JnO7LgMJTwgJeTH6gPwckpm3fy7FjvM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CpgcFNJNZP8R5sU4bJ5+ZlVcRSmiJicZwdWxvdrlmEUklQ8rv4KQh+BcP7DlgJcHoNYokutdfU6YKNrQ2I+l9u9zQUObQNPtlwpcGEp7iWDL3yJ4eaucnbUCa6OZo8JfkrieEJKHvwh0IR8cZ5C7AASMVX7LkP083MrWstqd+FE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=jHPcvSMr; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="jHPcvSMr" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-8423f869421so1673924b3a.3 for ; Tue, 02 Jun 2026 03:11:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395060; x=1780999860; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aE8gFeezLetNlZxC9KnzUv4webDUXLM7puFdMIz+EH0=; b=jHPcvSMr8ZsMMYkh1utr9XgVHFYQudBwyK301qRWtoXdacWHM89e9DKUHIMF4N3s20 zv1jiThHUJRXfy8zsYNizYV3/L092glPf9FsdxrRpnL19AHl5YYsSvLHGOGRBLg8bSkQ xlFbO46BckkyWYHgKJCFmuQ5u+YZ3p/u5nt/XTIYAjGdc5ToqYZh3SoQSrYxN7Q9tu6u 0rbc2tGmqzNDDulgt8ra3uFaIKtop95PA85gezp7xtW9HYoL6oJS148hTt6EkXu0sxQ5 DNpqgPEs7IEDNU+nPNPitJJTfmdNXw4aDRbrseeKk940DsWCboFWwwyuw35gnq8XJYWp EMbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395060; x=1780999860; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=aE8gFeezLetNlZxC9KnzUv4webDUXLM7puFdMIz+EH0=; b=mMFfwFkxMMIrq6fx+JobU7HxcVCuxSMM2A6XSEzjzjSVkUmFsGxkaub2c2Fr49qmMB 3370oNYSdUyni7Crt5pTqe0uWQ4igkmNL3+JkgPj93CZGllWYiWiVM8hXKQD3U/S68yt qBKVVI1J7PCQ5VRLLSxpXzBMaGQhuSLIiqme+9UOfNXyN9pw5KbJW8MUD0SAt+wbKIbD 2OtNV9rIxngi2O01Cvs0oQMXIYU4f3SUL1Y5rBg9mRadF8zQ2SAcQVvW/uXBQFZBp7hY VYXap+19MqF8loVKNmf5UDLc8AbbR5PHeouoYADhuPna5Kb5UNQ+06Pp9AC1VjmBDPiR /YVg== X-Forwarded-Encrypted: i=1; AFNElJ8aWZIJDcfU3y7RYrl4N/10f0wb2aJ9DoBo08DBi9JxVUyT49bckgnCuDUez7kK/P0Zh8lw702CGSn57dc=@vger.kernel.org X-Gm-Message-State: AOJu0Yw6IXR0Z96xQiEDKO1WkPKNrbMCMYjve4XCb54OmMvChosiiTl0 VO/a/ILEX2U7UtBlY/WFL+3q2/DWrfbhmDKSoZSA72WcLul/iPFHFJz5W0IiLwIY4AA= X-Gm-Gg: Acq92OGggRfChmVVbudx6yqOgdJ/Y71PGsefjhFKCGcEx65xfhzeLmB5uHFSz03bsK9 6yg4L9kxb/YkGjx0nZ4a6vJ1Ezl3qmEI6QLBp0PkO6XRAAFgtfGDoNju7zUEYqRdb42FdBB1T6r faHE8y4nTxTnsgZXBAyc7aruJUBc/Hc0KdHmE1B1mQUrvC1fQ/pcTIXF8X37W20JyNSrxlnt+lW KgbXhKDEwq1wt76VJbQZg+mP7+Lg/ZgI3CJr2UHF9QssdO+NOWTcHshnIUVFceIJhDhUThg8JH5 k3plosZRau/wwyFwPp4ce5F0vdXUrcMbUIgPY3rtFuOzOQlEbS4VtVSp9l3SXFtVS9VtIbKIGld 51QiNL4wN4m/HvITFxPK9pwY7bwn68zM+M/dUYP9ObepYuIu3EMMnNAO/81KeIDAUQ0hZDOgiqa GM9OynvieFxD7JrnPXkagtFik5I+EhNMDnSZL5Eq5axlLuXa3dKa5AijU= X-Received: by 2002:a05:6a00:1ad4:b0:842:5d9b:d590 with SMTP id d2e1a72fcca58-8425d9bd996mr6238843b3a.30.1780395059676; Tue, 02 Jun 2026 03:10:59 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.10.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:10:59 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 02/19] mm/hugetlb_vmemmap: Fix __hugetlb_vmemmap_optimize_folios() Date: Tue, 2 Jun 2026 18:10:22 +0800 Message-ID: <20260602101039.1867613-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" __hugetlb_vmemmap_optimize_folios() uses incorrect arguments when handling bootmem HugeTLB folios. The section number passed to register_page_bootmem_memmap() is derived from the vmemmap virtual address of folio->page instead of the folio PFN, so the bootmem memmap metadata can be registered against the wrong section. The helper is also given HUGETLB_VMEMMAP_RESERVE_SIZE even though it expects a page count, not a size in bytes. In addition, the write-protect range is based on pages_per_huge_page(h), which does not cover the full HugeTLB vmemmap area and can leave part of the shared tail vmemmap mapping writable. Fix the section lookup to use folio_pfn(folio), use HUGETLB_VMEMMAP_RESERVE_PAGES when registering the reserved memmap pages, a= nd use hugetlb_vmemmap_size(h) for the write-protect range. Fixes: 752fe17af693 ("mm/hugetlb: add pre-HVO framework") Signed-off-by: Muchun Song Acked-by: Oscar Salvador --- v2->v3: - collect Acked-by from Oscar Salvador --- mm/hugetlb_vmemmap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index c713c0d2593a..ea6af85bfec1 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -635,12 +635,12 @@ static void __hugetlb_vmemmap_optimize_folios(struct = hstate *h, * mirrored tail page structs RO. */ spfn =3D (unsigned long)&folio->page; - epfn =3D spfn + pages_per_huge_page(h); + epfn =3D spfn + hugetlb_vmemmap_size(h); vmemmap_wrprotect_hvo(spfn, epfn, folio_nid(folio), HUGETLB_VMEMMAP_RESERVE_SIZE); - register_page_bootmem_memmap(pfn_to_section_nr(spfn), + register_page_bootmem_memmap(pfn_to_section_nr(folio_pfn(folio)), &folio->page, - HUGETLB_VMEMMAP_RESERVE_SIZE); + HUGETLB_VMEMMAP_RESERVE_PAGES); continue; } =20 --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C14353D7D63 for ; Tue, 2 Jun 2026 10:11:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395067; cv=none; b=T2Hpo9rIbd8CdWAf1sVcGfUIsVoJnB2hODwoJr4cWgRc9Oxapx3nNlpUBvLBZLijnZMQWqO4ylw7nNYv+ZwNZsiRIPcd3eoNK45klGcqVf1GUOszPXKLGpeBpqACfruhdj2pu+gLOcSA0pQ8N+2SfavQj2E90Ibqf7VeCA/LGxY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395067; c=relaxed/simple; bh=s6Iz7RUTIQy+cRaqvGFmzxOTy37x/2FAJ2Uw8UZqIvI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iO+tsVI9snioaOhmBjGnyLJWF3RKTRk0xsGl1Ln8aG/IARMlxGbZGMOfR888280C13/uGtOpHQ1IjCfiVcbECFpkhffwcYafV4IEvKgqqh3RrD1oltZ3obk+NsgpFcN2OUiljOhmPohnNHEslN5becGmLYsYPZBzyuQkIPbrzLs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=Z9kFn5yB; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="Z9kFn5yB" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-842358aaf36so900574b3a.2 for ; Tue, 02 Jun 2026 03:11:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395065; x=1780999865; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y0lfog+XMb7vxf8z5NRAN8W4fHvuXnUL6M7vrh2PtXs=; b=Z9kFn5yBQLCS8PCbh/9mzFftm/4VVCLg1B3+WytNvLKKP5nF3/SHUKM+77OUUUGPbK DD8WmhuVuo+vELPSbSepT/PHI0OORFk6wA1nqkt6MWBENsba5Fk6kC36fV11o1dsaQMP 4cQn13Iks+N1zbOWjzOlCmlOp8Djv/950exfH+O+uLCnHb/4WuSPZs3yz2keicFx/4n6 h7m6Ra3y1BgdhmnVEJSns8/vEXrYXEoftTULUuJzpIh5aXQuLH5cQZ9No6zyHc7AjDSR 1P9eAhulYTZ9GA6iFHMaNRt1fk2kioV3sY1xKWCqMFEI7QcZTP+DeqHxNN+PFuqjLG/W r7jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395065; x=1780999865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Y0lfog+XMb7vxf8z5NRAN8W4fHvuXnUL6M7vrh2PtXs=; b=I8wPNdA2IZ9s4EoWQGgsjTN5kXKXkHS5n373HoHjYAT63bGNrOOIpGcE2xatp16bJz VrwQXFrRogYvHuzsW2gWcVhiw5vwNVPViTyMwF2ZJg4Ef7fWQTXK+unrSqdDp2zm4l2o SwdGek/gwRTV9SLc8aMLXjyfR4FyVVdu378stTuOyvl1csfve5ZUjltC6o+tq4EuAetL gPVWPgK+BS4WSDod3hCKf9TdLC1Xj8nZb0CddFZ9M276V6A+S3xEvtSuYPVrDaorc0Uf aV1Hr/uOZoQaGezLHQepeFrAvl9YFuRqUL7wMsE45D5jUgr2xFDdZKANY3akQCk9G5n2 V5NQ== X-Forwarded-Encrypted: i=1; AFNElJ9bMryjRypUj6LpHq51hmadeX1YBvoYzoibo0Hk28V+IDdHY6s2T0buXtN5wjC/EW+YPVzb0y0uvwd6kgo=@vger.kernel.org X-Gm-Message-State: AOJu0YxQMFqVPb4+TQ/MoNm0tBZ+NBuc+Zth2W+dhgFAEUTxGPFLpsAw 4jaIX1qgx2hlHqykSjry8BgmAQWiWgblIkbL8IZIPYAjviAKK3Mhv4TSJKr3p7v3UGc= X-Gm-Gg: Acq92OHfUr+wp7TVgbbD0gdj4OBNRGVyB6VUWLXwQD1ozGUOaMBwheAP0mSZkXOLJIm 3/i+NZPWZExrSQLHxnYs/7Mic1Q381GDmubIRb72fWdFK/5QK3GHPkxDzIS78Jzpqm9gjTZM1Rd PTARAj5zoAx9v5O2r00ft4XrvAGByhzBdVK4F5oyP6GEaDrKw9Cc00H2VZttR3W9cPWcIf+t7ag JKNYaDPxVsqpSK9pEEueIYb9CMobGaEyfzrpTov7Q3BwmZL7D06IwGM1Jyy/0t/phHeT+xuTY3h RBzu7+Cv9/hjVfNqzAxarBGYE1xnXs8XOV5WJmfnFNqufq0/TE0WBg5gYPaE/PirH1HkFIXYxj3 Q2QKaM5x0reP7g1KS4+Ss7v7O2oSb8PEjC0DOe7q4jKHpYPBfFGgxr1wX2z2RKUGQLcPdcY6aG9 5ZygPeBRNx98LERZqjFPootoEYpdwlQ2KBB3Lacj59Kp/J/igHenQgtd8= X-Received: by 2002:a05:6a00:1a86:b0:83a:7565:3505 with SMTP id d2e1a72fcca58-842253d0cb1mr14807615b3a.8.1780395064752; Tue, 02 Jun 2026 03:11:04 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:04 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 03/19] powerpc/mm: Fix wrong addr_pfn tracking in compound vmemmap population Date: Tue, 2 Jun 2026 18:10:23 +0800 Message-ID: <20260602101039.1867613-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" vmemmap_populate_compound_pages() uses addr_pfn to determine the PFN offset within a compound page and to decide whether the current vmemmap slot should be populated as a head page mapping or should reuse a tail page mapping. However, addr_pfn is advanced manually in parallel with addr. The loop itself progresses in vmemmap address space, so each PAGE_SIZE step in addr covers PAGE_SIZE / sizeof(struct page) struct page slots. Since addr_pfn is compared against nr_pages in data-PFN units, it should advance by the same number of PFNs. The existing manual increments do not match that and therefore do not reliably track the PFN corresponding to the current addr. As a result, pfn_offset can be computed from the wrong PFN and the code can make the head/tail decision for the wrong compound-page position. Fix this by deriving addr_pfn directly from the current vmemmap address instead of carrying it as loop state. Fixes: f2b79c0d7968 ("powerpc/book3s64/radix: add support for vmemmap optim= ization for radix") Signed-off-by: Muchun Song Acked-by: Oscar Salvador Reviewed-by: Ritesh Harjani (IBM) --- v2->v3: - collect Acked-by from Oscar Salvador --- arch/powerpc/mm/book3s64/radix_pgtable.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/boo= k3s64/radix_pgtable.c index 10aced261cff..cf692b2b5f7b 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -1314,7 +1314,6 @@ int __meminit vmemmap_populate_compound_pages(unsigne= d long start_pfn, * covering out both edges. */ unsigned long addr; - unsigned long addr_pfn =3D start_pfn; unsigned long next; pgd_t *pgd; p4d_t *p4d; @@ -1335,7 +1334,6 @@ int __meminit vmemmap_populate_compound_pages(unsigne= d long start_pfn, =20 if (pmd_leaf(READ_ONCE(*pmd))) { /* existing huge mapping. Skip the range */ - addr_pfn +=3D (PMD_SIZE >> PAGE_SHIFT); next =3D pmd_addr_end(addr, end); continue; } @@ -1348,11 +1346,11 @@ int __meminit vmemmap_populate_compound_pages(unsig= ned long start_pfn, * page whose VMEMMAP_RESERVE_NR pages were mapped and * this request fall in those pages. */ - addr_pfn +=3D 1; next =3D addr + PAGE_SIZE; continue; } else { unsigned long nr_pages =3D pgmap_vmemmap_nr(pgmap); + unsigned long addr_pfn =3D page_to_pfn((struct page *)addr); unsigned long pfn_offset =3D addr_pfn - ALIGN_DOWN(addr_pfn, nr_pages); pte_t *tail_page_pte; =20 @@ -1376,7 +1374,6 @@ int __meminit vmemmap_populate_compound_pages(unsigne= d long start_pfn, if (!pte) return -ENOMEM; =20 - addr_pfn +=3D 2; next =3D addr + 2 * PAGE_SIZE; continue; } @@ -1392,7 +1389,6 @@ int __meminit vmemmap_populate_compound_pages(unsigne= d long start_pfn, return -ENOMEM; vmemmap_verify(pte, node, addr, addr + PAGE_SIZE); =20 - addr_pfn +=3D 1; next =3D addr + PAGE_SIZE; continue; } @@ -1402,7 +1398,6 @@ int __meminit vmemmap_populate_compound_pages(unsigne= d long start_pfn, return -ENOMEM; vmemmap_verify(pte, node, addr, addr + PAGE_SIZE); =20 - addr_pfn +=3D 1; next =3D addr + PAGE_SIZE; continue; } --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A40903D7A10 for ; Tue, 2 Jun 2026 10:11:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395073; cv=none; b=dwCKTgsQRcqmouCWrCeDn6ay+TBzqbYetfUtiaCW8bicdkh/p2fRZQPnHE9FfRmWlBj9sKvCLD88dHPRMFmbYoop8pkziWImkWz+kuYsvLLymsrlzQRQySbhPGtJ/CemLIGQFtGDgtRfOOp1tutluYAwFp6AQb8DNamOCMSfngs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395073; c=relaxed/simple; bh=/qq1FBjZdQOD03rmTq1CAAJ+RdRSe1d934emQevskYM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vt/oW/JCrPsO0fosS4U2a5/Rw7y7uwSh4YxteXhulW/Qmis6y6AKO4oT5l7zlcdDrD/mn3kmZ6cJp/yfkCIhyF8WWx+duKCDcyKA07DKjK+4k16SKMxGsHvV2TdCCoWs2Ca99qNw3nePgCuhskDu+TmMW2MRTpB7jfDFktrssEQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=bj2iZ177; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="bj2iZ177" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-8423f52af13so1536559b3a.2 for ; Tue, 02 Jun 2026 03:11:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395071; x=1780999871; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6ifAYajDzE3bfTkaM223T1RUSqkPH2sIJK1XHV+rXLE=; b=bj2iZ177xf5KO9owcjnIhCZg83P8rHfXDJIH0zQNIjYLJoHhyeyJHQMPNmDWRzTV2F 6U47t2j9B0xwXaqPanU0Ye1W9yMyOEAZCAs9i5LF53+U2G0XKaHfuql36SozvEqIlgxE S7ndJOTXB7ZqneTkGlYE6o9Z+xYQO37osLyhaNRa8Eoub7mF91Jp4Mkzg9oPk7pSbtvc YtW2NyS7PdrehyEnzOHvQjIFBNKtIwTR7/eGszDW18lDC+5XTBbGpIC73l1F4iPWJwIy oQXFTt/U7P4GFWCIUN22HZbYno2ktBRvXFhdU8fXKcNmGfLHlAWVOQLJYCFVu4Gkc6aE wZVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395071; x=1780999871; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=6ifAYajDzE3bfTkaM223T1RUSqkPH2sIJK1XHV+rXLE=; b=qLmvmIjXEqMAitMT0gDhoBca1/LhcotfbsUNihyCLs9Tb+32K22wyk8shVkUKt8hfk hODsUb5jl63LxqJWdDXpnHbL/XG+UY2rUHEkQbrATOZHw5lxlr5lEAgYwpFledHBclnH 1AI7l5wASMu8m5RDSBYxmW+kEvev+lSWjNLzEzIYQwh8dzv81JNnxRf6B31BpIWXwpM2 88Sv7od/oxazRFr/VY4Ck+o8Bp9THpRC0npykGnBrpXKmja1H2KnsJ367NpIjOJCCCNs juEAJs6PzMiwenz48d/2i2172h7QE/QoKbZ+xwIZfL1YCu/owL9SzviPBU7akgnEFTwF FqqQ== X-Forwarded-Encrypted: i=1; AFNElJ8MUp0/DsUakwMOLDQKc/GIR9+w+RUSKvx40GNcdt3gd5o67nxXWjvKGZhOCZZjtGDhYb/eKoQRY8zK/BE=@vger.kernel.org X-Gm-Message-State: AOJu0Yw9zJJQKBjIOQiHRkbpjgpPcFyyfkTyPuR/gw5eG0wsXoZqzsSW VhGc6dU3xpxSmf55kYS9H+a7Y41kUeYrjoMr9ZfUWsrXJ8ICHBNBDD47vSvMRKy8q/8= X-Gm-Gg: Acq92OEqnzViLt47roVCLUOhWadfnX+/pvmBcCfA50QYYrTpdpTSyx900c0a0aLh6Pr Ijx/g/+rlO3WSKVNnh/a5Dt5ZAhCoKdjc/6rDyRYultv05YhScxkSEpeEFG2wdwkRs/CYFuKPKO Wb8VeoVmJTpIUAxg7aacOb+qVGAPQUqyamVDQpugXLGsOtcVKqnRXNt4NcisUWaMyMWWg5cwN1W /ZQuv2XYVpZYwvyEv7KTNoo0SBOxSwdBXxrrK0I+PVAp1qexl2R+McmD+Vf1uTCn53xXnLV3Etm FbGpLAC5SHG+4z6X7Qq/YLbAs6AOdNKYOBu5XXd6w9Cu85RQu5Smi+e7WwR+pybR5N/O0fuOXb6 j3wUyQW7NgdkJdiSo9VNVngSeXQxWPbyKhdJHwORx/zmXDz80/u8uvX+erfCMQsM8VXidhWgu+b /TN1H99lv+fSaO9l0uiF0cKPY7VnE9m5G06PIHejbAWzHq9fdqTCXZlIs= X-Received: by 2002:a05:6a00:3c91:b0:842:1052:5fe7 with SMTP id d2e1a72fcca58-8422532c951mr13639719b3a.6.1780395070652; Tue, 02 Jun 2026 03:11:10 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:10 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 04/19] mm/hugetlb: Initialize gigantic bootmem hugepage struct pages earlier Date: Tue, 2 Jun 2026 18:10:24 +0800 Message-ID: <20260602101039.1867613-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Gigantic bootmem HugeTLB pages are currently initialized from hugetlb_init(= ), but page_alloc_init_late() runs earlier and walks pageblocks to determine zone contiguity. If a bootmem HugeTLB region is marked noinit, set_zone_contiguous() can observe still-uninitialized struct pages through __pageblock_pfn_to_page(). This may not trigger an immediate failure, but it can make set_zone_contiguous() compute the wrong zone contiguity state. If extra poisoned-page checks are added in this path, such as PF_POISONED_CHECK() in page_zone_id(), it can also trigger an early boot panic. Initialize gigantic bootmem HugeTLB struct pages from page_alloc_init_late(= ), before zone contiguity is evaluated, so later page allocator setup only sees valid struct page state. This also makes the initialization order more natural, as struct pages should be initialized before later code inspects them. Fixes: fde1c4ecf916 ("mm: hugetlb: skip initialization of gigantic tail str= uct pages if freed by HVO") Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) Acked-by: Oscar Salvador --- v2->v3: - rename the helper to hugetlb_bootmem_struct_page_init() to make the bootmem-only scope explicit (per Oscar Salvador) --- include/linux/hugetlb.h | 5 +++++ mm/hugetlb.c | 5 ++--- mm/mm_init.c | 1 + mm/sparse-vmemmap.c | 4 ++-- 4 files changed, 10 insertions(+), 5 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 2abaf99321e9..3700c0a1f6ff 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -171,6 +171,7 @@ extern int movable_gigantic_pages __read_mostly; extern int sysctl_hugetlb_shm_group __read_mostly; extern struct list_head huge_boot_pages[MAX_NUMNODES]; =20 +void hugetlb_bootmem_struct_page_init(void); void hugetlb_bootmem_alloc(void); extern nodemask_t hugetlb_bootmem_nodes; void hugetlb_bootmem_set_nodes(void); @@ -1293,6 +1294,10 @@ static inline bool hugetlbfs_pagecache_present( static inline void hugetlb_bootmem_alloc(void) { } + +static inline void hugetlb_bootmem_struct_page_init(void) +{ +} #endif /* CONFIG_HUGETLB_PAGE */ =20 static inline spinlock_t *huge_pte_lock(struct hstate *h, diff --git a/mm/hugetlb.c b/mm/hugetlb.c index cd55524c7e30..2bf9fe16abb9 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3353,7 +3353,7 @@ static void __init gather_bootmem_prealloc_parallel(u= nsigned long start, gather_bootmem_prealloc_node(nid); } =20 -static void __init gather_bootmem_prealloc(void) +void __init hugetlb_bootmem_struct_page_init(void) { struct padata_mt_job job =3D { .thread_fn =3D gather_bootmem_prealloc_parallel, @@ -3582,7 +3582,7 @@ static unsigned long __init hugetlb_pages_alloc_boot(= struct hstate *h) * - For gigantic pages, this is called early in the boot process and * pages are allocated from memblock allocated or something similar. * Gigantic pages are actually added to pools later with the routine - * gather_bootmem_prealloc. + * hugetlb_bootmem_struct_page_init. * - For non-gigantic pages, this is called later in the boot process after * all of mm is up and functional. Pages are allocated from buddy and * then added to hugetlb pools. @@ -4152,7 +4152,6 @@ static int __init hugetlb_init(void) } =20 hugetlb_init_hstates(); - gather_bootmem_prealloc(); report_hugepages(); =20 hugetlb_sysfs_init(); diff --git a/mm/mm_init.c b/mm/mm_init.c index 6de3a77eb9ae..1890bda948b8 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -2338,6 +2338,7 @@ void __init page_alloc_init_late(void) /* Reinit limits that are based on free pages after the kernel is up */ files_maxfiles_init(); #endif + hugetlb_bootmem_struct_page_init(); =20 /* Accounting of total+free memory is stable at this point. */ mem_init_print_info(); diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 8f41b73fb674..db9cfe57e827 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -342,8 +342,8 @@ static __meminit struct page *vmemmap_get_tail(unsigned= int order, struct zone * * * Any initialization done here will be overwritten by memmap_init(). * - * gather_bootmem_prealloc() will take care of initialization after - * memmap_init(). + * hugetlb_bootmem_struct_page_init() will take care of initialization + * after memmap_init(). */ =20 p =3D vmemmap_alloc_block_zero(PAGE_SIZE, node); --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDBF93E274A for ; Tue, 2 Jun 2026 10:11:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395078; cv=none; b=bamh3iSv9catwFpjXHhgkzKphcGzHjGnEjtyHOBrMiiOe5g39vouuXuLSvVd73Q1h2j1XImLjBD17gedSn4nBUIWci/jCA6BNENbrvuWZMbonNbakPQKkvtiCkGaEAwtsN7AYANLb+pbfDkBOC2Y7TbgDkuSTS907jQTk8cYh/c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395078; c=relaxed/simple; bh=Nu4rAOHahg8gdCQTidyLhz9TeGPi9mI96jP01x6OJvY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lCIge+RGxTr709ulNwq9I1Gm2gB/uS1rWsm6+MX5SgrdHprj25WhtCmBcazt63/Igxo8d2Z7Ly232qCXhkheIaMoBOnM/6A5dc9UmT0LhoXk1hJBU2kFghSRqwfq4CmMygMurZjYTmRSnCCeu4igfpL108T4ZnNApX7BZUJc3Kc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=N+Fh5U1l; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="N+Fh5U1l" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-8423efd76c8so1439256b3a.0 for ; Tue, 02 Jun 2026 03:11:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395076; x=1780999876; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Btf8iCCXb7pZCMHTCvc15KHxJTAr77nR7rRKFiPgBRM=; b=N+Fh5U1lJiy5A4anXFqwxzpey7d6jBKaps94ritnwGG4kI/ivpJqGpQIkZxMbAEZ/c am+rBCpPYamf1woGoXycezV+AnylDj9C40kSuEa6faebxA+Fq9AdpA+MyhjeXBHpqWll dxmtPtRPcMshI0CKWc4mvwH42UzZ9rzTSUXMc5aLAwsSM4KXf/E31Ogtc5fOihtNea+S /e4jorODy2UHwl5MvvKVc8szMnjLcwmsdsmmDdH2oLovkSmmvyPO2B6D0bGGACs/skM7 HVMwakRwdhNr1sd44hHbSK2AF2mtNJib3LhrYABuO5RPDZBjPv5QbZTDzEAtJ8Vsem0Z 7sFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395076; x=1780999876; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Btf8iCCXb7pZCMHTCvc15KHxJTAr77nR7rRKFiPgBRM=; b=Yb4f27109CgXur2xdpxfJt6pwEy3d7P7bdrw+xaDm4sKStFjH4+wniZb27eqsqYc6x JoMAA7o/RYl4tztrqFTskFV3wpL/7690BA25952Uw5oP/SD9C+RaEwfDFxcKxOZAmHq5 WIE7lw/VWSDNCjZNIa7a4hF7UWXzq5zckHArQPPCdwR6LpVw/iYHNSfqlwadkYYoweAZ 5BG4iPCgKR1nFpeB7N7RN94O1VRNHD4pG6tf0NTv5d8DB93Eeu04rpn0AW+Ru0njprG7 ASw58kPvoQsn1RqPh+W/H/ZA7VDOMPjgGdG7mk/oP6+oYjhvO/ItVtpp0kXYxyg/OSB4 Q+Tg== X-Forwarded-Encrypted: i=1; AFNElJ8FLJepnJnC+kcbB5Ksn6QcwEhLMNc/kiYy7JgyKADIiKiuJJBHLNNOHb0QgCa9fnV6pZRphlZgne7vCzU=@vger.kernel.org X-Gm-Message-State: AOJu0YygGictPk3+IOmpu2cIeNdXaF5EkevYbvfiNJwJvjuzG+hvsHqQ 6S0407yn7u6Oqx9kEtc7fTlW26OgiHHEk6ry472JwVD9IP58kTqXxyY6J9P+gRsph7Q= X-Gm-Gg: Acq92OGIYAYfJyzLEO//0KQnjJMJ4tbPr9xYlWRgisOZchSAq3lF3MZOpeTLWhOsnvf +eB7JEt/OXT1FF4/o/5Sy0hmuxKiCKRzuPf1Uhos1HIMOjdHKdiCh53006oneaFqCeVCZ3UTla4 NufZ2+pa/0P9twRzE89/YohJZttVL+hEBiAlsX+3kFLtymedbBCbquCIrQAK/n0qVc5eJ1HV7x3 KDvYquMKCGnHa5VKh4wDovXF9rq00CX5hzvvuwur+UMwZEmtzgoyOIQT9chC08dMDMPb9CYC1Bv GOhth+RxqeoAB9uE7jaFoMLCGiA91jhoxDuwlUg46ABqBvLks2zt+vYvD2v/SUeXJKo0CGO+hKL Vh/xYO5BNAxCcYrDL4y8z/51MKxhGSZ1w1KIkyimwr7Ke5MlghzKCrqbBLC1ARF6/QEL7QCcQnD GEMRFxw5v2owUc4evmUK+ci+e33b8U36ALRf2m/4k7EuCq0//JVJ+w7Go= X-Received: by 2002:a05:6a00:3e0f:b0:842:4f49:71bc with SMTP id d2e1a72fcca58-8424f4972femr8017729b3a.46.1780395075647; Tue, 02 Jun 2026 03:11:15 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:15 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 05/19] mm/mm_init: Simplify deferred_free_pages() migratetype init Date: Tue, 2 Jun 2026 18:10:25 +0800 Message-ID: <20260602101039.1867613-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" deferred_free_pages() open-codes two loops to initialize the pageblock migratetype for a range of pages. Replace them with pageblock_migratetype_init_range() to remove the duplication and make the code clearer (Note that deferred_free_pages() may be called from atomic context). Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) Acked-by: Oscar Salvador --- v2->v3: - collect Acked-by from Oscar Salvador --- mm/mm_init.c | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-) diff --git a/mm/mm_init.c b/mm/mm_init.c index 1890bda948b8..be652b6990a2 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -674,15 +674,15 @@ static inline void fixup_hashdist(void) static inline void fixup_hashdist(void) {} #endif /* CONFIG_NUMA */ =20 -#ifdef CONFIG_ZONE_DEVICE +#if defined(CONFIG_ZONE_DEVICE) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INI= T) static __meminit void pageblock_migratetype_init_range(unsigned long pfn, - unsigned long nr_pages, int migratetype) + unsigned long nr_pages, int migratetype, bool atomic) { const unsigned long end =3D pfn + nr_pages; =20 for (pfn =3D pageblock_align(pfn); pfn < end; pfn +=3D pageblock_nr_pages= ) { init_pageblock_migratetype(pfn_to_page(pfn), migratetype, false); - if (IS_ALIGNED(pfn, PAGES_PER_SECTION)) + if (!atomic && IS_ALIGNED(pfn, PAGES_PER_SECTION)) cond_resched(); } } @@ -1142,7 +1142,7 @@ void __ref memmap_init_zone_device(struct zone *zone, compound_nr_pages(pfn, altmap, pgmap)); } =20 - pageblock_migratetype_init_range(start_pfn, nr_pages, MIGRATE_MOVABLE); + pageblock_migratetype_init_range(start_pfn, nr_pages, MIGRATE_MOVABLE, fa= lse); =20 pr_debug("%s initialised %lu pages in %ums\n", __func__, nr_pages, jiffies_to_msecs(jiffies - start)); @@ -1996,12 +1996,12 @@ static void __init deferred_free_pages(unsigned lon= g pfn, if (!nr_pages) return; =20 + pageblock_migratetype_init_range(pfn, nr_pages, mt, true); + page =3D pfn_to_page(pfn); =20 /* Free a large naturally-aligned chunk if possible */ if (nr_pages =3D=3D MAX_ORDER_NR_PAGES && IS_MAX_ORDER_ALIGNED(pfn)) { - for (i =3D 0; i < nr_pages; i +=3D pageblock_nr_pages) - init_pageblock_migratetype(page + i, mt, false); __free_pages_core(page, MAX_PAGE_ORDER, MEMINIT_EARLY); return; } @@ -2009,11 +2009,8 @@ static void __init deferred_free_pages(unsigned long= pfn, /* Accept chunks smaller than MAX_PAGE_ORDER upfront */ accept_memory(PFN_PHYS(pfn), nr_pages * PAGE_SIZE); =20 - for (i =3D 0; i < nr_pages; i++, page++, pfn++) { - if (pageblock_aligned(pfn)) - init_pageblock_migratetype(page, mt, false); - __free_pages_core(page, 0, MEMINIT_EARLY); - } + for (i =3D 0; i < nr_pages; i++) + __free_pages_core(page + i, 0, MEMINIT_EARLY); } =20 /* Completion tracking for deferred_init_memmap() threads */ --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBE483D7D9E for ; Tue, 2 Jun 2026 10:11:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395084; cv=none; b=PceCghiCOohuE0nPxKsZLlJDeoB/CQV2zv5SJzHX9C7P6eY/5NFHAtupB+y4Nsg99MbN7yKtgTArVDU/1B3CCVMHo/7roLihpWQE0aNX/ADPRP+L45PWVt1gy+FQNuKfYa8ZCqXj8m1V3pGrVBKRKlOMyyOzOOKT9yNgBL7TzOo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395084; c=relaxed/simple; bh=HQzneXZzThZINJZfPM7MoNkfiWwwG0PyooFZF2tS/vY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mhdH96dV540x+Itz5DNNX7pKhEncTyokhm/0osh6UKLqby83gb+WvGRuiZ2HKiq9VZX4MCuSrzSinnjDWlVBbaKBTPF5JhyUskpo2r78EPeDYse5scorWrRT1s4RWnT1ZqT8ethU2PL0FDp0R/JzDlRoRKUC0GvqoY1JYSuA0qw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=EvLBbeTH; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="EvLBbeTH" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-8423f869421so1674149b3a.3 for ; Tue, 02 Jun 2026 03:11:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395082; x=1780999882; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=m1Kna2jMMM5bJ6rAGniT5H8QaKGxjLzFWpw04bCbO68=; b=EvLBbeTHbe41cp1c6EaXdfSdrKp7G1BNi3+f/p1ztRTFK52Il8PB/oguHZufb3FMB8 kHPOOeuUErVTmheB5TQH15WrnhIN4s0O3DtHpGdgTZrEkB+GDp2W6olSw3ZVipxRgCRQ TN6fJuuyIaPRbBsdbL1ny3fcdAxPT8tehJX0U3jz4zmcUMPYuLypg73gcKX+ixhSARGY FeXOcmaVQmJHoaoVib5BBnij9jVFDDAEdBPHpES47S7oUTsMk3JlXxDmVqD7ZTC/Ydoy YauLNclViugRmgiALj7Dq9rAbw19S5/N4fFMysntLOsTCUFl98T38eu11VjkP771723k EQ8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395082; x=1780999882; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=m1Kna2jMMM5bJ6rAGniT5H8QaKGxjLzFWpw04bCbO68=; b=Y/inH15TfspA64LFq+49mddHy7L60gkL4ez2mHkEjTsR6BITx9e7jTwI7GEjOwW5eA GfV0GtlDR5nJhR1N+T8u0PEUx7V21yLEqulJfwS28ovAN+i5yFS+qEh2C2+cUqMCLCSX PXzA54ZFb/2MXZ6oYkNlcCeUHKlEBA2fp9iwDS22gca0/pziM44EnD3AQHD8sg50/dyy E+yvieEq/uJBW0i7ePMa0WLdOt10MjaKXat6Etn3MtcIkGhsP8Yd1crSjCe/T0O3+vNT hYOHtGEJcnMvXjzC6s+uwbsXu2kwKU/N/52fGdm41uAzkOkLDyzcVf/qTWdJ71ODBMkT +bEg== X-Forwarded-Encrypted: i=1; AFNElJ/PadN2F8Y0V6hHMpZ0+0vYyhrND6GVfXODr/2L2LuAe1iaX9M7NlqD2I0G/3uD23Nona+egU6hgWBRpGQ=@vger.kernel.org X-Gm-Message-State: AOJu0YycRFv1r9TFUFJsMS4fcEc27/H3qfchqFKHHsUeX6pflLhrn10U e07OGnFoCHhYkbiUZhEj/Nb0MOJQ6eH2UdPPv0lvvURd+Ouc8V8fUZs7my4/q8M89Yk= X-Gm-Gg: Acq92OEij4Sru/FEIlF1eBhAgIX4fEfMKEj4DbvMLQLGkx1Ev21Sg2957kCclPwZu0X KgYTzZb2R9bglo6MO2uBkEwLnkdeZvGKC/YoGZKiP4BBMz2tejHIIqLeuEJaSjAo8lEEqbupzCS jeSLIdoCnFt97YpNZ9b66hsScJwZtInJFVNvKNh7w386FKGfG4FO7IlIq/6I06ydW5Lf4smVFzN +IpbOo5a5aNW+EnYAAPRnawdMjcQ6qhru4+VtvgOmaalvMerZFz67Z87GHgosjXV6iCfwxARa04 PtCONmKljZGnCh52mrTAVXJkkxi/htI4gg2HmLwRA4BR+meUoXWic+2yiI+drzujxL6A4nKIs36 7L1VXcYwamewkA4Uyfb/TdsDGbX2EY84Fdv2d1PlRP8sr+wyc0ilYNZ34B9Oz5fPHau1Hj31yLr 7+/pLUJodGlKfKSl1nv/NB3UqpAuE5gQZtlB43p/kq3JLVyVvxytTO6f8= X-Received: by 2002:a05:6a00:4087:b0:842:708f:39a0 with SMTP id d2e1a72fcca58-842708f49cemr2632750b3a.8.1780395081825; Tue, 02 Jun 2026 03:11:21 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:21 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 06/19] mm/sparse: Panic on memmap and usemap allocation failure Date: Tue, 2 Jun 2026 18:10:26 +0800 Message-ID: <20260602101039.1867613-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When vmemmap or usemap allocation fails, sparse_init_nid() currently marks the section non-present and continues. Later boot-time code can still walk PFNs in that section without checking for this partial setup, which leads to invalid accesses. subsection_map_init() can also touch an unallocated usemap. Auditing and fixing all early PFN walkers for this case is not worth the complexity. These allocation failures are expected to be fatal anyway, and other memory models already treat them that way. Make memmap and usemap allocation failures panic immediately instead of trying to recover and crashing later in less obvious ways. This is also consistent with how other memory model configurations handle memmap allocation failures. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) Acked-by: Oscar Salvador --- v2->v3: - collect Acked-by from Oscar Salvador --- mm/sparse.c | 44 +++++++++----------------------------------- 1 file changed, 9 insertions(+), 35 deletions(-) diff --git a/mm/sparse.c b/mm/sparse.c index 16ac6df3c89f..c92bbc3f3aa3 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -239,15 +239,8 @@ struct page __init *__populate_section_memmap(unsigned= long pfn, struct dev_pagemap *pgmap) { unsigned long size =3D section_map_size(); - struct page *map; - phys_addr_t addr =3D __pa(MAX_DMA_ADDRESS); =20 - map =3D memmap_alloc(size, size, addr, nid, false); - if (!map) - panic("%s: Failed to allocate %lu bytes align=3D0x%lx nid=3D%d from=3D%p= a\n", - __func__, size, PAGE_SIZE, nid, &addr); - - return map; + return memmap_alloc(size, size, __pa(MAX_DMA_ADDRESS), nid, false); } #endif /* !CONFIG_SPARSEMEM_VMEMMAP */ =20 @@ -300,17 +293,14 @@ static void __init sparse_init_nid(int nid, unsigned = long pnum_begin, unsigned long map_count) { unsigned long pnum; - struct page *map; - struct mem_section *ms; =20 - if (sparse_usage_init(nid, map_count)) { - pr_err("%s: node[%d] usemap allocation failed", __func__, nid); - goto failed; - } + if (sparse_usage_init(nid, map_count)) + panic("Failed to allocate usemap for node %d\n", nid); =20 sparse_vmemmap_init_nid_early(nid); =20 for_each_present_section_nr(pnum_begin, pnum) { + struct mem_section *ms; unsigned long pfn =3D section_nr_to_pfn(pnum); =20 if (pnum >=3D pnum_end) @@ -318,34 +308,18 @@ static void __init sparse_init_nid(int nid, unsigned = long pnum_begin, =20 ms =3D __nr_to_section(pnum); if (!preinited_vmemmap_section(ms)) { + struct page *map; + map =3D __populate_section_memmap(pfn, PAGES_PER_SECTION, - nid, NULL, NULL); - if (!map) { - pr_err("%s: node[%d] memory map backing failed. Some memory will not b= e available.", - __func__, nid); - pnum_begin =3D pnum; - sparse_usage_fini(); - goto failed; - } + nid, NULL, NULL); + if (!map) + panic("Failed to allocate memmap for section %lu\n", pnum); memmap_boot_pages_add(DIV_ROUND_UP(PAGES_PER_SECTION * sizeof(struct pa= ge), PAGE_SIZE)); sparse_init_early_section(nid, map, pnum, 0); } } sparse_usage_fini(); - return; -failed: - /* - * We failed to allocate, mark all the following pnums as not present, - * except the ones already initialized earlier. - */ - for_each_present_section_nr(pnum_begin, pnum) { - if (pnum >=3D pnum_end) - break; - ms =3D __nr_to_section(pnum); - if (!preinited_vmemmap_section(ms)) - ms->section_mem_map =3D 0; - } } =20 /* --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9DF3C3D566F for ; Tue, 2 Jun 2026 10:11:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395090; cv=none; b=RoCb36YPHbz0GG8eRrklbvseX82WcU95N4qWiUXNj7C3nLeZE7zY/3rw/SWs6o1651zlvBvCS4Mt+OX/QpFsM8uiK6kaA+tOVGWWx2GNPwXGqJIHSyGl+J06FyAVrKjm6yi9/Ih3SznJBB4fUIff59b4zJXMqj+bgnhjxzI33So= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395090; c=relaxed/simple; bh=jbrNh9zzlKcYl0EdK1KtlQ3ZUXHT3LWDMPzD/sT6Upo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZU20gC5oFjUnSeIQfSs3KaEO8VmreqanRMUceegKlb7N7VkfWS5t2VSL/LIgzmlFTJojzUggMNaizWzI9Hm2mXZFwReBtXW2K8Z+VqBj3I/y6silth+IJn/DcaRxk8L2+R6FLYQ1BAUobrgYIGCRWg7VF42sfnes3H5jOENhyf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=dHvY9f1G; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="dHvY9f1G" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-84226d0f1d2so1968652b3a.1 for ; Tue, 02 Jun 2026 03:11:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395088; x=1780999888; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DnGxO1Udfe58Wuis4xgFsB98Q+Og1N+BThJGTGl24kM=; b=dHvY9f1G6meH/Inks6ImV1kF2ugDQ780MPwNRkB5rvwS3Ru3v7GemtTetbbkiNG+T8 dhtNxWM0A35LwZLauQzNuJFSaBKADP4v+kBV1hL25oksIzfgVFZle4Ads8yYfCYL2OQL 8g6ezOeqCeW6YRAsIHZBbWYAEhi7JC/EVCTHQSX7uSpPOG8ZC7X8tSR4EzrsNf9FxSqd NIT1Y0EEo+X2mTSZFX3Ih0FZb6qFJewkOOv452HfAQuZ3NsMM+UuGrTfO4Z02HfEQlxP cAD80Rlq0PFa3y+twOs38aCEJl932kvYpbcKZkntrLc+uvh5U3EHF1DoTOPOZXWu5FKc xhlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395088; x=1780999888; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=DnGxO1Udfe58Wuis4xgFsB98Q+Og1N+BThJGTGl24kM=; b=O/gNVYaqhXPRpmbPzSfp7IfNn5XrvGUuwwjDjaAxhxMFSnIOdlwW2ImQzfvlHfiqj1 dtbQMUYa9dtVPc8iRAoOZMK/dns05xwcQWi/+aPjeUY0LpsyKu3ZUfJmeeORVI8L1YkP DwFAGS24HrH7LdjrO5UMneVh701HIzceB95UfwMx0XAyMK8Qgn2LnTE0HvSVAUHnz9zU 7B6wBhPgwHCOo1I/a+4E7uL3VVavNp+/Wy6d0DjqHNjQgC2zLsaSKLX8Zrn92g1UDEA4 bh9qEqEQ3JOw0L6WkoqFKwT6ysjXre1AnS/bNkl/QED3ibsL5lbt1t29OM+ZUCF7TZzk KW2A== X-Forwarded-Encrypted: i=1; AFNElJ9F5u5I/2jb2SHhsqWEwdnK323xvBOXYKge4/yJ1dL9RZPW7aiDXgWt5y+eijCOz90/oQCWCzg2tWkz5xY=@vger.kernel.org X-Gm-Message-State: AOJu0YxVY14k9pUPR8gwyWMqQVOMhabkliWcjn8NdEpNbB6W7ZTMvpQT OkkP7kjkDI0WN0uHSLJLG2JPDbjLnfxUTXD6QqL9CO6jXkLBCx2YLugRIhbNciDGa+8= X-Gm-Gg: Acq92OG7oeGf72/PpbxmtJ1N8RuNkwe2jDzjVdKj0AILfYEq8rvW0WFnkSVZ3jOOjS1 1XStmCIYzvFRuabh7pUjf/qRNaLBU7IdBS0CRkzBXnm2cqJQK5dqBI4GVMR5yvJ5sIJnWW+DPd+ iRhARJ8MWG72pmOQ1dmA5iGorjidpORVhARB/u8B/tmVbl37OOzZK3ViQ/LmkIiUhhy6DKtVyf1 Fm+HYt7r4se+/zFTjfnu0hBzAoontS787xvqym0PRboLfGnY0gayBERNEWyV1Da7KmvqpPC0I5C 7qy//tcdI0EHTiZejevg7VmjgGz6Sw+hwG+j6tYCnMpehwscfZzimGDY+anE6UNJzLjGETi2JBX Q9dqjk5HztmUCxnEi4dHURJ8NUk0YKLVVwlS8HKg86cc4oxV8vfcRR8Ziim9IFMsNnpgmjOIVne 6Gv/RNnKRYB0nRWXsD40OvKPp4VuRApZt6Qif5fMqwjSTzX7fuJrhmZXyQAfNQBBrlpw== X-Received: by 2002:a05:6a00:348c:b0:842:5c60:513 with SMTP id d2e1a72fcca58-8425c600b94mr6788288b3a.30.1780395087753; Tue, 02 Jun 2026 03:11:27 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:27 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 07/19] mm/sparse: Move subsection_map_init() into sparse_init() Date: Tue, 2 Jun 2026 18:10:27 +0800 Message-ID: <20260602101039.1867613-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" subsection_map_init() is part of sparse memory initialization, but it is currently called from free_area_init(). Move it into sparse_init() so the sparse-specific setup stays together instead of being split across the generic free_area_init() path. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) Acked-by: Oscar Salvador --- v2->v3: - collect Acked-by from Oscar Salvador --- mm/internal.h | 5 ++--- mm/mm_init.c | 10 ++-------- mm/sparse-vmemmap.c | 11 ++++++++++- mm/sparse.c | 1 + 4 files changed, 15 insertions(+), 12 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 5602393054f3..e71ba519f7f2 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -994,10 +994,9 @@ static inline void sparse_init(void) {} * mm/sparse-vmemmap.c */ #ifdef CONFIG_SPARSEMEM_VMEMMAP -void sparse_init_subsection_map(unsigned long pfn, unsigned long nr_pages); +void sparse_init_subsection_map(void); #else -static inline void sparse_init_subsection_map(unsigned long pfn, - unsigned long nr_pages) +static inline void sparse_init_subsection_map(void) { } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ diff --git a/mm/mm_init.c b/mm/mm_init.c index be652b6990a2..3a57bf5a9b46 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1879,18 +1879,12 @@ static void __init free_area_init(void) (u64)zone_movable_pfn[i] << PAGE_SHIFT); } =20 - /* - * Print out the early node map, and initialize the - * subsection-map relative to active online memory ranges to - * enable future "sub-section" extensions of the memory map. - */ + /* Print out the early node map. */ pr_info("Early memory node ranges\n"); - for_each_mem_pfn_range(i, MAX_NUMNODES, &start_pfn, &end_pfn, &nid) { + for_each_mem_pfn_range(i, MAX_NUMNODES, &start_pfn, &end_pfn, &nid) pr_info(" node %3d: [mem %#018Lx-%#018Lx]\n", nid, (u64)start_pfn << PAGE_SHIFT, ((u64)end_pfn << PAGE_SHIFT) - 1); - sparse_init_subsection_map(start_pfn, end_pfn - start_pfn); - } =20 /* Initialise every node */ mminit_verify_pageflags_layout(); diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index db9cfe57e827..3b036251a2f4 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -596,7 +596,7 @@ static void subsection_mask_set(unsigned long *map, uns= igned long pfn, bitmap_set(map, idx, end - idx + 1); } =20 -void __init sparse_init_subsection_map(unsigned long pfn, unsigned long nr= _pages) +static void __init sparse_init_subsection_map_range(unsigned long pfn, uns= igned long nr_pages) { int end_sec_nr =3D pfn_to_section_nr(pfn + nr_pages - 1); unsigned long nr, start_sec_nr =3D pfn_to_section_nr(pfn); @@ -619,6 +619,15 @@ void __init sparse_init_subsection_map(unsigned long p= fn, unsigned long nr_pages } } =20 +void __init sparse_init_subsection_map(void) +{ + int i, nid; + unsigned long start, end; + + for_each_mem_pfn_range(i, MAX_NUMNODES, &start, &end, &nid) + sparse_init_subsection_map_range(start, end - start); +} + #ifdef CONFIG_MEMORY_HOTPLUG =20 /* Mark all memory sections within the pfn range as online */ diff --git a/mm/sparse.c b/mm/sparse.c index c92bbc3f3aa3..85557ef387c7 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -361,5 +361,6 @@ void __init sparse_init(void) } /* cover the last node */ sparse_init_nid(nid_begin, pnum_begin, pnum_end, map_count); + sparse_init_subsection_map(); vmemmap_populate_print_last(); } --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B9503D566F for ; Tue, 2 Jun 2026 10:11:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395095; cv=none; b=S3iM0aXtvnP81krEy1mvwFomwbeKU7hcxahT7ru0FtuWgxBWPiiX8XEyCUdoSsSd8c+JnHUcQfI/rsRzJPXawMEnm81g3VrvnI0L3iNntLhbVt4SyMbgcc4AGEWGKdD6NVIUrRCWchdzmoYadA67JqtAsiNoSDxizqIWWxZb/nI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395095; c=relaxed/simple; bh=dYe7aiqFYcOjFi1G/CU8eNdOQm6ogGYmEZdcC7M63F4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZwVQ6BKPM5Y3m6qsR2+D/wXbtQGZwKi7G6e9qwmGUzuJyjbMQsXvl8Jixd9Fifps6RhH/WC6b+xjRIAAEdyd1R4ZkhBkd3oe6/yEtbQSc50JQIJgoPWzmv2qGLZZyM9XMS85cjCU8KvYHQY4uS8IggqjQPEsqCKqZc3eU9pHXWA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=QAZW25Xk; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="QAZW25Xk" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-8423b08b293so953071b3a.3 for ; Tue, 02 Jun 2026 03:11:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395094; x=1780999894; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r2XnUrRZ1W0TfW9t5CZRJtozKMCRYAjD9rqyDyxjcoI=; b=QAZW25XkVvMTprhPjwcVw2RtDA1c9F/5eyTRfMDidPM+q2Ompipoeg3yFznfIGJjFJ q96c3QV6FtQZsYXC/NNXSQfG2nV+b2MuyZlDPnmrBNL681RVtmIXWcZlxfgz+lt2qyz6 wjvKMU7lUhFimXOfqsQrIyzHe9rrn6NsTd8lTxl3zNKSJ4Fw+VKpBGAhg+Drw/QnW5Wt 9ti9MLZkRmISQYiNEQWC/rVeJTSp8NfdtPS1tKYWtgrePUl1tuuZjTl4qbQzD0A+4wHq D66QUUok/ODd4ZzI/XNBK2rIhz+vO4xLSUT5hrLw5W0mTKoJZ8NFR9HfmGAgeHFkf3Mn LeNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395094; x=1780999894; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=r2XnUrRZ1W0TfW9t5CZRJtozKMCRYAjD9rqyDyxjcoI=; b=og2GtW2D7Sb7/uj8TyiJOcpZij++2zA+zA2IUgo9GuaPrQAQjJtWuBuDLw2q8+Yq2A c50LUkIDB/97uBKswTHtIaMJhvD0/oBG6idGDdv7iNV4qF1FODZ3Oql+vbJd5BM3JORQ vsyYGtda6tb1tdwS2wwNvkgBXBI9O+BFIompnh1HaQ2VWZHb71wg3wMBw+sHvADzeH0J 9hX8zUG/blsZHpvHKJZ1bmWDR0hYv0aXhP0WgwT/kS0CcX08gJWA0TuhbQyZCFAUiffV q3idxYB0/NYkuGqcXe88ThHT3mAgPQvRamX97uWNl2fWAK4Sl3u8edXMZFJvsdBTvCmY mslQ== X-Forwarded-Encrypted: i=1; AFNElJ9kkOqCblmshOYs5c2kX7A4rB+zjamwqSXs9EB1ECtWPnlqNObPt+NeyQuapuiEb/UytNUuo5xok0OrMvs=@vger.kernel.org X-Gm-Message-State: AOJu0Yz0CZ3nJ+3d2DeYvQ2LGHpj8idnD40Kpg9n/Ys6vIHLJsdxAIs8 tUibFxRgyvdIV0J81z56dfo4GM/hbDVeyJ0MdMbCC+6/f1hWvBbj4YL5AhBp3ImbDhw= X-Gm-Gg: Acq92OEMRrpjMey5KyfBRd8zBWoDAXT/yza/Bh9A8/ROTU/xVDpfJPOlFKX7pIVQKJp sgA+MobPAuTyBUE5gFyxkrSvJc33PsAf4H66UTz6015yx/fhs+A9Fsmiv7AsBE+0fkikGNkPv4k RBFfpGw4LLEmV/Yhao2WG945Da0aemUScdTbuMorP7BjU7IpIkTbKHU7FGsSqQZa8lAhvfGzsUx ESmiku+nUoMtEjyaLMfFzfi724cWyGMIinKJz0mtN9JA/rnso3Rbla3euK0KpcdrlEzcql4AgUA DVpqwZnPlduBzzFtWkLNKxqTvsx1ry5HubSdhv9mMANHTo0zMaeQzUAaDke7+rBZCcTPNzexlYh BtcvvwgZ1FIrV8UxPr5TPdiPnMb73ZRvDIkusL4B2Oz/HO8tnj8HWh4CkTYjIx8CvVx5fBGhfwk wNS4B6WZ7VghmY4URvyYO61CFwX7e8krwlXYBEWc7bnON0iMlEjtATEL7x9M8no3+EKQ== X-Received: by 2002:a05:6a00:4b50:b0:841:edbf:6424 with SMTP id d2e1a72fcca58-842253d6616mr12648288b3a.13.1780395093559; Tue, 02 Jun 2026 03:11:33 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:33 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song , "Oscar Salvador (SUSE)" Subject: [PATCH v3 08/19] mm/mm_init: Defer sparse_init() until after zone initialization Date: Tue, 2 Jun 2026 18:10:28 +0800 Message-ID: <20260602101039.1867613-9-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" free_area_init() is responsible for initializing pgdat and zone state. Calling sparse_init() from there mixes in later vmemmap and struct page setup, which makes the initialization flow less clear. Defer sparse_init(), sparse_vmemmap_init_nid_late(), and memmap_init() until after free_area_init() completes, when zone initialization is fully done. This keeps free_area_init() focused on zone setup and ensures that sparse_init() runs with the relevant zone state already available. This is also a prerequisite for later hugetlb vmemmap changes that need zone information during early sparse vmemmap setup. Signed-off-by: Muchun Song Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Oscar Salvador (SUSE) --- v2->v3: - collect Reviewed-by from Oscar Salvador --- mm/mm_init.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/mm_init.c b/mm/mm_init.c index 3a57bf5a9b46..f349a6f34139 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1829,7 +1829,6 @@ static void __init free_area_init(void) bool descending; =20 arch_zone_limits_init(max_zone_pfn); - sparse_init(); =20 start_pfn =3D PHYS_PFN(memblock_start_of_DRAM()); descending =3D arch_has_descending_max_zone_pfns(); @@ -1918,11 +1917,7 @@ static void __init free_area_init(void) } } =20 - for_each_node_state(nid, N_MEMORY) - sparse_vmemmap_init_nid_late(nid); - calc_nr_kernel_pages(); - memmap_init(); =20 /* disable hash distribution for systems with a single node */ fixup_hashdist(); @@ -2694,10 +2689,17 @@ void __init __weak mem_init(void) =20 void __init mm_core_init_early(void) { + int nid; + hugetlb_cma_reserve(); hugetlb_bootmem_alloc(); =20 free_area_init(); + + sparse_init(); + for_each_node_state(nid, N_MEMORY) + sparse_vmemmap_init_nid_late(nid); + memmap_init(); } =20 /* --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 878FF3D9050 for ; Tue, 2 Jun 2026 10:11:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395102; cv=none; b=kwba5UvPrPFd+Pf4tGZHJUjBsZC2QZtQHuto4bvJkwoB9d08hXY/ysMmO1NGJS2wqxnbkksbRBVoxHWLbR01kVw26HxyjN2e+uexuOMv09cnmZ/C0LIkaaDtS+mi4NYmndZzd9W0QBUXonl8HapG1GRL1VykrBeAm5CQW6kdyWk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395102; c=relaxed/simple; bh=4LsJ2UWfsT/Pv/Q2SbHbwoezh4Xmh8U+dJ1J/rj7woE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XA5MO4oRTH1Q+Mur86GQewWZ2uIDJ7Sc+Up7t3XsW3yny6OemPuVkpTKecL8Mk7PwWII+i3im/lZjcEdLjdyQLBKQ22XPZr78AJT0/Aw97jikhCTtkBKPVFzlz4lw0oiihKgrUNV/Ss3EAKbYNGbOgfIdgT2rpmSJCjwmMymj04= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=J2BohsJp; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="J2BohsJp" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-8423efad617so1079220b3a.0 for ; Tue, 02 Jun 2026 03:11:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395100; x=1780999900; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sxbi5h4Z+JKwQ2t0bJ1zO9+zcXybTFpZA7Fr3QeLXFU=; b=J2BohsJpVJO2O2JujBT00qQ/F6/iqAszC4DjtrP6nWLs7BzAWjgD0XWNmBZIS+c7Ns pwhlX57pJV2duzcE25b8ZUFfEB0IqVQINfgfKnnBBqMFikqn/T70wBWCGB7xBBuOllI0 7YWPPsxKzZU+y1i5w1b66PiCC6Sju4A8UdbaiZT3JTdV1DU/UDWNLsQnU+SN5ReW1Iru hZDnWzbeXGsjyeQ2+pTF0Tss3njp6wtClzV8hjZwD9kdg0yTvrU9vbxnu42aXKIQuKpf DauWF1ZD52rAyv//B72/Gw1iqMZIlgXjWlQw/aV18Ez8yX/3LINORg8atJqXRwxuDsrR 1C9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395100; x=1780999900; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=sxbi5h4Z+JKwQ2t0bJ1zO9+zcXybTFpZA7Fr3QeLXFU=; b=BV9wbK9gr9EqzQ5AkSmI0xhiC9x/BfFv3+t3vl3yRymImDkTaU8zercsgWTCGwqNBS W0wOiKYH19MbklIYjiWTTP0SMMZ95HFB4dX9U4poFjHibfEedBy5NrQevgAlUkzIMHYd vdMBZw4hYomQ03tjAWvkgD81e4p+9br7F8wy8XC+hZdv82CFxCp2zKHneVnZvnCo336g 3qQc6eFDXdu182M4b4HqXY3RvlotJXFpJGO4e73NxL/SsWzHmW9JR6ZcXi8aINtRn7tZ BA+LAWRK7014Xk2t8SmZtXWvgn6uONslEdUrjH9q7Lr+czdRTZIww9p6AA3RcxRpmQx9 29/Q== X-Forwarded-Encrypted: i=1; AFNElJ/zhfw1dDfBEMuFrRXAZjACiabXkk9BpylFfOCjwEVLWp1cCnh2F5vvmRSSp6yT2kmT8XSWcUzTpdZpTms=@vger.kernel.org X-Gm-Message-State: AOJu0YxvO1N/fjpPVtK7lmJuqMYNIlK6X40vneXJEtn6imGBJkf66MRb EDK+ts7/Dynd/M0B/CSCJn804fD0RgiFfxz+fC7iFywUKaMqfB5Dq8t2mMUbezXyt/Q= X-Gm-Gg: Acq92OE9fygd3TdXStj2f2d7ZS9OQW2DGh1xwfSbfHeokItwtUegSFw4TgYr3mVAHyN hj8fNH00dLaisM6Uezd/wstdNq9XOWF2vKLKfIkOSErDZ8sCO4abuBpTX5YjBPrmzKJhlemhoSl P4UhHyKfC3WNP+VcvWYh6yVO5p0OfohgIYLH4Ydn6CwwaWro+KkHDVm76lMfJRx/Ycb70zkK+vq dqIGuZbIXB0CawHa/eAvMKOu5ur0vNvrm1knrZMDX9UtgQgZQHdYCui4AlKmRQy6LF2iq3g3wCd OpqCDH2xI7oKkWSANC/eBeYrtSHKOZJRqBSznIP6JKAdtZ+VSTATEq6z2SXyeOqYb2Cj/buehBU k2pxuLRkvdbv7/KBK8tMaDeLbPGTPdpmKFIT6Ab7jZv69fCS8xD0V+0psac1A6MBGOh4PKRxhs+ YFUPYIggZof9quukdFkXO9VFV8PITXQMeY7dLMsmBatwk/bpFwTc9xkNo= X-Received: by 2002:a05:6a00:1741:b0:842:6004:3fb9 with SMTP id d2e1a72fcca58-84260044470mr5719945b3a.25.1780395099558; Tue, 02 Jun 2026 03:11:39 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:39 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song , "Oscar Salvador (SUSE)" Subject: [PATCH v3 09/19] mm/mm_init: Defer hugetlb reservation until after zone initialization Date: Tue, 2 Jun 2026 18:10:29 +0800 Message-ID: <20260602101039.1867613-10-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" hugetlb_cma_reserve() and hugetlb_bootmem_alloc() currently run before free_area_init(), so HugeTLB reservation happens before zone state is initialized. Move the reservation step after free_area_init() so the relevant zone information is available before HugeTLB reserves memory. This is needed for later hugetlb changes that validate boot-time HugeTLB reservations against zone boundaries. Signed-off-by: Muchun Song Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Oscar Salvador (SUSE) --- v2->v3: - collect Reviewed-by from Mike Rapoport - collect Reviewed-by from Oscar Salvador --- mm/mm_init.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/mm_init.c b/mm/mm_init.c index f349a6f34139..4601e5d659eb 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -2691,11 +2691,11 @@ void __init mm_core_init_early(void) { int nid; =20 + free_area_init(); + hugetlb_cma_reserve(); hugetlb_bootmem_alloc(); =20 - free_area_init(); - sparse_init(); for_each_node_state(nid, N_MEMORY) sparse_vmemmap_init_nid_late(nid); --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AB9B3D6497 for ; Tue, 2 Jun 2026 10:11:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395106; cv=none; b=R5Yy7rRCWUYFGHBXOx3PhVFBOOuf4JeF4LLmA6F+/EwVRbMJKzS7AqMW+YicFhhbe7vWq0wEIPz5KS2TGVC+mUJMh9ranJ1fIEkqAml9S3myvp3tC6PVSoB9W7/k6+kwjSZZtAryblEcyZbJ0oSK4QBliSH3M1umDYjckgslrLQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395106; c=relaxed/simple; bh=gYEvJ/8075x7WH+GgXiHA0urgdVKgapkoLUPlfUHXQc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bexfZzvoJnOseFS2ihsO4b6LTdQiRqTI/cDBVvgR4wwRM9vtUzz6pwnu1bgWwhuuQClu55qRtqxo+qfPYhg+H5oUGz4QOkUhd49pTbJ5YWhqABqo4B41lJcsPeuqEg/L0/3JtUq+yLo8ku6yQXqGJc9vmQks9Y31er3dyyKYFrM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=JqKfrbL3; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="JqKfrbL3" Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-8423f236418so1016660b3a.1 for ; Tue, 02 Jun 2026 03:11:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395105; x=1780999905; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lRWl52YiNF8NqlpcvOldJSrMkxHitZpwQxluu3UebDE=; b=JqKfrbL3ek1D5zVyw6Pq369uhaxdjpUMFMyM0AwpYtFxA3P1OCEmnqioB9cQuifbrL rKE3sBy8ow6SiFaX4BOmROUXKpNV5eXZLCFiu0MlPBX+HVdRp3POVHwODm5+BiQQiyza klZUZDT7PcjL++45NDKM+mdBmOndwXrYn5jnhYRE+PCoKZAVsx+L2QMEDn3Slrp0ZIQt uyMYA7dnnOH1SBhQbpsnDOrWWRbjjo6vhe9m6fiJxXxBzUHYyViAWWR3Blcvb89SVpsn FCeO6xNNhw8Xjj0WY2jLpTQF0771JvDuHSFStCz/E1P6elSbmu+WmijBEqXXNWDriyXM h35Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395105; x=1780999905; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=lRWl52YiNF8NqlpcvOldJSrMkxHitZpwQxluu3UebDE=; b=Sn+J+CORmGuK2jV47EZSB6e9JHlXGvNV6tsya4ENUopIfLsBpY0FseBS+IYRoeE0uh 8cUoVzwf6gCf7yI0VU4VjowhMIcRSvk5lLETetuBTPt8LN+hVLvS/w/J+uXotwujJWhu 6wBytotW0yHOQZ35/CYkug2DQzsxSlQch27P8GHPJNUVvSJTYt6EkNwNx6PBej70lhPT pOKhMIgVmnyWiuSeA/poXmnU70ZR0u+5kaFsGlyRpgMDKeSG08qje2H0eoWu80ci8jGM 1flo5pxyFycI7zCK3McJ0WE0Ve9eC/3bdfsBZd2zKe/cxNAMyxCRcBD0Vw5UXSaqnvjw i6Cw== X-Forwarded-Encrypted: i=1; AFNElJ/aoBYgB4RijYnJNbo/LSSfCexuPBBrWG5t/8Mka9xcDPG397wbDjMVjYdUedfYLei78yN7UH/h3P3UhcY=@vger.kernel.org X-Gm-Message-State: AOJu0YycnW2rcaaNMdt+lvw1lDSfpP4hvRkCAUviaJPIe/PPHTYkSArU zOKN2BaX3r5XW1Uu2yej+4pUA922awjdpR4ks7D13+8W41vOWuRVFAc/QHWrO8+N2oE= X-Gm-Gg: Acq92OFgwCtMbTNyoyCGv64J+9PBDpccD8OBQZ8T4luBSNzY1DO/7zNdXFmLsjx7q3g zlxLln9aTv4V+ukC/msw52m/eUov50nu8vgv+J8xTXndpuieOyxGcn+iDzBQnGxeenOpx3wZo9M c+R8lOMnt16ajSAM1m14jqu+dzxEYsg+7ZaDiRtaBsb9Onj0fbO/HnvPqABAPufPC1PZuFxPHmM FM9UOtQ5gM35VBtUyrRJ0JAhIkK6ayLZu+krH/aE5sZNYCNb1/7LF+aFgCGKSHHd80CiBcONEFF 31gAqt5uDMDFSvhFbFgaSP2Q3PTG+Pc3+F0vfxjIvGQt+gjsK44nfpL4gd4z44n5axJaK0KSA4U OHcskFTHY+wnSxUFTD58i16hk0olN0R//AyyOPXFDeRWGXKkFMVls+qaajZ3XGvqGI7Pt8Qn/06 horS3XV64/SafbAGLaWclSrb6IHypJ2JUAY2EygmC3nhRsVPsS97VeOF12u0eGeRezbw== X-Received: by 2002:a05:6a00:22ca:b0:842:4469:9202 with SMTP id d2e1a72fcca58-84244699f1bmr9235454b3a.12.1780395104666; Tue, 02 Jun 2026 03:11:44 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:44 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 10/19] mm/mm_init: Remove set_pageblock_order() call from sparse_init() Date: Tue, 2 Jun 2026 18:10:30 +0800 Message-ID: <20260602101039.1867613-11-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" free_area_init() already sets pageblock_order before sparse_init() runs for CONFIG_HUGETLB_PAGE_SIZE_VARIABLE, so sparse_init() does not need to call set_pageblock_order() again. With that call removed, set_pageblock_order() is only used in mm/mm_init.c. Make it static. Reviewed-by: Mike Rapoport (Microsoft) Signed-off-by: Muchun Song Reviewed-by: Oscar Salvador (SUSE) --- v2->v3: - collect Reviewed-by from Oscar Salvador --- mm/internal.h | 1 - mm/mm_init.c | 4 ++-- mm/sparse.c | 3 --- 3 files changed, 2 insertions(+), 6 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index e71ba519f7f2..004a3f1d5006 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1435,7 +1435,6 @@ extern unsigned long __must_check vm_mmap_pgoff(stru= ct file *, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long); =20 -extern void set_pageblock_order(void); unsigned long reclaim_pages(struct list_head *folio_list); unsigned int reclaim_clean_pages_from_list(struct zone *zone, struct list_head *folio_list); diff --git a/mm/mm_init.c b/mm/mm_init.c index 4601e5d659eb..44512f3b3544 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1511,7 +1511,7 @@ static inline void setup_usemap(struct zone *zone) {} #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE =20 /* Initialise the number of pages represented by NR_PAGEBLOCK_BITS */ -void __init set_pageblock_order(void) +static void __init set_pageblock_order(void) { unsigned int order =3D PAGE_BLOCK_MAX_ORDER; =20 @@ -1537,7 +1537,7 @@ void __init set_pageblock_order(void) * include/linux/pageblock-flags.h for the values of pageblock_order based= on * the kernel config */ -void __init set_pageblock_order(void) +static inline void __init set_pageblock_order(void) { } =20 diff --git a/mm/sparse.c b/mm/sparse.c index 85557ef387c7..324213d8bdcb 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -343,9 +343,6 @@ void __init sparse_init(void) pnum_begin =3D first_present_section_nr(); nid_begin =3D sparse_early_nid(__nr_to_section(pnum_begin)); =20 - /* Setup pageblock_order for HUGETLB_PAGE_SIZE_VARIABLE */ - set_pageblock_order(); - for_each_present_section_nr(pnum_begin + 1, pnum_end) { int nid =3D sparse_early_nid(__nr_to_section(pnum_end)); =20 --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3A373D9DCC for ; Tue, 2 Jun 2026 10:11:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395113; cv=none; b=TCQHe4vC/ahB69eWFVSMjufdsA0wh650eM+vRIR77B0rKuezxYDtnl/gDLDJbFLB708vedvtXNvYrc+lhF5RfYAIG3sJprJagtoKHa9nqcFkcWrFBqCOMIbGfpr1PC5DT0DjqlJ0oCtuZGiuTMyEn97D72YibMraJXP4X98+QYc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395113; c=relaxed/simple; bh=6YEn5mTtNtNOs/vkHbwfozvj0hVUloezrimvM3JFLVY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZFM0biHAfDl+ViA/GzlIPuMEtNDiL9TiCBw7xGHRdC2j7lLNTyHIfo9zjb2T1h/MtjCtMsKWc8rkn8sJCURaHO51zwZdDEIqyVx05R/kbuok4AlTD4cz2GgbR5iWYr/I6DUs57EtXz4RvqiEfjWYokJgwZtKiLtc+ZLmQwprGJQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=jyOT3y/f; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="jyOT3y/f" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-8423b08b293so953165b3a.3 for ; Tue, 02 Jun 2026 03:11:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395111; x=1780999911; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ePux0+p+Er7H/YD/Nlp/8ECZIcHxOs0LOb73zwAatVo=; b=jyOT3y/fHpo1eOYMAyafTwbBPa9tLq/xNJqeEcatp5dCrKuaITTDUf3Kv+60lJAfyi ozpHxn6Ytdr2w4JfSk8vdxTrW2AXz5bUvns/9s8gF+cteJBRLm1DXVIox3Nx7xkJslhk 9WUo46pc+4LHtIFV71NtViLmzFjqPbtIOzlT/aOVtUsvoUpkOTTFyl0pJhtYvkdU35l+ jbD9xxI4uzkqrRxuR1jKICtsxo3S+N7PTmlRVatvcJik97aZPanrd52Ti1oVIv3PiOCW 7abAinT5ltbJ+KFYKU6vtwPcaqHvcmEip5f7JdHvV4SyydDIvl8DKSdSAq+hGOEJbulf iQ4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395111; x=1780999911; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ePux0+p+Er7H/YD/Nlp/8ECZIcHxOs0LOb73zwAatVo=; b=TvkNklEf7MWkmGPwpHxPag3q8bCEED2CxjuC9yMGuF9hjI+mH0bsNNK2Mbf+s9IcRA nEZ/3EOMMzDvguZtZ2++5zOeK2EVN03kKIjO9RAv7xS6IY9z/tuWjkYp6volwUYOWsIH DFcd0IQHz365Af4tm29zlpOFe7n1EsE36VNLApVwlcIve9CRPbFO8rvzrKrzLMUTShBx J7VXrKd7S5bxx9Az+WDTmODSUuAigAZ0vL+wFiWAbQHthHhixxel+XyUq2ryEHSnNSTV f5g9ukdougXu39FFZo531I7NZ3vpLvXJGZiRf8zJv2D5z1QedpS0W7H0RtF5MWIathsY BK+g== X-Forwarded-Encrypted: i=1; AFNElJ85ANfB/mr9AVRncWqHd3/4mALcbaZRY5R0dbN+01czPZp5XRH5Ft2l1ZEJnd95Es5iPRAzWKVQJDI6vuE=@vger.kernel.org X-Gm-Message-State: AOJu0YzXodoHb+KMh2UH7d9IbVl3bmBqrgeW4aVxtb5el1kk7KUfivHy 7pSWliy5ifEZoDuC4F3dUsxtPHmjtHkTWqnHQ7kVb36IVE3RBtHkZPPvLBAEhLh+7jo= X-Gm-Gg: Acq92OEeTcWdIVtrqBn/q79QvTljN1JDLwV2q8CLHy3uqz1aw5sH/dnE4XFsq4UifE9 iZyGeD3tGOxWG0GoDtawDUqJbsY70tGQ31H7a+z9olTH3aqjAHtKi8sh65iMVLeqWOzwKSJ4uKx RR6GPTwKicBwCUcD9ncjZA28Jq6eHYPIeNE++dJ58JMmfDsKphnxd53a1uGU0olEFbM3QkgiWxz gqT0NWbtQ/NKg/wi/TH2qELTdrwPQX2vkYotRFQn9IFfB6gYcoBwdd/9TzaUEjsesEDDZjTv5Vz Tvrf0sFVm/Toy5Tcpn/s4aSCmMMZrnQJxm4zNx9sc6J81FSdaxDv4haNGX6iUgoyjhBVGOUOJaG EVsaJJXSVeTNImRhC4MBBmWudjMpCintAejzO4/U6OBiiGqoZtszcDGZ11FKJz4PkAQDli3klaj IHXWwvq5EtScg9yl/UY9SfE73LIaujZj4gjf8vNx8dzOWJp1T1h3NoLOeP9qkro1odsg== X-Received: by 2002:a05:6a00:22c5:b0:842:7d47:28bb with SMTP id d2e1a72fcca58-8427d473274mr311986b3a.47.1780395110901; Tue, 02 Jun 2026 03:11:50 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:50 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song , "Oscar Salvador (SUSE)" Subject: [PATCH v3 11/19] mm/sparse: Move sparse_vmemmap_init_nid_late() into sparse_init_nid() Date: Tue, 2 Jun 2026 18:10:31 +0800 Message-ID: <20260602101039.1867613-12-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" sparse_vmemmap_init_nid_late() is still called separately from mm_core_init_early(), away from the rest of the sparse initialization path. Now that sparse_init() runs after zone initialization, call sparse_vmemmap_init_nid_late() from sparse_init_nid() instead. This keeps both sparse_vmemmap_init_nid_early() and sparse_vmemmap_init_nid_late() in the sparse setup path. Signed-off-by: Muchun Song Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Oscar Salvador (SUSE) --- v2->v3: - collect Reviewed-by from Oscar Salvador --- mm/mm_init.c | 4 ---- mm/sparse.c | 1 + 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/mm/mm_init.c b/mm/mm_init.c index 44512f3b3544..41b83dd18c01 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -2689,16 +2689,12 @@ void __init __weak mem_init(void) =20 void __init mm_core_init_early(void) { - int nid; - free_area_init(); =20 hugetlb_cma_reserve(); hugetlb_bootmem_alloc(); =20 sparse_init(); - for_each_node_state(nid, N_MEMORY) - sparse_vmemmap_init_nid_late(nid); memmap_init(); } =20 diff --git a/mm/sparse.c b/mm/sparse.c index 324213d8bdcb..3917a47153d8 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -320,6 +320,7 @@ static void __init sparse_init_nid(int nid, unsigned lo= ng pnum_begin, } } sparse_usage_fini(); + sparse_vmemmap_init_nid_late(nid); } =20 /* --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1F023DA5B4 for ; Tue, 2 Jun 2026 10:12:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395122; cv=none; b=FvXFijJXILyzkmuybrqo0/kOFPLRNe1RhxNL7U7mQqCTtMcZARZoFhJXbcfwjJyqvbBpxI0JAV4P/AsbcL60a6pn27KBuGpPdt1jfOweSF2r65Lz+s6gQFe8qBG8XZ0jyFfJ9I5luypPG/XhJsMOve/E7ANEY6te1DHOne7Uv2c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395122; c=relaxed/simple; bh=tGDjWR//Z0ExgY5X8xfzTayfFuNUGXS7Is6BW41xo1A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fB/PlGVqestwu63d/3pY4EQeut9/G3mKsZ6+uZp1AVcTzXSXyg1DfMD4eZ21FJ1Ct5c25DQyO0R6vk3asqsfiUjfZ+Al+uvi3+Zf6vZkoNd/AR9Pien104SFLnK+ewYVqOMPcJLsT1jxym+oUlStTjywlvcbcA8cc9mbr9Mc5rY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=SfTkx/5a; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="SfTkx/5a" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-84275887a3fso228856b3a.1 for ; Tue, 02 Jun 2026 03:12:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395120; x=1780999920; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VWUO3+cl7MWVyS5m22bSK5o78fH+CF71Q5m4ETc4a2A=; b=SfTkx/5ayg/P67dhgrij2D1Fek0/aU8XvyphWubuU1plczJWHdQi9R0hkZSLgG+PEG e8Mn4GdNBFzgREueSx0UE6OMQVa8I5T/3w3jEMFTVCot27eyglDt7jE+4YRGEV83whV1 ALq2H9Xh7WFaeYRP3j4g7K4jNRTCCqIMqXEKgwAFGRTddMuIfLUx9rPbwV6Tz858qTEB mvlXL5bPrGaCj71sV8tp5kwfHTloXp2KPq0OInfrRI2BLdRzNk/2W8vQoEfWAft8j3IM G06BV2v4QzJ8wFA0XSYLNbIMNBeMmFdvpu255OEcKswXKavAe1m3OnIo51PV+8/QcIBm i6Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395120; x=1780999920; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=VWUO3+cl7MWVyS5m22bSK5o78fH+CF71Q5m4ETc4a2A=; b=eGUV+WLkaS3n1lFYcmqjB5gfynLRPu6UqgmtJn221c6h9L/Pn0dFsW5BS4RCahXNJp h1pMxEltOOr5fj2GFjHOsCjUz2+hIULRnwc18Il42AsBhejTXufNMDXPDp3Dq3S9rJo9 52FC3Rh6rK+8knxITzED6aYbL3oXXpoVjD3dCmluaoZnG8j56C6EZQ8NgzixD6VHRkOa pCAIcWOwO3zjrMDCrQhcFOP2GWTM2neBF7vKKVkLXuxg+lqaXJ1I8Rqrejya47VzRW+l oKDq/1L9riwO5+xkgkLJ7XWZOGGxXR6C0aNlo+ApwH/eQuQy1wmj0COl7dCQezJFqhrh gVrw== X-Forwarded-Encrypted: i=1; AFNElJ9Ohr+Rraqufs3dfhCD/E5IaTr28roX90KHWMH4ICIL2rNsmA/sh5ZWs8O+e51RFhBg9RMCLOdRA7yA7MI=@vger.kernel.org X-Gm-Message-State: AOJu0YyXqptpOcW1kFAqhxh73BUohmLlIX637hVFxaOrRk4Ikp/nGAvS /BPtvil/H136xuDFrWv2X/jYaOVbQyQD2DHg4HKGN0pFe6YyPm4xK3icUgnhDZI4XII= X-Gm-Gg: Acq92OFDaTv344xHPQXspb+e4wr6plFIN0/ZJ/dnAfctDfYswoC/WDd/KkcPDP1hzSW Hjea0mVWor1a0c8yH3duLq1KP+4yXfXqtnF5HS/yzc2AoLFKy5N9Z/xPGpdbtkXvsouiJ/AIc7g cjm27qy8QstbEcfMMPwcTasqow+/hNefVPCw+1wdMZCPdegkmK0JaeDE5g3Oa8zEeOgKx5NI9O9 eEQrHXdDZNN+hT2nlfrVHxe3pWk8alc2MHyxXtN7odfT6GMQ8ORTP8JeDVKwe7MTWeZiaec+8EW yMeMZPHryDiPDaCBxmErH9euhjLub6wB81LzEmkaqi5sHi0FWYR1O9WxA0FYLxbdLiK+QTaL9B8 017lSUD/3V4ExcIkl0Kiz+RwRUlYs9etgU1/suC8prmwr/y5NrA5VAgSG1hYHmfTx2pnNEo8l2p kqDJAxNbEwwreXBhfMzc+hbo0LCvVVcu096IvvzLNMDD0QeKQuXLXvSYo= X-Received: by 2002:a05:6a20:c79a:b0:3b4:613f:90b1 with SMTP id adf61e73a8af0-3b478a4a8ffmr3176243637.16.1780395116512; Tue, 02 Jun 2026 03:11:56 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.11.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:11:55 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 12/19] mm/hugetlb_cma: Validate hugetlb CMA range by zone at reserve time Date: Tue, 2 Jun 2026 18:10:32 +0800 Message-ID: <20260602101039.1867613-13-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Hugetlb CMA allocation currently has to cope with CMA areas that span multiple zones. Validate the reserved CMA range up front in hugetlb_cma_reserve() so later hugetlb CMA allocations can assume a zone-consistent area. Also drop the pfn_valid() check from cma_validate_zones(). mem_section is not fully initialized at this point, so the check can trigger false warnings. Keep the sanity check in cma_activate_area() instead. Acked-by: Mike Rapoport (Microsoft) Signed-off-by: Muchun Song Reviewed-by: Oscar Salvador (SUSE) --- v2->v3: - collect Reviewed-by from Oscar Salvador --- mm/cma.c | 3 ++- mm/hugetlb_cma.c | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/cma.c b/mm/cma.c index a13ce4999b39..31073738f2ac 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -126,7 +126,6 @@ bool cma_validate_zones(struct cma *cma) * to be in the same zone. Simplify by forcing the entire * CMA resv range to be in the same zone. */ - WARN_ON_ONCE(!pfn_valid(base_pfn)); if (pfn_range_intersects_zones(cma->nid, base_pfn, cmr->count)) { set_bit(CMA_ZONES_INVALID, &cma->flags); return false; @@ -165,6 +164,8 @@ static void __init cma_activate_area(struct cma *cma) bitmap_set(cmr->bitmap, 0, bitmap_count); } =20 + WARN_ON_ONCE(!pfn_valid(cmr->base_pfn)); + for (pfn =3D early_pfn[r]; pfn < cmr->base_pfn + cmr->count; pfn +=3D pageblock_nr_pages) init_cma_reserved_pageblock(pfn_to_page(pfn)); diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c index 39344d6c78d8..ce999391cc14 100644 --- a/mm/hugetlb_cma.c +++ b/mm/hugetlb_cma.c @@ -231,9 +231,11 @@ void __init hugetlb_cma_reserve(void) res =3D cma_declare_contiguous_multi(size, gigantic_page_size, HUGETLB_PAGE_ORDER, name, &hugetlb_cma[nid], nid); - if (res) { - pr_warn("hugetlb_cma: reservation failed: err %d, node %d", + if (res || !cma_validate_zones(hugetlb_cma[nid])) { + pr_warn("hugetlb_cma: %s: err %d, node %d\n", + res ? "reservation failed" : "reserved area spans zones", res, nid); + hugetlb_cma[nid] =3D NULL; continue; } =20 --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BA5B3DA7CE for ; Tue, 2 Jun 2026 10:12:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395128; cv=none; b=fvGvIpiYVc0916gHzh2nu1MIOWsxQxPCOq8T7LSBJsMASYWTdBepvoaftUAs1f9tURnq+2/ZmLR0pQ16Nlp1YfTgrZQ3vdqO+GDrKGoq/Q1p9F1AJVkT6lvv+QbcO3pbLCEs53ePHjHknDJwFoAidaMjlGUiYt098ffbOTBcTQs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395128; c=relaxed/simple; bh=qRRCdi5IilGYBq+Ne7Tno3ZTnKD/ocdySkDrQCOGESU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TyQzek333tBupLQG3vYppYZA8s96WK+85AkV8Ol1h4CNebKHkIBc/mRiSMkTUC0zKT4n4tutsbpcvwWArUlyHRw/opntQn3hTgw+vpYpQZrsX9gRaHjIN+OPi/n9JGDJ5sZav6SM/g/k5ClaovQ456e5SgnsHOruatU2h7UFLsY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=K9S6cAh1; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="K9S6cAh1" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-84234c83142so986115b3a.1 for ; Tue, 02 Jun 2026 03:12:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395126; x=1780999926; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=izaqgnnNIWyz1SuOc+YrUva6s2Nz3vC7JJgyPQLHyJ8=; b=K9S6cAh1Xr1WQ14vgBzKD4R2iR/TtyYCRlZYXI1cdbBIloQIVE7dJ9RwKeWxFGCc8u 9l+qDyCBaOHHvX2D6J7QfflOlLIKNUXLdkooj+v82GqaEZ8e06gGnUNVr39dwPBA+4Ut K+kum1cGn6lG7jquH9341ozMdHIGZEuoL0Ftso8GS7PpOkUz15gMspYdpl5jId1/tjp2 DUWzdwK/r0oN3nTtH0OczJLNpJjHPYQNGm1CBCmtEs1z8TErHzgGFJpMrqwRbX9IobY0 UNTkj4/A57yAmDENUVx8WOTMRJXmhGW6dK+8/owc1AhVOv9nWnJ6CPHZPPXz4Y+oXYdK eugw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395126; x=1780999926; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=izaqgnnNIWyz1SuOc+YrUva6s2Nz3vC7JJgyPQLHyJ8=; b=oEqyVYHiIZoi6SpnN14NrxkJKUjvdX07U9Ws0JyHLKWQKfUvZp6cvNb024HOEvT8a4 Cphvxy1dg9TuzW929ny+TLue4p+7kq3uymgFUi9qkl9ThbZ6IagmkZxfCylIlovuaQM/ ctiS9rU09qFgXkcR2uGQhPLokxGar/Olr2vros4HuuGlbN7NtVrvk9iDGLRLn92DRT9h L5q8raagYiuwWEUgigyMC8Af/F1FNlp4OXp+bAcBMq8wg6Rl+FaT4A0Wgn/lh222XpNe pHaOHQN3oabSnszBPc39V+/46IO+nhYB5T/8oukfNipz2BqUHkxawSKsXPusrYpcWWRf 6BGg== X-Forwarded-Encrypted: i=1; AFNElJ+3cETxEuxlkysSxdLgrLXEWkzS30G8ArPYN4yx7GD7H0L56FhXIvDCxc1FdIg+knfIGyKp+r8F2gQOOfE=@vger.kernel.org X-Gm-Message-State: AOJu0YyQRgTEEXRiRy1Mm0uGcPBDaL6MDQJNGkEEzx5T4NIBVLRH8/DB yW9hGcE0gLDwP9UJ0A8/stxyQUCRtkeBqkJhyxPqz6q5/lnVwFRD8sTzCqe9MV7OZ2g= X-Gm-Gg: Acq92OFhXL0M2MuilMRVUnqpmjY24UbyOksFVhw/LZmDv8c1fbk8wnaL2BNieMO78aB 0xBg7ErE1B7snKczx/JA4vaeEzduem2WFB2Sdeyj2sYA+9kysDRum3FX4mNAeYxW83HzcjSiLKA CoVQFoYrTPKpiREoNe8iDAUxnOADlINPPhpig8TfyibD2cOmTtwp6vOzMn7a+Ngw7BxhScF5bM3 0nguDAymx0HPNnE/Cx+NXAWlbmmB03Kz9s36k3kYq7Gzx3Rldz5y/S9K6iXeno2NM5+u/H0s4Ds 6PbJvM5JSjSrjYb5D4XsWcJRF0qrlI3Ph4KvajtSdftoCOjkv4qKqZG2vjI8M+xpXeJxzH30QtM uoxyKRm2R6Xo9wGN4gG8VCjyCJ8erml6MgCIj2a81/qQ6NcXRY207/DUFCSrzpnOhlklLl7ASwc brHwuRynYiQVXpUyYGUS0sOLK7xs1zjIFstibeOP/ogYfCeVAuBYxTcscYFEJGt703Hg== X-Received: by 2002:a05:6a00:744c:b0:842:3aee:12c1 with SMTP id d2e1a72fcca58-8423aee321dmr9318619b3a.18.1780395125556; Tue, 02 Jun 2026 03:12:05 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.12.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:12:05 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 13/19] mm/hugetlb: Refactor early boot gigantic hugepage allocation Date: Tue, 2 Jun 2026 18:10:33 +0800 Message-ID: <20260602101039.1867613-14-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The early boot gigantic hugepage allocation helpers currently mix allocation with huge_bootmem_page setup, and leave part of the initialization flow in architecture code. Refactor the interface to return the allocated huge page pointer and move the huge_bootmem_page setup into the generic hugetlb code. This makes the architecture-specific paths focus only on finding memory, while the common code handles node placement and early page metadata setup in one place. This also lets powerpc benefit from memblock_reserved_mark_noinit(), which it did not enable before. In addition, upcoming cross-zone validation for boot-time gigantic hugetlb reservation is common logic. With this refactoring, that logic can stay in the generic code instead of being duplicated in architecture-specific paths. Signed-off-by: Muchun Song Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Oscar Salvador (SUSE) --- v2->v3: - keep powerpc code independent of struct huge_bootmem_page by switching it to void * (per Mike Rapoport) - move huge_bootmem_page internals out of include/linux/hugetlb.h and keep them in mm-private scope so the arch code does not need to see the type (per Mike Rapoport, echoed by Oscar Salvador) --- arch/powerpc/mm/hugetlbpage.c | 13 ++--- include/linux/hugetlb.h | 18 ++----- mm/hugetlb.c | 95 ++++++++++++++--------------------- mm/hugetlb_cma.c | 13 ++--- mm/hugetlb_cma.h | 8 ++- mm/internal.h | 9 ++++ 6 files changed, 64 insertions(+), 92 deletions(-) diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c index 558fafb82b8a..a298746dc143 100644 --- a/arch/powerpc/mm/hugetlbpage.c +++ b/arch/powerpc/mm/hugetlbpage.c @@ -104,17 +104,14 @@ void __init pseries_add_gpage(u64 addr, u64 page_size= , unsigned long number_of_p } } =20 -static int __init pseries_alloc_bootmem_huge_page(struct hstate *hstate) +static __init void *pseries_alloc_bootmem_huge_page(struct hstate *hstate) { - struct huge_bootmem_page *m; + void *m; if (nr_gpages =3D=3D 0) - return 0; + return NULL; m =3D phys_to_virt(gpage_freearray[--nr_gpages]); gpage_freearray[nr_gpages] =3D 0; - list_add(&m->list, &huge_boot_pages[0]); - m->hstate =3D hstate; - m->flags =3D 0; - return 1; + return m; } =20 bool __init hugetlb_node_alloc_supported(void) @@ -124,7 +121,7 @@ bool __init hugetlb_node_alloc_supported(void) #endif =20 =20 -int __init alloc_bootmem_huge_page(struct hstate *h, int nid) +void *__init arch_alloc_bootmem_huge_page(struct hstate *h, int nid) { =20 #ifdef CONFIG_PPC_BOOK3S_64 diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 3700c0a1f6ff..09f28dd773b7 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -674,19 +674,11 @@ struct hstate { char name[HSTATE_NAME_LEN]; }; =20 -struct cma; - -struct huge_bootmem_page { - struct list_head list; - struct hstate *hstate; - unsigned long flags; - struct cma *cma; -}; - #define HUGE_BOOTMEM_HVO 0x0001 #define HUGE_BOOTMEM_ZONES_VALID 0x0002 #define HUGE_BOOTMEM_CMA 0x0004 =20 +struct huge_bootmem_page; bool hugetlb_bootmem_page_zones_valid(int nid, struct huge_bootmem_page *m= ); =20 int isolate_or_dissolve_huge_folio(struct folio *folio, struct list_head *= list); @@ -706,8 +698,8 @@ void restore_reserve_on_error(struct hstate *h, struct = vm_area_struct *vma, unsigned long address, struct folio *folio); =20 /* arch callback */ -int __init __alloc_bootmem_huge_page(struct hstate *h, int nid); -int __init alloc_bootmem_huge_page(struct hstate *h, int nid); +void *__init __alloc_bootmem_huge_page(struct hstate *h, int nid); +void *__init arch_alloc_bootmem_huge_page(struct hstate *h, int nid); bool __init hugetlb_node_alloc_supported(void); =20 void __init hugetlb_add_hstate(unsigned order); @@ -1138,9 +1130,9 @@ alloc_hugetlb_folio_nodemask(struct hstate *h, int pr= eferred_nid, return NULL; } =20 -static inline int __alloc_bootmem_huge_page(struct hstate *h) +static inline void *__alloc_bootmem_huge_page(struct hstate *h, int nid) { - return 0; + return NULL; } =20 static inline struct hstate *hstate_file(struct file *f) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 2bf9fe16abb9..5e557c05d80a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3027,79 +3027,58 @@ struct folio *alloc_hugetlb_folio(struct vm_area_st= ruct *vma, =20 static __init void *alloc_bootmem(struct hstate *h, int nid, bool node_exa= ct) { - struct huge_bootmem_page *m; - int listnode =3D nid; - if (hugetlb_early_cma(h)) - m =3D hugetlb_cma_alloc_bootmem(h, &listnode, node_exact); - else { - if (node_exact) - m =3D memblock_alloc_exact_nid_raw(huge_page_size(h), + return hugetlb_cma_alloc_bootmem(h, nid, node_exact); + + if (node_exact) + return memblock_alloc_exact_nid_raw(huge_page_size(h), huge_page_size(h), 0, MEMBLOCK_ALLOC_ACCESSIBLE, nid); - else { - m =3D memblock_alloc_try_nid_raw(huge_page_size(h), + + return memblock_alloc_try_nid_raw(huge_page_size(h), huge_page_size(h), 0, MEMBLOCK_ALLOC_ACCESSIBLE, nid); - /* - * For pre-HVO to work correctly, pages need to be on - * the list for the node they were actually allocated - * from. That node may be different in the case of - * fallback by memblock_alloc_try_nid_raw. So, - * extract the actual node first. - */ - if (m) - listnode =3D early_pfn_to_nid(PHYS_PFN(__pa(m))); - } - - if (m) { - m->flags =3D 0; - m->cma =3D NULL; - } - } - - if (m) { - /* - * Use the beginning of the huge page to store the - * huge_bootmem_page struct (until gather_bootmem - * puts them into the mem_map). - * - * Put them into a private list first because mem_map - * is not up yet. - */ - INIT_LIST_HEAD(&m->list); - list_add(&m->list, &huge_boot_pages[listnode]); - m->hstate =3D h; - } - - return m; } =20 -int alloc_bootmem_huge_page(struct hstate *h, int nid) +void *__init arch_alloc_bootmem_huge_page(struct hstate *h, int nid) __attribute__ ((weak, alias("__alloc_bootmem_huge_page"))); -int __alloc_bootmem_huge_page(struct hstate *h, int nid) +void *__init __alloc_bootmem_huge_page(struct hstate *h, int nid) { - struct huge_bootmem_page *m =3D NULL; /* initialize for clang */ int nr_nodes, node =3D nid; =20 /* do node specific alloc */ - if (nid !=3D NUMA_NO_NODE) { - m =3D alloc_bootmem(h, node, true); - if (!m) - return 0; - goto found; - } + if (nid !=3D NUMA_NO_NODE) + return alloc_bootmem(h, node, true); =20 /* allocate from next node when distributing huge pages */ for_each_node_mask_to_alloc(&h->next_nid_to_alloc, nr_nodes, node, - &hugetlb_bootmem_nodes) { - m =3D alloc_bootmem(h, node, false); - if (!m) - return 0; - goto found; - } + &hugetlb_bootmem_nodes) + return alloc_bootmem(h, node, false); =20 -found: + return NULL; +} + +static bool __init alloc_bootmem_huge_page(struct hstate *h, int nid) +{ + struct huge_bootmem_page *m =3D arch_alloc_bootmem_huge_page(h, nid); + + if (!m) + return false; + + nid =3D early_pfn_to_nid(PHYS_PFN(__pa(m))); + /* + * Use the beginning of the huge page to store the huge_bootmem_page + * struct (until gather_bootmem puts them into the mem_map). + * + * Put them into a private list first because mem_map is not up yet. + */ + INIT_LIST_HEAD(&m->list); + list_add(&m->list, &huge_boot_pages[nid]); + m->hstate =3D h; + if (!hugetlb_early_cma(h)) { + m->cma =3D NULL; + m->flags =3D 0; + } =20 /* * Only initialize the head struct page in memmap_init_reserved_pages, @@ -3111,7 +3090,7 @@ int __alloc_bootmem_huge_page(struct hstate *h, int n= id) memblock_reserved_mark_noinit(__pa((void *)m + PAGE_SIZE), huge_page_size(h) - PAGE_SIZE); =20 - return 1; + return true; } =20 /* Initialize [start_page:end_page_number] tail struct pages of a hugepage= */ diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c index ce999391cc14..e487d0ffffc0 100644 --- a/mm/hugetlb_cma.c +++ b/mm/hugetlb_cma.c @@ -56,14 +56,13 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order,= gfp_t gfp_mask, return folio; } =20 -struct huge_bootmem_page * __init -hugetlb_cma_alloc_bootmem(struct hstate *h, int *nid, bool node_exact) +void * __init hugetlb_cma_alloc_bootmem(struct hstate *h, int nid, bool no= de_exact) { struct cma *cma; struct huge_bootmem_page *m; - int node =3D *nid; + int node; =20 - cma =3D hugetlb_cma[*nid]; + cma =3D hugetlb_cma[nid]; m =3D cma_reserve_early(cma, huge_page_size(h)); if (!m) { if (node_exact) @@ -71,13 +70,11 @@ hugetlb_cma_alloc_bootmem(struct hstate *h, int *nid, b= ool node_exact) =20 for_each_node_mask(node, hugetlb_bootmem_nodes) { cma =3D hugetlb_cma[node]; - if (!cma || node =3D=3D *nid) + if (!cma || node =3D=3D nid) continue; m =3D cma_reserve_early(cma, huge_page_size(h)); - if (m) { - *nid =3D node; + if (m) break; - } } } =20 diff --git a/mm/hugetlb_cma.h b/mm/hugetlb_cma.h index c619c394b1ae..3aa483573d17 100644 --- a/mm/hugetlb_cma.h +++ b/mm/hugetlb_cma.h @@ -6,8 +6,7 @@ void hugetlb_cma_free_frozen_folio(struct folio *folio); struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, int nid, nodemask_t *nodemask); -struct huge_bootmem_page *hugetlb_cma_alloc_bootmem(struct hstate *h, int = *nid, - bool node_exact); +void *hugetlb_cma_alloc_bootmem(struct hstate *h, int nid, bool node_exact= ); bool hugetlb_cma_exclusive_alloc(void); unsigned long hugetlb_cma_total_size(void); void hugetlb_cma_validate_params(void); @@ -23,9 +22,8 @@ static inline struct folio *hugetlb_cma_alloc_frozen_foli= o(int order, return NULL; } =20 -static inline -struct huge_bootmem_page *hugetlb_cma_alloc_bootmem(struct hstate *h, int = *nid, - bool node_exact) +static inline void *hugetlb_cma_alloc_bootmem(struct hstate *h, int nid, + bool node_exact) { return NULL; } diff --git a/mm/internal.h b/mm/internal.h index 004a3f1d5006..6b9802460a7c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -23,6 +23,15 @@ #include "vma.h" =20 struct folio_batch; +struct hstate; +struct cma; + +struct huge_bootmem_page { + struct list_head list; + struct hstate *hstate; + unsigned long flags; + struct cma *cma; +}; =20 /* * Maintains state across a page table move. The operation assumes both so= urce --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C13E93DA7FF for ; Tue, 2 Jun 2026 10:12:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395133; cv=none; b=hz+dQSFaD7lN/07lOl9zHPEQRHjLMgekMhUnQaAEP/yDVL0BccLktholQemB9PvxuIpM29LQVXPF8KLJ9OnaEfEy6SHGAJR7fZokK53us4I2xBy3TVTexm05iekhZ993PRbMGg5M7AWG/eyIODkx883DI4DsLxpd2vX+UbuXKoU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395133; c=relaxed/simple; bh=udw/SeGPA6CD6uaMzNqCLS4SxbhM3Wn0KnnJN2byGmk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FZKMVW0xSdrjy3AftTmpIrytGdG5HoTsy98smeT/gWcrNI5P4/clarfyl3boXekM91ftZjRVTIXkG+DQmPrKTslr1YrodMbCD8c7VlEYfWYNeU+Iuo1wAs39IV2164TU9zgQU/XGV5LhKMD6KxVKIOFs6NNQC24zXU7dtA/Qq54= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=Yv3FvX3w; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="Yv3FvX3w" Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-842288702fbso1145122b3a.1 for ; Tue, 02 Jun 2026 03:12:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395131; x=1780999931; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E/u6lKC7hQ/GhzoNdcnEA7gND1myfeYXkd1NgoaUr0Y=; b=Yv3FvX3w2n1f8d1IYrIRqmEFr5vn3emjtqadAOFJCS6oCSOhCIKluNEHo9UsMvyO1Q LxO86YuHsK4kD7XRmGLKs2WqmXtuOB6KR3FduwZQ9gZu82VjbT44yyWF+Jxr6TuXVfi6 Ha+Cj//tEGWVitOIG0dwJxNa6JMwWh3zMBlZWDN1RYzzLmMnOzJy7v7W8JzhEmqZfR7s sWGupvIFnnEcy370lqt3yfQ1Jct6JQsuRmYKK5cMJQkWMVarpJgVf6+c7087AWhBcJWQ a4lXM7/9hxUSvnN1gLJARN88O4QKajMPXO4kOCUly3VniUJbCTmIO5Lb9Hj1Dw4bcOtC 41mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395131; x=1780999931; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=E/u6lKC7hQ/GhzoNdcnEA7gND1myfeYXkd1NgoaUr0Y=; b=EsBJXeIrBGSydHKZ5e8I5u9It6d8u76xqA91G2miZXN433eS7knq4tjioGY+b5aCSM jBfEtvLebZY7B4p/dF16s+1o7f+c8dnFzCU+JdPYfUZHH7DOXjEtdKIO2DgoOQfiUZ7l J6TmihONygfv5iCf/UZzj3xqsB3ehK4DXJRLgunUSpV2cjbCuRrcQC+4ORwfONCl0e+N S3kbYxAGlvPMG5YJNmxOYBqGf9plb+tCb8+PXuzxiY7EUhHtdlEY3F0WUwSDAyV9zP+3 25j2c64VwxqPAylA+pYlStL9PyevnFMctA0H63Etk+eTEHMeNKLTTfljIh1p0CF0v+1L xu6w== X-Forwarded-Encrypted: i=1; AFNElJ9X2XkIW+LRbWnvvtl7sktzPHRqfAsCN7BM6b7zDAXLj5/ToKVIxsfMseJ67UbmIIDraJ8ZTCPROaPmn4U=@vger.kernel.org X-Gm-Message-State: AOJu0YzDbLbAbF0eDwpCAolg6JNpDDyEAT+f+1pGd0gLU0xbBdSvjsuK xWFv3FzZv9kLeTC7y72I0ToUNlZ3NhVeWRwAUapq+Ck9h2bQYQgcnsANAgV85c3x/X0= X-Gm-Gg: Acq92OGTp7ZjvbLr7RzHUcEZf+Do6lnMCy2ZbKmW2Ng9jbOsAc2MmCRime2fHYPYLCJ J/+s7YLlubQnoRmG/Jw2v+6CZbXQAwACZOaR6na2YOBiYWXpyG1PIGFdnbSlmg3jgsuAET/j/YR +y7/eSgFyYGDaKjDrv8ZrYhEYganXzPcgFdNGTYpEm5AgyzaHpdTg/GQpIiL6Cp+NayMl26T8fj IzFFfamsYbfHAEyb3Y6KNDXvyfSXVOJ+piLUhdJSIzGrhzCHZIg4S3X97fV30u7MlrWdxeqvpLL r8iZLPVNM8WfxCb077l+bvonZJ5zWgZxDT7z3O1OABXDjySIuFQMy6+1qktZ4mASeIIGet/LoOl rQSswGEtnHCQ1/jW8An5xhYx4j+6+zT6iBaOwVa07E8yHps3ha/97FujPxPXC/Hw96pamxVLQAy jvxwpR+X+CNlUR0STgGx9U0EhOveHc8tlVLFxE7Pk1jxDE9sQw2Z3uUIk= X-Received: by 2002:a05:6a00:2da7:b0:842:7a78:8f14 with SMTP id d2e1a72fcca58-8427a789a5cmr998275b3a.6.1780395130964; Tue, 02 Jun 2026 03:12:10 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.12.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:12:10 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 14/19] mm/hugetlb: Free cross-zone bootmem gigantic pages after allocation Date: Tue, 2 Jun 2026 18:10:34 +0800 Message-ID: <20260602101039.1867613-15-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that hugetlb reservation runs after zone initialization, bootmem gigantic page allocation can detect pages that span multiple zones. Keep those cross-zone pages separate during allocation and free them after allocation completes, so later hugetlb initialization only sees zone-valid gigantic pages. This chooses to free cross-zone gigantic pages directly instead of retrying allocation. In practice, such cross-zone cases are expected to be very rare, so adding retry logic does not seem justified at this point. Keeping the handling simple also preserves the previous behavior. If similar real-world reports show up later, retry support can be reconsidered then. Signed-off-by: Muchun Song --- mm/hugetlb.c | 75 ++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 64 insertions(+), 11 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 5e557c05d80a..218fb1ca45f4 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3060,12 +3060,15 @@ void *__init __alloc_bootmem_huge_page(struct hstat= e *h, int nid) =20 static bool __init alloc_bootmem_huge_page(struct hstate *h, int nid) { + unsigned long pfn; + unsigned int nid_request =3D nid; struct huge_bootmem_page *m =3D arch_alloc_bootmem_huge_page(h, nid); =20 if (!m) return false; =20 - nid =3D early_pfn_to_nid(PHYS_PFN(__pa(m))); + pfn =3D PHYS_PFN(__pa(m)); + nid =3D early_pfn_to_nid(pfn); /* * Use the beginning of the huge page to store the huge_bootmem_page * struct (until gather_bootmem puts them into the mem_map). @@ -3073,22 +3076,38 @@ static bool __init alloc_bootmem_huge_page(struct h= state *h, int nid) * Put them into a private list first because mem_map is not up yet. */ INIT_LIST_HEAD(&m->list); - list_add(&m->list, &huge_boot_pages[nid]); m->hstate =3D h; if (!hugetlb_early_cma(h)) { m->cma =3D NULL; m->flags =3D 0; } =20 - /* - * Only initialize the head struct page in memmap_init_reserved_pages, - * rest of the struct pages will be initialized by the HugeTLB - * subsystem itself. - * The head struct page is used to get folio information by the HugeTLB - * subsystem like zone id and node id. - */ - memblock_reserved_mark_noinit(__pa((void *)m + PAGE_SIZE), - huge_page_size(h) - PAGE_SIZE); + /* CMA pages: zone-crossing is validated in hugetlb_cma_reserve(). */ + if (!hugetlb_early_cma(h) && + pfn_range_intersects_zones(nid, pfn, pages_per_huge_page(h))) { + /* + * If the allocated page is on a different node than requested + * (e.g., on PowerPC LPARs), put it on the requested node's list, + * because hugetlb_free_cross_zone_pages() only frees cross-zone + * pages belonging to the requested node. + */ + if (WARN_ON_ONCE(nid_request !=3D NUMA_NO_NODE && nid !=3D nid_request)) + list_add(&m->list, &huge_boot_pages[nid_request]); + else + list_add(&m->list, &huge_boot_pages[nid]); + } else { + list_add_tail(&m->list, &huge_boot_pages[nid]); + m->flags |=3D HUGE_BOOTMEM_ZONES_VALID; + /* + * Only initialize the head struct page in memmap_init_reserved_pages, + * rest of the struct pages will be initialized by the HugeTLB + * subsystem itself. + * The head struct page is used to get folio information by the HugeTLB + * subsystem like zone id and node id. + */ + memblock_reserved_mark_noinit(__pa((void *)m + PAGE_SIZE), + huge_page_size(h) - PAGE_SIZE); + } =20 return true; } @@ -3373,6 +3392,34 @@ void __init hugetlb_bootmem_struct_page_init(void) padata_do_multithreaded(&job); } =20 +static unsigned long __init hugetlb_free_cross_zone_pages(struct hstate *h= , int nid) +{ + unsigned long freed =3D 0; + struct huge_bootmem_page *m, *tmp; + + if (!hstate_is_gigantic(h)) + return freed; + + list_for_each_entry_safe(m, tmp, &huge_boot_pages[nid], list) { + if (m->flags & HUGE_BOOTMEM_ZONES_VALID) + break; + + list_del(&m->list); + memblock_free(m, huge_page_size(h)); + freed++; + } + + if (freed) { + char buf[32]; + + string_get_size(huge_page_size(h), 1, STRING_UNITS_2, buf, sizeof(buf)); + pr_warn("HugeTLB: freed %lu cross-zone hugepages of size %s on node %d.\= n", + freed, buf, nid); + } + + return freed; +} + static void __init hugetlb_hstate_alloc_pages_onenode(struct hstate *h, in= t nid) { unsigned long i; @@ -3403,6 +3450,8 @@ static void __init hugetlb_hstate_alloc_pages_onenode= (struct hstate *h, int nid) cond_resched(); } =20 + i -=3D hugetlb_free_cross_zone_pages(h, nid); + if (!list_empty(&folio_list)) prep_and_add_allocated_folios(h, &folio_list); =20 @@ -3476,6 +3525,7 @@ static void __init hugetlb_pages_alloc_boot_node(unsi= gned long start, unsigned l =20 static unsigned long __init hugetlb_gigantic_pages_alloc_boot(struct hstat= e *h) { + int nid; unsigned long i; =20 for (i =3D 0; i < h->max_huge_pages; ++i) { @@ -3484,6 +3534,9 @@ static unsigned long __init hugetlb_gigantic_pages_al= loc_boot(struct hstate *h) cond_resched(); } =20 + for_each_node(nid) + i -=3D hugetlb_free_cross_zone_pages(h, nid); + return i; } =20 --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 279163DA7FF for ; Tue, 2 Jun 2026 10:12:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395139; cv=none; b=cNl6QQaVeAyj6CFbf9qzWPbthG29GoRn+FqIAJh9sIZyOgVBpfed2uGVDhbIlS8on2z/WYo0wVN2ekvXeAY6r9fUgHSm06nfT89ZL7SE7SpxasJ6sEoo/yEY0It2a0hmDt43io+hE1KdLz/qju2VUAY4lwLSU/uXlGOMxzTBWV8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395139; c=relaxed/simple; bh=QxooAqDLcZijFwCaQ33+GIDEUN8jnYpNUv7liuynEkU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XEQh+ZzG7B52eFODqXkwy+BpIqPim3v4jaCiFuz1E8WHAjFAcIEPkVmM586ZmkuM1CbZmciGvxfZsNLu1QA5Y8Z84gKw3jg7JK8jFOh8d7SXSpmd/KZFwiJjocKPKtSXepL4bQ232/YY7RWlf8De5qbz0yXucIOJJMOfN9/yqoY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=TZS7+wCr; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="TZS7+wCr" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-8423f236418so1016869b3a.1 for ; Tue, 02 Jun 2026 03:12:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395136; x=1780999936; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oJaZoYtZ6J61gHq76uPvOqmXDVgTt0Xo6SthcejktT8=; b=TZS7+wCr3yoO02DpDi7Xp2nYEssIBWWScwX0oDVeM5SixSw8oXN5QbPEtNYCSMGFCv U/L6h9YpPjkqiPmhhs6DRjPOu3u1r8YLeHAi5qKcaes/RJtrxeIc9qEvcN3NcxHtQwsN jviNNZd6ygKdiLghxyr/v9KmCh8GM3L+Boi/bolFHdptaoyiZbZnlK7/qMBfRe7VTWeZ P4AjcYKQGUGNCaSeJLsDXePzkrhOI+M4i6X43euvTIkHQXrvlLtFFkBMLudq/UNkMwX5 2OwdUtNUlOjZevC4nfLDlD/moDHQH5A9SejGx2t1ZwenBglNVZvVzidnqBI8wy5H5F5f 6Ncg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395136; x=1780999936; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=oJaZoYtZ6J61gHq76uPvOqmXDVgTt0Xo6SthcejktT8=; b=I/YpIgVPjDroHq/C1Q9ExTOXg6UXfhNyoITDd/nhBOpticakJ5K+9clYMCyEnCzcqo 8gN/wuw/LBF0Y/E6ovdzSpDzOJ/MwTE6jcODcFP6hCly+aDlCX8TCMesUhvRe7Z3I2t0 +9nnyJ8B3rT4cV8OS/oA3r+QdXLclNjc2NaxRQ4fNPLTa5ybLOd7yRz4GwsKhzJlBPS8 XoKXk8wfok9vM5pcDZwCurYunyREGOwQVZDAn8ks/KgmMcAWFqDXBDgQjXvS0buFqHPK eda/wRIOFTZILsvcoLj1M0/3qg3W6U4NBYvYCMhn8mXCqvyA4Zll4pmfPFFj1pTOIOAw O7zQ== X-Forwarded-Encrypted: i=1; AFNElJ+i5Jqex9CFgAKOZA1gJBvXnisoSsBy4qg/84VO4TJ2AaW7aNoHKRxR7mqDoBlMn6uaN4LfXOCKHlnTCRM=@vger.kernel.org X-Gm-Message-State: AOJu0YwUM4NAe99kePDahMkFt41cOWJSafd7DIzOSBFfO0rDm8FN1sXO VhMD6/8VT7eiq5C9+2CLXZ34aVaI4Fy5dAIbQ1BPgErdrCOGioCz4xXdRrDASUn/Fys= X-Gm-Gg: Acq92OFE73AkslntrhPC9WDmB9O4UTiN5yHvl2aMt+Vh9MuzhsTHC4ZQtzNOqGwOau7 QWX1i15pibuetsy0pNg0r+lJAXhadzQAcsVe8nTyBrT3QKV4tEFjcbBtZHncQ3astKtuA6sGcsF XwbbOlL7LQvx/1JPacSr2gsUsARuWlYOzrg50m4pLpgHFFsqbkbG4zsLgPs4pVy6v9vpsIe+Tit itML6Da6dkd41NzgEvWY8SK7JDqe5zpgiBMpbmJXkeocmX4jJRq6whKoXlppXt4rwwM/oXBlsOP 0DxpPYCFww8zkTtoUQthTZLGhv4XumZvP/r8KzXzlXHG+CGhv1FM33itGskek99ZDAcFE0uy/jG mXe6jHcVrxZ7J1GO2Wie/txeUhL27iHgPHtDAjZeIW8f0zGqFnSHMFygEEz0dovvkkC0pckDXeV cCWqIJVIbaN/01b/spMaq1kV1Y19VIJpPlitu7dfQrG6vvqKnOKAgMti4= X-Received: by 2002:a05:6a00:b806:b0:842:655d:1232 with SMTP id d2e1a72fcca58-842655d14a9mr4627780b3a.0.1780395136206; Tue, 02 Jun 2026 03:12:16 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.12.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:12:15 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 15/19] mm/hugetlb_vmemmap: Move bootmem HVO setup to early init Date: Tue, 2 Jun 2026 18:10:35 +0800 Message-ID: <20260602101039.1867613-16-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Bootmem HugeTLB pages currently defer HVO setup to hugetlb_vmemmap_init_late(), because the optimization needs zone information. Now that zone initialization is available earlier, the bootmem HVO setup can be done directly from hugetlb_vmemmap_init_early(). This lets gigantic HugeTLB pages apply HVO as soon as they are allocated. Bootmem gigantic pages that span multiple zones are now filtered out when they are allocated, so the remaining bootmem gigantic pages seen by later hugetlb initialization are already zone-valid. As a result, hugetlb_vmemmap_init_late() no longer needs to handle bootmem HVO setup. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) Acked-by: Usama Arif --- mm/hugetlb_vmemmap.c | 67 +++++++++----------------------------------- 1 file changed, 13 insertions(+), 54 deletions(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index ea6af85bfec1..464578ee246e 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -745,6 +745,8 @@ static bool vmemmap_should_optimize_bootmem_page(struct= huge_bootmem_page *m) return true; } =20 +static struct zone *pfn_to_zone(unsigned nid, unsigned long pfn); + /* * Initialize memmap section for a gigantic page, HVO-style. */ @@ -752,6 +754,7 @@ void __init hugetlb_vmemmap_init_early(int nid) { unsigned long psize, paddr, section_size; unsigned long ns, i, pnum, pfn, nr_pages; + unsigned long start, end; struct huge_bootmem_page *m =3D NULL; void *map; =20 @@ -761,6 +764,8 @@ void __init hugetlb_vmemmap_init_early(int nid) section_size =3D (1UL << PA_SECTION_SHIFT); =20 list_for_each_entry(m, &huge_boot_pages[nid], list) { + struct zone *zone; + if (!vmemmap_should_optimize_bootmem_page(m)) continue; =20 @@ -769,6 +774,14 @@ void __init hugetlb_vmemmap_init_early(int nid) paddr =3D virt_to_phys(m); pfn =3D PHYS_PFN(paddr); map =3D pfn_to_page(pfn); + start =3D (unsigned long)map; + end =3D start + hugetlb_vmemmap_size(m->hstate); + zone =3D pfn_to_zone(nid, pfn); + + if (vmemmap_populate_hvo(start, end, huge_page_order(m->hstate), + zone, HUGETLB_VMEMMAP_RESERVE_SIZE)) + panic("Failed to allocate memmap for HugeTLB page\n"); + memmap_boot_pages_add(DIV_ROUND_UP(HUGETLB_VMEMMAP_RESERVE_SIZE, PAGE_SI= ZE)); =20 pnum =3D pfn_to_section_nr(pfn); ns =3D psize / section_size; @@ -800,60 +813,6 @@ static struct zone *pfn_to_zone(unsigned nid, unsigned= long pfn) =20 void __init hugetlb_vmemmap_init_late(int nid) { - struct huge_bootmem_page *m, *tm; - unsigned long phys, nr_pages, start, end; - unsigned long pfn, nr_mmap; - struct zone *zone =3D NULL; - struct hstate *h; - void *map; - - if (!READ_ONCE(vmemmap_optimize_enabled)) - return; - - list_for_each_entry_safe(m, tm, &huge_boot_pages[nid], list) { - if (!(m->flags & HUGE_BOOTMEM_HVO)) - continue; - - phys =3D virt_to_phys(m); - h =3D m->hstate; - pfn =3D PHYS_PFN(phys); - nr_pages =3D pages_per_huge_page(h); - map =3D pfn_to_page(pfn); - start =3D (unsigned long)map; - end =3D start + nr_pages * sizeof(struct page); - - if (!hugetlb_bootmem_page_zones_valid(nid, m)) { - /* - * Oops, the hugetlb page spans multiple zones. - * Remove it from the list, and populate it normally. - */ - list_del(&m->list); - - vmemmap_populate(start, end, nid, NULL); - nr_mmap =3D end - start; - memmap_boot_pages_add(DIV_ROUND_UP(nr_mmap, PAGE_SIZE)); - - memblock_phys_free(phys, huge_page_size(h)); - continue; - } - - if (!zone || !zone_spans_pfn(zone, pfn)) - zone =3D pfn_to_zone(nid, pfn); - if (WARN_ON_ONCE(!zone)) - continue; - - if (vmemmap_populate_hvo(start, end, huge_page_order(h), zone, - HUGETLB_VMEMMAP_RESERVE_SIZE) < 0) { - /* Fallback if HVO population fails */ - vmemmap_populate(start, end, nid, NULL); - nr_mmap =3D end - start; - } else { - m->flags |=3D HUGE_BOOTMEM_ZONES_VALID; - nr_mmap =3D HUGETLB_VMEMMAP_RESERVE_SIZE; - } - - memmap_boot_pages_add(DIV_ROUND_UP(nr_mmap, PAGE_SIZE)); - } } #endif =20 --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3350D3DB339 for ; Tue, 2 Jun 2026 10:12:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395143; cv=none; b=a6TctVpPs1jMe8Wzzam2IvnZvoDOexBa8NF5lIywE/pBRCix9Eu6DwbMi3BJwuxSSAjjS2bkenFuYRD+TCmWw1R10+uLCABmaN4QJ9cmU0XZ475QXhlxS/z+Nfs1vHt8H5Dccg+fof0M2xqWijRM2wPvl2h/2dZKj3ukzbJfLms= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395143; c=relaxed/simple; bh=4euGeIhA0G5fwYuiFPTrlspUhDT24nNRsfmE7y0FOzM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nP7IQsM9wiWyC6v+/PskuO4G5NzYFmh1XAfux3h/4DDD2XQSiD48+AE2tPWwS4aTk39WEsXZrZusZxA95ZbcGqMN9vZzPEVGLJi0p4xtwL+IdSHbmLvQiN9lTP04Tydz8sW8kcC/hJq5kZUTRn77f2enSILU+uWrFBoC8qUdQC8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=bCAcneo3; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="bCAcneo3" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-84237c55ef9so1009684b3a.0 for ; Tue, 02 Jun 2026 03:12:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395141; x=1780999941; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aqtFBpfBq6FrNHW9ccKxFbBIkIWdyfnMbN3qSInC4vE=; b=bCAcneo3gIgF/fXoYUY/Zxe6fihL1pqwm8SbyjAScOJL/F1xw8hvthnf0MUI0SVPQ4 DK6I0Jo8UtREbdMuZG7Gz1YCQk7YxqG48DJekLZ8LBHfBxXrq6WChAeQubSeJ2e4PvQo K6vnDzUkYQOV9NBX4Jd9XsM2EPUaNGE9BXYWULXqe3MEGW2/adbHejD1sT4K1RRedavU 48xg5EzhAwXHl7bGUoQeYbY937YT/B4BLAvmuNPOAnCZJ0MXmNilmdB2YW0WlE2Z2gtL F/mN30x++E1+7je+zuocWaPMCAF5UsvTypFJdE/uHclrnxOV//04IS7Zgtn61eZLY8WW MoJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395141; x=1780999941; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=aqtFBpfBq6FrNHW9ccKxFbBIkIWdyfnMbN3qSInC4vE=; b=k4LUgJh158Ac9j3BpmZLcjTjyW5YbT599f1UKfXYM8nf0z9YkoKYx4iKN6sTm2Xz0m yO6ypgYLr8gDMgXb3jPehWkgAr42tXrQ/vZNb1foBWrap0fpqT5bRnMy3AIC2naV6wa3 gL07TqjkOW/IWfV0/3LPKE7vABFviPHQ7Rzd7zt5k7fab5O2CLVo4boBCa3Yi8uUlnmP Mv8fP5cxR+amjWpDZQVYvgB6jR3iJ2w3vHjujwnZjva1pQqRZa1hK7JmyisfspwrliMs 6Ndm4P9+pFIlHy4v/lKG9g6kqJvSZcswLOQKeQGP8LVdyJdmIEA53pheYHRcl0Otuglv /eww== X-Forwarded-Encrypted: i=1; AFNElJ8ISa20nthomzzCPkIDSw+wXu69qMyz1EWnjMHo1CqCqWHtbjOVa4h7kj7aOu29RiegU0RfYpqzPsuCurw=@vger.kernel.org X-Gm-Message-State: AOJu0YwiNQUdDfAYz7MDZs0VIjF3Ze7zewo/lEevo6UM3a2RwcG0jbkl RnLAKD6WdxwyAxf0YLZqXowiaTKJzH1fzkpEq26uOCa6UwZX2GrGArSskTUwx1K8SOs= X-Gm-Gg: Acq92OFwrP8VUQM/fcYeOdM7kZd4qCR/CyA4574RCLnUoxkUOWMN8TfNcAq0AsZlxYp b9PPb2aNLLY6u1TICbMzE85g+Z1Y/Gs8nejkjqhS9MKAtiNv4K8VO/AQqhHDv4HpHUIWMVAvaY4 wwg5YvdsFAEmlEVPJmM4BHBaDq5RUnkFEEsLEWV6T9PLyhdiEdtNazSPuvo01VKIxNsLwM97h9L tUt6Bt5DdeAw5nfcX2EcMalwgw4W3cQdDPuOKs6bjTC2giC9DXcK+6ANsTYiqaOKCfruhFUfIzh 7ZWr06gEW4WF9eEE5PAzVsV13JcOILyNnfi6uYAWztep6mwVg6BCHjwBjGP/rd5TTGh6/bPDR8s 8ogVFx94SiuQEJmTBYAQ8zyAkFP3bL4fEbfLC0bQL5pDGigAU82O9um3zQqCluVedRmi8QW1JmC TValmNUYr6y+O8VZygdaiJdVBOJk06jT6s5VeP40kM3lcK9+ENGbWODQ8= X-Received: by 2002:a05:6a00:92a9:b0:842:6f13:1461 with SMTP id d2e1a72fcca58-8426f131bffmr2620970b3a.11.1780395141260; Tue, 02 Jun 2026 03:12:21 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.12.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:12:20 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 16/19] mm/hugetlb: Remove obsolete bootmem cross-zone checks Date: Tue, 2 Jun 2026 18:10:36 +0800 Message-ID: <20260602101039.1867613-17-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Bootmem gigantic HugeTLB pages used to be validated again during gather_bootmem_prealloc_node() and any cross-zone pages were discarded there. That validation is no longer needed. Cross-zone bootmem gigantic pages are now detected during allocation and freed before they reach the later bootmem gathering path, so the remaining pages are already zone-valid. Remove the obsolete cross-zone validation, invalid-page freeing, and the associated discarded-page accounting. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- include/linux/hugetlb.h | 3 -- mm/hugetlb.c | 70 ----------------------------------------- 2 files changed, 73 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 09f28dd773b7..f68a390d43bd 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -678,9 +678,6 @@ struct hstate { #define HUGE_BOOTMEM_ZONES_VALID 0x0002 #define HUGE_BOOTMEM_CMA 0x0004 =20 -struct huge_bootmem_page; -bool hugetlb_bootmem_page_zones_valid(int nid, struct huge_bootmem_page *m= ); - int isolate_or_dissolve_huge_folio(struct folio *folio, struct list_head *= list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); void wait_for_freed_hugetlb_folios(void); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 218fb1ca45f4..47c3d6d11c58 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -58,7 +58,6 @@ struct hstate hstates[HUGE_MAX_HSTATE]; =20 __initdata nodemask_t hugetlb_bootmem_nodes; __initdata struct list_head huge_boot_pages[MAX_NUMNODES]; -static unsigned long hstate_boot_nrinvalid[HUGE_MAX_HSTATE] __initdata; =20 /* * Due to ordering constraints across the init code for various @@ -3221,57 +3220,6 @@ static void __init prep_and_add_bootmem_folios(struc= t hstate *h, } } =20 -bool __init hugetlb_bootmem_page_zones_valid(int nid, - struct huge_bootmem_page *m) -{ - unsigned long start_pfn; - bool valid; - - if (m->flags & HUGE_BOOTMEM_ZONES_VALID) { - /* - * Already validated, skip check. - */ - return true; - } - - if (hugetlb_bootmem_page_earlycma(m)) { - valid =3D cma_validate_zones(m->cma); - goto out; - } - - start_pfn =3D virt_to_phys(m) >> PAGE_SHIFT; - - valid =3D !pfn_range_intersects_zones(nid, start_pfn, - pages_per_huge_page(m->hstate)); -out: - if (!valid) - hstate_boot_nrinvalid[hstate_index(m->hstate)]++; - - return valid; -} - -/* - * Free a bootmem page that was found to be invalid (intersecting with - * multiple zones). - * - * Since it intersects with multiple zones, we can't just do a free - * operation on all pages at once, but instead have to walk all - * pages, freeing them one by one. - */ -static void __init hugetlb_bootmem_free_invalid_page(int nid, struct page = *page, - struct hstate *h) -{ - unsigned long npages =3D pages_per_huge_page(h); - unsigned long pfn; - - while (npages--) { - pfn =3D page_to_pfn(page); - __init_page_from_nid(pfn, nid); - free_reserved_page(page); - page++; - } -} - /* * Put bootmem huge pages into the standard lists after mem_map is up. * Note: This only applies to gigantic (order > MAX_PAGE_ORDER) pages. @@ -3287,17 +3235,6 @@ static void __init gather_bootmem_prealloc_node(unsi= gned long nid) struct folio *folio =3D (void *)page; =20 h =3D m->hstate; - if (!hugetlb_bootmem_page_zones_valid(nid, m)) { - /* - * Can't use this page. Initialize the - * page structures if that hasn't already - * been done, and give them to the page - * allocator. - */ - hugetlb_bootmem_free_invalid_page(nid, page, h); - continue; - } - /* * It is possible to have multiple huge page sizes (hstates) * in this list. If so, process each size separately. @@ -3692,20 +3629,13 @@ static void __init hugetlb_init_hstates(void) static void __init report_hugepages(void) { struct hstate *h; - unsigned long nrinvalid; =20 for_each_hstate(h) { char buf[32]; =20 - nrinvalid =3D hstate_boot_nrinvalid[hstate_index(h)]; - h->max_huge_pages -=3D nrinvalid; - string_get_size(huge_page_size(h), 1, STRING_UNITS_2, buf, 32); pr_info("HugeTLB: registered %s page size, pre-allocated %ld pages\n", buf, h->nr_huge_pages); - if (nrinvalid) - pr_info("HugeTLB: %s page size: %lu invalid page%s discarded\n", - buf, nrinvalid, str_plural(nrinvalid)); pr_info("HugeTLB: %d KiB vmemmap can be freed for a %s page\n", hugetlb_vmemmap_optimizable_size(h) / SZ_1K, buf); } --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A82A3DB339 for ; Tue, 2 Jun 2026 10:12:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395150; cv=none; b=SsYIXSdGZcHy447A5KJFog2MnGGaGFgSnRMwN2GahVTE3u6adY6sA/GQsmJdsTqYq44qub4pxt4RR8JG/Vm5dFlXb+Dw3yd7qbNkzPYGEzk9WIpS4NF2SPa3185gae7W/CjvTcZWFFyBGn7IuWMlU8QJBoEYqOZW4hBFgejGj8I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395150; c=relaxed/simple; bh=1T8j5kD2dBSggKbtwOLW/8J1vgDVh2H5v4T3x5/pK+E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ScbqEMdE7AjCTCLvg8161QAzrczflf89NN+1k884oI9Bgik6pNJ7T1DzV6Y6iQ3nfesANJThYP42bHuVhDPQ9UdRWmIbyvXZtbjordo3t4o/IqjCPPwKXnukj9K9DgDIRDe68B6pZtvyWfoUF/2CxdP69yShWV5GmrhZAeVTZLE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=PALS8+8B; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="PALS8+8B" Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-8419ab3a297so3540203b3a.2 for ; Tue, 02 Jun 2026 03:12:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395146; x=1780999946; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=26MXCqLFrwb2c3VcF20KKUb5g0TkMABKfAG/bHmkXiE=; b=PALS8+8Br/u+rl2RadYAg/wxjlShkkR0mQTfwKrB4m+lOxvw/6IUyGgNlLOE4md82B Cl0gTr0dsdx2mvzBOGqhIYQIP5QdoBDLb1f72I/sGW8+/vXWqFRRNrfLyRkHXh50X2p+ t/Zv9x2T8fLjj3N/urKV9EHSkf2q58rTI13O3A+i+NtGXUCZAdf8+RH2FSSPij81UoJZ hVMUAFYhGGefiajIxPCl4FN727stmKohhkuD3DqraEERwsLMQjfxcXBzEPFKGXbclAhA spLKtOF/sWvcugheq8p1yONzBxBx/SlouwtBDDhiP2d2FS5126VLnfPro00wp3jH8eZ7 tQgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395146; x=1780999946; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=26MXCqLFrwb2c3VcF20KKUb5g0TkMABKfAG/bHmkXiE=; b=lVh9wOiyjcJowLQwPPRS9jv/YGLCJ4UiUgg65zNkWgWOlhN8zawcj4TR1C7L3SSbl9 jB53wneiq73X/cCCDw3H8P4yfY/vLSV/D2fWtsFilakZegDWcdspmVRDtINyHRxNs1I7 jP2agLFTUyjO41PXvtu0EFknb+4d5qK3mbiet8IXRvpsZDLqKvdYbA42nf0gogB7yQ2S pQzSZ/6CuCQvYLCAE4c4qrci/2/whgFYLT5ZHK/DZ7xrcjEgXdd/mDrDisLRprt0g0VO DCcC/U1i3LomRn+uf0u3gUMg0WXDsJXZxcGn1/B3ftSnLi/0OylHYbs7ZGcNx5PmFIH+ IkyQ== X-Forwarded-Encrypted: i=1; AFNElJ+IRPxm0YqObgMUF/MIdb6Av/X5Jx0a/uzdYHHB1vG4DKGDG7Tqzdu8ObMIYQ4J/n4KM0AD+9pBDEzQz64=@vger.kernel.org X-Gm-Message-State: AOJu0YwsVtpcFXFgLouhOUEneNzqjmkzXhSf5zvbbTPg1ax+tNcBKW1d ukuzvJNHETKBoCr/MZkUJdcV9P9qdz2QpkUNYwFDblz6Im2nllt6FeNzhypJy6U1dtg= X-Gm-Gg: Acq92OFhE9J3odyQOlngi/jB8EVJuv2/P5YRt8XBfHknwulq3lSvr5oGy9pkBdEOH7V VPNxM02IreKzIgYfp3WiN9aLtLg06g5N/jBExe/xmsZbLTnLaifGQd858YN13mkqkcsf+uaZi78 PqqgZZlaxT1OBgBm+9I9Aipq1T6dCoOFzqkJwMayTGMNwxnZfkqpkZEitk/H7MlqTHm9UEnLYgH cqOcSS7uOWJWqysaRcn42BuFFX9t3ZbljTUcE+TW4Ed4QbexBzNz1ytIuOMh8L//YujDgVqmvFm d8DH4rOBmICRXnMUpMCL7/bOICNCWX/FweyORWot67JXCLf9FclbzoXdSYARaraFDAZ3aY2HuCS y5FfsW13zEChr9fJhqNCXbAsUB+u5W2jfxKZUEyLfbyBdYNPjGhSr54WbNW79bVMLs97Q22ivW9 n+gvtZT4rjn5zZCNs0XdTHVown4X60ss1KR69l+fphX/fxFMJgPqCnQEo= X-Received: by 2002:a05:6a00:4c8b:b0:842:43d1:4b41 with SMTP id d2e1a72fcca58-84243d15007mr9808067b3a.47.1780395146453; Tue, 02 Jun 2026 03:12:26 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.12.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:12:25 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 17/19] mm/sparse-vmemmap: Remove sparse_vmemmap_init_nid_late() Date: Tue, 2 Jun 2026 18:10:37 +0800 Message-ID: <20260602101039.1867613-18-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" hugetlb_vmemmap_init_late() no longer has any users, so the remaining late-init path in sparse_vmemmap_init_nid_late() is dead code. Remove sparse_vmemmap_init_nid_late() and its declarations. Signed-off-by: Muchun Song Acked-by: Mike Rapoport (Microsoft) --- include/linux/mmzone.h | 7 ------- mm/hugetlb_vmemmap.c | 4 ---- mm/hugetlb_vmemmap.h | 5 ----- mm/sparse-vmemmap.c | 11 ----------- mm/sparse.c | 1 - 5 files changed, 28 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 1331a7b93f33..72883df17c72 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2170,8 +2170,6 @@ static inline int preinited_vmemmap_section(const str= uct mem_section *section) } =20 void sparse_vmemmap_init_nid_early(int nid); -void sparse_vmemmap_init_nid_late(int nid); - #else static inline int preinited_vmemmap_section(const struct mem_section *sect= ion) { @@ -2180,10 +2178,6 @@ static inline int preinited_vmemmap_section(const st= ruct mem_section *section) static inline void sparse_vmemmap_init_nid_early(int nid) { } - -static inline void sparse_vmemmap_init_nid_late(int nid) -{ -} #endif =20 static inline int online_section_nr(unsigned long nr) @@ -2388,7 +2382,6 @@ static inline unsigned long next_present_section_nr(u= nsigned long section_nr) =20 #else #define sparse_vmemmap_init_nid_early(_nid) do {} while (0) -#define sparse_vmemmap_init_nid_late(_nid) do {} while (0) #define pfn_in_present_section pfn_valid #endif /* CONFIG_SPARSEMEM */ =20 diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 464578ee246e..cde6f3aba87b 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -810,10 +810,6 @@ static struct zone *pfn_to_zone(unsigned nid, unsigned= long pfn) =20 return NULL; } - -void __init hugetlb_vmemmap_init_late(int nid) -{ -} #endif =20 static const struct ctl_table hugetlb_vmemmap_sysctls[] =3D { diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index 18b490825215..7ac49c52457d 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -29,7 +29,6 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, st= ruct list_head *folio_l void hugetlb_vmemmap_optimize_bootmem_folios(struct hstate *h, struct list= _head *folio_list); #ifdef CONFIG_SPARSEMEM_VMEMMAP_PREINIT void hugetlb_vmemmap_init_early(int nid); -void hugetlb_vmemmap_init_late(int nid); #endif =20 =20 @@ -81,10 +80,6 @@ static inline void hugetlb_vmemmap_init_early(int nid) { } =20 -static inline void hugetlb_vmemmap_init_late(int nid) -{ -} - static inline unsigned int hugetlb_vmemmap_optimizable_size(const struct h= state *h) { return 0; diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 3b036251a2f4..077686af394b 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -574,17 +574,6 @@ void __init sparse_vmemmap_init_nid_early(int nid) { hugetlb_vmemmap_init_early(nid); } - -/* - * This is called just before the initialization of page structures - * through memmap_init. Zones are now initialized, so any work that - * needs to be done that needs zone information can be done from - * here. - */ -void __init sparse_vmemmap_init_nid_late(int nid) -{ - hugetlb_vmemmap_init_late(nid); -} #endif =20 static void subsection_mask_set(unsigned long *map, unsigned long pfn, diff --git a/mm/sparse.c b/mm/sparse.c index 3917a47153d8..324213d8bdcb 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -320,7 +320,6 @@ static void __init sparse_init_nid(int nid, unsigned lo= ng pnum_begin, } } sparse_usage_fini(); - sparse_vmemmap_init_nid_late(nid); } =20 /* --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A5232C11E2 for ; Tue, 2 Jun 2026 10:12:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395156; cv=none; b=MXilKObCjK8C9sKWfx+OYJLNfOnJggtCuIydSVCrUF1g02Espf+EiGY+UjhLA7d0I7XHLothMAK2CNbXO2MLiuUKe7g097WD61fJHQajhof6cp5jPpXexOD+14WJOOGrs0E7dTYDOpqVgQiBzHOtiXuroMm4kgwazw/AcJU4wnk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395156; c=relaxed/simple; bh=RULGxljhycusfKnRNN/CCMkodyXxkrbyshdNT0VgxQA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aDtmtu0tE4BusTT0vIw6mwAhg1nS0R282ti2ieCeFLlZiL6C4aL5QMozw5QsdpVOeapIU6zihuRMeGCQ+41Q4BDEa8JzvhVSatRhHUPX/8FC0QsRPIudfEQDavoCQjmFyFHU0qCqlMqfZoSiQLHQZfALnfdKaIc8iCDPl2oFO54= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=U77kGtIY; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="U77kGtIY" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-8423610ec93so1970124b3a.2 for ; Tue, 02 Jun 2026 03:12:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395152; x=1780999952; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1RCwi6UfDOd+MCdFxykpsofvVeescXETLa8L+kiLjU4=; b=U77kGtIYCC4Zwm7K0HD4VKaWPnkdO4qxEtwHqvOEBpQS1oJ32XObpSr8HfT3e63jaJ xCbrYP1yuGgWMvuWESNoJurszUsiBpI3bQh6T5SyJXUrGFncHWMNXQzUtbOEIjDV4Z39 gM1ZhRapu850LqDCBLGaZF0ehTa3IhH8hLPKqbZgm8K9HaRy88X2gDH4NMFAuh3Nyb8x DIz9/epeBtoOk1EYO8q4vTs6I0EcjS5qkI5QHwbY1lYo4XNwRzqnGCrNObpVY1BwXbQ2 z1Z2i3PhcON4DLHYRj0G46btm6mm7lPIMuZjgl7XJiUepcn3ILWBn00cedvQG6ZnCvsk ji4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395152; x=1780999952; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=1RCwi6UfDOd+MCdFxykpsofvVeescXETLa8L+kiLjU4=; b=gmwcES383uXOsHciBx+ECZNbCLBH4X9WOc2emJFKkhlYJKVi2X5u68aoS65m/gKWHE jLMSMhzC+nz3hcqYUY/8VNivKPXfZVz+79+muZJkNNCBL40EpwgmCT3gs3H6ngK0v1cF 185sRR0wqTER+lKo+wtwygEnagqcNVrxp+cmQfatONMXdAby6BWIIcJSUf5ihZ58xwQR FfohVkjUAZ2j0aCGYNFpp9jo3vUJoCKDaUHD3SNZ9V1pqila8fUymwBQ+Pn0zXiVgLvQ IHHfrRToHCOmoIWn7X31Gp0+JARLeuCl0NNr8isVvvHp+TLFzylL7x8njSV0rlA1fyfY iEnA== X-Forwarded-Encrypted: i=1; AFNElJ9vTf6Og/1ZPZdXjsLvZBT0lWDFKTaBsI6SZMjfiE4vgNcxvIGeB1g26JDXnc78g/Jhvza1fA/+J38c0+c=@vger.kernel.org X-Gm-Message-State: AOJu0YyGcrWJ29QDhaIXu3EBvlMnh4l46G+UcPJEssBE/28gVw0hnTzs YX0HKv45RR+EmyIK5dokZRn3ySnD4ZnsOcZRRCAQYOn+8hEEOkcE6FQuZ936IJdqHPs= X-Gm-Gg: Acq92OH2xdAS2paB+hjt3ySxsDRLKY4949i0SBI9x6Ik3w4s3hDWuDq5sDYb0sE00MH HQq5mxxCMgOzGB7XknnYa9o349BJwLujkNJisMe9hyZX4hKZIySbrlbGx7ud7WNhdkeSIGKTBQY P5Fr0n9wfuYoUfgERDaVs2z0Yj/WU260QYjp9qxTs7C5yvVV+sJB0VQEUg6suns2Rb2kV5W+sC8 ieaMunh+PMnshNMQBUoMqnCj6FoJatbUznblpK5CEOAVWW8SHa5OI7mtSPd/N1SMd/vatmoH+hE MvqxAg0WAq9JCxsaEo5zt+nAKgzMJ3LRPEmpOxN7uUkCwW6h0P8ZQlljbcCHxFdQ8OwF+ZRqahN UO56DZK5EEc08nMIcAH2ZrGFrqwJcjbToYus7Jf2xVa47sCnOUvvf8mFE8NKvcAJfgW2SYVZG6X RiYxbRvm+pFNltJLMrkoKWaAXxs4b+7djmvt55Ipy8A2b24GlrPP4yjL4= X-Received: by 2002:a05:6a00:3c91:b0:841:58b0:82c7 with SMTP id d2e1a72fcca58-84225327e9amr13634126b3a.8.1780395152058; Tue, 02 Jun 2026 03:12:32 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.12.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:12:31 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 18/19] mm/hugetlb: Remove unused bootmem cma field Date: Tue, 2 Jun 2026 18:10:38 +0800 Message-ID: <20260602101039.1867613-19-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" struct huge_bootmem_page no longer needs to keep the CMA pointer. The bootmem path only needs to remember whether a huge page came from CMA, which is already encoded in the flags field. Set HUGE_BOOTMEM_CMA when the page is allocated and drop the unused cma field together with the redundant assignments. Signed-off-by: Muchun Song --- mm/hugetlb.c | 5 +---- mm/hugetlb_cma.c | 29 +++++++++++------------------ mm/internal.h | 2 -- 3 files changed, 12 insertions(+), 24 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 47c3d6d11c58..fb7ad2a4a26b 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3076,10 +3076,7 @@ static bool __init alloc_bootmem_huge_page(struct hs= tate *h, int nid) */ INIT_LIST_HEAD(&m->list); m->hstate =3D h; - if (!hugetlb_early_cma(h)) { - m->cma =3D NULL; - m->flags =3D 0; - } + m->flags =3D hugetlb_early_cma(h) ? HUGE_BOOTMEM_CMA : 0; =20 /* CMA pages: zone-crossing is validated in hugetlb_cma_reserve(). */ if (!hugetlb_early_cma(h) && diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c index e487d0ffffc0..4dfce68b354a 100644 --- a/mm/hugetlb_cma.c +++ b/mm/hugetlb_cma.c @@ -59,31 +59,24 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order,= gfp_t gfp_mask, void * __init hugetlb_cma_alloc_bootmem(struct hstate *h, int nid, bool no= de_exact) { struct cma *cma; - struct huge_bootmem_page *m; + void *m; int node; =20 cma =3D hugetlb_cma[nid]; m =3D cma_reserve_early(cma, huge_page_size(h)); - if (!m) { - if (node_exact) - return NULL; + if (m || node_exact) + return m; =20 - for_each_node_mask(node, hugetlb_bootmem_nodes) { - cma =3D hugetlb_cma[node]; - if (!cma || node =3D=3D nid) - continue; - m =3D cma_reserve_early(cma, huge_page_size(h)); - if (m) - break; - } - } - - if (m) { - m->flags =3D HUGE_BOOTMEM_CMA; - m->cma =3D cma; + for_each_node_mask(node, hugetlb_bootmem_nodes) { + cma =3D hugetlb_cma[node]; + if (!cma || node =3D=3D nid) + continue; + m =3D cma_reserve_early(cma, huge_page_size(h)); + if (m) + return m; } =20 - return m; + return NULL; } =20 static int __init cmdline_parse_hugetlb_cma(char *p) diff --git a/mm/internal.h b/mm/internal.h index 6b9802460a7c..8497673d0ac3 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -24,13 +24,11 @@ =20 struct folio_batch; struct hstate; -struct cma; =20 struct huge_bootmem_page { struct list_head list; struct hstate *hstate; unsigned long flags; - struct cma *cma; }; =20 /* --=20 2.54.0 From nobody Mon Jun 8 03:22:20 2026 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1818E3E7164 for ; Tue, 2 Jun 2026 10:12:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395161; cv=none; b=VIcJlWGRCBpC28OpIMx47Bs3hE2C1TPZLN8BoYJJP7ZBCe0B+rm87QvyEyKz6mPF3siYAe7LsfXTlYS9Yx2ZUpnRqz+oyII9EE0dhs8yjsu1Lp6uHuUfoA2/FjL1+KxBgHLtPxGgWxR/Gdq4VpVI1GjXbViw3Si7abR7UfxIy9Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780395161; c=relaxed/simple; bh=nMn+/0CcgJtqXCFVO1ubwkSBMh//lHF+AGiZcDEWDOo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tDVGQbrdvXJLf0+apfAOJ0kHpNr/ZBY/gJ1qCRyTPYW9zcNVC0A1D6N6qjr58hs/GOwlQJ3ImStzcqBhg3CpKO+ESVytsNya/F1cCs42CjJ9P2PUBYH97Izu/SwUGY26PlHN7V51qjcVa/yUZ0kkdhsUt4DrqFHrJyYgwzYtTuI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=XT22Wd2B; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="XT22Wd2B" Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-842307472d4so668628b3a.0 for ; Tue, 02 Jun 2026 03:12:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1780395158; x=1780999958; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=J5Q2gGk4UZAL2GpP2RnQJwNvQrGN9WYj/IZDEx+Kuy0=; b=XT22Wd2BDdQxN049I/EJAoxwxDb7HrCfEy7jAQWKCi/JdH5LWIi6QrRx5veivg6f42 V0hBoYxCG8KdpPC7hwLOPGI8hGzOY+8+Wj3C6HrWVCOmVHASe4RzIgX/fJfyUec4eMxM oVbdIYmHyXyiSzvDAoJCOcL/RC8E+oLrhfnVzAUAmN+VzV4fLFOJYRZrWtjPVJ/i83Y7 l4H1+Dlrc9DA59ygl5x6BxiU8vQaW45/QeSlUkvXnGysZ61yB46q34tTgLqHGLoBcvW/ jFE3VZdebxNT/y1AXoIwmzpvWfnxH1WWvprxbQJgrm3GswMnRFhLFx2aMr5VkCLsX1b2 DsZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780395158; x=1780999958; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=J5Q2gGk4UZAL2GpP2RnQJwNvQrGN9WYj/IZDEx+Kuy0=; b=AU33uw1yJ7cC3ZDWt/47izUdkdha8OtHqUUUK0PjMtfxQFC7I2yuXz1guhpVjLK4Rj WSxUkodcU+EppRWzsJrPUDmBwwthhM4sibuA375tGzh4yOO5s0aatY1xRZElU+d0m1G5 Fxe8V7NshRX4lVHZ0+3v/gVr8Ttco/HjQd1LD5ZlxLMdoL7XPLDJ1b882xkAoJTBXGuD WcktH/qOR5wc6OGF4+i2F747erKAIELX29LAfHpgs+AiJA4tFhZb7YF8XNZLURlLz8yO dttRuhP83L3H/ElBf9tdRbmlasjNqKU7adGsqbSJfH/pC/Kr4zgWoG0XqJBK85Vng1FU Kq3A== X-Forwarded-Encrypted: i=1; AFNElJ8NxGK3diYwsd7xed5U83Z7tCH9td2O1V0bxDITtvhQECCHe8lrkpCRcm+ZLgZIFzMKyivW77WCwUeDrV0=@vger.kernel.org X-Gm-Message-State: AOJu0YxdTm6ksMZncNMnA6rLBMJLT8LWpdSToy8cmIYX7csOxOZJgU+I y1RA/ZRsWmHUsnNJPq9L6yevpLvk+OhV6k77o7uAB4UYYzCP5P7FIGB4BykPnYFwcHo= X-Gm-Gg: Acq92OHtkFQ89FRFFrZxxZDPLJYUZX9Ljqh2vZG4JqOkTr+0cfsBq8HBD9z2q9NjvXg bWWJAx/ggNV7EMh0OtP3Ry2m4fzrgHK6l3iOQEv9fKZZ8xzLvqtf6whmcaPpR1DA8uEWSTUvlts tHYBHkgZkiCG7GwBABVhqdBIVFjGXyfrQW+Fga51T9T+EZAPse2qwOgf2Ag2fy9NT5DfqzXydMA u4sp8ELWyTJLmH8gbUF78Q8ypS4uIt0DkLiQmxRHZkC1dcZP6xQl3IH8okc3AbFJhRWvzRQz4DU DvdFD+zcjBAJ2FGiARwuKamK2nOusjBWp9UsuGEF7jc0iAaUQCuN5XVc0+q2e2NPltpVHmfzPIK sxgPfVawlPAl0kIZ5NNlFaHMxtdHiHp5eL4mW1ymXtFI7qjo3ymm66dPRlTaGOT7v+fxoNeJ3Z1 RgNFcj+ocm1DeiXce2x2ViHB4inOEGfkUG6D6GDsRZT+tX2/WlaX6eA6Q= X-Received: by 2002:a05:6a00:8d96:b0:835:366f:5da2 with SMTP id d2e1a72fcca58-842254980a1mr14651687b3a.37.1780395158008; Tue, 02 Jun 2026 03:12:38 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.102]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84214d018fcsm12015507b3a.60.2026.06.02.03.12.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2026 03:12:37 -0700 (PDT) From: Muchun Song To: Oscar Salvador , David Hildenbrand , Andrew Morton , Madhavan Srinivasan , Michael Ellerman Cc: Muchun Song , Mike Rapoport , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "Ritesh Harjani (IBM)" , "Aneesh Kumar K.V" , linuxppc-dev@lists.ozlabs.org, Mike Kravetz , Muchun Song Subject: [PATCH v3 19/19] mm/mm_init: Fold __init_page_from_nid() into __init_deferred_page() Date: Tue, 2 Jun 2026 18:10:39 +0800 Message-ID: <20260602101039.1867613-20-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260602101039.1867613-1-songmuchun@bytedance.com> References: <20260602101039.1867613-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" __init_page_from_nid() no longer has external users and is only used locally in mm/mm_init.c under CONFIG_DEFERRED_STRUCT_PAGE_INIT. Fold it into its sole caller __init_deferred_page() and remove the separate helper declaration. Signed-off-by: Muchun Song Reviewed-by: Mike Rapoport (Microsoft) --- v2->v3: - fold __init_page_from_nid() into __init_deferred_page() since it only has a single caller (suggested by Mike Rapoport) --- mm/internal.h | 1 - mm/mm_init.c | 44 ++++++++++++++++++-------------------------- 2 files changed, 18 insertions(+), 27 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 8497673d0ac3..b33fc87e4555 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1760,7 +1760,6 @@ static inline bool pte_needs_soft_dirty_wp(struct vm_= area_struct *vma, pte_t pte =20 void __meminit __init_single_page(struct page *page, unsigned long pfn, unsigned long zone, int nid); -void __meminit __init_page_from_nid(unsigned long pfn, int nid); =20 /* shrinker related functions */ unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memc= g, diff --git a/mm/mm_init.c b/mm/mm_init.c index 41b83dd18c01..f1bbf3b9a321 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -688,31 +688,6 @@ static __meminit void pageblock_migratetype_init_range= (unsigned long pfn, } #endif =20 -/* - * Initialize a reserved page unconditionally, finding its zone first. - */ -void __meminit __init_page_from_nid(unsigned long pfn, int nid) -{ - pg_data_t *pgdat; - int zid; - - pgdat =3D NODE_DATA(nid); - - for (zid =3D 0; zid < MAX_NR_ZONES; zid++) { - struct zone *zone =3D &pgdat->node_zones[zid]; - - if (zone_spans_pfn(zone, pfn)) - break; - } - __init_single_page(pfn_to_page(pfn), pfn, zid, nid); - - if (pageblock_aligned(pfn)) { - enum migratetype mt =3D - kho_scratch_migratetype(pfn, MIGRATE_MOVABLE); - init_pageblock_migratetype(pfn_to_page(pfn), mt, false); - } -} - #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT static inline void pgdat_set_deferred_range(pg_data_t *pgdat) { @@ -771,10 +746,27 @@ defer_init(int nid, unsigned long pfn, unsigned long = end_pfn) =20 static void __meminit __init_deferred_page(unsigned long pfn, int nid) { + pg_data_t *pgdat; + int zid; + if (early_page_initialised(pfn, nid)) return; =20 - __init_page_from_nid(pfn, nid); + pgdat =3D NODE_DATA(nid); + + for (zid =3D 0; zid < MAX_NR_ZONES; zid++) { + struct zone *zone =3D &pgdat->node_zones[zid]; + + if (zone_spans_pfn(zone, pfn)) + break; + } + __init_single_page(pfn_to_page(pfn), pfn, zid, nid); + + if (pageblock_aligned(pfn)) { + enum migratetype mt =3D + kho_scratch_migratetype(pfn, MIGRATE_MOVABLE); + init_pageblock_migratetype(pfn_to_page(pfn), mt, false); + } } #else static inline void pgdat_set_deferred_range(pg_data_t *pgdat) {} --=20 2.54.0