From nobody Sat Sep 26 14:39:46 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00F7A435A9E for ; Mon, 31 Aug 2026 13:36:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788183387; cv=none; b=DOeDJPaM2SnahY+69tTZUqIJbfyCBLQZBBLSp6mqnW3j+nMhTW8A9H001//PSB1WC7BGWzBVQduVLYUlDbGa+fhCoteiuBkvlAqOwqfIkKPaKoHvQ8V51PAK1n+Z+vvurE2GlAwbsWz4nOLFDZs/y9rqc/n7rwCG9MI+Y8c5/N0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788183387; c=relaxed/simple; bh=4qLV+I2TLdLxH1jJbYPeqrjBblREqvDLagnJNPs3g1E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bs7svYl/b0Ou1ts1hcttZqRAm3BXLbTjwAxf2B0gmu4ytLcmD5IuasLcWO7ofJVxZI5V0vqmgzbsgPphluGb3ygtGP+KBL7x98t61XvKycTL4+yg2h3ztVQ5OAmfU28kV1GLI3RmrnNIdU6KtVj31vjtGtxT4OFBTgseLscxhGQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=m2SKSnTB; arc=none smtp.client-ip=117.135.210.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="m2SKSnTB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=Se R1+mALksfzVZt4agnwdff+4ehIP+rnZ95J6spXEug=; b=m2SKSnTBFVwCNze2g6 hr979eG4m1npQqij6JEbCGskhQSApcqT1y/qW9WJ2rnN7ePjrdL6AvDwADMP9FS7 TyXUzEh7QdGfdy/PQW1wT71AUgm22mTrfRKUugjOanCjj5RzuVGdLwCH5P8VdQ2J 7M6RBeFwmBNrn4lJ+KtyYRXRM= Received: from XLL-9950X.localdomain (unknown []) by gzga-smtp-mtada-g0-3 (Coremail) with SMTP id _____wBHHmEbg5VqFKS5Sg--.13223S3; Mon, 31 Aug 2026 21:35:26 +0800 (CST) From: Longlong Xia To: muchun.song@linux.dev, osalvador@suse.de, akpm@linux-foundation.org Cc: david@kernel.org, mike.kravetz@oracle.com, yuzhao@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Longlong Xia Subject: [PATCH v2 1/2] mm/hugetlb: preserve source surplus accounting during demotion Date: Mon, 31 Aug 2026 21:35:18 +0800 Message-ID: <20260831133519.2505020-2-xialonglong2025@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260831133519.2505020-1-xialonglong2025@163.com> References: <20260831133519.2505020-1-xialonglong2025@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wBHHmEbg5VqFKS5Sg--.13223S3 X-Coremail-Antispam: 1Uf129KBjvJXoWxZFykGr4fWFyDJw4ktryfXrb_yoWrCw17pa yYkwnYkrW8Ar9xGr4Sq34qywn8G340vrWjya4fKr43J3ZxZ34UKr13Xw1UJayv9rWfJF43 Xw4DZFyDWF43ZaUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07j2v38UUUUU= X-CM-SenderInfo: x0ldz0pqjo00rjsqjki6rwjhhfrp/xtbC3B7n4GqVgx6CpgAA3D Content-Type: text/plain; charset="utf-8" From: Longlong Xia demote_pool_huge_page() currently removes every source folio as a persistent folio. A free folio can instead account for one of the source hstate's surplus pages, for example after a vmemmap restoration failure. Removing such a folio without adjusting surplus_huge_pages makes the persistent count underflow, and later subtracting it from max_huge_pages can underflow that counter as well. Classify selected folios against the node's surplus count while holding hugetlb_lock, and preserve that classification on rollback. Track the number of successfully demoted persistent folios separately so only those folios reduce the source max_huge_pages target. All successfully demoted folios still increase the destination target because the new destination folios are added as persistent pages. Testing: Tested on an x86_64 QEMU guest booted with: hugepagesz=3D1G hugepages=3D0 hugetlb_cma=3D1G hugetlb_free_vmemmap=3Don For testing only, add a one-shot fault injection that makes the first call to __hugetlb_vmemmap_restore_folio() for an optimized 1 GiB folio return -ENOMEM. Set nr_overcommit_hugepages to 1, then allocate one 1 GiB huge page: nr=3D1 surplus=3D1 free=3D0 resv=3D0 Unmap it. The failed restoration leaves the folio on the freelist while it is still accounted as surplus: nr=3D1 surplus=3D1 free=3D1 resv=3D0 Demote one page. Before this fix, the result is: nr=3D0 surplus=3D1 free=3D0 resv=3D0 After this fix, the result is: nr=3D0 surplus=3D0 free=3D0 resv=3D0 The fault injection is one-shot, so the restore performed during demotion can succeed. Fixes: 8531fc6f52f5 ("hugetlb: add hugetlb demote page support") Assisted-by: Codex:gpt-5.6-sol Signed-off-by: Longlong Xia --- mm/hugetlb.c | 35 +++++++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index ed26105b84de..640df58be4e5 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3983,6 +3983,7 @@ long demote_pool_huge_page(struct hstate *src, nodema= sk_t *nodes_allowed, struct hstate *dst; long rc =3D 0; long nr_demoted =3D 0; + long nr_persistent =3D 0; =20 lockdep_assert_held(&hugetlb_lock); =20 @@ -3995,22 +3996,40 @@ long demote_pool_huge_page(struct hstate *src, node= mask_t *nodes_allowed, =20 for_each_node_mask_to_free(src, nr_nodes, node, nodes_allowed) { LIST_HEAD(list); + LIST_HEAD(surplus_list); struct folio *folio, *next; =20 list_for_each_entry_safe(folio, next, &src->hugepage_freelists[node], lr= u) { + bool adjust_surplus; + if (folio_test_hwpoison(folio)) continue; =20 - remove_hugetlb_folio(src, folio, false); - list_add(&folio->lru, &list); + /* Surplus accounting is maintained per node, not per folio. */ + adjust_surplus =3D src->surplus_huge_pages_node[node] > 0; + remove_hugetlb_folio(src, folio, adjust_surplus); + list_add(&folio->lru, adjust_surplus ? &surplus_list : &list); + if (!adjust_surplus) + nr_persistent++; =20 if (++nr_demoted =3D=3D nr_to_demote) break; } =20 + if (list_empty(&list) && list_empty(&surplus_list)) + continue; + spin_unlock_irq(&hugetlb_lock); =20 - rc =3D demote_free_hugetlb_folios(src, dst, &list); + if (!list_empty(&list)) + rc =3D demote_free_hugetlb_folios(src, dst, &list); + if (!list_empty(&surplus_list)) { + long tmp_rc; + + tmp_rc =3D demote_free_hugetlb_folios(src, dst, &surplus_list); + if (rc >=3D 0) + rc =3D tmp_rc; + } =20 spin_lock_irq(&hugetlb_lock); =20 @@ -4018,6 +4037,14 @@ long demote_pool_huge_page(struct hstate *src, nodem= ask_t *nodes_allowed, list_del(&folio->lru); add_hugetlb_folio(src, folio, false); =20 + nr_demoted--; + nr_persistent--; + } + + list_for_each_entry_safe(folio, next, &surplus_list, lru) { + list_del(&folio->lru); + add_hugetlb_folio(src, folio, true); + nr_demoted--; } =20 @@ -4029,7 +4056,7 @@ long demote_pool_huge_page(struct hstate *src, nodema= sk_t *nodes_allowed, * Not absolutely necessary, but for consistency update max_huge_pages * based on pool changes for the demoted page. */ - src->max_huge_pages -=3D nr_demoted; + src->max_huge_pages -=3D nr_persistent; dst->max_huge_pages +=3D nr_demoted << (huge_page_order(src) - huge_page_= order(dst)); =20 if (rc < 0) --=20 2.43.0 From nobody Sat Sep 26 14:39:46 2026 Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D06F7438FE6 for ; Mon, 31 Aug 2026 13:36:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=117.135.210.3 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788183388; cv=none; b=S5vAubHiQ84E6RliImlsDUwL6dDJGJ+AbJ3JdQ9UznAzXlYESIElQkfU+tZV5i5RzQhy8mdqZ6JSWTrzbhgZBMCPLjBuBIBpqWLQmf2atSaRjbVB4RT3Wude0Oo5rSTgIr1wGHLEeIcvk4w5xg10xp88LMxsOozVunY7MjFm+k4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788183388; c=relaxed/simple; bh=c3XB6pclgUDulfG0w1HZ57HAEcJOtSCEIStcIHB0wrc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NSruIpdrmhQV/MVnpkifalKFTKGrdI8WtXW9c+vE9DQ73rqZQgv2ST8qvuytCmyRBBoVqsqIjIHDoveaXft1RxxlYYqXFnoO6FbFiyXnsLb1NyrWjoiVBoqo8WK9YVbxjibaXxRh/rXaV3QE9HPTRJxRsHo9E2jbdH28tyigKI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=LWrUCsyp; arc=none smtp.client-ip=117.135.210.3 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="LWrUCsyp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=w/ ZOvr0cFnO+KxNXzKLYo9hHd8muTFSoABjiNPb5SQo=; b=LWrUCsyp6sBJRjuFFH WoX/i60LcZJIhpl9tb8texE4kWXekPjEjDwGD0GBLWceXAlSipmfOb1zX3SH5iGW Nndy4j6SMPn2S98e9bLe+YVaEsNI7xnQl0qloYEDZnJH2MgTjYLRYO3oxjymQtLj 9Ag1QdX40jVFjzUjGlD6nfAW8= Received: from XLL-9950X.localdomain (unknown []) by gzga-smtp-mtada-g0-3 (Coremail) with SMTP id _____wBHHmEbg5VqFKS5Sg--.13223S4; Mon, 31 Aug 2026 21:35:27 +0800 (CST) From: Longlong Xia To: muchun.song@linux.dev, osalvador@suse.de, akpm@linux-foundation.org Cc: david@kernel.org, mike.kravetz@oracle.com, yuzhao@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Longlong Xia Subject: [PATCH v2 2/2] mm/hugetlb: cap demotion at currently available free pages Date: Mon, 31 Aug 2026 21:35:19 +0800 Message-ID: <20260831133519.2505020-3-xialonglong2025@163.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260831133519.2505020-1-xialonglong2025@163.com> References: <20260831133519.2505020-1-xialonglong2025@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wBHHmEbg5VqFKS5Sg--.13223S4 X-Coremail-Antispam: 1Uf129KBjvJXoWxZr17Cw1xArW5Aw45tF1rCrg_yoWrJFy5pr W8tw1Sk3ykZF9rGr1Sqw1Dt3WfZw4xKrWrA34fJ3y5Zw13X3sF9r1xJryUZa4xuFWfJanI 9F4DZ34DZry5XaUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jc6p9UUUUU= X-CM-SenderInfo: x0ldz0pqjo00rjsqjki6rwjhhfrp/xtbC+B-n4GqVgx+NqAAA3m Content-Type: text/plain; charset="utf-8" From: Longlong Xia Demotion must not remove free huge pages that back existing reservations. The sysfs path checks whether any page is available, but passes the entire request to demote_pool_huge_page(). For example, with two free pages and one reservation, a request for two pages removes both and leaves the reservation without a backing page. Cap the sysfs request by both global availability and the selected node's free pages. Recheck global availability in demote_pool_huge_page() before each node batch because that function drops hugetlb_lock while restoring vmemmap and reservations can change before the next batch. Testing: Tested on an x86_64 QEMU guest booted with: hugepagesz=3D1G hugepages=3D2 Reserve one 1 GiB huge page with an untouched hugetlbfs mapping: nr=3D2 surplus=3D0 free=3D2 resv=3D1 Request demotion of two pages. Before this fix, both free pages are demoted: nr=3D0 surplus=3D0 free=3D0 resv=3D1 Touching the reserved mapping then fails with SIGBUS. After this fix, the request is capped at the single available page: nr=3D1 surplus=3D0 free=3D1 resv=3D1 Touching the reserved mapping succeeds. After the process exits, the counters are: nr=3D1 surplus=3D0 free=3D1 resv=3D0 Fixes: c0f398c3b2cf ("mm/hugetlb_vmemmap: batch HVO work when demoting") Assisted-by: Codex:gpt-5.6-sol Signed-off-by: Longlong Xia --- mm/hugetlb.c | 22 +++++++++++++++++++++- mm/hugetlb_sysfs.c | 10 +++++----- 2 files changed, 26 insertions(+), 6 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 640df58be4e5..ae26d400ad31 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3998,6 +3998,26 @@ long demote_pool_huge_page(struct hstate *src, nodem= ask_t *nodes_allowed, LIST_HEAD(list); LIST_HEAD(surplus_list); struct folio *folio, *next; + unsigned long nr_available, nr_target; + + /* + * Re-check available each node batch: the previous + * batch released hugetlb_lock for vmemmap restore/split, + * and a new reservation could have been added in that + * window, shrinking the budget. available is global + * (resv is not per-node), so 0 means no node can + * contribute -- stop the whole scan. + */ + nr_available =3D available_huge_pages(src); + if (!nr_available) + break; + + /* + * Cap this batch at the current budget; expressed as a + * cumulative stop point because nr_demoted is running. + */ + nr_target =3D nr_demoted + min_t(unsigned long, + nr_to_demote - nr_demoted, nr_available); =20 list_for_each_entry_safe(folio, next, &src->hugepage_freelists[node], lr= u) { bool adjust_surplus; @@ -4012,7 +4032,7 @@ long demote_pool_huge_page(struct hstate *src, nodema= sk_t *nodes_allowed, if (!adjust_surplus) nr_persistent++; =20 - if (++nr_demoted =3D=3D nr_to_demote) + if (++nr_demoted =3D=3D nr_target) break; } =20 diff --git a/mm/hugetlb_sysfs.c b/mm/hugetlb_sysfs.c index 79ece91406bf..326a54b4d991 100644 --- a/mm/hugetlb_sysfs.c +++ b/mm/hugetlb_sysfs.c @@ -211,15 +211,15 @@ static ssize_t demote_store(struct kobject *kobj, * Check for available pages to demote each time thorough the * loop as demote_pool_huge_page will drop hugetlb_lock. */ + nr_available =3D h->free_huge_pages - h->resv_huge_pages; if (nid !=3D NUMA_NO_NODE) - nr_available =3D h->free_huge_pages_node[nid]; - else - nr_available =3D h->free_huge_pages; - nr_available -=3D h->resv_huge_pages; + nr_available =3D min(nr_available, + h->free_huge_pages_node[nid]); if (!nr_available) break; =20 - rc =3D demote_pool_huge_page(h, n_mask, nr_demote); + rc =3D demote_pool_huge_page(h, n_mask, + min(nr_demote, nr_available)); if (rc < 0) { err =3D rc; break; --=20 2.43.0