From nobody Sun Feb 8 05:30:29 2026 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E7681E2606 for ; Mon, 12 Jan 2026 00:49:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768178970; cv=none; b=PvGn6VkxW9PA2lywoIh8Qjp0fOmENLiUDGwH9PwENawyP2eCS3Hjro0ScLY2ZkXMiDs+niOXSA9tJTgo6UbHnxCK/1CW2Iet7lwgNvLAJNfd3tKq35eCzrYFI9XXsWI6PoytrUalee/XX9RmUh3zOK+PteRMp7C03wRbRA1PO9Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768178970; c=relaxed/simple; bh=Q+sritPehEeS6wAIA6c4wtQmtWYoZZ9O37BCaRGN1P8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GToBs5dTwygbmoTazo7vxTTVCUqNjfCuAm7g+Vpd2+YnSPsBKiZNld8FCVF/2WRx6u+GxBaVk2Gb0H3cbG73lTKMpEuozUdN2CuYd17ZIDF4njSzJB2NBkoJ/BD2ShQ99KnSrPphfiuehmlx1MNtbMhIeY8ToAFFBp8td9uAl0U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jiaqiyan.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=i5oHGA96; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jiaqiyan.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="i5oHGA96" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-34ea5074935so5788171a91.0 for ; Sun, 11 Jan 2026 16:49:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1768178969; x=1768783769; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=gjAce3C3Y1QIKd40PiLPufwi5ToI+2zeU/k8z/0R9VU=; b=i5oHGA96WVwlGnYZWoa58/zaQebDelHS5yXemcxXZtbLKf4WuvcJxxWSfoySzBvFpe UpdOFrtoqXQxiORO4XnIK0RzMIR0xYSN9yQRskTfie9vatpZyfxl2VkXtdj5ybStPVXr C9QiQE/CJUOytQM5VHUugIL/ACVScirldXFi0HEOhTLWY02YsFIcCEQEK6K9JRirhySc Zm/VPA2F/jt6ZsjZ0gLGKWujpHrYM4ulZ8kyQ81Bu3ikB+Nb/SaMSNn1FR3pPHYrVHaR DB8WJCAFR2/3a9EU5xrJ188YR3iEn6tLR5qZlg+2ZQo+pjHZF/KQQr8Nh7/38vx51kHU TFXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768178969; x=1768783769; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gjAce3C3Y1QIKd40PiLPufwi5ToI+2zeU/k8z/0R9VU=; b=JDxId/V+lxvFocGbppU2hmMXzdV8HTsRGABFJSqqQ+v1t6zx2nfB/ZBLIWUfc2RwTB /r5REIIiOK3RW5FRl/Vt9S5uJGi/k/l4V6NAJUMG5Wmr8Qs9uVnUrQHTRJeRHEQP65w6 HU1tuHqYN0KRwnLB3JjsoRNea9Nw2I21NBCEjKGYUJEvycjYvsLAZi6/NEX+flt4aod8 hAlbeFyGPBi1xryR15/eFa/pQVCV9O2JdhkyyPYNwowkdKSCElugLecTs6jSPpE6MTdZ oQH5x6b+PT50KYdJqkdn9kJr59IyI/QLo4le6SsHX+GyUPx5hSlerYcJuaR24a59oCR8 qyaA== X-Forwarded-Encrypted: i=1; AJvYcCU4TUoo3O2Cu4VYTcGEtQSPL7Vl1yiIMSAPSXIc7Sd83Bg1DjT1jrXn7GJvyGG0JVKfC6oUsH3X7RJAA0c=@vger.kernel.org X-Gm-Message-State: AOJu0YzoDT3+V2ESVRBWnJhScIoOLWk85Pz9Ro1HKJS+g0JHFACi4Uvx CyNu1e6y8UudVRjXHE//2pksjVMDjBOaqbiLhGZhFX8ZNvqAH+u3SE/ux9btTZ+kYhpYFAGzzW3 6wUHTQtSWhOBBcw== X-Google-Smtp-Source: AGHT+IGRsYmD0Owt99yy0ghEM70oDPPy8Wm8X//vCefVX5nk+u6cdO2UbDFZj27GMxMpcJ/JK+UWe9vqv3woFg== X-Received: from pjbgk1.prod.google.com ([2002:a17:90b:1181:b0:327:50fa:eff9]) (user=jiaqiyan job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2642:b0:340:d578:f2a2 with SMTP id 98e67ed59e1d1-34f68c335e4mr15145403a91.6.1768178968964; Sun, 11 Jan 2026 16:49:28 -0800 (PST) Date: Mon, 12 Jan 2026 00:49:21 +0000 In-Reply-To: <20260112004923.888429-1-jiaqiyan@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260112004923.888429-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.52.0.457.g6b5491de43-goog Message-ID: <20260112004923.888429-2-jiaqiyan@google.com> Subject: [PATCH v3 1/3] mm/memory-failure: set has_hwpoisoned flags on HugeTLB folio From: Jiaqi Yan To: jackmanb@google.com, hannes@cmpxchg.org, linmiaohe@huawei.com, ziy@nvidia.com, harry.yoo@oracle.com, willy@infradead.org Cc: nao.horiguchi@gmail.com, david@redhat.com, lorenzo.stoakes@oracle.com, william.roche@oracle.com, tony.luck@intel.com, wangkefeng.wang@huawei.com, jane.chu@oracle.com, akpm@linux-foundation.org, osalvador@suse.de, muchun.song@linux.dev, rientjes@google.com, duenwen@google.com, jthoughton@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, Jiaqi Yan Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a free HWPoison HugeTLB folio is dissolved, it becomes non-HugeTLB and is released to buddy allocator as a high-order folio. Set has_hwpoisoned flags on the high-order folio so that buddy allocator can tell that it contains certain HWPoison page(s). This is a prepare change for buddy allocator to handle only the high-order HWPoison folio differently. This cannot be done with hwpoison flag because users cannot tell from the case that the page with hwpoison is hardware corrupted. Signed-off-by: Jiaqi Yan --- include/linux/page-flags.h | 2 +- mm/memory-failure.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index f7a0e4af0c734..d13835e265952 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -904,7 +904,7 @@ static inline int PageTransCompound(const struct page *= page) TESTPAGEFLAG_FALSE(TransCompound, transcompound) #endif =20 -#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_TRANSPARENT_HUGEPAGE) +#if defined(CONFIG_MEMORY_FAILURE) && (defined(CONFIG_TRANSPARENT_HUGEPAGE= ) || defined(CONFIG_HUGETLB_PAGE)) /* * PageHasHWPoisoned indicates that at least one subpage is hwpoisoned in = the * compound page. diff --git a/mm/memory-failure.c b/mm/memory-failure.c index fbc5a01260c89..d204de6c9792a 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1952,6 +1952,7 @@ void folio_clear_hugetlb_hwpoison(struct folio *folio) if (folio_test_hugetlb_vmemmap_optimized(folio)) return; folio_clear_hwpoison(folio); + folio_set_has_hwpoisoned(folio); folio_free_raw_hwp(folio, true); } =20 --=20 2.52.0.457.g6b5491de43-goog From nobody Sun Feb 8 05:30:29 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48EFE20010A for ; Mon, 12 Jan 2026 00:49:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768178972; cv=none; b=GNSImOGwJ54VbIStfiascvEvzMQq3KvYHa4mIGadURrtrToWRsS4OLSLNMb811NzpBTyZgvvwyfYGrncuN+0nExqNBjrocYNihW6gQ82k65FOCpx+HortMO4fze+K1wpvPKs5igcHW6M3SA/oCWs7+so+yitd3Dz/Etg6mCZQzs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768178972; c=relaxed/simple; bh=BeycCS36b/oT9UtTuqLnX3mpVbKss9EYi+BomrbRUvM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=d0dL5dRXtNXMNmjWFFz0fA2HLSlJxKmPFwhNiOxRLEYr6OfqE3YST2/VeWW2bfE3rlkv+BgnVrhuk0uS9SGFd6/umjDld12ga8Ca3Go/CnGVMGFrgTvE8HB6/5pZ7JWrXp6ObPADxIwpx+9Z0hTn/ZylzqrgkGqA7tYqdAOtQVI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jiaqiyan.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VzZAA9Av; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jiaqiyan.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VzZAA9Av" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-29f1f79d6afso73993085ad.0 for ; Sun, 11 Jan 2026 16:49:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1768178970; x=1768783770; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uwmWF2Bg3a4xbvi54Ji/t7adbofzJTtZzLagbDRk3qs=; b=VzZAA9AvgJyAVFsWgCngzw7TvWzj3HYX0jTUTvFvFO/uxVgTcbT9IcZAGeE7KtETlA YOCijkVMsjBd+rNDEYR8yLxuPTMdPSCkE9c6hc+YcWM/w5d+vcs59yuKrpaJLZLvaqjr mAsclPb0Uo36Eyk/rTS7WT1avgvW0MNGe7B6cb8nndzhOL46x6jECs4bh+pSP82ql0RV D5GTkr1LTm+cNDqxLtwBEcvYxWpWuglsYMzlL222tzOt++mYbiebVfeLSrUuXNjqFqTf NbKutteM2GGW3KOhtvqY8JHceReFn5yAbxdaifeC+C5KXeEyWY78sBe+1ljRD/8xbb4r rDHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768178970; x=1768783770; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uwmWF2Bg3a4xbvi54Ji/t7adbofzJTtZzLagbDRk3qs=; b=vqE2+4S0Da1OEgCKcS8fNcOE6ubsse4fCTfejj+w+tIlESspEDrsv/T9Ku+Bs9nJni 1rrOeXYbVSF3liZ4lRrJQvHTeTkZ1I6ziGrN5NXundeerHEXfElosYHXsUN5NZzJq5cs UiA9x8tQC96vgCjxqQRAIEFm24Uvf2VtLVI6qUw9JKUbzrOV78bXYLnkCFvAehU4x0fe iyJLYJur7RYaRKjpv+7c0u6XJ9CJfZFtR7SZw4bdDxL02vxDw84DO/5A0iarX8O12Zlu Z3xvh9iCDINo0xB3jNg7cT6M4mjBacAAZobxkT1c7+zs6BgKjxNmK/xHNic5T1rriVZ0 gMOA== X-Forwarded-Encrypted: i=1; AJvYcCXfhShUOk9SbNQ0y6V1WjiIjm1wfld9LhPG4ZxBr7AbEXLpIDqLFnVLJLN+u+gZ9Xo+o8EpK7PN0uPNYM0=@vger.kernel.org X-Gm-Message-State: AOJu0YziI20KL5A/7PttjBsskbpRpmz/QLPL8lIp5UPPQj6U2mlEb94z +ORom3p/sPS4eUtRJvpaQI3tXitJI4JcbLgxQlDHgJc8T80uWbRIwKI2/j4jfjYxkPma9L8rgCr OzxxFzaf8Ip6vVA== X-Google-Smtp-Source: AGHT+IFL3OjBX45+6Y4kyGYmIkZajwGcoQAeVv1gmRs2jlg58e65eo6WLqHWTKXBSZfAGu8Fskh2r/bvzC2oFA== X-Received: from plde19.prod.google.com ([2002:a17:902:d393:b0:29e:fb92:99f6]) (user=jiaqiyan job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ef49:b0:2a0:c1f5:c695 with SMTP id d9443c01a7336-2a3ee434048mr164800025ad.16.1768178970528; Sun, 11 Jan 2026 16:49:30 -0800 (PST) Date: Mon, 12 Jan 2026 00:49:22 +0000 In-Reply-To: <20260112004923.888429-1-jiaqiyan@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260112004923.888429-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.52.0.457.g6b5491de43-goog Message-ID: <20260112004923.888429-3-jiaqiyan@google.com> Subject: [PATCH v3 2/3] mm/page_alloc: only free healthy pages in high-order has_hwpoisoned folio From: Jiaqi Yan To: jackmanb@google.com, hannes@cmpxchg.org, linmiaohe@huawei.com, ziy@nvidia.com, harry.yoo@oracle.com, willy@infradead.org Cc: nao.horiguchi@gmail.com, david@redhat.com, lorenzo.stoakes@oracle.com, william.roche@oracle.com, tony.luck@intel.com, wangkefeng.wang@huawei.com, jane.chu@oracle.com, akpm@linux-foundation.org, osalvador@suse.de, muchun.song@linux.dev, rientjes@google.com, duenwen@google.com, jthoughton@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, Jiaqi Yan Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" At the end of dissolve_free_hugetlb_folio(), a free HugeTLB folio becomes non-HugeTLB, and it is released to buddy allocator as a high-order folio, e.g. a folio that contains 262144 pages if the folio was a 1G HugeTLB hugepage. This is problematic if the HugeTLB hugepage contained HWPoison subpages. In that case, since buddy allocator does not check HWPoison for non-zero-order folio, the raw HWPoison page can be given out with its buddy page and be re-used by either kernel or userspace. Memory failure recovery (MFR) in kernel does attempt to take raw HWPoison page off buddy allocator after dissolve_free_hugetlb_folio(). However, there is always a time window between dissolve_free_hugetlb_folio() frees a HWPoison high-order folio to buddy allocator and MFR takes HWPoison raw page off buddy allocator. One obvious way to avoid this problem is to add page sanity checks in page allocate or free path. However, it is against the past efforts to reduce sanity check overhead [1,2,3]. Introduce free_has_hwpoisoned() to only free the healthy pages and to exclude the HWPoison ones in the high-order folio. The idea is to iterate through the sub-pages of the folio to identify contiguous ranges of healthy pages. Instead of freeing pages one by one, decompose healthy ranges into the largest possible blocks having different orders. Every block meets the requirements to be freed via __free_one_page(). free_has_hwpoisoned() has linear time complexity wrt the number of pages in the folio. While the power-of-two decomposition ensures that the number of calls to the buddy allocator is logarithmic for each contiguous healthy range, the mandatory linear scan of pages to identify PageHWPoison() defines the overall time complexity. For a 1G hugepage having several HWPoison pages, free_has_hwpoisoned() takes around 2ms on average. Since free_has_hwpoisoned() has nontrivial overhead, it is wrapped inside free_pages_prepare_has_hwpoisoned() and done only PG_has_hwpoisoned indicates HWPoison page exists and after free_pages_prepare() succeeded. [1] https://lore.kernel.org/linux-mm/1460711275-1130-15-git-send-email-mgor= man@techsingularity.net [2] https://lore.kernel.org/linux-mm/1460711275-1130-16-git-send-email-mgor= man@techsingularity.net [3] https://lore.kernel.org/all/20230216095131.17336-1-vbabka@suse.cz Signed-off-by: Jiaqi Yan --- mm/page_alloc.c | 157 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 154 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 822e05f1a9646..9393589118604 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -215,6 +215,9 @@ gfp_t gfp_allowed_mask __read_mostly =3D GFP_BOOT_MASK; unsigned int pageblock_order __read_mostly; #endif =20 +static bool free_pages_prepare_has_hwpoisoned(struct page *page, + unsigned int order, + fpi_t fpi_flags); static void __free_pages_ok(struct page *page, unsigned int order, fpi_t fpi_flags); =20 @@ -1568,8 +1571,10 @@ static void __free_pages_ok(struct page *page, unsig= ned int order, unsigned long pfn =3D page_to_pfn(page); struct zone *zone =3D page_zone(page); =20 - if (free_pages_prepare(page, order)) - free_one_page(zone, page, pfn, order, fpi_flags); + if (!free_pages_prepare_has_hwpoisoned(page, order, fpi_flags)) + return; + + free_one_page(zone, page, pfn, order, fpi_flags); } =20 void __meminit __free_pages_core(struct page *page, unsigned int order, @@ -2923,6 +2928,152 @@ static bool free_frozen_page_commit(struct zone *zo= ne, return ret; } =20 +/* + * Given a range of physically contiguous pages, efficiently free them + * block by block. Block order is chosen to meet the PFN alignment + * requirement in __free_one_page(). + */ +static void free_contiguous_pages(struct page *curr, unsigned long nr_page= s, + fpi_t fpi_flags) +{ + unsigned int order; + unsigned int align_order; + unsigned int size_order; + unsigned long remaining; + unsigned long pfn =3D page_to_pfn(curr); + const unsigned long end_pfn =3D pfn + nr_pages; + struct zone *zone =3D page_zone(curr); + + /* + * This decomposition algorithm at every iteration chooses the + * order to be the minimum of two constraints: + * - Alignment: the largest power-of-two that divides current pfn. + * - Size: the largest power-of-two that fits in the current + * remaining number of pages. + */ + while (pfn < end_pfn) { + remaining =3D end_pfn - pfn; + align_order =3D ffs(pfn) - 1; + size_order =3D fls_long(remaining) - 1; + order =3D min(align_order, size_order); + + free_one_page(zone, curr, pfn, order, fpi_flags); + curr +=3D (1UL << order); + pfn +=3D (1UL << order); + } + + VM_WARN_ON(pfn !=3D end_pfn); +} + +/* + * Given a high-order compound page containing certain number of HWPoison + * pages, free only the healthy ones to buddy allocator. + * + * Pages must have passed free_pages_prepare(). Even if having HWPoison + * pages, breaking down compound page and updating metadata (e.g. page + * owner, alloc tag) can be done together during free_pages_prepare(), + * which simplifies the splitting here: unlike __split_unmapped_folio(), + * there is no need to turn split pages into a compound page or to carry + * metadata. + * + * It calls free_one_page O(2^order) times and cause nontrivial overhead. + * So only use this when the compound page really contains HWPoison. + * + * This implementation doesn't work in memdesc world. + */ +static void free_has_hwpoisoned(struct page *page, unsigned int order, + fpi_t fpi_flags) +{ + struct page *curr =3D page; + struct page *next; + unsigned long nr_pages; + /* + * Don't assume end points to a valid page. It is only used + * here for pointer arithmetic. + */ + struct page *end =3D page + (1 << order); + unsigned long total_freed =3D 0; + unsigned long total_hwp =3D 0; + + VM_WARN_ON(order =3D=3D 0); + VM_WARN_ON(page->flags.f & PAGE_FLAGS_CHECK_AT_PREP); + + while (curr < end) { + next =3D curr; + nr_pages =3D 0; + + while (next < end && !PageHWPoison(next)) { + ++next; + ++nr_pages; + } + + if (next !=3D end && PageHWPoison(next)) { + clear_page_tag_ref(next); + ++total_hwp; + } + + free_contiguous_pages(curr, nr_pages, fpi_flags); + total_freed +=3D nr_pages; + if (next =3D=3D end) + break; + + curr =3D PageHWPoison(next) ? next + 1 : next; + } + + VM_WARN_ON(total_freed + total_hwp !=3D (1 << order)); + pr_info("Freed %#lx pages, excluded %lu hwpoison pages\n", + total_freed, total_hwp); +} + +static bool compound_has_hwpoisoned(struct page *page, unsigned int order) +{ + if (order =3D=3D 0 || !PageCompound(page)) + return false; + + return folio_test_has_hwpoisoned(page_folio(page)); +} + +/* + * Do free_has_hwpoisoned() when needed after free_pages_prepare(). + * Returns + * - true: free_pages_prepare() is good and caller can proceed freeing. + * - false: caller should not free pages for one of the two reasons: + * 1. free_pages_prepare() failed so it is not safe to proceed freeing. + * 2. this is a compound page having some HWPoison pages, and healthy + * pages are already safely freed. + */ +static bool free_pages_prepare_has_hwpoisoned(struct page *page, + unsigned int order, + fpi_t fpi_flags) +{ + /* + * free_pages_prepare() clears PAGE_FLAGS_SECOND flags on the + * first tail page of a compound page, which clears PG_has_hwpoisoned. + * So this call must be before free_pages_prepare(). + * + * Note we can't exclude PG_has_hwpoisoned from PAGE_FLAGS_SECOND. + * Because PG_has_hwpoisoned =3D=3D PG_active, free_page_is_bad() will + * confuse and complaint that the first tail page is still active. + */ + bool should_fhh =3D compound_has_hwpoisoned(page, order); + + if (!free_pages_prepare(page, order)) + return false; + + /* + * After free_pages_prepare() breaks down compound page and deals + * with page metadata (e.g. page owner and page alloc tags), + * free_has_hwpoisoned() can directly use free_one_page() whenever + * it knows the appropriate orders of page blocks to free. + */ + if (should_fhh) { + free_has_hwpoisoned(page, order, fpi_flags); + return false; + } + + return true; +} + /* * Free a pcp page */ @@ -2940,7 +3091,7 @@ static void __free_frozen_pages(struct page *page, un= signed int order, return; } =20 - if (!free_pages_prepare(page, order)) + if (!free_pages_prepare_has_hwpoisoned(page, order, fpi_flags)) return; =20 /* --=20 2.52.0.457.g6b5491de43-goog From nobody Sun Feb 8 05:30:29 2026 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC5B521C16E for ; Mon, 12 Jan 2026 00:49:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768178974; cv=none; b=gdr/5YrFQbRR4WG5j+LK32OqcMdHR728igZQ97TAit3GvcjRhtnX3ef1uNMdOMSbWgBjsTvL/tOcOej/N60OREcg9emUwm/0gN7jeIyzoOzDUb3aO7VEcykyjsLcgbNzE1R1LeeGFE+nLhnjGWSpF6VQ32z7h4l2dSf1+cEOeFY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768178974; c=relaxed/simple; bh=gDsNc4WRZ7bXIoryYSTAbaJUY95PrLrIDuXElm3sNrM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cj9HxD0HbLsXDsDkvRtd83ZwChNKjySz2WAFxn5gnGBXUa15BimkL5CMvxVXXLwc/f3JAKPLykoriAhE5eVaqDZBJB3kuqLRLdpTfbilTC1H5n+Zu+fkRwkQcjMuXQ/AL6wTIWM7xK4+YzQDlC9xbs0RvRotYRqflY1bMmT0Jcc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jiaqiyan.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fDEE6qnc; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jiaqiyan.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fDEE6qnc" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-81ea3358dd3so1514705b3a.3 for ; Sun, 11 Jan 2026 16:49:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1768178972; x=1768783772; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tOfkfgub/NaDAEiuWC6wAGOFEvU0NuNfiux/MMIfgrM=; b=fDEE6qncv0M5g1RfwdR0fZTURYRSuZcHmyp/S993bAxW4TDCbtvy8hffm8s1lpfTYi N6KA1LglOmVHF1N+sTHDsRDLLfYHUDPPkfDwOt6GdHqhZe67kMXh/daNKowknSCqg6q4 wInvNjdMu2MWAGFSsDzpvp/fvr1z7IOH2YOyAf0XRTVKzWpJsuUnqtELl9X5pxdgV+x5 jLIAVMgRdFm+AyCghRbfyRuDriNjSUSp2DB9wfYva0xRgFdpqYuIhkIG+62E+mXK+lFN yZA2Pg3EgWr6DR/JrPQA2GoJ1TSQtgXLKFARKdfOtYBctSLxCTnMqOecXJrkaMnfD9c/ 6RWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768178972; x=1768783772; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tOfkfgub/NaDAEiuWC6wAGOFEvU0NuNfiux/MMIfgrM=; b=hqXONJsNvKFMtjp6tfNOhIHmY1Y/LNz/3f/F7EId1yqAf2sJFYvZiy39mrXAMgwAy8 zAYgcwpvyWiRngeyqEhVBKfBIuYpsUpE5JOptDD7ua+rYmIbpmkP+HXXWgIbTabxjZrA um7n68yAmZSvbTdOFm+HuGB/pv25cX0XvzTH2phOpf3YOKhT6JDy9OVaEcdtyJ8cFmPb Fs2OFimwe8S9wa10BUJ0CYaQ4VqaHfO6UWXylli0ZSD86Qpl+ZzVR7prhTLGHM1hlTPm yVXQbHFB637vjJ2X6pklmfLsVpTnx21YcLr4dYEeOnNwO56Cx8KugW4fk/Ft5xiLZwzn YGgw== X-Forwarded-Encrypted: i=1; AJvYcCV2cF2ligJRFtNg6v8EK5EA5090tHr26M1B8piXeBBJGp5G7rmwRnDg6AdE38fXLZgNrjsRh3vtM/2YX6U=@vger.kernel.org X-Gm-Message-State: AOJu0YxgXMQhyG9L9grsNoF9vpALWYr4DPuRfxZbp5Z4qpmJIqVzcNHE ihy7SUALs1iETjIXVzZ+62UouZohH3tax6/wKbpbYKH+XH0jIoNPgQVGKMSlNYuKIoPt6Sy5SqA 6XMqbuQ0jSIcoWg== X-Google-Smtp-Source: AGHT+IE3mm0jWQWNZBJBvrBw2R/8R6/gLaYBFwhcilPPZzp5OWEiyGaY2M5Ofj7hXuABGa1C9diKGvBO24qNGg== X-Received: from pfib20.prod.google.com ([2002:aa7:8114:0:b0:7b0:bc2e:9595]) (user=jiaqiyan job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:4516:b0:81f:50b1:51fc with SMTP id d2e1a72fcca58-81f50b171b3mr2334123b3a.10.1768178972099; Sun, 11 Jan 2026 16:49:32 -0800 (PST) Date: Mon, 12 Jan 2026 00:49:23 +0000 In-Reply-To: <20260112004923.888429-1-jiaqiyan@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260112004923.888429-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.52.0.457.g6b5491de43-goog Message-ID: <20260112004923.888429-4-jiaqiyan@google.com> Subject: [PATCH v3 3/3] mm/memory-failure: refactor page_handle_poison() From: Jiaqi Yan To: jackmanb@google.com, hannes@cmpxchg.org, linmiaohe@huawei.com, ziy@nvidia.com, harry.yoo@oracle.com, willy@infradead.org Cc: nao.horiguchi@gmail.com, david@redhat.com, lorenzo.stoakes@oracle.com, william.roche@oracle.com, tony.luck@intel.com, wangkefeng.wang@huawei.com, jane.chu@oracle.com, akpm@linux-foundation.org, osalvador@suse.de, muchun.song@linux.dev, rientjes@google.com, duenwen@google.com, jthoughton@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, Jiaqi Yan Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that HWPoison page(s) within HugeTLB page will be rejected by buddy allocator during dissolve_free_hugetlb_folio(), there is no need to drain_all_pages() and take_page_off_buddy() anymore. In fact, calling take_page_off_buddy() after dissolve_free_hugetlb_folio() succeeded returns false, making caller think page_handl_poion() failed. On the other hand, for hardware corrupted pages in buddy allocator, take_page_off_buddy() is still a must-have. Given hugepage and free buddy page should be treated differently, refactor page_handle_poison() and __page_handle_poison(): - __page_handle_poison() is unwind into page_handle_poison(). - Callers of page_handle_poison() also need to explicitly tell if page is HugeTLB hugepage or free buddy page. - Add helper hugepage_handle_poison() for several existing HugeTLB specific callsites. Signed-off-by: Jiaqi Yan --- mm/memory-failure.c | 84 ++++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 43 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index d204de6c9792a..1fdaee1e48bb8 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -162,54 +162,48 @@ static struct rb_root_cached pfn_space_itree =3D RB_R= OOT_CACHED; =20 static DEFINE_MUTEX(pfn_space_lock); =20 -/* - * Return values: - * 1: the page is dissolved (if needed) and taken off from buddy, - * 0: the page is dissolved (if needed) and not taken off from buddy, - * < 0: failed to dissolve. +/** + * Handle the HugeTLB hugepage that @page belongs to. Return values: + * =3D 0: the hugepage is free hugepage and is dissolved. + * < 0: hugepage is in-use or failed to dissolve. */ -static int __page_handle_poison(struct page *page) +static int hugepage_handle_poison(struct page *page) { - int ret; + return dissolve_free_hugetlb_folio(page_folio(page)); +} + +/** + * Helper at the end of handling @page having hardware errors. + * @huge: @page is part of a HugeTLB hugepage. + * @free: @page is free buddy page. + * @release: memory-failure module should release a pending refcount. + */ +static bool page_handle_poison(struct page *page, bool huge, bool free, + bool release) +{ + int ret =3D 0; =20 /* - * zone_pcp_disable() can't be used here. It will - * hold pcp_batch_high_lock and dissolve_free_hugetlb_folio() might hold - * cpu_hotplug_lock via static_key_slow_dec() when hugetlb vmemmap - * optimization is enabled. This will break current lock dependency - * chain and leads to deadlock. - * Disabling pcp before dissolving the page was a deterministic - * approach because we made sure that those pages cannot end up in any - * PCP list. Draining PCP lists expels those pages to the buddy system, - * but nothing guarantees that those pages do not get back to a PCP - * queue if we need to refill those. + * Buddy allocator will exclude the HWPoison page after hugepage + * is successfully dissolved. */ - ret =3D dissolve_free_hugetlb_folio(page_folio(page)); - if (!ret) { + if (huge) + ret =3D hugepage_handle_poison(page); + + if (free) { drain_all_pages(page_zone(page)); - ret =3D take_page_off_buddy(page); + ret =3D take_page_off_buddy(page) ? 0 : -1; } =20 - return ret; -} - -static bool page_handle_poison(struct page *page, bool hugepage_or_freepag= e, bool release) -{ - if (hugepage_or_freepage) { + if ((huge || free) && ret < 0) /* - * Doing this check for free pages is also fine since - * dissolve_free_hugetlb_folio() returns 0 for non-hugetlb folios as wel= l. + * We could fail to take off the target page from buddy + * for example due to racy page allocation, but that's + * acceptable because soft-offlined page is not broken + * and if someone really want to use it, they should + * take it. */ - if (__page_handle_poison(page) <=3D 0) - /* - * We could fail to take off the target page from buddy - * for example due to racy page allocation, but that's - * acceptable because soft-offlined page is not broken - * and if someone really want to use it, they should - * take it. - */ - return false; - } + return false; =20 SetPageHWPoison(page); if (release) @@ -1174,7 +1168,7 @@ static int me_huge_page(struct page_state *ps, struct= page *p) * subpages. */ folio_put(folio); - if (__page_handle_poison(p) > 0) { + if (!hugepage_handle_poison(p)) { page_ref_inc(p); res =3D MF_RECOVERED; } else { @@ -2067,7 +2061,7 @@ static int try_memory_failure_hugetlb(unsigned long p= fn, int flags, int *hugetlb */ if (res =3D=3D 0) { folio_unlock(folio); - if (__page_handle_poison(p) > 0) { + if (!hugepage_handle_poison(p)) { page_ref_inc(p); res =3D MF_RECOVERED; } else { @@ -2815,7 +2809,7 @@ static int soft_offline_in_use_page(struct page *page) =20 if (ret) { pr_info("%#lx: invalidated\n", pfn); - page_handle_poison(page, false, true); + page_handle_poison(page, false, false, true); return 0; } =20 @@ -2836,7 +2830,7 @@ static int soft_offline_in_use_page(struct page *page) if (!ret) { bool release =3D !huge; =20 - if (!page_handle_poison(page, huge, release)) + if (!page_handle_poison(page, huge, false, release)) ret =3D -EBUSY; } else { if (!list_empty(&pagelist)) @@ -2884,6 +2878,8 @@ int soft_offline_page(unsigned long pfn, int flags) { int ret; bool try_again =3D true; + bool huge; + bool free; struct page *page; =20 if (!pfn_valid(pfn)) { @@ -2929,7 +2925,9 @@ int soft_offline_page(unsigned long pfn, int flags) if (ret > 0) { ret =3D soft_offline_in_use_page(page); } else if (ret =3D=3D 0) { - if (!page_handle_poison(page, true, false)) { + huge =3D folio_test_hugetlb(page_folio(page)); + free =3D is_free_buddy_page(page); + if (!page_handle_poison(page, huge, free, false)) { if (try_again) { try_again =3D false; flags &=3D ~MF_COUNT_INCREASED; --=20 2.52.0.457.g6b5491de43-goog