From nobody Wed Apr 8 12:13:15 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EF83C54EE9 for ; Mon, 5 Sep 2022 06:22:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236556AbiIEGWQ (ORCPT ); Mon, 5 Sep 2022 02:22:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236540AbiIEGWN (ORCPT ); Mon, 5 Sep 2022 02:22:13 -0400 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A34E92CE22 for ; Sun, 4 Sep 2022 23:22:12 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1662358931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ucAIXNnH5R2gOfgMvHbR1y7ug9n/ezUmfveCIJw4IxY=; b=ASM6HcWZzDLxIDXs3ez6SRtQhvGY3pPYeVjp5UhQSb4qv3mL+xT5hh7Qb0N6USjo4XDIWS JIO6ZYdRQqpOMgdLzA+JiCmflMW8UrAFrA3yqYD3NyoawZpIBB1E6LRtznziKUchM2IOm4 6GyjKk9KlgbmHYdoU+2LrbRLBKQ9gpY= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 1/4] mm,hwpoison,hugetlb,memory_hotplug: hotremove memory section with hwpoisoned hugepage Date: Mon, 5 Sep 2022 15:21:34 +0900 Message-Id: <20220905062137.1455537-2-naoya.horiguchi@linux.dev> In-Reply-To: <20220905062137.1455537-1-naoya.horiguchi@linux.dev> References: <20220905062137.1455537-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Naoya Horiguchi HWPoisoned page is not supposed to be accessed once marked, but currently such accesses can happen during memory hotremove because do_migrate_range() can be called before dissolve_free_huge_pages() is called. Move dissolve_free_huge_pages() before scan_movable_pages(). Recently delayed dissolve has been implemented, so the dissolving can turn a hwpoisoned hugepage into 4kB hwpoison page, which memory hotplug can handle safely. Reported-by: Miaohe Lin Signed-off-by: Naoya Horiguchi --- mm/memory_hotplug.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index fad6d1f2262a..c24735d63b25 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1880,6 +1880,17 @@ int __ref offline_pages(unsigned long start_pfn, uns= igned long nr_pages, =20 cond_resched(); =20 + /* + * Dissolve free hugepages in the memory block before doing + * offlining actually in order to make hugetlbfs's object + * counting consistent. + */ + ret =3D dissolve_free_huge_pages(start_pfn, end_pfn); + if (ret) { + reason =3D "failure to dissolve huge pages"; + goto failed_removal_isolated; + } + ret =3D scan_movable_pages(pfn, end_pfn, &pfn); if (!ret) { /* @@ -1895,17 +1906,6 @@ int __ref offline_pages(unsigned long start_pfn, uns= igned long nr_pages, goto failed_removal_isolated; } =20 - /* - * Dissolve free hugepages in the memory block before doing - * offlining actually in order to make hugetlbfs's object - * counting consistent. - */ - ret =3D dissolve_free_huge_pages(start_pfn, end_pfn); - if (ret) { - reason =3D "failure to dissolve huge pages"; - goto failed_removal_isolated; - } - ret =3D test_pages_isolated(start_pfn, end_pfn, MEMORY_OFFLINE); =20 } while (ret); --=20 2.25.1 From nobody Wed Apr 8 12:13:15 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59658ECAAD5 for ; Mon, 5 Sep 2022 06:22:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236567AbiIEGWV (ORCPT ); Mon, 5 Sep 2022 02:22:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236540AbiIEGWS (ORCPT ); Mon, 5 Sep 2022 02:22:18 -0400 Received: from out2.migadu.com (out2.migadu.com [IPv6:2001:41d0:2:aacc::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D5AD2CE22 for ; Sun, 4 Sep 2022 23:22:17 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1662358935; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KJKmQBcrfqjRJ7vJe0VgHKDJG/ZTJxuNlianmOytPC4=; b=GqjZ2HY6ixn3Q6PsBSb1oIPpXpTKNnhcPJ+LjiO6VJRmjy3qgnpcVjTqKzisan0dnC7an0 dRzhW9LH3rrXfBUpKvST7f/WeZiCwGcoeqr8heKUJFQvtmLc+gWnP/0xKbbZ0GIxmgywpb tw5sxGPxHjKhNTRggiM5Mpdcbdi/7cU= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 2/4] mm/hwpoison: move definitions of num_poisoned_pages_* to memory-failure.c Date: Mon, 5 Sep 2022 15:21:35 +0900 Message-Id: <20220905062137.1455537-3-naoya.horiguchi@linux.dev> In-Reply-To: <20220905062137.1455537-1-naoya.horiguchi@linux.dev> References: <20220905062137.1455537-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Naoya Horiguchi These interfaces will be used by drivers/base/core.c by later patch, so as a preparatory work move them to more common header file visible to the file. Signed-off-by: Naoya Horiguchi --- arch/parisc/kernel/pdt.c | 3 +-- include/linux/mm.h | 4 ++++ include/linux/swapops.h | 25 ------------------------- mm/memory-failure.c | 15 +++++++++++++++ 4 files changed, 20 insertions(+), 27 deletions(-) diff --git a/arch/parisc/kernel/pdt.c b/arch/parisc/kernel/pdt.c index e391b175f5ec..fdc880e2575a 100644 --- a/arch/parisc/kernel/pdt.c +++ b/arch/parisc/kernel/pdt.c @@ -18,8 +18,7 @@ #include #include #include -#include -#include +#include =20 #include #include diff --git a/include/linux/mm.h b/include/linux/mm.h index 21f8b27bd9fd..b81dd600e51a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3202,6 +3202,10 @@ static inline int __get_huge_page_for_hwpoison(unsig= ned long pfn, int flags) { return 0; } + +static inline void num_poisoned_pages_inc() +{ +} #endif =20 #ifndef arch_memory_failure diff --git a/include/linux/swapops.h b/include/linux/swapops.h index ddc98f96ad2c..55afc2aaba6b 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -459,8 +459,6 @@ static inline int is_pmd_migration_entry(pmd_t pmd) =20 #ifdef CONFIG_MEMORY_FAILURE =20 -extern atomic_long_t num_poisoned_pages __read_mostly; - /* * Support for hardware poisoned pages */ @@ -488,21 +486,6 @@ static inline struct page *hwpoison_entry_to_page(swp_= entry_t entry) return p; } =20 -static inline void num_poisoned_pages_inc(void) -{ - atomic_long_inc(&num_poisoned_pages); -} - -static inline void num_poisoned_pages_dec(void) -{ - atomic_long_dec(&num_poisoned_pages); -} - -static inline void num_poisoned_pages_sub(long i) -{ - atomic_long_sub(i, &num_poisoned_pages); -} - #else =20 static inline swp_entry_t make_hwpoison_entry(struct page *page) @@ -519,14 +502,6 @@ static inline struct page *hwpoison_entry_to_page(swp_= entry_t entry) { return NULL; } - -static inline void num_poisoned_pages_inc(void) -{ -} - -static inline void num_poisoned_pages_sub(long i) -{ -} #endif =20 static inline int non_swap_entry(swp_entry_t entry) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 7b077da568ff..b6236c721f54 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -74,6 +74,21 @@ atomic_long_t num_poisoned_pages __read_mostly =3D ATOMI= C_LONG_INIT(0); =20 static bool hw_memory_failure __read_mostly =3D false; =20 +static inline void num_poisoned_pages_inc(void) +{ + atomic_long_inc(&num_poisoned_pages); +} + +static inline void num_poisoned_pages_dec(void) +{ + atomic_long_dec(&num_poisoned_pages); +} + +static inline void num_poisoned_pages_sub(long i) +{ + atomic_long_sub(i, &num_poisoned_pages); +} + /* * Return values: * 1: the page is dissolved (if needed) and taken off from buddy, --=20 2.25.1 From nobody Wed Apr 8 12:13:15 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEFF6ECAAD5 for ; Mon, 5 Sep 2022 06:22:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236589AbiIEGW0 (ORCPT ); Mon, 5 Sep 2022 02:22:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236085AbiIEGWV (ORCPT ); Mon, 5 Sep 2022 02:22:21 -0400 Received: from out2.migadu.com (out2.migadu.com [IPv6:2001:41d0:2:aacc::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC17A2B261 for ; Sun, 4 Sep 2022 23:22:20 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1662358939; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/BbMhqxmYuU5COA45Jj7QmdMCnLs62eTq+39BWVy6eY=; b=IjFxF5qczN8wu+FeEj3Qhjhq/1V0oaMQc25LmtpYuj1uUY4VP8dD5LVY9YyQZ9mNfHuVzH UspenGcv02ijkXrI63eUFKBY422C5EGcmiv4obWhAk/rglpi/RPOUFdvNnL6gZFRCcvtmk +PZG4ZxxnvmSjxm3py1cyS6xQ+czEPc= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 3/4] mm/hwpoison: pass pfn to num_poisoned_pages_*() Date: Mon, 5 Sep 2022 15:21:36 +0900 Message-Id: <20220905062137.1455537-4-naoya.horiguchi@linux.dev> In-Reply-To: <20220905062137.1455537-1-naoya.horiguchi@linux.dev> References: <20220905062137.1455537-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Naoya Horiguchi No functional change. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- arch/parisc/kernel/pdt.c | 2 +- include/linux/mm.h | 2 +- mm/memory-failure.c | 16 ++++++++-------- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/arch/parisc/kernel/pdt.c b/arch/parisc/kernel/pdt.c index fdc880e2575a..80943a00e245 100644 --- a/arch/parisc/kernel/pdt.c +++ b/arch/parisc/kernel/pdt.c @@ -231,7 +231,7 @@ void __init pdc_pdt_init(void) =20 /* mark memory page bad */ memblock_reserve(pdt_entry[i] & PAGE_MASK, PAGE_SIZE); - num_poisoned_pages_inc(); + num_poisoned_pages_inc(addr >> PAGE_SHIFT); } } =20 diff --git a/include/linux/mm.h b/include/linux/mm.h index b81dd600e51a..6316973afd1d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3203,7 +3203,7 @@ static inline int __get_huge_page_for_hwpoison(unsign= ed long pfn, int flags) return 0; } =20 -static inline void num_poisoned_pages_inc() +static inline void num_poisoned_pages_inc(unsigned long pfn) { } #endif diff --git a/mm/memory-failure.c b/mm/memory-failure.c index b6236c721f54..7dd4e403e634 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -74,17 +74,17 @@ atomic_long_t num_poisoned_pages __read_mostly =3D ATOM= IC_LONG_INIT(0); =20 static bool hw_memory_failure __read_mostly =3D false; =20 -static inline void num_poisoned_pages_inc(void) +static inline void num_poisoned_pages_inc(unsigned long pfn) { atomic_long_inc(&num_poisoned_pages); } =20 -static inline void num_poisoned_pages_dec(void) +static inline void num_poisoned_pages_dec(unsigned long pfn) { atomic_long_dec(&num_poisoned_pages); } =20 -static inline void num_poisoned_pages_sub(long i) +static inline void num_poisoned_pages_sub(unsigned long pfn, long i) { atomic_long_sub(i, &num_poisoned_pages); } @@ -130,7 +130,7 @@ static bool page_handle_poison(struct page *page, bool = hugepage_or_freepage, boo if (release) put_page(page); page_ref_inc(page); - num_poisoned_pages_inc(); + num_poisoned_pages_inc(page_to_pfn(page)); =20 return true; } @@ -1196,7 +1196,7 @@ static void action_result(unsigned long pfn, enum mf_= action_page_type type, { trace_memory_failure_event(pfn, type, result); =20 - num_poisoned_pages_inc(); + num_poisoned_pages_inc(pfn); pr_err("%#lx: recovery action for %s: %s\n", pfn, action_page_types[type], action_name[result]); } @@ -1743,7 +1743,7 @@ static int hugetlb_set_page_hwpoison(struct page *hpa= ge, struct page *page) llist_add(&raw_hwp->node, head); /* the first error event will be counted in action_result(). */ if (ret) - num_poisoned_pages_inc(); + num_poisoned_pages_inc(page_to_pfn(page)); } else { /* * Failed to save raw error info. We no longer trace all @@ -2408,7 +2408,7 @@ int unpoison_memory(unsigned long pfn) unlock_mutex: mutex_unlock(&mf_mutex); if (!ret || freeit) { - num_poisoned_pages_sub(count); + num_poisoned_pages_sub(pfn, count); unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", page_to_pfn(p), &unpoison_rs); } @@ -2625,7 +2625,7 @@ void clear_hwpoisoned_pages(struct page *memmap, int = nr_pages) =20 for (i =3D 0; i < nr_pages; i++) { if (PageHWPoison(&memmap[i])) { - num_poisoned_pages_dec(); + num_poisoned_pages_dec(page_to_pfn(&memmap[i])); ClearPageHWPoison(&memmap[i]); } } --=20 2.25.1 From nobody Wed Apr 8 12:13:15 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8E8AECAAD5 for ; Mon, 5 Sep 2022 06:22:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235616AbiIEGWc (ORCPT ); Mon, 5 Sep 2022 02:22:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236590AbiIEGW0 (ORCPT ); Mon, 5 Sep 2022 02:22:26 -0400 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1099F2D1F8 for ; Sun, 4 Sep 2022 23:22:25 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1662358943; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TlE1DFZak7ox33QEkAGjI26kF8cTbXvD0IXu7We9lGI=; b=WdSyPOOz6wbs25kknHNuAXt/eyuI8K+6rafCoflwSy7uxe9dFzVuRejvVmp9k2bqJUNn2T 1iEgkl03EfiXDfAMXUM2URIBaEFt6bwKG/zZ+TXLPR34Om9yQaz+LXwEIy73xJ6IL6T6qd yUbLpJ/tapOFE6clrIt5AA8XbgfZMeg= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 4/4] mm/hwpoison: introduce per-memory_block hwpoison counter Date: Mon, 5 Sep 2022 15:21:37 +0900 Message-Id: <20220905062137.1455537-5-naoya.horiguchi@linux.dev> In-Reply-To: <20220905062137.1455537-1-naoya.horiguchi@linux.dev> References: <20220905062137.1455537-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Naoya Horiguchi Currently PageHWPoison flag does not behave well when experiencing memory hotremove/hotplug. Any data field in struct page is unreliable when the associated memory is offlined, and the current mechanism can't tell whether a memory section is onlined because a new memory devices is installed or because previous failed offline operations are undone. Especially if there's a hwpoisoned memory, it's unclear what the best option is. So introduce a new mechanism to make struct memory_block remember that a memory block has hwpoisoned memory inside it. And make any online event fail if the onlined memory block contains hwpoison. struct memory_block is freed and reallocated over ACPI-based hotremove/hotplug, but not over sysfs-based hotremove/hotplug. So it's desirable to implement hwpoison counter on this struct. Note that clear_hwpoisoned_pages() is relocated to be called earlier than now, just before unregistering struct memory_block. Otherwise, the per-memory_block hwpoison counter is freed and we fail to adjust global hwpoison counter properly. Signed-off-by: Naoya Horiguchi --- drivers/base/memory.c | 36 ++++++++++++++++++++++++++++++++++++ include/linux/memory.h | 3 +++ include/linux/mm.h | 8 ++++++++ mm/internal.h | 8 -------- mm/memory-failure.c | 36 +++++++++++------------------------- mm/sparse.c | 2 -- 6 files changed, 58 insertions(+), 35 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index bc60c9cd3230..10e45083af52 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -183,6 +183,9 @@ static int memory_block_online(struct memory_block *mem) struct zone *zone; int ret; =20 + if (atomic_long_read(&mem->nr_hwpoison)) + return -EHWPOISON; + zone =3D zone_for_pfn_range(mem->online_type, mem->nid, mem->group, start_pfn, nr_pages); =20 @@ -864,6 +867,7 @@ void remove_memory_block_devices(unsigned long start, u= nsigned long size) mem =3D find_memory_block_by_id(block_id); if (WARN_ON_ONCE(!mem)) continue; + clear_hwpoisoned_pages(atomic_long_read(&mem->nr_hwpoison)); unregister_memory_block_under_nodes(mem); remove_memory_block(mem); } @@ -1170,3 +1174,35 @@ int walk_dynamic_memory_groups(int nid, walk_memory_= groups_func_t func, } return ret; } + +#ifdef CONFIG_MEMORY_FAILURE + +void memblk_nr_poison_inc(unsigned long pfn) +{ + const unsigned long block_id =3D pfn_to_block_id(pfn); + struct memory_block *mem =3D find_memory_block_by_id(block_id); + + if (mem) + atomic_long_inc(&mem->nr_hwpoison); +} + +void memblk_nr_poison_sub(unsigned long pfn, long i) +{ + const unsigned long block_id =3D pfn_to_block_id(pfn); + struct memory_block *mem =3D find_memory_block_by_id(block_id); + + if (mem) + atomic_long_sub(i, &mem->nr_hwpoison); +} + +unsigned long memblk_nr_poison(unsigned long pfn) +{ + const unsigned long block_id =3D pfn_to_block_id(pfn); + struct memory_block *mem =3D find_memory_block_by_id(block_id); + + if (mem) + return atomic_long_read(&mem->nr_hwpoison); + return 0; +} + +#endif diff --git a/include/linux/memory.h b/include/linux/memory.h index aa619464a1df..74e6b3ad947f 100644 --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -85,6 +85,9 @@ struct memory_block { unsigned long nr_vmemmap_pages; struct memory_group *group; /* group (if any) for this block */ struct list_head group_next; /* next block inside memory group */ +#ifdef CONFIG_MEMORY_FAILURE + atomic_long_t nr_hwpoison; +#endif }; =20 int arch_get_memory_phys_device(unsigned long start_pfn); diff --git a/include/linux/mm.h b/include/linux/mm.h index 6316973afd1d..951c3cdd7683 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3197,6 +3197,10 @@ extern atomic_long_t num_poisoned_pages __read_mostl= y; extern int soft_offline_page(unsigned long pfn, int flags); #ifdef CONFIG_MEMORY_FAILURE extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags); +extern void memblk_nr_poison_inc(unsigned long pfn); +extern void memblk_nr_poison_sub(unsigned long pfn, long i); +extern unsigned long memblk_nr_poison(unsigned long pfn); +extern void clear_hwpoisoned_pages(long nr_poison); #else static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flag= s) { @@ -3206,6 +3210,10 @@ static inline int __get_huge_page_for_hwpoison(unsig= ned long pfn, int flags) static inline void num_poisoned_pages_inc(unsigned long pfn) { } + +static inline void clear_hwpoisoned_pages(long nr_poison) +{ +} #endif =20 #ifndef arch_memory_failure diff --git a/mm/internal.h b/mm/internal.h index 785409805ed7..fa481fc04fb7 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -738,14 +738,6 @@ extern u64 hwpoison_filter_flags_value; extern u64 hwpoison_filter_memcg; extern u32 hwpoison_filter_enable; =20 -#ifdef CONFIG_MEMORY_FAILURE -void clear_hwpoisoned_pages(struct page *memmap, int nr_pages); -#else -static inline void clear_hwpoisoned_pages(struct page *memmap, int nr_page= s) -{ -} -#endif - extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned l= ong, unsigned long, unsigned long, unsigned long, unsigned long); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 7dd4e403e634..5f3a0351a200 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -74,19 +74,17 @@ atomic_long_t num_poisoned_pages __read_mostly =3D ATOM= IC_LONG_INIT(0); =20 static bool hw_memory_failure __read_mostly =3D false; =20 -static inline void num_poisoned_pages_inc(unsigned long pfn) +void num_poisoned_pages_inc(unsigned long pfn) { atomic_long_inc(&num_poisoned_pages); + memblk_nr_poison_inc(pfn); } =20 -static inline void num_poisoned_pages_dec(unsigned long pfn) -{ - atomic_long_dec(&num_poisoned_pages); -} - -static inline void num_poisoned_pages_sub(unsigned long pfn, long i) +void num_poisoned_pages_sub(unsigned long pfn, long i) { atomic_long_sub(i, &num_poisoned_pages); + if (pfn !=3D -1UL) + memblk_nr_poison_sub(pfn, i); } =20 /* @@ -2408,6 +2406,10 @@ int unpoison_memory(unsigned long pfn) unlock_mutex: mutex_unlock(&mf_mutex); if (!ret || freeit) { + /* + * TODO: per-memory_block counter might break when the page + * size to be unpoisoned is larger than a memory_block. + */ num_poisoned_pages_sub(pfn, count); unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", page_to_pfn(p), &unpoison_rs); @@ -2610,23 +2612,7 @@ int soft_offline_page(unsigned long pfn, int flags) return ret; } =20 -void clear_hwpoisoned_pages(struct page *memmap, int nr_pages) +void clear_hwpoisoned_pages(long nr_poison) { - int i; - - /* - * A further optimization is to have per section refcounted - * num_poisoned_pages. But that would need more space per memmap, so - * for now just do a quick global check to speed up this routine in the - * absence of bad pages. - */ - if (atomic_long_read(&num_poisoned_pages) =3D=3D 0) - return; - - for (i =3D 0; i < nr_pages; i++) { - if (PageHWPoison(&memmap[i])) { - num_poisoned_pages_dec(page_to_pfn(&memmap[i])); - ClearPageHWPoison(&memmap[i]); - } - } + num_poisoned_pages_sub(-1UL, nr_poison); } diff --git a/mm/sparse.c b/mm/sparse.c index e5a8a3a0edd7..2779b419ef2a 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -926,8 +926,6 @@ void sparse_remove_section(struct mem_section *ms, unsi= gned long pfn, unsigned long nr_pages, unsigned long map_offset, struct vmem_altmap *altmap) { - clear_hwpoisoned_pages(pfn_to_page(pfn) + map_offset, - nr_pages - map_offset); section_deactivate(pfn, nr_pages, altmap); } #endif /* CONFIG_MEMORY_HOTPLUG */ --=20 2.25.1