From nobody Sat Apr 11 06:35:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9995EC25B08 for ; Mon, 15 Aug 2022 01:03:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231747AbiHOBDo (ORCPT ); Sun, 14 Aug 2022 21:03:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229441AbiHOBDk (ORCPT ); Sun, 14 Aug 2022 21:03:40 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3EAF4DFC2 for ; Sun, 14 Aug 2022 18:03:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1660525419; x=1692061419; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SnO+KuyY2Ig6pp+7jgz3mFNr913uLAxnryQVGv4z4qw=; b=TfgyAjTcLo34s2I8WfF5bBGn+TaA2C8c7VhMHpKpIe6ekij2VBVBVP1H p8Jq1lNeXQb4LPu3ODePz5njFjwTzAfOtS+MeSPvYB1G40DuOaOh29RGr bH6tYnWEQGH1btFSaYjdEvntyB7EWnI8l89g07/jbMOBQAM8cNXjelMps VL9Svp9oE4P9p80/yVleYN1fsezrvBn1WA5O0wLRIRu8mufxQmZq0gcZz kIH+BDKVN9D3DBKslknsgBOKLPjU2/2bBJXY4Q9Stxq5pS9ZxI7qOcLBv 89oSxIqCN11oAhE7jlCuMvU7fzAB+42jE+b8GZrsjb7bMfrWz5S0L6otN Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10439"; a="355872161" X-IronPort-AV: E=Sophos;i="5.93,237,1654585200"; d="scan'208";a="355872161" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Aug 2022 18:03:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,237,1654585200"; d="scan'208";a="695823581" Received: from sse-cse-haiyue-nuc.sh.intel.com ([10.239.241.114]) by FMSMGA003.fm.intel.com with ESMTP; 14 Aug 2022 18:03:36 -0700 From: Haiyue Wang To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, ying.huang@intel.com, songmuchun@bytedance.com, naoya.horiguchi@linux.dev, alex.sierra@amd.com, Haiyue Wang Subject: [PATCH v3 1/2] mm: migration: fix the FOLL_GET failure on following huge page Date: Mon, 15 Aug 2022 09:03:48 +0800 Message-Id: <20220815010349.432313-2-haiyue.wang@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220815010349.432313-1-haiyue.wang@intel.com> References: <20220812084921.409142-1-haiyue.wang@intel.com> <20220815010349.432313-1-haiyue.wang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Not all huge page APIs support FOLL_GET option, so the __NR_move_pages will fail to get the page node information for huge page. This is an temporary solution to mitigate the racing fix. After supporting follow huge page by FOLL_GET is done, this fix can be reverted safely. Fixes: 4cd614841c06 ("mm: migration: fix possible do_pages_stat_array racin= g with memory offline") Signed-off-by: Haiyue Wang --- mm/migrate.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 6a1597c92261..581dfaad9257 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1848,6 +1848,7 @@ static void do_pages_stat_array(struct mm_struct *mm,= unsigned long nr_pages, =20 for (i =3D 0; i < nr_pages; i++) { unsigned long addr =3D (unsigned long)(*pages); + unsigned int foll_flags =3D FOLL_DUMP; struct vm_area_struct *vma; struct page *page; int err =3D -EFAULT; @@ -1856,8 +1857,12 @@ static void do_pages_stat_array(struct mm_struct *mm= , unsigned long nr_pages, if (!vma) goto set_status; =20 + /* Not all huge page follow APIs support 'FOLL_GET' */ + if (!is_vm_hugetlb_page(vma)) + foll_flags |=3D FOLL_GET; + /* FOLL_DUMP to ignore special (like zero) pages */ - page =3D follow_page(vma, addr, FOLL_GET | FOLL_DUMP); + page =3D follow_page(vma, addr, foll_flags); =20 err =3D PTR_ERR(page); if (IS_ERR(page)) @@ -1865,7 +1870,8 @@ static void do_pages_stat_array(struct mm_struct *mm,= unsigned long nr_pages, =20 if (page && !is_zone_device_page(page)) { err =3D page_to_nid(page); - put_page(page); + if (foll_flags & FOLL_GET) + put_page(page); } else { err =3D -ENOENT; } --=20 2.37.2 From nobody Sat Apr 11 06:35:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8CCFC25B06 for ; Mon, 15 Aug 2022 01:03:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231751AbiHOBDr (ORCPT ); Sun, 14 Aug 2022 21:03:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229714AbiHOBDn (ORCPT ); Sun, 14 Aug 2022 21:03:43 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E35EDFEC for ; Sun, 14 Aug 2022 18:03:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1660525422; x=1692061422; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=g1Gz490N3X+xinSJ0Yj8+3Bx6+nLZMmSv2ioQICZ2Ro=; b=HEhY/es2e8AsHgiR/+L7+YZYBOMSTrGlhUd93ZFjb704oVAzKfVuWf9x zhaDjppVwCNIcnCvrCySovqwJhBgSvLyzIY3sKYpSANgbuw/xSLULWwB/ iAuCLB9BLYwE9ChHscicbSU4fq7KSQcTiIxHFztWk7u/uPF02lU0Tvg+r nKLHl5XHR7Bz25A9KVvhdp6JAuEkmsMoIDxPAVJz9Ktf8rkUTnJX6ZrcL zNDMa+O9SMXy6thJKK+R7d6jE3sR2K3ez/IpHwwLYRHWzsra8+aUA8+D/ 2YT4cVsL6ldCFUQh5Muqdz+g6yj+ibQyNnxdK0+hjEOC6FuTMEp4F8NmC Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10439"; a="355872165" X-IronPort-AV: E=Sophos;i="5.93,237,1654585200"; d="scan'208";a="355872165" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Aug 2022 18:03:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,237,1654585200"; d="scan'208";a="695823591" Received: from sse-cse-haiyue-nuc.sh.intel.com ([10.239.241.114]) by FMSMGA003.fm.intel.com with ESMTP; 14 Aug 2022 18:03:39 -0700 From: Haiyue Wang To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, ying.huang@intel.com, songmuchun@bytedance.com, naoya.horiguchi@linux.dev, alex.sierra@amd.com, Haiyue Wang , Felix Kuehling , Alistair Popple Subject: [PATCH v3 2/2] mm: fix the handling Non-LRU pages returned by follow_page Date: Mon, 15 Aug 2022 09:03:49 +0800 Message-Id: <20220815010349.432313-3-haiyue.wang@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220815010349.432313-1-haiyue.wang@intel.com> References: <20220812084921.409142-1-haiyue.wang@intel.com> <20220815010349.432313-1-haiyue.wang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The page returned by follow_page with 'FOLL_GET' has get_page called, so it needs to call put_page for handling the reference count correctly. And as David reviewed, "device pages are never PageKsm pages". Drop this zone device page check for break_ksm. Fixes: 3218f8712d6b ("mm: handling Non-LRU pages returned by vm_normal_page= s") Signed-off-by: Haiyue Wang --- mm/huge_memory.c | 4 ++-- mm/ksm.c | 12 +++++++++--- mm/migrate.c | 10 +++++++--- 3 files changed, 18 insertions(+), 8 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 8a7c1b344abe..b2ba17c3dcd7 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2963,10 +2963,10 @@ static int split_huge_pages_pid(int pid, unsigned l= ong vaddr_start, /* FOLL_DUMP to ignore special (like zero) pages */ page =3D follow_page(vma, addr, FOLL_GET | FOLL_DUMP); =20 - if (IS_ERR_OR_NULL(page) || is_zone_device_page(page)) + if (IS_ERR_OR_NULL(page)) continue; =20 - if (!is_transparent_hugepage(page)) + if (is_zone_device_page(page) || !is_transparent_hugepage(page)) goto next; =20 total++; diff --git a/mm/ksm.c b/mm/ksm.c index 42ab153335a2..e26f57fc1f0e 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -475,7 +475,7 @@ static int break_ksm(struct vm_area_struct *vma, unsign= ed long addr) cond_resched(); page =3D follow_page(vma, addr, FOLL_GET | FOLL_MIGRATION | FOLL_REMOTE); - if (IS_ERR_OR_NULL(page) || is_zone_device_page(page)) + if (IS_ERR_OR_NULL(page)) break; if (PageKsm(page)) ret =3D handle_mm_fault(vma, addr, @@ -560,12 +560,15 @@ static struct page *get_mergeable_page(struct rmap_it= em *rmap_item) goto out; =20 page =3D follow_page(vma, addr, FOLL_GET); - if (IS_ERR_OR_NULL(page) || is_zone_device_page(page)) + if (IS_ERR_OR_NULL(page)) goto out; + if (is_zone_device_page(page)) + goto out_putpage; if (PageAnon(page)) { flush_anon_page(vma, page, addr); flush_dcache_page(page); } else { +out_putpage: put_page(page); out: page =3D NULL; @@ -2308,11 +2311,13 @@ static struct rmap_item *scan_get_next_rmap_item(st= ruct page **page) if (ksm_test_exit(mm)) break; *page =3D follow_page(vma, ksm_scan.address, FOLL_GET); - if (IS_ERR_OR_NULL(*page) || is_zone_device_page(*page)) { + if (IS_ERR_OR_NULL(*page)) { ksm_scan.address +=3D PAGE_SIZE; cond_resched(); continue; } + if (is_zone_device_page(*page)) + goto next_page; if (PageAnon(*page)) { flush_anon_page(vma, *page, ksm_scan.address); flush_dcache_page(*page); @@ -2327,6 +2332,7 @@ static struct rmap_item *scan_get_next_rmap_item(stru= ct page **page) mmap_read_unlock(mm); return rmap_item; } +next_page: put_page(*page); ksm_scan.address +=3D PAGE_SIZE; cond_resched(); diff --git a/mm/migrate.c b/mm/migrate.c index 581dfaad9257..fee12cd2f294 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1672,9 +1672,12 @@ static int add_page_for_migration(struct mm_struct *= mm, unsigned long addr, goto out; =20 err =3D -ENOENT; - if (!page || is_zone_device_page(page)) + if (!page) goto out; =20 + if (is_zone_device_page(page)) + goto out_putpage; + err =3D 0; if (page_to_nid(page) =3D=3D node) goto out_putpage; @@ -1868,8 +1871,9 @@ static void do_pages_stat_array(struct mm_struct *mm,= unsigned long nr_pages, if (IS_ERR(page)) goto set_status; =20 - if (page && !is_zone_device_page(page)) { - err =3D page_to_nid(page); + if (page) { + err =3D !is_zone_device_page(page) ? page_to_nid(page) + : -ENOENT; if (foll_flags & FOLL_GET) put_page(page); } else { --=20 2.37.2