From nobody Sun May 10 16:26:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E93A0C433F5 for ; Fri, 29 Apr 2022 06:40:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354684AbiD2GoJ (ORCPT ); Fri, 29 Apr 2022 02:44:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354649AbiD2Gnv (ORCPT ); Fri, 29 Apr 2022 02:43:51 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE637B9F1A for ; Thu, 28 Apr 2022 23:40:33 -0700 (PDT) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4KqNBb5XYmz1JBqW; Fri, 29 Apr 2022 14:39:35 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 29 Apr 2022 14:40:31 +0800 From: Miaohe Lin To: , CC: , , Subject: [PATCH 1/9] mm/z3fold: fix sheduling while atomic Date: Fri, 29 Apr 2022 14:40:43 +0800 Message-ID: <20220429064051.61552-2-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220429064051.61552-1-linmiaohe@huawei.com> References: <20220429064051.61552-1-linmiaohe@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" z3fold's page_lock is always held when calling alloc_slots. So gfp should be GFP_ATOMIC to avoid "scheduling while atomic" bug. Fixes: fc5488651c7d ("z3fold: simplify freeing slots") Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool --- mm/z3fold.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index 83b5a3514427..c2260f5a5885 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -941,8 +941,7 @@ static inline struct z3fold_header *__z3fold_alloc(stru= ct z3fold_pool *pool, } =20 if (zhdr && !zhdr->slots) - zhdr->slots =3D alloc_slots(pool, - can_sleep ? GFP_NOIO : GFP_ATOMIC); + zhdr->slots =3D alloc_slots(pool, GFP_ATOMIC); return zhdr; } =20 --=20 2.23.0 From nobody Sun May 10 16:26:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 684A3C433F5 for ; Fri, 29 Apr 2022 06:40:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354662AbiD2Gnx (ORCPT ); Fri, 29 Apr 2022 02:43:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354650AbiD2Gnv (ORCPT ); Fri, 29 Apr 2022 02:43:51 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D045B9F1B for ; Thu, 28 Apr 2022 23:40:34 -0700 (PDT) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4KqNCP0CRZzhYq6; Fri, 29 Apr 2022 14:40:17 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 29 Apr 2022 14:40:32 +0800 From: Miaohe Lin To: , CC: , , Subject: [PATCH 2/9] mm/z3fold: fix possible null pointer dereferencing Date: Fri, 29 Apr 2022 14:40:44 +0800 Message-ID: <20220429064051.61552-3-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220429064051.61552-1-linmiaohe@huawei.com> References: <20220429064051.61552-1-linmiaohe@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" alloc_slots could fail to allocate memory under heavy memory pressure. So we should check zhdr->slots against NULL to avoid future null pointer dereferencing. Fixes: fc5488651c7d ("z3fold: simplify freeing slots") Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool --- mm/z3fold.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index c2260f5a5885..5d8c21f2bc59 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -940,9 +940,19 @@ static inline struct z3fold_header *__z3fold_alloc(str= uct z3fold_pool *pool, } } =20 - if (zhdr && !zhdr->slots) + if (zhdr && !zhdr->slots) { zhdr->slots =3D alloc_slots(pool, GFP_ATOMIC); + if (!zhdr->slots) + goto out_fail; + } return zhdr; + +out_fail: + if (!kref_put(&zhdr->refcount, release_z3fold_page_locked)) { + add_to_unbuddied(pool, zhdr); + z3fold_page_unlock(zhdr); + } + return NULL; } =20 /* --=20 2.23.0 From nobody Sun May 10 16:26:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD582C433EF for ; Fri, 29 Apr 2022 06:41:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354732AbiD2GoY (ORCPT ); Fri, 29 Apr 2022 02:44:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51924 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354665AbiD2Gnx (ORCPT ); Fri, 29 Apr 2022 02:43:53 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31D81B9F31 for ; Thu, 28 Apr 2022 23:40:35 -0700 (PDT) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4KqN8f4HK2zGpPK; Fri, 29 Apr 2022 14:37:54 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 29 Apr 2022 14:40:32 +0800 From: Miaohe Lin To: , CC: , , Subject: [PATCH 3/9] mm/z3fold: remove buggy use of stale list for allocation Date: Fri, 29 Apr 2022 14:40:45 +0800 Message-ID: <20220429064051.61552-4-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220429064051.61552-1-linmiaohe@huawei.com> References: <20220429064051.61552-1-linmiaohe@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Currently if z3fold couldn't find an unbuddied page it would first try to pull a page off the stale list. But this approach is problematic. If init z3fold page fails later, the page should be freed via free_z3fold_page to clean up the relevant resource instead of using __free_page directly. And if page is successfully reused, it will BUG_ON later in __SetPageMovable because it's already non-lru movable page, i.e. PAGE_MAPPING_MOVABLE is already set in page->mapping. In order to fix all of these issues, we can simply remove the buggy use of stale list for allocation because can_sleep should always be false and we never really hit the reusing code path now. Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool --- mm/z3fold.c | 23 +---------------------- 1 file changed, 1 insertion(+), 22 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index 5d8c21f2bc59..4e6814c5694f 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -1102,28 +1102,7 @@ static int z3fold_alloc(struct z3fold_pool *pool, si= ze_t size, gfp_t gfp, bud =3D FIRST; } =20 - page =3D NULL; - if (can_sleep) { - spin_lock(&pool->stale_lock); - zhdr =3D list_first_entry_or_null(&pool->stale, - struct z3fold_header, buddy); - /* - * Before allocating a page, let's see if we can take one from - * the stale pages list. cancel_work_sync() can sleep so we - * limit this case to the contexts where we can sleep - */ - if (zhdr) { - list_del(&zhdr->buddy); - spin_unlock(&pool->stale_lock); - cancel_work_sync(&zhdr->work); - page =3D virt_to_page(zhdr); - } else { - spin_unlock(&pool->stale_lock); - } - } - if (!page) - page =3D alloc_page(gfp); - + page =3D alloc_page(gfp); if (!page) return -ENOMEM; =20 --=20 2.23.0 From nobody Sun May 10 16:26:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51854C433FE for ; Fri, 29 Apr 2022 06:40:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354701AbiD2GoO (ORCPT ); Fri, 29 Apr 2022 02:44:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234036AbiD2Gnv (ORCPT ); Fri, 29 Apr 2022 02:43:51 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC721B9F29 for ; Thu, 28 Apr 2022 23:40:34 -0700 (PDT) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4KqNCP6JCPzhYqr; Fri, 29 Apr 2022 14:40:17 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 29 Apr 2022 14:40:33 +0800 From: Miaohe Lin To: , CC: , , Subject: [PATCH 4/9] mm/z3fold: throw warning on failure of trylock_page in z3fold_alloc Date: Fri, 29 Apr 2022 14:40:46 +0800 Message-ID: <20220429064051.61552-5-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220429064051.61552-1-linmiaohe@huawei.com> References: <20220429064051.61552-1-linmiaohe@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" If trylock_page fails, the page won't be non-lru movable page. When this page is freed via free_z3fold_page, it will trigger bug on PageMovable check in __ClearPageMovable. Throw warning on failure of trylock_page to guard against such rare case just as what zsmalloc does. Signed-off-by: Miaohe Lin --- mm/z3fold.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index 4e6814c5694f..b3b4e65c107f 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -1122,10 +1122,9 @@ static int z3fold_alloc(struct z3fold_pool *pool, si= ze_t size, gfp_t gfp, __SetPageMovable(page, pool->inode->i_mapping); unlock_page(page); } else { - if (trylock_page(page)) { - __SetPageMovable(page, pool->inode->i_mapping); - unlock_page(page); - } + WARN_ON(!trylock_page(page)); + __SetPageMovable(page, pool->inode->i_mapping); + unlock_page(page); } z3fold_page_lock(zhdr); =20 --=20 2.23.0 From nobody Sun May 10 16:26:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B955C433EF for ; Fri, 29 Apr 2022 06:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354739AbiD2Go2 (ORCPT ); Fri, 29 Apr 2022 02:44:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354668AbiD2Gnx (ORCPT ); Fri, 29 Apr 2022 02:43:53 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F743B9F29 for ; Thu, 28 Apr 2022 23:40:36 -0700 (PDT) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4KqNCJ4bp4zhYWh; Fri, 29 Apr 2022 14:40:12 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 29 Apr 2022 14:40:33 +0800 From: Miaohe Lin To: , CC: , , Subject: [PATCH 5/9] revert "mm/z3fold.c: allow __GFP_HIGHMEM in z3fold_alloc" Date: Fri, 29 Apr 2022 14:40:47 +0800 Message-ID: <20220429064051.61552-6-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220429064051.61552-1-linmiaohe@huawei.com> References: <20220429064051.61552-1-linmiaohe@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Revert commit f1549cb5ab2b ("mm/z3fold.c: allow __GFP_HIGHMEM in z3fold_alloc"). z3fold can't support GFP_HIGHMEM page now. page_address is used directly at all places. Moreover, z3fold_header is on per cpu unbuddied list which could be access anytime. So we should rid the support of GFP_HIGHMEM allocation for z3fold. Signed-off-by: Miaohe Lin --- mm/z3fold.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index b3b4e65c107f..5f5d5f1556be 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -212,10 +212,8 @@ static int size_to_chunks(size_t size) static inline struct z3fold_buddy_slots *alloc_slots(struct z3fold_pool *p= ool, gfp_t gfp) { - struct z3fold_buddy_slots *slots; - - slots =3D kmem_cache_zalloc(pool->c_handle, - (gfp & ~(__GFP_HIGHMEM | __GFP_MOVABLE))); + struct z3fold_buddy_slots *slots =3D kmem_cache_zalloc(pool->c_handle, + gfp); =20 if (slots) { /* It will be freed separately in free_handle(). */ @@ -1075,7 +1073,7 @@ static int z3fold_alloc(struct z3fold_pool *pool, siz= e_t size, gfp_t gfp, enum buddy bud; bool can_sleep =3D gfpflags_allow_blocking(gfp); =20 - if (!size) + if (!size || (gfp & __GFP_HIGHMEM)) return -EINVAL; =20 if (size > PAGE_SIZE) --=20 2.23.0 From nobody Sun May 10 16:26:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2081C433F5 for ; Fri, 29 Apr 2022 06:41:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354710AbiD2GoS (ORCPT ); Fri, 29 Apr 2022 02:44:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354655AbiD2Gnx (ORCPT ); Fri, 29 Apr 2022 02:43:53 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49570B9F28 for ; Thu, 28 Apr 2022 23:40:36 -0700 (PDT) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4KqNBd3mZLzfZqG; Fri, 29 Apr 2022 14:39:37 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 29 Apr 2022 14:40:33 +0800 From: Miaohe Lin To: , CC: , , Subject: [PATCH 6/9] mm/z3fold: put z3fold page back into unbuddied list when reclaim or migration fails Date: Fri, 29 Apr 2022 14:40:48 +0800 Message-ID: <20220429064051.61552-7-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220429064051.61552-1-linmiaohe@huawei.com> References: <20220429064051.61552-1-linmiaohe@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When doing z3fold page reclaim or migration, the page is removed from unbuddied list. If reclaim or migration succeeds, it's fine as page is released. But in case it fails, the page is not put back into unbuddied list now. The page will be leaked until next compaction work, reclaim or migration is done. Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool --- mm/z3fold.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/z3fold.c b/mm/z3fold.c index 5f5d5f1556be..a1c150fc8def 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -1422,6 +1422,8 @@ static int z3fold_reclaim_page(struct z3fold_pool *po= ol, unsigned int retries) spin_lock(&pool->lock); list_add(&page->lru, &pool->lru); spin_unlock(&pool->lock); + if (list_empty(&zhdr->buddy)) + add_to_unbuddied(pool, zhdr); z3fold_page_unlock(zhdr); clear_bit(PAGE_CLAIMED, &page->private); } @@ -1638,6 +1640,8 @@ static void z3fold_page_putback(struct page *page) spin_lock(&pool->lock); list_add(&page->lru, &pool->lru); spin_unlock(&pool->lock); + if (list_empty(&zhdr->buddy)) + add_to_unbuddied(pool, zhdr); clear_bit(PAGE_CLAIMED, &page->private); z3fold_page_unlock(zhdr); } --=20 2.23.0 From nobody Sun May 10 16:26:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCA26C433FE for ; Fri, 29 Apr 2022 06:41:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350978AbiD2GoU (ORCPT ); Fri, 29 Apr 2022 02:44:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51878 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354656AbiD2Gnx (ORCPT ); Fri, 29 Apr 2022 02:43:53 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28783B9F1B for ; Thu, 28 Apr 2022 23:40:36 -0700 (PDT) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4KqNCR148kzhYrG; Fri, 29 Apr 2022 14:40:19 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 29 Apr 2022 14:40:34 +0800 From: Miaohe Lin To: , CC: , , Subject: [PATCH 7/9] mm/z3fold: always clear PAGE_CLAIMED under z3fold page lock Date: Fri, 29 Apr 2022 14:40:49 +0800 Message-ID: <20220429064051.61552-8-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220429064051.61552-1-linmiaohe@huawei.com> References: <20220429064051.61552-1-linmiaohe@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Think about the below race window: CPU1 CPU2 z3fold_reclaim_page z3fold_free test_and_set_bit PAGE_CLAIMED failed to reclaim page z3fold_page_lock(zhdr); add back to the lru list; z3fold_page_unlock(zhdr); get_z3fold_header page_claimed=3Dtest_and_set_bit PAGE_CLAIMED clear_bit(PAGE_CLAIMED, &page->private); if (!page_claimed) /* it's false true */ free_handle is not called free_handle won't be called in this case. So z3fold_buddy_slots will leak. Fix it by always clear PAGE_CLAIMED under z3fold page lock. Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool --- mm/z3fold.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index a1c150fc8def..4a3cd2ff15b0 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -1221,8 +1221,8 @@ static void z3fold_free(struct z3fold_pool *pool, uns= igned long handle) return; } if (test_and_set_bit(NEEDS_COMPACTING, &page->private)) { - put_z3fold_header(zhdr); clear_bit(PAGE_CLAIMED, &page->private); + put_z3fold_header(zhdr); return; } if (zhdr->cpu < 0 || !cpu_online(zhdr->cpu)) { @@ -1424,8 +1424,8 @@ static int z3fold_reclaim_page(struct z3fold_pool *po= ol, unsigned int retries) spin_unlock(&pool->lock); if (list_empty(&zhdr->buddy)) add_to_unbuddied(pool, zhdr); - z3fold_page_unlock(zhdr); clear_bit(PAGE_CLAIMED, &page->private); + z3fold_page_unlock(zhdr); } =20 /* We started off locked to we need to lock the pool back */ @@ -1577,8 +1577,8 @@ static int z3fold_page_migrate(struct address_space *= mapping, struct page *newpa if (!z3fold_page_trylock(zhdr)) return -EAGAIN; if (zhdr->mapped_count !=3D 0 || zhdr->foreign_handles !=3D 0) { - z3fold_page_unlock(zhdr); clear_bit(PAGE_CLAIMED, &page->private); + z3fold_page_unlock(zhdr); return -EBUSY; } if (work_pending(&zhdr->work)) { --=20 2.23.0 From nobody Sun May 10 16:26:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00219C433EF for ; Fri, 29 Apr 2022 06:41:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354721AbiD2GoV (ORCPT ); Fri, 29 Apr 2022 02:44:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354669AbiD2Gnx (ORCPT ); Fri, 29 Apr 2022 02:43:53 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A0252BABBD for ; Thu, 28 Apr 2022 23:40:36 -0700 (PDT) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4KqNBf5Dt7z1JBqc; Fri, 29 Apr 2022 14:39:38 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 29 Apr 2022 14:40:34 +0800 From: Miaohe Lin To: , CC: , , Subject: [PATCH 8/9] mm/z3fold: fix z3fold_reclaim_page races with z3fold_free Date: Fri, 29 Apr 2022 14:40:50 +0800 Message-ID: <20220429064051.61552-9-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220429064051.61552-1-linmiaohe@huawei.com> References: <20220429064051.61552-1-linmiaohe@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Think about the below scene: CPU1 CPU2 z3fold_reclaim_page z3fold_free spin_lock(&pool->lock) get_z3fold_header -- hold page_lock kref_get_unless_zero kref_put--zhdr->refcount can be 1 now !z3fold_page_trylock kref_put -- zhdr->refcount is 0 now release_z3fold_page WARN_ON(!list_empty(&zhdr->buddy)); -- we're on buddy now! spin_lock(&pool->lock); -- deadlock here! z3fold_reclaim_page might race with z3fold_free and will lead to pool lock deadlock and zhdr buddy non-empty warning. To fix this, defer getting the refcount until page_lock is held just like what __z3fold_alloc does. Note this has the side effect that we won't break the reclaim if we meet a soon to be released z3fold page now. Fixes: dcf5aedb24f8 ("z3fold: stricter locking and more careful reclaim") Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool --- mm/z3fold.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index 4a3cd2ff15b0..a7769befd74e 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -519,13 +519,6 @@ static void __release_z3fold_page(struct z3fold_header= *zhdr, bool locked) atomic64_dec(&pool->pages_nr); } =20 -static void release_z3fold_page(struct kref *ref) -{ - struct z3fold_header *zhdr =3D container_of(ref, struct z3fold_header, - refcount); - __release_z3fold_page(zhdr, false); -} - static void release_z3fold_page_locked(struct kref *ref) { struct z3fold_header *zhdr =3D container_of(ref, struct z3fold_header, @@ -1317,12 +1310,7 @@ static int z3fold_reclaim_page(struct z3fold_pool *p= ool, unsigned int retries) break; } =20 - if (kref_get_unless_zero(&zhdr->refcount) =3D=3D 0) { - zhdr =3D NULL; - break; - } if (!z3fold_page_trylock(zhdr)) { - kref_put(&zhdr->refcount, release_z3fold_page); zhdr =3D NULL; continue; /* can't evict at this point */ } @@ -1333,14 +1321,14 @@ static int z3fold_reclaim_page(struct z3fold_pool *= pool, unsigned int retries) */ if (zhdr->foreign_handles || test_and_set_bit(PAGE_CLAIMED, &page->private)) { - if (!kref_put(&zhdr->refcount, - release_z3fold_page_locked)) - z3fold_page_unlock(zhdr); + z3fold_page_unlock(zhdr); zhdr =3D NULL; continue; /* can't evict such page */ } list_del_init(&zhdr->buddy); zhdr->cpu =3D -1; + /* See comment in __z3fold_alloc. */ + kref_get(&zhdr->refcount); break; } =20 --=20 2.23.0 From nobody Sun May 10 16:26:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C54B8C433F5 for ; Fri, 29 Apr 2022 06:41:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354748AbiD2Goc (ORCPT ); Fri, 29 Apr 2022 02:44:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354682AbiD2Gn4 (ORCPT ); Fri, 29 Apr 2022 02:43:56 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A499AB9F1B for ; Thu, 28 Apr 2022 23:40:37 -0700 (PDT) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.54]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4KqN6S5MmNzCsLJ; Fri, 29 Apr 2022 14:36:00 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 29 Apr 2022 14:40:35 +0800 From: Miaohe Lin To: , CC: , , Subject: [PATCH 9/9] mm/z3fold: fix z3fold_page_migrate races with z3fold_map Date: Fri, 29 Apr 2022 14:40:51 +0800 Message-ID: <20220429064051.61552-10-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220429064051.61552-1-linmiaohe@huawei.com> References: <20220429064051.61552-1-linmiaohe@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Think about the below scene: CPU1 CPU2 z3fold_page_migrate z3fold_map z3fold_page_trylock ... z3fold_page_unlock /* slots still points to old zhdr*/ get_z3fold_header get slots from handle get old zhdr from slots z3fold_page_trylock return *old* zhdr encode_handle(new_zhdr, FIRST|LAST|MIDDLE) put_page(page) /* zhdr is freed! */ but zhdr is still used by caller! z3fold_map can map freed z3fold page and lead to use-after-free bug. To fix it, we add PAGE_MIGRATED to indicate z3fold page is migrated and soon to be released. So get_z3fold_header won't return such page. Fixes: 1f862989b04a ("mm/z3fold.c: support page migration") Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool --- mm/z3fold.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index a7769befd74e..f41f8b0d9e9a 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -181,6 +181,7 @@ enum z3fold_page_flags { NEEDS_COMPACTING, PAGE_STALE, PAGE_CLAIMED, /* by either reclaim or free */ + PAGE_MIGRATED, /* page is migrated and soon to be released */ }; =20 /* @@ -270,8 +271,13 @@ static inline struct z3fold_header *get_z3fold_header(= unsigned long handle) zhdr =3D (struct z3fold_header *)(addr & PAGE_MASK); locked =3D z3fold_page_trylock(zhdr); read_unlock(&slots->lock); - if (locked) - break; + if (locked) { + struct page *page =3D virt_to_page(zhdr); + + if (!test_bit(PAGE_MIGRATED, &page->private)) + break; + z3fold_page_unlock(zhdr); + } cpu_relax(); } while (true); } else { @@ -389,6 +395,7 @@ static struct z3fold_header *init_z3fold_page(struct pa= ge *page, bool headless, clear_bit(NEEDS_COMPACTING, &page->private); clear_bit(PAGE_STALE, &page->private); clear_bit(PAGE_CLAIMED, &page->private); + clear_bit(PAGE_MIGRATED, &page->private); if (headless) return zhdr; =20 @@ -1576,7 +1583,7 @@ static int z3fold_page_migrate(struct address_space *= mapping, struct page *newpa new_zhdr =3D page_address(newpage); memcpy(new_zhdr, zhdr, PAGE_SIZE); newpage->private =3D page->private; - page->private =3D 0; + set_bit(PAGE_MIGRATED, &page->private); z3fold_page_unlock(zhdr); spin_lock_init(&new_zhdr->page_lock); INIT_WORK(&new_zhdr->work, compact_page_work); @@ -1606,7 +1613,8 @@ static int z3fold_page_migrate(struct address_space *= mapping, struct page *newpa =20 queue_work_on(new_zhdr->cpu, pool->compact_wq, &new_zhdr->work); =20 - clear_bit(PAGE_CLAIMED, &page->private); + /* PAGE_CLAIMED and PAGE_MIGRATED are cleared now. */ + page->private =3D 0; put_page(page); return 0; } --=20 2.23.0