From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C00A1F934 for ; Fri, 28 Jun 2024 03:06:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544013; cv=none; b=ip50CXDHpsQdohYjjpih0qV2cQlm9xUkN34n9sKDNEO8pcXbLSZcc9wFvWgIxVEWRWkhmbtH1G0bu+5x6t+GCbJi1NtcA8DwkJ47UW+YMPA3mYBDWcV7NO5gIbgLuLBdygviIuurn9prFgG31ldPTGH+H3CppzPPexbx3/jYRJk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544013; c=relaxed/simple; bh=Cr0rk228Nf4Ejmcf123tMGr+wVgBtKk7ChztlR6oqcc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UhGgOWJ/NnYRjFtQ9auYU0yUrQd2KuoykgqME+ROqw0ex03iTif0MHxTK/7rAclIlAzOBZxc7oEfW832dv5voSgdYdWL9itA7jVDOM8mxEvlWd52Ep0D+79odJV1Osova+7KlxxyD2OgnXhtjAeDJtEhlm6LMga0VciQjh03KxA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=C87rd9lC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="C87rd9lC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 01F3FC4AF09; Fri, 28 Jun 2024 03:06:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544012; bh=Cr0rk228Nf4Ejmcf123tMGr+wVgBtKk7ChztlR6oqcc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C87rd9lCAIEm3gmgN1HsqdNpdHVQnk+7Q4Nv5dOer+u/gx8+vllp84o4qmsNwEaZg +Fe1YLW2saEdNkRrbeqC8q7l72FvR+c0OCa5xkz+05L0MavtAPM7hr/UDfktBh/QSb oJjlWdO597SbcB/MMgnDSkhzMxI2B6jMBPx9yG08XhP6Go5Ay+7NS4e/baBvqUc1sO eFslcQG/0/9QdUnqkjnahncWxOulT4eXMZVUN0hMXgwqy8SzeqVX0Aj7BXIoByXD3F WgfD365aGDEcOL1toHZ63a0fAY+j4Rit+f8tKukzw1F/O63OwIGXMV+jk/g1251juN woROqUiGIiSlQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 01/20] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Fri, 28 Jun 2024 11:11:16 +0800 Message-ID: <20240628031138.429622-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi The 1st patch introduces new memory decriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 19 ++++++++--------- 2 files changed, 66 insertions(+), 9 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..a1ab5ebaa936 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirected used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Pointer to zspage in zsmalloc + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + unsigned long _zp_pad_1; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) =3D=3D offsetof(struct zpdesc, zp= )) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <=3D sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index fec1a39e5bbe..67bb80b7413a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,17 +13,17 @@ =20 /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. * page->page_type: PG_zsmalloc, lower 16 bit locate the first object * offset in a subpage of a zspage * - * Usage of struct page flags: + * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page * PG_owner_priv_1: identifies the huge component page * @@ -64,6 +64,7 @@ #include #include #include +#include "zpdesc.h" =20 #define ZSPAGE_MAGIC 0x58 =20 @@ -253,7 +254,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -448,7 +449,7 @@ static inline void mod_zspage_inuse(struct zspage *zspa= ge, int val) =20 static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page =3D zspage->first_page; + struct page *first_page =3D zpdesc_page(zspage->first_zpdesc); =20 VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -948,7 +949,7 @@ static void create_page_chain(struct size_class *class,= struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index =3D 0; if (i =3D=3D 0) { - zspage->first_page =3D page; + zspage->first_zpdesc =3D page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage =3D=3D 1 && class->pages_per_zspage =3D=3D 1)) @@ -1325,7 +1326,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle =3D handle; else /* record handle to page->index */ - zspage->first_page->index =3D handle; + zspage->first_zpdesc->handle =3D handle; =20 kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3895225D4 for ; Fri, 28 Jun 2024 03:06:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544016; cv=none; b=YiPKRc+SzhgAxVlpN5NJCGmge5wKU4NECL0ywPRwPD4q2nE3UUH7Dey9Up0uADXTSMm5AFIj8k8F1On5pdkdtzc+4JTjGsxBnNTmEYEMHH8wZA1N84eMlzYLFHbD5cgYLl3OlzHMHaXr0rbBqG4pAf7LlX+lk8h4KkOkhoGmkmM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544016; c=relaxed/simple; bh=wX8wNBVCAeYaipWwjXbga4FkUGc1GqAIKlVyc5psJK0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i9VXVu41oqlNa1M7mCsny6uQzc2HEk69KZb8XqkAIjN5nlxCeGEYgNqxP7dNaL73q88LXZr8L7o0SaYbKFE3GqAern6g0faDnVFj304O+YB6J1kWXqnd1ICjle/4FLD5fpOOCvurvEeMTXk02UW8Ri+tGpmRXlJmKAROZdocOPk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uxYPxwOw; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uxYPxwOw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89772C2BBFC; Fri, 28 Jun 2024 03:06:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544016; bh=wX8wNBVCAeYaipWwjXbga4FkUGc1GqAIKlVyc5psJK0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uxYPxwOwKZkbAWcRCcj/faaq1syYlYM8BWU58yRNfmwar7N4AMMqbCcR61gQJ3hmJ nB9GYWciKXfEiUS4NLgEoiQquUcjB/0zyoiAKOBO8+xHDR4G1NEdIJ+wzy6h/WKB6H CLp9kQKd6Nbc+hxoc8kVP2YhPDyc6geIASE7dPiNVOMTMETM/IOUEOHBiw4hZ7rRVu 65oZiddBo5ncFNhCp1hfrtQm0GC4UA4uaVNCI03ksluRAywiDglD098AbmGckA0pU3 FHTvWD16NC64U9SYB4VXsPbRUDHmAuFRN4VfeFfhLySAXPbaYWFPevnO5EHDrJ8et0 7d+6X6G15pkQg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 02/20] mm/zsmalloc: use zpdesc in trylock_zspage/lock_zspage Date: Fri, 28 Jun 2024 11:11:17 +0800 Message-ID: <20240628031138.429622-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi To use zpdesc in trylock_zspage/lock_zspage funcs, we add couple of helpers: zpdesc_lock/zpdesc_unlock/zpdesc_trylock/zpdesc_wait_locked and zpdesc_get/zpdesc_put for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index a1ab5ebaa936..fd95277843d5 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -53,4 +53,34 @@ static_assert(sizeof(struct zpdesc) <=3D sizeof(struct p= age)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) =20 +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 67bb80b7413a..9835121109d1 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -435,13 +435,17 @@ static __maybe_unused int is_first_page(struct page *= page) return PagePrivate(page); } =20 +static int is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } =20 - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse +=3D val; @@ -455,6 +459,14 @@ static inline struct page *get_first_page(struct zspag= e *zspage) return first_page; } =20 +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc =3D zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff =20 static inline void reset_first_obj_offset(struct page *page) @@ -747,6 +759,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } =20 +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -817,11 +839,11 @@ static void reset_page(struct page *page) =20 static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; =20 - for (cursor =3D get_first_page(zspage); cursor !=3D NULL; cursor =3D - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor =3D get_first_zpdesc(zspage); cursor !=3D NULL; cursor =3D + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail =3D cursor; goto unlock; } @@ -829,9 +851,9 @@ static int trylock_zspage(struct zspage *zspage) =20 return 1; unlock: - for (cursor =3D get_first_page(zspage); cursor !=3D fail; cursor =3D - get_next_page(cursor)) - unlock_page(cursor); + for (cursor =3D get_first_zpdesc(zspage); cursor !=3D fail; cursor =3D + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); =20 return 0; } @@ -1663,7 +1685,7 @@ static int putback_zspage(struct size_class *class, s= truct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; =20 /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1675,24 +1697,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page =3D get_first_page(zspage); - if (trylock_page(page)) + zpdesc =3D get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } =20 - curr_page =3D page; - while ((page =3D get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page =3D page; + curr_zpdesc =3D zpdesc; + while ((zpdesc =3D get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc =3D zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEC7033086 for ; Fri, 28 Jun 2024 03:07:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544020; cv=none; b=OYyezng4+LWKuq80nskEwe1GlxsiELD5DZyc3J1XEI0tTJ8p8ImGG92bjvCTtijJlx5mwGOsEE5nZI72zdciwz/M+MpNaV19IOrme9rV0cXNe4BAhtE+gQkdBN6Am7y56xT/HV9nkjXfNZNI6JMNx3P5ovaSKgK58J9IXgI1OsA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544020; c=relaxed/simple; bh=la1BOuF9dyPKXa6VzFValIggtH+1hQli1SJEv8vNjvY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZkjFoe8dSoavltgVPr5An+OA6r6jHg/E9heWsi0eLPtSZ3pN6kjMz/coIBAY/YdCB496INjVDuPrgOgBdcYM1XGerDyuIdVA6qjNmHfAyQE6020ZPnkjlwKkgB4NU+qamjPuWH9QnDEaXc8aOECnu5tVqvJ6QVg2uIGTuJFI2Mk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kePCV/TI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kePCV/TI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 47CB1C2BBFC; Fri, 28 Jun 2024 03:06:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544020; bh=la1BOuF9dyPKXa6VzFValIggtH+1hQli1SJEv8vNjvY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kePCV/TI59oEiDU6RiCVWdhIVIIKK6xMrJRCLPWy/EAVyv+bgOGbbgE/wlA1MOr7d rQzOIbgaBwAOnvtUtf19hEHYCrJ4dovLCdTK8438MXdWmSM58iKRvGF9DbM3lXfOYp rKGIs0YnIzt4nnDUsyin6MVfxZaBbsF+kCD+jDs9TxrG+UIVVJ7vYbIlnowNDwFzVO 3js5naNvOwhjn7Am/WMKBsjRm83kLOsGAJuAjxrSQQiy+GQDMucwfZSIA9QICY4h8P 0uTp0tQREmNquy0fJ0kNTYxSLm+8YiI3moo28X2+FqrPCZLhgupx7dSpIyxsczlkRE 349W7Hg/LeVMA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 03/20] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Fri, 28 Jun 2024 11:11:18 +0800 Message-ID: <20240628031138.429622-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 9835121109d1..cedd3dfb9124 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -245,6 +245,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; =20 +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1063,7 +1068,7 @@ static inline void __zs_cpu_down(struct mapping_area = *area) } =20 static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1080,10 +1085,10 @@ static void *__zs_map_object(struct mapping_area *a= rea, sizes[1] =3D size - sizes[0]; =20 /* copy object to per-cpu buffer */ - addr =3D kmap_atomic(pages[0]); + addr =3D zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr =3D kmap_atomic(pages[1]); + addr =3D zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1091,7 +1096,7 @@ static void *__zs_map_object(struct mapping_area *are= a, } =20 static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1110,10 +1115,10 @@ static void __zs_unmap_object(struct mapping_area *= area, sizes[1] =3D size - sizes[0]; =20 /* copy per-cpu buffer to object */ - addr =3D kmap_atomic(pages[0]); + addr =3D zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr =3D kmap_atomic(pages[1]); + addr =3D zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); =20 @@ -1254,7 +1259,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, pages[1] =3D get_next_page(page); BUG_ON(!pages[1]); =20 - ret =3D __zs_map_object(area, pages, off, class->size); + ret =3D __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret +=3D ZS_HANDLE_SIZE; @@ -1289,7 +1294,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) pages[1] =3D get_next_page(page); BUG_ON(!pages[1]); =20 - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); =20 --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D22F3FBA5 for ; Fri, 28 Jun 2024 03:07:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544024; cv=none; b=mpo2fv2tXDHrUOxruOZcEZ97PBFo8ve3Jged/aGQsLY2VUlY2G8LYqKYRpdZegVCy49ltlAPlRugq/71iOr6hVRAXh66UX4Q82EvrbKDzJ63eXnO1dud1oTJpVBrYqkeThCL+mpwI2KDsgzaL2da+Un5jRXxvGBOng9stoaBQRI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544024; c=relaxed/simple; bh=Lq3Glkac4UQFmimamfx1rDNuTCRcH+b9V+Y0ngU8Y20=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FXKyJ1bhogD/pHoGMUsKzSNXddb3xfuvcUoheE+hkNxHhA/tEMFKirgOjVtIvuuv9k45FW3PRnT7mxCn7J8Vj4xGZEGD+phlJMifWC4QB86xll35R7Z4zWNwHSvFuSsEU6BDjMfp5CvqFrR1xhAXlklWl5zErJpq2Pv4lGPSy38= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=R5F7BJaN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="R5F7BJaN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 04EBFC32789; Fri, 28 Jun 2024 03:07:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544024; bh=Lq3Glkac4UQFmimamfx1rDNuTCRcH+b9V+Y0ngU8Y20=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R5F7BJaN3GA1L3ByUBwnS7JM2lJN1vZN2c9HhkzE/Knj//eX18PCVW+lJuPlcT35H Iodhqv5pltSEAMkGC+6DjTEXr5R/GE4wHanKSeG5Ve4n4uNJMu2DUsG7MPVSUVKab9 MN3HrGqv5T8xa+asV67FicGmnNgJXTGEH5M6QrH6ku1wgYja7QO+IB8lwxePqvcG/q zGq9JPJHhUehDaQp14KK4n2LvR7iWqOriqHcMFzQMWpClHFxNvzk05tMzk/VMyc1Ua SjQPZBjwy4zMx4VtSi9X1IqbtrTV2Wt0KTVwt4ecKnVIiCXM9o75u3xgKWvv2C7bUq GYAjfC8JgH6zA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 04/20] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Fri, 28 Jun 2024 11:11:19 +0800 Message-ID: <20240628031138.429622-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index fd95277843d5..3c0ebdc78ff8 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -83,4 +83,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } =20 +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index cedd3dfb9124..efb1d58b3c36 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -775,15 +775,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *= zpdesc) } =20 /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page =3D pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx =3D (obj & OBJ_INDEX_MASK); } =20 @@ -1210,13 +1210,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned = long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; =20 struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; =20 /* @@ -1229,8 +1229,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage =3D get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage =3D get_zspage(zpdesc_page(zpdesc)); =20 /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1249,17 +1249,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned = long handle, area->vm_mm =3D mm; if (off + class->size <=3D PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr =3D kmap_atomic(page); + area->vm_addr =3D zpdesc_kmap_atomic(zpdesc); ret =3D area->vm_addr + off; goto out; } =20 /* this object spans two pages */ - pages[0] =3D page; - pages[1] =3D get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] =3D zpdesc; + zpdescs[1] =3D get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); =20 - ret =3D __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret =3D __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret +=3D ZS_HANDLE_SIZE; @@ -1271,7 +1271,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; =20 @@ -1279,8 +1279,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) struct mapping_area *area; =20 obj =3D handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage =3D get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage =3D get_zspage(zpdesc_page(zpdesc)); class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 @@ -1288,13 +1288,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned= long handle) if (off + class->size <=3D PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; =20 - pages[0] =3D page; - pages[1] =3D get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] =3D zpdesc; + zpdescs[1] =3D get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); =20 - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); =20 @@ -1438,23 +1438,24 @@ static void obj_free(int class_size, unsigned long = obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; =20 - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset =3D offset_in_page(class_size * f_objidx); - zspage =3D get_zspage(f_page); + zspage =3D get_zspage(zpdesc_page(f_zpdesc)); =20 - vaddr =3D kmap_atomic(f_page); + vaddr =3D zpdesc_kmap_atomic(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); =20 /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next =3D get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index =3D 0; + f_zpdesc->next =3D NULL; set_freeobj(zspage, f_objidx); =20 kunmap_atomic(vaddr); @@ -1499,7 +1500,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1508,8 +1509,8 @@ static void zs_object_copy(struct size_class *class, = unsigned long dst, =20 s_size =3D d_size =3D class->size; =20 - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); =20 s_off =3D offset_in_page(class->size * s_objidx); d_off =3D offset_in_page(class->size * d_objidx); @@ -1520,8 +1521,8 @@ static void zs_object_copy(struct size_class *class, = unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size =3D PAGE_SIZE - d_off; =20 - s_addr =3D kmap_atomic(s_page); - d_addr =3D kmap_atomic(d_page); + s_addr =3D zpdesc_kmap_atomic(s_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); =20 while (1) { size =3D min(s_size, d_size); @@ -1546,17 +1547,17 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, if (s_off >=3D PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page =3D get_next_page(s_page); - s_addr =3D kmap_atomic(s_page); - d_addr =3D kmap_atomic(d_page); + s_zpdesc =3D get_next_zpdesc(s_zpdesc); + s_addr =3D zpdesc_kmap_atomic(s_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); s_size =3D class->size - written; s_off =3D 0; } =20 if (d_off >=3D PAGE_SIZE) { kunmap_atomic(d_addr); - d_page =3D get_next_page(d_page); - d_addr =3D kmap_atomic(d_page); + d_zpdesc =3D get_next_zpdesc(d_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); d_size =3D class->size - written; d_off =3D 0; } @@ -1796,7 +1797,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3854D46453 for ; Fri, 28 Jun 2024 03:07:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544028; cv=none; b=KeiSrJoIeAhupPFSAlJWHSeaLIn9VxVELZlulgCTkG6chjIOrjfhFnipDLQl+LajXauIk+Wq0mNEmpLyAxei6lHfg0k/UBynAXfjD/hUOKFx1Zkogkd4Nc6fOQHdbXQGYlnQoZ/bGh11wCReS8r8rPCvKW/ixqK9wt39AGC0GsI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544028; c=relaxed/simple; bh=XjQFvUcS1LHsRYGu9jnBiByPT0wYykOdlBmIkN2FtBk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m4FRovMSYZXr/0X41VM3nL/ghdWdyQnesgZy44ghrpO8LvwCTVzq5AaTxyDnLiZt2JCVJZ/Y6ownDDXSx75R4eBlrC0CLwKIUeDTP9eiAgDhhB1k1PF+TSmBjsDDf1mIGm7Cq/0UVy2pXM+8+tAb++c+2p5hVAgeTDhOshFYukQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fLwz/ccg; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fLwz/ccg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BBB16C4AF0D; Fri, 28 Jun 2024 03:07:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544027; bh=XjQFvUcS1LHsRYGu9jnBiByPT0wYykOdlBmIkN2FtBk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fLwz/ccgEUbpiJOrFdS0JKWIACGuXo9B+lq1ll4r21yihrI4awNMbyxM+XE5ayHG3 XmK0O9G0zjrm7ibjLcKlK7WBFSzMn2Gj8217qv2X5CDFdM/u9I3EbeRkl+JrtaKfhk fQ6Wq3h+qXPYeahBSmHYSfSArtmajGSvBL7BsbIMmiX7e2BxiJFzWn5ngTydX03V+G LktDfp1xBOQ/s1nuxJRcdFMD2B//pwJQcQlM4wn4D+o3HzI8hd6yKYVAbQ7DLK6KOs dKisWok1zVs65YDd7PhvkNMpwpefhHE0rYPzWAo64qeHnYsA5F9AxbF+bN17yxu3iJ +IwgjYNBRM6JA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 05/20] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Fri, 28 Jun 2024 11:11:20 +0800 Message-ID: <20240628031138.429622-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index efb1d58b3c36..137b36515acf 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1324,12 +1324,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; =20 - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; =20 @@ -1338,14 +1338,14 @@ static unsigned long obj_malloc(struct zs_pool *poo= l, obj =3D get_freeobj(zspage); =20 offset =3D obj * class->size; - nr_page =3D offset >> PAGE_SHIFT; + nr_zpdesc =3D offset >> PAGE_SHIFT; m_offset =3D offset_in_page(offset); - m_page =3D get_first_page(zspage); + m_zpdesc =3D get_first_zpdesc(zspage); =20 - for (i =3D 0; i < nr_page; i++) - m_page =3D get_next_page(m_page); + for (i =3D 0; i < nr_zpdesc; i++) + m_zpdesc =3D get_next_zpdesc(m_zpdesc); =20 - vaddr =3D kmap_atomic(m_page); + vaddr =3D zpdesc_kmap_atomic(m_zpdesc); link =3D (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1358,7 +1358,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(m_page, obj); + obj =3D location_to_obj(zpdesc_page(m_zpdesc), obj); =20 return obj; } --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5B324CE04 for ; Fri, 28 Jun 2024 03:07:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544032; cv=none; b=EvoQo+71EghF7CUmIJpF0dx2AlxKsyfXYcm4dmqaBhlDPt79OwZxRH4O0X6on23Y2ATJqOxwhM5Xur5rtMPYP1m9gaiwxEIl36p5KGplg93ygb58U44rZ/evGVvpzN/shStwROF7Ct7gkBz2ZqWRv8fwB+RNyaWjk45H6bpDpEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544032; c=relaxed/simple; bh=DEgsEkg1N6dj0FWoDYLMEtx39UiAuN8BZowdME7NzSQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=R8DNtuDwTvf5TubkyhBdN9FC1GMnYH0ZaytmebwEt6ZQKVdTFXz2/IzUKzygYa6QA/CkBlPtmLgJ+S/BPRFgJWM+y/3VkNSQgrquy/+bx3nv017u+RzWncxjIFafdX+Mt4so6ss2eQ6ppVuzpVT1xMxjjYo4tKUyIXKfiQ58gR0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FSZsq3px; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FSZsq3px" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79717C2BBFC; Fri, 28 Jun 2024 03:07:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544031; bh=DEgsEkg1N6dj0FWoDYLMEtx39UiAuN8BZowdME7NzSQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FSZsq3pxtjEamZ0NO6vE08PTTiK4G3H8yUs8rLnV+WbysjiW+mZFZE0UGYeBNYpEP YB0gFO+qwGQWdfZX90SJHf9eCmRkjHRsMqgymWr9cgaXlHreST2W1QzDkqYI/sT1Ix ufqYD566GPxPVQ6LSh6bpClqC6aovilg0E20nW5vrxy2VV7iyWKMTpS658xJMnCVBU evSs3Cj4wdRZqcZsvcvaTUaOjHcXo5GZd3Xknyqg0RItte8+C0tIQnOO5QwFdQr+eP BL9EsN2UzrnTXDT4FkJJAjvPqkcs7l5c9m7jsvXXgyGFckIJes7Tw/OJ1Rm60vIol8 +bKRDIY5Blg9A== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 06/20] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Fri, 28 Jun 2024 11:11:21 +0800 Message-ID: <20240628031138.429622-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chai= n() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 +++ mm/zsmalloc.c | 115 +++++++++++++++++++++++++++++++++----------------- 2 files changed, 82 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 3c0ebdc78ff8..f998d65c59d6 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -92,4 +92,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long pf= n) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 137b36515acf..ee890d513f6f 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -250,6 +250,41 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *= zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } =20 +static inline void zpdesc_set_zspage(struct zpdesc *zpdesc, + struct zspage *zspage) +{ + zpdesc->zspage =3D zspage; +} + +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline struct zpdesc *alloc_zpdesc(gfp_t gfp) +{ + struct page *page =3D alloc_page(gfp); + + return page_zpdesc(page); +} + +static inline void free_zpdesc(struct zpdesc *zpdesc) +{ + struct page *page =3D zpdesc_page(zpdesc); + + __free_page(page); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -956,35 +991,35 @@ static void init_zspage(struct size_class *class, str= uct zspage *zspage) } =20 static void create_page_chain(struct size_class *class, struct zspage *zsp= age, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page =3D NULL; - int nr_pages =3D class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc =3D NULL; + int nr_zpdescs =3D class->pages_per_zspage; =20 /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i =3D 0; i < nr_pages; i++) { - page =3D pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index =3D 0; + for (i =3D 0; i < nr_zpdescs; i++) { + zpdesc =3D zpdescs[i]; + zpdesc_set_zspage(zpdesc, zspage); + zpdesc->next =3D NULL; if (i =3D=3D 0) { - zspage->first_zpdesc =3D page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc =3D zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage =3D=3D 1 && class->pages_per_zspage =3D=3D 1)) SetZsHugePage(zspage); } else { - prev_page->index =3D (unsigned long)page; + prev_zpdesc->next =3D zpdesc; } - prev_page =3D page; + prev_zpdesc =3D zpdesc; } } =20 @@ -996,7 +1031,7 @@ static struct zspage *alloc_zspage(struct zs_pool *poo= l, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage =3D cache_alloc_zspage(pool, gfp); =20 if (!zspage) @@ -1006,25 +1041,25 @@ static struct zspage *alloc_zspage(struct zs_pool *= pool, migrate_lock_init(zspage); =20 for (i =3D 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; =20 - page =3D alloc_page(gfp); - if (!page) { + zpdesc =3D alloc_zpdesc(gfp); + if (!zpdesc) { while (--i >=3D 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); =20 - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] =3D page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] =3D zpdesc; } =20 - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool =3D pool; zspage->class =3D class->index; @@ -1758,26 +1793,28 @@ static void migrate_write_unlock(struct zspage *zsp= age) static const struct movable_operations zsmalloc_mops; =20 static void replace_sub_page(struct size_class *class, struct zspage *zspa= ge, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] =3D {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] =3D {NULL, }; + unsigned int first_obj_offset; int idx =3D 0; =20 - page =3D get_first_page(zspage); + zpdesc =3D get_first_zpdesc(zspage); do { - if (page =3D=3D oldpage) - pages[idx] =3D newpage; + if (zpdesc =3D=3D oldzpdesc) + zpdescs[idx] =3D newzpdesc; else - pages[idx] =3D page; + zpdescs[idx] =3D zpdesc; idx++; - } while ((page =3D get_next_page(page)) !=3D NULL); + } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset =3D get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index =3D oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle =3D oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } =20 static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1850,7 +1887,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, } kunmap_atomic(s_addr); =20 - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BA0350275 for ; Fri, 28 Jun 2024 03:07:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544035; cv=none; b=K6ZhLr1sfgsHRAORjC+nuPwlCt3jQPo/hwnaoT1qJwkvZdV7Iy/nmL7h4jXKp28wm8Tuzbc/aD1O8ajLZC93ZfAd7LFxg+qoXwqw/L2Ma5Hx6IAhrJpLqEkv34piH0O6L6faAOEkHuW644BjJ2mufaYZ3HE82QQOPDoWfYeFxhk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544035; c=relaxed/simple; bh=6Pm9+ydZIsO5c9qRiniJl+L1KPC6XDYzIyrZkkU9aKI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ffTp9Htq8n9W8drrmgJCYxmi3AF0354hrYe0+l4Ev3VrnLQx/L7vV3a1MCAeMLXLgquRsDCzUFyf1bowjC0Tug133uxtjgI4UTeOTZhKBjCpVRI36d2uCIZ+Us41XWDTV66UIfjY294FNLzh+8H6xoB74rNQvUG9kxGqMYH+jyg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EsLIVXmX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EsLIVXmX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 370D3C2BBFC; Fri, 28 Jun 2024 03:07:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544035; bh=6Pm9+ydZIsO5c9qRiniJl+L1KPC6XDYzIyrZkkU9aKI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EsLIVXmXFHNSPRxWd3m4+VHHl+VjFRdCHM8BTCvp4ASArnVdJg45eBwEpv1PTcc4+ eEJ1NHhHKtGm0C4A8uG2fnOzsLTe/goC7j/F4+vvDSYh3WgGjhkYDVegoswWQD2w4w prk/ypbbDn0J4scwSNmm3arQFeDX6m7bbKum0YUg/hBmQqvW5hthU1GNKPG49VCX5B 49mb0ZY9H9gEpbKIwx/47UF2WqV/ndYyAlhSva5OnEzxDIigPsBJWNOxaqEhf5HIGL Lxx2fFFVNA5XFHqBlSh73eJ7YpfzwDn8cAGEfxmnV26rCFrqUT3RiR/gVjKJDXYw9s cStrQQm3EJOJQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 07/20] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Fri, 28 Jun 2024 11:11:22 +0800 Message-ID: <20240628031138.429622-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index ee890d513f6f..6efaa6279687 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -847,15 +847,15 @@ static unsigned long handle_to_obj(unsigned long hand= le) return *(unsigned long *)handle; } =20 -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage =3D get_zspage(page); + struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); =20 if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle =3D page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle =3D zpdesc->handle; } else handle =3D *(unsigned long *)obj; =20 @@ -1607,18 +1607,18 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index =3D *obj_idx; unsigned long handle =3D 0; - void *addr =3D kmap_atomic(page); + void *addr =3D zpdesc_kmap_atomic(zpdesc); =20 - offset =3D get_first_obj_offset(page); + offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; =20 offset +=3D class->size; @@ -1642,7 +1642,7 @@ static void migrate_zspage(struct zs_pool *pool, stru= ct zspage *src_zspage, struct size_class *class =3D pool->size_class[src_zspage->class]; =20 while (1) { - handle =3D find_alloced_obj(class, s_page, &obj_idx); + handle =3D find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page =3D get_next_page(s_page); if (!s_page) @@ -1876,7 +1876,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 for (addr =3D s_addr + offset; addr < s_addr + PAGE_SIZE; addr +=3D class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55D7455897 for ; Fri, 28 Jun 2024 03:07:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544039; cv=none; b=e7drSOndr5Fo4z5Q/WQjdIAsJZBwe9CYmAdNaA6CnW87NaJe9+gDBWiwcKWM3l+P94UKYX6x2IfzZQ9b4CCfzra2lgiu2//3TUg6VfGQswmnLaIoccnzz1V45gR6gMmbtlQR1fQZduliUIE2MZp5iNeroVUHaBBHKUMzFgClwqQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544039; c=relaxed/simple; bh=KxgG3cMNh41F6OEFoev4W8KoSH3YuCbiC9yyKNDszWM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SfzlKZ/kFhzYQ9eQffQSz/HEymow9eESVYLFPkeSSVrLKKB5EtS/4fTs7jkceGMBBX6uzaFn/DeqfYY2PJw1gCVff9m1H/ro4vNjQJkhJw8HZmA/O+sinaubwaHCfGGYsYSKHs21DpNIJy71CFSR3/nDS95r+OQ/qB/YMNYs+yc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=T/NycspK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="T/NycspK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BECADC116B1; Fri, 28 Jun 2024 03:07:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544038; bh=KxgG3cMNh41F6OEFoev4W8KoSH3YuCbiC9yyKNDszWM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=T/NycspKY6Agf3MdDX1SDFtv0XHJhE1/1Ar+c+DWV0XEmuLaiYftnIc+JENyd4fOp daPtvZ95Qs3DI6xb3UCA8xlF8P5PlQ/U/k4ab1NL2hTXfOZIGYOVgEu4+CUaGkdhD7 tnIc259NZ3i5cYSavDbG2dHMBBjGtPv+RDgT2nUnnw/BAADyjfgVLMBjIpzXz5UpxI XDHb0hrUPW3d/n2EGhhU1NGpmpuktWX3xvOYAhE08BtUB6xV847wDyefrPg9uwS2fC 4rwIcyKebgAB+DWqa/OMETiqGnf3sC98/n1MmdU220AjgU75XO+WRb8rdkVVaC1/UZ ZRaOiC+UnURdg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 08/20] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Fri, 28 Jun 2024 11:11:23 +0800 Message-ID: <20240628031138.429622-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6efaa6279687..e90e3a638068 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -950,16 +950,16 @@ static void init_zspage(struct size_class *class, str= uct zspage *zspage) { unsigned int freeobj =3D 1; unsigned long off =3D 0; - struct page *page =3D get_first_page(zspage); + struct zpdesc *zpdesc =3D get_first_zpdesc(zspage); =20 - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; =20 - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); =20 - vaddr =3D kmap_atomic(page); + vaddr =3D zpdesc_kmap_atomic(zpdesc); link =3D (struct link_free *)vaddr + off / sizeof(*link); =20 while ((off +=3D class->size) < PAGE_SIZE) { @@ -972,8 +972,8 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page =3D get_next_page(page); - if (next_page) { + next_zpdesc =3D get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next =3D freeobj++ << OBJ_TAG_BITS; } else { /* @@ -983,7 +983,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) link->next =3D -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page =3D next_page; + zpdesc =3D next_zpdesc; off %=3D PAGE_SIZE; } =20 --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3CB659147 for ; Fri, 28 Jun 2024 03:07:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544043; cv=none; b=av69WvjYAUKvwuyuKtEhmwE+P8but1uAW50I+PRzQNNyd1gUPxE7MyV/tUb/6XIbp/odX9EPFEYxfiemT6p40fOBJAC9cuB/+YLcoA+jtZGjjW2phZRtOQX3Mm1PW7HwZ+/Am4WtQUOUMXxzfHyjEuART2sK4V2LJpRNH0Ha+nI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544043; c=relaxed/simple; bh=f1FIB9i8kXKQZtzU8kTMr1QDsZX4ItRyZ1ReJH3drr0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h65sz/Buw3JQKn7fyDtuWfkcPVzzmJAsxwtYirsdIh+vWMFNkr9rUzlPG00qdCseSFzZ6bD3+kU8An4hXLesysMkEvlkLgaCBRbwpIpuNKlBoE6+I64Uu0x2ndTyY1b2QEKajvYpyH1DtFK24e3bXMyyYcnAy/++SUjy8dssfcI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=asuZ1JAY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="asuZ1JAY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7F2EEC4AF0C; Fri, 28 Jun 2024 03:07:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544042; bh=f1FIB9i8kXKQZtzU8kTMr1QDsZX4ItRyZ1ReJH3drr0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=asuZ1JAYGbKula5yKxocWKi3XyNBBP9kf53ah/IL0JHq81kjKvUKY3mJqyO7C04nj 55mdth8Bvias1bnoJbA9Yl82dad+PZtD0AdoIoQCbd/rEiLLmUVt7peD2R1uj4Z+kh BbN31tKf37MrNy9lcLxSbysArq5k0GH2C6A/e3qUjSoUddir51w3pGo6kBtLKfhBSB 39GgUBPrsDKVMJ+53C1BLXuD93wxHgw3Se1SSYCtGZTJWr3Tstzvbm8LDFTQIZH8M2 3ppPPAsXRhXnYvn8MycTdHq/au5R9QsL3F0ULCwRm7axJh4eFmoRLDYbAdbETSgYvu k1jLByx8bIr4Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 09/20] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Fri, 28 Jun 2024 11:11:24 +0800 Message-ID: <20240628031138.429622-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e90e3a638068..3ef8e13fb9e6 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -822,9 +822,9 @@ static void obj_to_location(unsigned long obj, struct z= pdesc **zpdesc, *obj_idx =3D (obj & OBJ_INDEX_MASK); } =20 -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page =3D pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); } =20 /** @@ -1500,7 +1500,7 @@ static void obj_free(int class_size, unsigned long ob= j) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1514,8 +1514,8 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) */ read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage =3D get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage =3D get_zspage(zpdesc_page(f_zpdesc)); class =3D zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8DD56F2E5 for ; Fri, 28 Jun 2024 03:07:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544046; cv=none; b=LpcKlatxlfArtflVyoSq2xf55pFU2NMFoU5Triz0LEtXOttVVC5/ybv5z4c9+QzNaZHpflT4DY2q2qAKzEDjE/WlcGR+j4aXHH8j8ePqzm8GnQlnPsS5VOctU5swsElTZS3BXTcmLQ6ThDiuD+3yndwBrl2syvDmdeB4hAi/K3E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544046; c=relaxed/simple; bh=lrAcBtSPY8wF6yEhTYORr+zHxr4lEQSsuXyA2QNdVVw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qe5/+RauRDafjl/jVNhfBmaE8WJ91MUZg+SpQdZcJ777wtmuYtFslpuLH7NUAfZ/MSkhQsFZVk0jxFTFfQD3Ojlhzly9muVg2XGCP/XyQO41efJZ8IMcSoVS3CG1VbkoA4N5XfQW8YwOh9xr0t1YJEuhYrgfmzuQWIQhYANFvHE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OYpxeUZE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OYpxeUZE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3DA8BC116B1; Fri, 28 Jun 2024 03:07:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544046; bh=lrAcBtSPY8wF6yEhTYORr+zHxr4lEQSsuXyA2QNdVVw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OYpxeUZEAarVD5SnSCfgAbdk57grg3isCoQNmyTfbmHWGcxRADl1KA0GOLSwNsskB 1LQ/VpAl4JEWfjQiJXUpI80DZ98V1A82SK/ZeiZy4Dik+eGYGzuIR5W4BVCiaZ3uT6 Bp4+4/OlTxkjz2cxCoEmrLAnnozkhD8LBnhHpPn+mBir2MUoS3t31oZxuqRGdiXY9+ 7YHLfuEPWLEtKBZLqm2gMIR/DXTs7xnUEYLYYZ5m6qjLL0rczZzWptVCpk7eykyEof 41Hd/z51hpHJWSSAZXHTHQuNpIjqUI+MH33JNtZgqR8PQmPy0us8dObV2536bZ6yNP 03ZYClyRuJqKg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 10/20] mm/zsmalloc: add zpdesc_is_isolated/zpdesc_zone helper for zs_page_migrate Date: Fri, 28 Jun 2024 11:11:25 +0800 Message-ID: <20240628031138.429622-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated/zpdesc_zone helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index f998d65c59d6..2e35b8220e05 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -98,4 +98,15 @@ static inline void __zpdesc_set_movable(struct zpdesc *z= pdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3ef8e13fb9e6..251a6e63e58a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1835,19 +1835,21 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc =3D page_zpdesc(newpage); + struct zpdesc *zpdesc =3D page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; =20 - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); =20 /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); =20 /* The page is locked, so this pointer must remain valid */ - zspage =3D get_zspage(page); + zspage =3D get_zspage(zpdesc_page(zpdesc)); pool =3D zspage->pool; =20 /* @@ -1864,30 +1866,30 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); =20 - offset =3D get_first_obj_offset(page); - s_addr =3D kmap_atomic(page); + offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr =3D zpdesc_kmap_atomic(zpdesc); =20 /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr =3D kmap_atomic(newpage); + d_addr =3D zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); =20 for (addr =3D s_addr + offset; addr < s_addr + PAGE_SIZE; addr +=3D class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(newpage, + new_obj =3D (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); =20 - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1896,14 +1898,14 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); =20 - get_page(newpage); - if (page_zone(newpage) !=3D page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) !=3D zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } =20 reset_page(page); - put_page(page); + zpdesc_put(zpdesc); =20 return MIGRATEPAGE_SUCCESS; } --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8CD01CA8A for ; Fri, 28 Jun 2024 03:07:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544050; cv=none; b=N8X/rg1rNX737cLmJhEFIpXcphiYlu1APZrGAAZ8/o3xon7Xu5rWq/T+WeX62AKIOFvJJrGl1trjdNnJWJS6jfGpqFyN57NFJhL8oh3Xk/8skIbLTDrH6UU03UNLbdHBeeOzlI9kN2BJqWiUN7BG5XAIU49WUbVOC9QGwBi/e9Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544050; c=relaxed/simple; bh=WVCo5FocoDi8mJGyEARAUtW59QMXLmGoSt30iD6IwAg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GIqzvCJBbToR1mpOsSMhC0fPkeEZrt2t51hiIJpWMfoFLQbhSm46H2UCkxAjSMSkvaaTVrgODB5uT6aLAC5bLKa6VdGw/3FmqjrqLkpF+SzkzlXPlCMvCVnF8mYWGFZ03sLKsWTmWCXhVxYRdoBSVPwRaY9++Pg+BPy2bXBQ2tc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=G93JlV8Y; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="G93JlV8Y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF836C116B1; Fri, 28 Jun 2024 03:07:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544049; bh=WVCo5FocoDi8mJGyEARAUtW59QMXLmGoSt30iD6IwAg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G93JlV8YYzxWgG9GYE5JjU2ygzVYPZRfSW0u+Lv+XKD+uGl5mDGQzi1sW2Ses9l1a jAqs4GDKakVW2T7olETXuSfQGUxe3N79I+c8/cGoDeT2dP/Qw0Y2SuS5rGQLiufXU5 cOLPgU28kk/KZq11ozpMKKNWpsjd4HHmGqk0MlEem4feI7RiQOLPstDBogmGArrQPa LcgEy1h/AuWnFZj+aEkAbS3r6jb/tOYSB09HoP/8yXn74PQtbDa7t5ldfdFxCMsP7Q a2HDHZ12nEeMpUneZSsUIak2KUFzhCLVi5SF4Or2usMTBhpCCoUqLLHZLhaZQkpljK xnmC1xGKfZrYA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 11/20] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Fri, 28 Jun 2024 11:11:26 +0800 Message-ID: <20240628031138.429622-12-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() wich is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 251a6e63e58a..216f8e6950ef 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -867,12 +867,14 @@ static inline bool obj_allocated(struct zpdesc *zpdes= c, void *obj, return true; } =20 -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page =3D zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index =3D 0; + zpdesc->zspage =3D NULL; + zpdesc->next =3D NULL; reset_first_obj_offset(page); __ClearPageZsmalloc(page); } @@ -912,7 +914,7 @@ static void __free_zspage(struct zs_pool *pool, struct = size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next =3D get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1904,7 +1906,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, zpdesc_inc_zone_page_state(newzpdesc); } =20 - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); =20 return MIGRATEPAGE_SUCCESS; --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C0E421342 for ; Fri, 28 Jun 2024 03:07:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544053; cv=none; b=A82lG483coMeXPjjYAEX0Kx76tUe00DpqwP9U2YA6YJA3i1SucIQr2kCowpKtHw6w8IzPXPyJErqg3CNqfZxAAe8zcKNApMplP2n2iEXE2wMraeWDxbLvxETLodEkrXvcd/71IyNnxTKi2oY4Pxo6n4PdyTrmH3mXkbh8o+0gSI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544053; c=relaxed/simple; bh=rDBA9PQhfrX2sld8/x0O9UG/mVNkTt8p9w0g00Zl/Is=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pqiL6F2cG+4gza4y28kg7/8sOAsJiiocKLMv8lH46b1yWaV9xNrANIK0XpPz4azzD3Nf3HYCE4rS9s/yAiD6nnMwoSmXZYVhOLp3mdqqBO7oK619k/Zznu0GPfVsgGq9F/8zPuCEgotRxNYwXyBkerf2cqK8zEPjS5PtdDQpVg0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mLYMc/7u; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mLYMc/7u" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 812C8C32789; Fri, 28 Jun 2024 03:07:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544053; bh=rDBA9PQhfrX2sld8/x0O9UG/mVNkTt8p9w0g00Zl/Is=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mLYMc/7u901OlvwzejVebGRJEyPAEygoLhV89brKlhhYjxZ0icIH+7gy5VLMEhstY nC7sgqj0XdoqBL5QlBfnBN+qP+BIpEJN0PjpCOSOS2WuR74IrOaEn/iLirkfQxTzS+ UkAx8p2uK7Z8em0W7vDigSNNjGWkjfZxdeqM90qox2/0MhzJ8EBVvPhWtBO936eT5i AnutNkwsiJvqnDczkg2qwn5VxLl1uIwZDool0UvbpMRsJO7a5R6tn9KT0sHs1oQWWC 7L62XReP/pejWZpdCCQx8KXAehDgaiohDLewX5nNd2thVBKgKwWv6LbGWHl/ZVXaNP WDThjKrP9FPBg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 12/20] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Fri, 28 Jun 2024 11:11:27 +0800 Message-ID: <20240628031138.429622-13-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 2e35b8220e05..186ff9711ffb 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -109,4 +109,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *z= pdesc) return page_zone(zpdesc_page(zpdesc)); } =20 +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 216f8e6950ef..bf5a1b63bb17 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -903,23 +903,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; =20 assert_spin_locked(&class->lock); =20 VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness !=3D ZS_INUSE_RATIO_0); =20 - next =3D page =3D get_first_page(zspage); + next =3D zpdesc =3D get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next =3D get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page =3D next; - } while (page !=3D NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next =3D get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc =3D next; + } while (zpdesc !=3D NULL); =20 cache_free_zspage(pool, zspage); =20 --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C9CF21342 for ; Fri, 28 Jun 2024 03:07:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544057; cv=none; b=HWSc4mxBnXWvHVRNemNiu+EnpIzheMQtVVCoIrPNNFoWvcOshKMB4U7kuewIkeq5eAQYJ432zUDsXSxkF8tr+xJF+b0FDQTU7+fUpyECzW/+Q5gWrqeL7I0JsMO+Y8ijuLIxSd8uA6ijVpB21TUijI8TviuZdT/DBBJ7cNfdaSs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544057; c=relaxed/simple; bh=gedDp0RHU7Zb19ZIAumudrHJ1RWmViS9C8nGeod0zds=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Rzl34+byiKnyFu3YnStKPzInhHH0RPZlpO3krSnMmLAmRd2i86QqMyG77VQe8ZRr8DJnRW7foD4So7P6gnjVFE8T5z6V4YeQ84Bwd0oFFHe30kGiwhUaPV25OcfL/6n5Xjivov2HzqdEz3aIZVLNPsb2fMNJiHnaq9FsSEAUbtk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oIUTcWXK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oIUTcWXK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 14C8CC116B1; Fri, 28 Jun 2024 03:07:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544057; bh=gedDp0RHU7Zb19ZIAumudrHJ1RWmViS9C8nGeod0zds=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oIUTcWXK+xs8W0Bn+nZd+E2zqjNEJ8xxmE/ynQKaenuBx0s++2m7yO67tPFf51YlB QNtpHUsRZ3/KM9uiJkqmmWApsGNcD5Z4brLMBJm7rD9XT6E+m8POORQcq0+eM6IZhX A3xTV3/hWWTqz+QxjbnhnqWH3t0NQdJefRAGWm4AaAoiasyGi7TJXpct7XkGKoyPEf bo8vRCiWJxJtEFuXHFms5scP7tUJ2i+4wyOJBLnYWfUmoIVDKACDbY8HbwIfv5bR8q OPBReFRDgjZSfMO/MF1XAy2s/p8kAdKbeWbZLIC+70s8CeBaHItKYNJoWCiVttKUU3 6qDrwLqLmgX3w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 13/20] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Fri, 28 Jun 2024 11:11:28 +0800 Message-ID: <20240628031138.429622-14-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bf5a1b63bb17..e8af01768ae0 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -828,15 +828,15 @@ static void obj_to_zpdesc(unsigned long obj, struct z= pdesc **zpdesc) } =20 /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_i= dx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx) { unsigned long obj; =20 - obj =3D page_to_pfn(page) << OBJ_INDEX_BITS; + obj =3D zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |=3D obj_idx & OBJ_INDEX_MASK; =20 return obj; @@ -1395,7 +1395,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(zpdesc_page(m_zpdesc), obj); + obj =3D location_to_obj(m_zpdesc, obj); =20 return obj; } @@ -1884,8 +1884,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj =3D (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8F9534545 for ; Fri, 28 Jun 2024 03:07:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544061; cv=none; b=DqNoL4EmpoZazNUSRNogEliAG7ca+Gnp0JKJWuKfPNXhcfX3Z5RNwk1L6rtJ7a0B8YpiiWBaGFMcCKmjxZJxXl7fkK1pW0MZIopQdIID9vWVTzh7CsfC3rsSSrP+w/BIP0flAmMsoqvBB0M7e1tKSezZqxSDbe0TGwT7FdhtPzU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544061; c=relaxed/simple; bh=6AxrbjsULXI2U/XKKk6mycuO4iJeWrzhsSGBcy6kdOQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZCuqqxPEXe01gOeVGJC/W4JdoSH7KRSz1bzh5hzeA5XiDv1F7wRH2+cXpWr0ErF17aQrRMsMtwJPtivquZfDgeCEDvlVo+bG6hoTL+YvipmuV2DqUjjnmZFeBJKE/dkqnkhBaLnniitVHFQh3wzM1z1NcV89U7TdPxKsNunmafs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aECRD5kR; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aECRD5kR" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C6B02C32789; Fri, 28 Jun 2024 03:07:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544060; bh=6AxrbjsULXI2U/XKKk6mycuO4iJeWrzhsSGBcy6kdOQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aECRD5kR9xoAvSpndBIFlgKo2JqqxBip4pTFrYHYczi9pt2fEs0YDBL3FL9ihGLPN dglkNp8v2wVgW4fRUFamWff3pQLICuo6u4rcQytZJhWVKePCQKo2Z1pc4Keqr24Min kCxKz85MfgScoKYa8Jmkhm5LUaoIo6ll2NeyVgC8MU1CIjZATofW4L/z07K/VsOoNp Hg0xPuVEK8REdGAmwy7pa3YxAZ/RfCoVSfYm+87q30iT1p6LByqLPnnSQGNB3DHnmn L5Ph4XaeiMvUxy/hGgtGom2xh/BcTHyMdCpa9KAncBtu+ct1oNvBGLFPyrramw0rrJ Gi41UA6Yh6veA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 14/20] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Fri, 28 Jun 2024 11:11:29 +0800 Message-ID: <20240628031138.429622-15-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e8af01768ae0..a10af49d8d08 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1640,14 +1640,14 @@ static void migrate_zspage(struct zs_pool *pool, st= ruct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx =3D 0; - struct page *s_page =3D get_first_page(src_zspage); + struct zpdesc *s_zpdesc =3D get_first_zpdesc(src_zspage); struct size_class *class =3D pool->size_class[src_zspage->class]; =20 while (1) { - handle =3D find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle =3D find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page =3D get_next_page(s_page); - if (!s_page) + s_zpdesc =3D get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx =3D 0; continue; --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08F444D8C2 for ; Fri, 28 Jun 2024 03:07:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544065; cv=none; b=JoPLPrAnhjduLDNcJIoi+hklNxSu7fOFLTjqBSItSRBWVII7S5MqJ2Sn+NI9duH9zlt9Kh2Ark/6pOGhCGBmdBoBzvHu+jKv075FtG+VUcT7rzBHT95oQtWmkYqGCQy51Kt4EpR7LF72KCK44NBtb4PJp+oyG+XJ+lptWlnO4wk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544065; c=relaxed/simple; bh=vusdBHzfqHkaDK8baoOjr47f4AAjSRZFJhrtyFn5Ac8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ocHnMChfoH3Por8OMrZqrnikMamat1y7DB4GoriRgA1sg6oiisjPYBl0ML6C/YyaiOZx8iCoCetoJ9bonfLDyMPjh1SeiTvu64AKO4EUEgSAx28j34EzCY56ENqaI0S2fuUmo6YZWZUrNHXIdhRCddx7A7KUjcQf6TV/0QITMa8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZZ/oDvEH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZZ/oDvEH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 84569C116B1; Fri, 28 Jun 2024 03:07:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544064; bh=vusdBHzfqHkaDK8baoOjr47f4AAjSRZFJhrtyFn5Ac8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZZ/oDvEHhBu8s6dToln3pesB6GaFXokhxzB7lNQwP7xSG6j/NBhslRWc1e3jMBzEO kiPYgyN2CWYo+tKF0U246DL7nSEXR4GnlsXz+SrRPmAc2lbptCrK5YiJF2xKiZdbnZ hv+O/C6NuSsMex3Y3mffzQo0IqaUOvYQc3E9iDlogODu7Y6/LHWRVBJwFSQnHwjGE7 GPzFRFyB0QXF9PAMKQW1jWAmIuF5gguivX0DnSVuME3aIdikODNiyyUvaqE/uv4A/9 PxxgTIaZaU8gQeEhxtl35LiMSPjI6rNW8ra32YGAVam6NcgKExAEY3Xxq9dui01BGi 68jFL4HIjdE6g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 15/20] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Fri, 28 Jun 2024 11:11:30 +0800 Message-ID: <20240628031138.429622-16-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a10af49d8d08..c136124adb10 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -781,9 +781,9 @@ static int fix_fullness_group(struct size_class *class,= struct zspage *zspage) return newfg; } =20 -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage =3D (struct zspage *)page_private(page); + struct zspage *zspage =3D zpdesc->zspage; =20 BUG_ON(zspage->magic !=3D ZSPAGE_MAGIC); return zspage; @@ -791,7 +791,7 @@ static struct zspage *get_zspage(struct page *page) =20 static struct page *get_next_page(struct page *page) { - struct zspage *zspage =3D get_zspage(page); + struct zspage *zspage =3D get_zspage(page_zpdesc(page)); =20 if (unlikely(ZsHugePage(zspage))) return NULL; @@ -801,7 +801,7 @@ static struct page *get_next_page(struct page *page) =20 static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage =3D get_zspage(zpdesc); =20 if (unlikely(ZsHugePage(zspage))) return NULL; @@ -851,7 +851,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc,= void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage =3D get_zspage(zpdesc); =20 if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1267,7 +1267,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); =20 /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1317,7 +1317,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) =20 obj =3D handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 @@ -1483,7 +1483,7 @@ static void obj_free(int class_size, unsigned long ob= j) =20 obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset =3D offset_in_page(class_size * f_objidx); - zspage =3D get_zspage(zpdesc_page(f_zpdesc)); + zspage =3D get_zspage(f_zpdesc); =20 vaddr =3D zpdesc_kmap_atomic(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); @@ -1517,7 +1517,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage =3D get_zspage(zpdesc_page(f_zpdesc)); + zspage =3D get_zspage(f_zpdesc); class =3D zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1851,7 +1851,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); =20 /* The page is locked, so this pointer must remain valid */ - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); pool =3D zspage->pool; =20 /* --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A2206F06D for ; Fri, 28 Jun 2024 03:07:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544068; cv=none; b=ZrtmnQZGr7uS9XogIdKAKT1ZIQzepDjb/jb0I9fKOqsGe1zuxzDK+Y8H1qk4a56V7++u/R3L+tI06EMtdvOu3mXJphpvTwLXJii1uFLxPXpHC/7qYw5L9drWZzghvmjDt/zQv/OCtcfthVN1nRNs0tBBNvlyICuiMkxC+D+GAus= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544068; c=relaxed/simple; bh=kldqwAh9cZgKWc/qnQGuGYAkpd0utSr37gCIdaPuGYU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FSbNL+wAoxKOhUS7yIoD6OqSkQlIN2LwKf/zdeYYFzUXe80yPLs/4AXEvffh7zGl9FCZRvlYBr2d7MgGJ4eeHHesAcw5F8GSsH1TY/XUStqgaEC/gYOlfBwakJyKHxZqAxmOIEY5eIHjif+Ek9t30QK8NowotKjdZ8MU4lI/lvk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cnRaB13D; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cnRaB13D" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 41863C116B1; Fri, 28 Jun 2024 03:07:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544068; bh=kldqwAh9cZgKWc/qnQGuGYAkpd0utSr37gCIdaPuGYU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cnRaB13DEdjhJTai8RSwypjifYR0A/D24GZqzbB/taIZOGJfx5ukrATQw9p8OyWzM iAAxQfNiplB4VzN0rtbVW16IMBdZ/RsWZzSYGoCNXE7P0FcRAh0awod5grcab4giG/ H9STRRKZnmZoXd6hwu7YPDdhTGj85zNkM/x+g2gmKq3nr/+1Yqa5SxXdBusHA0/gHA gavi3MwMdlDst2QYjmWdcNLkXxzrNsfj+AkN1SOSbDG0fQAH2wr8szA84Zub72pyP6 UeXApiZ2X+rmvh2mr8QAuT5tuyHFj2yDnC3+V4z8vxOPqUv+lI8tAvJlryj3mSk/gL rH841R6zzOW+w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 16/20] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Fri, 28 Jun 2024 11:11:31 +0800 Message-ID: <20240628031138.429622-17-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs:get_next_page/get_first_page/is_first_page. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index c136124adb10..951292dbcae4 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -470,11 +470,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area= ) =3D { .lock =3D INIT_LOCAL_LOCK(lock), }; =20 -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static int is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -491,14 +486,6 @@ static inline void mod_zspage_inuse(struct zspage *zsp= age, int val) zspage->inuse +=3D val; } =20 -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page =3D zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc =3D zspage->first_zpdesc; @@ -789,16 +776,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } =20 -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage =3D get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage =3D get_zspage(zpdesc); @@ -1974,13 +1951,13 @@ static void init_deferred_free(struct zs_pool *pool) =20 static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page =3D get_first_page(zspage); + struct zpdesc *zpdesc =3D get_first_zpdesc(zspage); =20 do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page =3D get_next_page(page)) !=3D NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA92B4436B for ; Fri, 28 Jun 2024 03:07:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544072; cv=none; b=MWWMwTg9Y46NX9HPAJ8JbDR2uuPMZ9NhNZc3X15T2T8Tpv2gkFdIqVzDHarZJxNEBKfrFNDE7/+bnJT+Ro5HAvc2UeSNF5MpuNBB2y7/UMgZhTp52SVKsNSdiUWVlLHD1F6Kl1BmfIAXz4tB7SbIICW5ci/5vceunO7Cm/RAa3A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544072; c=relaxed/simple; bh=BULUfQtc3EjNlGEEdYDtph/07BaO7bkwBtlyH1/5iBg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gNKW02bl5r4NmwajdZokq1llqdSOOph3X1G+kSVzkaP06YTtukssBBxJJBTlve/h6FGICw6yccSC6+NPC9cUiBy238QWv4sRab1xPgKbFF+hGPYm5suPyWzJFuR5hFvjF3GlpLOtUffba+iXDJU7NjWM9SgN1RKtlDQYMGAHYUI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Kn8GoGgZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Kn8GoGgZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C7603C116B1; Fri, 28 Jun 2024 03:07:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544071; bh=BULUfQtc3EjNlGEEdYDtph/07BaO7bkwBtlyH1/5iBg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Kn8GoGgZJ1gvzJR2ncaJOTr4cft0ptFYqqqvbtAwTpCfjfvz4TvWV5uAi2Eez6HFe fKUWmp/Ts+D6L4Djx91kxmxRNiKIpJnkx8+z1/ZcZdL8U1zcxymua+7Gi4mAGVdSPM JORUVVR6FgEOCUCxesPq43knn7yJph28Reeb3A8CaTzJ0LJYlKmw1JaZJKOtO85Pce R/zlvqlS7vVfrxk14GzwY82GipQfBVN9n7LaPjttYrKJmpNLAq4aXSGvIaD6Gh5t+u QRlJ7PvfPSXp5TlA6TInbxbPPsrHJyyGJD7s2T5vpgnRSaqu8yy/uWEXSp7AbPLic2 7bibTnUJh3iLA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 17/20] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Fri, 28 Jun 2024 11:11:32 +0800 Message-ID: <20240628031138.429622-18-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 3 +++ mm/zsmalloc.c | 32 ++++++++++++++++---------------- 2 files changed, 19 insertions(+), 16 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 186ff9711ffb..17941774c46e 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -14,6 +14,7 @@ * @next: Next zpdesc in a zspage in zsmalloc zpool * @handle: For huge zspage in zsmalloc zpool * @zspage: Pointer to zspage in zsmalloc + * @first_obj_offset: First object offset in zsmalloc zpool * * This struct overlays struct page for now. Do not modify without a good * understanding of the issues. @@ -29,6 +30,7 @@ struct zpdesc { unsigned long handle; }; struct zspage *zspage; + unsigned int first_obj_offset; }; #define ZPDESC_MATCH(pg, zp) \ static_assert(offsetof(struct page, pg) =3D=3D offsetof(struct zpdesc, zp= )) @@ -38,6 +40,7 @@ ZPDESC_MATCH(lru, lru); ZPDESC_MATCH(index, next); ZPDESC_MATCH(index, handle); ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); #undef ZPDESC_MATCH static_assert(sizeof(struct zpdesc) <=3D sizeof(struct page)); =20 diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 951292dbcae4..3ce49f0372cc 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -496,26 +496,26 @@ static struct zpdesc *get_first_zpdesc(struct zspage = *zspage) =20 #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff =20 -static inline void reset_first_obj_offset(struct page *page) +static inline void reset_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - page->page_type |=3D FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + zpdesc->first_obj_offset |=3D FIRST_OBJ_PAGE_TYPE_MASK; } =20 -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } =20 -static inline void set_first_obj_offset(struct page *page, unsigned int of= fset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned in= t offset) { /* With 16 bit available, we can support offsets into 64 KiB pages. */ BUILD_BUG_ON(PAGE_SIZE > SZ_64K); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &=3D ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |=3D offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &=3D ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |=3D offset & FIRST_OBJ_PAGE_TYPE_MASK; } =20 static inline unsigned int get_freeobj(struct zspage *zspage) @@ -852,7 +852,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; - reset_first_obj_offset(page); + reset_first_obj_offset(zpdesc); __ClearPageZsmalloc(page); } =20 @@ -936,7 +936,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) struct link_free *link; void *vaddr; =20 - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); =20 vaddr =3D zpdesc_kmap_atomic(zpdesc); link =3D (struct link_free *)vaddr + off / sizeof(*link); @@ -1593,7 +1593,7 @@ static unsigned long find_alloced_obj(struct size_cla= ss *class, unsigned long handle =3D 0; void *addr =3D zpdesc_kmap_atomic(zpdesc); =20 - offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + offset =3D get_first_obj_offset(zpdesc); offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { @@ -1789,8 +1789,8 @@ static void replace_sub_page(struct size_class *class= , struct zspage *zspage, } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 create_page_chain(class, zspage, zpdescs); - first_obj_offset =3D get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset =3D get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle =3D oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1845,7 +1845,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); =20 - offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + offset =3D get_first_obj_offset(zpdesc); s_addr =3D zpdesc_kmap_atomic(zpdesc); =20 /* --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7913A7441A for ; Fri, 28 Jun 2024 03:07:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544075; cv=none; b=owJId4deZTph7+B/mppObMsO7Tvkr0SBc6+2wIVhOHKNst6n+9u49m/KpTuwfVgkqJq307hkVSLIeJ1wtN7UdDsn0m8pFDJNdNUtAw83YCrM12vVnF8d5Kn0qa+A+Nha6ywDNFyhZkT0rOmf9pXa78gUlIXWwK4F9JLWgluOYuQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544075; c=relaxed/simple; bh=fr0OSLBdVEu4PiREH2sgZbNiunlztN2rB5Cg12p8Bfw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QUzJBIhYqSenohi9jCH+ODyaeeCWmn37/7nitMe/Z352ZFitUo8MviDUbozh1SCvYNv8qDecNOMK5aMF01N4LUw5GbvFY4ud+do+EDFq2LWohrbLa8PfxN4Ey6+727i11jsnH47rXWawPe86axNmhS5kF04GTJCAWgxqTaJ6VSo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DvOnLEDk; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DvOnLEDk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8258FC32781; Fri, 28 Jun 2024 03:07:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544075; bh=fr0OSLBdVEu4PiREH2sgZbNiunlztN2rB5Cg12p8Bfw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DvOnLEDkbTMaJdXPPoE1HQN7ICZRZxu/wx2iuZ1D7NCvHtiBu/yFUSmpp5eoK+R+Z TvQqHcw2vcNpwhurl3xIfOeKvMoBgVhkiFsLwk0VU4++awCje1qxplJZfPZn0cbjYD 5x1ZXJrtgAeHdcFkb8/i8YpR1zJv1PKsrAXbqKRKqZPO0plgI5cGurmLzHbirBSPEc dhk0stZxfzY2xA42dD8R6xyMrrBjAVjxbF2X/2AT1IGkp3TraKKTbrLLOCjiOnZ3+K 08leqFILaTWgWWtRcj3nawG+h99Vs1nRjoc37yqyeL2bdDuvr6V5RNmcnnWddzGbGs yfGXWvqCkDi6g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 18/20] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Fri, 28 Jun 2024 11:11:33 +0800 Message-ID: <20240628031138.429622-19-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 17941774c46e..c6a44fe04f97 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -102,6 +102,11 @@ static inline void __zpdesc_set_movable(struct zpdesc = *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } =20 +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3ce49f0372cc..abecc8a5c566 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -848,7 +848,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page =3D zpdesc_page(zpdesc); =20 - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 541337605E for ; Fri, 28 Jun 2024 03:07:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544079; cv=none; b=L0f7OGuysewkj5ekbL3lZvNot08jUb1oy0ByMl5+TQ3SnIblGcCE0FvInjKn57gUBuvtAGfWgW8tF/x8/YjY4elBdykvA+3hpGY07Y1r7Ge/3/p5fXuE8S5+QRPeggRI1BLcoera/DL8QmmyAukl7d/7IiVu9CeTKWfScTJeQX8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544079; c=relaxed/simple; bh=1b2JRlKwTFXQCKXls2arw2ZYXd32r3yXKlFqVVcujQo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AKIBRq9jVEsLOCot5BApZdFgik6WbLYKde/w/W3BcuyBlcZWhwVf9iEFhc/oe8BEgMeTQbGKRuYvV4f0dkcYD1PiKuJkovzHpkuCIhtbrphAEACAg+7aWciCWby15X3DqAgDW647NH/FXezdR/4L/yIVSLLB+IPg4xIKbaOhwE4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Mk0Y3czk; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Mk0Y3czk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13BFEC2BBFC; Fri, 28 Jun 2024 03:07:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544078; bh=1b2JRlKwTFXQCKXls2arw2ZYXd32r3yXKlFqVVcujQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Mk0Y3czkLU95tgGbkMh1dc7Ujh5UVx4rFxHwXuxVsgEsk/u+S4AReBO1g56nW+Nhh 3YmhlRIT03b0vDYM21OStct7U9LXxp1hcClg6Ir+2o7e3MxGkjqKbb5TODMs8B/69m s+9yoffNOB1dnVjRcLRz7Za2COjRzhajsk6g2M9mBVyu1yDZ0XyVjnPwzMhEO2SUBx oWnWAdZdNEGIBe+7XR2MbwfFfO024TyLiN67b3jo42V6s/nL20YHE/H5HsXiKGt9jr 6ax09zDtLSK7qDG6FRD4gBONZiiKEkcRz4bX6SQ4ufwhNd7vi0wDAs2fplpKZqB0GJ 479dAcAhfe1uA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 19/20] mm/zsmalloc: introduce __zpdesc_clear_zsmalloc Date: Fri, 28 Jun 2024 11:11:34 +0800 Message-ID: <20240628031138.429622-20-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add a helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index c6a44fe04f97..e346b39aba59 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -107,6 +107,11 @@ static inline void __zpdesc_clear_movable(struct zpdes= c *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } =20 +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index abecc8a5c566..eaf8dad04f2c 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -853,7 +853,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; reset_first_obj_offset(zpdesc); - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } =20 static int trylock_zspage(struct zspage *zspage) @@ -1026,7 +1026,7 @@ static struct zspage *alloc_zspage(struct zs_pool *po= ol, if (!zpdesc) { while (--i >=3D 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); --=20 2.43.0 From nobody Fri Dec 19 04:51:19 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23790446BF for ; Fri, 28 Jun 2024 03:08:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544083; cv=none; b=WDqPJMxWLFnUb9yleIDjPS2pekpeJ9EDgh0oV94tn8vogtKOUN44T5eh0WH5JzXqQfovVaQuCZzXwZLfgkSiUVoARGlT96q7i5kXuF/1jUinG31WZybvYQdZHvAFBC0pzvcWq6kB4sIckUAgAk8tA2X3Sl4EjEGUMIuwY6fRqCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719544083; c=relaxed/simple; bh=4MKTXWqqoUrRELMNSTMj+qMl+m60BitHu1nDeXKuUgs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jH7sloLmkN0UqAgPU27NqZNgDa8NkkYvta5qLQKGo3STQemnUvPNp4JFTmRR9LhZe5VHSf9OyelIgZFK7gnqsZA3wPdwEwFClkD2fkgSrKIIqdPpIVcUgC7g5AYJyxjX/mIjGtFadHyDcgBJv4TjjYc9eFvoIsSQZCAp8zDE4HA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cbU3x2Dd; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cbU3x2Dd" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 97FBBC32789; Fri, 28 Jun 2024 03:07:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719544082; bh=4MKTXWqqoUrRELMNSTMj+qMl+m60BitHu1nDeXKuUgs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cbU3x2Ddh8UT1d2qoVfthnhMM+GJKywAez5C89nw56pBJLHMwtkvKj/JzIwEwMFRk wzmpdRUrNHYRvjA6DyL9hLgF2gtdCELLqOT0Ac2uw6PTQ1z+HCS0CPop6sQ5JrxKaL H6DNIP2g8MFFZxd1i8wYTSKZYOUXhQ+x0WRhjXVCeosCcHVhUCN33Xlf7v6uT8OUDw i/46Pz0IAIjX67naNUMaB2vjL8T6s7Fh8s8wMLWP9XALYQ8N1EidkmLu2LRwvgLF0J yE2ElJWpvcfyCD92w/NKVFID3nhC2abtUH7BW1bSfu8OtUlsZUF724zhamgTRYry3Z 7pie/+HAHWdWg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com Cc: Alex Shi Subject: [PATCH 20/20] mm/zsmalloc: introduce __zpdesc_set_zsmalloc() Date: Fri, 28 Jun 2024 11:11:35 +0800 Message-ID: <20240628031138.429622-21-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240628031138.429622-1-alexs@kernel.org> References: <20240628031138.429622-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add a helper __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index e346b39aba59..ef36fc3de75d 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -107,6 +107,11 @@ static inline void __zpdesc_clear_movable(struct zpdes= c *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } =20 +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) { __ClearPageZsmalloc(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index eaf8dad04f2c..e2a506f53957 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1032,7 +1032,7 @@ static struct zspage *alloc_zspage(struct zs_pool *po= ol, cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); =20 zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] =3D zpdesc; @@ -1825,7 +1825,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); =20 /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); =20 /* The page is locked, so this pointer must remain valid */ zspage =3D get_zspage(zpdesc); --=20 2.43.0