From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A293F2B9BB for ; Mon, 2 Sep 2024 07:16:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261396; cv=none; b=JBwvxeYEhOcB/E1pEEyN/xyUDBFEcPqNWZxNfLsAiUVN5MKaQ0ZBLKUH3BuOX8Ixyil4hXr5nLCcKEMJ17CMZgurk7n+mTLPhEfnU8Lw1HdiCvsMchzNWQ86ol0blXoBrpvKViFALr0QOCBny9fdVf84rYqEekyX+oD0fyQJEeQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261396; c=relaxed/simple; bh=35oS5cqvsOoGfro8SlCHvjRjEwfOQy4xgC0vABlxtJk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dujYXoB1lhG3XrTMMvZsuSxMIcT4QReqhJ4qn0z5bVHo6GAW6CisAA0OxRVppSXOJOnlG1MATpnqxjQ0pKDUJfNmuwPZEeRUg0R0W82Z9SdI1cLtQFh+DX+8LCyMJoWFOFupqt7jTjoeD8QMpyFgr5xaoF+sXaXIb747nNma0Nc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=agYKSAil; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="agYKSAil" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5BAB1C4CEC2; Mon, 2 Sep 2024 07:16:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261396; bh=35oS5cqvsOoGfro8SlCHvjRjEwfOQy4xgC0vABlxtJk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=agYKSAilfuYO5J7eGOLpdxzw52rI9hxDd1DKce1cIpm9Fjt0aSVLplL3wmg3sHr/G HcnCqtqK+RRS9/TCwa5CYuM7L/NqpTsTB4pDnREzgFVbLECV8qmZhLyt0sh+yUJ0TS FRx3J5kf7s+dBjFhx4zq5oHD+lnChGZzvxxnvQkR9PX/G6oLAS0039chUbgygF+8tp 8ZPsA6lo80t4DbiB6RX9oxVQLDuR6S6ekvtWBhywuT7ZpbwpxXZEVZhDdCjGsMqA2d 2YOFTQU8dknUtnJQPcYzUM2Dpjx2Gq7vW03RUznWSZB8bJNXXeiZPGqYcGs17eJXul wy6Zp76/km4Ug== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 01/21] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Mon, 2 Sep 2024 15:21:12 +0800 Message-ID: <20240902072136.578720-2-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi The 1st patch introduces new memory descriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). And keep the memcg_data member, since as Yosry pointed out: "When the pages are freed, put_page() -> folio_put() -> __folio_put() will call mem_cgroup_uncharge(). The latter will call folio_memcg() (which reads folio->memcg_data) to figure out if uncharging needs to be done. There are also other similar code paths that will check folio->memcg_data. It is currently expected to be present for all folios. So until we have custom code paths per-folio type for allocation/freeing/etc, we need to keep folio->memcg_data present and properly initialized." Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 72 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 9 ++++--- 2 files changed, 77 insertions(+), 4 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..721ef8861131 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirectly used by page migration + * @mops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Points to the zspage this zpdesc is a part of + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly use by page migration + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + struct movable_operations *mops; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; + unsigned int first_obj_offset; + atomic_t _refcount; +#ifdef CONFIG_MEMCG + unsigned long memcg_data; +#endif +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) =3D=3D offsetof(struct zpdesc, zp= )) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, mops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); +ZPDESC_MATCH(_refcount, _refcount); +#ifdef CONFIG_MEMCG +ZPDESC_MATCH(memcg_data, memcg_data); +#endif +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <=3D sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +/* Using folio conversion to skip compound_head checking */ +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 73a3ec5b21ad..5d3e27083c72 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -64,6 +64,7 @@ #include #include #include +#include "zpdesc.h" =20 #define ZSPAGE_MAGIC 0x58 =20 @@ -251,7 +252,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -446,7 +447,7 @@ static inline void mod_zspage_inuse(struct zspage *zspa= ge, int val) =20 static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page =3D zspage->first_page; + struct page *first_page =3D zpdesc_page(zspage->first_zpdesc); =20 VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -939,7 +940,7 @@ static void create_page_chain(struct size_class *class,= struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index =3D 0; if (i =3D=3D 0) { - zspage->first_page =3D page; + zspage->first_zpdesc =3D page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage =3D=3D 1 && class->pages_per_zspage =3D=3D 1)) @@ -1315,7 +1316,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle =3D handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index =3D handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle =3D handle | OBJ_ALLOCATED_TAG; =20 kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 549FA2B9BB for ; Mon, 2 Sep 2024 07:16:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261400; cv=none; b=lBvAFBqsdLOQ369dSLNxDL51MsFWM5dvixKtHWKzbuKK3ORG+CwyJ/Xy+ZOqTIzE+oC44ZRyEEFRbQZulMUbZsY9kCsbp5rHs9UBcTDYtrnseY5FwGUxkqKcMglkPu695Ne6egjZo77lo63AYBQz/UmgG4Rq7Kv9TEi5mDgbxB0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261400; c=relaxed/simple; bh=uRakrlKF0QiEotK1S+ipaw/WhogyCqf9q/BFIecbO0w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SRZ1BRbntMmj6mX0C2fH1lIF9WKHl1JEc2XZAKtxV0UeiKs+MiesNOWn8EQgvrjCE+EzSdiZiwsoLSnvSAc0P92GdWGLAWNIvSgn/x9ONojLIlcotYEC8SQ17fP4qHnhm81x/v5bqsCnN06fYUYNvtGkrnwSHPCcQab62+BpnZo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=omLdEnIe; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="omLdEnIe" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E224BC4CEC2; Mon, 2 Sep 2024 07:16:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261399; bh=uRakrlKF0QiEotK1S+ipaw/WhogyCqf9q/BFIecbO0w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=omLdEnIeCSt4q9xhdyugvCd7eKaYl3jjcrrGs/5B5FQqUj6Mo5C+3aXl5bnh88LRb Kf5fnjA7jl1SR+iGOEUAKiCBuhlX1I/qHaw1wslfDP8slOpfXwmt2YsRgTHwbbOb4d 4a6sOIw5yI4d2mi9KKiLXgYak6mKpr+3ra02wNFlO+5HtvQibAw1un3yWbZlePLsyT XoBvbOMXFe/4sBiEHiPWG6H+JEsRShAaEAXDfmzMxiN0u8coenLq398BsAGcojPe0L xuRvTQSzBgZiepFORp39R6M7m5ShGBV1PKoQqBUqM+k1aYTQp09JtmAGIYTKWGNOM6 nDfHfMcBz4kXw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 02/21] mm/zsmalloc: use zpdesc in trylock_zspage()/lock_zspage() Date: Mon, 2 Sep 2024 15:21:13 +0800 Message-ID: <20240902072136.578720-3-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi To use zpdesc in trylock_zspage()/lock_zspage() funcs, we add couple of hel= pers: zpdesc_lock()/zpdesc_unlock()/zpdesc_trylock()/zpdesc_wait_locked() and zpdesc_get()/zpdesc_put() for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. BTW, thanks Intel LKP found a build warning on the patch. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 721ef8861131..782b5ad67cda 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -69,4 +69,34 @@ static_assert(sizeof(struct zpdesc) <=3D sizeof(struct p= age)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) =20 +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d3e27083c72..9806b1629880 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -433,13 +433,17 @@ static __maybe_unused int is_first_page(struct page *= page) return PagePrivate(page); } =20 +static inline bool is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } =20 - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse +=3D val; @@ -453,6 +457,14 @@ static inline struct page *get_first_page(struct zspag= e *zspage) return first_page; } =20 +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc =3D zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffffff =20 static inline unsigned int get_first_obj_offset(struct page *page) @@ -739,6 +751,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } =20 +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -808,11 +830,11 @@ static void reset_page(struct page *page) =20 static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; =20 - for (cursor =3D get_first_page(zspage); cursor !=3D NULL; cursor =3D - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor =3D get_first_zpdesc(zspage); cursor !=3D NULL; cursor =3D + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail =3D cursor; goto unlock; } @@ -820,9 +842,9 @@ static int trylock_zspage(struct zspage *zspage) =20 return 1; unlock: - for (cursor =3D get_first_page(zspage); cursor !=3D fail; cursor =3D - get_next_page(cursor)) - unlock_page(cursor); + for (cursor =3D get_first_zpdesc(zspage); cursor !=3D fail; cursor =3D + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); =20 return 0; } @@ -1651,7 +1673,7 @@ static int putback_zspage(struct size_class *class, s= truct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; =20 /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1663,24 +1685,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page =3D get_first_page(zspage); - if (trylock_page(page)) + zpdesc =3D get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } =20 - curr_page =3D page; - while ((page =3D get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page =3D page; + curr_zpdesc =3D zpdesc; + while ((zpdesc =3D get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc =3D zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A202317F505 for ; Mon, 2 Sep 2024 07:16:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261403; cv=none; b=qXGR7t9AQmZW0VdCtDE8z+4du82MaJdDtLdW+s8wCkG368lUQerSuxu6rIKSdxG+CUKJvpPH/bskyYHsdas/xglIKyY9CiUa6AQ17WknoZFREUzh3+AvqXoZvLSPIBSethr1Yk0EacrnNkSCGj3034thKUeXUm3wwv//IwobI2A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261403; c=relaxed/simple; bh=WINuZGeFx9tI8KdY19QN4OBBU7X+cSgJrgZBdb4m5I0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bpXI/kJmZrkOZmOsxaXYRc4IHL8HscKoNld2nA1QLUQQGkhNixaIXP6O30P0fvhN4SA7/etYl7WmUKHi8b8LUztLczDnufYbSx8hoBLSDAKhvcjXLYwSGj1pBqj3HfLjLIqWmHGORbUrOMjIXpICkg/MJ+UWoQ//xEH6n+Kvj6I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Hwc3Rqz3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Hwc3Rqz3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 72301C4CEC9; Mon, 2 Sep 2024 07:16:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261403; bh=WINuZGeFx9tI8KdY19QN4OBBU7X+cSgJrgZBdb4m5I0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hwc3Rqz3ixvX3jviF7KczOLII9qkX8ppPQNRek7rHhU8uwwExWu9TBfPEG0Z+d9b5 d1gtCEaQNeEsMRzGfr7gUwCFA1q0jlki4TeEvvDrJaXFnyrwiK2qkSTY3NscwUMmFL V8YeGKG+79ICG94273EdMqDP7cIBali7OEv5KU2y8SpOdC0m5Pv93IVlaBBvAg8goK VDA0Cp+KGinMewjf5gDY3forURqjs1gT8nJ/EqzDzoFaz7AGVA7wePPX69MVBXyFVf 2/az9VA4ty2O5MCtt+zhpVcXthqY6Zq0BoyOOC85PHRjAG4EtRmvCtTYtqX4D5aW+P HNBJVQlZhwv/Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 03/21] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Mon, 2 Sep 2024 15:21:14 +0800 Message-ID: <20240902072136.578720-4-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 9806b1629880..1b49f74fd728 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -243,6 +243,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; =20 +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1054,7 +1059,7 @@ static inline void __zs_cpu_down(struct mapping_area = *area) } =20 static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1071,10 +1076,10 @@ static void *__zs_map_object(struct mapping_area *a= rea, sizes[1] =3D size - sizes[0]; =20 /* copy object to per-cpu buffer */ - addr =3D kmap_atomic(pages[0]); + addr =3D zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr =3D kmap_atomic(pages[1]); + addr =3D zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1082,7 +1087,7 @@ static void *__zs_map_object(struct mapping_area *are= a, } =20 static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1101,10 +1106,10 @@ static void __zs_unmap_object(struct mapping_area *= area, sizes[1] =3D size - sizes[0]; =20 /* copy per-cpu buffer to object */ - addr =3D kmap_atomic(pages[0]); + addr =3D zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr =3D kmap_atomic(pages[1]); + addr =3D zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); =20 @@ -1245,7 +1250,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, pages[1] =3D get_next_page(page); BUG_ON(!pages[1]); =20 - ret =3D __zs_map_object(area, pages, off, class->size); + ret =3D __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret +=3D ZS_HANDLE_SIZE; @@ -1280,7 +1285,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) pages[1] =3D get_next_page(page); BUG_ON(!pages[1]); =20 - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); =20 --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B9781714CC for ; Mon, 2 Sep 2024 07:16:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261407; cv=none; b=ZXjbpOJdWzozwUijAqaZE1icqjsUuQvJfuUn9jozpZzLHMP/bXnKV9A6ncwzroc3RHe2wRPC4GyMctvJadmFJEHTeH6tzFPo2Mu/U3dWGqvrWP6OSIkXNH7p9egTt0IG5y5cgzE/Q0D/RvAXfcP/ubV1eW5k1kN2nrPAxbEa/nU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261407; c=relaxed/simple; bh=XsUVOcBfW2JeyjUAJrM1VbT2++Hcy1Zh8J3Xk4CJ6fs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eT3YdWe9SqFJqloEbogK4IFSTjUlvHtQNEunnsx8rk2/67qeZOmGbkRU/HjaSO+srxubExp0zUe1QuwsoNjG84hsPJy6isS59UsBMjVKhZ0Q23Fjg/iNVuDFVc0ORrOidPLXGkMLDJ2Q86pDC92+CZ+PIrsX/uJipAkmNyovF/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WTSddoPE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WTSddoPE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CFA11C4CEC2; Mon, 2 Sep 2024 07:16:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261406; bh=XsUVOcBfW2JeyjUAJrM1VbT2++Hcy1Zh8J3Xk4CJ6fs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WTSddoPEbD5kSFh8DW5pYD4ONuFe79WI4ETo4U7BQjI1bST4rPU5bcbMIuyNXKMZN bNkFla+us9idV8Q6bk/xzhSxvRVyFkHC1XBCvLLJiz2CDIof7H2AaHdKdiAlLaEokN FBf1IH1mbZtOu4IsLPkCiWx8lU8624B4xKqIwEZ4Rq2nEwz9qzoAjwwHyk+OQ9GX0Q w1KoJNH5pGaKuNymf+b1bHL7jZPtsXyOW1IZ95yYbVEvJW2K0fsiiB4sYRoL8ZKJkN Dluez4d3GOKyB9AfxW54urTAXehe81vQgiYWrnQ+gUOcMQx3OxbTm7WAbQH498x/4V QUbQiC8L55T9w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 04/21] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Mon, 2 Sep 2024 15:21:15 +0800 Message-ID: <20240902072136.578720-5-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 782b5ad67cda..11083a1c2464 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -99,4 +99,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } =20 +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1b49f74fd728..a7e73c0de039 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -767,15 +767,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *= zpdesc) } =20 /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page =3D pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx =3D (obj & OBJ_INDEX_MASK); } =20 @@ -1201,13 +1201,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned = long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; =20 struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; =20 /* @@ -1220,8 +1220,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage =3D get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage =3D get_zspage(zpdesc_page(zpdesc)); =20 /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1240,17 +1240,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned = long handle, area->vm_mm =3D mm; if (off + class->size <=3D PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr =3D kmap_atomic(page); + area->vm_addr =3D zpdesc_kmap_atomic(zpdesc); ret =3D area->vm_addr + off; goto out; } =20 /* this object spans two pages */ - pages[0] =3D page; - pages[1] =3D get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] =3D zpdesc; + zpdescs[1] =3D get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); =20 - ret =3D __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret =3D __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret +=3D ZS_HANDLE_SIZE; @@ -1262,7 +1262,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; =20 @@ -1270,8 +1270,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) struct mapping_area *area; =20 obj =3D handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage =3D get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage =3D get_zspage(zpdesc_page(zpdesc)); class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 @@ -1279,13 +1279,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned= long handle) if (off + class->size <=3D PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; =20 - pages[0] =3D page; - pages[1] =3D get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] =3D zpdesc; + zpdescs[1] =3D get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); =20 - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); =20 @@ -1427,23 +1427,24 @@ static void obj_free(int class_size, unsigned long = obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; =20 - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset =3D offset_in_page(class_size * f_objidx); - zspage =3D get_zspage(f_page); + zspage =3D get_zspage(zpdesc_page(f_zpdesc)); =20 - vaddr =3D kmap_atomic(f_page); + vaddr =3D zpdesc_kmap_atomic(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); =20 /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next =3D get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index =3D 0; + f_zpdesc->next =3D NULL; set_freeobj(zspage, f_objidx); =20 kunmap_atomic(vaddr); @@ -1488,7 +1489,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1497,8 +1498,8 @@ static void zs_object_copy(struct size_class *class, = unsigned long dst, =20 s_size =3D d_size =3D class->size; =20 - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); =20 s_off =3D offset_in_page(class->size * s_objidx); d_off =3D offset_in_page(class->size * d_objidx); @@ -1509,8 +1510,8 @@ static void zs_object_copy(struct size_class *class, = unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size =3D PAGE_SIZE - d_off; =20 - s_addr =3D kmap_atomic(s_page); - d_addr =3D kmap_atomic(d_page); + s_addr =3D zpdesc_kmap_atomic(s_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); =20 while (1) { size =3D min(s_size, d_size); @@ -1535,17 +1536,17 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, if (s_off >=3D PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page =3D get_next_page(s_page); - s_addr =3D kmap_atomic(s_page); - d_addr =3D kmap_atomic(d_page); + s_zpdesc =3D get_next_zpdesc(s_zpdesc); + s_addr =3D zpdesc_kmap_atomic(s_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); s_size =3D class->size - written; s_off =3D 0; } =20 if (d_off >=3D PAGE_SIZE) { kunmap_atomic(d_addr); - d_page =3D get_next_page(d_page); - d_addr =3D kmap_atomic(d_page); + d_zpdesc =3D get_next_zpdesc(d_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); d_size =3D class->size - written; d_off =3D 0; } @@ -1784,7 +1785,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47A74185B50 for ; Mon, 2 Sep 2024 07:16:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261410; cv=none; b=kdrOYlo4ZAFCC9odS/S/zRrruJZYrqUPrlxmETIBGKAP7hZc/rRUvho1Bf8coHzY7IQvrjQjqy43+xTx4MlPek7fL2fjEQeTMRHsKonGZap399/YDIR0Ev8avr79zNeDKMuyc60JA8QFxxByLVGw+++FgjYClumyhWS4bwqPUoY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261410; c=relaxed/simple; bh=8cxk8W9Xo69SGYBFRex8FDtZ09OrBx21iVwdBQgpsTo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hL3F9kQKZmF8ifwgRn8/FWBZnRgoC3urPqZv16iLui+VGSohKowoX5pzUfpqqswAp5jZX3aTfke2vk5yZTHo4k901mM9piqCbf1FH3qatvN0I0v4crlp+rz3Z8/oAjI1HC8iMK4CVuU7QyUChq/ynic3x7go9BzagjwLQzuebHA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=a24F2VQn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="a24F2VQn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 61482C4CEC2; Mon, 2 Sep 2024 07:16:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261410; bh=8cxk8W9Xo69SGYBFRex8FDtZ09OrBx21iVwdBQgpsTo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a24F2VQnLxBeAfFgzkncMF3h1rCsz0+HZE0mPCQdzHFzcAAnrpDMYveicOyyg990A HmRfhw0p7bfJqGm+RryWkBgUWvMhBgTW/uBJbjvf7YLCMIMB0Vhm/l8hsaxn/+mhnw 7WmVG5f14hJSFBysA0DWgujyfNTa86E7yzPq54eu5mJsGTnETUbsPO3OMT5UMdw1Gg cz7Ui+tx7hmuxYk8vVOCa9FxXonF+j/ayOHUXTuIvnRQlVH2Xg9xmlOKZML4wtJLfF zwn28zh8Yv7cNLIzHsX8TjaUmmFlNH8WAh1EUFEf6gINm6luljXGdCM4Tb2JV9Vs6P eYoSK2APZFQcg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 05/21] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Mon, 2 Sep 2024 15:21:16 +0800 Message-ID: <20240902072136.578720-6-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a7e73c0de039..664fff88a7f0 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1315,12 +1315,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; =20 - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; =20 @@ -1328,14 +1328,14 @@ static unsigned long obj_malloc(struct zs_pool *poo= l, obj =3D get_freeobj(zspage); =20 offset =3D obj * class->size; - nr_page =3D offset >> PAGE_SHIFT; + nr_zpdesc =3D offset >> PAGE_SHIFT; m_offset =3D offset_in_page(offset); - m_page =3D get_first_page(zspage); + m_zpdesc =3D get_first_zpdesc(zspage); =20 - for (i =3D 0; i < nr_page; i++) - m_page =3D get_next_page(m_page); + for (i =3D 0; i < nr_zpdesc; i++) + m_zpdesc =3D get_next_zpdesc(m_zpdesc); =20 - vaddr =3D kmap_atomic(m_page); + vaddr =3D zpdesc_kmap_atomic(m_zpdesc); link =3D (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1348,7 +1348,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(m_page, obj); + obj =3D location_to_obj(zpdesc_page(m_zpdesc), obj); record_obj(handle, obj); =20 return obj; --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32537176ADF for ; Mon, 2 Sep 2024 07:16:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261414; cv=none; b=dicz4E7/ixiz9dotQRLKGqIZScd08+5781DHdxBTK2bedGexTbFRPoZUeMYzFWVbwYs2CggBpRsLreYtnDaSB/zun2OTkY7vxe0HOaNKtzlyDljmYbPNwOSL8Wr1DUuimr1dNBUVi2O9xowZRNziBbYyp2eAbFIVNMPB1vzHtCI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261414; c=relaxed/simple; bh=LtPxxmWpxhvwgd6evBIlBSmgc0+AxTDG0ppcncG0AUE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YwNiMKRiNlF40LuuupLRqHuYnC7Pau9bFqw15hbAMitxO//ahmrFo9bjR4N3MEPjA2NRqVvVBjWFhy7XTNHm3zF2UgMGq0Gvidt6087b2tNixkk0xMmY1qc4yIWP5+PYvcGTNtmZ1flDxFM3SXo4iHQSMwYtn66x0kMbnUx9fYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oeTHe/G/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oeTHe/G/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C2D43C4CECA; Mon, 2 Sep 2024 07:16:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261413; bh=LtPxxmWpxhvwgd6evBIlBSmgc0+AxTDG0ppcncG0AUE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oeTHe/G/ry/99t6lVEMFyFRpCzzGHE4tmMQdoPie6y3EhRhkY0ZQ9axzBbdD0XDjP DusqDDUtP11Vi9AmxWicp9ESzmFmMArZFk5ta8D+vj+PcVoHoTOvclom6F9CNE2Q7x NqwNMZzRt9JsUw5dEEDcysjJhNRyxXPTvBFenOH81yRxmGnvRGU2LF/3KPmDe3Wtte Bep//k78yEXeZGCB12dLrYyMyvMsmeuk3xdkjYyg60AhADg1syXKU9jHxJGRmkA+Ko TLgnEDcV+YakWCfs/M4A6tOi/arCYKTRpuW890jGwlxCqTnDoM78WCg/un3uzb//Yk W3QSnOsWnKmcA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 06/21] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Mon, 2 Sep 2024 15:21:17 +0800 Message-ID: <20240902072136.578720-7-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chai= n() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 +++ mm/zsmalloc.c | 109 ++++++++++++++++++++++++++++++++------------------ 2 files changed, 76 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 11083a1c2464..3a65a7d494b7 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -108,4 +108,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long = pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 664fff88a7f0..6afc3e642e02 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -248,6 +248,35 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *= zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } =20 +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline struct zpdesc *alloc_zpdesc(gfp_t gfp) +{ + struct page *page =3D alloc_page(gfp); + + return page_zpdesc(page); +} + +static inline void free_zpdesc(struct zpdesc *zpdesc) +{ + struct page *page =3D zpdesc_page(zpdesc); + + __free_page(page); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -947,35 +976,35 @@ static void init_zspage(struct size_class *class, str= uct zspage *zspage) } =20 static void create_page_chain(struct size_class *class, struct zspage *zsp= age, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page =3D NULL; - int nr_pages =3D class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc =3D NULL; + int nr_zpdescs =3D class->pages_per_zspage; =20 /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i =3D 0; i < nr_pages; i++) { - page =3D pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index =3D 0; + for (i =3D 0; i < nr_zpdescs; i++) { + zpdesc =3D zpdescs[i]; + zpdesc->zspage =3D zspage; + zpdesc->next =3D NULL; if (i =3D=3D 0) { - zspage->first_zpdesc =3D page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc =3D zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage =3D=3D 1 && class->pages_per_zspage =3D=3D 1)) SetZsHugePage(zspage); } else { - prev_page->index =3D (unsigned long)page; + prev_zpdesc->next =3D zpdesc; } - prev_page =3D page; + prev_zpdesc =3D zpdesc; } } =20 @@ -987,7 +1016,7 @@ static struct zspage *alloc_zspage(struct zs_pool *poo= l, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage =3D cache_alloc_zspage(pool, gfp); =20 if (!zspage) @@ -997,25 +1026,25 @@ static struct zspage *alloc_zspage(struct zs_pool *p= ool, migrate_lock_init(zspage); =20 for (i =3D 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; =20 - page =3D alloc_page(gfp); - if (!page) { + zpdesc =3D alloc_zpdesc(gfp); + if (!zpdesc) { while (--i >=3D 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); =20 - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] =3D page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] =3D zpdesc; } =20 - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool =3D pool; zspage->class =3D class->index; @@ -1746,26 +1775,28 @@ static void migrate_write_unlock(struct zspage *zsp= age) static const struct movable_operations zsmalloc_mops; =20 static void replace_sub_page(struct size_class *class, struct zspage *zspa= ge, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] =3D {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] =3D {NULL, }; + unsigned int first_obj_offset; int idx =3D 0; =20 - page =3D get_first_page(zspage); + zpdesc =3D get_first_zpdesc(zspage); do { - if (page =3D=3D oldpage) - pages[idx] =3D newpage; + if (zpdesc =3D=3D oldzpdesc) + zpdescs[idx] =3D newzpdesc; else - pages[idx] =3D page; + zpdescs[idx] =3D zpdesc; idx++; - } while ((page =3D get_next_page(page)) !=3D NULL); + } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset =3D get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index =3D oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle =3D oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } =20 static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1838,7 +1869,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, } kunmap_atomic(s_addr); =20 - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90DA6185B7B for ; Mon, 2 Sep 2024 07:16:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261417; cv=none; b=Yo8AKddlPIDQnsS9322aoc+7rCTm4kYkZDKrqYF4NWOIL6hp9JL4VF2YXxs7sHDNs10TP1yidELf7mAxxiIv/N25qLrUwAUHZ0IdPmE2DlYL/sX6LqCvr1Fas+pssJ05CwJzQ4a3HQ4M9XhjDCrZ3yTsnsi11gkHJl9iWQ6iWkQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261417; c=relaxed/simple; bh=Br29/5+PsdLeZ/2tyuIzR5WeYQvl1vCMxw0jQ4y7I0c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EvlszZXmkdNVymcEQ+nQKcw16H15l3UguM9B2UJZZ6M8f4IXJ1DvLRzMB+90mXuPnJ9y4BQ1nSllclRCOpKgM5EFgI7Xjr4y+sFk0Q0lRkQABr+ECrpRnqxJ8QfVy0qnfjNRbwkLixXyyP+OdcddzTQy/awaszkf12+S8qsKwnA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CAfKDYNX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CAfKDYNX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 533AAC4CEC8; Mon, 2 Sep 2024 07:16:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261417; bh=Br29/5+PsdLeZ/2tyuIzR5WeYQvl1vCMxw0jQ4y7I0c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CAfKDYNXjkGUMjjB3LlqQwolWE5LfDVmqwftcV6wvGJkYybv1hqip4//3RUWCXBE0 YopBu9YKpW1ldrbR0e9ShcXXpwmaI6gIc/4ttV2oKrteRG4M1HKS65E20+U+R3o7yM OAltuYv3c4EPwZjcbEtD7NJKJ3LGTe3YNRi3Oa5DHNCXVhHT29BWWzkUcwAuyPHaUG t/uhkkC3MUTgkk3pxWxY+yC6uFQ5ms5l1hG+8G7ElOlF8KBnl1Qga5dNieECRmUhUk 4jwlkCj/prBwfMUZYq9Ht59LMc5RBEF9mj17eEC2zhUSdTCdi4syzKXAvjIeSQEr+b d/CkesoRyrHHg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 07/21] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Mon, 2 Sep 2024 15:21:18 +0800 Message-ID: <20240902072136.578720-8-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6afc3e642e02..46d8be5cb4b7 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -833,15 +833,15 @@ static unsigned long handle_to_obj(unsigned long hand= le) return *(unsigned long *)handle; } =20 -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage =3D get_zspage(page); + struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); =20 if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle =3D page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle =3D zpdesc->handle; } else handle =3D *(unsigned long *)obj; =20 @@ -1590,18 +1590,18 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index =3D *obj_idx; unsigned long handle =3D 0; - void *addr =3D kmap_atomic(page); + void *addr =3D zpdesc_kmap_atomic(zpdesc); =20 - offset =3D get_first_obj_offset(page); + offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; =20 offset +=3D class->size; @@ -1625,7 +1625,7 @@ static void migrate_zspage(struct zs_pool *pool, stru= ct zspage *src_zspage, struct size_class *class =3D pool->size_class[src_zspage->class]; =20 while (1) { - handle =3D find_alloced_obj(class, s_page, &obj_idx); + handle =3D find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page =3D get_next_page(s_page); if (!s_page) @@ -1858,7 +1858,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 for (addr =3D s_addr + offset; addr < s_addr + PAGE_SIZE; addr +=3D class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6A5F185B7B for ; Mon, 2 Sep 2024 07:17:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261420; cv=none; b=Ppdr7O46LtTcJFcQL8Fvv9QV4C3dBb5GTxsdj/IjqfYevtpGRrtVstmvgCGw7PLwNfLB6B/2JEQxDRnTGzwlHS1uI6JkHC/9N6aAXtlaR8f3+6bz7MAD68cmb/vphYlz0Y/vPO7InlU3ZNI58+NpZX9r0D86amk7pXZuQxNio+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261420; c=relaxed/simple; bh=PNe+PCWyOI4z40ocTHR27DhffZO/+69yc298dwmlSes=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tlO3g3RwFrfX9KZmxlIhOJCGHfIQCiIceCeOB+NWskgszd10axW5TlAINnhwZGZWqja6EMUJhyjKJbgyxmQYBGEnMLgFiB1moloxJuI9dTIOEMZX8JxCxvNUTioqCZKTATmUcLXHkrMoa749Nf2YG3WY2M08fpvXlXWX9zOnU3k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Tjz9s9qe; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Tjz9s9qe" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3069C4CEC9; Mon, 2 Sep 2024 07:16:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261420; bh=PNe+PCWyOI4z40ocTHR27DhffZO/+69yc298dwmlSes=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Tjz9s9qeCkxm3K4mcUE5debLCDqO25YRhRFWJ/avcEDhpY4b8AFlwwm6cq1qeRVSM TGzZdBTjO0kyKpbF/Z7DIxsvEizBwlGCw9sJadLQCvWyAzoTTek19qzxsiyEkUhEak u0KMdnQ6O+BTfJOhcIz2D04qnh0DCeQ4IOyceUrOokKwmSLO7zYm58lT2aGv38l27n ISkuZaS3xI44QuXK7+MXr0ymkpKO43OncUVwMQJoGL/cZpb3Er7s5xJojvxNtbKHEN vMl9o5p/VP+DNo/4sqQvC5TySyWuu0k5MImEY96NjGNU+enX0pdUQJ91RPuVGMeLfg dviBzU7NaaG7w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 08/21] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Mon, 2 Sep 2024 15:21:19 +0800 Message-ID: <20240902072136.578720-9-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 46d8be5cb4b7..01c5be6f4c96 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -935,16 +935,16 @@ static void init_zspage(struct size_class *class, str= uct zspage *zspage) { unsigned int freeobj =3D 1; unsigned long off =3D 0; - struct page *page =3D get_first_page(zspage); + struct zpdesc *zpdesc =3D get_first_zpdesc(zspage); =20 - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; =20 - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); =20 - vaddr =3D kmap_atomic(page); + vaddr =3D zpdesc_kmap_atomic(zpdesc); link =3D (struct link_free *)vaddr + off / sizeof(*link); =20 while ((off +=3D class->size) < PAGE_SIZE) { @@ -957,8 +957,8 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page =3D get_next_page(page); - if (next_page) { + next_zpdesc =3D get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next =3D freeobj++ << OBJ_TAG_BITS; } else { /* @@ -968,7 +968,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) link->next =3D -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page =3D next_page; + zpdesc =3D next_zpdesc; off %=3D PAGE_SIZE; } =20 --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A490D185B7B for ; Mon, 2 Sep 2024 07:17:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261424; cv=none; b=Hcr08RDjI03m9aKBnoNHMlPzErxulbXENCNXHWpCJQvVIaPGpRnGTqns3JkrF+56SNZtf2Mam5iyvparUiAhpYKz0Ck7l2iUD0d7IthhE5Sngwg+KKujoTTVG4ZxmkDJlVkAxNJuxAxeLbJ6bgXIJyy70Av3SH90Evdjf2qMXXE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261424; c=relaxed/simple; bh=sa8ddOMDXRaYhp/IOg2wSEnMgXawFHsto5brOpBu4jE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dg8eJuMONhczNToyZaK/Ah7zGZylbOTHor9RfwlU/cazi8Z+lb5pE8l/KhKLUeXxLbZjiFqZgFXldFapJopULkllDkLObeAZnrE6nOg1FcFqnCnLGQhBAe+3F7h6mKIhmzG4sW/TfQBZcoeLRvFmZ1o18s5GVWBu2EEKX7nQDn0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WbsWZzPz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WbsWZzPz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 422A2C4CEC8; Mon, 2 Sep 2024 07:17:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261424; bh=sa8ddOMDXRaYhp/IOg2wSEnMgXawFHsto5brOpBu4jE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WbsWZzPzOYnq9VACFjLbHT0QtTboQMrs9Cl5BnEg42AvkulcCszPJZC4ZgWJ7VBIs aa6jHlKKy7UdoTM3zYo5H5sY1gxJVq8u6gEGeIGVjHyKyvlMx7Jw+fs2WXoRrqOR6k B3lHLvwe6ydPue5clUvlCNMVXV1PbfHaTfNFXvS5C78imrXR4D+wTnakGT8ZwdzMak H1fg402rc99rP4R//IJ5X02R3KMzo1XrNoEQMHSIEhTNiwT7e4352mPk+9CfuKWovi EMzxRlUou3vQ/ZS/xQiscEZrpsDV8+bThb50Mn1FvN2/rNaA8KQJEsw34L71nnOBAY L0XGUwN8m4lWw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 09/21] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Mon, 2 Sep 2024 15:21:20 +0800 Message-ID: <20240902072136.578720-10-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 01c5be6f4c96..90d93cdc5df6 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -808,9 +808,9 @@ static void obj_to_location(unsigned long obj, struct z= pdesc **zpdesc, *obj_idx =3D (obj & OBJ_INDEX_MASK); } =20 -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page =3D pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); } =20 /** @@ -1483,7 +1483,7 @@ static void obj_free(int class_size, unsigned long ob= j) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1497,8 +1497,8 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) */ read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage =3D get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage =3D get_zspage(zpdesc_page(f_zpdesc)); class =3D zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E572186E52 for ; Mon, 2 Sep 2024 07:17:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261428; cv=none; b=Jg0wBQvJ9frBWnUpdzMvOim790nZFsz5U4DJpjzlMEpbsPQnTgIreuz+LggANROUC8j9RUMDE87aglKjZgeVbh5Eu8SYkMFWPXBNn2oRrdY/y73B150OAI4Y43aTV6/fohpFgVXDJVFvnemsufUfODnSoWc8BGayQYBS8UksezM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261428; c=relaxed/simple; bh=oCzUVJExiVdlJN3ac9k05YJBYpT4iWle/rruJLC4yF4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B34VKwYwMaxH2zifaWraK8CJa2ko9f7YWbNCnG/Vheh0xhqivFB0oKhY4/c5nYQLdA15OxBbhoqBYsnZpsAYhs86vBehu23JYlk6GzweF3J4PiOnMKj/5+whTIdl3AYXPo2HHT63ekI6yv6nGt8j+KrbL5FpyzwBexlxEn5tmwA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rkUy6HuW; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rkUy6HuW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8A34C4CEC8; Mon, 2 Sep 2024 07:17:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261427; bh=oCzUVJExiVdlJN3ac9k05YJBYpT4iWle/rruJLC4yF4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rkUy6HuWPoGU4zFzaR50FOgnKCKF9Tv5JOIFvN6EGC+Wo0ag94CXHRxJhF/CSJ7Qq cDZxI3rBPwkmIZ4vAujby84oFn+WBybmfq1QQgNg+9K/3edwYfXgwyuzm6jDI1bVyQ y3m4XjueWEln4Vb7BgU/ODrYz9R0Sfj9cnTbhQaG8sPmEE/xrMSmTsdr+lb1yds84+ eGBorMA2OYL0Idrh4/9Eol5VGZ1XidpF+qyqo4xROKcLJtzGYV0iU40pCRuQQ7OenL gSHTH5L8jd9BzD/QVi97ByljTvtkpMyZLlNKKpOfgytuIfhVQ0LiRbgPi/doqJx79o RJnhD2a/uTqEA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 10/21] mm/zsmalloc: add zpdesc_is_isolated()/zpdesc_zone() helper for zs_page_migrate() Date: Mon, 2 Sep 2024 15:21:21 +0800 Message-ID: <20240902072136.578720-11-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated()/zpdesc_zone() helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 3a65a7d494b7..4b42d8517fcb 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -114,4 +114,15 @@ static inline void __zpdesc_set_movable(struct zpdesc = *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 90d93cdc5df6..caa4a7883ab8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1817,19 +1817,21 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc =3D page_zpdesc(newpage); + struct zpdesc *zpdesc =3D page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; =20 - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); =20 /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); =20 /* The page is locked, so this pointer must remain valid */ - zspage =3D get_zspage(page); + zspage =3D get_zspage(zpdesc_page(zpdesc)); pool =3D zspage->pool; =20 /* @@ -1846,30 +1848,30 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); =20 - offset =3D get_first_obj_offset(page); - s_addr =3D kmap_atomic(page); + offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr =3D zpdesc_kmap_atomic(zpdesc); =20 /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr =3D kmap_atomic(newpage); + d_addr =3D zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); =20 for (addr =3D s_addr + offset; addr < s_addr + PAGE_SIZE; addr +=3D class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(newpage, + new_obj =3D (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); =20 - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1878,14 +1880,14 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); =20 - get_page(newpage); - if (page_zone(newpage) !=3D page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) !=3D zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } =20 reset_page(page); - put_page(page); + zpdesc_put(zpdesc); =20 return MIGRATEPAGE_SUCCESS; } --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3FD04187326 for ; Mon, 2 Sep 2024 07:17:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261431; cv=none; b=k/7sKZMs2INqMbKu9r3y3+OiT9VzoDIQ1AbMPBG8VZ+eQyPDAM4a40WuuqaHYHW08dpSUNLNLasv5fHWl3MiCYwQqJRJWsLPUVcgCtOCwddgXwoHa+ZeHqKFyZsKsG+YTnrML97nDmWtFP1zeBesxhaBWj9CZ8gBc2zu/pLRFLo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261431; c=relaxed/simple; bh=TqaE/5WJpT1xBdjh2dqdooKapipE1l2P0P7kGVoluZA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SP7UHn5iYXn9iW5ORb6jj0ljFv/2u52QqO8OrReVjInFVwTMhkendhKJPx1rrPkLPI+Dqtxq+vkMG/ojvJLrNnciYhpm3D59xDv+f9+E7DUwEpop8vt1x6D/AtdL8fH+j6SEBcOXX3tjNFY1Zse5BM1YEdx7XAIhLx5T7LEWdKE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=tINHAOSK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="tINHAOSK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 59271C4CEC2; Mon, 2 Sep 2024 07:17:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261431; bh=TqaE/5WJpT1xBdjh2dqdooKapipE1l2P0P7kGVoluZA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tINHAOSKO69ja64wk58zkCcRcnjShQs6ujw7E9y0QqjTQ2Y9taiVSVw1xtraDIUOF NzJ4/jDvLlYoO4zZIiSJqmsszuMAbnuyMTsOCbm3B+BJIsl+16BsZ+omQXqmXSglnm v0yWrwoutoQuD4ye6gqAqFgIJmwaWmBXfx+V26fOfvQcdN6QCRaB8Tubehki32vm70 NbcjUFnlub6UuwvekHSYnnKakVcPvNyKfOZ9EjH2oC/AFPoZAGVtaVXAdiHExx3WCi rQciGFPkmq1gCe+us9jvCXv6Fm8P71mQUUJ1oqY7Z2FfKKB2hmIgeUX36hKv70OFoS aL3Y2UJ4+oxiQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 11/21] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Mon, 2 Sep 2024 15:21:22 +0800 Message-ID: <20240902072136.578720-12-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() which is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index caa4a7883ab8..b03bd7e07e98 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -853,12 +853,14 @@ static inline bool obj_allocated(struct zpdesc *zpdes= c, void *obj, return true; } =20 -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page =3D zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index =3D 0; + zpdesc->zspage =3D NULL; + zpdesc->next =3D NULL; __ClearPageZsmalloc(page); } =20 @@ -897,7 +899,7 @@ static void __free_zspage(struct zs_pool *pool, struct = size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next =3D get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1886,7 +1888,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, zpdesc_inc_zone_page_state(newzpdesc); } =20 - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); =20 return MIGRATEPAGE_SUCCESS; --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2730117D378 for ; Mon, 2 Sep 2024 07:17:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261435; cv=none; b=R03vlyWXqI2GLaEDz7higFlfVbcP6fQMlhdNMQGWKPPUQVvysgYFs94sS3SBs+a9ikjlmXAQxMJueHFEkRmxN1Y0cnGoaKa4IDX0msuT3xm/4xldalRzVapNB4LYo+rTyHHfBpaQcJ/s0/T3wyfmR6T511mSfTgzYiNeaP0CrvQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261435; c=relaxed/simple; bh=V2EFOKt+6az/uMmwc5vttTlHqgkkiN0OISIMWGbt/wQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TXsQesSMWxJ6vtU6JsqrcMDIqHISG4IzDM0b0Pzds30b2AubV3lDBMhVvE1WJnbdyZJKjBNAI2mGkNhywwFetHvtf/86Q7VxciSUXAnS+ushiqH1DIIaAPSeQYZsl/xgWVLELPlotX3J1dg5voq4mq9XEO/bhpA1+aA6/vUGnqI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QibvnDvI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QibvnDvI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BA1A4C4CEC4; Mon, 2 Sep 2024 07:17:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261434; bh=V2EFOKt+6az/uMmwc5vttTlHqgkkiN0OISIMWGbt/wQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QibvnDvIwyvgilFEHEmKNY6ncRz9JForNYuHKUV/SasiGH8CLsNIj6aePH+ZrSs3z hUios/48n2louj3GjCRjC0Oawhx0jRpwnhG35lr8yyZApFk47hBbbHO6FkP2hMeGcl VnE8LOmNM1GyTTtZngN4I4ZEWpAG4b0JKq1MykngZEfVoGoJ8ucTz62ynqEVofLygl vd4Y/uW/yblxNd5EQIvkSlGfKW6yOalNJd/t4sSqw34G5Hamy4yimVdGIf54eP5o2e 4jkfeedFF91ZcBTI1Fn/R6LQhRNayDr4GqPp11hZJBPM/uFsHukJ3ZsF1e9RxkbWv3 vmzcC3wTHfggg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 12/21] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Mon, 2 Sep 2024 15:21:23 +0800 Message-ID: <20240902072136.578720-13-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 4b42d8517fcb..a1834d36ccfc 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -125,4 +125,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *z= pdesc) return page_zone(zpdesc_page(zpdesc)); } =20 +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index b03bd7e07e98..5a8fd5fbc962 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -888,23 +888,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; =20 assert_spin_locked(&class->lock); =20 VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness !=3D ZS_INUSE_RATIO_0); =20 - next =3D page =3D get_first_page(zspage); + next =3D zpdesc =3D get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next =3D get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page =3D next; - } while (page !=3D NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next =3D get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc =3D next; + } while (zpdesc !=3D NULL); =20 cache_free_zspage(pool, zspage); =20 --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3697B187552 for ; Mon, 2 Sep 2024 07:17:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261438; cv=none; b=CK9xjgsFN9/7pCJnZeL6dl3iDJciptBn5kUqvNo2/CcuBAco4UIq8tam0fzqrpwiBeOGzBxiKHSeiBS02sr0FyvBRnQYEcMKjbizSDhqjvjSmHI807TI1lOHHttfeuPMJ/2yluKToseDioaAF8GrR5rOqtAEuObB8Gf5s2ziDgY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261438; c=relaxed/simple; bh=XMY5jg562sPF99BloXKZJR4HKiDGqicXgdq+2ws9Iy0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VQD+xxmUDoXuxuQANOMP+27YcQff8VP3a32Vw2hmEjsbTnLPTMs1JHd6OHKXEdnr8VF5vE2PG4H+/Hl3rUIrUxBe2o44/jMWnJTT3z9rosl6i3oYsExRMSl2FtKjy3fmVY1rY6HEnniVAl/Ee/UXSBpD1Ml8J0YLjGkgzXnOudk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NPeK2NyV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NPeK2NyV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B362C4CEC4; Mon, 2 Sep 2024 07:17:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261438; bh=XMY5jg562sPF99BloXKZJR4HKiDGqicXgdq+2ws9Iy0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NPeK2NyVAp/tWckPu9912WQybh4F4gsYJTs+6ho+SST7R9QNtr+Wb0XlHgJI0twbc 3bINCAfA5kKpMF2Q7PrN+D2xXHoHSQd9M2nsfbhEmvC/kMGUJtZQ40YI+NCbthEAvB 0XPZVe6lyFovwfqWR2rNwDjMaRrcBQisjgCtJWR3reZwYuJFQnCsYbgAbxvefF059f UZ/3lSpf7g0T4mVBjpcJUPdNbwoy2hnea6sYvK8VwRlZ8WNWuyv4oGjYdw/iHVS6Z9 2kAImmM3/NlKqL3WG288/KRhAfKLnwJPAbm9U7O9TIdjYTmFD4EvUA93+Y3WuOibN1 KTCCtWBhvfP2w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 13/21] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Mon, 2 Sep 2024 15:21:24 +0800 Message-ID: <20240902072136.578720-14-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5a8fd5fbc962..f94c3a6653c7 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -814,15 +814,15 @@ static void obj_to_zpdesc(unsigned long obj, struct z= pdesc **zpdesc) } =20 /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_i= dx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx) { unsigned long obj; =20 - obj =3D page_to_pfn(page) << OBJ_INDEX_BITS; + obj =3D zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |=3D obj_idx & OBJ_INDEX_MASK; =20 return obj; @@ -1379,7 +1379,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(zpdesc_page(m_zpdesc), obj); + obj =3D location_to_obj(m_zpdesc, obj); record_obj(handle, obj); =20 return obj; @@ -1866,8 +1866,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj =3D (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B368515FA92 for ; Mon, 2 Sep 2024 07:17:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261441; cv=none; b=D8idqp1q5RYglP364lNqIPiTBQ9Q0oSlVImMScP2+0PekVL4ENGBW8yRe9Q3laX0vyWnAkgwFJYMmo6x+4dbLRwNRHOtSqsKBLgsg9SGUtzShYk/eq7HCjVqsPOGw2ZWhwIEVEdDf9RPbB53FcrJDAvQBlIwirQupqDsjhZjURE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261441; c=relaxed/simple; bh=oMrU85lTLRKqEwjZvX+/RAn/IsBRgGpZpbsgEGeKZYk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SEXxDNVyW6kW30Ci3YT4hC4+i3w8awSB0zRNzOzxiJ4ho9gnkpmloXBXQMFAcF4yLbJm6JiTUPQ3+83waKZ3KFu2n8FAysF8pbi5wPQfoqBfYcmXj100bN4oUjSXdoBVd2JI+zL7aSj7UeivnX8j1rJLbUTQzwtt0H5RXvRLcEQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uTv2JYWK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uTv2JYWK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id ACE76C4CEC2; Mon, 2 Sep 2024 07:17:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261441; bh=oMrU85lTLRKqEwjZvX+/RAn/IsBRgGpZpbsgEGeKZYk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uTv2JYWKBWs/eY3vQwV4b7d46Dw+8FnfttVYa627PvUXCjgj8YFZiXNE2cONgQ2vD JkgfML6hEagHjvBeo08sMYm5L9jLkSjtkTdRS7T2LkNHQcFsDx2bVCHMuvp3eznYve SC/wsNxQ0jT08WxCgk5l1VnUbaEx4JcQxRDMcaxevO06zXMJubQ+lKPSu3nErUeF92 QQGh6bVLa6es1W1dR3HomaK6ZvUGMjggVhm7u6kO/swf/AcMtavkx6lsFCqJXQU6Mi 1Vo67rxmdi0vb2Ei+Mo06rE6WDhBAFTKNqdPX1iPelhAwfZsjP4PMzXFTcnadVGpX5 LyF4L2q+hsbNQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 14/21] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Mon, 2 Sep 2024 15:21:25 +0800 Message-ID: <20240902072136.578720-15-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index f94c3a6653c7..aeb8cab5767b 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1623,14 +1623,14 @@ static void migrate_zspage(struct zs_pool *pool, st= ruct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx =3D 0; - struct page *s_page =3D get_first_page(src_zspage); + struct zpdesc *s_zpdesc =3D get_first_zpdesc(src_zspage); struct size_class *class =3D pool->size_class[src_zspage->class]; =20 while (1) { - handle =3D find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle =3D find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page =3D get_next_page(s_page); - if (!s_page) + s_zpdesc =3D get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx =3D 0; continue; --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21C0B15FA92 for ; Mon, 2 Sep 2024 07:17:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261445; cv=none; b=b+7IzR09VyGJfzUNO+tJveLijJkLxc/0NB19S58G5roxiiLO1MEoYM5BJDKaieh3q0kmNxO/vYGYAWSBIykm2Nz3YvJp0MXiZrC/PFLqedM5HjGDCWT/Hw0TeiFGyOD6S5U80kU9xoZpk9XN9v5CjQIKRcO2C72naLYPyhQTbKA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261445; c=relaxed/simple; bh=5d5O3WmlEwe4HKiK+a7YzCdtF3HLgTzqKDcXKeTJtNE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PvbURFQuvZtb6Y+6fmgIqe9tkO/0f6Gb5kYscuViq0/IUkvtt8U/SKFJAXZSQj4ZJtIgBeubLioqjvbfl1hpGRPY8TfIR2QNu+p7irEyzq3c5GUzYZFc/dlNsQLXiNQiSOJ9rYDJ0GvCJEgE8sRVIZHKo7dpllVS0r8xCFcDLxo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GXjsatA/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GXjsatA/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3AC57C4CEC9; Mon, 2 Sep 2024 07:17:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261445; bh=5d5O3WmlEwe4HKiK+a7YzCdtF3HLgTzqKDcXKeTJtNE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GXjsatA/b+bhLB5tHwCAiwV9pwnZATA6bAxMquy1hdL0LTxEf+G8D8b14lHW4PpbA YCLIrtwj9QXxUugNM0EKIeD4C5K/78gOBane3bfpBjUBmWXw40jJbTHEZGGyShhWtE a+OLLpl14bHKuvWxt+6wkxNlOTZg0oeM/kzbgkP2bXfa5YfwkdT0bHCLWPzBBXBTYM bY9oCR7Z8pWjz7hOpbswP+6qh7eAVoHu9J7os3pkqWqR8j2vllzd0uD0RFTmr6iLW6 3TAImOXHbZFvxvvi0yyL8lHiQSwH7gMFC9Nx50eEFSXu2sFl51dpXJ+o5Ba5b5mgh4 PRgEY1VF82sGQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 15/21] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Mon, 2 Sep 2024 15:21:26 +0800 Message-ID: <20240902072136.578720-16-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index aeb8cab5767b..d9251450ec60 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -767,9 +767,9 @@ static int fix_fullness_group(struct size_class *class,= struct zspage *zspage) return newfg; } =20 -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage =3D (struct zspage *)page_private(page); + struct zspage *zspage =3D zpdesc->zspage; =20 BUG_ON(zspage->magic !=3D ZSPAGE_MAGIC); return zspage; @@ -777,7 +777,7 @@ static struct zspage *get_zspage(struct page *page) =20 static struct page *get_next_page(struct page *page) { - struct zspage *zspage =3D get_zspage(page); + struct zspage *zspage =3D get_zspage(page_zpdesc(page)); =20 if (unlikely(ZsHugePage(zspage))) return NULL; @@ -787,7 +787,7 @@ static struct page *get_next_page(struct page *page) =20 static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage =3D get_zspage(zpdesc); =20 if (unlikely(ZsHugePage(zspage))) return NULL; @@ -837,7 +837,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc,= void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage =3D get_zspage(zpdesc); =20 if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1252,7 +1252,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); =20 /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1302,7 +1302,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) =20 obj =3D handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 @@ -1466,7 +1466,7 @@ static void obj_free(int class_size, unsigned long ob= j) =20 obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset =3D offset_in_page(class_size * f_objidx); - zspage =3D get_zspage(zpdesc_page(f_zpdesc)); + zspage =3D get_zspage(f_zpdesc); =20 vaddr =3D zpdesc_kmap_atomic(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); @@ -1500,7 +1500,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage =3D get_zspage(zpdesc_page(f_zpdesc)); + zspage =3D get_zspage(f_zpdesc); class =3D zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1833,7 +1833,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); =20 /* The page is locked, so this pointer must remain valid */ - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); pool =3D zspage->pool; =20 /* --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF02715FA92 for ; Mon, 2 Sep 2024 07:17:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261449; cv=none; b=n+hWSlsobLG5V+k/giI4NTa1MOwwEOUvU+RDDRfLgi315/THfRxb8jMU9c0+4IN/E5vfsnKoX2nO2QPkl9J1oOh3xjpZv+MNIHLABEISzCnU6bCwzbS089b0BcbeTUFX2aXZv60EbpWkWGPsR/5rekovfF7YkXy9/iqglC+C5OE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261449; c=relaxed/simple; bh=JuDze2waT5auO9Z7Ml30M0i2Ht/5hJoDWbD0pOFWjSk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VwjYlKiRU35iYNDO/mAbY5FYa6aGIz+V+MzMcVBKq+L6RI5cg8PD2f/vtDsBfscMjEa1Re+X8lnL6qsqco9cTz2PUHVvJuABqTF66NUp7yTjQntAlhkz+OSecQyihkP1l4m+BlbpmDBsZWc63OooGVwkAiQKr/8dV40jmRCtwoA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ERPokKEV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ERPokKEV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 99803C4CEC4; Mon, 2 Sep 2024 07:17:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261448; bh=JuDze2waT5auO9Z7Ml30M0i2Ht/5hJoDWbD0pOFWjSk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ERPokKEVB7btxb1uvOqV12Md3j6Hiln/4Oe6BMGQ0KkLLvi6CvWIloMINQAnte7U7 elqpjG8Xnkla09nh2xTF0/JOW/ym6N4MKk36KVw3BiAIRpJZ9pcJMC3UsMGtOdlnqI if720f3F4KWiQWy84ic+ZksiAvMwBi1dSI/sAKv7FheYygvZZ+iDYNfsHbex8/h9qJ y/OXCEYiHNDN6tbacLZFXUhsNGNqvdN/w6Y82L/yLEqJP6G69cqslsWgZsIh1UUNOB a8U1Xqnj4M60MqOk0doSkUH2avB5I+3G/EckGtiFPlz4Bh+ZpGMsQK7m4iw2GeKdmj 6MeoSLhIzMHxA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 16/21] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Mon, 2 Sep 2024 15:21:27 +0800 Message-ID: <20240902072136.578720-17-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs: get_next_page()/get_first_page()/is_first_page(). Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d9251450ec60..3b98daa63ece 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -462,11 +462,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area= ) =3D { .lock =3D INIT_LOCAL_LOCK(lock), }; =20 -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static inline bool is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -483,14 +478,6 @@ static inline void mod_zspage_inuse(struct zspage *zsp= age, int val) zspage->inuse +=3D val; } =20 -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page =3D zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc =3D zspage->first_zpdesc; @@ -775,16 +762,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } =20 -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage =3D get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage =3D get_zspage(zpdesc); @@ -1957,13 +1934,13 @@ static void init_deferred_free(struct zs_pool *pool) =20 static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page =3D get_first_page(zspage); + struct zpdesc *zpdesc =3D get_first_zpdesc(zspage); =20 do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page =3D get_next_page(page)) !=3D NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36CA715FA92 for ; Mon, 2 Sep 2024 07:17:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261452; cv=none; b=gpowSe6ISp7ghqKtmR1Pn0WqbE941plVXeXUsbVvihfvuV1gpxwZp6bnbAea+hz605ag6DXIQcA5AAupM9CgzpCNz/6jjJrk0YKX0dDdJR6BD7zRJJx4s9Ea/yBoqIZzPJRM3XTLOXX2Vt/nf1b1zTn2yu9DRYM6cAdj31YAmDY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261452; c=relaxed/simple; bh=mUt+2jl57a1O3x/5Q9wSnZvk6+ZzUwzDrkQBYoscUAE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rFNltA5HOJnGAD980hjSj05u5e9qoSYbJlNzrLO8tiMYJ2/wilw1R1LgOJ6ky4Z5J2P+UJPSoRvOaN8jlv2OdgSCC4tIAtNAqTMoAS11yn0ZikRkhqGQw3fRo0NKszGTlNMpJrWIADAGyj6yRx7AzpiENZI+biB/2uyfOUItPTk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gQujTYAA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gQujTYAA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A176C4CEC4; Mon, 2 Sep 2024 07:17:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261452; bh=mUt+2jl57a1O3x/5Q9wSnZvk6+ZzUwzDrkQBYoscUAE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gQujTYAAUF38D+/6f3rqz71xwGDllZgvbzpatRIu87HIi30swSwSE/K0ycx4vmIfl oqE0HY3KHgIE50k9iKI7PrQXI8TUzZPYgpAjpYabGLaVvtX4eEVL8YXBVMy0OBYb5x DpNOctRFE/8cBs7OB5UGid0JG7sHC1jSAOTOdVaee2LWSE6ozbnSHn+RgjVD2KI3fo KrMqJoCwDNOUe+Uz+r2k/5fJzP2DdEmhbb2/VhBxuab4pmB5bfiXjdaRYI2KKOWJTX FPGJ3k9WoBD36mKBgspStCsQzlzxftZjTo8/Hx22ZT38TQFA0rdz/+7aYpDWxNCHVG 3JwYu5XumfIUA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 17/21] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Mon, 2 Sep 2024 15:21:28 +0800 Message-ID: <20240902072136.578720-18-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3b98daa63ece..839fe7eaca8a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -488,20 +488,20 @@ static struct zpdesc *get_first_zpdesc(struct zspage = *zspage) =20 #define FIRST_OBJ_PAGE_TYPE_MASK 0xffffff =20 -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } =20 -static inline void set_first_obj_offset(struct page *page, unsigned int of= fset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned in= t offset) { - /* With 24 bits available, we can support offsets into 16 MiB pages. */ - BUILD_BUG_ON(PAGE_SIZE > SZ_16M); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + /* With 16 bit available, we can support offsets into 64 KiB pages. */ + BUILD_BUG_ON(PAGE_SIZE > SZ_64K); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &=3D ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |=3D offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &=3D ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |=3D offset & FIRST_OBJ_PAGE_TYPE_MASK; } =20 static inline unsigned int get_freeobj(struct zspage *zspage) @@ -921,7 +921,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) struct link_free *link; void *vaddr; =20 - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); =20 vaddr =3D zpdesc_kmap_atomic(zpdesc); link =3D (struct link_free *)vaddr + off / sizeof(*link); @@ -1576,7 +1576,7 @@ static unsigned long find_alloced_obj(struct size_cla= ss *class, unsigned long handle =3D 0; void *addr =3D zpdesc_kmap_atomic(zpdesc); =20 - offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + offset =3D get_first_obj_offset(zpdesc); offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { @@ -1771,8 +1771,8 @@ static void replace_sub_page(struct size_class *class= , struct zspage *zspage, } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 create_page_chain(class, zspage, zpdescs); - first_obj_offset =3D get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset =3D get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle =3D oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1827,7 +1827,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); =20 - offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + offset =3D get_first_obj_offset(zpdesc); s_addr =3D zpdesc_kmap_atomic(zpdesc); =20 /* --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98E4615FA92 for ; Mon, 2 Sep 2024 07:17:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261455; cv=none; b=ApfGiCit0J/qkihxDTmDU0Hmr1VnWqb+WBu88GGGUp/84wCoOm/1I6CCdn5MNtEyJBXwqUU6qAKENOcLlXS2zYxF7lQCRF4493zgnVHnijvvaHUiAB5VT6y4/wTkMNWvhLgpusH3IffNbIhLCIfOmtpv6vYzV2udPtjd+nu44dk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261455; c=relaxed/simple; bh=kXYRElA12DbhEH97D9heS99VK1p14zBkU2t+ZW1MGTY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dANwoH7XZD+0IkF+Id6kpStxYiIL37gnpBku0QgGd6nVDO3Q7NVSnhiNvf/R6Fuzn+Ru8RMrhr7PC7gC6PykEL+WJZnznwhayFd/gxPogybr5n1mLS67WsJ2TkKom1O92SGL3wEObuV+CqC98myDI3J63+4oUJykJpdg8ZvdHYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EXNl4KFK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EXNl4KFK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3294C4CEC4; Mon, 2 Sep 2024 07:17:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261455; bh=kXYRElA12DbhEH97D9heS99VK1p14zBkU2t+ZW1MGTY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EXNl4KFKTK0kmfyfg0PFMIaOD3FttgGZCH0MnmSSo5zAW04bquMAE2xDfbjMNNVIR xyOchytZAcoeIyzGiM6gh7DHnGBRVYsm0uukeKA7UiBdDmv3Bna/wVYMnlbvQ/wF2G mHIEqY6gnf8yj3FLghQIzw7pDazFPq9rtnwYI/Oxk7Lc9kLqjGKxucW3wXOgHl5Ox+ 1Oe3QJVe/XVTHUZlHu9WVghZq/41pyRXmUhbaLI+563RJskdW+XeKdRFKxczmafI+1 VJ3pKjjHnlgQRmX8cbEnLMbzcoVpRjFIEki8S7ZigdUbQW9rPHfj4lPtEWfk+s2mrS Zza1TaewLy57Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 18/21] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Mon, 2 Sep 2024 15:21:29 +0800 Message-ID: <20240902072136.578720-19-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index a1834d36ccfc..747a2d410a35 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -115,6 +115,11 @@ static inline void __zpdesc_set_movable(struct zpdesc = *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } =20 +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 839fe7eaca8a..107973a2a36e 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -834,7 +834,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page =3D zpdesc_page(zpdesc); =20 - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 288B915FA92 for ; Mon, 2 Sep 2024 07:17:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261459; cv=none; b=M+b94KV5/zJwjWblvE1KLDMOuHoUU6YQoKOSw9jwErlljdhCkIUlGj7j1lOSw58/Keu88IhtJf+KzFnmqY7NB29Dw50sXZEm3/yiSaQPm6RfH49VNqg1ILwnkguL2k68Kl7fn/c3B1mRCs4Y6YRMGz432osf826fwvuyIpPlpgI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261459; c=relaxed/simple; bh=qBTLwe3n68h/nHaRcrmhMxfrsTQQMe/QD9FAr8WVzS4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jdMStZc4Y6tYP4MqDIo72EwKF01gO6Hs7LRJaIy3zX9YNXChfRcOF+GUkdKrUTvY4ren53UXbKFaN1asLOqdopLuDstEE7Vs4HLUhCK5rDJfz8h6sGN57Z9yRKvJo3V7dcCHE9HYyzBd33dgA4VCoNPwMb9VMm1sk84izOp09A0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rmCwtlM6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rmCwtlM6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1DA3BC4CEC4; Mon, 2 Sep 2024 07:17:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261459; bh=qBTLwe3n68h/nHaRcrmhMxfrsTQQMe/QD9FAr8WVzS4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rmCwtlM67H383ue/d1A7wHWuRCZDakd/TZzSnpJeA4VFGc32hDbeOS8FTOWo5sRb0 ViqeWCYjWICvSaWY72rsoZUG4M6YX+PGvWQMmy3jIv7c7QCQS60yusdDIeSZK812FQ WLRmX14gVqvnz4Cdklb3OlLQTXo9nCCtLvzPXK8uSEjWqTwzP+fhJ4ututCYbSksjy f7Osa+8CG1OpTfqZwFyicoXReh1YIGFAQgIjmm0MeiZif5Ahk46Xc59XHm09Urv5Zk lBJVGSAjYGJy3i6OlV8Yy5/NI7Ll8uGLNkXK40cfZieLx7/rjrLINdxsuvXMIZaOKL uY/1WpeDffh0Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 19/21] mm/zsmalloc: introduce __zpdesc_clear/set_zsmalloc() Date: Mon, 2 Sep 2024 15:21:30 +0800 Message-ID: <20240902072136.578720-20-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use them in callers. Signed-off-by: Alex Shi --- mm/zpdesc.h | 10 ++++++++++ mm/zsmalloc.c | 8 ++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 747a2d410a35..33f599081281 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -120,6 +120,16 @@ static inline void __zpdesc_clear_movable(struct zpdes= c *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } =20 +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 107973a2a36e..411621f00a02 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -838,7 +838,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } =20 static int trylock_zspage(struct zspage *zspage) @@ -1011,13 +1011,13 @@ static struct zspage *alloc_zspage(struct zs_pool *= pool, if (!zpdesc) { while (--i >=3D 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); =20 zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] =3D zpdesc; @@ -1807,7 +1807,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); =20 /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); =20 /* The page is locked, so this pointer must remain valid */ zspage =3D get_zspage(zpdesc); --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEBDA18595B for ; Mon, 2 Sep 2024 07:17:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261463; cv=none; b=oHvwWADIb7vjIBDMsOpVx7ZSMeTm2bD6QsdrpRReF2P7TkAGYjj/+giZ8/QBqLitbn5EK9NpgkEzlPKZM5T5BHa7PGcfLI1x5j+YO6hwq2cOnEBhM5JAtmEb/EeF8EJoRYsq46SNnbSIEQ1KBaQovL5kBCv19gnGINFhPX+o5Gg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261463; c=relaxed/simple; bh=CFikcVVBkO0fkQwhPbTnJD5oTr7539f6hYvWeSZonkA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kM657irNOKGmPXuL0eVqAccgucCiiWkdaHPWdAQ/MNOM+nPwoW3oCnURwFF+vQPy66Pk0XE1z5JGq07vJOrLE7MV0YKTWJLh3xmNvPj2jvKLtwmXJ6b2qQGFMSUNTy3DpDdnjH9We/XvYF6UkY8Aui8aEMg4k+JIaxGKq8EazoM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=eXaocgJo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="eXaocgJo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3AEAC4CECA; Mon, 2 Sep 2024 07:17:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261462; bh=CFikcVVBkO0fkQwhPbTnJD5oTr7539f6hYvWeSZonkA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eXaocgJoKA1eC/mSBuf6h0+NhqRTKodMvnfs6tgPCGpmDK4W4Jfjf9Czj01vbzrB/ DwYWluMcU0nesvSRU1Oqa1i029m3hnIFjyRTlL7TRvoigsU1Si26+BxCZc1SMjXIrF m8JNHWz09WGPalcymX1yQO9CKN4uFTcN5ZwiKrsNbby8/9FA6yhG34TCuT7gTWM6JZ 4eNIM4jaA9E2e8kdSXAVYxorYAyipYDD1upXQF8a+L4fO1c3qgzq4wHVID461UIOfs gualGxbb0eMWhlydjmBwg9c78Xyb5/cKmORueJ+b6iqsiOJdA1wuJN/Osv7wlfjcdS AHBIjdJyXo26Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 20/21] mm/zsmalloc: introduce zpdesc_clear_first() helper Date: Mon, 2 Sep 2024 15:21:31 +0800 Message-ID: <20240902072136.578720-21-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Like the zpdesc_set_first(), introduce zpdesc_clear_first() helper for ClearPagePrivate(), then clean up a 'struct page' usage in reset_zpdesc(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 411621f00a02..849f192d4937 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -253,6 +253,11 @@ static inline void zpdesc_set_first(struct zpdesc *zpd= esc) SetPagePrivate(zpdesc_page(zpdesc)); } =20 +static inline void zpdesc_clear_first(struct zpdesc *zpdesc) +{ + ClearPagePrivate(zpdesc_page(zpdesc)); +} + static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) { inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); @@ -832,10 +837,8 @@ static inline bool obj_allocated(struct zpdesc *zpdesc= , void *obj, =20 static void reset_zpdesc(struct zpdesc *zpdesc) { - struct page *page =3D zpdesc_page(zpdesc); - __zpdesc_clear_movable(zpdesc); - ClearPagePrivate(page); + zpdesc_clear_first(zpdesc); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; __zpdesc_clear_zsmalloc(zpdesc); --=20 2.46.0 From nobody Thu Dec 18 19:04:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90A5718595B for ; Mon, 2 Sep 2024 07:17:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261466; cv=none; b=UcLvGY5vfllXJuF7vow73gQiBqDwjRFKfqMM+jr0FjSKU4oN7ovAee7Ypfo52COq+rEgkj5eQCx2E7wnPuoUTPaoSGqB0qV/jA5uN/hQ4bnQfIsLHT3MVDasuMGObHEQ+GWf8FG+dFr2p+htdidcljpNlMSDEjadlx94FsaGQhk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725261466; c=relaxed/simple; bh=8NP5E97AtP/kem4IUjjLi+FAumtWmwCjNYRbQZ4Zmwk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sEfumxtfGx3cagMFFOlpsUNms3ktLLQqQv29rFOMAGA3C7C+lKlaMlZPbqyjonmeCnxcwYYaIwsPigD2+YxlQUF8dSfASE/TCHxcIRfbODIAitYVlZY35rOpA9YhIqenXqBq/8XYaftsdzH8VD3pC3cBJ1JH5IU5FYeBB2XVpE0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=luyybGbQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="luyybGbQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1060BC4CEC2; Mon, 2 Sep 2024 07:17:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725261466; bh=8NP5E97AtP/kem4IUjjLi+FAumtWmwCjNYRbQZ4Zmwk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=luyybGbQP4t9sYIyrJgDKuirEOXTx1BKnq3ONcswGeFiP+L0aMFR8Iec24a+I+X6u DrKhRpC/OLMgaJziuSmQlKpnzOYKyvR/Vh1yQ7aDSjErHfjApKkVdb/q/2ag0XSK/B chTwLoZUDipz8l1hXp7+h3YMah7dJUleabEjOxfGT3ru9pVV5+oNWZ8b0EElUFt3q3 ZcD8H2E6p/WxIAyU7fXdmEmIjqSnXdOZHhVx6nBIht4bV2himfEqtGPEx2/cx23eoq is6MR0afprQ7E1x9B9EySSyJw7/VDVGw+49zdYW7k/vImFfc7UupAym3vgCuVPrMtk pjSj/AX+ltH6w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v7 21/21] mm/zsmalloc: update comments for page->zpdesc changes Date: Mon, 2 Sep 2024 15:21:32 +0800 Message-ID: <20240902072136.578720-22-alexs@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240902072136.578720-1-alexs@kernel.org> References: <20240902072136.578720-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi After the page to zpdesc conversion, there still left few comments or function named with page not zpdesc, let's update the comments and rename function create_page_chain() as create_zpdesc_chain(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 61 ++++++++++++++++++++++++++------------------------- 1 file changed, 31 insertions(+), 30 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 849f192d4937..1e47cda05c83 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,20 +13,19 @@ =20 /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component zpdescs of a zspage * For the huge page, this is always 0, so we use this field * to store handle. - * page->page_type: PGTY_zsmalloc, lower 24 bits locate the first object - * offset in a subpage of a zspage - * - * Usage of struct page flags: - * PG_private: identifies the first component page - * PG_owner_priv_1: identifies the huge component page + * zpdesc->first_obj_offset: PGTY_zsmalloc, lower 24 bits locate the first + * object offset in a subpage of a zspage * + * Usage of struct zpdesc(page) flags: + * PG_private: identifies the first component zpdesc + * PG_lock: lock all component zpdescs for a zspage free, serialize with */ =20 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -191,7 +190,10 @@ struct size_class { */ int size; int objs_per_zspage; - /* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */ + /* + * Number of PAGE_SIZE sized zpdescs/pages to combine to + * form a 'zspage' + */ int pages_per_zspage; =20 unsigned int index; @@ -900,7 +902,7 @@ static void free_zspage(struct zs_pool *pool, struct si= ze_class *class, =20 /* * Since zs_free couldn't be sleepable, this function cannot call - * lock_page. The page locks trylock_zspage got will be released + * lock_page. The zpdesc locks trylock_zspage got will be released * by __free_zspage. */ if (!trylock_zspage(zspage)) { @@ -957,7 +959,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) set_freeobj(zspage, 0); } =20 -static void create_page_chain(struct size_class *class, struct zspage *zsp= age, +static void create_zpdesc_chain(struct size_class *class, struct zspage *z= spage, struct zpdesc *zpdescs[]) { int i; @@ -966,9 +968,9 @@ static void create_page_chain(struct size_class *class,= struct zspage *zspage, int nr_zpdescs =3D class->pages_per_zspage; =20 /* - * Allocate individual pages and link them together as: - * 1. all pages are linked together using zpdesc->next - * 2. each sub-page point to zspage using zpdesc->zspage + * Allocate individual zpdescs and link them together as: + * 1. all zpdescs are linked together using zpdesc->next + * 2. each sub-zpdesc point to zspage using zpdesc->zspage * * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). @@ -1026,7 +1028,7 @@ static struct zspage *alloc_zspage(struct zs_pool *po= ol, zpdescs[i] =3D zpdesc; } =20 - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool =3D pool; zspage->class =3D class->index; @@ -1353,7 +1355,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, /* record handle in the header of allocated chunk */ link->handle =3D handle | OBJ_ALLOCATED_TAG; else - /* record handle to page->index */ + /* record handle to zpdesc->handle */ zspage->first_zpdesc->handle =3D handle | OBJ_ALLOCATED_TAG; =20 kunmap_atomic(vaddr); @@ -1443,7 +1445,6 @@ static void obj_free(int class_size, unsigned long ob= j) unsigned int f_objidx; void *vaddr; =20 - obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset =3D offset_in_page(class_size * f_objidx); zspage =3D get_zspage(f_zpdesc); @@ -1686,19 +1687,19 @@ static int putback_zspage(struct size_class *class,= struct zspage *zspage) #ifdef CONFIG_COMPACTION /* * To prevent zspage destroy during migration, zspage freeing should - * hold locks of all pages in the zspage. + * hold locks of all component zpdesc in the zspage. */ static void lock_zspage(struct zspage *zspage) { struct zpdesc *curr_zpdesc, *zpdesc; =20 /* - * Pages we haven't locked yet can be migrated off the list while we're + * Zpdesc we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock - * may no longer belong to the zspage. This means that we may wait for - * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * lock each zpdesc under migrate_read_lock(). Otherwise, the zpdesc we + * lock may no longer belong to the zspage. This means that we may wait + * for the wrong zpdesc to unlock, so we must take a reference to the + * zpdesc prior to waiting for it to unlock outside migrate_read_lock(). */ while (1) { migrate_read_lock(zspage); @@ -1773,7 +1774,7 @@ static void replace_sub_page(struct size_class *class= , struct zspage *zspage, idx++; } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); first_obj_offset =3D get_first_obj_offset(oldzpdesc); set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) @@ -1784,8 +1785,8 @@ static void replace_sub_page(struct size_class *class= , struct zspage *zspage, static bool zs_page_isolate(struct page *page, isolate_mode_t mode) { /* - * Page is locked so zspage couldn't be destroyed. For detail, look at - * lock_zspage in free_zspage. + * Page/zpdesc is locked so zspage couldn't be destroyed. For detail, + * look at lock_zspage in free_zspage. */ VM_BUG_ON_PAGE(PageIsolated(page), page); =20 @@ -1812,7 +1813,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, /* We're committed, tell the world that this is a Zsmalloc page. */ __zpdesc_set_zsmalloc(newzpdesc); =20 - /* The page is locked, so this pointer must remain valid */ + /* The zpdesc/page is locked, so this pointer must remain valid */ zspage =3D get_zspage(zpdesc); pool =3D zspage->pool; =20 @@ -1885,7 +1886,7 @@ static const struct movable_operations zsmalloc_mops = =3D { }; =20 /* - * Caller should hold page_lock of all pages in the zspage + * Caller should hold zpdesc locks of all in the zspage * In here, we cannot use zspage meta data. */ static void async_free_zspage(struct work_struct *work) --=20 2.46.0