From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9BD518622 for ; Tue, 6 Aug 2024 02:17:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910678; cv=none; b=KjPTMujZ6+78J59h5vGAeCDFBeFd1nOcOgzlAwyCELYX0pGF1VCd/4eDDCkJcb7elBZO2FC3Y7LWbIEZI31lNdSmhFokVNYP0TTx8aGApOCzhnG6noiHL7KabSZxCzizJXAZj6NjtF0kG7t5Gwns7mPmGVQx59gQn1MT5vRL9y8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910678; c=relaxed/simple; bh=QBwdQc1dwuHyM9NzB3E244MQf6Dsu1USXdYwK6ATMGw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WhpCQMyUDys+9hokUUjZmNu/S02Gjl/gYzbr5e3haPXUBxNcDEWcrGCiwzaQ6ksFycjBl/lcs8wLnABQ7rdE+952ODYi31vDKRUh6I82yDVVPCojRx1OT7F9SX1KWMtfmNEqRpoVlvSi8oZMJ7crM3Q64ot1PasJXSut2ucebbk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mv6dhj49; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mv6dhj49" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 33563C32782; Tue, 6 Aug 2024 02:17:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910678; bh=QBwdQc1dwuHyM9NzB3E244MQf6Dsu1USXdYwK6ATMGw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mv6dhj491G+wvqkCVOQPcZFOz90e8QwbdU1OqQYGUfz7bXpOGtE/EKXo3Wl+4C3V6 BmgsoLz8/5KNioCraKsuYBX68DG/tHGuT80cKnWOFrFUl24nhSPxsbQT+IpZqhyw9s djar2TNNIMLvYlhpdzqvaZu1mGka6ad2IKV6/YAuqvOyajAPMt403Lx9onkOcbne3o 2Ak4H1jmzpVLx9QzvOTp6sWdpHB/Rwn1koZaJ7boU1PW5DpL8llCiW7Sjk42llhlEg fNniplVkiQ/TRyTiJj4PXqqq7f2EISBarWVIHCgoKWQOnpMSNON9/kEp551CIq9iWz 2izL8p4n04eGw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 01/21] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Tue, 6 Aug 2024 10:22:48 +0800 Message-ID: <20240806022311.3924442-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi The 1st patch introduces new memory decriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). And keep the memcg_data member, since as Yosry pointed out: "When the pages are freed, put_page() -> folio_put() -> __folio_put() will = call mem_cgroup_uncharge(). The latter will call folio_memcg() (which reads folio->memcg_data) to figure out if uncharging needs to be done. There are also other similar code paths that will check folio->memcg_data. It is currently expected to be present for all folios. So until we have custom code paths per-folio type for allocation/freeing/etc, we need to keep folio->memcg_data present and properly initialized." Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 25 +++++++++--------- 2 files changed, 83 insertions(+), 13 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..0df54a5b1670 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirectly used by page migration + * @mops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Points to the zspage this zpdesc is a part of + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly use by page migration + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + struct movable_operations *mops; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; + unsigned int first_obj_offset; + atomic_t _refcount; +#ifdef CONFIG_MEMCG + unsigned long memcg_data; +#endif +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) =3D=3D offsetof(struct zpdesc, zp= )) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, mops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); +ZPDESC_MATCH(_refcount, _refcount); +#ifdef CONFIG_MEMCG +ZPDESC_MATCH(memcg_data, memcg_data); +#endif +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <=3D sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d6581ab7c07..30f0a7abbda3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,20 +13,18 @@ =20 /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. - * page->page_type: PG_zsmalloc, lower 16 bit locate the first object - * offset in a subpage of a zspage + * zpdesc->first_obj_offset: PG_zsmalloc, lower 16 bit locate the first + * object offset in a subpage of a zspage * - * Usage of struct page flags: + * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page - * PG_owner_priv_1: identifies the huge component page - * */ =20 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -64,6 +62,7 @@ #include #include #include +#include "zpdesc.h" =20 #define ZSPAGE_MAGIC 0x58 =20 @@ -253,7 +252,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -448,7 +447,7 @@ static inline void mod_zspage_inuse(struct zspage *zspa= ge, int val) =20 static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page =3D zspage->first_page; + struct page *first_page =3D zpdesc_page(zspage->first_zpdesc); =20 VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -948,7 +947,7 @@ static void create_page_chain(struct size_class *class,= struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index =3D 0; if (i =3D=3D 0) { - zspage->first_page =3D page; + zspage->first_zpdesc =3D page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage =3D=3D 1 && class->pages_per_zspage =3D=3D 1)) @@ -1324,7 +1323,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle =3D handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index =3D handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle =3D handle | OBJ_ALLOCATED_TAG; =20 kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8DFB723776 for ; Tue, 6 Aug 2024 02:18:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910682; cv=none; b=P27IT23EvTkevtI74Y0FbuChW09YL0KhBIicOcBVmZIojgHb5xsmsJg0yBtr//1TIa0/R20bRiXa14cP/m3ou5gb7sLoB2us3ntTnhgRGgdsT/yjU+2AopJpFmhzGKenCOUzdyLKS7ClRqW1yVf6JxzMtoMEQPnYn0fDROGuBfk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910682; c=relaxed/simple; bh=W2CmxaC60Muhq4cmp7NdT+PQ9UwnM6tRFRsuxLeQy0s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NmLvgcMWuyp7hGNULrSoaCkS1Ql65gMskmnQHS7LEvwIw36VEMCXOSvqyfOPuD0IlGE8bKYPTGPjGfXHM2bI4eqCQMVrcmD8ZxOb/ag92AjOizmZTfgiygTFjhiNKIbV2qgXosF2sA9Zbn7fyIrC/Xch/tg/k2DQih/fjAKS4Qs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rc1NJYVS; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rc1NJYVS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 07238C32782; Tue, 6 Aug 2024 02:17:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910682; bh=W2CmxaC60Muhq4cmp7NdT+PQ9UwnM6tRFRsuxLeQy0s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rc1NJYVSBoTuAOpI656XakHjjozJdwgj0cfNQKibJWY1LL8jXwY1WbssbpvpbER1c 073HAUwpSPdOO/ObCGVEy/V2WrdPUN9CE0imEwfF24LSChnSVfkxdzE3Hi/81h+Qgw Tv72ubRfzJgh7QGq3G9bcSqWtpY2v/me/APYMaPnJ/tiUbMf3WZkk3CD8BZ4SjGTI5 T9YRt/NxHjIMqT9neXat/llYvBdantwiqmgBW8/yvpPxzyqg4Lvr2NqYnBteDvTNtS a/Ur5BNX8XYzh08MQfeB+baYhmi02LXVkt5rTusJok2SIRQvbcncnRMzdl7hZA8hUM 3vGbZy+RQvA3Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 02/21] mm/zsmalloc: use zpdesc in trylock_zspage()/lock_zspage() Date: Tue, 6 Aug 2024 10:22:49 +0800 Message-ID: <20240806022311.3924442-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi To use zpdesc in trylock_zspage()/lock_zspage() funcs, we add couple of hel= pers: zpdesc_lock()/zpdesc_unlock()/zpdesc_trylock()/zpdesc_wait_locked() and zpdesc_get()/zpdesc_put() for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. BTW, thanks Intel LKP found a build warning on the patch. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 0df54a5b1670..421eeeef6f8f 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -68,4 +68,34 @@ static_assert(sizeof(struct zpdesc) <=3D sizeof(struct p= age)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) =20 +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 30f0a7abbda3..25c90224f21f 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -433,13 +433,17 @@ static __maybe_unused int is_first_page(struct page *= page) return PagePrivate(page); } =20 +static inline bool is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } =20 - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse +=3D val; @@ -453,6 +457,14 @@ static inline struct page *get_first_page(struct zspag= e *zspage) return first_page; } =20 +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc =3D zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff =20 static inline void reset_first_obj_offset(struct page *page) @@ -745,6 +757,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } =20 +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -815,11 +837,11 @@ static void reset_page(struct page *page) =20 static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; =20 - for (cursor =3D get_first_page(zspage); cursor !=3D NULL; cursor =3D - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor =3D get_first_zpdesc(zspage); cursor !=3D NULL; cursor =3D + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail =3D cursor; goto unlock; } @@ -827,9 +849,9 @@ static int trylock_zspage(struct zspage *zspage) =20 return 1; unlock: - for (cursor =3D get_first_page(zspage); cursor !=3D fail; cursor =3D - get_next_page(cursor)) - unlock_page(cursor); + for (cursor =3D get_first_zpdesc(zspage); cursor !=3D fail; cursor =3D + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); =20 return 0; } @@ -1658,7 +1680,7 @@ static int putback_zspage(struct size_class *class, s= truct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; =20 /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1670,24 +1692,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page =3D get_first_page(zspage); - if (trylock_page(page)) + zpdesc =3D get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } =20 - curr_page =3D page; - while ((page =3D get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page =3D page; + curr_zpdesc =3D zpdesc; + while ((zpdesc =3D get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc =3D zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 281CC2B9C6 for ; Tue, 6 Aug 2024 02:18:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910686; cv=none; b=EH3RJ5lgHLMP7nMoRs8I6Vv7/k04yiLosAhZqwCLYI8bB96u5pjFkRgv6MK0ji2HLP6qaTOZZKOb1NQzV8vWIQU6CkHnPmXgKNq2B4qXNQaiO1IXZjw7d/wxSSWldW+ydOUmCSx8vWY1JrVJV0BMMqsuIGYcdhhQWm7sGdYVR94= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910686; c=relaxed/simple; bh=rMm13SX4aLf3oZ/lBmcjJeeV3CGO+2RCAQN5DP+xGmo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EZZD9GUG4U9WPtZh48jcRz7exOzz8Mc0jOjgDpF4IjTAo89a+kiGeaeBB/PED19PIWMntKlhjM1I2sdQmwknBKnExc2tYSgpSBuxydE+D5LdBpPy3bLivRqorccd2D95nIqRUHaey0Jrfbp0nr5lpOhGlKbL4+UmTZ8H8SWkWBI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y9bMfZD/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y9bMfZD/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CD578C32782; Tue, 6 Aug 2024 02:18:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910686; bh=rMm13SX4aLf3oZ/lBmcjJeeV3CGO+2RCAQN5DP+xGmo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y9bMfZD/5HIEGcjAndQ/lypfE4f+loM6ikbINUx7e4ZVxqmabtcM3IJIvPbKyohqh R0lCElOP1IqSvv4eN2T5KrP+x7QIi/l/v+tKP2fas0PoG5toYLpB4I8hyrj3HHzO4l i3LfF0/shOAC/fAypNubA8AmCHtevJ6wxFiamGL7VuIWUQzJUzakpEWwkMwS6V270S 6YI85bUWbzpkTNsuW+b7q0KKgWs4xbBRXMUf/5dTRjxYEjlzTdU1434myFq2/48qy8 9UhQf2lLEHTOM1xIgk/f1Tgn8WKUcPUokYFcZtIK4WiIPOvpjGiU92Bq4SJtbV7Sl1 uNm1Wpab5IGyw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 03/21] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Tue, 6 Aug 2024 10:22:50 +0800 Message-ID: <20240806022311.3924442-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 25c90224f21f..b9b5e2824f2c 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -243,6 +243,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; =20 +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1061,7 +1066,7 @@ static inline void __zs_cpu_down(struct mapping_area = *area) } =20 static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1078,10 +1083,10 @@ static void *__zs_map_object(struct mapping_area *a= rea, sizes[1] =3D size - sizes[0]; =20 /* copy object to per-cpu buffer */ - addr =3D kmap_atomic(pages[0]); + addr =3D zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr =3D kmap_atomic(pages[1]); + addr =3D zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1089,7 +1094,7 @@ static void *__zs_map_object(struct mapping_area *are= a, } =20 static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1108,10 +1113,10 @@ static void __zs_unmap_object(struct mapping_area *= area, sizes[1] =3D size - sizes[0]; =20 /* copy per-cpu buffer to object */ - addr =3D kmap_atomic(pages[0]); + addr =3D zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr =3D kmap_atomic(pages[1]); + addr =3D zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); =20 @@ -1252,7 +1257,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, pages[1] =3D get_next_page(page); BUG_ON(!pages[1]); =20 - ret =3D __zs_map_object(area, pages, off, class->size); + ret =3D __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret +=3D ZS_HANDLE_SIZE; @@ -1287,7 +1292,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) pages[1] =3D get_next_page(page); BUG_ON(!pages[1]); =20 - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); =20 --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A56638396 for ; Tue, 6 Aug 2024 02:18:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910690; cv=none; b=ECsjgw7hrdUfUg5frLY1rglpxVufL4l1nXiSbju6XAh2UG36OuEmJDD6BQJsMPLsVENThIHK78WipxPidG5kOXwnhSMCk521+Ch/gIyRbVQVVWQTiMyvJ/bPohMcQru8v6rGPGDRfu7AAYspHdsya0HMhxDp5H7SQapqEyrPg4o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910690; c=relaxed/simple; bh=xRbL5zVs+Qjgf9i92JU0jHMe8Ob3o1agljGLF7p0KNc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HHkGN2bKFhsosoNjdPisr7aydPJtsFDzh0Jm9FVP+9YqZVrA385l0sE982WkmSKKAtj67hZsEbGhnJ+zV7hRtYGgC4UOke1AR4PMJ2dosH1LgWBKzKZZgHZG/8lJqDgk7k8RobD+C9/XoDCLlYeivnE9+gCQDLcJFKYZ69b4ecs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kWdNfAWj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kWdNfAWj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F567C32782; Tue, 6 Aug 2024 02:18:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910689; bh=xRbL5zVs+Qjgf9i92JU0jHMe8Ob3o1agljGLF7p0KNc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kWdNfAWjBaZIXiutIjz8LzEoghILI0Dj9+anUzmCrCkxd2qtOypIhrUAueUbH9/AB SUaM6AzVwPMda0h0Fp24pvVUNJSHvPsPeONOsGBBdfOM6WKARobWbhHgGiw0N24ciB s3peCG93Il1VzKl2xancD0/l4WcPam3WIJ6VxrY7s64MRlHHqfgauOTcbXQdTGX+0V jFm+qWpku0GRsDctyTcWfoTMSz3ku0661lDy50QNwj+ridqElSa1flaoSpkm7/wTvk 9T7sWf9dGhH1BuNHnBlTr/MEgx52hGdXT2HzQRDzx2fMk8DowDnotoKNHgihfWmglX 0GFSUutoDztSw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 04/21] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Tue, 6 Aug 2024 10:22:51 +0800 Message-ID: <20240806022311.3924442-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 421eeeef6f8f..2101de23d16d 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -98,4 +98,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } =20 +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index b9b5e2824f2c..384a5ba49788 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -773,15 +773,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *= zpdesc) } =20 /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page =3D pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx =3D (obj & OBJ_INDEX_MASK); } =20 @@ -1208,13 +1208,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned = long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; =20 struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; =20 /* @@ -1227,8 +1227,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage =3D get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage =3D get_zspage(zpdesc_page(zpdesc)); =20 /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1247,17 +1247,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned = long handle, area->vm_mm =3D mm; if (off + class->size <=3D PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr =3D kmap_atomic(page); + area->vm_addr =3D zpdesc_kmap_atomic(zpdesc); ret =3D area->vm_addr + off; goto out; } =20 /* this object spans two pages */ - pages[0] =3D page; - pages[1] =3D get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] =3D zpdesc; + zpdescs[1] =3D get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); =20 - ret =3D __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret =3D __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret +=3D ZS_HANDLE_SIZE; @@ -1269,7 +1269,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; =20 @@ -1277,8 +1277,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) struct mapping_area *area; =20 obj =3D handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage =3D get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage =3D get_zspage(zpdesc_page(zpdesc)); class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 @@ -1286,13 +1286,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned= long handle) if (off + class->size <=3D PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; =20 - pages[0] =3D page; - pages[1] =3D get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] =3D zpdesc; + zpdescs[1] =3D get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); =20 - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); =20 @@ -1434,23 +1434,24 @@ static void obj_free(int class_size, unsigned long = obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; =20 - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset =3D offset_in_page(class_size * f_objidx); - zspage =3D get_zspage(f_page); + zspage =3D get_zspage(zpdesc_page(f_zpdesc)); =20 - vaddr =3D kmap_atomic(f_page); + vaddr =3D zpdesc_kmap_atomic(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); =20 /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next =3D get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index =3D 0; + f_zpdesc->next =3D NULL; set_freeobj(zspage, f_objidx); =20 kunmap_atomic(vaddr); @@ -1495,7 +1496,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1504,8 +1505,8 @@ static void zs_object_copy(struct size_class *class, = unsigned long dst, =20 s_size =3D d_size =3D class->size; =20 - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); =20 s_off =3D offset_in_page(class->size * s_objidx); d_off =3D offset_in_page(class->size * d_objidx); @@ -1516,8 +1517,8 @@ static void zs_object_copy(struct size_class *class, = unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size =3D PAGE_SIZE - d_off; =20 - s_addr =3D kmap_atomic(s_page); - d_addr =3D kmap_atomic(d_page); + s_addr =3D zpdesc_kmap_atomic(s_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); =20 while (1) { size =3D min(s_size, d_size); @@ -1542,17 +1543,17 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, if (s_off >=3D PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page =3D get_next_page(s_page); - s_addr =3D kmap_atomic(s_page); - d_addr =3D kmap_atomic(d_page); + s_zpdesc =3D get_next_zpdesc(s_zpdesc); + s_addr =3D zpdesc_kmap_atomic(s_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); s_size =3D class->size - written; s_off =3D 0; } =20 if (d_off >=3D PAGE_SIZE) { kunmap_atomic(d_addr); - d_page =3D get_next_page(d_page); - d_addr =3D kmap_atomic(d_page); + d_zpdesc =3D get_next_zpdesc(d_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); d_size =3D class->size - written; d_off =3D 0; } @@ -1791,7 +1792,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95CA43A8E4 for ; Tue, 6 Aug 2024 02:18:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910693; cv=none; b=R7/YJR3H9IN4T5YvXQe72piBRAhJX7TIbIiEiKH5k/XjUHGYMU6EE/+YZWR5KKql2zNsDRXvuUGKrK4epKceb2ntofC5DHVPx/infyXQh8WKDjfU7HEHYWx635TZbui/SMWJNr6oaSGjpIiRVTQd69088+3Jnd34HYOEu2KT2NQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910693; c=relaxed/simple; bh=j9RlWiCV98maWH46GH8LQqP9KXOBrIEnE2eMblguIA0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DUiDBc0EyCsKxfNiRHCrBZognl/qvLecOTzLpEa++u8jtfcXiLmme/Y6p3HNYhkQSATBAQH07PnVBg9JF2iaNOhsJV1UTlY7qsdA8mKyJrg+j+1p9CBR1rs3msJUY60s/PqPAwr9sGCrnoqDEC2SPr8R2BWNTSgCVZAUZunX8Ls= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SxdVNQz1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SxdVNQz1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 735E6C4AF0F; Tue, 6 Aug 2024 02:18:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910693; bh=j9RlWiCV98maWH46GH8LQqP9KXOBrIEnE2eMblguIA0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SxdVNQz1yw2axgmM9F16czobZ/4nc0Pb+j/p+kCd1AYPxtcjl3v5C8Ac1KS+WxEJA qHbVrLZrGxWROP255woP/8UgVivsjW7fQKSmK/u3U1YhM2Y5yfKSBcPBnm9hSvwETb tHwuQb3AVPme+ZKcFSzgpl9kj5oiBQM4daNIhzVFI3bO980a3e7XE7995BirXv6581 ZNtaCZgVAX8hzxySY1qD5hfTenjzEHUfm9R+fwANUH/T/lBx0ZWNyrtlZH/O8s3ftT moOpR1aUdc1DFPo49MVi4sKcDA2ExbcNzbWbRSz81ROx6l14RmY1ZjK+/txBfF/I2J DfRyE8ct66jZQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 05/21] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Tue, 6 Aug 2024 10:22:52 +0800 Message-ID: <20240806022311.3924442-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 384a5ba49788..7421d7678880 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1322,12 +1322,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; =20 - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; =20 @@ -1335,14 +1335,14 @@ static unsigned long obj_malloc(struct zs_pool *poo= l, obj =3D get_freeobj(zspage); =20 offset =3D obj * class->size; - nr_page =3D offset >> PAGE_SHIFT; + nr_zpdesc =3D offset >> PAGE_SHIFT; m_offset =3D offset_in_page(offset); - m_page =3D get_first_page(zspage); + m_zpdesc =3D get_first_zpdesc(zspage); =20 - for (i =3D 0; i < nr_page; i++) - m_page =3D get_next_page(m_page); + for (i =3D 0; i < nr_zpdesc; i++) + m_zpdesc =3D get_next_zpdesc(m_zpdesc); =20 - vaddr =3D kmap_atomic(m_page); + vaddr =3D zpdesc_kmap_atomic(m_zpdesc); link =3D (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1355,7 +1355,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(m_page, obj); + obj =3D location_to_obj(zpdesc_page(m_zpdesc), obj); record_obj(handle, obj); =20 return obj; --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6254140858 for ; Tue, 6 Aug 2024 02:18:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910697; cv=none; b=f+nui5EP3S7pGz3NslH9kiDk1Hcb4OTKaWJUwkHBY2CxqEEpR0qMR6WjbIy9Rq7pyFu4qbN/NtUraXwkmJk7zr/RUxXHSdpSYTsyhB1pLUrj1MxBB/iEcKikp3/PLD1pfTC0VXqT0cGjPcCO39xV9opDjx8p1i+oJQNzPtAyY48= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910697; c=relaxed/simple; bh=Hcz6YDLKpn+zwSbhAqxdxpFHMpHIDCe8idbfi+gvSSM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WHqKTxKx7fFEbwHka0F6KXdfGso75U3EC3f/LRUCpXiXTmBH9Pm2Cv65mGWFA9GLFfHDQigMSoBRS1e1fhRTd7RV5nlpSJyu1ihyS/BGlczmjdvN0zbkp97AzTwYEDgvl7o+dFCVprwdmW5d8OCvvN6XuDg8Am31s0DwvL6CKSA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CNjv4Yt9; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CNjv4Yt9" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1EF94C4AF0B; Tue, 6 Aug 2024 02:18:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910697; bh=Hcz6YDLKpn+zwSbhAqxdxpFHMpHIDCe8idbfi+gvSSM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CNjv4Yt9eT8l1U83xyPDy6BZuirk5dnxQU81nTlT/y5ULIt1rjedy+R5ACeEBWVBi VjVKk5NmdQP/WWEUzGqrzT/Ogyykzw74HoNE1d2Bb7YfG/8xWI5ax3dCf6Xr7roZDJ Dk0iu3z0Ib7D4ATLidLr6RPt6z1oEzNj6mPt1ao36Kolg7F5BG4mpOYNep/gw6xur2 NfdoPAuQsnTHxwzxtvJfi1g/1TMDQVBATGFt2nKp1jWMG6DhjD9cuTQKfitgFhUrXi I8gLccD4SMhAWkrUe+LA6YXXw2sZJsJLCqF2lkEQ87ruSuH8FI/KSKK6CgG2hLMUbn LHxQHsK6dtWzQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 06/21] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Tue, 6 Aug 2024 10:22:53 +0800 Message-ID: <20240806022311.3924442-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chai= n() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 ++++ mm/zsmalloc.c | 95 ++++++++++++++++++++++++++++++--------------------- 2 files changed, 62 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 2101de23d16d..6a01e09e4beb 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -107,4 +107,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long = pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7421d7678880..33a4f7d026e3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -248,6 +248,21 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *= zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } =20 +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -954,35 +969,35 @@ static void init_zspage(struct size_class *class, str= uct zspage *zspage) } =20 static void create_page_chain(struct size_class *class, struct zspage *zsp= age, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page =3D NULL; - int nr_pages =3D class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc =3D NULL; + int nr_zpdescs =3D class->pages_per_zspage; =20 /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i =3D 0; i < nr_pages; i++) { - page =3D pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index =3D 0; + for (i =3D 0; i < nr_zpdescs; i++) { + zpdesc =3D zpdescs[i]; + zpdesc->zspage =3D zspage; + zpdesc->next =3D NULL; if (i =3D=3D 0) { - zspage->first_zpdesc =3D page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc =3D zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage =3D=3D 1 && class->pages_per_zspage =3D=3D 1)) SetZsHugePage(zspage); } else { - prev_page->index =3D (unsigned long)page; + prev_zpdesc->next =3D zpdesc; } - prev_page =3D page; + prev_zpdesc =3D zpdesc; } } =20 @@ -994,7 +1009,7 @@ static struct zspage *alloc_zspage(struct zs_pool *poo= l, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage =3D cache_alloc_zspage(pool, gfp); =20 if (!zspage) @@ -1004,25 +1019,25 @@ static struct zspage *alloc_zspage(struct zs_pool *= pool, migrate_lock_init(zspage); =20 for (i =3D 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; =20 - page =3D alloc_page(gfp); - if (!page) { + zpdesc =3D page_zpdesc(alloc_page(gfp)); + if (!zpdesc) { while (--i >=3D 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __free_page(zpdesc_page(zpdescs[i])); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); =20 - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] =3D page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] =3D zpdesc; } =20 - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool =3D pool; zspage->class =3D class->index; @@ -1753,26 +1768,28 @@ static void migrate_write_unlock(struct zspage *zsp= age) static const struct movable_operations zsmalloc_mops; =20 static void replace_sub_page(struct size_class *class, struct zspage *zspa= ge, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] =3D {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] =3D {NULL, }; + unsigned int first_obj_offset; int idx =3D 0; =20 - page =3D get_first_page(zspage); + zpdesc =3D get_first_zpdesc(zspage); do { - if (page =3D=3D oldpage) - pages[idx] =3D newpage; + if (zpdesc =3D=3D oldzpdesc) + zpdescs[idx] =3D newzpdesc; else - pages[idx] =3D page; + zpdescs[idx] =3D zpdesc; idx++; - } while ((page =3D get_next_page(page)) !=3D NULL); + } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset =3D get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index =3D oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle =3D oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } =20 static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1845,7 +1862,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, } kunmap_atomic(s_addr); =20 - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5796243ABC for ; Tue, 6 Aug 2024 02:18:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910701; cv=none; b=DEGX3hPmf2HQUTPqUCR+bYs/vCn7KoPKvWoULBmNw5R1IAgeFlcnWZCKVjRGv579NkyZp5ibD49NJUzM+lsvqj5FLgxjXAz306RmHcIKhoYneDg7i/djw/ugjgFnsFpL3dL9Ew2a9rBndz7if9rLsVqSTlc8WoAnEBuH5QiX8JA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910701; c=relaxed/simple; bh=S9lb/ZM2ddFk5ykU4pBIY4i0BSbXkh97cFi/e+1K9AA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TKaowldX/iLMGROe7ywTnR+aHZY3v2F6E7kfFc4zoyyFO/vooFi6WWIx3iteAIMhkdM8dfg+fNO7/Hz16rhIW5jS65W8F0e5Y+YwxlhSZJAR+RPEBjf48/ZxwWirspOdm/PAPhAmTSJ3tYqrDwUpOLHNQsLQ+OVfabiQKSQ+0tQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FoXry/KR; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FoXry/KR" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7EBCC4AF0B; Tue, 6 Aug 2024 02:18:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910700; bh=S9lb/ZM2ddFk5ykU4pBIY4i0BSbXkh97cFi/e+1K9AA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FoXry/KRy6McNtYGfI2Z6enXmj2PWAy7b9Lk8XnIwOINqaCLUi2tLUaqmmtgbrpG/ MxbLCcc0DyuZARER7WhzK6gSVOwnOsHFc6/RIAwicCSRIGdGeoRD1WcpwBuRgSYNlB kH1juNG7IOd1xdgQDziBMjpAYa4BgcHE3F6FXY4SanlKYCBX8Dg00AfglkRQ9Md6O4 lxZENUCGEbNYf4+TLt3zOZ2H3MKXoY9KXp+kd38tme2pa0wo3SFHn68t6XoGliUpbi mfFhW6n+LPNt4zSKnSof8Uoueji5zXnGJ0yBZbj3+XJqo46Rfrfv+PmvZHTQMa1OTC 4fc970Hz8u/Hg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 07/21] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Tue, 6 Aug 2024 10:22:54 +0800 Message-ID: <20240806022311.3924442-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 33a4f7d026e3..3c1755227706 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -825,15 +825,15 @@ static unsigned long handle_to_obj(unsigned long hand= le) return *(unsigned long *)handle; } =20 -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage =3D get_zspage(page); + struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); =20 if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle =3D page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle =3D zpdesc->handle; } else handle =3D *(unsigned long *)obj; =20 @@ -1583,18 +1583,18 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index =3D *obj_idx; unsigned long handle =3D 0; - void *addr =3D kmap_atomic(page); + void *addr =3D zpdesc_kmap_atomic(zpdesc); =20 - offset =3D get_first_obj_offset(page); + offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; =20 offset +=3D class->size; @@ -1618,7 +1618,7 @@ static void migrate_zspage(struct zs_pool *pool, stru= ct zspage *src_zspage, struct size_class *class =3D pool->size_class[src_zspage->class]; =20 while (1) { - handle =3D find_alloced_obj(class, s_page, &obj_idx); + handle =3D find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page =3D get_next_page(s_page); if (!s_page) @@ -1851,7 +1851,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 for (addr =3D s_addr + offset; addr < s_addr + PAGE_SIZE; addr +=3D class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C38547F7A for ; Tue, 6 Aug 2024 02:18:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910705; cv=none; b=Q8WEBiRU6A5Qemcx76cAgisXWcqmhwqbhnBk5l1l696I8AFnPOPyUfHYtyFLi69okyPqdVy29shPTCvwFUEznki54fcMfge0Zozjn3lEoIWv6++np6fzTG233K05ukQ5zI3ohq5RIDDMPouwIQZH2/dx5HZ1XWE3Efr1Eih9924= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910705; c=relaxed/simple; bh=vNwIGyy8uUag3zQP5bXF/PNe/niCN1bQWoZ3dk5XDpg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j1p2nqn2xd/zaT+FoVMR0Qtvs7Fb7Pnm70hhZA+2vQMZ+sj1WpWDYmDZ7f+KIqaDD8pZo+7SCqfK66qNkSCB+Pt+yvrOfHaeDQsABvaKXn51WZS2sLlVwMhjfbjWlKPSKml4gWFriifWuv7P2bnQdM7ProM/KWnJcpcYJIFFis8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HsMWYRZJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HsMWYRZJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9359EC32782; Tue, 6 Aug 2024 02:18:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910704; bh=vNwIGyy8uUag3zQP5bXF/PNe/niCN1bQWoZ3dk5XDpg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HsMWYRZJUqU0Itrt3J8567cHgcx7ZWcgbPSnbOQZy70q7mwkndUTL+/3BqxOm3jLe 1JRXz3aiwZLokB5iJOIAxSzflNxhwrdBNUmQ0yr+fTHmP6UHPnfZXf7ixPXMndCYtR Xvx0NYmeEH8VFJaWIPEcpkE44VAf3Iq0589/x9KYqXkzoR1v4hFCn9gXAp43FRTRuB 3gwiRk+9fT3CtV1V1KDMvvYsPFFrtc+3sl1sU18WkpAkEgFNlSYItJMcheDweCDUcj x6cS/1tCwsSUmImOLOvc1iIP45X2TJ1nhJOZhMkYGSHW81OmWHotixv+CqQ+uO3/q4 WUgiwGNqY5X+w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 08/21] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Tue, 6 Aug 2024 10:22:55 +0800 Message-ID: <20240806022311.3924442-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3c1755227706..d7400acd0f01 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -928,16 +928,16 @@ static void init_zspage(struct size_class *class, str= uct zspage *zspage) { unsigned int freeobj =3D 1; unsigned long off =3D 0; - struct page *page =3D get_first_page(zspage); + struct zpdesc *zpdesc =3D get_first_zpdesc(zspage); =20 - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; =20 - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); =20 - vaddr =3D kmap_atomic(page); + vaddr =3D zpdesc_kmap_atomic(zpdesc); link =3D (struct link_free *)vaddr + off / sizeof(*link); =20 while ((off +=3D class->size) < PAGE_SIZE) { @@ -950,8 +950,8 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page =3D get_next_page(page); - if (next_page) { + next_zpdesc =3D get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next =3D freeobj++ << OBJ_TAG_BITS; } else { /* @@ -961,7 +961,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) link->next =3D -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page =3D next_page; + zpdesc =3D next_zpdesc; off %=3D PAGE_SIZE; } =20 --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C855051C3E for ; Tue, 6 Aug 2024 02:18:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910708; cv=none; b=JyPaZok1XPVhUECKoJ+bPlcxt7XDsoJ+DfaTkXJOf3R+k3FgTIaMmGH/FUwRHwu2p63UtTUyL5WCCtNZ6HR7eGJ3M3j2pvOwDktY1zcjPI3iYnn64SnTxRKMbZyQx/COcOZcZsQ8W6/nJBEv4fxLe36YfMEResKB9iVaoK1yT1c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910708; c=relaxed/simple; bh=2XuUwCpmZ+NvY5bJ3CJbiV5pb57fdunEaUBcKFqNcHQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CDW3Ej4ijvXl2OcfyVRn4y/iBSsQ776rgoaZDVHLP87FqC0Pv54zrdJRY029fjaR3gorCt081ocTXGOeD62QppIfrEIvNLW18v2/i6V+276bYZJx/xTGcJcr41yBF3W14ca0Rx/DVRBhmzkD5pKHOgEwTh6owFoOH0YzZ1Ys9bE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QBV8krES; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QBV8krES" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 66673C4AF0F; Tue, 6 Aug 2024 02:18:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910708; bh=2XuUwCpmZ+NvY5bJ3CJbiV5pb57fdunEaUBcKFqNcHQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QBV8krESkVQBI1btdudXp3KnYlXCSSkzzl+6/65HHV3ctpQCaq1VOxmims7maxwHY wrBU3toUz1xv2/xTaEdWgIGqSdlbQT8ALHlvL6k9uSg7FUZoD2n3R9jk12A6+w3PPG 4p9Qjo2J8NR/4mJT1z2wuIu0hKQsn8J025ArD7UIENklvSYRCOSEarp8LcmPNjEITN fnG0iA0wTar2Lb2bdYWWc9r/imZYQ83co9ypl4Ct5PTBSrK62sZ/wIQDLljIBPQAks EsRG+glMxynyTvNGO7qfPVnLA3IEmk1uoQC882anPHSZqeNAp7qpEJjUJ8+3Lp09gw whxLi1wfKYSfw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 09/21] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Tue, 6 Aug 2024 10:22:56 +0800 Message-ID: <20240806022311.3924442-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d7400acd0f01..5fac2bb436f8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -800,9 +800,9 @@ static void obj_to_location(unsigned long obj, struct z= pdesc **zpdesc, *obj_idx =3D (obj & OBJ_INDEX_MASK); } =20 -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page =3D pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); } =20 /** @@ -1476,7 +1476,7 @@ static void obj_free(int class_size, unsigned long ob= j) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1490,8 +1490,8 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) */ read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage =3D get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage =3D get_zspage(zpdesc_page(f_zpdesc)); class =3D zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5901FDF5C for ; Tue, 6 Aug 2024 02:18:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910712; cv=none; b=uUzqA2twP3KekbFEZ2+0Ky7XalVNcZre+LP9VItqIN3O+XWJ7QcrPbhtohmUaoV8XeWOLHQCmGvyzDy4sX69/6JcHNEoMs0RwyKL3C+oQU/R3T/wQ4cUAXsNsUfoG4uAEdXaZdlduqCyQYrwlmBForWFHiqSIpM95AceTObLVjU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910712; c=relaxed/simple; bh=jd//d/JKjS/cL1w5x1hSjGz2Vv8a8C5I3yz/l4s7SBA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bqko8Ifi/zBkBSOL3H7e3PJ6vwBHDIRQEue9aOQwb6fGsCwnCTHkdGIFiGR994I8bAUb2dV/wpPOpd09ejD58JAR4KQX2OTmJXVAJOeV5MiMsfxNa5PCF0yb/sPDJHvP6MbPlmTFibYLolTJ5uoOw2XGQnrK9vEDb1aApW5zr+g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=O01JxTsF; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="O01JxTsF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13C64C32782; Tue, 6 Aug 2024 02:18:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910712; bh=jd//d/JKjS/cL1w5x1hSjGz2Vv8a8C5I3yz/l4s7SBA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O01JxTsFcnRCOHKuPEnby1tCq8aWX74P/c0exfYMK/J/CPxNyd8q5/bPamjiRPu/D Jbdq5g/rE4MKhzhnGw6Y+VEGx66ZKCcPQDJlAoDr/Ho7DFKt3N+Ej0/Y+xm+s8hkZ1 2pbU44Dg9XOKAaX+Ya+IpbEvo2BwfU0yv8ESApwHCdGk424aYgHyqyBtoK8f/UUPay XPT+/plQr4/OmfpRfFVDj3hPa3nI0wbRBKhBeb0/doYypbQPMnKCJu4N90NLDyF7ER gCMoDrX+hji7qleXNI8IYn1jvUXVJzmAni/q1ZINV1LIEht6CBgEA6xQtXk/vEztSv 0Xwt2TumKKccg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 10/21] mm/zsmalloc: add zpdesc_is_isolated()/zpdesc_zone() helper for zs_page_migrate() Date: Tue, 6 Aug 2024 10:22:57 +0800 Message-ID: <20240806022311.3924442-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated()/zpdesc_zone() helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 6a01e09e4beb..20881e7e42be 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -113,4 +113,15 @@ static inline void __zpdesc_set_movable(struct zpdesc = *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5fac2bb436f8..398b602a3b4d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1810,19 +1810,21 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc =3D page_zpdesc(newpage); + struct zpdesc *zpdesc =3D page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; =20 - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); =20 /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); =20 /* The page is locked, so this pointer must remain valid */ - zspage =3D get_zspage(page); + zspage =3D get_zspage(zpdesc_page(zpdesc)); pool =3D zspage->pool; =20 /* @@ -1839,30 +1841,30 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); =20 - offset =3D get_first_obj_offset(page); - s_addr =3D kmap_atomic(page); + offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr =3D zpdesc_kmap_atomic(zpdesc); =20 /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr =3D kmap_atomic(newpage); + d_addr =3D zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); =20 for (addr =3D s_addr + offset; addr < s_addr + PAGE_SIZE; addr +=3D class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(newpage, + new_obj =3D (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); =20 - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1871,14 +1873,14 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); =20 - get_page(newpage); - if (page_zone(newpage) !=3D page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) !=3D zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } =20 reset_page(page); - put_page(page); + zpdesc_put(zpdesc); =20 return MIGRATEPAGE_SUCCESS; } --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BED21BF37 for ; Tue, 6 Aug 2024 02:18:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910716; cv=none; b=HLqB3enHILyAs1m9Sbpo1QmujMFAnBc0zPKMc5UGIRcwzBnKm85MeWJrnaIubsG8pacE4Kngpuq6gj6/zsqPCzWmDrre3978rPfT1QGlDkdkAf+Bh/dxFUIGE10JzTVWwdhKWhJveAxerFsnh24qkb9+pjWHoDFb0aHKPvamMaA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910716; c=relaxed/simple; bh=lIAmwXpKjxS2Hsoj/dTwcnmyf7UY5ner31n75hmLtY4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Jz+Se64/hhU2k/WsPVRkp5Xbv1l9F90iscCE73dPBvlKrH+DlklS/PrG5PyfOmB+kv884Bkhm83mq2OqZLt0j1BJ7Bt4zKlBJuYw4pNCHNADW7sOaJTVuHPBo1RGhUzKCzy54TilsTFaW4E/gHOjD20jJhA+qzppG0OfTaskW8M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=eumQagZu; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="eumQagZu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB7A6C4AF15; Tue, 6 Aug 2024 02:18:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910715; bh=lIAmwXpKjxS2Hsoj/dTwcnmyf7UY5ner31n75hmLtY4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eumQagZuRPiE1JGpb1pdxG+/99Ht5pD7RRzlqeURY/va7iPdIX7vB4gKhr06VJQ3r UWoA8msGRp4eA8sccL2/GzO5mQxkVqzheMiDuAC3F5rJW5N+SCFZtKyAtFO+Lf67rc KTHxlNvMvXZB/J55XLBEOsY745Nro5/dQ9+M95DGwMx671Ot4J8rtSAseOVLtktMQF oFfqu8itoW6TWi2Fvx8KvB0l7TgUW74oWiWh598DDOWMLJ7jNCPpg2iwUlurfumYlt StcwxXKGxg+yrDJ9bj0qQnx/7/ncGtbXfNpvFRVea1X2Pg4QthRUrw4kec6lJhMnIZ D6Wh2QaMAul8g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 11/21] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Tue, 6 Aug 2024 10:22:58 +0800 Message-ID: <20240806022311.3924442-12-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() wich is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 398b602a3b4d..f4fc79c0bef5 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -845,12 +845,14 @@ static inline bool obj_allocated(struct zpdesc *zpdes= c, void *obj, return true; } =20 -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page =3D zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index =3D 0; + zpdesc->zspage =3D NULL; + zpdesc->next =3D NULL; reset_first_obj_offset(page); __ClearPageZsmalloc(page); } @@ -890,7 +892,7 @@ static void __free_zspage(struct zs_pool *pool, struct = size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next =3D get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1879,7 +1881,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, zpdesc_inc_zone_page_state(newzpdesc); } =20 - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); =20 return MIGRATEPAGE_SUCCESS; --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DFD6F28E0F for ; Tue, 6 Aug 2024 02:18:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910720; cv=none; b=SGoE/uS8Eh945ggiQRT3rlO6CHI39X5la19F9rVdip1/qgKVwv4ZfvH06mEXWnr4zZYkmUpGBgeGiDrX23xFJxMWnkyyfxmZ5dsYX+r7kTVUCOJZ3srVJXFu698qSPYBNp6CxKPAKI8txc8eZIdu65pWj5rSseIByyxnYL2ZA0M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910720; c=relaxed/simple; bh=aFVskGOXvp/05Uc/SGs92DOGYm3KvLJCDxXlSNzTuX4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nbWl4pZ7Y2yNa+UzYoYCNcP1CPxhYFUXTuzajvag3xDLqk0YuC2GGvJsy5SSg3g5IB3eB1oF04AZ508n4zikUAiOntFOzP+Za+GS99Visx6fM9pmBaQyKOZgS2Ohs5Hcwa8KK85/I8jTH/9woFs4G+GzTT5hbyc9YbHeEIaq4/U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cvE9YuJi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cvE9YuJi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88D72C32782; Tue, 6 Aug 2024 02:18:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910719; bh=aFVskGOXvp/05Uc/SGs92DOGYm3KvLJCDxXlSNzTuX4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cvE9YuJiftrDSE1MdCbV7FmQDTvQ0+TQ0arxnPZndZbNu/MTxGTtjGpfe109H63gU EYlozhNKkNouVY2+GUzBSvklk1lOwJte5Wbw/cdE+8jzyW43VHDfNpeanSC+8PlmFA T1z6KNjfNtMt5abA5/7hG2ehSUx5Ru2Mv+oQLyP1YuD7GTKasFewskGQUq8gJrEt/8 5VzOBU3CEAoX4BAIxOH4oznwxYA5GhplD0r91533J0skWDt1FnOyWfJlOEvCQOx7AZ jotgu/PQ420IcxYrbHzQKf4mQnk4BqKD21NVym2jYYv8egk9cSHF8LUQayTgt8GFm+ 0L1DkJwRroClw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 12/21] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Tue, 6 Aug 2024 10:22:59 +0800 Message-ID: <20240806022311.3924442-13-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 20881e7e42be..640f33b5c8bd 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -124,4 +124,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *z= pdesc) return page_zone(zpdesc_page(zpdesc)); } =20 +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index f4fc79c0bef5..4b948b84e7f3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -881,23 +881,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; =20 assert_spin_locked(&class->lock); =20 VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness !=3D ZS_INUSE_RATIO_0); =20 - next =3D page =3D get_first_page(zspage); + next =3D zpdesc =3D get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next =3D get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page =3D next; - } while (page !=3D NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next =3D get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc =3D next; + } while (zpdesc !=3D NULL); =20 cache_free_zspage(pool, zspage); =20 --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79FC3381B9 for ; Tue, 6 Aug 2024 02:18:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910723; cv=none; b=MAXVOUgpHNlCz5G8JcsBRCED/2tasTC27GRcRzLFEWbo/wa0wVQfD4jP757YbSNSJ8ZgdmPbLi3mfi+/fofbNBgFyRg3UoL+IW4IesqnB11/R0l38OKNQZw1x+OnEBoOGtvm4zdT/rDUZncyrfIV0j3+Fbc7Smi1ChC/sVGb2YE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910723; c=relaxed/simple; bh=7R27PI6t8PPGIzjMbda+XNUteaxs5t0PfeOafdtKA9c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Hh0BngyBfqLb7BdTvsmYPTptZHKv2dUztuNsKqgtq9RYaiarELWEdZNx+YIVxbXgRzKm+8NSEA6VCh8uS6KLAHto9SwkaDaUHZx1jRf4CsSEf8cOQRx0H19CaiTHsFSC5hkCH2UpL9+B1cYoy37V8wnYuiBRgw824hGy254XuRg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CRmUrjTQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CRmUrjTQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 34CA4C4AF0B; Tue, 6 Aug 2024 02:18:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910723; bh=7R27PI6t8PPGIzjMbda+XNUteaxs5t0PfeOafdtKA9c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CRmUrjTQbKnuPQmOwufiAx4cYw8y+lYQTG38+7FslMxb3MSpH7GyZODkN1/ZB59fB s8imNPAIKMRXmNKRZK0mUiaLhD+0PgbBtXwT168RS2gd2u58Rg3m104ej3wIe/snzs NjG9ikTjq5ki9Gnay0a4shcsPz1sNZb4rTFy/MQsccnSfB32pVwCaqCPwnUL4La9DS TWtv6u+zy+LKEbXrV6o32O46jbrvR2Vg73CGnzTQFa+/sr9+klLHfr9ul2ZUl55YGl dcLUtMdq/ti4ucNX5c9ViRpDob04PaaSR2Fjff3vT5pmKjMGlolcDuytY0hd9wT7Fe 85JxlEARg1HcQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 13/21] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Tue, 6 Aug 2024 10:23:00 +0800 Message-ID: <20240806022311.3924442-14-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 4b948b84e7f3..e6d1bd8969e5 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -806,15 +806,15 @@ static void obj_to_zpdesc(unsigned long obj, struct z= pdesc **zpdesc) } =20 /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_i= dx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx) { unsigned long obj; =20 - obj =3D page_to_pfn(page) << OBJ_INDEX_BITS; + obj =3D zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |=3D obj_idx & OBJ_INDEX_MASK; =20 return obj; @@ -1372,7 +1372,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(zpdesc_page(m_zpdesc), obj); + obj =3D location_to_obj(m_zpdesc, obj); record_obj(handle, obj); =20 return obj; @@ -1859,8 +1859,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj =3D (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7244AAD5E for ; Tue, 6 Aug 2024 02:18:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910727; cv=none; b=DxcTe8CutOMOMeN470ATDmJ+40DObt1RxDXP1UwkY6nN24+cBvdtgBcQVTw0oB5Fr5bPkK5234KU4LfDaay38a4BYTvty5HhFtA1+0gv58YYmWGaxFlLFbdDBe2Tr/aiimpMcOG0ev2yf/rRl2RKMjJMhV0WnlyVuv5etziTWR0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910727; c=relaxed/simple; bh=UE8t+seeM35rl2H7JTm6eRVAxobMBm5PBazGBdplkkE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uwThjgBOUsOSPZ7wTPH3e/be/RH0Gn42fG8Ro1DWa83cNVyPIZnSEdKJYnqPUOhfTGgpA9lZmWQTjL+tQ57TpbVGTMHVKbnhaG1yWsbl2jQlnr7vfhd/+MfU1ajCo9nmEpmFp66zU6uEAcsWDnI8KQz8D3CaH767ctWVKK5+gdo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QC1dagSP; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QC1dagSP" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0788FC4AF0F; Tue, 6 Aug 2024 02:18:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910727; bh=UE8t+seeM35rl2H7JTm6eRVAxobMBm5PBazGBdplkkE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QC1dagSPDYyeNnZc8QopLpLhTX37estqkQpAdmcGKNJbVDWh/qqd4LXDnB9rPkwo8 a5aYS0P0SofZMckIif6UjPr4WkZzfpHrUzYIMglACX2iFlDgqrD7MmCC1PSWaPV2XR 4s7xbWyqt0Pf2yz0mPj8FaMGg0br36C84Q2I2Sc6N0UVKDmLr9mQ91j5HduyA7RXQf AJgeqEdBcb0yfzmxjDNiltYEvbGvoZFOfiIJiYLj8FzzUrjCgp90Qm3abdIcIP1/U7 vHzJUWdv9NoTlwyaXv/hHPPUokBHRBunGFV9102zXdOcI3xZq4MGWE5tUhw0uLYS/A Koq9n6XXYdWFA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 14/21] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Tue, 6 Aug 2024 10:23:01 +0800 Message-ID: <20240806022311.3924442-15-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e6d1bd8969e5..c8a533b516ea 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1616,14 +1616,14 @@ static void migrate_zspage(struct zs_pool *pool, st= ruct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx =3D 0; - struct page *s_page =3D get_first_page(src_zspage); + struct zpdesc *s_zpdesc =3D get_first_zpdesc(src_zspage); struct size_class *class =3D pool->size_class[src_zspage->class]; =20 while (1) { - handle =3D find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle =3D find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page =3D get_next_page(s_page); - if (!s_page) + s_zpdesc =3D get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx =3D 0; continue; --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3896136AF2 for ; Tue, 6 Aug 2024 02:18:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910731; cv=none; b=QR1gP1binUxPkF6PbgHJkPbpTFP7LxPk6Fb21thZIC3fts42tjsHMszLY0hfF/7AkEdu5hXC8nsXIlkEd86EVo7I3aR3QPj5pnJy6COqLdDqK25IXUftp1SFoylvvCo9o6M1hlcqlQm3x/UUyCrkLlxg3VuDchSIUB/Rbaz0Iqg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910731; c=relaxed/simple; bh=ogZXEVMMW4vu3mmfL4ddTVfOtlodq8fb2Ek8tntegjk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=C7HBcSErssquljakW/ES1G+1ScPEO5qQvA5VRTR/HQRWVA+bN48o5kBOuvC3gu0OZ/Eeks1OLzKeidzUFLBba7gkfT9qH0AL5y23DGwlgacTgZshFALobrbwPvPE+XeS221hr52x/Sl7hPAnGwqvyKdJ6SuFWm7voGichZlSpNk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DWgzmYgv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DWgzmYgv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7538C4AF0C; Tue, 6 Aug 2024 02:18:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910730; bh=ogZXEVMMW4vu3mmfL4ddTVfOtlodq8fb2Ek8tntegjk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DWgzmYgv43cIb1Yiwb7X3nxE/pvVIH2czvwhcvBxw+2XWnAWknGn5a5R2zBnvcHX2 AN2G5vS7GEbGmHtuuwf4IQ+El+/Yo9rRX+lAukVEUWkiDn5Bk0XTE61vlWrhSNXtsw h0RBltoj7fNzr7nTdqweZjWmT4+rVA5Ye+/YKu7LpAc6OM+i0DtFuwv/Xjziz4XPMJ yykfc6l+Byh1PYyMscKofhgssFWWTA1a90SfMRPHgJDCsBwDIKx9Y7d+Jtld/GETLC eaOgOHDsyvsotZXHSHWJd33trvlCMq2Ie47dpzUtR/Gs3xRP3Vg2pqkk72SXbUh0fa OCih3mDoloK4w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 15/21] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Tue, 6 Aug 2024 10:23:02 +0800 Message-ID: <20240806022311.3924442-16-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index c8a533b516ea..7ae98d048590 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -759,9 +759,9 @@ static int fix_fullness_group(struct size_class *class,= struct zspage *zspage) return newfg; } =20 -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage =3D (struct zspage *)page_private(page); + struct zspage *zspage =3D zpdesc->zspage; =20 BUG_ON(zspage->magic !=3D ZSPAGE_MAGIC); return zspage; @@ -769,7 +769,7 @@ static struct zspage *get_zspage(struct page *page) =20 static struct page *get_next_page(struct page *page) { - struct zspage *zspage =3D get_zspage(page); + struct zspage *zspage =3D get_zspage(page_zpdesc(page)); =20 if (unlikely(ZsHugePage(zspage))) return NULL; @@ -779,7 +779,7 @@ static struct page *get_next_page(struct page *page) =20 static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage =3D get_zspage(zpdesc); =20 if (unlikely(ZsHugePage(zspage))) return NULL; @@ -829,7 +829,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc,= void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage =3D get_zspage(zpdesc); =20 if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1245,7 +1245,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); =20 /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1295,7 +1295,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) =20 obj =3D handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 @@ -1459,7 +1459,7 @@ static void obj_free(int class_size, unsigned long ob= j) =20 obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset =3D offset_in_page(class_size * f_objidx); - zspage =3D get_zspage(zpdesc_page(f_zpdesc)); + zspage =3D get_zspage(f_zpdesc); =20 vaddr =3D zpdesc_kmap_atomic(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); @@ -1493,7 +1493,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage =3D get_zspage(zpdesc_page(f_zpdesc)); + zspage =3D get_zspage(f_zpdesc); class =3D zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1826,7 +1826,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); =20 /* The page is locked, so this pointer must remain valid */ - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); pool =3D zspage->pool; =20 /* --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BEFFF54656 for ; Tue, 6 Aug 2024 02:18:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910734; cv=none; b=HmHFf2soARxvA/NxvzHbWzLVBj8vDneJNHwd0rGdxcpUV5gcOr6OBQZgvqpCrDeI8tYcovx4srBfJB9UDiUyb3XO+4OC8Cu7pzcdTcZfTjNmgsHkawgzutg8hZtqe8fBwxsEJf1LmMrsEhi/a+cXw3mSWuMVX8pwGq9XKmHEmeg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910734; c=relaxed/simple; bh=dtYWibYOHtP7k2L7bRrRJ0o8M+qJDQY9usNXaIwbwBY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=spVTFBDfhfuyiGXS3hO7f2IqK5Yzn1XYb5J06tA1mxeMvKS+SZojex+3JbTRVTszKLc/b53xEfA/IYe+uaeXEJeeZmIrdCXrBXsOp1ZHbhhgBFipvFR1sQ6clppzSefydmRCw83AzOp+X+g+tsvIXvc80l5P3EbaTXhCmk7nxCQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lO72cahT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lO72cahT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79ED9C32782; Tue, 6 Aug 2024 02:18:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910734; bh=dtYWibYOHtP7k2L7bRrRJ0o8M+qJDQY9usNXaIwbwBY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lO72cahTLoWkgC7Fa1GVC+0hzqrLY7guqKdalHp+19GYloDBXVGgnVMkA6OEbgxLz I0omHi39eDE1hwAIsmiyNC3lQerM4A6pVo0uXUNxZeUyQTI5Rll9kQOReJhxdbTnse 3riuCQoqgpy1oFQmmD0ZEN/HioRasC0sVhguV0gCV6kwUReZeO65WOweOVH7DebCig buBiZCpWOGWi7uSflgP1duhhn/JiZAMVXhBSAcntuFNtApPZZlo5fb18wvMFe1NzvE xLJCqvCx1jHwdpI7Qb+YpqvafIVMoIj7yj4TEfqqTAuyAcFqW0znvZNr7xRenq/Hh5 ERruHjKKT35BA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 16/21] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Tue, 6 Aug 2024 10:23:03 +0800 Message-ID: <20240806022311.3924442-17-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs: get_next_page()/get_first_page()/is_first_page(). Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7ae98d048590..1c53a58e6473 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -448,11 +448,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area= ) =3D { .lock =3D INIT_LOCAL_LOCK(lock), }; =20 -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static inline bool is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -469,14 +464,6 @@ static inline void mod_zspage_inuse(struct zspage *zsp= age, int val) zspage->inuse +=3D val; } =20 -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page =3D zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc =3D zspage->first_zpdesc; @@ -767,16 +754,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } =20 -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage =3D get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage =3D get_zspage(zpdesc); @@ -1950,13 +1927,13 @@ static void init_deferred_free(struct zs_pool *pool) =20 static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page =3D get_first_page(zspage); + struct zpdesc *zpdesc =3D get_first_zpdesc(zspage); =20 do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page =3D get_next_page(page)) !=3D NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F66538DC7 for ; Tue, 6 Aug 2024 02:18:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910738; cv=none; b=Mfd9DVY66lzIaiCstOIDWp4z2br3zynaqEAjiZjfapIifNku1FPqniHMsWYR2yO5ztIdOIWcwuNJBDIQFgRNlHX6iusNRXZCnh5KQr+zoP7Fs6e2blB8GzE2IBi/SaQecigXjfDJfqaPRUyziQo6OkV4RXn7ydMTxybA6WIPxbs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910738; c=relaxed/simple; bh=xBRbiPI9Fj5vFVKbb/yDx6Gk1e8RF8irjBVrIsL9n4o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Lq+5bsSEpiu/BferpQRmstCztoEWDTNsDRqvWqxnMpUwwIO2M7wOkKIDU3/Zc6wZEc8SwmzuV29oPecwRyBm6KZWtGmQQ+AocrMl8wiqt420vZP0HhjliXXtnMnasfLJbP4GNIk6lm0fer2mWTQYvjxm52cKGbJGse3p2woHRuQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BaWMFlpO; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BaWMFlpO" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B6A3C4AF0C; Tue, 6 Aug 2024 02:18:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910738; bh=xBRbiPI9Fj5vFVKbb/yDx6Gk1e8RF8irjBVrIsL9n4o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BaWMFlpOOvQxiuUrEdt0K4kqxnxvptrUSrrBTxZQhjnf0InUS+rHp/YA9L18z0U/j o5GFJeiz6YFu/eejUX1D3OwapL+PjeF8zf/2vZEs8ccrA84UfOySV0QI9jatpp15Az xz8nX1jKA+eAf3fiP9Br2fwstLZFEXiPeFHtLhgnOBmcy9ACATYNVDLet8rtdgtY2p fwnDQi+SpyOlv71NA4smr8Y+8jIBLZ8Pa3+WRph3nNejjXqF9EnM9X3i52KFE/XAX7 PJ0RCylZSMz2WHISlIgceqb5JLJl4q0/+QKv7mFavS/oXXL4FUDFILThvNu7v/30Tu xYIIdvUjvEPvQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 17/21] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Tue, 6 Aug 2024 10:23:04 +0800 Message-ID: <20240806022311.3924442-18-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1c53a58e6473..e2cfee57a39a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -474,26 +474,26 @@ static struct zpdesc *get_first_zpdesc(struct zspage = *zspage) =20 #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff =20 -static inline void reset_first_obj_offset(struct page *page) +static inline void reset_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - page->page_type |=3D FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + zpdesc->first_obj_offset |=3D FIRST_OBJ_PAGE_TYPE_MASK; } =20 -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } =20 -static inline void set_first_obj_offset(struct page *page, unsigned int of= fset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned in= t offset) { /* With 16 bit available, we can support offsets into 64 KiB pages. */ BUILD_BUG_ON(PAGE_SIZE > SZ_64K); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &=3D ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |=3D offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &=3D ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |=3D offset & FIRST_OBJ_PAGE_TYPE_MASK; } =20 static inline unsigned int get_freeobj(struct zspage *zspage) @@ -830,7 +830,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; - reset_first_obj_offset(page); + reset_first_obj_offset(zpdesc); __ClearPageZsmalloc(page); } =20 @@ -914,7 +914,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) struct link_free *link; void *vaddr; =20 - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); =20 vaddr =3D zpdesc_kmap_atomic(zpdesc); link =3D (struct link_free *)vaddr + off / sizeof(*link); @@ -1569,7 +1569,7 @@ static unsigned long find_alloced_obj(struct size_cla= ss *class, unsigned long handle =3D 0; void *addr =3D zpdesc_kmap_atomic(zpdesc); =20 - offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + offset =3D get_first_obj_offset(zpdesc); offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { @@ -1764,8 +1764,8 @@ static void replace_sub_page(struct size_class *class= , struct zspage *zspage, } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 create_page_chain(class, zspage, zpdescs); - first_obj_offset =3D get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset =3D get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle =3D oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1820,7 +1820,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); =20 - offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + offset =3D get_first_obj_offset(zpdesc); s_addr =3D zpdesc_kmap_atomic(zpdesc); =20 /* --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B22238DC7 for ; Tue, 6 Aug 2024 02:19:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910742; cv=none; b=kGWIEsndlz5TABRoPgkbJ6KVrKeehYK+Aipl32IWYFp5f4iTren7JZ1ma8fNbiLYbk9tl7h+w6u2Z4qnMHM+Z6aLoqZe7gvHtUP+jl740Iy0RE9j/InRkWU33oOCiSJBIFMQKGDWb1KV0bkXMdb/ONubOxkcUNegpKtLVk8rmKE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910742; c=relaxed/simple; bh=gO5yNLlLwsCfZqvTP2RVzms7OfZiyAMCAW6p5U5JaUg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=o9rdlWT6OD7tMTM33EAPKOen5y+waVpz/18wQr2LBH78YG4PMyizVnnYo3CfOtOGh9ajgffnIfLgNQhEG3HF6lRtNikduWMMIqF71sqPSXEeTbqGYisiT9cBkIonczDrU7W6L/2Onw2GpbtsTobzdYCvXE0OroH5JgKxR5jmEds= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GJ4wy4eK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GJ4wy4eK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1C9B0C4AF0C; Tue, 6 Aug 2024 02:18:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910742; bh=gO5yNLlLwsCfZqvTP2RVzms7OfZiyAMCAW6p5U5JaUg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GJ4wy4eK4qW7kRgbBTYCXZtWl8AumXoo42ILqjgP/YgqVhQuJCoip2c2h9Nz7JxV/ Y+POjNOqPqoYGnzttSb8PWI6rk2905cz6tpbM6fPqqHkcZAESSGqoduGWf9qcJC5uy fdj1He4fUsZ+XfS2s6kIB6S2Vcb7Bxox8DH2QYPvI1HsZZTySzchbOkR4W7LNgBlzA VMUbijwI55mKeVswvC6FPobdGxl25MGMZkfiag52ZV5yHMqh0c+r1IyRmuyzBvSkot lmUQeCUa0om8RhsQedOxNuawS8lii5OvwvYr7I3/LzmSnTIKMnm+jenS/B2PwIE8So gKZLae5Vm8ryQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 18/21] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Tue, 6 Aug 2024 10:23:05 +0800 Message-ID: <20240806022311.3924442-19-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 640f33b5c8bd..1ab47faed49b 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -114,6 +114,11 @@ static inline void __zpdesc_set_movable(struct zpdesc = *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } =20 +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e2cfee57a39a..7c7432a00a07 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -826,7 +826,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page =3D zpdesc_page(zpdesc); =20 - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C1646F076 for ; Tue, 6 Aug 2024 02:19:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910746; cv=none; b=W0iuWJfjpt+elQX9CYP+eQkCFDJm1u77WgHvmL7rmTadHhlQvmgJWWq1jC+XkHaj6onFUBqOtwXYZD4BxOxscZ9BCG8Ig9xCdmv0U6GaPIwEm+T/hBOgtDJZRuzCZPo7uRm0ES+6d5OB5MuP+hDd2Kzb2jWVG4thH8+3A8Grx/8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910746; c=relaxed/simple; bh=oL5cTZYBz5hcMd4ibMjrzRAOxUIAWeTC3GGP5qzznjU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KYjvVZxGB0MNS/L0bZCAG/i3uCPhk99ClwXWOb9bVhFW3+EbKOcpwd2tgbW3Nna5haPAm0IbsahJ6MXDb5OkqjUFrheN2JaMh9rteyC91jyQox8dWTDQXES1EkMX4FQKcoGXbJVJqO5tM5//azmFgSawv/YW2mNcIe3vOg4hfHk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HznYQbSG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HznYQbSG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC641C4AF11; Tue, 6 Aug 2024 02:19:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910745; bh=oL5cTZYBz5hcMd4ibMjrzRAOxUIAWeTC3GGP5qzznjU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HznYQbSGB5gQ2kI4wcaUCOnPgGomh9VJ4uFtU8YG1LSpwabhCQ5mxFNi8WpsZ6exU K2rMxdEaj7xFao9BCCnDrHgaXr9hQ4jYCcgFfLBKdAml2YEKJ69MLvRMrXZQgAQxqX RONxiHPQHIUEX/m1A/ofr7XuFO/YWOznmfR9zypSe7GLU6pMZS8Xx3aN/pypoyEzh6 kLyjt2xVtNmiSUqQtoB2AjBnSdr4Dcc/epjg/GoMU4SEg/Xv2KyIEHTI6BaDEVxS8k gT+v+/dqRQT+ymOFOQz2ckMqoZBnAoBylS3mn63NoXSeKUbYwjnzgDGBG/r5ZOElwK iG0VmI/vVGPxQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 19/21] mm/zsmalloc: introduce __zpdesc_clear/set_zsmalloc() Date: Tue, 6 Aug 2024 10:23:06 +0800 Message-ID: <20240806022311.3924442-20-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use them in callers. Signed-off-by: Alex Shi --- mm/zpdesc.h | 10 ++++++++++ mm/zsmalloc.c | 8 ++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 1ab47faed49b..d0a1af0ee424 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -119,6 +119,16 @@ static inline void __zpdesc_clear_movable(struct zpdes= c *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } =20 +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7c7432a00a07..0b318db894a8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -831,7 +831,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; reset_first_obj_offset(zpdesc); - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } =20 static int trylock_zspage(struct zspage *zspage) @@ -1004,13 +1004,13 @@ static struct zspage *alloc_zspage(struct zs_pool *= pool, if (!zpdesc) { while (--i >=3D 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); __free_page(zpdesc_page(zpdescs[i])); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); =20 zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] =3D zpdesc; @@ -1800,7 +1800,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); =20 /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); =20 /* The page is locked, so this pointer must remain valid */ zspage =3D get_zspage(zpdesc); --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E59B0762C1 for ; Tue, 6 Aug 2024 02:19:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910750; cv=none; b=BAbBGqZRUtTmBbXYmC/Fc6iQ9gX+y0qrz+2bi4ZjLXTzjuVxrg05cs/2C1Kl2KqN6qV8/KyeIwvNK1n5qTDK5y6ZhiF2M2iF8D/O0KEaphAX4MyYAmRqUsrzIDG+EJfrPuj5efgw83s1truR1Y3Vkk+zEkO40WkU9Ux1/oEfwM0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910750; c=relaxed/simple; bh=SIgiyPCW8QCBZtqr5fnMzvbajnde7CasC3gp99UMX7E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=StnfxI8cPYVc7qR5dx7JMHZY3RBYU/p2sjBEA6jxgKG9Bp6pKYYb8q8EOTp8KChZr4kvY1cZizkGTfOGAB0U/EG2VgXp6C627pppOo8c9vCvgGqtVVjLLyVRytN+S8hATlQssj09X2DJlJe0iNS6GolQtExCBaJH52Iqv0IBwxc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=eogsWgAJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="eogsWgAJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E4D5C4AF0C; Tue, 6 Aug 2024 02:19:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910749; bh=SIgiyPCW8QCBZtqr5fnMzvbajnde7CasC3gp99UMX7E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eogsWgAJOVMVgMm+88DiD7XkroE1wr8TVvpEQjp6NmnLl30eBFsFWj/nQJK+I8c5I jUhC6+ElxHccYMtnUAdfDvOCrIXJ1xMAi+HSzAgR5yAn1y686RgXFAQo4iYVVx44kJ dBNVnqwpiPqTyTKh3OA5LoUrYXaseNQYH3N0a+Z8E9/IdcZJ72fPC9/LMMjOC5NpMH h46X+x/17nvesgtbZ3mxjDf/f1XGF2Wp8TGFIPucQKhRAFPIUBJxVLCHLlEozMXN/2 yHKU2MvKQY37DOnJnxKAUQXYl65JxIApCyH3Cvfj9VuUr49mJFH14C+pH9E5/7j6Ly xUWeJVNSWL3aw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 20/21] mm/zsmalloc: introduce zpdesc_clear_first() helper Date: Tue, 6 Aug 2024 10:23:07 +0800 Message-ID: <20240806022311.3924442-21-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Like the zpdesc_set_first(), introduce zpdesc_clear_first() helper for ClearPagePrivate(), then clean up a 'struct page' usage in reset_zpdesc(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 0b318db894a8..e92451c6b69c 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -253,6 +253,11 @@ static inline void zpdesc_set_first(struct zpdesc *zpd= esc) SetPagePrivate(zpdesc_page(zpdesc)); } =20 +static inline void zpdesc_clear_first(struct zpdesc *zpdesc) +{ + ClearPagePrivate(zpdesc_page(zpdesc)); +} + static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) { inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); @@ -824,10 +829,8 @@ static inline bool obj_allocated(struct zpdesc *zpdesc= , void *obj, =20 static void reset_zpdesc(struct zpdesc *zpdesc) { - struct page *page =3D zpdesc_page(zpdesc); - __zpdesc_clear_movable(zpdesc); - ClearPagePrivate(page); + zpdesc_clear_first(zpdesc); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; reset_first_obj_offset(zpdesc); --=20 2.43.0 From nobody Fri Dec 19 19:16:35 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86DB93C485 for ; Tue, 6 Aug 2024 02:19:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910753; cv=none; b=akyqQGm9kEtPBLmvwoYfcnINZcJL2ENbaFHFpXQ711R6biGjJU8hdgBd42l4vjD2L7PZKIndIbhN6I7TXVmmIjxkYBjSM2NL7JeuvGONFLKmtr6M5jFvfYPFKkM1FD7eOl1pcPsQi5kVq4oEi8NekFj0bpPfkRMAQ74bNLOLttI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722910753; c=relaxed/simple; bh=8EJ/o5yp3ZapndwWQeMuiNEuMIoUNdxMr4p7pDAhjMQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gZ+IKPFM1qUlURlBTroJwiIet6GCb2zBZ+TJ361tCIT4x10d7ttKBA00QmXtE7RYUX4w87YijyM4g3HGiMjvlGTgkvF/DymMGz0VnV4P52aIAq+Obj6A6IXZ2PFX109FwqdL1XTCvxwgtg5uC3pu2C52hLwH3dbgq38xOglzYVw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fHD0Nind; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fHD0Nind" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3BEC0C4AF11; Tue, 6 Aug 2024 02:19:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722910753; bh=8EJ/o5yp3ZapndwWQeMuiNEuMIoUNdxMr4p7pDAhjMQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fHD0NindhMQW6QUkJ2u9oDL3HNMi/H8qnJWxEJrGbwtH1YlIb2amxcYwVifocYTdU S4LnhS1E+vxEX5IUdJBshGvy/4Dkn7bBOdoq6v4vqZpYCkKdW+tzpO/HAsF430o7YK iCQpenKaRCiHhk+pkrD9d6b1W7PAfmGD87rWCRlLrvfE8NInQwIObWeiw3kX9eifcF H/PzSqlHXVcXLUFXOgsJxQfLjOW1W40R4RhUgtvXZDEalOqqkVePdXSRBv33b1YYmV i6P2PWItaOhhBbbw2/aX1FW7p0MeXuJp6GEMv+4xwNRv0u7yh3UTkHwfIBgu1KrIoI QmZak2SKy6ucg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v5 21/21] mm/zsmalloc: update comments for page->zpdesc changes Date: Tue, 6 Aug 2024 10:23:08 +0800 Message-ID: <20240806022311.3924442-22-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240806022311.3924442-1-alexs@kernel.org> References: <20240806022143.3924396-1-alexs@kernel.org> <20240806022311.3924442-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi After the page to zpdesc conversion, there still left few comments or function named with page not zpdesc, let's update the comments and rename function create_page_chain() as create_zpdesc_chain(). Signed-off-by: Alex Shi --- mm/zsmalloc.c | 47 ++++++++++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 21 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e92451c6b69c..a3d9431d29ec 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -17,14 +17,16 @@ * * Usage of struct zpdesc fields: * zpdesc->zspage: points to zspage - * zpdesc->next: links together all component pages of a zspage + * zpdesc->next: links together all component zpdescs of a zspage * For the huge page, this is always 0, so we use this field * to store handle. * zpdesc->first_obj_offset: PG_zsmalloc, lower 16 bit locate the first * object offset in a subpage of a zspage * * Usage of struct zpdesc(page) flags: - * PG_private: identifies the first component page + * PG_private: identifies the first component zpdesc + * PG_lock: lock all component zpdescs for a zspage free, serialize with + * migration */ =20 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -191,7 +193,10 @@ struct size_class { */ int size; int objs_per_zspage; - /* Number of PAGE_SIZE sized pages to combine to form a 'zspage' */ + /* + * Number of PAGE_SIZE sized zpdescs/pages to combine to + * form a 'zspage' + */ int pages_per_zspage; =20 unsigned int index; @@ -893,7 +898,7 @@ static void free_zspage(struct zs_pool *pool, struct si= ze_class *class, =20 /* * Since zs_free couldn't be sleepable, this function cannot call - * lock_page. The page locks trylock_zspage got will be released + * lock_page. The zpdesc locks trylock_zspage got will be released * by __free_zspage. */ if (!trylock_zspage(zspage)) { @@ -950,7 +955,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) set_freeobj(zspage, 0); } =20 -static void create_page_chain(struct size_class *class, struct zspage *zsp= age, +static void create_zpdesc_chain(struct size_class *class, struct zspage *z= spage, struct zpdesc *zpdescs[]) { int i; @@ -959,9 +964,9 @@ static void create_page_chain(struct size_class *class,= struct zspage *zspage, int nr_zpdescs =3D class->pages_per_zspage; =20 /* - * Allocate individual pages and link them together as: - * 1. all pages are linked together using zpdesc->next - * 2. each sub-page point to zspage using zpdesc->zspage + * Allocate individual zpdescs and link them together as: + * 1. all zpdescs are linked together using zpdesc->next + * 2. each sub-zpdesc point to zspage using zpdesc->zspage * * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). @@ -1019,7 +1024,7 @@ static struct zspage *alloc_zspage(struct zs_pool *po= ol, zpdescs[i] =3D zpdesc; } =20 - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool =3D pool; zspage->class =3D class->index; @@ -1346,7 +1351,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, /* record handle in the header of allocated chunk */ link->handle =3D handle | OBJ_ALLOCATED_TAG; else - /* record handle to page->index */ + /* record handle to zpdesc->handle */ zspage->first_zpdesc->handle =3D handle | OBJ_ALLOCATED_TAG; =20 kunmap_atomic(vaddr); @@ -1679,19 +1684,19 @@ static int putback_zspage(struct size_class *class,= struct zspage *zspage) #ifdef CONFIG_COMPACTION /* * To prevent zspage destroy during migration, zspage freeing should - * hold locks of all pages in the zspage. + * hold locks of all component zpdesc in the zspage. */ static void lock_zspage(struct zspage *zspage) { struct zpdesc *curr_zpdesc, *zpdesc; =20 /* - * Pages we haven't locked yet can be migrated off the list while we're + * Zpdesc we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock - * may no longer belong to the zspage. This means that we may wait for - * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * lock each zpdesc under migrate_read_lock(). Otherwise, the zpdesc we + * lock may no longer belong to the zspage. This means that we may wait + * for the wrong zpdesc to unlock, so we must take a reference to the + * zpdesc prior to waiting for it to unlock outside migrate_read_lock(). */ while (1) { migrate_read_lock(zspage); @@ -1766,7 +1771,7 @@ static void replace_sub_page(struct size_class *class= , struct zspage *zspage, idx++; } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); first_obj_offset =3D get_first_obj_offset(oldzpdesc); set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) @@ -1777,8 +1782,8 @@ static void replace_sub_page(struct size_class *class= , struct zspage *zspage, static bool zs_page_isolate(struct page *page, isolate_mode_t mode) { /* - * Page is locked so zspage couldn't be destroyed. For detail, look at - * lock_zspage in free_zspage. + * Page/zpdesc is locked so zspage couldn't be destroyed. For detail, + * look at lock_zspage in free_zspage. */ VM_BUG_ON_PAGE(PageIsolated(page), page); =20 @@ -1805,7 +1810,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, /* We're committed, tell the world that this is a Zsmalloc page. */ __zpdesc_set_zsmalloc(newzpdesc); =20 - /* The page is locked, so this pointer must remain valid */ + /* The zpdesc/page is locked, so this pointer must remain valid */ zspage =3D get_zspage(zpdesc); pool =3D zspage->pool; =20 @@ -1878,7 +1883,7 @@ static const struct movable_operations zsmalloc_mops = =3D { }; =20 /* - * Caller should hold page_lock of all pages in the zspage + * Caller should hold zpdesc locks of all in the zspage * In here, we cannot use zspage meta data. */ static void async_free_zspage(struct work_struct *work) --=20 2.43.0