From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 787EA21373 for ; Mon, 29 Jul 2024 11:20:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252021; cv=none; b=n2rivR5JA/1JbW0dt4qsgZxcEjqG/PFnyySppm8TDl/QAV5pdvJ2JADxJ6DFRTAQthJINaZbrBWC0+KDLfuCrPV1J2NGYMpnfjPVhGG33ZGMKeKHckHpPE/jMc1G2cuK2eri3miorIOFUDAoQQjlDw0mRe8eKhGCcL8watSzSsU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252021; c=relaxed/simple; bh=ZMy0HVEu+XzzoekAuFVehV3NQWY5KpoqAmg4pIiNEm0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RUuwLWDTi7B2FNbzd0AurZN4YYC4jGYy17LYnJzS20OD+ykf0mrX3+GQRc2FZMh6ThIbKFFHK39fPOreDFqCWEYHu5aoAS5TPwKf8BCsBRUggONnN8j1RaqXotImZzI6Uq9LZEM0WqZVK0KrqTF4oTnabFffReMniaJN+gigo9U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nBwIehPf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nBwIehPf" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 51CFDC32786; Mon, 29 Jul 2024 11:20:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252021; bh=ZMy0HVEu+XzzoekAuFVehV3NQWY5KpoqAmg4pIiNEm0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nBwIehPfLpmA1kyjsm46mnAqNpzYgrAXYCZIcwst0anGWvE9xf0m0R8uFaRtIPafu t/td39zcrbnZCdeZ8Hng4wHXf3YwDD9eNyWYe7W/Im049pUfREJaRBDouTeuse56Oz 3mv1N7T6lkm9KCFwe/zaANt0mHEKazgD2gaWuOI+J0vIETg6lJKpHz3LCbsGPlWfnp XXVMKZHwtpMB3p6VUEVxW3PWo4gfcqwMrF5lSlgTSSYkDC1C5g1FF+UxfKElPkb7Fe dsSS/yGrJCevPh2RsZ/SVQbLQ1CIU9NGDRhATvT33AuoL53PAg2gwql7mm17WwLswY 6VhUZbMkF/G+g== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 01/22] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Mon, 29 Jul 2024 19:25:13 +0800 Message-ID: <20240729112534.3416707-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi The 1st patch introduces new memory decriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). And keep the memcg_data member, since as Yosry pointed out: "When the pages are freed, put_page() -> folio_put() -> __folio_put() will = call mem_cgroup_uncharge(). The latter will call folio_memcg() (which reads folio->memcg_data) to figure out if uncharging needs to be done. There are also other similar code paths that will check folio->memcg_data. It is currently expected to be present for all folios. So until we have custom code paths per-folio type for allocation/freeing/etc, we need to keep folio->memcg_data present and properly initialized." Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 21 ++++++++-------- 2 files changed, 76 insertions(+), 11 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..2dbef231f616 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory, now is for zsmalloc + * @flags: Page flags, PG_private: identifies the first component page + * @lru: Indirectly used by page migration + * @mops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Pointer to zspage in zsmalloc + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + struct movable_operations *mops; + union { + /* Next zpdescs in a zspage in zsmalloc zpool */ + struct zpdesc *next; + /* For huge zspage in zsmalloc zpool */ + unsigned long handle; + }; + struct zspage *zspage; + unsigned long _zp_pad_1; +#ifdef CONFIG_MEMCG + unsigned long memcg_data; +#endif +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) =3D=3D offsetof(struct zpdesc, zp= )) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, mops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +#ifdef CONFIG_MEMCG +ZPDESC_MATCH(memcg_data, memcg_data); +#endif +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <=3D sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d6581ab7c07..a532851025f9 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -13,20 +13,18 @@ =20 /* * Following is how we use various fields and flags of underlying - * struct page(s) to form a zspage. + * struct zpdesc(page) to form a zspage. * - * Usage of struct page fields: - * page->private: points to zspage - * page->index: links together all component pages of a zspage + * Usage of struct zpdesc fields: + * zpdesc->zspage: points to zspage + * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. * page->page_type: PG_zsmalloc, lower 16 bit locate the first object * offset in a subpage of a zspage * - * Usage of struct page flags: + * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page - * PG_owner_priv_1: identifies the huge component page - * */ =20 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -64,6 +62,7 @@ #include #include #include +#include "zpdesc.h" =20 #define ZSPAGE_MAGIC 0x58 =20 @@ -253,7 +252,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -448,7 +447,7 @@ static inline void mod_zspage_inuse(struct zspage *zspa= ge, int val) =20 static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page =3D zspage->first_page; + struct page *first_page =3D zpdesc_page(zspage->first_zpdesc); =20 VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -948,7 +947,7 @@ static void create_page_chain(struct size_class *class,= struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index =3D 0; if (i =3D=3D 0) { - zspage->first_page =3D page; + zspage->first_zpdesc =3D page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage =3D=3D 1 && class->pages_per_zspage =3D=3D 1)) @@ -1324,7 +1323,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle =3D handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index =3D handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle =3D handle | OBJ_ALLOCATED_TAG; =20 kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8652F146015 for ; Mon, 29 Jul 2024 11:20:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252024; cv=none; b=dLUcB2JxBY/acNYuVAYRMifnakfDSN7kefaQVWyG8MDzowONr22n9YRRJQxa+CIYG6SJOfSTOaaGn0YBB10H2c59HTxXAfrykbofCFC9XVXHQdJxYDRFVrGzazaG0TOg1FjPkJXRIS88BtztOXCfXWRxKRrLEG4oHvtbq2Sg4B4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252024; c=relaxed/simple; bh=L8DShu1UmY1Sg2Z983VpR54BaIaM2jEwz6eoX7cilko=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gWc1qqDFVy4n3eqXreA61ZkK0/u6hODKpW0c0M4qr8Cu+/Z/XSsLAwoydUktu03nzBhFFM6eZPzYIX3xpAd/PPqVNGGT7Go0oqD7D8Z8PrSSXNt8Ll8lg6wKHhsDGlkXCjL3NR04VBdr1vlylamMbQReiHbEoCjk3fCiYqVTzsw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SbdRtdLa; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SbdRtdLa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 93387C4AF0A; Mon, 29 Jul 2024 11:20:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252024; bh=L8DShu1UmY1Sg2Z983VpR54BaIaM2jEwz6eoX7cilko=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SbdRtdLaduX5nsjwmdcLwtIXUdqB9/5/aj+4DgJZnt9EekflUIzv6SzENAzlwRCi9 bxzItDWLB6yhsPVpdvv8mnTXpf7D5SjuYrX9Tnl3TOBe8LwXBhKHftAIyo0EYV8hWr mnRISrYNV++OhSkCDDIhUuSDsFULCjGdsn4onyrDXtrhUBIN9bsSzY30N5F83xePIT 9SkVVPfAHjX/xlHXgxuaCPMA3feETv5Qa/fdcUUSBq9HHUpJ/1G3kLAROE9f+2yH/c D2RqzCjcFi4ArUlsgDblEBw56WCu1A2XmrSFk8+Z4f2QzAmKm7olzai1YNIquJJJB0 fcojyZZx5zzUA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 02/22] mm/zsmalloc: use zpdesc in trylock_zspage/lock_zspage Date: Mon, 29 Jul 2024 19:25:14 +0800 Message-ID: <20240729112534.3416707-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi To use zpdesc in trylock_zspage/lock_zspage funcs, we add couple of helpers: zpdesc_lock/zpdesc_unlock/zpdesc_trylock/zpdesc_wait_locked and zpdesc_get/zpdesc_put for this purpose. Here we use the folio series func in guts for 2 reasons, one zswap.zpool only get single page, and use folio could save some compound_head checking; two, folio_put could bypass devmap checking that we don't need. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 30 ++++++++++++++++++++++++ mm/zsmalloc.c | 64 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 21 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 2dbef231f616..3b04197cec9d 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -63,4 +63,34 @@ static_assert(sizeof(struct zpdesc) <=3D sizeof(struct p= age)); const struct page *: (const struct zpdesc *)(p), \ struct page *: (struct zpdesc *)(p))) =20 +static inline void zpdesc_lock(struct zpdesc *zpdesc) +{ + folio_lock(zpdesc_folio(zpdesc)); +} + +static inline bool zpdesc_trylock(struct zpdesc *zpdesc) +{ + return folio_trylock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_unlock(struct zpdesc *zpdesc) +{ + folio_unlock(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_wait_locked(struct zpdesc *zpdesc) +{ + folio_wait_locked(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_get(struct zpdesc *zpdesc) +{ + folio_get(zpdesc_folio(zpdesc)); +} + +static inline void zpdesc_put(struct zpdesc *zpdesc) +{ + folio_put(zpdesc_folio(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a532851025f9..243677a9c6d2 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -433,13 +433,17 @@ static __maybe_unused int is_first_page(struct page *= page) return PagePrivate(page); } =20 +static int is_first_zpdesc(struct zpdesc *zpdesc) +{ + return PagePrivate(zpdesc_page(zpdesc)); +} + /* Protected by class->lock */ static inline int get_zspage_inuse(struct zspage *zspage) { return zspage->inuse; } =20 - static inline void mod_zspage_inuse(struct zspage *zspage, int val) { zspage->inuse +=3D val; @@ -453,6 +457,14 @@ static inline struct page *get_first_page(struct zspag= e *zspage) return first_page; } =20 +static struct zpdesc *get_first_zpdesc(struct zspage *zspage) +{ + struct zpdesc *first_zpdesc =3D zspage->first_zpdesc; + + VM_BUG_ON_PAGE(!is_first_zpdesc(first_zpdesc), zpdesc_page(first_zpdesc)); + return first_zpdesc; +} + #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff =20 static inline void reset_first_obj_offset(struct page *page) @@ -745,6 +757,16 @@ static struct page *get_next_page(struct page *page) return (struct page *)page->index; } =20 +static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) +{ + struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + + if (unlikely(ZsHugePage(zspage))) + return NULL; + + return zpdesc->next; +} + /** * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value @@ -815,11 +837,11 @@ static void reset_page(struct page *page) =20 static int trylock_zspage(struct zspage *zspage) { - struct page *cursor, *fail; + struct zpdesc *cursor, *fail; =20 - for (cursor =3D get_first_page(zspage); cursor !=3D NULL; cursor =3D - get_next_page(cursor)) { - if (!trylock_page(cursor)) { + for (cursor =3D get_first_zpdesc(zspage); cursor !=3D NULL; cursor =3D + get_next_zpdesc(cursor)) { + if (!zpdesc_trylock(cursor)) { fail =3D cursor; goto unlock; } @@ -827,9 +849,9 @@ static int trylock_zspage(struct zspage *zspage) =20 return 1; unlock: - for (cursor =3D get_first_page(zspage); cursor !=3D fail; cursor =3D - get_next_page(cursor)) - unlock_page(cursor); + for (cursor =3D get_first_zpdesc(zspage); cursor !=3D fail; cursor =3D + get_next_zpdesc(cursor)) + zpdesc_unlock(cursor); =20 return 0; } @@ -1658,7 +1680,7 @@ static int putback_zspage(struct size_class *class, s= truct zspage *zspage) */ static void lock_zspage(struct zspage *zspage) { - struct page *curr_page, *page; + struct zpdesc *curr_zpdesc, *zpdesc; =20 /* * Pages we haven't locked yet can be migrated off the list while we're @@ -1670,24 +1692,24 @@ static void lock_zspage(struct zspage *zspage) */ while (1) { migrate_read_lock(zspage); - page =3D get_first_page(zspage); - if (trylock_page(page)) + zpdesc =3D get_first_zpdesc(zspage); + if (zpdesc_trylock(zpdesc)) break; - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); } =20 - curr_page =3D page; - while ((page =3D get_next_page(curr_page))) { - if (trylock_page(page)) { - curr_page =3D page; + curr_zpdesc =3D zpdesc; + while ((zpdesc =3D get_next_zpdesc(curr_zpdesc))) { + if (zpdesc_trylock(zpdesc)) { + curr_zpdesc =3D zpdesc; } else { - get_page(page); + zpdesc_get(zpdesc); migrate_read_unlock(zspage); - wait_on_page_locked(page); - put_page(page); + zpdesc_wait_locked(zpdesc); + zpdesc_put(zpdesc); migrate_read_lock(zspage); } } --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6A7D146595 for ; Mon, 29 Jul 2024 11:20:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252027; cv=none; b=k6l4lrxirqtEV9eg+V9BSwmtCq39N/la7Iva5eGBDeypGc0xxWL0W82s53ICziauV8vaJyanOEtvm43GmyDSt8gYIYloTIh5DQ6qfnalbjnE3FjwSrfwhVhwKWnc4CtzeTRy8k2/WAbUKI2U/UD2dx1LImDow7XnDmuZCUthFr8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252027; c=relaxed/simple; bh=ezRZsoVNxvO9p/Es1oKbiCO8zRXU9uaLAm4pSSvGkg4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bXm7S+2RcGOhHq8sSZHf7lrroH+NGpfvnEfXFhOgdez/msv8hjW5XEwVUhlRtO1x9el6LYt39LDb28Jl0TXnoz625KlUIsrRkmeh8pa+YhG4YwAfS/qMfGkagu+n1dOE4xwH3fnZ3KfGpDTbDZZznsMwj0Ew2fe5T42agUL+y9Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mPBuc1PI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mPBuc1PI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D32CFC4AF0A; Mon, 29 Jul 2024 11:20:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252027; bh=ezRZsoVNxvO9p/Es1oKbiCO8zRXU9uaLAm4pSSvGkg4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mPBuc1PIJz8zxeEieUPstihcjHXEabomtxQv0t1FNmEjidfXx8YbWGSAUWn5iN7D7 DEyiE44MryR2s94k9jM8AXG6j1u5h/1k3RSSZAOfk3Y/EF+LxFI07oscV3CGh0J6ed B6cj5rJspAo6bNylOz4erxIr8aBwqP+bNW919IGYX/RCEGVxm5tx2i77vy+ZpHd3eT gGh2JRbA9G2mi9iKh2gmOKthWRciie29ZHJjBEseRwcP+hIxFTykM9M4wYVvG1xTBs r6o/CobB/cF4rZDnpSBwcspMXsRequ6xuCHD3GU91Cq49XYzsMeyHEKs66Twok2ApT mJcO8nA17CJeQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 03/22] mm/zsmalloc: convert __zs_map_object/__zs_unmap_object to use zpdesc Date: Mon, 29 Jul 2024 19:25:15 +0800 Message-ID: <20240729112534.3416707-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> These two functions take pointer to an array of struct page. Introduce zpdesc_kmap_atomic() and make __zs_{map,unmap}_object() take pointer to an array of zpdesc instead of page. Add silly type casting when calling them. Casting will be removed late. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 243677a9c6d2..68a39c233d34 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -243,6 +243,11 @@ struct zs_pool { atomic_t compaction_in_progress; }; =20 +static inline void *zpdesc_kmap_atomic(struct zpdesc *zpdesc) +{ + return kmap_atomic(zpdesc_page(zpdesc)); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -1061,7 +1066,7 @@ static inline void __zs_cpu_down(struct mapping_area = *area) } =20 static void *__zs_map_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1078,10 +1083,10 @@ static void *__zs_map_object(struct mapping_area *a= rea, sizes[1] =3D size - sizes[0]; =20 /* copy object to per-cpu buffer */ - addr =3D kmap_atomic(pages[0]); + addr =3D zpdesc_kmap_atomic(zpdescs[0]); memcpy(buf, addr + off, sizes[0]); kunmap_atomic(addr); - addr =3D kmap_atomic(pages[1]); + addr =3D zpdesc_kmap_atomic(zpdescs[1]); memcpy(buf + sizes[0], addr, sizes[1]); kunmap_atomic(addr); out: @@ -1089,7 +1094,7 @@ static void *__zs_map_object(struct mapping_area *are= a, } =20 static void __zs_unmap_object(struct mapping_area *area, - struct page *pages[2], int off, int size) + struct zpdesc *zpdescs[2], int off, int size) { int sizes[2]; void *addr; @@ -1108,10 +1113,10 @@ static void __zs_unmap_object(struct mapping_area *= area, sizes[1] =3D size - sizes[0]; =20 /* copy per-cpu buffer to object */ - addr =3D kmap_atomic(pages[0]); + addr =3D zpdesc_kmap_atomic(zpdescs[0]); memcpy(addr + off, buf, sizes[0]); kunmap_atomic(addr); - addr =3D kmap_atomic(pages[1]); + addr =3D zpdesc_kmap_atomic(zpdescs[1]); memcpy(addr, buf + sizes[0], sizes[1]); kunmap_atomic(addr); =20 @@ -1252,7 +1257,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, pages[1] =3D get_next_page(page); BUG_ON(!pages[1]); =20 - ret =3D __zs_map_object(area, pages, off, class->size); + ret =3D __zs_map_object(area, (struct zpdesc **)pages, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret +=3D ZS_HANDLE_SIZE; @@ -1287,7 +1292,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) pages[1] =3D get_next_page(page); BUG_ON(!pages[1]); =20 - __zs_unmap_object(area, pages, off, class->size); + __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); } local_unlock(&zs_map_area.lock); =20 --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04164146A60 for ; Mon, 29 Jul 2024 11:20:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252031; cv=none; b=MyF2AyofTWrm5wkaoIoQ23BPaLz6W/J9tUY+OZ7JkQQ10hkDtr2L9rEZePsRJwcSlro7XQdktsVq6sPdmLb66Q5M0ptsAIhb2ILIdCFmFOExJiyye/i8TYmqMQ/KXxx8cMYoDYOGNm5qSLDHWZvbeothweiqtqy3Y+5U9XoZTo8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252031; c=relaxed/simple; bh=L0Y7PVm5cpbz2IFM45IR0tDhRxs7uvign9JpFjcXT0w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iITtWn651CWb4Fkg3PAGM6m0/SYm0JrMyMDG0me3Pi+K/XD8LHyYKXDvLJF8/IzgovaDCmCJb/iMqyoMieyTquNbzY43E816kGRe9Ue8oxVDqjiufhZqqmKi3d4Fgz+8pRCWl5Eg3D3cMO9fUq+d+sl8ieJOaAd0HOwaD+XFe+s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=toQpXt7a; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="toQpXt7a" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1D765C32786; Mon, 29 Jul 2024 11:20:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252030; bh=L0Y7PVm5cpbz2IFM45IR0tDhRxs7uvign9JpFjcXT0w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=toQpXt7a1O6odufEDjhGAzZYu3VuMAKim87S6w38HBpq3NyDrVubRh0vzjPWIoUq7 Kmp9LObgQS7DZcmOp8oE0SVc3ZgKc7hKBsyLCsSpkSfTj+7Jzd93aYoImSQntG4eF9 9QgWjRSpjW8+BrUmAgIZAHwJ9Zat1lil6FYLUCOmyAATd5uLdfLC+S5ohT3hKbPs7J mdBlovYhV6Wh/rF5+xEP4NsMbLYiNZ10tEhT7o5PrnGdBqESs3jpayId6AXUn4fFKf Ose/yAf/SYm5b9XQOOEUtSOwh7XiYz99icq5AyR8Dx5gENUdGpIq0Gj4h/sy3ExVyg N802hFeXmjcvA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 04/22] mm/zsmalloc: add and use pfn/zpdesc seeking funcs Date: Mon, 29 Jul 2024 19:25:16 +0800 Message-ID: <20240729112534.3416707-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Add pfn_zpdesc conversion, convert obj_to_location() to take zpdesc and also convert its users to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 9 +++++++ mm/zsmalloc.c | 75 ++++++++++++++++++++++++++------------------------- 2 files changed, 47 insertions(+), 37 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 3b04197cec9d..79ec40b03956 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -93,4 +93,13 @@ static inline void zpdesc_put(struct zpdesc *zpdesc) folio_put(zpdesc_folio(zpdesc)); } =20 +static inline unsigned long zpdesc_pfn(struct zpdesc *zpdesc) +{ + return page_to_pfn(zpdesc_page(zpdesc)); +} + +static inline struct zpdesc *pfn_zpdesc(unsigned long pfn) +{ + return page_zpdesc(pfn_to_page(pfn)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 68a39c233d34..149fe2b332cb 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -773,15 +773,15 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *= zpdesc) } =20 /** - * obj_to_location - get (, ) from encoded object value + * obj_to_location - get (, ) from encoded object value * @obj: the encoded object value - * @page: page object resides in zspage + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static void obj_to_location(unsigned long obj, struct page **page, +static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *page =3D pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); *obj_idx =3D (obj & OBJ_INDEX_MASK); } =20 @@ -1208,13 +1208,13 @@ void *zs_map_object(struct zs_pool *pool, unsigned = long handle, enum zs_mapmode mm) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; =20 struct size_class *class; struct mapping_area *area; - struct page *pages[2]; + struct zpdesc *zpdescs[2]; void *ret; =20 /* @@ -1227,8 +1227,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, /* It guarantees it can get zspage from handle safely */ read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage =3D get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage =3D get_zspage(zpdesc_page(zpdesc)); =20 /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1247,17 +1247,17 @@ void *zs_map_object(struct zs_pool *pool, unsigned = long handle, area->vm_mm =3D mm; if (off + class->size <=3D PAGE_SIZE) { /* this object is contained entirely within a page */ - area->vm_addr =3D kmap_atomic(page); + area->vm_addr =3D zpdesc_kmap_atomic(zpdesc); ret =3D area->vm_addr + off; goto out; } =20 /* this object spans two pages */ - pages[0] =3D page; - pages[1] =3D get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] =3D zpdesc; + zpdescs[1] =3D get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); =20 - ret =3D __zs_map_object(area, (struct zpdesc **)pages, off, class->size); + ret =3D __zs_map_object(area, zpdescs, off, class->size); out: if (likely(!ZsHugePage(zspage))) ret +=3D ZS_HANDLE_SIZE; @@ -1269,7 +1269,7 @@ EXPORT_SYMBOL_GPL(zs_map_object); void zs_unmap_object(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *page; + struct zpdesc *zpdesc; unsigned long obj, off; unsigned int obj_idx; =20 @@ -1277,8 +1277,8 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) struct mapping_area *area; =20 obj =3D handle_to_obj(handle); - obj_to_location(obj, &page, &obj_idx); - zspage =3D get_zspage(page); + obj_to_location(obj, &zpdesc, &obj_idx); + zspage =3D get_zspage(zpdesc_page(zpdesc)); class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 @@ -1286,13 +1286,13 @@ void zs_unmap_object(struct zs_pool *pool, unsigned= long handle) if (off + class->size <=3D PAGE_SIZE) kunmap_atomic(area->vm_addr); else { - struct page *pages[2]; + struct zpdesc *zpdescs[2]; =20 - pages[0] =3D page; - pages[1] =3D get_next_page(page); - BUG_ON(!pages[1]); + zpdescs[0] =3D zpdesc; + zpdescs[1] =3D get_next_zpdesc(zpdesc); + BUG_ON(!zpdescs[1]); =20 - __zs_unmap_object(area, (struct zpdesc **)pages, off, class->size); + __zs_unmap_object(area, zpdescs, off, class->size); } local_unlock(&zs_map_area.lock); =20 @@ -1434,23 +1434,24 @@ static void obj_free(int class_size, unsigned long = obj) { struct link_free *link; struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long f_offset; unsigned int f_objidx; void *vaddr; =20 - obj_to_location(obj, &f_page, &f_objidx); + + obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset =3D offset_in_page(class_size * f_objidx); - zspage =3D get_zspage(f_page); + zspage =3D get_zspage(zpdesc_page(f_zpdesc)); =20 - vaddr =3D kmap_atomic(f_page); + vaddr =3D zpdesc_kmap_atomic(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); =20 /* Insert this object in containing zspage's freelist */ if (likely(!ZsHugePage(zspage))) link->next =3D get_freeobj(zspage) << OBJ_TAG_BITS; else - f_page->index =3D 0; + f_zpdesc->next =3D NULL; set_freeobj(zspage, f_objidx); =20 kunmap_atomic(vaddr); @@ -1495,7 +1496,7 @@ EXPORT_SYMBOL_GPL(zs_free); static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { - struct page *s_page, *d_page; + struct zpdesc *s_zpdesc, *d_zpdesc; unsigned int s_objidx, d_objidx; unsigned long s_off, d_off; void *s_addr, *d_addr; @@ -1504,8 +1505,8 @@ static void zs_object_copy(struct size_class *class, = unsigned long dst, =20 s_size =3D d_size =3D class->size; =20 - obj_to_location(src, &s_page, &s_objidx); - obj_to_location(dst, &d_page, &d_objidx); + obj_to_location(src, &s_zpdesc, &s_objidx); + obj_to_location(dst, &d_zpdesc, &d_objidx); =20 s_off =3D offset_in_page(class->size * s_objidx); d_off =3D offset_in_page(class->size * d_objidx); @@ -1516,8 +1517,8 @@ static void zs_object_copy(struct size_class *class, = unsigned long dst, if (d_off + class->size > PAGE_SIZE) d_size =3D PAGE_SIZE - d_off; =20 - s_addr =3D kmap_atomic(s_page); - d_addr =3D kmap_atomic(d_page); + s_addr =3D zpdesc_kmap_atomic(s_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); =20 while (1) { size =3D min(s_size, d_size); @@ -1542,17 +1543,17 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, if (s_off >=3D PAGE_SIZE) { kunmap_atomic(d_addr); kunmap_atomic(s_addr); - s_page =3D get_next_page(s_page); - s_addr =3D kmap_atomic(s_page); - d_addr =3D kmap_atomic(d_page); + s_zpdesc =3D get_next_zpdesc(s_zpdesc); + s_addr =3D zpdesc_kmap_atomic(s_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); s_size =3D class->size - written; s_off =3D 0; } =20 if (d_off >=3D PAGE_SIZE) { kunmap_atomic(d_addr); - d_page =3D get_next_page(d_page); - d_addr =3D kmap_atomic(d_page); + d_zpdesc =3D get_next_zpdesc(d_zpdesc); + d_addr =3D zpdesc_kmap_atomic(d_zpdesc); d_size =3D class->size - written; d_off =3D 0; } @@ -1791,7 +1792,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, struct zs_pool *pool; struct size_class *class; struct zspage *zspage; - struct page *dummy; + struct zpdesc *dummy; void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 439A5146A8C for ; Mon, 29 Jul 2024 11:20:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252034; cv=none; b=lu9Z+TEKF2uo+UenyEDY7jJmpx5tf+4knmg4G5emWzayl+zJZzsph4iSEU5dTBtsAtzcYl0M5FmfuiGH9TnCKQSiiNYVLQ+s7pvFjMEMED4kPvy+Wh9g7x6HORFs5zqOkBrBB5XyMsLBpOiTtyT1Q3C9SQRcKxoh77fz0AGA3D4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252034; c=relaxed/simple; bh=OUw0L1+EEkxStexhrY6FwqktPPZ5HwiknqjqfzU6PXk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AtYjHkhfzfyV9gRyRGDBlp4agjA0VoTvTe9WeJSKgKINeZKRXNAj+hgRWWoP953nROcbhrW0IoHdSaljNvLeYMvyObI5R5lDopLuNb1PESe4a8ShgzrMHFZvEVmLEEZsBY+oxcyfMea4bj9D2/mZxU6X77R59IhZVTKEymLGAuU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=N1dl0pOc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="N1dl0pOc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5F00FC32786; Mon, 29 Jul 2024 11:20:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252034; bh=OUw0L1+EEkxStexhrY6FwqktPPZ5HwiknqjqfzU6PXk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N1dl0pOcTFgdmJMCVClV0yXxSZWm7Os6wsGJ8ws8QQxeG1j3yfc4V0ckuKv4YRMv/ N5hoiL5nFmBw9n5kMO//6XdEuvI1hUezsU07uTtuoLOWXZf9d7IUGz7qr8PrG7Ou9P oz3DyQNNb766EmVqx/Ph2tCIOxiJYm8TzWmi0nRSA3dytbAskWr3IWIfrIMpzw9y/D oVY3NFI/tP0Wjs+PVcxCpfRBSSKz1lDKt0FCv6Aznv+WWyn5nND3SBVMP8rLdaRzof H7D20CRe1/5dp+pGZF7Nyz3t615VHG60okruZFTRx/u9cA4QYBXyElY4tQZktkitMc /UvAS8vIlC8Bw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 05/22] mm/zsmalloc: convert obj_malloc() to use zpdesc Date: Mon, 29 Jul 2024 19:25:17 +0800 Message-ID: <20240729112534.3416707-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first_page/get_next_page. no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 149fe2b332cb..bbc165cb587d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1322,12 +1322,12 @@ EXPORT_SYMBOL_GPL(zs_huge_class_size); static unsigned long obj_malloc(struct zs_pool *pool, struct zspage *zspage, unsigned long handle) { - int i, nr_page, offset; + int i, nr_zpdesc, offset; unsigned long obj; struct link_free *link; struct size_class *class; =20 - struct page *m_page; + struct zpdesc *m_zpdesc; unsigned long m_offset; void *vaddr; =20 @@ -1335,14 +1335,14 @@ static unsigned long obj_malloc(struct zs_pool *poo= l, obj =3D get_freeobj(zspage); =20 offset =3D obj * class->size; - nr_page =3D offset >> PAGE_SHIFT; + nr_zpdesc =3D offset >> PAGE_SHIFT; m_offset =3D offset_in_page(offset); - m_page =3D get_first_page(zspage); + m_zpdesc =3D get_first_zpdesc(zspage); =20 - for (i =3D 0; i < nr_page; i++) - m_page =3D get_next_page(m_page); + for (i =3D 0; i < nr_zpdesc; i++) + m_zpdesc =3D get_next_zpdesc(m_zpdesc); =20 - vaddr =3D kmap_atomic(m_page); + vaddr =3D zpdesc_kmap_atomic(m_zpdesc); link =3D (struct link_free *)vaddr + m_offset / sizeof(*link); set_freeobj(zspage, link->next >> OBJ_TAG_BITS); if (likely(!ZsHugePage(zspage))) @@ -1355,7 +1355,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(m_page, obj); + obj =3D location_to_obj(zpdesc_page(m_zpdesc), obj); record_obj(handle, obj); =20 return obj; --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E43311474AE for ; Mon, 29 Jul 2024 11:20:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252038; cv=none; b=julfWF3sCLNTZTwZhW9EO9XP0B5WYoOb+XRQ8jtzxQhy9XHpJclaYn5nqWg0RAxaxeLhr7hJ77qeoPhWpkt0qoSBdqRZOZrwszJCVvW0Dl0COQ2Hq0M7MCK23wqw9vomX3yIYBiQ/4eDHtEkU+qmdvmvp1o/O6NYnzLwRFCEJLc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252038; c=relaxed/simple; bh=4GoIroOrsiTuTdQywHvfrAqEQ0msvRBa8bIywcf/abg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ib7q5jgP7IgEcw6YbuSkbXNjtd+OUflNbL2fzMvUUTKB/ZRmpJ0zTDJj6FsjS/P4pI82gcFW+OaylmhxDTBa6Xi+cXqv+P1QUOZf9xJg2ZkqcSWIv6ifXX8vOl/fF79skpaJ41L3aOqFb9xabybpmW9kdVeJ3cOHAGCpJT9xpFI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CHDQSHaQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CHDQSHaQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9DF16C32786; Mon, 29 Jul 2024 11:20:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252037; bh=4GoIroOrsiTuTdQywHvfrAqEQ0msvRBa8bIywcf/abg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CHDQSHaQ7YakefYJHb5Fg8lwtzCCaRG0VB351O0fHGxYrATxRQrDZhI6rXJ3dYnNK Cs/mmOJEmxG2Aa1cu/6/PsIqH4iL7wmLNMzzgMSyyw6nxdlJ7TUkJefkCoFSJ2U4XS IMEeeC0++d0mDQXFOPSmTGpPyQZyQAKDz/oPl24dNOK4QRKxHMwM0ex8RFI3nGBYJw pknR9QDjYq8runDJIu6KXyqIB3Ocw+VmQeuFt5ONCeMIVGQmaQGQYkBstduSvvwdHA ncnyZj0t/SqxNDRf721evVt/lqIhGFRrvpfpJa6Vz1uJAWL6rzHStGwxIs063ARWFF exuUydPFMljvw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 06/22] mm/zsmalloc: convert create_page_chain() and its users to use zpdesc Date: Mon, 29 Jul 2024 19:25:18 +0800 Message-ID: <20240729112534.3416707-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Introduce a few helper functions for conversion to convert create_page_chai= n() to use zpdesc, then use zpdesc in replace_sub_page() too. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 6 +++ mm/zsmalloc.c | 115 +++++++++++++++++++++++++++++++++----------------- 2 files changed, 82 insertions(+), 39 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 79ec40b03956..2293453f5d57 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -102,4 +102,10 @@ static inline struct zpdesc *pfn_zpdesc(unsigned long = pfn) { return page_zpdesc(pfn_to_page(pfn)); } + +static inline void __zpdesc_set_movable(struct zpdesc *zpdesc, + const struct movable_operations *mops) +{ + __SetPageMovable(zpdesc_page(zpdesc), mops); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bbc165cb587d..a8f390beeab8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -248,6 +248,41 @@ static inline void *zpdesc_kmap_atomic(struct zpdesc *= zpdesc) return kmap_atomic(zpdesc_page(zpdesc)); } =20 +static inline void zpdesc_set_zspage(struct zpdesc *zpdesc, + struct zspage *zspage) +{ + zpdesc->zspage =3D zspage; +} + +static inline void zpdesc_set_first(struct zpdesc *zpdesc) +{ + SetPagePrivate(zpdesc_page(zpdesc)); +} + +static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) +{ + inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline void zpdesc_dec_zone_page_state(struct zpdesc *zpdesc) +{ + dec_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); +} + +static inline struct zpdesc *alloc_zpdesc(gfp_t gfp) +{ + struct page *page =3D alloc_page(gfp); + + return page_zpdesc(page); +} + +static inline void free_zpdesc(struct zpdesc *zpdesc) +{ + struct page *page =3D zpdesc_page(zpdesc); + + __free_page(page); +} + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -954,35 +989,35 @@ static void init_zspage(struct size_class *class, str= uct zspage *zspage) } =20 static void create_page_chain(struct size_class *class, struct zspage *zsp= age, - struct page *pages[]) + struct zpdesc *zpdescs[]) { int i; - struct page *page; - struct page *prev_page =3D NULL; - int nr_pages =3D class->pages_per_zspage; + struct zpdesc *zpdesc; + struct zpdesc *prev_zpdesc =3D NULL; + int nr_zpdescs =3D class->pages_per_zspage; =20 /* * Allocate individual pages and link them together as: - * 1. all pages are linked together using page->index - * 2. each sub-page point to zspage using page->private + * 1. all pages are linked together using zpdesc->next + * 2. each sub-page point to zspage using zpdesc->zspage * - * we set PG_private to identify the first page (i.e. no other sub-page + * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). */ - for (i =3D 0; i < nr_pages; i++) { - page =3D pages[i]; - set_page_private(page, (unsigned long)zspage); - page->index =3D 0; + for (i =3D 0; i < nr_zpdescs; i++) { + zpdesc =3D zpdescs[i]; + zpdesc_set_zspage(zpdesc, zspage); + zpdesc->next =3D NULL; if (i =3D=3D 0) { - zspage->first_zpdesc =3D page_zpdesc(page); - SetPagePrivate(page); + zspage->first_zpdesc =3D zpdesc; + zpdesc_set_first(zpdesc); if (unlikely(class->objs_per_zspage =3D=3D 1 && class->pages_per_zspage =3D=3D 1)) SetZsHugePage(zspage); } else { - prev_page->index =3D (unsigned long)page; + prev_zpdesc->next =3D zpdesc; } - prev_page =3D page; + prev_zpdesc =3D zpdesc; } } =20 @@ -994,7 +1029,7 @@ static struct zspage *alloc_zspage(struct zs_pool *poo= l, gfp_t gfp) { int i; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE]; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE]; struct zspage *zspage =3D cache_alloc_zspage(pool, gfp); =20 if (!zspage) @@ -1004,25 +1039,25 @@ static struct zspage *alloc_zspage(struct zs_pool *= pool, migrate_lock_init(zspage); =20 for (i =3D 0; i < class->pages_per_zspage; i++) { - struct page *page; + struct zpdesc *zpdesc; =20 - page =3D alloc_page(gfp); - if (!page) { + zpdesc =3D alloc_zpdesc(gfp); + if (!zpdesc) { while (--i >=3D 0) { - dec_zone_page_state(pages[i], NR_ZSPAGES); - __ClearPageZsmalloc(pages[i]); - __free_page(pages[i]); + zpdesc_dec_zone_page_state(zpdescs[i]); + __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(page); + __SetPageZsmalloc(zpdesc_page(zpdesc)); =20 - inc_zone_page_state(page, NR_ZSPAGES); - pages[i] =3D page; + zpdesc_inc_zone_page_state(zpdesc); + zpdescs[i] =3D zpdesc; } =20 - create_page_chain(class, zspage, pages); + create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool =3D pool; zspage->class =3D class->index; @@ -1753,26 +1788,28 @@ static void migrate_write_unlock(struct zspage *zsp= age) static const struct movable_operations zsmalloc_mops; =20 static void replace_sub_page(struct size_class *class, struct zspage *zspa= ge, - struct page *newpage, struct page *oldpage) + struct zpdesc *newzpdesc, struct zpdesc *oldzpdesc) { - struct page *page; - struct page *pages[ZS_MAX_PAGES_PER_ZSPAGE] =3D {NULL, }; + struct zpdesc *zpdesc; + struct zpdesc *zpdescs[ZS_MAX_PAGES_PER_ZSPAGE] =3D {NULL, }; + unsigned int first_obj_offset; int idx =3D 0; =20 - page =3D get_first_page(zspage); + zpdesc =3D get_first_zpdesc(zspage); do { - if (page =3D=3D oldpage) - pages[idx] =3D newpage; + if (zpdesc =3D=3D oldzpdesc) + zpdescs[idx] =3D newzpdesc; else - pages[idx] =3D page; + zpdescs[idx] =3D zpdesc; idx++; - } while ((page =3D get_next_page(page)) !=3D NULL); + } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 - create_page_chain(class, zspage, pages); - set_first_obj_offset(newpage, get_first_obj_offset(oldpage)); + create_page_chain(class, zspage, zpdescs); + first_obj_offset =3D get_first_obj_offset(zpdesc_page(oldzpdesc)); + set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); if (unlikely(ZsHugePage(zspage))) - newpage->index =3D oldpage->index; - __SetPageMovable(newpage, &zsmalloc_mops); + newzpdesc->handle =3D oldzpdesc->handle; + __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); } =20 static bool zs_page_isolate(struct page *page, isolate_mode_t mode) @@ -1845,7 +1882,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, } kunmap_atomic(s_addr); =20 - replace_sub_page(class, zspage, newpage, page); + replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08E711474DA for ; Mon, 29 Jul 2024 11:20:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252041; cv=none; b=opaaInJ09GzwOq1JL86pgUUs6wlK+nEZEigX6DEQCMV3KGjvppHZPrRoQLnaFcJshILAnRLW5s8D2WJ6gnEdBuOIFT8Lr7hjiEQFByWcop+tiZEMXB85xtrO2zYHSNXBbHBNsZ3mgN3mYZ7o1IasdznhIixksK3DacQPc/K6doM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252041; c=relaxed/simple; bh=82zz9HxUIo/V08TGVJ1E1ZkMfjqggCMIVYH80TRsdU8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jEkJ38chF1L+nIIXhWuPl55nzDaC8SuF6yPotdyffpQp3E8DDUya4fgg21Na5S4EwNxzyqftnaYeRegFNM2pJ/9Pv1A1YF1GNWQjGr8V0fSKHVqyZkgjuvzm2P0o/HAYDsPjo+7Bzlnli11T7/lw5L2FoN8Ucp2DF/BNbaGSEj4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kogUr8J+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kogUr8J+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DE9DCC4AF07; Mon, 29 Jul 2024 11:20:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252040; bh=82zz9HxUIo/V08TGVJ1E1ZkMfjqggCMIVYH80TRsdU8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kogUr8J+ASWDsTSUahBnjIKsYaKvi+MjsPcBvHhGeMdzD9/bY1wyw1uJjMFXqaZqa dbzYtQVeJrFawJPLm0Az6yN9N4HdL1MVHpK6luozdkWZsUBI0HDrA4b3KXItLrQxnt FMtVzoqoh1N2uK9Bv+aIKlt2fPU64fk92RzxzWQkEwNG+BtJhMhRhQmjjLweH2jQys o1NhJCqu93uTeCGJp77cZB5zYrqiDCLwwhIGCzsuyKh0QWkprw7eZWTRkq0VdpizXt oif7Tr3J8T6GX5EOfUt0ZLw+Egnrb/dD4h3giVED052va3Wk+2pYPUamRUWeo2rTld Hc9VV7OuD9jXA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 07/22] mm/zsmalloc: convert obj_allocated() and related helpers to use zpdesc Date: Mon, 29 Jul 2024 19:25:19 +0800 Message-ID: <20240729112534.3416707-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Convert obj_allocated(), and related helpers to take zpdesc. Also make its callers to cast (struct page *) to (struct zpdesc *) when calling them. The users will be converted gradually as there are many. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a8f390beeab8..29b9fa5baa46 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -845,15 +845,15 @@ static unsigned long handle_to_obj(unsigned long hand= le) return *(unsigned long *)handle; } =20 -static inline bool obj_allocated(struct page *page, void *obj, +static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage =3D get_zspage(page); + struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); =20 if (unlikely(ZsHugePage(zspage))) { - VM_BUG_ON_PAGE(!is_first_page(page), page); - handle =3D page->index; + VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); + handle =3D zpdesc->handle; } else handle =3D *(unsigned long *)obj; =20 @@ -1603,18 +1603,18 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, * return handle. */ static unsigned long find_alloced_obj(struct size_class *class, - struct page *page, int *obj_idx) + struct zpdesc *zpdesc, int *obj_idx) { unsigned int offset; int index =3D *obj_idx; unsigned long handle =3D 0; - void *addr =3D kmap_atomic(page); + void *addr =3D zpdesc_kmap_atomic(zpdesc); =20 - offset =3D get_first_obj_offset(page); + offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { - if (obj_allocated(page, addr + offset, &handle)) + if (obj_allocated(zpdesc, addr + offset, &handle)) break; =20 offset +=3D class->size; @@ -1638,7 +1638,7 @@ static void migrate_zspage(struct zs_pool *pool, stru= ct zspage *src_zspage, struct size_class *class =3D pool->size_class[src_zspage->class]; =20 while (1) { - handle =3D find_alloced_obj(class, s_page, &obj_idx); + handle =3D find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); if (!handle) { s_page =3D get_next_page(s_page); if (!s_page) @@ -1871,7 +1871,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 for (addr =3D s_addr + offset; addr < s_addr + PAGE_SIZE; addr +=3D class->size) { - if (obj_allocated(page, addr, &handle)) { + if (obj_allocated(page_zpdesc(page), addr, &handle)) { =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B6F1146A60 for ; Mon, 29 Jul 2024 11:20:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252044; cv=none; b=P0ZIit9Wbj9o2SNha7fRFfCgEUYWpD4InrbMKwwSmyXrafMDvajXqVUPBxoufkNpMGFNuO/vB4Gj4kFTFnJOXQ0HAkAKR9/ulgbY4q96kOMcmK4IicEDDcP+bC73fwsmIhkCXEuP2YJtiLD937K9uvSI0FZzvOZl+ph4wW2m2b4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252044; c=relaxed/simple; bh=hCA+Wp2koQFPErK04B2puEI5lWDR1Rh4V60AMISo3bA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VbjjQ/mRaXjNz/jkArCzdeOWaM+uya82qod5WDQ6FxDhlFTDa8O9G2PfeJKzANyHvqWxYFQQbiexC2z1EWGSJEwtWfnO0MK2JGyNZ4rM8ftFHo72IqSRxmQv4PHscx4pADM2/xhQu0chx81KhkyAXgM3wko4kU/v9xQyA710eik= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RxZcivsx; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RxZcivsx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 232D0C4AF0A; Mon, 29 Jul 2024 11:20:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252043; bh=hCA+Wp2koQFPErK04B2puEI5lWDR1Rh4V60AMISo3bA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RxZcivsxHrV7ru1KjcEzOlOvSeKDf/VpS35R1Jm/ckHCF5FkBHfPZZlKMLo1HZlJZ e+mZIYJdkYGOoowl74xkpbxMv+5NuCg1KbSFCL4kZ00Tqa3dTGnIiwH50thl+3pYVa Geh9a7OSmgGpOqv0ZOW2hZIUlBaMFwvD0xV8t4bdMzCjwlGp7rgp+nn3EGq3oahway 16a0c1+b0SjHzRW7oSgs1/gNsL+p0Ht9S6hgw8Abb/W7bj3eI4av9N4IaeoQx0Ogp7 ZP2Xcf2MLAk4ozA193cBosfrjL0qJE8Q7nrWXzuslYTjAjYJOrSlnhZ9DJHaPEbYhC GG/xWcMb4MZ4Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 08/22] mm/zsmalloc: convert init_zspage() to use zpdesc Date: Mon, 29 Jul 2024 19:25:20 +0800 Message-ID: <20240729112534.3416707-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Replace get_first/next_page func series and kmap_atomic to new helper, no functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 29b9fa5baa46..d3558f3f8bc3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -948,16 +948,16 @@ static void init_zspage(struct size_class *class, str= uct zspage *zspage) { unsigned int freeobj =3D 1; unsigned long off =3D 0; - struct page *page =3D get_first_page(zspage); + struct zpdesc *zpdesc =3D get_first_zpdesc(zspage); =20 - while (page) { - struct page *next_page; + while (zpdesc) { + struct zpdesc *next_zpdesc; struct link_free *link; void *vaddr; =20 - set_first_obj_offset(page, off); + set_first_obj_offset(zpdesc_page(zpdesc), off); =20 - vaddr =3D kmap_atomic(page); + vaddr =3D zpdesc_kmap_atomic(zpdesc); link =3D (struct link_free *)vaddr + off / sizeof(*link); =20 while ((off +=3D class->size) < PAGE_SIZE) { @@ -970,8 +970,8 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) * page, which must point to the first object on the next * page (if present) */ - next_page =3D get_next_page(page); - if (next_page) { + next_zpdesc =3D get_next_zpdesc(zpdesc); + if (next_zpdesc) { link->next =3D freeobj++ << OBJ_TAG_BITS; } else { /* @@ -981,7 +981,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) link->next =3D -1UL << OBJ_TAG_BITS; } kunmap_atomic(vaddr); - page =3D next_page; + zpdesc =3D next_zpdesc; off %=3D PAGE_SIZE; } =20 --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8AE86146013 for ; Mon, 29 Jul 2024 11:20:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252047; cv=none; b=An1REGyO32fSXgBsztKj48IGTdWkEexwq1VB/kQbm2iBYbth+guvWFb3XMq/0JMZHrHGblWwJKEaJ/Vjjo8tuaZ+k3EMK8rM2QivBqEVnB6XBrq9473KwwG4tThCnSxkTGzFFHSVGS/LbW8WtL+R+XDNTKZkKOkj0PI/pdwM8dw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252047; c=relaxed/simple; bh=cpsRKinL3S7LbxZnzQMdHFXcR2a2bxLsePv8cUuJa+o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LvZqkjkhG4IPC6fVt5NBn9XgHDlIZ9QgsME7AnSAmBmKfmjjT8ICb/WnNKcnwDerUhE7kzKk8iVl3gqJvVjNx/H1QnBvMkDBPcJpWiJ0sXQJk4cpK0aaMAh+lboWVnUtgV+UD6bd7C+81qyNmtTgWf80j7cmgO33lISI+VbC8BY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=T0HH8kYx; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="T0HH8kYx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 60074C32786; Mon, 29 Jul 2024 11:20:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252047; bh=cpsRKinL3S7LbxZnzQMdHFXcR2a2bxLsePv8cUuJa+o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=T0HH8kYxUU1m2iRXQy2rq8UBV0w9GOrcvJ5BKUKIZP0MxXQhIiJHVXkOYVVvUrFc/ rbr+b4ZBiHZ0MUo9TvktNqnYHiM4n+NSgjKL43RRGvYjmPnPKvw+jCbLo9v9YaBpOV F8ve7JyaLosgbP5LMe+oLr2On8GDoBALuQBVqDlOiAmOJRfKpSmn7ZybSNJ+bx+ffS kGkruVAKs3HzjR0cHAJy5RkX9mjF8+0IztbMD5XpjxbMs7PiqUDSQvpqpyofs8/FWt clA7JVTFSXxI9MlNfs/KV9XHHCns4sVh32aFpY9QJcsQ2236XureRRXkHCFAjdfjc5 Mr+RgjTe17w4w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 09/22] mm/zsmalloc: convert obj_to_page() and zs_free() to use zpdesc Date: Mon, 29 Jul 2024 19:25:21 +0800 Message-ID: <20240729112534.3416707-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Rename obj_to_page() to obj_to_zpdesc() and also convert it and its user zs_free() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d3558f3f8bc3..7aa4a4acaec9 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -820,9 +820,9 @@ static void obj_to_location(unsigned long obj, struct z= pdesc **zpdesc, *obj_idx =3D (obj & OBJ_INDEX_MASK); } =20 -static void obj_to_page(unsigned long obj, struct page **page) +static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *page =3D pfn_to_page(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); } =20 /** @@ -1496,7 +1496,7 @@ static void obj_free(int class_size, unsigned long ob= j) void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct page *f_page; + struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1510,8 +1510,8 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) */ read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); - obj_to_page(obj, &f_page); - zspage =3D get_zspage(f_page); + obj_to_zpdesc(obj, &f_zpdesc); + zspage =3D get_zspage(zpdesc_page(f_zpdesc)); class =3D zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D0A5148820 for ; Mon, 29 Jul 2024 11:20:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252050; cv=none; b=RxukP21VFtaISFJHwdcfMgoylOwAa9pmL18T+sarnEU8489qQuB2QvJggpF+lYnJZZmRmEmdGsTHlKEW4zcwzwGz+Ite6/1PXsLF03BbU0p3XuBmw2AOjlF1eLHkzonrh08JvDbk2zYZeAv13qR4jNLdKemWZeB2oDZUNnCQBrQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252050; c=relaxed/simple; bh=WyPUKBHEIKzHOQMb2khWX8zwblQV8unnw77vQM7bDF8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fij5LFRlWDeC/Da+jNPq1JgzCUx+byn3CqjEj+irB1oLpXSFiG2tlfESAUAF63p3HigOzvFd50J9Y/IwO9k7q9K84hmncnY8lDLbwIIbf8uyt4f7eg3dAjvplsCFjfRNnW9Bim0AX4wnZ87cHVHn4f8EskPjRyRBeVHfd02OpQ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=E5iyLufh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="E5iyLufh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E330C4AF0E; Mon, 29 Jul 2024 11:20:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252050; bh=WyPUKBHEIKzHOQMb2khWX8zwblQV8unnw77vQM7bDF8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E5iyLufhwEDkZ+aY5DI547cHfuH0SyU0cZnKUBidJpNkKOkO5+26+kAm6UVZbixNO njVNEpfbytbIXh+8dvyp2agedOkshPsMVpKvAE8d3t5isV/D/XUHFz12c6g17JUgqq JEIr6IMFyMo6wMOYJApbAtqri95Ugy2upf77adlVIFdVbKOypNKYyistCUwc/ndMip ob4vjCmg1y8Ifvm3jCtN9ViV5NE5fHZmq/WCaKwx4mRXgOwGgZ4Xi3sfywdy2c7cwX CACAKi4+a6TxHXlxR9n+RwdFtn44i7Y7Tcaw5m5yjVWIwJ4SAUHNTl+xl7Qt+wr3OB Y7DEGDaoJUl6w== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 10/22] mm/zsmalloc: add zpdesc_is_isolated/zpdesc_zone helper for zs_page_migrate Date: Mon, 29 Jul 2024 19:25:22 +0800 Message-ID: <20240729112534.3416707-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To convert page to zpdesc in zs_page_migrate(), we added zpdesc_is_isolated/zpdesc_zone helpers. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 11 +++++++++++ mm/zsmalloc.c | 30 ++++++++++++++++-------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 2293453f5d57..ad04c8337cae 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -108,4 +108,15 @@ static inline void __zpdesc_set_movable(struct zpdesc = *zpdesc, { __SetPageMovable(zpdesc_page(zpdesc), mops); } + +static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) +{ + return PageIsolated(zpdesc_page(zpdesc)); +} + +static inline struct zone *zpdesc_zone(struct zpdesc *zpdesc) +{ + return page_zone(zpdesc_page(zpdesc)); +} + #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7aa4a4acaec9..9bc9b14187ed 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1830,19 +1830,21 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, struct size_class *class; struct zspage *zspage; struct zpdesc *dummy; + struct zpdesc *newzpdesc =3D page_zpdesc(newpage); + struct zpdesc *zpdesc =3D page_zpdesc(page); void *s_addr, *d_addr, *addr; unsigned int offset; unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; =20 - VM_BUG_ON_PAGE(!PageIsolated(page), page); + VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); =20 /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(newpage); + __SetPageZsmalloc(zpdesc_page(newzpdesc)); =20 /* The page is locked, so this pointer must remain valid */ - zspage =3D get_zspage(page); + zspage =3D get_zspage(zpdesc_page(zpdesc)); pool =3D zspage->pool; =20 /* @@ -1859,30 +1861,30 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); =20 - offset =3D get_first_obj_offset(page); - s_addr =3D kmap_atomic(page); + offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + s_addr =3D zpdesc_kmap_atomic(zpdesc); =20 /* * Here, any user cannot access all objects in the zspage so let's move. */ - d_addr =3D kmap_atomic(newpage); + d_addr =3D zpdesc_kmap_atomic(newzpdesc); copy_page(d_addr, s_addr); kunmap_atomic(d_addr); =20 for (addr =3D s_addr + offset; addr < s_addr + PAGE_SIZE; addr +=3D class->size) { - if (obj_allocated(page_zpdesc(page), addr, &handle)) { + if (obj_allocated(zpdesc, addr, &handle)) { =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(newpage, + new_obj =3D (unsigned long)location_to_obj(zpdesc_page(newzpdesc), obj_idx); record_obj(handle, new_obj); } } kunmap_atomic(s_addr); =20 - replace_sub_page(class, zspage, page_zpdesc(newpage), page_zpdesc(page)); + replace_sub_page(class, zspage, newzpdesc, zpdesc); /* * Since we complete the data copy and set up new zspage structure, * it's okay to release migration_lock. @@ -1891,14 +1893,14 @@ static int zs_page_migrate(struct page *newpage, st= ruct page *page, spin_unlock(&class->lock); migrate_write_unlock(zspage); =20 - get_page(newpage); - if (page_zone(newpage) !=3D page_zone(page)) { - dec_zone_page_state(page, NR_ZSPAGES); - inc_zone_page_state(newpage, NR_ZSPAGES); + zpdesc_get(newzpdesc); + if (zpdesc_zone(newzpdesc) !=3D zpdesc_zone(zpdesc)) { + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_inc_zone_page_state(newzpdesc); } =20 reset_page(page); - put_page(page); + zpdesc_put(zpdesc); =20 return MIGRATEPAGE_SUCCESS; } --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C29E5148FF5 for ; Mon, 29 Jul 2024 11:20:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252053; cv=none; b=Aj2fZlYfWrsPECqHwxcsuMu9MoK35fUGFfM8VVFwbMCeqtG0u76LfbNPYU2i3QkiYfo/ETGau0vcAR5K4ruiwA7/s3tFzhEr9hlnTDVLUMOf24tCE7vHPtk78O7LK8BY/wwci7bODjZYY2vj6G2JtvQrzcwmzSEP+SoOhV/rvmo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252053; c=relaxed/simple; bh=d7GxkLwHXdQyb6XsJEXUauGOBeH6/SyL3Iuz9vNzPwE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kMfKC4clz1olYaEowG1UMDnw7sjuXE7BJcwLSPIVQI6czB6gOZnUVtus6Y5W8rBNfaBX8gqaiKoV2BdsVB4uLWSPyxrC8vJ0tJH7rOonkYmqGNxWGJyDjOq+7R4l0+Fb4O9akNpfh9EzVcCHD1d5zV0pwAPboDsKoSdNFNvVS4k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y1xfuqoT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y1xfuqoT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DD9A5C4AF0A; Mon, 29 Jul 2024 11:20:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252053; bh=d7GxkLwHXdQyb6XsJEXUauGOBeH6/SyL3Iuz9vNzPwE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y1xfuqoTF8RBa6k5ApOc1e9IzEm4y5wIehqq1Lak4a/tussklfMkbwl3E865wK7Sg NAqDweP7jioqqU8PNGlclmJy+bVTfwgSxrpH6Bn7Cs8W3/dLWLQpvp+BcZMpqoO2qN 9d8znmyUTNzACuqjIZz0RwW+widCJiDk03ZWFhnpT0Q3dPKfYpQAwwAS3wU3LIj0Yw 7YVZuuB/lWRsQeG2lg59DOcbTCrKpJI5IUFSFF4RsrlXmif7ZmLyWSf4nerSPMXzZm o594WC/rvrMnYCSgW/pSL0QNyaXpuXUSFzjnSM+XwWbty1FHL/hvjGQSyRY2kdY73J IySG9ESyAF6tg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 11/22] mm/zsmalloc: rename reset_page to reset_zpdesc and use zpdesc in it Date: Mon, 29 Jul 2024 19:25:23 +0800 Message-ID: <20240729112534.3416707-12-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi zpdesc.zspage matches with page.private, zpdesc.next matches with page.index. They will be reset in reset_page() wich is called prior to free base pages of a zspage. Use zpdesc to replace page struct and rename it to reset_zpdesc(), few page helper still left since they are used too widely. Signed-off-by: Alex Shi --- mm/zsmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 9bc9b14187ed..6d1971836391 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -865,12 +865,14 @@ static inline bool obj_allocated(struct zpdesc *zpdes= c, void *obj, return true; } =20 -static void reset_page(struct page *page) +static void reset_zpdesc(struct zpdesc *zpdesc) { + struct page *page =3D zpdesc_page(zpdesc); + __ClearPageMovable(page); ClearPagePrivate(page); - set_page_private(page, 0); - page->index =3D 0; + zpdesc->zspage =3D NULL; + zpdesc->next =3D NULL; reset_first_obj_offset(page); __ClearPageZsmalloc(page); } @@ -910,7 +912,7 @@ static void __free_zspage(struct zs_pool *pool, struct = size_class *class, do { VM_BUG_ON_PAGE(!PageLocked(page), page); next =3D get_next_page(page); - reset_page(page); + reset_zpdesc(page_zpdesc(page)); unlock_page(page); dec_zone_page_state(page, NR_ZSPAGES); put_page(page); @@ -1899,7 +1901,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, zpdesc_inc_zone_page_state(newzpdesc); } =20 - reset_page(page); + reset_zpdesc(zpdesc); zpdesc_put(zpdesc); =20 return MIGRATEPAGE_SUCCESS; --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5770C149DFA for ; Mon, 29 Jul 2024 11:20:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252057; cv=none; b=r/Fvjp1dUlXeGtMCMf6kUhsQKxSa2RY7BgWyxlZwofFMUDP51NXqic3h4E58bm8yLH4wInc3bqpM+COALftWJ/1RwkTcXaY9JPdrFmtZhM7xFUY5k2HZ65ODEnHFfq9hllz0+cy2UQsX51/3cDBvlY9Ap+DGzKS/NrQWdubHlDM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252057; c=relaxed/simple; bh=/reSYv2I3LBpywbB8eW2CIXO42cQ1NynjFn2dBL53LI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DvbTDP7Tkk+DZfVxHmgFgILIGTG3mMfd6izbF2eDJqY7JGIL8Pl51AHI0zE+I1hwC2mZhIh641mCsaKoPl0M2YOxYkMk3k7kuE7801vznoQsGEluYtuAL04TSBuyKKtExnKKVb5EySCFY6eqbNUmTJdCHl1W90ahZ7ABtqMorpc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Q6Z5z02c; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q6Z5z02c" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 278FFC32786; Mon, 29 Jul 2024 11:20:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252056; bh=/reSYv2I3LBpywbB8eW2CIXO42cQ1NynjFn2dBL53LI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q6Z5z02cPsiz04Im8TmiPZ0LRtDPvb7j/Z7gsVKUC+vY0bQTeAukeCpSUVaodx8qH 7CIS0LLlHtEwTCDlK7Odzu8VbidIbZIBvA+q3YjUryx+a4UHVnii/qXAjvSBqdjqvR 4tD5ERJ0Pywjn30AShewyQPZTcxGolFIewnVmUTha0nTO+e+wgxNPbfao/L22WkvZ1 QIRFfedl+/ckWEC0oJ4OGBcsN0u7rvH6OsXtffWwPRJe4Y1RQMf034J5/jRZhT06aC Q6nd6bZ+aSHBwOqbIJ778lEu/5fNxKYZs9Xzu4YeC2QTW6M27X8mr9eUEenpkCqCLq YG69szOAWaPKw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 12/22] mm/zsmalloc: convert __free_zspage() to use zdsesc Date: Mon, 29 Jul 2024 19:25:24 +0800 Message-ID: <20240729112534.3416707-13-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Introduce zpdesc_is_locked() and convert __free_zspage() to use zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 4 ++++ mm/zsmalloc.c | 20 ++++++++++---------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index ad04c8337cae..72c8c072b4c8 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -119,4 +119,8 @@ static inline struct zone *zpdesc_zone(struct zpdesc *z= pdesc) return page_zone(zpdesc_page(zpdesc)); } =20 +static inline bool zpdesc_is_locked(struct zpdesc *zpdesc) +{ + return PageLocked(zpdesc_page(zpdesc)); +} #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6d1971836391..68fdea7b6e0d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -901,23 +901,23 @@ static int trylock_zspage(struct zspage *zspage) static void __free_zspage(struct zs_pool *pool, struct size_class *class, struct zspage *zspage) { - struct page *page, *next; + struct zpdesc *zpdesc, *next; =20 assert_spin_locked(&class->lock); =20 VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness !=3D ZS_INUSE_RATIO_0); =20 - next =3D page =3D get_first_page(zspage); + next =3D zpdesc =3D get_first_zpdesc(zspage); do { - VM_BUG_ON_PAGE(!PageLocked(page), page); - next =3D get_next_page(page); - reset_zpdesc(page_zpdesc(page)); - unlock_page(page); - dec_zone_page_state(page, NR_ZSPAGES); - put_page(page); - page =3D next; - } while (page !=3D NULL); + VM_BUG_ON_PAGE(!zpdesc_is_locked(zpdesc), zpdesc_page(zpdesc)); + next =3D get_next_zpdesc(zpdesc); + reset_zpdesc(zpdesc); + zpdesc_unlock(zpdesc); + zpdesc_dec_zone_page_state(zpdesc); + zpdesc_put(zpdesc); + zpdesc =3D next; + } while (zpdesc !=3D NULL); =20 cache_free_zspage(pool, zspage); =20 --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E09814659C for ; Mon, 29 Jul 2024 11:21:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252060; cv=none; b=Cr87Ow7Vq35prP9FiCo8ER6eBMVFWmp1escTKBdjD5zQGCKLBYhMlniKRh/HJZcHm/A8tAhMYEMEVET45vU+P1XZJ/OfMLVaqWiSJZfDo/bTueBkiaUzFB9hERm1Pxe/4y4Mapy0xlAEEIzy37sWd2mIy3sTxHw/i9rqd3GoBlE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252060; c=relaxed/simple; bh=wTDI0eYHYt7QQuqfYVSh3nHEsVuSO77v1MHzFFaeoFg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=utmWBfS23nBMQyNJ2+Q57eZnWMEtcTSOZxR/d4M20x0SSonPWtS8Dih6GFnNht0aOqKaHQROYX0oJQT1e/983iBw+2U/Y1Ll2uwi/f5gPAHaQ/4+GUL+wwHE+aKpr10kqShHqJKVazbNhgsdrn3BjDulsbQ0P2zUOeVVnL/fL44= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=soKprBYq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="soKprBYq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 64837C4AF07; Mon, 29 Jul 2024 11:20:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252060; bh=wTDI0eYHYt7QQuqfYVSh3nHEsVuSO77v1MHzFFaeoFg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=soKprBYqWURfg25sfItqMlUmjCEOrgiYtNAVU0eDQr68G1KVfp/73qZOeW7TOTupC a4MzcCvbu8bHoU4pupUZtFb30ceJnWgg7D5Dgq74Dc8Lu42X2pxdIrA1XU8XJoRuTr XKCxYMaMOprZnX8ZS/+ih6s8whodTe32MyaDzTAgQr/CuJ/erKwRgmD2ZTw0b73hGS 1rQiKJMZNSv8vii8ZRFqvcMR1hnxy9BF4gx+gEwTHVcXLWhB4Fap9uUH9Om8FWwo13 O6qln7YSujzOPOMEVbMY6RhnaWzjgGqu53rtHRuM8XNeT3KFntHVNBXeVkhf4nO+Ob 8tHZuDQsRuUMQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 13/22] mm/zsmalloc: convert location_to_obj() to take zpdesc Date: Mon, 29 Jul 2024 19:25:25 +0800 Message-ID: <20240729112534.3416707-14-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> As all users of location_to_obj() now use zpdesc, convert location_to_obj() to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 68fdea7b6e0d..e291c7319485 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -826,15 +826,15 @@ static void obj_to_zpdesc(unsigned long obj, struct z= pdesc **zpdesc) } =20 /** - * location_to_obj - get obj value encoded from (, ) - * @page: page object resides in zspage + * location_to_obj - get obj value encoded from (, ) + * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index */ -static unsigned long location_to_obj(struct page *page, unsigned int obj_i= dx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx) { unsigned long obj; =20 - obj =3D page_to_pfn(page) << OBJ_INDEX_BITS; + obj =3D zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; obj |=3D obj_idx & OBJ_INDEX_MASK; =20 return obj; @@ -1392,7 +1392,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_atomic(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(zpdesc_page(m_zpdesc), obj); + obj =3D location_to_obj(m_zpdesc, obj); record_obj(handle, obj); =20 return obj; @@ -1879,8 +1879,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(zpdesc_page(newzpdesc), - obj_idx); + new_obj =3D (unsigned long)location_to_obj(newzpdesc, obj_idx); record_obj(handle, new_obj); } } --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EF701465A1 for ; Mon, 29 Jul 2024 11:21:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252063; cv=none; b=aNiX8FpQa9WfrH4pyrNOj9S1pPL1t/cKLTshqo485efxAD5gBiylwlZc1Bopp5WsgfjLosNpQwd/bNEMZGu4GNrph7Tu8jp/BuwZuzRysdxEj0Hi43WbYMck+RHkBTCVOgfxLhrJqeQQfazPbggCt8hge34YoS2I8euBRjsWn+8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252063; c=relaxed/simple; bh=K3GUIiHemMLy8fa8RNH072Iq/pXEu/mTjv8z/3myXkU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MNljwLlWCzrYw5Iagc+T4/1UKkmjwMbUjzFFOUdEO9G0mUbxadosey1ueXF0lpWocxYIpOOHURMApaWglBiXd27oDaidHxG2LV9Dci5CXXhLHSGAqiV+mNcBP2Soj1b9/oqabJr85uJQjNpvXcpt+vy+S8ucxKCEnf0tbV06LNQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=o5gCrlI0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="o5gCrlI0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A282AC4AF0C; Mon, 29 Jul 2024 11:21:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252063; bh=K3GUIiHemMLy8fa8RNH072Iq/pXEu/mTjv8z/3myXkU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=o5gCrlI0q81yaT0XVXU2oqnN965qfPvIFAFcpm5o8+/9uozJOlLo7KU+gwBQ+kwOP iAnLxsVxjYVw3zhS0k4gOkFU4woJ9SVcQP/bLhknsUR1zUBYaLZMii9yqmeKAuapKI U/yfn6gsE25TokO5tBSpDviaM4NXHWSY5+3PCs9Yvqwi0cNmLTS6l+swpRFFoUEw47 glc2GKCDOuvKDd4g1VWHlltI66gYTXQC67wpWqGAQwrBq4RLqouYP96rjmrEw+3uab QDMOv8TaAYsoO6R5qrWC/i34oo75n8qUzH2KXjSeYp09IZ7uZoph5jQun18uSGhPaj mQRK84Tevde6Q== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 14/22] mm/zsmalloc: convert migrate_zspage() to use zpdesc Date: Mon, 29 Jul 2024 19:25:26 +0800 Message-ID: <20240729112534.3416707-15-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Use get_first_zpdesc/get_next_zpdesc to replace get_first/next_page. No functional change. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e291c7319485..7d039b0c66db 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1636,14 +1636,14 @@ static void migrate_zspage(struct zs_pool *pool, st= ruct zspage *src_zspage, unsigned long used_obj, free_obj; unsigned long handle; int obj_idx =3D 0; - struct page *s_page =3D get_first_page(src_zspage); + struct zpdesc *s_zpdesc =3D get_first_zpdesc(src_zspage); struct size_class *class =3D pool->size_class[src_zspage->class]; =20 while (1) { - handle =3D find_alloced_obj(class, page_zpdesc(s_page), &obj_idx); + handle =3D find_alloced_obj(class, s_zpdesc, &obj_idx); if (!handle) { - s_page =3D get_next_page(s_page); - if (!s_page) + s_zpdesc =3D get_next_zpdesc(s_zpdesc); + if (!s_zpdesc) break; obj_idx =3D 0; continue; --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C677A14F133 for ; Mon, 29 Jul 2024 11:21:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252066; cv=none; b=uGEgWn99UvPFOGks6LDyPJR/4xaYu3tFrVbmaO85kW8R3Fxcej6iuoHxKyD571m53tvBxIeNJN82Zw+uW5MCzL1TcTc74cXqC3mePvsdYWOYIh1ZPvn1WXN95lbshCxVatnQn0+URztijK8OdZAh+m/yZxinmbNICdVjZHZl2DU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252066; c=relaxed/simple; bh=0AvWtekA1fBYWEoFvuJjRv8WAqdxESdkBRdAezOabRE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=M1ygd6FFd0zwY9kpJ6BjdCAUMNXfeNQ3SvHAa+1qYU0jc1ZkQaHiAQKmr4imzFE7Ulrny+X6ZyaEaOpvBaEx5RiDp2gu0lQgW7mhlT/MXkHz+6kwSZL9j82MaUlZaM4wmnrtiKZBJUPPq9tRa73YMmTXzh1WY7+q2JJGQKqRcjs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NUq3ixaQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NUq3ixaQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DE97DC32786; Mon, 29 Jul 2024 11:21:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252066; bh=0AvWtekA1fBYWEoFvuJjRv8WAqdxESdkBRdAezOabRE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NUq3ixaQ6u5c4OlwYwdVAJz07eKqVSBQDfBqsWK7nIM0s3VhqwQM/W5WH+38OmX5Q NdDZZABPrBrX/v4zQp5TFRvQHj1pd9c6rvOsn9KYIALCUJ57wdxf4CZQLZQzP23tEk uWbK0FAj5zbEjBN/yhkX4BM13sZH5rmKIGH1X6Zo/yGti7w4jcxTITcf5U6AC04r+i 7J1jFuwdbJdJ3f/JJlBlQ5q7HgEhUuAMtDiij8FsSs48lAraDh/Y92I/uVRmZgL62L wa3A9LbcmwL5fg6tBqdWjMkMRqO662420Fdy4oeamfSZBwv5R2p73tbcBwAqB351ox /xeYlJgosEmEA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 15/22] mm/zsmalloc: convert get_zspage() to take zpdesc Date: Mon, 29 Jul 2024 19:25:27 +0800 Message-ID: <20240729112534.3416707-16-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Now that all users except get_next_page() (which will be removed in later patch) use zpdesc, convert get_zspage() to take zpdesc instead of page. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7d039b0c66db..458ad696b473 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -779,9 +779,9 @@ static int fix_fullness_group(struct size_class *class,= struct zspage *zspage) return newfg; } =20 -static struct zspage *get_zspage(struct page *page) +static struct zspage *get_zspage(struct zpdesc *zpdesc) { - struct zspage *zspage =3D (struct zspage *)page_private(page); + struct zspage *zspage =3D zpdesc->zspage; =20 BUG_ON(zspage->magic !=3D ZSPAGE_MAGIC); return zspage; @@ -789,7 +789,7 @@ static struct zspage *get_zspage(struct page *page) =20 static struct page *get_next_page(struct page *page) { - struct zspage *zspage =3D get_zspage(page); + struct zspage *zspage =3D get_zspage(page_zpdesc(page)); =20 if (unlikely(ZsHugePage(zspage))) return NULL; @@ -799,7 +799,7 @@ static struct page *get_next_page(struct page *page) =20 static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { - struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage =3D get_zspage(zpdesc); =20 if (unlikely(ZsHugePage(zspage))) return NULL; @@ -849,7 +849,7 @@ static inline bool obj_allocated(struct zpdesc *zpdesc,= void *obj, unsigned long *phandle) { unsigned long handle; - struct zspage *zspage =3D get_zspage(zpdesc_page(zpdesc)); + struct zspage *zspage =3D get_zspage(zpdesc); =20 if (unlikely(ZsHugePage(zspage))) { VM_BUG_ON_PAGE(!is_first_zpdesc(zpdesc), zpdesc_page(zpdesc)); @@ -1265,7 +1265,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned lo= ng handle, read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); =20 /* * migration cannot move any zpages in this zspage. Here, class->lock @@ -1315,7 +1315,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned l= ong handle) =20 obj =3D handle_to_obj(handle); obj_to_location(obj, &zpdesc, &obj_idx); - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 @@ -1479,7 +1479,7 @@ static void obj_free(int class_size, unsigned long ob= j) =20 obj_to_location(obj, &f_zpdesc, &f_objidx); f_offset =3D offset_in_page(class_size * f_objidx); - zspage =3D get_zspage(zpdesc_page(f_zpdesc)); + zspage =3D get_zspage(f_zpdesc); =20 vaddr =3D zpdesc_kmap_atomic(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); @@ -1513,7 +1513,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) read_lock(&pool->migrate_lock); obj =3D handle_to_obj(handle); obj_to_zpdesc(obj, &f_zpdesc); - zspage =3D get_zspage(zpdesc_page(f_zpdesc)); + zspage =3D get_zspage(f_zpdesc); class =3D zspage_class(pool, zspage); spin_lock(&class->lock); read_unlock(&pool->migrate_lock); @@ -1846,7 +1846,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, __SetPageZsmalloc(zpdesc_page(newzpdesc)); =20 /* The page is locked, so this pointer must remain valid */ - zspage =3D get_zspage(zpdesc_page(zpdesc)); + zspage =3D get_zspage(zpdesc); pool =3D zspage->pool; =20 /* --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1203B153598 for ; Mon, 29 Jul 2024 11:21:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252070; cv=none; b=dYAmD5GN7Z4gdHxVu7FWlQ+2rPwcCuwPdHguBk3AeRCzyw76gOqwRkQC/XQzOfINLzDxdOMu7nH9Ooi2YtNv07fcIqn+M0CRuFo0+ReYfN5G2/KE3HGeD5r/aVdySzo9RJHgEnWvnHIzVRXgTu+gzNPxwEygNA3cn3yZlsZGdRg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252070; c=relaxed/simple; bh=uH0rZNnpHFMljEmSyKQuSSlLph2dNP0xTWvseJvNDaY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LOb+uuWF782ZTDCBMpclSeyKLbsCqUcKSj3yRBHLaJGJLwY3UiLgzIHaH563LZqr+4qCNUz77V1QI8qMGPVAkmel8Q3ZrWMz8Uj6cy2B8Q3B/gOZsI0pQQtGi7tfYOGchEmxxrysBI1TN8abpdnxWNRHLWbFPMa0motsvCVfats= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Cis/sQct; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Cis/sQct" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2CC73C32786; Mon, 29 Jul 2024 11:21:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252069; bh=uH0rZNnpHFMljEmSyKQuSSlLph2dNP0xTWvseJvNDaY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Cis/sQct2oie4isbX0P4WgsHRrGtWLVh7YFLlWHN4Hqxb4Ps0mGmDN5s2E3j4uoLw hcbPLuM3mQCdC8wAZKHt/Fi5LPEaPrAd5zhVLwPvCiqEEsOOljP8QGnJnBIP7iuO2o denYHvyN+8K1G9sUD4xLP82ZAtOnzeMNsYkan3a5EtkxsS9BZddKDmb03Z5zXVaANg aZe32kqPkj3x+SuS/BhLrA2bB+Mxs7Lto2RiN/O3apyJO7inUD8qPFSXMkT1gMuLKV 1rf5SV/fqXZ108XIFo7MPKqAEtCY8JczZCXeoWKST8sS8Bl1na7887/WCHXFmUE+/3 CBqLzo08RhhPg== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 16/22] mm/zsmalloc: convert SetZsPageMovable and remove unused funcs Date: Mon, 29 Jul 2024 19:25:28 +0800 Message-ID: <20240729112534.3416707-17-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Convert SetZsPageMovable() to use zpdesc, and then remove unused funcs:get_next_page/get_first_page/is_first_page. Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zsmalloc.c | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 458ad696b473..8b713ac03902 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -468,11 +468,6 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area= ) =3D { .lock =3D INIT_LOCAL_LOCK(lock), }; =20 -static __maybe_unused int is_first_page(struct page *page) -{ - return PagePrivate(page); -} - static int is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); @@ -489,14 +484,6 @@ static inline void mod_zspage_inuse(struct zspage *zsp= age, int val) zspage->inuse +=3D val; } =20 -static inline struct page *get_first_page(struct zspage *zspage) -{ - struct page *first_page =3D zpdesc_page(zspage->first_zpdesc); - - VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); - return first_page; -} - static struct zpdesc *get_first_zpdesc(struct zspage *zspage) { struct zpdesc *first_zpdesc =3D zspage->first_zpdesc; @@ -787,16 +774,6 @@ static struct zspage *get_zspage(struct zpdesc *zpdesc) return zspage; } =20 -static struct page *get_next_page(struct page *page) -{ - struct zspage *zspage =3D get_zspage(page_zpdesc(page)); - - if (unlikely(ZsHugePage(zspage))) - return NULL; - - return (struct page *)page->index; -} - static struct zpdesc *get_next_zpdesc(struct zpdesc *zpdesc) { struct zspage *zspage =3D get_zspage(zpdesc); @@ -1970,13 +1947,13 @@ static void init_deferred_free(struct zs_pool *pool) =20 static void SetZsPageMovable(struct zs_pool *pool, struct zspage *zspage) { - struct page *page =3D get_first_page(zspage); + struct zpdesc *zpdesc =3D get_first_zpdesc(zspage); =20 do { - WARN_ON(!trylock_page(page)); - __SetPageMovable(page, &zsmalloc_mops); - unlock_page(page); - } while ((page =3D get_next_page(page)) !=3D NULL); + WARN_ON(!zpdesc_trylock(zpdesc)); + __zpdesc_set_movable(zpdesc, &zsmalloc_mops); + zpdesc_unlock(zpdesc); + } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); } #else static inline void zs_flush_migration(struct zs_pool *pool) { } --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E393153598 for ; Mon, 29 Jul 2024 11:21:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252073; cv=none; b=MQi1C2NIo0yS4oE0pu51g/8ajtWPRlF0aG5T9+6tsVWdCDEYoVOVe8tfLQJaq2sDqm0MyWGJqjgg0BLfjWOnpsogb+SA+LZB6mld4leY1nrAf1NvMByFZUTEgg2Pe8cpBSu5j16Lkd/WPLOW5CQahMpFXKkfQuXQJhJwr15WGYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252073; c=relaxed/simple; bh=Itfg7R6iNeKIMJsSCXaa7VK0PRdQ2psxEdNvilX7MGw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GRTFBgbKpAa64+MLGPA3EcBvI404/hwZWHu6toQ03uIGnuKWcFDt+CSG5D6nFa3nL1VHy9e7uqhehNdWo57C92NQg/K3k2nY4dNXRQZvZmRRLCpTaqyAwVV20SaZPqZ/c0xv9RdeBZ8asmO7GjLVJnRfI6IjQPH+XyaLnlmHPvo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WKtVL170; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WKtVL170" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 693D1C4AF11; Mon, 29 Jul 2024 11:21:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252073; bh=Itfg7R6iNeKIMJsSCXaa7VK0PRdQ2psxEdNvilX7MGw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WKtVL170fOP1O4XLSDWo1F250iq5RJ80EenjRyd7ZFOT2WvuJuInoIyx92SwRS7Cc pbrGzXYKmIIAT4QtnvMB8ThaarUUedL3E5+TlS+UtYM+YJsxN2p5XSlso+e1SHSb5M xTUP2ijw2w7jJF69q2NiOoesk4j1NPgHOtJ1w0TS2tjj2RqoWf5SEtWI/DS+hgTA/t jwIzd+zFQpYJUj4XC3xJkx35hp9rlt9Ju0MpE+vnq/J8BM6koz0K0BkDwwpac68UpE 0W/f4fyzHaM9zCNTqtvtVyYatxW07+UpA2zA8fi8x/CPXDjyEwZFw9NR4VBwQLbWRv TGyKYh/tyozJw== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 17/22] mm/zsmalloc: convert get/set_first_obj_offset() to take zpdesc Date: Mon, 29 Jul 2024 19:25:29 +0800 Message-ID: <20240729112534.3416707-18-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Now that all users of get/set_first_obj_offset() are converted to use zpdesc, convert them to take zpdesc. Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 7 ++++++- mm/zsmalloc.c | 36 ++++++++++++++++++------------------ 2 files changed, 24 insertions(+), 19 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 72c8c072b4c8..f64e813f4847 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -15,6 +15,8 @@ * @next: Next zpdesc in a zspage in zsmalloc zpool * @handle: For huge zspage in zsmalloc zpool * @zspage: Pointer to zspage in zsmalloc + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly use by page migration * @memcg_data: Memory Control Group data. * * This struct overlays struct page for now. Do not modify without a good @@ -31,7 +33,8 @@ struct zpdesc { unsigned long handle; }; struct zspage *zspage; - unsigned long _zp_pad_1; + unsigned int first_obj_offset; + atomic_t _refcount; #ifdef CONFIG_MEMCG unsigned long memcg_data; #endif @@ -45,6 +48,8 @@ ZPDESC_MATCH(mapping, mops); ZPDESC_MATCH(index, next); ZPDESC_MATCH(index, handle); ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); +ZPDESC_MATCH(_refcount, _refcount); #ifdef CONFIG_MEMCG ZPDESC_MATCH(memcg_data, memcg_data); #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 8b713ac03902..bb8b5f13a966 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -20,8 +20,8 @@ * zpdesc->next: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. - * page->page_type: PG_zsmalloc, lower 16 bit locate the first object - * offset in a subpage of a zspage + * zpdesc->first_obj_offset: PG_zsmalloc, lower 16 bit locate the first + * object offset in a subpage of a zspage * * Usage of struct zpdesc(page) flags: * PG_private: identifies the first component page @@ -494,26 +494,26 @@ static struct zpdesc *get_first_zpdesc(struct zspage = *zspage) =20 #define FIRST_OBJ_PAGE_TYPE_MASK 0xffff =20 -static inline void reset_first_obj_offset(struct page *page) +static inline void reset_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - page->page_type |=3D FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + zpdesc->first_obj_offset |=3D FIRST_OBJ_PAGE_TYPE_MASK; } =20 -static inline unsigned int get_first_obj_offset(struct page *page) +static inline unsigned int get_first_obj_offset(struct zpdesc *zpdesc) { - VM_WARN_ON_ONCE(!PageZsmalloc(page)); - return page->page_type & FIRST_OBJ_PAGE_TYPE_MASK; + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); + return zpdesc->first_obj_offset & FIRST_OBJ_PAGE_TYPE_MASK; } =20 -static inline void set_first_obj_offset(struct page *page, unsigned int of= fset) +static inline void set_first_obj_offset(struct zpdesc *zpdesc, unsigned in= t offset) { /* With 16 bit available, we can support offsets into 64 KiB pages. */ BUILD_BUG_ON(PAGE_SIZE > SZ_64K); - VM_WARN_ON_ONCE(!PageZsmalloc(page)); + VM_WARN_ON_ONCE(!PageZsmalloc(zpdesc_page(zpdesc))); VM_WARN_ON_ONCE(offset & ~FIRST_OBJ_PAGE_TYPE_MASK); - page->page_type &=3D ~FIRST_OBJ_PAGE_TYPE_MASK; - page->page_type |=3D offset & FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset &=3D ~FIRST_OBJ_PAGE_TYPE_MASK; + zpdesc->first_obj_offset |=3D offset & FIRST_OBJ_PAGE_TYPE_MASK; } =20 static inline unsigned int get_freeobj(struct zspage *zspage) @@ -850,7 +850,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) ClearPagePrivate(page); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; - reset_first_obj_offset(page); + reset_first_obj_offset(zpdesc); __ClearPageZsmalloc(page); } =20 @@ -934,7 +934,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) struct link_free *link; void *vaddr; =20 - set_first_obj_offset(zpdesc_page(zpdesc), off); + set_first_obj_offset(zpdesc, off); =20 vaddr =3D zpdesc_kmap_atomic(zpdesc); link =3D (struct link_free *)vaddr + off / sizeof(*link); @@ -1589,7 +1589,7 @@ static unsigned long find_alloced_obj(struct size_cla= ss *class, unsigned long handle =3D 0; void *addr =3D zpdesc_kmap_atomic(zpdesc); =20 - offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + offset =3D get_first_obj_offset(zpdesc); offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { @@ -1784,8 +1784,8 @@ static void replace_sub_page(struct size_class *class= , struct zspage *zspage, } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 create_page_chain(class, zspage, zpdescs); - first_obj_offset =3D get_first_obj_offset(zpdesc_page(oldzpdesc)); - set_first_obj_offset(zpdesc_page(newzpdesc), first_obj_offset); + first_obj_offset =3D get_first_obj_offset(oldzpdesc); + set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) newzpdesc->handle =3D oldzpdesc->handle; __zpdesc_set_movable(newzpdesc, &zsmalloc_mops); @@ -1840,7 +1840,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); =20 - offset =3D get_first_obj_offset(zpdesc_page(zpdesc)); + offset =3D get_first_obj_offset(zpdesc); s_addr =3D zpdesc_kmap_atomic(zpdesc); =20 /* --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAA32145B37 for ; Mon, 29 Jul 2024 11:21:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252076; cv=none; b=rT0yNRHMzDvy+Onnk4DNHhNf/1ClD9c/SOVb5ImPBB6X3wFhMWELeVcsuTCd5NtdVAbIY4Z5gyXnWvTLcVqldP+BMKI1bJT3dTG/6g8YOrNObVUHEyJLy2p6eELHZhK6rsdIrNgQM/q1y5kr4DeUwTnJ9qrx7ksG0KCPKuv9htY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252076; c=relaxed/simple; bh=WC1zVtU8HoKTgwyGG1oBnxz/TQGKFBik73v06jNNZ/o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GDFmxIhaycwmSxCkmwXbLDuIAbODdz9rJUR8NuAIsuGCJ9L3Bcj83wEjPpTgClvKd23zq+gwGUnafqWvcx8NVKf4GAhjO1eJdgVZrHeCoDN42Q24c6iNK0JSgPViFZG35fdukXUEyE5290ktGq971EygEmpcwKKpVk4iqx+g0kA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Zp7crNGG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Zp7crNGG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id ABF10C32786; Mon, 29 Jul 2024 11:21:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252076; bh=WC1zVtU8HoKTgwyGG1oBnxz/TQGKFBik73v06jNNZ/o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Zp7crNGGj8tFuQ+KKFsA4x+ASc1R4kKGuxUypiz+7x5QReIhm0x2m4fG7Jjw/wU+2 8asbu0zMOQYtzwO97Hj1gyyuvmxrSeh3uHcz1xtom7DMR1m4wA6K2gKnUWMi44cLC9 4RsUSKHEarV7mkuTqCVzvlvPMh9hpvgdyVTqCbqSIzcbTBHmMMk83DNuMPyA1s37Yw HgeUQ654r3M6IAG2fRbj+a5gogYPOcjTax9pkat/I3+H6NocMAmoG8dLxWd6Ke1Fmd TAyY1llDoLoeJjnLyQASqhE6ahUVKMGbJsipzXyx5hy5YgFAomQu8YaHzpo3BHs5Rz 8/Ugyd+q+VGcA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 18/22] mm/zsmalloc: introduce __zpdesc_clear_movable Date: Mon, 29 Jul 2024 19:25:30 +0800 Message-ID: <20240729112534.3416707-19-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add a helper __zpdesc_clear_movable() for __ClearPageMovable(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index f64e813f4847..5db4fbe2d139 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -114,6 +114,11 @@ static inline void __zpdesc_set_movable(struct zpdesc = *zpdesc, __SetPageMovable(zpdesc_page(zpdesc), mops); } =20 +static inline void __zpdesc_clear_movable(struct zpdesc *zpdesc) +{ + __ClearPageMovable(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index bb8b5f13a966..e1d3ad50538c 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -846,7 +846,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) { struct page *page =3D zpdesc_page(zpdesc); =20 - __ClearPageMovable(page); + __zpdesc_clear_movable(zpdesc); ClearPagePrivate(page); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 246DC145B38 for ; Mon, 29 Jul 2024 11:21:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252080; cv=none; b=TOM1fpDWFgzmLl+YOhLBqHaccYkTRVtJicFeOPMW5k3ROzxkHyU0IvNXXI8yXyGBWAay0E4ZAw5Br9GDPkUYWwqerqmi/kvm1DcPtrgUadz+oYwpiFDQicQYiMEovxe5Eqor9GM1CnZ1kp4gmDZaMo9hzwC99+j5fEo49Wc7tyY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252080; c=relaxed/simple; bh=PC5jkQnfMoZxwZVAO0ES1OaCQkwLNtrVhI+Kr1f/doM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fBt/WgPXlYbJtyajLevKgLv0GFStcQjlaefM8t1lAMxWMECgVqz3CNG6izCvruBi5lXI6jaJU7Bp5AWwxaTi8H5y/hJudd/2/yl+T9WbVr2NgsYR+mpT+60SNrQWdu+bPX7dV/DIEiXyQf1Jm/foNy+4WtKj9jWHZXzMnRgaH/c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NpQwJ0V/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NpQwJ0V/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7CF5C4AF07; Mon, 29 Jul 2024 11:21:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252079; bh=PC5jkQnfMoZxwZVAO0ES1OaCQkwLNtrVhI+Kr1f/doM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NpQwJ0V/9xj+pd3NV/uPFqOuWZuUCH5VTltgNIR8JOiw68vnQc/mmjRL0R+hJWPxS Dqj7sdu4p9fgtW71tq2acbFtvgIwIS+KcTUjHrotVN/yVR/dZRkf2/NTlNH88FSujg vs/cboP6T/die3KPnUcQ3a7vLIHFQAd7vhxELPhr0uAgH0yLI9u4oj4KCueAQQtQ+j hfJa1OSHq5027FOLujL3t3jPAv+nnvPLuFDnOI8ZBS8pVQPq+XweejJbEpa+AAFBUE 9ODKZcowCZRBUoWE/Qunc4kTxmTQBQGxTG7Ktvz9eiAMTsjs7iqp18x5xpqq67Z7H0 NU6nTU1sOW4uQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 19/22] mm/zsmalloc: introduce __zpdesc_clear_zsmalloc Date: Mon, 29 Jul 2024 19:25:31 +0800 Message-ID: <20240729112534.3416707-20-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add a helper __zpdesc_clear_zsmalloc() for __ClearPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 5db4fbe2d139..05def4d45265 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -119,6 +119,11 @@ static inline void __zpdesc_clear_movable(struct zpdes= c *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } =20 +static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) +{ + __ClearPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline bool zpdesc_is_isolated(struct zpdesc *zpdesc) { return PageIsolated(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index e1d3ad50538c..d88602fb0233 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -851,7 +851,7 @@ static void reset_zpdesc(struct zpdesc *zpdesc) zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; reset_first_obj_offset(zpdesc); - __ClearPageZsmalloc(page); + __zpdesc_clear_zsmalloc(zpdesc); } =20 static int trylock_zspage(struct zspage *zspage) @@ -1024,7 +1024,7 @@ static struct zspage *alloc_zspage(struct zs_pool *po= ol, if (!zpdesc) { while (--i >=3D 0) { zpdesc_dec_zone_page_state(zpdescs[i]); - __ClearPageZsmalloc(zpdesc_page(zpdescs[i])); + __zpdesc_clear_zsmalloc(zpdescs[i]); free_zpdesc(zpdescs[i]); } cache_free_zspage(pool, zspage); --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51E1F1553BB for ; Mon, 29 Jul 2024 11:21:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252083; cv=none; b=GUUXC6Ho+pLZj7Jr8pTL5gi3Tv4rkP1MMR/XeDvl/TvYEjVYVFSssfq/QbI7tThL0TsDslWT3wM7rVU3sjS26IMs4SJQZAvQ+jBJishifnQhYULv3ABP2NPbuZxJcG/gkA/mZ8+U0r/8dKYPCt2F8A3x1OShRuQKI60+oJbKiFY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252083; c=relaxed/simple; bh=U4TlC31Eh5dq6Szo/Iu40dA9+YSQztS7ohMuHhtrtaU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h4l5nLcIkXNjYhThn4TN3GQUmZMpl4Sgy1kw8jl+PwQbrk+WhmzP65asYtq1gN5KqVNFeJmE9t2/wHuo6L7f1gsJm6Hzq5Ah+2yEwxiRaAjU/75J14Pw0gE8pkFuHcEwdSOrHOajs5VoNrE1o15WArS5RD9VtWa3IMSAnP6mauc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hUHlbM/T; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hUHlbM/T" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 33FA2C32786; Mon, 29 Jul 2024 11:21:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252083; bh=U4TlC31Eh5dq6Szo/Iu40dA9+YSQztS7ohMuHhtrtaU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hUHlbM/T2v2RtXsHmp6QIE6F/mi7q6Je8OLzlD/lexCb2P1pX21JgHzDv/TbkSYP7 j/i58sijth6hxlaay3bpDMXYY1XTUB32nZXeDQI4c0iNeq3R8hqJif09oqqGFIHW03 ycgzafFlbbUM7z1E/+is1BED+Izlg3fW34+e+sdv7n+eawGF0JgSqqNp/X6hI06t5h 5wOzG3uFnpYyb7tHw6LyNAC3dMteobqFfLN1K5RoZB9LszFDuRCtlc7BElwgP1BgeK EliDMwNkMg93GXpaKvhxPKwZSgj8Go4CcBcdrVaY57+nI4Kzh5Ueooo/GqGxUhgsZ9 XljEaxsXtUaRA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 20/22] mm/zsmalloc: introduce __zpdesc_set_zsmalloc() Date: Mon, 29 Jul 2024 19:25:32 +0800 Message-ID: <20240729112534.3416707-21-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Add a helper __zpdesc_set_zsmalloc() for __SetPageZsmalloc(), and use it in callers to make code clear. Signed-off-by: Alex Shi --- mm/zpdesc.h | 5 +++++ mm/zsmalloc.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/zpdesc.h b/mm/zpdesc.h index 05def4d45265..06371ce60cd1 100644 --- a/mm/zpdesc.h +++ b/mm/zpdesc.h @@ -119,6 +119,11 @@ static inline void __zpdesc_clear_movable(struct zpdes= c *zpdesc) __ClearPageMovable(zpdesc_page(zpdesc)); } =20 +static inline void __zpdesc_set_zsmalloc(struct zpdesc *zpdesc) +{ + __SetPageZsmalloc(zpdesc_page(zpdesc)); +} + static inline void __zpdesc_clear_zsmalloc(struct zpdesc *zpdesc) { __ClearPageZsmalloc(zpdesc_page(zpdesc)); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d88602fb0233..7f8e02df4e3e 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1030,7 +1030,7 @@ static struct zspage *alloc_zspage(struct zs_pool *po= ol, cache_free_zspage(pool, zspage); return NULL; } - __SetPageZsmalloc(zpdesc_page(zpdesc)); + __zpdesc_set_zsmalloc(zpdesc); =20 zpdesc_inc_zone_page_state(zpdesc); zpdescs[i] =3D zpdesc; @@ -1820,7 +1820,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); =20 /* We're committed, tell the world that this is a Zsmalloc page. */ - __SetPageZsmalloc(zpdesc_page(newzpdesc)); + __zpdesc_set_zsmalloc(newzpdesc); =20 /* The page is locked, so this pointer must remain valid */ zspage =3D get_zspage(zpdesc); --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F38315574C for ; Mon, 29 Jul 2024 11:21:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252086; cv=none; b=QjwrlgL7tdI1cmFgRcvbVJqvcsEcL9xJtaq8zymX/bi+MlHJAaYHkeglL/N6ffm3qbs2x+i7vmTt1ayUkjOkQeK8bsntJ1RMBb/ZM9GKWk0OVEcE07tdw42acTMykdMz5BkLEJvvctVIIXvQ4fTyuk/xN8anb3iRFRK9l0PnEgQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252086; c=relaxed/simple; bh=lHz4+kv9JsOYqs60tdoZ6ATt9ROlCYt2854vH8gmrb4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V9kkTaMlhGlUdSEF2W+VQLqM4coXK7bsOmyA63qD+FN74nrk2lYeGgIv6cUF9ajH/yiuD8dnXDRJDeqjQvedlTr+ygcYA2Ogye0UGrXJKtxfs1sDEqxf8s7gi5/wRiEA4Dyvu9npI4fGPneG+qyVEWOUS/aTNirDxbi5Nzdc+G8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Q1CgdvR2; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q1CgdvR2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 707BAC4AF07; Mon, 29 Jul 2024 11:21:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252086; bh=lHz4+kv9JsOYqs60tdoZ6ATt9ROlCYt2854vH8gmrb4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q1CgdvR29F1u5T1ro7OnwYP8ZI6Z8m5hqoCiuU6o2r6/Llckc5OZClzt+xtMn3slx AFvT91UPiTj2FxS6Alp1vxaZ3e++kWdf5oS7M3Sj3DUCwEoJth7Ceb7JnkuabRtR0l R5iECzCnEGJmmx8aarCKgA4UNXSt71i1vHs3BXW+pArZP4RaS+ZMO6PZT+855B6LaB 9BGgI25HnzxXeJBvgkW+SHFyRqJ+tu43jYM6AEzD62G+fWy02YX00rzqkL0+2y4ACp 7UEsjmUXSG3nhoO+rx5BV2wgsV31LrkbcwddZpOqClNWsrD/liVk7o5QAksSAwuHQZ Db5SyjD/FlHIA== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi , kernel test robot Subject: [PATCH v4 21/22] mm/zsmalloc: fix build warning from lkp testing Date: Mon, 29 Jul 2024 19:25:33 +0800 Message-ID: <20240729112534.3416707-22-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi LKP reported the following warning w/o CONFIG_DEBUG_VM: mm/zsmalloc.c:471:12: warning: function 'is_first_zpdesc' is not needed and will not be emitted [-Wunneeded-internal-declaration] To remove this warning, better to incline the function is_first_zpdesc Reported-by: kernel test robot Closes: https://lore.kernel.org/oe-kbuild-all/202407052102.qbT7nLMK-lkp@int= el.com/ Signed-off-by: Alex Shi --- mm/zsmalloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7f8e02df4e3e..64e523ae71f8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -468,7 +468,7 @@ static DEFINE_PER_CPU(struct mapping_area, zs_map_area)= =3D { .lock =3D INIT_LOCAL_LOCK(lock), }; =20 -static int is_first_zpdesc(struct zpdesc *zpdesc) +static inline bool is_first_zpdesc(struct zpdesc *zpdesc) { return PagePrivate(zpdesc_page(zpdesc)); } --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7AE3155CB3 for ; Mon, 29 Jul 2024 11:21:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252089; cv=none; b=GYv7ZBnkq5WF8bjEaH1LQDdOFAZY+UpKe8nKVMjPlBZ7m1U/ePHTPzWzF80Btg0gHJ8TlkwxxfXBNwgMOyiuMS3xVBXT618x+iyYCI5VVfracJhBlHj4Mu6jw4wVA+YCpNYz9q3e+2HO6CvoIz3QB7YGAMg0KZ/tkL01WN5+fFo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722252089; c=relaxed/simple; bh=rElfU0d71iABwZ67rvpmRaqaEhOUs/Njjel6X7x+ZZU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p7ksORC8/0ieUVy5tMyLSQlXFAFCdTlVB1mKGlw/mfUML2l5/yxVZwx1b/3elyccGhrHh2giA9hA8+D+zNZFsCWsllLXGGBC1jUYLrl7022Okq1dZpVtm1fcBuyk5JR0Bz4HP67IinCkTtVbdLm8g1vmTOkAvTy4POOFlpGMU2s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hRs0YtQX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hRs0YtQX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D8204C32786; Mon, 29 Jul 2024 11:21:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722252089; bh=rElfU0d71iABwZ67rvpmRaqaEhOUs/Njjel6X7x+ZZU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hRs0YtQX+izREwBhLyXvqNE4b9sY2v23cdCN/KVwRcfz69oPiD+Hvnh90SwhTsaPb Vkx9r2ZgMQTy4rmd0MnKCimmI3w9vx/KaFmcPpT3SCdLMXKPYzhSEKbXTYzzNr3oN7 m26rXcx/pzAXE1R5k5eAqJYKMvinugxhBLA8twMfBgA+zJMoWYPl02pDo+cWmUbHLa Dm5UOuCX5Ae3mcSJ3o7Ckxn/IDqSbGnvEiu5bcPkEHVnYDBb3x2T6p1MDJsrdwE0E3 yqlmrYcH1BamgQHFciiWbXWkrO8PfJIgvAik/Cpka48aj+BCo5XT+UZrbc0mlt8Fu+ Okqo2hQrbz3mQ== From: alexs@kernel.org To: Vitaly Wool , Miaohe Lin , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, willy@infradead.org, senozhatsky@chromium.org, david@redhat.com, 42.hyeyoo@gmail.com, Yosry Ahmed , nphamcs@gmail.com Cc: Alex Shi Subject: [PATCH v4 22/22] mm/zsmalloc: update comments for page->zpdesc changes Date: Mon, 29 Jul 2024 19:25:34 +0800 Message-ID: <20240729112534.3416707-23-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Signed-off-by: Alex Shi --- mm/zsmalloc.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 64e523ae71f8..50ce4a3b8279 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -967,7 +967,7 @@ static void init_zspage(struct size_class *class, struc= t zspage *zspage) set_freeobj(zspage, 0); } =20 -static void create_page_chain(struct size_class *class, struct zspage *zsp= age, +static void create_zpdesc_chain(struct size_class *class, struct zspage *z= spage, struct zpdesc *zpdescs[]) { int i; @@ -976,9 +976,9 @@ static void create_page_chain(struct size_class *class,= struct zspage *zspage, int nr_zpdescs =3D class->pages_per_zspage; =20 /* - * Allocate individual pages and link them together as: - * 1. all pages are linked together using zpdesc->next - * 2. each sub-page point to zspage using zpdesc->zspage + * Allocate individual zpdescs and link them together as: + * 1. all zpdescs are linked together using zpdesc->next + * 2. each sub-zpdesc point to zspage using zpdesc->zspage * * we set PG_private to identify the first zpdesc (i.e. no other zpdesc * has this flag set). @@ -1036,7 +1036,7 @@ static struct zspage *alloc_zspage(struct zs_pool *po= ol, zpdescs[i] =3D zpdesc; } =20 - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); init_zspage(class, zspage); zspage->pool =3D pool; zspage->class =3D class->index; @@ -1363,7 +1363,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, /* record handle in the header of allocated chunk */ link->handle =3D handle | OBJ_ALLOCATED_TAG; else - /* record handle to page->index */ + /* record handle to zpdesc->handle */ zspage->first_zpdesc->handle =3D handle | OBJ_ALLOCATED_TAG; =20 kunmap_atomic(vaddr); @@ -1783,7 +1783,7 @@ static void replace_sub_page(struct size_class *class= , struct zspage *zspage, idx++; } while ((zpdesc =3D get_next_zpdesc(zpdesc)) !=3D NULL); =20 - create_page_chain(class, zspage, zpdescs); + create_zpdesc_chain(class, zspage, zpdescs); first_obj_offset =3D get_first_obj_offset(oldzpdesc); set_first_obj_offset(newzpdesc, first_obj_offset); if (unlikely(ZsHugePage(zspage))) --=20 2.43.0 From nobody Sat Feb 7 08:15:25 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12AB419FA8D for ; Tue, 30 Jul 2024 12:27:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722342462; cv=none; b=phmv5L6fPQtL9D8v8RIGJxHEHNke+3DPKNdJXG770Ix/VbDob5D0h3goh2huzHdsfogAG5SHUNm1kcNJbqDMW3JRfjJ++DYMqv5/tgB01NOR5okK7GJHlIRmbEwchZRtKLCFkC8vW2pL9SIWEysmfaQOr+4uAyvYbfbkFhqnLcA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722342462; c=relaxed/simple; bh=rBOTxVukObDt26c+er7K24D5TzmL/+0zcqF1Wh3ZGHE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AdggzB7R+SBhfiUFSCwjFyy/zdNsReaTdTMn/AOSKzp7R5HRO5QaisFDJS1ASulEbtX1jc2CfvRQ2uMLCAWD+SGvAeQDTUR5mjOcFN9U/cbyhGdnacWq+lDSI4Uo4xJUYvn4amtsVs7jxXu7FwReKTCu9oT1oVpfZhz4HqJUGyA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CL4YamNW; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CL4YamNW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8BC51C32782; Tue, 30 Jul 2024 12:27:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722342461; bh=rBOTxVukObDt26c+er7K24D5TzmL/+0zcqF1Wh3ZGHE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CL4YamNWq/Sh4OpNbR8MweHnWdrmkAREEsaoqPREX07MePcgTdhGqtesC1js1Awty qc2hh95WqP5Pu2/aFf9j/Xg+TNY67MMwk9HzZFVMivzAFZogpcGCd76qJ25KXj82d1 T3vhlM//Y30II7zafEEvIifpvK/iPqFtr5k+J9fmGCHE9rpIKuPbur7mU/3eyUg4pz VoPNv5r87i0T4jm2kHnDon8VU8AJG4VUOBIUvTgeDhaP9Ym5aL6N1B0iMt2ExmNbnt QwzHKMzrZVJCeO6AAliH+4JF/TsjzXH+2OkCp603oaAukOdkyqqSWT7Iv39JtP/ULI 1KluTFaLBQB9A== From: alexs@kernel.org To: alexs@kernel.org Cc: 42.hyeyoo@gmail.com, akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, minchan@kernel.org, nphamcs@gmail.com, senozhatsky@chromium.org, vitaly.wool@konsulko.com, willy@infradead.org, yosryahmed@google.com Subject: [PATCH 23/23] mm/zsmalloc: introduce zpdesc_clear_first() helper Date: Tue, 30 Jul 2024 20:31:57 +0800 Message-ID: <20240730123157.3761782-1-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240729112534.3416707-1-alexs@kernel.org> References: <20240729112534.3416707-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alex Shi Like the zpdesc_set_first(), introduce zpdesc_clear_first() helper for ClearPagePrivate(), then clean up a 'struct page' usage in reset_zpdesc(). Signed-off-by: Alex Shi To: linux-kernel@vger.kernel.org To: linux-mm@kvack.org To: Andrew Morton To: Sergey Senozhatsky To: Minchan Kim --- mm/zsmalloc.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 50ce4a3b8279..731055ccef23 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -259,6 +259,11 @@ static inline void zpdesc_set_first(struct zpdesc *zpd= esc) SetPagePrivate(zpdesc_page(zpdesc)); } =20 +static inline void zpdesc_clear_first(struct zpdesc *zpdesc) +{ + ClearPagePrivate(zpdesc_page(zpdesc)); +} + static inline void zpdesc_inc_zone_page_state(struct zpdesc *zpdesc) { inc_zone_page_state(zpdesc_page(zpdesc), NR_ZSPAGES); @@ -844,10 +849,8 @@ static inline bool obj_allocated(struct zpdesc *zpdesc= , void *obj, =20 static void reset_zpdesc(struct zpdesc *zpdesc) { - struct page *page =3D zpdesc_page(zpdesc); - __zpdesc_clear_movable(zpdesc); - ClearPagePrivate(page); + zpdesc_clear_first(zpdesc); zpdesc->zspage =3D NULL; zpdesc->next =3D NULL; reset_first_obj_offset(zpdesc); --=20 2.43.0