From nobody Tue Apr 7 18:51:34 2026 Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06FA444D6BA for ; Thu, 26 Feb 2026 19:29:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772134190; cv=none; b=roIk4rPXRFL787zhIA+9cMqlz1ZuO6EmmYAnfnw6vklpmYxTm4Cy1XeBH43/gHQKCPo5faPyepKcrHYYiaSYeVxeeKPPlra9K4V3KLsXaDqKDywu4fQhS7bl5PrrdZLKeELqjKnvy7fUwr/TLchQ5yEdQTlEmBkXCh3ZGDJJnCk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772134190; c=relaxed/simple; bh=wfhNq3yitHAZ8XpYXoCk88fkKDKZU9yo7nZb3MeoDNk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UW6ycgj9v+JzpRf68o2WBkQjEYZgBljMrwo0VZPjzbL62rRUGKXHcZxt09L1OZGOWxRRhm6bxizodqJK8m2tZTd6NC9tusHjeq6UUfgiC5QbOgDN7JmUkL+CFr8wXHT9196MQ7qadcY0l4pBA61kSsycrclS0gRfwKPnFEKehdg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lyU8G3QK; arc=none smtp.client-ip=209.85.210.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lyU8G3QK" Received: by mail-ot1-f41.google.com with SMTP id 46e09a7af769-7d18d02af68so949581a34.2 for ; Thu, 26 Feb 2026 11:29:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772134186; x=1772738986; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MTFgJFJwcADBwm8xdyLgLzJR5Ju+0hDvZcOEI0tPhVw=; b=lyU8G3QKqXOrqG+pgD5g2k1/qy/eledBVYavkCp4YD3z5HAkZoQ0KtqH6PTCesqi0/ rJrjggHz2AGrIm8qig45DSzgskxQqZw5yGr5GF7mVNpxz8xJRcy7cXqfoHjXvNr7/irN PxpsnQYwoWsqheOTDzJ2B38B0oWKzeUT0kM/jqmZkFyai6w8PSslNjxTibWyOWf57ML4 YlimCqA0zprTFOLRt6RBMExF9EfzxOYQT5rCAOJWpE85eKkS3EeFluc4f9zcaYq6usc4 OLWD30HGWwKJ6Qn4YB3I49PlNQAmdSuGSkmpo7C68upaHCMNYkhcTzELRlKp1GsQq3bc HJew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772134186; x=1772738986; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=MTFgJFJwcADBwm8xdyLgLzJR5Ju+0hDvZcOEI0tPhVw=; b=pCRtIdo1QmywVEd2mAUQipphSJOPrClFRIMmJWskcYpkGFc2vC0Jl5LOUtbuQmpbQM +GplFW4S90ZlavBlJV1sllI5lu2kfFbGCn1FbTNv7XH4KzpjcdYHqj5KmXGVPCtsi9LL P22vzcUQYL0b96I2dfo/ko9Cv2wHnXeTPMZ8CRYuEJtMt6yFxsdjjxLSgUUSlziGBWZN fem47vt6bpmjuw84Uj89w7U2l5Z9uPHAfPZfJbXah/6xZCaeJNAeBOYJNe/NuzqRMdd2 r7ZaNh5+7x4DSfiIFBi1nQ/+zOmKaPiRMx8PnKNG+aNdNK8bEMev7GGxJM4UmLiHVRVa QMoA== X-Forwarded-Encrypted: i=1; AJvYcCXPQA+gIma0uxLVN8bQBE1yrjCLjxsMg78yyTnju4lMe5lBo+u1AtBIY2JlEjzCvQbVTRN0CqHhc4uafiU=@vger.kernel.org X-Gm-Message-State: AOJu0YxcVgEoxZ369Na9j879v7jl5qhCoPMt5ISpWFftpxaiji+bo8z3 P9PmM3aCgUTrJmDeoWH4tOsXZZuSq+XaMDgTGWls931AKR3cgg1uolvh X-Gm-Gg: ATEYQzwi9nH+Aa+RjJurIUPdYzbIG43gFoNIpUL51+K9bvkyZ5sRu5R5gMeo/QjoSZy 33S6bd3ZbTZy1Gk3KP/VhbSC7Zk4470SBaOEzaM6LJ89Txrr4pxdzq+TfXhZfkjsVetYPTXi7Cv Zx+Hq+63QHJQWLkCNnXYKnrYUZdgf35S6KjIHtszlX7doNikVhItyvQHqbHUOXKxd4dcjYidTPT mZ2vrabIT+fMXfivAfyw+klsTheEbXGcH1IhvqiV1pJFTHlYE14+9VFVuAjFnCC64DLx5nXIBbL bPVvt9oV4J2Pnuijb4TScxf3IezRLfXvsuNRng+kKq4twhkE/NjEUWwcOm2ZR2QmqfhT403rFWZ ANHobDmH4kuFqJrEDYQ0yLVoLbgVzV9VtF0Vh/AaUiimPcI+ON3L5kKro6XpRon/oxDYRquSEp3 pHIsZEz/myPMB6CsPz1jzX3w== X-Received: by 2002:a05:6871:7402:b0:40e:9550:3242 with SMTP id 586e51a60fabf-41626de807cmr252273fac.16.1772134186118; Thu, 26 Feb 2026 11:29:46 -0800 (PST) Received: from localhost ([2a03:2880:10ff:5f::]) by smtp.gmail.com with ESMTPSA id 586e51a60fabf-4160d21196csm2606670fac.12.2026.02.26.11.29.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Feb 2026 11:29:45 -0800 (PST) From: Joshua Hahn To: Minchan Kim , Sergey Senozhatsky Cc: Johannes Weiner , Jens Axboe , Yosry Ahmed , Nhat Pham , Nhat Pham , Chengming Zhou , Andrew Morton , linux-mm@kvack.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Subject: [PATCH 4/8] mm/zsmalloc: Store obj_cgroup pointer in zpdesc Date: Thu, 26 Feb 2026 11:29:27 -0800 Message-ID: <20260226192936.3190275-5-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260226192936.3190275-1-joshua.hahnjy@gmail.com> References: <20260226192936.3190275-1-joshua.hahnjy@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With each zswap-backing zpdesc now having an array of obj_cgroup pointers, plumb the obj_cgroup pointer from the zswap / zram layer down to zsmalloc. Introduce two helper functions zpdesc_obj_cgroup and zpdesc_set_obj_cgroup, which abstract the conversion of an object's zspage idx to its zpdesc idx and the retrieval of the obj_cgroup pointer from the zpdesc. From the zswap path, store the obj_cgroup pointer after compression when writing the object and free when the object gets freed. Also handle the migration of an object across zpdescs. The lifetime and charging of the obj_cgroup is still handled in the zswap layer. Suggested-by: Johannes Weiner Signed-off-by: Joshua Hahn --- drivers/block/zram/zram_drv.c | 7 ++-- include/linux/zsmalloc.h | 3 +- mm/zsmalloc.c | 71 ++++++++++++++++++++++++++++++++++- mm/zswap.c | 6 +-- 4 files changed, 79 insertions(+), 8 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 60ee85679730..209668b14428 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2231,7 +2231,7 @@ static int write_incompressible_page(struct zram *zra= m, struct page *page, } =20 src =3D kmap_local_page(page); - zs_obj_write(zram->mem_pool, handle, src, PAGE_SIZE); + zs_obj_write(zram->mem_pool, handle, src, PAGE_SIZE, NULL); kunmap_local(src); =20 slot_lock(zram, index); @@ -2296,7 +2296,7 @@ static int zram_write_page(struct zram *zram, struct = page *page, u32 index) return -ENOMEM; } =20 - zs_obj_write(zram->mem_pool, handle, zstrm->buffer, comp_len); + zs_obj_write(zram->mem_pool, handle, zstrm->buffer, comp_len, NULL); zcomp_stream_put(zstrm); =20 slot_lock(zram, index); @@ -2520,7 +2520,8 @@ static int recompress_slot(struct zram *zram, u32 ind= ex, struct page *page, return PTR_ERR((void *)handle_new); } =20 - zs_obj_write(zram->mem_pool, handle_new, zstrm->buffer, comp_len_new); + zs_obj_write(zram->mem_pool, handle_new, zstrm->buffer, + comp_len_new, NULL); zcomp_stream_put(zstrm); =20 slot_free(zram, index); diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index 8ef28b964bb0..22f3baa13f24 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -15,6 +15,7 @@ #define _ZS_MALLOC_H_ =20 #include +#include =20 struct zs_pool_stats { /* How many pages were migrated (freed) */ @@ -48,7 +49,7 @@ void zs_obj_read_sg_begin(struct zs_pool *pool, unsigned = long handle, struct scatterlist *sg, size_t mem_len); void zs_obj_read_sg_end(struct zs_pool *pool, unsigned long handle); void zs_obj_write(struct zs_pool *pool, unsigned long handle, - void *handle_mem, size_t mem_len); + void *handle_mem, size_t mem_len, struct obj_cgroup *objcg); =20 extern const struct movable_operations zsmalloc_mops; =20 diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7d56bb700e11..e5ae9a0fc78a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -899,6 +899,41 @@ static void init_zspage(struct size_class *class, stru= ct zspage *zspage) } =20 #ifdef CONFIG_MEMCG +/* idx is indexed per-zspage, not per-zpdesc. */ +static inline struct obj_cgroup *zpdesc_obj_cgroup(struct zpdesc *zpdesc, + unsigned int idx, + int size) +{ + struct obj_cgroup **objcgs =3D zpdesc_objcgs(zpdesc); + unsigned int off =3D offset_in_page(size * idx); + unsigned int zpdesc_idx =3D DIV_ROUND_UP(off, size); + + if (!objcgs) + return NULL; + + return objcgs[zpdesc_idx]; +} + +/* idx is indexed per-zspage, not per-zpdesc. */ +static inline void zpdesc_set_obj_cgroup(struct zpdesc *zpdesc, + unsigned int idx, int size, + struct obj_cgroup *objcg) +{ + struct obj_cgroup **objcgs =3D zpdesc_objcgs(zpdesc); + unsigned int off =3D offset_in_page(size * idx); + unsigned int zpdesc_idx =3D DIV_ROUND_UP(off, size); + + if (!objcgs) + return; + + objcgs[zpdesc_idx] =3D objcg; + if (off + size > PAGE_SIZE) { + /* object spans two pages */ + objcgs =3D zpdesc_objcgs(get_next_zpdesc(zpdesc)); + objcgs[0] =3D objcg; + } +} + static bool alloc_zspage_objcgs(struct size_class *class, gfp_t gfp, struct zpdesc *zpdescs[]) { @@ -927,12 +962,40 @@ static bool alloc_zspage_objcgs(struct size_class *cl= ass, gfp_t gfp, =20 return true; } + +static void migrate_obj_objcg(unsigned long used_obj, unsigned long free_o= bj, + int size) +{ + unsigned int s_obj_idx, d_obj_idx; + struct zpdesc *s_zpdesc, *d_zpdesc; + struct obj_cgroup *objcg; + + obj_to_location(used_obj, &s_zpdesc, &s_obj_idx); + obj_to_location(free_obj, &d_zpdesc, &d_obj_idx); + objcg =3D zpdesc_obj_cgroup(s_zpdesc, s_obj_idx, size); + + zpdesc_set_obj_cgroup(d_zpdesc, d_obj_idx, size, objcg); + zpdesc_set_obj_cgroup(s_zpdesc, s_obj_idx, size, NULL); +} #else +static inline struct obj_cgroup *zpdesc_obj_cgroup(struct zpdesc *zpdesc, + unsigned int offset, + int size) +{ + return NULL; +} + +static inline void zpdesc_set_obj_cgroup(struct zpdesc *zpdesc, + unsigned int offset, int size, + struct obj_cgroup *objcg) {} static bool alloc_zspage_objcgs(struct size_class *class, gfp_t gfp, struct zpdesc *zpdescs[]) { return true; } + +static void migrate_obj_objcg(unsigned long used_obj, unsigned long free_o= bj, + int size) {} #endif =20 static void create_page_chain(struct size_class *class, struct zspage *zsp= age, @@ -1221,7 +1284,7 @@ void zs_obj_read_sg_end(struct zs_pool *pool, unsigne= d long handle) EXPORT_SYMBOL_GPL(zs_obj_read_sg_end); =20 void zs_obj_write(struct zs_pool *pool, unsigned long handle, - void *handle_mem, size_t mem_len) + void *handle_mem, size_t mem_len, struct obj_cgroup *objcg) { struct zspage *zspage; struct zpdesc *zpdesc; @@ -1242,6 +1305,9 @@ void zs_obj_write(struct zs_pool *pool, unsigned long= handle, class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 + if (objcg) + zpdesc_set_obj_cgroup(zpdesc, obj_idx, class->size, objcg); + if (!ZsHugePage(zspage)) off +=3D ZS_HANDLE_SIZE; =20 @@ -1415,6 +1481,8 @@ static void obj_free(int class_size, unsigned long ob= j) f_offset =3D offset_in_page(class_size * f_objidx); zspage =3D get_zspage(f_zpdesc); =20 + zpdesc_set_obj_cgroup(f_zpdesc, f_objidx, class_size, NULL); + vaddr =3D kmap_local_zpdesc(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); =20 @@ -1587,6 +1655,7 @@ static void migrate_zspage(struct zs_pool *pool, stru= ct zspage *src_zspage, used_obj =3D handle_to_obj(handle); free_obj =3D obj_malloc(pool, dst_zspage, handle); zs_obj_copy(class, free_obj, used_obj); + migrate_obj_objcg(used_obj, free_obj, class->size); obj_idx++; obj_free(class->size, used_obj); =20 diff --git a/mm/zswap.c b/mm/zswap.c index dd083110bfa0..1e2d60f47919 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -851,7 +851,7 @@ static void acomp_ctx_put_unlock(struct crypto_acomp_ct= x *acomp_ctx) } =20 static bool zswap_compress(struct page *page, struct zswap_entry *entry, - struct zswap_pool *pool) + struct zswap_pool *pool, struct obj_cgroup *objcg) { struct crypto_acomp_ctx *acomp_ctx; struct scatterlist input, output; @@ -911,7 +911,7 @@ static bool zswap_compress(struct page *page, struct zs= wap_entry *entry, goto unlock; } =20 - zs_obj_write(pool->zs_pool, handle, dst, dlen); + zs_obj_write(pool->zs_pool, handle, dst, dlen, objcg); entry->handle =3D handle; entry->length =3D dlen; =20 @@ -1413,7 +1413,7 @@ static bool zswap_store_page(struct page *page, return false; } =20 - if (!zswap_compress(page, entry, pool)) + if (!zswap_compress(page, entry, pool, objcg)) goto compress_failed; =20 old =3D xa_store(swap_zswap_tree(page_swpentry), --=20 2.47.3