From nobody Tue Apr 7 18:51:34 2026 Received: from mail-oi1-f173.google.com (mail-oi1-f173.google.com [209.85.167.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1421A44DB89 for ; Thu, 26 Feb 2026 19:29:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772134196; cv=none; b=BKeY7qGWL2kzkffXwFyJbIq224ZtZHBwOcL2AE6GVBhWw1+AioFEsVjd8YGOwp/OUoMAsVzK+jwCYj32eVvicKrBr8joOAMrS5oelBGLn1YnSz8XYu+IsfbMGcRrnAyNAMmek5ZSAuLqvwLQbBZHIKlKJ6BMTpAE9c1Wq2mJuk8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772134196; c=relaxed/simple; bh=yJ8hDpck92sFoRQLA5ASgE0Grx5gw3Jrij4YOy9XvEA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OtlA/t2Ursd6pVlNbu+3GTx0yZQQqSbKclPx3An2aTxsvSN8LP+73fkKCdmITCZtWZwm8NUasGBf7bafP7ghJTrTq8IpazJ2kUxtDx82kgAt723Xu5I6b2o3KrGQjBxogtv4upj0fq4uOr5jqS5ktKF+e9cwjaQtX0HFxx0oW8c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WL5z6dpN; arc=none smtp.client-ip=209.85.167.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WL5z6dpN" Received: by mail-oi1-f173.google.com with SMTP id 5614622812f47-4648448e387so986045b6e.1 for ; Thu, 26 Feb 2026 11:29:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772134190; x=1772738990; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BQnJpZ9tlMxlAvSH4/u9efMTjBoGOkCwurLY+eANTp4=; b=WL5z6dpNtNOKuc28cuEB/g1Uxf7EYrfOUmTcSj7a3S5i9agIvvk0n92/iebzw/vqzR L0wZebi7eNeJPzDqLiNRYP2TTIyT/50ZQiDawAkc3+N/NBUQbOtYlHXaNZoWjbomcfIu GhsONQwAt0L5dEP2iTIbzixk/XfeXPXO8yLGQjy5vRbyZGG5lvIS/s1ziDbtB177eGRd +ODjomDPIB78K81ptkMsqhnPSDMJRKBbMvzD/Ndy6CkJQsiWrY3i7D0R7l8/ClVqEkS9 Z+7jXQsLigvOpbtEkP8LPceFcWS8d/kCtZBJ/Q/LJGHd4Q1Fx6wXxGvKlW1LakXqHgcf WwAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772134190; x=1772738990; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=BQnJpZ9tlMxlAvSH4/u9efMTjBoGOkCwurLY+eANTp4=; b=ekRcxIlNjHoJLNuiOxrNgI50rvTYOa/mvORGkB1otmtBYR3l20ergEI5FPUtFnoRxz whsTJbxk6ifyze3YXpib/Y20sZH/UGGXeG3aR++A10yaojyKDwDPgYiyQBPvvPDNZ+M/ rP7fsdC9aH1GNu3yGAPjl2/NBGZGqjCgMkdvkR2shg7h/hn8Wx6s+8utpEXGUlN61SkL Nwnx8+kxRa6D4trgqIuPvjh1tKhtZIkRin97AZNYN4GunS4VqTJiX5OPpMtNgIoIXaqI 9Rzz29P5vNDmumr0NVaH6YvBw81JEJu8p7lxbykoDiUBtgS5Gz6eMSIzMBt+y5WY775m Je0g== X-Forwarded-Encrypted: i=1; AJvYcCUwJn+hQ4n4a5FrpgDl1Kdau2XT+qaTMcU815WxRhiyVT5VpNLiHjc2e0Y+br6Y08n/ctkqWQ0EPEnHHC8=@vger.kernel.org X-Gm-Message-State: AOJu0YzbdXPBdK7Re4LqaN6Nw2pDmdt0aX/JodvmKTQrlOK2drBbbGHB LaxnQmpVgmjy80LtZP+DPMPQTtD6oqOUFQZnhsLa4tMBp9jO89FSdW0k X-Gm-Gg: ATEYQzwHNfwNrsaqQ+Vi1Sc9xxbi6X+dx5uAMctd1lI/7VfZUqnwamGo9RuiSgkEzBS E6L/ZEHU08EZcZScOaY9pJpBWpBXy1+4BIDii+i8wyJD+3d+z9YizP7MOgj5pzrbDfIfxAYnjXZ zlgw/ItB/yo8ixvO/Y8MXfn/Rg8+LjCm8lKCzI5ZYj6VocMvqG2CUu+YlTxu4hWrJrz7N2SnJVV ZvH5VJebKQxtnqkr4mQitYw5CTZbpki3nnhwKRLF+pb5bOi3nKVrBUI7mXAb9ddVAXztyK7EV9/ ikiMsXVZL+UMRyt+jz/+p/P1LrZPZTkVA+MLQ9Pn2wNnhj9RFtsk8oKdtvMCVeMZ0ExWcsmJCux +d4+GIneTEzzSnL8+I6rvHWp59bkkQzfb8VUYr7RO4ASXfJpMX8JvsEYVwIiMpYMO0aKlHaJPNs 1Mx2GsHdM5HOI2PelK8YPmhA== X-Received: by 2002:a05:6808:c1e2:b0:45c:8c23:12b3 with SMTP id 5614622812f47-464bec872bdmr142745b6e.61.1772134189833; Thu, 26 Feb 2026 11:29:49 -0800 (PST) Received: from localhost ([2a03:2880:10ff:4f::]) by smtp.gmail.com with ESMTPSA id 5614622812f47-464bb0991f5sm498612b6e.0.2026.02.26.11.29.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Feb 2026 11:29:48 -0800 (PST) From: Joshua Hahn To: Minchan Kim , Sergey Senozhatsky Cc: Johannes Weiner , Yosry Ahmed , Nhat Pham , Nhat Pham , Chengming Zhou , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Subject: [PATCH 6/8] mm/zsmalloc, zswap: Handle objcg charging and lifetime in zsmalloc Date: Thu, 26 Feb 2026 11:29:29 -0800 Message-ID: <20260226192936.3190275-7-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260226192936.3190275-1-joshua.hahnjy@gmail.com> References: <20260226192936.3190275-1-joshua.hahnjy@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that zswap_entries do not directly track obj_cgroups of the entries, handle the lifetime management and charging of these entries into the zsmalloc layer. One functional change is that zswap entries are now no longer accounted by the size of the compressed object, but by the size of the size_class slot they occupy. This brings the charging one step closer to an accurate representation of the memory consumed in the zpdesc; even if a compressed object doesn't consume the entirety of a obj slot, the hole it creates between the objects is dead space the obj is accountable for. Thus, account the memory each object makes unusable, not the amount of memory each object takes up. Signed-off-by: Joshua Hahn --- include/linux/memcontrol.h | 10 ------- mm/memcontrol.c | 51 ---------------------------------- mm/zsmalloc.c | 57 ++++++++++++++++++++++++++++++++++++-- mm/zswap.c | 8 ------ 4 files changed, 55 insertions(+), 71 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index b6c82c8f73e1..dd4278b1ca35 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1824,22 +1824,12 @@ static inline bool memcg_is_dying(struct mem_cgroup= *memcg) =20 #if defined(CONFIG_MEMCG) && defined(CONFIG_ZSWAP) bool obj_cgroup_may_zswap(struct obj_cgroup *objcg); -void obj_cgroup_charge_zswap(struct obj_cgroup *objcg, size_t size); -void obj_cgroup_uncharge_zswap(struct obj_cgroup *objcg, size_t size); bool mem_cgroup_zswap_writeback_enabled(struct mem_cgroup *memcg); #else static inline bool obj_cgroup_may_zswap(struct obj_cgroup *objcg) { return true; } -static inline void obj_cgroup_charge_zswap(struct obj_cgroup *objcg, - size_t size) -{ -} -static inline void obj_cgroup_uncharge_zswap(struct obj_cgroup *objcg, - size_t size) -{ -} static inline bool mem_cgroup_zswap_writeback_enabled(struct mem_cgroup *m= emcg) { /* if zswap is disabled, do not block pages going to the swapping device = */ diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 007413a53b45..3432e1afc037 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5433,57 +5433,6 @@ bool obj_cgroup_may_zswap(struct obj_cgroup *objcg) return ret; } =20 -/** - * obj_cgroup_charge_zswap - charge compression backend memory - * @objcg: the object cgroup - * @size: size of compressed object - * - * This forces the charge after obj_cgroup_may_zswap() allowed - * compression and storage in zswap for this cgroup to go ahead. - */ -void obj_cgroup_charge_zswap(struct obj_cgroup *objcg, size_t size) -{ - struct mem_cgroup *memcg; - - if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) - return; - - VM_WARN_ON_ONCE(!(current->flags & PF_MEMALLOC)); - - /* PF_MEMALLOC context, charging must succeed */ - if (obj_cgroup_charge(objcg, GFP_KERNEL, size)) - VM_WARN_ON_ONCE(1); - - rcu_read_lock(); - memcg =3D obj_cgroup_memcg(objcg); - mod_memcg_state(memcg, MEMCG_ZSWAP_B, size); - mod_memcg_state(memcg, MEMCG_ZSWAPPED, 1); - rcu_read_unlock(); -} - -/** - * obj_cgroup_uncharge_zswap - uncharge compression backend memory - * @objcg: the object cgroup - * @size: size of compressed object - * - * Uncharges zswap memory on page in. - */ -void obj_cgroup_uncharge_zswap(struct obj_cgroup *objcg, size_t size) -{ - struct mem_cgroup *memcg; - - if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) - return; - - obj_cgroup_uncharge(objcg, size); - - rcu_read_lock(); - memcg =3D obj_cgroup_memcg(objcg); - mod_memcg_state(memcg, MEMCG_ZSWAP_B, -size); - mod_memcg_state(memcg, MEMCG_ZSWAPPED, -1); - rcu_read_unlock(); -} - bool mem_cgroup_zswap_writeback_enabled(struct mem_cgroup *memcg) { /* if zswap is disabled, do not block pages going to the swapping device = */ diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 067215a6ddcc..88c7cd399261 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -963,6 +963,44 @@ static bool alloc_zspage_objcgs(struct size_class *cla= ss, gfp_t gfp, return true; } =20 +static void zs_charge_objcg(struct zpdesc *zpdesc, struct obj_cgroup *objc= g, + int size, unsigned long offset) +{ + struct mem_cgroup *memcg; + + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) + return; + + VM_WARN_ON_ONCE(!(current->flags & PF_MEMALLOC)); + + /* PF_MEMALLOC context, charging must succeed */ + if (obj_cgroup_charge(objcg, GFP_KERNEL, size)) + VM_WARN_ON_ONCE(1); + + rcu_read_lock(); + memcg =3D obj_cgroup_memcg(objcg); + mod_memcg_state(memcg, MEMCG_ZSWAP_B, size); + mod_memcg_state(memcg, MEMCG_ZSWAPPED, 1); + rcu_read_unlock(); +} + +static void zs_uncharge_objcg(struct zpdesc *zpdesc, struct obj_cgroup *ob= jcg, + int size, unsigned long offset) +{ + struct mem_cgroup *memcg; + + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) + return; + + obj_cgroup_uncharge(objcg, size); + + rcu_read_lock(); + memcg =3D obj_cgroup_memcg(objcg); + mod_memcg_state(memcg, MEMCG_ZSWAP_B, -size); + mod_memcg_state(memcg, MEMCG_ZSWAPPED, -1); + rcu_read_unlock(); +} + static void migrate_obj_objcg(unsigned long used_obj, unsigned long free_o= bj, int size) { @@ -1018,6 +1056,12 @@ static bool alloc_zspage_objcgs(struct size_class *c= lass, gfp_t gfp, return true; } =20 +static void zs_charge_objcg(struct zpdesc *zpdesc, struct obj_cgroup *objc= g, + int size, unsigned long offset) {} + +static void zs_uncharge_objcg(struct zpdesc *zpdesc, struct obj_cgroup *ob= jcg, + int size, unsigned long offset) {} + static void migrate_obj_objcg(unsigned long used_obj, unsigned long free_o= bj, int size) {} =20 @@ -1334,8 +1378,11 @@ void zs_obj_write(struct zs_pool *pool, unsigned lon= g handle, class =3D zspage_class(pool, zspage); off =3D offset_in_page(class->size * obj_idx); =20 - if (objcg) + if (objcg) { + obj_cgroup_get(objcg); + zs_charge_objcg(zpdesc, objcg, class->size, off); zpdesc_set_obj_cgroup(zpdesc, obj_idx, class->size, objcg); + } =20 if (!ZsHugePage(zspage)) off +=3D ZS_HANDLE_SIZE; @@ -1501,6 +1548,7 @@ static void obj_free(int class_size, unsigned long ob= j) struct link_free *link; struct zspage *zspage; struct zpdesc *f_zpdesc; + struct obj_cgroup *objcg; unsigned long f_offset; unsigned int f_objidx; void *vaddr; @@ -1510,7 +1558,12 @@ static void obj_free(int class_size, unsigned long o= bj) f_offset =3D offset_in_page(class_size * f_objidx); zspage =3D get_zspage(f_zpdesc); =20 - zpdesc_set_obj_cgroup(f_zpdesc, f_objidx, class_size, NULL); + objcg =3D zpdesc_obj_cgroup(f_zpdesc, f_objidx, class_size); + if (objcg) { + zs_uncharge_objcg(f_zpdesc, objcg, class_size, f_offset); + obj_cgroup_put(objcg); + zpdesc_set_obj_cgroup(f_zpdesc, f_objidx, class_size, NULL); + } =20 vaddr =3D kmap_local_zpdesc(f_zpdesc); link =3D (struct link_free *)(vaddr + f_offset); diff --git a/mm/zswap.c b/mm/zswap.c index 55161a5c9d4c..77d3c6516ed3 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -711,10 +711,6 @@ static void zswap_entry_free(struct zswap_entry *entry) zs_free(entry->pool->zs_pool, entry->handle); zswap_pool_put(entry->pool); =20 - if (objcg) { - obj_cgroup_uncharge_zswap(objcg, entry->length); - obj_cgroup_put(objcg); - } if (entry->length =3D=3D PAGE_SIZE) atomic_long_dec(&zswap_stored_incompressible_pages); zswap_entry_cache_free(entry); @@ -1437,10 +1433,6 @@ static bool zswap_store_page(struct page *page, * when the entry is removed from the tree. */ zswap_pool_get(pool); - if (objcg) { - obj_cgroup_get(objcg); - obj_cgroup_charge_zswap(objcg, entry->length); - } atomic_long_inc(&zswap_stored_pages); if (entry->length =3D=3D PAGE_SIZE) atomic_long_inc(&zswap_stored_incompressible_pages); --=20 2.47.3