From nobody Wed Feb 11 10:02:34 2026 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2A7F6A33F for ; Wed, 29 Jan 2025 06:49:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738133356; cv=none; b=koyQfccG0q5qE2/CnRho40Y8U13yICqsFQmnuJZMpNDmTu/MLzKBPkI51J2SSK54JRVUHmzYuH7BVh53JuPx60a3Po9YY3lJc1qZJ0PjdopJeA5mCd1aSVwF9SLXwivP5/weOVzWQwvA3dGD7yT/2AbRwtF8PrwdjM2UZWaIlrw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738133356; c=relaxed/simple; bh=jT3RS7060nG9mjcjdxoe/GfXqx37HRR/2sZkVogWWqY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DSPhB1vgLSo/VC5GtyWlrheT+ZIrymoU6UA3e9Cbgpzx+WBgWlSZxfCgcfEVCltbTwMvIPxvRXhFa8L43Oc6NnJTORp3MwLpL5c8ZK6ZZLPl5zqK8kLjYbAtu31+4iQikp2qf73PFTJExKnuS6ZEWhi0X9hQcgZqhhD2DBOGmg0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=gRcbw7dT; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="gRcbw7dT" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-21c2f1b610dso151660355ad.0 for ; Tue, 28 Jan 2025 22:49:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738133354; x=1738738154; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U5xlt93WdnDLIhYO6MmK7r5usvow5aZkNdIQfTNWjy0=; b=gRcbw7dTQhcGCUlwgO2iLF1Z3h70lx5D2iHTHjodt/40kRuASmLH3wIVaWweDPgmRt Jx384Rw0YrSWB7rUaK+iNiXp2m9KTZNW99G5yxoWIoBxn25YSpv+WV44/P+brawZYl6T Sn6ZXU5++BUsUsmjiSz6y6z7rV+lLORBq33+g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738133354; x=1738738154; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U5xlt93WdnDLIhYO6MmK7r5usvow5aZkNdIQfTNWjy0=; b=XrIsxv+sselymG6t3vziFvvVzaoqzLb2qXPmNhC8K857+VmuSFPY0sIHG5QNAIn4oc zi0b+THiPzSYWYymPSYnDxhYDOJpDeMD3jryS09P17uT42NDfaMlxJTeC2UdYFj5pYDe cy+i0KFth/xltlcfkM7KT4JvwbwRIfp6K+UnpdZWyubhAB+hAjl5SvmhEMTU3uVm/b/b 6nT9SoLnUMZV1PJuGmwt4DVuNqZDLro7givMIJEFkVo+FSwRYmS10DDIIWBlEoJs1T48 kz0EfdSLxUBWZ0M1UGCHeAy5QL6nIjDtYmFM+bw6S6iB31ymsLfraC5X6wcoOT426l1q uzbg== X-Forwarded-Encrypted: i=1; AJvYcCVneJ0/mcA2caGxb9RsXcQtsr/Mf1Q40SBhiZxTr19lN7W/gTBWa8t0/mtn5rwqcZTi4smFDNeRfmHbkWc=@vger.kernel.org X-Gm-Message-State: AOJu0YzbxbBkuDgpNWt8tl2a6nkHLbFfV9UfmH3v6EPIqvbBFH83CUpx taRX76B/1kmd4MK0WdIGX/yONzQhBasx5mH9iygO4gVow6FPZGTHbZPc1kzK0A== X-Gm-Gg: ASbGncsBJ2lIdRf41nYBDn9rg4hSOWTN0tmfzKIqUM0zl+8UffZ76PkNcHdyu851bFt heNWQ58nSY5/lDHFGmyw4GxqXYDeP7ZiYO7/04XxIKrI8v6Yh9K4ePRRaSfAWmLloJTULAEtYI9 iOPvraRcgSncTZJ1uDWKNBoMNhfrmjiHh1jGGjEfIkcHMxkaxGc3VQAfHXLnAxZELWsOnO8VCxB pk8nwjft7lgfNVVcn85/ie9ZJwueX/fhf5Hq45Uf1UAPAMdZvm0FJTlNt0XxJt3O9dZ3IhsuoeT yR97IvyD2HOy0poZPA== X-Google-Smtp-Source: AGHT+IFzqhCwAv4hpOOWLJMonKQwPRQTh3PdlNYEJTZLHBEywcIgCKkqWY9wtUBWb0Oa2eHoYdi0kA== X-Received: by 2002:a17:902:da85:b0:215:6489:cfbf with SMTP id d9443c01a7336-21dd7c44a4dmr33973515ad.11.1738133354160; Tue, 28 Jan 2025 22:49:14 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:b323:d70b:a1b8:1683]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21da414d87fsm92309165ad.172.2025.01.28.22.49.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 28 Jan 2025 22:49:13 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim , Johannes Weiner , Yosry Ahmed , Nhat Pham Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv1 2/6] zsmalloc: factor out size-class locking helpers Date: Wed, 29 Jan 2025 15:43:48 +0900 Message-ID: <20250129064853.2210753-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250129064853.2210753-1-senozhatsky@chromium.org> References: <20250129064853.2210753-1-senozhatsky@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move open-coded size-class locking to dedicated helpers. Signed-off-by: Sergey Senozhatsky Reviewed-by: Yosry Ahmed --- mm/zsmalloc.c | 47 ++++++++++++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 19 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 2f8a2b139919..0f575307675d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -254,6 +254,16 @@ static bool pool_lock_is_contended(struct zs_pool *poo= l) return rwlock_is_contended(&pool->migrate_lock); } =20 +static void size_class_lock(struct size_class *class) +{ + spin_lock(&class->lock); +} + +static void size_class_unlock(struct size_class *class) +{ + spin_unlock(&class->lock); +} + static inline void zpdesc_set_first(struct zpdesc *zpdesc) { SetPagePrivate(zpdesc_page(zpdesc)); @@ -614,8 +624,7 @@ static int zs_stats_size_show(struct seq_file *s, void = *v) if (class->index !=3D i) continue; =20 - spin_lock(&class->lock); - + size_class_lock(class); seq_printf(s, " %5u %5u ", i, class->size); for (fg =3D ZS_INUSE_RATIO_10; fg < NR_FULLNESS_GROUPS; fg++) { inuse_totals[fg] +=3D class_stat_read(class, fg); @@ -625,7 +634,7 @@ static int zs_stats_size_show(struct seq_file *s, void = *v) obj_allocated =3D class_stat_read(class, ZS_OBJS_ALLOCATED); obj_used =3D class_stat_read(class, ZS_OBJS_INUSE); freeable =3D zs_can_compact(class); - spin_unlock(&class->lock); + size_class_unlock(class); =20 objs_per_zspage =3D class->objs_per_zspage; pages_used =3D obj_allocated / objs_per_zspage * @@ -1400,7 +1409,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t = size, gfp_t gfp) class =3D pool->size_class[get_size_class_index(size)]; =20 /* class->lock effectively protects the zpage migration */ - spin_lock(&class->lock); + size_class_lock(class); zspage =3D find_get_zspage(class); if (likely(zspage)) { obj_malloc(pool, zspage, handle); @@ -1411,7 +1420,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t = size, gfp_t gfp) goto out; } =20 - spin_unlock(&class->lock); + size_class_unlock(class); =20 zspage =3D alloc_zspage(pool, class, gfp); if (!zspage) { @@ -1419,7 +1428,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t = size, gfp_t gfp) return (unsigned long)ERR_PTR(-ENOMEM); } =20 - spin_lock(&class->lock); + size_class_lock(class); obj_malloc(pool, zspage, handle); newfg =3D get_fullness_group(class, zspage); insert_zspage(class, zspage, newfg); @@ -1430,7 +1439,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t = size, gfp_t gfp) /* We completely set up zspage so mark them as movable */ SetZsPageMovable(pool, zspage); out: - spin_unlock(&class->lock); + size_class_unlock(class); =20 return handle; } @@ -1484,7 +1493,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) obj_to_zpdesc(obj, &f_zpdesc); zspage =3D get_zspage(f_zpdesc); class =3D zspage_class(pool, zspage); - spin_lock(&class->lock); + size_class_lock(class); pool_read_unlock(pool); =20 class_stat_sub(class, ZS_OBJS_INUSE, 1); @@ -1494,7 +1503,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) if (fullness =3D=3D ZS_INUSE_RATIO_0) free_zspage(pool, class, zspage); =20 - spin_unlock(&class->lock); + size_class_unlock(class); cache_free_handle(pool, handle); } EXPORT_SYMBOL_GPL(zs_free); @@ -1828,7 +1837,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, /* * the class lock protects zpage alloc/free in the zspage. */ - spin_lock(&class->lock); + size_class_lock(class); /* the migrate_write_lock protects zpage access via zs_map_object */ migrate_write_lock(zspage); =20 @@ -1860,7 +1869,7 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, * it's okay to release migration_lock. */ pool_write_unlock(pool); - spin_unlock(&class->lock); + size_class_unlock(class); migrate_write_unlock(zspage); =20 zpdesc_get(newzpdesc); @@ -1904,10 +1913,10 @@ static void async_free_zspage(struct work_struct *w= ork) if (class->index !=3D i) continue; =20 - spin_lock(&class->lock); + size_class_lock(class); list_splice_init(&class->fullness_list[ZS_INUSE_RATIO_0], &free_pages); - spin_unlock(&class->lock); + size_class_unlock(class); } =20 list_for_each_entry_safe(zspage, tmp, &free_pages, list) { @@ -1915,10 +1924,10 @@ static void async_free_zspage(struct work_struct *w= ork) lock_zspage(zspage); =20 class =3D zspage_class(pool, zspage); - spin_lock(&class->lock); + size_class_lock(class); class_stat_sub(class, ZS_INUSE_RATIO_0, 1); __free_zspage(pool, class, zspage); - spin_unlock(&class->lock); + size_class_unlock(class); } }; =20 @@ -1983,7 +1992,7 @@ static unsigned long __zs_compact(struct zs_pool *poo= l, * as well as zpage allocation/free */ pool_write_lock(pool); - spin_lock(&class->lock); + size_class_lock(class); while (zs_can_compact(class)) { int fg; =20 @@ -2013,11 +2022,11 @@ static unsigned long __zs_compact(struct zs_pool *p= ool, putback_zspage(class, dst_zspage); dst_zspage =3D NULL; =20 - spin_unlock(&class->lock); + size_class_unlock(class); pool_write_unlock(pool); cond_resched(); pool_write_lock(pool); - spin_lock(&class->lock); + size_class_lock(class); } } =20 @@ -2027,7 +2036,7 @@ static unsigned long __zs_compact(struct zs_pool *poo= l, if (dst_zspage) putback_zspage(class, dst_zspage); =20 - spin_unlock(&class->lock); + size_class_unlock(class); pool_write_unlock(pool); =20 return pages_freed; --=20 2.48.1.262.g85cc9f2d1e-goog