From nobody Sat May 30 12:36:04 2026 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 26AB8371CE0 for ; Fri, 8 May 2026 06:08:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778220528; cv=none; b=Abt9dnTXGjqXu4YZssR6+aCYShLryzXexSdMSw/ypYBDQj2+/uOIAcRyvmfRn10MEdYYk9rjfaoH23KbS5ZIb7OGTxGmXv51icOaSru/GLWBx/1lWYKOdvs9gYJZNlx8+rWc+W7MUnPBMS9MpM4OJ5HJQbcFmG+KV++JcHLVWwg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778220528; c=relaxed/simple; bh=Usoh5lWjXatpqgQTm5AyAXJm7U5sj7zKAQ3IUH2LUhA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=qzFUZXObvMqaTRf1ZxWFAIkz/p7FztySaYroHpuoZhFF8I3qEfpoN7MZv2xm8o/Cvvbg2zhwnv3+dR5w2HJxmoKpU/7UMTgtQA0A5tSTOEuMTq6nMq0h4F1KhTKBWlZd0dDZyyEK4TbfbfUjxyrpMqOp8PcWLXkla24qods6FOI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hO8G6m9N; arc=none smtp.client-ip=209.85.216.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hO8G6m9N" Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-3660ab73adbso893004a91.1 for ; Thu, 07 May 2026 23:08:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778220516; x=1778825316; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=daHGfdxKfsNdfAdW1qTjK0dkvxPd9/EICvnVIHa78Qk=; b=hO8G6m9Ng0J/R2TAAZwBTQG4xe9bFfPbxns9MkRLnBjiO3q3SbQtGC8VAjB5QOSodt BO4g8S0vT53LR73bm4nidva+H8jJTktb8ZJweB/JRyuwetNlBDH/3h+qaGiCSDKYvtHU phwPU8mG3otKvmsXVpYRXWOdehE84FubEWvLzDKWAJ6+6ENx6hKw8xO92QgWkGP4VCv1 DL9MYVdSazLHN6i0nZNodKKrtANoKgFPfZD78skksp5g5VIeYPOoZc/L1exkYdOu+55P DWK84AiZ8+FtiRhQeyLhiUf4a474Bua0mZ2VW+xjTCXWCkZPMoOJlxChMreJJNtNGBLL qbZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778220516; x=1778825316; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=daHGfdxKfsNdfAdW1qTjK0dkvxPd9/EICvnVIHa78Qk=; b=WHsh4aIZeOuV4K0gxCuROPS5jKPn1K7CjN/YRl8cuThcm+uvGEts8tB7EXIvpjRgLM wZlmiy7UclMn2e+H63xrlOkxwV6ECVDCpXlebpgsXE+Sd0MYH9ZABaLJgkTC+FTb85iT IjThL1yTP8BqBGuG3DGbDYQ/0kABqItDUEL6XXf2KynxKDpZ2O/zwRy9bSXsVudPVyib l4Ddzs9VzI/qbwYfbUIaXlEzdqgekrqW++yiRfZiXOrjICP+bjh4ZBfCRfNQhyAwyoBv nM50OEAwsDcD1Dtr+VoMTsXiXebxMal4Pp6CDzmtT99Eb/ZBJ/gkLMKzWiX//FVYesMI iC+g== X-Forwarded-Encrypted: i=1; AFNElJ/CDBCV8/9rFMrxsAZeSUAspeo1+DY/4IHsIeIEkYoPsaudrIv+5QjD621fLJ0I4UJEwacDQ+clql74ZfM=@vger.kernel.org X-Gm-Message-State: AOJu0YwPVioA/19sXaGMNYTLZnaAkfxM4mRDJBVVbZdYEgs4nPzEIKEz K8RTIb8NBOK8awQiThX72BAAS1xfaBrAyks3/QunLTHBCy1xQQxw6k3Z X-Gm-Gg: Acq92OE1QbWTGjCKM69qYhmzbWknLShV6vzs/URP3FGVmqxJEAjvqKJZ50VLJKYgkyT ZVlgBBZlXBn8hMqpChx/vEYFbEXzVOHAoi72ZAE9ESTgXBur9G0iPnE3gwvs1ZCRgmeDRDTKn3I eLQ/6RYQwORXMi9ZIorcXJjLXN/5wTB9QqdmpZjLknci3oTcOmQdwqnh/aCU63gymFSoLeet84X pUh8JQbgkemkYDcV1Hd9SUKJciFR4Hw54hKvV5cyrzMvw9x8kXqs11/OrHsR00a1g8YTI+RVk9A Yi0SS+LsOaGO5jyr5E5ui/2gMzs5Xl4v6THLIAcFExXQKc22GdXrGGEOh/1GPAKCKCnWcsjXsuy 3zfP/Q3ofFdE69Gc8OwZu5mVLjWI0b0leUVcVf7cK38Lr2ym1e8sawf9Y5CtXX1vP/3t6Gl6iSA Zy7gnWscnAQXII+ka+JfV0IltID+fE4odkmi18zOKMyFouK+A9 X-Received: by 2002:a17:90b:5486:b0:35f:b230:5889 with SMTP id 98e67ed59e1d1-366053e90a9mr5119623a91.6.1778220515779; Thu, 07 May 2026 23:08:35 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36645bb9c02sm673651a91.1.2026.05.07.23.08.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 23:08:35 -0700 (PDT) From: Wenchao Hao X-Google-Original-From: Wenchao Hao To: Andrew Morton , Barry Song <21cnbao@gmail.com>, Chengming Zhou , Jens Axboe , Johannes Weiner , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Nhat Pham , Sergey Senozhatsky , Yosry Ahmed Cc: Wenchao Hao , Wenchao Hao Subject: [RFC PATCH v3 1/4] mm/zsmalloc: introduce deferred free framework with callback ops Date: Fri, 8 May 2026 14:07:21 +0800 Message-Id: <20260508060724.3810904-2-haowenchao@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260508060724.3810904-1-haowenchao@xiaomi.com> References: <20260508060724.3810904-1-haowenchao@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Add a per-cpu deferred free mechanism to zsmalloc with a callback interface that lets callers (zram, zswap) customize push and drain behavior. Each CPU owns a single-page buffer. The hot path (zs_free_deferred) writes a value into the current CPU's buffer via the push callback with preemption disabled =E2=80=94 no locks, no atomics. When the buffer fills, it is swapped with a fresh page from a pre-allocated page pool and the full page is queued to a WQ_UNBOUND worker for drain. The drain worker invokes the drain callback which performs the actual expensive work (zs_free, slot_free, etc.) in batch, away from the original hot path. Page pool management: - Pool is pre-allocated at enable time (ZS_DEFERRED_POOL_SIZE pages) - Full buffers are drained and returned to the pool - If no free page is available when buffer is full, the push falls back to synchronous processing by the caller Signed-off-by: Wenchao Hao --- include/linux/zsmalloc.h | 16 +++ mm/zsmalloc.c | 208 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 223 insertions(+), 1 deletion(-) diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index 478410c880b1..8d6c675b10dc 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -24,12 +24,28 @@ struct zs_pool_stats { struct zs_pool; struct scatterlist; =20 +enum zs_push_ret { + ZS_PUSH_OK =3D 0, + ZS_PUSH_FULL, + ZS_PUSH_FULL_QUEUED, +}; + +struct zs_deferred_ops { + enum zs_push_ret (*push)(void *buf, unsigned int count, + unsigned long value); + void (*drain)(void *private, void *buf, unsigned int count); +}; + struct zs_pool *zs_create_pool(const char *name); void zs_destroy_pool(struct zs_pool *pool); =20 unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t flags, const int nid); void zs_free(struct zs_pool *pool, unsigned long obj); +int zs_pool_enable_deferred_free(struct zs_pool *pool, + const struct zs_deferred_ops *ops, + void *private); +bool zs_free_deferred(struct zs_pool *pool, unsigned long value); =20 size_t zs_huge_class_size(struct zs_pool *pool); =20 diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 63128ddb7959..d8220a8753a7 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -196,6 +196,13 @@ struct link_free { static struct kmem_cache *handle_cachep; static struct kmem_cache *zspage_cachep; =20 +#define ZS_DEFERRED_POOL_SIZE (256 * 1024 / PAGE_SIZE) + +struct zs_deferred_percpu { + unsigned int count; + void *buf; +}; + struct zs_pool { const char *name; =20 @@ -217,6 +224,18 @@ struct zs_pool { /* protect zspage migration/compaction */ rwlock_t lock; atomic_t compaction_in_progress; + + /* per-cpu deferred free */ + const struct zs_deferred_ops *deferred_ops; + void *deferred_private; + struct zs_deferred_percpu __percpu *deferred; + struct work_struct deferred_work; + struct workqueue_struct *deferred_wq; + struct list_head deferred_pool; + unsigned int deferred_pool_count; + spinlock_t deferred_pool_lock; + struct list_head deferred_drain_list; + spinlock_t deferred_drain_lock; }; =20 static inline void zpdesc_set_first(struct zpdesc *zpdesc) @@ -1416,6 +1435,171 @@ void zs_free(struct zs_pool *pool, unsigned long ha= ndle) } EXPORT_SYMBOL_GPL(zs_free); =20 +static struct page *deferred_pool_get(struct zs_pool *pool) +{ + struct page *page =3D NULL; + + spin_lock(&pool->deferred_pool_lock); + if (!list_empty(&pool->deferred_pool)) { + page =3D list_first_entry(&pool->deferred_pool, struct page, lru); + list_del(&page->lru); + pool->deferred_pool_count--; + } + spin_unlock(&pool->deferred_pool_lock); + return page; +} + +static void deferred_pool_put(struct zs_pool *pool, struct page *page) +{ + spin_lock(&pool->deferred_pool_lock); + list_add_tail(&page->lru, &pool->deferred_pool); + pool->deferred_pool_count++; + spin_unlock(&pool->deferred_pool_lock); +} + +static void zs_deferred_work_fn(struct work_struct *work) +{ + struct zs_pool *pool =3D container_of(work, struct zs_pool, deferred_work= ); + struct page *page; + + while (true) { + unsigned int count; + + spin_lock(&pool->deferred_drain_lock); + if (list_empty(&pool->deferred_drain_list)) { + spin_unlock(&pool->deferred_drain_lock); + break; + } + page =3D list_first_entry(&pool->deferred_drain_list, + struct page, lru); + list_del(&page->lru); + count =3D page_private(page); + spin_unlock(&pool->deferred_drain_lock); + + pool->deferred_ops->drain(pool->deferred_private, + page_address(page), count); + deferred_pool_put(pool, page); + cond_resched(); + } +} + +bool zs_free_deferred(struct zs_pool *pool, unsigned long value) +{ + struct zs_deferred_percpu *def; + struct page *new_page, *full_page; + enum zs_push_ret ret; + + if (!pool->deferred) + return false; + + def =3D get_cpu_ptr(pool->deferred); + + ret =3D pool->deferred_ops->push(def->buf, def->count, value); + if (ret =3D=3D ZS_PUSH_OK) { + def->count++; + put_cpu_ptr(pool->deferred); + return true; + } + + if (ret =3D=3D ZS_PUSH_FULL_QUEUED) + def->count++; + + new_page =3D deferred_pool_get(pool); + if (new_page) { + full_page =3D virt_to_page(def->buf); + set_page_private(full_page, def->count); + def->buf =3D page_address(new_page); + def->count =3D 0; + + if (ret =3D=3D ZS_PUSH_FULL) { + pool->deferred_ops->push(def->buf, 0, value); + def->count =3D 1; + } + put_cpu_ptr(pool->deferred); + + spin_lock(&pool->deferred_drain_lock); + list_add_tail(&full_page->lru, &pool->deferred_drain_list); + spin_unlock(&pool->deferred_drain_lock); + queue_work(pool->deferred_wq, &pool->deferred_work); + return true; + } + put_cpu_ptr(pool->deferred); + + /* ret=3D=3D2: value already queued, will be drained eventually */ + if (ret =3D=3D 2) + return true; + + /* ret=3D=3D1: value not queued, caller must fallback */ + return false; +} +EXPORT_SYMBOL_GPL(zs_free_deferred); + +int zs_pool_enable_deferred_free(struct zs_pool *pool, + const struct zs_deferred_ops *ops, + void *private) +{ + int cpu; + unsigned int pg_idx; + struct page *page, *tmp; + + pool->deferred_ops =3D ops; + pool->deferred_private =3D private; + + INIT_WORK(&pool->deferred_work, zs_deferred_work_fn); + pool->deferred_wq =3D alloc_workqueue("zs_drain", WQ_UNBOUND, 0); + if (!pool->deferred_wq) + return -ENOMEM; + + INIT_LIST_HEAD(&pool->deferred_pool); + spin_lock_init(&pool->deferred_pool_lock); + pool->deferred_pool_count =3D 0; + INIT_LIST_HEAD(&pool->deferred_drain_list); + spin_lock_init(&pool->deferred_drain_lock); + + for (pg_idx =3D 0; pg_idx < ZS_DEFERRED_POOL_SIZE; pg_idx++) { + page =3D alloc_page(GFP_KERNEL); + if (!page) + goto err_pages; + list_add_tail(&page->lru, &pool->deferred_pool); + pool->deferred_pool_count++; + } + + pool->deferred =3D alloc_percpu(struct zs_deferred_percpu); + if (!pool->deferred) + goto err_pages; + + for_each_possible_cpu(cpu) { + struct zs_deferred_percpu *def =3D per_cpu_ptr(pool->deferred, cpu); + + page =3D deferred_pool_get(pool); + if (!page) + goto err_percpu; + def->buf =3D page_address(page); + def->count =3D 0; + } + + return 0; + +err_percpu: + for_each_possible_cpu(cpu) { + struct zs_deferred_percpu *def =3D per_cpu_ptr(pool->deferred, cpu); + + if (def->buf) + deferred_pool_put(pool, virt_to_page(def->buf)); + } + free_percpu(pool->deferred); + pool->deferred =3D NULL; +err_pages: + list_for_each_entry_safe(page, tmp, &pool->deferred_pool, lru) { + list_del(&page->lru); + __free_page(page); + } + destroy_workqueue(pool->deferred_wq); + pool->deferred_wq =3D NULL; + return -ENOMEM; +} +EXPORT_SYMBOL_GPL(zs_pool_enable_deferred_free); + static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { @@ -2182,9 +2366,31 @@ EXPORT_SYMBOL_GPL(zs_create_pool); =20 void zs_destroy_pool(struct zs_pool *pool) { - int i; + int i, cpu; + struct page *page, *tmp; =20 zs_unregister_shrinker(pool); + + if (pool->deferred) { + flush_work(&pool->deferred_work); + for_each_possible_cpu(cpu) { + struct zs_deferred_percpu *def =3D + per_cpu_ptr(pool->deferred, cpu); + + if (def->buf && def->count) + pool->deferred_ops->drain(pool->deferred_private, + def->buf, def->count); + if (def->buf) + deferred_pool_put(pool, virt_to_page(def->buf)); + } + free_percpu(pool->deferred); + list_for_each_entry_safe(page, tmp, &pool->deferred_pool, lru) { + list_del(&page->lru); + __free_page(page); + } + destroy_workqueue(pool->deferred_wq); + } + zs_flush_migration(pool); zs_pool_stat_destroy(pool); =20 --=20 2.34.1 From nobody Sat May 30 12:36:04 2026 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79011343D86 for ; Fri, 8 May 2026 06:08:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778220527; cv=none; b=pmZCUm4NL4+Hg15o4aWZr8V3iY8sqYabwSavxDkPB2fhQpoS7udkeVFva8cniujApgTy4yyD2UDZrmhrZYidxG7kHTCIiX3P3zKkn0Occz8+BxC4GuUgmxW8Qs4LOjcXKc9C4QIfHyZySkGhHdy2FDB0ePSFp5F7Nvs4ZgWJiSk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778220527; c=relaxed/simple; bh=FyN27PCxe7idHEb0M9IbW6EmU5wzpxP1oM6GSGDaxlE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=A3MA15FllnU85uEwEIkSPCsC9HvfkyAhdRPALJQXYZICY8K9dDk1wKPc6ocvmRnnGeIpNuT9UDmP7KN6JRY4eDIPoILveRVWwCxcIXBVzzp/xyTt5xepzx8X7t0Kd3KnydxBLdSW/cDXgsrX3WFLxQEP4wM/FkuQh+qxs2AKYhs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Uq8LMJCy; arc=none smtp.client-ip=209.85.216.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Uq8LMJCy" Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-365d4d2f9c1so1057293a91.1 for ; Thu, 07 May 2026 23:08:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778220520; x=1778825320; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r2lOcE/NP78qCrPw4J8Q2tX1lxKRSjJyhRJEj/5K+f4=; b=Uq8LMJCyhNVwF6uiTHJWYg5x3MdhlHE2y3y26N52ZLn2c83Syil7b6CpO8ueQZnDsZ 0IKI/qZ8QRT7zm6j+KyXiw7cATrWv2qYwvaCjgYcwAdQEnDZ/OwXYObtapWFWJHGVppD TuXRq+hWfRUdsWLtclU3EoFiZMehvHRWDQX7HEyGpdZVRldLDMS00PjUEywXxloA/HJx PppzounWEzqdysqUzz/57FEtcX/n49YTVL1ODvOS+orzzrmBRLhSc4GADoVzTqtamLwe aOynqz4YTqqb1QaWuZ4Vn1SZiOl4sWocN/atToRNLcrEGzR7KRCORyemFc+miR0VGKS8 bZwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778220520; x=1778825320; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=r2lOcE/NP78qCrPw4J8Q2tX1lxKRSjJyhRJEj/5K+f4=; b=fBWFDLfghXD3B48+BdxTbljN9+59mtjMwyde+HeGmKE/6DR0/UnjEAuJG4hSkotKe+ EipQEnlKRy30Po1CverdUWJuUU3T+IDjxcNjsFFmrIdgBa3yerOYYbGWtJdbsq3GWNqB wxBDm0RRsiwTP9rt3oo8g8VeIdcG59n4f8onyWP0PLPvxCWqF7k0qMy/q3S3/pk7jy7Y 06lE3az+aGNFvgmVVQyBMYKqxBSKU/g0yLAxcZecc6roqbUVgPfDKm2ZX0UbZ94c0kLs hB6UP9ssg6KsG2KDywsrqQVnMU7Ld4SFFdMgs4A7g0AaXnV3c9QuFYAPevmGXuuetqwz mnfQ== X-Forwarded-Encrypted: i=1; AFNElJ+WKjzN4byQI1kHWnSDTaiCyUEjBE139FETUlC7jpyWJXC9d3hjC/cfFdiun813NMm/8uZBPdMWsDV4XHE=@vger.kernel.org X-Gm-Message-State: AOJu0Yz4jmIMrRG75k4b+eU2NSEyno2MH9VUytlybpEZMCRH+rD4FOeu RqmOT10buAQ4Se1A4Ckm3bkYspc6HACt7HeNM1KLxrTAKKfdiVmXMTo4 X-Gm-Gg: Acq92OHgem/0XZoDo2jKD7BQFLIpbBqFfxh/iFsamFO8FyooSH9NLWOuPX/eSGY2v0y ezssnBIrQ0N7yYnL7Jk9Vi3NvReYB80jRuG37Fgq5lXUdmUPPlTrIxfg8ujd/LvmoiAOpXdkbcD 99ZeChTtc7oHzwY38l8zCAtKfk0FtLh40Wjj/IIGChWc1Fn7Ui84vMiwguB4w5X/HnQRwajhomX w3owzChYhJppqZm4xf4+QQhdwVZp23F3Rb0fCUoUxuRVHvo6Y203QTBvgHHjHbwr8HMLUZdgw+q sOkpL9bNp8JZA2dGOi8H7Gh+UkUrXQoJY0f9MCMzqzkLYze7FstMp6K6O/NhMdnoRSDJ1n01IYO uxB+ZqvsfXsFMeT2Uf+SrsF4L+EseL5GWTkeXSzBQxdTN7E+bkPkir7TTY1DgmR5cMA+Xa+iePG OkDZIOT4N89hnHjrsS45S3jqYOL58FLlq/T8xUfw== X-Received: by 2002:a17:90b:1c8b:b0:364:534b:a898 with SMTP id 98e67ed59e1d1-365abad59c2mr10570996a91.8.1778220519914; Thu, 07 May 2026 23:08:39 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36645bb9c02sm673651a91.1.2026.05.07.23.08.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 23:08:39 -0700 (PDT) From: Wenchao Hao X-Google-Original-From: Wenchao Hao To: Andrew Morton , Barry Song <21cnbao@gmail.com>, Chengming Zhou , Jens Axboe , Johannes Weiner , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Nhat Pham , Sergey Senozhatsky , Yosry Ahmed Cc: Wenchao Hao , Wenchao Hao Subject: [RFC PATCH v3 2/4] mm/zswap: use zsmalloc deferred free callback for async invalidate Date: Fri, 8 May 2026 14:07:22 +0800 Message-Id: <20260508060724.3810904-3-haowenchao@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260508060724.3810904-1-haowenchao@xiaomi.com> References: <20260508060724.3810904-1-haowenchao@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Register zswap_deferred_ops to defer the entire zswap_entry_free() to the WQ_UNBOUND worker. The invalidate hot path only stores the entry pointer into the per-cpu buffer (512 entries/page). The drain callback performs the full entry teardown: lru_del, zs_free, memcg uncharge, cache_free, and stats update. On deferred failure, fallback to synchronous zswap_entry_free(). Signed-off-by: Wenchao Hao --- mm/zswap.c | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/mm/zswap.c b/mm/zswap.c index 4b5149173b0e..3f23ddbe525c 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -270,6 +270,8 @@ static void acomp_ctx_free(struct crypto_acomp_ctx *aco= mp_ctx) acomp_ctx->buffer =3D NULL; } =20 +static const struct zs_deferred_ops zswap_deferred_ops; + static struct zswap_pool *zswap_pool_create(char *compressor) { struct zswap_pool *pool; @@ -289,6 +291,8 @@ static struct zswap_pool *zswap_pool_create(char *compr= essor) if (!pool->zs_pool) goto error; =20 + zs_pool_enable_deferred_free(pool->zs_pool, &zswap_deferred_ops, pool); + strscpy(pool->tfm_name, compressor, sizeof(pool->tfm_name)); =20 /* Many things rely on the zero-initialization. */ @@ -777,6 +781,36 @@ static void zswap_entry_free(struct zswap_entry *entry) atomic_long_dec(&zswap_stored_pages); } =20 +static enum zs_push_ret zswap_deferred_push(void *buf, + unsigned int count, unsigned long value) +{ + unsigned long *entries =3D buf; + + if (count >=3D PAGE_SIZE / sizeof(unsigned long)) + return ZS_PUSH_FULL; + entries[count] =3D value; + if (count + 1 >=3D PAGE_SIZE / sizeof(unsigned long)) + return ZS_PUSH_FULL_QUEUED; + return ZS_PUSH_OK; +} + +static void zswap_deferred_drain(void *private, void *buf, unsigned int co= unt) +{ + unsigned long *entries =3D buf; + unsigned int i; + + for (i =3D 0; i < count; i++) { + struct zswap_entry *entry =3D (struct zswap_entry *)entries[i]; + + zswap_entry_free(entry); + } +} + +static const struct zs_deferred_ops zswap_deferred_ops =3D { + .push =3D zswap_deferred_push, + .drain =3D zswap_deferred_drain, +}; + /********************************* * compressed storage functions **********************************/ @@ -1647,7 +1681,9 @@ void zswap_invalidate(swp_entry_t swp) return; =20 entry =3D xa_erase(tree, offset); - if (entry) + if (!entry) + return; + if (!zs_free_deferred(entry->pool->zs_pool, (unsigned long)entry)) zswap_entry_free(entry); } =20 --=20 2.34.1 From nobody Sat May 30 12:36:04 2026 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1920735CB8B for ; Fri, 8 May 2026 06:08:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778220536; cv=none; b=C3tPhBtSdPCW0DYC40BxegUauai9i1fSCaO24nTvZl0aHd+n1dOMhfnryG5/G+HAmpSGo0Vjs2So4/vIGGOCgKw3ptkiaBW2fWEcZ9nzpcCXjN9jvq+JfgNvdhVat20N9C0Eb1qwBPgXGcXjmpsizExKPsAVYtV1hi1GAWWN170= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778220536; c=relaxed/simple; bh=QFVoHteQkcIT+51G9kCmVEfuqfNqbnC+Yohfl+w3FDc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iwIh7ld2b2ZjOG6UUdepQysG4FNKqkQLYQEJ58jA+IeF5JxonrfPjTCJeRiO+vtz7+p5nVhA2qDRI0wIhQ6weDeapiZvNwvLJkET5KkAtqTJfoqOxA0ra/2MAakFPd9PYGVXrFn8iI/WwDDvpZomY6Rp36ttq9TpJgqUWmluQpY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iSuHa/Sn; arc=none smtp.client-ip=209.85.216.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iSuHa/Sn" Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-365212191f6so1201524a91.3 for ; Thu, 07 May 2026 23:08:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778220524; x=1778825324; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+yZbCZTgQWjeOZMMSiccr5Z+zvHQMRYXTy9f6X1ybwU=; b=iSuHa/Sn5HvUlCHS4qwIZhc5EaB7lLK+3AMg+jnT25+mGURBpPsjDhqF0C72wXJdeW KDzwpTpYQkDwsgvxaOAsQebSzvUG1TZKehIvqNXMBodRp58J1ALIO4DwVF8ADIv4jNOI P0VFj5FyjmwLNRWL0BP4mcglV9wpYQtBYhJCp/7QPIt/hd/+EJvXqijy4UZoQnzL8lk1 kqU1Zg1lFEpLwfix2lnay2+XjKeD1cfEqUVSuHXdZmWb76EC6afsKYv2Px1bZokSUBeD SJ+h32F1WSotPh6vZ8Z1tml3jxTbL15Hsece+hVS2i2vPsxgJH6UveP2AngELq1/YYAj mnPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778220524; x=1778825324; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=+yZbCZTgQWjeOZMMSiccr5Z+zvHQMRYXTy9f6X1ybwU=; b=JLXMV0qXWphSI0WcTWA3FaHdoT64VkxVanVICIxZBOp8IO5CadZku9Vv431fIRwO5Z ZgROe2rvYSD4RWOCss7v0e3u/nui337gcdKKWZpmArGaFBiJsikHAsezHQN7xw2lhPI1 mg7KCNvE1wyVyjj+E+hasbH1Tc5HBnxEq6/QiX49M7qP+FdNQlOl5wNfPf9BkyW8YD9u 6aEiINIOPFabZs8SwEn4dYG5wHyQUIbcLc+n8Lvlu+BcRuY65CImBm9n4nM24coK/prC n/FzGEV+v3d1u1LkM3nw4HTQQXGxoXOLcvScuJRBCsyCrD58K//5XIlSF4UaVOzca6JV Ulsw== X-Forwarded-Encrypted: i=1; AFNElJ+bER4b7rlvElnjmZwsxAqcx1pqZwPRnWg7WB5cJtgv8lJgEGgtmKjmw5ackyg/weGS8PWtJUqv14oh3iw=@vger.kernel.org X-Gm-Message-State: AOJu0YxZZitQzRnoC/oHk2T1X+r0Y6rLVqaLRgCjBrECsG4NyAAq44CQ jV6KvZm1F3J0przg5ioIUnIqslQPDxm/LjNBHxUHpQc4ZLaBOC2P29hr X-Gm-Gg: Acq92OHi9hUP1FaSpYmhKfJjgJdusc+58zWpub45zqRYovz6w544ArZKXBFybqnz29S 3LvdtXXmMItbGeBXNayHoRPCFGXrKU0jdKSvC8TrpjI0lps/VyKfjw24YIJoiUl21P5jZ90+PMR VIEV6FJU2QOyW7WyyD3dlIjQsVbyS2c4+7NwEPiYevAPjT1Q6SnydZWxJylFl7Riy3Nba/MyUJO x7DqkZz7uybLPjKMCmE1Aqc6QEdXQlG6Mn4FAcZWZ1Vy9gdql4U5+gFz1bEh41qI5XOXlUUC2Gm YZRsKLje2X+rJT40mVkYNk48HUiQmhHAmT6D9hxzL45dVP/ZjSekhhe43TLSRX1gsqTSDJ5vJ7A n3NPcOHL+vmIY33ShJZiwJQioS2sltWqHXWGmygnBlcUhYTCa6sOP9JLMx/GSh+Au0tNyzKiTjy gDE4Y8dSmcML9Jz5FAtmKkIrW9SsZ4oCAAF5W1OZJZp5uz38R8 X-Received: by 2002:a17:90b:3d0b:b0:35b:952c:43b9 with SMTP id 98e67ed59e1d1-365abcf0698mr11808706a91.10.1778220524197; Thu, 07 May 2026 23:08:44 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36645bb9c02sm673651a91.1.2026.05.07.23.08.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 23:08:43 -0700 (PDT) From: Wenchao Hao X-Google-Original-From: Wenchao Hao To: Andrew Morton , Barry Song <21cnbao@gmail.com>, Chengming Zhou , Jens Axboe , Johannes Weiner , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Nhat Pham , Sergey Senozhatsky , Yosry Ahmed Cc: Wenchao Hao , Barry Song , Wenchao Hao Subject: [RFC PATCH v3 3/4] zram: use zsmalloc deferred free callback for async slot free Date: Fri, 8 May 2026 14:07:23 +0800 Message-Id: <20260508060724.3810904-4-haowenchao@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260508060724.3810904-1-haowenchao@xiaomi.com> References: <20260508060724.3810904-1-haowenchao@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Barry Song Register zram_deferred_ops with zs_pool_enable_deferred_free() to defer slot freeing to a WQ_UNBOUND worker. The notify hot path only stores a u32 slot index into the per-cpu buffer (1024 entries/page). The drain callback does slot_lock + slot_free + slot_unlock for each index. On deferred failure (no free page), fallback to synchronous slot_lock + slot_free + slot_unlock. Signed-off-by: Barry Song Signed-off-by: Wenchao Hao --- drivers/block/zram/zram_drv.c | 39 +++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index aebc710f0d6a..0d07f0901e55 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -56,6 +56,7 @@ static size_t huge_class_size; static const struct block_device_operations zram_devops; =20 static void slot_free(struct zram *zram, u32 index); +static const struct zs_deferred_ops zram_deferred_ops; #define slot_dep_map(zram, index) (&(zram)->table[(index)].dep_map) =20 static void slot_lock_init(struct zram *zram, u32 index) @@ -1994,6 +1995,8 @@ static bool zram_meta_alloc(struct zram *zram, u64 di= sksize) if (!huge_class_size) huge_class_size =3D zs_huge_class_size(zram->mem_pool); =20 + zs_pool_enable_deferred_free(zram->mem_pool, &zram_deferred_ops, zram); + for (index =3D 0; index < num_pages; index++) slot_lock_init(zram, index); =20 @@ -2784,6 +2787,39 @@ static void zram_submit_bio(struct bio *bio) } } =20 +static enum zs_push_ret zram_deferred_push(void *buf, + unsigned int count, unsigned long value) +{ + u32 *indices =3D buf; + + if (count >=3D PAGE_SIZE / sizeof(u32)) + return ZS_PUSH_FULL; + indices[count] =3D (u32)value; + if (count + 1 >=3D PAGE_SIZE / sizeof(u32)) + return ZS_PUSH_FULL_QUEUED; + return ZS_PUSH_OK; +} + +static void zram_deferred_drain(void *private, void *buf, unsigned int cou= nt) +{ + struct zram *zram =3D private; + u32 *indices =3D buf; + unsigned int i; + + for (i =3D 0; i < count; i++) { + u32 index =3D indices[i]; + + slot_lock(zram, index); + slot_free(zram, index); + slot_unlock(zram, index); + } +} + +static const struct zs_deferred_ops zram_deferred_ops =3D { + .push =3D zram_deferred_push, + .drain =3D zram_deferred_drain, +}; + static void zram_slot_free_notify(struct block_device *bdev, unsigned long index) { @@ -2792,6 +2828,9 @@ static void zram_slot_free_notify(struct block_device= *bdev, zram =3D bdev->bd_disk->private_data; =20 atomic64_inc(&zram->stats.notify_free); + if (zs_free_deferred(zram->mem_pool, (unsigned long)index)) + return; + if (!slot_trylock(zram, index)) { atomic64_inc(&zram->stats.miss_free); return; --=20 2.34.1 From nobody Sat May 30 12:36:04 2026 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 274B73783B1 for ; Fri, 8 May 2026 06:08:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778220539; cv=none; b=R08uFsRZZEvVZjvQovQcYHgAmj99/r2Ly7KK8NiiX2x9oiQkLttbamCGkjX0ZXrOXtbQY2aJx4bp9v7wlL3oMc9iABVz9fF8tDyxVbTwj6aGi7ctVq3CR5PwgvqXmb9cJ7/kt0zjC+PQ/pwL1nGrwGBHOJVB1YNYyp90akPmEXM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778220539; c=relaxed/simple; bh=SvrLVwrtLzVKQiQQgO56qCvdjLHcnaINUCgpfFTOyjs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=asRELUJgHxgbSCwrxjbRGMU5WRECpul0GGDuUNviduqcjAEnQONXcjsRVywyCs4h9K0K1rG8ng22NwHnQY79az2rKjVi49VNLFIS5axRAtCJEfRnlOLS8b1nSc7ZGapQkTjAiN0bDEAA5CaE1gWrOkWp4vtNP+Wpe6JMoweBJW4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RfJUfGiY; arc=none smtp.client-ip=209.85.216.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RfJUfGiY" Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-3650453fb28so1007596a91.2 for ; Thu, 07 May 2026 23:08:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778220528; x=1778825328; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lYRirjFTjUvafOj+0puY9l5IWTCm5/8QMipbtzzjFcI=; b=RfJUfGiY2opvt5Qy6ZHH4g++pvIOLkEE8ACFgd7fvUDbLFn1HNh8jhoQT+YQsdu5mb Cj7opou6xaZLwb3ZQKGcbimxnUS5HdtJuUkS4a0KPMN3C6f5uFxOuirvEMlPk2c0T93e npAFkjAU/KULLcjKY10KwqyLaLykBeSOhCX5p/2GLV5j1o2gsxSE5UTgQMGfSa209DlK cX46FxiWL1Vnswa0Luk5btm+1fNIQ6FfTbsbwNCAlhonN56vZjEmrpEg0hS/Mu/LNPAs VJ73pE+/n1+PKwnyeNC12ndWEcgVTDaJI4hdDY4x5xCSjxD0nV4kSJcC3jsByZ6O/nZV WWtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778220528; x=1778825328; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=lYRirjFTjUvafOj+0puY9l5IWTCm5/8QMipbtzzjFcI=; b=e5Dc+Ucl7ruvhlJFYsFnpurBdvymQZmjf0RdyhTClLlzo1Wo/z3LUiNc4/5hpCK0VR KdVlMsaKecqooQRwJyNDFPFRzIBsqIj6S6QGiA1r+sDX9a98wbIxugtX9dxJ2FI6ALSh 98xAvfuXxxD9QGCJ99dm4yQDy32khTdu7/WcWlsSJ09ppofWUlwnOjNKIjt9jdRazcld iJSRsSa7DYINiK9WochV5MM3RyQd13DBl1vN0WJrxffBNvFGIb+xKxv48ocdFGAYj71v 8tIrt5O5XFCNbPhfT80pOXK59I314JVcftXOzEwTKIlcJksAbC4GV0cO2oRp9ZkCl/il PjAQ== X-Forwarded-Encrypted: i=1; AFNElJ/zFo2VPCOPr9AiSOJfsmd5KuFKS0ZmcLhdQ0dfaCaMrPL29PQrUYJG3shd3gXY0Nd+9aw3f/UI4JMt++Y=@vger.kernel.org X-Gm-Message-State: AOJu0YyEixDkgPZMqZd63NU1iS+lfo0iVkRn2tNHI0HHWw+HAmAVEyzK JBVPxMHQCm08TuqRUgW1XMJKWKm+wHUaln0wo5mOWV+LcBcXeNdxLKSX X-Gm-Gg: Acq92OFAvgK6XvzS0opUmSE7NRUuUkiBfTaK6GvJdF+9DnYhVKBmT1sFkV78/CdQaUd xxp9YexPki2aL/d7lReT17q498PiIJ5HRhD4XlMV63FzByWcCxj0yL+/sULaOdHXlmZ00Y0v08R vTye4/W0YR+EJQyavZU/AOfM3QDZUFCtoTOkBmXlO2ABNH6L4mvD78aVBIVRxbW/yhUWbJoMEwD bh4urjwcD2/3BQWHOs/ijo97W7E7fv0Xln3AsTx4Kh0wHoACYv4hIvBfS9JbpCpaFr53mle01Ya ctfQ0JLXmGSYvFOvd5xYjpxHRbDeW+h07a47j6cGmQuQeNGQ129izZu+lh+i0PDwbD/lkX6gjqp za4QdNA5f3Mg0xnSzkTyhdv+Ub4vbv5OjESsmIX4c4l04KKMAMu4RS2dsJ3lNd2HRJmDIbbTv8+ Y6M/5djzsdbUaH0Y/ZaB3PmA2rocPg1QuRs5EBoQ== X-Received: by 2002:a17:90b:3805:b0:35e:b017:da6a with SMTP id 98e67ed59e1d1-365aa73cc81mr11744580a91.0.1778220528288; Thu, 07 May 2026 23:08:48 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36645bb9c02sm673651a91.1.2026.05.07.23.08.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 23:08:47 -0700 (PDT) From: Wenchao Hao X-Google-Original-From: Wenchao Hao To: Andrew Morton , Barry Song <21cnbao@gmail.com>, Chengming Zhou , Jens Axboe , Johannes Weiner , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Nhat Pham , Sergey Senozhatsky , Yosry Ahmed Cc: Wenchao Hao , Wenchao Hao Subject: [RFC PATCH v3 4/4] zram: batch clear flags in slot_free with single write Date: Fri, 8 May 2026 14:07:24 +0800 Message-Id: <20260508060724.3810904-5-haowenchao@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260508060724.3810904-1-haowenchao@xiaomi.com> References: <20260508060724.3810904-1-haowenchao@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace four separate flag clear operations in slot_free() with a single mask write. This reduces redundant read-modify-write cycles on the same flags word. Signed-off-by: Wenchao Hao --- drivers/block/zram/zram_drv.c | 5 +---- drivers/block/zram/zram_drv.h | 6 ++++++ 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 0d07f0901e55..b1a565d35567 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2011,10 +2011,7 @@ static void slot_free(struct zram *zram, u32 index) zram->table[index].attr.ac_time =3D 0; #endif =20 - clear_slot_flag(zram, index, ZRAM_IDLE); - clear_slot_flag(zram, index, ZRAM_INCOMPRESSIBLE); - clear_slot_flag(zram, index, ZRAM_PP_SLOT); - set_slot_comp_priority(zram, index, 0); + zram->table[index].attr.flags &=3D ~ZRAM_SLOT_FREE_CLEAR_MASK; =20 if (test_slot_flag(zram, index, ZRAM_HUGE)) { /* diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 08d1774c15db..89a7e39a2f4b 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -57,6 +57,12 @@ enum zram_pageflags { __NR_ZRAM_PAGEFLAGS, }; =20 +#define ZRAM_SLOT_FREE_CLEAR_MASK (BIT(ZRAM_IDLE) | \ + BIT(ZRAM_INCOMPRESSIBLE) | \ + BIT(ZRAM_PP_SLOT) | \ + (ZRAM_COMP_PRIORITY_MASK << \ + ZRAM_COMP_PRIORITY_BIT1)) + /* * Allocated for each disk page. We use bit-lock (ZRAM_ENTRY_LOCK bit * of flags) to save memory. There can be plenty of entries and standard --=20 2.34.1