From nobody Wed Jun 17 01:42:35 2026 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BAD7F3BE63F for ; Tue, 21 Apr 2026 12:16:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776773803; cv=none; b=jo7iFzyMhFLyYWlO16IQnRSqVKZMzZJAPx//EImWPKRpAlN5Y1yXUqrP1lbDTgSqS0X+w7EJeQ8gKR9G+tBGTciMwPXXr4MPL6C+e1Y69tJgNNeM1ZA1lV8vqhZc4GukdGjBhZt581NfYXMCs1fPqY7vPVtpbhboOqkE7H2JKnA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776773803; c=relaxed/simple; bh=kLgOVy+bATsUzIamzLMfOptz5iAJtTXdDloAK01IeDI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qbAPtAcZwYZd0Kj6fp+bym9ULu3tDh7Q9ChWWypNl2u7fDq3DA9bLYdUAteSLctWh1uc0SuGPwaZgE9Ivhi4o63cUE0sp9HOmEieFYeDVebG+mFoRk54eL7jnXf5KC5XAgTZS/1/0LhZu9Q+fMn7Ai6zIY3WwM/v8v4KWqZvL2s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=N7U4/zMr; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="N7U4/zMr" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-8296dabef74so3674805b3a.1 for ; Tue, 21 Apr 2026 05:16:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1776773797; x=1777378597; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jm1uZfhnuhLjYk2Ukt+cLbqh8Cqoh4c1TPve8K7i0ls=; b=N7U4/zMr7DoPqj/U9MJ7LNFajgi/eNAG5nQH0lS2HEmXZqz+P1ng/TJhULW5tXxJsg WFYqsHe4B34+HDRZCeFpUHmxV0l/aySpgPnCldH3wkWD1YURcDXC1Kt5HE7KawAOwGTv BUHOjJBStSyVoP4DznSEflE+WmSu8NbhJ9kJW5FlnPlGk+xu8FnsjC+sHlk4Lty35hPm gbc6u4XDtwaNrzov8n6yamlsrMVUefECyFP9lxwPQz18qCi1EYufaGEzeHzGoCKIZNHK 172wGktkoVO8sOhVy7GF+cqqXUgQUuYJ78X9jm30yTR7U46huxLYpyCvOTSH9L+KZu1V yHxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776773797; x=1777378597; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Jm1uZfhnuhLjYk2Ukt+cLbqh8Cqoh4c1TPve8K7i0ls=; b=BXh2N6hvKMHWs3VmHgpV0kM8yq9MLLH9xNHOsobLPt4/lreOA71p7owTVQBs55Hn51 0a8BQO2wVFVMIcTzPZEfJ8l308ZeAzt2vLMck/2dNCnTDL6DfNHJJmjfDY6O8HdEubUY h72svTpnbXnGed/xLbX0HTGq71I/jmWa+TilQJWyOfhvnMhK3NARAmydGlS97dUmKSBn SfeqjfA+1J/zcE0/whCcFeiamGO0MD7p7BEjnzo+eNZsl5pw8S8ksVBoxj98hrlo+QD7 /d7sFjZCwi7GVq5e3N0OKoBa/w7nGV2re2NH+V5vNweFG5POAMjxWie8++PutHqclhSR IqmA== X-Forwarded-Encrypted: i=1; AFNElJ/YYAiOJmdFJ4ZQM1nyb9EGyROAwUo+j6VAodr/cVEBgdUjdez98V8bPWEOeIpS4WqgMnFPNjCqjtSyNuA=@vger.kernel.org X-Gm-Message-State: AOJu0YyfAsFNpQg0J0oWCx21sDub2q2qEYorV1lk3uKM7Usa4K8mTY0c uu3SEMjqNRDcGwY+DAIIfq2SExbwPlLZWTr6tNHHSsyNe+QXJzAPFWHz X-Gm-Gg: AeBDietuV4C7OBsN1MWQxXSqKwPgdZizAef3N+JYLwlRVVdXZIW9nZ2NLf2WyGVxb+r RyEvdW3YearhByrVoTlqetxwFdSpbgdI/xrHN4P5ulfibaeW9hh/ifsCXk6pw92GSv5zxM6DvwK AK/hppeEWoVVahQs93IFX4POUMiqfs5yVI2kU1ML58E2hps5d/sf6YA4ednPipfT4UcTzbZvypz 1Q9chEQk9KdJpyo2+ZQPvvODmNRfuRbccQC0T+1yjpF7fLqlIPhYUzgEU88YeG3cThDVcUwMCrz FxNcZqhma+KyhfKlps9qAWlcLkAySQeKz2vKW5vQtFIFeu7haJAM3Ar9rjGXCwwbOrCNxFckuzi 4Kr/lGBk5dYcVGzmKwt0Ud9VaifGGM7Al3ce9k8Qd/CAJBY09SZUVklb8eWH9t7iU1ySRH5YhPq +zvdxt4H/xXkyEu6JxNz8MPlNXdzv0zHUho1Z96doFUmNSe+8= X-Received: by 2002:a05:6a00:2e91:b0:829:9a7b:db84 with SMTP id d2e1a72fcca58-82f8c97e604mr19692584b3a.49.1776773796869; Tue, 21 Apr 2026 05:16:36 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-82f8ec037e1sm16371071b3a.54.2026.04.21.05.16.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 05:16:36 -0700 (PDT) From: Wenchao Hao X-Google-Original-From: Wenchao Hao To: Andrew Morton , Chengming Zhou , Jens Axboe , Johannes Weiner , Minchan Kim , Nhat Pham , Sergey Senozhatsky , Yosry Ahmed , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Barry Song , Xueyuan Chen , Wenchao Hao Subject: [RFC PATCH v2 1/4] mm:zsmalloc: drop class lock before freeing zspage Date: Tue, 21 Apr 2026 20:16:13 +0800 Message-Id: <20260421121616.3298845-2-haowenchao@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260421121616.3298845-1-haowenchao@xiaomi.com> References: <20260421121616.3298845-1-haowenchao@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xueyuan Chen Currently in zs_free(), the class->lock is held until the zspage is completely freed and the counters are updated. However, freeing pages back to the buddy allocator requires acquiring the zone lock. Under heavy memory pressure, zone lock contention can be severe. When this happens, the CPU holding the class->lock will stall waiting for the zone lock, thereby blocking all other CPUs attempting to acquire the same class->lock. This patch shrinks the critical section of the class->lock to reduce lock contention. By moving the actual page freeing process outside the class->lock, we can improve the concurrency performance of zs_free(). Testing on the RADXA O6 platform shows that with 12 CPUs concurrently performing zs_free() operations, the execution time is reduced by 20%. Signed-off-by: Xueyuan Chen Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 63128ddb7959..40687c8a7469 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -801,13 +801,10 @@ static int trylock_zspage(struct zspage *zspage) return 0; } =20 -static void __free_zspage(struct zs_pool *pool, struct size_class *class, - struct zspage *zspage) +static inline void __free_zspage_lockless(struct zs_pool *pool, struct zsp= age *zspage) { struct zpdesc *zpdesc, *next; =20 - assert_spin_locked(&class->lock); - VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness !=3D ZS_INUSE_RATIO_0); =20 @@ -823,7 +820,13 @@ static void __free_zspage(struct zs_pool *pool, struct= size_class *class, } while (zpdesc !=3D NULL); =20 cache_free_zspage(zspage); +} =20 +static void __free_zspage(struct zs_pool *pool, struct size_class *class, + struct zspage *zspage) +{ + assert_spin_locked(&class->lock); + __free_zspage_lockless(pool, zspage); class_stat_sub(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); } @@ -1388,6 +1391,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) unsigned long obj; struct size_class *class; int fullness; + struct zspage *zspage_to_free =3D NULL; =20 if (IS_ERR_OR_NULL((void *)handle)) return; @@ -1408,10 +1412,22 @@ void zs_free(struct zs_pool *pool, unsigned long ha= ndle) obj_free(class->size, obj); =20 fullness =3D fix_fullness_group(class, zspage); - if (fullness =3D=3D ZS_INUSE_RATIO_0) - free_zspage(pool, class, zspage); + if (fullness =3D=3D ZS_INUSE_RATIO_0) { + if (trylock_zspage(zspage)) { + remove_zspage(class, zspage); + class_stat_sub(class, ZS_OBJS_ALLOCATED, + class->objs_per_zspage); + zspage_to_free =3D zspage; + } else + kick_deferred_free(pool); + } =20 spin_unlock(&class->lock); + + if (likely(zspage_to_free)) { + __free_zspage_lockless(pool, zspage_to_free); + atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); + } cache_free_handle(handle); } EXPORT_SYMBOL_GPL(zs_free); --=20 2.34.1 From nobody Wed Jun 17 01:42:35 2026 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D95EA3BE633 for ; Tue, 21 Apr 2026 12:16:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776773804; cv=none; b=X1mYn+LyaWIJcEvXw2dKstagi4dBjv6hu6sTYqd/kLJJb5tDKU02IwMdhEkQkDbkzK20KNAsGRouQV4T7V4YtI6BTMmvqFmxLgzAKerLJtkS9HnagCfcnVavN8zP9d8ZOwYzynqQwIK9Onp11n8EUn7v7+PYCS1U6yCb8Qx/QgQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776773804; c=relaxed/simple; bh=yQJQB/l54vycNoNGy/3OE0/3ou0HbdUNcc5tdjcARNg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LGhWxk1B8lIXDcc6vACzs0GFOKgPi11mnzdsqYyktq4FkX7KLtBuQF0KPMM0/ccx+tXNJbNmiGSbpeAIiBQ4eQijCzFC7XXveUhCI5IjDtADtYDrMyAwSbuOExQcbdOMMP0yNcGQOcfDvvwlvDstmedXWKeqMnCcRPuHTuLOh94= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MD6189Ud; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MD6189Ud" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-82fa8d6425bso1130696b3a.0 for ; Tue, 21 Apr 2026 05:16:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1776773801; x=1777378601; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Yjoi/ADx361WB7o/C1DRNygxoHsfgIft6bO7Dodrjv4=; b=MD6189UdxlZR5HE48zVm2wQoDqo1n4rJrBJLy2SUXrzgY0DeayLHRc13rf82mtduv6 stxvNm4+k6/OBii6xKVotG+nyg8RhZDWDyygMhm99YqLobLoSW8TVuFIqxvQvHSH24PC O6CD5TL5mT5Z7PJ2SbNSBQFlmvfi44SKCzfn/2K+QDX0+zlv8TIGRtB61Ya8jRL6rCZY Kq4k/ATkAWw7ZMeTcjQc0p7GhV9FjKR+dLrnWIYMcaU3gikhVFbJddA78TMxLrxrd6Ql XEOrb1wAV/uxvMvyzFu+ZDD5CqxhfonoBlre3z3l6SeewiNj64tcJBk3Ws2jV0jqkJsI /MMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776773801; x=1777378601; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Yjoi/ADx361WB7o/C1DRNygxoHsfgIft6bO7Dodrjv4=; b=pK3odIfFlBp80jlTqFzW9xUWgE3yt6b3Ypf3fxiin9z97MEenIG8dijKJ6YJomYWbS aLyhIwZ7g5EDWhJTPXIh/S8/++JyYNQGElmjyAN3gqleXbBURJcEw96NVDfx+0d7071y bmlwvLaQiNIW2ENQ89jmLvn0ttX6aPMhDEQ2TRlMcNM9jT3B5VeEwlVSwPk50+QPLiXd tkw6NJelX/rE42bHWRrGrqlgwge4zkrIQdMptG92ajFmVwHLKwEKweGFLBEazGETDtHQ JYC9koSb7s1SeWJysYgTkuE0WhNoG405IyYSLc4e6UkzSy778BjJzIEfNbGSRGXrOWVv 5ZNw== X-Forwarded-Encrypted: i=1; AFNElJ+867WCXhKZCEXgGMdSYBKU0Iel5pmhBBtLiSgbkiRhhXy5zRsCsQ9vdJqdHMXTEVAfJkP5eU0hUufW8GY=@vger.kernel.org X-Gm-Message-State: AOJu0YzNHvvQQJXoHjCmUdeon55zFmdWHV1Z8eacs9j5oFA6JBi4eR+B 2CfOzfoao0xKTUEXvcpcSp7fYOkxEDppZNJErNAbpx6zUeGkPqVf0eXO X-Gm-Gg: AeBDieu72pcGecaUtzGku1ywQJ3VnPhHi6Fvn8LC8CdH2XShWv2Uku107llgmAVbSx/ ib3uebta7kl0lmXoffjCa3QvqiGXNo8rm4MDWJn2Hr8fy0esN9md3iZ8GqLNAFY46D2HnRq3l1Y norxqEpSZOhwKTWNpI3RHXoU/yDUJEbJ2bYI+PFpmjruPKNhb/v1LHighlC+juikXMU5WbJBt3o zN4KkTyvIR1o2XXiJf0zT1MvZ1PJ7fk1OoLsu1KN3MKV6TAFXnZA/A438qvkT2l4CkOIhzk2pHO 89T0V+W5hTttZ3jAG2DCFIJHyEK9SsTYgO4vIlGuILjZbRJTRHntiLG743U6/4RLnyEZyTha4tZ yWJz7dDnOEIf0suOInfmFXPhYmrTENQjnBrtdhy4T48EBwkAasoiAvHAVkrUUo947OFkUO7yX1d /51fUd4PoCKJp/xlNvtc1MblxeAs/L8oDAjst/beJ1fE9LAVSaZAYNhrn/Eg== X-Received: by 2002:a05:6a00:3d16:b0:82d:24f:2511 with SMTP id d2e1a72fcca58-82f8c8522f9mr19548140b3a.12.1776773801023; Tue, 21 Apr 2026 05:16:41 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-82f8ec037e1sm16371071b3a.54.2026.04.21.05.16.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 05:16:40 -0700 (PDT) From: Wenchao Hao X-Google-Original-From: Wenchao Hao To: Andrew Morton , Chengming Zhou , Jens Axboe , Johannes Weiner , Minchan Kim , Nhat Pham , Sergey Senozhatsky , Yosry Ahmed , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Barry Song , Xueyuan Chen , Wenchao Hao Subject: [RFC PATCH v2 2/4] mm/zsmalloc: introduce zs_free_deferred() for async handle freeing Date: Tue, 21 Apr 2026 20:16:14 +0800 Message-Id: <20260421121616.3298845-3-haowenchao@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260421121616.3298845-1-haowenchao@xiaomi.com> References: <20260421121616.3298845-1-haowenchao@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" zs_free() is expensive due to internal locking (pool->lock, class->lock) and potential zspage freeing. On the process exit path, the slow zs_free() blocks memory reclamation, delaying overall memory release. This has been reported to significantly impact Android low-memory killing where slot_free() accounts for over 80% of the total swap entry freeing cost. Introduce zs_free_deferred() which queues handles into a fixed-size per-pool array for later processing by a workqueue. This allows callers to defer the expensive zs_free() and return quickly, so the process exit path can release memory faster. The array capacity is derived from a 128MB uncompressed data budget (128MB >> PAGE_SHIFT entries), which scales naturally with PAGE_SIZE. When the array reaches half capacity, the workqueue is scheduled to drain pending handles. zs_free_deferred() uses spin_trylock() to access the deferred queue. If the lock is contended (e.g. drain in progress) or the queue is full, it falls back to synchronous zs_free() to guarantee correctness. Also introduce zs_free_deferred_flush() for use during pool teardown to ensure all pending handles are freed. Signed-off-by: Wenchao Hao --- include/linux/zsmalloc.h | 2 + mm/zsmalloc.c | 111 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 113 insertions(+) diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index 478410c880b1..1e5ac1a39d41 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -30,6 +30,8 @@ void zs_destroy_pool(struct zs_pool *pool); unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t flags, const int nid); void zs_free(struct zs_pool *pool, unsigned long obj); +void zs_free_deferred(struct zs_pool *pool, unsigned long handle); +void zs_free_deferred_flush(struct zs_pool *pool); =20 size_t zs_huge_class_size(struct zs_pool *pool); =20 diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 40687c8a7469..defc892555e4 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -53,6 +53,10 @@ =20 #define ZS_HANDLE_SIZE (sizeof(unsigned long)) =20 +#define ZS_DEFERRED_FREE_MAX_BYTES (128 << 20) +#define ZS_DEFERRED_FREE_CAPACITY (ZS_DEFERRED_FREE_MAX_BYTES >> PAGE_SHIF= T) +#define ZS_DEFERRED_FREE_THRESHOLD (ZS_DEFERRED_FREE_CAPACITY / 2) + /* * Object location (, ) is encoded as * a single (unsigned long) handle value. @@ -217,6 +221,13 @@ struct zs_pool { /* protect zspage migration/compaction */ rwlock_t lock; atomic_t compaction_in_progress; + + /* deferred free support */ + spinlock_t deferred_lock; + unsigned long *deferred_handles; + unsigned int deferred_count; + unsigned int deferred_capacity; + struct work_struct deferred_free_work; }; =20 static inline void zpdesc_set_first(struct zpdesc *zpdesc) @@ -579,6 +590,19 @@ static int zs_stats_size_show(struct seq_file *s, void= *v) } DEFINE_SHOW_ATTRIBUTE(zs_stats_size); =20 +static int zs_stats_deferred_show(struct seq_file *s, void *v) +{ + struct zs_pool *pool =3D s->private; + + spin_lock(&pool->deferred_lock); + seq_printf(s, "pending: %u\n", pool->deferred_count); + seq_printf(s, "capacity: %u\n", pool->deferred_capacity); + spin_unlock(&pool->deferred_lock); + + return 0; +} +DEFINE_SHOW_ATTRIBUTE(zs_stats_deferred); + static void zs_pool_stat_create(struct zs_pool *pool, const char *name) { if (!zs_stat_root) { @@ -590,6 +614,9 @@ static void zs_pool_stat_create(struct zs_pool *pool, c= onst char *name) =20 debugfs_create_file("classes", S_IFREG | 0444, pool->stat_dentry, pool, &zs_stats_size_fops); + debugfs_create_file("deferred_free", S_IFREG | 0444, + pool->stat_dentry, pool, + &zs_stats_deferred_fops); } =20 static void zs_pool_stat_destroy(struct zs_pool *pool) @@ -1432,6 +1459,76 @@ void zs_free(struct zs_pool *pool, unsigned long han= dle) } EXPORT_SYMBOL_GPL(zs_free); =20 +static void zs_deferred_free_work(struct work_struct *work) +{ + struct zs_pool *pool =3D container_of(work, struct zs_pool, + deferred_free_work); + unsigned long handle; + + while (1) { + spin_lock(&pool->deferred_lock); + if (pool->deferred_count =3D=3D 0) { + spin_unlock(&pool->deferred_lock); + break; + } + handle =3D pool->deferred_handles[--pool->deferred_count]; + spin_unlock(&pool->deferred_lock); + + zs_free(pool, handle); + cond_resched(); + } +} + +/** + * zs_free_deferred - queue a handle for asynchronous freeing + * @pool: pool to free from + * @handle: handle to free + * + * Place @handle into a deferred free queue for later processing by a + * workqueue. This is intended for callers that are in atomic context + * (e.g. under a spinlock) and cannot afford the cost of zs_free() + * directly. When the queue reaches a threshold the work is scheduled. + * Falls back to synchronous zs_free() if the lock is contended (drain + * in progress) or if the queue is full. + */ +void zs_free_deferred(struct zs_pool *pool, unsigned long handle) +{ + if (IS_ERR_OR_NULL((void *)handle)) + return; + + if (!spin_trylock(&pool->deferred_lock)) + goto sync_free; + + if (pool->deferred_count >=3D pool->deferred_capacity) { + spin_unlock(&pool->deferred_lock); + goto sync_free; + } + + pool->deferred_handles[pool->deferred_count++] =3D handle; + if (pool->deferred_count >=3D ZS_DEFERRED_FREE_THRESHOLD) + queue_work(system_wq, &pool->deferred_free_work); + spin_unlock(&pool->deferred_lock); + return; + +sync_free: + zs_free(pool, handle); +} +EXPORT_SYMBOL_GPL(zs_free_deferred); + +/** + * zs_free_deferred_flush - flush all pending deferred frees + * @pool: pool to flush + * + * Wait for any scheduled work to complete, then drain any remaining + * handles. Must be called from process context. + */ +void zs_free_deferred_flush(struct zs_pool *pool) +{ + flush_work(&pool->deferred_free_work); + zs_deferred_free_work(&pool->deferred_free_work); +} +EXPORT_SYMBOL_GPL(zs_free_deferred_flush); + static void zs_object_copy(struct size_class *class, unsigned long dst, unsigned long src) { @@ -2099,6 +2196,18 @@ struct zs_pool *zs_create_pool(const char *name) rwlock_init(&pool->lock); atomic_set(&pool->compaction_in_progress, 0); =20 + spin_lock_init(&pool->deferred_lock); + pool->deferred_capacity =3D ZS_DEFERRED_FREE_CAPACITY; + pool->deferred_handles =3D kvmalloc_array(pool->deferred_capacity, + sizeof(unsigned long), + GFP_KERNEL); + if (!pool->deferred_handles) { + kfree(pool); + return NULL; + } + pool->deferred_count =3D 0; + INIT_WORK(&pool->deferred_free_work, zs_deferred_free_work); + pool->name =3D kstrdup(name, GFP_KERNEL); if (!pool->name) goto err; @@ -2201,6 +2310,7 @@ void zs_destroy_pool(struct zs_pool *pool) int i; =20 zs_unregister_shrinker(pool); + zs_free_deferred_flush(pool); zs_flush_migration(pool); zs_pool_stat_destroy(pool); =20 @@ -2224,6 +2334,7 @@ void zs_destroy_pool(struct zs_pool *pool) kfree(class); } =20 + kvfree(pool->deferred_handles); kfree(pool->name); kfree(pool); } --=20 2.34.1 From nobody Wed Jun 17 01:42:35 2026 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4205D3A1685 for ; Tue, 21 Apr 2026 12:16:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776773809; cv=none; b=YL0zuCjFaSO2aY7mePxEljgpGsjPbSNnEAtmwoQzwAaXvW/UKniODxUDlsMCr+ajr+cUqQou4jxgHwpT7DDiHVYW3n+3tyruYr+yV+CD40FCfJ0VzXS4jcFyi5ieHMuVJ8cBeBHzc1Zg1DNXPWfPcA3rMsMr0z/U9xlMN/z/2DE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776773809; c=relaxed/simple; bh=i7ThFkM/qArd39OgyIXeWRiSqkQW27QZPpZUUMok1TY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PcCHO9USlGHLWsk9hqvRSggFDd5GA5qH02l50Vac4UBge0lT82SO+WppLnH4ExsshCnjL/KjposNZ/hVvibCC2OD9zZ6cKwF2CGLgsUx+PWXdsoygzQkEZWtDcCx+Xr8suLkBIUZz9Sbacw6VfOcYPBZVJyuVg6ijAoDbvZwD9s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=g8jQ5wLY; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="g8jQ5wLY" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-8296dabef74so3674906b3a.1 for ; Tue, 21 Apr 2026 05:16:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1776773805; x=1777378605; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0Eih9BBnnHT00t9rzMlOXMywyDxDk82fhUWxfKmMkyw=; b=g8jQ5wLYrBAr9uK7Wtb2oS6Q8Sb1wnKwTRD23rrwIJE5N7WTq0V+m9WZSdIO/ijLjI 31VG1CyMtT++jHqzCSMqRqDTSU+KKB5MKYjHdGzcYiDv0efoP1fNl6nRDp8OueKE9wAm A+N1lLajRFdacj0uH3+CKvdaXtqTTnWyxAOGgYSQ/QUAO2ruqukywlkKfYKbURQoSzMa B9MpIh9jHLn8J80oZA7aze+RcuzGslnTeY1e1gGlmTp58jFC2boRh6Z3XCWn5qMmwsTO OKDy5B5ZV9QVCyYqi3LafB2NeMNHp39SCH7r439sYfxOM+aBrXf9Rn22/5KkyuSp0fkT Zwjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776773805; x=1777378605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=0Eih9BBnnHT00t9rzMlOXMywyDxDk82fhUWxfKmMkyw=; b=ZmiNpoM8IYjczfG3Ulh4rP/ZspB6wt+OyX+UY7QyzGi7g6gx5Wyo+m1Gyoy+th8N5E NyFz82mlWVffSCPPO7s1Xlgmc5l8MQMoxhwhwp5C5wZKnfBu8vPgKSP7lZX2jm9vyjdr +HldyibFoOCmkD83kH9ZlvTyz12F23pQUFwm0nuYywqqlhPipnUXoc/xJT2O+1KnjVZy xk258d34wwGuxCmT/gENk+DzshMvYpERu66V1UnU52R/okfN7r7e3JdvVSVkybvTgx6t PojxSC1xeuhdLsU21+5o0i6w3KNyQ2crmiKqmCNyC3jST0PjyjGGZ1Rye5Ube5Ffpf8M EOng== X-Forwarded-Encrypted: i=1; AFNElJ99V+YbGIkv9nHIrvrmkz7ZxIG3aKFWTaRvVvIfx2dtUZQNV3rlIdnvpX6dUSi6Ij7MiZRilUf/7SwO61s=@vger.kernel.org X-Gm-Message-State: AOJu0YwNFmROvxYReT24zru9dkWLyZqVZeNIrpq8jlAhgUikQshXqa+9 RjDuyS+AYNIyWN2GaSA+MKKQWiRf6kSRUhY4gMOwWRUe4IcCC/+WJrTklBkP5Gb5 X-Gm-Gg: AeBDieuHlylj+qp0FI3ngTcllEpZYRAK3N+79RUbcD64GMEyLzK1sWBGKIrwsa6ngwm /Y5FAxKwJs+5Z2UUTH8Hrt9AU9JkPCgJxK3l75/2+9L1ylkdAqpjxh9mLVD3mEiPsdMTk/DwyNG uzTGvZ5zzOUBcFy17/FQdxNNTZRI8lON24aPF99J0RsKiPpHhyHiJpOovaqUjmMxZtbtBDkXBhS sRT+WWoP5EECHmHCp6cRao3GlTDgRlkA2BZiywmnHiC2z7vkDCBb91DrPABnxW1UjtEpRJydmmj vv+N1cpxFwr2SQdiI/W2YQ+EgMJ1OG8sU8UJrhMowuajlRQTaUDb50cJatgPanFVLj12SyPA8nQ BTymI9vptArG7eY95d6n5jItWeJGliVGJPkWWHg6HOBKvraWP3bRYogV34frG57YIg9ZR/CzdY1 BsAwatGyprgQ2bAI+joPh4ZUVx42X5I6SdOe8RSXKYJeHKz4Y= X-Received: by 2002:a05:6a00:2d19:b0:82d:5da9:adc9 with SMTP id d2e1a72fcca58-82f8c856e60mr19094536b3a.12.1776773805202; Tue, 21 Apr 2026 05:16:45 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-82f8ec037e1sm16371071b3a.54.2026.04.21.05.16.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 05:16:44 -0700 (PDT) From: Wenchao Hao X-Google-Original-From: Wenchao Hao To: Andrew Morton , Chengming Zhou , Jens Axboe , Johannes Weiner , Minchan Kim , Nhat Pham , Sergey Senozhatsky , Yosry Ahmed , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Barry Song , Xueyuan Chen , Wenchao Hao Subject: [RFC PATCH v2 3/4] zram: defer zs_free() in swap slot free notification path Date: Tue, 21 Apr 2026 20:16:15 +0800 Message-Id: <20260421121616.3298845-4-haowenchao@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260421121616.3298845-1-haowenchao@xiaomi.com> References: <20260421121616.3298845-1-haowenchao@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Barry Song (Xiaomi)" zram_slot_free_notify() is called on the process exit path when unmapping swap entries. The slot_free() it calls internally invokes zs_free(), which accounts for ~87% of slot_free() cost due to zsmalloc internal locking (pool->lock, class->lock) and potential zspage freeing. This blocks the process exit path, delaying overall memory release during Android low-memory killing. Split slot_free() into slot_free_extract() and the actual zs_free() call. slot_free_extract() handles all slot metadata cleanup (clearing flags, updating stats, zeroing handle/size) and returns the zsmalloc handle that needs freeing. This separation has two benefits: 1. It makes the two responsibilities of slot_free() explicit: slot metadata management (must be done under slot lock) vs zsmalloc memory release (can be deferred). 2. It allows zram_slot_free_notify() to use zs_free_deferred() for the handle, deferring the expensive zs_free() to a workqueue so the exit path can release memory faster. While at it, merge three separate clear_slot_flag() calls for ZRAM_IDLE, ZRAM_INCOMPRESSIBLE, and ZRAM_PP_SLOT into a single bitmask operation via clear_slot_flags_on_free(), reducing redundant read-modify-write cycles on the same flags word. All other slot_free() callers (write, discard, meta_free) continue to use synchronous zs_free() through the unchanged slot_free() wrapper. Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wenchao Hao --- drivers/block/zram/zram_drv.c | 37 ++++++++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 9 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index c2afd1c34f4a..382c4dc57c8d 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -165,6 +165,15 @@ static inline bool slot_allocated(struct zram *zram, u= 32 index) test_slot_flag(zram, index, ZRAM_WB); } =20 +#define ZRAM_FLAGS_TO_CLEAR_ON_FREE (BIT(ZRAM_IDLE) | \ + BIT(ZRAM_INCOMPRESSIBLE) | \ + BIT(ZRAM_PP_SLOT)) + +static inline void clear_slot_flags_on_free(struct zram *zram, u32 index) +{ + zram->table[index].attr.flags &=3D ~ZRAM_FLAGS_TO_CLEAR_ON_FREE; +} + static inline void set_slot_comp_priority(struct zram *zram, u32 index, u32 prio) { @@ -2000,17 +2009,20 @@ static bool zram_meta_alloc(struct zram *zram, u64 = disksize) return true; } =20 -static void slot_free(struct zram *zram, u32 index) +/* + * Clear slot metadata and extract the zsmalloc handle for freeing. + * Returns the handle that needs to be freed via zs_free(), or 0 if + * no zsmalloc freeing is needed (e.g. same-filled or writeback slots). + */ +static unsigned long slot_free_extract(struct zram *zram, u32 index) { - unsigned long handle; + unsigned long handle =3D 0; =20 #ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME zram->table[index].attr.ac_time =3D 0; #endif =20 - clear_slot_flag(zram, index, ZRAM_IDLE); - clear_slot_flag(zram, index, ZRAM_INCOMPRESSIBLE); - clear_slot_flag(zram, index, ZRAM_PP_SLOT); + clear_slot_flags_on_free(zram, index); set_slot_comp_priority(zram, index, 0); =20 if (test_slot_flag(zram, index, ZRAM_HUGE)) { @@ -2041,9 +2053,7 @@ static void slot_free(struct zram *zram, u32 index) =20 handle =3D get_slot_handle(zram, index); if (!handle) - return; - - zs_free(zram->mem_pool, handle); + return 0; =20 atomic64_sub(get_slot_size(zram, index), &zram->stats.compr_data_size); @@ -2051,6 +2061,15 @@ static void slot_free(struct zram *zram, u32 index) atomic64_dec(&zram->stats.pages_stored); set_slot_handle(zram, index, 0); set_slot_size(zram, index, 0); + + return handle; +} + +static void slot_free(struct zram *zram, u32 index) +{ + unsigned long handle =3D slot_free_extract(zram, index); + + zs_free(zram->mem_pool, handle); } =20 static int read_same_filled_page(struct zram *zram, struct page *page, @@ -2794,7 +2813,7 @@ static void zram_slot_free_notify(struct block_device= *bdev, return; } =20 - slot_free(zram, index); + zs_free_deferred(zram->mem_pool, slot_free_extract(zram, index)); slot_unlock(zram, index); } =20 --=20 2.34.1 From nobody Wed Jun 17 01:42:35 2026 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3AB9E3BED78 for ; Tue, 21 Apr 2026 12:16:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776773816; cv=none; b=bDvb3ecoI3S4PILfJOSR3mN87wlE3aK0ovTNPNtS9tvNd8v/Af+Y6/nP71w6Z4n9VM9wuc0fsHJZpYGAtZHhqEOvLO3uTTOLoMyCQluiWf6ImruL2HvLdHYWIa1h+JfiOVk3Ex1CQKNT/c1YwwlBTf7/haRETZiEI4jioCeFizw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776773816; c=relaxed/simple; bh=6rKmLIe4zCY6D4i9zpxvV+Dzmm4hxKvdTnkjV+5cCJM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ey+hZCzU8KkQLZBOPBSvUCR5NIQzZEn1nwvafsnmcXYqSZn+L2fTosb8pU/WQlEIOD0CRfX6r5NXXJzyHW1Fv/2rgz82gbG6aOCl1qp9yduoe6Cru3E/Ub+qoFkZANwTT652D9pioWDYl/8fkQrx2du4NKiPbqHUFZl1cN6bajM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=d23g07LQ; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="d23g07LQ" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-82f351ca23cso2027959b3a.2 for ; Tue, 21 Apr 2026 05:16:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1776773809; x=1777378609; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FuBH0tSA4QMEqer5rgkERxzfsnQAmO+99E5ku8qz2ok=; b=d23g07LQm3+TGeN2gKLl4InjSUO0r8W6mtsZA0/7YNWWPW6qGFbwhOGq48J6lhbY/S /mX6bqM5TpmAEicsI7FaYzUIz0xSntEkLhI0AK0BivUbfbcw0HNYAZ+usqO/rI1C/rse zZ6cvOfB3yWkqZPgJeDK4/JmsCbpcZpm+OE65bFPYmp0T9/45tGMAIgwcdiyMYUwnSYM y/ll1Aaen86lj0umRy0wV7a/hANP/6jyJJVVvfb5q/VHOswJ+M04Rk8ZpORMFfKg2TiA AgPpPLUUfmgEGX76mvxf+ZPsd9Q4cwpj0Nr8hqWJyloaIBXPGwgS5wh8q+lNR7H5onjU CLAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776773809; x=1777378609; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=FuBH0tSA4QMEqer5rgkERxzfsnQAmO+99E5ku8qz2ok=; b=BTZUfd2GCa2T1HQdXntdHYr8mHC1wprMwyHDKJ2u3pQUcKdyjGLA/nYP4Vv7q+MkdY jVLmqaDVb52VVTjeXR5GGn5XF8ULqJfuAD3mDN7wOyln56DzFLhboNora/FQZo/PIw9K 24YsBG7HMzX7JbH6c+27CiEQgEJdnH3CEBFBWk1Af4wDddUjBJ6FVn00je7Stkni5nUc C3jumn81BSS/U5RYJsZplMBixjwErIX80YtFnS6JXXrRrTy3aQcj2PGxYh3BxeDrHGyo Gt0MbOgMzPBHLkeDuQyCFBE04TDA532SR0evvUDgIPfGKHYkCh0sTN0aGVugYOHwPFQ0 Ws8A== X-Forwarded-Encrypted: i=1; AFNElJ/dAGQiMlzXyw4SHWjZHNCg33qMtcXbGkdwQQBbOgSkkSJL3RyzntYHRkQLQbKv3iawZ1PF4lDmZ66qd8c=@vger.kernel.org X-Gm-Message-State: AOJu0YwuZkfHNvLBi3vQPtV2aaVz2npkdLxt3dIlJo76dj2e6o8t1l7a xTpVCLtubGmoCUmUPufUccqIXBbe3mA0dX/Qpyk+Wvd/lRiUCYhpxH2B X-Gm-Gg: AeBDiet6yfE0nW6lzv0AXMT7J5L/MkFBBVqZL0Pxx0q7ZEpjKOBmQZ4wIy6DRpJW0fA ffX8GxIjaH6EuPr7ncf9qIZ+S/o1brX+VGs3UE39ikTOYaFdHVrykWRjZUSYJZwbzelWNGfF+Sh 0goSY1birV3ZVddhQNY5mKzDEelGrQoEEyIJ2PdFfF7LFpjbhvbhnEp9bOsiugTQ6ZjUI+s1GcJ zWZ4QW+fUt4kyvZUtre24EsMpSuvjRMnkRVXRAULg7j2LrChXtzjMsVWnhPKVqrUjBDew6KUJJ5 u/9Gdv2giA3B+UxWGX0v/fj2E13IhNO4dXmsdOayH3L2erD/tLhp6GliYSRpZjdqySmro/v1bc8 pvBYYv0Th/WORGyWXaI7ieClUl4Rkrvk3jiXqpOnelTP0dLMnNxvPPZemJwQxM0boEC7viIh583 PHIZj8w4nKRvG1ncq7N8yapvZ4JR7IqTpNVLZSCwRBGUZIz0Q= X-Received: by 2002:aa7:88ca:0:b0:82c:d9d0:f482 with SMTP id d2e1a72fcca58-82f8c976ff1mr18659748b3a.46.1776773809369; Tue, 21 Apr 2026 05:16:49 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-82f8ec037e1sm16371071b3a.54.2026.04.21.05.16.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 05:16:49 -0700 (PDT) From: Wenchao Hao X-Google-Original-From: Wenchao Hao To: Andrew Morton , Chengming Zhou , Jens Axboe , Johannes Weiner , Minchan Kim , Nhat Pham , Sergey Senozhatsky , Yosry Ahmed , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Barry Song , Xueyuan Chen , Wenchao Hao Subject: [RFC PATCH v2 4/4] mm/zswap: defer zs_free() in zswap_invalidate() path Date: Tue, 21 Apr 2026 20:16:16 +0800 Message-Id: <20260421121616.3298845-5-haowenchao@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260421121616.3298845-1-haowenchao@xiaomi.com> References: <20260421121616.3298845-1-haowenchao@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" zswap_invalidate() is called on the same process exit path as zram_slot_free_notify(). The zswap_entry_free() it calls internally performs zs_free() which is expensive due to zsmalloc internal locking. Unlike zram which has a trylock fallback, zswap_invalidate() executes unconditionally, making the latency impact potentially worse. Like zram, the expensive zs_free() here blocks the process exit path, delaying overall memory release. Additionally, zswap_entry_free() performs extra work beyond zs_free(): list_lru_del() (takes its own spinlock), obj_cgroup accounting, and kmem_cache_free for the entry itself. Use zs_free_deferred() in zswap_invalidate() path to defer the expensive zsmalloc handle freeing to a workqueue, allowing the exit path to release memory faster. All other callers (zswap_load, zswap_writeback_entry, zswap_store error paths) run in process context and continue to use synchronous zs_free(). Signed-off-by: Wenchao Hao --- mm/zswap.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 0823cadd02b6..7291f6deb5b6 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -713,11 +713,16 @@ static void zswap_entry_cache_free(struct zswap_entry= *entry) /* * Carries out the common pattern of freeing an entry's zsmalloc allocatio= n, * freeing the entry itself, and decrementing the number of stored pages. + * When @deferred is true, the zsmalloc handle is queued for async freeing + * instead of being freed immediately. */ -static void zswap_entry_free(struct zswap_entry *entry) +static void __zswap_entry_free(struct zswap_entry *entry, bool deferred) { zswap_lru_del(&zswap_list_lru, entry); - zs_free(entry->pool->zs_pool, entry->handle); + if (deferred) + zs_free_deferred(entry->pool->zs_pool, entry->handle); + else + zs_free(entry->pool->zs_pool, entry->handle); zswap_pool_put(entry->pool); if (entry->objcg) { obj_cgroup_uncharge_zswap(entry->objcg, entry->length); @@ -729,6 +734,11 @@ static void zswap_entry_free(struct zswap_entry *entry) atomic_long_dec(&zswap_stored_pages); } =20 +static void zswap_entry_free(struct zswap_entry *entry) +{ + __zswap_entry_free(entry, false); +} + /********************************* * compressed storage functions **********************************/ @@ -1655,7 +1665,7 @@ void zswap_invalidate(swp_entry_t swp) =20 entry =3D xa_erase(tree, offset); if (entry) - zswap_entry_free(entry); + __zswap_entry_free(entry, true); } =20 int zswap_swapon(int type, unsigned long nr_pages) --=20 2.34.1