From nobody Sun Oct 5 18:12:36 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E583C28D8D9 for ; Thu, 31 Jul 2025 06:50:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753944660; cv=none; b=C2sFQ9nboT/1+aEqys8KrGLUI6ph2VNvWJTFt7Qo4yv6tV3vYpgDQ3TYYA2+V4bnQiYhb520PhIDyz/UNenh76tsncwqE7rIrNCLK1vOQkM2r5aKsn1wEcTVnFVGcbdJLHCqhoDsehWS11iYHvXJRaKDQEWGAChC11gYvifoFZs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753944660; c=relaxed/simple; bh=G+UT1S3qj26BV2xnMB/dporrcHS4uOhh/Cg7uwxTjro=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=o/YbZCjrsp3H8ArlWgeOQ45R4bpf2Bz36AFxuStjpbseL1+qlbMsGr0Q+Oa9spOdDIJtqPzjrxIk8p4XcEpeEl+YrAWqN4lzwVabq76dXXHd0xkVvO9kKGGKYJ6a2mFofyhPsiIEMUIHO82Fuc2OywYxQwP+e17dN0xkNPGJ8kA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--richardycc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WJe+p2O4; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--richardycc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WJe+p2O4" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-76bcf2cac36so315162b3a.0 for ; Wed, 30 Jul 2025 23:50:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753944658; x=1754549458; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dGUE3+fkLdFpOKkrsZLlRFaze4JHwy3kWJ1YQNILIOw=; b=WJe+p2O4ETZ2eFhYQwXIBVoXUgd3Smrd/PHICyWg8Gjp/dUgRKxBF3HmWuDpKs+c97 w1KIt+cEmOdaNqOjXpdI+Pk6KzsXVSxJwOMsCGmEyvnrGJx6d4aLN1+3e63ssdR74HIk 5WJgxqxr3z7jD9xNxoiR4A5oZd27iG0O+YQwu+87vwH55L4ynW0H/rWfX+F0i/v6T3II Y/wvVfOVI9NAAe0LbACxGPCzE+3sULJXWVxNp7P1gMhnT/0PNJ41b0ab0cL0kv+Hzd3Z fQ0RobyuCAMbVmg6fGUSD/zX5/zWhsNpMzZMYfuYt8aFit6Zy3Gvrv1L+eL1Rg1IvqEV IGtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753944658; x=1754549458; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dGUE3+fkLdFpOKkrsZLlRFaze4JHwy3kWJ1YQNILIOw=; b=GYpLrI7+z1IpHs9xzjO3RtogTLvq9Dt3vLFd8E8isasK2i9+OqM4cgx2qWBgcbbOC/ VI5c6WjDSCJXQvMq22E2QN7oa9LYcajZTx+wJkkdZDELc2GMFy4P0n7FR2FtV1BWz48N +DQWn6/BDRWJjl3NnWzJg1XwqKYJSobJzc4DXIFHr80kZEvcEJTF7PLjR+tOpZYo62Z+ 2f2gKkTyQe8V/j1lvsc4VY6PrEt0doy8cv8OAdME1Lnd9w/QuH3CibWvEgrq+qrhQvsJ RlU1/giJZn4FiJ26ndkuzrUO8y6Xm/5+H2lwYc9Jmgxy4BEJpjwW0O0Ve2q0yfeuYdgv 9mjg== X-Forwarded-Encrypted: i=1; AJvYcCW5dHpragHDLV0b3mHtydx1Nkq0Io7conFjaOtFpTGSxAt84XZ7//VFagFcWb/L3u/cQsxP6aHCVBTYyjI=@vger.kernel.org X-Gm-Message-State: AOJu0YxEqOvVwH52MXIxsOVBKg47UUUB5vPr0hTX6r9Cfm9eKtlDenkX +V9Dox+QJX0Y1OSiE9Wk0IhRLor6pTpzHChNCI1MgOq7nOAxIQhJXGNeAGg8mnqH5Ddh57Jx3uk gYjyhulD+nuAEh/bRlop8 X-Google-Smtp-Source: AGHT+IEgVBWm/BLKTiVc6BPX/+JCFhdiQ8GmUIXhRNeX1+TmocJCuFg4OY1N6Lq5dY3Gt40NAXUyRmKvEykmSYEi X-Received: from pgah15.prod.google.com ([2002:a05:6a02:4e8f:b0:b42:2a6f:4fcc]) (user=richardycc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:6da1:b0:233:d85e:a69b with SMTP id adf61e73a8af0-23dc0eba156mr9493234637.39.1753944658322; Wed, 30 Jul 2025 23:50:58 -0700 (PDT) Date: Thu, 31 Jul 2025 06:49:47 +0000 In-Reply-To: <20250731064949.1690732-1-richardycc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250731064949.1690732-1-richardycc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250731064949.1690732-2-richardycc@google.com> Subject: [PATCH v2 1/3] zram: refactor writeback helpers From: Richard Chang To: Minchan Kim , Sergey Senozhatsky , Jens Axboe , Andrew Morton Cc: bgeffon@google.com, liumartin@google.com, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, Richard Chang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move writeback-related functions and data structures from zram_drv.[ch] to a new zram_wb.[ch] file. This is a pure refactoring patch with no functional changes, preparing the ground for the upcoming asynchronous writeback implementation. Signed-off-by: Richard Chang --- drivers/block/zram/Makefile | 1 + drivers/block/zram/zram_drv.c | 44 +---------------------------------- drivers/block/zram/zram_drv.h | 19 +++++++++++++++ drivers/block/zram/zram_wb.c | 35 ++++++++++++++++++++++++++++ drivers/block/zram/zram_wb.h | 18 ++++++++++++++ 5 files changed, 74 insertions(+), 43 deletions(-) create mode 100644 drivers/block/zram/zram_wb.c create mode 100644 drivers/block/zram/zram_wb.h diff --git a/drivers/block/zram/Makefile b/drivers/block/zram/Makefile index 0fdefd576691..31ee1ed34e17 100644 --- a/drivers/block/zram/Makefile +++ b/drivers/block/zram/Makefile @@ -8,5 +8,6 @@ zram-$(CONFIG_ZRAM_BACKEND_LZ4HC) +=3D backend_lz4hc.o zram-$(CONFIG_ZRAM_BACKEND_ZSTD) +=3D backend_zstd.o zram-$(CONFIG_ZRAM_BACKEND_DEFLATE) +=3D backend_deflate.o zram-$(CONFIG_ZRAM_BACKEND_842) +=3D backend_842.o +zram-$(CONFIG_ZRAM_WRITEBACK) +=3D zram_wb.o =20 obj-$(CONFIG_ZRAM) +=3D zram.o diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 8acad3cc6e6e..ec8649cad21e 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -36,6 +36,7 @@ #include =20 #include "zram_drv.h" +#include "zram_wb.h" =20 static DEFINE_IDR(zram_index_idr); /* idr index must be protected */ @@ -233,22 +234,6 @@ static void zram_accessed(struct zram *zram, u32 index) } =20 #if defined CONFIG_ZRAM_WRITEBACK || defined CONFIG_ZRAM_MULTI_COMP -struct zram_pp_slot { - unsigned long index; - struct list_head entry; -}; - -/* - * A post-processing bucket is, essentially, a size class, this defines - * the range (in bytes) of pp-slots sizes in particular bucket. - */ -#define PP_BUCKET_SIZE_RANGE 64 -#define NUM_PP_BUCKETS ((PAGE_SIZE / PP_BUCKET_SIZE_RANGE) + 1) - -struct zram_pp_ctl { - struct list_head pp_buckets[NUM_PP_BUCKETS]; -}; - static struct zram_pp_ctl *init_pp_ctl(void) { struct zram_pp_ctl *ctl; @@ -697,31 +682,6 @@ static ssize_t backing_dev_store(struct device *dev, return err; } =20 -static unsigned long alloc_block_bdev(struct zram *zram) -{ - unsigned long blk_idx =3D 1; -retry: - /* skip 0 bit to confuse zram.handle =3D 0 */ - blk_idx =3D find_next_zero_bit(zram->bitmap, zram->nr_pages, blk_idx); - if (blk_idx =3D=3D zram->nr_pages) - return 0; - - if (test_and_set_bit(blk_idx, zram->bitmap)) - goto retry; - - atomic64_inc(&zram->stats.bd_count); - return blk_idx; -} - -static void free_block_bdev(struct zram *zram, unsigned long blk_idx) -{ - int was_set; - - was_set =3D test_and_clear_bit(blk_idx, zram->bitmap); - WARN_ON_ONCE(!was_set); - atomic64_dec(&zram->stats.bd_count); -} - static void read_from_bdev_async(struct zram *zram, struct page *page, unsigned long entry, struct bio *parent) { @@ -1111,8 +1071,6 @@ static int read_from_bdev(struct zram *zram, struct p= age *page, { return -EIO; } - -static void free_block_bdev(struct zram *zram, unsigned long blk_idx) {}; #endif =20 #ifdef CONFIG_ZRAM_MEMORY_TRACKING diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 6cee93f9c0d0..2c7bc05fb186 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -139,4 +139,23 @@ struct zram { #endif atomic_t pp_in_progress; }; + +#if defined CONFIG_ZRAM_WRITEBACK || defined CONFIG_ZRAM_MULTI_COMP +struct zram_pp_slot { + unsigned long index; + struct list_head entry; +}; + +/* + * A post-processing bucket is, essentially, a size class, this defines + * the range (in bytes) of pp-slots sizes in particular bucket. + */ +#define PP_BUCKET_SIZE_RANGE 64 +#define NUM_PP_BUCKETS ((PAGE_SIZE / PP_BUCKET_SIZE_RANGE) + 1) + +struct zram_pp_ctl { + struct list_head pp_buckets[NUM_PP_BUCKETS]; +}; +#endif + #endif diff --git a/drivers/block/zram/zram_wb.c b/drivers/block/zram/zram_wb.c new file mode 100644 index 000000000000..0bc10f725939 --- /dev/null +++ b/drivers/block/zram/zram_wb.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +#define KMSG_COMPONENT "zram_wb" +#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt + +#include +#include + +#include "zram_wb.h" + +unsigned long alloc_block_bdev(struct zram *zram) +{ + unsigned long blk_idx =3D 1; +retry: + /* skip 0 bit to confuse zram.handle =3D 0 */ + blk_idx =3D find_next_zero_bit(zram->bitmap, zram->nr_pages, blk_idx); + if (blk_idx =3D=3D zram->nr_pages) + return 0; + + if (test_and_set_bit(blk_idx, zram->bitmap)) + goto retry; + + atomic64_inc(&zram->stats.bd_count); + return blk_idx; +} + +void free_block_bdev(struct zram *zram, unsigned long blk_idx) +{ + int was_set; + + was_set =3D test_and_clear_bit(blk_idx, zram->bitmap); + WARN_ON_ONCE(!was_set); + atomic64_dec(&zram->stats.bd_count); +} + diff --git a/drivers/block/zram/zram_wb.h b/drivers/block/zram/zram_wb.h new file mode 100644 index 000000000000..c2f5984e7aa2 --- /dev/null +++ b/drivers/block/zram/zram_wb.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef _ZRAM_WRITEBACK_H_ +#define _ZRAM_WRITEBACK_H_ + +#include +#include "zram_drv.h" + +#if IS_ENABLED(CONFIG_ZRAM_WRITEBACK) +unsigned long alloc_block_bdev(struct zram *zram); +void free_block_bdev(struct zram *zram, unsigned long blk_idx); +#else +inline unsigned long alloc_block_bdev(struct zram *zram) { return 0; } +inline void free_block_bdev(struct zram *zram, unsigned long blk_idx) {}; +#endif + +#endif /* _ZRAM_WRITEBACK_H_ */ + --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 18:12:36 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2267528DB54 for ; Thu, 31 Jul 2025 06:51:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753944662; cv=none; b=kQTLalqTBaCQVDVxkJ+rRTSkt5gVM/GTQJ660f5eaR2hDJWfLWTOAfeXEdAaQsCuwuhq9wsNgnq/PlVqR35neESgaDQlU6a7Ylv2y4xCSXx8tiULR4N4Ixqxh7tI7ENR/4aLiMTIPgRH//q3cSbr4Mqn6OsymIKKGpVK2t1cN68= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753944662; c=relaxed/simple; bh=vrxT16m5cp8hnooQKhpGXNSMbSVI+ItCWpisQc7udsI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dQFK0e6lq/JkMLadDtpCevJdvElxWtIkDr+IVFP+LoQqAvU8bFMUbTFjkdBGSu0duJPPbHSGi51Bj4J+Gj2yE0yAeOc945Lecttn3Vu8p8d4ZC0BqOJPU2zGDXKmLNeRfS6rhtXtbAog2LPYJSG4fisfXV5Wi03vt9j32Y9Sn4w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--richardycc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ij4ZtqMC; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--richardycc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ij4ZtqMC" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-74b29ee4f8bso710616b3a.2 for ; Wed, 30 Jul 2025 23:51:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753944660; x=1754549460; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=S355pAkI7NEC7JgrgQtUxkZyhDtDgj0z6R5QsB/4a/A=; b=ij4ZtqMCS2Tu2g5qcHgKq/vZuW5fKz/yPklC3OA+dClZOthBBYHqlX5P5VAvA9+ofv OKZXxiK3NbmeA28dT64fgoT7W979NTNS16QT2u6wyiA0fx4qeCoI+5y/JwZ4Iovp7Kv8 YpY69aZ5Lwnl7KEr1JxyDxfsRqZpfe4BTo1rLh4WCknNG7fmfIUqk0oDwTOmOby78mlL YWegZp2BJXvAtVtpdSkAw+F4GzwYFkl/ZIdQQJesnc/8IVJzdOJ/Ni9i93XEdtkazPMj Y9Ugrf/SbLPp4h/ava9cqlYbIZjtR9NPq+iqr0cQCE/xVwqV6Nmtdx4uLh0ffjZ1bHWD xvoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753944660; x=1754549460; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=S355pAkI7NEC7JgrgQtUxkZyhDtDgj0z6R5QsB/4a/A=; b=qANiECoh6AjTRT2yzKN8hFbYttXKfLeX5mAcm2usRcZ2tBjkhhQptkK/vwIo8eREyn px+qWEcahhKedKxCqBg3rrYvP+WcXelFaHu4dzpQRo7ErC/lqxt5XUjyMHYsD4qJYtRu UNo111ZQzNWTLkQerq//seNgryzX2758Cv9/aH6sByfsndF+0iCL6LVbbhwtsFV/B5lI /n0GnNqUB6xIPUB4BwXUIEl+bcjQY/lDtCMVm6TQLgwrQ0dzUw5/5kvdmxeKZYVQL/Qw YDNPTNUeyDPoaRMrj2wB17Ls6VDSHAFFPwtjd3t3PHAQrEsCysBE7UAeOI/no8tPl05/ sk1Q== X-Forwarded-Encrypted: i=1; AJvYcCV5/qBUs2TAv4y1K+yAFon5jSE/kO9OFC3OIRA/AZDjFo1xrAPFe+jnQnf2tXIec5BHwws2MuekfJ+H83o=@vger.kernel.org X-Gm-Message-State: AOJu0YyyK2uy/rJTStUC/jjE8DQ3iTPU9qkoLskVQEvjm/Ay50ObJ7/n ZezWhyXJT+9+vAvIs9WaAaU5UGkeLBD1m6df4yKBJ+fO24nfRmV7msvXjR2nXyBtwjibMXauuXV DRG6BgCw/H2ZZjipey9Y2 X-Google-Smtp-Source: AGHT+IFMUHJAFt8LKadusLSxWtpDyV1hCcQ6KorKLDjQu0fTBgiS1ZlW3Ro3SPn2MxUk0sYyQq397lMznVYTEcB5 X-Received: from pgah15.prod.google.com ([2002:a05:6a02:4e8f:b0:b42:2a6f:4fcc]) (user=richardycc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:2450:b0:233:951f:a572 with SMTP id adf61e73a8af0-23dc0d58b95mr10119563637.16.1753944660091; Wed, 30 Jul 2025 23:51:00 -0700 (PDT) Date: Thu, 31 Jul 2025 06:49:48 +0000 In-Reply-To: <20250731064949.1690732-1-richardycc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250731064949.1690732-1-richardycc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250731064949.1690732-3-richardycc@google.com> Subject: [PATCH v2 2/3] zram: add async writeback infrastructure From: Richard Chang To: Minchan Kim , Sergey Senozhatsky , Jens Axboe , Andrew Morton Cc: bgeffon@google.com, liumartin@google.com, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, Richard Chang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce the necessary infrastructure for performing writeback operations asynchronously. It adds a dedicated kernel thread (`zram_wb_thread`), a request queue for managing pending writebacks, and helper functions to deal with the writeback requests. This patch lays the foundation for enabling asynchronous writeback in a subsequent change. Signed-off-by: Richard Chang --- drivers/block/zram/zram_drv.c | 4 ++ drivers/block/zram/zram_wb.c | 114 ++++++++++++++++++++++++++++++++++ drivers/block/zram/zram_wb.h | 22 +++++++ 3 files changed, 140 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index ec8649cad21e..6098c0bb773c 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2871,6 +2871,9 @@ static int __init zram_init(void) num_devices--; } =20 + if (setup_zram_writeback()) + goto out_error; + return 0; =20 out_error: @@ -2880,6 +2883,7 @@ static int __init zram_init(void) =20 static void __exit zram_exit(void) { + destroy_zram_writeback(); destroy_devices(); } =20 diff --git a/drivers/block/zram/zram_wb.c b/drivers/block/zram/zram_wb.c index 0bc10f725939..63a16dae5796 100644 --- a/drivers/block/zram/zram_wb.c +++ b/drivers/block/zram/zram_wb.c @@ -5,9 +5,16 @@ =20 #include #include +#include +#include +#include =20 #include "zram_wb.h" =20 +static struct task_struct *wb_thread; +static DECLARE_WAIT_QUEUE_HEAD(wb_wq); +static struct zram_wb_request_list wb_req_list; + unsigned long alloc_block_bdev(struct zram *zram) { unsigned long blk_idx =3D 1; @@ -33,3 +40,110 @@ void free_block_bdev(struct zram *zram, unsigned long b= lk_idx) atomic64_dec(&zram->stats.bd_count); } =20 +static void complete_wb_request(struct zram_wb_request *req) +{ + struct zram *zram =3D req->zram; + unsigned long blk_idx =3D req->blk_idx; + + free_block_bdev(zram, blk_idx); + free_wb_request(req); +} + +void enqueue_wb_request(struct zram_wb_request_list *req_list, + struct zram_wb_request *req) +{ + spin_lock_bh(&req_list->lock); + list_add_tail(&req->node, &req_list->head); + req_list->count++; + spin_unlock_bh(&req_list->lock); +} + +static struct zram_wb_request *dequeue_wb_request( + struct zram_wb_request_list *req_list) +{ + struct zram_wb_request *req =3D NULL; + + spin_lock_bh(&req_list->lock); + if (!list_empty(&req_list->head)) { + req =3D list_first_entry(&req_list->head, + struct zram_wb_request, + node); + list_del(&req->node); + req_list->count--; + } + spin_unlock_bh(&req_list->lock); + + return req; +} + +static void destroy_wb_request_list(struct zram_wb_request_list *req_list) +{ + struct zram_wb_request *req; + + while (!list_empty(&req_list->head)) { + req =3D dequeue_wb_request(req_list); + free_block_bdev(req->zram, req->blk_idx); + free_wb_request(req); + } +} + +static bool wb_ready_to_run(void) +{ + int count; + + spin_lock_bh(&wb_req_list.lock); + count =3D wb_req_list.count; + spin_unlock_bh(&wb_req_list.lock); + + return count > 0; +} + +static int wb_thread_func(void *data) +{ + set_freezable(); + + while (!kthread_should_stop()) { + wait_event_freezable(wb_wq, wb_ready_to_run()); + + while (1) { + struct zram_wb_request *req; + + req =3D dequeue_wb_request(&wb_req_list); + if (!req) + break; + complete_wb_request(req); + } + } + return 0; +} + +void free_wb_request(struct zram_wb_request *req) +{ + struct bio *bio =3D req->bio; + struct page *page =3D bio_first_page_all(bio); + + __free_page(page); + bio_put(bio); + kfree(req); +} + +int setup_zram_writeback(void) +{ + spin_lock_init(&wb_req_list.lock); + INIT_LIST_HEAD(&wb_req_list.head); + wb_req_list.count =3D 0; + + wb_thread =3D kthread_run(wb_thread_func, NULL, "zram_wb_thread"); + if (IS_ERR(wb_thread)) { + pr_err("Unable to create zram_wb_thread\n"); + return -1; + } + return 0; +} + +void destroy_zram_writeback(void) +{ + kthread_stop(wb_thread); + destroy_wb_request_list(&wb_req_list); +} + diff --git a/drivers/block/zram/zram_wb.h b/drivers/block/zram/zram_wb.h index c2f5984e7aa2..b86de0398346 100644 --- a/drivers/block/zram/zram_wb.h +++ b/drivers/block/zram/zram_wb.h @@ -6,12 +6,34 @@ #include #include "zram_drv.h" =20 +struct zram_wb_request { + struct zram *zram; + unsigned long blk_idx; + struct zram_pp_slot *pps; + struct zram_pp_ctl *ppctl; + struct bio *bio; + struct list_head node; +}; + +struct zram_wb_request_list { + struct list_head head; + int count; + spinlock_t lock; +}; + #if IS_ENABLED(CONFIG_ZRAM_WRITEBACK) unsigned long alloc_block_bdev(struct zram *zram); void free_block_bdev(struct zram *zram, unsigned long blk_idx); +int setup_zram_writeback(void); +void destroy_zram_writeback(void); +void free_wb_request(struct zram_wb_request *req); +void enqueue_wb_request(struct zram_wb_request_list *req_list, + struct zram_wb_request *req); #else inline unsigned long alloc_block_bdev(struct zram *zram) { return 0; } inline void free_block_bdev(struct zram *zram, unsigned long blk_idx) {}; +inline int setup_zram_writeback(void) { return 0; } +inline void destroy_zram_writeback(void) {} #endif =20 #endif /* _ZRAM_WRITEBACK_H_ */ --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 18:12:36 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A1C928DB77 for ; Thu, 31 Jul 2025 06:51:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753944664; cv=none; b=qAKUCSb724y93MO4IV/xA53B+wSFtHiRQUc4NfYSFl3tZMxFyMiF7FXP6P07kkFd3+f8tpa8K2HTayIahdNeV32AEhTuhl7yP9LgITj8XOxdSdXlaKZT+gDM6EY66jrQY5Lrby+hIkSseCC9JYraI5ZuafiO8/trLT1wKEqLYlE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753944664; c=relaxed/simple; bh=x5WHeabDW4C0/9wkwfhqLtOyfNkukvgRdDHl7/2SHkk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Fk+CdM17/AMMq78msbEZASkEws8ea15lZC2fduRWit6JsVV8RTjhRHVLQsygqvGbR653WDQ64P5xr14Bz3l6D7wcD4BCUAcrViFOQnM7BCCM1zi54aGZa7b+75L9Y4iPIDKRDiUHsF0GfH8Vyy2iowpQjwjixspoHYW6sCgjT7k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--richardycc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=S3qzKEPW; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--richardycc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="S3qzKEPW" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b3bde4d4d5cso461151a12.2 for ; Wed, 30 Jul 2025 23:51:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753944662; x=1754549462; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=J9HBBhNz4JX/02iQNqHxMkOYIhfKJVrRepRJjtX/gUI=; b=S3qzKEPWHwrBoziGB55W2voZZJxWW1Z8KPh/jyK6xSq3AfsVIgDQp0lopqJXYUZgfm w4t1/4CZqKX/WASpiRegjLI1lGoLE/u98Jw2Z8dhyKtTJYYiFmxZ5Do3QT2HaOgItShZ cLETZGAEm1wGdpgspqwN8k7mrl22QX7zRML0hdg17riF1HtN6dOhlCWL/rcGUpaLEZCr XUM+sULGaeSe0HZe/2eIv4kn1SoHpnfirWoGTnKK9GpIu++1jYuVbagr9BScx31gy7qi EJiwDFzYXE8eEKTsJKOcaUpsGy4r+3sSVmoE8FHfn2EwTyyLNUDWLGwEpky35UjD50/5 0Plg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753944662; x=1754549462; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=J9HBBhNz4JX/02iQNqHxMkOYIhfKJVrRepRJjtX/gUI=; b=ecgDrUfrK1pe3ET/QJ/H145LJz03lu/s57boQfKSwAaortShus/h+sqY66r1qu+YIs ++RwHDU1IIeDKdeG1KdS13kdGPAORYqSndZ5aLmqqlUbjmU7UiFjtbeceXl/D4Ez49bS mzrrUQlc1rxNlX6WtZ/MCbv5T1XB2zxsRk4sHxA+8QhswEmCbmVVRJoyQF9MH5q8h7jD Vz94hjTc6jErI2i0r6rRTsKqyHS930zy1xO/khiCuLKfiOx+mBnmLkPlQ1L4YZqy6woU lmwcVkNuwcw5qRcGF017LsLOiet+53mwPvF5MGh2nvZY0hbiy/ga4wgJ6b2WT5Fjl78s jBRA== X-Forwarded-Encrypted: i=1; AJvYcCW6NC55wzdk4GC1AM1riR4aNN7u1wFM4Rlz5BQ9HkJkErYS2DQ6Goq0C6xQAZ80c+QPwGPo5yGZ+bQtmR0=@vger.kernel.org X-Gm-Message-State: AOJu0YzJZdwA75ZcYgJQ4NLfnsQT916Eed/458CcuTHlePqol69MK2SS HU/Tw6ybF0AkcdRgyRFUTnddZ37aqBIUZuYyi4dvVlyp+YWPE1XsEVcT7s+jEUpPNmg4h9MejlA pEJQmerLUyEJaKLAu6In8 X-Google-Smtp-Source: AGHT+IHwD0DQee4gELDZbn1uwpxGmpeIotkHWXAUCxbVBgG5s7/AREeRYCR5fiI/9tCEtbg+9OUAskPwVsXk7or5 X-Received: from pgbfq19.prod.google.com ([2002:a05:6a02:2993:b0:b1f:fd39:8314]) (user=richardycc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:4322:b0:231:e2e:95e4 with SMTP id adf61e73a8af0-23dc0d57f48mr10285018637.16.1753944661925; Wed, 30 Jul 2025 23:51:01 -0700 (PDT) Date: Thu, 31 Jul 2025 06:49:49 +0000 In-Reply-To: <20250731064949.1690732-1-richardycc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250731064949.1690732-1-richardycc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250731064949.1690732-4-richardycc@google.com> Subject: [PATCH v2 3/3] zram: enable asynchronous writeback From: Richard Chang To: Minchan Kim , Sergey Senozhatsky , Jens Axboe , Andrew Morton Cc: bgeffon@google.com, liumartin@google.com, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, Richard Chang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace the synchronous `submit_bio_wait()` with the non-blocking `submit_bio()` to perform writeback operations asynchronously. This leverages the infrastructure added in the previous commit to handle the completion of I/O operations in a dedicated kthread. This change shows 27% speed improvement of idle writeback on Android platform. Test Flow: - mkfs on the zram device, mount it - cp three linux-6.16-rc2.tar.gz tarball files as the data - do idle writeback - check bd_stat writes 185072 pages Test Result: idle writeback for 185072 4k-pages (~723 MiB) $ echo all > /sys/block/zram0/idle $ time echo idle > /sys/block/zram0/writeback Async writeback: 0m02.49s real 0m00.00s user 0m01.19s system 0m02.32s real 0m00.00s user 0m00.89s system 0m02.35s real 0m00.00s user 0m00.93s system 0m02.29s real 0m00.00s user 0m00.88s system Sync writeback: 0m03.09s real 0m00.00s user 0m01.07s system 0m03.18s real 0m00.00s user 0m01.12s system 0m03.47s real 0m00.00s user 0m01.16s system 0m03.36s real 0m00.00s user 0m01.27s system Signed-off-by: Richard Chang --- drivers/block/zram/zram_drv.c | 109 ++++++++++++++-------------------- drivers/block/zram/zram_drv.h | 11 ++++ drivers/block/zram/zram_wb.c | 103 +++++++++++++++++++++++++++++++- drivers/block/zram/zram_wb.h | 6 +- 4 files changed, 161 insertions(+), 68 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 6098c0bb773c..f459d889fa44 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -57,7 +57,6 @@ static size_t huge_class_size; =20 static const struct block_device_operations zram_devops; =20 -static void zram_free_page(struct zram *zram, size_t index); static int zram_read_from_zspool(struct zram *zram, struct page *page, u32 index); =20 @@ -96,7 +95,7 @@ static __must_check bool zram_slot_trylock(struct zram *z= ram, u32 index) return false; } =20 -static void zram_slot_lock(struct zram *zram, u32 index) +void zram_slot_lock(struct zram *zram, u32 index) { unsigned long *lock =3D &zram->table[index].flags; =20 @@ -105,7 +104,7 @@ static void zram_slot_lock(struct zram *zram, u32 index) lock_acquired(slot_dep_map(zram, index), _RET_IP_); } =20 -static void zram_slot_unlock(struct zram *zram, u32 index) +void zram_slot_unlock(struct zram *zram, u32 index) { unsigned long *lock =3D &zram->table[index].flags; =20 @@ -128,19 +127,17 @@ static unsigned long zram_get_handle(struct zram *zra= m, u32 index) return zram->table[index].handle; } =20 -static void zram_set_handle(struct zram *zram, u32 index, unsigned long ha= ndle) +void zram_set_handle(struct zram *zram, u32 index, unsigned long handle) { zram->table[index].handle =3D handle; } =20 -static bool zram_test_flag(struct zram *zram, u32 index, - enum zram_pageflags flag) +bool zram_test_flag(struct zram *zram, u32 index, enum zram_pageflags flag) { return zram->table[index].flags & BIT(flag); } =20 -static void zram_set_flag(struct zram *zram, u32 index, - enum zram_pageflags flag) +void zram_set_flag(struct zram *zram, u32 index, enum zram_pageflags flag) { zram->table[index].flags |=3D BIT(flag); } @@ -243,15 +240,20 @@ static struct zram_pp_ctl *init_pp_ctl(void) if (!ctl) return NULL; =20 + init_completion(&ctl->all_done); + atomic_set(&ctl->num_pp_slots, 0); for (idx =3D 0; idx < NUM_PP_BUCKETS; idx++) INIT_LIST_HEAD(&ctl->pp_buckets[idx]); return ctl; } =20 -static void release_pp_slot(struct zram *zram, struct zram_pp_slot *pps) +static void remove_pp_slot_from_ctl(struct zram_pp_slot *pps) { list_del_init(&pps->entry); +} =20 +void free_pp_slot(struct zram *zram, struct zram_pp_slot *pps) +{ zram_slot_lock(zram, pps->index); zram_clear_flag(zram, pps->index, ZRAM_PP_SLOT); zram_slot_unlock(zram, pps->index); @@ -259,6 +261,12 @@ static void release_pp_slot(struct zram *zram, struct = zram_pp_slot *pps) kfree(pps); } =20 +static void release_pp_slot(struct zram *zram, struct zram_pp_slot *pps) +{ + remove_pp_slot_from_ctl(pps); + free_pp_slot(zram, pps); +} + static void release_pp_ctl(struct zram *zram, struct zram_pp_ctl *ctl) { u32 idx; @@ -297,6 +305,7 @@ static bool place_pp_slot(struct zram *zram, struct zra= m_pp_ctl *ctl, list_add(&pps->entry, &ctl->pp_buckets[bid]); =20 zram_set_flag(zram, pps->index, ZRAM_PP_SLOT); + atomic_inc(&ctl->num_pp_slots); return true; } =20 @@ -697,18 +706,18 @@ static void read_from_bdev_async(struct zram *zram, s= truct page *page, static int zram_writeback_slots(struct zram *zram, struct zram_pp_ctl *ctl) { unsigned long blk_idx =3D 0; - struct page *page =3D NULL; struct zram_pp_slot *pps; - struct bio_vec bio_vec; - struct bio bio; - int ret =3D 0, err; + int ret =3D 0; u32 index; + int nr_pps =3D atomic_read(&ctl->num_pp_slots); =20 - page =3D alloc_page(GFP_KERNEL); - if (!page) - return -ENOMEM; + if (!nr_pps) + return 0; =20 while ((pps =3D select_pp_slot(ctl))) { + struct zram_wb_request *req; + struct page *page; + spin_lock(&zram->wb_limit_lock); if (zram->wb_limit_enable && !zram->bd_wb_limit) { spin_unlock(&zram->wb_limit_lock); @@ -725,6 +734,13 @@ static int zram_writeback_slots(struct zram *zram, str= uct zram_pp_ctl *ctl) } } =20 + req =3D alloc_wb_request(zram, pps, ctl, blk_idx); + if (IS_ERR(req)) { + ret =3D PTR_ERR(req); + break; + } + page =3D bio_first_page_all(req->bio); + index =3D pps->index; zram_slot_lock(zram, index); /* @@ -739,63 +755,28 @@ static int zram_writeback_slots(struct zram *zram, st= ruct zram_pp_ctl *ctl) goto next; zram_slot_unlock(zram, index); =20 - bio_init(&bio, zram->bdev, &bio_vec, 1, - REQ_OP_WRITE | REQ_SYNC); - bio.bi_iter.bi_sector =3D blk_idx * (PAGE_SIZE >> 9); - __bio_add_page(&bio, page, PAGE_SIZE, 0); - - /* - * XXX: A single page IO would be inefficient for write - * but it would be not bad as starter. - */ - err =3D submit_bio_wait(&bio); - if (err) { - release_pp_slot(zram, pps); - /* - * BIO errors are not fatal, we continue and simply - * attempt to writeback the remaining objects (pages). - * At the same time we need to signal user-space that - * some writes (at least one, but also could be all of - * them) were not successful and we do so by returning - * the most recent BIO error. - */ - ret =3D err; - continue; - } - - atomic64_inc(&zram->stats.bd_writes); - zram_slot_lock(zram, index); - /* - * Same as above, we release slot lock during writeback so - * slot can change under us: slot_free() or slot_free() and - * reallocation (zram_write_page()). In both cases slot loses - * ZRAM_PP_SLOT flag. No concurrent post-processing can set - * ZRAM_PP_SLOT on such slots until current post-processing - * finishes. - */ - if (!zram_test_flag(zram, index, ZRAM_PP_SLOT)) - goto next; - - zram_free_page(zram, index); - zram_set_flag(zram, index, ZRAM_WB); - zram_set_handle(zram, index, blk_idx); + nr_pps--; + remove_pp_slot_from_ctl(pps); blk_idx =3D 0; - atomic64_inc(&zram->stats.pages_stored); - spin_lock(&zram->wb_limit_lock); - if (zram->wb_limit_enable && zram->bd_wb_limit > 0) - zram->bd_wb_limit -=3D 1UL << (PAGE_SHIFT - 12); - spin_unlock(&zram->wb_limit_lock); + submit_bio(req->bio); + continue; + next: zram_slot_unlock(zram, index); release_pp_slot(zram, pps); + free_wb_request(req); =20 cond_resched(); } =20 if (blk_idx) free_block_bdev(zram, blk_idx); - if (page) - __free_page(page); + + if (nr_pps && atomic_sub_and_test(nr_pps, &ctl->num_pp_slots)) + complete(&ctl->all_done); + + /* wait until all async bios completed */ + wait_for_completion(&ctl->all_done); =20 return ret; } @@ -1579,7 +1560,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 di= sksize) return true; } =20 -static void zram_free_page(struct zram *zram, size_t index) +void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; =20 diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 2c7bc05fb186..ab5ceecc8ed8 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -140,6 +140,13 @@ struct zram { atomic_t pp_in_progress; }; =20 +void zram_slot_lock(struct zram *zram, u32 index); +void zram_slot_unlock(struct zram *zram, u32 index); +void zram_set_handle(struct zram *zram, u32 index, unsigned long handle); +bool zram_test_flag(struct zram *zram, u32 index, enum zram_pageflags flag= ); +void zram_set_flag(struct zram *zram, u32 index, enum zram_pageflags flag); +void zram_free_page(struct zram *zram, size_t index); + #if defined CONFIG_ZRAM_WRITEBACK || defined CONFIG_ZRAM_MULTI_COMP struct zram_pp_slot { unsigned long index; @@ -155,7 +162,11 @@ struct zram_pp_slot { =20 struct zram_pp_ctl { struct list_head pp_buckets[NUM_PP_BUCKETS]; + struct completion all_done; + atomic_t num_pp_slots; }; + +void free_pp_slot(struct zram *zram, struct zram_pp_slot *pps); #endif =20 #endif diff --git a/drivers/block/zram/zram_wb.c b/drivers/block/zram/zram_wb.c index 63a16dae5796..3b8edd048dcb 100644 --- a/drivers/block/zram/zram_wb.c +++ b/drivers/block/zram/zram_wb.c @@ -43,15 +43,60 @@ void free_block_bdev(struct zram *zram, unsigned long b= lk_idx) static void complete_wb_request(struct zram_wb_request *req) { struct zram *zram =3D req->zram; + struct zram_pp_slot *pps =3D req->pps; + struct zram_pp_ctl *ctl =3D req->ppctl; + unsigned long index =3D pps->index; unsigned long blk_idx =3D req->blk_idx; + struct bio *bio =3D req->bio; + + if (bio->bi_status) + goto out_err; + + atomic64_inc(&zram->stats.bd_writes); + zram_slot_lock(zram, index); + + /* + * We release slot lock during writeback so slot can change + * under us: slot_free() or slot_free() and reallocation + * (zram_write_page()). In both cases slot loses + * ZRAM_PP_SLOT flag. No concurrent post-processing can set + * ZRAM_PP_SLOT on such slots until current post-processing + * finishes. + */ + if (!zram_test_flag(zram, index, ZRAM_PP_SLOT)) { + zram_slot_unlock(zram, index); + goto out_err; + } =20 + zram_free_page(zram, index); + zram_set_flag(zram, index, ZRAM_WB); + zram_set_handle(zram, index, blk_idx); + atomic64_inc(&zram->stats.pages_stored); + spin_lock(&zram->wb_limit_lock); + if (zram->wb_limit_enable && zram->bd_wb_limit > 0) + zram->bd_wb_limit -=3D 1UL << (PAGE_SHIFT - 12); + spin_unlock(&zram->wb_limit_lock); + zram_slot_unlock(zram, index); + goto end; + +out_err: free_block_bdev(zram, blk_idx); +end: + free_pp_slot(zram, pps); free_wb_request(req); + + if (atomic_dec_and_test(&ctl->num_pp_slots)) + complete(&ctl->all_done); } =20 -void enqueue_wb_request(struct zram_wb_request_list *req_list, - struct zram_wb_request *req) +static void enqueue_wb_request(struct zram_wb_request_list *req_list, + struct zram_wb_request *req) { + /* + * The enqueue path comes from softirq context: + * blk_done_softirq -> bio_endio -> zram_writeback_end_io + * Use spin_lock_bh for locking. + */ spin_lock_bh(&req_list->lock); list_add_tail(&req->node, &req_list->head); req_list->count++; @@ -83,6 +128,7 @@ static void destroy_wb_request_list(struct zram_wb_reque= st_list *req_list) while (!list_empty(&req_list->head)) { req =3D dequeue_wb_request(req_list); free_block_bdev(req->zram, req->blk_idx); + free_pp_slot(req->zram, req->pps); free_wb_request(req); } } @@ -117,6 +163,59 @@ static int wb_thread_func(void *data) return 0; } =20 +static void zram_writeback_end_io(struct bio *bio) +{ + struct zram_wb_request *req =3D + (struct zram_wb_request *)bio->bi_private; + + enqueue_wb_request(&wb_req_list, req); + wake_up(&wb_wq); +} + +struct zram_wb_request *alloc_wb_request(struct zram *zram, + struct zram_pp_slot *pps, + struct zram_pp_ctl *ppctl, + unsigned long blk_idx) +{ + struct zram_wb_request *req; + struct page *page; + struct bio *bio; + int err =3D 0; + + page =3D alloc_page(GFP_NOIO | __GFP_NOWARN); + if (!page) + return ERR_PTR(-ENOMEM); + + bio =3D bio_alloc(zram->bdev, 1, REQ_OP_WRITE, GFP_NOIO | __GFP_NOWARN); + if (!bio) { + err =3D -ENOMEM; + goto out_free_page; + } + + req =3D kmalloc(sizeof(struct zram_wb_request), GFP_NOIO | __GFP_NOWARN); + if (!req) { + err =3D -ENOMEM; + goto out_free_bio; + } + req->zram =3D zram; + req->pps =3D pps; + req->ppctl =3D ppctl; + req->blk_idx =3D blk_idx; + req->bio =3D bio; + + bio->bi_iter.bi_sector =3D blk_idx * (PAGE_SIZE >> 9); + __bio_add_page(bio, page, PAGE_SIZE, 0); + bio->bi_private =3D req; + bio->bi_end_io =3D zram_writeback_end_io; + return req; + +out_free_bio: + bio_put(bio); +out_free_page: + __free_page(page); + return ERR_PTR(err); +} + void free_wb_request(struct zram_wb_request *req) { struct bio *bio =3D req->bio; diff --git a/drivers/block/zram/zram_wb.h b/drivers/block/zram/zram_wb.h index b86de0398346..f6fe7c51d496 100644 --- a/drivers/block/zram/zram_wb.h +++ b/drivers/block/zram/zram_wb.h @@ -26,9 +26,11 @@ unsigned long alloc_block_bdev(struct zram *zram); void free_block_bdev(struct zram *zram, unsigned long blk_idx); int setup_zram_writeback(void); void destroy_zram_writeback(void); +struct zram_wb_request *alloc_wb_request(struct zram *zram, + struct zram_pp_slot *pps, + struct zram_pp_ctl *ppctl, + unsigned long blk_idx); void free_wb_request(struct zram_wb_request *req); -void enqueue_wb_request(struct zram_wb_request_list *req_list, - struct zram_wb_request *req); #else inline unsigned long alloc_block_bdev(struct zram *zram) { return 0; } inline void free_block_bdev(struct zram *zram, unsigned long blk_idx) {}; --=20 2.50.1.565.gc32cd1483b-goog