From nobody Tue Apr 7 15:56:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C5B2C43219 for ; Tue, 18 Oct 2022 04:56:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229824AbiJREz7 (ORCPT ); Tue, 18 Oct 2022 00:55:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229554AbiJREzq (ORCPT ); Tue, 18 Oct 2022 00:55:46 -0400 Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBA2C95E61 for ; Mon, 17 Oct 2022 21:55:44 -0700 (PDT) Received: by mail-pl1-x62d.google.com with SMTP id f23so12748629plr.6 for ; Mon, 17 Oct 2022 21:55:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jrgty8apOrCiteWZXp6cJje3TbByVDrSP3kfA6Yguxs=; b=A6PtYP4e6nUYFbJ4+qL+YXYG9Y0rfnMuZPCpE8XkO0YwqAWChGhcGU4CmsYC9wXhXj qpDpEPhPy9anNRnlSo+n6LTO/jY0HDT26iB9XeTUJNMvOjmmd6watayZOx+r8Hbzh6Rf cYCxCOaQ31nZhNHAPdoFWY0ii+GqoLGKJW4jQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jrgty8apOrCiteWZXp6cJje3TbByVDrSP3kfA6Yguxs=; b=jpNqc/focSR5XaeL0yk/g90Bsr/qT9X9NFVY+BRqB86+GjSEElYy1q8GoKeG/VbdJU okUoJ2uhVODvZr/0aLNlwgt1jbkLu5k9TEzXuM5wS4RDPMEEzF9QZqwqt188+b9+jrJJ NaWCjDSSRtUTz/1uJpDFdYiFbcYbiRK+Su9EMoQuQPzFcA38u/Y9NTr7R+L5SdyDFU6w o5zj6+5UjQeQESzuN/ikhZPycMponT3N3PVcMYb6k/n8g+PpyhRiHJTSfVklSZqr7Xnr vnh5lDQqTaTSVdUoCHUQ/AuFmg4vB2mcjJgdC+jNb/3MxSO/7yjxEOn+iXCfa+n0xyzN IFXA== X-Gm-Message-State: ACrzQf1D5SX7ho/eKnskq8yc0xBUXRu/ut24JMz2g8Bv2/eTMhqZJEak NThGHf53lvOYdpsOhZwl1q47Li7uFpqKyw== X-Google-Smtp-Source: AMsMyM66inxn0epWk6ZT/hKerMGY//31WjBrk8xQugG/Yr9rmeuT1n1bS16U1TfGgwCFbH0pqO6HYw== X-Received: by 2002:a17:902:da86:b0:183:e2a9:63e6 with SMTP id j6-20020a170902da8600b00183e2a963e6mr1199250plx.105.1666068944372; Mon, 17 Oct 2022 21:55:44 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:17a9:73b0:c262:eccd]) by smtp.gmail.com with ESMTPSA id p4-20020a170902e74400b0017b69f99321sm7549220plf.219.2022.10.17.21.55.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 21:55:44 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 1/9] zram: Preparation for multi-zcomp support Date: Tue, 18 Oct 2022 13:55:25 +0900 Message-Id: <20221018045533.2396670-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog In-Reply-To: <20221018045533.2396670-1-senozhatsky@chromium.org> References: <20221018045533.2396670-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The patch turns compression streams and compressor algorithm name struct zram members into arrays, so that we can have multiple compression streams support (in the next patches). The patch uses a rather explicit API for compressor selection: - Get primary (default) compression stream zcomp_stream_get(zram->comps[ZRAM_PRIMARY_ZCOMP]) - Get secondary compression stream zcomp_stream_get(zram->comps[ZRAM_SECONDARY_ZCOMP]) We use similar API for compression streams put(). At this point we always have just one compression stream, since CONFIG_ZRAM_MULTI_COMP is not yet defined. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 6 +-- drivers/block/zram/zcomp.h | 2 +- drivers/block/zram/zram_drv.c | 87 ++++++++++++++++++++++++----------- drivers/block/zram/zram_drv.h | 14 +++++- 4 files changed, 77 insertions(+), 32 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index 0916de952e09..55af4efd7983 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -206,7 +206,7 @@ void zcomp_destroy(struct zcomp *comp) * case of allocation error, or any other error potentially * returned by zcomp_init(). */ -struct zcomp *zcomp_create(const char *compress) +struct zcomp *zcomp_create(const char *alg) { struct zcomp *comp; int error; @@ -216,14 +216,14 @@ struct zcomp *zcomp_create(const char *compress) * is not loaded yet. We must do it here, otherwise we are about to * call /sbin/modprobe under CPU hot-plug lock. */ - if (!zcomp_available_algorithm(compress)) + if (!zcomp_available_algorithm(alg)) return ERR_PTR(-EINVAL); =20 comp =3D kzalloc(sizeof(struct zcomp), GFP_KERNEL); if (!comp) return ERR_PTR(-ENOMEM); =20 - comp->name =3D compress; + comp->name =3D alg; error =3D zcomp_init(comp); if (error) { kfree(comp); diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h index 40f6420f4b2e..cdefdef93da8 100644 --- a/drivers/block/zram/zcomp.h +++ b/drivers/block/zram/zcomp.h @@ -27,7 +27,7 @@ int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *n= ode); ssize_t zcomp_available_show(const char *comp, char *buf); bool zcomp_available_algorithm(const char *comp); =20 -struct zcomp *zcomp_create(const char *comp); +struct zcomp *zcomp_create(const char *alg); void zcomp_destroy(struct zcomp *comp); =20 struct zcomp_strm *zcomp_stream_get(struct zcomp *comp); diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 966aab902d19..770ea3489eb6 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1007,36 +1007,53 @@ static ssize_t comp_algorithm_show(struct device *d= ev, struct zram *zram =3D dev_to_zram(dev); =20 down_read(&zram->init_lock); - sz =3D zcomp_available_show(zram->compressor, buf); + sz =3D zcomp_available_show(zram->comp_algs[ZRAM_PRIMARY_ZCOMP], buf); up_read(&zram->init_lock); =20 return sz; } =20 +static void comp_algorithm_set(struct zram *zram, u32 idx, const char *alg) +{ + /* Do not kfree() algs that we didn't allocate, IOW the default ones */ + if (zram->comp_algs[idx] !=3D default_compressor) + kfree(zram->comp_algs[idx]); + zram->comp_algs[idx] =3D alg; +} + static ssize_t comp_algorithm_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { struct zram *zram =3D dev_to_zram(dev); - char compressor[ARRAY_SIZE(zram->compressor)]; + char *compressor; size_t sz; =20 - strscpy(compressor, buf, sizeof(compressor)); + sz =3D strlen(buf); + if (sz >=3D CRYPTO_MAX_ALG_NAME) + return -E2BIG; + + compressor =3D kstrdup(buf, GFP_KERNEL); + if (!compressor) + return -ENOMEM; + /* ignore trailing newline */ - sz =3D strlen(compressor); if (sz > 0 && compressor[sz - 1] =3D=3D '\n') compressor[sz - 1] =3D 0x00; =20 - if (!zcomp_available_algorithm(compressor)) + if (!zcomp_available_algorithm(compressor)) { + kfree(compressor); return -EINVAL; + } =20 down_write(&zram->init_lock); if (init_done(zram)) { up_write(&zram->init_lock); + kfree(compressor); pr_info("Can't change algorithm for initialized device\n"); return -EBUSY; } =20 - strcpy(zram->compressor, compressor); + comp_algorithm_set(zram, ZRAM_PRIMARY_ZCOMP, compressor); up_write(&zram->init_lock); return len; } @@ -1284,7 +1301,7 @@ static int __zram_bvec_read(struct zram *zram, struct= page *page, u32 index, size =3D zram_get_obj_size(zram, index); =20 if (size !=3D PAGE_SIZE) - zstrm =3D zcomp_stream_get(zram->comp); + zstrm =3D zcomp_stream_get(zram->comps[ZRAM_PRIMARY_ZCOMP]); =20 src =3D zs_map_object(zram->mem_pool, handle, ZS_MM_RO); if (size =3D=3D PAGE_SIZE) { @@ -1296,7 +1313,7 @@ static int __zram_bvec_read(struct zram *zram, struct= page *page, u32 index, dst =3D kmap_atomic(page); ret =3D zcomp_decompress(zstrm, src, size, dst); kunmap_atomic(dst); - zcomp_stream_put(zram->comp); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_ZCOMP]); } zs_unmap_object(zram->mem_pool, handle); zram_slot_unlock(zram, index); @@ -1363,13 +1380,13 @@ static int __zram_bvec_write(struct zram *zram, str= uct bio_vec *bvec, kunmap_atomic(mem); =20 compress_again: - zstrm =3D zcomp_stream_get(zram->comp); + zstrm =3D zcomp_stream_get(zram->comps[ZRAM_PRIMARY_ZCOMP]); src =3D kmap_atomic(page); ret =3D zcomp_compress(zstrm, src, &comp_len); kunmap_atomic(src); =20 if (unlikely(ret)) { - zcomp_stream_put(zram->comp); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_ZCOMP]); pr_err("Compression failed! err=3D%d\n", ret); zs_free(zram->mem_pool, handle); return ret; @@ -1397,7 +1414,7 @@ static int __zram_bvec_write(struct zram *zram, struc= t bio_vec *bvec, __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR((void *)handle)) { - zcomp_stream_put(zram->comp); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_ZCOMP]); atomic64_inc(&zram->stats.writestall); handle =3D zs_malloc(zram->mem_pool, comp_len, GFP_NOIO | __GFP_HIGHMEM | @@ -1414,14 +1431,14 @@ static int __zram_bvec_write(struct zram *zram, str= uct bio_vec *bvec, * zstrm buffer back. It is necessary that the dereferencing * of the zstrm variable below occurs correctly. */ - zstrm =3D zcomp_stream_get(zram->comp); + zstrm =3D zcomp_stream_get(zram->comps[ZRAM_PRIMARY_ZCOMP]); } =20 alloced_pages =3D zs_get_total_pages(zram->mem_pool); update_used_max(zram, alloced_pages); =20 if (zram->limit_pages && alloced_pages > zram->limit_pages) { - zcomp_stream_put(zram->comp); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_ZCOMP]); zs_free(zram->mem_pool, handle); return -ENOMEM; } @@ -1435,7 +1452,7 @@ static int __zram_bvec_write(struct zram *zram, struc= t bio_vec *bvec, if (comp_len =3D=3D PAGE_SIZE) kunmap_atomic(src); =20 - zcomp_stream_put(zram->comp); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_ZCOMP]); zs_unmap_object(zram->mem_pool, handle); atomic64_add(comp_len, &zram->stats.compr_data_size); out: @@ -1710,6 +1727,20 @@ static int zram_rw_page(struct block_device *bdev, s= ector_t sector, return ret; } =20 +static void zram_destroy_comps(struct zram *zram) +{ + u32 idx; + + for (idx =3D 0; idx < ZRAM_MAX_ZCOMPS; idx++) { + struct zcomp *comp =3D zram->comps[idx]; + + zram->comps[idx] =3D NULL; + if (IS_ERR_OR_NULL(comp)) + continue; + zcomp_destroy(comp); + } +} + static void zram_reset_device(struct zram *zram) { down_write(&zram->init_lock); @@ -1727,11 +1758,11 @@ static void zram_reset_device(struct zram *zram) /* I/O operation under all of CPU are done so let's free */ zram_meta_free(zram, zram->disksize); zram->disksize =3D 0; + zram_destroy_comps(zram); memset(&zram->stats, 0, sizeof(zram->stats)); - zcomp_destroy(zram->comp); - zram->comp =3D NULL; reset_bdev(zram); =20 + comp_algorithm_set(zram, ZRAM_PRIMARY_ZCOMP, default_compressor); up_write(&zram->init_lock); } =20 @@ -1742,6 +1773,7 @@ static ssize_t disksize_store(struct device *dev, struct zcomp *comp; struct zram *zram =3D dev_to_zram(dev); int err; + u32 idx; =20 disksize =3D memparse(buf, NULL); if (!disksize) @@ -1760,22 +1792,25 @@ static ssize_t disksize_store(struct device *dev, goto out_unlock; } =20 - comp =3D zcomp_create(zram->compressor); - if (IS_ERR(comp)) { - pr_err("Cannot initialise %s compressing backend\n", - zram->compressor); - err =3D PTR_ERR(comp); - goto out_free_meta; - } + for (idx =3D 0; idx < ZRAM_MAX_ZCOMPS; idx++) { + comp =3D zcomp_create(zram->comp_algs[idx]); + if (IS_ERR(comp)) { + pr_err("Cannot initialise %s compressing backend\n", + zram->comp_algs[idx]); + err =3D PTR_ERR(comp); + goto out_free_comps; + } =20 - zram->comp =3D comp; + zram->comps[idx] =3D comp; + } zram->disksize =3D disksize; set_capacity_and_notify(zram->disk, zram->disksize >> SECTOR_SHIFT); up_write(&zram->init_lock); =20 return len; =20 -out_free_meta: +out_free_comps: + zram_destroy_comps(zram); zram_meta_free(zram, disksize); out_unlock: up_write(&zram->init_lock); @@ -1962,7 +1997,7 @@ static int zram_add(void) if (ret) goto out_cleanup_disk; =20 - strscpy(zram->compressor, default_compressor, sizeof(zram->compressor)); + zram->comp_algs[ZRAM_PRIMARY_ZCOMP] =3D default_compressor; =20 zram_debugfs_register(zram); pr_info("Added device: %s\n", zram->disk->disk_name); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index a2bda53020fd..4044ddbb2326 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -89,10 +89,20 @@ struct zram_stats { #endif }; =20 +#ifdef CONFIG_ZRAM_MULTI_COMP +#define ZRAM_PRIMARY_ZCOMP 0 +#define ZRAM_SECONDARY_ZCOMP 1 +#define ZRAM_MAX_ZCOMPS 2 +#else +#define ZRAM_PRIMARY_ZCOMP 0 +#define ZRAM_SECONDARY_ZCOMP 0 +#define ZRAM_MAX_ZCOMPS 1 +#endif + struct zram { struct zram_table_entry *table; struct zs_pool *mem_pool; - struct zcomp *comp; + struct zcomp *comps[ZRAM_MAX_ZCOMPS]; struct gendisk *disk; /* Prevent concurrent execution of device init */ struct rw_semaphore init_lock; @@ -107,7 +117,7 @@ struct zram { * we can store in a disk. */ u64 disksize; /* bytes */ - char compressor[CRYPTO_MAX_ALG_NAME]; + const char *comp_algs[ZRAM_MAX_ZCOMPS]; /* * zram is claimed so open request will be failed */ --=20 2.38.0.413.g74048e4d9e-goog From nobody Tue Apr 7 15:56:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CDCDC4332F for ; Tue, 18 Oct 2022 04:56:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229798AbiJRE4D (ORCPT ); Tue, 18 Oct 2022 00:56:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229905AbiJREzs (ORCPT ); Tue, 18 Oct 2022 00:55:48 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52E6A45F7C for ; Mon, 17 Oct 2022 21:55:47 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id o17-20020a17090aac1100b0020d98b0c0f4so14860921pjq.4 for ; Mon, 17 Oct 2022 21:55:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JFk+eNUXynOg9IsBl0rempe5RZozp8YcQAw5rmNlx7E=; b=iCGJlVBxWRXpUIMQsucXw6YsDvNwLojnvvHiyGxBI3A/0Gu/oRlJYCwhdk46CgBA65 G6Rw7zcaSq3YmH1fwtlCqV00WL7d4C9edtJW+4+FF8gYtdPa3nCAphCfMsd0+7AXr5+5 D+9Azge2plTkQRyHGI6hzyYkTVAtM2V+Jzlo0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JFk+eNUXynOg9IsBl0rempe5RZozp8YcQAw5rmNlx7E=; b=UDjrEsxklTC+6BaRp/dlIQ76MaPDqsM65G4eozGhIx4HWi+bgMIoK16OVUNbFvVoGk fdMTGK9ao3+XFA7sd905eL4UWhNOVGG41uOG+9/pbwRM2xqBNKwhEnoxosjNx9QWjgsc NxhCNVjgEzkuH/q7ImN9mtRONHIRgGF6alt9Pj30ClTqRrOaJ8LoZ3iF6blz7eTQc1Di ZZg6ZHhg5+20kvrvOreIqT9mAHEznF7nVU5hOysUY5cjELyFttw1m9OfrKEqH9VeC4U0 OE2iDPFn0GdBe1FjZSPGBx2P4vDwpoU4Gujy//amecL/5fFv/0GYhYweMS+78CXV1GXT 9Pwg== X-Gm-Message-State: ACrzQf2EMbwpEOnwYrztPEDiWiVrfhyIdL63nZ2Nnc9pEIC5ftcBPsgl SJ5Mofg12gg8jZZDtm+aW96IAw== X-Google-Smtp-Source: AMsMyM4qw2KQWg1A7qtGIj8f0nG7xq3qtR/3RNoXQuSnVGVdaqSzgZw9BR0qelI0lapfk4C3bR5Dlw== X-Received: by 2002:a17:903:124b:b0:179:da2f:2457 with SMTP id u11-20020a170903124b00b00179da2f2457mr1175806plh.156.1666068946825; Mon, 17 Oct 2022 21:55:46 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:17a9:73b0:c262:eccd]) by smtp.gmail.com with ESMTPSA id p4-20020a170902e74400b0017b69f99321sm7549220plf.219.2022.10.17.21.55.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 21:55:46 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 2/9] zram: Add recompression algorithm sysfs knob Date: Tue, 18 Oct 2022 13:55:26 +0900 Message-Id: <20221018045533.2396670-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog In-Reply-To: <20221018045533.2396670-1-senozhatsky@chromium.org> References: <20221018045533.2396670-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Introduce recomp_algorithm sysfs knob that controls secondary algorithm selection used for recompression. This device attribute works in a similar way with comp_algorithm attribute. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 111 +++++++++++++++++++++++++++------- 1 file changed, 90 insertions(+), 21 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 770ea3489eb6..a8ef3c0c3dae 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -41,7 +41,12 @@ static DEFINE_IDR(zram_index_idr); static DEFINE_MUTEX(zram_index_mutex); =20 static int zram_major; -static const char *default_compressor =3D CONFIG_ZRAM_DEF_COMP; +static const char *default_comp_algs[ZRAM_MAX_ZCOMPS] =3D { + CONFIG_ZRAM_DEF_COMP, +#ifdef CONFIG_ZRAM_MULTI_COMP + "zstd", +#endif +}; =20 /* Module params (documentation at end) */ static unsigned int num_devices =3D 1; @@ -1000,31 +1005,37 @@ static ssize_t max_comp_streams_store(struct device= *dev, return len; } =20 -static ssize_t comp_algorithm_show(struct device *dev, - struct device_attribute *attr, char *buf) +static void comp_algorithm_set(struct zram *zram, u32 idx, const char *alg) { - size_t sz; - struct zram *zram =3D dev_to_zram(dev); + bool default_alg =3D false; + int i; =20 - down_read(&zram->init_lock); - sz =3D zcomp_available_show(zram->comp_algs[ZRAM_PRIMARY_ZCOMP], buf); - up_read(&zram->init_lock); + /* Do not kfree() algs that we didn't allocate, IOW the default ones */ + for (i =3D 0; i < ZRAM_MAX_ZCOMPS; i++) { + if (zram->comp_algs[idx] =3D=3D default_comp_algs[i]) { + default_alg =3D true; + break; + } + } =20 - return sz; + if (!default_alg) + kfree(zram->comp_algs[idx]); + zram->comp_algs[idx] =3D alg; } =20 -static void comp_algorithm_set(struct zram *zram, u32 idx, const char *alg) +static ssize_t __comp_algorithm_show(struct zram *zram, u32 idx, char *buf) { - /* Do not kfree() algs that we didn't allocate, IOW the default ones */ - if (zram->comp_algs[idx] !=3D default_compressor) - kfree(zram->comp_algs[idx]); - zram->comp_algs[idx] =3D alg; + ssize_t sz; + + down_read(&zram->init_lock); + sz =3D zcomp_available_show(zram->comp_algs[idx], buf); + up_read(&zram->init_lock); + + return sz; } =20 -static ssize_t comp_algorithm_store(struct device *dev, - struct device_attribute *attr, const char *buf, size_t len) +static int __comp_algorithm_store(struct zram *zram, u32 idx, const char *= buf) { - struct zram *zram =3D dev_to_zram(dev); char *compressor; size_t sz; =20 @@ -1053,11 +1064,55 @@ static ssize_t comp_algorithm_store(struct device *= dev, return -EBUSY; } =20 - comp_algorithm_set(zram, ZRAM_PRIMARY_ZCOMP, compressor); + comp_algorithm_set(zram, idx, compressor); up_write(&zram->init_lock); - return len; + return 0; +} + +static ssize_t comp_algorithm_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct zram *zram =3D dev_to_zram(dev); + + return __comp_algorithm_show(zram, ZRAM_PRIMARY_ZCOMP, buf); +} + +static ssize_t comp_algorithm_store(struct device *dev, + struct device_attribute *attr, + const char *buf, + size_t len) +{ + struct zram *zram =3D dev_to_zram(dev); + int ret; + + ret =3D __comp_algorithm_store(zram, ZRAM_PRIMARY_ZCOMP, buf); + return ret ? ret : len; } =20 +#ifdef CONFIG_ZRAM_MULTI_COMP +static ssize_t recomp_algorithm_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct zram *zram =3D dev_to_zram(dev); + + return __comp_algorithm_show(zram, ZRAM_SECONDARY_ZCOMP, buf); +} + +static ssize_t recomp_algorithm_store(struct device *dev, + struct device_attribute *attr, + const char *buf, + size_t len) +{ + struct zram *zram =3D dev_to_zram(dev); + int ret; + + ret =3D __comp_algorithm_store(zram, ZRAM_SECONDARY_ZCOMP, buf); + return ret ? ret : len; +} +#endif + static ssize_t compact_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { @@ -1762,7 +1817,11 @@ static void zram_reset_device(struct zram *zram) memset(&zram->stats, 0, sizeof(zram->stats)); reset_bdev(zram); =20 - comp_algorithm_set(zram, ZRAM_PRIMARY_ZCOMP, default_compressor); + comp_algorithm_set(zram, ZRAM_PRIMARY_ZCOMP, + default_comp_algs[ZRAM_PRIMARY_ZCOMP]); + if (IS_ENABLED(CONFIG_ZRAM_MULTI_COMP)) + comp_algorithm_set(zram, ZRAM_SECONDARY_ZCOMP, + default_comp_algs[ZRAM_SECONDARY_ZCOMP]); up_write(&zram->init_lock); } =20 @@ -1895,6 +1954,9 @@ static DEVICE_ATTR_WO(writeback); static DEVICE_ATTR_RW(writeback_limit); static DEVICE_ATTR_RW(writeback_limit_enable); #endif +#ifdef CONFIG_ZRAM_MULTI_COMP +static DEVICE_ATTR_RW(recomp_algorithm); +#endif =20 static struct attribute *zram_disk_attrs[] =3D { &dev_attr_disksize.attr, @@ -1918,6 +1980,9 @@ static struct attribute *zram_disk_attrs[] =3D { &dev_attr_bd_stat.attr, #endif &dev_attr_debug_stat.attr, +#ifdef CONFIG_ZRAM_MULTI_COMP + &dev_attr_recomp_algorithm.attr, +#endif NULL, }; =20 @@ -1997,7 +2062,11 @@ static int zram_add(void) if (ret) goto out_cleanup_disk; =20 - zram->comp_algs[ZRAM_PRIMARY_ZCOMP] =3D default_compressor; + zram->comp_algs[ZRAM_PRIMARY_ZCOMP] =3D + default_comp_algs[ZRAM_PRIMARY_ZCOMP]; + if (IS_ENABLED(CONFIG_ZRAM_MULTI_COMP)) + zram->comp_algs[ZRAM_SECONDARY_ZCOMP] =3D + default_comp_algs[ZRAM_SECONDARY_ZCOMP]; =20 zram_debugfs_register(zram); pr_info("Added device: %s\n", zram->disk->disk_name); --=20 2.38.0.413.g74048e4d9e-goog From nobody Tue Apr 7 15:56:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E09F0C4332F for ; Tue, 18 Oct 2022 04:56:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229956AbiJRE4J (ORCPT ); Tue, 18 Oct 2022 00:56:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229718AbiJREzv (ORCPT ); Tue, 18 Oct 2022 00:55:51 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E416C6525F for ; Mon, 17 Oct 2022 21:55:49 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id d7-20020a17090a2a4700b0020d268b1f02so16252218pjg.1 for ; Mon, 17 Oct 2022 21:55:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lPOwozwW566MrhSDQ5Hik/2oBaG/MJdP0TGgYtIKPis=; b=YnRifkaNMN9xeU3dP05q8Ea3BFeTmAimQBXUIgaGkiE2mSiMswcN01xw+OrW2yPYCF OvIXJvFjayfS/j734H+93oHgzpWOVPR33IaJNC/+srLLlOZGNDfcMf0kZlVFSBcKh4MV dADJd8226DEIE85679/r2I8nGKw2AK68/UaEo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lPOwozwW566MrhSDQ5Hik/2oBaG/MJdP0TGgYtIKPis=; b=3wecTXYPpgSKFeUuItCY7ZprOFecyn/IKah4f7Gay0xQrtP7LlvqgqEqvSVfY0VVh9 UmvgP5lYa5yXFqtyn2OssjlN75eYN6aIN9w1mFtiQLZD4xxNBsSidQ6NdAit8MPxKRLH j7vKTWRjaRmw08ZdWkLwiXZmHPSvFoTNL819bXEqPn58NdtnSuzqTmhVKyaVkdGtNL/w 2P3u2AS4WUl+pZJmTf4Pptc0M6uqKo3inB/GoZmrRuUOSbKyL9b47PRakAtWaYQ1X5DR dExIAtCcX0JRqHMdr/hY/tkHdjXwasHKL2vcyBEp+xT8GGBiGdMvJfq2ZjY9gmTKORG6 owVw== X-Gm-Message-State: ACrzQf3BLra82wJMCNy8WE8ngToogRmKp9HuZpQYWzx1GKrnybKk9vuG gP1Tdo4HaDeChvZA7dGsetIf4g== X-Google-Smtp-Source: AMsMyM68b3LFlOFKZ5ZmhuvTT7t7kmp/l6V3wTu+yDELsOlS3kOPAXwmAmfP+TaiyKDtir3UTdwQrA== X-Received: by 2002:a17:902:c405:b0:181:83e4:490e with SMTP id k5-20020a170902c40500b0018183e4490emr1173860plk.4.1666068949191; Mon, 17 Oct 2022 21:55:49 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:17a9:73b0:c262:eccd]) by smtp.gmail.com with ESMTPSA id p4-20020a170902e74400b0017b69f99321sm7549220plf.219.2022.10.17.21.55.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 21:55:48 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 3/9] zram: Factor out WB and non-WB zram read functions Date: Tue, 18 Oct 2022 13:55:27 +0900 Message-Id: <20221018045533.2396670-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog In-Reply-To: <20221018045533.2396670-1-senozhatsky@chromium.org> References: <20221018045533.2396670-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" We will use non-WB variant in ZRAM page recompression path. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 73 ++++++++++++++++++++++++----------- 1 file changed, 50 insertions(+), 23 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index a8ef3c0c3dae..94c62d7ea818 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1314,8 +1314,30 @@ static void zram_free_page(struct zram *zram, size_t= index) ~(1UL << ZRAM_LOCK | 1UL << ZRAM_UNDER_WB)); } =20 -static int __zram_bvec_read(struct zram *zram, struct page *page, u32 inde= x, - struct bio *bio, bool partial_io) +/* + * Reads a page from the writeback devices. Corresponding ZRAM slot + * should be unlocked. + */ +static int zram_read_from_writeback(struct zram *zram, struct page *page, + u32 index, struct bio *bio, + bool partial_io) +{ + struct bio_vec bvec; + + bvec.bv_page =3D page; + bvec.bv_len =3D PAGE_SIZE; + bvec.bv_offset =3D 0; + return read_from_bdev(zram, &bvec, + zram_get_element(zram, index), + bio, partial_io); +} + +/* + * Reads (decompresses if needed) a page from zspool (zsmalloc). + * Corresponding ZRAM slot should be locked. + */ +static int zram_read_from_zspool(struct zram *zram, struct page *page, + u32 index) { struct zcomp_strm *zstrm; unsigned long handle; @@ -1323,23 +1345,6 @@ static int __zram_bvec_read(struct zram *zram, struc= t page *page, u32 index, void *src, *dst; int ret; =20 - zram_slot_lock(zram, index); - if (zram_test_flag(zram, index, ZRAM_WB)) { - struct bio_vec bvec; - - zram_slot_unlock(zram, index); - /* A null bio means rw_page was used, we must fallback to bio */ - if (!bio) - return -EOPNOTSUPP; - - bvec.bv_page =3D page; - bvec.bv_len =3D PAGE_SIZE; - bvec.bv_offset =3D 0; - return read_from_bdev(zram, &bvec, - zram_get_element(zram, index), - bio, partial_io); - } - handle =3D zram_get_handle(zram, index); if (!handle || zram_test_flag(zram, index, ZRAM_SAME)) { unsigned long value; @@ -1349,7 +1354,6 @@ static int __zram_bvec_read(struct zram *zram, struct= page *page, u32 index, mem =3D kmap_atomic(page); zram_fill_page(mem, PAGE_SIZE, value); kunmap_atomic(mem); - zram_slot_unlock(zram, index); return 0; } =20 @@ -1371,17 +1375,40 @@ static int __zram_bvec_read(struct zram *zram, stru= ct page *page, u32 index, zcomp_stream_put(zram->comps[ZRAM_PRIMARY_ZCOMP]); } zs_unmap_object(zram->mem_pool, handle); - zram_slot_unlock(zram, index); + return ret; +} + +static int __zram_bvec_read(struct zram *zram, struct page *page, u32 inde= x, + struct bio *bio, bool partial_io) +{ + int ret; + + zram_slot_lock(zram, index); + if (!zram_test_flag(zram, index, ZRAM_WB)) { + /* Slot should be locked through out the function call */ + ret =3D zram_read_from_zspool(zram, page, index); + zram_slot_unlock(zram, index); + } else { + /* Slot should be unlocked before the function call */ + zram_slot_unlock(zram, index); + + /* A null bio means rw_page was used, we must fallback to bio */ + if (!bio) + return -EOPNOTSUPP; + + ret =3D zram_read_from_writeback(zram, page, index, bio, + partial_io); + } =20 /* Should NEVER happen. Return bio error if it does. */ - if (WARN_ON(ret)) + if (WARN_ON(ret < 0)) pr_err("Decompression failed! err=3D%d, page=3D%u\n", ret, index); =20 return ret; } =20 static int zram_bvec_read(struct zram *zram, struct bio_vec *bvec, - u32 index, int offset, struct bio *bio) + u32 index, int offset, struct bio *bio) { int ret; struct page *page; --=20 2.38.0.413.g74048e4d9e-goog From nobody Tue Apr 7 15:56:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C3F6C4332F for ; Tue, 18 Oct 2022 04:56:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230139AbiJRE4R (ORCPT ); Tue, 18 Oct 2022 00:56:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229848AbiJREzx (ORCPT ); Tue, 18 Oct 2022 00:55:53 -0400 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 610019C2E4 for ; Mon, 17 Oct 2022 21:55:52 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id h185so12310850pgc.10 for ; Mon, 17 Oct 2022 21:55:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KakqBjSQ6Ep6MOW9Qh1kHbwwQF5mRCkC8aIFa9DdqUw=; b=DX29os8zmQ9tNTIbLain0oTGkNhWorNiLSIbcEIS9zUjBxum+HzT1PGjwzHnll2TuV MqPcv8tyqGOAFrrZccUSo2Z3xwIaCBZsW8BvoGjUzCcrBjkpSNgY2OtfvPvYIKlJBHpP C04ZJeFLG5EK2OFDtZAEPWZCQa/fCWdyAXVwA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KakqBjSQ6Ep6MOW9Qh1kHbwwQF5mRCkC8aIFa9DdqUw=; b=jDbCeRWOcQO/bKqeZCyrJPBSc985tQNeigWVFt2sPPGCCdTLy23PzS1BCCM0O6bWmk Eon+GVaOvU1yR4JwXG79km5FVW03q1WLSUVMmcfJmeA0A2pdfMQBjy3fUcDE3W56O5Dq Qs9teuhz5ztFsXbvP5O4iWkYhbWdFK9WDAblmYjgUr7sol3gsznkWCUBz6+atP7pcDGh 5HEnoW/fK8Z8BHH47/TPqo+DUcbnVVw/1p4GRDSoQBI8yZaFiTLvt5qr9viYrKV5UFz5 w5I017zxzTFmuzBcy53HeVtDTzZWM8ZjtCF6y0cEKLGyeE+7zBHsgEQP792RBLx5r2Ge Nwig== X-Gm-Message-State: ACrzQf1weD1zFWaPJ6gm6G5wlvMNRKNx4l+9NE314lgZQOU2CyQxvUyB qyBmhArdB/fsckgMzHgxVFz8vw== X-Google-Smtp-Source: AMsMyM6XYl3cSzycod2x/jFmz2cyJvc8ZV9FNrvk80nWi+NhE2+4JQi8JbVJFgRyYotfkEIzaj7ofQ== X-Received: by 2002:a05:6a00:3249:b0:565:fc2c:ad79 with SMTP id bn9-20020a056a00324900b00565fc2cad79mr1332858pfb.72.1666068951586; Mon, 17 Oct 2022 21:55:51 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:17a9:73b0:c262:eccd]) by smtp.gmail.com with ESMTPSA id p4-20020a170902e74400b0017b69f99321sm7549220plf.219.2022.10.17.21.55.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 21:55:51 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 4/9] zram: Introduce recompress sysfs knob Date: Tue, 18 Oct 2022 13:55:28 +0900 Message-Id: <20221018045533.2396670-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog In-Reply-To: <20221018045533.2396670-1-senozhatsky@chromium.org> References: <20221018045533.2396670-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Allow zram to recompress (using secondary compression streams) pages. We support three modes: 1) IDLE pages recompression is activated by `idle` mode echo idle > /sys/block/zram0/recompress 2) Since there may be many idle pages user-space may pass a size watermark value (in bytes) and we will recompress IDLE pages only of equal or greater size: echo 888 > /sys/block/zram0/recompress 3) HUGE pages recompression is activated by `huge` mode echo huge > /sys/block/zram0/recompress 4) HUGE_IDLE pages recompression is activated by `huge_idle` mode echo huge_idle > /sys/block/zram0/recompress Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/Kconfig | 15 +++ drivers/block/zram/zram_drv.c | 196 +++++++++++++++++++++++++++++++++- drivers/block/zram/zram_drv.h | 2 + 3 files changed, 210 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig index d4100b0c083e..3e00656a6f8a 100644 --- a/drivers/block/zram/Kconfig +++ b/drivers/block/zram/Kconfig @@ -78,3 +78,18 @@ config ZRAM_MEMORY_TRACKING /sys/kernel/debug/zram/zramX/block_state. =20 See Documentation/admin-guide/blockdev/zram.rst for more information. + +config ZRAM_MULTI_COMP + bool "Enable multiple per-CPU compression streams" + depends on ZRAM + help + This will enable per-CPU multi-compression streams, so that ZRAM + can re-compress IDLE/huge pages, using a potentially slower but + more effective compression algorithm. Note, that IDLE page support + requires ZRAM_MEMORY_TRACKING. + + echo TIMEOUT > /sys/block/zramX/idle + echo SIZE > /sys/block/zramX/recompress + + SIZE (in bytes) parameter sets the object size watermark: idle + objects that are of a smaller size will not get recompressed. diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 94c62d7ea818..da11560ecf70 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1282,6 +1282,12 @@ static void zram_free_page(struct zram *zram, size_t= index) atomic64_dec(&zram->stats.huge_pages); } =20 + if (zram_test_flag(zram, index, ZRAM_RECOMP)) + zram_clear_flag(zram, index, ZRAM_RECOMP); + + if (zram_test_flag(zram, index, ZRAM_RECOMP_SKIP)) + zram_clear_flag(zram, index, ZRAM_RECOMP_SKIP); + if (zram_test_flag(zram, index, ZRAM_WB)) { zram_clear_flag(zram, index, ZRAM_WB); free_block_bdev(zram, zram_get_element(zram, index)); @@ -1343,6 +1349,7 @@ static int zram_read_from_zspool(struct zram *zram, s= truct page *page, unsigned long handle; unsigned int size; void *src, *dst; + u32 idx; int ret; =20 handle =3D zram_get_handle(zram, index); @@ -1359,8 +1366,13 @@ static int zram_read_from_zspool(struct zram *zram, = struct page *page, =20 size =3D zram_get_obj_size(zram, index); =20 - if (size !=3D PAGE_SIZE) - zstrm =3D zcomp_stream_get(zram->comps[ZRAM_PRIMARY_ZCOMP]); + if (size !=3D PAGE_SIZE) { + idx =3D ZRAM_PRIMARY_ZCOMP; + if (zram_test_flag(zram, index, ZRAM_RECOMP)) + idx =3D ZRAM_SECONDARY_ZCOMP; + + zstrm =3D zcomp_stream_get(zram->comps[idx]); + } =20 src =3D zs_map_object(zram->mem_pool, handle, ZS_MM_RO); if (size =3D=3D PAGE_SIZE) { @@ -1372,7 +1384,7 @@ static int zram_read_from_zspool(struct zram *zram, s= truct page *page, dst =3D kmap_atomic(page); ret =3D zcomp_decompress(zstrm, src, size, dst); kunmap_atomic(dst); - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_ZCOMP]); + zcomp_stream_put(zram->comps[idx]); } zs_unmap_object(zram->mem_pool, handle); return ret; @@ -1603,6 +1615,182 @@ static int zram_bvec_write(struct zram *zram, struc= t bio_vec *bvec, return ret; } =20 +#ifdef CONFIG_ZRAM_MULTI_COMP +/* + * This function will decompress (unless it's ZRAM_HUGE) the page and then + * attempt to compress it using secondary compression algorithm (which is + * potentially more effective). + * + * Corresponding ZRAM slot should be locked. + */ +static int zram_recompress(struct zram *zram, u32 index, struct page *page, + int size_watermark) +{ + unsigned long handle_prev; + unsigned long handle_next; + unsigned int comp_len_next; + unsigned int comp_len_prev; + struct zcomp_strm *zstrm; + void *src, *dst; + int ret; + + handle_prev =3D zram_get_handle(zram, index); + if (!handle_prev) + return -EINVAL; + + comp_len_prev =3D zram_get_obj_size(zram, index); + /* + * Do not recompress objects that are already "small enough". + */ + if (comp_len_prev < size_watermark) + return 0; + + ret =3D zram_read_from_zspool(zram, page, index); + if (ret) + return ret; + + zstrm =3D zcomp_stream_get(zram->comps[ZRAM_SECONDARY_ZCOMP]); + src =3D kmap_atomic(page); + ret =3D zcomp_compress(zstrm, src, &comp_len_next); + kunmap_atomic(src); + + /* + * Either a compression error or we failed to compressed the object + * in a way that will save us memory. Mark the object so that we + * don't attempt to re-compress it again (RECOMP_SKIP). + */ + if (comp_len_next >=3D huge_class_size || + comp_len_next >=3D comp_len_prev || + ret) { + zram_set_flag(zram, index, ZRAM_RECOMP_SKIP); + zram_clear_flag(zram, index, ZRAM_IDLE); + zcomp_stream_put(zram->comps[ZRAM_SECONDARY_ZCOMP]); + return ret; + } + + /* + * No direct reclaim (slow path) for handle allocation and no + * re-compression attempt (unlike in __zram_bvec_write()) since + * we already have stored that object in zsmalloc. If we cannot + * alloc memory for recompressed object then we bail out and + * simply keep the old (existing) object in zsmalloc. + */ + handle_next =3D zs_malloc(zram->mem_pool, comp_len_next, + __GFP_KSWAPD_RECLAIM | + __GFP_NOWARN | + __GFP_HIGHMEM | + __GFP_MOVABLE); + if (IS_ERR((void *)handle_next)) { + zcomp_stream_put(zram->comps[ZRAM_SECONDARY_ZCOMP]); + return PTR_ERR((void *)handle_next); + } + + dst =3D zs_map_object(zram->mem_pool, handle_next, ZS_MM_WO); + memcpy(dst, zstrm->buffer, comp_len_next); + zcomp_stream_put(zram->comps[ZRAM_SECONDARY_ZCOMP]); + + zs_unmap_object(zram->mem_pool, handle_next); + + zram_free_page(zram, index); + zram_set_handle(zram, index, handle_next); + zram_set_obj_size(zram, index, comp_len_next); + + zram_set_flag(zram, index, ZRAM_RECOMP); + atomic64_add(comp_len_next, &zram->stats.compr_data_size); + atomic64_inc(&zram->stats.pages_stored); + + return 0; +} + +#define RECOMPRESS_IDLE (1 << 0) +#define RECOMPRESS_HUGE (1 << 1) + +static ssize_t recompress_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct zram *zram =3D dev_to_zram(dev); + unsigned long nr_pages =3D zram->disksize >> PAGE_SHIFT; + unsigned long index; + struct page *page; + ssize_t ret; + int mode, size_watermark =3D 0; + + if (sysfs_streq(buf, "idle")) { + mode =3D RECOMPRESS_IDLE; + } else if (sysfs_streq(buf, "huge")) { + mode =3D RECOMPRESS_HUGE; + } else if (sysfs_streq(buf, "huge_idle")) { + mode =3D RECOMPRESS_IDLE | RECOMPRESS_HUGE; + } else { + /* + * We will re-compress only idle objects equal or greater + * in size than watermark. + */ + ret =3D kstrtoint(buf, 10, &size_watermark); + if (ret) + return ret; + mode =3D RECOMPRESS_IDLE; + } + + if (size_watermark > PAGE_SIZE) + return -EINVAL; + + down_read(&zram->init_lock); + if (!init_done(zram)) { + ret =3D -EINVAL; + goto release_init_lock; + } + + page =3D alloc_page(GFP_KERNEL); + if (!page) { + ret =3D -ENOMEM; + goto release_init_lock; + } + + ret =3D len; + for (index =3D 0; index < nr_pages; index++) { + int err =3D 0; + + zram_slot_lock(zram, index); + + if (!zram_allocated(zram, index)) + goto next; + + if (mode & RECOMPRESS_IDLE && + !zram_test_flag(zram, index, ZRAM_IDLE)) + goto next; + + if (mode & RECOMPRESS_HUGE && + !zram_test_flag(zram, index, ZRAM_HUGE)) + goto next; + + if (zram_test_flag(zram, index, ZRAM_WB) || + zram_test_flag(zram, index, ZRAM_UNDER_WB) || + zram_test_flag(zram, index, ZRAM_SAME) || + zram_test_flag(zram, index, ZRAM_RECOMP) || + zram_test_flag(zram, index, ZRAM_RECOMP_SKIP)) + goto next; + + err =3D zram_recompress(zram, index, page, size_watermark); +next: + zram_slot_unlock(zram, index); + if (err) { + ret =3D err; + break; + } + + cond_resched(); + } + + __free_page(page); + +release_init_lock: + up_read(&zram->init_lock); + return ret; +} +#endif + /* * zram_bio_discard - handler on discard request * @index: physical block index in PAGE_SIZE units @@ -1983,6 +2171,7 @@ static DEVICE_ATTR_RW(writeback_limit_enable); #endif #ifdef CONFIG_ZRAM_MULTI_COMP static DEVICE_ATTR_RW(recomp_algorithm); +static DEVICE_ATTR_WO(recompress); #endif =20 static struct attribute *zram_disk_attrs[] =3D { @@ -2009,6 +2198,7 @@ static struct attribute *zram_disk_attrs[] =3D { &dev_attr_debug_stat.attr, #ifdef CONFIG_ZRAM_MULTI_COMP &dev_attr_recomp_algorithm.attr, + &dev_attr_recompress.attr, #endif NULL, }; diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 4044ddbb2326..09b9ceb5dfa3 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -49,6 +49,8 @@ enum zram_pageflags { ZRAM_UNDER_WB, /* page is under writeback */ ZRAM_HUGE, /* Incompressible page */ ZRAM_IDLE, /* not accessed page since last idle marking */ + ZRAM_RECOMP, /* page was recompressed */ + ZRAM_RECOMP_SKIP, /* secondary algorithm cannot compress this page */ =20 __NR_ZRAM_PAGEFLAGS, }; --=20 2.38.0.413.g74048e4d9e-goog From nobody Tue Apr 7 15:56:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14B94C4332F for ; Tue, 18 Oct 2022 04:56:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229794AbiJRE4X (ORCPT ); Tue, 18 Oct 2022 00:56:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229943AbiJREzz (ORCPT ); Tue, 18 Oct 2022 00:55:55 -0400 Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3647A0256 for ; Mon, 17 Oct 2022 21:55:54 -0700 (PDT) Received: by mail-pg1-x534.google.com with SMTP id 129so12329364pgc.5 for ; Mon, 17 Oct 2022 21:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zqbNRdZv0AEZA2o65O/7d+PvuPWf+J/YFOhKZdVnIao=; b=ds+HJ6GR0TTrsjhdVBePPTb+ZYdPlv5VzdAyfO5kycvLRMKU33bufTdI0konnlBlZM 2jNp8xV7OmYFzIYYik/sjHdf7RJiNFshnSc4wMS+NII79AKetCRjFskA10EFZ/6QjeS5 QQZnpbF4K4nH6Tm/zfaat/KILJuoWiu3I4ZG4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zqbNRdZv0AEZA2o65O/7d+PvuPWf+J/YFOhKZdVnIao=; b=pRxDUrTNMDUsdtVj9pojf6akSsBL8ufzaAVVPyntw7pzwj6gHJiCvKq5hAIUg73XYd 6Zv0Xge7EnKtdJ18etzGardj7zJvWUlSu2BOFbqVTzFQrqCBKDaXf6g39ELaShMSBPPQ QyBo+NfvoF1TYMhW9Wect2eelYe4zano3wVkbzjc1MCeTf2EpQE4H+eGdgcgRd38RZTf KySzvNd7ceP5ufc30Mtx3iNCivMjo+GnPj4yM7ww0Lud56Vc9IwiZKOOQ3b0gdgQ6j1I 0jnfxek4nMcMTS+pxvDEjIEcUp+G/jS7EvLQqJMkHFGtBseiwkEKFXv62gMAp9ymm9KI BEEA== X-Gm-Message-State: ACrzQf1xwiS+MEKKfBOC9vE2lOglSzWlpM5wR2E9SLBhcOoUkTWHuy3W ejZMr3dfNWBSbMfZM1GTULAFxg== X-Google-Smtp-Source: AMsMyM4gF8hnVrDD4vEMZDKoG10O5uE+BlIbYnroail7BaHbMICR6KtHATfACK4nJDhUHhjX+lvO1g== X-Received: by 2002:a05:6a00:1a92:b0:565:d5c0:f627 with SMTP id e18-20020a056a001a9200b00565d5c0f627mr1383823pfv.10.1666068953954; Mon, 17 Oct 2022 21:55:53 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:17a9:73b0:c262:eccd]) by smtp.gmail.com with ESMTPSA id p4-20020a170902e74400b0017b69f99321sm7549220plf.219.2022.10.17.21.55.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 21:55:53 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 5/9] documentation: Add recompression documentation Date: Tue, 18 Oct 2022 13:55:29 +0900 Message-Id: <20221018045533.2396670-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog In-Reply-To: <20221018045533.2396670-1-senozhatsky@chromium.org> References: <20221018045533.2396670-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Document user-space visible device attributes that are enabled by ZRAM_MULTI_COMP. Signed-off-by: Sergey Senozhatsky --- Documentation/admin-guide/blockdev/zram.rst | 55 +++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/ad= min-guide/blockdev/zram.rst index c73b16930449..c916c2b9da55 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -401,6 +401,61 @@ budget in next setting is user's job. If admin wants to measure writeback count in a certain period, they could know it via /sys/block/zram0/bd_stat's 3rd column. =20 +recompression +------------- + +With CONFIG_ZRAM_MULTI_COMP, zram can recompress idle/huge pages using +alternative (secondary) compression algorithm. The basic idea is that +alternative compression algorithm can provide better compression ratio +at a price of (potentially) slower compression/decompression speeds. +Alternative compression algorithm can, for example, be more successful +compressing huge pages (those that default algorithm failed to compress). +Another application is idle pages recompression - pages that are cold and +sit in the memory can be recompressed using more effective algorithm and, +hence, reduce zsmalloc memory usage. + +With CONFIG_ZRAM_MULTI_COMP, zram will setup two compression algorithms +per-CPU: primary and secondary ones. Primary zram compressor is explained +in "3) Select compression algorithm", the secondary algorithm is configured +in a similar way, using recomp_algorithm device attribute: + +Examples:: + + #show supported recompression algorithms + cat /sys/block/zramX/recomp_algorithm + zstd [lzo] + + #select zstd recompression algorithm + echo zstd > /sys/block/zramX/recomp_algorithm + +Another device attribute that CONFIG_ZRAM_MULTI_COMP enables is recompress, +which controls recompression: + +Examples:: + + #IDLE pages recompression is activated by `idle` mode + echo idle > /sys/block/zramX/recompress + + #HUGE pages recompression is activated by `huge` mode + echo huge > /sys/block/zram0/recompress + + #HUGE_IDLE pages recompression is activated by `huge_idle` mode + echo huge_idle > /sys/block/zramX/recompress + +The number of idle pages can be significant, so user-space can pass a size +watermark value (in bytes) to the recompress knob, to filter out idle pages +for recompression: zram will recompress only idle pages of equal or greater +size::: + + #recompress idle pages larger than 3000 bytes + echo 3000 > /sys/block/zramX/recompress + + #recompress idle pages larger than 2000 bytes + echo 2000 > /sys/block/zramX/recompress + +Recompression is mostly focused on idle pages (except for huge pages +recompression), so it works better in conjunction with memory tracking. + memory tracking =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 --=20 2.38.0.413.g74048e4d9e-goog From nobody Tue Apr 7 15:56:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF6FEC433FE for ; Tue, 18 Oct 2022 04:56:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230097AbiJRE4p (ORCPT ); Tue, 18 Oct 2022 00:56:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230084AbiJRE4B (ORCPT ); Tue, 18 Oct 2022 00:56:01 -0400 Received: from mail-pg1-x52a.google.com (mail-pg1-x52a.google.com [IPv6:2607:f8b0:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14CF4A02C0 for ; Mon, 17 Oct 2022 21:55:57 -0700 (PDT) Received: by mail-pg1-x52a.google.com with SMTP id e129so12315121pgc.9 for ; Mon, 17 Oct 2022 21:55:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=u6FdlTaAeoJZnsdClOVvKEq4lJg+XPWe1oKWdw8/hCk=; b=mGht8IGExOGxSDdSigMACLR7r+oqEPBhqmdFZwzcCZbnhwEIwDlx8IUzwUJ2oNN4mf yI7zRshfGlUr4VjyEIYcQm9A+xqAcUqq0SHRcig+dl2PYPG/RiYazZ4zPKWAYc9FFzju 6vDXIdBf+phqB93FKQYUwUmhyALJr9U293+wA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u6FdlTaAeoJZnsdClOVvKEq4lJg+XPWe1oKWdw8/hCk=; b=6q00BR2suOJifYFsWMFM8Ktm96gW+5lpXiIbbwxd0K2sD9md9r4uSadVMg53o+VO0a 0OoCYOASM3a6DmnFEtbmYB56I9mwYwhiVSFBANOh87u58VUKfI0YnAVv8ClEe7aFxwOI Lb9eOYGlmWy5PVWbzuCIUlkQzyyQpx0/hN71zw2AaUfYLj18ie50SMO4StXF02DNrE5H 66gLAp1Q+En+mLUEnxJC4796D9PGN7hOQXW4zF4zZ3PwoXEhD3piVVYjoHclN5yOLWOr XsL9igpexD13c9WZJqVYYKqj1mHdW8fcGJZY0UICWCEfx4+f6aZ5+bq4NMF+VXrYFDPF wKpA== X-Gm-Message-State: ACrzQf1OXPOxNgE49vD8qgkapKcIZAfiBuwDOrvQnj3QQM5kOcVZLypD YLonoAV+cYX53SkTwyESHvVRnw== X-Google-Smtp-Source: AMsMyM6g/QL3RczfjsbLh8zvYrP191iu/ZlPb5PzgxBWDVBC98pOVFGn60QYUSh3PKf4EuHzV0UVLw== X-Received: by 2002:a05:6a00:1ad0:b0:564:a791:42dc with SMTP id f16-20020a056a001ad000b00564a79142dcmr1116320pfv.50.1666068956740; Mon, 17 Oct 2022 21:55:56 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:17a9:73b0:c262:eccd]) by smtp.gmail.com with ESMTPSA id p4-20020a170902e74400b0017b69f99321sm7549220plf.219.2022.10.17.21.55.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 21:55:56 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 6/9] zram: Add recompression algorithm choice to Kconfig Date: Tue, 18 Oct 2022 13:55:30 +0900 Message-Id: <20221018045533.2396670-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog In-Reply-To: <20221018045533.2396670-1-senozhatsky@chromium.org> References: <20221018045533.2396670-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Make (secondary) recompression algorithm selectable just like we do it for the (primary) default one. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/Kconfig | 40 +++++++++++++++++++++++++++++++++++ drivers/block/zram/zram_drv.c | 2 +- 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig index 3e00656a6f8a..076a76cd1664 100644 --- a/drivers/block/zram/Kconfig +++ b/drivers/block/zram/Kconfig @@ -93,3 +93,43 @@ config ZRAM_MULTI_COMP =20 SIZE (in bytes) parameter sets the object size watermark: idle objects that are of a smaller size will not get recompressed. + +choice + prompt "Default zram recompression algorithm" + default ZRAM_DEF_RECOMP_ZSTD + depends on ZRAM && ZRAM_MULTI_COMP + +config ZRAM_DEF_RECOMP_LZORLE + bool "lzo-rle" + depends on CRYPTO_LZO + +config ZRAM_DEF_RECOMP_ZSTD + bool "zstd" + depends on CRYPTO_ZSTD + +config ZRAM_DEF_RECOMP_LZ4 + bool "lz4" + depends on CRYPTO_LZ4 + +config ZRAM_DEF_RECOMP_LZO + bool "lzo" + depends on CRYPTO_LZO + +config ZRAM_DEF_RECOMP_LZ4HC + bool "lz4hc" + depends on CRYPTO_LZ4HC + +config ZRAM_DEF_RECOMP_842 + bool "842" + depends on CRYPTO_842 + +endchoice + +config ZRAM_DEF_RECOMP + string + default "lzo-rle" if ZRAM_DEF_RECOMP_LZORLE + default "zstd" if ZRAM_DEF_RECOMP_ZSTD + default "lz4" if ZRAM_DEF_RECOMP_LZ4 + default "lzo" if ZRAM_DEF_RECOMP_LZO + default "lz4hc" if ZRAM_DEF_RECOMP_LZ4HC + default "842" if ZRAM_DEF_RECOMP_842 diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index da11560ecf70..1e9561217466 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -44,7 +44,7 @@ static int zram_major; static const char *default_comp_algs[ZRAM_MAX_ZCOMPS] =3D { CONFIG_ZRAM_DEF_COMP, #ifdef CONFIG_ZRAM_MULTI_COMP - "zstd", + CONFIG_ZRAM_DEF_RECOMP, #endif }; =20 --=20 2.38.0.413.g74048e4d9e-goog From nobody Tue Apr 7 15:56:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BB38C4332F for ; Tue, 18 Oct 2022 04:56:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230121AbiJRE4u (ORCPT ); Tue, 18 Oct 2022 00:56:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229869AbiJRE4D (ORCPT ); Tue, 18 Oct 2022 00:56:03 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67DF3A220A for ; Mon, 17 Oct 2022 21:55:59 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id y191so13056696pfb.2 for ; Mon, 17 Oct 2022 21:55:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xtQh7ZmjoyZo2F0qXeywbofevhcfT59MQNGpxOyNbx8=; b=Ez9/9h1lS/+dC1G6E7jdHeTIahEsqMGdrK+uwr+wyLQfJq+YLjzffrRXXk0guSR6Ye VzU5kwyeOgaqpli2PFTCX+fpqvNyEHxZoVc7VzgNecYXJnNNROFa4+6BvojEANHXNLHN Udu8/kOoLQRV0SokqX/ZR/orQ8KcP4PTdY21k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xtQh7ZmjoyZo2F0qXeywbofevhcfT59MQNGpxOyNbx8=; b=CH9trWJhAo6hSop840Y6vivx5xop8jrgeJ3pOwASu3xaXjktllEcKdAGaz92nFn7HM dZ1DzFTgpIGPZAX0rBe5OOOsw7EETS44yNKpCLiLUfzjYsnXGc7CofPOLP/fL5xJNcRm dN9tV+tWE13pP5yuEUSE6DB3FTpNo+eP5aDszu5rM5TVkwQgiMA6zKP2KLk33d+CG/W6 fh3eJm4gV5un76D0wdDM17I6yPH71Sh3nCT4DHFjiIs4q3ERYyjugo6Takx2OfMGtLVa WTTC9spVuWGjZRHy9c7q54Fcy/PlbRsOSxLxXkIroCU+fDwpUjTTSUBzOMkrPHDDen0F jvmg== X-Gm-Message-State: ACrzQf2yf0vo0jVXZkhO7U8VpNWhwtj4zZXa12tRxPYTfJt5Mq/jLWW4 t+Fas6g7p1najzcfz2Ik/H4gPg== X-Google-Smtp-Source: AMsMyM4FOU2/bLgIX42hGV0iONzrzomt7CCu65KWnilogKYyQZoRpCn/IH6cUR0nYi3r5vr3skYaXQ== X-Received: by 2002:a05:6a00:1781:b0:561:7f7f:dc38 with SMTP id s1-20020a056a00178100b005617f7fdc38mr1384946pfg.42.1666068958685; Mon, 17 Oct 2022 21:55:58 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:17a9:73b0:c262:eccd]) by smtp.gmail.com with ESMTPSA id p4-20020a170902e74400b0017b69f99321sm7549220plf.219.2022.10.17.21.55.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 21:55:58 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 7/9] zram: Add recompress flag to read_block_state() Date: Tue, 18 Oct 2022 13:55:31 +0900 Message-Id: <20221018045533.2396670-8-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog In-Reply-To: <20221018045533.2396670-1-senozhatsky@chromium.org> References: <20221018045533.2396670-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a new flag to zram block state that shows if the page was recompressed (using alternative compression algorithm). Signed-off-by: Sergey Senozhatsky --- Documentation/admin-guide/blockdev/zram.rst | 9 ++++++--- drivers/block/zram/zram_drv.c | 5 +++-- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/ad= min-guide/blockdev/zram.rst index c916c2b9da55..010fb05a5999 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -466,9 +466,10 @@ pages of the process with*pagemap. If you enable the feature, you could see block state via /sys/kernel/debug/zram/zram0/block_state". The output is as follows:: =20 - 300 75.033841 .wh. - 301 63.806904 s... - 302 63.806919 ..hi + 300 75.033841 .wh.. + 301 63.806904 s.... + 302 63.806919 ..hi. + 303 62.801919 ....r =20 First column zram's block index. @@ -485,6 +486,8 @@ Third column huge page i: idle page + r: + recompressed page (secondary compression algorithm) =20 First line of above example says 300th block is accessed at 75.033841sec and the block's state is huge so it is written back to the backing diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 1e9561217466..5d760467e0bc 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -925,13 +925,14 @@ static ssize_t read_block_state(struct file *file, ch= ar __user *buf, =20 ts =3D ktime_to_timespec64(zram->table[index].ac_time); copied =3D snprintf(kbuf + written, count, - "%12zd %12lld.%06lu %c%c%c%c\n", + "%12zd %12lld.%06lu %c%c%c%c%c\n", index, (s64)ts.tv_sec, ts.tv_nsec / NSEC_PER_USEC, zram_test_flag(zram, index, ZRAM_SAME) ? 's' : '.', zram_test_flag(zram, index, ZRAM_WB) ? 'w' : '.', zram_test_flag(zram, index, ZRAM_HUGE) ? 'h' : '.', - zram_test_flag(zram, index, ZRAM_IDLE) ? 'i' : '.'); + zram_test_flag(zram, index, ZRAM_IDLE) ? 'i' : '.', + zram_test_flag(zram, index, ZRAM_RECOMP) ? 'r' : '.'); =20 if (count <=3D copied) { zram_slot_unlock(zram, index); --=20 2.38.0.413.g74048e4d9e-goog From nobody Tue Apr 7 15:56:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E668C433FE for ; Tue, 18 Oct 2022 04:56:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230273AbiJRE4y (ORCPT ); Tue, 18 Oct 2022 00:56:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230117AbiJRE4H (ORCPT ); Tue, 18 Oct 2022 00:56:07 -0400 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6493A2AAE for ; Mon, 17 Oct 2022 21:56:01 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id h185so12311080pgc.10 for ; Mon, 17 Oct 2022 21:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=z6j2M3ywUHsS1+ykQ/OfNhi067lR0Bt99BN92Wk+zcw=; b=H4tQqYyGb10hilriYTv+DvIcK69suR/MknRv/uI/c7Sq8IVi/fkI5CgIYlaPZTn3R3 fcnApKLzJ2JC3QxaMOGgeKe9UHO+kkbWQ0logwCwtTSA1l45M7ttQfjxjlmDarqRhC/q +YH2hyuuTFmys4kuUH7wG9CPoEgVf6/DURZSo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=z6j2M3ywUHsS1+ykQ/OfNhi067lR0Bt99BN92Wk+zcw=; b=s5uiRztdyFh6JDyd40dx7FiK2xryZfvNxNa5yLUMZy2UuqQfMcPYn2C1pkgcv5j+K5 fcVfStp5WpoID7ok/iCugO47uLMnkvaG8rVUqWUao66aqgzNIvjLivH2K75g/jDw0DW8 EumFSBxUYmWPo1rqCm54PdTnGNJWJp+pujPYjnSI6xf5u8akFwbqTJAWXMoQhOWN9o3r m9NaJeKtBZ1vTwMOOUtwMsdm/rx/RstnOGRWsBq9+YgqLOK+LGBuS3qunx0BP8N6iYWr W1qB6iZvyfO/aQWegx3BpKayaGtbsKL5DxFuLShdZ1bWOQ4KZz1tFDBxyM9JQRRusFHx lLJA== X-Gm-Message-State: ACrzQf0dkoA2ONEkZM4oGGas5IrMsTk0s6TUNpyu/xS9XRUYtM/bQG9H 77Dm6rjB8IJLo84FY4D7+QSK5Q== X-Google-Smtp-Source: AMsMyM4x6naz3nZNmfa/NQmCK6M9uAvOmZsk0imfFXfuxX/g0/1/X6ammsTwTiolzSPSQ6r1RUIKpQ== X-Received: by 2002:a65:6849:0:b0:461:8779:2452 with SMTP id q9-20020a656849000000b0046187792452mr1089661pgt.383.1666068961162; Mon, 17 Oct 2022 21:56:01 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:17a9:73b0:c262:eccd]) by smtp.gmail.com with ESMTPSA id p4-20020a170902e74400b0017b69f99321sm7549220plf.219.2022.10.17.21.55.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 21:56:00 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 8/9] zram: Clarify writeback_store() comment Date: Tue, 18 Oct 2022 13:55:32 +0900 Message-Id: <20221018045533.2396670-9-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog In-Reply-To: <20221018045533.2396670-1-senozhatsky@chromium.org> References: <20221018045533.2396670-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Re-phrase writeback BIO error comment. Reported-by: Andrew Morton Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5d760467e0bc..6b56f6f2ce82 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -758,8 +758,12 @@ static ssize_t writeback_store(struct device *dev, zram_clear_flag(zram, index, ZRAM_IDLE); zram_slot_unlock(zram, index); /* - * Return last IO error unless every IO were - * not suceeded. + * BIO errors are not fatal, we continue and simply + * attempt to writeback the remaining objects (pages). + * At the same time we need to signal user-space that + * some writes (at least one, but also could be all of + * them) were not successful and we do so by returning + * the most recent BIO error. */ ret =3D err; continue; --=20 2.38.0.413.g74048e4d9e-goog From nobody Tue Apr 7 15:56:55 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E89CC433FE for ; Tue, 18 Oct 2022 04:57:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230288AbiJRE5P (ORCPT ); Tue, 18 Oct 2022 00:57:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230162AbiJRE4c (ORCPT ); Tue, 18 Oct 2022 00:56:32 -0400 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E3DDA344A for ; Mon, 17 Oct 2022 21:56:06 -0700 (PDT) Received: by mail-pl1-x630.google.com with SMTP id c24so12738440pls.9 for ; Mon, 17 Oct 2022 21:56:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5M5hW54OP3FcHBFbXUW0BvyRXPUP5sF32vTgPzuINJw=; b=OpPjLqj+j0EF5h4VtuAZp6f+eGdl/5fSvLw56hi7HO01ZV6gqL3LhfNt4tuqvbFZWL B9Zo2V1IhStyc8xGnpweJqdk5JOPtoGtNyRZvNKx+nA/GCWy7rPkstTlJfT4GGbEyOzL khin31KlV69YEer+c0KA1hT4XmJY8qBrHD7bg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5M5hW54OP3FcHBFbXUW0BvyRXPUP5sF32vTgPzuINJw=; b=mSJf3fII1EzdyIqhmJCm1b5h3fZo7jkWmbPJjnQrG2wxPRQyXZBafUEUVD5/Lp9U6f Zp4T5u+7obVYrace19qMkzd6KQgYu4CaBZIneVEUaHXEbeB/5uyuCx8/GQN5vMHFwUNJ MDAgFnrn6M4O8HlN9k+1/Y2FLEzvTHdyMXvwl4E+tcjVbhxGOg2Yxx74ALst+KiBIao7 +6cNMI67zgzJoZFSsHqKMGNF9V/j9IVMdGZDk6fbFgBVtrR4ODg01SH6JFo/mSknDvNw GWek+I6KzwNl9wNKcd7gf1OqRwBOEd4ICm4PByUwPe0XvMmqa5KPBtdnwtqU8NOTcxep 1Zfg== X-Gm-Message-State: ACrzQf2n3RTG1f2NAhK10wQsVD1CrIl0/4MBosdvrhdUmFP0hCL7e9gO EWy/xNl8WsYNjGAmxSg896jCpw== X-Google-Smtp-Source: AMsMyM6MKM49SbK5pAtcJa9D193BGaOIJJk0SqOeD46rdK86Nh8j/RagnI/d8Ulw0ikvj1yWjSgb+w== X-Received: by 2002:a17:902:e5c3:b0:185:53ee:60e3 with SMTP id u3-20020a170902e5c300b0018553ee60e3mr1397002plf.71.1666068964804; Mon, 17 Oct 2022 21:56:04 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:17a9:73b0:c262:eccd]) by smtp.gmail.com with ESMTPSA id p4-20020a170902e74400b0017b69f99321sm7549220plf.219.2022.10.17.21.56.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 21:56:04 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 9/9] zram: Use IS_ERR_VALUE() to check for zs_malloc() errors Date: Tue, 18 Oct 2022 13:55:33 +0900 Message-Id: <20221018045533.2396670-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog In-Reply-To: <20221018045533.2396670-1-senozhatsky@chromium.org> References: <20221018045533.2396670-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Avoid type casts that are needed for IS_ERR() and use IS_ERR_VALUE() instead. Suggested-by: Andrew Morton Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 6b56f6f2ce82..422e3cfd2ddc 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1506,19 +1506,19 @@ static int __zram_bvec_write(struct zram *zram, str= uct bio_vec *bvec, * if we have a 'non-null' handle here then we are coming * from the slow path and handle has already been allocated. */ - if (IS_ERR((void *)handle)) + if (IS_ERR_VALUE(handle)) handle =3D zs_malloc(zram->mem_pool, comp_len, __GFP_KSWAPD_RECLAIM | __GFP_NOWARN | __GFP_HIGHMEM | __GFP_MOVABLE); - if (IS_ERR((void *)handle)) { + if (IS_ERR_VALUE(handle)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_ZCOMP]); atomic64_inc(&zram->stats.writestall); handle =3D zs_malloc(zram->mem_pool, comp_len, GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); - if (IS_ERR((void *)handle)) + if (IS_ERR_VALUE(handle)) return PTR_ERR((void *)handle); =20 if (comp_len !=3D PAGE_SIZE) @@ -1685,7 +1685,7 @@ static int zram_recompress(struct zram *zram, u32 ind= ex, struct page *page, __GFP_NOWARN | __GFP_HIGHMEM | __GFP_MOVABLE); - if (IS_ERR((void *)handle_next)) { + if (IS_ERR_VALUE(handle_next)) { zcomp_stream_put(zram->comps[ZRAM_SECONDARY_ZCOMP]); return PTR_ERR((void *)handle_next); } --=20 2.38.0.413.g74048e4d9e-goog