From nobody Sun Feb 8 18:20:20 2026 Received: from unicom146.biz-email.net (unicom146.biz-email.net [210.51.26.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25CA9224B1C for ; Fri, 16 May 2025 08:30:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.51.26.146 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747384225; cv=none; b=ru63znz4fCkofyZbDi+NL96+XNw+7WIyC/Yn0PGKz0SU6E6kCJheEV8/fj/JRC51LYxGuJEiU5ZauwQ68bH4+e3FIfQDLtAaoIarney5+U9tk9c9QslVaJKU4wdygiTlJ7kgan98t6AKQ7Zlj179Sq9daKpWU6ywbc9JQD8znz0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747384225; c=relaxed/simple; bh=ErY6nz941wKMOUhC2DrxeBtcdq+eadoPHdYMamSQA60=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=rLkkC6FRQgXupGLJE0a+3e9fvgOnyhqB11blw1y0zfiNHTgFNUj62xomIFrYqMWTRucim14kPbUhfdbHxaYGAGfOzNKaMCLIFdRcX9s+E/tMl8KTwE/BZKIK1mUr2chO1mgvvlbyyCdGQCuMGbML5Jr8nfJR/R8ZiuYLy7Tw6Zo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=inspur.com; spf=pass smtp.mailfrom=inspur.com; arc=none smtp.client-ip=210.51.26.146 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=inspur.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=inspur.com Received: from jtjnmail201610.home.langchao.com by unicom146.biz-email.net ((D)) with ASMTP (SSL) id 202505161629018466; Fri, 16 May 2025 16:29:01 +0800 Received: from localhost.localdomain (10.94.17.51) by jtjnmail201610.home.langchao.com (10.100.2.10) with Microsoft SMTP Server id 15.1.2507.39; Fri, 16 May 2025 16:29:01 +0800 From: Bo Liu To: , CC: , , Bo Liu Subject: [PATCH v3] erofs: support deflate decompress by using Intel QAT Date: Fri, 16 May 2025 04:26:34 -0400 Message-ID: <20250516082634.3801-1-liubo03@inspur.com> X-Mailer: git-send-email 2.18.2 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 tUid: 202551616290110875e0ba356e9856e2240661ed931c2 X-Abuse-Reports-To: service@corp-email.com Abuse-Reports-To: service@corp-email.com X-Complaints-To: service@corp-email.com X-Report-Abuse-To: service@corp-email.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This patch introdueces the use of the Intel QAT to decompress compressed data in the EROFS filesystem, aiming to improve the decompression speed of compressed datea. We created a 285MiB compressed file and then used the following command to create EROFS images with different cluster size. # mkfs.erofs -zdeflate,level=3D9 -C16384 fio command was used to test random read and small random read(~5%) and sequential read performance. # fio -filename=3Dtestfile -bs=3D4k -rw=3Dread -name=3Djob1 # fio -filename=3Dtestfile -bs=3D4k -rw=3Drandread -name=3Djob1 # fio -filename=3Dtestfile -bs=3D4k -rw=3Drandread --io_size=3D14m -n= ame=3Djob1 Here are some performance numbers for reference: Processors: Intel(R) Xeon(R) 6766E(144 core) Memory: 521 GiB |--------------------------------------------------------------------------= ---| | | Cluster size | sequential read | randread | small randread(5= %) | |-----------|--------------|-----------------|-----------|-----------------= ---| | Intel QAT | 4096 | 538 MiB/s | 112 MiB/s | 20.76 MiB/s = | | Intel QAT | 16384 | 699 MiB/s | 158 MiB/s | 21.02 MiB/s = | | Intel QAT | 65536 | 917 MiB/s | 278 MiB/s | 20.90 MiB/s = | | Intel QAT | 131072 | 1056 MiB/s | 351 MiB/s | 23.36 MiB/s = | | Intel QAT | 262144 | 1145 MiB/s | 431 MiB/s | 26.66 MiB/s = | | deflate | 4096 | 499 MiB/s | 108 MiB/s | 21.50 MiB/s = | | deflate | 16384 | 422 MiB/s | 125 MiB/s | 18.94 MiB/s = | | deflate | 65536 | 452 MiB/s | 159 MiB/s | 13.02 MiB/s = | | deflate | 131072 | 452 MiB/s | 177 MiB/s | 11.44 MiB/s = | | deflate | 262144 | 466 MiB/s | 194 MiB/s | 10.60 MiB/s = | Signed-off-by: Bo Liu --- v1: https://lore.kernel.org/linux-erofs/20250410042048.3044-1-liubo03@inspu= r.com/ v2: https://lore.kernel.org/linux-erofs/20250410042048.3044-1-liubo03@inspu= r.com/T/#t Changes since v2: - Add a new kernel config option CONFIG_EROFS_FS_ZIP_CRYPTO.=20 - Modify the sysfs interface to merge enable and disable into a single entry point, and add a read function to view the current state. - Optimize the relevant code,use the sg_alloc_table_from_pages_segment int= erface. fs/erofs/Kconfig | 14 +++ fs/erofs/Makefile | 1 + fs/erofs/compress.h | 14 +++ fs/erofs/decompressor_crypto.c | 199 ++++++++++++++++++++++++++++++++ fs/erofs/decompressor_deflate.c | 16 ++- fs/erofs/sysfs.c | 36 ++++++ 6 files changed, 279 insertions(+), 1 deletion(-) create mode 100644 fs/erofs/decompressor_crypto.c diff --git a/fs/erofs/Kconfig b/fs/erofs/Kconfig index 331e49cd1b8d..db49ae3c1922 100644 --- a/fs/erofs/Kconfig +++ b/fs/erofs/Kconfig @@ -144,6 +144,20 @@ config EROFS_FS_ZIP_ZSTD =20 If unsure, say N. =20 +config EROFS_FS_ZIP_CRYPTO + bool "EROFS hardware decompression support (crypto interface)" + depends on EROFS_FS_ZIP + help + Saying Y here includes support for reading EROFS file systems + containing crypto compressed data. It gives better decompression + speed than the software-implemented compression, and it costs + lower CPU overhead. + + Crypto support is an experimental feature for now and so most + file systems will be readable without selecting this option. + + If unsure, say N. + config EROFS_FS_ONDEMAND bool "EROFS fscache-based on-demand read support (deprecated)" depends on EROFS_FS diff --git a/fs/erofs/Makefile b/fs/erofs/Makefile index 4331d53c7109..247b263eb667 100644 --- a/fs/erofs/Makefile +++ b/fs/erofs/Makefile @@ -7,5 +7,6 @@ erofs-$(CONFIG_EROFS_FS_ZIP) +=3D decompressor.o zmap.o zda= ta.o zutil.o erofs-$(CONFIG_EROFS_FS_ZIP_LZMA) +=3D decompressor_lzma.o erofs-$(CONFIG_EROFS_FS_ZIP_DEFLATE) +=3D decompressor_deflate.o erofs-$(CONFIG_EROFS_FS_ZIP_ZSTD) +=3D decompressor_zstd.o +erofs-$(CONFIG_EROFS_FS_ZIP_CRYPTO) +=3D decompressor_crypto.o erofs-$(CONFIG_EROFS_FS_BACKED_BY_FILE) +=3D fileio.o erofs-$(CONFIG_EROFS_FS_ONDEMAND) +=3D fscache.o diff --git a/fs/erofs/compress.h b/fs/erofs/compress.h index 2704d7a592a5..356ae2473bd5 100644 --- a/fs/erofs/compress.h +++ b/fs/erofs/compress.h @@ -70,10 +70,24 @@ struct z_erofs_stream_dctx { bool bounced; /* is the bounce buffer used now? */ }; =20 +struct z_erofs_crypto_engine { + char *crypto_name; + bool enabled; + struct crypto_acomp *erofs_tfm; +}; + +extern struct z_erofs_crypto_engine z_erofs_crypto[Z_EROFS_COMPRESSION_MAX= ][2]; + int z_erofs_stream_switch_bufs(struct z_erofs_stream_dctx *dctx, void **ds= t, void **src, struct page **pgpl); int z_erofs_fixup_insize(struct z_erofs_decompress_req *rq, const char *pa= dbuf, unsigned int padbufsize); int __init z_erofs_init_decompressor(void); void z_erofs_exit_decompressor(void); +int z_erofs_crypto_decompress(struct z_erofs_decompress_req *rq, + struct crypto_acomp *erofs_tfm, struct page **pgpl); +struct crypto_acomp *z_erofs_crypto_get_engine(int type); +int z_erofs_crypto_enable_engine(const char *name); +void z_erofs_crypto_disable_engine(void); +int z_erofs_crypto_engine_format(char *buf); #endif diff --git a/fs/erofs/decompressor_crypto.c b/fs/erofs/decompressor_crypto.c new file mode 100644 index 000000000000..55f6a9e6dcf6 --- /dev/null +++ b/fs/erofs/decompressor_crypto.c @@ -0,0 +1,199 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +#include +#include + +#include "compress.h" + +static int z_erofs_crypto_decompress_mem(struct z_erofs_decompress_req *rq, + struct crypto_acomp *erofs_tfm) +{ + struct sg_table st_src, st_dst; + struct acomp_req *req; + struct crypto_wait wait; + u8 *headpage; + int ret; + + headpage =3D kmap_local_page(*rq->in); + ret =3D z_erofs_fixup_insize(rq, headpage + rq->pageofs_in, + min_t(unsigned int, rq->inputsize, + rq->sb->s_blocksize - rq->pageofs_in)); + kunmap_local(headpage); + if (ret) + return ret; + + req =3D acomp_request_alloc(erofs_tfm); + if (!req) { + erofs_err(rq->sb, "failed to alloc decompress request"); + return -ENOMEM; + } + + ret =3D sg_alloc_table_from_pages_segment(&st_src, + rq->in, rq->inpages, rq->pageofs_in, + rq->inputsize, UINT_MAX, GFP_KERNEL); + if (ret < 0) + goto failed_src_alloc; + + ret =3D sg_alloc_table_from_pages_segment(&st_dst, + rq->out, rq->outpages, rq->pageofs_out, + rq->outputsize, UINT_MAX, GFP_KERNEL); + if (ret < 0) + goto failed_dst_alloc; + + acomp_request_set_params(req, st_src.sgl, + st_dst.sgl, rq->inputsize, rq->outputsize); + + crypto_init_wait(&wait); + acomp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, + crypto_req_done, &wait); + + ret =3D crypto_wait_req(crypto_acomp_decompress(req), &wait); + if (ret < 0) { + erofs_err(rq->sb, "failed to decompress %d in[%u, %u] out[%u]", + ret, rq->inputsize, rq->pageofs_in, rq->outputsize); + ret =3D -EIO; + } else + ret =3D 0; + + sg_free_table(&st_dst); +failed_dst_alloc: + sg_free_table(&st_src); +failed_src_alloc: + acomp_request_free(req); + return ret; +} + +int z_erofs_crypto_decompress(struct z_erofs_decompress_req *rq, + struct crypto_acomp *erofs_tfm, struct page **pgpl) +{ + int i; + + for (i =3D 0; i < rq->outpages; i++) { + struct page *const page =3D rq->out[i]; + struct page *victim; + + if (!page) { + victim =3D __erofs_allocpage(pgpl, rq->gfp, true); + if (!victim) + return -ENOMEM; + set_page_private(victim, Z_EROFS_SHORTLIVED_PAGE); + rq->out[i] =3D victim; + } + } + + return z_erofs_crypto_decompress_mem(rq, erofs_tfm); +} + +struct crypto_acomp *z_erofs_crypto_get_engine(int type) +{ + int i =3D 0; + + while (z_erofs_crypto[type][i].crypto_name) { + if (z_erofs_crypto[type][i].enabled) + return z_erofs_crypto[type][i].erofs_tfm; + i++; + } + return NULL; +} + +static int z_erofs_crypto_get_compress_type(const char *name) +{ + int i, j; + + for (i =3D 0; i < Z_EROFS_COMPRESSION_MAX; i++) { + j =3D 0; + while (z_erofs_crypto[i][j].crypto_name) { + if (!strcmp(name, z_erofs_crypto[i][j].crypto_name)) + return i; + + j++; + } + } + return -EINVAL; +} + +int z_erofs_crypto_enable_engine(const char *name) +{ + int i =3D 0, type; + + type =3D z_erofs_crypto_get_compress_type(name); + if (type < 0) + return -EINVAL; + + while (z_erofs_crypto[type][i].crypto_name) { + if (!strcmp(z_erofs_crypto[type][i].crypto_name, name)) { + if (z_erofs_crypto[type][i].enabled) + break; + + z_erofs_crypto[type][i].erofs_tfm =3D + crypto_alloc_acomp(z_erofs_crypto[type][i].crypto_name, 0, 0); + if (IS_ERR(z_erofs_crypto[type][i].erofs_tfm)) { + z_erofs_crypto[type][i].erofs_tfm =3D NULL; + return -ENXIO; + } + z_erofs_crypto[type][i].enabled =3D true; + } else if (z_erofs_crypto[type][i].enabled) { + if (z_erofs_crypto[type][i].erofs_tfm) + crypto_free_acomp(z_erofs_crypto[type][i].erofs_tfm); + z_erofs_crypto[type][i].enabled =3D false; + } + i++; + } + return 0; +} + +void z_erofs_crypto_disable_engine(void) +{ + int i =3D 0, type; + + for (type =3D 0; type < Z_EROFS_COMPRESSION_MAX; type++) { + i =3D 0; + while (z_erofs_crypto[type][i].crypto_name) { + if (z_erofs_crypto[type][i].enabled && + z_erofs_crypto[type][i].erofs_tfm) { + crypto_free_acomp(z_erofs_crypto[type][i].erofs_tfm); + z_erofs_crypto[type][i].erofs_tfm =3D NULL; + z_erofs_crypto[type][i].enabled =3D false; + } + i++; + } + } +} + +int z_erofs_crypto_engine_format(char *buf) +{ + int type, i, len =3D 0; + + for (type =3D 0; type < Z_EROFS_COMPRESSION_MAX; type++) { + i =3D 0; + while (z_erofs_crypto[type][i].crypto_name) { + if (z_erofs_crypto[type][i].enabled) + len +=3D scnprintf(buf + len, PATH_MAX - len, "%s ", + z_erofs_crypto[type][i].crypto_name); + i++; + } + } + return len; +} + +struct z_erofs_crypto_engine z_erofs_crypto[Z_EROFS_COMPRESSION_MAX][2] = =3D { + [Z_EROFS_COMPRESSION_LZ4] =3D { + (struct z_erofs_crypto_engine) {NULL}, + (struct z_erofs_crypto_engine) {NULL}, + }, + [Z_EROFS_COMPRESSION_LZMA] =3D { + (struct z_erofs_crypto_engine) {NULL}, + (struct z_erofs_crypto_engine) {NULL}, + }, + [Z_EROFS_COMPRESSION_DEFLATE] =3D { + (struct z_erofs_crypto_engine) { + .crypto_name =3D "qat_deflate", + .enabled =3D false, + .erofs_tfm =3D NULL, + }, + (struct z_erofs_crypto_engine) {NULL}, + }, + [Z_EROFS_COMPRESSION_ZSTD] =3D { + (struct z_erofs_crypto_engine) {NULL}, + (struct z_erofs_crypto_engine) {NULL}, + }, +}; diff --git a/fs/erofs/decompressor_deflate.c b/fs/erofs/decompressor_deflat= e.c index c6908a487054..bebbf1eccd3d 100644 --- a/fs/erofs/decompressor_deflate.c +++ b/fs/erofs/decompressor_deflate.c @@ -97,7 +97,7 @@ static int z_erofs_load_deflate_config(struct super_block= *sb, return -ENOMEM; } =20 -static int z_erofs_deflate_decompress(struct z_erofs_decompress_req *rq, +static int __z_erofs_deflate_decompress(struct z_erofs_decompress_req *rq, struct page **pgpl) { struct super_block *sb =3D rq->sb; @@ -178,6 +178,20 @@ static int z_erofs_deflate_decompress(struct z_erofs_d= ecompress_req *rq, return err; } =20 +static int z_erofs_deflate_decompress(struct z_erofs_decompress_req *rq, + struct page **pgpl) +{ +#ifdef CONFIG_EROFS_FS_ZIP_CRYPTO + struct crypto_acomp *erofs_tfm =3D NULL; + + erofs_tfm =3D z_erofs_crypto_get_engine(Z_EROFS_COMPRESSION_DEFLATE); + if (erofs_tfm && !rq->partial_decoding) + return z_erofs_crypto_decompress(rq, erofs_tfm, pgpl); + else +#endif + return __z_erofs_deflate_decompress(rq, pgpl); +} + const struct z_erofs_decompressor z_erofs_deflate_decomp =3D { .config =3D z_erofs_load_deflate_config, .decompress =3D z_erofs_deflate_decompress, diff --git a/fs/erofs/sysfs.c b/fs/erofs/sysfs.c index dad4e6c6c155..00123c4d3196 100644 --- a/fs/erofs/sysfs.c +++ b/fs/erofs/sysfs.c @@ -7,12 +7,14 @@ #include =20 #include "internal.h" +#include "compress.h" =20 enum { attr_feature, attr_drop_caches, attr_pointer_ui, attr_pointer_bool, + attr_crypto, }; =20 enum { @@ -60,6 +62,9 @@ static struct erofs_attr erofs_attr_##_name =3D { \ EROFS_ATTR_RW_UI(sync_decompress, erofs_mount_opts); EROFS_ATTR_FUNC(drop_caches, 0200); #endif +#ifdef CONFIG_EROFS_FS_ZIP_CRYPTO +EROFS_ATTR_FUNC(crypto, 0644); +#endif =20 static struct attribute *erofs_attrs[] =3D { #ifdef CONFIG_EROFS_FS_ZIP @@ -95,6 +100,9 @@ static struct attribute *erofs_feat_attrs[] =3D { ATTR_LIST(fragments), ATTR_LIST(dedupe), ATTR_LIST(48bit), +#ifdef CONFIG_EROFS_FS_ZIP_CRYPTO + ATTR_LIST(crypto), +#endif NULL, }; ATTRIBUTE_GROUPS(erofs_feat); @@ -128,6 +136,10 @@ static ssize_t erofs_attr_show(struct kobject *kobj, if (!ptr) return 0; return sysfs_emit(buf, "%d\n", *(bool *)ptr); +#ifdef CONFIG_EROFS_FS_ZIP_CRYPTO + case attr_crypto: + return z_erofs_crypto_engine_format(buf); +#endif } return 0; } @@ -141,6 +153,10 @@ static ssize_t erofs_attr_store(struct kobject *kobj, = struct attribute *attr, unsigned char *ptr =3D __struct_ptr(sbi, a->struct_type, a->offset); unsigned long t; int ret; +#ifdef CONFIG_EROFS_FS_ZIP_CRYPTO + char *crypto_name; + size_t sz; +#endif =20 switch (a->attr_id) { case attr_pointer_ui: @@ -181,6 +197,26 @@ static ssize_t erofs_attr_store(struct kobject *kobj, = struct attribute *attr, if (t & 1) invalidate_mapping_pages(MNGD_MAPPING(sbi), 0, -1); return len; +#endif +#ifdef CONFIG_EROFS_FS_ZIP_CRYPTO + case attr_crypto: + buf =3D skip_spaces(buf); + sz =3D strlen(buf); + crypto_name =3D kstrdup(buf, GFP_KERNEL); + if (!crypto_name) + return -ENOMEM; + + /* ignore trailing newline */ + if (sz > 0 && crypto_name[sz - 1] =3D=3D '\n') + crypto_name[sz - 1] =3D 0x00; + + if (strlen(crypto_name) > 0) { + ret =3D z_erofs_crypto_enable_engine(crypto_name); + if (ret < 0) + return ret; + } else + z_erofs_crypto_disable_engine(); + return len; #endif } return 0; --=20 2.31.1