From nobody Wed Dec 17 10:48:54 2025 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7CE211D6DD4 for ; Mon, 2 Sep 2024 10:57:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725274672; cv=none; b=PBmbKpz/bHuv+5X5Wy3qs5OSL4LNTtmPPa5qG2R8RodddfKU6ZVdj7VOitqo9Grx7O2N56Pn1uUkO+5OI5ZOKjyjdmodSJLBs+FNtTrAaBV4NMNQFoy5OYyzLTe+OC9dHQ1kPDJdy9SV5ai0U6tILXHiUa/jorUuuO+cpKT3ycc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725274672; c=relaxed/simple; bh=LxK7yxVOuU/Mf7+dft522QNvM0qcckmP8rj7nOwAApM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=osObUw71EjJpM8cD9aCLl3dIpDXa23R5zP9inlATYyZra8uHRR/Lcn7K2751KII/sWuFusSMn/WdMdxZH3GxX6+O+H2dsNP2aKFM+90DoyzGvDSSu1LDOS9FzD7XP3DNDYXwxKIf8EYUMvOAiEfBWyTM3D/DGoWWYmF/yBBHEgg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=QYw+kCS/; arc=none smtp.client-ip=209.85.215.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="QYw+kCS/" Received: by mail-pg1-f193.google.com with SMTP id 41be03b00d2f7-7cda0453766so1297775a12.2 for ; Mon, 02 Sep 2024 03:57:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1725274670; x=1725879470; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cbR1gFo0yq5XvwmrFtnpkMRAOk31bpE/Kw6nP8TwsWk=; b=QYw+kCS/twM1soIvTM8zf+jEQuFYosUPLpI2tUwHtZ8arinyPZ1Z3LbJbHJwtyoE6d G2cZqU6ZolMTLfrB9fDhas5LHobhwJ5yIq/xdllMyAPrrtdTkIOU0XOmw+Royn4Z6MuT aDGhVCjGmbonkk0nUxw23GxoCWszqQ73B/Oa8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725274670; x=1725879470; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cbR1gFo0yq5XvwmrFtnpkMRAOk31bpE/Kw6nP8TwsWk=; b=U1OOjhg9YWFBW0ATC6vNFyT7b/LIsKpDYusIfnzZjAa42QyKVsdZ7Zd3UDTIiRu8TM 4TPlG5WBwmwulA1eCHpTypQ7XUibfwgguBB6doL6aGAix4xPn2jZoE66eU+fEMMPXUxJ UChAHEWuGCRDnUfgLFskr04bh2fWZmTFZS9fwRr0Z5vmeV4f2I63yTMKi2ymr5FJI28z sGLSKaUi2Yt/8uXpuRQxUXOlEXEQHX3QuNjZT81ELySimC/45N/GN5JYAbdrjsDeO4re aV78Eo0eq8Nba7LCOkr47vPs/3aQswlMjPwyI82lsIS59+dG5Ipak9SIkYhiFUt95b77 XEzQ== X-Forwarded-Encrypted: i=1; AJvYcCXlVuMUZ8SyIRCEqg4VMftdkHcAryMeDiGdLmyxMQyJNPIqzyHBdewugdgC2ti4ZyeeD8c4LE/38kAcXMo=@vger.kernel.org X-Gm-Message-State: AOJu0YzEOMyY6rqQ7A0t2B47sRBg2NJsYrDs234/WMLItLOOsBqbtPuX Iit0rK0NibghmBmo1XZ6s51x0AqoyiVb+XZ4jnooak0xqkcwLdDrKapDkhXu5WEyMol03D0oV3v eWbXn X-Google-Smtp-Source: AGHT+IFwfHrqJ2bHzNS/FwApFbvO9Bo6qKdoRwnyVi41LA71Z2a0ug42iBISEEf2FYdsuSJN+d+54A== X-Received: by 2002:a05:6a21:151a:b0:1c6:ae03:6882 with SMTP id adf61e73a8af0-1cce10038ebmr14285361637.9.1725274669771; Mon, 02 Sep 2024 03:57:49 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:43a1:71e3:7464:e91b]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-715e55a51e9sm6633769b3a.78.2024.09.02.03.57.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Sep 2024 03:57:49 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv7 22/24] zram: add dictionary support to zstd backend Date: Mon, 2 Sep 2024 19:56:10 +0900 Message-ID: <20240902105656.1383858-23-senozhatsky@chromium.org> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog In-Reply-To: <20240902105656.1383858-1-senozhatsky@chromium.org> References: <20240902105656.1383858-1-senozhatsky@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This adds support for pre-trained zstd dictionaries [1] Dictionary is setup in params once (per-comp) and loaded to Cctx and Dctx by reference, so we don't allocate extra memory. TEST =3D=3D=3D=3D *** zstd /sys/block/zram0/mm_stat 1750654976 504565092 514203648 0 514203648 1 0 3420= 4 34204 *** zstd dict=3D/etc/zstd-dict-amd64 /sys/block/zram0/mm_stat 1750638592 465851259 475373568 0 475373568 1 0 3418= 5 34185 *** zstd level=3D8 dict=3D/etc/zstd-dict-amd64 /sys/block/zram0/mm_stat 1750642688 430765171 439955456 0 439955456 1 0 3418= 5 34185 [1] https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md#dictionary= -builder Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/backend_zstd.c | 150 ++++++++++++++++++++++++------ 1 file changed, 124 insertions(+), 26 deletions(-) diff --git a/drivers/block/zram/backend_zstd.c b/drivers/block/zram/backend= _zstd.c index 5b33daf4f645..1184c0036f44 100644 --- a/drivers/block/zram/backend_zstd.c +++ b/drivers/block/zram/backend_zstd.c @@ -15,29 +15,87 @@ struct zstd_ctx { }; =20 struct zstd_params { + zstd_custom_mem custom_mem; + zstd_cdict *cdict; + zstd_ddict *ddict; zstd_parameters cprm; }; =20 +/* + * For C/D dictionaries we need to provide zstd with zstd_custom_mem, + * which zstd uses internally to allocate/free memory when needed. + * + * This means that allocator.customAlloc() can be called from zcomp_compre= ss() + * under local-lock (per-CPU compression stream), in which case we must use + * GFP_ATOMIC. + * + * Another complication here is that we can be configured as a swap device. + */ +static void *zstd_custom_alloc(void *opaque, size_t size) +{ + if (!preemptible()) + return kvzalloc(size, GFP_ATOMIC); + + return kvzalloc(size, __GFP_KSWAPD_RECLAIM | __GFP_NOWARN); +} + +static void zstd_custom_free(void *opaque, void *address) +{ + kvfree(address); +} + static void zstd_release_params(struct zcomp_params *params) { - kfree(params->drv_data); + struct zstd_params *zp =3D params->drv_data; + + params->drv_data =3D NULL; + if (!zp) + return; + + zstd_free_cdict(zp->cdict); + zstd_free_ddict(zp->ddict); + kfree(zp); } =20 static int zstd_setup_params(struct zcomp_params *params) { + zstd_compression_parameters prm; struct zstd_params *zp; =20 zp =3D kzalloc(sizeof(*zp), GFP_KERNEL); if (!zp) return -ENOMEM; =20 + params->drv_data =3D zp; if (params->level =3D=3D ZCOMP_PARAM_NO_LEVEL) params->level =3D zstd_default_clevel(); =20 zp->cprm =3D zstd_get_params(params->level, PAGE_SIZE); - params->drv_data =3D zp; + + zp->custom_mem.customAlloc =3D zstd_custom_alloc; + zp->custom_mem.customFree =3D zstd_custom_free; + + prm =3D zstd_get_cparams(params->level, PAGE_SIZE, + params->dict_sz); + + zp->cdict =3D zstd_create_cdict_byreference(params->dict, + params->dict_sz, + prm, + zp->custom_mem); + if (!zp->cdict) + goto error; + + zp->ddict =3D zstd_create_ddict_byreference(params->dict, + params->dict_sz, + zp->custom_mem); + if (!zp->ddict) + goto error; =20 return 0; + +error: + zstd_release_params(params); + return -EINVAL; } =20 static void zstd_destroy(struct zcomp_ctx *ctx) @@ -47,8 +105,23 @@ static void zstd_destroy(struct zcomp_ctx *ctx) if (!zctx) return; =20 - vfree(zctx->cctx_mem); - vfree(zctx->dctx_mem); + /* + * If ->cctx_mem and ->dctx_mem were allocated then we didn't use + * C/D dictionary and ->cctx / ->dctx were "embedded" into these + * buffers. + * + * If otherwise then we need to explicitly release ->cctx / ->dctx. + */ + if (zctx->cctx_mem) + vfree(zctx->cctx_mem); + else + zstd_free_cctx(zctx->cctx); + + if (zctx->dctx_mem) + vfree(zctx->dctx_mem); + else + zstd_free_dctx(zctx->dctx); + kfree(zctx); } =20 @@ -63,28 +136,41 @@ static int zstd_create(struct zcomp_params *params, st= ruct zcomp_ctx *ctx) return -ENOMEM; =20 ctx->context =3D zctx; - prm =3D zstd_get_params(params->level, PAGE_SIZE); - sz =3D zstd_cctx_workspace_bound(&prm.cParams); - zctx->cctx_mem =3D vzalloc(sz); - if (!zctx->cctx_mem) - goto error; - - zctx->cctx =3D zstd_init_cctx(zctx->cctx_mem, sz); - if (!zctx->cctx) - goto error; - - sz =3D zstd_dctx_workspace_bound(); - zctx->dctx_mem =3D vzalloc(sz); - if (!zctx->dctx_mem) - goto error; - - zctx->dctx =3D zstd_init_dctx(zctx->dctx_mem, sz); - if (!zctx->dctx) - goto error; + if (params->dict_sz =3D=3D 0) { + prm =3D zstd_get_params(params->level, PAGE_SIZE); + sz =3D zstd_cctx_workspace_bound(&prm.cParams); + zctx->cctx_mem =3D vzalloc(sz); + if (!zctx->cctx_mem) + goto error; + + zctx->cctx =3D zstd_init_cctx(zctx->cctx_mem, sz); + if (!zctx->cctx) + goto error; + + sz =3D zstd_dctx_workspace_bound(); + zctx->dctx_mem =3D vzalloc(sz); + if (!zctx->dctx_mem) + goto error; + + zctx->dctx =3D zstd_init_dctx(zctx->dctx_mem, sz); + if (!zctx->dctx) + goto error; + } else { + struct zstd_params *zp =3D params->drv_data; + + zctx->cctx =3D zstd_create_cctx_advanced(zp->custom_mem); + if (!zctx->cctx) + goto error; + + zctx->dctx =3D zstd_create_dctx_advanced(zp->custom_mem); + if (!zctx->dctx) + goto error; + } =20 return 0; =20 error: + zstd_release_params(params); zstd_destroy(ctx); return -EINVAL; } @@ -96,8 +182,14 @@ static int zstd_compress(struct zcomp_params *params, s= truct zcomp_ctx *ctx, struct zstd_ctx *zctx =3D ctx->context; size_t ret; =20 - ret =3D zstd_compress_cctx(zctx->cctx, req->dst, req->dst_len, - req->src, req->src_len, &zp->cprm); + if (params->dict_sz =3D=3D 0) + ret =3D zstd_compress_cctx(zctx->cctx, req->dst, req->dst_len, + req->src, req->src_len, &zp->cprm); + else + ret =3D zstd_compress_using_cdict(zctx->cctx, req->dst, + req->dst_len, req->src, + req->src_len, + zp->cdict); if (zstd_is_error(ret)) return -EINVAL; req->dst_len =3D ret; @@ -107,11 +199,17 @@ static int zstd_compress(struct zcomp_params *params,= struct zcomp_ctx *ctx, static int zstd_decompress(struct zcomp_params *params, struct zcomp_ctx *= ctx, struct zcomp_req *req) { + struct zstd_params *zp =3D params->drv_data; struct zstd_ctx *zctx =3D ctx->context; size_t ret; =20 - ret =3D zstd_decompress_dctx(zctx->dctx, req->dst, req->dst_len, - req->src, req->src_len); + if (params->dict_sz =3D=3D 0) + ret =3D zstd_decompress_dctx(zctx->dctx, req->dst, req->dst_len, + req->src, req->src_len); + else + ret =3D zstd_decompress_using_ddict(zctx->dctx, req->dst, + req->dst_len, req->src, + req->src_len, zp->ddict); if (zstd_is_error(ret)) return -EINVAL; return 0; --=20 2.46.0.469.g59c65b2a67-goog