From nobody Mon Apr 29 13:19:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1549357870530880.5431854149165; Tue, 5 Feb 2019 01:11:10 -0800 (PST) Received: from localhost ([127.0.0.1]:56802 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gqwky-0001QW-IL for importer@patchew.org; Tue, 05 Feb 2019 04:11:04 -0500 Received: from eggs.gnu.org ([209.51.188.92]:39019) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gqwiu-0000NJ-Tb for qemu-devel@nongnu.org; Tue, 05 Feb 2019 04:08:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gqwit-00037L-Hq for qemu-devel@nongnu.org; Tue, 05 Feb 2019 04:08:56 -0500 Received: from relay.sw.ru ([185.231.240.75]:54986) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gqwin-0002yb-4Y; Tue, 05 Feb 2019 04:08:50 -0500 Received: from [10.94.4.71] (helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1gqwiV-0001FT-TN; Tue, 05 Feb 2019 12:08:32 +0300 From: Denis Plotnikov To: kwolf@redhat.com, mreitz@redhat.com Date: Tue, 5 Feb 2019 12:08:25 +0300 Message-Id: <20190205090825.14059-1-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH] [RFC] qcow2: add compression type feature X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, den@virtuozzo.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The patch adds some preparation parts for incompatible compression type feature into QCOW2 header that indicates that *all* compressed clusters must be (de)compressed using a certain compression type. It is implied that the compression type is set on the image creation and can be changed only later by image convertion, thus the only compression algorithm is used for the image. The plan is to add support for ZSTD and then may be something more effective in the future. ZSDT compression algorithm consumes 3-5 times less CPU power with a comparable comression ratio with zlib. It would be wise to use it for data compression f.e. for backups. The default compression is ZLIB. Signed-off-by: Denis Plotnikov --- block/qcow2.c | 25 +++++++++++++++++++++++++ block/qcow2.h | 26 ++++++++++++++++++++++---- 2 files changed, 47 insertions(+), 4 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 8c91b92865..cb3d6cc1c0 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -73,6 +73,7 @@ typedef struct { #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 #define QCOW2_EXT_MAGIC_CRYPTO_HEADER 0x0537be77 #define QCOW2_EXT_MAGIC_BITMAPS 0x23852875 +#define QCOW2_EXT_MAGIC_COMPRESSION_TYPE 0x434D5052 =20 static int coroutine_fn qcow2_co_preadv_compressed(BlockDriverState *bs, @@ -397,6 +398,9 @@ static int qcow2_read_extensions(BlockDriverState *bs, = uint64_t start_offset, #endif break; =20 + case QCOW2_EXT_MAGIC_COMPRESSION_TYPE: + /* Setting compression type to BDRVQcow2State->compression_typ= e */ + /* from the image header is going to be here */ default: /* unknown magic - save it in case we need to rewrite the head= er */ /* If you add a new feature, make sure to also update the fast @@ -2431,6 +2435,11 @@ int qcow2_update_header(BlockDriverState *bs) .bit =3D QCOW2_COMPAT_LAZY_REFCOUNTS_BITNR, .name =3D "lazy refcounts", }, + { + .type =3D QCOW2_FEAT_TYPE_INCOMPATIBLE, + .bit =3D QCOW2_INCOMPAT_COMPRESSION_TYPE_BITNR, + .name =3D "compression type", + }, }; =20 ret =3D header_ext_add(buf, QCOW2_EXT_MAGIC_FEATURE_TABLE, @@ -2461,6 +2470,22 @@ int qcow2_update_header(BlockDriverState *bs) buflen -=3D ret; } =20 + /* Compression type extension */ + if (s->compression_type !=3D 0) { + Qcow2CompressionTypeExt comp_header =3D { + .compression_type =3D cpu_to_be32(s->compression_type), + }; + ret =3D header_ext_add(buf, QCOW2_EXT_MAGIC_COMPRESSION_TYPE, + &comp_header, + cpu_to_be64(sizeof(comp_header)), + buflen); + if (ret < 0) { + goto fail; + } + buf +=3D ret; + buflen -=3D ret; + } + /* Keep unknown header extensions */ QLIST_FOREACH(uext, &s->unknown_header_ext, next) { ret =3D header_ext_add(buf, uext->magic, uext->data, uext->len, bu= flen); diff --git a/block/qcow2.h b/block/qcow2.h index 32cce9eee2..fdde5bbefd 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -112,6 +112,10 @@ #define QCOW2_OPT_REFCOUNT_CACHE_SIZE "refcount-cache-size" #define QCOW2_OPT_CACHE_CLEAN_INTERVAL "cache-clean-interval" =20 +/* Compression types */ +#define QCOW2_COMPRESSION_TYPE_ZLIB 0 +#define QCOW2_COMPRESSION_TYPE_ZSTD 1 + typedef struct QCowHeader { uint32_t magic; uint32_t version; @@ -197,10 +201,13 @@ enum { =20 /* Incompatible feature bits */ enum { - QCOW2_INCOMPAT_DIRTY_BITNR =3D 0, - QCOW2_INCOMPAT_CORRUPT_BITNR =3D 1, - QCOW2_INCOMPAT_DIRTY =3D 1 << QCOW2_INCOMPAT_DIRTY_BITNR, - QCOW2_INCOMPAT_CORRUPT =3D 1 << QCOW2_INCOMPAT_CORRUPT_BITNR, + QCOW2_INCOMPAT_DIRTY_BITNR =3D 0, + QCOW2_INCOMPAT_CORRUPT_BITNR =3D 1, + QCOW2_INCOMPAT_COMPRESSION_TYPE_BITNR =3D 2, + QCOW2_INCOMPAT_DIRTY =3D 1 << QCOW2_INCOMPAT_DIRTY_BI= TNR, + QCOW2_INCOMPAT_CORRUPT =3D 1 << QCOW2_INCOMPAT_CORRUPT_= BITNR, + QCOW2_INCOMPAT_COMPRESSION_TYPE =3D + 1 << QCOW2_INCOMPAT_COMPRESSION_TYPE_B= ITNR, =20 QCOW2_INCOMPAT_MASK =3D QCOW2_INCOMPAT_DIRTY | QCOW2_INCOMPAT_CORRUPT, @@ -256,6 +263,10 @@ typedef struct Qcow2BitmapHeaderExt { uint64_t bitmap_directory_offset; } QEMU_PACKED Qcow2BitmapHeaderExt; =20 +typedef struct Qcow2CompressionTypeExt { + uint32_t compression_type; +} QEMU_PACKED Qcow2CompressionTypeExt; + typedef struct BDRVQcow2State { int cluster_bits; int cluster_size; @@ -340,6 +351,13 @@ typedef struct BDRVQcow2State { =20 CoQueue compress_wait_queue; int nb_compress_threads; + /** + * Compression type used for the image. Default: 0 - ZLIB + * The image compression type is set on image creation. + * The only way to change the compression type is to convert the image + * with the desired compresion type set + */ + uint32_t compression_type; } BDRVQcow2State; =20 typedef struct Qcow2COWRegion { --=20 2.17.0