From nobody Fri May 3 10:13:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498734126389425.5989056664206; Thu, 29 Jun 2017 04:02:06 -0700 (PDT) Received: from localhost ([::1]:38330 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQXD0-0002be-VE for importer@patchew.org; Thu, 29 Jun 2017 07:02:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51640) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQX8d-00071m-Kg for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQX8W-0007vR-Hq for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:28 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:42868 helo=mx01.kamp.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQX8W-0007r3-8N for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:24 -0400 Received: (qmail 8776 invoked by uid 89); 29 Jun 2017 10:57:19 -0000 Received: from [195.62.97.28] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.99.2/23517. avast: 1.2.2/17010300. spamassassin: 3.4.1. Clear:RC:1(195.62.97.28):. Processed in 0.296473 secs); 29 Jun 2017 10:57:19 -0000 Received: from smtp.kamp.de (HELO submission.kamp.de) ([195.62.97.28]) by mx01.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted); 29 Jun 2017 10:57:16 -0000 Received: (qmail 12757 invoked from network); 29 Jun 2017 10:57:15 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 29 Jun 2017 10:57:15 -0000 Received: by lieven-pc (Postfix, from userid 1000) id 9EC69208AB; Thu, 29 Jun 2017 12:57:15 +0200 (CEST) X-GL_Whitelist: yes From: Peter Lieven To: qemu-block@nongnu.org Date: Thu, 29 Jun 2017 12:57:04 +0200 Message-Id: <1498733831-15254-2-git-send-email-pl@kamp.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498733831-15254-1-git-send-email-pl@kamp.de> References: <1498733831-15254-1-git-send-email-pl@kamp.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:248:0:51::16 Subject: [Qemu-devel] [PATCH V2 1/8] docs: add compress format extension to qcow2 spec X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, mreitz@redhat.com, den@openvz.org, lersek@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Peter Lieven --- docs/interop/qcow2.txt | 43 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 80cdfd0..c01daf3 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -85,7 +85,12 @@ in the description of a field. be written to (unless for regaining consistency). =20 - Bits 2-63: Reserved (set to 0) + Bit 2: Compress format bit. If and only if this = bit + is set then the compress format extension + MUST be present and MUST be parsed and che= cked + for compatibility. + + Bits 3-63: Reserved (set to 0) =20 80 - 87: compatible_features Bitmask of compatible features. An implementation can @@ -135,6 +140,7 @@ be stored. Each extension has a structure like the foll= owing: 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension + 0xC03183A3 - Compress format extension other - Unknown header extension, can be safe= ly ignored =20 @@ -208,6 +214,41 @@ The fields of the bitmaps extension are: starts. Must be aligned to a cluster boundary. =20 =20 +=3D=3D Compress format extension =3D=3D + +The compress format extension is an optional header extension. It provides +the ability to specify the compress algorithm and compress parameters +that are used for compressed clusters. This new header MUST be present if +the incompatible-feature bit "compress format bit" is set and MUST be abse= nt +otherwise. + +The fields of the compress format extension are: + + Byte 0 - 15: compress_format_name (padded with zeros, but not + necessarily null terminated if it has full length) + + 16: compress_level (uint8_t) + 0 =3D default compress level + 1 =3D lowest compress level + x =3D highest compress level (the highest compress + level may vary for different compress formats) + + 17 - 19: Reserved for future use, must be zero. + + 20 - 23: extra_data_size + Size of compress format specific extra data. + For now, as no format specifies extra data, + extra_data_size is reserved and should be zero. + + variable: extra_data + Extra data with additional parameters for the compress + format, occupying extra_data_size bytes. + + variable: Padding to round up the size of compress format extensi= on + to the next multiple of 8. All bytes of the padding mus= t be + zero. + + =3D=3D Host cluster management =3D=3D =20 qcow2 manages the allocation of host clusters by maintaining a reference c= ount --=20 1.9.1 From nobody Fri May 3 10:13:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498734119296268.60282873039887; Thu, 29 Jun 2017 04:01:59 -0700 (PDT) Received: from localhost ([::1]:38329 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQXCt-0002Wc-Fq for importer@patchew.org; Thu, 29 Jun 2017 07:01:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51644) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQX8d-00071t-A8 for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQX8a-0007zp-2r for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:29 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:49001 helo=mx01.kamp.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQX8Z-0007vC-Qy for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:28 -0400 Received: (qmail 8894 invoked by uid 89); 29 Jun 2017 10:57:23 -0000 Received: from [195.62.97.28] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.99.2/23517. avast: 1.2.2/17010300. spamassassin: 3.4.1. Clear:RC:1(195.62.97.28):. Processed in 0.236644 secs); 29 Jun 2017 10:57:23 -0000 Received: from smtp.kamp.de (HELO submission.kamp.de) ([195.62.97.28]) by mx01.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted); 29 Jun 2017 10:57:20 -0000 Received: (qmail 12761 invoked from network); 29 Jun 2017 10:57:15 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 29 Jun 2017 10:57:15 -0000 Received: by lieven-pc (Postfix, from userid 1000) id A286926C35; Thu, 29 Jun 2017 12:57:15 +0200 (CEST) X-GL_Whitelist: yes From: Peter Lieven To: qemu-block@nongnu.org Date: Thu, 29 Jun 2017 12:57:05 +0200 Message-Id: <1498733831-15254-3-git-send-email-pl@kamp.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498733831-15254-1-git-send-email-pl@kamp.de> References: <1498733831-15254-1-git-send-email-pl@kamp.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:248:0:51::16 Subject: [Qemu-devel] [PATCH V2 2/8] qapi: add compress parameters to Qcow2 Blockdev options X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, mreitz@redhat.com, den@openvz.org, lersek@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Peter Lieven --- qapi/block-core.json | 44 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index f85c223..1574ffb 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2282,6 +2282,43 @@ 'mode': 'Qcow2OverlapCheckMode' } } =20 ## +# @Qcow2CompressFormat: +# @zlib: standard zlib deflate compression +# +# Since: 2.10 +## +{ 'enum': 'Qcow2CompressFormat', + 'data': [ 'zlib' ] } + +## +# @Qcow2CompressZLib: +# +# Since: 2.10 +## +{ 'struct': 'Qcow2CompressZLib', + 'data': { } } + +## +# @Qcow2Compress: +# +# Specifies the compression format and compression level that should +# be used for compressed Qcow2 clusters. +# +# @format: specifies the compression format to use. (defaults to zlib) +# +# @level: specifies the compression level. 0 =3D default compression, +# 1 =3D fastest compression, x =3D highest compresion (x may very = between +# different compression formats) +# +# Since: 2.10 +## +{ 'union': 'Qcow2Compress', + 'base': { 'format': 'Qcow2CompressFormat', + 'level': 'uint8' }, + 'discriminator': 'format', + 'data': { 'zlib': 'Qcow2CompressZLib' } } + +## # @BlockdevOptionsQcow2: # # Driver specific block device options for qcow2. @@ -2316,6 +2353,10 @@ # caches. The interval is in seconds. The default = value # is 0 and it disables this feature (since 2.5) # +# @compress: which format and compression level to use for +# compressed clusters. Defaults to zlib with defau= lt +# compression level (since 2.10) +# # Since: 2.9 ## { 'struct': 'BlockdevOptionsQcow2', @@ -2328,7 +2369,8 @@ '*cache-size': 'int', '*l2-cache-size': 'int', '*refcount-cache-size': 'int', - '*cache-clean-interval': 'int' } } + '*cache-clean-interval': 'int', + '*compress': 'Qcow2Compress' } } =20 =20 ## --=20 1.9.1 From nobody Fri May 3 10:13:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149873397003052.749822211143055; Thu, 29 Jun 2017 03:59:30 -0700 (PDT) Received: from localhost ([::1]:38314 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQXAV-0000TY-I4 for importer@patchew.org; Thu, 29 Jun 2017 06:59:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51656) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQX8d-00071z-KL for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQX8a-000803-K0 for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:30 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:36025 helo=mx01.kamp.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQX8a-0007vc-AY for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:28 -0400 Received: (qmail 8964 invoked by uid 89); 29 Jun 2017 10:57:24 -0000 Received: from [195.62.97.28] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.99.2/23517. avast: 1.2.2/17010300. spamassassin: 3.4.1. Clear:RC:1(195.62.97.28):. Processed in 0.29144 secs); 29 Jun 2017 10:57:24 -0000 Received: from smtp.kamp.de (HELO submission.kamp.de) ([195.62.97.28]) by mx01.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted); 29 Jun 2017 10:57:20 -0000 Received: (qmail 12764 invoked from network); 29 Jun 2017 10:57:15 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 29 Jun 2017 10:57:15 -0000 Received: by lieven-pc (Postfix, from userid 1000) id A4B2526C38; Thu, 29 Jun 2017 12:57:15 +0200 (CEST) X-GL_Whitelist: yes From: Peter Lieven To: qemu-block@nongnu.org Date: Thu, 29 Jun 2017 12:57:06 +0200 Message-Id: <1498733831-15254-4-git-send-email-pl@kamp.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498733831-15254-1-git-send-email-pl@kamp.de> References: <1498733831-15254-1-git-send-email-pl@kamp.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:248:0:51::16 Subject: [Qemu-devel] [PATCH V2 3/8] block/qcow2: parse compress create options X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, mreitz@redhat.com, den@openvz.org, lersek@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" this adds parsing and validation for the compress create options. They are only validated but not yet used. Signed-off-by: Peter Lieven --- block/qcow2.c | 56 +++++++++++++++++++++++++++++++++++++++++++= ++-- block/qcow2.h | 9 ++++++++ include/block/block_int.h | 2 ++ 3 files changed, 65 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 2f94f03..308121a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2144,7 +2144,8 @@ static int qcow2_create2(const char *filename, int64_= t total_size, const char *backing_file, const char *backing_for= mat, int flags, size_t cluster_size, PreallocMode prea= lloc, QemuOpts *opts, int version, int refcount_order, - Error **errp) + const char *compress_format_name, + uint8_t compress_level, Error **errp) { int cluster_bits; QDict *options; @@ -2390,11 +2391,24 @@ out: return ret; } =20 +static int qcow2_compress_format_from_name(char *fmt) +{ + if (!fmt || !fmt[0]) { + return QCOW2_COMPRESS_ZLIB_COMPAT; + } else if (g_str_equal(fmt, "zlib")) { + return QCOW2_COMPRESS_ZLIB; + } else { + return -EINVAL; + } +} + static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp) { char *backing_file =3D NULL; char *backing_fmt =3D NULL; char *buf =3D NULL; + char *compress_format_name =3D NULL; + uint64_t compress_level =3D 0; uint64_t size =3D 0; int flags =3D 0; size_t cluster_size =3D DEFAULT_CLUSTER_SIZE; @@ -2475,15 +2489,40 @@ static int qcow2_create(const char *filename, QemuO= pts *opts, Error **errp) =20 refcount_order =3D ctz32(refcount_bits); =20 + compress_format_name =3D qemu_opt_get_del(opts, + BLOCK_OPT_COMPRESS_FORMAT); + ret =3D qcow2_compress_format_from_name(compress_format_name); + if (ret < 0) { + error_setg(errp, "Compress format '%s' is not supported", + compress_format_name); + goto finish; + } + compress_level =3D qemu_opt_get_number_del(opts, BLOCK_OPT_COMPRESS_LE= VEL, + compress_level); + if (ret =3D=3D QCOW2_COMPRESS_ZLIB_COMPAT && compress_level > 0) { + error_setg(errp, "Compress level can only be defined in conjunctio= n" + " with compress format"); + ret =3D -EINVAL; + goto finish; + } + if ((ret =3D=3D QCOW2_COMPRESS_ZLIB && compress_level > 9) || + compress_level > 0xff) { + error_setg(errp, "Compress level %" PRIu64 " is not supported for" + " format '%s'", compress_level, compress_format_name); + ret =3D -EINVAL; + goto finish; + } + ret =3D qcow2_create2(filename, size, backing_file, backing_fmt, flags, cluster_size, prealloc, opts, version, refcount_or= der, - &local_err); + compress_format_name, compress_level, &local_err); error_propagate(errp, local_err); =20 finish: g_free(backing_file); g_free(backing_fmt); g_free(buf); + g_free(compress_format_name); return ret; } =20 @@ -3458,6 +3497,19 @@ static QemuOptsList qcow2_create_opts =3D { .help =3D "Width of a reference count entry in bits", .def_value_str =3D "16" }, + { + .name =3D BLOCK_OPT_COMPRESS_FORMAT, + .type =3D QEMU_OPT_STRING, + .help =3D "Compress format used for compressed clusters (zlib)= ", + .def_value_str =3D "" + }, + { + .name =3D BLOCK_OPT_COMPRESS_LEVEL, + .type =3D QEMU_OPT_NUMBER, + .help =3D "Compress level used for compressed clusters (0 =3D = default," + " 1=3Dfastest, x=3Dbest; x varies depending on compres= s.format)", + .def_value_str =3D "0" + }, { /* end of list */ } } }; diff --git a/block/qcow2.h b/block/qcow2.h index 87b15eb..d21da33 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -171,6 +171,15 @@ typedef struct Qcow2UnknownHeaderExtension { } Qcow2UnknownHeaderExtension; =20 enum { + /* QCOW2_COMPRESS_ZLIB_COMPAT specifies to use the old standard + * zlib compression with a smaller window size that is compatible with + * old QEMU versions. This compression is used if no compression format + * is specified at create time */ + QCOW2_COMPRESS_ZLIB_COMPAT =3D 0, + QCOW2_COMPRESS_ZLIB =3D 1, +}; + +enum { QCOW2_FEAT_TYPE_INCOMPATIBLE =3D 0, QCOW2_FEAT_TYPE_COMPATIBLE =3D 1, QCOW2_FEAT_TYPE_AUTOCLEAR =3D 2, diff --git a/include/block/block_int.h b/include/block/block_int.h index 15fa602..49811b0 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -57,6 +57,8 @@ #define BLOCK_OPT_NOCOW "nocow" #define BLOCK_OPT_OBJECT_SIZE "object_size" #define BLOCK_OPT_REFCOUNT_BITS "refcount_bits" +#define BLOCK_OPT_COMPRESS_FORMAT "compress.format" +#define BLOCK_OPT_COMPRESS_LEVEL "compress.level" =20 #define BLOCK_PROBE_BUF_SIZE 512 =20 --=20 1.9.1 From nobody Fri May 3 10:13:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498733987320665.6721514224013; Thu, 29 Jun 2017 03:59:47 -0700 (PDT) Received: from localhost ([::1]:38317 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQXAm-0000kQ-P0 for importer@patchew.org; Thu, 29 Jun 2017 06:59:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51824) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQX8k-0007A0-St for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQX8h-00089X-8m for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:38 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:60888 helo=mx01.kamp.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQX8g-00087Z-V4 for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:35 -0400 Received: (qmail 9380 invoked by uid 89); 29 Jun 2017 10:57:33 -0000 Received: from [195.62.97.28] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.99.2/23517. avast: 1.2.2/17010300. spamassassin: 3.4.1. Clear:RC:1(195.62.97.28):. Processed in 0.243147 secs); 29 Jun 2017 10:57:33 -0000 Received: from smtp.kamp.de (HELO submission.kamp.de) ([195.62.97.28]) by mx01.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted); 29 Jun 2017 10:57:30 -0000 Received: (qmail 12772 invoked from network); 29 Jun 2017 10:57:15 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 29 Jun 2017 10:57:15 -0000 Received: by lieven-pc (Postfix, from userid 1000) id A78CE26C5D; Thu, 29 Jun 2017 12:57:15 +0200 (CEST) X-GL_Whitelist: yes From: Peter Lieven To: qemu-block@nongnu.org Date: Thu, 29 Jun 2017 12:57:07 +0200 Message-Id: <1498733831-15254-5-git-send-email-pl@kamp.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498733831-15254-1-git-send-email-pl@kamp.de> References: <1498733831-15254-1-git-send-email-pl@kamp.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:248:0:51::16 Subject: [Qemu-devel] [PATCH V2 4/8] qemu-img: add documentation for compress settings X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, mreitz@redhat.com, den@openvz.org, lersek@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Peter Lieven --- qemu-img.texi | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/qemu-img.texi b/qemu-img.texi index 5b925ec..430f0b9 100644 --- a/qemu-img.texi +++ b/qemu-img.texi @@ -621,6 +621,27 @@ file which is COW and has data blocks already, it coul= dn't be changed to NOCOW by setting @code{nocow=3Don}. One can issue @code{lsattr filename} to chec= k if the NOCOW flag is set or not (Capital 'C' is NOCOW flag). =20 +@item compress.format +Defines which compression algorithm is should be used for compressed clust= ers. +The following options are available if support for the respective libraries +has been enabled at compile time: + + zlib Uses standard zlib compression + +The compression algorithm can only be defined at image create time and can= not +be changed later. + +Note: defining a compression format will result in the compression format + extension being written to the Qcow2 image. Older versions of QEMU w= ill + not be able to open images with this extension. + +@item compress.level +Defines which compression level is used for the selected compression forma= t. +The default of @code{compress.level=3D0} will use the default compression = level +for the format. Alternate values range from 1 for fastest compression to +x for the best compression (x max vary between compression formats). This = is +always a trade in of compression speed against compressed size. + @end table =20 @item Other --=20 1.9.1 From nobody Fri May 3 10:13:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498734255315979.0507709472321; Thu, 29 Jun 2017 04:04:15 -0700 (PDT) Received: from localhost ([::1]:38338 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQXF5-0004Dm-0c for importer@patchew.org; Thu, 29 Jun 2017 07:04:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51786) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQX8h-00074z-8Z for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQX8d-000848-PG for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:35 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:34450 helo=mx01.kamp.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQX8d-00081f-EV for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:31 -0400 Received: (qmail 9193 invoked by uid 89); 29 Jun 2017 10:57:28 -0000 Received: from [195.62.97.28] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.99.2/23517. avast: 1.2.2/17010300. spamassassin: 3.4.1. Clear:RC:1(195.62.97.28):. Processed in 0.351141 secs); 29 Jun 2017 10:57:28 -0000 Received: from smtp.kamp.de (HELO submission.kamp.de) ([195.62.97.28]) by mx01.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted); 29 Jun 2017 10:57:25 -0000 Received: (qmail 12783 invoked from network); 29 Jun 2017 10:57:15 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 29 Jun 2017 10:57:15 -0000 Received: by lieven-pc (Postfix, from userid 1000) id A9B6126C28; Thu, 29 Jun 2017 12:57:15 +0200 (CEST) X-GL_Whitelist: yes From: Peter Lieven To: qemu-block@nongnu.org Date: Thu, 29 Jun 2017 12:57:08 +0200 Message-Id: <1498733831-15254-6-git-send-email-pl@kamp.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498733831-15254-1-git-send-email-pl@kamp.de> References: <1498733831-15254-1-git-send-email-pl@kamp.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:248:0:51::16 Subject: [Qemu-devel] [PATCH V2 5/8] block/qcow2: read and write the compress format extension X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, mreitz@redhat.com, den@openvz.org, lersek@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" we now read the extension on open and write it on update, but do not yet use it. Signed-off-by: Peter Lieven --- block/qcow2.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++----= ---- block/qcow2.h | 23 +++++++++++--- 2 files changed, 104 insertions(+), 19 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 308121a..39a8afc 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -63,6 +63,7 @@ typedef struct { #define QCOW2_EXT_MAGIC_END 0 #define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xE2792ACA #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 +#define QCOW2_EXT_MAGIC_COMPRESS_FORMAT 0xC03183A3 =20 static int qcow2_probe(const uint8_t *buf, int buf_size, const char *filen= ame) { @@ -76,6 +77,26 @@ static int qcow2_probe(const uint8_t *buf, int buf_size,= const char *filename) return 0; } =20 +static int qcow2_compress_format_from_name(char *fmt) +{ + if (!fmt || !fmt[0]) { + return QCOW2_COMPRESS_ZLIB_COMPAT; + } else if (g_str_equal(fmt, "zlib")) { + return QCOW2_COMPRESS_ZLIB; + } else { + return -EINVAL; + } +} + +static int qcow2_compress_level_supported(int id, uint64_t level) +{ + if ((id =3D=3D QCOW2_COMPRESS_ZLIB_COMPAT && level > 0) || + (id =3D=3D QCOW2_COMPRESS_ZLIB && level > 9) || + level > 0xff) { + return -EINVAL; + } + return 0; +} =20 /*=20 * read qcow2 extension and fill bs @@ -148,6 +169,43 @@ static int qcow2_read_extensions(BlockDriverState *bs,= uint64_t start_offset, #endif break; =20 + case QCOW2_EXT_MAGIC_COMPRESS_FORMAT: + if (ext.len !=3D sizeof(s->compress_format)) { + error_setg(errp, "ERROR: ext_compress_format: len=3D%" + PRIu32 " invalid (!=3D%zu)", ext.len, + sizeof(s->compress_format)); + return 2; + } + ret =3D bdrv_pread(bs->file, offset, &s->compress_format, + ext.len); + if (ret < 0) { + error_setg_errno(errp, -ret, "ERROR: ext_compress_fromat:" + " Could not read extension"); + return 3; + } + s->compress_format_id =3D + qcow2_compress_format_from_name(s->compress_format.name); + if (s->compress_format_id < 0) { + error_setg(errp, "ERROR: compression algorithm '%s' is " + " unsupported", s->compress_format.name); + return 4; + } + if (qcow2_compress_level_supported(s->compress_format_id, + s->compress_format.level) <= 0) { + error_setg(errp, "ERROR: compress level %" PRIu8 " is not" + " supported for format '%s'", + s->compress_format.level, s->compress_format.na= me); + return 5; + } + +#ifdef DEBUG_EXT + printf("Qcow2: Got compress format %s with compress level %" + PRIu8 "\n", s->compress_format.name, + s->compress_format.level); +#endif + break; + + case QCOW2_EXT_MAGIC_FEATURE_TABLE: if (p_feature_table !=3D NULL) { void* feature_table =3D g_malloc0(ext.len + 2 * sizeof(Qco= w2Feature)); @@ -1981,6 +2039,20 @@ int qcow2_update_header(BlockDriverState *bs) buflen -=3D ret; } =20 + /* Compress Format header extension */ + if (s->compress_format.name[0]) { + assert(!s->compress_format.extra_data_size); + ret =3D header_ext_add(buf, QCOW2_EXT_MAGIC_COMPRESS_FORMAT, + &s->compress_format, sizeof(s->compress_forma= t), + buflen); + if (ret < 0) { + goto fail; + } + buf +=3D ret; + buflen -=3D ret; + header->incompatible_features |=3D cpu_to_be64(QCOW2_INCOMPAT_COMP= RESS); + } + /* Feature table */ if (s->qcow_version >=3D 3) { Qcow2Feature features[] =3D { @@ -1995,6 +2067,11 @@ int qcow2_update_header(BlockDriverState *bs) .name =3D "corrupt bit", }, { + .type =3D QCOW2_FEAT_TYPE_INCOMPATIBLE, + .bit =3D QCOW2_INCOMPAT_COMPRESS_BITNR, + .name =3D "compress format bit", + }, + { .type =3D QCOW2_FEAT_TYPE_COMPATIBLE, .bit =3D QCOW2_COMPAT_LAZY_REFCOUNTS_BITNR, .name =3D "lazy refcounts", @@ -2333,6 +2410,13 @@ static int qcow2_create2(const char *filename, int64= _t total_size, abort(); } =20 + if (compress_format_name[0]) { + BDRVQcow2State *s =3D blk_bs(blk)->opaque; + memcpy(s->compress_format.name, compress_format_name, + strlen(compress_format_name)); + s->compress_format.level =3D compress_level; + } + /* Create a full header (including things like feature table) */ ret =3D qcow2_update_header(blk_bs(blk)); if (ret < 0) { @@ -2391,17 +2475,6 @@ out: return ret; } =20 -static int qcow2_compress_format_from_name(char *fmt) -{ - if (!fmt || !fmt[0]) { - return QCOW2_COMPRESS_ZLIB_COMPAT; - } else if (g_str_equal(fmt, "zlib")) { - return QCOW2_COMPRESS_ZLIB; - } else { - return -EINVAL; - } -} - static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp) { char *backing_file =3D NULL; @@ -2505,11 +2578,10 @@ static int qcow2_create(const char *filename, QemuO= pts *opts, Error **errp) ret =3D -EINVAL; goto finish; } - if ((ret =3D=3D QCOW2_COMPRESS_ZLIB && compress_level > 9) || - compress_level > 0xff) { + ret =3D qcow2_compress_level_supported(ret, compress_level); + if (ret < 0) { error_setg(errp, "Compress level %" PRIu64 " is not supported for" " format '%s'", compress_level, compress_format_name); - ret =3D -EINVAL; goto finish; } =20 diff --git a/block/qcow2.h b/block/qcow2.h index d21da33..4ceaba1 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -187,13 +187,16 @@ enum { =20 /* Incompatible feature bits */ enum { - QCOW2_INCOMPAT_DIRTY_BITNR =3D 0, - QCOW2_INCOMPAT_CORRUPT_BITNR =3D 1, - QCOW2_INCOMPAT_DIRTY =3D 1 << QCOW2_INCOMPAT_DIRTY_BITNR, - QCOW2_INCOMPAT_CORRUPT =3D 1 << QCOW2_INCOMPAT_CORRUPT_BITNR, + QCOW2_INCOMPAT_DIRTY_BITNR =3D 0, + QCOW2_INCOMPAT_CORRUPT_BITNR =3D 1, + QCOW2_INCOMPAT_COMPRESS_BITNR =3D 2, + QCOW2_INCOMPAT_DIRTY =3D 1 << QCOW2_INCOMPAT_DIRTY_BITNR, + QCOW2_INCOMPAT_CORRUPT =3D 1 << QCOW2_INCOMPAT_CORRUPT_BITNR, + QCOW2_INCOMPAT_COMPRESS =3D 1 << QCOW2_INCOMPAT_COMPRESS_BITNR, =20 QCOW2_INCOMPAT_MASK =3D QCOW2_INCOMPAT_DIRTY - | QCOW2_INCOMPAT_CORRUPT, + | QCOW2_INCOMPAT_CORRUPT + | QCOW2_INCOMPAT_COMPRESS, }; =20 /* Compatible feature bits */ @@ -219,6 +222,13 @@ typedef struct Qcow2Feature { char name[46]; } QEMU_PACKED Qcow2Feature; =20 +typedef struct Qcow2CompressFormatExt { + char name[16]; + uint8_t level; + char res[3]; + uint32_t extra_data_size; +} QEMU_PACKED Qcow2CompressFormatExt; + typedef struct Qcow2DiscardRegion { BlockDriverState *bs; uint64_t offset; @@ -303,6 +313,9 @@ typedef struct BDRVQcow2State { * override) */ char *image_backing_file; char *image_backing_format; + + Qcow2CompressFormatExt compress_format; + int compress_format_id; } BDRVQcow2State; =20 typedef struct Qcow2COWRegion { --=20 1.9.1 From nobody Fri May 3 10:13:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498734241576687.928595845863; Thu, 29 Jun 2017 04:04:01 -0700 (PDT) Received: from localhost ([::1]:38337 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQXEq-00043a-CS for importer@patchew.org; Thu, 29 Jun 2017 07:03:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51823) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQX8k-00079z-Sn for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQX8g-000895-VE for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:38 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:34692 helo=mx01.kamp.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQX8g-00087O-N9 for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:34 -0400 Received: (qmail 9361 invoked by uid 89); 29 Jun 2017 10:57:33 -0000 Received: from [195.62.97.28] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.99.2/23517. avast: 1.2.2/17010300. spamassassin: 3.4.1. Clear:RC:1(195.62.97.28):. Processed in 0.342327 secs); 29 Jun 2017 10:57:33 -0000 Received: from smtp.kamp.de (HELO submission.kamp.de) ([195.62.97.28]) by mx01.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted); 29 Jun 2017 10:57:30 -0000 Received: (qmail 12784 invoked from network); 29 Jun 2017 10:57:15 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 29 Jun 2017 10:57:15 -0000 Received: by lieven-pc (Postfix, from userid 1000) id ACDF426C60; Thu, 29 Jun 2017 12:57:15 +0200 (CEST) X-GL_Whitelist: yes From: Peter Lieven To: qemu-block@nongnu.org Date: Thu, 29 Jun 2017 12:57:09 +0200 Message-Id: <1498733831-15254-7-git-send-email-pl@kamp.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498733831-15254-1-git-send-email-pl@kamp.de> References: <1498733831-15254-1-git-send-email-pl@kamp.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:248:0:51::16 Subject: [Qemu-devel] [PATCH V2 6/8] block/qcow2: optimize qcow2_co_pwritev_compressed X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, mreitz@redhat.com, den@openvz.org, lersek@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" if we specify exactly one iov of s->cluster_size bytes we can avoid the bounce buffer. Signed-off-by: Peter Lieven --- block/qcow2.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 39a8afc..0a7202a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2750,7 +2750,7 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uin= t64_t offset, struct iovec iov; z_stream strm; int ret, out_len; - uint8_t *buf, *out_buf; + uint8_t *buf, *out_buf, *local_buf =3D NULL; uint64_t cluster_offset; =20 if (bytes =3D=3D 0) { @@ -2760,8 +2760,8 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uin= t64_t offset, return bdrv_truncate(bs->file, cluster_offset, NULL); } =20 - buf =3D qemu_blockalign(bs, s->cluster_size); - if (bytes !=3D s->cluster_size) { + if (bytes !=3D s->cluster_size || qiov->niov !=3D 1) { + buf =3D local_buf =3D qemu_blockalign(bs, s->cluster_size); if (bytes > s->cluster_size || offset + bytes !=3D bs->total_sectors << BDRV_SECTOR_BITS) { @@ -2770,8 +2770,10 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, ui= nt64_t offset, } /* Zero-pad last write if image size is not cluster aligned */ memset(buf + bytes, 0, s->cluster_size - bytes); + qemu_iovec_to_buf(qiov, 0, buf, bytes); + } else { + buf =3D qiov->iov[0].iov_base; } - qemu_iovec_to_buf(qiov, 0, buf, bytes); =20 out_buf =3D g_malloc(s->cluster_size); =20 @@ -2839,7 +2841,7 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uin= t64_t offset, success: ret =3D 0; fail: - qemu_vfree(buf); + qemu_vfree(local_buf); g_free(out_buf); return ret; } --=20 1.9.1 From nobody Fri May 3 10:13:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498734139997108.28781008350154; Thu, 29 Jun 2017 04:02:19 -0700 (PDT) Received: from localhost ([::1]:38332 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQXDD-0002lc-KJ for importer@patchew.org; Thu, 29 Jun 2017 07:02:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51701) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQX8f-00072N-3C for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQX8d-00083u-N9 for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:33 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:39176 helo=mx01.kamp.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQX8d-00081A-EO for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:31 -0400 Received: (qmail 9154 invoked by uid 89); 29 Jun 2017 10:57:28 -0000 Received: from [195.62.97.28] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.99.2/23517. avast: 1.2.2/17010300. spamassassin: 3.4.1. Clear:RC:1(195.62.97.28):. Processed in 0.236141 secs); 29 Jun 2017 10:57:28 -0000 Received: from smtp.kamp.de (HELO submission.kamp.de) ([195.62.97.28]) by mx01.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted); 29 Jun 2017 10:57:25 -0000 Received: (qmail 12785 invoked from network); 29 Jun 2017 10:57:15 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 29 Jun 2017 10:57:15 -0000 Received: by lieven-pc (Postfix, from userid 1000) id AF8A326C61; Thu, 29 Jun 2017 12:57:15 +0200 (CEST) X-GL_Whitelist: yes From: Peter Lieven To: qemu-block@nongnu.org Date: Thu, 29 Jun 2017 12:57:10 +0200 Message-Id: <1498733831-15254-8-git-send-email-pl@kamp.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498733831-15254-1-git-send-email-pl@kamp.de> References: <1498733831-15254-1-git-send-email-pl@kamp.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:248:0:51::16 Subject: [Qemu-devel] [PATCH V2 7/8] block/qcow2: start using the compress format extension X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, mreitz@redhat.com, den@openvz.org, lersek@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" we now pass the parameters to the zlib compressor if the extension is present and use the old default values if the extension is absent. Signed-off-by: Peter Lieven --- block/qcow2-cluster.c | 58 ++++++++++++++++++++++++++++++-----------------= ---- block/qcow2.c | 57 +++++++++++++++++++++++++++--------------------= --- 2 files changed, 65 insertions(+), 50 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 3d341fd..353ac87 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1521,30 +1521,39 @@ again: } =20 static int decompress_buffer(uint8_t *out_buf, int out_buf_size, - const uint8_t *buf, int buf_size) + const uint8_t *buf, int buf_size, + uint32_t compress_format_id) { - z_stream strm1, *strm =3D &strm1; - int ret, out_len; - - memset(strm, 0, sizeof(*strm)); - - strm->next_in =3D (uint8_t *)buf; - strm->avail_in =3D buf_size; - strm->next_out =3D out_buf; - strm->avail_out =3D out_buf_size; - - ret =3D inflateInit2(strm, -12); - if (ret !=3D Z_OK) - return -1; - ret =3D inflate(strm, Z_FINISH); - out_len =3D strm->next_out - out_buf; - if ((ret !=3D Z_STREAM_END && ret !=3D Z_BUF_ERROR) || - out_len !=3D out_buf_size) { - inflateEnd(strm); - return -1; - } - inflateEnd(strm); - return 0; + int ret =3D 0, out_len; + + switch (compress_format_id) { + case QCOW2_COMPRESS_ZLIB: + case QCOW2_COMPRESS_ZLIB_COMPAT: { + z_stream z_strm =3D {}; + + z_strm.next_in =3D (uint8_t *)buf; + z_strm.avail_in =3D buf_size; + z_strm.next_out =3D out_buf; + z_strm.avail_out =3D out_buf_size; + + ret =3D inflateInit2(&z_strm, -15); + if (ret !=3D Z_OK) { + return -1; + } + ret =3D inflate(&z_strm, Z_FINISH); + out_len =3D z_strm.next_out - out_buf; + ret =3D -(ret !=3D Z_STREAM_END); + inflateEnd(&z_strm); + break; + } + default: + abort(); /* should never reach this point */ + } + + if (out_len !=3D out_buf_size) { + ret =3D -1; + } + return ret; } =20 int qcow2_decompress_cluster(BlockDriverState *bs, uint64_t cluster_offset) @@ -1565,7 +1574,8 @@ int qcow2_decompress_cluster(BlockDriverState *bs, ui= nt64_t cluster_offset) return ret; } if (decompress_buffer(s->cluster_cache, s->cluster_size, - s->cluster_data + sector_offset, csize) < 0)= { + s->cluster_data + sector_offset, csize, + s->compress_format_id) < 0) { return -EIO; } s->cluster_cache_offset =3D coffset; diff --git a/block/qcow2.c b/block/qcow2.c index 0a7202a..b41e58d 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2748,9 +2748,10 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, ui= nt64_t offset, BDRVQcow2State *s =3D bs->opaque; QEMUIOVector hd_qiov; struct iovec iov; - z_stream strm; - int ret, out_len; - uint8_t *buf, *out_buf, *local_buf =3D NULL; + z_stream z_strm =3D {}; + int z_windowBits =3D -15, z_level =3D Z_DEFAULT_COMPRESSION; + int ret, out_len =3D 0; + uint8_t *buf, *out_buf =3D NULL, *local_buf =3D NULL; uint64_t cluster_offset; =20 if (bytes =3D=3D 0) { @@ -2775,34 +2776,38 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, u= int64_t offset, buf =3D qiov->iov[0].iov_base; } =20 - out_buf =3D g_malloc(s->cluster_size); + switch (s->compress_format_id) { + case QCOW2_COMPRESS_ZLIB_COMPAT: + z_windowBits =3D -12; + case QCOW2_COMPRESS_ZLIB: + out_buf =3D g_malloc(s->cluster_size); + if (s->compress_format.level > 0) { + z_level =3D s->compress_format.level; + } =20 - /* best compression, small window, no zlib header */ - memset(&strm, 0, sizeof(strm)); - ret =3D deflateInit2(&strm, Z_DEFAULT_COMPRESSION, - Z_DEFLATED, -12, - 9, Z_DEFAULT_STRATEGY); - if (ret !=3D 0) { - ret =3D -EINVAL; - goto fail; - } + ret =3D deflateInit2(&z_strm, z_level, Z_DEFLATED, z_windowBits, 9, + Z_DEFAULT_STRATEGY); + if (ret !=3D Z_OK) { + ret =3D -EINVAL; + goto fail; + } =20 - strm.avail_in =3D s->cluster_size; - strm.next_in =3D (uint8_t *)buf; - strm.avail_out =3D s->cluster_size; - strm.next_out =3D out_buf; + z_strm.avail_in =3D s->cluster_size; + z_strm.next_in =3D (uint8_t *)buf; + z_strm.avail_out =3D s->cluster_size; + z_strm.next_out =3D out_buf; =20 - ret =3D deflate(&strm, Z_FINISH); - if (ret !=3D Z_STREAM_END && ret !=3D Z_OK) { - deflateEnd(&strm); - ret =3D -EINVAL; - goto fail; - } - out_len =3D strm.next_out - out_buf; + ret =3D deflate(&z_strm, Z_FINISH); + out_len =3D z_strm.next_out - out_buf; + deflateEnd(&z_strm); =20 - deflateEnd(&strm); + ret =3D ret !=3D Z_STREAM_END; + break; + default: + abort(); /* should never reach this point */ + } =20 - if (ret !=3D Z_STREAM_END || out_len >=3D s->cluster_size) { + if (ret || out_len >=3D s->cluster_size) { /* could not compress: write normal cluster */ ret =3D qcow2_co_pwritev(bs, offset, bytes, qiov, 0); if (ret < 0) { --=20 1.9.1 From nobody Fri May 3 10:13:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498733969919771.7598922820014; Thu, 29 Jun 2017 03:59:29 -0700 (PDT) Received: from localhost ([::1]:38315 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQXAU-0000Ty-44 for importer@patchew.org; Thu, 29 Jun 2017 06:59:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51641) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQX8d-00071o-A8 for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQX8b-00080f-2c for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:30 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:42015 helo=mx01.kamp.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQX8a-0007wn-Pz for qemu-devel@nongnu.org; Thu, 29 Jun 2017 06:57:29 -0400 Received: (qmail 8998 invoked by uid 89); 29 Jun 2017 10:57:24 -0000 Received: from [195.62.97.28] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.99.2/23517. avast: 1.2.2/17010300. spamassassin: 3.4.1. Clear:RC:1(195.62.97.28):. Processed in 0.329586 secs); 29 Jun 2017 10:57:24 -0000 Received: from smtp.kamp.de (HELO submission.kamp.de) ([195.62.97.28]) by mx01.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted); 29 Jun 2017 10:57:21 -0000 Received: (qmail 12788 invoked from network); 29 Jun 2017 10:57:15 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 29 Jun 2017 10:57:15 -0000 Received: by lieven-pc (Postfix, from userid 1000) id B223626C62; Thu, 29 Jun 2017 12:57:15 +0200 (CEST) X-GL_Whitelist: yes From: Peter Lieven To: qemu-block@nongnu.org Date: Thu, 29 Jun 2017 12:57:11 +0200 Message-Id: <1498733831-15254-9-git-send-email-pl@kamp.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498733831-15254-1-git-send-email-pl@kamp.de> References: <1498733831-15254-1-git-send-email-pl@kamp.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:248:0:51::16 Subject: [Qemu-devel] [PATCH V2 8/8] block/qcow2: add lzo compress format X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, mreitz@redhat.com, den@openvz.org, lersek@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Peter Lieven --- block/qcow2-cluster.c | 15 +++++++++++++++ block/qcow2.c | 26 +++++++++++++++++++++++++- block/qcow2.h | 1 + configure | 2 +- qapi/block-core.json | 14 ++++++++++++-- qemu-img.texi | 1 + 6 files changed, 55 insertions(+), 4 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 353ac87..666d090 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -24,6 +24,9 @@ =20 #include "qemu/osdep.h" #include +#ifdef CONFIG_LZO +#include +#endif =20 #include "qapi/error.h" #include "qemu-common.h" @@ -1546,6 +1549,18 @@ static int decompress_buffer(uint8_t *out_buf, int o= ut_buf_size, inflateEnd(&z_strm); break; } +#ifdef CONFIG_LZO + case QCOW2_COMPRESS_LZO: + out_len =3D out_buf_size; + ret =3D lzo1x_decompress_safe(buf, buf_size, out_buf, + (lzo_uint *) &out_len, NULL); + if (ret =3D=3D LZO_E_INPUT_NOT_CONSUMED) { + /* We always read up to the next sector boundary. Thus + * buf_size may be larger than the original compressed size. */ + ret =3D 0; + } + break; +#endif default: abort(); /* should never reach this point */ } diff --git a/block/qcow2.c b/block/qcow2.c index b41e58d..ef94193 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -26,6 +26,9 @@ #include "sysemu/block-backend.h" #include "qemu/module.h" #include +#ifdef CONFIG_LZO +#include +#endif #include "block/qcow2.h" #include "qemu/error-report.h" #include "qapi/qmp/qerror.h" @@ -83,6 +86,10 @@ static int qcow2_compress_format_from_name(char *fmt) return QCOW2_COMPRESS_ZLIB_COMPAT; } else if (g_str_equal(fmt, "zlib")) { return QCOW2_COMPRESS_ZLIB; +#ifdef CONFIG_LZO + } else if (g_str_equal(fmt, "lzo")) { + return QCOW2_COMPRESS_LZO; +#endif } else { return -EINVAL; } @@ -92,6 +99,7 @@ static int qcow2_compress_level_supported(int id, uint64_= t level) { if ((id =3D=3D QCOW2_COMPRESS_ZLIB_COMPAT && level > 0) || (id =3D=3D QCOW2_COMPRESS_ZLIB && level > 9) || + (id =3D=3D QCOW2_COMPRESS_LZO && level > 0) || level > 0xff) { return -EINVAL; } @@ -197,6 +205,13 @@ static int qcow2_read_extensions(BlockDriverState *bs,= uint64_t start_offset, s->compress_format.level, s->compress_format.na= me); return 5; } +#ifdef CONFIG_LZO + if (s->compress_format_id =3D=3D QCOW2_COMPRESS_LZO && + lzo_init() !=3D LZO_E_OK) { + error_setg(errp, "ERROR: internal error - lzo_init() faile= d"); + return 6; + } +#endif =20 #ifdef DEBUG_EXT printf("Qcow2: Got compress format %s with compress level %" @@ -2751,7 +2766,7 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uin= t64_t offset, z_stream z_strm =3D {}; int z_windowBits =3D -15, z_level =3D Z_DEFAULT_COMPRESSION; int ret, out_len =3D 0; - uint8_t *buf, *out_buf =3D NULL, *local_buf =3D NULL; + uint8_t *buf, *out_buf =3D NULL, *local_buf =3D NULL, *work_buf =3D NU= LL; uint64_t cluster_offset; =20 if (bytes =3D=3D 0) { @@ -2803,6 +2818,14 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, ui= nt64_t offset, =20 ret =3D ret !=3D Z_STREAM_END; break; +#ifdef CONFIG_LZO + case QCOW2_COMPRESS_LZO: + out_buf =3D g_malloc(s->cluster_size + s->cluster_size / 16 + 64 += 3); + work_buf =3D g_malloc(LZO1X_1_MEM_COMPRESS); + ret =3D lzo1x_1_compress(buf, s->cluster_size, out_buf, + (lzo_uint *) &out_len, work_buf); + break; +#endif default: abort(); /* should never reach this point */ } @@ -2848,6 +2871,7 @@ success: fail: qemu_vfree(local_buf); g_free(out_buf); + g_free(work_buf); return ret; } =20 diff --git a/block/qcow2.h b/block/qcow2.h index 4ceaba1..038688d 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -177,6 +177,7 @@ enum { * is specified at create time */ QCOW2_COMPRESS_ZLIB_COMPAT =3D 0, QCOW2_COMPRESS_ZLIB =3D 1, + QCOW2_COMPRESS_LZO =3D 2, }; =20 enum { diff --git a/configure b/configure index c571ad1..81d3286 100755 --- a/configure +++ b/configure @@ -1890,7 +1890,7 @@ if test "$lzo" !=3D "no" ; then int main(void) { lzo_version(); return 0; } EOF if compile_prog "" "-llzo2" ; then - libs_softmmu=3D"$libs_softmmu -llzo2" + LIBS=3D"$LIBS -llzo2" lzo=3D"yes" else if test "$lzo" =3D "yes"; then diff --git a/qapi/block-core.json b/qapi/block-core.json index 1574ffb..736073a 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2284,11 +2284,12 @@ ## # @Qcow2CompressFormat: # @zlib: standard zlib deflate compression +# @lzo: lzo1x compression # # Since: 2.10 ## { 'enum': 'Qcow2CompressFormat', - 'data': [ 'zlib' ] } + 'data': [ 'zlib', 'lzo' ] } =20 ## # @Qcow2CompressZLib: @@ -2299,6 +2300,14 @@ 'data': { } } =20 ## +# @Qcow2CompressLZO: +# +# Since: 2.10 +## +{ 'struct': 'Qcow2CompressLZO', + 'data': { } } + +## # @Qcow2Compress: # # Specifies the compression format and compression level that should @@ -2316,7 +2325,8 @@ 'base': { 'format': 'Qcow2CompressFormat', 'level': 'uint8' }, 'discriminator': 'format', - 'data': { 'zlib': 'Qcow2CompressZLib' } } + 'data': { 'zlib': 'Qcow2CompressZLib', + 'lzo': 'Qcow2CompressLZO' } } =20 ## # @BlockdevOptionsQcow2: diff --git a/qemu-img.texi b/qemu-img.texi index 430f0b9..fda0e50 100644 --- a/qemu-img.texi +++ b/qemu-img.texi @@ -627,6 +627,7 @@ The following options are available if support for the = respective libraries has been enabled at compile time: =20 zlib Uses standard zlib compression + lzo Uses LZO1X compression =20 The compression algorithm can only be defined at image create time and can= not be changed later. --=20 1.9.1