From nobody Sat May 4 14:06:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1579540943; cv=none; d=zohomail.com; s=zohoarc; b=XTIdldoH0OEbLsAqitg621pdwD2kXVUo1XzzELgL3B0g25TPgbw0QjYtLhBh2geDUhvgZAYZ6U8EJbkZxNuDfH8/Jj9HCY7HPVS4BAILHLUJmsdyI8DJsgaL/+MD2aS7zrcGjv2SINGWDZHN0kFubn5wrNMthyhXGr1aCY0OzPs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579540943; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PnvRpCuKMabuMLo50QShgCN2gw5ZNkckD6cUo92y7nI=; b=Qcp4KSrYVYr5BBf1Zf5g6uLSU/m3LfuPaQy3vHydl5we1BiBXuA9MH68CRrrBDbWw6UO7IJwwVSHd7DkIAfgRLTQcDQeDIIN7kmxbXszIyYXIdX/lKO0Kb56pwj2Cv2A6zSpZyu5BrzdQ40WDm6MTE8xb8zSx1AN2IpmsVovTcM= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157954094388043.101325402977295; Mon, 20 Jan 2020 09:22:23 -0800 (PST) Received: from localhost ([::1]:40506 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1itako-0005Pu-9H for importer@patchew.org; Mon, 20 Jan 2020 12:22:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:36994) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1itacu-0005Mw-6W for qemu-devel@nongnu.org; Mon, 20 Jan 2020 12:14:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1itacs-0004Qn-Sf for qemu-devel@nongnu.org; Mon, 20 Jan 2020 12:14:12 -0500 Received: from relay.sw.ru ([185.231.240.75]:37506) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1itacp-0004PD-U2; Mon, 20 Jan 2020 12:14:08 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1itaco-0002hc-Gg; Mon, 20 Jan 2020 20:14:06 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v10 1/2] docs: improve qcow2 spec about extending image header Date: Mon, 20 Jan 2020 20:13:44 +0300 Message-Id: <20200120171345.24345-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200120171345.24345-1-vsementsov@virtuozzo.com> References: <20200120171345.24345-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, dplotnikov@virtuozzo.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Make it more obvious how to add new fields to the version 3 header and how to interpret them. The specification is adjusted so for new defined optional fields: 1. Software may support some of these optional fields and ignore the others, which means that features may be backported to downstream Qemu independently. 2. If we want to add incompatible field (or a field, for which some its values would be incompatible), it must be accompanied by incompatible feature bit. Also the concept of "default is zero" is clarified, as it's strange to say that the value of the field is assumed to be zero for the software version which don't know about the field at all and don't know how to treat it be it zero or not. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- docs/interop/qcow2.txt | 44 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 41 insertions(+), 3 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index af5711e533..355925c35e 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -79,9 +79,9 @@ The first cluster of a qcow2 image contains the file head= er: Offset into the image file at which the snapshot table starts. Must be aligned to a cluster boundary. =20 -If the version is 3 or higher, the header has the following additional fie= lds. -For version 2, the values are assumed to be zero, unless specified otherwi= se -in the description of a field. +For version 2, the header is exactly 72 bytes in length, and finishes here. +For version 3 or higher, the header length is at least 104 bytes, including +the next fields through header_length. =20 72 - 79: incompatible_features Bitmask of incompatible features. An implementation mu= st @@ -164,6 +164,44 @@ in the description of a field. 100 - 103: header_length Length of the header structure in bytes. For version 2 images, the length is always assumed to be 72 bytes. + For version 3 it's at least 104 bytes and must be a mu= ltiple + of 8. + + +=3D=3D=3D Additional fields (version 3 and higher) =3D=3D=3D + +In general, these fields are optional and may be safely ignored by the sof= tware, +as well as filled by zeros (which is equal to field absence), if software = needs +to set field B, but does not care about field A, which precedes B. More +formally, additional fields have the following compatibility rules: + +1. If the value of the additional field must not be ignored for correct +handling of the file, it will be accompanied by a corresponding incompatib= le +feature bit. + +2. If there are no unrecognized incompatible feature bits set, an unknown +additional field may be safely ignored other than preserving its value when +rewriting the image header. + +3. An explicit value of 0 will have the same behavior as when the field is= not +present*, if not altered by a specific incompatible bit. + +*. A field is considered not present when header_length is less or equal t= o the +field's offset. Also, all additional fields are not present for version 2. + + < ... No additional fields in the header currently ... > + + +=3D=3D=3D Header padding =3D=3D=3D + +@header_length must be a multiple of 8, which means that if the end of the= last +additional field is not aligned, some padding is needed. This padding must= be +zeroed, so that, if some existing (or future) additional field will fall i= nto +the padding, it will be interpreted accordingly to point [3.] of the previ= ous +paragraph, i.e. in the same manner as when this field is not present. + + +=3D=3D=3D Header extensions =3D=3D=3D =20 Directly after the image header, optional sections called header extension= s can be stored. Each extension has a structure like the following: --=20 2.21.0 From nobody Sat May 4 14:06:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1579540968; cv=none; d=zohomail.com; s=zohoarc; b=nZ1bDoTIeyQ/PUlc1wkfYxuzdkSYKbDQJ7/e7+91leiWqNGHi5NndBtS0lfnoai5hWxWuHyySh0LMJFjGh9QBWwtwVEI6A58q9cgyh20TvLDXxLNWdNSOOU46Q8dXUYaw1pcxKNUgcjeGamvVYGHOnZuBEojul1miAkmo2b4qzk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579540968; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=WO8kNTYxYFxfKcT9JhL/9gTG+HbrKUbbdsUDuzUP01A=; b=MvJaF3s2KxjiWsQXalPCsHM4yW67Nrhq2q+P/mpYo1BelKi0lOlgQm8NqdqXKw8UEiHLAkTIde8zmbUYCDooHF67ADdUtBDUbe0OUPXOPU95Rn3F1wLYNepyB/6W8+QMTYLcm64sDkuWgn2c3X6JbpbMsh8SenxnpRLVweCaHgk= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1579540968382489.8890578048771; Mon, 20 Jan 2020 09:22:48 -0800 (PST) Received: from localhost ([::1]:40510 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1italD-00066B-3W for importer@patchew.org; Mon, 20 Jan 2020 12:22:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37021) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1itacz-0005W0-4g for qemu-devel@nongnu.org; Mon, 20 Jan 2020 12:14:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1itacx-0004SY-Ta for qemu-devel@nongnu.org; Mon, 20 Jan 2020 12:14:16 -0500 Received: from relay.sw.ru ([185.231.240.75]:37530) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1itacu-0004R9-GU; Mon, 20 Jan 2020 12:14:12 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1itact-0002hc-0z; Mon, 20 Jan 2020 20:14:11 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v10 2/2] docs: qcow2: introduce compression type feature Date: Mon, 20 Jan 2020 20:13:45 +0300 Message-Id: <20200120171345.24345-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200120171345.24345-1-vsementsov@virtuozzo.com> References: <20200120171345.24345-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, dplotnikov@virtuozzo.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" The patch add new additional field to qcow2 header: compression_type, which specifies compression type. If field is absent or zero, default compression type is set: ZLIB, which corresponds to current behavior. New compression type (ZSTD) is to be added in further commit. Suggested-by: Denis Plotnikov Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/interop/qcow2.txt | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 355925c35e..4569f0dba3 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -109,6 +109,11 @@ the next fields through header_length. An External Data File Name header extensio= n may be present if this bit is set. =20 + Bit 3: Compression type bit. If this bit is set, + a non-default compression is used for comp= ressed + clusters. The compression_type field must = be + present and not zero. + Bits 3-63: Reserved (set to 0) =20 80 - 87: compatible_features @@ -189,7 +194,16 @@ present*, if not altered by a specific incompatible bi= t. *. A field is considered not present when header_length is less or equal t= o the field's offset. Also, all additional fields are not present for version 2. =20 - < ... No additional fields in the header currently ... > + 104: compression_type + Defines the compression method used for compressed clu= sters. + All compressed clusters in an image use the same compr= ession + type. + If the incompatible bit "Compression type" is set: the= field + must be present and non-zero (which means non-zlib + compression type). Otherwise, this field must not be p= resent + or must be zero (which means zlib). + Available compression type values: + 0: zlib =20 =20 =3D=3D=3D Header padding =3D=3D=3D --=20 2.21.0