From nobody Fri Mar 29 10:07:27 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1570464597; cv=none; d=zoho.com; s=zohoarc; b=W0FM9PJP3hAeG4F3gIfP3SYvLCDVJ0eNUsV8wV7pp6iZF/Lf5BJlqzwc6eBjvS/w9GD3vuLoba7Bv0r2nbXGSssBBPsQNRxqBKjOy89u4yTRKUA4ijZPShnqfYaas8csLPF0Frzc1PPb91kcxzPec0iyBB/mQRPPxSF7QcbVuO8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1570464597; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=7APLR01gud+8beFnLjbF+bBba5oO7ArFAlQvVTQEIDc=; b=g3ubxYQTrbvWiSyMnofZsjl89iSWbi3ft+q77+uqUkikNT1HTd5/5h/a8d58unhsCFeSIUcFfHddMf3nCYbpgHdd3SVbCLc2ESJVi+WqtgUCB4+CffuGGXg28g5Nf22BM5rGdpcsDZ+nz/5S0O/7En5Y+NlrzEwaYJj3Ek/i2jc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1570464596930923.4764643214527; Mon, 7 Oct 2019 09:09:56 -0700 (PDT) Received: from localhost ([::1]:47016 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHVa4-0006f1-Lj for importer@patchew.org; Mon, 07 Oct 2019 12:09:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54747) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHVVN-0001d9-Hh for qemu-devel@nongnu.org; Mon, 07 Oct 2019 12:05:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iHVVL-0006Xa-W8 for qemu-devel@nongnu.org; Mon, 07 Oct 2019 12:05:01 -0400 Received: from relay.sw.ru ([185.231.240.75]:56606) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iHVVH-0006SH-Ef; Mon, 07 Oct 2019 12:04:55 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iHVVD-0003oB-Uk; Mon, 07 Oct 2019 19:04:52 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v7 1/2] docs: improve qcow2 spec about extending image header Date: Mon, 7 Oct 2019 19:04:50 +0300 Message-Id: <20191007160451.27334-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191007160451.27334-1-vsementsov@virtuozzo.com> References: <20191007160451.27334-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, den@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, dplotnikov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Make it more obvious how to add new fields to the version 3 header and how to interpret them. Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/interop/qcow2.txt | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index af5711e533..3f2855593f 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -79,9 +79,9 @@ The first cluster of a qcow2 image contains the file head= er: Offset into the image file at which the snapshot table starts. Must be aligned to a cluster boundary. =20 -If the version is 3 or higher, the header has the following additional fie= lds. -For version 2, the values are assumed to be zero, unless specified otherwi= se -in the description of a field. +For version 2, header is always 72 bytes length and finishes here. +For version 3 or higher the header length is at least 104 bytes and has at +least next five fields, up to the @header_length field. =20 72 - 79: incompatible_features Bitmask of incompatible features. An implementation mu= st @@ -165,6 +165,26 @@ in the description of a field. Length of the header structure in bytes. For version 2 images, the length is always assumed to be 72 bytes. =20 +Additional fields (version 3 and higher) + +The following fields of the header are optional: if software don't know ho= w to +interpret the field, it may safely ignore it. Still the field must be kept= as is +when rewriting the image. @header_length must be bound to the end of one of +these fields (or to @header_length field end itself, to be 104 bytes). +This definition implies the following: +1. Software may support some of these optional fields and ignore the other= s, + which means that features may be backported to downstream Qemu independ= ently. +2. Software may check @header_length, if it knows optional fields specific= ation + enough (knows about the field which exceeds @header_length). +3. If @header_length is higher than the highest field end that software kn= ows, + it should assume that additional fields are correct, @header_length is + correct and keep @header_length and additional unknown fields as is on + rewriting the image. +3. If we want to add incompatible field (or a field, for which some its va= lues + would be incompatible), it must be accompanied by incompatible feature = bit. + + < ... No additional fields in the header currently ... > + Directly after the image header, optional sections called header extension= s can be stored. Each extension has a structure like the following: =20 --=20 2.21.0 From nobody Fri Mar 29 10:07:27 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1570464761; cv=none; d=zoho.com; s=zohoarc; b=mDloBOcf973JP8GRtPGuktfUhD2VoFaDfAwY/mC+mj/C7yRuRd93vB0asB7LbymigRYx000dmReNl9+waV68OyWytWj7zh0c0qDVwUvsymUh81coRMaxz/hQc5lGspATVCrj84FJHncDnYBZvODAvPb34SyI4nYJv7JY0bmaCKU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1570464761; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=SNYjQwMBRdf2+2qVsytTX0kRiUl0VtlOE8GLJFQ1ttQ=; b=FDk2CDDknprJInXyFOkehSQ0TLR6IFBjtR2nOwgtWEpk2uYGwqWnI/XBNObs4IOMy+gYmKGaVVRZSVO+qhloG1/ZVlbG2KYlZ0Ia/aEjfc72KzqYSmocuT0DBi5Mvj/04zLe205IyEHBke90d5oU5H+QHtTEmMkyRhhrqHZWubM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15704647611413.146557025677339; Mon, 7 Oct 2019 09:12:41 -0700 (PDT) Received: from localhost ([::1]:47092 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHVck-0000sl-V3 for importer@patchew.org; Mon, 07 Oct 2019 12:12:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54743) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHVVN-0001cT-3I for qemu-devel@nongnu.org; Mon, 07 Oct 2019 12:05:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iHVVL-0006XK-L4 for qemu-devel@nongnu.org; Mon, 07 Oct 2019 12:05:00 -0400 Received: from relay.sw.ru ([185.231.240.75]:56598) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iHVVH-0006SG-7I; Mon, 07 Oct 2019 12:04:55 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iHVVE-0003oB-8Y; Mon, 07 Oct 2019 19:04:52 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v7 2/2] docs: qcow2: introduce compression type feature Date: Mon, 7 Oct 2019 19:04:51 +0300 Message-Id: <20191007160451.27334-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191007160451.27334-1-vsementsov@virtuozzo.com> References: <20191007160451.27334-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, den@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, dplotnikov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" The patch add new additional field to qcow2 header: compression_type, which specifies compression type. If field is absent or zero, default compression type is set: ZLIB, which corresponds to current behavior. New compression type (ZSTD) is to be added in further commit. Suggested-by: Denis Plotnikov Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/interop/qcow2.txt | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 3f2855593f..82ec0845d8 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -109,6 +109,12 @@ least next five fields, up to the @header_length field. An External Data File Name header extensio= n may be present if this bit is set. =20 + Bit 3: Compression type bit. If this bit is set, + non-default compression is used for compre= ssed + clusters. In this case, @header_length must + be at least 108 and @compression_type field + must be non-zero. + Bits 3-63: Reserved (set to 0) =20 80 - 87: compatible_features @@ -183,7 +189,18 @@ This definition implies the following: 3. If we want to add incompatible field (or a field, for which some its va= lues would be incompatible), it must be accompanied by incompatible feature = bit. =20 - < ... No additional fields in the header currently ... > + 104: compression_type + Defines the compression method used for compressed clu= sters. + A single compression type is applied to all compressed= image + clusters. + If incompatible compression type bit is set: the field= must + exist (i.e. @header_length >=3D 105) and must be non-z= ero ( + which means non-zlib compression type) + If incompatible compression type bit is unset: the fie= ld + may not exist (if @header_length < 105) or it must be = zero + (which means zlib). + Available compression type values: + 0: zlib =20 Directly after the image header, optional sections called header extension= s can be stored. Each extension has a structure like the following: --=20 2.21.0