From nobody Sun Oct 5 21:14:38 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1542151092959889.1985788284829; Tue, 13 Nov 2018 15:18:12 -0800 (PST) Received: from localhost ([::1]:56390 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMhwh-00076v-P3 for importer@patchew.org; Tue, 13 Nov 2018 18:18:11 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41977) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMhu6-0005Tr-H2 for qemu-devel@nongnu.org; Tue, 13 Nov 2018 18:15:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMhij-0003wo-0W for qemu-devel@nongnu.org; Tue, 13 Nov 2018 18:03:53 -0500 Received: from mx1.redhat.com ([209.132.183.28]:45194) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gMhiY-0003af-Pu; Tue, 13 Nov 2018 18:03:35 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EA3603082E63; Tue, 13 Nov 2018 23:03:28 +0000 (UTC) Received: from red.redhat.com (ovpn-123-32.rdu2.redhat.com [10.10.123.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 72BCD600D6; Tue, 13 Nov 2018 23:03:27 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Date: Tue, 13 Nov 2018 17:03:17 -0600 Message-Id: <20181113230319.1008531-2-eblake@redhat.com> In-Reply-To: <20181113230319.1008531-1-eblake@redhat.com> References: <20181113230319.1008531-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 13 Nov 2018 23:03:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 1/3] qcow2: Document some maximum size constraints X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, berto@igalia.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Although off_t permits up to 63 bits (8EB) of file offsets, in practice, we're going to hit other limits first. Document some of those limits in the qcow2 spec, and how choice of cluster size can influence some of the limits. While we cannot map any virtual cluster to any address higher than 64 PB (56 bits) (due to the current L1/L2 field encoding stopping at bit 55), the refcount table can currently be sized larger. For comparison, ext4 with 4k blocks caps files at 16PB. Another interesting limit: for compressed clusters, the L2 layout requires an ever-smaller maximum host offset as cluster size gets larger, down to a 512 TB maximum with 2M clusters. Signed-off-by: Eric Blake -- v8: don't try and limit refcount (R-b dropped) v5: even more wording tweaks v4: more wording tweaks v3: new patch --- docs/interop/qcow2.txt | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 845d40a086d..89faf7b99f3 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -40,7 +40,16 @@ The first cluster of a qcow2 image contains the file hea= der: with larger cluster sizes. 24 - 31: size - Virtual disk size in bytes + Virtual disk size in bytes. + + Note: with a 2 MB cluster size, the maximum + virtual size is 2 EB (61 bits) for a fully sparse + file; however, L1/L2 table layouts limit an image + to no more than 64 PB (56 bits) of populated + clusters, and an image may hit other limits first + (such as a file system's maximum size). With a + 512 byte cluster size, the maximum virtual size + drops to 128 GB (37 bits). 32 - 35: crypt_method 0 for no encryption @@ -326,6 +335,11 @@ in the image file. It contains pointers to the second level structures which are called refco= unt blocks and are exactly one cluster in size. +Although the refcount table can reserve clusters past 64 PB (56 bits) +(assuming the underlying protocol can even be sized that large), note +that some qcow2 metadata such as L1/L2 tables must point to clusters +prior to that point. + Given an offset into the image file, the refcount of its cluster can be obtained as follows: @@ -365,6 +379,16 @@ The L1 table has a variable size (stored in the header= ) and may use multiple clusters, however it must be contiguous in the image file. L2 tables are exactly one cluster in size. +The L1 and L2 tables have implications on the maximum virtual file +size; a larger cluster size is required for the guest to have access +to more space. Furthermore, a virtual cluster must currently map to a +host offset below 64 PB (56 bits) (although this limit could be +relaxed by putting reserved bits into use). Additionally, as cluster +size increases, the maximum host offset for a compressed cluster is +reduced (a 2M cluster size requires compressed clusters to reside +below 512 TB (49 bits), and this limit cannot be relaxed without an +incompatible layout change). + Given an offset into the virtual disk, the offset into the image file can = be obtained as follows: @@ -427,7 +451,9 @@ Standard Cluster Descriptor: Compressed Clusters Descriptor (x =3D 62 - (cluster_bits - 8)): Bit 0 - x-1: Host cluster offset. This is usually _not_ aligned to a - cluster or sector boundary! + cluster or sector boundary! If cluster_bits is + small enough that this field includes bits beyond + 55, those upper bits must be set to 0. x - 61: Number of additional 512-byte sectors used for the compressed data, beyond the sector containing the offs= et --=20 2.17.2