From nobody Mon Feb 9 07:57:10 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1577016134; cv=none; d=zohomail.com; s=zohoarc; b=AzKIk7XRt2QuG1npCgJg7+dY8S5sMudQeMxiA9BoYX26zv7Q0SmhPbTO0EivN+M2XbDZwt95B8KPYZtaSP5d7Z4EU0W8+Sai5tf8q5/eRfLEQn4wIVAUnv7lkfwEAlAMezYfTO9EYLd5w0FvMhy/5lyjOg1O7gPmEXhpLXrzvC0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1577016134; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HiUDYygHbreOJ4VpTqnZuGeG1uwk96nU2SwXCxyXjEw=; b=hWL7jRM3/5WczA4ZnAFEfdUzBL6QjFb9PFswVIdVhLJ2qLu5gqcHokiEf7sMp7wOp0IFjsAbWd+DRKD5qoBvJmBwHglaYQLk8mNxQfB7SoqvNEoEU7P3+QvE0wxgrGU/WjsfcdlIifFry/CWD9JKcrwYX8jP7FS6Er7fDoqEiyk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157701613467442.77618265505453; Sun, 22 Dec 2019 04:02:14 -0800 (PST) Received: from localhost ([::1]:47074 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iizw5-0002rq-Ky for importer@patchew.org; Sun, 22 Dec 2019 07:02:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:44283) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iizZE-0003VR-FT for qemu-devel@nongnu.org; Sun, 22 Dec 2019 06:38:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iizZC-00074S-3B for qemu-devel@nongnu.org; Sun, 22 Dec 2019 06:38:36 -0500 Received: from fanzine.igalia.com ([178.60.130.6]:35047) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iizZB-0004no-JC; Sun, 22 Dec 2019 06:38:33 -0500 Received: from [80.30.182.172] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1iizYU-0005d7-FQ; Sun, 22 Dec 2019 12:37:52 +0100 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1iizXu-0001Vc-Bw; Sun, 22 Dec 2019 12:37:14 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=HiUDYygHbreOJ4VpTqnZuGeG1uwk96nU2SwXCxyXjEw=; b=gCUQRQQmE6sV3CANOS4KIIp1mvmOGau7BqrarvBzkfKuqBZJNQz+9YRCp7ijiPlykD9NM80skRoqgrmRIe67zLVkQCPlanZFve9E4ysV+IdfA+DoGSbNcL4n8WNcWuL81327ml79yWgYMz3Nw0fJoFpKc47bagkp9Wd4ZHLjdPBxf/0ykzn/w8orBmd+1ARDW8sMlWcr7M0ipWYzil4l4yooDGrduMNdCz7FNWpnkGEVV1AAxCWhoZLupytXgqu0O8IGrLlvFb92cqmIP8Sb72mNAH63L6EFJfmURoRwJxOGqzsfW0ubbQlcfKE5P/SiuqZafs0AjNbYg6VaQa9AUg==; From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [RFC PATCH v3 05/27] qcow2: Document the Extended L2 Entries feature Date: Sun, 22 Dec 2019 12:36:46 +0100 Message-Id: <0b884ddcd0ac3a3c0b8cdd9d09c74566ac107c9a.1577014346.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Received-From: 178.60.130.6 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Anton Nefedov , Alberto Garcia , qemu-block@nongnu.org, Max Reitz , Vladimir Sementsov-Ogievskiy , "Denis V . Lunev" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Subcluster allocation in qcow2 is implemented by extending the existing L2 table entries and adding additional information to indicate the allocation status of each subcluster. This patch documents the changes to the qcow2 format and how they affect the calculation of the L2 cache size. Signed-off-by: Alberto Garcia --- docs/interop/qcow2.txt | 68 ++++++++++++++++++++++++++++++++++++++++-- docs/qcow2-cache.txt | 19 +++++++++++- 2 files changed, 83 insertions(+), 4 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index af5711e533..d34261f955 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -39,6 +39,9 @@ The first cluster of a qcow2 image contains the file head= er: as the maximum cluster size and won't be able to open = images with larger cluster sizes. =20 + Note: if the image has Extended L2 Entries then cluste= r_bits + must be at least 14 (i.e. 16384 byte clusters). + 24 - 31: size Virtual disk size in bytes. =20 @@ -109,7 +112,12 @@ in the description of a field. An External Data File Name header extensio= n may be present if this bit is set. =20 - Bits 3-63: Reserved (set to 0) + Bit 3: Extended L2 Entries. If this bit is set t= hen + L2 table entries use an extended format th= at + allows subcluster-based allocation. See the + Extended L2 Entries section for more detai= ls. + + Bits 4-63: Reserved (set to 0) =20 80 - 87: compatible_features Bitmask of compatible features. An implementation can @@ -437,7 +445,7 @@ cannot be relaxed without an incompatible layout change= ). Given an offset into the virtual disk, the offset into the image file can = be obtained as follows: =20 - l2_entries =3D (cluster_size / sizeof(uint64_t)) + l2_entries =3D (cluster_size / sizeof(uint64_t)) [*] =20 l2_index =3D (offset / cluster_size) % l2_entries l1_index =3D (offset / cluster_size) / l2_entries @@ -447,6 +455,8 @@ obtained as follows: =20 return cluster_offset + (offset % cluster_size) =20 + [*] this changes if Extended L2 Entries are enabled, see next section + L1 table entry: =20 Bit 0 - 8: Reserved (set to 0) @@ -487,7 +497,8 @@ Standard Cluster Descriptor: nor is data read from the backing file if the cluster = is unallocated. =20 - With version 2, this is always 0. + With version 2 or with extended L2 entries (see the ne= xt + section), this is always 0. =20 1 - 8: Reserved (set to 0) =20 @@ -524,6 +535,57 @@ file (except if bit 0 in the Standard Cluster Descript= or is set). If there is no backing file or the backing file is smaller than the image, they shall = read zeros for all parts that are not covered by the backing file. =20 +=3D=3D Extended L2 Entries =3D=3D + +An image uses Extended L2 Entries if bit 3 is set on the incompatible_feat= ures +field of the header. + +In these images standard data clusters are divided into 32 subclusters of = the +same size. They are contiguous and start from the beginning of the cluster. +Subclusters can be allocated independently and the L2 entry contains infor= mation +indicating the status of each one of them. Compressed data clusters don't = have +subclusters so they are treated like in images without this feature. + +The size of an extended L2 entry is 128 bits so the number of entries per = table +is calculated using this formula: + + l2_entries =3D (cluster_size / (2 * sizeof(uint64_t))) + +The first 64 bits have the same format as the standard L2 table entry desc= ribed +in the previous section, with the exception of bit 0 of the standard clust= er +descriptor. + +The last 64 bits contain a subcluster allocation bitmap with this format: + +Subcluster Allocation Bitmap (for standard clusters): + + Bit 0 - 31: Allocation status (one bit per subcluster) + + 1: the subcluster is allocated. In this case the + host cluster offset field must contain a valid + offset. + 0: the subcluster is not allocated. In this case + read requests shall go to the backing file or + return zeros if there is no backing file data. + + Bits are assigned starting from the most significant o= ne. + (i.e. bit x is used for subcluster 31 - x) + + 32 - 63 Subcluster reads as zeros (one bit per subcluster) + + 1: the subcluster reads as zeros. In this case the + allocation status bit must be unset. The host + cluster offset field may or may not be set. + 0: no effect. + + Bits are assigned starting from the most significant o= ne. + (i.e. bit x is used for subcluster 63 - x) + +Subcluster Allocation Bitmap (for compressed clusters): + + Bit 0 - 63: Reserved (set to 0) + Compressed clusters don't have subclusters, + so this field is not used. =20 =3D=3D Snapshots =3D=3D =20 diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt index d57f409861..04eb4ce2f1 100644 --- a/docs/qcow2-cache.txt +++ b/docs/qcow2-cache.txt @@ -1,6 +1,6 @@ qcow2 L2/refcount cache configuration =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Copyright (C) 2015, 2018 Igalia, S.L. +Copyright (C) 2015, 2018-2019 Igalia, S.L. Author: Alberto Garcia =20 This work is licensed under the terms of the GNU GPL, version 2 or @@ -222,3 +222,20 @@ support this functionality, and is 0 (disabled) on oth= er platforms. This functionality currently relies on the MADV_DONTNEED argument for madvise() to actually free the memory. This is a Linux-specific feature, so cache-clean-interval is not supported on other systems. + + +Extended L2 Entries +------------------- +All numbers shown in this document are valid for qcow2 images with normal +64-bit L2 entries. + +Images with extended L2 entries need twice as much L2 metadata, so the L2 +cache size must be twice as large for the same disk space. + + disk_size =3D l2_cache_size * cluster_size / 16 + +i.e. + + l2_cache_size =3D disk_size * 16 / cluster_size + +Refcount blocks are not affected by this. --=20 2.20.1