From nobody Sat May 4 19:08:14 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1493853191532237.52498022208215; Wed, 3 May 2017 16:13:11 -0700 (PDT) Received: from localhost ([::1]:38999 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d63SG-00039H-48 for importer@patchew.org; Wed, 03 May 2017 19:13:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59768) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d63Qj-0001zH-1Z for qemu-devel@nongnu.org; Wed, 03 May 2017 19:11:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d63Qi-0005w7-2t for qemu-devel@nongnu.org; Wed, 03 May 2017 19:11:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52286) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d63Qf-0005so-JT; Wed, 03 May 2017 19:11:29 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9756C8046A; Wed, 3 May 2017 23:11:28 +0000 (UTC) Received: from localhost (ovpn-204-47.brq.redhat.com [10.40.204.47]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2DF7B182E2; Wed, 3 May 2017 23:11:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 9756C8046A Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=mreitz@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 9756C8046A From: Max Reitz To: qemu-block@nongnu.org Date: Thu, 4 May 2017 01:11:17 +0200 Message-Id: <20170503231120.23507-2-mreitz@redhat.com> In-Reply-To: <20170503231120.23507-1-mreitz@redhat.com> References: <20170503231120.23507-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 03 May 2017 23:11:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 1/4] qcow2: Fix preallocation size formula X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When calculating the number of reftable entries, we should actually use the number of refblocks and not (wrongly[1]) re-calculate it. [1] "Wrongly" means: Dividing the number of clusters by the number of entries per refblock and rounding down instead of up. Reported-by: Eric Blake Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 5c1573c999..f5a72a4d2d 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2139,7 +2139,7 @@ static int qcow2_create2(const char *filename, int64_= t total_size, * too, as long as the bulk is allocated here). Therefore, using * floating point arithmetic is fine. */ int64_t meta_size =3D 0; - uint64_t nreftablee, nrefblocke, nl1e, nl2e; + uint64_t nreftablee, nrefblocke, nl1e, nl2e, refblock_count; int64_t aligned_total_size =3D align_offset(total_size, cluster_si= ze); int refblock_bits, refblock_size; /* refcount entry size in bytes */ @@ -2182,11 +2182,12 @@ static int qcow2_create2(const char *filename, int6= 4_t total_size, nrefblocke =3D (aligned_total_size + meta_size + cluster_size) / (cluster_size - rces - rces * sizeof(uint64_t) / cluster_size); - meta_size +=3D DIV_ROUND_UP(nrefblocke, refblock_size) * cluster_s= ize; + refblock_count =3D DIV_ROUND_UP(nrefblocke, refblock_size); + meta_size +=3D refblock_count * cluster_size; =20 /* total size of refcount tables */ - nreftablee =3D nrefblocke / refblock_size; - nreftablee =3D align_offset(nreftablee, cluster_size / sizeof(uint= 64_t)); + nreftablee =3D align_offset(refblock_count, + cluster_size / sizeof(uint64_t)); meta_size +=3D nreftablee * sizeof(uint64_t); =20 qemu_opt_set_number(opts, BLOCK_OPT_SIZE, --=20 2.12.2 From nobody Sat May 4 19:08:14 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 14938532721108.960220092904251; Wed, 3 May 2017 16:14:32 -0700 (PDT) Received: from localhost ([::1]:39006 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d63Ta-0004H6-Rq for importer@patchew.org; Wed, 03 May 2017 19:14:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59811) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d63Qm-00021q-B4 for qemu-devel@nongnu.org; Wed, 03 May 2017 19:11:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d63Ql-0005yp-3x for qemu-devel@nongnu.org; Wed, 03 May 2017 19:11:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59632) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d63Qh-0005vK-Qe; Wed, 03 May 2017 19:11:32 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EBF72C059733; Wed, 3 May 2017 23:11:30 +0000 (UTC) Received: from localhost (ovpn-204-47.brq.redhat.com [10.40.204.47]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7DEDB18242; Wed, 3 May 2017 23:11:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com EBF72C059733 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=mreitz@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com EBF72C059733 From: Max Reitz To: qemu-block@nongnu.org Date: Thu, 4 May 2017 01:11:18 +0200 Message-Id: <20170503231120.23507-3-mreitz@redhat.com> In-Reply-To: <20170503231120.23507-1-mreitz@redhat.com> References: <20170503231120.23507-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 03 May 2017 23:11:31 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 2/4] qcow2: Reuse preallocated zero clusters X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Instead of just freeing preallocated zero clusters and completely allocating them from scratch, reuse them. We cannot do this in handle_copied(), however, since this is a COW operation. Therefore, we have to add the new logic to handle_alloc() and simply return the existing offset if it exists. The only catch is that we have to convince qcow2_alloc_cluster_link_l2() not to free the old clusters (because we have reused them). Reported-by: Eric Blake Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.h | 3 ++ block/qcow2-cluster.c | 80 +++++++++++++++++++++++++++++++++++------------= ---- 2 files changed, 59 insertions(+), 24 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index f8aeb08794..8731f24b82 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -322,6 +322,9 @@ typedef struct QCowL2Meta /** Number of newly allocated clusters */ int nb_clusters; =20 + /** Do not free the old clusters */ + bool keep_old_clusters; + /** * Requests that overlap with this allocation and wait to be restarted * when the allocating request has completed. diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 100398c565..fb91fd8979 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -309,14 +309,20 @@ static int count_contiguous_clusters(int nb_clusters,= int cluster_size, uint64_t *l2_table, uint64_t stop_flags) { int i; + int first_cluster_type; uint64_t mask =3D stop_flags | L2E_OFFSET_MASK | QCOW_OFLAG_COMPRESSED; uint64_t first_entry =3D be64_to_cpu(l2_table[0]); uint64_t offset =3D first_entry & mask; =20 - if (!offset) + if (!offset) { return 0; + } =20 - assert(qcow2_get_cluster_type(first_entry) =3D=3D QCOW2_CLUSTER_NORMAL= ); + /* must be allocated */ + first_cluster_type =3D qcow2_get_cluster_type(first_entry); + assert(first_cluster_type =3D=3D QCOW2_CLUSTER_NORMAL || + (first_cluster_type =3D=3D QCOW2_CLUSTER_ZERO && + (first_entry & L2E_OFFSET_MASK) !=3D 0)); =20 for (i =3D 0; i < nb_clusters; i++) { uint64_t l2_entry =3D be64_to_cpu(l2_table[i]) & mask; @@ -835,7 +841,7 @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, Q= CowL2Meta *m) * Don't discard clusters that reach a refcount of 0 (e.g. compressed * clusters), the next write will reuse them anyway. */ - if (j !=3D 0) { + if (!m->keep_old_clusters && j !=3D 0) { for (i =3D 0; i < j; i++) { qcow2_free_any_clusters(bs, be64_to_cpu(old_cluster[i]), 1, QCOW2_DISCARD_NEVER); @@ -1132,8 +1138,9 @@ static int handle_alloc(BlockDriverState *bs, uint64_= t guest_offset, uint64_t entry; uint64_t nb_clusters; int ret; + bool keep_old_clusters =3D false; =20 - uint64_t alloc_cluster_offset; + uint64_t alloc_cluster_offset =3D 0; =20 trace_qcow2_handle_alloc(qemu_coroutine_self(), guest_offset, *host_of= fset, *bytes); @@ -1170,31 +1177,54 @@ static int handle_alloc(BlockDriverState *bs, uint6= 4_t guest_offset, * wrong with our code. */ assert(nb_clusters > 0); =20 - qcow2_cache_put(bs, s->l2_table_cache, (void **) &l2_table); + if (qcow2_get_cluster_type(entry) =3D=3D QCOW2_CLUSTER_ZERO && + (entry & L2E_OFFSET_MASK) !=3D 0 && (entry & QCOW_OFLAG_COPIED) && + (!*host_offset || + start_of_cluster(s, *host_offset) =3D=3D (entry & L2E_OFFSET_MASK= ))) + { + /* Try to reuse preallocated zero clusters; contiguous normal clus= ters + * would be fine, too, but count_cow_clusters() above has limited + * nb_clusters already to a range of COW clusters */ + int preallocated_nb_clusters =3D + count_contiguous_clusters(nb_clusters, s->cluster_size, + &l2_table[l2_index], QCOW_OFLAG_COPI= ED); + assert(preallocated_nb_clusters > 0); =20 - /* Allocate, if necessary at a given offset in the image file */ - alloc_cluster_offset =3D start_of_cluster(s, *host_offset); - ret =3D do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_offse= t, - &nb_clusters); - if (ret < 0) { - goto fail; - } + nb_clusters =3D preallocated_nb_clusters; + alloc_cluster_offset =3D entry & L2E_OFFSET_MASK; =20 - /* Can't extend contiguous allocation */ - if (nb_clusters =3D=3D 0) { - *bytes =3D 0; - return 0; + /* We want to reuse these clusters, so qcow2_alloc_cluster_link_l2= () + * should not free them. */ + keep_old_clusters =3D true; } =20 - /* !*host_offset would overwrite the image header and is reserved for = "no - * host offset preferred". If 0 was a valid host offset, it'd trigger = the - * following overlap check; do that now to avoid having an invalid val= ue in - * *host_offset. */ + qcow2_cache_put(bs, s->l2_table_cache, (void **) &l2_table); + if (!alloc_cluster_offset) { - ret =3D qcow2_pre_write_overlap_check(bs, 0, alloc_cluster_offset, - nb_clusters * s->cluster_size); - assert(ret < 0); - goto fail; + /* Allocate, if necessary at a given offset in the image file */ + alloc_cluster_offset =3D start_of_cluster(s, *host_offset); + ret =3D do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_o= ffset, + &nb_clusters); + if (ret < 0) { + goto fail; + } + + /* Can't extend contiguous allocation */ + if (nb_clusters =3D=3D 0) { + *bytes =3D 0; + return 0; + } + + /* !*host_offset would overwrite the image header and is reserved = for + * "no host offset preferred". If 0 was a valid host offset, it'd + * trigger the following overlap check; do that now to avoid havin= g an + * invalid value in *host_offset. */ + if (!alloc_cluster_offset) { + ret =3D qcow2_pre_write_overlap_check(bs, 0, alloc_cluster_off= set, + nb_clusters * s->cluster_s= ize); + assert(ret < 0); + goto fail; + } } =20 /* @@ -1225,6 +1255,8 @@ static int handle_alloc(BlockDriverState *bs, uint64_= t guest_offset, .offset =3D start_of_cluster(s, guest_offset), .nb_clusters =3D nb_clusters, =20 + .keep_old_clusters =3D keep_old_clusters, + .cow_start =3D { .offset =3D 0, .nb_bytes =3D offset_into_cluster(s, guest_offset), --=20 2.12.2 From nobody Sat May 4 19:08:14 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1493853333773596.0531568265213; Wed, 3 May 2017 16:15:33 -0700 (PDT) Received: from localhost ([::1]:39013 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d63Ua-0004sQ-IV for importer@patchew.org; Wed, 03 May 2017 19:15:32 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59830) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d63Qp-00023S-Kn for qemu-devel@nongnu.org; Wed, 03 May 2017 19:11:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d63Qo-000617-7U for qemu-devel@nongnu.org; Wed, 03 May 2017 19:11:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44274) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d63Qk-0005xS-8V; Wed, 03 May 2017 19:11:34 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5C96F201FB; Wed, 3 May 2017 23:11:33 +0000 (UTC) Received: from localhost (ovpn-204-47.brq.redhat.com [10.40.204.47]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D065F182E5; Wed, 3 May 2017 23:11:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5C96F201FB Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=mreitz@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 5C96F201FB From: Max Reitz To: qemu-block@nongnu.org Date: Thu, 4 May 2017 01:11:19 +0200 Message-Id: <20170503231120.23507-4-mreitz@redhat.com> In-Reply-To: <20170503231120.23507-1-mreitz@redhat.com> References: <20170503231120.23507-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 03 May 2017 23:11:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 3/4] qcow2: Discard preallocated zero clusters X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" In discard_single_l2(), we completely discard normal clusters instead of simply turning them into preallocated zero clusters. That means we should probably do the same with such preallocated zero clusters: Discard them instead of keeping them allocated. Reported-by: Eric Blake Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index fb91fd8979..31077d8102 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1511,7 +1511,8 @@ static int discard_single_l2(BlockDriverState *bs, ui= nt64_t offset, break; =20 case QCOW2_CLUSTER_ZERO: - if (!full_discard) { + /* Preallocated zero clusters should be discarded in any c= ase */ + if (!full_discard && (old_l2_entry & L2E_OFFSET_MASK) =3D= =3D 0) { continue; } break; --=20 2.12.2 From nobody Sat May 4 19:08:14 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1493853401074547.1075996256857; Wed, 3 May 2017 16:16:41 -0700 (PDT) Received: from localhost ([::1]:39016 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d63Vf-0005Oe-08 for importer@patchew.org; Wed, 03 May 2017 19:16:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59847) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d63Qs-00026F-BG for qemu-devel@nongnu.org; Wed, 03 May 2017 19:11:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d63Qr-00063C-1n for qemu-devel@nongnu.org; Wed, 03 May 2017 19:11:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:13139) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d63Qm-0005zi-RH; Wed, 03 May 2017 19:11:37 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E7DE0201FB; Wed, 3 May 2017 23:11:35 +0000 (UTC) Received: from localhost (ovpn-204-47.brq.redhat.com [10.40.204.47]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4353C7EBC7; Wed, 3 May 2017 23:11:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E7DE0201FB Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=mreitz@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com E7DE0201FB From: Max Reitz To: qemu-block@nongnu.org Date: Thu, 4 May 2017 01:11:20 +0200 Message-Id: <20170503231120.23507-5-mreitz@redhat.com> In-Reply-To: <20170503231120.23507-1-mreitz@redhat.com> References: <20170503231120.23507-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 03 May 2017 23:11:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 4/4] iotests: Extend test 066 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" 066 was supposed to be a test "for discarding preallocated zero clusters", but it did so incompletely: While it did check the image file's integrity after the operation, it did not confirm that the clusters are indeed freed. This patch adds this test. In addition, new cases for writing to preallocated zero clusters are added. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/066 | 128 +++++++++++++++++++++++++++++++++++++++++= +++- tests/qemu-iotests/066.out | 46 ++++++++++++++++ 2 files changed, 173 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/066 b/tests/qemu-iotests/066 index c2116a3088..8638217736 100755 --- a/tests/qemu-iotests/066 +++ b/tests/qemu-iotests/066 @@ -1,6 +1,6 @@ #!/bin/bash # -# Test case for discarding preallocated zero clusters in qcow2 +# Test case for preallocated zero clusters in qcow2 # # Copyright (C) 2013 Red Hat, Inc. # @@ -55,8 +55,134 @@ _make_test_img $IMG_SIZE $QEMU_IO -c "write 0 256k" -c "write -z 0 256k" -c "write 64M 512" \ -c "discard 0 $IMG_SIZE" -c "read -P 0 0 $IMG_SIZE" "$TEST_IMG" \ | _filter_qemu_io + # Check the image (there shouldn't be any leaks) _check_test_img +# Map the image (we want all clusters to be gone) +$QEMU_IMG map "$TEST_IMG" + +_cleanup_test_img + + +echo +echo '=3D=3D=3D Writing to preallocated zero clusters =3D=3D=3D' +echo + +_make_test_img $IMG_SIZE + +# Create data clusters (not aligned to an L2 table) +$QEMU_IO -c 'write -P 42 1M 256k' "$TEST_IMG" | _filter_qemu_io +orig_map=3D$($QEMU_IMG map --output=3Djson "$TEST_IMG") + +# Convert the data clusters to preallocated zero clusters +$QEMU_IO -c 'write -z 1M 256k' "$TEST_IMG" | _filter_qemu_io + +# Now write to them (with a COW needed for the head and tail) +$QEMU_IO -c "write -P 23 $(((1024 + 32) * 1024)) 192k" "$TEST_IMG" \ + | _filter_qemu_io + +# Check metadata correctness +_check_test_img + +# Check data correctness +$QEMU_IO -c "read -P 0 $(( 1024 * 1024)) 32k" \ + -c "read -P 23 $(((1024 + 32) * 1024)) 192k" \ + -c "read -P 0 $(((1024 + 32 + 192) * 1024)) 32k" \ + "$TEST_IMG" \ + | _filter_qemu_io + +# Check that we have actually reused the original area +new_map=3D$($QEMU_IMG map --output=3Djson "$TEST_IMG") +if [ "$new_map" =3D "$orig_map" ]; then + echo 'Successfully reused original clusters.' +else + echo 'Failed to reuse original clusters.' + echo 'Original map:' + echo "$orig_map" + echo 'New map:' + echo "$new_map" +fi + +_cleanup_test_img + + +echo +echo '=3D=3D=3D Writing to a snapshotted preallocated zero cluster =3D=3D= =3D' +echo + +_make_test_img 64k + +# Create a preallocated zero cluster +$QEMU_IO -c 'write -P 42 0 64k' -c 'write -z 0 64k' "$TEST_IMG" \ + | _filter_qemu_io + +# Snapshot it +$QEMU_IMG snapshot -c foo "$TEST_IMG" + +# Write to the cluster +$QEMU_IO -c 'write -P 23 0 64k' "$TEST_IMG" | _filter_qemu_io + +# Check metadata correctness +_check_test_img + +# Check data correctness +$QEMU_IO -c 'read -P 23 0 64k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG snapshot -a foo "$TEST_IMG" +$QEMU_IO -c 'read -P 0 0 64k' "$TEST_IMG" | _filter_qemu_io + +_cleanup_test_img + + +echo +echo '=3D=3D=3D Consecutive write to a preallocated zero cluster =3D=3D=3D' +echo + +_make_test_img 192k + +# Create three normal clusters +$QEMU_IO -c 'write -P 42 0 192k' "$TEST_IMG" | _filter_qemu_io +orig_map=3D$($QEMU_IMG map --output=3Djson "$TEST_IMG") + +# Make the middle cluster a preallocated zero cluster +$QEMU_IO -c 'write -z 64k 64k' "$TEST_IMG" | _filter_qemu_io + +# Try to overwrite everything: This should reuse the whole range. To test = that +# this only issues a single continuous write request, use blkdebug. +$QEMU_IO -c 'write -P 42 0 192k' \ + "json:{ + 'driver': '$IMGFMT', + 'file': { + 'driver': 'blkdebug', + 'image.filename': '$TEST_IMG', + 'set-state': [{ + 'event': 'write_aio', + 'new_state': 2 + }], + 'inject-error': [{ + 'event': 'write_aio', + 'state': 2 + }] + } + }" \ + | _filter_qemu_io + +# Check metadata correctness +_check_test_img + +# Check that we have actually reused the original area +new_map=3D$($QEMU_IMG map --output=3Djson "$TEST_IMG") +if [ "$new_map" =3D "$orig_map" ]; then + echo 'Successfully reused original clusters.' +else + echo 'Failed to reuse original clusters.' + echo 'Original map:' + echo "$orig_map" + echo 'New map:' + echo "$new_map" +fi + +_cleanup_test_img + =20 # success, all done echo "*** done" diff --git a/tests/qemu-iotests/066.out b/tests/qemu-iotests/066.out index 7c1f31a1b1..3d9da9bd0b 100644 --- a/tests/qemu-iotests/066.out +++ b/tests/qemu-iotests/066.out @@ -14,4 +14,50 @@ discard 67109376/67109376 bytes at offset 0 read 67109376/67109376 bytes at offset 0 64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) No errors were found on the image. +Offset Length Mapped to File + +=3D=3D=3D Writing to preallocated zero clusters =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67109376 +wrote 262144/262144 bytes at offset 1048576 +256 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 262144/262144 bytes at offset 1048576 +256 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 196608/196608 bytes at offset 1081344 +192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +No errors were found on the image. +read 32768/32768 bytes at offset 1048576 +32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 196608/196608 bytes at offset 1081344 +192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 32768/32768 bytes at offset 1277952 +32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Successfully reused original clusters. + +=3D=3D=3D Writing to a snapshotted preallocated zero cluster =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D65536 +wrote 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +No errors were found on the image. +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=3D=3D=3D Consecutive write to a preallocated zero cluster =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D196608 +wrote 196608/196608 bytes at offset 0 +192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 65536/65536 bytes at offset 65536 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 196608/196608 bytes at offset 0 +192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +No errors were found on the image. +Successfully reused original clusters. *** done --=20 2.12.2