From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680419841431.64964990331964; Tue, 14 Nov 2017 09:26:59 -0800 (PST) Received: from localhost ([::1]:60988 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEeyl-0007Qe-3U for importer@patchew.org; Tue, 14 Nov 2017 12:26:31 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53518) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEewn-0005Om-Ex for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEewm-00047Y-9S for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:29 -0500 Received: from mx1.redhat.com ([209.132.183.28]:46314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEewh-00043F-Rs; Tue, 14 Nov 2017 12:24:24 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 08BF6C047B8C; Tue, 14 Nov 2017 17:24:23 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 891E04C7; Tue, 14 Nov 2017 17:24:22 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:23:58 +0100 Message-Id: <20171114172417.7654-2-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 14 Nov 2017 17:24:23 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 01/20] qcow2: Prevent allocating refcount blocks at offset 0 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Alberto Garcia , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia Each entry in the qcow2 cache contains an offset field indicating the location of the data in the qcow2 image. If the offset is 0 then it means that the entry contains no data and is available to be used when needed. Because of that it is not possible to store in the cache the first cluster of the qcow2 image (offset =3D 0). This is not a problem because that cluster always contains the qcow2 header and we're not using this cache for that. However, if the qcow2 image is corrupted it can happen that we try to allocate a new refcount block at offset 0, triggering this assertion and crashing QEMU: qcow2_cache_entry_mark_dirty: Assertion `c->entries[i].offset !=3D 0' fai= led This patch adds an explicit check for this scenario and a new test case. This problem was originally reported here: https://bugs.launchpad.net/qemu/+bug/1728615 Reported-by: R.Nageswara Sastry Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-id: 92a2fadd10d58b423f269c1d1a309af161cdc73f.1509718618.git.berto@i= galia.com Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 7 +++++++ tests/qemu-iotests/060 | 11 +++++++++++ tests/qemu-iotests/060.out | 8 ++++++++ 3 files changed, 26 insertions(+) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index aa3fd6cf17..9059996c4b 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -367,6 +367,13 @@ static int alloc_refcount_block(BlockDriverState *bs, return new_block; } =20 + /* If we're allocating the block at offset 0 then something is wrong */ + if (new_block =3D=3D 0) { + qcow2_signal_corruption(bs, true, -1, -1, "Preventing invalid " + "allocation of refcount block at offset 0"= ); + return -EIO; + } + #ifdef DEBUG_ALLOC2 fprintf(stderr, "qcow2: Allocate refcount block %d for %" PRIx64 " at %" PRIx64 "\n", diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index 8e95c450eb..dead26aeaf 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -242,6 +242,17 @@ poke_file "$TEST_IMG" "$(($l2_offset+8))" "\x80\x00\x0= 0\x00\x00\x06\x2a\x00" # Should emit two error messages $QEMU_IO -c "discard 0 64k" -c "read 64k 64k" "$TEST_IMG" | _filter_qemu_io =20 +echo +echo "=3D=3D=3D Testing empty refcount table with valid L1 and L2 tables = =3D=3D=3D" +echo +_make_test_img 64M +$QEMU_IO -c "write 0 64k" "$TEST_IMG" | _filter_qemu_io +poke_file "$TEST_IMG" "$rt_offset" "\x00\x00\x00\x00\x00\x00\x00\x0= 0" +# Since the first data cluster is already allocated this triggers an +# allocation with an explicit offset (using qcow2_alloc_clusters_at()) +# causing a refcount block to be allocated at offset 0 +$QEMU_IO -c "write 0 128k" "$TEST_IMG" | _filter_qemu_io + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index 5ca3af491f..872719009c 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -181,4 +181,12 @@ qcow2: Marking image as corrupt: Cluster allocation of= fset 0x62a00 unaligned (L2 discard 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read failed: Input/output error + +=3D=3D=3D Testing empty refcount table with valid L1 and L2 tables =3D=3D= =3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +wrote 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qcow2: Marking image as corrupt: Preventing invalid allocation of refcount= block at offset 0; further corruption events will be suppressed +write failed: Input/output error *** done --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680575018242.43474162049984; Tue, 14 Nov 2017 09:29:35 -0800 (PST) Received: from localhost ([::1]:32768 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf1I-0002Lv-IW for importer@patchew.org; Tue, 14 Nov 2017 12:29:08 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53539) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEewo-0005Pd-ES for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEewn-00048k-8y for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:30 -0500 Received: from mx1.redhat.com ([209.132.183.28]:41290) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEewk-00044b-Jb; Tue, 14 Nov 2017 12:24:26 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BE3C05DA03; Tue, 14 Nov 2017 17:24:25 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 393564C7; Tue, 14 Nov 2017 17:24:24 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:23:59 +0100 Message-Id: <20171114172417.7654-3-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 14 Nov 2017 17:24:25 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 02/20] qcow2: Prevent allocating L2 tables at offset 0 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Alberto Garcia , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia If the refcount data is corrupted then we can end up trying to allocate a new L2 table at offset 0 in the image, triggering an assertion in the qcow2 cache that would crash QEMU: qcow2_cache_entry_mark_dirty: Assertion `c->entries[i].offset !=3D 0' fai= led This patch adds an explicit check for this scenario and a new test case. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-id: 92dac37191ae7844a2da22c122204eb493cc3133.1509718618.git.berto@i= galia.com Signed-off-by: Max Reitz --- block/qcow2-cluster.c | 8 ++++++++ tests/qemu-iotests/060 | 7 +++++++ tests/qemu-iotests/060.out | 6 ++++++ 3 files changed, 21 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index fb10e26068..2e072ed155 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -278,6 +278,14 @@ static int l2_allocate(BlockDriverState *bs, int l1_in= dex, uint64_t **table) goto fail; } =20 + /* If we're allocating the table at offset 0 then something is wrong */ + if (l2_offset =3D=3D 0) { + qcow2_signal_corruption(bs, true, -1, -1, "Preventing invalid " + "allocation of L2 table at offset 0"); + ret =3D -EIO; + goto fail; + } + ret =3D qcow2_cache_flush(bs, s->refcount_block_cache); if (ret < 0) { goto fail; diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index dead26aeaf..40f85cc216 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -253,6 +253,13 @@ poke_file "$TEST_IMG" "$rt_offset" "\x00\x00\x0= 0\x00\x00\x00\x00\x00" # causing a refcount block to be allocated at offset 0 $QEMU_IO -c "write 0 128k" "$TEST_IMG" | _filter_qemu_io =20 +echo +echo "=3D=3D=3D Testing empty refcount block =3D=3D=3D" +echo +_make_test_img 64M +poke_file "$TEST_IMG" "$rb_offset" "\x00\x00\x00\x00\x00\x00\x00\x0= 0" +$QEMU_IO -c "write 0 64k" "$TEST_IMG" | _filter_qemu_io + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index 872719009c..5b8b518486 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -189,4 +189,10 @@ wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) qcow2: Marking image as corrupt: Preventing invalid allocation of refcount= block at offset 0; further corruption events will be suppressed write failed: Input/output error + +=3D=3D=3D Testing empty refcount block =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +qcow2: Marking image as corrupt: Preventing invalid allocation of L2 table= at offset 0; further corruption events will be suppressed +write failed: Input/output error *** done --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680729911873.6876047417537; Tue, 14 Nov 2017 09:32:09 -0800 (PST) Received: from localhost ([::1]:32789 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf3p-0004ce-Or for importer@patchew.org; Tue, 14 Nov 2017 12:31:45 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53572) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEewq-0005S8-Ry for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:37 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEewp-0004AM-Ov for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:32 -0500 Received: from mx1.redhat.com ([209.132.183.28]:41878) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEewn-00047w-6C; Tue, 14 Nov 2017 12:24:29 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 60D5961D07; Tue, 14 Nov 2017 17:24:28 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CFF6087108; Tue, 14 Nov 2017 17:24:27 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:00 +0100 Message-Id: <20171114172417.7654-4-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 14 Nov 2017 17:24:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 03/20] qcow2: Prevent allocating compressed clusters at offset 0 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Alberto Garcia , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia If the refcount data is corrupted then we can end up trying to allocate a new compressed cluster at offset 0 in the image, triggering an assertion in qcow2_alloc_bytes() that would crash QEMU: qcow2_alloc_bytes: Assertion `offset' failed. This patch adds an explicit check for this scenario and a new test case. Signed-off-by: Alberto Garcia Message-id: fb53467cf48e95ff3330def1cf1003a5b862b7d9.1509718618.git.berto@i= galia.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 7 +++++++ tests/qemu-iotests/060 | 10 ++++++++++ tests/qemu-iotests/060.out | 8 ++++++++ 3 files changed, 25 insertions(+) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 9059996c4b..60b8eef3e8 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1082,6 +1082,13 @@ int64_t qcow2_alloc_bytes(BlockDriverState *bs, int = size) return new_cluster; } =20 + if (new_cluster =3D=3D 0) { + qcow2_signal_corruption(bs, true, -1, -1, "Preventing inva= lid " + "allocation of compressed cluster " + "at offset 0"); + return -EIO; + } + if (!offset || ROUND_UP(offset, s->cluster_size) !=3D new_clus= ter) { offset =3D new_cluster; free_in_cluster =3D s->cluster_size; diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index 40f85cc216..c3bce27b33 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -260,6 +260,16 @@ _make_test_img 64M poke_file "$TEST_IMG" "$rb_offset" "\x00\x00\x00\x00\x00\x00\x00\x0= 0" $QEMU_IO -c "write 0 64k" "$TEST_IMG" | _filter_qemu_io =20 +echo +echo "=3D=3D=3D Testing empty refcount block with compressed write =3D=3D= =3D" +echo +_make_test_img 64M +$QEMU_IO -c "write 64k 64k" "$TEST_IMG" | _filter_qemu_io +poke_file "$TEST_IMG" "$rb_offset" "\x00\x00\x00\x00\x00\x00\x00\x0= 0" +# The previous write already allocated an L2 table, so now this new +# write will try to allocate a compressed data cluster at offset 0. +$QEMU_IO -c "write -c 0k 64k" "$TEST_IMG" | _filter_qemu_io + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index 5b8b518486..cf8790ff57 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -195,4 +195,12 @@ write failed: Input/output error Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 qcow2: Marking image as corrupt: Preventing invalid allocation of L2 table= at offset 0; further corruption events will be suppressed write failed: Input/output error + +=3D=3D=3D Testing empty refcount block with compressed write =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +wrote 65536/65536 bytes at offset 65536 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qcow2: Marking image as corrupt: Preventing invalid allocation of compress= ed cluster at offset 0; further corruption events will be suppressed +write failed: Input/output error *** done --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680436684497.95865825846045; Tue, 14 Nov 2017 09:27:16 -0800 (PST) Received: from localhost ([::1]:60990 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEezL-0007vy-KS for importer@patchew.org; Tue, 14 Nov 2017 12:27:07 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53660) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEewx-0005YP-Gm for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEeww-0004Dn-IE for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:39 -0500 Received: from mx1.redhat.com ([209.132.183.28]:46526) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEewp-00049w-Ur; Tue, 14 Nov 2017 12:24:32 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 29EF0C04AC7C; Tue, 14 Nov 2017 17:24:31 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6DB6087105; Tue, 14 Nov 2017 17:24:30 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:01 +0100 Message-Id: <20171114172417.7654-5-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 14 Nov 2017 17:24:31 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 04/20] qcow2: Don't open images with header.refcount_table_clusters == 0 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Alberto Garcia , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia qcow2_do_open() is checking that header.refcount_table_clusters is not too large, but it doesn't check that it's greater than zero. Apart from the fact that an image like that is obviously corrupted, trying to use it crashes QEMU since we end up with a null s->refcount_table after qcow2_refcount_init(). These images can however be repaired, so allow opening them if the BDRV_O_CHECK flag is set. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-id: f9750f50c80359babba11062e88f5075a47e8e16.1509718618.git.berto@i= galia.com Signed-off-by: Max Reitz --- block/qcow2.c | 6 ++++++ tests/qemu-iotests/060 | 7 +++++++ tests/qemu-iotests/060.out | 5 +++++ 3 files changed, 18 insertions(+) diff --git a/block/qcow2.c b/block/qcow2.c index 92cb9f9bfa..defc1fe49f 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1280,6 +1280,12 @@ static int qcow2_do_open(BlockDriverState *bs, QDict= *options, int flags, goto fail; } =20 + if (header.refcount_table_clusters =3D=3D 0 && !(flags & BDRV_O_CHECK)= ) { + error_setg(errp, "Image does not contain a reference count table"); + ret =3D -EINVAL; + goto fail; + } + ret =3D validate_table_offset(bs, s->refcount_table_offset, s->refcount_table_size, sizeof(uint64_t)); if (ret < 0) { diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index c3bce27b33..656af50883 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -270,6 +270,13 @@ poke_file "$TEST_IMG" "$rb_offset" "\x00\x00\x0= 0\x00\x00\x00\x00\x00" # write will try to allocate a compressed data cluster at offset 0. $QEMU_IO -c "write -c 0k 64k" "$TEST_IMG" | _filter_qemu_io =20 +echo +echo "=3D=3D=3D Testing zero refcount table size =3D=3D=3D" +echo +_make_test_img 64M +poke_file "$TEST_IMG" "56" "\x00\x00\x00\x00" +$QEMU_IO -c "write 0 64k" "$TEST_IMG" 2>&1 | _filter_testdir | _filter_img= fmt + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index cf8790ff57..58456e8487 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -203,4 +203,9 @@ wrote 65536/65536 bytes at offset 65536 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) qcow2: Marking image as corrupt: Preventing invalid allocation of compress= ed cluster at offset 0; further corruption events will be suppressed write failed: Input/output error + +=3D=3D=3D Testing zero refcount table size =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +can't open device TEST_DIR/t.IMGFMT: Image does not contain a reference co= unt table *** done --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680566134189.87312920655359; Tue, 14 Nov 2017 09:29:26 -0800 (PST) Received: from localhost ([::1]:32769 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf1K-0002NP-6v for importer@patchew.org; Tue, 14 Nov 2017 12:29:10 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53657) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEewx-0005YG-D5 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEeww-0004Di-GN for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:39 -0500 Received: from mx1.redhat.com ([209.132.183.28]:41450) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEews-0004BO-Ee; Tue, 14 Nov 2017 12:24:34 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A190981E1A; Tue, 14 Nov 2017 17:24:33 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 30FD1183D0; Tue, 14 Nov 2017 17:24:33 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:02 +0100 Message-Id: <20171114172417.7654-6-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 14 Nov 2017 17:24:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 05/20] qcow2: Add iotest for an image with header.refcount_table_offset == 0 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Alberto Garcia , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia This patch adds a simple iotest in which we try to write to an image with the refcount table offset set to 0. This scenario was already handled by the existing consistency checks, but we add an explicit test case for completeness. Signed-off-by: Alberto Garcia Message-id: feeceada92486bb8790b90f303fc9fe82a27391a.1509718618.git.berto@i= galia.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- tests/qemu-iotests/060 | 7 +++++++ tests/qemu-iotests/060.out | 6 ++++++ 2 files changed, 13 insertions(+) diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index 656af50883..dc5a517673 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -277,6 +277,13 @@ _make_test_img 64M poke_file "$TEST_IMG" "56" "\x00\x00\x00\x00" $QEMU_IO -c "write 0 64k" "$TEST_IMG" 2>&1 | _filter_testdir | _filter_img= fmt =20 +echo +echo "=3D=3D=3D Testing incorrect refcount table offset =3D=3D=3D" +echo +_make_test_img 64M +poke_file "$TEST_IMG" "48" "\x00\x00\x00\x00\x00\x00\x00\x0= 0" +$QEMU_IO -c "write 0 64k" "$TEST_IMG" | _filter_qemu_io + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index 58456e8487..98f314c16d 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -208,4 +208,10 @@ write failed: Input/output error =20 Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 can't open device TEST_DIR/t.IMGFMT: Image does not contain a reference co= unt table + +=3D=3D=3D Testing incorrect refcount table offset =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +qcow2: Marking image as corrupt: Preventing invalid allocation of L2 table= at offset 0; further corruption events will be suppressed +write failed: Input/output error *** done --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680891550818.5871426278079; Tue, 14 Nov 2017 09:34:51 -0800 (PST) Received: from localhost ([::1]:32810 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf6e-0007Of-Pc for importer@patchew.org; Tue, 14 Nov 2017 12:34:40 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53684) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEewy-0005ZQ-8Y for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEewx-0004Ed-CM for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:40 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37524) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEewv-0004CR-1i; Tue, 14 Nov 2017 12:24:37 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 321E8C045741; Tue, 14 Nov 2017 17:24:36 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9F64296B60; Tue, 14 Nov 2017 17:24:35 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:03 +0100 Message-Id: <20171114172417.7654-7-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 14 Nov 2017 17:24:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 06/20] qcow2: Add iotest for an empty refcount table X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Alberto Garcia , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia This patch adds a simple iotest in which we try to write to an image with an empty refcount table (i.e. with all entries set to 0). This scenario was already handled by the existing consistency checks, but we add an explicit test case for completeness. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-id: 7e48b0e2ae1a0a18e0ee303b3045f130feec0474.1509718618.git.berto@i= galia.com Signed-off-by: Max Reitz --- tests/qemu-iotests/060 | 7 +++++++ tests/qemu-iotests/060.out | 6 ++++++ 2 files changed, 13 insertions(+) diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index dc5a517673..66a8fa4aea 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -243,6 +243,13 @@ poke_file "$TEST_IMG" "$(($l2_offset+8))" "\x80\x00\x0= 0\x00\x00\x06\x2a\x00" $QEMU_IO -c "discard 0 64k" -c "read 64k 64k" "$TEST_IMG" | _filter_qemu_io =20 echo +echo "=3D=3D=3D Testing empty refcount table =3D=3D=3D" +echo +_make_test_img 64M +poke_file "$TEST_IMG" "$rt_offset" "\x00\x00\x00\x00\x00\x00\x00\x0= 0" +$QEMU_IO -c "write 0 64k" "$TEST_IMG" | _filter_qemu_io + +echo echo "=3D=3D=3D Testing empty refcount table with valid L1 and L2 tables = =3D=3D=3D" echo _make_test_img 64M diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index 98f314c16d..cfd78f87a9 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -182,6 +182,12 @@ discard 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read failed: Input/output error =20 +=3D=3D=3D Testing empty refcount table =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +qcow2: Marking image as corrupt: Preventing invalid write on metadata (ove= rlaps with refcount table); further corruption events will be suppressed +write failed: Input/output error + =3D=3D=3D Testing empty refcount table with valid L1 and L2 tables =3D=3D= =3D =20 Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680620256939.5048934318902; Tue, 14 Nov 2017 09:30:20 -0800 (PST) Received: from localhost ([::1]:32771 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf24-00031r-Bn for importer@patchew.org; Tue, 14 Nov 2017 12:29:56 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53777) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEex5-0005ra-TM for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEex1-0004Gd-0l for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:47 -0500 Received: from mx1.redhat.com ([209.132.183.28]:53360) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEewx-0004EZ-RL; Tue, 14 Nov 2017 12:24:39 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 147BE15447; Tue, 14 Nov 2017 17:24:39 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 82B6491834; Tue, 14 Nov 2017 17:24:38 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:04 +0100 Message-Id: <20171114172417.7654-8-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 14 Nov 2017 17:24:39 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 07/20] qcow2: Assert that the crypto header does not overlap other metadata X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Alberto Garcia , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia The crypto header is initialized only when QEMU is creating a new image, so there's no chance of this happening on a corrupted image. If QEMU is really trying to allocate the header overlapping other existing metadata sections then this is a serious bug in QEMU itself so let's add an assertion. Signed-off-by: Alberto Garcia Message-id: ae3d77f312fc0c5e0ac2bbd71676c0112eebe2e5.1509718618.git.berto@i= galia.com Reviewed-by: Daniel P. Berrange Signed-off-by: Max Reitz --- block/qcow2.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/qcow2.c b/block/qcow2.c index defc1fe49f..b3d66a0e88 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -126,6 +126,7 @@ static ssize_t qcow2_crypto_hdr_init_func(QCryptoBlock = *block, size_t headerlen, /* Zero fill remaining space in cluster so it has predictable * content in case of future spec changes */ clusterlen =3D size_to_clusters(s, headerlen) * s->cluster_size; + assert(qcow2_pre_write_overlap_check(bs, 0, ret, clusterlen) =3D=3D 0); ret =3D bdrv_pwrite_zeroes(bs->file, ret + headerlen, clusterlen - headerlen, 0); --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680724909274.45133246559146; Tue, 14 Nov 2017 09:32:04 -0800 (PST) Received: from localhost ([::1]:32790 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf3s-0004ds-FJ for importer@patchew.org; Tue, 14 Nov 2017 12:31:48 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53834) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEex8-0005yJ-7S for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEex7-0004JG-4x for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:50 -0500 Received: from mx1.redhat.com ([209.132.183.28]:46648) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEex0-0004GD-Lc; Tue, 14 Nov 2017 12:24:42 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D29D2C04AC64; Tue, 14 Nov 2017 17:24:41 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 33F31183D4; Tue, 14 Nov 2017 17:24:40 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:05 +0100 Message-Id: <20171114172417.7654-9-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 14 Nov 2017 17:24:41 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 08/20] iotests: Make 030 less flaky X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch fixes two race conditions in 030: 1. The first is in TestENOSPC.test_enospc(). After resuming the job, querying it to confirm it is no longer paused may fail because in the meantime it might have completed already. The same was fixed in TestEIO.test_ignore() already (in commit 2c3b44da07d341557a8203cc509ea07fe3605e11). 2. The second is in TestSetSpeed.test_set_speed_invalid(): Here, a stream job is started on a drive without any break points, with a block-job-set-speed invoked subsequently. However, without any break points, the job might have completed in the meantime (on tmpfs at least); or it might complete before cancel_and_wait() which expects the job to still exist. This can be fixed like everywhere else by pausing the drive (installing break points) before starting the job and letting cancel_and_wait() resume it. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi Message-id: 20171109203025.27493-2-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/030 | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index 18838948fa..457984b8e9 100755 --- a/tests/qemu-iotests/030 +++ b/tests/qemu-iotests/030 @@ -666,6 +666,7 @@ class TestENOSPC(TestErrors): if event['event'] =3D=3D 'BLOCK_JOB_ERROR': self.assert_qmp(event, 'data/device', 'drive0') self.assert_qmp(event, 'data/operation', 'read') + error =3D True =20 result =3D self.vm.qmp('query-block-jobs') self.assert_qmp(result, 'return[0]/paused', True) @@ -676,9 +677,11 @@ class TestENOSPC(TestErrors): self.assert_qmp(result, 'return', {}) =20 result =3D self.vm.qmp('query-block-jobs') + if result =3D=3D {'return': []}: + # Race; likely already finished. Check. + continue self.assert_qmp(result, 'return[0]/paused', False) self.assert_qmp(result, 'return[0]/io-status', 'ok') - error =3D True elif event['event'] =3D=3D 'BLOCK_JOB_COMPLETED': self.assertTrue(error, 'job completed unexpectedly') self.assert_qmp(event, 'data/type', 'stream') @@ -792,13 +795,14 @@ class TestSetSpeed(iotests.QMPTestCase): =20 self.assert_no_active_block_jobs() =20 + self.vm.pause_drive('drive0') result =3D self.vm.qmp('block-stream', device=3D'drive0') self.assert_qmp(result, 'return', {}) =20 result =3D self.vm.qmp('block-job-set-speed', device=3D'drive0', s= peed=3D-1) self.assert_qmp(result, 'error/class', 'GenericError') =20 - self.cancel_and_wait() + self.cancel_and_wait(resume=3DTrue) =20 if __name__ =3D=3D '__main__': iotests.main(supported_fmts=3D['qcow2', 'qed']) --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510681026085459.157829076801; Tue, 14 Nov 2017 09:37:06 -0800 (PST) Received: from localhost ([::1]:32831 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf8w-0000uh-74 for importer@patchew.org; Tue, 14 Nov 2017 12:37:02 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53821) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEex7-0005xA-TV for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEex7-0004JN-8q for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:49 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59838) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEex4-0004Hf-8u; Tue, 14 Nov 2017 12:24:46 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6C84D883BF; Tue, 14 Nov 2017 17:24:45 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DA37794B5A; Tue, 14 Nov 2017 17:24:44 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:06 +0100 Message-Id: <20171114172417.7654-10-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 14 Nov 2017 17:24:45 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 09/20] iotests: Add missing 'blkdebug::' in 040 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" 040 tries to invoke pause_drive() on a drive that does not use blkdebug. Good idea, but let's use blkdebug to make it actually work. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi Message-id: 20171109203025.27493-3-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/040 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index c284d08796..90b5b4f2ad 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -289,7 +289,7 @@ class TestSetSpeed(ImageCommitTestCase): qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=3D%s'= % mid_img, test_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 0 512', test_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xef 524288 524288',= mid_img) - self.vm =3D iotests.VM().add_drive(test_img) + self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img) self.vm.launch() =20 def tearDown(self): --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680436683139.91987793851615; Tue, 14 Nov 2017 09:27:16 -0800 (PST) Received: from localhost ([::1]:60989 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEezE-0007qM-Ic for importer@patchew.org; Tue, 14 Nov 2017 12:27:00 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53934) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexG-0006JB-7k for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexC-0004Ns-VU for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:58 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37734) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEex7-0004Is-41; Tue, 14 Nov 2017 12:24:49 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4C5B8C04D94D; Tue, 14 Nov 2017 17:24:48 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B9F208A9D8; Tue, 14 Nov 2017 17:24:47 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:07 +0100 Message-Id: <20171114172417.7654-11-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 14 Nov 2017 17:24:48 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 10/20] iotests: Make 055 less flaky X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" First of all, test 055 does a valiant job of invoking pause_drive() sometimes, but that is worth nothing without blkdebug. So the first thing to do is to sprinkle a couple of "blkdebug::" in there -- with the exception of the transaction tests, because the blkdebug break points make the transaction QMP command hang (which is bad). In that case, we can get away with throttling the block job that it effectively is paused. Then, 055 usually does not pause the drive before starting a block job that should be cancelled. This means that the backup job might be completed already before block-job-cancel is invoked; thus making the test either fail (currently) or moot if cancel_and_wait() ignored this condition. Fix this by pausing the drive before starting the job. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi Message-id: 20171109203025.27493-4-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/055 | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/tests/qemu-iotests/055 b/tests/qemu-iotests/055 index e1206caf9b..8a5d9fd269 100755 --- a/tests/qemu-iotests/055 +++ b/tests/qemu-iotests/055 @@ -48,7 +48,7 @@ class TestSingleDrive(iotests.QMPTestCase): def setUp(self): qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(= image_len)) =20 - self.vm =3D iotests.VM().add_drive(test_img) + self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img) self.vm.add_drive(blockdev_target_img, interface=3D"none") if iotests.qemu_default_machine =3D=3D 'pc': self.vm.add_drive(None, 'media=3Dcdrom', 'ide') @@ -65,10 +65,11 @@ class TestSingleDrive(iotests.QMPTestCase): def do_test_cancel(self, cmd, target): self.assert_no_active_block_jobs() =20 + self.vm.pause_drive('drive0') result =3D self.vm.qmp(cmd, device=3D'drive0', target=3Dtarget, sy= nc=3D'full') self.assert_qmp(result, 'return', {}) =20 - event =3D self.cancel_and_wait() + event =3D self.cancel_and_wait(resume=3DTrue) self.assert_qmp(event, 'data/type', 'backup') =20 def test_cancel_drive_backup(self): @@ -166,7 +167,7 @@ class TestSetSpeed(iotests.QMPTestCase): def setUp(self): qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(= image_len)) =20 - self.vm =3D iotests.VM().add_drive(test_img) + self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img) self.vm.add_drive(blockdev_target_img, interface=3D"none") self.vm.launch() =20 @@ -246,6 +247,8 @@ class TestSetSpeed(iotests.QMPTestCase): def test_set_speed_invalid_blockdev_backup(self): self.do_test_set_speed_invalid('blockdev-backup', 'drive1') =20 +# Note: We cannot use pause_drive() here, or the transaction command +# would stall. Instead, we limit the block job speed here. class TestSingleTransaction(iotests.QMPTestCase): def setUp(self): qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(= image_len)) @@ -271,7 +274,8 @@ class TestSingleTransaction(iotests.QMPTestCase): 'type': cmd, 'data': { 'device': 'drive0', 'target': target, - 'sync': 'full' }, + 'sync': 'full', + 'speed': 64 * 1024 }, } ]) =20 @@ -289,12 +293,12 @@ class TestSingleTransaction(iotests.QMPTestCase): def do_test_pause(self, cmd, target, image): self.assert_no_active_block_jobs() =20 - self.vm.pause_drive('drive0') result =3D self.vm.qmp('transaction', actions=3D[{ 'type': cmd, 'data': { 'device': 'drive0', 'target': target, - 'sync': 'full' }, + 'sync': 'full', + 'speed': 64 * 1024 }, } ]) self.assert_qmp(result, 'return', {}) @@ -302,7 +306,9 @@ class TestSingleTransaction(iotests.QMPTestCase): result =3D self.vm.qmp('block-job-pause', device=3D'drive0') self.assert_qmp(result, 'return', {}) =20 - self.vm.resume_drive('drive0') + result =3D self.vm.qmp('block-job-set-speed', device=3D'drive0', s= peed=3D0) + self.assert_qmp(result, 'return', {}) + self.pause_job('drive0') =20 result =3D self.vm.qmp('query-block-jobs') @@ -461,7 +467,7 @@ class TestDriveCompression(iotests.QMPTestCase): pass =20 def do_prepare_drives(self, fmt, args, attach_target): - self.vm =3D iotests.VM().add_drive(test_img) + self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img) =20 qemu_img('create', '-f', fmt, blockdev_target_img, str(TestDriveCompression.image_len), *args) @@ -500,10 +506,11 @@ class TestDriveCompression(iotests.QMPTestCase): =20 self.assert_no_active_block_jobs() =20 + self.vm.pause_drive('drive0') result =3D self.vm.qmp(cmd, device=3D'drive0', sync=3D'full', comp= ress=3DTrue, **args) self.assert_qmp(result, 'return', {}) =20 - event =3D self.cancel_and_wait() + event =3D self.cancel_and_wait(resume=3DTrue) self.assert_qmp(event, 'data/type', 'backup') =20 self.vm.shutdown() --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680601174566.664379208034; Tue, 14 Nov 2017 09:30:01 -0800 (PST) Received: from localhost ([::1]:32770 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf1q-0002pu-AC for importer@patchew.org; Tue, 14 Nov 2017 12:29:42 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53962) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexH-0006KF-6U for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexG-0004S6-D6 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:24:59 -0500 Received: from mx1.redhat.com ([209.132.183.28]:53658) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexA-0004LG-3z; Tue, 14 Nov 2017 12:24:52 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 41AAC13AB3; Tue, 14 Nov 2017 17:24:51 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C8ACD93539; Tue, 14 Nov 2017 17:24:50 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:08 +0100 Message-Id: <20171114172417.7654-12-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 14 Nov 2017 17:24:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 11/20] iotests: Make 083 less flaky X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" 083 has (at least) two issues: 1. By launching the nbd-fault-injector in background, it may not be scheduled until the first grep on its output file is executed. However, until then, that file may not have been created yet -- so it either does not exist yet (thus making the grep emit an error), or it does exist but contains stale data (thus making the rest of the test case work connect to a wrong address). Fix this by explicitly overwriting the output file before executing nbd-fault-injector. 2. The nbd-fault-injector prints things other than "Listening on...". It also prints a "Closing connection" message from time to time. We currently invoke sed on the whole file in the hope of it only containing the "Listening on..." line yet. That hope is sometimes shattered by the brutal reality of race conditions, so make the sed script more robust. Signed-off-by: Max Reitz Message-id: 20171109203025.27493-5-mreitz@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- tests/qemu-iotests/083 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/083 b/tests/qemu-iotests/083 index 0306f112da..3c1adbf0fb 100755 --- a/tests/qemu-iotests/083 +++ b/tests/qemu-iotests/083 @@ -86,6 +86,7 @@ EOF =20 rm -f "$TEST_DIR/nbd.sock" =20 + echo > "$TEST_DIR/nbd-fault-injector.out" $PYTHON nbd-fault-injector.py $extra_args "$nbd_addr" "$TEST_DIR/nbd-faul= t-injector.conf" >"$TEST_DIR/nbd-fault-injector.out" 2>&1 & =20 # Wait for server to be ready @@ -94,7 +95,8 @@ EOF done =20 # Extract the final address (port number has now been assigned in tcp cas= e) - nbd_addr=3D$(sed 's/Listening on \(.*\)$/\1/' "$TEST_DIR/nbd-fault-inject= or.out") + nbd_addr=3D$(sed -n 's/^Listening on //p' \ + "$TEST_DIR/nbd-fault-injector.out") =20 if [ "$proto" =3D "tcp" ]; then nbd_url=3D"nbd+tcp://$nbd_addr/$export_name" --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680789731554.0568752478589; Tue, 14 Nov 2017 09:33:09 -0800 (PST) Received: from localhost ([::1]:32792 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf4v-0005ko-F2 for importer@patchew.org; Tue, 14 Nov 2017 12:32:53 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54027) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexL-0006Oq-OO for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexK-0004V0-CP for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:03 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43966) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexD-0004NH-71; Tue, 14 Nov 2017 12:24:55 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5A91F5016E; Tue, 14 Nov 2017 17:24:54 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AB35493539; Tue, 14 Nov 2017 17:24:53 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:09 +0100 Message-Id: <20171114172417.7654-13-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 14 Nov 2017 17:24:54 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 12/20] iotests: Make 136 less flaky X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" 136 executes some AIO requests without a final aio_flush; then it advances the virtual clock and thus expects the last access time of the device to be less than the current time when queried (i.e. idle_time_ns to be greater than 0). However, without the aio_flush, some requests may be settled after the clock_step invocation. In that case, idle_time_ns would be 0 and the test fails. Fix this by adding an aio_flush if any AIO request other than some other aio_flush has been executed. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi Message-id: 20171109203025.27493-6-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/136 | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/136 b/tests/qemu-iotests/136 index 4b994897af..88b97ea7c6 100644 --- a/tests/qemu-iotests/136 +++ b/tests/qemu-iotests/136 @@ -238,6 +238,18 @@ sector =3D "%d" for i in range(failed_wr_ops): ops.append("aio_write %d 512" % bad_offset) =20 + # We need an extra aio_flush to settle all outstanding AIO + # operations before we can advance the virtual clock, so that + # the last access happens before clock_step and idle_time_ns + # will be greater than 0 + extra_flush =3D 0 + if rd_ops + wr_ops + invalid_rd_ops + invalid_wr_ops + \ + failed_rd_ops + failed_wr_ops > 0: + extra_flush =3D 1 + + if extra_flush > 0: + ops.append("aio_flush") + if failed_wr_ops > 0: highest_offset =3D max(highest_offset, bad_offset + 512) =20 @@ -251,7 +263,7 @@ sector =3D "%d" self.total_wr_bytes +=3D wr_ops * wr_size self.total_wr_ops +=3D wr_ops self.total_wr_merged +=3D wr_merged - self.total_flush_ops +=3D flush_ops + self.total_flush_ops +=3D flush_ops + extra_flush self.invalid_rd_ops +=3D invalid_rd_ops self.invalid_wr_ops +=3D invalid_wr_ops self.failed_rd_ops +=3D failed_rd_ops --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510680899589617.1224762528482; Tue, 14 Nov 2017 09:34:59 -0800 (PST) Received: from localhost ([::1]:32813 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf6m-0007YE-QK for importer@patchew.org; Tue, 14 Nov 2017 12:34:48 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54116) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexS-0006ir-T8 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexN-0004X4-AK for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:10 -0500 Received: from mx1.redhat.com ([209.132.183.28]:60232) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexG-0004RV-9s; Tue, 14 Nov 2017 12:24:58 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7FE5D368E6; Tue, 14 Nov 2017 17:24:57 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D712887102; Tue, 14 Nov 2017 17:24:56 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:10 +0100 Message-Id: <20171114172417.7654-14-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 14 Nov 2017 17:24:57 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 13/20] iotests: Use new-style NBD connections X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Old-style NBD is deprecated upstream (it is documented, but no longer implemented in the reference implementation), and it is severely limited (it cannot support structured replies, which means it cannot support efficient handling of zeroes), when compared to new-style NBD. We are better off having our iotests favor new-style everywhere (although some explicit tests, particularly 83, still cover old-style for back-compat reasons); this is as simple as supplying the empty string as the default export name, as it does not change the URI needed to connect a client to the server. This also gives us more coverage of the just-added structured reply code, when not overriding $QEMU_NBD to intentionally point to an older server. Signed-off-by: Eric Blake Message-id: 20171109221216.10248-1-eblake@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/common.rc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 0e8a33c696..dbae7d74ba 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -242,7 +242,7 @@ _make_test_img() if [ $IMGPROTO =3D "nbd" ]; then # Pass a sufficiently high number to -e that should be enough for = all # tests - eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT -e 42 $TES= T_IMG_FILE >/dev/null &" + eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT -e 42 -x ''= $TEST_IMG_FILE >/dev/null &" sleep 1 # FIXME: qemu-nbd needs to be listening before we continue fi =20 --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 151068079979561.30952978998664; Tue, 14 Nov 2017 09:33:19 -0800 (PST) Received: from localhost ([::1]:32791 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf4s-0005hh-TC for importer@patchew.org; Tue, 14 Nov 2017 12:32:50 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54127) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexT-0006ji-7N for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexR-0004cD-Sh for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:11 -0500 Received: from mx1.redhat.com ([209.132.183.28]:15568) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexJ-0004U1-C6; Tue, 14 Nov 2017 12:25:01 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 92170C05689C; Tue, 14 Nov 2017 17:25:00 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0BFCA88E2E; Tue, 14 Nov 2017 17:24:59 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:11 +0100 Message-Id: <20171114172417.7654-15-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 14 Nov 2017 17:25:00 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 14/20] qcow2: Check that corrupted images can be repaired in iotest 060 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Alberto Garcia , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia We just fixed a few bugs that caused QEMU to crash when trying to write to corrupted qcow2 images, and iotest 060 was expanded to test all those scenarios. In almost all cases the corrupted images can be repaired using qemu-img, so this patch verifies that. Signed-off-by: Alberto Garcia Message-id: 0b1b95340ecdfbc6927e36adf2fd42ae6198747a.1510143008.git.berto@i= galia.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- tests/qemu-iotests/060 | 10 ++++++++ tests/qemu-iotests/060.out | 64 ++++++++++++++++++++++++++++++++++++++++++= ++++ 2 files changed, 74 insertions(+) diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index 66a8fa4aea..fae08b03bf 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -248,6 +248,8 @@ echo _make_test_img 64M poke_file "$TEST_IMG" "$rt_offset" "\x00\x00\x00\x00\x00\x00\x00\x0= 0" $QEMU_IO -c "write 0 64k" "$TEST_IMG" | _filter_qemu_io +# Repair the image +_check_test_img -r all =20 echo echo "=3D=3D=3D Testing empty refcount table with valid L1 and L2 tables = =3D=3D=3D" @@ -259,6 +261,8 @@ poke_file "$TEST_IMG" "$rt_offset" "\x00\x00\x00= \x00\x00\x00\x00\x00" # allocation with an explicit offset (using qcow2_alloc_clusters_at()) # causing a refcount block to be allocated at offset 0 $QEMU_IO -c "write 0 128k" "$TEST_IMG" | _filter_qemu_io +# Repair the image +_check_test_img -r all =20 echo echo "=3D=3D=3D Testing empty refcount block =3D=3D=3D" @@ -266,6 +270,8 @@ echo _make_test_img 64M poke_file "$TEST_IMG" "$rb_offset" "\x00\x00\x00\x00\x00\x00\x00\x0= 0" $QEMU_IO -c "write 0 64k" "$TEST_IMG" | _filter_qemu_io +# Repair the image +_check_test_img -r all =20 echo echo "=3D=3D=3D Testing empty refcount block with compressed write =3D=3D= =3D" @@ -276,6 +282,8 @@ poke_file "$TEST_IMG" "$rb_offset" "\x00\x00\x00= \x00\x00\x00\x00\x00" # The previous write already allocated an L2 table, so now this new # write will try to allocate a compressed data cluster at offset 0. $QEMU_IO -c "write -c 0k 64k" "$TEST_IMG" | _filter_qemu_io +# Repair the image +_check_test_img -r all =20 echo echo "=3D=3D=3D Testing zero refcount table size =3D=3D=3D" @@ -283,6 +291,8 @@ echo _make_test_img 64M poke_file "$TEST_IMG" "56" "\x00\x00\x00\x00" $QEMU_IO -c "write 0 64k" "$TEST_IMG" 2>&1 | _filter_testdir | _filter_img= fmt +# Repair the image +_check_test_img -r all =20 echo echo "=3D=3D=3D Testing incorrect refcount table offset =3D=3D=3D" diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index cfd78f87a9..62c22701b8 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -187,6 +187,18 @@ read failed: Input/output error Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 qcow2: Marking image as corrupt: Preventing invalid write on metadata (ove= rlaps with refcount table); further corruption events will be suppressed write failed: Input/output error +ERROR cluster 0 refcount=3D0 reference=3D1 +ERROR cluster 1 refcount=3D0 reference=3D1 +ERROR cluster 3 refcount=3D0 reference=3D1 +Rebuilding refcount structure +Repairing cluster 1 refcount=3D1 reference=3D0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 3 corruptions + +Double checking the fixed image now... +No errors were found on the image. =20 =3D=3D=3D Testing empty refcount table with valid L1 and L2 tables =3D=3D= =3D =20 @@ -195,12 +207,40 @@ wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) qcow2: Marking image as corrupt: Preventing invalid allocation of refcount= block at offset 0; further corruption events will be suppressed write failed: Input/output error +ERROR cluster 0 refcount=3D0 reference=3D1 +ERROR cluster 1 refcount=3D0 reference=3D1 +ERROR cluster 3 refcount=3D0 reference=3D1 +ERROR cluster 4 refcount=3D0 reference=3D1 +ERROR cluster 5 refcount=3D0 reference=3D1 +Rebuilding refcount structure +Repairing cluster 1 refcount=3D1 reference=3D0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 5 corruptions + +Double checking the fixed image now... +No errors were found on the image. =20 =3D=3D=3D Testing empty refcount block =3D=3D=3D =20 Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 qcow2: Marking image as corrupt: Preventing invalid allocation of L2 table= at offset 0; further corruption events will be suppressed write failed: Input/output error +ERROR cluster 0 refcount=3D0 reference=3D1 +ERROR cluster 1 refcount=3D0 reference=3D1 +ERROR cluster 2 refcount=3D0 reference=3D1 +ERROR cluster 3 refcount=3D0 reference=3D1 +Rebuilding refcount structure +Repairing cluster 1 refcount=3D1 reference=3D0 +Repairing cluster 2 refcount=3D1 reference=3D0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 4 corruptions + +Double checking the fixed image now... +No errors were found on the image. =20 =3D=3D=3D Testing empty refcount block with compressed write =3D=3D=3D =20 @@ -209,11 +249,35 @@ wrote 65536/65536 bytes at offset 65536 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) qcow2: Marking image as corrupt: Preventing invalid allocation of compress= ed cluster at offset 0; further corruption events will be suppressed write failed: Input/output error +ERROR cluster 0 refcount=3D0 reference=3D1 +ERROR cluster 1 refcount=3D0 reference=3D1 +ERROR cluster 2 refcount=3D0 reference=3D1 +ERROR cluster 3 refcount=3D0 reference=3D1 +Rebuilding refcount structure +Repairing cluster 1 refcount=3D1 reference=3D0 +Repairing cluster 2 refcount=3D1 reference=3D0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 4 corruptions + +Double checking the fixed image now... +No errors were found on the image. =20 =3D=3D=3D Testing zero refcount table size =3D=3D=3D =20 Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 can't open device TEST_DIR/t.IMGFMT: Image does not contain a reference co= unt table +ERROR cluster 0 refcount=3D0 reference=3D1 +ERROR cluster 3 refcount=3D0 reference=3D1 +Rebuilding refcount structure +The following inconsistencies were found and repaired: + + 0 leaked clusters + 2 corruptions + +Double checking the fixed image now... +No errors were found on the image. =20 =3D=3D=3D Testing incorrect refcount table offset =3D=3D=3D =20 --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510681189015583.5920744933868; Tue, 14 Nov 2017 09:39:49 -0800 (PST) Received: from localhost ([::1]:32849 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEfBZ-0004yh-0M for importer@patchew.org; Tue, 14 Nov 2017 12:39:45 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54251) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexc-0006rX-04 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexX-0004jZ-L1 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:19 -0500 Received: from mx1.redhat.com ([209.132.183.28]:53866) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexM-0004W5-LD; Tue, 14 Nov 2017 12:25:04 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C3F2F128B; Tue, 14 Nov 2017 17:25:03 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 37F108B164; Tue, 14 Nov 2017 17:25:02 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:12 +0100 Message-Id: <20171114172417.7654-16-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 14 Nov 2017 17:25:03 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 15/20] block/snapshot: dirty all dirty bitmaps on snapshot-switch X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Snapshot-switch actually changes active state of disk so it should reflect on dirty bitmaps. Otherwise next incremental backup using these bitmaps will be invalid. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20171023092945.54532-1-vsementsov@virtuozzo.com Reviewed-by: Eric Blake Reviewed-by: John Snow Signed-off-by: Max Reitz --- block/snapshot.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/block/snapshot.c b/block/snapshot.c index a46564e7b7..1d5ab5f90f 100644 --- a/block/snapshot.c +++ b/block/snapshot.c @@ -181,10 +181,24 @@ int bdrv_snapshot_goto(BlockDriverState *bs, { BlockDriver *drv =3D bs->drv; int ret, open_ret; + int64_t len; =20 if (!drv) { return -ENOMEDIUM; } + + len =3D bdrv_getlength(bs); + if (len < 0) { + return len; + } + /* We should set all bits in all enabled dirty bitmaps, because dirty + * bitmaps reflect active state of disk and snapshot switch operation + * actually dirties active state. + * TODO: It may make sense not to set all bits but analyze block statu= s of + * current state and destination snapshot and do not set bits correspo= nding + * to both-zero or both-unallocated areas. */ + bdrv_set_dirty(bs, 0, len); + if (drv->bdrv_snapshot_goto) { return drv->bdrv_snapshot_goto(bs, snapshot_id); } --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 151068097071620.86544305492123; Tue, 14 Nov 2017 09:36:10 -0800 (PST) Received: from localhost ([::1]:32821 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf7i-0008NA-Uf for importer@patchew.org; Tue, 14 Nov 2017 12:35:46 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54250) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexb-0006rV-WC for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexX-0004jR-Iz for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:19 -0500 Received: from mx1.redhat.com ([209.132.183.28]:36854) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexQ-0004Yj-6U; Tue, 14 Nov 2017 12:25:08 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4649D80481; Tue, 14 Nov 2017 17:25:07 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1697487100; Tue, 14 Nov 2017 17:25:05 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:13 +0100 Message-Id: <20171114172417.7654-17-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 14 Nov 2017 17:25:07 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 16/20] iotests: 077: Filter out 'resume' lines X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Fam Zheng , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Fam Zheng In the "Overlapping multiple requests" cases, the 3rd reqs (the break point B) doesn't wait for the 2nd, and once resumed the I/O will just continue. This is because the 2nd is already waiting for the 1st, and in wait_serialising_requests() there is: /* If the request is already (indirectly) waiting for us, or * will wait for us as soon as it wakes up, then just go on * (instead of producing a deadlock in the former case). */ if (!req->waiting_for) { /* actually break */ ... } Consequently, the following "sleep 100; resume A" command races with the completion of that request, and sometimes results in an unexpected order of output: > @@ -56,9 +56,9 @@ > wrote XXX/XXX bytes at offset XXX > XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > blkdebug: Resuming request 'B' > +blkdebug: Resuming request 'A' > wrote XXX/XXX bytes at offset XXX > XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > -blkdebug: Resuming request 'A' > wrote XXX/XXX bytes at offset XXX > XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > wrote XXX/XXX bytes at offset XXX Filter out the "Resuming request" lines to make the output deterministic. Reported-by: Patchew Signed-off-by: Fam Zheng Message-id: 20171113150026.4743-1-famz@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- tests/qemu-iotests/077 | 3 ++- tests/qemu-iotests/077.out | 16 ---------------- 2 files changed, 2 insertions(+), 17 deletions(-) diff --git a/tests/qemu-iotests/077 b/tests/qemu-iotests/077 index d2d2a2d687..b3c6fb1370 100755 --- a/tests/qemu-iotests/077 +++ b/tests/qemu-iotests/077 @@ -188,7 +188,8 @@ EOF test_io | $QEMU_IO | _filter_qemu_io | \ sed -e 's,[0-9/]* bytes at offset [0-9]*,XXX/XXX bytes at offset XXX,g= ' \ -e 's/^[0-9]* \(bytes\|KiB\)/XXX bytes/' \ - -e '/Suspended/d' + -e '/Suspended/d' \ + -e '/blkdebug: Resuming request/d' =20 echo echo "=3D=3D Verify image content =3D=3D" diff --git a/tests/qemu-iotests/077.out b/tests/qemu-iotests/077.out index 16f951fd3d..4aae82f2e2 100644 --- a/tests/qemu-iotests/077.out +++ b/tests/qemu-iotests/077.out @@ -4,17 +4,14 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D13421= 7728 =3D=3D Some concurrent requests involving RMW =3D=3D wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX @@ -31,51 +28,38 @@ wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'B' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'B' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'B' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'B' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' -blkdebug: Resuming request 'C' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'B' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -blkdebug: Resuming request 'A' wrote XXX/XXX bytes at offset XXX XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote XXX/XXX bytes at offset XXX --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 151068109424353.59284527726368; Tue, 14 Nov 2017 09:38:14 -0800 (PST) Received: from localhost ([::1]:32835 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf9y-0002Yr-9O for importer@patchew.org; Tue, 14 Nov 2017 12:38:06 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54252) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexc-0006rY-0j for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexX-0004jk-TI for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:19 -0500 Received: from mx1.redhat.com ([209.132.183.28]:42082) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexS-0004c6-En; Tue, 14 Nov 2017 12:25:10 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9970181DE2; Tue, 14 Nov 2017 17:25:09 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0F25D93539; Tue, 14 Nov 2017 17:25:08 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:14 +0100 Message-Id: <20171114172417.7654-18-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 14 Nov 2017 17:25:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 17/20] block/vhdx.c: Don't blindly update the header X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Jeff Cody , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Jeff Cody The VHDX specification requires that before user data modification of the vhdx image, the VHDX header file and data GUIDs need to be updated. In vhdx_open(), if the image is set to RDWR, we go ahead and update the header. However, just because the image is set to RDWR does not mean we can go ahead and write at this point - specifically, if the QEMU run state is INMIGRATE, the underlying file BS may be set to inactive via the BDS open flag of BDRV_O_INACTIVE. Attempting to write under this condition will cause an assert in bdrv_co_pwritev(). We can alternatively latch the first time the image is written. And lo and behold, we do just that, via vhdx_user_visible_write() in vhdx_co_writev(). This means the call to vhdx_update_headers() in vhdx_open() is likely just vestigial, and can be removed. Reported-by: Alexey Kardashevskiy Tested-by: Alexey Kardashevskiy Signed-off-by: Jeff Cody Message-id: 659e4cdba6ef4c651737852777c8c93d27b38040.1510059970.git.jcody@r= edhat.com Reviewed-by: Stefan Hajnoczi Reviewed-by: Denis V. Lunev Signed-off-by: Max Reitz --- block/vhdx.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/block/vhdx.c b/block/vhdx.c index 7ae4589879..9956933da6 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -1008,13 +1008,6 @@ static int vhdx_open(BlockDriverState *bs, QDict *op= tions, int flags, goto fail; } =20 - if (flags & BDRV_O_RDWR) { - ret =3D vhdx_update_headers(bs, s, false, NULL); - if (ret < 0) { - goto fail; - } - } - /* TODO: differencing files */ =20 return 0; --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510681025282903.7235576017313; Tue, 14 Nov 2017 09:37:05 -0800 (PST) Received: from localhost ([::1]:32830 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEf8v-0000tp-BV for importer@patchew.org; Tue, 14 Nov 2017 12:37:01 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54265) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexc-0006s3-H4 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexb-0004m0-7t for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:20 -0500 Received: from mx1.redhat.com ([209.132.183.28]:60474) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexV-0004hk-Q2; Tue, 14 Nov 2017 12:25:13 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EF65C368E6; Tue, 14 Nov 2017 17:25:12 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 525C8D767F; Tue, 14 Nov 2017 17:25:12 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:15 +0100 Message-Id: <20171114172417.7654-19-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 14 Nov 2017 17:25:13 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 18/20] block/parallels: Do not update header or truncate image when INMIGRATE X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Jeff Cody , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Jeff Cody If we write or modify the image file while the QEMU run state is INMIGRATE, then the BDRV_O_INACTIVE BDS flag is set. This will cause an assert, since the image is marked inactive. Make sure we obey this flag. Tested-by: Alexey Kardashevskiy Signed-off-by: Jeff Cody Message-id: 3996c930fa8cde8570b7a63032720d76a28fd78b.1510059970.git.jcody@r= edhat.com Reviewed-by: Stefan Hajnoczi Reviewed-by: Denis V. Lunev Signed-off-by: Max Reitz --- block/parallels.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 2b6c6e5709..7b7a3efa1d 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -708,7 +708,7 @@ static int parallels_open(BlockDriverState *bs, QDict *= options, int flags, s->prealloc_mode =3D PRL_PREALLOC_MODE_FALLOCATE; } =20 - if (flags & BDRV_O_RDWR) { + if ((flags & BDRV_O_RDWR) && !(flags & BDRV_O_INACTIVE)) { s->header->inuse =3D cpu_to_le32(HEADER_INUSE_MAGIC); ret =3D parallels_update_header(bs); if (ret < 0) { @@ -741,12 +741,9 @@ static void parallels_close(BlockDriverState *bs) { BDRVParallelsState *s =3D bs->opaque; =20 - if (bs->open_flags & BDRV_O_RDWR) { + if ((bs->open_flags & BDRV_O_RDWR) && !(bs->open_flags & BDRV_O_INACTI= VE)) { s->header->inuse =3D 0; parallels_update_header(bs); - } - - if (bs->open_flags & BDRV_O_RDWR) { bdrv_truncate(bs->file, s->data_end << BDRV_SECTOR_BITS, PREALLOC_MODE_OFF, NULL); } --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510681235882989.0454052689835; Tue, 14 Nov 2017 09:40:35 -0800 (PST) Received: from localhost ([::1]:32850 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEfC8-0005SY-8L for importer@patchew.org; Tue, 14 Nov 2017 12:40:20 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54410) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexn-00071n-Ny for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexi-0004ry-S6 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:31 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59513) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexY-0004jw-PF; Tue, 14 Nov 2017 12:25:17 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EB5C8C04AC57; Tue, 14 Nov 2017 17:25:15 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6444887104; Tue, 14 Nov 2017 17:25:15 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:16 +0100 Message-Id: <20171114172417.7654-20-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 14 Nov 2017 17:25:16 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 19/20] block/parallels: add migration blocker X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Jeff Cody , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Jeff Cody Migration does not work for parallels, and has been broken for a while (see patch 'block/parallels: Do not update header or truncate image when INMIGRATE'). The bdrv_invalidate_cache() method needs to be added for migration to be supported. Until this is done, prohibit migration. Signed-off-by: Jeff Cody Reviewed-by: Fam Zheng Message-id: 5e04a7c8a3089913fa58d484af42dab7993984ad.1510059970.git.jcody@r= edhat.com Reviewed-by: Stefan Hajnoczi Reviewed-by: Denis V. Lunev Signed-off-by: Max Reitz --- block/parallels.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/block/parallels.c b/block/parallels.c index 7b7a3efa1d..9545761f49 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -35,6 +35,7 @@ #include "qemu/module.h" #include "qemu/bswap.h" #include "qemu/bitmap.h" +#include "migration/blocker.h" =20 /**************************************************************/ =20 @@ -100,6 +101,7 @@ typedef struct BDRVParallelsState { unsigned int tracks; =20 unsigned int off_multiplier; + Error *migration_blocker; } BDRVParallelsState; =20 =20 @@ -720,6 +722,16 @@ static int parallels_open(BlockDriverState *bs, QDict = *options, int flags, s->bat_dirty_bmap =3D bitmap_new(DIV_ROUND_UP(s->header_size, s->bat_dirty_block)); =20 + /* Disable migration until bdrv_invalidate_cache method is added */ + error_setg(&s->migration_blocker, "The Parallels format used by node '= %s' " + "does not support live migration", + bdrv_get_device_or_node_name(bs)); + ret =3D migrate_add_blocker(s->migration_blocker, &local_err); + if (local_err) { + error_propagate(errp, local_err); + error_free(s->migration_blocker); + goto fail; + } qemu_co_mutex_init(&s->lock); return 0; =20 @@ -750,6 +762,9 @@ static void parallels_close(BlockDriverState *bs) =20 g_free(s->bat_dirty_bmap); qemu_vfree(s->header); + + migrate_del_blocker(s->migration_blocker); + error_free(s->migration_blocker); } =20 static QemuOptsList parallels_create_opts =3D { --=20 2.13.6 From nobody Sun May 5 21:35:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1510681304999641.104989748962; Tue, 14 Nov 2017 09:41:44 -0800 (PST) Received: from localhost ([::1]:32862 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEfDK-0006Rn-8d for importer@patchew.org; Tue, 14 Nov 2017 12:41:34 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54334) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEexf-0006uk-AR for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEexe-0004o2-BK for qemu-devel@nongnu.org; Tue, 14 Nov 2017 12:25:23 -0500 Received: from mx1.redhat.com ([209.132.183.28]:54148) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEexb-0004lw-PX; Tue, 14 Nov 2017 12:25:19 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 07D4019D4CA; Tue, 14 Nov 2017 17:25:19 +0000 (UTC) Received: from localhost (unknown [10.40.205.72]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7562B87100; Tue, 14 Nov 2017 17:25:18 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 14 Nov 2017 18:24:17 +0100 Message-Id: <20171114172417.7654-21-mreitz@redhat.com> In-Reply-To: <20171114172417.7654-1-mreitz@redhat.com> References: <20171114172417.7654-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 14 Nov 2017 17:25:19 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 20/20] qemu-iotests: update unsupported image formats in 194 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Jeff Cody , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Jeff Cody Test 194 checks for 'luks' to exclude as an unsupported format, However, most formats are unsupported, due to migration blockers. Rather than specifying a blacklist of unsupported formats, whitelist supported formats (specifically, qcow2, qed, raw, dmg). Tested-by: Alexey Kardashevskiy Signed-off-by: Jeff Cody Message-id: 23ca18c7f843c86a28b1529ca9ac6db4b35ca0e4.1510059970.git.jcody@r= edhat.com Reviewed-by: Stefan Hajnoczi Reviewed-by: Denis V. Lunev Signed-off-by: Max Reitz --- tests/qemu-iotests/194 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194 index 8d973b440f..1d4214aca3 100755 --- a/tests/qemu-iotests/194 +++ b/tests/qemu-iotests/194 @@ -21,7 +21,7 @@ =20 import iotests =20 -iotests.verify_image_format(unsupported_fmts=3D['luks']) +iotests.verify_image_format(supported_fmts=3D['qcow2', 'qed', 'raw', 'dmg'= ]) iotests.verify_platform(['linux']) =20 with iotests.FilePath('source.img') as source_img_path, \ --=20 2.13.6