From nobody Fri Apr 19 11:35:57 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1523882436159863.7434316713878; Mon, 16 Apr 2018 05:40:36 -0700 (PDT) Received: from localhost ([::1]:43492 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f83Qe-00011G-4V for importer@patchew.org; Mon, 16 Apr 2018 08:40:16 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49318) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f83P5-00009h-Tm for qemu-devel@nongnu.org; Mon, 16 Apr 2018 08:38:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f83P4-0008JN-Tj for qemu-devel@nongnu.org; Mon, 16 Apr 2018 08:38:39 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:39010 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f83Oy-0008Fs-LM; Mon, 16 Apr 2018 08:38:32 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AF5FF722FE; Mon, 16 Apr 2018 12:38:31 +0000 (UTC) Received: from localhost (unknown [10.40.205.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 79176215CDC8; Mon, 16 Apr 2018 12:38:30 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Mon, 16 Apr 2018 14:38:21 +0200 Message-Id: <20180416123822.11744-2-mreitz@redhat.com> In-Reply-To: <20180416123822.11744-1-mreitz@redhat.com> References: <20180416123822.11744-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Mon, 16 Apr 2018 12:38:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Mon, 16 Apr 2018 12:38:31 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'mreitz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 1/2] qcow2: try load bitmaps only once X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Checking reopen by existence of some bitmaps is wrong, as it may be some other bitmaps, or on the other hand, user may remove bitmaps. This criteria is bad. To simplify things and make behavior more predictable let's just add a flag to remember, that we've already tried to load bitmaps on open and do not want do it again. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20180411122606.367301-2-vsementsov@virtuozzo.com [mreitz: Changed comment wording according to Eric Blake's suggestion] Signed-off-by: Max Reitz --- block/qcow2.h | 1 + block/qcow2.c | 16 ++++++++-------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index d301f77cea..adf5c3950f 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -298,6 +298,7 @@ typedef struct BDRVQcow2State { uint32_t nb_bitmaps; uint64_t bitmap_directory_size; uint64_t bitmap_directory_offset; + bool dirty_bitmaps_loaded; =20 int flags; int qcow_version; diff --git a/block/qcow2.c b/block/qcow2.c index 486f3e83b7..ef68772aca 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1142,6 +1142,7 @@ static int coroutine_fn qcow2_do_open(BlockDriverStat= e *bs, QDict *options, uint64_t ext_end; uint64_t l1_vm_state_index; bool update_header =3D false; + bool header_updated =3D false; =20 ret =3D bdrv_pread(bs->file, 0, &header, sizeof(header)); if (ret < 0) { @@ -1480,10 +1481,9 @@ static int coroutine_fn qcow2_do_open(BlockDriverSta= te *bs, QDict *options, s->autoclear_features &=3D QCOW2_AUTOCLEAR_MASK; } =20 - if (bdrv_dirty_bitmap_next(bs, NULL)) { - /* It's some kind of reopen with already existing dirty bitmaps. T= here - * are no known cases where we need loading bitmaps in such situat= ion, - * so it's safer don't load them. + if (s->dirty_bitmaps_loaded) { + /* It's some kind of reopen. There are no known cases where we nee= d to + * reload bitmaps in such a situation, so it's safer to skip them. * * Moreover, if we have some readonly bitmaps and we are reopening= for * rw we should reopen bitmaps correspondingly. @@ -1491,13 +1491,13 @@ static int coroutine_fn qcow2_do_open(BlockDriverSt= ate *bs, QDict *options, if (bdrv_has_readonly_bitmaps(bs) && !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTI= VE)) { - bool header_updated =3D false; qcow2_reopen_bitmaps_rw_hint(bs, &header_updated, &local_err); - update_header =3D update_header && !header_updated; } - } else if (qcow2_load_dirty_bitmaps(bs, &local_err)) { - update_header =3D false; + } else { + header_updated =3D qcow2_load_dirty_bitmaps(bs, &local_err); + s->dirty_bitmaps_loaded =3D true; } + update_header =3D update_header && !header_updated; if (local_err !=3D NULL) { error_propagate(errp, local_err); ret =3D -EINVAL; --=20 2.14.3 From nobody Fri Apr 19 11:35:57 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1523882525770957.9335888859306; Mon, 16 Apr 2018 05:42:05 -0700 (PDT) Received: from localhost ([::1]:43587 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f83SC-0002Co-Pt for importer@patchew.org; Mon, 16 Apr 2018 08:41:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49359) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f83P9-0000DN-9J for qemu-devel@nongnu.org; Mon, 16 Apr 2018 08:38:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f83P5-0008Jl-7O for qemu-devel@nongnu.org; Mon, 16 Apr 2018 08:38:43 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:52848 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f83P0-0008Gj-Aw; Mon, 16 Apr 2018 08:38:34 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DA89741662E0; Mon, 16 Apr 2018 12:38:33 +0000 (UTC) Received: from localhost (unknown [10.40.205.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 864562026DFD; Mon, 16 Apr 2018 12:38:33 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Mon, 16 Apr 2018 14:38:22 +0200 Message-Id: <20180416123822.11744-3-mreitz@redhat.com> In-Reply-To: <20180416123822.11744-1-mreitz@redhat.com> References: <20180416123822.11744-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Mon, 16 Apr 2018 12:38:33 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Mon, 16 Apr 2018 12:38:33 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'mreitz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 2/2] iotests: fix 169 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Improve and fix 169: - use MIGRATION events instead of RESUME - make a TODO: enable dirty-bitmaps capability for offline case - recreate vm_b without -incoming near test end This (likely) fixes racy faults at least of the following types: - timeout on waiting for RESUME event - sha256 mismatch on line 136 (142 after this patch) - fail to self.vm_b.launch() on line 135 (141 now after this patch) And surely fixes cat processes, left after test finish. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20180411122606.367301-3-vsementsov@virtuozzo.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- tests/qemu-iotests/169 | 48 +++++++++++++++++++++++++++-------------------= -- 1 file changed, 27 insertions(+), 21 deletions(-) diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 index 153b10b6e7..f243db9955 100755 --- a/tests/qemu-iotests/169 +++ b/tests/qemu-iotests/169 @@ -31,6 +31,8 @@ disk_a =3D os.path.join(iotests.test_dir, 'disk_a') disk_b =3D os.path.join(iotests.test_dir, 'disk_b') size =3D '1M' mig_file =3D os.path.join(iotests.test_dir, 'mig_file') +mig_cmd =3D 'exec: cat > ' + mig_file +incoming_cmd =3D 'exec: cat ' + mig_file =20 =20 class TestDirtyBitmapMigration(iotests.QMPTestCase): @@ -49,7 +51,6 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): self.vm_a.launch() =20 self.vm_b =3D iotests.VM(path_suffix=3D'b') - self.vm_b.add_incoming("exec: cat '" + mig_file + "'") =20 def add_bitmap(self, vm, granularity, persistent): params =3D {'node': 'drive0', @@ -86,36 +87,30 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): (0xa0201, 0x1000)) =20 should_migrate =3D migrate_bitmaps or persistent and shared_storage + mig_caps =3D [{'capability': 'events', 'state': True}] + if migrate_bitmaps: + mig_caps.append({'capability': 'dirty-bitmaps', 'state': True}) =20 + result =3D self.vm_a.qmp('migrate-set-capabilities', + capabilities=3Dmig_caps) + self.assert_qmp(result, 'return', {}) + + self.vm_b.add_incoming(incoming_cmd if online else "defer") self.vm_b.add_drive(disk_a if shared_storage else disk_b) =20 if online: os.mkfifo(mig_file) self.vm_b.launch() + result =3D self.vm_b.qmp('migrate-set-capabilities', + capabilities=3Dmig_caps) + self.assert_qmp(result, 'return', {}) =20 self.add_bitmap(self.vm_a, granularity, persistent) for r in regions: self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % r) sha256 =3D self.get_bitmap_hash(self.vm_a) =20 - if migrate_bitmaps: - capabilities =3D [{'capability': 'dirty-bitmaps', 'state': Tru= e}] - - result =3D self.vm_a.qmp('migrate-set-capabilities', - capabilities=3Dcapabilities) - self.assert_qmp(result, 'return', {}) - - if online: - result =3D self.vm_b.qmp('migrate-set-capabilities', - capabilities=3Dcapabilities) - self.assert_qmp(result, 'return', {}) - - result =3D self.vm_a.qmp('migrate-set-capabilities', - capabilities=3D[{'capability': 'events', - 'state': True}]) - self.assert_qmp(result, 'return', {}) - - result =3D self.vm_a.qmp('migrate', uri=3D'exec:cat>' + mig_file) + result =3D self.vm_a.qmp('migrate', uri=3Dmig_cmd) while True: event =3D self.vm_a.event_wait('MIGRATION') if event['data']['status'] =3D=3D 'completed': @@ -124,14 +119,25 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): if not online: self.vm_a.shutdown() self.vm_b.launch() - # TODO enable bitmap capability for vm_b in this case + result =3D self.vm_b.qmp('migrate-set-capabilities', + capabilities=3Dmig_caps) + self.assert_qmp(result, 'return', {}) + result =3D self.vm_b.qmp('migrate-incoming', uri=3Dincoming_cm= d) + self.assert_qmp(result, 'return', {}) =20 - self.vm_b.event_wait("RESUME", timeout=3D10.0) + while True: + event =3D self.vm_b.event_wait('MIGRATION') + if event['data']['status'] =3D=3D 'completed': + break =20 self.check_bitmap(self.vm_b, sha256 if should_migrate else False) =20 if should_migrate: self.vm_b.shutdown() + # recreate vm_b, as we don't want -incoming option (this will = lead + # to "cat" process left alive after test finish) + self.vm_b =3D iotests.VM(path_suffix=3D'b') + self.vm_b.add_drive(disk_a if shared_storage else disk_b) self.vm_b.launch() self.check_bitmap(self.vm_b, sha256 if persistent else False) =20 --=20 2.14.3