From nobody Wed May 1 08:55:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 151309496802348.610474535651406; Tue, 12 Dec 2017 08:09:28 -0800 (PST) Received: from localhost ([::1]:59165 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eOn7R-0005UI-3J for importer@patchew.org; Tue, 12 Dec 2017 11:09:21 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55257) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eOn3n-0001Ud-Fh for qemu-devel@nongnu.org; Tue, 12 Dec 2017 11:05:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eOn3f-000673-Qt for qemu-devel@nongnu.org; Tue, 12 Dec 2017 11:05:35 -0500 Received: from [195.214.232.25] (port=15694 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eOn3f-0005xt-0F for qemu-devel@nongnu.org; Tue, 12 Dec 2017 11:05:27 -0500 Received: from kvm.sw.ru ([195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id vBCG4owZ030408; Tue, 12 Dec 2017 19:04:52 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 12 Dec 2017 19:04:48 +0300 Message-Id: <20171212160450.17510-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20171212160450.17510-1-vsementsov@virtuozzo.com> References: <20171212160450.17510-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v2 1/3] qcow2-bitmap: add qcow2_reopen_bitmaps_rw_hint() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, mreitz@redhat.com, crosa@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add version of qcow2_reopen_bitmaps_rw, which do the same work but also return a hint about was header updated or not. This will be used in the following fix for bitmaps reloading after migration. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- block/qcow2.h | 2 ++ block/qcow2-bitmap.c | 15 ++++++++++++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index 6f0ff15dd0..40fa5b7cfe 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -667,6 +667,8 @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs,= BdrvCheckResult *res, void **refcount_table, int64_t *refcount_table_size); bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); +int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_update= d, + Error **errp); int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp); diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index f45e46cfbd..6c80dcda69 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -1002,7 +1002,8 @@ fail: return false; } =20 -int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp) +int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_update= d, + Error **errp) { BDRVQcow2State *s =3D bs->opaque; Qcow2BitmapList *bm_list; @@ -1010,6 +1011,10 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Er= ror **errp) GSList *ro_dirty_bitmaps =3D NULL; int ret =3D 0; =20 + if (header_updated !=3D NULL) { + *header_updated =3D false; + } + if (s->nb_bitmaps =3D=3D 0) { /* No bitmaps - nothing to do */ return 0; @@ -1053,6 +1058,9 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Err= or **errp) error_setg_errno(errp, -ret, "Can't update bitmap directory"); goto out; } + if (header_updated !=3D NULL) { + *header_updated =3D true; + } g_slist_foreach(ro_dirty_bitmaps, set_readonly_helper, false); } =20 @@ -1063,6 +1071,11 @@ out: return ret; } =20 +int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp) +{ + return qcow2_reopen_bitmaps_rw_hint(bs, NULL, errp); +} + /* store_bitmap_data() * Store bitmap to image, filling bitmap table accordingly. */ --=20 2.11.1 From nobody Wed May 1 08:55:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1513094872697395.92311599693835; Tue, 12 Dec 2017 08:07:52 -0800 (PST) Received: from localhost ([::1]:59148 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eOn5l-0002cC-Pj for importer@patchew.org; Tue, 12 Dec 2017 11:07:37 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55148) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eOn3h-0001Sk-JE for qemu-devel@nongnu.org; Tue, 12 Dec 2017 11:05:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eOn3e-00064V-Bx for qemu-devel@nongnu.org; Tue, 12 Dec 2017 11:05:29 -0500 Received: from [195.214.232.25] (port=1317 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eOn3d-000625-Vr for qemu-devel@nongnu.org; Tue, 12 Dec 2017 11:05:26 -0500 Received: from kvm.sw.ru ([195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id vBCG4owa030408; Tue, 12 Dec 2017 19:04:52 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 12 Dec 2017 19:04:49 +0300 Message-Id: <20171212160450.17510-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20171212160450.17510-1-vsementsov@virtuozzo.com> References: <20171212160450.17510-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v2 2/3] qcow2: handle reopening bitmaps on bdrv_invalidate_cache X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, mreitz@redhat.com, crosa@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Consider migration with shared storage. Persistent bitmaps are stored on bdrv_inactivate. Then, on destination process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps are already loaded on destination start. In this case we should call qcow2_reopen_bitmaps_rw instead. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- block/qcow2.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c index 1914a940e5..f5f205e1b7 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1450,7 +1450,13 @@ static int qcow2_do_open(BlockDriverState *bs, QDict= *options, int flags, s->autoclear_features &=3D QCOW2_AUTOCLEAR_MASK; } =20 - if (qcow2_load_autoloading_dirty_bitmaps(bs, &local_err)) { + if (bdrv_has_readonly_bitmaps(bs)) { + if (!bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTI= VE)) { + bool header_updated =3D false; + qcow2_reopen_bitmaps_rw_hint(bs, &header_updated, &local_err); + update_header =3D update_header && !header_updated; + } + } else if (qcow2_load_autoloading_dirty_bitmaps(bs, &local_err)) { update_header =3D false; } if (local_err !=3D NULL) { --=20 2.11.1 From nobody Wed May 1 08:55:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1513094863327769.3004579139248; Tue, 12 Dec 2017 08:07:43 -0800 (PST) Received: from localhost ([::1]:59146 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eOn5b-0002U4-8A for importer@patchew.org; Tue, 12 Dec 2017 11:07:27 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55150) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eOn3h-0001Sl-JQ for qemu-devel@nongnu.org; Tue, 12 Dec 2017 11:05:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eOn3f-00066t-Pu for qemu-devel@nongnu.org; Tue, 12 Dec 2017 11:05:29 -0500 Received: from [195.214.232.25] (port=19424 helo=relay.sw.ru) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eOn3f-00063t-6i for qemu-devel@nongnu.org; Tue, 12 Dec 2017 11:05:27 -0500 Received: from kvm.sw.ru ([195.214.232.6]) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id vBCG4owb030408; Tue, 12 Dec 2017 19:04:52 +0300 (MSK) From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 12 Dec 2017 19:04:50 +0300 Message-Id: <20171212160450.17510-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20171212160450.17510-1-vsementsov@virtuozzo.com> References: <20171212160450.17510-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: OpenBSD 3.x [fuzzy] X-Received-From: 195.214.232.25 Subject: [Qemu-devel] [PATCH v2 3/3] iotests: add dirty bitmap migration test X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, mreitz@redhat.com, crosa@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The test creates two vms (vm_a, vm_b), create dirty bitmap in the first one, do several writes to corresponding device and then migrate vm_a to vm_b with dirty bitmaps. For now, only migration through shared storage for persistent bitmaps is available, so it is tested here. Only offline variant is tested for now (a kind of suspend-resume), as it is needed to test that this case is successfully fixed by recent patch. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- tests/qemu-iotests/169 | 82 ++++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/169.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 88 insertions(+) create mode 100755 tests/qemu-iotests/169 create mode 100644 tests/qemu-iotests/169.out diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 new file mode 100755 index 0000000000..282970df8a --- /dev/null +++ b/tests/qemu-iotests/169 @@ -0,0 +1,82 @@ +#!/usr/bin/env python +# +# Tests for dirty bitmaps migration. +# +# Copyright (c) 2016-2017 Virtuozzo International GmbH. All rights reserve= d. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import os +import iotests +import time +from iotests import qemu_img + +disk =3D os.path.join(iotests.test_dir, 'disk') +migfile =3D os.path.join(iotests.test_dir, 'migfile') + +class TestPersistentDirtyBitmapSuspendResume(iotests.QMPTestCase): + + def tearDown(self): + self.vm_a.shutdown() + self.vm_b.shutdown() + os.remove(disk) + os.remove(migfile) + + def setUp(self): + qemu_img('create', '-f', iotests.imgfmt, disk, '1M') + + self.vm_a =3D iotests.VM(path_suffix=3D'a').add_drive(disk) + self.vm_a.launch() + + self.vm_b =3D iotests.VM(path_suffix=3D'b').add_drive(disk) + self.vm_b.add_incoming("exec: cat '" + migfile + "'") + + def test_migration_persistent_shared_offline(self): + """ A kind of suspend-resume """ + granularity =3D 512 + regions =3D [ + { 'start': 0, 'count': 0x10000 }, + { 'start': 0xf0000, 'count': 0x10000 }, + { 'start': 0xa0201, 'count': 0x1000 } + ] + + result =3D self.vm_a.qmp('block-dirty-bitmap-add', node=3D'drive0', + name=3D'bitmap0', granularity=3Dgranularity, + persistent=3DTrue, autoload=3DTrue) + self.assert_qmp(result, 'return', {}); + + for r in regions: + self.vm_a.hmp_qemu_io('drive0', + 'write %d %d' % (r['start'], r['count'])) + + result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap0') + sha256 =3D result['return']['sha256'] + + result =3D self.vm_a.qmp('migrate', uri=3D'exec:cat>' + migfile) + self.assert_qmp(result, 'return', {}); + self.assertNotEqual(self.vm_a.event_wait("STOP"), None) + self.vm_a.shutdown() + + self.vm_b.launch() + self.vm_b.event_wait("RESUME", timeout=3D10.0) + + result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', + node=3D'drive0', name=3D'bitmap0') + self.assert_qmp(result, 'return/sha256', sha256); + + +if __name__ =3D=3D '__main__': + iotests.main(supported_fmts=3D['qcow2']) diff --git a/tests/qemu-iotests/169.out b/tests/qemu-iotests/169.out new file mode 100644 index 0000000000..ae1213e6f8 --- /dev/null +++ b/tests/qemu-iotests/169.out @@ -0,0 +1,5 @@ +. +---------------------------------------------------------------------- +Ran 1 tests + +OK diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 3e688678dd..e879c7ebc7 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -169,6 +169,7 @@ 162 auto quick 163 rw auto quick 165 rw auto quick +169 rw auto quick 170 rw auto quick 171 rw auto quick 172 auto --=20 2.11.1