From nobody Tue May 14 07:41:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521565748616491.3045724253943; Tue, 20 Mar 2018 10:09:08 -0700 (PDT) Received: from localhost ([::1]:50753 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyKl1-0003v4-Pp for importer@patchew.org; Tue, 20 Mar 2018 13:09:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42632) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyKhW-0001Mf-7d for qemu-devel@nongnu.org; Tue, 20 Mar 2018 13:05:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eyKhS-0001KU-7w for qemu-devel@nongnu.org; Tue, 20 Mar 2018 13:05:30 -0400 Received: from relay.sw.ru ([185.231.240.75]:45494) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eyKhR-0001IY-Ld; Tue, 20 Mar 2018 13:05:25 -0400 Received: from msk-vpn.virtuozzo.com ([195.214.232.6] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1eyKhO-0001rl-L8; Tue, 20 Mar 2018 20:05:22 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 20 Mar 2018 20:05:18 +0300 Message-Id: <20180320170521.32152-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180320170521.32152-1-vsementsov@virtuozzo.com> References: <20180320170521.32152-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 1/3] qcow2-bitmap: add qcow2_reopen_bitmaps_rw_hint() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add version of qcow2_reopen_bitmaps_rw, which do the same work but also return a hint about was header updated or not. This will be used in the following fix for bitmaps reloading after migration. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz --- block/qcow2.h | 2 ++ block/qcow2-bitmap.c | 15 ++++++++++++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index ccb92a9696..d301f77cea 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -671,6 +671,8 @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs,= BdrvCheckResult *res, void **refcount_table, int64_t *refcount_table_size); bool qcow2_load_dirty_bitmaps(BlockDriverState *bs, Error **errp); +int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_update= d, + Error **errp); int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp); void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **er= rp); int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp); diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 3010adb909..6e93ec43e1 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -1004,7 +1004,8 @@ fail: return false; } =20 -int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp) +int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_update= d, + Error **errp) { BDRVQcow2State *s =3D bs->opaque; Qcow2BitmapList *bm_list; @@ -1012,6 +1013,10 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Er= ror **errp) GSList *ro_dirty_bitmaps =3D NULL; int ret =3D 0; =20 + if (header_updated !=3D NULL) { + *header_updated =3D false; + } + if (s->nb_bitmaps =3D=3D 0) { /* No bitmaps - nothing to do */ return 0; @@ -1055,6 +1060,9 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Err= or **errp) error_setg_errno(errp, -ret, "Can't update bitmap directory"); goto out; } + if (header_updated !=3D NULL) { + *header_updated =3D true; + } g_slist_foreach(ro_dirty_bitmaps, set_readonly_helper, false); } =20 @@ -1065,6 +1073,11 @@ out: return ret; } =20 +int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp) +{ + return qcow2_reopen_bitmaps_rw_hint(bs, NULL, errp); +} + /* store_bitmap_data() * Store bitmap to image, filling bitmap table accordingly. */ --=20 2.11.1 From nobody Tue May 14 07:41:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521565849581403.45915214757224; Tue, 20 Mar 2018 10:10:49 -0700 (PDT) Received: from localhost ([::1]:50770 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyKme-0005Qs-PL for importer@patchew.org; Tue, 20 Mar 2018 13:10:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42633) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyKhW-0001Mg-7z for qemu-devel@nongnu.org; Tue, 20 Mar 2018 13:05:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eyKhS-0001KN-7N for qemu-devel@nongnu.org; Tue, 20 Mar 2018 13:05:30 -0400 Received: from relay.sw.ru ([185.231.240.75]:45484) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eyKhR-0001IW-Lp; Tue, 20 Mar 2018 13:05:25 -0400 Received: from msk-vpn.virtuozzo.com ([195.214.232.6] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1eyKhO-0001rl-Sh; Tue, 20 Mar 2018 20:05:22 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 20 Mar 2018 20:05:19 +0300 Message-Id: <20180320170521.32152-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180320170521.32152-1-vsementsov@virtuozzo.com> References: <20180320170521.32152-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 2/3] qcow2: fix bitmaps loading when bitmaps already exist X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" On reopen with existing bitmaps, instead of loading bitmaps, lets reopen them if needed. This also fixes bitmaps migration through shared storage. Consider the case. Persistent bitmaps are stored on bdrv_inactivate. Then, on destination process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps are already loaded on destination start. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c index 7472af6931..d2d129a2e6 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1480,7 +1480,22 @@ static int coroutine_fn qcow2_do_open(BlockDriverSta= te *bs, QDict *options, s->autoclear_features &=3D QCOW2_AUTOCLEAR_MASK; } =20 - if (qcow2_load_dirty_bitmaps(bs, &local_err)) { + if (bdrv_dirty_bitmap_next(bs, NULL)) { + /* It's some kind of reopen with already existing dirty bitmaps. T= here + * are no known cases where we need loading bitmaps in such situat= ion, + * so it's safer don't load them. + * + * Moreover, if we have some readonly bitmaps and we are reopening= for + * rw we should reopen bitmaps correspondingly. + */ + if (bdrv_has_readonly_bitmaps(bs) && + !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTI= VE)) + { + bool header_updated =3D false; + qcow2_reopen_bitmaps_rw_hint(bs, &header_updated, &local_err); + update_header =3D update_header && !header_updated; + } + } else if (qcow2_load_dirty_bitmaps(bs, &local_err)) { update_header =3D false; } if (local_err !=3D NULL) { --=20 2.11.1 From nobody Tue May 14 07:41:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521565660492999.4390046145918; Tue, 20 Mar 2018 10:07:40 -0700 (PDT) Received: from localhost ([::1]:50744 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyKjb-0002aI-LF for importer@patchew.org; Tue, 20 Mar 2018 13:07:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42631) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyKhW-0001Me-7b for qemu-devel@nongnu.org; Tue, 20 Mar 2018 13:05:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eyKhS-0001Kh-8p for qemu-devel@nongnu.org; Tue, 20 Mar 2018 13:05:30 -0400 Received: from relay.sw.ru ([185.231.240.75]:45488) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eyKhR-0001Ia-Lr; Tue, 20 Mar 2018 13:05:25 -0400 Received: from msk-vpn.virtuozzo.com ([195.214.232.6] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1eyKhP-0001rl-3M; Tue, 20 Mar 2018 20:05:23 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 20 Mar 2018 20:05:20 +0300 Message-Id: <20180320170521.32152-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180320170521.32152-1-vsementsov@virtuozzo.com> References: <20180320170521.32152-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 2/3] qcow2: handle reopening bitmaps on bdrv_invalidate_cache X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" On reopen with existing bitmaps, instead of loading bitmaps, lets reopen them if needed. This also fixes bitmaps migration through shared storage. Consider the case. Persistent bitmaps are stored on bdrv_inactivate. Then, on destination process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps are already loaded on destination start. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c index 7472af6931..d2d129a2e6 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1480,7 +1480,22 @@ static int coroutine_fn qcow2_do_open(BlockDriverSta= te *bs, QDict *options, s->autoclear_features &=3D QCOW2_AUTOCLEAR_MASK; } =20 - if (qcow2_load_dirty_bitmaps(bs, &local_err)) { + if (bdrv_dirty_bitmap_next(bs, NULL)) { + /* It's some kind of reopen with already existing dirty bitmaps. T= here + * are no known cases where we need loading bitmaps in such situat= ion, + * so it's safer don't load them. + * + * Moreover, if we have some readonly bitmaps and we are reopening= for + * rw we should reopen bitmaps correspondingly. + */ + if (bdrv_has_readonly_bitmaps(bs) && + !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTI= VE)) + { + bool header_updated =3D false; + qcow2_reopen_bitmaps_rw_hint(bs, &header_updated, &local_err); + update_header =3D update_header && !header_updated; + } + } else if (qcow2_load_dirty_bitmaps(bs, &local_err)) { update_header =3D false; } if (local_err !=3D NULL) { --=20 2.11.1 From nobody Tue May 14 07:41:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521565797959199.27845486625677; Tue, 20 Mar 2018 10:09:57 -0700 (PDT) Received: from localhost ([::1]:50760 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyKll-0004gc-21 for importer@patchew.org; Tue, 20 Mar 2018 13:09:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42636) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyKhW-0001Mi-86 for qemu-devel@nongnu.org; Tue, 20 Mar 2018 13:05:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eyKhS-0001KT-7x for qemu-devel@nongnu.org; Tue, 20 Mar 2018 13:05:30 -0400 Received: from relay.sw.ru ([185.231.240.75]:45486) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eyKhR-0001IX-Lw; Tue, 20 Mar 2018 13:05:25 -0400 Received: from msk-vpn.virtuozzo.com ([195.214.232.6] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1eyKhP-0001rl-At; Tue, 20 Mar 2018 20:05:23 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 20 Mar 2018 20:05:21 +0300 Message-Id: <20180320170521.32152-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180320170521.32152-1-vsementsov@virtuozzo.com> References: <20180320170521.32152-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 3/3] iotests: enable shared migration cases in 169 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Shared migration for dirty bitmaps is fixed by previous patches, so we can enable the test. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/169 | 8 +++----- tests/qemu-iotests/169.out | 4 ++-- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 index 3a8db91f6f..153b10b6e7 100755 --- a/tests/qemu-iotests/169 +++ b/tests/qemu-iotests/169 @@ -140,16 +140,14 @@ def inject_test_case(klass, name, method, *args, **kw= args): mc =3D operator.methodcaller(method, *args, **kwargs) setattr(klass, 'test_' + name, new.instancemethod(mc, None, klass)) =20 -for cmb in list(itertools.product((True, False), repeat=3D3)): +for cmb in list(itertools.product((True, False), repeat=3D4)): name =3D ('_' if cmb[0] else '_not_') + 'persistent_' name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap_' name +=3D '_online' if cmb[2] else '_offline' - - # TODO fix shared-storage bitmap migration and enable cases for it - args =3D list(cmb) + [False] + name +=3D '_shared' if cmb[3] else '_nonshared' =20 inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', - *args) + *list(cmb)) =20 =20 if __name__ =3D=3D '__main__': diff --git a/tests/qemu-iotests/169.out b/tests/qemu-iotests/169.out index 594c16f49f..b6f257674e 100644 --- a/tests/qemu-iotests/169.out +++ b/tests/qemu-iotests/169.out @@ -1,5 +1,5 @@ -........ +................ ---------------------------------------------------------------------- -Ran 8 tests +Ran 16 tests =20 OK --=20 2.11.1