From nobody Mon May 6 21:04:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1574775086; cv=none; d=zohomail.com; s=zohoarc; b=ArKmQEvCbyf3vk5stXmbG4cHwZFDUkSqnK+6Da2DMqk6xSlUFstj4ZSurw8QbldbcCJ/DRRLTqzXvE800LJylYwJmVHg4uNLywWq4a+D21bbeaB52Hc2jiMmg/+lmoui0aTmqc+GzyO7XZqsEDLFOqQeXL43HI59IdSLUrXhC8E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1574775086; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=glXsvTzAqSGFnaZ7yF21zgS+6qUqfNQK+PSP38hQfcI=; b=kiq33luUyMGInhtwqtXNiYNxzaQX3u6pON6gR4wjDV5umv2Cc832RMde8SVvqzA5b+eD2B1eLVbZViGyjcZ9H9bsao2up1JBDWAMpX27yEopi9qZMmzDxV9bpNz2H/gVlVPNQZszfFsHBIsQlxx+qO0ep2W+a/7CefJSzGLxUe0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1574775086560862.9224581632307; Tue, 26 Nov 2019 05:31:26 -0800 (PST) Received: from localhost ([::1]:54910 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iZaw9-0005GD-5Y for importer@patchew.org; Tue, 26 Nov 2019 08:31:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39891) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iZauc-00049M-8p for qemu-devel@nongnu.org; Tue, 26 Nov 2019 08:29:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iZaua-0000wJ-Be for qemu-devel@nongnu.org; Tue, 26 Nov 2019 08:29:50 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:20701 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iZaua-0000vG-42 for qemu-devel@nongnu.org; Tue, 26 Nov 2019 08:29:48 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-269-7vltM02XPIubPBtDbuCzYQ-1; Tue, 26 Nov 2019 08:29:43 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4D25E132914; Tue, 26 Nov 2019 13:29:41 +0000 (UTC) Received: from localhost (ovpn-204-240.brq.redhat.com [10.40.204.240]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DAE3E60BE2; Tue, 26 Nov 2019 13:29:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574774987; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=glXsvTzAqSGFnaZ7yF21zgS+6qUqfNQK+PSP38hQfcI=; b=IOdkr+AZ0KyQUMoeA9dmNjn85knJwJKno36JKmCdmt1i4KdSbhYOpvSA4DeeYfm9J6JcSk X6hCzoVR/PpP6ojgw/Z22aY1OLAjfFZ/i7T0wNPmBALS2pKNmSyuAa3TRVyc7yM0TGcNeo HBZd6COfgKB9aPiCVzprSas1xm4U7ek= From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 1/2] block/qcow2-bitmap: fix bitmap migration Date: Tue, 26 Nov 2019 14:29:35 +0100 Message-Id: <20191126132936.1141588-2-mreitz@redhat.com> In-Reply-To: <20191126132936.1141588-1-mreitz@redhat.com> References: <20191126132936.1141588-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: 7vltM02XPIubPBtDbuCzYQ-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Fix bitmap migration with dirty-bitmaps capability enabled and shared storage. We should ignore IN_USE bitmaps in the image on target, when migrating bitmaps through migration channel, see new comment below. Fixes: 74da6b943565c451 Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20191125125229.13531-2-vsementsov@virtuozzo.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- block/qcow2-bitmap.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 809bbc5d20..8abaf632fc 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -988,7 +988,26 @@ bool qcow2_load_dirty_bitmaps(BlockDriverState *bs, Er= ror **errp) } =20 QSIMPLEQ_FOREACH(bm, bm_list, entry) { - BdrvDirtyBitmap *bitmap =3D load_bitmap(bs, bm, errp); + BdrvDirtyBitmap *bitmap; + + if ((bm->flags & BME_FLAG_IN_USE) && + bdrv_find_dirty_bitmap(bs, bm->name)) + { + /* + * We already have corresponding BdrvDirtyBitmap, and bitmap i= n the + * image is marked IN_USE. Firstly, this state is valid, no re= ason + * to consider existing BdrvDirtyBitmap to be bad. Secondly it= 's + * absolutely possible, when we do migration with shared stora= ge + * with dirty-bitmaps capability enabled: if the bitmap was lo= aded + * from this storage before migration start, the storage will + * of-course contain IN_USE outdated version of the bitmap, an= d we + * should not load it on migration target, as we already have = this + * bitmap, being migrated. + */ + continue; + } + + bitmap =3D load_bitmap(bs, bm, errp); if (bitmap =3D=3D NULL) { goto fail; } --=20 2.23.0 From nobody Mon May 6 21:04:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1574775245; cv=none; d=zohomail.com; s=zohoarc; b=gFV3/UejkdPvExNNKkdvdnK40utR6XVc6LO81i/NDABDvp4Ib9x00/9aoZsrQfb7Md59qBCvPC7QlQWb0l3hdR5CYmPNeSYuUamhVlt+CBuYYUNVmEzWk7LssEgZib31fUopwWW/Ho78rSjw41uU9FBRY+vUSOkQLzB0D3ISMjs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1574775245; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5AFYkr7AE9YDxHQ9t5BowRjTy1X9gyqLwc17mwhpohU=; b=eroGBToCAzq2FCmV9EaTzCRn2giJ5IGoWgBzUQo5dFY5sR969pqg3DvzZxNtpz9n4GkV/OGpWHu34LEl0xFmewnovDeUPINueecn+Gdojq3nabDiOp8/eA22scWYqQFRZhCD/BrmV6gK8xQsaTWA6hS9wq3780yIaMc7M7KLpNI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1574775245260641.0335204604958; Tue, 26 Nov 2019 05:34:05 -0800 (PST) Received: from localhost ([::1]:54938 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iZayg-0006rC-Kc for importer@patchew.org; Tue, 26 Nov 2019 08:34:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39890) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iZauc-00049L-8P for qemu-devel@nongnu.org; Tue, 26 Nov 2019 08:29:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iZaua-0000wD-B2 for qemu-devel@nongnu.org; Tue, 26 Nov 2019 08:29:50 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:42884 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iZauZ-0000uz-0N for qemu-devel@nongnu.org; Tue, 26 Nov 2019 08:29:48 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-213-vaNRyB5UM7yY9x9DCj4hDg-1; Tue, 26 Nov 2019 08:29:44 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 98F69805A8D; Tue, 26 Nov 2019 13:29:43 +0000 (UTC) Received: from localhost (ovpn-204-240.brq.redhat.com [10.40.204.240]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 328BE196AE; Tue, 26 Nov 2019 13:29:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574774986; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5AFYkr7AE9YDxHQ9t5BowRjTy1X9gyqLwc17mwhpohU=; b=NsZX0i59DOHkgbItDHd4cIJ5re7qeKogyjGHLlnzbd0Ke6hZjZjXAg0+rbLsxVCo8CHqJR HAveng81/Rg3/W9oXMwThMjyb8NiSCw/GpKagvYlzDxfkhUKKSqhw5WME7rzmUXj1+/E7i ztFh3mJBAhbBgt1Mt0yNSTr5yi44DgE= From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 2/2] iotests: add new test cases to bitmap migration Date: Tue, 26 Nov 2019 14:29:36 +0100 Message-Id: <20191126132936.1141588-3-mreitz@redhat.com> In-Reply-To: <20191126132936.1141588-1-mreitz@redhat.com> References: <20191126132936.1141588-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: vaNRyB5UM7yY9x9DCj4hDg-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Add optional pre-shutdown: shutdown/launch vm before migration. This leads to storing persistent bitmap to the storage, which breaks migration with dirty-bitmaps capability enabled and shared storage until fixed by previous commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20191125125229.13531-3-vsementsov@virtuozzo.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- tests/qemu-iotests/169 | 22 +++++++++++++++------- tests/qemu-iotests/169.out | 4 ++-- 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 index 8c204caf20..9656a7f620 100755 --- a/tests/qemu-iotests/169 +++ b/tests/qemu-iotests/169 @@ -134,7 +134,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): self.check_bitmap(self.vm_a, sha256 if persistent else False) =20 def do_test_migration(self, persistent, migrate_bitmaps, online, - shared_storage): + shared_storage, pre_shutdown): granularity =3D 512 =20 # regions =3D ((start, count), ...) @@ -142,15 +142,13 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): (0xf0000, 0x10000), (0xa0201, 0x1000)) =20 - should_migrate =3D migrate_bitmaps or persistent and shared_storage + should_migrate =3D \ + (migrate_bitmaps and (persistent or not pre_shutdown)) or \ + (persistent and shared_storage) mig_caps =3D [{'capability': 'events', 'state': True}] if migrate_bitmaps: mig_caps.append({'capability': 'dirty-bitmaps', 'state': True}) =20 - result =3D self.vm_a.qmp('migrate-set-capabilities', - capabilities=3Dmig_caps) - self.assert_qmp(result, 'return', {}) - self.vm_b.add_incoming(incoming_cmd if online else "defer") self.vm_b.add_drive(disk_a if shared_storage else disk_b) =20 @@ -166,6 +164,14 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % r) sha256 =3D self.get_bitmap_hash(self.vm_a) =20 + if pre_shutdown: + self.vm_a.shutdown() + self.vm_a.launch() + + result =3D self.vm_a.qmp('migrate-set-capabilities', + capabilities=3Dmig_caps) + self.assert_qmp(result, 'return', {}) + result =3D self.vm_a.qmp('migrate', uri=3Dmig_cmd) while True: event =3D self.vm_a.event_wait('MIGRATION') @@ -210,11 +216,13 @@ def inject_test_case(klass, name, method, *args, **kw= args): mc =3D operator.methodcaller(method, *args, **kwargs) setattr(klass, 'test_' + method + name, lambda self: mc(self)) =20 -for cmb in list(itertools.product((True, False), repeat=3D4)): +for cmb in list(itertools.product((True, False), repeat=3D5)): name =3D ('_' if cmb[0] else '_not_') + 'persistent_' name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap_' name +=3D '_online' if cmb[2] else '_offline' name +=3D '_shared' if cmb[3] else '_nonshared' + if (cmb[4]): + name +=3D '__pre_shutdown' =20 inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', *list(cmb)) diff --git a/tests/qemu-iotests/169.out b/tests/qemu-iotests/169.out index 3a89159833..5c26d15c0d 100644 --- a/tests/qemu-iotests/169.out +++ b/tests/qemu-iotests/169.out @@ -1,5 +1,5 @@ -.................... +.................................... ---------------------------------------------------------------------- -Ran 20 tests +Ran 36 tests =20 OK --=20 2.23.0