From nobody Sun Feb 8 23:05:54 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1558011928; cv=none; d=zoho.com; s=zohoarc; b=mhhMK43UJF8oc1sv0kkhXtKn7o9pMq5rdLsi8P87z43igjb+ZKwy1bemO4Z6PHHw5zQUWLPZ0rY8R+JucK+7R5E3xDwB2Pp7tboOe9LIUmXFeAs4wnyoGwKdBkC2JboBhYh28WNB3cDQ+OCqmGNk3NwBOuoNCC6v5JEPPDti9DQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1558011928; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=yAhFLCc3UnX5WxsossQ/PllzWxp4bW2rhn/9qjXzsC0=; b=E7vZ5UmGMC5oPPv3Uvk54+T+4w3gDXoBRvVpk9edAUx5KrIo+yUhpkiyoDlfgo0ARPyAAZY13ajqFHxAw4/WqrOmm/NWGjeDd5cyqss8di2gz2yX5NCEkcrkaIta4t130n6/0Gyrk6EcyIQ2jYCoHzufhpVV1yWIPebIa2B6rQk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1558011928720463.9352602731053; Thu, 16 May 2019 06:05:28 -0700 (PDT) Received: from localhost ([127.0.0.1]:55229 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRG4Y-0005Ko-DO for importer@patchew.org; Thu, 16 May 2019 09:05:22 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50561) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRFTy-0007uh-JC for qemu-devel@nongnu.org; Thu, 16 May 2019 08:27:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hRFTv-00087H-HK for qemu-devel@nongnu.org; Thu, 16 May 2019 08:27:34 -0400 Received: from relay.sw.ru ([185.231.240.75]:33894) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hRFTv-00085E-A2; Thu, 16 May 2019 08:27:31 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hRFTp-0006Eb-Vd; Thu, 16 May 2019 15:27:26 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 16 May 2019 15:27:24 +0300 Message-Id: <20190516122725.132334-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190516122725.132334-1-vsementsov@virtuozzo.com> References: <20190516122725.132334-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 1/2] qapi: support external bitmaps in block-dirty-bitmap-merge X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add new optional parameter making possible to merge bitmaps from different nodes. It is needed to maintain external snapshots during incremental backup chain history. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- qapi/block-core.json | 13 ++++++++++--- block/dirty-bitmap.c | 9 ++++++--- blockdev.c | 46 ++++++++++++++++++++++++++++++-------------- 3 files changed, 48 insertions(+), 20 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 7ccbfff9d0..933b50771a 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2006,16 +2006,23 @@ ## # @BlockDirtyBitmapMerge: # -# @node: name of device/node which the bitmap is tracking +# @node: name of device/node which the @target and @bitmaps bitmaps are +# tracking # # @target: name of the destination dirty bitmap # -# @bitmaps: name(s) of the source dirty bitmap(s) +# @bitmaps: name(s) of the source dirty bitmap(s). The field is optional +# since 4.1. +# +# @external-bitmaps: additional list of source dirty bitmaps with specified +# nodes, which allows merging bitmaps between different +# nodes. (Since: 4.1) # # Since: 4.0 ## { 'struct': 'BlockDirtyBitmapMerge', - 'data': { 'node': 'str', 'target': 'str', 'bitmaps': ['str'] } } + 'data': { 'node': 'str', 'target': 'str', '*bitmaps': ['str'], + '*external-bitmaps': ['BlockDirtyBitmap'] } } =20 ## # @block-dirty-bitmap-add: diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 59e6ebb861..49646a30e6 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -816,10 +816,10 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, c= onst BdrvDirtyBitmap *src, { bool ret; =20 - /* only bitmaps from one bds are supported */ - assert(dest->mutex =3D=3D src->mutex); - qemu_mutex_lock(dest->mutex); + if (src->mutex !=3D dest->mutex) { + qemu_mutex_lock(src->mutex); + } =20 if (bdrv_dirty_bitmap_check(dest, BDRV_BITMAP_DEFAULT, errp)) { goto out; @@ -845,4 +845,7 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, con= st BdrvDirtyBitmap *src, =20 out: qemu_mutex_unlock(dest->mutex); + if (src->mutex !=3D dest->mutex) { + qemu_mutex_unlock(src->mutex); + } } diff --git a/blockdev.c b/blockdev.c index 79fbac8450..8d37ce5943 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2112,11 +2112,9 @@ static void block_dirty_bitmap_disable_abort(BlkActi= onState *common) } } =20 -static BdrvDirtyBitmap *do_block_dirty_bitmap_merge(const char *node, - const char *target, - strList *bitmaps, - HBitmap **backup, - Error **errp); +static BdrvDirtyBitmap *do_block_dirty_bitmap_merge( + const char *node, const char *target, strList *bitmaps, + BlockDirtyBitmapList *external_bitmaps, HBitmap **backup, Error **= errp); =20 static void block_dirty_bitmap_merge_prepare(BlkActionState *common, Error **errp) @@ -2132,8 +2130,9 @@ static void block_dirty_bitmap_merge_prepare(BlkActio= nState *common, action =3D common->action->u.block_dirty_bitmap_merge.data; =20 state->bitmap =3D do_block_dirty_bitmap_merge(action->node, action->ta= rget, - action->bitmaps, &state->b= ackup, - errp); + action->bitmaps, + action->external_bitmaps, + &state->backup, errp); } =20 static void abort_prepare(BlkActionState *common, Error **errp) @@ -2965,15 +2964,14 @@ void qmp_block_dirty_bitmap_disable(const char *nod= e, const char *name, bdrv_disable_dirty_bitmap(bitmap); } =20 -static BdrvDirtyBitmap *do_block_dirty_bitmap_merge(const char *node, - const char *target, - strList *bitmaps, - HBitmap **backup, - Error **errp) +static BdrvDirtyBitmap *do_block_dirty_bitmap_merge( + const char *node, const char *target, strList *bitmaps, + BlockDirtyBitmapList *external_bitmaps, HBitmap **backup, Error **= errp) { BlockDriverState *bs; BdrvDirtyBitmap *dst, *src, *anon; strList *lst; + BlockDirtyBitmapList *ext_lst; Error *local_err =3D NULL; =20 dst =3D block_dirty_bitmap_lookup(node, target, &bs, errp); @@ -3003,6 +3001,22 @@ static BdrvDirtyBitmap *do_block_dirty_bitmap_merge(= const char *node, } } =20 + for (ext_lst =3D external_bitmaps; ext_lst; ext_lst =3D ext_lst->next)= { + src =3D block_dirty_bitmap_lookup(ext_lst->value->node, + ext_lst->value->name, NULL, errp); + if (!src) { + dst =3D NULL; + goto out; + } + + bdrv_merge_dirty_bitmap(anon, src, NULL, &local_err); + if (local_err) { + error_propagate(errp, local_err); + dst =3D NULL; + goto out; + } + } + /* Merge into dst; dst is unchanged on failure. */ bdrv_merge_dirty_bitmap(dst, anon, backup, errp); =20 @@ -3012,9 +3026,13 @@ static BdrvDirtyBitmap *do_block_dirty_bitmap_merge(= const char *node, } =20 void qmp_block_dirty_bitmap_merge(const char *node, const char *target, - strList *bitmaps, Error **errp) + bool has_bitmaps, strList *bitmaps, + bool has_external_bitmaps, + BlockDirtyBitmapList *external_bitmaps, + Error **errp) { - do_block_dirty_bitmap_merge(node, target, bitmaps, NULL, errp); + do_block_dirty_bitmap_merge(node, target, bitmaps, external_bitmaps, N= ULL, + errp); } =20 BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *= node, --=20 2.18.0 From nobody Sun Feb 8 23:05:54 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1558010941; cv=none; d=zoho.com; s=zohoarc; b=mZcPEPzP2auINplJOOdf73jrWRclzdUIpVG9iLaL4VVYje+K0L5b38R0IrQyxHYk2FQ+fLKzLit1+M9as+0cSZ419aobUPc9DAFctGIjW6VhbIZUIvzKWLR6FzsJrQZZC6nvk9DIlHeGLErDn0UPjywICcdT2sehhr/PcJmuuGk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1558010941; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=4xocE78qPWleCP2foHRGPIf5p44VJ8uif1R+sB5+Kqk=; b=SC/BEY4ojKlL6psjOwd9L99P1kyTsvqKUxKiCeKaKZsIoN5OWh+y3dp/eAElfHUNJgaa3f3dqEr7fRrgsKqkwHHSOdwOSzWU7f9do+vJOr/JhHEN/hb0si5fk1Ng3U9L8g1Pt8bz1qyymG+j7vhnDfhEE4KmYVUvigMW4o+9RQw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1558010941656633.3721586273252; Thu, 16 May 2019 05:49:01 -0700 (PDT) Received: from localhost ([127.0.0.1]:54449 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRFod-0008D2-Ih for importer@patchew.org; Thu, 16 May 2019 08:48:55 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50557) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRFTy-0007uY-Ib for qemu-devel@nongnu.org; Thu, 16 May 2019 08:27:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hRFTv-000877-H1 for qemu-devel@nongnu.org; Thu, 16 May 2019 08:27:34 -0400 Received: from relay.sw.ru ([185.231.240.75]:33896) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hRFTv-00085F-9v; Thu, 16 May 2019 08:27:31 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hRFTq-0006Eb-1S; Thu, 16 May 2019 15:27:26 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 16 May 2019 15:27:25 +0300 Message-Id: <20190516122725.132334-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190516122725.132334-1-vsementsov@virtuozzo.com> References: <20190516122725.132334-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 2/2] iotests: test external snapshot with bitmap copying X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This test shows that external snapshots and incremental backups are friends. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/254 | 52 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/254.out | 53 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 106 insertions(+) create mode 100755 tests/qemu-iotests/254 create mode 100644 tests/qemu-iotests/254.out diff --git a/tests/qemu-iotests/254 b/tests/qemu-iotests/254 new file mode 100755 index 0000000000..15688f2c29 --- /dev/null +++ b/tests/qemu-iotests/254 @@ -0,0 +1,52 @@ +#!/usr/bin/env python +# +# Test external snapshot with bitmap copying. +# +# Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import iotests +from iotests import qemu_img_create, file_path, log + +disk, top =3D file_path('disk', 'top') +size =3D 1024 * 1024 + +qemu_img_create('-f', iotests.imgfmt, disk, str(size)) + +vm =3D iotests.VM().add_drive(disk, opts=3D'node-name=3Dbase') +vm.launch() + +vm.qmp_log('block-dirty-bitmap-add', node=3D'drive0', name=3D'bitmap0') + +vm.hmp_qemu_io('drive0', 'write 0 512K') + +vm.qmp_log('transaction', indent=3D2, actions=3D[ + {'type': 'blockdev-snapshot-sync', + 'data': {'device': 'drive0', 'snapshot-file': top, + 'snapshot-node-name': 'snap'}}, + {'type': 'block-dirty-bitmap-add', + 'data': {'node': 'snap', 'name': 'bitmap0'}}, + {'type': 'block-dirty-bitmap-merge', + 'data': {'node': 'snap', 'target': 'bitmap0', 'bitmaps': [], + 'external-bitmaps': [{'node': 'base', 'name': 'bitmap0'}]}} +], filters=3D[iotests.filter_qmp_testfiles]) + +result =3D vm.qmp('query-block')['return'][0] +log("query-block: device =3D {}, node-name =3D {}, dirty-bitmaps:".format( + result['device'], result['inserted']['node-name'])) +log(result['dirty-bitmaps'], indent=3D2) + +vm.shutdown() diff --git a/tests/qemu-iotests/254.out b/tests/qemu-iotests/254.out new file mode 100644 index 0000000000..14a9cd3d71 --- /dev/null +++ b/tests/qemu-iotests/254.out @@ -0,0 +1,53 @@ +{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmap0", "no= de": "drive0"}} +{"return": {}} +{ + "execute": "transaction", + "arguments": { + "actions": [ + { + "data": { + "device": "drive0", + "snapshot-file": "TEST_DIR/PID-top", + "snapshot-node-name": "snap" + }, + "type": "blockdev-snapshot-sync" + }, + { + "data": { + "name": "bitmap0", + "node": "snap" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "bitmaps": [], + "external-bitmaps": [ + { + "name": "bitmap0", + "node": "base" + } + ], + "node": "snap", + "target": "bitmap0" + }, + "type": "block-dirty-bitmap-merge" + } + ] + } +} +{ + "return": {} +} +query-block: device =3D drive0, node-name =3D snap, dirty-bitmaps: +[ + { + "busy": false, + "count": 524288, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } +] diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 00e474ab0a..5552d0153c 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -250,3 +250,4 @@ 248 rw auto quick 249 rw auto quick 252 rw auto backing quick +254 rw auto backing quick --=20 2.18.0