From nobody Sun Feb 8 12:33:24 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1558106763; cv=none; d=zoho.com; s=zohoarc; b=RERETgd06u5hVSIad9SIjbuywcNIWZdxmr0J7rz2rOxbXWhkb5HkuE4Dvg5kMYryRWu5Oug1Eu5NaTfFB8xc42fLZPUnDBaXG8odWa294cAeUK+FzSHKY5TFSJY7eQYsiM6JPIUPhrsNg3pAiKqbJYWyPGkFEM+KzI6rOf8xPIM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1558106763; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=OIwZzes1rWBr3Zw0t/yZ9Bcq+FdtIktv2tM2umKBeVk=; b=V9dHAhfac7CCSkgeA2toRMkz4EEE8np7lnxHSsfiZOkuG1zqK+PEHpNDSxCylq8lpSiVC37j8bjSIWHN8eJT+bvuNz7CA+r2l9u9B7IGdM18PtSxYGWywVKoi/iwdzDZpnT412lIqsKdLR726RpYNLKREwmpaeUWijqsbxj6hUg= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1558106763211840.5325652910174; Fri, 17 May 2019 08:26:03 -0700 (PDT) Received: from localhost ([127.0.0.1]:50156 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRek8-0005hJ-63 for importer@patchew.org; Fri, 17 May 2019 11:25:56 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51902) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRefg-0001i5-B8 for qemu-devel@nongnu.org; Fri, 17 May 2019 11:21:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hReff-0003eX-0H for qemu-devel@nongnu.org; Fri, 17 May 2019 11:21:20 -0400 Received: from relay.sw.ru ([185.231.240.75]:33968) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hRefe-0003PU-Ou; Fri, 17 May 2019 11:21:18 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hRefZ-0005qt-Qj; Fri, 17 May 2019 18:21:13 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 17 May 2019 18:21:10 +0300 Message-Id: <20190517152111.206494-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190517152111.206494-1-vsementsov@virtuozzo.com> References: <20190517152111.206494-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 1/2] qapi: support external bitmaps in block-dirty-bitmap-merge X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add new optional parameter making possible to merge bitmaps from different nodes. It is needed to maintain external snapshots during incremental backup chain history. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- qapi/block-core.json | 22 ++++++++++++++++--- block/dirty-bitmap.c | 9 +++++--- blockdev.c | 50 +++++++++++++++++++++++++++++--------------- 3 files changed, 58 insertions(+), 23 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 7ccbfff9d0..dcc935d655 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2003,19 +2003,35 @@ 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32', '*persistent': 'bool', '*autoload': 'bool', '*disabled': 'bool= ' } } =20 +## +# @BlockDirtyBitmapMergeSource: +# +# @local: name of the bitmap, attached to the same node as target bitmap. +# +# @external: bitmap with specified node +# +# Since: 4.1 +## +{ 'alternate': 'BlockDirtyBitmapMergeSource', + 'data': { 'local': 'str', + 'external': 'BlockDirtyBitmap' } } + ## # @BlockDirtyBitmapMerge: # -# @node: name of device/node which the bitmap is tracking +# @node: name of device/node which the @target bitmap is tracking # # @target: name of the destination dirty bitmap # -# @bitmaps: name(s) of the source dirty bitmap(s) +# @bitmaps: name(s) of the source dirty bitmap(s) at @node and/or fully +# specifed BlockDirtyBitmap elements. The latter are supported +# since 4.1. # # Since: 4.0 ## { 'struct': 'BlockDirtyBitmapMerge', - 'data': { 'node': 'str', 'target': 'str', 'bitmaps': ['str'] } } + 'data': { 'node': 'str', 'target': 'str', + 'bitmaps': ['BlockDirtyBitmapMergeSource'] } } =20 ## # @block-dirty-bitmap-add: diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 59e6ebb861..49646a30e6 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -816,10 +816,10 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, c= onst BdrvDirtyBitmap *src, { bool ret; =20 - /* only bitmaps from one bds are supported */ - assert(dest->mutex =3D=3D src->mutex); - qemu_mutex_lock(dest->mutex); + if (src->mutex !=3D dest->mutex) { + qemu_mutex_lock(src->mutex); + } =20 if (bdrv_dirty_bitmap_check(dest, BDRV_BITMAP_DEFAULT, errp)) { goto out; @@ -845,4 +845,7 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, con= st BdrvDirtyBitmap *src, =20 out: qemu_mutex_unlock(dest->mutex); + if (src->mutex !=3D dest->mutex) { + qemu_mutex_unlock(src->mutex); + } } diff --git a/blockdev.c b/blockdev.c index 79fbac8450..64ccef735b 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2112,11 +2112,10 @@ static void block_dirty_bitmap_disable_abort(BlkAct= ionState *common) } } =20 -static BdrvDirtyBitmap *do_block_dirty_bitmap_merge(const char *node, - const char *target, - strList *bitmaps, - HBitmap **backup, - Error **errp); +static BdrvDirtyBitmap *do_block_dirty_bitmap_merge( + const char *node, const char *target, + BlockDirtyBitmapMergeSourceList *bitmaps, + HBitmap **backup, Error **errp); =20 static void block_dirty_bitmap_merge_prepare(BlkActionState *common, Error **errp) @@ -2965,15 +2964,14 @@ void qmp_block_dirty_bitmap_disable(const char *nod= e, const char *name, bdrv_disable_dirty_bitmap(bitmap); } =20 -static BdrvDirtyBitmap *do_block_dirty_bitmap_merge(const char *node, - const char *target, - strList *bitmaps, - HBitmap **backup, - Error **errp) +static BdrvDirtyBitmap *do_block_dirty_bitmap_merge( + const char *node, const char *target, + BlockDirtyBitmapMergeSourceList *bitmaps, + HBitmap **backup, Error **errp) { BlockDriverState *bs; BdrvDirtyBitmap *dst, *src, *anon; - strList *lst; + BlockDirtyBitmapMergeSourceList *lst; Error *local_err =3D NULL; =20 dst =3D block_dirty_bitmap_lookup(node, target, &bs, errp); @@ -2988,11 +2986,28 @@ static BdrvDirtyBitmap *do_block_dirty_bitmap_merge= (const char *node, } =20 for (lst =3D bitmaps; lst; lst =3D lst->next) { - src =3D bdrv_find_dirty_bitmap(bs, lst->value); - if (!src) { - error_setg(errp, "Dirty bitmap '%s' not found", lst->value); - dst =3D NULL; - goto out; + switch (lst->value->type) { + const char *name, *node; + case QTYPE_QSTRING: + name =3D lst->value->u.local; + src =3D bdrv_find_dirty_bitmap(bs, name); + if (!src) { + error_setg(errp, "Dirty bitmap '%s' not found", name); + dst =3D NULL; + goto out; + } + break; + case QTYPE_QDICT: + node =3D lst->value->u.external.node; + name =3D lst->value->u.external.name; + src =3D block_dirty_bitmap_lookup(node, name, NULL, errp); + if (!src) { + dst =3D NULL; + goto out; + } + break; + default: + abort(); } =20 bdrv_merge_dirty_bitmap(anon, src, NULL, &local_err); @@ -3012,7 +3027,8 @@ static BdrvDirtyBitmap *do_block_dirty_bitmap_merge(c= onst char *node, } =20 void qmp_block_dirty_bitmap_merge(const char *node, const char *target, - strList *bitmaps, Error **errp) + BlockDirtyBitmapMergeSourceList *bitmaps, + Error **errp) { do_block_dirty_bitmap_merge(node, target, bitmaps, NULL, errp); } --=20 2.18.0 From nobody Sun Feb 8 12:33:24 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1558106597; cv=none; d=zoho.com; s=zohoarc; b=YLj41nE64aSfPo8zEDVTO47zUDCBs90o8L/zIUZfuMDiqhlCZij/1dAg2Fx9fNhJLTHXWmdSWVwSzz8H3sjz1mdiY7WmVrwgrep+BXhMhMob0ARXYQV8FLrkkvjl9iD9b7N888Ns1pN5LjFOLpOUR0wSin4Bx+L+ej8wqqgmQl0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1558106597; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=oaYp28UH4gp8iXyLuFoctDfKG/r012Zlu83MtBHFnHU=; b=krzE+y8sbbtDTSP4mVJVA1oIJ1N7Pvat0h+i/LQfAjRre9l/MVacPLB1+1/EkJwwyY+zrpthcMNBPkuouzG4WS/CnlD/7lNq9rBws7MZEUiFUe6OuOyCz3OcHFHcLpuGNePrAe5tGyUdanyy97X8ty9FGZy2D18zt2skqbQG4bY= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1558106597290777.7548520496243; Fri, 17 May 2019 08:23:17 -0700 (PDT) Received: from localhost ([127.0.0.1]:50093 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRehR-0003B7-Vy for importer@patchew.org; Fri, 17 May 2019 11:23:10 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51904) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRefg-0001iD-C4 for qemu-devel@nongnu.org; Fri, 17 May 2019 11:21:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hRefe-0003eQ-Vw for qemu-devel@nongnu.org; Fri, 17 May 2019 11:21:20 -0400 Received: from relay.sw.ru ([185.231.240.75]:33978) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hRefe-0003PR-NT; Fri, 17 May 2019 11:21:18 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hRefZ-0005qt-W7; Fri, 17 May 2019 18:21:14 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 17 May 2019 18:21:11 +0300 Message-Id: <20190517152111.206494-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190517152111.206494-1-vsementsov@virtuozzo.com> References: <20190517152111.206494-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 2/2] iotests: test external snapshot with bitmap copying X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, den@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This test shows that external snapshots and incremental backups are friends. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- tests/qemu-iotests/254 | 52 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/254.out | 52 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 105 insertions(+) create mode 100755 tests/qemu-iotests/254 create mode 100644 tests/qemu-iotests/254.out diff --git a/tests/qemu-iotests/254 b/tests/qemu-iotests/254 new file mode 100755 index 0000000000..33cb80a512 --- /dev/null +++ b/tests/qemu-iotests/254 @@ -0,0 +1,52 @@ +#!/usr/bin/env python +# +# Test external snapshot with bitmap copying. +# +# Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import iotests +from iotests import qemu_img_create, file_path, log + +disk, top =3D file_path('disk', 'top') +size =3D 1024 * 1024 + +qemu_img_create('-f', iotests.imgfmt, disk, str(size)) + +vm =3D iotests.VM().add_drive(disk, opts=3D'node-name=3Dbase') +vm.launch() + +vm.qmp_log('block-dirty-bitmap-add', node=3D'drive0', name=3D'bitmap0') + +vm.hmp_qemu_io('drive0', 'write 0 512K') + +vm.qmp_log('transaction', indent=3D2, actions=3D[ + {'type': 'blockdev-snapshot-sync', + 'data': {'device': 'drive0', 'snapshot-file': top, + 'snapshot-node-name': 'snap'}}, + {'type': 'block-dirty-bitmap-add', + 'data': {'node': 'snap', 'name': 'bitmap0'}}, + {'type': 'block-dirty-bitmap-merge', + 'data': {'node': 'snap', 'target': 'bitmap0', + 'bitmaps': [{'node': 'base', 'name': 'bitmap0'}]}} +], filters=3D[iotests.filter_qmp_testfiles]) + +result =3D vm.qmp('query-block')['return'][0] +log("query-block: device =3D {}, node-name =3D {}, dirty-bitmaps:".format( + result['device'], result['inserted']['node-name'])) +log(result['dirty-bitmaps'], indent=3D2) + +vm.shutdown() diff --git a/tests/qemu-iotests/254.out b/tests/qemu-iotests/254.out new file mode 100644 index 0000000000..d7394cf002 --- /dev/null +++ b/tests/qemu-iotests/254.out @@ -0,0 +1,52 @@ +{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmap0", "no= de": "drive0"}} +{"return": {}} +{ + "execute": "transaction", + "arguments": { + "actions": [ + { + "data": { + "device": "drive0", + "snapshot-file": "TEST_DIR/PID-top", + "snapshot-node-name": "snap" + }, + "type": "blockdev-snapshot-sync" + }, + { + "data": { + "name": "bitmap0", + "node": "snap" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "bitmaps": [ + { + "name": "bitmap0", + "node": "base" + } + ], + "node": "snap", + "target": "bitmap0" + }, + "type": "block-dirty-bitmap-merge" + } + ] + } +} +{ + "return": {} +} +query-block: device =3D drive0, node-name =3D snap, dirty-bitmaps: +[ + { + "busy": false, + "count": 524288, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } +] diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 00e474ab0a..5552d0153c 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -250,3 +250,4 @@ 248 rw auto quick 249 rw auto quick 252 rw auto backing quick +254 rw auto backing quick --=20 2.18.0