From nobody Thu May 2 01:27:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1544829495820425.300573171113; Fri, 14 Dec 2018 15:18:15 -0800 (PST) Received: from localhost ([::1]:35934 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwia-00025H-Oo for importer@patchew.org; Fri, 14 Dec 2018 18:18:04 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39761) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwgN-0000p8-Ji for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXwgL-0002cz-Hi for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:47 -0500 Received: from mx1.redhat.com ([209.132.183.28]:48966) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXwg0-0001N5-Dg; Fri, 14 Dec 2018 18:15:25 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BD15480468; Fri, 14 Dec 2018 23:15:20 +0000 (UTC) Received: from probe.redhat.com (ovpn-122-54.rdu2.redhat.com [10.10.122.54]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6F08219C65; Fri, 14 Dec 2018 23:15:19 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 18:15:06 -0500 Message-Id: <20181214231512.5295-2-jsnow@redhat.com> In-Reply-To: <20181214231512.5295-1-jsnow@redhat.com> References: <20181214231512.5295-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 14 Dec 2018 23:15:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 1/7] blockdev: abort transactions in reverse order X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , vsementsov@virtuozzo.com, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Presently, we abort transactions in the same order they were processed in. Bitmap commands, though, attempt to restore backup data structures on abort. To that end, though, they need to be aborted in reverse chronological order. Replace the QSIMPLEQ data structure with a QTAILQ one, so we can iterate in reverse for the abort phase of the transaction. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- blockdev.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/blockdev.c b/blockdev.c index 81f95d920b..1ba706df8b 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1341,7 +1341,7 @@ struct BlkActionState { const BlkActionOps *ops; JobTxn *block_job_txn; TransactionProperties *txn_props; - QSIMPLEQ_ENTRY(BlkActionState) entry; + QTAILQ_ENTRY(BlkActionState) entry; }; =20 /* internal snapshot private data */ @@ -2269,8 +2269,8 @@ void qmp_transaction(TransactionActionList *dev_list, BlkActionState *state, *next; Error *local_err =3D NULL; =20 - QSIMPLEQ_HEAD(snap_bdrv_states, BlkActionState) snap_bdrv_states; - QSIMPLEQ_INIT(&snap_bdrv_states); + QTAILQ_HEAD(snap_bdrv_states, BlkActionState) snap_bdrv_states; + QTAILQ_INIT(&snap_bdrv_states); =20 /* Does this transaction get canceled as a group on failure? * If not, we don't really need to make a JobTxn. @@ -2301,7 +2301,7 @@ void qmp_transaction(TransactionActionList *dev_list, state->action =3D dev_info; state->block_job_txn =3D block_job_txn; state->txn_props =3D props; - QSIMPLEQ_INSERT_TAIL(&snap_bdrv_states, state, entry); + QTAILQ_INSERT_TAIL(&snap_bdrv_states, state, entry); =20 state->ops->prepare(state, &local_err); if (local_err) { @@ -2310,7 +2310,7 @@ void qmp_transaction(TransactionActionList *dev_list, } } =20 - QSIMPLEQ_FOREACH(state, &snap_bdrv_states, entry) { + QTAILQ_FOREACH(state, &snap_bdrv_states, entry) { if (state->ops->commit) { state->ops->commit(state); } @@ -2321,13 +2321,13 @@ void qmp_transaction(TransactionActionList *dev_lis= t, =20 delete_and_fail: /* failure, and it is all-or-none; roll back all operations */ - QSIMPLEQ_FOREACH(state, &snap_bdrv_states, entry) { + QTAILQ_FOREACH_REVERSE(state, &snap_bdrv_states, snap_bdrv_states, ent= ry) { if (state->ops->abort) { state->ops->abort(state); } } exit: - QSIMPLEQ_FOREACH_SAFE(state, &snap_bdrv_states, entry, next) { + QTAILQ_FOREACH_SAFE(state, &snap_bdrv_states, entry, next) { if (state->ops->clean) { state->ops->clean(state); } --=20 2.17.2 From nobody Thu May 2 01:27:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1544829496844833.4338222256364; Fri, 14 Dec 2018 15:18:16 -0800 (PST) Received: from localhost ([::1]:35935 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwig-0002B4-Gw for importer@patchew.org; Fri, 14 Dec 2018 18:18:10 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39796) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwgO-0000qW-Rf for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:53 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXwgN-0002oX-LB for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:48 -0500 Received: from mx1.redhat.com ([209.132.183.28]:48994) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXwg2-0001RZ-LX; Fri, 14 Dec 2018 18:15:29 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2F805420B3; Fri, 14 Dec 2018 23:15:22 +0000 (UTC) Received: from probe.redhat.com (ovpn-122-54.rdu2.redhat.com [10.10.122.54]) by smtp.corp.redhat.com (Postfix) with ESMTP id F3E2719C65; Fri, 14 Dec 2018 23:15:20 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 18:15:07 -0500 Message-Id: <20181214231512.5295-3-jsnow@redhat.com> In-Reply-To: <20181214231512.5295-1-jsnow@redhat.com> References: <20181214231512.5295-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 14 Dec 2018 23:15:22 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 2/7] blockdev: n-ary bitmap merge X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , vsementsov@virtuozzo.com, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Especially outside of transactions, it is helpful to provide all-or-nothing semantics for bitmap merges. This facilitates the coalescing of multiple bitmaps into a single target for the "checkpoint" interpretation when assembling bitmaps that represent arbitrary points in time from component bitmaps. This is an incompatible change from the preliminary version of the API. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- blockdev.c | 64 +++++++++++++++++++++++++++++--------------- qapi/block-core.json | 22 +++++++-------- 2 files changed, 53 insertions(+), 33 deletions(-) diff --git a/blockdev.c b/blockdev.c index 1ba706df8b..0f740fd964 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2122,33 +2122,26 @@ static void block_dirty_bitmap_disable_abort(BlkAct= ionState *common) } } =20 +void do_block_dirty_bitmap_merge(const char *node, const char *target, + strList *bitmaps, HBitmap **backup, + Error **errp); + static void block_dirty_bitmap_merge_prepare(BlkActionState *common, Error **errp) { BlockDirtyBitmapMerge *action; BlockDirtyBitmapState *state =3D DO_UPCAST(BlockDirtyBitmapState, common, common); - BdrvDirtyBitmap *merge_source; =20 if (action_check_completion_mode(common, errp) < 0) { return; } =20 action =3D common->action->u.x_block_dirty_bitmap_merge.data; - state->bitmap =3D block_dirty_bitmap_lookup(action->node, - action->dst_name, - &state->bs, - errp); - if (!state->bitmap) { - return; - } =20 - merge_source =3D bdrv_find_dirty_bitmap(state->bs, action->src_name); - if (!merge_source) { - return; - } - - bdrv_merge_dirty_bitmap(state->bitmap, merge_source, &state->backup, e= rrp); + do_block_dirty_bitmap_merge(action->node, action->target, + action->bitmaps, &state->backup, + errp); } =20 static void abort_prepare(BlkActionState *common, Error **errp) @@ -2980,24 +2973,51 @@ void qmp_x_block_dirty_bitmap_disable(const char *n= ode, const char *name, bdrv_disable_dirty_bitmap(bitmap); } =20 -void qmp_x_block_dirty_bitmap_merge(const char *node, const char *dst_name, - const char *src_name, Error **errp) +void do_block_dirty_bitmap_merge(const char *node, const char *target, + strList *bitmaps, HBitmap **backup, + Error **errp) { BlockDriverState *bs; - BdrvDirtyBitmap *dst, *src; + BdrvDirtyBitmap *dst, *src, *anon; + strList *lst; + Error *local_err =3D NULL; =20 - dst =3D block_dirty_bitmap_lookup(node, dst_name, &bs, errp); + dst =3D block_dirty_bitmap_lookup(node, target, &bs, errp); if (!dst) { return; } =20 - src =3D bdrv_find_dirty_bitmap(bs, src_name); - if (!src) { - error_setg(errp, "Dirty bitmap '%s' not found", src_name); + anon =3D bdrv_create_dirty_bitmap(bs, bdrv_dirty_bitmap_granularity(ds= t), + NULL, errp); + if (!anon) { return; } =20 - bdrv_merge_dirty_bitmap(dst, src, NULL, errp); + for (lst =3D bitmaps; lst; lst =3D lst->next) { + src =3D bdrv_find_dirty_bitmap(bs, lst->value); + if (!src) { + error_setg(errp, "Dirty bitmap '%s' not found", lst->value); + goto out; + } + + bdrv_merge_dirty_bitmap(anon, src, NULL, &local_err); + if (local_err) { + error_propagate(errp, local_err); + goto out; + } + } + + /* Merge into dst; dst is unchanged on failure. */ + bdrv_merge_dirty_bitmap(dst, anon, backup, errp); + + out: + bdrv_release_dirty_bitmap(bs, anon); +} + +void qmp_x_block_dirty_bitmap_merge(const char *node, const char *target, + strList *bitmaps, Error **errp) +{ + do_block_dirty_bitmap_merge(node, target, bitmaps, NULL, errp); } =20 BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *= node, diff --git a/qapi/block-core.json b/qapi/block-core.json index d4fe710836..320d74ef34 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1818,14 +1818,14 @@ # # @node: name of device/node which the bitmap is tracking # -# @dst_name: name of the destination dirty bitmap +# @target: name of the destination dirty bitmap # -# @src_name: name of the source dirty bitmap +# @bitmaps: name(s) of the source dirty bitmap(s) # # Since: 3.0 ## { 'struct': 'BlockDirtyBitmapMerge', - 'data': { 'node': 'str', 'dst_name': 'str', 'src_name': 'str' } } + 'data': { 'node': 'str', 'target': 'str', 'bitmaps': ['str'] } } =20 ## # @block-dirty-bitmap-add: @@ -1940,23 +1940,23 @@ ## # @x-block-dirty-bitmap-merge: # -# FIXME: Rename @src_name and @dst_name to src-name and dst-name. -# -# Merge @src_name dirty bitmap to @dst_name dirty bitmap. @src_name dirty -# bitmap is unchanged. On error, @dst_name is unchanged. +# Merge dirty bitmaps listed in @bitmaps to the @target dirty bitmap. +# The @bitmaps dirty bitmaps are unchanged. +# On error, @target is unchanged. # # Returns: nothing on success # If @node is not a valid block device, DeviceNotFound -# If @dst_name or @src_name is not found, GenericError -# If bitmaps has different sizes or granularities, GenericError +# If any bitmap in @bitmaps or @target is not found, GenericError +# If any of the bitmaps have different sizes or granularities, +# GenericError # # Since: 3.0 # # Example: # # -> { "execute": "x-block-dirty-bitmap-merge", -# "arguments": { "node": "drive0", "dst_name": "bitmap0", -# "src_name": "bitmap1" } } +# "arguments": { "node": "drive0", "target": "bitmap0", +# "bitmaps": ["bitmap1"] } } # <- { "return": {} } # ## --=20 2.17.2 From nobody Thu May 2 01:27:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1544829531640729.3064346633654; Fri, 14 Dec 2018 15:18:51 -0800 (PST) Received: from localhost ([::1]:35936 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwjK-00035L-8f for importer@patchew.org; Fri, 14 Dec 2018 18:18:50 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39953) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwgW-0000yF-23 for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXwgU-0003Hj-9n for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:56 -0500 Received: from mx1.redhat.com ([209.132.183.28]:33086) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXwgN-0001X3-PM; Fri, 14 Dec 2018 18:15:47 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 33ED18E3DF; Fri, 14 Dec 2018 23:15:24 +0000 (UTC) Received: from probe.redhat.com (ovpn-122-54.rdu2.redhat.com [10.10.122.54]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5DD7219C65; Fri, 14 Dec 2018 23:15:22 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 18:15:08 -0500 Message-Id: <20181214231512.5295-4-jsnow@redhat.com> In-Reply-To: <20181214231512.5295-1-jsnow@redhat.com> References: <20181214231512.5295-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Fri, 14 Dec 2018 23:15:24 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 3/7] block: remove 'x' prefix from experimental bitmap APIs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , vsementsov@virtuozzo.com, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The 'x' prefix was added because I was uncertain of the direction we'd take for the libvirt API. With the general approach solidified, I feel comfortable committing to this API for 4.0. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- blockdev.c | 22 +++++++++++----------- qapi/block-core.json | 34 +++++++++++++++++----------------- qapi/transaction.json | 12 ++++++------ tests/qemu-iotests/223 | 4 ++-- 4 files changed, 36 insertions(+), 36 deletions(-) diff --git a/blockdev.c b/blockdev.c index 0f740fd964..da87aae5cf 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1966,7 +1966,7 @@ static void block_dirty_bitmap_add_prepare(BlkActionS= tate *common, action->has_granularity, action->granularit= y, action->has_persistent, action->persistent, action->has_autoload, action->autoload, - action->has_x_disabled, action->x_disabled, + action->has_disabled, action->disabled, &local_err); =20 if (!local_err) { @@ -2051,7 +2051,7 @@ static void block_dirty_bitmap_enable_prepare(BlkActi= onState *common, return; } =20 - action =3D common->action->u.x_block_dirty_bitmap_enable.data; + action =3D common->action->u.block_dirty_bitmap_enable.data; state->bitmap =3D block_dirty_bitmap_lookup(action->node, action->name, NULL, @@ -2092,7 +2092,7 @@ static void block_dirty_bitmap_disable_prepare(BlkAct= ionState *common, return; } =20 - action =3D common->action->u.x_block_dirty_bitmap_disable.data; + action =3D common->action->u.block_dirty_bitmap_disable.data; state->bitmap =3D block_dirty_bitmap_lookup(action->node, action->name, NULL, @@ -2137,7 +2137,7 @@ static void block_dirty_bitmap_merge_prepare(BlkActio= nState *common, return; } =20 - action =3D common->action->u.x_block_dirty_bitmap_merge.data; + action =3D common->action->u.block_dirty_bitmap_merge.data; =20 do_block_dirty_bitmap_merge(action->node, action->target, action->bitmaps, &state->backup, @@ -2205,17 +2205,17 @@ static const BlkActionOps actions[] =3D { .commit =3D block_dirty_bitmap_free_backup, .abort =3D block_dirty_bitmap_restore, }, - [TRANSACTION_ACTION_KIND_X_BLOCK_DIRTY_BITMAP_ENABLE] =3D { + [TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_ENABLE] =3D { .instance_size =3D sizeof(BlockDirtyBitmapState), .prepare =3D block_dirty_bitmap_enable_prepare, .abort =3D block_dirty_bitmap_enable_abort, }, - [TRANSACTION_ACTION_KIND_X_BLOCK_DIRTY_BITMAP_DISABLE] =3D { + [TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_DISABLE] =3D { .instance_size =3D sizeof(BlockDirtyBitmapState), .prepare =3D block_dirty_bitmap_disable_prepare, .abort =3D block_dirty_bitmap_disable_abort, }, - [TRANSACTION_ACTION_KIND_X_BLOCK_DIRTY_BITMAP_MERGE] =3D { + [TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_MERGE] =3D { .instance_size =3D sizeof(BlockDirtyBitmapState), .prepare =3D block_dirty_bitmap_merge_prepare, .commit =3D block_dirty_bitmap_free_backup, @@ -2931,7 +2931,7 @@ void qmp_block_dirty_bitmap_clear(const char *node, c= onst char *name, bdrv_clear_dirty_bitmap(bitmap, NULL); } =20 -void qmp_x_block_dirty_bitmap_enable(const char *node, const char *name, +void qmp_block_dirty_bitmap_enable(const char *node, const char *name, Error **errp) { BlockDriverState *bs; @@ -2952,7 +2952,7 @@ void qmp_x_block_dirty_bitmap_enable(const char *node= , const char *name, bdrv_enable_dirty_bitmap(bitmap); } =20 -void qmp_x_block_dirty_bitmap_disable(const char *node, const char *name, +void qmp_block_dirty_bitmap_disable(const char *node, const char *name, Error **errp) { BlockDriverState *bs; @@ -3014,8 +3014,8 @@ void do_block_dirty_bitmap_merge(const char *node, co= nst char *target, bdrv_release_dirty_bitmap(bs, anon); } =20 -void qmp_x_block_dirty_bitmap_merge(const char *node, const char *target, - strList *bitmaps, Error **errp) +void qmp_block_dirty_bitmap_merge(const char *node, const char *target, + strList *bitmaps, Error **errp) { do_block_dirty_bitmap_merge(node, target, bitmaps, NULL, errp); } diff --git a/qapi/block-core.json b/qapi/block-core.json index 320d74ef34..fde96fdb50 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1803,15 +1803,15 @@ # Currently, all dirty tracking bitmaps are loaded from Qcow2 on # open. # -# @x-disabled: the bitmap is created in the disabled state, which means th= at -# it will not track drive changes. The bitmap may be enabled = with -# x-block-dirty-bitmap-enable. Default is false. (Since: 3.0) +# @disabled: the bitmap is created in the disabled state, which means that +# it will not track drive changes. The bitmap may be enabled wi= th +# block-dirty-bitmap-enable. Default is false. (Since: 4.0) # # Since: 2.4 ## { 'struct': 'BlockDirtyBitmapAdd', 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32', - '*persistent': 'bool', '*autoload': 'bool', '*x-disabled': 'bo= ol' } } + '*persistent': 'bool', '*autoload': 'bool', '*disabled': 'bool= ' } } =20 ## # @BlockDirtyBitmapMerge: @@ -1822,7 +1822,7 @@ # # @bitmaps: name(s) of the source dirty bitmap(s) # -# Since: 3.0 +# Since: 4.0 ## { 'struct': 'BlockDirtyBitmapMerge', 'data': { 'node': 'str', 'target': 'str', 'bitmaps': ['str'] } } @@ -1896,7 +1896,7 @@ 'data': 'BlockDirtyBitmap' } =20 ## -# @x-block-dirty-bitmap-enable: +# @block-dirty-bitmap-enable: # # Enables a dirty bitmap so that it will begin tracking disk changes. # @@ -1904,20 +1904,20 @@ # If @node is not a valid block device, DeviceNotFound # If @name is not found, GenericError with an explanation # -# Since: 3.0 +# Since: 4.0 # # Example: # -# -> { "execute": "x-block-dirty-bitmap-enable", +# -> { "execute": "block-dirty-bitmap-enable", # "arguments": { "node": "drive0", "name": "bitmap0" } } # <- { "return": {} } # ## - { 'command': 'x-block-dirty-bitmap-enable', + { 'command': 'block-dirty-bitmap-enable', 'data': 'BlockDirtyBitmap' } =20 ## -# @x-block-dirty-bitmap-disable: +# @block-dirty-bitmap-disable: # # Disables a dirty bitmap so that it will stop tracking disk changes. # @@ -1925,20 +1925,20 @@ # If @node is not a valid block device, DeviceNotFound # If @name is not found, GenericError with an explanation # -# Since: 3.0 +# Since: 4.0 # # Example: # -# -> { "execute": "x-block-dirty-bitmap-disable", +# -> { "execute": "block-dirty-bitmap-disable", # "arguments": { "node": "drive0", "name": "bitmap0" } } # <- { "return": {} } # ## - { 'command': 'x-block-dirty-bitmap-disable', + { 'command': 'block-dirty-bitmap-disable', 'data': 'BlockDirtyBitmap' } =20 ## -# @x-block-dirty-bitmap-merge: +# @block-dirty-bitmap-merge: # # Merge dirty bitmaps listed in @bitmaps to the @target dirty bitmap. # The @bitmaps dirty bitmaps are unchanged. @@ -1950,17 +1950,17 @@ # If any of the bitmaps have different sizes or granularities, # GenericError # -# Since: 3.0 +# Since: 4.0 # # Example: # -# -> { "execute": "x-block-dirty-bitmap-merge", +# -> { "execute": "block-dirty-bitmap-merge", # "arguments": { "node": "drive0", "target": "bitmap0", # "bitmaps": ["bitmap1"] } } # <- { "return": {} } # ## - { 'command': 'x-block-dirty-bitmap-merge', + { 'command': 'block-dirty-bitmap-merge', 'data': 'BlockDirtyBitmapMerge' } =20 ## diff --git a/qapi/transaction.json b/qapi/transaction.json index 5875cdb16c..95edb78227 100644 --- a/qapi/transaction.json +++ b/qapi/transaction.json @@ -46,9 +46,9 @@ # - @abort: since 1.6 # - @block-dirty-bitmap-add: since 2.5 # - @block-dirty-bitmap-clear: since 2.5 -# - @x-block-dirty-bitmap-enable: since 3.0 -# - @x-block-dirty-bitmap-disable: since 3.0 -# - @x-block-dirty-bitmap-merge: since 3.1 +# - @block-dirty-bitmap-enable: since 4.0 +# - @block-dirty-bitmap-disable: since 4.0 +# - @block-dirty-bitmap-merge: since 4.0 # - @blockdev-backup: since 2.3 # - @blockdev-snapshot: since 2.5 # - @blockdev-snapshot-internal-sync: since 1.7 @@ -62,9 +62,9 @@ 'abort': 'Abort', 'block-dirty-bitmap-add': 'BlockDirtyBitmapAdd', 'block-dirty-bitmap-clear': 'BlockDirtyBitmap', - 'x-block-dirty-bitmap-enable': 'BlockDirtyBitmap', - 'x-block-dirty-bitmap-disable': 'BlockDirtyBitmap', - 'x-block-dirty-bitmap-merge': 'BlockDirtyBitmapMerge', + 'block-dirty-bitmap-enable': 'BlockDirtyBitmap', + 'block-dirty-bitmap-disable': 'BlockDirtyBitmap', + 'block-dirty-bitmap-merge': 'BlockDirtyBitmapMerge', 'blockdev-backup': 'BlockdevBackup', 'blockdev-snapshot': 'BlockdevSnapshot', 'blockdev-snapshot-internal-sync': 'BlockdevSnapshotInternal', diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223 index 397b865d34..5513dc6215 100755 --- a/tests/qemu-iotests/223 +++ b/tests/qemu-iotests/223 @@ -112,9 +112,9 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"qmp_capabiliti= es"}' "return" _send_qemu_cmd $QEMU_HANDLE '{"execute":"blockdev-add", "arguments":{"driver":"qcow2", "node-name":"n", "file":{"driver":"file", "filename":"'"$TEST_IMG"'"}}}' "return" -_send_qemu_cmd $QEMU_HANDLE '{"execute":"x-block-dirty-bitmap-disable", +_send_qemu_cmd $QEMU_HANDLE '{"execute":"block-dirty-bitmap-disable", "arguments":{"node":"n", "name":"b"}}' "return" -_send_qemu_cmd $QEMU_HANDLE '{"execute":"x-block-dirty-bitmap-disable", +_send_qemu_cmd $QEMU_HANDLE '{"execute":"block-dirty-bitmap-disable", "arguments":{"node":"n", "name":"b2"}}' "return" _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start", "arguments":{"addr":{"type":"unix", --=20 2.17.2 From nobody Thu May 2 01:27:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1544829659270271.366296335739; Fri, 14 Dec 2018 15:20:59 -0800 (PST) Received: from localhost ([::1]:35951 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwlO-0005XE-3n for importer@patchew.org; Fri, 14 Dec 2018 18:20:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39914) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwgU-0000xG-Uz for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:56 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXwgU-0003Gr-1b for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:54 -0500 Received: from mx1.redhat.com ([209.132.183.28]:60090) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXwgN-0001hp-O5; Fri, 14 Dec 2018 18:15:47 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CB94742BBB; Fri, 14 Dec 2018 23:15:25 +0000 (UTC) Received: from probe.redhat.com (ovpn-122-54.rdu2.redhat.com [10.10.122.54]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6494319C65; Fri, 14 Dec 2018 23:15:24 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 18:15:09 -0500 Message-Id: <20181214231512.5295-5-jsnow@redhat.com> In-Reply-To: <20181214231512.5295-1-jsnow@redhat.com> References: <20181214231512.5295-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 14 Dec 2018 23:15:25 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 4/7] iotests.py: don't abort if IMGKEYSECRET is undefined X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , vsementsov@virtuozzo.com, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Instead of using os.environ[], use .get with a default of empty string to match the setup in check to allow us to import the iotests module (for debugging, say) without needing a crafted environment just to import the module. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index d537538ba0..a34e66813a 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -63,7 +63,7 @@ socket_scm_helper =3D os.environ.get('SOCKET_SCM_HELPER',= 'socket_scm_helper') debug =3D False =20 luks_default_secret_object =3D 'secret,id=3Dkeysec0,data=3D' + \ - os.environ['IMGKEYSECRET'] + os.environ.get('IMGKEYSECRET', '') luks_default_key_secret_opt =3D 'key-secret=3Dkeysec0' =20 =20 --=20 2.17.2 From nobody Thu May 2 01:27:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1544830491857854.3764277508354; Fri, 14 Dec 2018 15:34:51 -0800 (PST) Received: from localhost ([::1]:36049 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwyf-000498-Ds for importer@patchew.org; Fri, 14 Dec 2018 18:34:41 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44246) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwxZ-0003MN-Ox for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:33:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXwxZ-0002Al-6E for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:33:33 -0500 Received: from mx1.redhat.com ([209.132.183.28]:40260) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXwxU-00023p-AT; Fri, 14 Dec 2018 18:33:28 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 91EF57FDE3; Fri, 14 Dec 2018 23:33:27 +0000 (UTC) Received: from probe.redhat.com (ovpn-122-54.rdu2.redhat.com [10.10.122.54]) by smtp.corp.redhat.com (Postfix) with ESMTP id D0F9160BF6; Fri, 14 Dec 2018 23:33:23 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 14 Dec 2018 18:32:57 -0500 Message-Id: <20181214233257.7543-1-jsnow@redhat.com> In-Reply-To: <20181214231512.5295-1-jsnow@redhat.com> References: <20181214231512.5295-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 14 Dec 2018 23:33:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 5/7 RESEND] iotests: add filter_generated_node_ids X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To mimic the common filter of the same name, but for the python tests. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index a34e66813a..9595429fea 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -239,6 +239,9 @@ def filter_testfiles(msg): prefix =3D os.path.join(test_dir, "%s-" % (os.getpid())) return msg.replace(prefix, 'TEST_DIR/PID-') =20 +def filter_generated_node_ids(msg): + return re.sub("#block[0-9]+", "NODE_NAME", msg) + def filter_img_info(output, filename): lines =3D [] for line in output.split('\n'): --=20 2.17.2 From nobody Thu May 2 01:27:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1544829682095266.49762541634357; Fri, 14 Dec 2018 15:21:22 -0800 (PST) Received: from localhost ([::1]:35955 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwli-0005pY-Ts for importer@patchew.org; Fri, 14 Dec 2018 18:21:18 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39916) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwgV-0000xX-76 for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:56 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXwgU-0003He-8v for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:55 -0500 Received: from mx1.redhat.com ([209.132.183.28]:40126) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXwgN-00024q-Pl; Fri, 14 Dec 2018 18:15:47 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B17A7138205; Fri, 14 Dec 2018 23:15:33 +0000 (UTC) Received: from probe.redhat.com (ovpn-122-54.rdu2.redhat.com [10.10.122.54]) by smtp.corp.redhat.com (Postfix) with ESMTP id CAF5718506; Fri, 14 Dec 2018 23:15:31 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 18:15:11 -0500 Message-Id: <20181214231512.5295-7-jsnow@redhat.com> In-Reply-To: <20181214231512.5295-1-jsnow@redhat.com> References: <20181214231512.5295-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 14 Dec 2018 23:15:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 6/7] iotests: allow pretty-print for qmp_log X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , vsementsov@virtuozzo.com, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" If iotests have lines exceeding >998 characters long, git doesn't want to send it plaintext to the list. We can solve this by allowing the iotests to use pretty printed QMP output that we can match against instead. As a bonus, it's much nicer for human eyes, too. Note that this changes the sort order for "command" and "arguments", so I restrict this reordering to occur only when the indent is specified. Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 9595429fea..ab5823c011 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -447,12 +447,21 @@ class VM(qtest.QEMUQtestMachine): result.append(filter_qmp_event(ev)) return result =20 - def qmp_log(self, cmd, filters=3D[filter_testfiles], **kwargs): - logmsg =3D '{"execute": "%s", "arguments": %s}' % \ - (cmd, json.dumps(kwargs, sort_keys=3DTrue)) + def qmp_log(self, cmd, indent=3DNone, filters=3D[filter_testfiles], **= kwargs): + # Python < 3.4 needs to know not to add whitespace when pretty-pri= nting: + separators =3D (',', ': ') if indent is not None else (',', ': ') + if indent is not None: + fullcmd =3D { "execute": cmd, + "arguments": kwargs } + logmsg =3D json.dumps(fullcmd, indent=3Dindent, separators=3Ds= eparators, + sort_keys=3DTrue) + else: + logmsg =3D '{"execute": "%s", "arguments": %s}' % \ + (cmd, json.dumps(kwargs, sort_keys=3DTrue)) log(logmsg, filters) result =3D self.qmp(cmd, **kwargs) - log(json.dumps(result, sort_keys=3DTrue), filters) + log(json.dumps(result, indent=3Dindent, separators=3Dseparators, + sort_keys=3DTrue), filters) return result =20 def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse): --=20 2.17.2 From nobody Thu May 2 01:27:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1544829722668902.0744229522744; Fri, 14 Dec 2018 15:22:02 -0800 (PST) Received: from localhost ([::1]:35957 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwmP-0006K9-BZ for importer@patchew.org; Fri, 14 Dec 2018 18:22:01 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39956) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXwgW-0000yI-4d for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXwgU-0003HZ-86 for qemu-devel@nongnu.org; Fri, 14 Dec 2018 18:15:56 -0500 Received: from mx1.redhat.com ([209.132.183.28]:34092) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXwgN-0002QM-Qm; Fri, 14 Dec 2018 18:15:48 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E655C7EBA9; Fri, 14 Dec 2018 23:15:40 +0000 (UTC) Received: from probe.redhat.com (ovpn-122-54.rdu2.redhat.com [10.10.122.54]) by smtp.corp.redhat.com (Postfix) with ESMTP id 143D419C65; Fri, 14 Dec 2018 23:15:33 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 18:15:12 -0500 Message-Id: <20181214231512.5295-8-jsnow@redhat.com> In-Reply-To: <20181214231512.5295-1-jsnow@redhat.com> References: <20181214231512.5295-1-jsnow@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 14 Dec 2018 23:15:40 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 7/7] iotests: add iotest 236 for testing bitmap merge X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , vsementsov@virtuozzo.com, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" New interface, new smoke test. Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/236 | 124 +++++++++++++++++++++++ tests/qemu-iotests/236.out | 200 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 325 insertions(+) create mode 100755 tests/qemu-iotests/236 create mode 100644 tests/qemu-iotests/236.out diff --git a/tests/qemu-iotests/236 b/tests/qemu-iotests/236 new file mode 100755 index 0000000000..edf16c4ab1 --- /dev/null +++ b/tests/qemu-iotests/236 @@ -0,0 +1,124 @@ +#!/usr/bin/env python +# +# Test bitmap merges. +# +# Copyright (c) 2018 John Snow for Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# owner=3Djsnow@redhat.com + +import json +import iotests +from iotests import log + +iotests.verify_image_format(supported_fmts=3D['qcow2']) + +patterns =3D [("0x5d", "0", "64k"), + ("0xd5", "1M", "64k"), + ("0xdc", "32M", "64k"), + ("0xcd", "0x3ff0000", "64k")] # 64M - 64K + +overwrite =3D [("0xab", "0", "64k"), # Full overwrite + ("0xad", "0x00f8000", "64k"), # Partial-left (1M-32K) + ("0x1d", "0x2008000", "64k"), # Partial-right (32M+32K) + ("0xea", "0x3fe0000", "64k")] # Adjacent-left (64M - 128K) + +def query_bitmaps(vm): + res =3D vm.qmp("query-block") + return {device['device']: device['dirty-bitmaps'] for + device in res['return']} + +with iotests.FilePath('img') as img_path, \ + iotests.VM() as vm: + + log('--- Preparing image & VM ---\n') + iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, img_path, '64M') + vm.add_drive(img_path) + vm.launch() + + log('--- Adding preliminary bitmaps A & B ---\n') + vm.qmp_log("block-dirty-bitmap-add", node=3D"drive0", name=3D"bitmapA") + vm.qmp_log("block-dirty-bitmap-add", node=3D"drive0", name=3D"bitmapB") + + # Dirties 4 clusters. count=3D262144 + log('') + log('--- Emulating writes ---\n') + for p in patterns: + cmd =3D "write -P%s %s %s" % p + log(cmd) + log(vm.hmp_qemu_io("drive0", cmd)) + + log(json.dumps(query_bitmaps(vm), indent=3D2, separators=3D(',', ': ')= )) + + log('') + log('--- Disabling B & Adding C ---\n') + vm.qmp_log("transaction", indent=3D2, actions=3D[ + { "type": "block-dirty-bitmap-disable", + "data": { "node": "drive0", "name": "bitmapB" }}, + { "type": "block-dirty-bitmap-add", + "data": { "node": "drive0", "name": "bitmapC" }}, + # Purely extraneous, but test that it works: + { "type": "block-dirty-bitmap-disable", + "data": { "node": "drive0", "name": "bitmapC" }}, + { "type": "block-dirty-bitmap-enable", + "data": { "node": "drive0", "name": "bitmapC" }}, + ]) + + log('') + log('--- Emulating further writes ---\n') + # Dirties 6 clusters, 3 of which are new in contrast to "A". + # A =3D 64 * 1024 * (4 + 3) =3D 458752 + # C =3D 64 * 1024 * 6 =3D 393216 + for p in overwrite: + cmd =3D "write -P%s %s %s" % p + log(cmd) + log(vm.hmp_qemu_io("drive0", cmd)) + + log('') + log('--- Disabling A & C ---\n') + vm.qmp_log("transaction", indent=3D2, actions=3D[ + { "type": "block-dirty-bitmap-disable", + "data": { "node": "drive0", "name": "bitmapA" }}, + { "type": "block-dirty-bitmap-disable", + "data": { "node": "drive0", "name": "bitmapC" }} + ]) + + # A: 7 clusters + # B: 4 clusters + # C: 6 clusters + log(json.dumps(query_bitmaps(vm), indent=3D2, separators=3D(',', ': ')= )) + + log('') + log('--- Creating D as a merge of B & C ---\n') + # Good hygiene: create a disabled bitmap as a merge target. + vm.qmp_log("transaction", indent=3D2, actions=3D[ + { "type": "block-dirty-bitmap-add", + "data": { "node": "drive0", "name": "bitmapD", "disabled": True = }}, + { "type": "block-dirty-bitmap-merge", + "data": { "node": "drive0", "target": "bitmapD", + "bitmaps": ["bitmapB", "bitmapC"] }} + ]) + + # A and D should now both have 7 clusters apiece. + # B and C remain unchanged with 4 and 6 respectively. + log(json.dumps(query_bitmaps(vm), indent=3D2, separators=3D(',', ': ')= )) + + # A and D should be equivalent. + vm.qmp_log('x-debug-block-dirty-bitmap-sha256', + node=3D"drive0", name=3D"bitmapA") + vm.qmp_log('x-debug-block-dirty-bitmap-sha256', + node=3D"drive0", name=3D"bitmapD") + + vm.shutdown() diff --git a/tests/qemu-iotests/236.out b/tests/qemu-iotests/236.out new file mode 100644 index 0000000000..42c131504d --- /dev/null +++ b/tests/qemu-iotests/236.out @@ -0,0 +1,200 @@ +--- Preparing image & VM --- + +--- Adding preliminary bitmaps A & B --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmapA", "no= de": "drive0"}} +{"return": {}} +{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmapB", "no= de": "drive0"}} +{"return": {}} + +--- Emulating writes --- + +write -P0x5d 0 64k +{"return": ""} +write -P0xd5 1M 64k +{"return": ""} +write -P0xdc 32M 64k +{"return": ""} +write -P0xcd 0x3ff0000 64k +{"return": ""} +{ + "drive0": [ + { + "status": "active", + "count": 262144, + "name": "bitmapB", + "granularity": 65536 + }, + { + "status": "active", + "count": 262144, + "name": "bitmapA", + "granularity": 65536 + } + ] +} + +--- Disabling B & Adding C --- + +{ + "arguments": { + "actions": [ + { + "data": { + "name": "bitmapB", + "node": "drive0" + }, + "type": "block-dirty-bitmap-disable" + }, + { + "data": { + "name": "bitmapC", + "node": "drive0" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "name": "bitmapC", + "node": "drive0" + }, + "type": "block-dirty-bitmap-disable" + }, + { + "data": { + "name": "bitmapC", + "node": "drive0" + }, + "type": "block-dirty-bitmap-enable" + } + ] + }, + "execute": "transaction" +} +{ + "return": {} +} + +--- Emulating further writes --- + +write -P0xab 0 64k +{"return": ""} +write -P0xad 0x00f8000 64k +{"return": ""} +write -P0x1d 0x2008000 64k +{"return": ""} +write -P0xea 0x3fe0000 64k +{"return": ""} + +--- Disabling A & C --- + +{ + "arguments": { + "actions": [ + { + "data": { + "name": "bitmapA", + "node": "drive0" + }, + "type": "block-dirty-bitmap-disable" + }, + { + "data": { + "name": "bitmapC", + "node": "drive0" + }, + "type": "block-dirty-bitmap-disable" + } + ] + }, + "execute": "transaction" +} +{ + "return": {} +} +{ + "drive0": [ + { + "status": "disabled", + "count": 393216, + "name": "bitmapC", + "granularity": 65536 + }, + { + "status": "disabled", + "count": 262144, + "name": "bitmapB", + "granularity": 65536 + }, + { + "status": "disabled", + "count": 458752, + "name": "bitmapA", + "granularity": 65536 + } + ] +} + +--- Creating D as a merge of B & C --- + +{ + "arguments": { + "actions": [ + { + "data": { + "disabled": true, + "name": "bitmapD", + "node": "drive0" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "bitmaps": [ + "bitmapB", + "bitmapC" + ], + "node": "drive0", + "target": "bitmapD" + }, + "type": "block-dirty-bitmap-merge" + } + ] + }, + "execute": "transaction" +} +{ + "return": {} +} +{ + "drive0": [ + { + "status": "disabled", + "count": 458752, + "name": "bitmapD", + "granularity": 65536 + }, + { + "status": "disabled", + "count": 393216, + "name": "bitmapC", + "granularity": 65536 + }, + { + "status": "disabled", + "count": 262144, + "name": "bitmapB", + "granularity": 65536 + }, + { + "status": "disabled", + "count": 458752, + "name": "bitmapA", + "granularity": 65536 + } + ] +} +{"execute": "x-debug-block-dirty-bitmap-sha256", "arguments": {"name": "bi= tmapA", "node": "drive0"}} +{"return": {"sha256": "7abe3d7e3c794cfaf9b08bc9ce599192c96a2206f07b42d9997= ff78fdd7af744"}} +{"execute": "x-debug-block-dirty-bitmap-sha256", "arguments": {"name": "bi= tmapD", "node": "drive0"}} +{"return": {"sha256": "7abe3d7e3c794cfaf9b08bc9ce599192c96a2206f07b42d9997= ff78fdd7af744"}} diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 61a6d98ebd..a61f9e83d6 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -233,3 +233,4 @@ 233 auto quick 234 auto quick migration 235 auto quick +236 auto quick \ No newline at end of file --=20 2.17.2