From nobody Fri May 17 12:36:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562715081; cv=none; d=zoho.com; s=zohoarc; b=hoKU31YgEuY8dHubbCjsNqVzTxxe9PWnYQ/DvlhBMLXcFSfxKAgDy71wkPWv/S+xNJJNa4894kKHOFmhEF3h0Rm/kVt0pXkg+8z0kYpvNr1YOTKM8i63+Tw57FA8ks3u7EuKOrhRYTWSO5LjMvFwZoz+qWMdijcpzwYsgyXhSAw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562715081; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=Ofc7FqxVtEJPaHWutO2QB/QD7QvGdWdVFvhc7jDKod4=; b=HCv0bRQ5IL/ON/HfENGNh5y+CbOQBfxxBPZg8a5WVZChR/SyrVAw2Wun/PtZdpYH02vN0vIxVSFdzCpfLHyZiZWHDk9dYQsJB/m63hpwmcS2di41UwLpoGT4TBIfkEmdxsNa/t18LqqTN7TbEQcQgWJN2XstPm4KQfudErwOXZE= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562715081676273.32955508592147; Tue, 9 Jul 2019 16:31:21 -0700 (PDT) Received: from localhost ([::1]:56802 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzZt-0001iz-Mz for importer@patchew.org; Tue, 09 Jul 2019 19:31:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48855) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVH-0007Iq-VR for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVD-00033K-UL for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48846) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV6-0002Wo-Vl; Tue, 09 Jul 2019 19:26:22 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8D54EC0495A6; Tue, 9 Jul 2019 23:25:58 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 12E755B810; Tue, 9 Jul 2019 23:25:56 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:33 -0400 Message-Id: <20190709232550.10724-2-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 09 Jul 2019 23:25:58 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 01/18] qapi/block-core: Introduce BackupCommon X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" drive-backup and blockdev-backup have an awful lot of things in common that are the same. Let's fix that. I don't deduplicate 'target', because the semantics actually did change between each structure. Leave that one alone so it can be documented separately. Signed-off-by: John Snow Reviewed-by: Max Reitz Reviewed-by: Markus Armbruster --- qapi/block-core.json | 103 ++++++++++++++----------------------------- 1 file changed, 33 insertions(+), 70 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 0d43d4f37c..0af3866015 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1315,32 +1315,23 @@ 'data': { 'node': 'str', 'overlay': 'str' } } =20 ## -# @DriveBackup: +# @BackupCommon: # # @job-id: identifier for the newly-created block job. If # omitted, the device name will be used. (Since 2.7) # # @device: the device name or node-name of a root node which should be cop= ied. # -# @target: the target of the new image. If the file exists, or if it -# is a device, the existing file/device will be used as the new -# destination. If it does not exist, a new file will be created. -# -# @format: the format of the new destination, default is to -# probe if @mode is 'existing', else the format of the source -# # @sync: what parts of the disk image should be copied to the destination # (all the disk, only the sectors allocated in the topmost image, f= rom a # dirty bitmap, or only new I/O). # -# @mode: whether and how QEMU should create a new image, default is -# 'absolute-paths'. -# -# @speed: the maximum speed, in bytes per second +# @speed: the maximum speed, in bytes per second. The default is 0, +# for unlimited. # # @bitmap: the name of dirty bitmap if sync is "incremental". # Must be present if sync is "incremental", must NOT be present -# otherwise. (Since 2.4) +# otherwise. (Since 2.4 (drive-backup), 3.1 (blockdev-backup)) # # @compress: true to compress data, if the target format supports it. # (default: false) (since 2.8) @@ -1370,75 +1361,47 @@ # I/O. If an error occurs during a guest write request, the device's # rerror/werror actions will be used. # +# Since: 4.2 +## +{ 'struct': 'BackupCommon', + 'data': { '*job-id': 'str', 'device': 'str', + 'sync': 'MirrorSyncMode', '*speed': 'int', + '*bitmap': 'str', '*compress': 'bool', + '*on-source-error': 'BlockdevOnError', + '*on-target-error': 'BlockdevOnError', + '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } + +## +# @DriveBackup: +# +# @target: the target of the new image. If the file exists, or if it +# is a device, the existing file/device will be used as the new +# destination. If it does not exist, a new file will be created. +# +# @format: the format of the new destination, default is to +# probe if @mode is 'existing', else the format of the source +# +# @mode: whether and how QEMU should create a new image, default is +# 'absolute-paths'. +# # Since: 1.6 ## { 'struct': 'DriveBackup', - 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str', - '*format': 'str', 'sync': 'MirrorSyncMode', - '*mode': 'NewImageMode', '*speed': 'int', - '*bitmap': 'str', '*compress': 'bool', - '*on-source-error': 'BlockdevOnError', - '*on-target-error': 'BlockdevOnError', - '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } + 'base': 'BackupCommon', + 'data': { 'target': 'str', + '*format': 'str', + '*mode': 'NewImageMode' } } =20 ## # @BlockdevBackup: # -# @job-id: identifier for the newly-created block job. If -# omitted, the device name will be used. (Since 2.7) -# -# @device: the device name or node-name of a root node which should be cop= ied. -# # @target: the device name or node-name of the backup target node. # -# @sync: what parts of the disk image should be copied to the destination -# (all the disk, only the sectors allocated in the topmost image, or -# only new I/O). -# -# @speed: the maximum speed, in bytes per second. The default is 0, -# for unlimited. -# -# @bitmap: the name of dirty bitmap if sync is "incremental". -# Must be present if sync is "incremental", must NOT be present -# otherwise. (Since 3.1) -# -# @compress: true to compress data, if the target format supports it. -# (default: false) (since 2.8) -# -# @on-source-error: the action to take on an error on the source, -# default 'report'. 'stop' and 'enospc' can only be used -# if the block device supports io-status (see BlockInfo). -# -# @on-target-error: the action to take on an error on the target, -# default 'report' (no limitations, since this applies to -# a different block device than @device). -# -# @auto-finalize: When false, this job will wait in a PENDING state after = it has -# finished its work, waiting for @block-job-finalize before -# making any block graph changes. -# When true, this job will automatically -# perform its abort or commit actions. -# Defaults to true. (Since 2.12) -# -# @auto-dismiss: When false, this job will wait in a CONCLUDED state after= it -# has completely ceased all work, and awaits @block-job-dis= miss. -# When true, this job will automatically disappear from the= query -# list without user intervention. -# Defaults to true. (Since 2.12) -# -# Note: @on-source-error and @on-target-error only affect background -# I/O. If an error occurs during a guest write request, the device's -# rerror/werror actions will be used. -# # Since: 2.3 ## { 'struct': 'BlockdevBackup', - 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str', - 'sync': 'MirrorSyncMode', '*speed': 'int', - '*bitmap': 'str', '*compress': 'bool', - '*on-source-error': 'BlockdevOnError', - '*on-target-error': 'BlockdevOnError', - '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } + 'base': 'BackupCommon', + 'data': { 'target': 'str' } } =20 ## # @blockdev-snapshot-sync: --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562716517; cv=none; d=zoho.com; s=zohoarc; b=XLPrUSFEuPGfjW58hCVxJkn7aQTGUWdC2yqud+l5AoTZwlxbp9Iz7krZufUqJanwwKPDXKPicMywAOMSQstOyOa7SPbjs5qIY7Hx3r4F5DQhA1RH/XLMX/HCGqPX55wXNZx6NYVl2BDgxXNfSRB5WvVLUwvYPrOZB8OiynuWyS8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562716517; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=+Bku5KZ+ewuk0uxkEnKmyXjRKuyNKJTS+TPPkG/hon4=; b=Oq3GBvsUUToHnxQ+uZfoQLh58eHaPR+vQZeYDjM0xvcXHh8XIkBkATb9/SR/0oVFb/wLpuQZblwfkCwGjYrHKkfJ6UXc+NeCmHsGsjq2ziL6Q95mQRfsUVMhkz6shqMAFw94yYB3K1H2EXJV8mNwW3mJpD2zCyyrgSwJs8cvHbI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156271651722819.24380779304238; Tue, 9 Jul 2019 16:55:17 -0700 (PDT) Received: from localhost ([::1]:56924 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzx3-0005hL-7h for importer@patchew.org; Tue, 09 Jul 2019 19:55:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49025) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVL-0007MW-CR for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVH-0003A8-Sc for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33656) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV9-0002Y5-Jw; Tue, 09 Jul 2019 19:26:25 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1865AF0D13; Tue, 9 Jul 2019 23:26:00 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id ADAEB5B80B; Tue, 9 Jul 2019 23:25:58 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:34 -0400 Message-Id: <20190709232550.10724-3-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 09 Jul 2019 23:26:00 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 02/18] drive-backup: create do_backup_common X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Create a common core that comprises the actual meat of what the backup API boundary needs to do, and then switch drive-backup to use it. Signed-off-by: John Snow Reviewed-by: Max Reitz --- blockdev.c | 122 +++++++++++++++++++++++++++++------------------------ 1 file changed, 67 insertions(+), 55 deletions(-) diff --git a/blockdev.c b/blockdev.c index 4d141e9a1f..5bc8ecd087 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3425,6 +3425,70 @@ out: aio_context_release(aio_context); } =20 +/* Common QMP interface for drive-backup and blockdev-backup */ +static BlockJob *do_backup_common(BackupCommon *backup, + BlockDriverState *bs, + BlockDriverState *target_bs, + AioContext *aio_context, + JobTxn *txn, Error **errp) +{ + BlockJob *job =3D NULL; + BdrvDirtyBitmap *bmap =3D NULL; + int job_flags =3D JOB_DEFAULT; + int ret; + + if (!backup->has_speed) { + backup->speed =3D 0; + } + if (!backup->has_on_source_error) { + backup->on_source_error =3D BLOCKDEV_ON_ERROR_REPORT; + } + if (!backup->has_on_target_error) { + backup->on_target_error =3D BLOCKDEV_ON_ERROR_REPORT; + } + if (!backup->has_job_id) { + backup->job_id =3D NULL; + } + if (!backup->has_auto_finalize) { + backup->auto_finalize =3D true; + } + if (!backup->has_auto_dismiss) { + backup->auto_dismiss =3D true; + } + if (!backup->has_compress) { + backup->compress =3D false; + } + + ret =3D bdrv_try_set_aio_context(target_bs, aio_context, errp); + if (ret < 0) { + return NULL; + } + + if (backup->has_bitmap) { + bmap =3D bdrv_find_dirty_bitmap(bs, backup->bitmap); + if (!bmap) { + error_setg(errp, "Bitmap '%s' could not be found", backup->bit= map); + return NULL; + } + if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) { + return NULL; + } + } + + if (!backup->auto_finalize) { + job_flags |=3D JOB_MANUAL_FINALIZE; + } + if (!backup->auto_dismiss) { + job_flags |=3D JOB_MANUAL_DISMISS; + } + + job =3D backup_job_create(backup->job_id, bs, target_bs, backup->speed, + backup->sync, bmap, backup->compress, + backup->on_source_error, backup->on_target_err= or, + job_flags, NULL, NULL, txn, errp); + return job; +} + static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn, Error **errp) { @@ -3432,39 +3496,16 @@ static BlockJob *do_drive_backup(DriveBackup *backu= p, JobTxn *txn, BlockDriverState *target_bs; BlockDriverState *source =3D NULL; BlockJob *job =3D NULL; - BdrvDirtyBitmap *bmap =3D NULL; AioContext *aio_context; QDict *options =3D NULL; Error *local_err =3D NULL; - int flags, job_flags =3D JOB_DEFAULT; + int flags; int64_t size; bool set_backing_hd =3D false; - int ret; =20 - if (!backup->has_speed) { - backup->speed =3D 0; - } - if (!backup->has_on_source_error) { - backup->on_source_error =3D BLOCKDEV_ON_ERROR_REPORT; - } - if (!backup->has_on_target_error) { - backup->on_target_error =3D BLOCKDEV_ON_ERROR_REPORT; - } if (!backup->has_mode) { backup->mode =3D NEW_IMAGE_MODE_ABSOLUTE_PATHS; } - if (!backup->has_job_id) { - backup->job_id =3D NULL; - } - if (!backup->has_auto_finalize) { - backup->auto_finalize =3D true; - } - if (!backup->has_auto_dismiss) { - backup->auto_dismiss =3D true; - } - if (!backup->has_compress) { - backup->compress =3D false; - } =20 bs =3D bdrv_lookup_bs(backup->device, backup->device, errp); if (!bs) { @@ -3541,12 +3582,6 @@ static BlockJob *do_drive_backup(DriveBackup *backup= , JobTxn *txn, goto out; } =20 - ret =3D bdrv_try_set_aio_context(target_bs, aio_context, errp); - if (ret < 0) { - bdrv_unref(target_bs); - goto out; - } - if (set_backing_hd) { bdrv_set_backing_hd(target_bs, source, &local_err); if (local_err) { @@ -3554,31 +3589,8 @@ static BlockJob *do_drive_backup(DriveBackup *backup= , JobTxn *txn, } } =20 - if (backup->has_bitmap) { - bmap =3D bdrv_find_dirty_bitmap(bs, backup->bitmap); - if (!bmap) { - error_setg(errp, "Bitmap '%s' could not be found", backup->bit= map); - goto unref; - } - if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) { - goto unref; - } - } - if (!backup->auto_finalize) { - job_flags |=3D JOB_MANUAL_FINALIZE; - } - if (!backup->auto_dismiss) { - job_flags |=3D JOB_MANUAL_DISMISS; - } - - job =3D backup_job_create(backup->job_id, bs, target_bs, backup->speed, - backup->sync, bmap, backup->compress, - backup->on_source_error, backup->on_target_err= or, - job_flags, NULL, NULL, txn, &local_err); - if (local_err !=3D NULL) { - error_propagate(errp, local_err); - goto unref; - } + job =3D do_backup_common(qapi_DriveBackup_base(backup), + bs, target_bs, aio_context, txn, errp); =20 unref: bdrv_unref(target_bs); --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562716476; cv=none; d=zoho.com; s=zohoarc; b=QzfSqXcd2QLEc92Z8ifN1F9t6FBMdmtGf5xTbW90kgLtzma34N4y1ABmGeZr5EC3L6BCNAFsmmLJjMujwVNDnj8Boq063pIZGNrHULsEqG3KtiQFT9q9/gptbyGVChENrhNtrHLvERjBmoNvdhkhvsgpiq+03oot76TnTMcx9I4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562716476; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=fFZkLeO1qIxMfvJvjMyxOnJdAm8Sl3wY7L/gUBQbbF0=; b=DG7msFwaUgAxvve9PSZoC0F8qiWKtCvVem9aP/rXGeb0YnfJi+MqzgBfoyp2MkZE2ayYfI1KBEPCM900cV+FgedOedPpznW5gzGqR61AZQ0z4KdmbYT49E5yQGq8LmU2I5+391aRKXLxY5PkEZK1+MmwtAq/SFk1lZnV6HJYMSs= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562716476219913.5427923917496; Tue, 9 Jul 2019 16:54:36 -0700 (PDT) Received: from localhost ([::1]:56922 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzwM-0005Xt-UM for importer@patchew.org; Tue, 09 Jul 2019 19:54:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48775) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVG-0007GG-5L for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVD-000330-SK for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33800) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV9-0002ZP-9C; Tue, 09 Jul 2019 19:26:23 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 89C9CCAA74; Tue, 9 Jul 2019 23:26:01 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 324EA5B0BD; Tue, 9 Jul 2019 23:26:00 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:35 -0400 Message-Id: <20190709232550.10724-4-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 09 Jul 2019 23:26:01 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 03/18] blockdev-backup: utilize do_backup_common X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Signed-off-by: John Snow Reviewed-by: Max Reitz --- blockdev.c | 65 +++++------------------------------------------------- 1 file changed, 6 insertions(+), 59 deletions(-) diff --git a/blockdev.c b/blockdev.c index 5bc8ecd087..77365d8166 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3624,78 +3624,25 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup= , JobTxn *txn, { BlockDriverState *bs; BlockDriverState *target_bs; - Error *local_err =3D NULL; - BdrvDirtyBitmap *bmap =3D NULL; AioContext *aio_context; - BlockJob *job =3D NULL; - int job_flags =3D JOB_DEFAULT; - int ret; - - if (!backup->has_speed) { - backup->speed =3D 0; - } - if (!backup->has_on_source_error) { - backup->on_source_error =3D BLOCKDEV_ON_ERROR_REPORT; - } - if (!backup->has_on_target_error) { - backup->on_target_error =3D BLOCKDEV_ON_ERROR_REPORT; - } - if (!backup->has_job_id) { - backup->job_id =3D NULL; - } - if (!backup->has_auto_finalize) { - backup->auto_finalize =3D true; - } - if (!backup->has_auto_dismiss) { - backup->auto_dismiss =3D true; - } - if (!backup->has_compress) { - backup->compress =3D false; - } + BlockJob *job; =20 bs =3D bdrv_lookup_bs(backup->device, backup->device, errp); if (!bs) { return NULL; } =20 - aio_context =3D bdrv_get_aio_context(bs); - aio_context_acquire(aio_context); - target_bs =3D bdrv_lookup_bs(backup->target, backup->target, errp); if (!target_bs) { - goto out; + return NULL; } =20 - ret =3D bdrv_try_set_aio_context(target_bs, aio_context, errp); - if (ret < 0) { - goto out; - } + aio_context =3D bdrv_get_aio_context(bs); + aio_context_acquire(aio_context); =20 - if (backup->has_bitmap) { - bmap =3D bdrv_find_dirty_bitmap(bs, backup->bitmap); - if (!bmap) { - error_setg(errp, "Bitmap '%s' could not be found", backup->bit= map); - goto out; - } - if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) { - goto out; - } - } + job =3D do_backup_common(qapi_BlockdevBackup_base(backup), + bs, target_bs, aio_context, txn, errp); =20 - if (!backup->auto_finalize) { - job_flags |=3D JOB_MANUAL_FINALIZE; - } - if (!backup->auto_dismiss) { - job_flags |=3D JOB_MANUAL_DISMISS; - } - job =3D backup_job_create(backup->job_id, bs, target_bs, backup->speed, - backup->sync, bmap, backup->compress, - backup->on_source_error, backup->on_target_err= or, - job_flags, NULL, NULL, txn, &local_err); - if (local_err !=3D NULL) { - error_propagate(errp, local_err); - } -out: aio_context_release(aio_context); return job; } --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562716077; cv=none; d=zoho.com; s=zohoarc; b=NgZ9UHpEZSnXg8ArfxssUIyOb5VF31/FGs4xRLQyN/0+bxP83keoOpNo6woJQLb+uPJXGeKzxCkpODXfFqsIUg23iNnkwMAN+IJRKo4rFloBUiVdfyRoyJ5cvjewgdvxeyne2jLBkzGALLPoS3Vt8Vam4e0kc1pW7XzeYTYNNnw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562716077; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=us76fjTF6VchOgWMbsNvH/xS0DcSlhXErgXC+9X+BBA=; b=PQgq4JF1rtegdHA/kgYPoiacx6K09GhJz28olPgEZUD9B15ajxUPnGnCDukmLPdXsZXJmIyvu6dFgmw1AzAJiLXrg6lQE9OW0oLjq9+U5TZWXLLXD4L7ekMOsBovjnG40IIXTWdON1nWYMfDJpqL2GSMt9J47/yqGmAdHjgpAW8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562716077189854.4265962640459; Tue, 9 Jul 2019 16:47:57 -0700 (PDT) Received: from localhost ([::1]:56880 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzpj-0001QL-3Q for importer@patchew.org; Tue, 09 Jul 2019 19:47:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48778) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVG-0007GH-6B for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVD-00033C-Vi for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33020) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV9-0002aZ-3l; Tue, 09 Jul 2019 19:26:23 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 134595AFF8; Tue, 9 Jul 2019 23:26:03 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE4625B810; Tue, 9 Jul 2019 23:26:01 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:36 -0400 Message-Id: <20190709232550.10724-5-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 09 Jul 2019 23:26:03 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 04/18] qapi: add BitmapSyncMode enum X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Depending on what a user is trying to accomplish, there might be a few bitmap cleanup actions that occur when an operation is finished that could be useful. I am proposing three: - NEVER: The bitmap is never synchronized against what was copied. - ALWAYS: The bitmap is always synchronized, even on failures. - ON-SUCCESS: The bitmap is synchronized only on success. The existing incremental backup modes use 'on-success' semantics, so add just that one for right now. Signed-off-by: John Snow Reviewed-by: Max Reitz Reviewed-by: Markus Armbruster --- qapi/block-core.json | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/qapi/block-core.json b/qapi/block-core.json index 0af3866015..0c853d00bd 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1134,6 +1134,20 @@ { 'enum': 'MirrorSyncMode', 'data': ['top', 'full', 'none', 'incremental'] } =20 +## +# @BitmapSyncMode: +# +# An enumeration of possible behaviors for the synchronization of a bitmap +# when used for data copy operations. +# +# @on-success: The bitmap is only synced when the operation is successful. +# This is the behavior always used for 'INCREMENTAL' backups. +# +# Since: 4.2 +## +{ 'enum': 'BitmapSyncMode', + 'data': ['on-success'] } + ## # @MirrorCopyMode: # --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562716116; cv=none; d=zoho.com; s=zohoarc; b=Ehk7kW5UM10ngZzkcsTOMCeDbf+c6SM6IoVJj9a/aZ6DcwcvFBQgflVeOmDmRhrzWBAYW/uYOYPNJZPLbrrm6rKkmmqIAPl4OcmCFiG6tMNLRR/Hp93yjpdhHfidwgnXryNV7st6bYM1m6jbugsj4/ApaJgpYD/leRXnZfxWLQw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562716116; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=fS40Du4j6srkOwHtHF9NAgG9oP87T5nntSoXTABGWoM=; b=ZL6mcZ/aPITYjDfkSgLz9EyeTZegBR0qMGMfov5Y8RE8zdIYMQeCzVYf3oGGQca3mtuF7pqYqZAAHDhWn0dUDnfmUlTMb/dTpVMTvxfS/fvw1OiI8crA1mHFUJXGQtM4cqGeQBxhmeamayhPnedJVcUgNTRhgCDWq7nvAAbfOsQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562716116856409.8781875534904; Tue, 9 Jul 2019 16:48:36 -0700 (PDT) Received: from localhost ([::1]:56882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzqW-0001YV-7K for importer@patchew.org; Tue, 09 Jul 2019 19:48:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49031) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVL-0007MZ-Ec for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVH-00039d-R9 for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:35 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58406) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV6-0002c7-W6; Tue, 09 Jul 2019 19:26:22 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8BC74307D88D; Tue, 9 Jul 2019 23:26:04 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 34B1E5B810; Tue, 9 Jul 2019 23:26:03 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:37 -0400 Message-Id: <20190709232550.10724-6-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Tue, 09 Jul 2019 23:26:04 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 05/18] block/backup: Add mirror sync mode 'bitmap' X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" We don't need or want a new sync mode for simple differences in semantics. Create a new mode simply named "BITMAP" that is designed to make use of the new Bitmap Sync Mode field. Because the only bitmap sync mode is 'on-success', this adds no new functionality to the backup job (yet). The old incremental backup mode is maintained as a syntactic sugar for sync=3Dbitmap, mode=3Don-success. Add all of the plumbing necessary to support this new instruction. Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/backup.c | 20 ++++++++++++-------- block/mirror.c | 6 ++++-- block/replication.c | 2 +- blockdev.c | 25 +++++++++++++++++++++++-- include/block/block_int.h | 4 +++- qapi/block-core.json | 21 +++++++++++++++------ 6 files changed, 58 insertions(+), 20 deletions(-) diff --git a/block/backup.c b/block/backup.c index 715e1d3be8..996941fa61 100644 --- a/block/backup.c +++ b/block/backup.c @@ -38,9 +38,9 @@ typedef struct CowRequest { typedef struct BackupBlockJob { BlockJob common; BlockBackend *target; - /* bitmap for sync=3Dincremental */ BdrvDirtyBitmap *sync_bitmap; MirrorSyncMode sync_mode; + BitmapSyncMode bitmap_mode; BlockdevOnError on_source_error; BlockdevOnError on_target_error; CoRwlock flush_rwlock; @@ -452,7 +452,7 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) =20 job_progress_set_remaining(job, s->len); =20 - if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { + if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_BITMAP) { backup_incremental_init_copy_bitmap(s); } else { hbitmap_set(s->copy_bitmap, 0, s->len); @@ -536,6 +536,7 @@ static int64_t backup_calculate_cluster_size(BlockDrive= rState *target, BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, + BitmapSyncMode bitmap_mode, bool compress, BlockdevOnError on_source_error, BlockdevOnError on_target_error, @@ -583,10 +584,13 @@ BlockJob *backup_job_create(const char *job_id, Block= DriverState *bs, return NULL; } =20 - if (sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { + /* QMP interface should have handled translating this to bitmap mode */ + assert(sync_mode !=3D MIRROR_SYNC_MODE_INCREMENTAL); + + if (sync_mode =3D=3D MIRROR_SYNC_MODE_BITMAP) { if (!sync_bitmap) { error_setg(errp, "must provide a valid bitmap name for " - "\"incremental\" sync mode"); + "'%s' sync mode", MirrorSyncMode_str(sync_mode)); return NULL; } =20 @@ -596,8 +600,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, } } else if (sync_bitmap) { error_setg(errp, - "a sync_bitmap was provided to backup_run, " - "but received an incompatible sync_mode (%s)", + "a bitmap was given to backup_job_create, " + "but it received an incompatible sync_mode (%s)", MirrorSyncMode_str(sync_mode)); return NULL; } @@ -639,8 +643,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, job->on_source_error =3D on_source_error; job->on_target_error =3D on_target_error; job->sync_mode =3D sync_mode; - job->sync_bitmap =3D sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL ? - sync_bitmap : NULL; + job->sync_bitmap =3D sync_bitmap; + job->bitmap_mode =3D bitmap_mode; job->compress =3D compress; =20 /* Detect image-fleecing (and similar) schemes */ diff --git a/block/mirror.c b/block/mirror.c index 2fcec70e35..75c8f38c6a 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1717,8 +1717,10 @@ void mirror_start(const char *job_id, BlockDriverSta= te *bs, bool is_none_mode; BlockDriverState *base; =20 - if (mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { - error_setg(errp, "Sync mode 'incremental' not supported"); + if ((mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) || + (mode =3D=3D MIRROR_SYNC_MODE_BITMAP)) { + error_setg(errp, "Sync mode '%s' not supported", + MirrorSyncMode_str(mode)); return; } is_none_mode =3D mode =3D=3D MIRROR_SYNC_MODE_NONE; diff --git a/block/replication.c b/block/replication.c index 23b2993d74..936b2f8b5a 100644 --- a/block/replication.c +++ b/block/replication.c @@ -543,7 +543,7 @@ static void replication_start(ReplicationState *rs, Rep= licationMode mode, =20 s->backup_job =3D backup_job_create( NULL, s->secondary_disk->bs, s->hidden_dis= k->bs, - 0, MIRROR_SYNC_MODE_NONE, NULL, false, + 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, backup_job_completed, bs, NULL, &local_err= ); diff --git a/blockdev.c b/blockdev.c index 77365d8166..5dfaa976c9 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3464,12 +3464,31 @@ static BlockJob *do_backup_common(BackupCommon *bac= kup, return NULL; } =20 + if (backup->sync =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { + if (backup->has_bitmap_mode && + backup->bitmap_mode !=3D BITMAP_SYNC_MODE_ON_SUCCESS) { + error_setg(errp, "Bitmap sync mode must be '%s' " + "when using sync mode '%s'", + BitmapSyncMode_str(BITMAP_SYNC_MODE_ON_SUCCESS), + MirrorSyncMode_str(backup->sync)); + return NULL; + } + backup->has_bitmap_mode =3D true; + backup->sync =3D MIRROR_SYNC_MODE_BITMAP; + backup->bitmap_mode =3D BITMAP_SYNC_MODE_ON_SUCCESS; + } + if (backup->has_bitmap) { bmap =3D bdrv_find_dirty_bitmap(bs, backup->bitmap); if (!bmap) { error_setg(errp, "Bitmap '%s' could not be found", backup->bit= map); return NULL; } + if (!backup->has_bitmap_mode) { + error_setg(errp, "Bitmap sync mode must be given " + "when providing a bitmap"); + return NULL; + } if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) { return NULL; } @@ -3483,8 +3502,10 @@ static BlockJob *do_backup_common(BackupCommon *back= up, } =20 job =3D backup_job_create(backup->job_id, bs, target_bs, backup->speed, - backup->sync, bmap, backup->compress, - backup->on_source_error, backup->on_target_err= or, + backup->sync, bmap, backup->bitmap_mode, + backup->compress, + backup->on_source_error, + backup->on_target_error, job_flags, NULL, NULL, txn, errp); return job; } diff --git a/include/block/block_int.h b/include/block/block_int.h index d6415b53c1..e1f2aa627e 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -1132,7 +1132,8 @@ void mirror_start(const char *job_id, BlockDriverStat= e *bs, * @target: Block device to write to. * @speed: The maximum speed, in bytes per second, or 0 for unlimited. * @sync_mode: What parts of the disk image should be copied to the destin= ation. - * @sync_bitmap: The dirty bitmap if sync_mode is MIRROR_SYNC_MODE_INCREME= NTAL. + * @sync_bitmap: The dirty bitmap if sync_mode is 'bitmap' or 'incremental' + * @bitmap_mode: The bitmap synchronization policy to use. * @on_source_error: The action to take upon error reading from the source. * @on_target_error: The action to take upon error writing to the target. * @creation_flags: Flags that control the behavior of the Job lifetime. @@ -1148,6 +1149,7 @@ BlockJob *backup_job_create(const char *job_id, Block= DriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, + BitmapSyncMode bitmap_mode, bool compress, BlockdevOnError on_source_error, BlockdevOnError on_target_error, diff --git a/qapi/block-core.json b/qapi/block-core.json index 0c853d00bd..99dcd5f099 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1127,12 +1127,15 @@ # # @none: only copy data written from now on # -# @incremental: only copy data described by the dirty bitmap. Since: 2.4 +# @incremental: only copy data described by the dirty bitmap. (since: 2.4) +# +# @bitmap: only copy data described by the dirty bitmap. (since: 4.2) +# Behavior on completion is determined by the BitmapSyncMode. # # Since: 1.3 ## { 'enum': 'MirrorSyncMode', - 'data': ['top', 'full', 'none', 'incremental'] } + 'data': ['top', 'full', 'none', 'incremental', 'bitmap'] } =20 ## # @BitmapSyncMode: @@ -1343,9 +1346,14 @@ # @speed: the maximum speed, in bytes per second. The default is 0, # for unlimited. # -# @bitmap: the name of dirty bitmap if sync is "incremental". -# Must be present if sync is "incremental", must NOT be present -# otherwise. (Since 2.4 (drive-backup), 3.1 (blockdev-backup)) +# @bitmap: the name of a dirty bitmap if sync is "bitmap" or "incremental". +# Must be present if sync is "bitmap" or "incremental". +# Must not be present otherwise. +# (Since 2.4 (drive-backup), 3.1 (blockdev-backup)) +# +# @bitmap-mode: Specifies the type of data the bitmap should contain after +# the operation concludes. Must be present if sync is "bitma= p". +# Must NOT be present otherwise. (Since 4.2) # # @compress: true to compress data, if the target format supports it. # (default: false) (since 2.8) @@ -1380,7 +1388,8 @@ { 'struct': 'BackupCommon', 'data': { '*job-id': 'str', 'device': 'str', 'sync': 'MirrorSyncMode', '*speed': 'int', - '*bitmap': 'str', '*compress': 'bool', + '*bitmap': 'str', '*bitmap-mode': 'BitmapSyncMode', + '*compress': 'bool', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562715514; cv=none; d=zoho.com; s=zohoarc; b=jghQB40nsqn/b4ECX3d6huoN5jsVuGlYoCv0yUoRhTRWQIy/pZ6pj5fklBdVcNibA3x/G2qb9FXc9PDUM6px6ioz3jtE+2QhsQIMWsSUFwsnhjy2TKhUVLrStdFbv/Di/sVY5Xjo8twRDgJEM1wSJL0jK2jcyufZ7xDQ38Fmo0Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562715514; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ljJMc2SmI38XU0xm7lJtXQVTOQF8C7npyx5TzfW+Cac=; b=ZdSfYJX3k4pBVPsPHIRlTjpPzVx3XKmzzynILMrVD2ketVj36Ohokhv1GoZ0Rat+HR/YNedtOQ7OPANJj0hXTnZUBjKLKggkdAgmf8m1KoL2tkRZFxlVPfEf9JVz6o96PROIH9nBrifeAZGGa5QIlV6iiEiakXRmLKKIf9YpjXM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562715514512375.96436842100957; Tue, 9 Jul 2019 16:38:34 -0700 (PDT) Received: from localhost ([::1]:56852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzgm-0005oX-Ir for importer@patchew.org; Tue, 09 Jul 2019 19:38:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48779) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVG-0007GI-6G for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVD-00033F-RH for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43974) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV9-0002da-B8; Tue, 09 Jul 2019 19:26:23 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 13B2E8831C; Tue, 9 Jul 2019 23:26:06 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE5465B810; Tue, 9 Jul 2019 23:26:04 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:38 -0400 Message-Id: <20190709232550.10724-7-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 09 Jul 2019 23:26:06 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 06/18] block/backup: add 'never' policy to bitmap sync mode X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" This adds a "never" policy for bitmap synchronization. Regardless of if the job succeeds or fails, we never update the bitmap. This can be used to perform differential backups, or simply to avoid the job modifying a bitmap. Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/backup.c | 7 +++++-- qapi/block-core.json | 5 ++++- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/block/backup.c b/block/backup.c index 996941fa61..efd0dcd2e7 100644 --- a/block/backup.c +++ b/block/backup.c @@ -265,8 +265,11 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob = *job, int ret) BdrvDirtyBitmap *bm; BlockDriverState *bs =3D blk_bs(job->common.blk); =20 - if (ret < 0) { - /* Merge the successor back into the parent, delete nothing. */ + if (ret < 0 || job->bitmap_mode =3D=3D BITMAP_SYNC_MODE_NEVER) { + /* + * Failure, or we don't want to synchronize the bitmap. + * Merge the successor back into the parent, delete nothing. + */ bm =3D bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL); assert(bm); } else { diff --git a/qapi/block-core.json b/qapi/block-core.json index 99dcd5f099..b1aaaaa98e 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1146,10 +1146,13 @@ # @on-success: The bitmap is only synced when the operation is successful. # This is the behavior always used for 'INCREMENTAL' backups. # +# @never: The bitmap is never synchronized with the operation, and is +# treated solely as a read-only manifest of blocks to copy. +# # Since: 4.2 ## { 'enum': 'BitmapSyncMode', - 'data': ['on-success'] } + 'data': ['on-success', 'never'] } =20 ## # @MirrorCopyMode: --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562714975; cv=none; d=zoho.com; s=zohoarc; b=CJAzP3TcCgAKznes6cJbSEF3YHqU4wqxxpxPDssh6g1weXnPcA9e3wDwnfqOaPXD7gMguhJqzhEJn6rqMjkyW4/rNA0a8EsSeCJJajMRWASQwwGp8OBdlV4LgRtkBf/nA97KM+SJo1Fey58Vam73VHrJmxcvZf03dFRbInsseQA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562714975; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=TFWmJPHleKcOV1n/SLm0w/mLyRD2wEEladQ9Gpe+H4I=; b=bnptjMO+spMw32tHb7y1tnqYMGweJdbtfM72BFi6+khRiCBXtcMM8zJNN1urFEvu8AQ4WAGE9hVzaajEHRuCKZBYtdHfrXY8VFvXuKpJHD1IvyHm+GRSXtDRFsDsKfpLVHXMfxKMY3E9vE0TfWQcoKRYtmsWWaAgh+hw79nIl54= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 156271497536836.32687081683491; Tue, 9 Jul 2019 16:29:35 -0700 (PDT) Received: from localhost ([::1]:56796 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzY8-0000kR-T4 for importer@patchew.org; Tue, 09 Jul 2019 19:29:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48507) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzV8-0007CS-Vb for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzV6-0002sP-RC for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:22 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52480) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzUy-0002eg-Ae; Tue, 09 Jul 2019 19:26:14 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8B900307E040; Tue, 9 Jul 2019 23:26:07 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 33D035B80B; Tue, 9 Jul 2019 23:26:06 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:39 -0400 Message-Id: <20190709232550.10724-8-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Tue, 09 Jul 2019 23:26:07 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 07/18] hbitmap: Fix merge when b is empty, and result is not an alias of a X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Nobody calls the function like this currently, but we neither prohibit or cope with this behavior. I decided to make the function cope with it. Signed-off-by: John Snow Reviewed-by: Max Reitz --- util/hbitmap.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/util/hbitmap.c b/util/hbitmap.c index 7905212a8b..3b6acae42b 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -781,8 +781,9 @@ bool hbitmap_can_merge(const HBitmap *a, const HBitmap = *b) } =20 /** - * Given HBitmaps A and B, let A :=3D A (BITOR) B. - * Bitmap B will not be modified. + * Given HBitmaps A and B, let R :=3D A (BITOR) B. + * Bitmaps A and B will not be modified, + * except when bitmap R is an alias of A or B. * * @return true if the merge was successful, * false if it was not attempted. @@ -797,7 +798,13 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b,= HBitmap *result) } assert(hbitmap_can_merge(b, result)); =20 - if (hbitmap_count(b) =3D=3D 0) { + if ((!hbitmap_count(a) && result =3D=3D b) || + (!hbitmap_count(b) && result =3D=3D a)) { + return true; + } + + if (!hbitmap_count(a) && !hbitmap_count(b)) { + hbitmap_reset_all(result); return true; } =20 --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562715058; cv=none; d=zoho.com; s=zohoarc; b=dQgsNHqb6zBSK8gg+dY1I+KDerg3QeHLb+YNLKp/KunLplbyAJe3+bZVmubIQ/1DAnHg8xlbkZojXL0WvxTMBvb3/52cQyeZa58m0gEZQR5ApF9ohdPeK+ez9IADm9VSQpy6ZWZ320N864DYJaYQY10Tl86d4t6yphfHZWQPQ9s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562715058; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=aK0QZEXEAUsEalt2U9VgRMO+PCEwb1K+u8537UUt+3M=; b=FLeuv1MoN5HtFLFHcOQVzMsm4LA/thERqPvF/uozTvw+4qSZVQQfC1B0CFofGG2TYdOFEktZdzWW3fXljHe+OXVkbjhZzoI+A6AI9qGz+LATXreQhcrPyfIWT+yrr4YAws3yK48c1LCYLtxqzZPJuWWFX3tezgbBGFtAqinXXdw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562715058701417.1363794043376; Tue, 9 Jul 2019 16:30:58 -0700 (PDT) Received: from localhost ([::1]:56800 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzZS-0001Qk-A8 for importer@patchew.org; Tue, 09 Jul 2019 19:30:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48680) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVD-0007Fh-Q7 for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVB-0002zu-KY for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:27 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35004) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV6-0002fz-1E; Tue, 09 Jul 2019 19:26:20 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 17DE68666A; Tue, 9 Jul 2019 23:26:09 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id AF0845B80B; Tue, 9 Jul 2019 23:26:07 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:40 -0400 Message-Id: <20190709232550.10724-9-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 09 Jul 2019 23:26:09 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 08/18] hbitmap: enable merging across granularities X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Signed-off-by: John Snow Reviewed-by: Max Reitz --- util/hbitmap.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/util/hbitmap.c b/util/hbitmap.c index 3b6acae42b..306bc4876d 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -777,7 +777,27 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size) =20 bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b) { - return (a->size =3D=3D b->size) && (a->granularity =3D=3D b->granulari= ty); + return (a->orig_size =3D=3D b->orig_size); +} + +/** + * hbitmap_sparse_merge: performs dst =3D dst | src + * works with differing granularities. + * best used when src is sparsely populated. + */ +static void hbitmap_sparse_merge(HBitmap *dst, const HBitmap *src) +{ + uint64_t offset =3D 0; + uint64_t count =3D src->orig_size; + + while (hbitmap_next_dirty_area(src, &offset, &count)) { + hbitmap_set(dst, offset, count); + offset +=3D count; + if (offset >=3D src->orig_size) { + break; + } + count =3D src->orig_size - offset; + } } =20 /** @@ -808,10 +828,24 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b= , HBitmap *result) return true; } =20 + if (a->granularity !=3D b->granularity) { + if ((a !=3D result) && (b !=3D result)) { + hbitmap_reset_all(result); + } + if (a !=3D result) { + hbitmap_sparse_merge(result, a); + } + if (b !=3D result) { + hbitmap_sparse_merge(result, b); + } + return true; + } + /* This merge is O(size), as BITS_PER_LONG and HBITMAP_LEVELS are cons= tant. * It may be possible to improve running times for sparsely populated = maps * by using hbitmap_iter_next, but this is suboptimal for dense maps. */ + assert(a->size =3D=3D b->size); for (i =3D HBITMAP_LEVELS - 1; i >=3D 0; i--) { for (j =3D 0; j < a->sizes[i]; j++) { result->levels[i][j] =3D a->levels[i][j] | b->levels[i][j]; --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562716155; cv=none; d=zoho.com; s=zohoarc; b=cPksCbTfwlHm535oDj0OCktszLWlsHMXrGKDMeha9KTGNh7GPqRelTufzeyPpLZDNrFmHkdlvjQNV3M76Kl24y6uRjRjzDe/ttbdne0uI8meScUas35zKKN7v96E0j35LqWDZxrCNxOzrK+558RHKe7ZT5Fu6kl40K0dY8e9hSk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562716155; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=+8EUmEVLRH/74ceQjm9Eio4Pcq3uhAd8WUjSh1yIIlI=; b=SXenGPJb4jb/RWO/v77gRv+Ua6CyunwJKJ7PToavnecnE0mw/9mEc6WpCEdqJuBbBfXWGvoJW1tBolH7+FTqpaHgFzve0nlpNQ8IZNfQOZRtjWX7Yn4Zz2YxfJ5y7co8qfqY3YYlwMgw2www9/nTea7HmCumszNqowcu62ieRj4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562716155583972.6409378225702; Tue, 9 Jul 2019 16:49:15 -0700 (PDT) Received: from localhost ([::1]:56890 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzrC-0001lA-I6 for importer@patchew.org; Tue, 09 Jul 2019 19:49:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49118) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVX-0007OZ-Eo for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVL-0003IO-CU for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37298) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzVD-0002mC-Sh; Tue, 09 Jul 2019 19:26:29 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3632FC05678B; Tue, 9 Jul 2019 23:26:15 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 369BE5B0BD; Tue, 9 Jul 2019 23:26:09 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:41 -0400 Message-Id: <20190709232550.10724-10-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 09 Jul 2019 23:26:15 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 09/18] block/dirty-bitmap: add bdrv_dirty_bitmap_merge_internal X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" I'm surprised it didn't come up sooner, but sometimes we have a +busy bitmap as a source. This is dangerous from the QMP API, but if we are the owner that marked the bitmap busy, it's safe to merge it using it as a read only source. It is not safe in the general case to allow users to read from in-use bitmaps, so create an internal variant that foregoes the safety checking. Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 54 +++++++++++++++++++++++++++++++++++---- include/block/block_int.h | 3 +++ 2 files changed, 52 insertions(+), 5 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 95a9c2a5d8..7881fea684 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -810,6 +810,12 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap= *bitmap, return hbitmap_next_dirty_area(bitmap->bitmap, offset, bytes); } =20 +/** + * bdrv_merge_dirty_bitmap: merge src into dest. + * Ensures permissions on bitmaps are reasonable; use for public API. + * + * @backup: If provided, make a copy of dest here prior to merge. + */ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap = *src, HBitmap **backup, Error **errp) { @@ -833,6 +839,42 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, co= nst BdrvDirtyBitmap *src, goto out; } =20 + ret =3D bdrv_dirty_bitmap_merge_internal(dest, src, backup, false); + assert(ret); + +out: + qemu_mutex_unlock(dest->mutex); + if (src->mutex !=3D dest->mutex) { + qemu_mutex_unlock(src->mutex); + } +} + +/** + * bdrv_dirty_bitmap_merge_internal: merge src into dest. + * Does NOT check bitmap permissions; not suitable for use as public API. + * + * @backup: If provided, make a copy of dest here prior to merge. + * @lock: If true, lock and unlock bitmaps on the way in/out. + * returns true if the merge succeeded; false if unattempted. + */ +bool bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest, + const BdrvDirtyBitmap *src, + HBitmap **backup, + bool lock) +{ + bool ret; + + assert(!bdrv_dirty_bitmap_readonly(dest)); + assert(!bdrv_dirty_bitmap_inconsistent(dest)); + assert(!bdrv_dirty_bitmap_inconsistent(src)); + + if (lock) { + qemu_mutex_lock(dest->mutex); + if (src->mutex !=3D dest->mutex) { + qemu_mutex_lock(src->mutex); + } + } + if (backup) { *backup =3D dest->bitmap; dest->bitmap =3D hbitmap_alloc(dest->size, hbitmap_granularity(*ba= ckup)); @@ -840,11 +882,13 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, c= onst BdrvDirtyBitmap *src, } else { ret =3D hbitmap_merge(dest->bitmap, src->bitmap, dest->bitmap); } - assert(ret); =20 -out: - qemu_mutex_unlock(dest->mutex); - if (src->mutex !=3D dest->mutex) { - qemu_mutex_unlock(src->mutex); + if (lock) { + qemu_mutex_unlock(dest->mutex); + if (src->mutex !=3D dest->mutex) { + qemu_mutex_unlock(src->mutex); + } } + + return ret; } diff --git a/include/block/block_int.h b/include/block/block_int.h index e1f2aa627e..83ffdf4950 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -1238,6 +1238,9 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t off= set, int64_t bytes); =20 void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out); void bdrv_restore_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap *backup); +bool bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest, + const BdrvDirtyBitmap *src, + HBitmap **backup, bool lock); =20 void bdrv_inc_in_flight(BlockDriverState *bs); void bdrv_dec_in_flight(BlockDriverState *bs); --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562715511; cv=none; d=zoho.com; s=zohoarc; b=XsBI/7uEMhK9Eb6bZ52Vp6c+M3rv30rNIMREhBhDrbv3cMrlIJyhU9RuKFdbc+rBvdvv+p8kHacacsMkWKm/QBU1/oYRX7vbXmlaoMrYt9an02Eo0BqEZXIf5IjLzbHFwZw/m1c3Sl1Ji3iCCAmeNugiTaSLaB19nr0nTLp7EBE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562715511; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=vjtWZQvbDAsaWwRli1c0ue24IfqP2NBTKj/b0blkeiA=; b=QFFYg/qFqc32TV7ArpW7HSz/0zsRrY7ki7bxDuafSNZN3eazOWYp/Me5UIMyqWI6BgOAD3GTw+HJlTpw0p+HSknbWfW+MTFm1AplBWi8FiVmif7gUj+aOCKTaKICB94RO26jSNFlyw7J0sltlcSV4Z738abQv/7erbtoqeE3CcU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562715511364839.3578142315585; Tue, 9 Jul 2019 16:38:31 -0700 (PDT) Received: from localhost ([::1]:56854 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzgp-0005qo-EK for importer@patchew.org; Tue, 09 Jul 2019 19:38:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48873) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVI-0007JX-5K for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVG-00036g-3w for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42796) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV8-0002oP-Vf; Tue, 09 Jul 2019 19:26:23 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AFACF3082E23; Tue, 9 Jul 2019 23:26:16 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 589D55B80B; Tue, 9 Jul 2019 23:26:15 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:42 -0400 Message-Id: <20190709232550.10724-11-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 09 Jul 2019 23:26:16 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 10/18] block/dirty-bitmap: add bdrv_dirty_bitmap_get X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Add a public interface for get. While we're at it, rename "bdrv_get_dirty_bitmap_locked" to "bdrv_dirty_bitmap_get_locked". (There are more functions to rename to the bdrv_dirty_bitmap_VERB form, but they will wait until the conclusion of this series.) Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 19 ++++++++++++------- block/mirror.c | 2 +- include/block/dirty-bitmap.h | 4 ++-- migration/block.c | 5 ++--- nbd/server.c | 2 +- 5 files changed, 18 insertions(+), 14 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 7881fea684..75a5daf116 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -509,14 +509,19 @@ BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDri= verState *bs) } =20 /* Called within bdrv_dirty_bitmap_lock..unlock */ -bool bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, - int64_t offset) +bool bdrv_dirty_bitmap_get_locked(BdrvDirtyBitmap *bitmap, int64_t offset) { - if (bitmap) { - return hbitmap_get(bitmap->bitmap, offset); - } else { - return false; - } + return hbitmap_get(bitmap->bitmap, offset); +} + +bool bdrv_dirty_bitmap_get(BdrvDirtyBitmap *bitmap, int64_t offset) +{ + bool ret; + bdrv_dirty_bitmap_lock(bitmap); + ret =3D bdrv_dirty_bitmap_get_locked(bitmap, offset); + bdrv_dirty_bitmap_unlock(bitmap); + + return ret; } =20 /** diff --git a/block/mirror.c b/block/mirror.c index 75c8f38c6a..63c3ead094 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -476,7 +476,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlo= ckJob *s) int64_t next_offset =3D offset + nb_chunks * s->granularity; int64_t next_chunk =3D next_offset / s->granularity; if (next_offset >=3D s->bdev_length || - !bdrv_get_dirty_locked(source, s->dirty_bitmap, next_offset)) { + !bdrv_dirty_bitmap_get_locked(s->dirty_bitmap, next_offset)) { break; } if (test_bit(next_chunk, s->in_flight_bitmap)) { diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 62682eb865..0120ef3f05 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -84,12 +84,12 @@ void bdrv_dirty_bitmap_set_busy(BdrvDirtyBitmap *bitmap= , bool busy); void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap = *src, HBitmap **backup, Error **errp); void bdrv_dirty_bitmap_set_migration(BdrvDirtyBitmap *bitmap, bool migrati= on); +bool bdrv_dirty_bitmap_get(BdrvDirtyBitmap *bitmap, int64_t offset); =20 /* Functions that require manual locking. */ void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_unlock(BdrvDirtyBitmap *bitmap); -bool bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, - int64_t offset); +bool bdrv_dirty_bitmap_get_locked(BdrvDirtyBitmap *bitmap, int64_t offset); void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t offset, int64_t bytes); void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, diff --git a/migration/block.c b/migration/block.c index 91f98ef44a..a5b60456ae 100644 --- a/migration/block.c +++ b/migration/block.c @@ -520,7 +520,6 @@ static int mig_save_device_dirty(QEMUFile *f, BlkMigDev= State *bmds, int is_async) { BlkMigBlock *blk; - BlockDriverState *bs =3D blk_bs(bmds->blk); int64_t total_sectors =3D bmds->total_sectors; int64_t sector; int nr_sectors; @@ -535,8 +534,8 @@ static int mig_save_device_dirty(QEMUFile *f, BlkMigDev= State *bmds, blk_mig_unlock(); } bdrv_dirty_bitmap_lock(bmds->dirty_bitmap); - if (bdrv_get_dirty_locked(bs, bmds->dirty_bitmap, - sector * BDRV_SECTOR_SIZE)) { + if (bdrv_dirty_bitmap_get_locked(bmds->dirty_bitmap, + sector * BDRV_SECTOR_SIZE)) { if (total_sectors - sector < BDRV_SECTORS_PER_DIRTY_CHUNK) { nr_sectors =3D total_sectors - sector; } else { diff --git a/nbd/server.c b/nbd/server.c index 10faedcfc5..fbd51b48a7 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -2003,7 +2003,7 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap= *bitmap, uint64_t offset, bdrv_dirty_bitmap_lock(bitmap); =20 it =3D bdrv_dirty_iter_new(bitmap); - dirty =3D bdrv_get_dirty_locked(NULL, bitmap, offset); + dirty =3D bdrv_dirty_bitmap_get_locked(bitmap, offset); =20 assert(begin < overall_end && nb_extents); while (begin < overall_end && i < nb_extents) { --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562717295; cv=none; d=zoho.com; s=zohoarc; b=F4HI/kOt9DSYDhnp8Q9xE3qUCWjeMvDs/BTId0AyVaiqhDajCYApL/H1hAJpV9HvA6ye2GdwJG8mhuSArhU04buo0wqzArp/5a2fqIwnwd+GL58HM3QIt3a/2AZAdCwU0lDAYS7T2H5+3q7B9NWZ7wf7QyE2pj7aQafCRyz/FYo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562717295; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=zVkTy8sqJD0ELFLC/8VDbl2RZBi7SXqVafCesBcuARk=; b=cgh5GMp+WBcmsG45/Za/IINlDWgW6c55woaoghk989YJ4zQOHtmFWQLMdBFCIxqC4bbQiDkabZuok+4zArnh6YVUZ891mlueFbbLEkGgE8wT9PaAfTphmdpH+FM0PGbO4tGms2bN9DO3dwZgGh3w0/neYc4fKJQiU3u8XcS9MRI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562717295452227.21433462611344; Tue, 9 Jul 2019 17:08:15 -0700 (PDT) Received: from localhost ([::1]:57026 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hl09c-0005Wt-8I for importer@patchew.org; Tue, 09 Jul 2019 20:08:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48856) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVH-0007Iu-Vb for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVD-000334-Pz for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42808) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV9-0002qJ-Au; Tue, 09 Jul 2019 19:26:23 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 386563082E51; Tue, 9 Jul 2019 23:26:18 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id D539D5B80B; Tue, 9 Jul 2019 23:26:16 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:43 -0400 Message-Id: <20190709232550.10724-12-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 09 Jul 2019 23:26:18 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 11/18] block/backup: upgrade copy_bitmap to BdrvDirtyBitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" This simplifies some interface matters; namely the initialization and (later) merging the manifest back into the sync_bitmap if it was provided. Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/backup.c | 81 ++++++++++++++++++++++++++------------------------ 1 file changed, 42 insertions(+), 39 deletions(-) diff --git a/block/backup.c b/block/backup.c index efd0dcd2e7..fb1b39c44e 100644 --- a/block/backup.c +++ b/block/backup.c @@ -38,7 +38,10 @@ typedef struct CowRequest { typedef struct BackupBlockJob { BlockJob common; BlockBackend *target; + BdrvDirtyBitmap *sync_bitmap; + BdrvDirtyBitmap *copy_bitmap; + MirrorSyncMode sync_mode; BitmapSyncMode bitmap_mode; BlockdevOnError on_source_error; @@ -51,7 +54,6 @@ typedef struct BackupBlockJob { NotifierWithReturn before_write; QLIST_HEAD(, CowRequest) inflight_reqs; =20 - HBitmap *copy_bitmap; bool use_copy_range; int64_t copy_range_size; =20 @@ -113,7 +115,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(B= ackupBlockJob *job, int write_flags =3D job->serialize_target_writes ? BDRV_REQ_SERIALISIN= G : 0; =20 assert(QEMU_IS_ALIGNED(start, job->cluster_size)); - hbitmap_reset(job->copy_bitmap, start, job->cluster_size); + bdrv_reset_dirty_bitmap(job->copy_bitmap, start, job->cluster_size); nbytes =3D MIN(job->cluster_size, job->len - start); if (!*bounce_buffer) { *bounce_buffer =3D blk_blockalign(blk, job->cluster_size); @@ -146,7 +148,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(B= ackupBlockJob *job, =20 return nbytes; fail: - hbitmap_set(job->copy_bitmap, start, job->cluster_size); + bdrv_set_dirty_bitmap(job->copy_bitmap, start, job->cluster_size); return ret; =20 } @@ -169,12 +171,14 @@ static int coroutine_fn backup_cow_with_offload(Backu= pBlockJob *job, assert(QEMU_IS_ALIGNED(start, job->cluster_size)); nbytes =3D MIN(job->copy_range_size, end - start); nr_clusters =3D DIV_ROUND_UP(nbytes, job->cluster_size); - hbitmap_reset(job->copy_bitmap, start, job->cluster_size * nr_clusters= ); + bdrv_reset_dirty_bitmap(job->copy_bitmap, start, + job->cluster_size * nr_clusters); ret =3D blk_co_copy_range(blk, start, job->target, start, nbytes, read_flags, write_flags); if (ret < 0) { trace_backup_do_cow_copy_range_fail(job, start, ret); - hbitmap_set(job->copy_bitmap, start, job->cluster_size * nr_cluste= rs); + bdrv_set_dirty_bitmap(job->copy_bitmap, start, + job->cluster_size * nr_clusters); return ret; } =20 @@ -202,7 +206,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *j= ob, cow_request_begin(&cow_request, job, start, end); =20 while (start < end) { - if (!hbitmap_get(job->copy_bitmap, start)) { + if (!bdrv_dirty_bitmap_get(job->copy_bitmap, start)) { trace_backup_do_cow_skip(job, start); start +=3D job->cluster_size; continue; /* already copied */ @@ -298,14 +302,16 @@ static void backup_abort(Job *job) static void backup_clean(Job *job) { BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); + BlockDriverState *bs =3D blk_bs(s->common.blk); + + if (s->copy_bitmap) { + bdrv_release_dirty_bitmap(bs, s->copy_bitmap); + s->copy_bitmap =3D NULL; + } + assert(s->target); blk_unref(s->target); s->target =3D NULL; - - if (s->copy_bitmap) { - hbitmap_free(s->copy_bitmap); - s->copy_bitmap =3D NULL; - } } =20 void backup_do_checkpoint(BlockJob *job, Error **errp) @@ -320,7 +326,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp) return; } =20 - hbitmap_set(backup_job->copy_bitmap, 0, backup_job->len); + bdrv_set_dirty_bitmap(backup_job->copy_bitmap, 0, backup_job->len); } =20 static void backup_drain(BlockJob *job) @@ -389,59 +395,52 @@ static bool bdrv_is_unallocated_range(BlockDriverStat= e *bs, =20 static int coroutine_fn backup_loop(BackupBlockJob *job) { - int ret; bool error_is_read; int64_t offset; - HBitmapIter hbi; + BdrvDirtyBitmapIter *bdbi; BlockDriverState *bs =3D blk_bs(job->common.blk); + int ret =3D 0; =20 - hbitmap_iter_init(&hbi, job->copy_bitmap, 0); - while ((offset =3D hbitmap_iter_next(&hbi)) !=3D -1) { + bdbi =3D bdrv_dirty_iter_new(job->copy_bitmap); + while ((offset =3D bdrv_dirty_iter_next(bdbi)) !=3D -1) { if (job->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP && bdrv_is_unallocated_range(bs, offset, job->cluster_size)) { - hbitmap_reset(job->copy_bitmap, offset, job->cluster_size); + bdrv_reset_dirty_bitmap(job->copy_bitmap, offset, + job->cluster_size); continue; } =20 do { if (yield_and_check(job)) { - return 0; + goto out; } ret =3D backup_do_cow(job, offset, job->cluster_size, &error_is_read, false); if (ret < 0 && backup_error_action(job, error_is_read, -ret) = =3D=3D BLOCK_ERROR_ACTION_REPORT) { - return ret; + goto out; } } while (ret < 0); } =20 - return 0; + out: + bdrv_dirty_iter_free(bdbi); + return ret; } =20 /* init copy_bitmap from sync_bitmap */ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) { - uint64_t offset =3D 0; - uint64_t bytes =3D job->len; - - while (bdrv_dirty_bitmap_next_dirty_area(job->sync_bitmap, - &offset, &bytes)) - { - hbitmap_set(job->copy_bitmap, offset, bytes); - - offset +=3D bytes; - if (offset >=3D job->len) { - break; - } - bytes =3D job->len - offset; - } + bool ret =3D bdrv_dirty_bitmap_merge_internal(job->copy_bitmap, + job->sync_bitmap, + NULL, true); + assert(ret); =20 /* TODO job_progress_set_remaining() would make more sense */ job_progress_update(&job->common.job, - job->len - hbitmap_count(job->copy_bitmap)); + job->len - bdrv_get_dirty_count(job->copy_bitmap)); } =20 static int coroutine_fn backup_run(Job *job, Error **errp) @@ -458,7 +457,7 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_BITMAP) { backup_incremental_init_copy_bitmap(s); } else { - hbitmap_set(s->copy_bitmap, 0, s->len); + bdrv_set_dirty_bitmap(s->copy_bitmap, 0, s->len); } =20 s->before_write.notify =3D backup_before_write_notify; @@ -551,7 +550,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, BackupBlockJob *job =3D NULL; int ret; int64_t cluster_size; - HBitmap *copy_bitmap =3D NULL; + BdrvDirtyBitmap *copy_bitmap =3D NULL; =20 assert(bs); assert(target); @@ -621,7 +620,11 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, goto error; } =20 - copy_bitmap =3D hbitmap_alloc(len, ctz32(cluster_size)); + copy_bitmap =3D bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp); + if (!copy_bitmap) { + goto error; + } + bdrv_disable_dirty_bitmap(copy_bitmap); =20 /* job->len is fixed, so we can't allow resize */ job =3D block_job_create(job_id, &backup_job_driver, txn, bs, @@ -672,7 +675,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, error: if (copy_bitmap) { assert(!job || !job->copy_bitmap); - hbitmap_free(copy_bitmap); + bdrv_release_dirty_bitmap(bs, copy_bitmap); } if (sync_bitmap) { bdrv_reclaim_dirty_bitmap(bs, sync_bitmap, NULL); --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562716768; cv=none; d=zoho.com; s=zohoarc; b=FYTaSv1noW41/g7zglLVZkiX+55MhxNysYk4SvIPpONnZ9ec+/vYdZ6J0kMaZuBSB7LA/61pKKWJ6NgriaIBcCY8u1bVwkjkUDL8UxieUyQE5GdcZ8hMlCp1y9ZFYloeJ0z2aOn4MiDvnOH4gGeICOVurOx295jSeZ7BzZgQnz4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562716768; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=4LizeWCix8VmjiuwVoT+eCGuabvJZWuSN23ozAnkBR4=; b=RNRX2VwzD+JfX1S2414rmXRmbsbV4W/12Jw6IJEepNvw2zLJVuGeocIhyI3/2xpqzH5GaRHaBV8dbCouht4lGC9A42ApeI3zFStZAub1lzulfCk9O6SFaY9BULVB7CDT9A6T5J1s6la2GrDhxYKC4M1VkNu+GhOagIDuY+xiqV4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15627167688351023.8859043377447; Tue, 9 Jul 2019 16:59:28 -0700 (PDT) Received: from localhost ([::1]:56962 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hl019-0000lo-FK for importer@patchew.org; Tue, 09 Jul 2019 19:59:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49124) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVX-0007Oa-FM for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVL-0003IA-C2 for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51114) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzVB-0002rn-K3; Tue, 09 Jul 2019 19:26:27 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B36F181DFC; Tue, 9 Jul 2019 23:26:19 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5C0645B80B; Tue, 9 Jul 2019 23:26:18 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:44 -0400 Message-Id: <20190709232550.10724-13-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 09 Jul 2019 23:26:19 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 12/18] block/backup: add 'always' bitmap sync policy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" This adds an "always" policy for bitmap synchronization. Regardless of if the job succeeds or fails, the bitmap is *always* synchronized. This means that for backups that fail part-way through, the bitmap retains a record of which sectors need to be copied out to accomplish a new backup using the old, partial result. In effect, this allows us to "resume" a failed backup; however the new back= up will be from the new point in time, so it isn't a "resume" as much as it is an "incremental retry." This can be useful in the case of extremely large backups that fail considerably through the operation and we'd like to not w= aste the work that was already performed. Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/backup.c | 27 +++++++++++++++++++-------- qapi/block-core.json | 5 ++++- 2 files changed, 23 insertions(+), 9 deletions(-) diff --git a/block/backup.c b/block/backup.c index fb1b39c44e..acfe87b756 100644 --- a/block/backup.c +++ b/block/backup.c @@ -268,18 +268,29 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob= *job, int ret) { BdrvDirtyBitmap *bm; BlockDriverState *bs =3D blk_bs(job->common.blk); + bool sync =3D (((ret =3D=3D 0) || (job->bitmap_mode =3D=3D BITMAP_SYNC= _MODE_ALWAYS)) \ + && (job->bitmap_mode !=3D BITMAP_SYNC_MODE_NEVER)); =20 - if (ret < 0 || job->bitmap_mode =3D=3D BITMAP_SYNC_MODE_NEVER) { + if (sync) { /* - * Failure, or we don't want to synchronize the bitmap. - * Merge the successor back into the parent, delete nothing. + * We succeeded, or we always intended to sync the bitmap. + * Delete this bitmap and install the child. */ - bm =3D bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL); - assert(bm); - } else { - /* Everything is fine, delete this bitmap and install the backup. = */ bm =3D bdrv_dirty_bitmap_abdicate(bs, job->sync_bitmap, NULL); - assert(bm); + } else { + /* + * We failed, or we never intended to sync the bitmap anyway. + * Merge the successor back into the parent, keeping all data. + */ + bm =3D bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL); + } + + assert(bm); + + if (ret < 0 && job->bitmap_mode =3D=3D BITMAP_SYNC_MODE_ALWAYS) { + /* If we failed and synced, merge in the bits we didn't copy: */ + bdrv_dirty_bitmap_merge_internal(bm, job->copy_bitmap, + NULL, true); } } =20 diff --git a/qapi/block-core.json b/qapi/block-core.json index b1aaaaa98e..5a578806c5 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1149,10 +1149,13 @@ # @never: The bitmap is never synchronized with the operation, and is # treated solely as a read-only manifest of blocks to copy. # +# @always: The bitmap is always synchronized with the operation, +# regardless of whether or not the operation was successful. +# # Since: 4.2 ## { 'enum': 'BitmapSyncMode', - 'data': ['on-success', 'never'] } + 'data': ['on-success', 'never', 'always'] } =20 ## # @MirrorCopyMode: --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562715515; cv=none; d=zoho.com; s=zohoarc; b=LYIzcDcRE4Ti/Lnl3mU6i2TtSfBN/FCXVgg6QAkGRlz7rZ0uI6NbF9Mxfx/c9dMfeEYn7pDZFHeW2hNvBihIuTmRgXq3JSF/ZUDapoRiyMbq1f4Fh5Qo1fldwllWRMD7KKe6UcH0XDBiUNOzbHCE92U8wrCcBhi0gYjsDyDtFjY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562715515; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=wNbr5zYc5ghH1NfhXnjbVrdjaHXVRVUjQ/kEi7EOMTU=; b=e66CRhnPgsze5uiOiNTiVBNNMJhsCPJI7qvVINEsJ5eLqzT59vwc0hDd/49T38hBGAI4u87AKyyFe/SIMCZ+txbKzJMqF22hQDisEG2Wglx+2DPWoV3KVR/Gm/ZDML2BjrZOSK9JjRMl2V+ucKr4i2r5/BeBxoTnxU2vtzK1Lwk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15627155158831001.5327591756949; Tue, 9 Jul 2019 16:38:35 -0700 (PDT) Received: from localhost ([::1]:56856 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzgq-0005v3-Nv for importer@patchew.org; Tue, 09 Jul 2019 19:38:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49029) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVL-0007MY-Dk for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVI-0003BA-4U for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:35 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48932) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV9-0002tN-F3; Tue, 09 Jul 2019 19:26:23 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3B2A6C0005DA; Tue, 9 Jul 2019 23:26:21 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id D5F825B80B; Tue, 9 Jul 2019 23:26:19 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:45 -0400 Message-Id: <20190709232550.10724-14-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 09 Jul 2019 23:26:21 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 13/18] iotests: add testing shim for script-style python tests X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Because the new-style python tests don't use the iotests.main() test launcher, we don't turn on the debugger logging for these scripts when invoked via ./check -d. Refactor the launcher shim into new and old style shims so that they share environmental configuration. Two cleanup notes: debug was not actually used as a global, and there was no reason to create a class in an inner scope just to achieve default variables; we can simply create an instance of the runner with the values we want instead. Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 40 +++++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 3ecef5bc90..fcad957d63 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -61,7 +61,6 @@ cachemode =3D os.environ.get('CACHEMODE') qemu_default_machine =3D os.environ.get('QEMU_DEFAULT_MACHINE') =20 socket_scm_helper =3D os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_help= er') -debug =3D False =20 luks_default_secret_object =3D 'secret,id=3Dkeysec0,data=3D' + \ os.environ.get('IMGKEYSECRET', '') @@ -834,11 +833,22 @@ def skip_if_unsupported(required_formats=3D[], read_o= nly=3DFalse): return func_wrapper return skip_test_decorator =20 -def main(supported_fmts=3D[], supported_oses=3D['linux'], supported_cache_= modes=3D[], - unsupported_fmts=3D[]): - '''Run tests''' +def execute_unittest(output, verbosity, debug): + runner =3D unittest.TextTestRunner(stream=3Doutput, descriptions=3DTru= e, + verbosity=3Dverbosity) + try: + # unittest.main() will use sys.exit(); so expect a SystemExit + # exception + unittest.main(testRunner=3Drunner) + finally: + if not debug: + sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', + r'Ran \1 tests', output.getvalue())) =20 - global debug +def execute_test(test_function=3DNone, + supported_fmts=3D[], supported_oses=3D['linux'], + supported_cache_modes=3D[], unsupported_fmts=3D[]): + """Run either unittest or script-style tests.""" =20 # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to # indicate that we're not being run via "check". There may be @@ -870,13 +880,15 @@ def main(supported_fmts=3D[], supported_oses=3D['linu= x'], supported_cache_modes=3D[], =20 logging.basicConfig(level=3D(logging.DEBUG if debug else logging.WARN)) =20 - class MyTestRunner(unittest.TextTestRunner): - def __init__(self, stream=3Doutput, descriptions=3DTrue, verbosity= =3Dverbosity): - unittest.TextTestRunner.__init__(self, stream, descriptions, v= erbosity) + if not test_function: + execute_unittest(output, verbosity, debug) + else: + test_function() =20 - # unittest.main() will use sys.exit() so expect a SystemExit exception - try: - unittest.main(testRunner=3DMyTestRunner) - finally: - if not debug: - sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran = \1 tests', output.getvalue())) +def script_main(test_function, *args, **kwargs): + """Run script-style tests outside of the unittest framework""" + execute_test(test_function, *args, **kwargs) + +def main(*args, **kwargs): + """Run tests using the unittest framework""" + execute_test(None, *args, **kwargs) --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562715502; cv=none; d=zoho.com; s=zohoarc; b=bvKddtN0+iKiM736xU7kF5zzgEUeGmZna7jMNZAqPnM9oRorNXoJuGGyY1ZnPbdGItXGENZz5gBHJ62+j55OXpkhXnYQUIxW6WbrfKUziRSBqYqe1lLDV7+JhVlyoVYy/vJgxYx/nDHaTYFf+UL+EM5a0kM/chz5fMlGGkR236Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562715502; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=05l0C2qktHVzgQPAUZlF/Gtwj+btEPUR7/v2jVc5snM=; b=fyNXhd4nmK3OusD4XgBeuRbTr25EVr7jFddnm7JeN5pMseKkLqBhoxUuBQa/31c1bYG8lRe45f3F0T/C/1s6UAlmPdLHEnqkBw2UKadZ1y+HhzFByk8judZVZSNI+cXGUllJ3m64gJOkDeOhOZ5uVyBBJsS2zG58be1ng+BTVFs= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562715502468322.92571592591855; Tue, 9 Jul 2019 16:38:22 -0700 (PDT) Received: from localhost ([::1]:56850 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzgP-0005P7-Qm for importer@patchew.org; Tue, 09 Jul 2019 19:38:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49028) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVL-0007MX-D7 for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVH-0003A9-Uh for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:13378) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzV9-0002ug-Mj; Tue, 09 Jul 2019 19:26:25 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B697E3D95A; Tue, 9 Jul 2019 23:26:22 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5D6755B80B; Tue, 9 Jul 2019 23:26:21 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:46 -0400 Message-Id: <20190709232550.10724-15-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 09 Jul 2019 23:26:22 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 14/18] iotests: teach run_job to cancel pending jobs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" run_job can cancel pending jobs to simulate failure. This lets us use the pending callback to issue test commands while the job is open, but then still have the job fail in the end. Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index fcad957d63..c544659ecb 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -541,7 +541,22 @@ class VM(qtest.QEMUQtestMachine): =20 # Returns None on success, and an error string on failure def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse, - pre_finalize=3DNone, wait=3D60.0): + pre_finalize=3DNone, cancel=3DFalse, wait=3D60.0): + """ + run_job moves a job from creation through to dismissal. + + :param job: String. ID of recently-launched job + :param auto_finalize: Bool. True if the job was launched with + auto_finalize. Defaults to True. + :param auto_dismiss: Bool. True if the job was launched with + auto_dismiss=3DTrue. Defaults to False. + :param pre_finalize: Callback. A callable that takes no arguments = to be + invoked prior to issuing job-finalize, if any. + :param cancel: Bool. When true, cancels the job after the pre_fina= lize + callback. + :param wait: Float. Timeout value specifying how long to wait for = any + event, in seconds. Defaults to 60.0. + """ match_device =3D {'data': {'device': job}} match_id =3D {'data': {'id': job}} events =3D [ @@ -568,7 +583,10 @@ class VM(qtest.QEMUQtestMachine): elif status =3D=3D 'pending' and not auto_finalize: if pre_finalize: pre_finalize() - self.qmp_log('job-finalize', id=3Djob) + if cancel: + self.qmp_log('job-cancel', id=3Djob) + else: + self.qmp_log('job-finalize', id=3Djob) elif status =3D=3D 'concluded' and not auto_dismiss: self.qmp_log('job-dismiss', id=3Djob) elif status =3D=3D 'null': --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562717138; cv=none; d=zoho.com; s=zohoarc; b=odPLP1bPsJX9RFwvQWJFuKc+B2y51ZPvk3cW36eFnyxsuRcE8zgxRnjDGEEtJ7GhAW1mJ0s1DebCeqIatA2E0/Xz6T1uEdd7JYPEE7h7xedaWYAqHPWmhZwxgzZGW06T4Jp4OAGHuXxW/0bglHwMOWDNykrYyW51ZPSDLUeO6yE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562717138; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=Rj/mW9gT1wP3gt2KTgjPB/uk5sg0I+TSebelG4TfWrc=; b=TCJCG+GeN8inzytjrQ8BSd3HT0f+DBokxoFpAJkYkUdt636nCI/tqQ6UlLvAe6pzUJM/y4bJ+WaQIHNhUntgKYm2MRiRINYdmRjK1oZ/kygk2yw7OBejo/XNmy63EHcbFZoWy4K8rC1iLX+C8+PttdgCEwcorb1h0EDaz30HRz8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562717138229736.2275375118672; Tue, 9 Jul 2019 17:05:38 -0700 (PDT) Received: from localhost ([::1]:57002 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hl073-0003w9-U5 for importer@patchew.org; Tue, 09 Jul 2019 20:05:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49157) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVf-0007PL-FC for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVQ-0003M6-Lt for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42834) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzVB-0002yA-LO; Tue, 09 Jul 2019 19:26:27 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3A67D3082E51; Tue, 9 Jul 2019 23:26:24 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id D88525B80B; Tue, 9 Jul 2019 23:26:22 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:47 -0400 Message-Id: <20190709232550.10724-16-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 09 Jul 2019 23:26:24 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 15/18] iotests: teach FilePath to produce multiple paths X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Use "FilePaths" instead of "FilePath" to request multiple files be cleaned up after we leave that object's scope. This is not crucial; but it saves a little typing. Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 34 ++++++++++++++++++++++++---------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index c544659ecb..6135c9663d 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -358,31 +358,45 @@ class Timeout: def timeout(self, signum, frame): raise Exception(self.errmsg) =20 +def file_pattern(name): + return "{0}-{1}".format(os.getpid(), name) =20 -class FilePath(object): - '''An auto-generated filename that cleans itself up. +class FilePaths(object): + """ + FilePaths is an auto-generated filename that cleans itself up. =20 Use this context manager to generate filenames and ensure that the file gets deleted:: =20 - with TestFilePath('test.img') as img_path: + with FilePaths(['test.img']) as img_path: qemu_img('create', img_path, '1G') # migration_sock_path is automatically deleted - ''' - def __init__(self, name): - filename =3D '{0}-{1}'.format(os.getpid(), name) - self.path =3D os.path.join(test_dir, filename) + """ + def __init__(self, names): + self.paths =3D [] + for name in names: + self.paths.append(os.path.join(test_dir, file_pattern(name))) =20 def __enter__(self): - return self.path + return self.paths =20 def __exit__(self, exc_type, exc_val, exc_tb): try: - os.remove(self.path) + for path in self.paths: + os.remove(path) except OSError: pass return False =20 +class FilePath(FilePaths): + """ + FilePath is a specialization of FilePaths that takes a single filename. + """ + def __init__(self, name): + super(FilePath, self).__init__([name]) + + def __enter__(self): + return self.paths[0] =20 def file_path_remover(): for path in reversed(file_path_remover.paths): @@ -407,7 +421,7 @@ def file_path(*names): =20 paths =3D [] for name in names: - filename =3D '{0}-{1}'.format(os.getpid(), name) + filename =3D file_pattern(name) path =3D os.path.join(test_dir, filename) file_path_remover.paths.append(path) paths.append(path) --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562715925; cv=none; d=zoho.com; s=zohoarc; b=IGKA4N4vovXFhvoCsQhGxwjVIyAQPuArz6xG5//hBubTIqw1XGZZmF363q7w1h0DB/RFqcesfWpNnZPWNm860Wan0mxCt0LPaT88RffPJdk6vvLLYTh4W5LDK/BUVcsV4EZS6mYxyuVbxlketuW2AL/8rncXK1wWJtzTjUzPtuQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562715925; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=7mpT6Z9j82kTDLu2ad668tTqcpwVXP50qe58+xXZlpk=; b=bvPCslzWEgKg6utsnqeIW55nqPsIQcRS+fi6o3q7RhDGgnJpFDm4wGJr8GhkPfZ7FYQZIbZ6nCKfE7LRDGSdGAzmfpQDCrEfCK/HT10B/wKSvepjBo6+W9AUp4KtU9u4+/3zMBo9+zQDfoN/jjNetJJFnLkfuyttamHawG+B5CI= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562715925024970.3422376240177; Tue, 9 Jul 2019 16:45:25 -0700 (PDT) Received: from localhost ([::1]:56878 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkznN-0000zJ-Eh for importer@patchew.org; Tue, 09 Jul 2019 19:45:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49178) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVh-0007QS-Pb for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVX-0003R0-Eu for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:26:53 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37442) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzVD-00030x-OT; Tue, 09 Jul 2019 19:26:27 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B238430C31B4; Tue, 9 Jul 2019 23:26:25 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5D0265B80B; Tue, 9 Jul 2019 23:26:24 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:48 -0400 Message-Id: <20190709232550.10724-17-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Tue, 09 Jul 2019 23:26:25 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 16/18] iotests: Add virtio-scsi device helper X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Seems that it comes up enough. Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/040 | 6 +----- tests/qemu-iotests/093 | 6 ++---- tests/qemu-iotests/139 | 7 ++----- tests/qemu-iotests/238 | 5 +---- tests/qemu-iotests/iotests.py | 4 ++++ 5 files changed, 10 insertions(+), 18 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index b81133a474..657b37103c 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -85,11 +85,7 @@ class TestSingleDrive(ImageCommitTestCase): qemu_io('-f', 'raw', '-c', 'write -P 0xab 0 524288', backing_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xef 524288 524288',= mid_img) self.vm =3D iotests.VM().add_drive(test_img, "node-name=3Dtop,back= ing.node-name=3Dmid,backing.backing.node-name=3Dbase", interface=3D"none") - if iotests.qemu_default_machine =3D=3D 's390-ccw-virtio': - self.vm.add_device("virtio-scsi-ccw") - else: - self.vm.add_device("virtio-scsi-pci") - + self.vm.add_device(iotests.get_virtio_scsi_device()) self.vm.add_device("scsi-hd,id=3Dscsi0,drive=3Ddrive0") self.vm.launch() =20 diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093 index d88fbc182e..46153220f8 100755 --- a/tests/qemu-iotests/093 +++ b/tests/qemu-iotests/093 @@ -366,10 +366,8 @@ class ThrottleTestGroupNames(iotests.QMPTestCase): class ThrottleTestRemovableMedia(iotests.QMPTestCase): def setUp(self): self.vm =3D iotests.VM() - if iotests.qemu_default_machine =3D=3D 's390-ccw-virtio': - self.vm.add_device("virtio-scsi-ccw,id=3Dvirtio-scsi") - else: - self.vm.add_device("virtio-scsi-pci,id=3Dvirtio-scsi") + self.vm.add_device("{},id=3Dvirtio-scsi".format( + iotests.get_virtio_scsi_device())) self.vm.launch() =20 def tearDown(self): diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139 index 933b45121a..2176ea51ba 100755 --- a/tests/qemu-iotests/139 +++ b/tests/qemu-iotests/139 @@ -35,11 +35,8 @@ class TestBlockdevDel(iotests.QMPTestCase): def setUp(self): iotests.qemu_img('create', '-f', iotests.imgfmt, base_img, '1M') self.vm =3D iotests.VM() - if iotests.qemu_default_machine =3D=3D 's390-ccw-virtio': - self.vm.add_device("virtio-scsi-ccw,id=3Dvirtio-scsi") - else: - self.vm.add_device("virtio-scsi-pci,id=3Dvirtio-scsi") - + self.vm.add_device("{},id=3Dvirtio-scsi".format( + iotests.get_virtio_scsi_device())) self.vm.launch() =20 def tearDown(self): diff --git a/tests/qemu-iotests/238 b/tests/qemu-iotests/238 index 1c0a46fa90..387a77b2cd 100755 --- a/tests/qemu-iotests/238 +++ b/tests/qemu-iotests/238 @@ -23,10 +23,7 @@ import os import iotests from iotests import log =20 -if iotests.qemu_default_machine =3D=3D 's390-ccw-virtio': - virtio_scsi_device =3D 'virtio-scsi-ccw' -else: - virtio_scsi_device =3D 'virtio-scsi-pci' +virtio_scsi_device =3D iotests.get_virtio_scsi_device() =20 vm =3D iotests.VM() vm.launch() diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6135c9663d..8ae7bc353e 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -164,6 +164,10 @@ def qemu_io_silent(*args): (-exitcode, ' '.join(args))) return exitcode =20 +def get_virtio_scsi_device(): + if qemu_default_machine =3D=3D 's390-ccw-virtio': + return 'virtio-scsi-ccw' + return 'virtio-scsi-pci' =20 class QemuIoInteractive: def __init__(self, *args): --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562717486; cv=none; d=zoho.com; s=zohoarc; b=DBeN9GKIoDBHDBcDuG8dO8xTPVvP8kQWFqtjdgRfOT+o6bo4OyuUzO59boVRY2pDDf0Mek6KaVEdyqVjNEcnj12RmFm5AzQ7FKIFms0OnazI8+0kDUyonIAeSHipRGYieyRsSwtHG4sujH4X8MjI/76DIO4wsRNTkiDCn7QRBRE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562717486; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=uTpfmtqo3uHmjxn3N0yYAIcL0PwXSI0Zs6dbgKembi4=; b=dQax4p1vW0IbvmnrsYWFu6TN7GnFlDrsnupMlLADNU2e6paCyr0nl2+qdge8o6lQmZsNmOcalZrHPq96jEa3/IA32wCXzGhtOsEIJGW8Wbc6euzWogVim2/BuC9dAuT7WLD8so541b5op74r4oCw3sQyD6Zg4sHAJde08O0fre0= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562717486037874.2348634102249; Tue, 9 Jul 2019 17:11:26 -0700 (PDT) Received: from localhost ([::1]:57038 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hl0Cf-00072a-0E for importer@patchew.org; Tue, 09 Jul 2019 20:11:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49473) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVx-0007fw-3n for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:27:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVp-0003iJ-MW for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:27:12 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42872) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzVH-00032C-Vr; Tue, 09 Jul 2019 19:26:32 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5F2403082E23; Tue, 9 Jul 2019 23:26:27 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id D60AD5B80B; Tue, 9 Jul 2019 23:26:25 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:49 -0400 Message-Id: <20190709232550.10724-18-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 09 Jul 2019 23:26:27 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 17/18] iotests: add test 257 for bitmap-mode backups X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/257 | 416 +++++++ tests/qemu-iotests/257.out | 2247 ++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 2664 insertions(+) create mode 100755 tests/qemu-iotests/257 create mode 100644 tests/qemu-iotests/257.out diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257 new file mode 100755 index 0000000000..75a651c7c3 --- /dev/null +++ b/tests/qemu-iotests/257 @@ -0,0 +1,416 @@ +#!/usr/bin/env python +# +# Test bitmap-sync backups (incremental, differential, and partials) +# +# Copyright (c) 2019 John Snow for Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# owner=3Djsnow@redhat.com + +from collections import namedtuple +import math +import os + +import iotests +from iotests import log, qemu_img + +SIZE =3D 64 * 1024 * 1024 +GRANULARITY =3D 64 * 1024 + +Pattern =3D namedtuple('Pattern', ['byte', 'offset', 'size']) +def mkpattern(byte, offset, size=3DGRANULARITY): + """Constructor for Pattern() with default size""" + return Pattern(byte, offset, size) + +class PatternGroup: + """Grouping of Pattern objects. Initialize with an iterable of Pattern= s.""" + def __init__(self, patterns): + self.patterns =3D patterns + + def bits(self, granularity): + """Calculate the unique bits dirtied by this pattern grouping""" + res =3D set() + for pattern in self.patterns: + lower =3D math.floor(pattern.offset / granularity) + upper =3D math.floor((pattern.offset + pattern.size - 1) / gra= nularity) + res =3D res | set(range(lower, upper + 1)) + return res + +GROUPS =3D [ + PatternGroup([ + # Batch 0: 4 clusters + mkpattern('0x49', 0x0000000), + mkpattern('0x6c', 0x0100000), # 1M + mkpattern('0x6f', 0x2000000), # 32M + mkpattern('0x76', 0x3ff0000)]), # 64M - 64K + PatternGroup([ + # Batch 1: 6 clusters (3 new) + mkpattern('0x65', 0x0000000), # Full overwrite + mkpattern('0x77', 0x00f8000), # Partial-left (1M-32K) + mkpattern('0x72', 0x2008000), # Partial-right (32M+32K) + mkpattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K) + PatternGroup([ + # Batch 2: 7 clusters (3 new) + mkpattern('0x74', 0x0010000), # Adjacent-right + mkpattern('0x69', 0x00e8000), # Partial-left (1M-96K) + mkpattern('0x6e', 0x2018000), # Partial-right (32M+96K) + mkpattern('0x67', 0x3fe0000, + 2*GRANULARITY)]), # Overwrite [(64M-128K)-64M) + PatternGroup([ + # Batch 3: 8 clusters (5 new) + # Carefully chosen such that nothing re-dirties the one cluster + # that copies out successfully before failure in Group #1. + mkpattern('0xaa', 0x0010000, + 3*GRANULARITY), # Overwrite and 2x Adjacent-right + mkpattern('0xbb', 0x00d8000), # Partial-left (1M-160K) + mkpattern('0xcc', 0x2028000), # Partial-right (32M+160K) + mkpattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right +] + +class Drive: + """Represents, vaguely, a drive attached to a VM. + Includes format, graph, and device information.""" + + def __init__(self, path, vm=3DNone): + self.path =3D path + self.vm =3D vm + self.fmt =3D None + self.size =3D None + self.node =3D None + self.device =3D None + + @property + def name(self): + return self.node or self.device + + def img_create(self, fmt, size): + self.fmt =3D fmt + self.size =3D size + iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size)) + + def create_target(self, name, fmt, size): + basename =3D os.path.basename(self.path) + file_node_name =3D "file_{}".format(basename) + vm =3D self.vm + + log(vm.command('blockdev-create', job_id=3D'bdc-file-job', + options=3D{ + 'driver': 'file', + 'filename': self.path, + 'size': 0, + })) + vm.run_job('bdc-file-job') + log(vm.command('blockdev-add', driver=3D'file', + node_name=3Dfile_node_name, filename=3Dself.path)) + + log(vm.command('blockdev-create', job_id=3D'bdc-fmt-job', + options=3D{ + 'driver': fmt, + 'file': file_node_name, + 'size': size, + })) + vm.run_job('bdc-fmt-job') + log(vm.command('blockdev-add', driver=3Dfmt, + node_name=3Dname, + file=3Dfile_node_name)) + self.fmt =3D fmt + self.size =3D size + self.node =3D name + +def query_bitmaps(vm): + res =3D vm.qmp("query-block") + return {"bitmaps": {device['device'] or device['qdev']: + device.get('dirty-bitmaps', []) for + device in res['return']}} + +def get_bitmap(bitmaps, drivename, name, recording=3DNone): + """ + get a specific bitmap from the object returned by query_bitmaps. + :param recording: If specified, filter results by the specified value. + """ + for bitmap in bitmaps['bitmaps'][drivename]: + if bitmap.get('name', '') =3D=3D name: + if recording is None: + return bitmap + elif bitmap.get('recording') =3D=3D recording: + return bitmap + return None + +def reference_backup(drive, n, filepath): + log("--- Reference Backup #{:d} ---\n".format(n)) + target_id =3D "ref_target_{:d}".format(n) + job_id =3D "ref_backup_{:d}".format(n) + target_drive =3D Drive(filepath, vm=3Ddrive.vm) + + target_drive.create_target(target_id, drive.fmt, drive.size) + drive.vm.qmp_log("blockdev-backup", + job_id=3Djob_id, device=3Ddrive.name, + target=3Dtarget_id, sync=3D"full") + drive.vm.run_job(job_id, auto_dismiss=3DTrue) + log('') + +def bitmap_backup(drive, n, filepath, bitmap, bitmap_mode): + log("--- Bitmap Backup #{:d} ---\n".format(n)) + target_id =3D "bitmap_target_{:d}".format(n) + job_id =3D "bitmap_backup_{:d}".format(n) + target_drive =3D Drive(filepath, vm=3Ddrive.vm) + + target_drive.create_target(target_id, drive.fmt, drive.size) + drive.vm.qmp_log("blockdev-backup", job_id=3Djob_id, device=3Ddrive.na= me, + target=3Dtarget_id, sync=3D"bitmap", + bitmap_mode=3Dbitmap_mode, + bitmap=3Dbitmap, + auto_finalize=3DFalse) + return job_id + +def perform_writes(drive, n): + log("--- Write #{:d} ---\n".format(n)) + for pattern in GROUPS[n].patterns: + cmd =3D "write -P{:s} 0x{:07x} 0x{:x}".format( + pattern.byte, + pattern.offset, + pattern.size) + log(cmd) + log(drive.vm.hmp_qemu_io(drive.name, cmd)) + bitmaps =3D query_bitmaps(drive.vm) + log(bitmaps, indent=3D2) + log('') + return bitmaps + +def calculate_bits(groups=3DNone): + """Calculate how many bits we expect to see dirtied.""" + if groups: + bits =3D set.union(*(GROUPS[group].bits(GRANULARITY) for group in = groups)) + return len(bits) + return 0 + +def bitmap_comparison(bitmap, groups=3DNone, want=3D0): + """ + Print a nice human-readable message checking that this bitmap has as + many bits set as we expect it to. + """ + log("=3D Checking Bitmap {:s} =3D".format(bitmap.get('name', '(anonymo= us)'))) + + if groups: + want =3D calculate_bits(groups) + have =3D bitmap['count'] // bitmap['granularity'] + + log("expecting {:d} dirty sectors; have {:d}. {:s}".format( + want, have, "OK!" if want =3D=3D have else "ERROR!")) + log('') + +def compare_images(image, reference, baseimg=3DNone, expected_match=3DTrue= ): + """ + Print a nice human-readable message comparing these images. + """ + expected_ret =3D 0 if expected_match else 1 + if baseimg: + assert qemu_img("rebase", "-u", "-b", baseimg, image) =3D=3D 0 + ret =3D qemu_img("compare", image, reference) + log('qemu_img compare "{:s}" "{:s}" =3D=3D> {:s}, {:s}'.format( + image, reference, + "Identical" if ret =3D=3D 0 else "Mismatch", + "OK!" if ret =3D=3D expected_ret else "ERROR!"), + filters=3D[iotests.filter_testfiles]) + +def test_bitmap_sync(bsync_mode, failure=3DNone): + """ + Test bitmap backup routines. + + :param bsync_mode: Is the Bitmap Sync mode, and can be any of: + - on-success: This is the "incremental" style mode. Bitmaps are + synchronized to what was copied out only on success. + (Partial images must be discarded.) + - never: This is the "differential" style mode. + Bitmaps are never synchronized. + - always: This is a "best effort" style mode. + Bitmaps are always synchronized, regardless of failu= re. + (Partial images must be kept.) + + :param failure: Is the (optional) failure mode, and can be any of: + - None: No failure. Test the normative path. Default. + - simulated: Cancel the job right before it completes. + This also tests writes "during" the job. + - intermediate: This tests a job that fails mid-process and produc= es + an incomplete backup. Testing limitations prevent + testing competing writes. + """ + with iotests.FilePaths(['img', 'bsync1', 'bsync2', + 'fbackup0', 'fbackup1', 'fbackup2']) as \ + (img_path, bsync1, bsync2, + fbackup0, fbackup1, fbackup2), \ + iotests.VM() as vm: + + mode =3D "Bitmap Sync Mode {:s}".format(bsync_mode) + preposition =3D "with" if failure else "without" + cond =3D "{:s} {:s}".format(preposition, + "{:s} failure".format(failure) if failure + else "failure") + log("\n=3D=3D=3D {:s} {:s} =3D=3D=3D\n".format(mode, cond)) + + log('--- Preparing image & VM ---\n') + drive0 =3D Drive(img_path, vm=3Dvm) + drive0.img_create(iotests.imgfmt, SIZE) + vm.add_device("{},id=3Dscsi0".format(iotests.get_virtio_scsi_devic= e())) + vm.launch() + + file_config =3D { + 'driver': 'file', + 'filename': drive0.path + } + + if failure =3D=3D 'intermediate': + file_config =3D { + 'driver': 'blkdebug', + 'image': file_config, + 'set-state': [{ + 'event': 'flush_to_disk', + 'state': 1, + 'new_state': 2 + }, { + 'event': 'read_aio', + 'state': 2, + 'new_state': 3 + }], + 'inject-error': [{ + 'event': 'read_aio', + 'errno': 5, + 'state': 3, + 'immediately': False, + 'once': True + }] + } + + vm.qmp_log('blockdev-add', + filters=3D[iotests.filter_qmp_testfiles], + node_name=3D"drive0", + driver=3Ddrive0.fmt, + file=3Dfile_config) + drive0.node =3D 'drive0' + drive0.device =3D 'device0' + # Use share-rw to allow writes directly to the node; + # The anonymous block-backend for this configuration prevents us + # from using HMP's qemu-io commands to address the device. + vm.qmp_log("device_add", id=3Ddrive0.device, + drive=3Ddrive0.name, driver=3D"scsi-hd", + share_rw=3DTrue) + log('') + + # 0 - Writes and Reference Backup + perform_writes(drive0, 0) + reference_backup(drive0, 0, fbackup0) + log('--- Add Bitmap ---\n') + vm.qmp_log("block-dirty-bitmap-add", node=3Ddrive0.name, + name=3D"bitmap0", granularity=3DGRANULARITY) + log('') + + # 1 - Writes and Reference Backup + bitmaps =3D perform_writes(drive0, 1) + dirty_groups =3D {1} + bitmap =3D get_bitmap(bitmaps, drive0.device, 'bitmap0') + bitmap_comparison(bitmap, groups=3Ddirty_groups) + reference_backup(drive0, 1, fbackup1) + + # 1 - Bitmap Backup (Optional induced failure) + if failure =3D=3D 'intermediate': + # Activate blkdebug induced failure for second-to-next read + log(vm.hmp_qemu_io(drive0.name, 'flush')) + log('') + job =3D bitmap_backup(drive0, 1, bsync1, "bitmap0", bsync_mode) + + def _callback(): + """Issue writes while the job is open to test bitmap divergenc= e.""" + # Note: when `failure` is 'intermediate', this isn't called. + log('') + bitmaps =3D perform_writes(drive0, 2) + # Named bitmap (static, should be unchanged) + bitmap_comparison(get_bitmap(bitmaps, drive0.device, 'bitmap0'= ), + groups=3Ddirty_groups) + # Anonymous bitmap (dynamic, shows new writes) + bitmap_comparison(get_bitmap(bitmaps, drive0.device, '', + recording=3DTrue), groups=3D{2}) + dirty_groups.add(2) + + vm.run_job(job, auto_dismiss=3DTrue, auto_finalize=3DFalse, + pre_finalize=3D_callback, + cancel=3D(failure =3D=3D 'simulated')) + bitmaps =3D query_bitmaps(vm) + bitmap =3D get_bitmap(bitmaps, drive0.device, 'bitmap0') + log(bitmaps, indent=3D2) + log('') + + if ((bsync_mode =3D=3D 'on-success' and not failure) or + (bsync_mode =3D=3D 'always' and failure !=3D 'intermediate= ')): + dirty_groups.remove(1) + + if bsync_mode =3D=3D 'always' and failure =3D=3D 'intermediate': + # We manage to copy one sector (one bit) before the error. + bitmap_comparison(bitmap, + want=3Dcalculate_bits(groups=3Ddirty_groups)= - 1) + else: + bitmap_comparison(bitmap, groups=3Ddirty_groups) + + # 2 - Writes and Reference Backup + bitmaps =3D perform_writes(drive0, 3) + dirty_groups.add(3) + bitmap =3D get_bitmap(bitmaps, drive0.device, 'bitmap0') + if bsync_mode =3D=3D 'always' and failure =3D=3D 'intermediate': + # We're one bit short, still. + bitmap_comparison(bitmap, + want=3Dcalculate_bits(groups=3Ddirty_groups)= - 1) + else: + bitmap_comparison(bitmap, groups=3Ddirty_groups) + reference_backup(drive0, 2, fbackup2) + + # 2 - Bitmap Backup (In failure modes, this is a recovery.) + job =3D bitmap_backup(drive0, 2, bsync2, "bitmap0", bsync_mode) + vm.run_job(job, auto_dismiss=3DTrue, auto_finalize=3DFalse) + bitmaps =3D query_bitmaps(vm) + bitmap =3D get_bitmap(bitmaps, drive0.device, 'bitmap0') + log(bitmaps, indent=3D2) + log('') + bitmap_comparison(bitmap, groups=3D{} + if bsync_mode !=3D 'never' + else dirty_groups) + + log('--- Cleanup ---\n') + vm.qmp_log("block-dirty-bitmap-remove", + node=3Ddrive0.name, name=3D"bitmap0") + log(query_bitmaps(vm), indent=3D2) + vm.shutdown() + log('') + + log('--- Verification ---\n') + # 'simulated' failures will actually all pass here because we canc= eled + # while "pending". This is actually undefined behavior, + # don't rely on this to be true! + compare_images(bsync1, fbackup1, baseimg=3Dfbackup0, + expected_match=3Dfailure !=3D 'intermediate') + if not failure or bsync_mode =3D=3D 'always': + # Always keep the last backup on success or when using 'always' + base =3D bsync1 + else: + base =3D fbackup0 + compare_images(bsync2, fbackup2, baseimg=3Dbase) + compare_images(img_path, fbackup2) + log('') + +def main(): + for bsync_mode in ("never", "on-success", "always"): + for failure in ("simulated", "intermediate", None): + test_bitmap_sync(bsync_mode, failure) + +if __name__ =3D=3D '__main__': + iotests.script_main(main, supported_fmts=3D['qcow2']) diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out new file mode 100644 index 0000000000..e0775d4815 --- /dev/null +++ b/tests/qemu-iotests/257.out @@ -0,0 +1,2247 @@ + +=3D=3D=3D Bitmap Sync Mode never with simulated failure =3D=3D=3D + +--- Preparing image & VM --- + +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}} +{"return": {}} +{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi= -hd", "id": "device0", "share-rw": true}} +{"return": {}} + +--- Write #0 --- + +write -P0x49 0x0000000 0x10000 +{"return": ""} +write -P0x6c 0x0100000 0x10000 +{"return": ""} +write -P0x6f 0x2000000 0x10000 +{"return": ""} +write -P0x76 0x3ff0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [] + } +} + +--- Reference Backup #0 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_0", "sync": "full", "target": "ref_target_0"}} +{"return": {}} +{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Add Bitmap --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, = "name": "bitmap0", "node": "drive0"}} +{"return": {}} + +--- Write #1 --- + +write -P0x65 0x0000000 0x10000 +{"return": ""} +write -P0x77 0x00f8000 0x10000 +{"return": ""} +write -P0x72 0x2008000 0x10000 +{"return": ""} +write -P0x69 0x3fe0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Reference Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_1", "sync": "full", "target": "ref_target_1"}} +{"return": {}} +{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitm= ap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}} +{"return": {}} + +--- Write #2 --- + +write -P0x74 0x0010000 0x10000 +{"return": ""} +write -P0x69 0x00e8000 0x10000 +{"return": ""} +write -P0x6e 0x2018000 0x10000 +{"return": ""} +write -P0x67 0x3fe0000 0x20000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "persistent": false, + "recording": false, + "status": "disabled" + }, + { + "busy": false, + "count": 458752, + "granularity": 65536, + "persistent": false, + "recording": true, + "status": "active" + }, + { + "busy": true, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "frozen" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +=3D Checking Bitmap (anonymous) =3D +expecting 7 dirty sectors; have 7. OK! + +{"execute": "job-cancel", "arguments": {"id": "bitmap_backup_1"}} +{"return": {}} +{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 655360, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 10 dirty sectors; have 10. OK! + +--- Write #3 --- + +write -P0xaa 0x0010000 0x30000 +{"return": ""} +write -P0xbb 0x00d8000 0x10000 +{"return": ""} +write -P0xcc 0x2028000 0x10000 +{"return": ""} +write -P0xdd 0x3fc0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 983040, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 15 dirty sectors; have 15. OK! + +--- Reference Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_2", "sync": "full", "target": "ref_target_2"}} +{"return": {}} +{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitm= ap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}} +{"return": {}} +{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 983040, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 15 dirty sectors; have 15. OK! + +--- Cleanup --- + +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", = "node": "drive0"}} +{"return": {}} +{ + "bitmaps": { + "device0": [] + } +} + +--- Verification --- + +qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" =3D=3D> Identi= cal, OK! + + +=3D=3D=3D Bitmap Sync Mode never with intermediate failure =3D=3D=3D + +--- Preparing image & VM --- + +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"return": {}} +{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi= -hd", "id": "device0", "share-rw": true}} +{"return": {}} + +--- Write #0 --- + +write -P0x49 0x0000000 0x10000 +{"return": ""} +write -P0x6c 0x0100000 0x10000 +{"return": ""} +write -P0x6f 0x2000000 0x10000 +{"return": ""} +write -P0x76 0x3ff0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [] + } +} + +--- Reference Backup #0 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_0", "sync": "full", "target": "ref_target_0"}} +{"return": {}} +{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Add Bitmap --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, = "name": "bitmap0", "node": "drive0"}} +{"return": {}} + +--- Write #1 --- + +write -P0x65 0x0000000 0x10000 +{"return": ""} +write -P0x77 0x00f8000 0x10000 +{"return": ""} +write -P0x72 0x2008000 0x10000 +{"return": ""} +write -P0x69 0x3fe0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Reference Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_1", "sync": "full", "target": "ref_target_1"}} +{"return": {}} +{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +{"return": ""} + +--- Bitmap Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitm= ap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}} +{"return": {}} +{"data": {"action": "report", "device": "bitmap_backup_1", "operation": "r= ead"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "= seconds": "SECS"}} +{"data": {"device": "bitmap_backup_1", "error": "Input/output error", "len= ": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "B= LOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SEC= S"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Write #3 --- + +write -P0xaa 0x0010000 0x30000 +{"return": ""} +write -P0xbb 0x00d8000 0x10000 +{"return": ""} +write -P0xcc 0x2028000 0x10000 +{"return": ""} +write -P0xdd 0x3fc0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 917504, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 14 dirty sectors; have 14. OK! + +--- Reference Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_2", "sync": "full", "target": "ref_target_2"}} +{"return": {}} +{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitm= ap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}} +{"return": {}} +{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 917504, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 14 dirty sectors; have 14. OK! + +--- Cleanup --- + +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", = "node": "drive0"}} +{"return": {}} +{ + "bitmaps": { + "device0": [] + } +} + +--- Verification --- + +qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" =3D=3D> Mis= match, OK! +qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" =3D=3D> Identi= cal, OK! + + +=3D=3D=3D Bitmap Sync Mode never without failure =3D=3D=3D + +--- Preparing image & VM --- + +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}} +{"return": {}} +{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi= -hd", "id": "device0", "share-rw": true}} +{"return": {}} + +--- Write #0 --- + +write -P0x49 0x0000000 0x10000 +{"return": ""} +write -P0x6c 0x0100000 0x10000 +{"return": ""} +write -P0x6f 0x2000000 0x10000 +{"return": ""} +write -P0x76 0x3ff0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [] + } +} + +--- Reference Backup #0 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_0", "sync": "full", "target": "ref_target_0"}} +{"return": {}} +{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Add Bitmap --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, = "name": "bitmap0", "node": "drive0"}} +{"return": {}} + +--- Write #1 --- + +write -P0x65 0x0000000 0x10000 +{"return": ""} +write -P0x77 0x00f8000 0x10000 +{"return": ""} +write -P0x72 0x2008000 0x10000 +{"return": ""} +write -P0x69 0x3fe0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Reference Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_1", "sync": "full", "target": "ref_target_1"}} +{"return": {}} +{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitm= ap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}} +{"return": {}} + +--- Write #2 --- + +write -P0x74 0x0010000 0x10000 +{"return": ""} +write -P0x69 0x00e8000 0x10000 +{"return": ""} +write -P0x6e 0x2018000 0x10000 +{"return": ""} +write -P0x67 0x3fe0000 0x20000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "persistent": false, + "recording": false, + "status": "disabled" + }, + { + "busy": false, + "count": 458752, + "granularity": 65536, + "persistent": false, + "recording": true, + "status": "active" + }, + { + "busy": true, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "frozen" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +=3D Checking Bitmap (anonymous) =3D +expecting 7 dirty sectors; have 7. OK! + +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_1"}} +{"return": {}} +{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 655360, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 10 dirty sectors; have 10. OK! + +--- Write #3 --- + +write -P0xaa 0x0010000 0x30000 +{"return": ""} +write -P0xbb 0x00d8000 0x10000 +{"return": ""} +write -P0xcc 0x2028000 0x10000 +{"return": ""} +write -P0xdd 0x3fc0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 983040, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 15 dirty sectors; have 15. OK! + +--- Reference Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_2", "sync": "full", "target": "ref_target_2"}} +{"return": {}} +{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitm= ap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}} +{"return": {}} +{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 983040, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 15 dirty sectors; have 15. OK! + +--- Cleanup --- + +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", = "node": "drive0"}} +{"return": {}} +{ + "bitmaps": { + "device0": [] + } +} + +--- Verification --- + +qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" =3D=3D> Identi= cal, OK! + + +=3D=3D=3D Bitmap Sync Mode on-success with simulated failure =3D=3D=3D + +--- Preparing image & VM --- + +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}} +{"return": {}} +{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi= -hd", "id": "device0", "share-rw": true}} +{"return": {}} + +--- Write #0 --- + +write -P0x49 0x0000000 0x10000 +{"return": ""} +write -P0x6c 0x0100000 0x10000 +{"return": ""} +write -P0x6f 0x2000000 0x10000 +{"return": ""} +write -P0x76 0x3ff0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [] + } +} + +--- Reference Backup #0 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_0", "sync": "full", "target": "ref_target_0"}} +{"return": {}} +{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Add Bitmap --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, = "name": "bitmap0", "node": "drive0"}} +{"return": {}} + +--- Write #1 --- + +write -P0x65 0x0000000 0x10000 +{"return": ""} +write -P0x77 0x00f8000 0x10000 +{"return": ""} +write -P0x72 0x2008000 0x10000 +{"return": ""} +write -P0x69 0x3fe0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Reference Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_1", "sync": "full", "target": "ref_target_1"}} +{"return": {}} +{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": = "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}} +{"return": {}} + +--- Write #2 --- + +write -P0x74 0x0010000 0x10000 +{"return": ""} +write -P0x69 0x00e8000 0x10000 +{"return": ""} +write -P0x6e 0x2018000 0x10000 +{"return": ""} +write -P0x67 0x3fe0000 0x20000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "persistent": false, + "recording": false, + "status": "disabled" + }, + { + "busy": false, + "count": 458752, + "granularity": 65536, + "persistent": false, + "recording": true, + "status": "active" + }, + { + "busy": true, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "frozen" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +=3D Checking Bitmap (anonymous) =3D +expecting 7 dirty sectors; have 7. OK! + +{"execute": "job-cancel", "arguments": {"id": "bitmap_backup_1"}} +{"return": {}} +{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 655360, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 10 dirty sectors; have 10. OK! + +--- Write #3 --- + +write -P0xaa 0x0010000 0x30000 +{"return": ""} +write -P0xbb 0x00d8000 0x10000 +{"return": ""} +write -P0xcc 0x2028000 0x10000 +{"return": ""} +write -P0xdd 0x3fc0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 983040, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 15 dirty sectors; have 15. OK! + +--- Reference Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_2", "sync": "full", "target": "ref_target_2"}} +{"return": {}} +{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": = "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}} +{"return": {}} +{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 0 dirty sectors; have 0. OK! + +--- Cleanup --- + +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", = "node": "drive0"}} +{"return": {}} +{ + "bitmaps": { + "device0": [] + } +} + +--- Verification --- + +qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" =3D=3D> Identi= cal, OK! + + +=3D=3D=3D Bitmap Sync Mode on-success with intermediate failure =3D=3D=3D + +--- Preparing image & VM --- + +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"return": {}} +{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi= -hd", "id": "device0", "share-rw": true}} +{"return": {}} + +--- Write #0 --- + +write -P0x49 0x0000000 0x10000 +{"return": ""} +write -P0x6c 0x0100000 0x10000 +{"return": ""} +write -P0x6f 0x2000000 0x10000 +{"return": ""} +write -P0x76 0x3ff0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [] + } +} + +--- Reference Backup #0 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_0", "sync": "full", "target": "ref_target_0"}} +{"return": {}} +{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Add Bitmap --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, = "name": "bitmap0", "node": "drive0"}} +{"return": {}} + +--- Write #1 --- + +write -P0x65 0x0000000 0x10000 +{"return": ""} +write -P0x77 0x00f8000 0x10000 +{"return": ""} +write -P0x72 0x2008000 0x10000 +{"return": ""} +write -P0x69 0x3fe0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Reference Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_1", "sync": "full", "target": "ref_target_1"}} +{"return": {}} +{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +{"return": ""} + +--- Bitmap Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": = "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}} +{"return": {}} +{"data": {"action": "report", "device": "bitmap_backup_1", "operation": "r= ead"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "= seconds": "SECS"}} +{"data": {"device": "bitmap_backup_1", "error": "Input/output error", "len= ": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "B= LOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SEC= S"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Write #3 --- + +write -P0xaa 0x0010000 0x30000 +{"return": ""} +write -P0xbb 0x00d8000 0x10000 +{"return": ""} +write -P0xcc 0x2028000 0x10000 +{"return": ""} +write -P0xdd 0x3fc0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 917504, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 14 dirty sectors; have 14. OK! + +--- Reference Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_2", "sync": "full", "target": "ref_target_2"}} +{"return": {}} +{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": = "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}} +{"return": {}} +{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 0 dirty sectors; have 0. OK! + +--- Cleanup --- + +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", = "node": "drive0"}} +{"return": {}} +{ + "bitmaps": { + "device0": [] + } +} + +--- Verification --- + +qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" =3D=3D> Mis= match, OK! +qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" =3D=3D> Identi= cal, OK! + + +=3D=3D=3D Bitmap Sync Mode on-success without failure =3D=3D=3D + +--- Preparing image & VM --- + +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}} +{"return": {}} +{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi= -hd", "id": "device0", "share-rw": true}} +{"return": {}} + +--- Write #0 --- + +write -P0x49 0x0000000 0x10000 +{"return": ""} +write -P0x6c 0x0100000 0x10000 +{"return": ""} +write -P0x6f 0x2000000 0x10000 +{"return": ""} +write -P0x76 0x3ff0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [] + } +} + +--- Reference Backup #0 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_0", "sync": "full", "target": "ref_target_0"}} +{"return": {}} +{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Add Bitmap --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, = "name": "bitmap0", "node": "drive0"}} +{"return": {}} + +--- Write #1 --- + +write -P0x65 0x0000000 0x10000 +{"return": ""} +write -P0x77 0x00f8000 0x10000 +{"return": ""} +write -P0x72 0x2008000 0x10000 +{"return": ""} +write -P0x69 0x3fe0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Reference Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_1", "sync": "full", "target": "ref_target_1"}} +{"return": {}} +{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": = "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}} +{"return": {}} + +--- Write #2 --- + +write -P0x74 0x0010000 0x10000 +{"return": ""} +write -P0x69 0x00e8000 0x10000 +{"return": ""} +write -P0x6e 0x2018000 0x10000 +{"return": ""} +write -P0x67 0x3fe0000 0x20000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "persistent": false, + "recording": false, + "status": "disabled" + }, + { + "busy": false, + "count": 458752, + "granularity": 65536, + "persistent": false, + "recording": true, + "status": "active" + }, + { + "busy": true, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "frozen" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +=3D Checking Bitmap (anonymous) =3D +expecting 7 dirty sectors; have 7. OK! + +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_1"}} +{"return": {}} +{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 458752, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 7 dirty sectors; have 7. OK! + +--- Write #3 --- + +write -P0xaa 0x0010000 0x30000 +{"return": ""} +write -P0xbb 0x00d8000 0x10000 +{"return": ""} +write -P0xcc 0x2028000 0x10000 +{"return": ""} +write -P0xdd 0x3fc0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 786432, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 12 dirty sectors; have 12. OK! + +--- Reference Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_2", "sync": "full", "target": "ref_target_2"}} +{"return": {}} +{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": = "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}} +{"return": {}} +{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 0 dirty sectors; have 0. OK! + +--- Cleanup --- + +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", = "node": "drive0"}} +{"return": {}} +{ + "bitmaps": { + "device0": [] + } +} + +--- Verification --- + +qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" =3D=3D> Identi= cal, OK! + + +=3D=3D=3D Bitmap Sync Mode always with simulated failure =3D=3D=3D + +--- Preparing image & VM --- + +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}} +{"return": {}} +{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi= -hd", "id": "device0", "share-rw": true}} +{"return": {}} + +--- Write #0 --- + +write -P0x49 0x0000000 0x10000 +{"return": ""} +write -P0x6c 0x0100000 0x10000 +{"return": ""} +write -P0x6f 0x2000000 0x10000 +{"return": ""} +write -P0x76 0x3ff0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [] + } +} + +--- Reference Backup #0 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_0", "sync": "full", "target": "ref_target_0"}} +{"return": {}} +{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Add Bitmap --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, = "name": "bitmap0", "node": "drive0"}} +{"return": {}} + +--- Write #1 --- + +write -P0x65 0x0000000 0x10000 +{"return": ""} +write -P0x77 0x00f8000 0x10000 +{"return": ""} +write -P0x72 0x2008000 0x10000 +{"return": ""} +write -P0x69 0x3fe0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Reference Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_1", "sync": "full", "target": "ref_target_1"}} +{"return": {}} +{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bit= map_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}} +{"return": {}} + +--- Write #2 --- + +write -P0x74 0x0010000 0x10000 +{"return": ""} +write -P0x69 0x00e8000 0x10000 +{"return": ""} +write -P0x6e 0x2018000 0x10000 +{"return": ""} +write -P0x67 0x3fe0000 0x20000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "persistent": false, + "recording": false, + "status": "disabled" + }, + { + "busy": false, + "count": 458752, + "granularity": 65536, + "persistent": false, + "recording": true, + "status": "active" + }, + { + "busy": true, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "frozen" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +=3D Checking Bitmap (anonymous) =3D +expecting 7 dirty sectors; have 7. OK! + +{"execute": "job-cancel", "arguments": {"id": "bitmap_backup_1"}} +{"return": {}} +{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 458752, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 7 dirty sectors; have 7. OK! + +--- Write #3 --- + +write -P0xaa 0x0010000 0x30000 +{"return": ""} +write -P0xbb 0x00d8000 0x10000 +{"return": ""} +write -P0xcc 0x2028000 0x10000 +{"return": ""} +write -P0xdd 0x3fc0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 786432, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 12 dirty sectors; have 12. OK! + +--- Reference Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_2", "sync": "full", "target": "ref_target_2"}} +{"return": {}} +{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bit= map_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}} +{"return": {}} +{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 0 dirty sectors; have 0. OK! + +--- Cleanup --- + +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", = "node": "drive0"}} +{"return": {}} +{ + "bitmaps": { + "device0": [] + } +} + +--- Verification --- + +qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" =3D=3D> Identi= cal, OK! + + +=3D=3D=3D Bitmap Sync Mode always with intermediate failure =3D=3D=3D + +--- Preparing image & VM --- + +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img= "}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false= , "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new= -state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]= }, "node-name": "drive0"}} +{"return": {}} +{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi= -hd", "id": "device0", "share-rw": true}} +{"return": {}} + +--- Write #0 --- + +write -P0x49 0x0000000 0x10000 +{"return": ""} +write -P0x6c 0x0100000 0x10000 +{"return": ""} +write -P0x6f 0x2000000 0x10000 +{"return": ""} +write -P0x76 0x3ff0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [] + } +} + +--- Reference Backup #0 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_0", "sync": "full", "target": "ref_target_0"}} +{"return": {}} +{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Add Bitmap --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, = "name": "bitmap0", "node": "drive0"}} +{"return": {}} + +--- Write #1 --- + +write -P0x65 0x0000000 0x10000 +{"return": ""} +write -P0x77 0x00f8000 0x10000 +{"return": ""} +write -P0x72 0x2008000 0x10000 +{"return": ""} +write -P0x69 0x3fe0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Reference Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_1", "sync": "full", "target": "ref_target_1"}} +{"return": {}} +{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +{"return": ""} + +--- Bitmap Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bit= map_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}} +{"return": {}} +{"data": {"action": "report", "device": "bitmap_backup_1", "operation": "r= ead"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "= seconds": "SECS"}} +{"data": {"device": "bitmap_backup_1", "error": "Input/output error", "len= ": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "B= LOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SEC= S"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 327680, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 5 dirty sectors; have 5. OK! + +--- Write #3 --- + +write -P0xaa 0x0010000 0x30000 +{"return": ""} +write -P0xbb 0x00d8000 0x10000 +{"return": ""} +write -P0xcc 0x2028000 0x10000 +{"return": ""} +write -P0xdd 0x3fc0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 851968, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 13 dirty sectors; have 13. OK! + +--- Reference Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_2", "sync": "full", "target": "ref_target_2"}} +{"return": {}} +{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bit= map_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}} +{"return": {}} +{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 0 dirty sectors; have 0. OK! + +--- Cleanup --- + +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", = "node": "drive0"}} +{"return": {}} +{ + "bitmaps": { + "device0": [] + } +} + +--- Verification --- + +qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" =3D=3D> Mis= match, OK! +qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" =3D=3D> Identi= cal, OK! + + +=3D=3D=3D Bitmap Sync Mode always without failure =3D=3D=3D + +--- Preparing image & VM --- + +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"dri= ver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}} +{"return": {}} +{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi= -hd", "id": "device0", "share-rw": true}} +{"return": {}} + +--- Write #0 --- + +write -P0x49 0x0000000 0x10000 +{"return": ""} +write -P0x6c 0x0100000 0x10000 +{"return": ""} +write -P0x6f 0x2000000 0x10000 +{"return": ""} +write -P0x76 0x3ff0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [] + } +} + +--- Reference Backup #0 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_0", "sync": "full", "target": "ref_target_0"}} +{"return": {}} +{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Add Bitmap --- + +{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, = "name": "bitmap0", "node": "drive0"}} +{"return": {}} + +--- Write #1 --- + +write -P0x65 0x0000000 0x10000 +{"return": ""} +write -P0x77 0x00f8000 0x10000 +{"return": ""} +write -P0x72 0x2008000 0x10000 +{"return": ""} +write -P0x69 0x3fe0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +--- Reference Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_1", "sync": "full", "target": "ref_target_1"}} +{"return": {}} +{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #1 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bit= map_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}} +{"return": {}} + +--- Write #2 --- + +write -P0x74 0x0010000 0x10000 +{"return": ""} +write -P0x69 0x00e8000 0x10000 +{"return": ""} +write -P0x6e 0x2018000 0x10000 +{"return": ""} +write -P0x67 0x3fe0000 0x20000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "persistent": false, + "recording": false, + "status": "disabled" + }, + { + "busy": false, + "count": 458752, + "granularity": 65536, + "persistent": false, + "recording": true, + "status": "active" + }, + { + "busy": true, + "count": 393216, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "frozen" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 6 dirty sectors; have 6. OK! + +=3D Checking Bitmap (anonymous) =3D +expecting 7 dirty sectors; have 7. OK! + +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_1"}} +{"return": {}} +{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 458752, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 7 dirty sectors; have 7. OK! + +--- Write #3 --- + +write -P0xaa 0x0010000 0x30000 +{"return": ""} +write -P0xbb 0x00d8000 0x10000 +{"return": ""} +write -P0xcc 0x2028000 0x10000 +{"return": ""} +write -P0xdd 0x3fc0000 0x10000 +{"return": ""} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 786432, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 12 dirty sectors; have 12. OK! + +--- Reference Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id":= "ref_backup_2", "sync": "full", "target": "ref_target_2"}} +{"return": {}} +{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} + +--- Bitmap Backup #2 --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} +{"return": {}} +{} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bit= map_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}} +{"return": {}} +{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}} +{"return": {}} +{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_= PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864= , "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp= ": {"microseconds": "USECS", "seconds": "SECS"}} +{ + "bitmaps": { + "device0": [ + { + "busy": false, + "count": 0, + "granularity": 65536, + "name": "bitmap0", + "persistent": false, + "recording": true, + "status": "active" + } + ] + } +} + +=3D Checking Bitmap bitmap0 =3D +expecting 0 dirty sectors; have 0. OK! + +--- Cleanup --- + +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", = "node": "drive0"}} +{"return": {}} +{ + "bitmaps": { + "device0": [] + } +} + +--- Verification --- + +qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" =3D=3D> Ide= ntical, OK! +qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" =3D=3D> Identi= cal, OK! + diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index b34c8e3c0c..b53afcd149 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -269,3 +269,4 @@ 254 rw auto backing quick 255 rw auto quick 256 rw auto quick +257 rw auto --=20 2.21.0 From nobody Fri May 17 12:36:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562716517; cv=none; d=zoho.com; s=zohoarc; b=RIdLF5eJn+PucFxhOeUOZrkjx08TfuQfCUMY3W+YZcSzNMGwObSuQQLVY6ljwrLyvmxoVvI8nzkUt6UsGKLWs3NfUUYQ00rET5RW1VbZFCBOT9xQLJivqY3SdCCWydsOBfnbThO4beOs5zwSCLfCqgC9xskmA83QZo83qYDeMP8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562716517; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=C0tOLy/wmyzIfdJYre/wpNHcAtTBu59ydk78A+5Nevo=; b=laokXi8/AW7kkDKBcAa99Q4oSHaHnsHqix8EP/GplN+T4Vk/IYNpLPohMmsYsuc0bFvA6sYaSYDExPQm2ELyVb5P+tdng2rP/1aGfhbGZ5g9zzbnaKfD4WHsSlOxTKiL/A5Lye2EAj5KeEnfcrPizpfd73U7qeZxose5pkN8qpQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562716517073294.3704409898312; Tue, 9 Jul 2019 16:55:17 -0700 (PDT) Received: from localhost ([::1]:56926 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzx3-0005hW-1q for importer@patchew.org; Tue, 09 Jul 2019 19:55:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49404) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkzVt-0007eA-0K for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:27:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkzVq-0003lN-M0 for qemu-devel@nongnu.org; Tue, 09 Jul 2019 19:27:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55726) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hkzVj-0003J0-KD; Tue, 09 Jul 2019 19:27:01 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 45F8930833A6; Tue, 9 Jul 2019 23:26:35 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-215.bos.redhat.com [10.18.17.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 819DD5B810; Tue, 9 Jul 2019 23:26:27 +0000 (UTC) From: John Snow To: qemu-block@nongnu.org, qemu-devel@nongnu.org Date: Tue, 9 Jul 2019 19:25:50 -0400 Message-Id: <20190709232550.10724-19-jsnow@redhat.com> In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com> References: <20190709232550.10724-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Tue, 09 Jul 2019 23:26:35 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 18/18] block/backup: loosen restriction on readonly bitmaps X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , vsementsov@virtuozzo.com, Juan Quintela , John Snow , Xie Changlong , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , Wen Congyang , Markus Armbruster Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" With the "never" sync policy, we actually can utilize readonly bitmaps now. Loosen the check at the QMP level, and tighten it based on provided arguments down at the job creation level instead. Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/backup.c | 6 ++++++ blockdev.c | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/block/backup.c b/block/backup.c index acfe87b756..e2729cf6fa 100644 --- a/block/backup.c +++ b/block/backup.c @@ -607,6 +607,12 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, return NULL; } =20 + /* If we need to write to this bitmap, check that we can: */ + if (bitmap_mode !=3D BITMAP_SYNC_MODE_NEVER && + bdrv_dirty_bitmap_check(sync_bitmap, BDRV_BITMAP_DEFAULT, errp= )) { + return NULL; + } + /* Create a new bitmap, and freeze/disable this one. */ if (bdrv_dirty_bitmap_create_successor(bs, sync_bitmap, errp) < 0)= { return NULL; diff --git a/blockdev.c b/blockdev.c index 5dfaa976c9..3e30bc2ca7 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3489,7 +3489,7 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, "when providing a bitmap"); return NULL; } - if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) { + if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_ALLOW_RO, errp)) { return NULL; } } --=20 2.21.0