From nobody Sat May 18 13:36:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=163.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1550127204961764.4725627361355; Wed, 13 Feb 2019 22:53:24 -0800 (PST) Received: from localhost ([127.0.0.1]:41352 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1guAtd-0003vw-Qi for importer@patchew.org; Thu, 14 Feb 2019 01:53:21 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43940) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1guArd-0002b8-1k for qemu-devel@nongnu.org; Thu, 14 Feb 2019 01:51:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1guAk3-0004TK-3B for qemu-devel@nongnu.org; Thu, 14 Feb 2019 01:43:30 -0500 Received: from m50-134.163.com ([123.125.50.134]:49273) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1guAk0-00042e-30; Thu, 14 Feb 2019 01:43:25 -0500 Received: from localhost.localdomain (unknown [111.202.166.3]) by smtp4 (Coremail) with SMTP id DtGowACHzNUCDmVce0y6AA--.228S3; Thu, 14 Feb 2019 14:43:16 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:Subject:Date:Message-Id; bh=NEvjgU9Wc6VR3DQn94 43eYm9EbCGzRTkcGzJ4b70Zq8=; b=BYU8LhsIFFiDzr+7wFV/SxYL0ECvlTozHC fDWpYmElPqr06LyaABnavvPAdCUzBaBq8jp0/ZxZ0RgqHQl7w7kmshN4ObYu4SQL sDdVkvADZVJ4sHlFts2wrvEQo7g9OtsnaNJfyXu8pvxN9jcsHb9+vAgvtOKVgQ6P E1AhVZR28= From: mahaocong To: qemu-block@nongnu.org Date: Thu, 14 Feb 2019 14:43:12 +0800 Message-Id: <20190214064312.44794-1-mahaocong_work@163.com> X-Mailer: git-send-email 2.14.3 (Apple Git-98) X-CM-TRANSID: DtGowACHzNUCDmVce0y6AA--.228S3 X-Coremail-Antispam: 1Uf129KBjvAXoWfGFW8uF45CF18KFyftFy3twb_yoW8Xr1DXo WfZrs093WYqrWY9ry2krn5Ar17Z3ZYgrn3Jw13WrZxCws2qa4YkF15Kw4fXa9xJF1vvFy8 t34xK3s8KFs7AFyfn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UbIYCTnIWIevJa73UjIFyTuYvjxUICJmUUUUU X-Originating-IP: [111.202.166.3] X-CM-SenderInfo: 5pdkt05frqwspzruyqqrwthudrp/1tbiRAVbClSIZpBboQAAsA X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 123.125.50.134 Subject: [Qemu-devel] [PATCH RESEND v4] drive-mirror: add incremental mode X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, jcody@redhat.com, mahaocong , qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: mahaocong This patch adds possibility to start mirroring with user-created-bitmap. On full mode, mirror create a non-named-bitmap by scanning whole block-chai= n, and on top mode, mirror create a bitmap by scanning the top block layer. So= I think I can copy a user-created-bitmap and use it as the initial state of t= he mirror, the same as incremental mode drive-backup, and I call this new mode as incremental mode drive-mirror. A possible usage scene of incremental mode mirror is live migration. For ma= intain the block data and recover after a malfunction, someone may backup data to = ceph or other distributed storage. On qemu incremental backup, we need to create= a new bitmap and attach to block device before the first backup job. Then the bit= map records the change after the backup job. If we want to migration this vm, w= e can migrate block data between source and destionation by using drive-mirror di= rectly, or use backup data and backup-bitmap to reduce the data should be synchroni= ze. To do this, we should first create a new image in destination and set backi= ng file as backup image, then set backup-bitmap as the initial state of incremental= mode drive-mirror, and synchronize dirty block starting with the state set by th= e last incremental backup job. When the mirror complete, we get an active layer on= destination, and its backing file is backup image on ceph. Then we can do live copy data= from backing files into overlay images by using block-stream, or do backup conti= nually. In this scene, It seems that If the guest os doesn't write too many data af= ter the last backup, the incremental mode may transmit less data than full mode or = top mode. However, if the write data is too many, there is no advantage on incr= emental mode compare with other mode. This scene can be described as following steps: 1.create rbd image in ceph, and map nbd device with rbd image. 2.create a new bitmap and attach to block device. 3.do full mode backup on nbd device and thus sync it to the rbd image. 4.severl times incremental mode backup. 5.create new image in destination and set its backing file as backup image. 6.do live-migration, and migrate block data by incremental mode drive-mirror with bitmap created from step 2. Signed-off-by: Ma Haocong --- compare with the version 3, there are following changes: 1. Adjust error checking form incremental mode to blockdev_mirror_common, = which lets drive-mirror and blockdev-mirror share error checking for incremental m= ode. 2. Delete error checking about granularity, which is seems to be a non-fat= al error, and add a warnning log to mark it. 3. Add incremental mode on blockdev-mirror by adding parameter 'bitmap' on= qmp command 'blockdev-mirror'. 4. Adjust the discription about parameter 'bitmap' on qmp command 'drive-m= irror' and 'blockdev-mirror'. I used old version libvirt (3.9.0 before) to test live migration, and I fo= und this version use 'qemuMigrationDriveMirror' to migrate block device. Therefore, I only = focused on drive-mirror. I add incremental mode on blockdev-mirror on new version pat= ch. However, I am not sure about the difference between 'drive-mirror' and 'blockdev-mi= rror'. I learn that I need to use 'blockdev-add' to add new node before using 'bl= ockdev-mirror'. Other than that, I don't know what else is different. Therefore, there may= be some defect on blockdev-mirror. I'm still in learning sth about 'blockdev-mirror', ple= ase give me some advices, thank you. block/mirror.c | 47 +++++++++++++++++++++++++++++++++++--------= ---- blockdev.c | 36 ++++++++++++++++++++++++++++++++++-- include/block/block_int.h | 3 ++- qapi/block-core.json | 14 ++++++++++++-- 4 files changed, 83 insertions(+), 17 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index ab59ad77e8..1fe0c767cf 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -50,6 +50,7 @@ typedef struct MirrorBlockJob { /* Used to block operations on the drive-mirror-replace target */ Error *replace_blocker; bool is_none_mode; + BdrvDirtyBitmap *src_bitmap; BlockMirrorBackingMode backing_mode; MirrorCopyMode copy_mode; BlockdevOnError on_source_error, on_target_error; @@ -814,6 +815,16 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJ= ob *s) return 0; } =20 +/* + * init dirty bitmap by using user bitmap. usr->hbitmap will be copy to + * mirror bitmap->hbitmap instead of reuse it. + */ +static void coroutine_fn mirror_dirty_init_incremental(MirrorBlockJob *s, + Error **errp) +{ + bdrv_merge_dirty_bitmap(s->dirty_bitmap, s->src_bitmap, NULL, errp); +} + /* Called when going out of the streaming phase to flush the bulk of the * data to the medium, or just before completing. */ @@ -839,6 +850,7 @@ static int coroutine_fn mirror_run(Job *job, Error **er= rp) char backing_filename[2]; /* we only need 2 characters because we are = only checking for a NULL string */ int ret =3D 0; + Error *local_err =3D NULL; =20 if (job_is_cancelled(&s->common.job)) { goto immediate_exit; @@ -913,9 +925,19 @@ static int coroutine_fn mirror_run(Job *job, Error **e= rrp) =20 s->last_pause_ns =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); if (!s->is_none_mode) { - ret =3D mirror_dirty_init(s); - if (ret < 0 || job_is_cancelled(&s->common.job)) { - goto immediate_exit; + /* incremental mode */ + if (s->src_bitmap) { + mirror_dirty_init_incremental(s, &local_err); + if (local_err) { + error_propagate(errp, local_err); + ret =3D -1; + goto immediate_exit; + } + } else { + ret =3D mirror_dirty_init(s); + if (ret < 0 || job_is_cancelled(&s->common.job)) { + goto immediate_exit; + } } } =20 @@ -1484,7 +1506,8 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, BlockCompletionFunc *cb, void *opaque, const BlockJobDriver *driver, - bool is_none_mode, BlockDriverState *base, + bool is_none_mode, BdrvDirtyBitmap *src_bitma= p, + BlockDriverState *base, bool auto_complete, const char *filter_node_n= ame, bool is_mirror, MirrorCopyMode copy_mode, Error **errp) @@ -1598,6 +1621,7 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, s->on_source_error =3D on_source_error; s->on_target_error =3D on_target_error; s->is_none_mode =3D is_none_mode; + s->src_bitmap =3D src_bitmap; s->backing_mode =3D backing_mode; s->copy_mode =3D copy_mode; s->base =3D base; @@ -1664,7 +1688,8 @@ void mirror_start(const char *job_id, BlockDriverStat= e *bs, BlockDriverState *target, const char *replaces, int creation_flags, int64_t speed, uint32_t granularity, int64_t buf_size, - MirrorSyncMode mode, BlockMirrorBackingMode backing_mode, + MirrorSyncMode mode, BdrvDirtyBitmap *src_bitmap, + BlockMirrorBackingMode backing_mode, BlockdevOnError on_source_error, BlockdevOnError on_target_error, bool unmap, const char *filter_node_name, @@ -1673,17 +1698,14 @@ void mirror_start(const char *job_id, BlockDriverSt= ate *bs, bool is_none_mode; BlockDriverState *base; =20 - if (mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { - error_setg(errp, "Sync mode 'incremental' not supported"); - return; - } is_none_mode =3D mode =3D=3D MIRROR_SYNC_MODE_NONE; base =3D mode =3D=3D MIRROR_SYNC_MODE_TOP ? backing_bs(bs) : NULL; mirror_start_job(job_id, bs, creation_flags, target, replaces, speed, granularity, buf_size, backing_mode, on_source_error, on_target_error, unmap, NULL, NULL, - &mirror_job_driver, is_none_mode, base, false, - filter_node_name, true, copy_mode, errp); + &mirror_job_driver, is_none_mode, + src_bitmap, base, false, filter_node_name, true, + copy_mode, errp); } =20 void commit_active_start(const char *job_id, BlockDriverState *bs, @@ -1707,7 +1729,8 @@ void commit_active_start(const char *job_id, BlockDri= verState *bs, mirror_start_job(job_id, bs, creation_flags, base, NULL, speed, 0, 0, MIRROR_LEAVE_BACKING_CHAIN, on_error, on_error, true, cb, opaque, - &commit_active_job_driver, false, base, auto_complete, + &commit_active_job_driver, false, + NULL, base, auto_complete, filter_node_name, false, MIRROR_COPY_MODE_BACKGROUND, &local_err); if (local_err) { diff --git a/blockdev.c b/blockdev.c index a6f71f9d83..9da06c0806 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3661,6 +3661,8 @@ static void blockdev_mirror_common(const char *job_id= , BlockDriverState *bs, BlockDriverState *target, bool has_replaces, const char *replaces, enum MirrorSyncMode sync, + bool has_bitmap, + const char *bitmap_name, BlockMirrorBackingMode backing_mode, bool has_speed, int64_t speed, bool has_granularity, uint32_t granular= ity, @@ -3678,6 +3680,7 @@ static void blockdev_mirror_common(const char *job_id= , BlockDriverState *bs, Error **errp) { int job_flags =3D JOB_DEFAULT; + BdrvDirtyBitmap *src_bitmap =3D NULL; =20 if (!has_speed) { speed =3D 0; @@ -3700,6 +3703,10 @@ static void blockdev_mirror_common(const char *job_i= d, BlockDriverState *bs, if (!has_filter_node_name) { filter_node_name =3D NULL; } + if (!has_bitmap) { + bitmap_name =3D NULL; + } + if (!has_copy_mode) { copy_mode =3D MIRROR_COPY_MODE_BACKGROUND; } @@ -3731,13 +3738,35 @@ static void blockdev_mirror_common(const char *job_= id, BlockDriverState *bs, if (!bs->backing && sync =3D=3D MIRROR_SYNC_MODE_TOP) { sync =3D MIRROR_SYNC_MODE_FULL; } + if (!bitmap_name && (sync =3D=3D MIRROR_SYNC_MODE_INCREMENTAL)) { + error_setg(errp, "incremental mode must specify the bitmap name"); + return; + } + /* + * In incremental mode, we should create null name bitmap by + * using user bitmap's granularity. + */ + if (sync =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { + assert(bitmap_name); + src_bitmap =3D bdrv_find_dirty_bitmap(bs, bitmap_name); + if (!src_bitmap) { + error_setg(errp, "Error: can't find dirty bitmap " + "before start incremental drive-mirror"); + return; + } + if (granularity) { + warn_report("On incremental mode, granularity is unused, " + "the bitmap's granularity is used instead"); + } + granularity =3D bdrv_dirty_bitmap_granularity(src_bitmap); + } =20 /* pass the node name to replace to mirror start since it's loose coup= ling * and will allow to check whether the node still exist at mirror comp= letion */ mirror_start(job_id, bs, target, has_replaces ? replaces : NULL, job_flags, - speed, granularity, buf_size, sync, backing_mode, + speed, granularity, buf_size, sync, src_bitmap, backing_m= ode, on_source_error, on_target_error, unmap, filter_node_name, copy_mode, errp); } @@ -3878,6 +3907,7 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp) =20 blockdev_mirror_common(arg->has_job_id ? arg->job_id : NULL, bs, targe= t_bs, arg->has_replaces, arg->replaces, arg->sync, + arg->has_bitmap, arg->bitmap, backing_mode, arg->has_speed, arg->speed, arg->has_granularity, arg->granularity, arg->has_buf_size, arg->buf_size, @@ -3899,6 +3929,7 @@ void qmp_blockdev_mirror(bool has_job_id, const char = *job_id, const char *device, const char *target, bool has_replaces, const char *replaces, MirrorSyncMode sync, + bool has_bitmap, const char *bitmap, bool has_speed, int64_t speed, bool has_granularity, uint32_t granularity, bool has_buf_size, int64_t buf_size, @@ -3935,7 +3966,8 @@ void qmp_blockdev_mirror(bool has_job_id, const char = *job_id, bdrv_set_aio_context(target_bs, aio_context); =20 blockdev_mirror_common(has_job_id ? job_id : NULL, bs, target_bs, - has_replaces, replaces, sync, backing_mode, + has_replaces, replaces, sync, has_bitmap, + bitmap, backing_mode, has_speed, speed, has_granularity, granularity, has_buf_size, buf_size, diff --git a/include/block/block_int.h b/include/block/block_int.h index f605622216..57a441f992 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -1054,7 +1054,8 @@ void mirror_start(const char *job_id, BlockDriverStat= e *bs, BlockDriverState *target, const char *replaces, int creation_flags, int64_t speed, uint32_t granularity, int64_t buf_size, - MirrorSyncMode mode, BlockMirrorBackingMode backing_mode, + MirrorSyncMode mode, BdrvDirtyBitmap *src_bitmap, + BlockMirrorBackingMode backing_mode, BlockdevOnError on_source_error, BlockdevOnError on_target_error, bool unmap, const char *filter_node_name, diff --git a/qapi/block-core.json b/qapi/block-core.json index 762000f31f..4be810132e 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1727,6 +1727,11 @@ # (all the disk, only the sectors allocated in the topmost image, or # only new I/O). # +# @bitmap: The name of a bitmap to use in incremental mode. This argument = must +# be present for incremental mode and absent otherwise. In increm= ental +# mode, granularity is unused, the bitmap's granularity is used i= nstead +# (since 4.0). +# # @granularity: granularity of the dirty bitmap, default is 64K # if the image format doesn't have clusters, 4K if the clust= ers # are smaller than that, else the cluster size. Must be a @@ -1768,7 +1773,7 @@ { 'struct': 'DriveMirror', 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str', '*format': 'str', '*node-name': 'str', '*replaces': 'str', - 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode', + 'sync': 'MirrorSyncMode', '*bitmap': 'str', '*mode': 'NewImage= Mode', '*speed': 'int', '*granularity': 'uint32', '*buf-size': 'int', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', @@ -2017,6 +2022,11 @@ # (all the disk, only the sectors allocated in the topmost image, or # only new I/O). # +# @bitmap: The name of a bitmap to use in incremental mode. This argument = must +# be present for incremental mode and absent otherwise. In increm= ental +# mode, granularity is unused, the bitmap's granularity is used i= nstead +# (since 4.0). +# # @granularity: granularity of the dirty bitmap, default is 64K # if the image format doesn't have clusters, 4K if the clust= ers # are smaller than that, else the cluster size. Must be a @@ -2069,7 +2079,7 @@ { 'command': 'blockdev-mirror', 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str', '*replaces': 'str', - 'sync': 'MirrorSyncMode', + 'sync': 'MirrorSyncMode', '*bitmap': 'str', '*speed': 'int', '*granularity': 'uint32', '*buf-size': 'int', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', --=20 2.14.1