From nobody Mon May 6 01:19:56 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1616232899; cv=none; d=zohomail.com; s=zohoarc; b=LDmuxMhgsjLsEDl+YiEghsXr5rSlKW2ZoBVPAzypjFIRF8pPT7ummkotlP30BMe5EoLnSUSlmCqjQ5DeymYy99RatvcXH6denFuhnZektHNf/FQvT+49dI02/D8gr5GndJcO+GWWvKk3OleAS+aDAVH1dXlXXqpC9gBVixGllPU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616232899; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LHBYKDT9hBV4qnHLdN4IFa4laURN8Txokf9XBcSA0Ww=; b=HGg6mJAmBZanYGWM2HA/oew1XCJX3EJHIrSPOZmewkIxhSVTygTDXn1iBCAMXP8oqcnOR/a68PpX/UJQ1wHp25phGjjOzcjowvdqrEO4hpIjTGpe0XRt8RZOShY4ijWYGHMRyVz56geiA4uDyOx1VESdlKSBaxhlnHlEAR9tQdI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1616232898877926.1725614567015; Sat, 20 Mar 2021 02:34:58 -0700 (PDT) Received: from localhost ([::1]:49428 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lNY0X-0004gT-Mq for importer@patchew.org; Sat, 20 Mar 2021 05:34:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55676) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNXzC-0003ov-RT for qemu-devel@nongnu.org; Sat, 20 Mar 2021 05:33:35 -0400 Received: from mxe2.seznam.cz ([2a02:598:2::34]:51339) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNXzA-0005ce-2i for qemu-devel@nongnu.org; Sat, 20 Mar 2021 05:33:34 -0400 Received: from email.seznam.cz by email-smtpc29a.ng.seznam.cz (email-smtpc29a.ng.seznam.cz [10.23.18.42]) id 6da514190daa33ab6b965f0e; Sat, 20 Mar 2021 10:33:28 +0100 (CET) Received: from archlinux.localdomain (2a01:510:d502:b200:c1b:ad27:bde0:341a [2a01:510:d502:b200:c1b:ad27:bde0:341a]) by email-relay4.ng.seznam.cz (Seznam SMTPD 1.3.124) with ESMTP; Sat, 20 Mar 2021 10:33:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=emailprofi.seznam.cz; s=beta; t=1616232808; bh=LHBYKDT9hBV4qnHLdN4IFa4laURN8Txokf9XBcSA0Ww=; h=Received:From:To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To: References:MIME-Version:Content-Type:Content-Transfer-Encoding; b=L6Rqpt1jVlWYp2cuqwsXLlyJB1xWjsDIN3rx3h4RC3W9gRtTsxz7m3d+A6GMZquoj Lj2xRblLh8b1dVpNRbu31gG+Z4fUyvk/LAJTmRGKzuhez9wv64Zq+j39e3NybLch1i reIoTX2AY7bzLrVFDX2LWgEEjL309DSKr5Udi+Xo= From: =?UTF-8?q?Patrik=20Janou=C5=A1ek?= To: qemu-devel@nongnu.org Subject: [PATCH 1/2] block/raw: added support of persistent dirty bitmaps Date: Sat, 20 Mar 2021 10:32:34 +0100 Message-Id: <20210320093235.461485-2-pj@patrikjanousek.cz> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210320093235.461485-1-pj@patrikjanousek.cz> References: <20210320093235.461485-1-pj@patrikjanousek.cz> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:598:2::34; envelope-from=pj@patrikjanousek.cz; helo=mxe2.seznam.cz X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Patrik=20Janou=C5=A1ek?= , lmatejka@kiv.zcu.cz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Current implementation of dirty bitmaps for raw format is very limited, because those bitmaps cannot be persistent. Basically it makes sense, because the raw format doesn't have space where could be dirty bitmap stored when QEMU is offline. This patch solves it by storing content of every dirty bitmap in separate file on the host filesystem. However, this only solves one part of the problem. We also have to store information about the existence of the dirty bitmap. This is solved by adding custom options, that stores all required metadata about dirty bitmap (filename where is the bitmap stored on the host filesystem, granularity, persistence, etc.). Signed-off-by: Patrik Janou=C5=A1ek --- block/meson.build | 1 + block/raw-format-bitmap.c | 163 ++++++++++++++++++++++++ block/raw-format.c | 256 ++++++++++++++++++++++++++++++++++++-- block/raw-format.h | 50 ++++++++ 4 files changed, 459 insertions(+), 11 deletions(-) create mode 100644 block/raw-format-bitmap.c create mode 100644 block/raw-format.h diff --git a/block/meson.build b/block/meson.build index 5dcc1e5cce..f163095574 100644 --- a/block/meson.build +++ b/block/meson.build @@ -30,6 +30,7 @@ block_ss.add(files( 'qcow2.c', 'quorum.c', 'raw-format.c', + 'raw-format-bitmap.c', 'snapshot.c', 'throttle-groups.c', 'throttle.c', diff --git a/block/raw-format-bitmap.c b/block/raw-format-bitmap.c new file mode 100644 index 0000000000..f622f0a700 --- /dev/null +++ b/block/raw-format-bitmap.c @@ -0,0 +1,163 @@ +#include + +#include "qemu/osdep.h" +#include "block/block_int.h" + +#include "raw-format.h" +#include "qapi/error.h" + +bool coroutine_fn raw_co_can_store_new_dirty_bitmap(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp) +{ + BDRVRawState *s =3D bs->opaque; + + RawDirtyBitmapOpts *opts; + BdrvDirtyBitmap *bm; + FILE *fp; + bool file_exists; + for (int i =3D 0; i < s->rdbol.n; i++) { + opts =3D &s->rdbol.opts[i]; + bm =3D bdrv_find_dirty_bitmap(bs, name); + if (!bm) { + error_setg(errp, "Dirty bitmap %s does not exists.", name); + return false; + } + + if (!opts->has_filename) { + error_setg(errp, "Dirty bitmap does not have filename defined.= "); + return false; + } + + if (!bdrv_dirty_bitmap_get_persistence(bm)) continue; + if (strcmp(opts->name, bdrv_dirty_bitmap_name(bm)) !=3D 0) + continue; + + file_exists =3D false; + fp =3D fopen(opts->filename, "r"); + if (fp) { + file_exists =3D true; + fclose(false); + } + + fp =3D fopen(opts->filename, "a"); + if (!fp) { + error_setg(errp, "Unable to open file %s for read and write", + opts->filename); + return false; + } else { + fclose(fp); + if (!file_exists) unlink(opts->filename); + } + } + + return true; +} + +bool raw_supports_persistent_dirty_bitmap(BlockDriverState *bs) +{ + return true; +} + +int coroutine_fn raw_co_remove_persistent_dirty_bitmap(BlockDriverState *b= s, + const char *name, + Error **errp) +{ + BDRVRawState *s =3D bs->opaque; + + RawDirtyBitmapOpts *opts; + BdrvDirtyBitmap *bm; + for (int i =3D 0; i < s->rdbol.n; i++) { + opts =3D &s->rdbol.opts[i]; + bm =3D bdrv_find_dirty_bitmap(bs, name); + if (!bm) { + error_setg(errp, "Dirty bitmap %s does not exists.", name); + return false; + } + + if (!opts->has_filename) { + error_setg(errp, "Dirty bitmap does not have filename defined.= "); + return false; + } + + if (!bdrv_dirty_bitmap_get_persistence(bm)) continue; + if (strcmp(opts->name, bdrv_dirty_bitmap_name(bm)) !=3D 0) + continue; + + return unlink(opts->filename); + } + + return 0; +} + +static void load_bitmap(BdrvDirtyBitmap *bm, FILE *fpbm) { + uint64_t bm_size =3D bdrv_dirty_bitmap_size(bm); + uint64_t tb_size =3D bdrv_dirty_bitmap_serialization_size(bm, 0, bm_si= ze); + uint8_t *buf =3D g_malloc(tb_size); + size_t n =3D fread(buf, 1, tb_size, fpbm); + + assert(n =3D=3D tb_size); + + bdrv_dirty_bitmap_deserialize_part(bm, buf, 0, bm_size, false); + bdrv_dirty_bitmap_deserialize_finish(bm); + + g_free(buf); +} + +void raw_load_persistent_dirty_bitmap(BlockDriverState *bs, FILE *fp, + uint32_t granularity, char* name, + bool persistent, bool disabled, + Error **errp) +{ + BdrvDirtyBitmap *bm; + + bm =3D bdrv_create_dirty_bitmap(bs, granularity, name, errp); + if (bm) { + bdrv_dirty_bitmap_set_persistence(bm, persistent); + + if (disabled) + bdrv_disable_dirty_bitmap(bm); + } + + if (fp) { + load_bitmap(bm, fp); + } +} + +static void store_bitmap(BdrvDirtyBitmap *bm, FILE *fpbm) { + uint64_t bm_size =3D bdrv_dirty_bitmap_size(bm); + uint64_t tb_size =3D bdrv_dirty_bitmap_serialization_size(bm, 0, bm_si= ze); + uint8_t *buf =3D g_malloc(tb_size); + + bdrv_dirty_bitmap_serialize_part(bm, buf, 0, bm_size); + + fwrite(buf, 1, tb_size, fpbm); + + g_free(buf); +} + +void raw_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp) +{ + BDRVRawState *s =3D bs->opaque; + + RawDirtyBitmapOpts *opts; + BdrvDirtyBitmap *bm; + for (int i =3D 0; i < s->rdbol.n; i++) { + opts =3D &s->rdbol.opts[i]; + + FOR_EACH_DIRTY_BITMAP(bs, bm) { + if (!bdrv_dirty_bitmap_get_persistence(bm)) continue; + assert(opts->has_filename); + + if (strcmp(opts->filename, bdrv_dirty_bitmap_name(bm)) !=3D 0) + continue; + + FILE *fp =3D fopen(opts->filename, "w"); + if(fp) { + store_bitmap(bm, fp); + fclose(fp); + } + } + } +} diff --git a/block/raw-format.c b/block/raw-format.c index 42ec50802b..901f7a9c33 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -28,15 +28,13 @@ =20 #include "qemu/osdep.h" #include "block/block_int.h" +#include "block/qdict.h" #include "qapi/error.h" +#include "qapi/qmp/qlist.h" #include "qemu/module.h" #include "qemu/option.h" =20 -typedef struct BDRVRawState { - uint64_t offset; - uint64_t size; - bool has_size; -} BDRVRawState; +#include "raw-format.h" =20 static const char *const mutable_opts[] =3D { "offset", "size", NULL }; =20 @@ -58,6 +56,40 @@ static QemuOptsList raw_runtime_opts =3D { }, }; =20 +static QemuOptsList raw_dirty_bitmap_opts =3D { + .name =3D "raw-dirty-bitmap-opts", + .head =3D QTAILQ_HEAD_INITIALIZER(raw_dirty_bitmap_opts.head), + .desc =3D { + { + .name =3D "filename", + .type =3D QEMU_OPT_STRING, + .help =3D "path to the persistent dirty bitmap file", + }, + { + .name =3D "name", + .type =3D QEMU_OPT_STRING, + .help =3D "name of the dirty bitmap", + }, + { + .name =3D "granularity", + .type =3D QEMU_OPT_NUMBER, + .help =3D "granularity of the dirty bitmap", + }, + { + .name =3D "persistent", + .type =3D QEMU_OPT_BOOL, + .help =3D "indicates whether the dirty bitmap is persistent" + "(requires filename)", + }, + { + .name =3D "disabled", + .type =3D QEMU_OPT_BOOL, + .help =3D "indicates whether the dirty bitmap is disabled", + }, + { /* end of list */ } + }, +}; + static QemuOptsList raw_create_opts =3D { .name =3D "raw-create-opts", .head =3D QTAILQ_HEAD_INITIALIZER(raw_create_opts.head), @@ -71,8 +103,144 @@ static QemuOptsList raw_create_opts =3D { } }; =20 +static int raw_read_dirty_bitmap_options(QDict *options, + bool *has_filename, char **filena= me, + bool *has_name, char **name, + bool *has_granularity, + uint32_t *granularity, + bool *has_persistent, bool *persi= stent, + bool *has_disabled, bool *disable= d, + Error **errp) +{ + QemuOpts *opts =3D NULL; + int ret; + + opts =3D qemu_opts_create(&raw_dirty_bitmap_opts, NULL, 0, &error_abor= t); + if (!qemu_opts_absorb_qdict(opts, options, errp)) { + ret =3D -EINVAL; + goto end; + } + + *has_filename =3D qemu_opt_find(opts, "filename"); + *filename =3D g_strdup(qemu_opt_get(opts, "filename")); + *has_name =3D qemu_opt_find(opts, "name"); + *name =3D g_strdup(qemu_opt_get(opts, "name")); + *has_granularity =3D qemu_opt_find(opts, "granularity"); + *granularity =3D qemu_opt_get_number(opts, "granularity", 65536); + *has_persistent =3D qemu_opt_find(opts, "persistent"); + *persistent =3D qemu_opt_get_bool(opts, "persistent", false); + *has_disabled =3D qemu_opt_find(opts, "disabled"); + *disabled =3D qemu_opt_get_bool(opts, "disabled", false); + + ret =3D 0; + end: + qemu_opts_del(opts); + return ret; +} + +static int raw_read_dirty_bitmaps_options(QDict *options, + RawDirtyBitmapOptsList *rdbol, + Error **errp) +{ + int ret; + QDict *db_options =3D NULL; + + QObject *qo; + char static_prefix[] =3D "dirty-bitmaps."; + uint static_prefix_len =3D strlen(static_prefix); + uint key_max_len =3D static_prefix_len; + char *key =3D g_malloc0(key_max_len + 1); + int i =3D 0, j; + char ibuff[16]; + size_t nkey, nname; + *rdbol =3D (RawDirtyBitmapOptsList){ + .n =3D 0, + .opts =3D NULL, + }; + while (true) { + nkey =3D snprintf(ibuff, sizeof(ibuff), "%d.", i); + + for (j =3D 0; raw_dirty_bitmap_opts.desc[j].name !=3D NULL; j++) { + nname =3D strlen(raw_dirty_bitmap_opts.desc[j].name); + if (key_max_len < static_prefix_len + nkey + nname) { + key_max_len =3D static_prefix_len + nkey + nname; + key =3D g_realloc(key, key_max_len + 1); + } + + strncpy(key, static_prefix, key_max_len); + strncpy(key + static_prefix_len, ibuff, + key_max_len - static_prefix_len); + strncpy(key + static_prefix_len + nkey, + raw_dirty_bitmap_opts.desc[j].name, + key_max_len - static_prefix_len - nkey); + *(key + static_prefix_len + nkey + nname) =3D '\0'; + + qo =3D qdict_get(options, key); + if (qo !=3D NULL) { + qobject_ref(qo); + qdict_del(options, key); + + if (db_options =3D=3D NULL) db_options =3D qdict_new(); + qdict_put_obj(db_options, raw_dirty_bitmap_opts.desc[j].na= me, + qo); + } + } + + if (db_options !=3D NULL && qdict_size(db_options) !=3D 0) { + rdbol->n++; + if (rdbol->n =3D=3D 1) + rdbol->opts =3D g_malloc0(sizeof(RawDirtyBitmapOpts) * rdb= ol->n); + else + rdbol->opts =3D g_realloc(rdbol->opts, + sizeof(RawDirtyBitmapOpts) * rdbol= ->n); + + bool has_filename; + char* filename; + bool has_name; + char *name; + bool has_granularity; + uint32_t granularity; + bool has_persistent; + bool persistent; + bool has_disabled; + bool disabled; + ret =3D raw_read_dirty_bitmap_options(db_options, + &has_filename, &filename, + &has_name, &name, + &has_granularity, &granula= rity, + &has_persistent, &persiste= nt, + &has_disabled, &disabled, + errp); + qobject_unref(db_options); + if (ret) { + goto end; + } + + rdbol->opts[rdbol->n - 1] =3D (RawDirtyBitmapOpts) { + .has_filename =3D has_filename, + .filename =3D filename, + .has_name =3D has_name || has_filename, + .name =3D has_name ? name : filename, + .has_granularity =3D has_granularity, + .granularity =3D granularity, + .has_persistent =3D has_persistent, + .persistent =3D persistent, + .has_disabled =3D has_disabled, + .disabled =3D disabled, + }; + } else { + break; + } + } + + ret =3D 0; + end: + return ret; +} + static int raw_read_options(QDict *options, uint64_t *offset, bool *has_si= ze, - uint64_t *size, Error **errp) + uint64_t *size, RawDirtyBitmapOptsList *rdbol, + Error **errp) { QemuOpts *opts =3D NULL; int ret; @@ -87,15 +255,35 @@ static int raw_read_options(QDict *options, uint64_t *= offset, bool *has_size, *has_size =3D qemu_opt_find(opts, "size"); *size =3D qemu_opt_get_size(opts, "size", 0); =20 + ret =3D raw_read_dirty_bitmaps_options(options, rdbol, errp); + if (ret < 0) { + ret =3D -EINVAL; + goto end; + } + ret =3D 0; end: qemu_opts_del(opts); return ret; } =20 +static void free_RawDirtyBitmapOptsList(RawDirtyBitmapOptsList *rdbol) { + if (rdbol->opts !=3D NULL) { + for (int i =3D 0; i < rdbol->n; i++) { + if (rdbol->opts[i].filename !=3D NULL) + g_free(rdbol->opts[i].filename); + if (rdbol->opts[i].name !=3D NULL) + g_free(rdbol->opts[i].name); + } + + g_free(rdbol->opts); + rdbol->opts =3D NULL; + } +} + static int raw_apply_options(BlockDriverState *bs, BDRVRawState *s, uint64_t offset, bool has_size, uint64_t size, - Error **errp) + RawDirtyBitmapOptsList rdbol, Error **errp) { int64_t real_size =3D 0; =20 @@ -132,6 +320,29 @@ static int raw_apply_options(BlockDriverState *bs, BDR= VRawState *s, s->offset =3D offset; s->has_size =3D has_size; s->size =3D has_size ? size : real_size - offset; + s->rdbol =3D rdbol; + + RawDirtyBitmapOpts *opts; + for (int i =3D 0; i < rdbol.n; i++) { + opts =3D &rdbol.opts[i]; + + if (opts->has_persistent && opts->persistent && !opts->has_filenam= e) { + error_setg(errp, "Filename is required for persistent bitmap"); + return -EINVAL; + } + + if (opts->has_filename) { + FILE *fp =3D fopen(opts->filename, "r"); + + raw_load_persistent_dirty_bitmap(bs, fp, opts->granularity, + opts->name, opts->persistent, + opts->disabled, errp); + + if (fp) { + fclose(fp); + } + } + } =20 return 0; } @@ -141,6 +352,7 @@ static int raw_reopen_prepare(BDRVReopenState *reopen_s= tate, { bool has_size; uint64_t offset, size; + RawDirtyBitmapOptsList rdbol =3D {0}; int ret; =20 assert(reopen_state !=3D NULL); @@ -149,13 +361,13 @@ static int raw_reopen_prepare(BDRVReopenState *reopen= _state, reopen_state->opaque =3D g_new0(BDRVRawState, 1); =20 ret =3D raw_read_options(reopen_state->options, &offset, &has_size, &s= ize, - errp); + &rdbol, errp); if (ret < 0) { return ret; } =20 ret =3D raw_apply_options(reopen_state->bs, reopen_state->opaque, - offset, has_size, size, errp); + offset, has_size, size, rdbol, errp); if (ret < 0) { return ret; } @@ -170,12 +382,14 @@ static void raw_reopen_commit(BDRVReopenState *state) =20 memcpy(s, new_s, sizeof(BDRVRawState)); =20 + free_RawDirtyBitmapOptsList(&((BDRVRawState *) state->opaque)->rdbol); g_free(state->opaque); state->opaque =3D NULL; } =20 static void raw_reopen_abort(BDRVReopenState *state) { + free_RawDirtyBitmapOptsList(&((BDRVRawState *) state->opaque)->rdbol); g_free(state->opaque); state->opaque =3D NULL; } @@ -438,10 +652,11 @@ static int raw_open(BlockDriverState *bs, QDict *opti= ons, int flags, BDRVRawState *s =3D bs->opaque; bool has_size; uint64_t offset, size; + RawDirtyBitmapOptsList rdbol =3D {0}; BdrvChildRole file_role; int ret; =20 - ret =3D raw_read_options(options, &offset, &has_size, &size, errp); + ret =3D raw_read_options(options, &offset, &has_size, &size, &rdbol, e= rrp); if (ret < 0) { return ret; } @@ -483,7 +698,7 @@ static int raw_open(BlockDriverState *bs, QDict *option= s, int flags, bs->file->bs->filename); } =20 - ret =3D raw_apply_options(bs, s, offset, has_size, size, errp); + ret =3D raw_apply_options(bs, s, offset, has_size, size, rdbol, errp); if (ret < 0) { return ret; } @@ -496,6 +711,19 @@ static int raw_open(BlockDriverState *bs, QDict *optio= ns, int flags, return 0; } =20 +static void raw_close(BlockDriverState *bs) +{ + Error *local_err =3D NULL; + + raw_store_persistent_dirty_bitmaps(bs, &local_err); + + if (local_err !=3D NULL) { + error_reportf_err(local_err, "Lost persistent bitmaps during " + "inactivation of node '%s': ", + bdrv_get_device_or_node_name(bs)); + } +} + static int raw_probe(const uint8_t *buf, int buf_size, const char *filenam= e) { /* smallest possible positive score so that raw is used if and only if= no @@ -583,6 +811,7 @@ BlockDriver bdrv_raw =3D { .bdrv_reopen_commit =3D &raw_reopen_commit, .bdrv_reopen_abort =3D &raw_reopen_abort, .bdrv_open =3D &raw_open, + .bdrv_close =3D &raw_close, .bdrv_child_perm =3D bdrv_default_perms, .bdrv_co_create_opts =3D &raw_co_create_opts, .bdrv_co_preadv =3D &raw_co_preadv, @@ -608,6 +837,11 @@ BlockDriver bdrv_raw =3D { .bdrv_has_zero_init =3D &raw_has_zero_init, .strong_runtime_opts =3D raw_strong_runtime_opts, .mutable_opts =3D mutable_opts, + .bdrv_co_can_store_new_dirty_bitmap =3D raw_co_can_store_new_dirty_bit= map, + .bdrv_supports_persistent_dirty_bitmap =3D + raw_supports_persistent_dirty_bitmap, + .bdrv_co_remove_persistent_dirty_bitmap =3D + raw_co_remove_persistent_dirty_bitmap, }; =20 static void bdrv_raw_init(void) diff --git a/block/raw-format.h b/block/raw-format.h new file mode 100644 index 0000000000..1ac765255e --- /dev/null +++ b/block/raw-format.h @@ -0,0 +1,50 @@ +#ifndef BLOCK_RAW_FORMAT_H +#define BLOCK_RAW_FORMAT_H + +#include "qemu/coroutine.h" +#include "qemu/osdep.h" + +typedef struct RawDirtyBitmapOpts { + bool has_filename; + char *filename; + bool has_name; + char *name; + bool has_granularity; + uint32_t granularity; + bool has_persistent; + bool persistent; + bool has_disabled; + bool disabled; +} RawDirtyBitmapOpts; + +typedef struct RawDirtyBitmapOptsList { + size_t n; + RawDirtyBitmapOpts *opts; +} RawDirtyBitmapOptsList; + +typedef struct BDRVRawState { + uint64_t offset; + uint64_t size; + bool has_size; + RawDirtyBitmapOptsList rdbol; +} BDRVRawState; + +bool raw_co_can_store_new_dirty_bitmap(BlockDriverState *bs, + const char *name, + uint32_t granularity, + Error **errp); + +bool raw_supports_persistent_dirty_bitmap(BlockDriverState *bs); + +int raw_co_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp); + +void raw_load_persistent_dirty_bitmap(BlockDriverState *bs, FILE *fp, + uint32_t granularity, char* name, + bool persistent, bool disabled, + Error **errp); + +void raw_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp= ); + +#endif --=20 2.31.0 From nobody Mon May 6 01:19:56 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1616233086; cv=none; d=zohomail.com; s=zohoarc; b=Z2V4oXnOUOy3o9ds9yfi4HqKI+2J35mJhnw5kYhFOExnDLzTB837EdaSjBknCPzbImGcnoryII2ay9VGp7NXUV2t5C6eQvEhVqUjfHprSjEJa7xgdju3v114ZwX5u0px5myu2zVBshD+uSUfZ5PvN96K1qYiXmzVKmBkrCOSE60= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616233086; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LOrFaL5skzHDoWoBjlljQTlgSVrL0mw690dXxzPM/wc=; b=fejW1JO3g3meo9pT3t0X9rUKKo16HfO/LPq7HHn2vd60FQPw/r4kPIzzGPteBICwqx0ALgImmfa4QfZGwR4vMZZ8t60DQrq26O1Yli6EJu6m9j6khJFYt9laT+PfvDIl2c0uQQXQZMIExBNfSmc8PZafE8sJG5gSMhJejDaVsUE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1616233086427851.8226854206283; Sat, 20 Mar 2021 02:38:06 -0700 (PDT) Received: from localhost ([::1]:53826 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lNY3Z-0006bh-F3 for importer@patchew.org; Sat, 20 Mar 2021 05:38:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55712) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNXzW-0003t9-7W for qemu-devel@nongnu.org; Sat, 20 Mar 2021 05:33:54 -0400 Received: from mxe2.seznam.cz ([2a02:598:2::34]:48899) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNXzF-0005gZ-7t for qemu-devel@nongnu.org; Sat, 20 Mar 2021 05:33:47 -0400 Received: from email.seznam.cz by email-smtpc15b.ng.seznam.cz (email-smtpc15b.ng.seznam.cz [10.23.14.195]) id 421e225b221105e9442d694c; Sat, 20 Mar 2021 10:33:35 +0100 (CET) Received: from archlinux.localdomain (2a01:510:d502:b200:c1b:ad27:bde0:341a [2a01:510:d502:b200:c1b:ad27:bde0:341a]) by email-relay4.ng.seznam.cz (Seznam SMTPD 1.3.124) with ESMTP; Sat, 20 Mar 2021 10:33:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=emailprofi.seznam.cz; s=beta; t=1616232815; bh=LOrFaL5skzHDoWoBjlljQTlgSVrL0mw690dXxzPM/wc=; h=Received:From:To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To: References:MIME-Version:Content-Type:Content-Transfer-Encoding; b=HWfrvA0xE/LU2zK+ElSvIU1XIEAWIMibqMlv8gBh/bH6xpUOH639dFxXSVWWqg+7O COnWr5gPNXOMfIZoq7aQH7tdIl6DHiZet1dSaz4omAgwgc90h8ImDXSKWnYLsPzu+p tFKDpU5X+m5lEO43Ha1qjuQaxeoembDwc+3SA95o= From: =?UTF-8?q?Patrik=20Janou=C5=A1ek?= To: qemu-devel@nongnu.org Subject: [PATCH 2/2] qapi: implementation of the block-dirty-bitmap-dump command Date: Sat, 20 Mar 2021 10:32:35 +0100 Message-Id: <20210320093235.461485-3-pj@patrikjanousek.cz> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210320093235.461485-1-pj@patrikjanousek.cz> References: <20210320093235.461485-1-pj@patrikjanousek.cz> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:598:2::34; envelope-from=pj@patrikjanousek.cz; helo=mxe2.seznam.cz X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Patrik=20Janou=C5=A1ek?= , lmatejka@kiv.zcu.cz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Currently, dirty bitmaps are for internal use only and there is no support for accessing their content from third party-apps. This patch implements new command block-dirty-bitmap-dump, which returns content of the dirty bitmap encoded in base64. This is very useful especially in combination with a drive that uses raw format because third-party apps can easily use it to create incremental or differential backup. Signed-off-by: Patrik Janou=C5=A1ek --- block/monitor/bitmap-qmp-cmds.c | 61 +++++++++++++++++++++++++++++++ qapi/block-core.json | 64 ++++++++++++++++++++++++++++++++- 2 files changed, 124 insertions(+), 1 deletion(-) diff --git a/block/monitor/bitmap-qmp-cmds.c b/block/monitor/bitmap-qmp-cmd= s.c index 9f11deec64..7f296e9ba7 100644 --- a/block/monitor/bitmap-qmp-cmds.c +++ b/block/monitor/bitmap-qmp-cmds.c @@ -146,6 +146,67 @@ out: aio_context_release(aio_context); } =20 +BlockDirtyBitmapContent *qmp_block_dirty_bitmap_dump(const char *node, + const char *name, + bool has_clear, bool = clear, + Error **errp) +{ + BlockDriverState *bs; + BdrvDirtyBitmap *bitmap; + BlockDirtyBitmapContent *bdbc; + HBitmap *hb; + AioContext *aio_context; + + if (!name || name[0] =3D=3D '\0') { + error_setg(errp, "Bitmap name cannot be empty"); + return NULL; + } + + bs =3D bdrv_lookup_bs(node, node, errp); + if (!bs) { + return NULL; + } + + aio_context =3D bdrv_get_aio_context(bs); + aio_context_acquire(aio_context); + + bitmap =3D block_dirty_bitmap_lookup(node, name, &bs, errp); + if (!bitmap || !bs) { + return NULL; + } + + if (has_clear && clear) { + /** + * Transactions cannot return value, so "clear" functionality must= be + * implemented here while holding AiO context + */ + + bdrv_clear_dirty_bitmap(bitmap, &hb); + + uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); + uint64_t tb_size =3D hbitmap_serialization_size(hb, 0, bm_size); + uint8_t *buf =3D g_malloc(tb_size); + + hbitmap_serialize_part(hb, buf, 0, bm_size); + + bdbc =3D g_new0(BlockDirtyBitmapContent, 1); + bdbc->content =3D g_base64_encode((guchar *) buf, tb_size); + } else { + uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); + uint64_t tb_size =3D bdrv_dirty_bitmap_serialization_size(bitmap, = 0, bm_size); + uint8_t *buf =3D g_malloc(tb_size); + + bdrv_dirty_bitmap_serialize_part(bitmap, buf, 0, bm_size); + + bdbc =3D g_new0(BlockDirtyBitmapContent, 1); + bdbc->content =3D g_base64_encode((guchar *) buf, tb_size); + } + + aio_context_release(aio_context); + + return bdbc; +} + BdrvDirtyBitmap *block_dirty_bitmap_remove(const char *node, const char *n= ame, bool release, BlockDriverState **bitmap_bs, diff --git a/qapi/block-core.json b/qapi/block-core.json index 04ad80bc1e..cbe3dac384 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2031,6 +2031,14 @@ { 'struct': 'BlockDirtyBitmap', 'data': { 'node': 'str', 'name': 'str' } } =20 +## +# @BlockDirtyBitmapContent: +# +# @content: content of dirty bitmap (encoded in base64) +## +{ 'struct': 'BlockDirtyBitmapContent', + 'data': { 'content': 'str' } } + ## # @BlockDirtyBitmapAdd: # @@ -2056,6 +2064,18 @@ 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32', '*persistent': 'bool', '*disabled': 'bool' } } =20 +## +# @BlockDirtyBitmapDump: +# +# @node: name of device/node which the bitmap is tracking +# +# @name: name of the dirty bitmap (must be less than 1024 bytes) +# +# @clear: true if bitmap should be cleared after dump +## +{ 'struct': 'BlockDirtyBitmapDump', + 'data': { 'node': 'str', 'name': 'str', '*clear': 'bool' } } + ## # @BlockDirtyBitmapMergeSource: # @@ -2086,6 +2106,26 @@ 'data': { 'node': 'str', 'target': 'str', 'bitmaps': ['BlockDirtyBitmapMergeSource'] } } =20 +## +# @block-dirty-bitmap-dump: +# +# Dump a dirty bitmap with a name on the node. +# +# Returns: - nothing on success +# - If @node is not a valid block device or node, DeviceNotFound +# - If @name is already taken, GenericError with an explanation +# +# Example: +# +# -> { "execute": "block-dirty-bitmap-dump", +# "arguments": { "node": "drive0", "name": "bitmap0" } } +# <- { "return": { "content": "TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFt... (trunc)= " } } +# +## +{ 'command': 'block-dirty-bitmap-dump', + 'data': 'BlockDirtyBitmapDump', + 'returns': 'BlockDirtyBitmapContent' } + ## # @block-dirty-bitmap-add: # @@ -3908,6 +3948,26 @@ '*x-dirty-bitmap': 'str', '*reconnect-delay': 'uint32' } } =20 +## +# @BlockdevOptionsRawDirtyBitmap: +# +# Dirty bitmap options for the raw driver. +# +# @name: the name of the dirty bitmap (Since 2.4) +# +# @filename: the filename of the dirty bitmap +# +# @granularity: granularity of the dirty bitmap in bytes (since 1.4) +# +# @persistent: true if the bitmap was stored on disk, is scheduled to be s= tored +# on disk, or both. (since 4.0) +# +# @disabled: true if the bitmap should not be loaded (and saved) automatic= ally +## +{ 'struct': 'BlockdevOptionsRawDirtyBitmap', + 'data': {'*name': 'str', 'filename': 'str', 'granularity': 'uint32', + 'persistent': 'bool', '*disabled': 'bool' } } + ## # @BlockdevOptionsRaw: # @@ -3915,12 +3975,14 @@ # # @offset: position where the block device starts # @size: the assumed size of the device +# @dirty-bitmaps: dirty bitmaps of the raw block device # # Since: 2.9 ## { 'struct': 'BlockdevOptionsRaw', 'base': 'BlockdevOptionsGenericFormat', - 'data': { '*offset': 'int', '*size': 'int' } } + 'data': { '*offset': 'int', '*size': 'int' , + '*dirty-bitmaps': ['BlockdevOptionsRawDirtyBitmap'] } } =20 ## # @BlockdevOptionsThrottle: --=20 2.31.0