From nobody Tue Nov 26 09:51:34 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1709129848; cv=none; d=zohomail.com; s=zohoarc; b=I43XAs6+3kfcTJAQ8VmOTl055nH8uYuWvOFAoqjBKsr5FqQS1csPGvNki+HPid2IWzXdL2HgThQ/0M69fXvsHkPFjvHfsAmYUCzZVIhyzZ9jl+to+YnMLwA4laMkUZtjiaI0TQuXQDIzLX/KFEEo9N+wQDu07wWDQsyjpoy3BWo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709129848; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Dwj8LSWlSbfVFEAeaG2oeScyzQx6/lTEouwrjgOLpHE=; b=jKFMEnRf8t6XAxow6RX4+5HIcI5wiZOatJXMPZdlaHgrxvkDngFBUt7q4Yzg3/Iap72jiBDR9bOT9yZd4pZsLEGXAp414g1NJJXTdBSdHVoSE6GtQ1p8m9WSRiyHigGYildzOaA+t2JV4zImAnpFgo+YddfbpnpTMWmL9w4y1ac= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709129848004973.1134593504773; Wed, 28 Feb 2024 06:17:28 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rfKim-0003y4-TZ; Wed, 28 Feb 2024 09:15:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfKic-0003qO-N1; Wed, 28 Feb 2024 09:15:38 -0500 Received: from forwardcorp1a.mail.yandex.net ([2a02:6b8:c0e:500:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfKiV-0002Gp-IC; Wed, 28 Feb 2024 09:15:34 -0500 Received: from mail-nwsmtp-smtp-corp-main-83.vla.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-83.vla.yp-c.yandex.net [IPv6:2a02:6b8:c18:486:0:640:cf34:0]) by forwardcorp1a.mail.yandex.net (Yandex) with ESMTPS id 0B2426090C; Wed, 28 Feb 2024 17:15:21 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:a522::1:14]) by mail-nwsmtp-smtp-corp-main-83.vla.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id 4FaAWa1Of0U0-wtWc0lXY; Wed, 28 Feb 2024 17:15:20 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1709129720; bh=Dwj8LSWlSbfVFEAeaG2oeScyzQx6/lTEouwrjgOLpHE=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=on5o3PXLOy8deJvJ/BeXROXbjd/nFdGjzDNMW+mldrWSWGNQIvLTJJAQ/yze9h0+x WyEJIcCgIjY+8vPeZb/PYQ76GlKlw08JWZssCbNfiWGlNa873A0hHD9rto7LM2qgfX Iz5uKkeguGi3kKOJM8LCK2UccaHJwgFm5kY9WW44= Authentication-Results: mail-nwsmtp-smtp-corp-main-83.vla.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, armbru@redhat.com, eblake@redhat.com, xiechanglong.d@gmail.com, wencongyang2@huawei.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, f.ebner@proxmox.com, den@virtuozzo.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v3 4/5] qapi: blockdev-backup: add discard-source parameter Date: Wed, 28 Feb 2024 17:15:00 +0300 Message-Id: <20240228141501.455989-5-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240228141501.455989-1-vsementsov@yandex-team.ru> References: <20240228141501.455989-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c0e:500:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1a.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1709129849221100003 Content-Type: text/plain; charset="utf-8" Add a parameter that enables discard-after-copy. That is mostly useful in "push backup with fleecing" scheme, when source is snapshot-access format driver node, based on copy-before-write filter snapshot-access API: [guest] [snapshot-access] ~~ blockdev-backup ~~> [backup target] | | | root | file v v [copy-before-write] | | | file | target v v [active disk] [temp.img] In this case discard-after-copy does two things: - discard data in temp.img to save disk space - avoid further copy-before-write operation in discarded area Note that we have to declare WRITE permission on source in copy-before-write filter, for discard to work. Still we can't take it unconditionally, as it will break normal backup from RO source. So, we have to add a parameter and pass it thorough bdrv_open flags. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Fiona Ebner --- block/backup.c | 5 +++-- block/block-copy.c | 9 +++++++++ block/copy-before-write.c | 15 +++++++++++++-- block/copy-before-write.h | 1 + block/replication.c | 4 ++-- blockdev.c | 2 +- include/block/block-common.h | 2 ++ include/block/block-copy.h | 1 + include/block/block_int-global-state.h | 2 +- qapi/block-core.json | 4 ++++ 10 files changed, 37 insertions(+), 8 deletions(-) diff --git a/block/backup.c b/block/backup.c index ec29d6b810..3dd2e229d2 100644 --- a/block/backup.c +++ b/block/backup.c @@ -356,7 +356,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, BitmapSyncMode bitmap_mode, - bool compress, + bool compress, bool discard_source, const char *filter_node_name, BackupPerf *perf, BlockdevOnError on_source_error, @@ -457,7 +457,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, goto error; } =20 - cbw =3D bdrv_cbw_append(bs, target, filter_node_name, &bcs, errp); + cbw =3D bdrv_cbw_append(bs, target, filter_node_name, discard_source, + &bcs, errp); if (!cbw) { goto error; } diff --git a/block/block-copy.c b/block/block-copy.c index 8fca2c3698..7e3b378528 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -137,6 +137,7 @@ typedef struct BlockCopyState { CoMutex lock; int64_t in_flight_bytes; BlockCopyMethod method; + bool discard_source; BlockReqList reqs; QLIST_HEAD(, BlockCopyCallState) calls; /* @@ -353,6 +354,7 @@ static int64_t block_copy_calculate_cluster_size(BlockD= riverState *target, BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, BlockDriverState *copy_bitmap_bs, const BdrvDirtyBitmap *bitmap, + bool discard_source, Error **errp) { ERRP_GUARD(); @@ -418,6 +420,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source,= BdrvChild *target, cluster_size), }; =20 + s->discard_source =3D discard_source; block_copy_set_copy_opts(s, false, false); =20 ratelimit_init(&s->rate_limit); @@ -589,6 +592,12 @@ static coroutine_fn int block_copy_task_entry(AioTask = *task) co_put_to_shres(s->mem, t->req.bytes); block_copy_task_end(t, ret); =20 + if (s->discard_source && ret =3D=3D 0) { + int64_t nbytes =3D + MIN(t->req.offset + t->req.bytes, s->len) - t->req.offset; + bdrv_co_pdiscard(s->source, t->req.offset, nbytes); + } + return ret; } =20 diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 44a9385aa7..dac57481c5 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -44,6 +44,7 @@ typedef struct BDRVCopyBeforeWriteState { BdrvChild *target; OnCbwError on_cbw_error; uint32_t cbw_timeout_ns; + bool discard_source; =20 /* * @lock: protects access to @access_bitmap, @done_bitmap and @@ -357,6 +358,8 @@ cbw_child_perm(BlockDriverState *bs, BdrvChild *c, Bdrv= ChildRole role, uint64_t perm, uint64_t shared, uint64_t *nperm, uint64_t *nshared) { + BDRVCopyBeforeWriteState *s =3D bs->opaque; + if (!(role & BDRV_CHILD_FILTERED)) { /* * Target child @@ -381,6 +384,10 @@ cbw_child_perm(BlockDriverState *bs, BdrvChild *c, Bdr= vChildRole role, * start */ *nperm =3D *nperm | BLK_PERM_CONSISTENT_READ; + if (s->discard_source) { + *nperm =3D *nperm | BLK_PERM_WRITE; + } + *nshared &=3D ~(BLK_PERM_WRITE | BLK_PERM_RESIZE); } } @@ -467,7 +474,9 @@ static int cbw_open(BlockDriverState *bs, QDict *option= s, int flags, ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags); =20 - s->bcs =3D block_copy_state_new(bs->file, s->target, bs, bitmap, errp); + s->discard_source =3D flags & BDRV_O_CBW_DISCARD_SOURCE; + s->bcs =3D block_copy_state_new(bs->file, s->target, bs, bitmap, + flags & BDRV_O_CBW_DISCARD_SOURCE, errp); if (!s->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); return -EINVAL; @@ -534,12 +543,14 @@ static BlockDriver bdrv_cbw_filter =3D { BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, + bool discard_source, BlockCopyState **bcs, Error **errp) { BDRVCopyBeforeWriteState *state; BlockDriverState *top; QDict *opts; + int flags =3D BDRV_O_RDWR | (discard_source ? BDRV_O_CBW_DISCARD_SOURC= E : 0); =20 assert(source->total_sectors =3D=3D target->total_sectors); GLOBAL_STATE_CODE(); @@ -552,7 +563,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *sou= rce, qdict_put_str(opts, "file", bdrv_get_node_name(source)); qdict_put_str(opts, "target", bdrv_get_node_name(target)); =20 - top =3D bdrv_insert_node(source, opts, BDRV_O_RDWR, errp); + top =3D bdrv_insert_node(source, opts, flags, errp); if (!top) { return NULL; } diff --git a/block/copy-before-write.h b/block/copy-before-write.h index 6e72bb25e9..01af0cd3c4 100644 --- a/block/copy-before-write.h +++ b/block/copy-before-write.h @@ -39,6 +39,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, + bool discard_source, BlockCopyState **bcs, Error **errp); void bdrv_cbw_drop(BlockDriverState *bs); diff --git a/block/replication.c b/block/replication.c index ca6bd0a720..0415a5e8b7 100644 --- a/block/replication.c +++ b/block/replication.c @@ -582,8 +582,8 @@ static void replication_start(ReplicationState *rs, Rep= licationMode mode, =20 s->backup_job =3D backup_job_create( NULL, s->secondary_disk->bs, s->hidden_dis= k->bs, - 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, = NULL, - &perf, + 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, = false, + NULL, &perf, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, backup_job_completed, bs, NULL, &local_err= ); diff --git a/blockdev.c b/blockdev.c index f8bb0932f8..daceb50460 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2726,7 +2726,7 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, =20 job =3D backup_job_create(backup->job_id, bs, target_bs, backup->speed, backup->sync, bmap, backup->bitmap_mode, - backup->compress, + backup->compress, backup->discard_source, backup->filter_node_name, &perf, backup->on_source_error, diff --git a/include/block/block-common.h b/include/block/block-common.h index a846023a09..338fe5ff7a 100644 --- a/include/block/block-common.h +++ b/include/block/block-common.h @@ -243,6 +243,8 @@ typedef enum { read-write fails */ #define BDRV_O_IO_URING 0x40000 /* use io_uring instead of the thread p= ool */ =20 +#define BDRV_O_CBW_DISCARD_SOURCE 0x80000 /* for copy-before-write filter = */ + #define BDRV_O_CACHE_MASK (BDRV_O_NOCACHE | BDRV_O_NO_FLUSH) =20 =20 diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 8b41643bfa..bdc703bacd 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -27,6 +27,7 @@ typedef struct BlockCopyCallState BlockCopyCallState; BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, BlockDriverState *copy_bitmap_bs, const BdrvDirtyBitmap *bitmap, + bool discard_source, Error **errp); =20 /* Function should be called prior any actual copy request */ diff --git a/include/block/block_int-global-state.h b/include/block/block_i= nt-global-state.h index d2201e27f4..eb2d92a226 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -193,7 +193,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, BitmapSyncMode bitmap_mode, - bool compress, + bool compress, bool discard_source, const char *filter_node_name, BackupPerf *perf, BlockdevOnError on_source_error, diff --git a/qapi/block-core.json b/qapi/block-core.json index 22b8634422..9cbe3988b4 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1611,6 +1611,9 @@ # node specified by @drive. If this option is not given, a node # name is autogenerated. (Since: 4.2) # +# @discard-source: Discard blocks on source which are already copied +# to the target. (Since 9.0) +# # @x-perf: Performance options. (Since 6.0) # # Features: @@ -1632,6 +1635,7 @@ '*on-target-error': 'BlockdevOnError', '*auto-finalize': 'bool', '*auto-dismiss': 'bool', '*filter-node-name': 'str', + '*discard-source': 'bool', '*x-perf': { 'type': 'BackupPerf', 'features': [ 'unstable' ] } } } =20 --=20 2.34.1