From nobody Tue Feb 10 17:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1714391650; cv=none; d=zohomail.com; s=zohoarc; b=Uq+gqRAUJzL2rx4NLyQUWinx+UKgIKhjQCPHdA50LhSy6NbZ4NitY72oBXly2jAbzFDFkevnxBX/zlwTQlSl/4etzeY0xTX3Gn2qsSq7J7gaisLodIpBwqtBzjlpRSF4WuXc6TjUqKmryRXMnioCxgvobGMYFI4wZe2SYCnPTnM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1714391650; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AdQfubhmAF0PPkZqpSttOuQyyd9tUV5FqMJvZMix/K0=; b=XaWK5JTxWWSzSmV1XaRpk0K4ZkAHZjwTRA7yKNB9giHpEo6nh+WZmaI7ldqT/k+0JdSJQ0dYwmmnqaH4qsiScMAuja/vwkLcRRxbZ/swXA+YG6bZ80uigtV0S5w4KREPPkRs3s5lGiXX4UBLouBfgJzyS262lEHIXJqllcz7H/M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1714391650826924.635273310411; Mon, 29 Apr 2024 04:54:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s1PYc-0008IH-Kw; Mon, 29 Apr 2024 07:52:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s1PYZ-0008GM-Nm; Mon, 29 Apr 2024 07:52:27 -0400 Received: from forwardcorp1b.mail.yandex.net ([2a02:6b8:c02:900:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s1PYQ-0001ol-GK; Mon, 29 Apr 2024 07:52:27 -0400 Received: from mail-nwsmtp-smtp-corp-main-66.iva.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-66.iva.yp-c.yandex.net [IPv6:2a02:6b8:c0c:2a2a:0:640:d546:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id 82BC260C36; Mon, 29 Apr 2024 14:52:15 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b739::1:30]) by mail-nwsmtp-smtp-corp-main-66.iva.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id 0qOEuD11MuQ0-VcFi89zK; Mon, 29 Apr 2024 14:52:14 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1714391534; bh=AdQfubhmAF0PPkZqpSttOuQyyd9tUV5FqMJvZMix/K0=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=mk46DQS0r2Tjj0VZ8lWQ9B+R+/9RioO3T+c3J2tNu0A615ZXxXT8BJGncsG1un6aU yAcQI5Lps4GfkXtMDcMFgrelIAiwxxpGVJd6ue592peaU9xE5fkcIRZpE/GfoZDOWr 3krL54VeSOsSwWjjFvCvUpxCGpI/J8NeoI93bllQ= Authentication-Results: mail-nwsmtp-smtp-corp-main-66.iva.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, Fiona Ebner , Markus Armbruster , John Snow , Kevin Wolf , Hanna Reitz , Wen Congyang , Xie Changlong , Eric Blake Subject: [PULL 5/6] qapi: blockdev-backup: add discard-source parameter Date: Mon, 29 Apr 2024 14:51:56 +0300 Message-Id: <20240429115157.2260885-6-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240429115157.2260885-1-vsementsov@yandex-team.ru> References: <20240429115157.2260885-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c02:900:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1714391652453100003 Content-Type: text/plain; charset="utf-8" Add a parameter that enables discard-after-copy. That is mostly useful in "push backup with fleecing" scheme, when source is snapshot-access format driver node, based on copy-before-write filter snapshot-access API: [guest] [snapshot-access] ~~ blockdev-backup ~~> [backup target] | | | root | file v v [copy-before-write] | | | file | target v v [active disk] [temp.img] In this case discard-after-copy does two things: - discard data in temp.img to save disk space - avoid further copy-before-write operation in discarded area Note that we have to declare WRITE permission on source in copy-before-write filter, for discard to work. Still we can't take it unconditionally, as it will break normal backup from RO source. So, we have to add a parameter and pass it thorough bdrv_open flags. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner Acked-by: Markus Armbruster Message-Id: <20240313152822.626493-5-vsementsov@yandex-team.ru> Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 5 +++-- block/block-copy.c | 9 +++++++++ block/copy-before-write.c | 15 +++++++++++++-- block/copy-before-write.h | 1 + block/replication.c | 4 ++-- blockdev.c | 2 +- include/block/block-common.h | 2 ++ include/block/block-copy.h | 1 + include/block/block_int-global-state.h | 2 +- qapi/block-core.json | 4 ++++ 10 files changed, 37 insertions(+), 8 deletions(-) diff --git a/block/backup.c b/block/backup.c index ec29d6b810..3dd2e229d2 100644 --- a/block/backup.c +++ b/block/backup.c @@ -356,7 +356,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, BitmapSyncMode bitmap_mode, - bool compress, + bool compress, bool discard_source, const char *filter_node_name, BackupPerf *perf, BlockdevOnError on_source_error, @@ -457,7 +457,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, goto error; } =20 - cbw =3D bdrv_cbw_append(bs, target, filter_node_name, &bcs, errp); + cbw =3D bdrv_cbw_append(bs, target, filter_node_name, discard_source, + &bcs, errp); if (!cbw) { goto error; } diff --git a/block/block-copy.c b/block/block-copy.c index 8fca2c3698..7e3b378528 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -137,6 +137,7 @@ typedef struct BlockCopyState { CoMutex lock; int64_t in_flight_bytes; BlockCopyMethod method; + bool discard_source; BlockReqList reqs; QLIST_HEAD(, BlockCopyCallState) calls; /* @@ -353,6 +354,7 @@ static int64_t block_copy_calculate_cluster_size(BlockD= riverState *target, BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, BlockDriverState *copy_bitmap_bs, const BdrvDirtyBitmap *bitmap, + bool discard_source, Error **errp) { ERRP_GUARD(); @@ -418,6 +420,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source,= BdrvChild *target, cluster_size), }; =20 + s->discard_source =3D discard_source; block_copy_set_copy_opts(s, false, false); =20 ratelimit_init(&s->rate_limit); @@ -589,6 +592,12 @@ static coroutine_fn int block_copy_task_entry(AioTask = *task) co_put_to_shres(s->mem, t->req.bytes); block_copy_task_end(t, ret); =20 + if (s->discard_source && ret =3D=3D 0) { + int64_t nbytes =3D + MIN(t->req.offset + t->req.bytes, s->len) - t->req.offset; + bdrv_co_pdiscard(s->source, t->req.offset, nbytes); + } + return ret; } =20 diff --git a/block/copy-before-write.c b/block/copy-before-write.c index ed2c228da7..cd65524e26 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -44,6 +44,7 @@ typedef struct BDRVCopyBeforeWriteState { BdrvChild *target; OnCbwError on_cbw_error; uint32_t cbw_timeout_ns; + bool discard_source; =20 /* * @lock: protects access to @access_bitmap, @done_bitmap and @@ -357,6 +358,8 @@ cbw_child_perm(BlockDriverState *bs, BdrvChild *c, Bdrv= ChildRole role, uint64_t perm, uint64_t shared, uint64_t *nperm, uint64_t *nshared) { + BDRVCopyBeforeWriteState *s =3D bs->opaque; + if (!(role & BDRV_CHILD_FILTERED)) { /* * Target child @@ -381,6 +384,10 @@ cbw_child_perm(BlockDriverState *bs, BdrvChild *c, Bdr= vChildRole role, * start */ *nperm =3D *nperm | BLK_PERM_CONSISTENT_READ; + if (s->discard_source) { + *nperm =3D *nperm | BLK_PERM_WRITE; + } + *nshared &=3D ~(BLK_PERM_WRITE | BLK_PERM_RESIZE); } } @@ -468,7 +475,9 @@ static int cbw_open(BlockDriverState *bs, QDict *option= s, int flags, ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags); =20 - s->bcs =3D block_copy_state_new(bs->file, s->target, bs, bitmap, errp); + s->discard_source =3D flags & BDRV_O_CBW_DISCARD_SOURCE; + s->bcs =3D block_copy_state_new(bs->file, s->target, bs, bitmap, + flags & BDRV_O_CBW_DISCARD_SOURCE, errp); if (!s->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); return -EINVAL; @@ -535,12 +544,14 @@ static BlockDriver bdrv_cbw_filter =3D { BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, + bool discard_source, BlockCopyState **bcs, Error **errp) { BDRVCopyBeforeWriteState *state; BlockDriverState *top; QDict *opts; + int flags =3D BDRV_O_RDWR | (discard_source ? BDRV_O_CBW_DISCARD_SOURC= E : 0); =20 assert(source->total_sectors =3D=3D target->total_sectors); GLOBAL_STATE_CODE(); @@ -553,7 +564,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *sou= rce, qdict_put_str(opts, "file", bdrv_get_node_name(source)); qdict_put_str(opts, "target", bdrv_get_node_name(target)); =20 - top =3D bdrv_insert_node(source, opts, BDRV_O_RDWR, errp); + top =3D bdrv_insert_node(source, opts, flags, errp); if (!top) { return NULL; } diff --git a/block/copy-before-write.h b/block/copy-before-write.h index 6e72bb25e9..01af0cd3c4 100644 --- a/block/copy-before-write.h +++ b/block/copy-before-write.h @@ -39,6 +39,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, + bool discard_source, BlockCopyState **bcs, Error **errp); void bdrv_cbw_drop(BlockDriverState *bs); diff --git a/block/replication.c b/block/replication.c index ca6bd0a720..0415a5e8b7 100644 --- a/block/replication.c +++ b/block/replication.c @@ -582,8 +582,8 @@ static void replication_start(ReplicationState *rs, Rep= licationMode mode, =20 s->backup_job =3D backup_job_create( NULL, s->secondary_disk->bs, s->hidden_dis= k->bs, - 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, = NULL, - &perf, + 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, = false, + NULL, &perf, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, backup_job_completed, bs, NULL, &local_err= ); diff --git a/blockdev.c b/blockdev.c index 08eccc9052..875a6a4ac1 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2726,7 +2726,7 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, =20 job =3D backup_job_create(backup->job_id, bs, target_bs, backup->speed, backup->sync, bmap, backup->bitmap_mode, - backup->compress, + backup->compress, backup->discard_source, backup->filter_node_name, &perf, backup->on_source_error, diff --git a/include/block/block-common.h b/include/block/block-common.h index a846023a09..338fe5ff7a 100644 --- a/include/block/block-common.h +++ b/include/block/block-common.h @@ -243,6 +243,8 @@ typedef enum { read-write fails */ #define BDRV_O_IO_URING 0x40000 /* use io_uring instead of the thread p= ool */ =20 +#define BDRV_O_CBW_DISCARD_SOURCE 0x80000 /* for copy-before-write filter = */ + #define BDRV_O_CACHE_MASK (BDRV_O_NOCACHE | BDRV_O_NO_FLUSH) =20 =20 diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 8b41643bfa..bdc703bacd 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -27,6 +27,7 @@ typedef struct BlockCopyCallState BlockCopyCallState; BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, BlockDriverState *copy_bitmap_bs, const BdrvDirtyBitmap *bitmap, + bool discard_source, Error **errp); =20 /* Function should be called prior any actual copy request */ diff --git a/include/block/block_int-global-state.h b/include/block/block_i= nt-global-state.h index d2201e27f4..eb2d92a226 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -193,7 +193,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, BitmapSyncMode bitmap_mode, - bool compress, + bool compress, bool discard_source, const char *filter_node_name, BackupPerf *perf, BlockdevOnError on_source_error, diff --git a/qapi/block-core.json b/qapi/block-core.json index 746d1694c2..df5e07debd 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1610,6 +1610,9 @@ # node specified by @drive. If this option is not given, a node # name is autogenerated. (Since: 4.2) # +# @discard-source: Discard blocks on source which have already been +# copied to the target. (Since 9.1) +# # @x-perf: Performance options. (Since 6.0) # # Features: @@ -1631,6 +1634,7 @@ '*on-target-error': 'BlockdevOnError', '*auto-finalize': 'bool', '*auto-dismiss': 'bool', '*filter-node-name': 'str', + '*discard-source': 'bool', '*x-perf': { 'type': 'BackupPerf', 'features': [ 'unstable' ] } } } =20 --=20 2.34.1