From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926586; cv=none; d=zohomail.com; s=zohoarc; b=Xu4+oJY0JRgLK3QdtBtiOpsTz+xxQC7Qk6ypYt9zlY+MPYD3Xth/UfsbBAap5H6GOVk+6FIMak/8JJT6uSw/nRNw0T30eYG/TgmvbcgBNhCK3yn0MiY069zplCWOOWxP9DjDZFVc9lTJ3Ny+afE/s7bx3O5OqHggqWgEPYYgI64= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926586; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5oseaxkYuuPVOra7hdGoqyPtzdfbXPYktFbnwhW/LzQ=; b=AuQePaQWwDVPfpHakMj3d7jG0kUd3Tc89LGjao0QyMj+UImA4pMasLoxyyhZyXXKeTv1ice6uf3NMrzAQnP+9GzQYIaaL0ESXq36CV9xUTt0i/HjYlkxHhzPWhe3IGaKkdHRfFR7ed5sEbKxapVNEmqE8q+xSKKo1sHNAa8N2QQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926585848639.9171779002236; Tue, 29 Apr 2025 04:36:25 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDr-0007AG-DD; Tue, 29 Apr 2025 07:33:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDh-0006vG-LJ; Tue, 29 Apr 2025 07:33:49 -0400 Received: from forwardcorp1d.mail.yandex.net ([178.154.239.200]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDd-0008Eg-G6; Tue, 29 Apr 2025 07:33:49 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id BC2336139C; Tue, 29 Apr 2025 14:33:38 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-et8shCl8; Tue, 29 Apr 2025 14:33:38 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926418; bh=5oseaxkYuuPVOra7hdGoqyPtzdfbXPYktFbnwhW/LzQ=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=Vhfbbb94yRIIRPX+ZZx/nsvF3UXTeFPyyufiRRhkmnPWZYySwwkd0/JJsQNP2xREy n1jTIf/xGRBYPxpQJ6xot+gyhZ1TNlXppFvOQ0gpc1tebmOJ+nKWujq/KGk/bCFySG BB/BXMqKS/p+x36OJMMMifoR17dxgkKMrDKr981o= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Vincent Vanlaer , Vladimir Sementsov-Ogievskiy Subject: [PULL 1/8] block: get type of block allocation in commit_run Date: Tue, 29 Apr 2025 14:33:23 +0300 Message-ID: <20250429113335.423535-2-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.200; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926588482019100 Content-Type: text/plain; charset="utf-8" From: Vincent Vanlaer bdrv_co_common_block_status_above not only returns whether the block is allocated, but also if it contains zeroes. Signed-off-by: Vincent Vanlaer Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <20241026163010.2865002-2-libvirt-e6954efa@volkihar.be> Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/commit.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/block/commit.c b/block/commit.c index 5df3d05346..ba0ba59316 100644 --- a/block/commit.c +++ b/block/commit.c @@ -15,6 +15,8 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" #include "trace.h" +#include "block/block-common.h" +#include "block/coroutines.h" #include "block/block_int.h" #include "block/blockjob_int.h" #include "qapi/error.h" @@ -167,9 +169,13 @@ static int coroutine_fn commit_run(Job *job, Error **e= rrp) break; } /* Copy if allocated above the base */ - ret =3D blk_co_is_allocated_above(s->top, s->base_overlay, true, - offset, COMMIT_BUFFER_SIZE, &n); - copy =3D (ret > 0); + WITH_GRAPH_RDLOCK_GUARD() { + ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), + s->base_overlay, true, true, offset, COMMIT_BUFFER_SIZE, + &n, NULL, NULL, NULL); + } + + copy =3D (ret >=3D 0 && ret & BDRV_BLOCK_ALLOCATED); trace_commit_one_iteration(s, offset, n, ret); if (copy) { assert(n < SIZE_MAX); --=20 2.48.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926488; cv=none; d=zohomail.com; s=zohoarc; b=n1xvmhUeTTDUoTotYGY1z6eUrcweFFk1KaNixAjwMHUvKDYyam8vB2i/1z9oxqsZAlOQnKT11Lh9/2FNjWIMNkC4T+NfH3j8PchlRu8PpSwLkt5S3hRj/vaF1YfU7P+RGVsDqEOdPexj1SN25gwaeb9/dTS5meecS+3QiywDUXc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926488; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CA0ozR2MoFs8iqJMkGwT6szoTB2uzbt1omtnLOIN2UU=; b=U49IBnKJjH87P1Nd2lYe/jQZyRMqWwJajvTglUy3bsLeQy0iC2W0KqHA69SHS8vBtMew3Dlax4f+Tc8gOadtH7VwJDQDsNhV55hYQuZY9Us3xMW0spXX1hNA2X3yglRulK7akWndX8xSLBCTu+2CgBuCB4+ihtQMfLWBP66L/gI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926487934466.83852545273646; Tue, 29 Apr 2025 04:34:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDw-0007Eg-Mj; Tue, 29 Apr 2025 07:34:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDi-0006vk-7X; Tue, 29 Apr 2025 07:33:50 -0400 Received: from forwardcorp1d.mail.yandex.net ([178.154.239.200]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDd-0008Et-Ef; Tue, 29 Apr 2025 07:33:49 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 81F94613AF; Tue, 29 Apr 2025 14:33:39 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-UX2nbXB5; Tue, 29 Apr 2025 14:33:38 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926418; bh=CA0ozR2MoFs8iqJMkGwT6szoTB2uzbt1omtnLOIN2UU=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=pFJ+W0hUNrFS2TV9xssxVvrMh+7rvXTlCCDhbDNnH2ivbwvsS81rQ1Kr0tzLapk+L /bWb0S0V98w68gVtmuMPv4fCFO9fsmLo7MhZ4dxKAr60vPvyW1iijbSmWIBMSl3Ibr WBk4Y3gqUUn3CFtN+11amRM92aUEkjGHUOtHmxkQ= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Fiona Ebner , Vladimir Sementsov-Ogievskiy , Markus Armbruster Subject: [PULL 1/5] copy-before-write: allow specifying minimum cluster size Date: Tue, 29 Apr 2025 14:33:24 +0300 Message-ID: <20250429113335.423535-3-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.200; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926490449019100 Content-Type: text/plain; charset="utf-8" From: Fiona Ebner In the context of backup fleecing, discarding the source will not work when the fleecing image has a larger granularity than the one used for block-copy operations (can happen if the backup target has smaller cluster size), because cbw_co_pdiscard_snapshot() will align down the discard requests and thus effectively ignore then. To make @discard-source work in such a scenario, allow specifying the minimum cluster size used for block-copy operations and thus in particular also the granularity for discard requests to the source. The type 'size' (corresponding to uint64_t in C) is used in QAPI to rule out negative inputs and for consistency with already existing @cluster-size parameters. Since block_copy_calculate_cluster_size() uses int64_t for its result, a check that the input is not too large is added in block_copy_state_new() before calling it. The calculation in block_copy_calculate_cluster_size() is done in the target int64_t type. Suggested-by: Vladimir Sementsov-Ogievskiy Acked-by: Markus Armbruster (QAPI schema) Signed-off-by: Fiona Ebner Message-Id: <20240711120915.310243-2-f.ebner@proxmox.com> [vsementsov: switch version to 9.2 in QAPI doc] Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 36 ++++++++++++++++++++++++++---------- block/copy-before-write.c | 5 ++++- include/block/block-copy.h | 1 + qapi/block-core.json | 8 +++++++- 4 files changed, 38 insertions(+), 12 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index cc618e4561..93eb1b2664 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -310,6 +310,7 @@ void block_copy_set_copy_opts(BlockCopyState *s, bool u= se_copy_range, } =20 static int64_t block_copy_calculate_cluster_size(BlockDriverState *target, + int64_t min_cluster_size, Error **errp) { int ret; @@ -319,6 +320,9 @@ static int64_t block_copy_calculate_cluster_size(BlockD= riverState *target, GLOBAL_STATE_CODE(); GRAPH_RDLOCK_GUARD_MAINLOOP(); =20 + min_cluster_size =3D MAX(min_cluster_size, + (int64_t)BLOCK_COPY_CLUSTER_SIZE_DEFAULT); + target_does_cow =3D bdrv_backing_chain_next(target); =20 /* @@ -329,13 +333,13 @@ static int64_t block_copy_calculate_cluster_size(Bloc= kDriverState *target, ret =3D bdrv_get_info(target, &bdi); if (ret =3D=3D -ENOTSUP && !target_does_cow) { /* Cluster size is not defined */ - warn_report("The target block device doesn't provide " - "information about the block size and it doesn't have = a " - "backing file. The default block size of %u bytes is " - "used. If the actual block size of the target exceeds " - "this default, the backup may be unusable", - BLOCK_COPY_CLUSTER_SIZE_DEFAULT); - return BLOCK_COPY_CLUSTER_SIZE_DEFAULT; + warn_report("The target block device doesn't provide information a= bout " + "the block size and it doesn't have a backing file. Th= e " + "(default) block size of %" PRIi64 " bytes is used. If= the " + "actual block size of the target exceeds this value, t= he " + "backup may be unusable", + min_cluster_size); + return min_cluster_size; } else if (ret < 0 && !target_does_cow) { error_setg_errno(errp, -ret, "Couldn't determine the cluster size of the target image, " @@ -345,16 +349,17 @@ static int64_t block_copy_calculate_cluster_size(Bloc= kDriverState *target, return ret; } else if (ret < 0 && target_does_cow) { /* Not fatal; just trudge on ahead. */ - return BLOCK_COPY_CLUSTER_SIZE_DEFAULT; + return min_cluster_size; } =20 - return MAX(BLOCK_COPY_CLUSTER_SIZE_DEFAULT, bdi.cluster_size); + return MAX(min_cluster_size, bdi.cluster_size); } =20 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, BlockDriverState *copy_bitmap_bs, const BdrvDirtyBitmap *bitmap, bool discard_source, + uint64_t min_cluster_size, Error **errp) { ERRP_GUARD(); @@ -365,7 +370,18 @@ BlockCopyState *block_copy_state_new(BdrvChild *source= , BdrvChild *target, =20 GLOBAL_STATE_CODE(); =20 - cluster_size =3D block_copy_calculate_cluster_size(target->bs, errp); + if (min_cluster_size > INT64_MAX) { + error_setg(errp, "min-cluster-size too large: %" PRIu64 " > %" PRI= i64, + min_cluster_size, INT64_MAX); + return NULL; + } else if (min_cluster_size && !is_power_of_2(min_cluster_size)) { + error_setg(errp, "min-cluster-size needs to be a power of 2"); + return NULL; + } + + cluster_size =3D block_copy_calculate_cluster_size(target->bs, + (int64_t)min_cluster_= size, + errp); if (cluster_size < 0) { return NULL; } diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 853e01a1eb..a919b1f41b 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -417,6 +417,7 @@ static BlockdevOptions *cbw_parse_options(QDict *option= s, Error **errp) qdict_extract_subqdict(options, NULL, "bitmap"); qdict_del(options, "on-cbw-error"); qdict_del(options, "cbw-timeout"); + qdict_del(options, "min-cluster-size"); =20 out: visit_free(v); @@ -476,8 +477,10 @@ static int cbw_open(BlockDriverState *bs, QDict *optio= ns, int flags, bs->file->bs->supported_zero_flags); =20 s->discard_source =3D flags & BDRV_O_CBW_DISCARD_SOURCE; + s->bcs =3D block_copy_state_new(bs->file, s->target, bs, bitmap, - flags & BDRV_O_CBW_DISCARD_SOURCE, errp); + flags & BDRV_O_CBW_DISCARD_SOURCE, + opts->min_cluster_size, errp); if (!s->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); return -EINVAL; diff --git a/include/block/block-copy.h b/include/block/block-copy.h index bdc703bacd..dd5cc82f3b 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -28,6 +28,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, B= drvChild *target, BlockDriverState *copy_bitmap_bs, const BdrvDirtyBitmap *bitmap, bool discard_source, + uint64_t min_cluster_size, Error **errp); =20 /* Function should be called prior any actual copy request */ diff --git a/qapi/block-core.json b/qapi/block-core.json index 9f6dd59298..6751022428 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -4639,12 +4639,18 @@ # @on-cbw-error parameter will decide how this failure is handled. # Default 0. (Since 7.1) # +# @min-cluster-size: Minimum size of blocks used by copy-before-write +# operations. Has to be a power of 2. No effect if smaller than +# the maximum of the target's cluster size and 64 KiB. Default 0. +# (Since 9.2) +# # Since: 6.2 ## { 'struct': 'BlockdevOptionsCbw', 'base': 'BlockdevOptionsGenericFormat', 'data': { 'target': 'BlockdevRef', '*bitmap': 'BlockDirtyBitmap', - '*on-cbw-error': 'OnCbwError', '*cbw-timeout': 'uint32' } } + '*on-cbw-error': 'OnCbwError', '*cbw-timeout': 'uint32', + '*min-cluster-size': 'size' } } =20 ## # @BlockdevOptions: --=20 2.34.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926492; cv=none; d=zohomail.com; s=zohoarc; b=nhLGtz0Tic4LvHya6+PXbiJrLRgUWSL6psVRYN+uTTmHRlmUmY0r8JnjKwQ5nFxoWZgh5fOLD3v6zytFoGK7JO9RkODmqEOqBcBoQulA+Q/lr87mNNH/AAhoPAY095CqoR0yZHm8MT7kl7LF1t72vjr1Kig+llbOE9b6+/9eR3g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926492; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=XKka9WznzaFH71i9QUb+uGa+QdEj4xDm/kPV4IEvdII=; b=AUNjgeZH26nBnACP1vyrjRqoV7PoQrXRb8u9Ov1D6T70/Qh2iiAn+1adU0WiWpDcrouoBrnUaorFvuFSpN6trL10zIM1hMy7IQmTPyK48hWdkfoKumua4Ouzqaah+1ZtDKzQlmmiYk+PqIpaFMMXI1CjQxU9HtCX5bStObLGgdI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926492407641.7610176509766; Tue, 29 Apr 2025 04:34:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDm-00072P-4s; Tue, 29 Apr 2025 07:33:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDg-0006sd-5V; Tue, 29 Apr 2025 07:33:48 -0400 Received: from forwardcorp1d.mail.yandex.net ([178.154.239.200]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDd-0008F6-9P; Tue, 29 Apr 2025 07:33:47 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 94774613CD; Tue, 29 Apr 2025 14:33:40 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-S9PIonOc; Tue, 29 Apr 2025 14:33:40 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926420; bh=XKka9WznzaFH71i9QUb+uGa+QdEj4xDm/kPV4IEvdII=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=SU16yhvdzlYnQLXOJcWWoDDJbL/IH7ARdSgk5a5ZBOw7N0xWQFKA0mdrkoPI7c9I1 xcYMFaFSWU1RTiTlkz8HbV2YiPBIDPL+F27Hgq5g/gef5WOaItYmwZ5UvA3ybEhsWZ Jwz0gP8NnjpeJJkxdb9npV4AYQvXDbtZQUMdxH0U= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Vincent Vanlaer , Vladimir Sementsov-Ogievskiy Subject: [PULL 2/8] block: move commit_run loop to separate function Date: Tue, 29 Apr 2025 14:33:26 +0300 Message-ID: <20250429113335.423535-5-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.200; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926495297019000 Content-Type: text/plain; charset="utf-8" From: Vincent Vanlaer Signed-off-by: Vincent Vanlaer Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <20241026163010.2865002-3-libvirt-e6954efa@volkihar.be> Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/commit.c | 89 +++++++++++++++++++++++++++++--------------------- 1 file changed, 52 insertions(+), 37 deletions(-) diff --git a/block/commit.c b/block/commit.c index ba0ba59316..3ee0ade7df 100644 --- a/block/commit.c +++ b/block/commit.c @@ -128,6 +128,55 @@ static void commit_clean(Job *job) blk_unref(s->top); } =20 +static int commit_iteration(CommitBlockJob *s, int64_t offset, + int64_t *n, void *buf) +{ + int ret =3D 0; + bool copy; + bool error_in_source =3D true; + + /* Copy if allocated above the base */ + WITH_GRAPH_RDLOCK_GUARD() { + ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), + s->base_overlay, true, true, offset, COMMIT_BUFFER_SIZE, + n, NULL, NULL, NULL); + } + + copy =3D (ret >=3D 0 && ret & BDRV_BLOCK_ALLOCATED); + trace_commit_one_iteration(s, offset, *n, ret); + if (copy) { + assert(*n < SIZE_MAX); + + ret =3D blk_co_pread(s->top, offset, *n, buf, 0); + if (ret >=3D 0) { + ret =3D blk_co_pwrite(s->base, offset, *n, buf, 0); + if (ret < 0) { + error_in_source =3D false; + } + } + } + if (ret < 0) { + BlockErrorAction action =3D block_job_error_action(&s->common, + s->on_error, + error_in_source, + -ret); + if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { + return ret; + } else { + *n =3D 0; + return 0; + } + } + /* Publish progress */ + job_progress_update(&s->common.job, *n); + + if (copy) { + block_job_ratelimit_processed_bytes(&s->common, *n); + } + + return 0; +} + static int coroutine_fn commit_run(Job *job, Error **errp) { CommitBlockJob *s =3D container_of(job, CommitBlockJob, common.job); @@ -158,9 +207,6 @@ static int coroutine_fn commit_run(Job *job, Error **er= rp) buf =3D blk_blockalign(s->top, COMMIT_BUFFER_SIZE); =20 for (offset =3D 0; offset < len; offset +=3D n) { - bool copy; - bool error_in_source =3D true; - /* Note that even when no rate limit is applied we need to yield * with no pending I/O here so that bdrv_drain_all() returns. */ @@ -168,42 +214,11 @@ static int coroutine_fn commit_run(Job *job, Error **= errp) if (job_is_cancelled(&s->common.job)) { break; } - /* Copy if allocated above the base */ - WITH_GRAPH_RDLOCK_GUARD() { - ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), - s->base_overlay, true, true, offset, COMMIT_BUFFER_SIZE, - &n, NULL, NULL, NULL); - } =20 - copy =3D (ret >=3D 0 && ret & BDRV_BLOCK_ALLOCATED); - trace_commit_one_iteration(s, offset, n, ret); - if (copy) { - assert(n < SIZE_MAX); - - ret =3D blk_co_pread(s->top, offset, n, buf, 0); - if (ret >=3D 0) { - ret =3D blk_co_pwrite(s->base, offset, n, buf, 0); - if (ret < 0) { - error_in_source =3D false; - } - } - } - if (ret < 0) { - BlockErrorAction action =3D - block_job_error_action(&s->common, s->on_error, - error_in_source, -ret); - if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { - return ret; - } else { - n =3D 0; - continue; - } - } - /* Publish progress */ - job_progress_update(&s->common.job, n); + ret =3D commit_iteration(s, offset, &n, buf); =20 - if (copy) { - block_job_ratelimit_processed_bytes(&s->common, n); + if (ret < 0) { + return ret; } } =20 --=20 2.48.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926573; cv=none; d=zohomail.com; s=zohoarc; b=eI71Tq9gzTHiWQiPBRg1blUvPySCkcJb7U1wFVsu2l3Bq4ZsWy/oIcqv4tO4Ib0eV9Fdtm7gb4jZsURm1h+RPCh/ajJtuYmZJmFwKbpvuE7kiNgCztX74VGkkHGMV7qWDAxuvSR/i/SHDH5vwXjdMAZ4VJ3kYnABVGexQE0t/nY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926573; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=A0n8HyYvSwxkRyzh1ThtysCFIAcf3fFl7IdICB5q2i0=; b=dn3AF4DPaS8MEU4t63s9zl7QXW7DzoDU7E7AdcPsd6ydlw6kXOulboQbQEMKw1uBjlvORPxBC+bs583AmlDUQ06uGxGbY5G8+x1B+ChSc9t24V/dQ82Z8Fsm0N/UrsmJOzpvxwOmrq96JgYufinoXJesSCoEfWH5lQ4y8JdAmqg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926573551697.418133421124; Tue, 29 Apr 2025 04:36:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDn-000750-Lb; Tue, 29 Apr 2025 07:33:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDi-0006vg-1w; Tue, 29 Apr 2025 07:33:50 -0400 Received: from forwardcorp1d.mail.yandex.net ([178.154.239.200]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDd-0008Ey-Em; Tue, 29 Apr 2025 07:33:49 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 21831613BD; Tue, 29 Apr 2025 14:33:40 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-LeCo85cr; Tue, 29 Apr 2025 14:33:39 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926419; bh=A0n8HyYvSwxkRyzh1ThtysCFIAcf3fFl7IdICB5q2i0=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=xjI4iZKNIPAr7m8UGHTTgHDB56NzNbkbdwcGxW5IdLF4jdkS4NCfvhliZ7AJNMyoU G2KeoG/o6EPvIIjO+hnbZXktHt/p3wjMCvmry9W0FX749UJqfrcF5/wsgRWQHssist DLvM0x8+95hSCJutUaLKKAXLXy3KFQUHzlP96IjI= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Fiona Ebner , Vladimir Sementsov-Ogievskiy , Markus Armbruster Subject: [PULL 2/5] backup: add minimum cluster size to performance options Date: Tue, 29 Apr 2025 14:33:25 +0300 Message-ID: <20250429113335.423535-4-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.200; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926575290019000 Content-Type: text/plain; charset="utf-8" From: Fiona Ebner In the context of backup fleecing, discarding the source will not work when the fleecing image has a larger granularity than the one used for block-copy operations (can happen if the backup target has smaller cluster size), because cbw_co_pdiscard_snapshot() will align down the discard requests and thus effectively ignore then. To make @discard-source work in such a scenario, allow specifying the minimum cluster size used for block-copy operations and thus in particular also the granularity for discard requests to the source. Suggested-by: Vladimir Sementsov-Ogievskiy Acked-by: Markus Armbruster (QAPI schema) Signed-off-by: Fiona Ebner Message-Id: <20240711120915.310243-3-f.ebner@proxmox.com> [vsementsov: switch version to 9.2 in QAPI doc] Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 2 +- block/copy-before-write.c | 9 +++++++++ block/copy-before-write.h | 1 + blockdev.c | 3 +++ qapi/block-core.json | 9 +++++++-- 5 files changed, 21 insertions(+), 3 deletions(-) diff --git a/block/backup.c b/block/backup.c index 3dd2e229d2..a1292c01ec 100644 --- a/block/backup.c +++ b/block/backup.c @@ -458,7 +458,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, } =20 cbw =3D bdrv_cbw_append(bs, target, filter_node_name, discard_source, - &bcs, errp); + perf->min_cluster_size, &bcs, errp); if (!cbw) { goto error; } diff --git a/block/copy-before-write.c b/block/copy-before-write.c index a919b1f41b..e835987e52 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -548,6 +548,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *sou= rce, BlockDriverState *target, const char *filter_node_name, bool discard_source, + uint64_t min_cluster_size, BlockCopyState **bcs, Error **errp) { @@ -567,6 +568,14 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *so= urce, qdict_put_str(opts, "file", bdrv_get_node_name(source)); qdict_put_str(opts, "target", bdrv_get_node_name(target)); =20 + if (min_cluster_size > INT64_MAX) { + error_setg(errp, "min-cluster-size too large: %" PRIu64 " > %" PRI= i64, + min_cluster_size, INT64_MAX); + qobject_unref(opts); + return NULL; + } + qdict_put_int(opts, "min-cluster-size", (int64_t)min_cluster_size); + top =3D bdrv_insert_node(source, opts, flags, errp); if (!top) { return NULL; diff --git a/block/copy-before-write.h b/block/copy-before-write.h index 01af0cd3c4..2a5d4ba693 100644 --- a/block/copy-before-write.h +++ b/block/copy-before-write.h @@ -40,6 +40,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *sourc= e, BlockDriverState *target, const char *filter_node_name, bool discard_source, + uint64_t min_cluster_size, BlockCopyState **bcs, Error **errp); void bdrv_cbw_drop(BlockDriverState *bs); diff --git a/blockdev.c b/blockdev.c index 835064ed03..6740663fda 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2655,6 +2655,9 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, if (backup->x_perf->has_max_chunk) { perf.max_chunk =3D backup->x_perf->max_chunk; } + if (backup->x_perf->has_min_cluster_size) { + perf.min_cluster_size =3D backup->x_perf->min_cluster_size; + } } =20 if ((backup->sync =3D=3D MIRROR_SYNC_MODE_BITMAP) || diff --git a/qapi/block-core.json b/qapi/block-core.json index 6751022428..c3b0a2376b 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1551,11 +1551,16 @@ # it should not be less than job cluster size which is calculated # as maximum of target image cluster size and 64k. Default 0. # +# @min-cluster-size: Minimum size of blocks used by copy-before-write +# and background copy operations. Has to be a power of 2. No +# effect if smaller than the maximum of the target's cluster size +# and 64 KiB. Default 0. (Since 9.2) +# # Since: 6.0 ## { 'struct': 'BackupPerf', - 'data': { '*use-copy-range': 'bool', - '*max-workers': 'int', '*max-chunk': 'int64' } } + 'data': { '*use-copy-range': 'bool', '*max-workers': 'int', + '*max-chunk': 'int64', '*min-cluster-size': 'size' } } =20 ## # @BackupCommon: --=20 2.34.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926586; cv=none; d=zohomail.com; s=zohoarc; b=Fq1gR4K+KDip/aTEmmIU4q0Q/rvFH8PSrKnkqqlROmL3IhXNzajJB0F0utbimznXBnbfnJ/kKcIKLZuVc79BYLNNrs7EfplWFO7VNvszCqWOtdhNfpJHucJsbX9xQcVHj+4jDJp/JzS+UwPYR11PvU513MFHTVmqEP1sqNiCl3Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926586; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xkLZ1GZTOtbVQt+osR2ziS0pLuj2G7hrkNYOXSXAPIU=; b=g4MK3dGF1DZpNjm/3+jQw00IEB6sz26xFOkkuljxaksG8QGeA4r4/rElS5U8uicLcd/y6jCbZxdkeQF3du5Etpv8JtCE9/GroSgG/I6uzvSrHYvMQUQeEBex+3jVb55zojaV8V9kUDLbwW1Jf5XR389HT2a+5vb3iLen7Pj0AxA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926585962673.1223042422808; Tue, 29 Apr 2025 04:36:25 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDx-0007G9-1S; Tue, 29 Apr 2025 07:34:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDj-0006wx-Py; Tue, 29 Apr 2025 07:33:52 -0400 Received: from forwardcorp1d.mail.yandex.net ([178.154.239.200]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDe-0008FT-7Z; Tue, 29 Apr 2025 07:33:51 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id A3A1C613D9; Tue, 29 Apr 2025 14:33:41 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-EKYKT2em; Tue, 29 Apr 2025 14:33:41 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926421; bh=xkLZ1GZTOtbVQt+osR2ziS0pLuj2G7hrkNYOXSXAPIU=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=Fzp22hLmF6e9sWVdT93i+PxIw/x4FZP033Gw0a+8mTSPxLBvhWZPsoZwZasTxgMnW vD2aX0qxYhaxShfEwgxRcCcCdnvaIK34tJrOPmo4/g7jqMnyGIxE03pTWTgJ0RddOo 8QvA+zuD7INGa8b8Vm+H4Kp9bjaOC1F3w6AndIBE= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Fiona Ebner , qemu-stable@nongnu.org, Vladimir Sementsov-Ogievskiy Subject: [PULL 3/5] block/reqlist: allow adding overlapping requests Date: Tue, 29 Apr 2025 14:33:28 +0300 Message-ID: <20250429113335.423535-7-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.200; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926588431019100 Content-Type: text/plain; charset="utf-8" From: Fiona Ebner Allow overlapping request by removing the assert that made it impossible. There are only two callers: 1. block_copy_task_create() It already asserts the very same condition before calling reqlist_init_req(). 2. cbw_snapshot_read_lock() There is no need to have read requests be non-overlapping in copy-before-write when used for snapshot-access. In fact, there was no protection against two callers of cbw_snapshot_read_lock() calling reqlist_init_req() with overlapping ranges and this could lead to an assertion failure [1]. In particular, with the reproducer script below [0], two cbw_co_snapshot_block_status() callers could race, with the second calling reqlist_init_req() before the first one finishes and removes its conflicting request. [0]: > #!/bin/bash -e > dd if=3D/dev/urandom of=3D/tmp/disk.raw bs=3D1M count=3D1024 > ./qemu-img create /tmp/fleecing.raw -f raw 1G > ( > ./qemu-system-x86_64 --qmp stdio \ > --blockdev raw,node-name=3Dnode0,file.driver=3Dfile,file.filename=3D/tmp/= disk.raw \ > --blockdev raw,node-name=3Dnode1,file.driver=3Dfile,file.filename=3D/tmp/= fleecing.raw \ > < {"execute": "qmp_capabilities"} > {"execute": "blockdev-add", "arguments": { "driver": "copy-before-write",= "file": "node0", "target": "node1", "node-name": "node3" } } > {"execute": "blockdev-add", "arguments": { "driver": "snapshot-access", "= file": "node3", "node-name": "snap0" } } > {"execute": "nbd-server-start", "arguments": {"addr": { "type": "unix", "= data": { "path": "/tmp/nbd.socket" } } } } > {"execute": "block-export-add", "arguments": {"id": "exp0", "node-name": = "snap0", "type": "nbd", "name": "exp0"}} > EOF > ) & > sleep 5 > while true; do > ./qemu-nbd -d /dev/nbd0 > ./qemu-nbd -c /dev/nbd0 nbd:unix:/tmp/nbd.socket:exportname=3Dexp0 -f raw= -r > nbdinfo --map 'nbd+unix:///exp0?socket=3D/tmp/nbd.socket' > done [1]: > #5 0x000071e5f0088eb2 in __GI___assert_fail (...) at ./assert/assert.c:1= 01 > #6 0x0000615285438017 in reqlist_init_req (...) at ../block/reqlist.c:23 > #7 0x00006152853e2d98 in cbw_snapshot_read_lock (...) at ../block/copy-b= efore-write.c:237 > #8 0x00006152853e3068 in cbw_co_snapshot_block_status (...) at ../block/= copy-before-write.c:304 > #9 0x00006152853f4d22 in bdrv_co_snapshot_block_status (...) at ../block= /io.c:3726 > #10 0x000061528543a63e in snapshot_access_co_block_status (...) at ../blo= ck/snapshot-access.c:48 > #11 0x00006152853f1a0a in bdrv_co_do_block_status (...) at ../block/io.c:= 2474 > #12 0x00006152853f2016 in bdrv_co_common_block_status_above (...) at ../b= lock/io.c:2652 > #13 0x00006152853f22cf in bdrv_co_block_status_above (...) at ../block/io= .c:2732 > #14 0x00006152853d9a86 in blk_co_block_status_above (...) at ../block/blo= ck-backend.c:1473 > #15 0x000061528538da6c in blockstatus_to_extents (...) at ../nbd/server.c= :2374 > #16 0x000061528538deb1 in nbd_co_send_block_status (...) at ../nbd/server= .c:2481 > #17 0x000061528538f424 in nbd_handle_request (...) at ../nbd/server.c:2978 > #18 0x000061528538f906 in nbd_trip (...) at ../nbd/server.c:3121 > #19 0x00006152855a7caf in coroutine_trampoline (...) at ../util/coroutine= -ucontext.c:175 Cc: qemu-stable@nongnu.org Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona Ebner Message-Id: <20240712140716.517911-1-f.ebner@proxmox.com> Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/copy-before-write.c | 3 ++- block/reqlist.c | 2 -- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index e835987e52..81afeff1c7 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -66,7 +66,8 @@ typedef struct BDRVCopyBeforeWriteState { =20 /* * @frozen_read_reqs: current read requests for fleecing user in bs->f= ile - * node. These areas must not be rewritten by guest. + * node. These areas must not be rewritten by guest. There can be mult= iple + * overlapping read requests. */ BlockReqList frozen_read_reqs; =20 diff --git a/block/reqlist.c b/block/reqlist.c index 08cb57cfa4..098e807378 100644 --- a/block/reqlist.c +++ b/block/reqlist.c @@ -20,8 +20,6 @@ void reqlist_init_req(BlockReqList *reqs, BlockReq *req, int64_t offset, int64_t bytes) { - assert(!reqlist_find_conflict(reqs, offset, bytes)); - *req =3D (BlockReq) { .offset =3D offset, .bytes =3D bytes, --=20 2.34.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926495; cv=none; d=zohomail.com; s=zohoarc; b=XlBKQAqjAMTTtkhpYcHKb+EGchFmCC9F9dxGVkO2XZEdM1LePk5PbydJL1I57gYXqBXNZXATmaQnzMmkOa2aXO8t3KOhvYzH5uri01vYjThE7c9TyVhzhiOAkfv0x9Vawi8uZ5yw6R4QYKZkNDVTSvJ9MVDpiUTM6scVggQBUHI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926495; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=/rG8UY5S3NYVU1597V40CSKWbjuXV6Z6w9PPCbf6m1E=; b=JUrFV8V5kPM3PAyWZytVaNhOdOQC+MNaBcNgzel+kSjU0RwT9Ci1syqjs+7d4Tog9qeJ/M0avbgtpkuPBx+vdDx2SPFKae6WTQGFKRzSPTfPWopKixooT7JP7WrfCzk3os5XcGjmRGeRNAE4TKYg1H3ILoK8ongxn3knJeBL+pk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926495417238.66363197568694; Tue, 29 Apr 2025 04:34:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDi-0006vb-9h; Tue, 29 Apr 2025 07:33:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDg-0006sT-1h; Tue, 29 Apr 2025 07:33:48 -0400 Received: from forwardcorp1d.mail.yandex.net ([178.154.239.200]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDd-0008FH-Ef; Tue, 29 Apr 2025 07:33:47 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 1399A611A0; Tue, 29 Apr 2025 14:33:41 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-NjKzBwzc; Tue, 29 Apr 2025 14:33:40 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926420; bh=/rG8UY5S3NYVU1597V40CSKWbjuXV6Z6w9PPCbf6m1E=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=oFB9TLz07EpVDUQTxJyVOKTrU+gKjSkbv7Bhvlf10R8bjMPWHBQzI6y+3ak9mXDQB 50OHyFnqzST1FIRM9gDJaUmjoR64n72KhF8buVCHhpzFAyCAMSsNETlafCBTg8BOjl 1MtrCF0NLqPyemUhvICY1tXPvEYhrjtnBxYHnkqg= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Vincent Vanlaer , Vladimir Sementsov-Ogievskiy Subject: [PULL 3/8] block: refactor error handling of commit_iteration Date: Tue, 29 Apr 2025 14:33:27 +0300 Message-ID: <20250429113335.423535-6-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.200; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926496634019000 Content-Type: text/plain; charset="utf-8" From: Vincent Vanlaer Signed-off-by: Vincent Vanlaer Message-Id: <20241026163010.2865002-4-libvirt-e6954efa@volkihar.be> Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/commit.c | 62 ++++++++++++++++++++++++++++---------------------- 1 file changed, 35 insertions(+), 27 deletions(-) diff --git a/block/commit.c b/block/commit.c index 3ee0ade7df..0fbf684728 100644 --- a/block/commit.c +++ b/block/commit.c @@ -129,51 +129,59 @@ static void commit_clean(Job *job) } =20 static int commit_iteration(CommitBlockJob *s, int64_t offset, - int64_t *n, void *buf) + int64_t *requested_bytes, void *buf) { + int64_t bytes =3D *requested_bytes; int ret =3D 0; - bool copy; bool error_in_source =3D true; =20 /* Copy if allocated above the base */ WITH_GRAPH_RDLOCK_GUARD() { ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), s->base_overlay, true, true, offset, COMMIT_BUFFER_SIZE, - n, NULL, NULL, NULL); + &bytes, NULL, NULL, NULL); } =20 - copy =3D (ret >=3D 0 && ret & BDRV_BLOCK_ALLOCATED); - trace_commit_one_iteration(s, offset, *n, ret); - if (copy) { - assert(*n < SIZE_MAX); + trace_commit_one_iteration(s, offset, bytes, ret); =20 - ret =3D blk_co_pread(s->top, offset, *n, buf, 0); - if (ret >=3D 0) { - ret =3D blk_co_pwrite(s->base, offset, *n, buf, 0); - if (ret < 0) { - error_in_source =3D false; - } - } - } if (ret < 0) { - BlockErrorAction action =3D block_job_error_action(&s->common, - s->on_error, - error_in_source, - -ret); - if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { - return ret; - } else { - *n =3D 0; - return 0; + goto fail; + } + + if (ret & BDRV_BLOCK_ALLOCATED) { + assert(bytes < SIZE_MAX); + + ret =3D blk_co_pread(s->top, offset, bytes, buf, 0); + if (ret < 0) { + goto fail; } + + ret =3D blk_co_pwrite(s->base, offset, bytes, buf, 0); + if (ret < 0) { + error_in_source =3D false; + goto fail; + } + + block_job_ratelimit_processed_bytes(&s->common, bytes); } + /* Publish progress */ - job_progress_update(&s->common.job, *n); =20 - if (copy) { - block_job_ratelimit_processed_bytes(&s->common, *n); + job_progress_update(&s->common.job, bytes); + + *requested_bytes =3D bytes; + + return 0; + +fail: + BlockErrorAction action =3D block_job_error_action(&s->common, s->on_e= rror, + error_in_source, -ret= ); + if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { + return ret; } =20 + *requested_bytes =3D 0; + return 0; } =20 --=20 2.48.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926544; cv=none; d=zohomail.com; s=zohoarc; b=EvBQmpwuVqnJx2mltm16gQloVxRDt7kbI0KV2Hk7DwE7gJUPzqjrDESqPAN2lugM24fNdsP+c9ZWugpkCJL0IKPSLTv+iANR2dtcI4lkq+Yxh0rE9SXffKTE3VZdlsMKF/jqjh0pKdIiyEYHGG/Nt3+G9Q992UEI/Cz0i5rUf6s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926544; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TNo371By6R1gWLzBQzjueDMSgHamik/HswYQWJjXTkA=; b=LzfrcSmPZONfhnJVufOskFuNXm70KfO2pOQkqSCaBvtDXqkGn1WOBV+NFEZzKIenfOhFYjMbZT+U5P/SQl+fADs5Y0gAYIFNvITsprJdCYrAyFr87DLrXrMD7+ctMpqViA62N3qnhbPSXgPPRVpxq64/MF9DXNgvXQ/Vg1+89WE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926544079350.51798081164577; Tue, 29 Apr 2025 04:35:44 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDo-00076E-Fi; Tue, 29 Apr 2025 07:33:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDh-0006vd-VM; Tue, 29 Apr 2025 07:33:50 -0400 Received: from forwardcorp1d.mail.yandex.net ([2a02:6b8:c41:1300:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDd-0008Fo-UD; Tue, 29 Apr 2025 07:33:49 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id D79AE613DA; Tue, 29 Apr 2025 14:33:42 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-DNZW9x1I; Tue, 29 Apr 2025 14:33:42 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926422; bh=TNo371By6R1gWLzBQzjueDMSgHamik/HswYQWJjXTkA=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=z6DwbRkHmibdE1uWvEEnhz8Oh7V9yjOUKJd3za6cT9GbbAwPTebBJqTpHEhHNKfYD M3HfYs66gc0Tj+wDfJ5sxXhNRJaaav7mh+zxKGSiXJAkVj1pA2jg3WYSqLEn8k3Qyt x/AWarqwIgilLy15+R6Muv8IsDd/VPvnqcTWlkjg= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, "Dr. David Alan Gilbert" , Vladimir Sementsov-Ogievskiy Subject: [PULL 4/5] block: Remove unused aio_task_pool_empty Date: Tue, 29 Apr 2025 14:33:30 +0300 Message-ID: <20250429113335.423535-9-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c41:1300:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926546460019100 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" aio_task_pool_empty has been unused since it was added in 6e9b225f73 ("block: introduce aio task pool") Remove it. Signed-off-by: Dr. David Alan Gilbert Message-Id: <20240917002007.330689-1-dave@treblig.org> Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/aio_task.c | 5 ----- include/block/aio_task.h | 2 -- 2 files changed, 7 deletions(-) diff --git a/block/aio_task.c b/block/aio_task.c index 9bd17ea2c1..bb5c05f455 100644 --- a/block/aio_task.c +++ b/block/aio_task.c @@ -119,8 +119,3 @@ int aio_task_pool_status(AioTaskPool *pool) =20 return pool->status; } - -bool aio_task_pool_empty(AioTaskPool *pool) -{ - return pool->busy_tasks =3D=3D 0; -} diff --git a/include/block/aio_task.h b/include/block/aio_task.h index 18a9c41f4e..c81d637617 100644 --- a/include/block/aio_task.h +++ b/include/block/aio_task.h @@ -40,8 +40,6 @@ void aio_task_pool_free(AioTaskPool *); /* error code of failed task or 0 if all is OK */ int aio_task_pool_status(AioTaskPool *pool); =20 -bool aio_task_pool_empty(AioTaskPool *pool); - /* User provides filled @task, however task->pool will be set automaticall= y */ void coroutine_fn aio_task_pool_start_task(AioTaskPool *pool, AioTask *tas= k); =20 --=20 2.34.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926589; cv=none; d=zohomail.com; s=zohoarc; b=XCztIwIp5sAEh1b0w/CDVCEfhQfPXK6RJhH8cYMyVpdZZMj+FrFQA5aplmbT2DWLwCh9PMUlWmQDwm5SZqRxilF9KDd4N9Jt7aHD6NfmjGTzAEGbO/bw0kC3I8YzCoJtraaeLkh/yHnOEz+reS1etB9mw9gFI465bU3qAO6VQCM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926589; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=klMsBhm4nQV+bJbcH/oarfEWSHGxoD1YQoQrrYEdnVM=; b=Hf2rn4uojs8otLVKIhl/h1d2h3GsA3M+/iOusGCct6wKUNqyGtECa/hD3WiL96QgUvt8oVNCvKMvL+cIUbzE0b1hvGs6K+05nIjpRVcWRRqe5rH1nhQQq83hx3h+eODa/rH0PjthGMwKOl7YMOHN5qpvdrkXc5fy/QDhQYQsND0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926589196799.0826734522657; Tue, 29 Apr 2025 04:36:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDv-0007DA-6n; Tue, 29 Apr 2025 07:34:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDj-0006wy-QV; Tue, 29 Apr 2025 07:33:52 -0400 Received: from forwardcorp1d.mail.yandex.net ([2a02:6b8:c41:1300:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDe-0008Fa-Du; Tue, 29 Apr 2025 07:33:51 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 2A4B461323; Tue, 29 Apr 2025 14:33:42 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-3PGDmJAX; Tue, 29 Apr 2025 14:33:41 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926421; bh=klMsBhm4nQV+bJbcH/oarfEWSHGxoD1YQoQrrYEdnVM=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=JsnQkAtcfsMB/5tBEjPhnSGYbtymjzCEF7cUWoYf3kAyxaiCurqs0iVV7F7CUHkgz WkLWjerGO2nGebNOK244WwQuudi0y4KbYE1iM7QVmLN1m2FOa75kru6whhm2Pk1Pea nr9x4nG7ZwTnsjTFpmEMY0/RsbwA34jkkMQkVXBA= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Vincent Vanlaer , Vladimir Sementsov-Ogievskiy Subject: [PULL 4/8] block: allow commit to unmap zero blocks Date: Tue, 29 Apr 2025 14:33:29 +0300 Message-ID: <20250429113335.423535-8-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c41:1300:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926591124019000 Content-Type: text/plain; charset="utf-8" From: Vincent Vanlaer Non-active block commits do not discard blocks only containing zeros, causing images to lose sparseness after the commit. This commit fixes that by writing zero blocks using blk_co_pwrite_zeroes rather than writing them out as any other arbitrary data. Signed-off-by: Vincent Vanlaer Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <20241026163010.2865002-5-libvirt-e6954efa@volkihar.be> Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/commit.c | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/block/commit.c b/block/commit.c index 0fbf684728..4938714e0f 100644 --- a/block/commit.c +++ b/block/commit.c @@ -149,19 +149,39 @@ static int commit_iteration(CommitBlockJob *s, int64_= t offset, } =20 if (ret & BDRV_BLOCK_ALLOCATED) { - assert(bytes < SIZE_MAX); + if (ret & BDRV_BLOCK_ZERO) { + /* + * If the top (sub)clusters are smaller than the base + * (sub)clusters, this will not unmap unless the underlying de= vice + * does some tracking of these requests. Ideally, we would find + * the maximal extent of the zero clusters. + */ + ret =3D blk_co_pwrite_zeroes(s->base, offset, bytes, + BDRV_REQ_MAY_UNMAP); + if (ret < 0) { + error_in_source =3D false; + goto fail; + } + } else { + assert(bytes < SIZE_MAX); =20 - ret =3D blk_co_pread(s->top, offset, bytes, buf, 0); - if (ret < 0) { - goto fail; - } + ret =3D blk_co_pread(s->top, offset, bytes, buf, 0); + if (ret < 0) { + goto fail; + } =20 - ret =3D blk_co_pwrite(s->base, offset, bytes, buf, 0); - if (ret < 0) { - error_in_source =3D false; - goto fail; + ret =3D blk_co_pwrite(s->base, offset, bytes, buf, 0); + if (ret < 0) { + error_in_source =3D false; + goto fail; + } } =20 + /* + * Whether zeroes actually end up on disk depends on the details of + * the underlying driver. Therefore, this might rate limit more th= an + * is necessary. + */ block_job_ratelimit_processed_bytes(&s->common, bytes); } =20 --=20 2.48.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926524; cv=none; d=zohomail.com; s=zohoarc; b=C7+Zpfy6YqCl3dy8SssufVstenT9xP4HMfDL0BVtEQmkcY3WVCWjz2w1pGikjipFmmicUUFEZg0CZyEzMmCcm7zZm8MPmJLYsvpqyBhf0SgizsPNz2oMO42X3maI0cN8P9w53e2CeYLrV0ql8ldkXRPduUDZK3I63YeqXh5lfw4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926524; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=BtuTV4i6Dso4pgdFfeN1B9eAAXqbLI7RCSe0Zk64D1I=; b=RZxaK2u3WRJG7YoCLSYyfGrP3ufplYXMvnDWI6anOlf7Tpsnn9QTUF/BqkqOmlILj0zo6fAWJcHHKk1Gxa59Ah4r3KYiHhm9AylGM5S32SDA3hBw4020nof3onmCi5fi9n8R8bwL9Zll/5Tck0QPgfa1zXD7K9jmtFDtH+9CSqA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926524545332.975041853676; Tue, 29 Apr 2025 04:35:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDq-00078c-GO; Tue, 29 Apr 2025 07:33:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDj-0006xE-Rz; Tue, 29 Apr 2025 07:33:52 -0400 Received: from forwardcorp1d.mail.yandex.net ([178.154.239.200]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDh-0008Gx-K1; Tue, 29 Apr 2025 07:33:51 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 1779461333; Tue, 29 Apr 2025 14:33:44 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-E099GDmw; Tue, 29 Apr 2025 14:33:43 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926423; bh=BtuTV4i6Dso4pgdFfeN1B9eAAXqbLI7RCSe0Zk64D1I=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=dlQe4M/FN6pErqV75MkagQYMuJbv8snm0aFLmTP4RUf7ALH1swMa1x4dl7IsTuqu/ /0qiCvEGmTzqj7isbmsORyD8ro/8Pd9fkeNFEE7lLaqRzAEZ0P006b7YiALmFGKBmM NHJLU5eeiE0qF0fkQC4Fx86gfTocOQ6TMMhvjt7g= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, "Dr. David Alan Gilbert" , Vladimir Sementsov-Ogievskiy Subject: [PULL 5/5] util/co-shared-resource: Remove unused co_try_get_from_shres Date: Tue, 29 Apr 2025 14:33:32 +0300 Message-ID: <20250429113335.423535-11-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.200; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926526733019000 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" co_try_get_from_shres hasn't been used since it was added in 55fa54a789 ("co-shared-resource: protect with a mutex") (Everyone uses the _locked version) Remove it. Signed-off-by: Dr. David Alan Gilbert Message-Id: <20240918124220.27871-1-dave@treblig.org> Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/qemu/co-shared-resource.h | 7 ------- util/qemu-co-shared-resource.c | 6 ------ 2 files changed, 13 deletions(-) diff --git a/include/qemu/co-shared-resource.h b/include/qemu/co-shared-res= ource.h index 78ca5850f8..41be1a8131 100644 --- a/include/qemu/co-shared-resource.h +++ b/include/qemu/co-shared-resource.h @@ -44,13 +44,6 @@ SharedResource *shres_create(uint64_t total); */ void shres_destroy(SharedResource *s); =20 -/* - * Try to allocate an amount of @n. Return true on success, and false - * if there is too little left of the collective resource to fulfill - * the request. - */ -bool co_try_get_from_shres(SharedResource *s, uint64_t n); - /* * Allocate an amount of @n, and, if necessary, yield until * that becomes possible. diff --git a/util/qemu-co-shared-resource.c b/util/qemu-co-shared-resource.c index a66cc07e75..752eb5a1c5 100644 --- a/util/qemu-co-shared-resource.c +++ b/util/qemu-co-shared-resource.c @@ -66,12 +66,6 @@ static bool co_try_get_from_shres_locked(SharedResource = *s, uint64_t n) return false; } =20 -bool co_try_get_from_shres(SharedResource *s, uint64_t n) -{ - QEMU_LOCK_GUARD(&s->lock); - return co_try_get_from_shres_locked(s, n); -} - void coroutine_fn co_get_from_shres(SharedResource *s, uint64_t n) { assert(n <=3D s->total); --=20 2.34.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926558; cv=none; d=zohomail.com; s=zohoarc; b=fGqtW9fANoUEXiLhbF1bVrPQWLinLTZ3faVHefBBV4VQm1HLBJEWmILnT7kITTLiDoitYsQwQjmqn9gZZyMKGrkckQFZX1a7G3c+u93IyxPWhDjcbm9w87TsU/5YxNTCUbCGn/3eVgm3Ys7DiAKx5wsQR4ZkI2QhgswSFawivsQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926558; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=z/6W6b1sM2VGsVoOmuFpEtt9gaZabUicXumqlhbDDEY=; b=TQwE01QBsdYfBuPuf2r8FcGUQdUOz34ghcwKKbFikeZ+qSv7doCfYaTlVmNMYb+Sip+lykuTyb3d0yg56YxBW10MqZ6fPLbuNmgDhP4Vg1wsPdru8k0SvipBo5POgNIp4iZqa1mZ3yc2iBeQKwb5cgbuyff7dCN0XefHLimohYU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926558235410.429943901368; Tue, 29 Apr 2025 04:35:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDo-00076L-LK; Tue, 29 Apr 2025 07:33:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDj-0006xL-UH; Tue, 29 Apr 2025 07:33:52 -0400 Received: from forwardcorp1d.mail.yandex.net ([2a02:6b8:c41:1300:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDe-0008Fu-CW; Tue, 29 Apr 2025 07:33:51 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 9D0B96133D; Tue, 29 Apr 2025 14:33:43 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-IGZN9tlF; Tue, 29 Apr 2025 14:33:42 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926422; bh=z/6W6b1sM2VGsVoOmuFpEtt9gaZabUicXumqlhbDDEY=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=nqmsxLQWLNsmde/SJMPkfA2Wjigr/pHBVQ1eAMRot5IouLDm8D8mauWTd1C1F/v+e zTmqUTEx5HAajj2+ZFEphu0EGrJa07+pc578MlhMl5Qfdbj0fj1iSIyzD+etNU921W 140CuRjWSfloZ45+CQOzYs8U2CfU/BOccjdEnuRU= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Vincent Vanlaer , Vladimir Sementsov-Ogievskiy Subject: [PULL 5/8] block: add test non-active commit with zeroed data Date: Tue, 29 Apr 2025 14:33:31 +0300 Message-ID: <20250429113335.423535-10-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c41:1300:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926559057019000 Content-Type: text/plain; charset="utf-8" From: Vincent Vanlaer Signed-off-by: Vincent Vanlaer Tested-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <20241026163010.2865002-6-libvirt-e6954efa@volkihar.be> Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/tests/commit-zero-blocks | 96 +++++++++++++++++++ .../qemu-iotests/tests/commit-zero-blocks.out | 54 +++++++++++ 2 files changed, 150 insertions(+) create mode 100755 tests/qemu-iotests/tests/commit-zero-blocks create mode 100644 tests/qemu-iotests/tests/commit-zero-blocks.out diff --git a/tests/qemu-iotests/tests/commit-zero-blocks b/tests/qemu-iotes= ts/tests/commit-zero-blocks new file mode 100755 index 0000000000..de00273e72 --- /dev/null +++ b/tests/qemu-iotests/tests/commit-zero-blocks @@ -0,0 +1,96 @@ +#!/usr/bin/env bash +# group: rw quick +# +# Test for commit of discarded blocks +# +# This tests committing a live snapshot where some of the blocks that +# are present in the base image are discarded in the intermediate image. +# This intends to check that these blocks are also discarded in the base +# image after the commit. +# +# Copyright (C) 2024 Vincent Vanlaer. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# creator +owner=3Dlibvirt-e6954efa@volkihar.be + +seq=3D`basename $0` +echo "QA output created by $seq" + +status=3D1 # failure is the default! + +_cleanup() +{ + _cleanup_qemu + _rm_test_img "${TEST_IMG}.base" + _rm_test_img "${TEST_IMG}.mid" + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +cd .. +. ./common.rc +. ./common.filter +. ./common.qemu + +_supported_fmt qcow2 +_supported_proto file + +size=3D"1M" + +TEST_IMG=3D"$TEST_IMG.base" _make_test_img $size +TEST_IMG=3D"$TEST_IMG.mid" _make_test_img -b "$TEST_IMG.base" -F $IMGFMT $= size +_make_test_img -b "${TEST_IMG}.mid" -F $IMGFMT $size + +$QEMU_IO -c "write -P 0x01 64k 128k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "discard 64k 64k" "$TEST_IMG.mid" | _filter_qemu_io + +echo +echo "=3D=3D=3D Base image info before commit =3D=3D=3D" +TEST_IMG=3D"${TEST_IMG}.base" _img_info | _filter_img_info +$QEMU_IMG map --output=3Djson "$TEST_IMG.base" | _filter_qemu_img_map + +echo +echo "=3D=3D=3D Middle image info before commit =3D=3D=3D" +TEST_IMG=3D"${TEST_IMG}.mid" _img_info | _filter_img_info +$QEMU_IMG map --output=3Djson "$TEST_IMG.mid" | _filter_qemu_img_map + +echo +echo =3D=3D=3D Running QEMU Live Commit Test =3D=3D=3D +echo + +qemu_comm_method=3D"qmp" +_launch_qemu -drive file=3D"${TEST_IMG}",if=3Dvirtio,id=3Dtest +h=3D$QEMU_HANDLE + +_send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" "return" + +_send_qemu_cmd $h "{ 'execute': 'block-commit', + 'arguments': { 'device': 'test', + 'top': '"${TEST_IMG}.mid"', + 'base': '"${TEST_IMG}.base"'} }" '"status= ": "null"' + +_cleanup_qemu + +echo +echo "=3D=3D=3D Base image info after commit =3D=3D=3D" +TEST_IMG=3D"${TEST_IMG}.base" _img_info | _filter_img_info +$QEMU_IMG map --output=3Djson "$TEST_IMG.base" | _filter_qemu_img_map + +# success, all done +echo "*** done" +rm -f $seq.full +status=3D0 diff --git a/tests/qemu-iotests/tests/commit-zero-blocks.out b/tests/qemu-i= otests/tests/commit-zero-blocks.out new file mode 100644 index 0000000000..85bdc46aaf --- /dev/null +++ b/tests/qemu-iotests/tests/commit-zero-blocks.out @@ -0,0 +1,54 @@ +QA output created by commit-zero-blocks +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D1048576 +Formatting 'TEST_DIR/t.IMGFMT.mid', fmt=3DIMGFMT size=3D1048576 backing_fi= le=3DTEST_DIR/t.IMGFMT.base backing_fmt=3DIMGFMT +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1048576 backing_file= =3DTEST_DIR/t.IMGFMT.mid backing_fmt=3DIMGFMT +wrote 131072/131072 bytes at offset 65536 +128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +discard 65536/65536 bytes at offset 65536 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=3D=3D=3D Base image info before commit =3D=3D=3D +image: TEST_DIR/t.IMGFMT.base +file format: IMGFMT +virtual size: 1 MiB (1048576 bytes) +[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": true= , "data": false, "compressed": false}, +{ "start": 65536, "length": 131072, "depth": 0, "present": true, "zero": f= alse, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 196608, "length": 851968, "depth": 0, "present": false, "zero":= true, "data": false, "compressed": false}] + +=3D=3D=3D Middle image info before commit =3D=3D=3D +image: TEST_DIR/t.IMGFMT.mid +file format: IMGFMT +virtual size: 1 MiB (1048576 bytes) +backing file: TEST_DIR/t.IMGFMT.base +backing file format: IMGFMT +[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true= , "data": false, "compressed": false}, +{ "start": 65536, "length": 65536, "depth": 0, "present": true, "zero": tr= ue, "data": false, "compressed": false}, +{ "start": 131072, "length": 65536, "depth": 1, "present": true, "zero": f= alse, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 196608, "length": 851968, "depth": 1, "present": false, "zero":= true, "data": false, "compressed": false}] + +=3D=3D=3D Running QEMU Live Commit Test =3D=3D=3D + +{ 'execute': 'qmp_capabilities' } +{"return": {}} +{ 'execute': 'block-commit', + 'arguments': { 'device': 'test', + 'top': 'TEST_DIR/t.IMGFMT.mid', + 'base': 'TEST_DIR/t.IMGFMT.base'} } +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "test"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "test"}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "test"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "test"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_COMPLETED", "data": {"device": "test", "len": 1048576, "offset= ": 1048576, "speed": 0, "type": "commit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "test"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "test"}} + +=3D=3D=3D Base image info after commit =3D=3D=3D +image: TEST_DIR/t.IMGFMT.base +file format: IMGFMT +virtual size: 1 MiB (1048576 bytes) +[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": true= , "data": false, "compressed": false}, +{ "start": 65536, "length": 65536, "depth": 0, "present": true, "zero": tr= ue, "data": false, "compressed": false}, +{ "start": 131072, "length": 65536, "depth": 0, "present": true, "zero": f= alse, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 196608, "length": 851968, "depth": 0, "present": false, "zero":= true, "data": false, "compressed": false}] +*** done --=20 2.48.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926471; cv=none; d=zohomail.com; s=zohoarc; b=RAXJ1QlFF6hMHqCK7MwbTQq30+MxfTAQj1WZKOo4KNKJt0em8ipo1jCMsdaaND4slAXE6Wyg/28Qg1jmOta2matQVm/Rb4A3tYHQ5IgybyeE0WPqrCJJZTz7TGv6Rk+gJb1BfBCsm1wzAJcvEpbpaxpG4m+sH+sEj72ZHVGcOzc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926471; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SijDt/tVC8RzUpiTA3pc0vLmFBOBKfrzJu2UzhRznAY=; b=c8eAcnuyWMCDjwhtA226f6LxqZ5dZ0nkhRdKWxVRThmVCmiVQq59xGNUQTecXmGTy9RSgPYouXyNJB/dvOZzt8mtkpezCLFO0mVpX21r9WMbMy4/WY0mUymo+Vy0m7ePKR3BI+Bj2VivyQPcF3i21hRYbWl/aQ9V0j94cq5dYf0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 174592647108725.64885349739791; Tue, 29 Apr 2025 04:34:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDp-00076x-CB; Tue, 29 Apr 2025 07:33:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDk-0006yJ-F8; Tue, 29 Apr 2025 07:33:53 -0400 Received: from forwardcorp1d.mail.yandex.net ([178.154.239.200]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDh-0008Gw-Q7; Tue, 29 Apr 2025 07:33:52 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 88FB5613E7; Tue, 29 Apr 2025 14:33:45 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-aBhvFnOW; Tue, 29 Apr 2025 14:33:45 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926425; bh=SijDt/tVC8RzUpiTA3pc0vLmFBOBKfrzJu2UzhRznAY=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=UcJG8IxxLjb14iTpfNu3G+wTg3zJtbmg8fu7BB1QJF5hchI8oS0WM4Dkrd885T3Kj hDnaAWZhEC0GQvWgVwZD59o0n/ekDfCj+myN70kcpgKS/6BhPRHqA0+U4j5kRrDSL/ 2MKIUMbmVUi2FC9+d+a2dTrmPAm5WmcwcFZshpg8= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Vladimir Sementsov-Ogievskiy , Markus Armbruster Subject: [PULL 6/8] qapi: synchronize jobs and block-jobs documentation Date: Tue, 29 Apr 2025 14:33:33 +0300 Message-ID: <20250429113335.423535-12-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.200; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926475246019100 Content-Type: text/plain; charset="utf-8" Actualize documentation and synchronize it for commands which actually call the same functions internally. Reviewed-by: Markus Armbruster Signed-off-by: Vladimir Sementsov-Ogievskiy Message-ID: <20250409084232.28201-2-vsementsov@yandex-team.ru> --- qapi/block-core.json | 61 ++++++++++++++++++++++++++------------------ qapi/job.json | 30 ++++++++++++++++++++-- 2 files changed, 64 insertions(+), 27 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index b1937780e1..6beab0dc12 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2956,13 +2956,14 @@ # # Pause an active background block operation. # -# This command returns immediately after marking the active background -# block operation for pausing. It is an error to call this command if -# no operation is in progress or if the job is already paused. +# This command returns immediately after marking the active job for +# pausing. Pausing an already paused job is an error. +# +# The job will pause as soon as possible, which means transitioning +# into the PAUSED state if it was RUNNING, or into STANDBY if it was +# READY. The corresponding JOB_STATUS_CHANGE event will be emitted. # -# The operation will pause as soon as possible. No event is emitted -# when the operation is actually paused. Cancelling a paused job -# automatically resumes it. +# Cancelling a paused job automatically resumes it. # # @device: The job identifier. This used to be a device name (hence # the name of the parameter), but since QEMU 2.7 it can have other @@ -2982,9 +2983,8 @@ # # Resume an active background block operation. # -# This command returns immediately after resuming a paused background -# block operation. It is an error to call this command if no -# operation is in progress or if the job is not paused. +# This command returns immediately after resuming a paused job. +# Resuming an already running job is an error. # # This command also clears the error status of the job. # @@ -3004,10 +3004,15 @@ ## # @block-job-complete: # -# Manually trigger completion of an active background block operation. -# This is supported for drive mirroring, where it also switches the -# device to write to the target path only. The ability to complete is -# signaled with a BLOCK_JOB_READY event. +# Manually trigger completion of an active job in the READY or STANDBY +# state. Completing the job in any other state is an error. +# +# This is supported only for drive mirroring, where it also switches +# the device to write to the target path only. Note that drive +# mirroring includes drive-mirror, blockdev-mirror and block-commit +# job (only in case of "active commit", when the node being commited +# is used by the guest). The ability to complete is signaled with a +# BLOCK_JOB_READY event. # # This command completes an active background block operation # synchronously. The ordering of this command's return with the @@ -3017,8 +3022,6 @@ # rerror/werror arguments that were specified when starting the # operation. # -# A cancelled or paused job cannot be completed. -# # @device: The job identifier. This used to be a device name (hence # the name of the parameter), but since QEMU 2.7 it can have other # values. @@ -3035,10 +3038,13 @@ ## # @block-job-dismiss: # -# For jobs that have already concluded, remove them from the -# block-job-query list. This command only needs to be run for jobs -# which were started with QEMU 2.12+ job lifetime management -# semantics. +# Deletes a job that is in the CONCLUDED state. This command only +# needs to be run explicitly for jobs that don't have automatic +# dismiss enabled. In turn, automatic dismiss may be enabled only +# for jobs that have @auto-dismiss option, which are drive-backup, +# blockdev-backup, drive-mirror, blockdev-mirror, block-commit and +# block-stream. @auto-dismiss is enabled by default for these +# jobs. # # This command will refuse to operate on any job that has not yet # reached its terminal state, JOB_STATUS_CONCLUDED. For jobs that @@ -3055,12 +3061,17 @@ ## # @block-job-finalize: # -# Once a job that has manual=3Dtrue reaches the pending state, it can be -# instructed to finalize any graph changes and do any necessary -# cleanup via this command. For jobs in a transaction, instructing -# one job to finalize will force ALL jobs in the transaction to -# finalize, so it is only necessary to instruct a single member job to -# finalize. +# Instructs all jobs in a transaction (or a single job if it is not +# part of any transaction) to finalize any graph changes and do any +# necessary cleanup. This command requires that all involved jobs are +# in the PENDING state. +# +# For jobs in a transaction, instructing one job to finalize will +# force ALL jobs in the transaction to finalize, so it is only +# necessary to instruct a single member job to finalize. +# +# The command is applicable only to jobs which have @auto-finalize option +# and only when this option is set to false. # # @id: The job identifier. # diff --git a/qapi/job.json b/qapi/job.json index cfc3beedd2..b03f80bc84 100644 --- a/qapi/job.json +++ b/qapi/job.json @@ -156,6 +156,9 @@ # This command returns immediately after resuming a paused job. # Resuming an already running job is an error. # +# This command also clears the error status for block-jobs (stream, +# commit, mirror, backup). +# # @id: The job identifier. # # Since: 3.0 @@ -184,7 +187,23 @@ ## # @job-complete: # -# Manually trigger completion of an active job in the READY state. +# Manually trigger completion of an active job in the READY or STANDBY +# state. Completing the job in any other state is an error. +# +# This is supported only for drive mirroring, where it also switches +# the device to write to the target path only. Note that drive +# mirroring includes drive-mirror, blockdev-mirror and block-commit +# job (only in case of "active commit", when the node being commited +# is used by the guest). The ability to complete is signaled with a +# BLOCK_JOB_READY event. +# +# This command completes an active background block operation +# synchronously. The ordering of this command's return with the +# BLOCK_JOB_COMPLETED event is not defined. Note that if an I/O error +# occurs during the processing of this command: 1) the command itself +# will fail; 2) the error will be processed according to the +# rerror/werror arguments that were specified when starting the +# operation. # # @id: The job identifier. # @@ -197,7 +216,11 @@ # # Deletes a job that is in the CONCLUDED state. This command only # needs to be run explicitly for jobs that don't have automatic -# dismiss enabled. +# dismiss enabled. In turn, automatic dismiss may be enabled only +# for jobs that have @auto-dismiss option, which are drive-backup, +# blockdev-backup, drive-mirror, blockdev-mirror, block-commit and +# block-stream. @auto-dismiss is enabled by default for these +# jobs. # # This command will refuse to operate on any job that has not yet # reached its terminal state, JOB_STATUS_CONCLUDED. For jobs that @@ -222,6 +245,9 @@ # force ALL jobs in the transaction to finalize, so it is only # necessary to instruct a single member job to finalize. # +# The command is applicable only to jobs which have @auto-finalize option +# and only when this option is set to false. +# # @id: The identifier of any job in the transaction, or of a job that # is not part of any transaction. # --=20 2.48.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926585; cv=none; d=zohomail.com; s=zohoarc; b=C8O+33H/zm7cS5nog7vbUn5AK0CNqJ3FGXQUIYzhypNec3X/SgmR1tNrNpqbz8ld67e9A3FwYE+5tgR+asiGE62SQ0vgGQfyxRPGSTnfJ1PqQXG8ugtze25y5cSuXUQVwd/BMmecg3MnWr/wuGBVG8B014QaBWKnLNY1971H7Ag= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926585; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QFP2bF/oCU/i3KsQH4XH+arblxug5ghRMFZFdOGqZ/Y=; b=eY0x1LoXru81rN8QkK88G7yFMWHoixQ20n6civflmPMb6SSsqAtpWHkMqY5Bgp3VRZ7PjU0UOPNdBft9kFqlVnOTP5KxMNuD3yV2nZcde22VLgvftlsIyuH+U++MzzwwBKnlToICG/0MJR2FGFkdwDUGuyurRwheTi2JuaNwTuk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926585579435.70689476469386; Tue, 29 Apr 2025 04:36:25 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jDx-0007Io-US; Tue, 29 Apr 2025 07:34:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDl-0006zl-3B; Tue, 29 Apr 2025 07:33:53 -0400 Received: from forwardcorp1d.mail.yandex.net ([178.154.239.200]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDi-0008H3-65; Tue, 29 Apr 2025 07:33:52 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 44354613ED; Tue, 29 Apr 2025 14:33:46 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-gsLH5Sh2; Tue, 29 Apr 2025 14:33:45 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926425; bh=QFP2bF/oCU/i3KsQH4XH+arblxug5ghRMFZFdOGqZ/Y=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=hU+Hcza7Q85TTgTkk4GpaMBbLvE3605tgOJej3Ho2i9pqHq00dg4TISNy/rQ0F3+S dZBoazqU9Cn3sodRUBD77how5ks4byEHDe7MLe7kq/uFvJ3bYK1pJPVn8l6Z+f3+if WNbSJ1luKoEOOITKl4ar6ZEeQnaCpumBqVkyJVCs= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Vladimir Sementsov-Ogievskiy , Peter Krempa , Eric Blake , Markus Armbruster Subject: [PULL 7/8] qapi/block-core: deprecate some block-job- APIs Date: Tue, 29 Apr 2025 14:33:34 +0300 Message-ID: <20250429113335.423535-13-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.200; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926587077019000 Content-Type: text/plain; charset="utf-8" For change, pause, resume, complete, dismiss and finalize actions corresponding job- and block-job commands are almost equal. The difference is in find_block_job_locked() vs find_job_locked() functions. What's different? 1. find_block_job_locked() checks whether the found job is a block-job. This is OK when moving to more generic API, no needs to document this change. 2. find_block_job_locked() reports DeviceNotActive on failure, when find_job_locked() reports GenericError. So, let's document this difference in deprecated.txt. Still, for dismiss and finalize errors are not documented at all, so be silent in deprecated.txt as well. ACKed-by: Peter Krempa Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster Signed-off-by: Vladimir Sementsov-Ogievskiy Message-ID: <20250409084232.28201-3-vsementsov@yandex-team.ru> --- docs/about/deprecated.rst | 31 +++++++++++++++++++++++++++++++ qapi/block-core.json | 30 ++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 05381441a9..b0d3ae6b32 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -148,6 +148,37 @@ options are removed in favor of using explicit ``block= dev-create`` and ``blockdev-add`` calls. See :doc:`/interop/live-block-operations` for details. =20 +``block-job-pause`` (since 10.1) +'''''''''''''''''''''''''''''''' + +Use ``job-pause`` instead. The only difference is that ``job-pause`` +always reports GenericError on failure when ``block-job-pause`` reports +DeviceNotActive when block-job is not found. + +``block-job-resume`` (since 10.1) +''''''''''''''''''''''''''''''''' + +Use ``job-resume`` instead. The only difference is that ``job-resume`` +always reports GenericError on failure when ``block-job-resume`` reports +DeviceNotActive when block-job is not found. + +``block-job-complete`` (since 10.1) +''''''''''''''''''''''''''''''''''' + +Use ``job-complete`` instead. The only difference is that ``job-complete`` +always reports GenericError on failure when ``block-job-complete`` reports +DeviceNotActive when block-job is not found. + +``block-job-dismiss`` (since 10.1) +'''''''''''''''''''''''''''''''''' + +Use ``job-dismiss`` instead. + +``block-job-finalize`` (since 10.1) +''''''''''''''''''''''''''''''''''' + +Use ``job-finalize`` instead. + ``query-migrationthreads`` (since 9.2) '''''''''''''''''''''''''''''''''''''' =20 diff --git a/qapi/block-core.json b/qapi/block-core.json index 6beab0dc12..22061227ca 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2969,6 +2969,11 @@ # the name of the parameter), but since QEMU 2.7 it can have other # values. # +# Features: +# +# @deprecated: This command is deprecated. Use @job-pause +# instead. +# # Errors: # - If no background operation is active on this device, # DeviceNotActive @@ -2976,6 +2981,7 @@ # Since: 1.3 ## { 'command': 'block-job-pause', 'data': { 'device': 'str' }, + 'features': ['deprecated'], 'allow-preconfig': true } =20 ## @@ -2992,6 +2998,11 @@ # the name of the parameter), but since QEMU 2.7 it can have other # values. # +# Features: +# +# @deprecated: This command is deprecated. Use @job-resume +# instead. +# # Errors: # - If no background operation is active on this device, # DeviceNotActive @@ -2999,6 +3010,7 @@ # Since: 1.3 ## { 'command': 'block-job-resume', 'data': { 'device': 'str' }, + 'features': ['deprecated'], 'allow-preconfig': true } =20 ## @@ -3026,6 +3038,11 @@ # the name of the parameter), but since QEMU 2.7 it can have other # values. # +# Features: +# +# @deprecated: This command is deprecated. Use @job-complete +# instead. +# # Errors: # - If no background operation is active on this device, # DeviceNotActive @@ -3033,6 +3050,7 @@ # Since: 1.3 ## { 'command': 'block-job-complete', 'data': { 'device': 'str' }, + 'features': ['deprecated'], 'allow-preconfig': true } =20 ## @@ -3053,9 +3071,15 @@ # # @id: The job identifier. # +# Features: +# +# @deprecated: This command is deprecated. Use @job-dismiss +# instead. +# # Since: 2.12 ## { 'command': 'block-job-dismiss', 'data': { 'id': 'str' }, + 'features': ['deprecated'], 'allow-preconfig': true } =20 ## @@ -3075,9 +3099,15 @@ # # @id: The job identifier. # +# Features: +# +# @deprecated: This command is deprecated. Use @job-finalize +# instead. +# # Since: 2.12 ## { 'command': 'block-job-finalize', 'data': { 'id': 'str' }, + 'features': ['deprecated'], 'allow-preconfig': true } =20 ## --=20 2.48.1 From nobody Sat Nov 15 22:26:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1745926483; cv=none; d=zohomail.com; s=zohoarc; b=dG9aKqz8CU8EbZLCsP+c6cOdSLSSzJKmQzrRFhqNdQiZ3sbFs49dGlZNa9OMHxtW4vFXwuiLYnU+iKir6V7R846faoyQ9ijN23W+a7yKb48Ij5w/w+3a0xlROkUGkcR6cRF3R/sHNfrJaHinU1sru2FnYJLHuLVYQe8pa6z8Z4s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745926483; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=U7aSoeMynX0Nus8SNvWltEhknHxNk61xBJbgIohDPBM=; b=I4rQlwfnHReDR/I4nDYRsaYh53LZwf/sqyQa0ZlXVp02EpgLPbF9wZETLDgMZTJdy+90ZsmhW3qGZ5LJIl9L5lEYHEZ2vqGb/rJLd5UooXeVBHmhU1++do5PL3HbTdmD7kZaNK0S0v98SZl+j6CyXnBuDYMNgL6+y7NUn70ZHQQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745926483758133.58807612306384; Tue, 29 Apr 2025 04:34:43 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9jE1-0007OW-1c; Tue, 29 Apr 2025 07:34:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDl-000713-IG; Tue, 29 Apr 2025 07:33:53 -0400 Received: from forwardcorp1d.mail.yandex.net ([2a02:6b8:c41:1300:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9jDi-0008H7-5u; Tue, 29 Apr 2025 07:33:53 -0400 Received: from mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net [IPv6:2a02:6b8:c42:cca4:0:640:432b:0]) by forwardcorp1d.mail.yandex.net (Yandex) with ESMTPS id 1D1C7613F4; Tue, 29 Apr 2025 14:33:47 +0300 (MSK) Received: from vsementsov-lin.. (unknown [2a02:6b8:b081:b423::1:3a]) by mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id bXTAnY0Fe0U0-72gCf8tQ; Tue, 29 Apr 2025 14:33:46 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1745926426; bh=U7aSoeMynX0Nus8SNvWltEhknHxNk61xBJbgIohDPBM=; h=Message-ID:Date:In-Reply-To:Cc:Subject:References:To:From; b=ELSPICGmk9QWZQB1cO3yYW2nU6lnXdzfrKb9zy/YpS5G8RpLvynvJi5TEHGQZNgob kecF5sshOK67EpLPlj0Fivh+62PXcsOsxF0bK+z32J17eSSmXT2HA4n0Cv/rD7CVm9 1gNZsd6qNXCUjaC9HcAxW0Ks+KCLmbWu2det3toA= Authentication-Results: mail-nwsmtp-smtp-corp-main-56.klg.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, Raman Dzehtsiar , Raman Dzehtsiar , Markus Armbruster , Vladimir Sementsov-Ogievskiy Subject: [PULL 8/8] blockdev-backup: Add error handling option for copy-before-write jobs Date: Tue, 29 Apr 2025 14:33:35 +0300 Message-ID: <20250429113335.423535-14-vsementsov@yandex-team.ru> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250429113335.423535-1-vsementsov@yandex-team.ru> References: <20250429113335.423535-1-vsementsov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c41:1300:1:45:d181:df01; envelope-from=vsementsov@yandex-team.ru; helo=forwardcorp1d.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1745926486583019100 Content-Type: text/plain; charset="utf-8" From: Raman Dzehtsiar This patch extends the blockdev-backup QMP command to allow users to specify how to behave when IO errors occur during copy-before-write operations. Previously, the behavior was fixed and could not be controlled by the user. The new 'on-cbw-error' option can be set to one of two values: - 'break-guest-write': Forwards the IO error to the guest and triggers the on-source-error policy. This preserves snapshot integrity at the expense of guest IO operations. - 'break-snapshot': Allows the guest OS to continue running normally, but invalidates the snapshot and aborts related jobs. This prioritizes guest operation over backup consistency. This enhancement provides more flexibility for backup operations in differe= nt environments where requirements for guest availability versus backup consistency may vary. The default behavior remains unchanged to maintain backward compatibility. Signed-off-by: Raman Dzehtsiar Message-ID: <20250414090025.828660-1-Raman.Dzehtsiar@gmail.com> Acked-by: Markus Armbruster Reviewed-by: Vladimir Sementsov-Ogievskiy Tested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 3 +- block/copy-before-write.c | 2 + block/copy-before-write.h | 1 + block/replication.c | 4 +- blockdev.c | 6 ++ include/block/block_int-global-state.h | 2 + qapi/block-core.json | 4 + tests/qemu-iotests/tests/copy-before-write | 90 +++++++++++++++++++ .../qemu-iotests/tests/copy-before-write.out | 4 +- 9 files changed, 112 insertions(+), 4 deletions(-) diff --git a/block/backup.c b/block/backup.c index 79652bf57b..0151e84395 100644 --- a/block/backup.c +++ b/block/backup.c @@ -361,6 +361,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, BackupPerf *perf, BlockdevOnError on_source_error, BlockdevOnError on_target_error, + OnCbwError on_cbw_error, int creation_flags, BlockCompletionFunc *cb, void *opaque, JobTxn *txn, Error **errp) @@ -458,7 +459,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, } =20 cbw =3D bdrv_cbw_append(bs, target, filter_node_name, discard_source, - perf->min_cluster_size, &bcs, errp); + perf->min_cluster_size, &bcs, on_cbw_error, errp= ); if (!cbw) { goto error; } diff --git a/block/copy-before-write.c b/block/copy-before-write.c index fd470f5f92..00af0b18ac 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -551,6 +551,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *sou= rce, bool discard_source, uint64_t min_cluster_size, BlockCopyState **bcs, + OnCbwError on_cbw_error, Error **errp) { BDRVCopyBeforeWriteState *state; @@ -568,6 +569,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *sou= rce, } qdict_put_str(opts, "file", bdrv_get_node_name(source)); qdict_put_str(opts, "target", bdrv_get_node_name(target)); + qdict_put_str(opts, "on-cbw-error", OnCbwError_str(on_cbw_error)); =20 if (min_cluster_size > INT64_MAX) { error_setg(errp, "min-cluster-size too large: %" PRIu64 " > %" PRI= i64, diff --git a/block/copy-before-write.h b/block/copy-before-write.h index 2a5d4ba693..eb93364e85 100644 --- a/block/copy-before-write.h +++ b/block/copy-before-write.h @@ -42,6 +42,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *sourc= e, bool discard_source, uint64_t min_cluster_size, BlockCopyState **bcs, + OnCbwError on_cbw_error, Error **errp); void bdrv_cbw_drop(BlockDriverState *bs); =20 diff --git a/block/replication.c b/block/replication.c index d6625c51fe..07f274de9e 100644 --- a/block/replication.c +++ b/block/replication.c @@ -583,7 +583,9 @@ static void replication_start(ReplicationState *rs, Rep= licationMode mode, 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, = false, NULL, &perf, BLOCKDEV_ON_ERROR_REPORT, - BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, + BLOCKDEV_ON_ERROR_REPORT, + ON_CBW_ERROR_BREAK_GUEST_WRITE, + JOB_INTERNAL, backup_job_completed, bs, NULL, &local_err= ); if (local_err) { error_propagate(errp, local_err); diff --git a/blockdev.c b/blockdev.c index 1d1f27cfff..818ec42511 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2641,6 +2641,7 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, BdrvDirtyBitmap *bmap =3D NULL; BackupPerf perf =3D { .max_workers =3D 64 }; int job_flags =3D JOB_DEFAULT; + OnCbwError on_cbw_error =3D ON_CBW_ERROR_BREAK_GUEST_WRITE; =20 if (!backup->has_speed) { backup->speed =3D 0; @@ -2745,6 +2746,10 @@ static BlockJob *do_backup_common(BackupCommon *back= up, job_flags |=3D JOB_MANUAL_DISMISS; } =20 + if (backup->has_on_cbw_error) { + on_cbw_error =3D backup->on_cbw_error; + } + job =3D backup_job_create(backup->job_id, bs, target_bs, backup->speed, backup->sync, bmap, backup->bitmap_mode, backup->compress, backup->discard_source, @@ -2752,6 +2757,7 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, &perf, backup->on_source_error, backup->on_target_error, + on_cbw_error, job_flags, NULL, NULL, txn, errp); return job; } diff --git a/include/block/block_int-global-state.h b/include/block/block_i= nt-global-state.h index eb2d92a226..0d93783763 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -179,6 +179,7 @@ void mirror_start(const char *job_id, BlockDriverState = *bs, * all ".has_*" fields are ignored. * @on_source_error: The action to take upon error reading from the source. * @on_target_error: The action to take upon error writing to the target. + * @on_cbw_error: The action to take upon error in copy-before-write opera= tions. * @creation_flags: Flags that control the behavior of the Job lifetime. * See @BlockJobCreateFlags * @cb: Completion function for the job. @@ -198,6 +199,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, BackupPerf *perf, BlockdevOnError on_source_error, BlockdevOnError on_target_error, + OnCbwError on_cbw_error, int creation_flags, BlockCompletionFunc *cb, void *opaque, JobTxn *txn, Error **errp); diff --git a/qapi/block-core.json b/qapi/block-core.json index 22061227ca..91c70e24a7 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1602,6 +1602,9 @@ # default 'report' (no limitations, since this applies to a # different block device than @device). # +# @on-cbw-error: policy defining behavior on I/O errors in +# copy-before-write jobs; defaults to break-guest-write. (Since 10.1) +# # @auto-finalize: When false, this job will wait in a PENDING state # after it has finished its work, waiting for @block-job-finalize # before making any block graph changes. When true, this job will @@ -1641,6 +1644,7 @@ '*compress': 'bool', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', + '*on-cbw-error': 'OnCbwError', '*auto-finalize': 'bool', '*auto-dismiss': 'bool', '*filter-node-name': 'str', '*discard-source': 'bool', diff --git a/tests/qemu-iotests/tests/copy-before-write b/tests/qemu-iotest= s/tests/copy-before-write index 498c558008..23d70c7fe7 100755 --- a/tests/qemu-iotests/tests/copy-before-write +++ b/tests/qemu-iotests/tests/copy-before-write @@ -99,6 +99,66 @@ class TestCbwError(iotests.QMPTestCase): log =3D iotests.filter_qemu_io(log) return log =20 + def do_cbw_error_via_blockdev_backup(self, on_cbw_error=3DNone): + self.vm.cmd('blockdev-add', { + 'node-name': 'source', + 'driver': iotests.imgfmt, + 'file': { + 'driver': 'file', + 'filename': source_img + } + }) + + self.vm.cmd('blockdev-add', { + 'node-name': 'target', + 'driver': iotests.imgfmt, + 'file': { + 'driver': 'blkdebug', + 'image': { + 'driver': 'file', + 'filename': temp_img + }, + 'inject-error': [ + { + 'event': 'write_aio', + 'errno': 5, + 'immediately': False, + 'once': True + } + ] + } + }) + + blockdev_backup_options =3D { + 'device': 'source', + 'target': 'target', + 'sync': 'none', + 'job-id': 'job-id', + 'filter-node-name': 'cbw' + } + + if on_cbw_error: + blockdev_backup_options['on-cbw-error'] =3D on_cbw_error + + self.vm.cmd('blockdev-backup', blockdev_backup_options) + + self.vm.cmd('blockdev-add', { + 'node-name': 'access', + 'driver': 'snapshot-access', + 'file': 'cbw' + }) + + result =3D self.vm.qmp('human-monitor-command', command_line=3D'qe= mu-io cbw "write 0 1M"') + self.assert_qmp(result, 'return', '') + + result =3D self.vm.qmp('human-monitor-command', command_line=3D'qe= mu-io access "read 0 1M"') + self.assert_qmp(result, 'return', '') + + self.vm.shutdown() + log =3D self.vm.get_log() + log =3D iotests.filter_qemu_io(log) + return log + def test_break_snapshot_on_cbw_error(self): """break-snapshot behavior: Guest write succeed, but further snapshot-read fails, as snapshot = is @@ -123,6 +183,36 @@ read failed: Permission denied write failed: Input/output error read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +""") + + def test_break_snapshot_policy_forwarding(self): + """Ensure CBW filter accepts break-snapshot policy specified in bl= ockdev-backup QMP command. + """ + log =3D self.do_cbw_error_via_blockdev_backup('break-snapshot') + self.assertEqual(log, """\ +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read failed: Permission denied +""") + + def test_break_guest_write_policy_forwarding(self): + """Ensure CBW filter accepts break-guest-write policy specified in= blockdev-backup QMP command. + """ + log =3D self.do_cbw_error_via_blockdev_backup('break-guest-write') + self.assertEqual(log, """\ +write failed: Input/output error +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +""") + + def test_default_on_cbw_error_policy_forwarding(self): + """Ensure break-guest-write policy is used by default when on-cbw-= error is not explicitly specified. + """ + log =3D self.do_cbw_error_via_blockdev_backup() + self.assertEqual(log, """\ +write failed: Input/output error +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) """) =20 def do_cbw_timeout(self, on_cbw_error): diff --git a/tests/qemu-iotests/tests/copy-before-write.out b/tests/qemu-io= tests/tests/copy-before-write.out index 89968f35d7..2f7d3902f2 100644 --- a/tests/qemu-iotests/tests/copy-before-write.out +++ b/tests/qemu-iotests/tests/copy-before-write.out @@ -1,5 +1,5 @@ -.... +....... ---------------------------------------------------------------------- -Ran 4 tests +Ran 7 tests =20 OK --=20 2.48.1