From nobody Fri Nov 14 00:48:49 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1583480410; cv=none; d=zohomail.com; s=zohoarc; b=lGwT84d3EoXaQjcfPBE4o8fFoh6V4iMpWowl1frD3RoVCTppVPv5xMdEl0mMtlG1xKke1SsdMSyzbwh2le47Si0nUsyJHxYdi4sNHwrL1yihnOGPN55NrbNsBrxhxpAnuEgD1CfQcvvwhKU8e1TCUeiGNBPYm6X9JI/uTx2QXkQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1583480410; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=TWkx4DRzed40+a2KJtc0Z7ofT9DXaI/klFil+5QwDjY=; b=H7/wi3eRFFNipBX69N8Ef6mUktioG4ztkI562tG/W/ptXEOSTQQ2EJSKnYLVzrh6DZa4C3hfiH+N9lqQf6BpHExABC+XFFhliE34TGpOiKASF4UFVQhWemwA4g5MDWdmBHzxwg1X0YXAsICn5kpm534Xd3BB7TbqjRttnYsPFIc= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 158348041071843.35632339900258; Thu, 5 Mar 2020 23:40:10 -0800 (PST) Received: from localhost ([::1]:60480 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jA7ab-0007b4-Hf for importer@patchew.org; Fri, 06 Mar 2020 02:40:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46315) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jA7ZI-00055Z-88 for qemu-devel@nongnu.org; Fri, 06 Mar 2020 02:38:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jA7ZG-0008M9-Ev for qemu-devel@nongnu.org; Fri, 06 Mar 2020 02:38:48 -0500 Received: from relay.sw.ru ([185.231.240.75]:37996) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1jA7ZC-0007x5-Kd; Fri, 06 Mar 2020 02:38:42 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1jA7Z8-0002Qu-4f; Fri, 06 Mar 2020 10:38:38 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v3 2/9] block/block-copy: fix progress calculation Date: Fri, 6 Mar 2020 10:38:24 +0300 Message-Id: <20200306073831.7737-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200306073831.7737-1-vsementsov@virtuozzo.com> References: <20200306073831.7737-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-stable@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Assume we have two regions, A and B, and region B is in-flight now, region A is not yet touched, but it is unallocated and should be skipped. Correspondingly, as progress we have total =3D A + B current =3D 0 If we reset unallocated region A and call progress_reset_callback, it will calculate 0 bytes dirty in the bitmap and call job_progress_set_remaining, which will set total =3D current + 0 =3D 0 + 0 =3D 0 So, B bytes are actually removed from total accounting. When job finishes we'll have total =3D 0 current =3D B , which doesn't sound good. This is because we didn't considered in-flight bytes, actually when calculating remaining, we should have set (in_flight + dirty_bytes) as remaining, not only dirty_bytes. To fix it, let's refactor progress calculation, moving it to block-copy itself instead of fixing callback. And, of course, track in_flight bytes count. We still have to keep one callback, to maintain backup job bytes_read calculation, but it will go on soon, when we turn the whole backup process into one block_copy call. Cc: qemu-stable@nongnu.org Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich --- include/block/block-copy.h | 14 +++++--------- block/backup.c | 13 ++----------- block/block-copy.c | 16 ++++++++++++---- 3 files changed, 19 insertions(+), 24 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 0a161724d7..dc68c8c54e 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -36,6 +36,7 @@ typedef struct BlockCopyState { BdrvChild *source; BdrvChild *target; BdrvDirtyBitmap *copy_bitmap; + int64_t in_flight_bytes; int64_t cluster_size; bool use_copy_range; int64_t copy_size; @@ -60,15 +61,9 @@ typedef struct BlockCopyState { */ bool skip_unallocated; =20 + ProgressMeter *progress; /* progress_bytes_callback: called when some copying progress is done.= */ ProgressBytesCallbackFunc progress_bytes_callback; - - /* - * progress_reset_callback: called when some bytes reset from copy_bit= map - * (see @skip_unallocated above). The callee is assumed to recalculate= how - * many bytes remain based on the dirty bit count of copy_bitmap. - */ - ProgressResetCallbackFunc progress_reset_callback; void *progress_opaque; =20 SharedResource *mem; @@ -79,12 +74,13 @@ BlockCopyState *block_copy_state_new(BdrvChild *source,= BdrvChild *target, BdrvRequestFlags write_flags, Error **errp); =20 -void block_copy_set_callbacks( +void block_copy_set_progress_callback( BlockCopyState *s, ProgressBytesCallbackFunc progress_bytes_callback, - ProgressResetCallbackFunc progress_reset_callback, void *progress_opaque); =20 +void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm); + void block_copy_state_free(BlockCopyState *s); =20 int64_t block_copy_reset_unallocated(BlockCopyState *s, diff --git a/block/backup.c b/block/backup.c index 1383e219f5..8694e0394b 100644 --- a/block/backup.c +++ b/block/backup.c @@ -57,15 +57,6 @@ static void backup_progress_bytes_callback(int64_t bytes= , void *opaque) BackupBlockJob *s =3D opaque; =20 s->bytes_read +=3D bytes; - job_progress_update(&s->common.job, bytes); -} - -static void backup_progress_reset_callback(void *opaque) -{ - BackupBlockJob *s =3D opaque; - uint64_t estimate =3D bdrv_get_dirty_count(s->bcs->copy_bitmap); - - job_progress_set_remaining(&s->common.job, estimate); } =20 static int coroutine_fn backup_do_cow(BackupBlockJob *job, @@ -464,8 +455,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, job->cluster_size =3D cluster_size; job->len =3D len; =20 - block_copy_set_callbacks(bcs, backup_progress_bytes_callback, - backup_progress_reset_callback, job); + block_copy_set_progress_callback(bcs, backup_progress_bytes_callback, = job); + block_copy_set_progress_meter(bcs, &job->common.job.progress); =20 /* Required permissions are already taken by backup-top target */ block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, diff --git a/block/block-copy.c b/block/block-copy.c index 79798a1567..e2d7b3b887 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -127,17 +127,20 @@ BlockCopyState *block_copy_state_new(BdrvChild *sourc= e, BdrvChild *target, return s; } =20 -void block_copy_set_callbacks( +void block_copy_set_progress_callback( BlockCopyState *s, ProgressBytesCallbackFunc progress_bytes_callback, - ProgressResetCallbackFunc progress_reset_callback, void *progress_opaque) { s->progress_bytes_callback =3D progress_bytes_callback; - s->progress_reset_callback =3D progress_reset_callback; s->progress_opaque =3D progress_opaque; } =20 +void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm) +{ + s->progress =3D pm; +} + /* * block_copy_do_copy * @@ -269,7 +272,9 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, =20 if (!ret) { bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); - s->progress_reset_callback(s->progress_opaque); + progress_set_remaining(s->progress, + bdrv_get_dirty_count(s->copy_bitmap) + + s->in_flight_bytes); } =20 *count =3D bytes; @@ -331,15 +336,18 @@ int coroutine_fn block_copy(BlockCopyState *s, trace_block_copy_process(s, start); =20 bdrv_reset_dirty_bitmap(s->copy_bitmap, start, chunk_end - start); + s->in_flight_bytes +=3D chunk_end - start; =20 co_get_from_shres(s->mem, chunk_end - start); ret =3D block_copy_do_copy(s, start, chunk_end, error_is_read); co_put_to_shres(s->mem, chunk_end - start); + s->in_flight_bytes -=3D chunk_end - start; if (ret < 0) { bdrv_set_dirty_bitmap(s->copy_bitmap, start, chunk_end - start= ); break; } =20 + progress_work_done(s->progress, chunk_end - start); s->progress_bytes_callback(chunk_end - start, s->progress_opaque); start =3D chunk_end; ret =3D 0; --=20 2.21.0