From nobody Sun Feb 8 18:43:15 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1583922977; cv=none; d=zohomail.com; s=zohoarc; b=EyLOIsJ55gIGwj50SeY92y0cS41GPERAkgUgNDP4K19/w8By/ep91/q3XIc1905wiunKoNszTy1fyLw/tuKKbaff0yT3dvfnYZHrXlKjnuln/LImCFFTkvR9ugiTx0MEYiHNHd4T/9NqFNgRV9FwNFj1GD8m9T+J+gCJpuSIfyM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1583922977; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=1haaOhSQet8Ak4eyr5yopGdEHB1al3VDklDufx0Sdvo=; b=A3rpJMhveQPq5Hgqy1g34ZbWdujaMIWWmE2ktEynFkiKtYOpU9QR2pSsUZC90LI+eGDZoHhU8BPqikIpI0vz4/ClpmEPOg+i1pDVY3iDPmGwzMYDq1VMgvZydSKk7PQ9ix1zP7q5a1AVgXLrN/lxgm6U0mjPwkz8Tr3/SaK5A9k= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1583922977566671.7337031711783; Wed, 11 Mar 2020 03:36:17 -0700 (PDT) Received: from localhost ([::1]:49388 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jByim-0004M1-Jd for importer@patchew.org; Wed, 11 Mar 2020 06:36:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43655) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jBydC-0002zs-K9 for qemu-devel@nongnu.org; Wed, 11 Mar 2020 06:30:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jBydB-0001Zp-0c for qemu-devel@nongnu.org; Wed, 11 Mar 2020 06:30:30 -0400 Received: from relay.sw.ru ([185.231.240.75]:33074) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1jByd3-0001Ro-Hu; Wed, 11 Mar 2020 06:30:21 -0400 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1jBycx-0005cJ-TW; Wed, 11 Mar 2020 13:30:16 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v4 2/9] block/block-copy: fix progress calculation Date: Wed, 11 Mar 2020 13:29:57 +0300 Message-Id: <20200311103004.7649-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200311103004.7649-1-vsementsov@virtuozzo.com> References: <20200311103004.7649-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-stable@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Assume we have two regions, A and B, and region B is in-flight now, region A is not yet touched, but it is unallocated and should be skipped. Correspondingly, as progress we have total =3D A + B current =3D 0 If we reset unallocated region A and call progress_reset_callback, it will calculate 0 bytes dirty in the bitmap and call job_progress_set_remaining, which will set total =3D current + 0 =3D 0 + 0 =3D 0 So, B bytes are actually removed from total accounting. When job finishes we'll have total =3D 0 current =3D B , which doesn't sound good. This is because we didn't considered in-flight bytes, actually when calculating remaining, we should have set (in_flight + dirty_bytes) as remaining, not only dirty_bytes. To fix it, let's refactor progress calculation, moving it to block-copy itself instead of fixing callback. And, of course, track in_flight bytes count. We still have to keep one callback, to maintain backup job bytes_read calculation, but it will go on soon, when we turn the whole backup process into one block_copy call. Cc: qemu-stable@nongnu.org Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich --- include/block/block-copy.h | 15 +++++---------- block/backup.c | 13 ++----------- block/block-copy.c | 16 ++++++++++++---- 3 files changed, 19 insertions(+), 25 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 0a161724d7..9def00068c 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -26,7 +26,6 @@ typedef struct BlockCopyInFlightReq { } BlockCopyInFlightReq; =20 typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque); -typedef void (*ProgressResetCallbackFunc)(void *opaque); typedef struct BlockCopyState { /* * BdrvChild objects are not owned or managed by block-copy. They are @@ -36,6 +35,7 @@ typedef struct BlockCopyState { BdrvChild *source; BdrvChild *target; BdrvDirtyBitmap *copy_bitmap; + int64_t in_flight_bytes; int64_t cluster_size; bool use_copy_range; int64_t copy_size; @@ -60,15 +60,9 @@ typedef struct BlockCopyState { */ bool skip_unallocated; =20 + ProgressMeter *progress; /* progress_bytes_callback: called when some copying progress is done.= */ ProgressBytesCallbackFunc progress_bytes_callback; - - /* - * progress_reset_callback: called when some bytes reset from copy_bit= map - * (see @skip_unallocated above). The callee is assumed to recalculate= how - * many bytes remain based on the dirty bit count of copy_bitmap. - */ - ProgressResetCallbackFunc progress_reset_callback; void *progress_opaque; =20 SharedResource *mem; @@ -79,12 +73,13 @@ BlockCopyState *block_copy_state_new(BdrvChild *source,= BdrvChild *target, BdrvRequestFlags write_flags, Error **errp); =20 -void block_copy_set_callbacks( +void block_copy_set_progress_callback( BlockCopyState *s, ProgressBytesCallbackFunc progress_bytes_callback, - ProgressResetCallbackFunc progress_reset_callback, void *progress_opaque); =20 +void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm); + void block_copy_state_free(BlockCopyState *s); =20 int64_t block_copy_reset_unallocated(BlockCopyState *s, diff --git a/block/backup.c b/block/backup.c index 1383e219f5..8694e0394b 100644 --- a/block/backup.c +++ b/block/backup.c @@ -57,15 +57,6 @@ static void backup_progress_bytes_callback(int64_t bytes= , void *opaque) BackupBlockJob *s =3D opaque; =20 s->bytes_read +=3D bytes; - job_progress_update(&s->common.job, bytes); -} - -static void backup_progress_reset_callback(void *opaque) -{ - BackupBlockJob *s =3D opaque; - uint64_t estimate =3D bdrv_get_dirty_count(s->bcs->copy_bitmap); - - job_progress_set_remaining(&s->common.job, estimate); } =20 static int coroutine_fn backup_do_cow(BackupBlockJob *job, @@ -464,8 +455,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, job->cluster_size =3D cluster_size; job->len =3D len; =20 - block_copy_set_callbacks(bcs, backup_progress_bytes_callback, - backup_progress_reset_callback, job); + block_copy_set_progress_callback(bcs, backup_progress_bytes_callback, = job); + block_copy_set_progress_meter(bcs, &job->common.job.progress); =20 /* Required permissions are already taken by backup-top target */ block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, diff --git a/block/block-copy.c b/block/block-copy.c index 79798a1567..e2d7b3b887 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -127,17 +127,20 @@ BlockCopyState *block_copy_state_new(BdrvChild *sourc= e, BdrvChild *target, return s; } =20 -void block_copy_set_callbacks( +void block_copy_set_progress_callback( BlockCopyState *s, ProgressBytesCallbackFunc progress_bytes_callback, - ProgressResetCallbackFunc progress_reset_callback, void *progress_opaque) { s->progress_bytes_callback =3D progress_bytes_callback; - s->progress_reset_callback =3D progress_reset_callback; s->progress_opaque =3D progress_opaque; } =20 +void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm) +{ + s->progress =3D pm; +} + /* * block_copy_do_copy * @@ -269,7 +272,9 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, =20 if (!ret) { bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); - s->progress_reset_callback(s->progress_opaque); + progress_set_remaining(s->progress, + bdrv_get_dirty_count(s->copy_bitmap) + + s->in_flight_bytes); } =20 *count =3D bytes; @@ -331,15 +336,18 @@ int coroutine_fn block_copy(BlockCopyState *s, trace_block_copy_process(s, start); =20 bdrv_reset_dirty_bitmap(s->copy_bitmap, start, chunk_end - start); + s->in_flight_bytes +=3D chunk_end - start; =20 co_get_from_shres(s->mem, chunk_end - start); ret =3D block_copy_do_copy(s, start, chunk_end, error_is_read); co_put_to_shres(s->mem, chunk_end - start); + s->in_flight_bytes -=3D chunk_end - start; if (ret < 0) { bdrv_set_dirty_bitmap(s->copy_bitmap, start, chunk_end - start= ); break; } =20 + progress_work_done(s->progress, chunk_end - start); s->progress_bytes_callback(chunk_end - start, s->progress_opaque); start =3D chunk_end; ret =3D 0; --=20 2.21.0