From nobody Tue Apr 23 19:50:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555339941; cv=none; d=zoho.com; s=zohoarc; b=PqoSnLuRx9xy8XQ6jLiC5MSJ43F0fkcrOy0qW0dPSoRyeu+WdH+Lc1OcP4dmIRmkklCr70kJvYLI33cSn3+gWdRA7ADL/ZA7TgmBCHTnZhz9xK4Y7rgQF9qOghr7pqTzE0UJOw4IjFlwaUFf+RFHZ4zKoFChaoHcsJpb5DoHDiI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555339941; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=qrHfwoG24ZFUNZAWf/RcED1H+f4XTk8a1hUagd6FM/4=; b=i95uWceebv5fGuxs7kA8KYwDxX0kloE/SPc+MFsyHrYFBIQc8yLD+RhNy1Lhkp0rMhnlM4dV5dinVf5lNR7bVWswxQKeDDBsXqKT0f+rxVmkQNukagYykZCAk1QTeVi3bU5D3JpJpm1PPhoGilfr29If/2kkg73QO2dePCoUhxQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555339941140241.94576547244867; Mon, 15 Apr 2019 07:52:21 -0700 (PDT) Received: from localhost ([127.0.0.1]:51281 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG2xs-0006rh-3X for importer@patchew.org; Mon, 15 Apr 2019 10:52:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58351) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG2vo-0005Uc-Uv for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:50:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hG2vk-0005S2-18 for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:49:59 -0400 Received: from relay.sw.ru ([185.231.240.75]:48874) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hG2vg-0005QH-T6; Mon, 15 Apr 2019 10:49:54 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hG2ve-0002By-6n; Mon, 15 Apr 2019 17:49:50 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 15 Apr 2019 17:49:44 +0300 Message-Id: <20190415144948.78568-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190415144948.78568-1-vsementsov@virtuozzo.com> References: <20190415144948.78568-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 1/5] block/backup: simplify backup_incremental_init_copy_bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Simplify backup_incremental_init_copy_bitmap using the function bdrv_dirty_bitmap_next_dirty_area. Note: move to job->len instead of bitmap size: it should not matter but less code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 40 ++++++++++++---------------------------- 1 file changed, 12 insertions(+), 28 deletions(-) diff --git a/block/backup.c b/block/backup.c index 9988753249..d9f5db18ac 100644 --- a/block/backup.c +++ b/block/backup.c @@ -403,43 +403,27 @@ static int coroutine_fn backup_run_incremental(Backup= BlockJob *job) /* init copy_bitmap from sync_bitmap */ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) { - BdrvDirtyBitmapIter *dbi; - int64_t offset; - int64_t end =3D DIV_ROUND_UP(bdrv_dirty_bitmap_size(job->sync_bitmap), - job->cluster_size); - - dbi =3D bdrv_dirty_iter_new(job->sync_bitmap); - while ((offset =3D bdrv_dirty_iter_next(dbi)) !=3D -1) { - int64_t cluster =3D offset / job->cluster_size; - int64_t next_cluster; - - offset +=3D bdrv_dirty_bitmap_granularity(job->sync_bitmap); - if (offset >=3D bdrv_dirty_bitmap_size(job->sync_bitmap)) { - hbitmap_set(job->copy_bitmap, cluster, end - cluster); - break; - } + uint64_t offset =3D 0; + uint64_t bytes =3D job->len; =20 - offset =3D bdrv_dirty_bitmap_next_zero(job->sync_bitmap, offset, - UINT64_MAX); - if (offset =3D=3D -1) { - hbitmap_set(job->copy_bitmap, cluster, end - cluster); - break; - } + while (bdrv_dirty_bitmap_next_dirty_area(job->sync_bitmap, + &offset, &bytes)) + { + uint64_t cluster =3D offset / job->cluster_size; + uint64_t end_cluster =3D DIV_ROUND_UP(offset + bytes, job->cluster= _size); =20 - next_cluster =3D DIV_ROUND_UP(offset, job->cluster_size); - hbitmap_set(job->copy_bitmap, cluster, next_cluster - cluster); - if (next_cluster >=3D end) { + hbitmap_set(job->copy_bitmap, cluster, end_cluster - cluster); + + offset =3D end_cluster * job->cluster_size; + if (offset >=3D job->len) { break; } - - bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size); + bytes =3D job->len - offset; } =20 /* TODO job_progress_set_remaining() would make more sense */ job_progress_update(&job->common.job, job->len - hbitmap_count(job->copy_bitmap) * job->cluster_size); - - bdrv_dirty_iter_free(dbi); } =20 static int coroutine_fn backup_run(Job *job, Error **errp) --=20 2.18.0 From nobody Tue Apr 23 19:50:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555340084; cv=none; d=zoho.com; s=zohoarc; b=nC9Dh04NAZywzTLg0igPu80vb7TU8xNIbrM4BiqfrI8stByfrAdjI8/KlS9z34zw8O3hMcCyk9X9UOJqTn9ck599e+Qs4aFA0yriXC7w9n3Jt/6MdiVVGb5abCgC+6ucv2AN3vthp0BEdyHrWSp7696Pp8W7AEFbds4Re0b65PU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555340084; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=4PeVipFSNvymMNQ22JKy6BcoeQ66p8Ut4/KR4JTovl8=; b=ocBySC/ShGrpuShqkOo0qK+bhuEIzEhBdtUkr8ow42N5tAWZ/T9/3y10xirYm+jIb6rkeUHkq+4O+4WWUDngi1vDYMMYlnYYEDDTYvLlWPy6UkTsJcoFRtk0Xp0IEg7OHBghVYWfMPSs6E+VdeCqjLmLtZvVb+K6wXcNsZNn67w= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555340084019735.6339569004934; Mon, 15 Apr 2019 07:54:44 -0700 (PDT) Received: from localhost ([127.0.0.1]:51307 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG30G-0000Vv-0q for importer@patchew.org; Mon, 15 Apr 2019 10:54:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58355) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG2vo-0005Ug-VJ for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:50:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hG2vk-0005S1-13 for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:50:00 -0400 Received: from relay.sw.ru ([185.231.240.75]:48870) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hG2vh-0005QI-0Z; Mon, 15 Apr 2019 10:49:54 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hG2ve-0002By-A1; Mon, 15 Apr 2019 17:49:50 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 15 Apr 2019 17:49:45 +0300 Message-Id: <20190415144948.78568-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190415144948.78568-1-vsementsov@virtuozzo.com> References: <20190415144948.78568-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 2/5] block/backup: move to copy_bitmap with granularity X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We are going to share this bitmap between backup and backup-top filter driver, so let's share something more meaningful. It also simplifies some calculations. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 48 +++++++++++++++++++++++------------------------- 1 file changed, 23 insertions(+), 25 deletions(-) diff --git a/block/backup.c b/block/backup.c index d9f5db18ac..510fc54f98 100644 --- a/block/backup.c +++ b/block/backup.c @@ -113,7 +113,8 @@ static int coroutine_fn backup_cow_with_bounce_buffer(B= ackupBlockJob *job, int read_flags =3D is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; int write_flags =3D job->serialize_target_writes ? BDRV_REQ_SERIALISIN= G : 0; =20 - hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1); + assert(QEMU_IS_ALIGNED(start, job->cluster_size)); + hbitmap_reset(job->copy_bitmap, start, job->cluster_size); nbytes =3D MIN(job->cluster_size, job->len - start); if (!*bounce_buffer) { *bounce_buffer =3D blk_blockalign(blk, job->cluster_size); @@ -147,7 +148,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(B= ackupBlockJob *job, =20 return nbytes; fail: - hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1); + hbitmap_set(job->copy_bitmap, start, job->cluster_size); return ret; =20 } @@ -167,16 +168,15 @@ static int coroutine_fn backup_cow_with_offload(Backu= pBlockJob *job, int write_flags =3D job->serialize_target_writes ? BDRV_REQ_SERIALISIN= G : 0; =20 assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size)); + assert(QEMU_IS_ALIGNED(start, job->cluster_size)); nbytes =3D MIN(job->copy_range_size, end - start); nr_clusters =3D DIV_ROUND_UP(nbytes, job->cluster_size); - hbitmap_reset(job->copy_bitmap, start / job->cluster_size, - nr_clusters); + hbitmap_reset(job->copy_bitmap, start, job->cluster_size * nr_clusters= ); ret =3D blk_co_copy_range(blk, start, job->target, start, nbytes, read_flags, write_flags); if (ret < 0) { trace_backup_do_cow_copy_range_fail(job, start, ret); - hbitmap_set(job->copy_bitmap, start / job->cluster_size, - nr_clusters); + hbitmap_set(job->copy_bitmap, start, job->cluster_size * nr_cluste= rs); return ret; } =20 @@ -204,7 +204,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *j= ob, cow_request_begin(&cow_request, job, start, end); =20 while (start < end) { - if (!hbitmap_get(job->copy_bitmap, start / job->cluster_size)) { + if (!hbitmap_get(job->copy_bitmap, start)) { trace_backup_do_cow_skip(job, start); start +=3D job->cluster_size; continue; /* already copied */ @@ -300,6 +300,11 @@ static void backup_clean(Job *job) assert(s->target); blk_unref(s->target); s->target =3D NULL; + + if (s->copy_bitmap) { + hbitmap_free(s->copy_bitmap); + s->copy_bitmap =3D NULL; + } } =20 static void backup_attached_aio_context(BlockJob *job, AioContext *aio_con= text) @@ -312,7 +317,6 @@ static void backup_attached_aio_context(BlockJob *job, = AioContext *aio_context) void backup_do_checkpoint(BlockJob *job, Error **errp) { BackupBlockJob *backup_job =3D container_of(job, BackupBlockJob, commo= n); - int64_t len; =20 assert(block_job_driver(job) =3D=3D &backup_job_driver); =20 @@ -322,8 +326,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp) return; } =20 - len =3D DIV_ROUND_UP(backup_job->len, backup_job->cluster_size); - hbitmap_set(backup_job->copy_bitmap, 0, len); + hbitmap_set(backup_job->copy_bitmap, 0, backup_job->len); } =20 static void backup_drain(BlockJob *job) @@ -378,16 +381,16 @@ static int coroutine_fn backup_run_incremental(Backup= BlockJob *job) { int ret; bool error_is_read; - int64_t cluster; + int64_t offset; HBitmapIter hbi; =20 hbitmap_iter_init(&hbi, job->copy_bitmap, 0); - while ((cluster =3D hbitmap_iter_next(&hbi)) !=3D -1) { + while ((offset =3D hbitmap_iter_next(&hbi)) !=3D -1) { do { if (yield_and_check(job)) { return 0; } - ret =3D backup_do_cow(job, cluster * job->cluster_size, + ret =3D backup_do_cow(job, offset, job->cluster_size, &error_is_read, false); if (ret < 0 && backup_error_action(job, error_is_read, -ret) = =3D=3D BLOCK_ERROR_ACTION_REPORT) @@ -409,12 +412,9 @@ static void backup_incremental_init_copy_bitmap(Backup= BlockJob *job) while (bdrv_dirty_bitmap_next_dirty_area(job->sync_bitmap, &offset, &bytes)) { - uint64_t cluster =3D offset / job->cluster_size; - uint64_t end_cluster =3D DIV_ROUND_UP(offset + bytes, job->cluster= _size); + hbitmap_set(job->copy_bitmap, offset, bytes); =20 - hbitmap_set(job->copy_bitmap, cluster, end_cluster - cluster); - - offset =3D end_cluster * job->cluster_size; + offset +=3D bytes; if (offset >=3D job->len) { break; } @@ -423,30 +423,27 @@ static void backup_incremental_init_copy_bitmap(Backu= pBlockJob *job) =20 /* TODO job_progress_set_remaining() would make more sense */ job_progress_update(&job->common.job, - job->len - hbitmap_count(job->copy_bitmap) * job->cluster_size); + job->len - hbitmap_count(job->copy_bitmap)); } =20 static int coroutine_fn backup_run(Job *job, Error **errp) { BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); BlockDriverState *bs =3D blk_bs(s->common.blk); - int64_t offset, nb_clusters; + int64_t offset; int ret =3D 0; =20 QLIST_INIT(&s->inflight_reqs); qemu_co_rwlock_init(&s->flush_rwlock); =20 - nb_clusters =3D DIV_ROUND_UP(s->len, s->cluster_size); job_progress_set_remaining(job, s->len); =20 - s->copy_bitmap =3D hbitmap_alloc(nb_clusters, 0); if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { backup_incremental_init_copy_bitmap(s); } else { - hbitmap_set(s->copy_bitmap, 0, nb_clusters); + hbitmap_set(s->copy_bitmap, 0, s->len); } =20 - s->before_write.notify =3D backup_before_write_notify; bdrv_add_before_write_notifier(bs, &s->before_write); =20 @@ -527,7 +524,6 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) /* wait until pending backup_do_cow() calls have completed */ qemu_co_rwlock_wrlock(&s->flush_rwlock); qemu_co_rwlock_unlock(&s->flush_rwlock); - hbitmap_free(s->copy_bitmap); =20 return ret; } @@ -678,6 +674,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, } else { job->cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster= _size); } + + job->copy_bitmap =3D hbitmap_alloc(len, ctz32(job->cluster_size)); job->use_copy_range =3D true; job->copy_range_size =3D MIN_NON_ZERO(blk_get_max_transfer(job->common= .blk), blk_get_max_transfer(job->target)); --=20 2.18.0 From nobody Tue Apr 23 19:50:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555340077; cv=none; d=zoho.com; s=zohoarc; b=iwCowWzB4A9lhhpvAFqF2JYcfTHuedj28e44jd7oDzqrqVbosAt8s5JFWlCaLtmyqZUKj5Pgnbj80DAukTo2stz6ENonNywBEGbRHD0ZkmXaPWUo3WGossKWLEa+sg7yIjDf4WH0UsJXnxB57nrue/TxrAjUXA+EPXvKceROmSY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555340077; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=TuK9/l1Vl63r/5B3LDlYX+KujfBHi7TuDF3PjN7t2Zk=; b=Q90I9QuQdRpkggsTVEiS07+1B4uiUPIaXtcBhlu1NEDHFyU4ksA/WiRzLhFmLZuEcq2eH5GRuZqEbK3y4gP8YBMPF+TZL55lMC32YEXe+g0kie2V+ynDkux6LQkSGdGjOqme1yVYjmOu4jwlvfDvBcZW4ypqxA+75GIE+HsbOPA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555340077960188.55958610575146; Mon, 15 Apr 2019 07:54:37 -0700 (PDT) Received: from localhost ([127.0.0.1]:51305 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG30F-0000Vg-1S for importer@patchew.org; Mon, 15 Apr 2019 10:54:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58354) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG2vo-0005Uf-V5 for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:50:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hG2vj-0005Rn-OO for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:49:59 -0400 Received: from relay.sw.ru ([185.231.240.75]:48876) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hG2vg-0005QG-PW; Mon, 15 Apr 2019 10:49:53 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hG2ve-0002By-Eb; Mon, 15 Apr 2019 17:49:50 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 15 Apr 2019 17:49:46 +0300 Message-Id: <20190415144948.78568-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190415144948.78568-1-vsementsov@virtuozzo.com> References: <20190415144948.78568-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 3/5] block/backup: refactor and tolerate unallocated cluster skipping X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Split allocation checking to separate function and reduce nesting. Consider bdrv_is_allocated() fail as allocated area, as copying more than needed is not wrong (and we do it anyway) and seems better than fail the whole job. And, most probably we will fail on the next read, if there are real problem with source. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 60 +++++++++++++++++++------------------------------- 1 file changed, 23 insertions(+), 37 deletions(-) diff --git a/block/backup.c b/block/backup.c index 510fc54f98..298e85f1a9 100644 --- a/block/backup.c +++ b/block/backup.c @@ -377,6 +377,22 @@ static bool coroutine_fn yield_and_check(BackupBlockJo= b *job) return false; } =20 +static bool bdrv_is_unallocated_range(BlockDriverState *bs, + int64_t offset, int64_t bytes) +{ + int64_t end =3D offset + bytes; + + while (offset < end && !bdrv_is_allocated(bs, offset, bytes, &bytes)) { + if (bytes =3D=3D 0) { + return true; + } + offset +=3D bytes; + bytes =3D end - offset; + } + + return offset >=3D end; +} + static int coroutine_fn backup_run_incremental(BackupBlockJob *job) { int ret; @@ -462,49 +478,19 @@ static int coroutine_fn backup_run(Job *job, Error **= errp) for (offset =3D 0; offset < s->len; offset +=3D s->cluster_size) { bool error_is_read; - int alloced =3D 0; =20 if (yield_and_check(s)) { break; } =20 - if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP) { - int i; - int64_t n; - - /* Check to see if these blocks are already in the - * backing file. */ - - for (i =3D 0; i < s->cluster_size;) { - /* bdrv_is_allocated() only returns true/false based - * on the first set of sectors it comes across that - * are are all in the same state. - * For that reason we must verify each sector in the - * backup cluster length. We end up copying more than - * needed but at some point that is always the case. */ - alloced =3D - bdrv_is_allocated(bs, offset + i, - s->cluster_size - i, &n); - i +=3D n; - - if (alloced || n =3D=3D 0) { - break; - } - } - - /* If the above loop never found any sectors that are in - * the topmost image, skip this backup. */ - if (alloced =3D=3D 0) { - continue; - } - } - /* FULL sync mode we copy the whole drive. */ - if (alloced < 0) { - ret =3D alloced; - } else { - ret =3D backup_do_cow(s, offset, s->cluster_size, - &error_is_read, false); + if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP && + bdrv_is_unallocated_range(bs, offset, s->cluster_size)) + { + continue; } + + ret =3D backup_do_cow(s, offset, s->cluster_size, + &error_is_read, false); if (ret < 0) { /* Depending on error action, fail now or retry cluster */ BlockErrorAction action =3D --=20 2.18.0 From nobody Tue Apr 23 19:50:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555339940; cv=none; d=zoho.com; s=zohoarc; b=C9O5xqW163NWXxfNL/H3qJVLeVbbmxGDZRg2wwOHWjys2EsT3VWTXXmgV6QtejtnQt8IAXM6YWeW+0YumLxSaCmPGxIBc8wVd2LDQVTjB95E52xqFAUFXLYF8RdBya9+29VDu905ZuuhFcvqRcCvKuF+0BRNFaOeUeC9QkrUrJg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555339940; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=A77+10UythBPb0Y5OEj3wCDQ3yDVZfEN8Dnvfx4myJA=; b=RDbxLLEsK8kPHl3x5Hwqb7F5npWJrYn+mRCWK/KpOa6e0gY3pUhFkb4KOIUxdW6fpz/q+G7fY6BwSMAIlmfenLSUeCelb0Wj418X+UDTYInuIG9vyQlwYruxraGU3Y2mO9+Hoz2RHUHTlZ0lcUDQIvugk+koCISfEv0iDKdhxLM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1555339940120915.7773915168564; Mon, 15 Apr 2019 07:52:20 -0700 (PDT) Received: from localhost ([127.0.0.1]:51279 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG2xs-0006rd-B2 for importer@patchew.org; Mon, 15 Apr 2019 10:52:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58350) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG2vo-0005Ub-Ut for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:50:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hG2vk-0005SB-1H for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:49:59 -0400 Received: from relay.sw.ru ([185.231.240.75]:48882) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hG2vg-0005QR-T2; Mon, 15 Apr 2019 10:49:54 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hG2ve-0002By-JB; Mon, 15 Apr 2019 17:49:50 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 15 Apr 2019 17:49:47 +0300 Message-Id: <20190415144948.78568-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190415144948.78568-1-vsementsov@virtuozzo.com> References: <20190415144948.78568-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 4/5] block/backup: unify different modes code path X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Do full, top and incremental mode copying all in one place. This unifies the code path and helps further improvements. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 43 ++++++++++--------------------------------- 1 file changed, 10 insertions(+), 33 deletions(-) diff --git a/block/backup.c b/block/backup.c index 298e85f1a9..b54386b699 100644 --- a/block/backup.c +++ b/block/backup.c @@ -393,15 +393,23 @@ static bool bdrv_is_unallocated_range(BlockDriverStat= e *bs, return offset >=3D end; } =20 -static int coroutine_fn backup_run_incremental(BackupBlockJob *job) +static int coroutine_fn backup_loop(BackupBlockJob *job) { int ret; bool error_is_read; int64_t offset; HBitmapIter hbi; + BlockDriverState *bs =3D blk_bs(job->common.blk); =20 hbitmap_iter_init(&hbi, job->copy_bitmap, 0); while ((offset =3D hbitmap_iter_next(&hbi)) !=3D -1) { + if (job->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP && + bdrv_is_unallocated_range(bs, offset, job->cluster_size)) + { + hbitmap_reset(job->copy_bitmap, offset, job->cluster_size); + continue; + } + do { if (yield_and_check(job)) { return 0; @@ -446,7 +454,6 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) { BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); BlockDriverState *bs =3D blk_bs(s->common.blk); - int64_t offset; int ret =3D 0; =20 QLIST_INIT(&s->inflight_reqs); @@ -471,38 +478,8 @@ static int coroutine_fn backup_run(Job *job, Error **e= rrp) * notify callback service CoW requests. */ job_yield(job); } - } else if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { - ret =3D backup_run_incremental(s); } else { - /* Both FULL and TOP SYNC_MODE's require copying.. */ - for (offset =3D 0; offset < s->len; - offset +=3D s->cluster_size) { - bool error_is_read; - - if (yield_and_check(s)) { - break; - } - - if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP && - bdrv_is_unallocated_range(bs, offset, s->cluster_size)) - { - continue; - } - - ret =3D backup_do_cow(s, offset, s->cluster_size, - &error_is_read, false); - if (ret < 0) { - /* Depending on error action, fail now or retry cluster */ - BlockErrorAction action =3D - backup_error_action(s, error_is_read, -ret); - if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { - break; - } else { - offset -=3D s->cluster_size; - continue; - } - } - } + ret =3D backup_loop(s); } =20 notifier_with_return_remove(&s->before_write); --=20 2.18.0 From nobody Tue Apr 23 19:50:58 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1555339941; cv=none; d=zoho.com; s=zohoarc; b=cS7qYcHT5xk1gCHsAG9EnqkQ3zN1qGzH89cErWb4zK1ezAZDfjraeI9yympkGQcmIr/PcFky2P2di29gkkOh9NANOlKLYdhszjatX1vRN1ToQp6AsNLUoiNOISlOtjrJG5qyNPOKl+DEUf5bK3QIN0WYsX6zqzYRWkXxeIDFA+g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555339941; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=qw3UOK0Jc/18MZDrThOQXOJojDv7sUm+z9BGfTxmC7o=; b=Y+zULNf5m64BlQcrSitjMs8+uYX/DbLS04rC129lrZN0sZGld7FwGi7TTJqt0cHoLo8uZMi4J+RcdXy6VFtPbTSUqvSXIy6gLRnhEVqmdCgTbXGeyFCLAmsMgrfHhBCAt9Q3CZv1zn4JU70jRrV/xOG/lX0R1M1jQpRm6dGvpQ8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155533994129011.98703021993299; Mon, 15 Apr 2019 07:52:21 -0700 (PDT) Received: from localhost ([127.0.0.1]:51283 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG2xu-0006sj-SG for importer@patchew.org; Mon, 15 Apr 2019 10:52:10 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58352) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hG2vo-0005Ud-V6 for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:50:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hG2vk-0005SF-15 for qemu-devel@nongnu.org; Mon, 15 Apr 2019 10:49:59 -0400 Received: from relay.sw.ru ([185.231.240.75]:48884) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hG2vg-0005QT-Sw; Mon, 15 Apr 2019 10:49:54 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hG2ve-0002By-Rc; Mon, 15 Apr 2019 17:49:50 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 15 Apr 2019 17:49:48 +0300 Message-Id: <20190415144948.78568-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190415144948.78568-1-vsementsov@virtuozzo.com> References: <20190415144948.78568-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 5/5] block/backup: refactor: split out backup_calculate_cluster_size X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Split out cluster_size calculation. Move copy-bitmap creation above block-job creation, as we are going to share it with upcoming backup-top filter, which also should be created before actual block job creation. Also, while being here, drop unnecessary "goto error" from bdrv_getlength error path. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 84 +++++++++++++++++++++++++++++++------------------- 1 file changed, 53 insertions(+), 31 deletions(-) diff --git a/block/backup.c b/block/backup.c index b54386b699..79db4fbb6d 100644 --- a/block/backup.c +++ b/block/backup.c @@ -507,6 +507,42 @@ static const BlockJobDriver backup_job_driver =3D { .drain =3D backup_drain, }; =20 +static int64_t backup_calculate_cluster_size(BlockDriverState *target, + Error **errp) +{ + int ret; + BlockDriverInfo bdi; + + /* + * If there is no backing file on the target, we cannot rely on COW if= our + * backup cluster size is smaller than the target cluster size. Even f= or + * targets with a backing file, try to avoid COW if possible. + */ + ret =3D bdrv_get_info(target, &bdi); + if (ret =3D=3D -ENOTSUP && !target->backing) { + /* Cluster size is not defined */ + warn_report("The target block device doesn't provide " + "information about the block size and it doesn't have = a " + "backing file. The default block size of %u bytes is " + "used. If the actual block size of the target exceeds " + "this default, the backup may be unusable", + BACKUP_CLUSTER_SIZE_DEFAULT); + return BACKUP_CLUSTER_SIZE_DEFAULT; + } else if (ret < 0 && !target->backing) { + error_setg_errno(errp, -ret, + "Couldn't determine the cluster size of the target image, " + "which has no backing file"); + error_append_hint(errp, + "Aborting, since this may create an unusable destination image= \n"); + return ret; + } else if (ret < 0 && target->backing) { + /* Not fatal; just trudge on ahead. */ + return BACKUP_CLUSTER_SIZE_DEFAULT; + } + + return MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size); +} + BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, @@ -518,9 +554,10 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, JobTxn *txn, Error **errp) { int64_t len; - BlockDriverInfo bdi; BackupBlockJob *job =3D NULL; int ret; + int64_t cluster_size; + HBitmap *copy_bitmap =3D NULL; =20 assert(bs); assert(target); @@ -579,9 +616,16 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, if (len < 0) { error_setg_errno(errp, -len, "unable to get length for '%s'", bdrv_get_device_name(bs)); - goto error; + return NULL; + } + + cluster_size =3D backup_calculate_cluster_size(target, errp); + if (cluster_size < 0) { + return NULL; } =20 + copy_bitmap =3D hbitmap_alloc(len, ctz32(cluster_size)); + /* job->len is fixed, so we can't allow resize */ job =3D block_job_create(job_id, &backup_job_driver, txn, bs, BLK_PERM_CONSISTENT_READ, @@ -610,35 +654,9 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, =20 /* Detect image-fleecing (and similar) schemes */ job->serialize_target_writes =3D bdrv_chain_contains(target, bs); - - /* If there is no backing file on the target, we cannot rely on COW if= our - * backup cluster size is smaller than the target cluster size. Even f= or - * targets with a backing file, try to avoid COW if possible. */ - ret =3D bdrv_get_info(target, &bdi); - if (ret =3D=3D -ENOTSUP && !target->backing) { - /* Cluster size is not defined */ - warn_report("The target block device doesn't provide " - "information about the block size and it doesn't have = a " - "backing file. The default block size of %u bytes is " - "used. If the actual block size of the target exceeds " - "this default, the backup may be unusable", - BACKUP_CLUSTER_SIZE_DEFAULT); - job->cluster_size =3D BACKUP_CLUSTER_SIZE_DEFAULT; - } else if (ret < 0 && !target->backing) { - error_setg_errno(errp, -ret, - "Couldn't determine the cluster size of the target image, " - "which has no backing file"); - error_append_hint(errp, - "Aborting, since this may create an unusable destination image= \n"); - goto error; - } else if (ret < 0 && target->backing) { - /* Not fatal; just trudge on ahead. */ - job->cluster_size =3D BACKUP_CLUSTER_SIZE_DEFAULT; - } else { - job->cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster= _size); - } - - job->copy_bitmap =3D hbitmap_alloc(len, ctz32(job->cluster_size)); + job->cluster_size =3D cluster_size; + job->copy_bitmap =3D copy_bitmap; + copy_bitmap =3D NULL; job->use_copy_range =3D true; job->copy_range_size =3D MIN_NON_ZERO(blk_get_max_transfer(job->common= .blk), blk_get_max_transfer(job->target)); @@ -654,6 +672,10 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, return &job->common; =20 error: + if (copy_bitmap) { + assert(!job || !job->copy_bitmap); + hbitmap_free(copy_bitmap); + } if (sync_bitmap) { bdrv_reclaim_dirty_bitmap(bs, sync_bitmap, NULL); } --=20 2.18.0