From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390342857395.1380022130734; Mon, 1 Oct 2018 03:39:02 -0700 (PDT) Received: from localhost ([::1]:37202 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vbJ-0002gw-G7 for importer@patchew.org; Mon, 01 Oct 2018 06:38:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40383) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSN-0004qd-MN for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002Kx-Dr for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:39654) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSK-0002JP-1n; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSD-0005Yj-N3; Mon, 01 Oct 2018 13:29:30 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:11 +0300 Message-Id: <20181001102928.20533-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 01/18] block/dirty-bitmap: allow set/reset bits in disabled bitmaps X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It is needed for use the bitmaps in backup. "disabled" means that bitmap is not auto set by writes. Let's allow changing bitmap for other uses. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 89c11111ae..65d2e92ec3 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -532,7 +532,6 @@ int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter) void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t offset, int64_t bytes) { - assert(bdrv_dirty_bitmap_enabled(bitmap)); assert(!bdrv_dirty_bitmap_readonly(bitmap)); hbitmap_set(bitmap->bitmap, offset, bytes); } @@ -549,7 +548,6 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap, void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t offset, int64_t bytes) { - assert(bdrv_dirty_bitmap_enabled(bitmap)); assert(!bdrv_dirty_bitmap_readonly(bitmap)); hbitmap_reset(bitmap->bitmap, offset, bytes); } --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538389948327129.14369968856386; Mon, 1 Oct 2018 03:32:28 -0700 (PDT) Received: from localhost ([::1]:37164 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vUv-0006DZ-6Y for importer@patchew.org; Mon, 01 Oct 2018 06:32:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40379) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSN-0004qa-MH for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002LM-PX for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:39686) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSK-0002JY-Dz; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSE-0005Yj-8d; Mon, 01 Oct 2018 13:29:30 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:12 +0300 Message-Id: <20181001102928.20533-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 02/18] block/io: allow BDRV_REQ_SERIALISING for read X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It will be used in further commit, changing backup architecture. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/io.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/block/io.c b/block/io.c index bd9d688f8b..b003b4d5bf 100644 --- a/block/io.c +++ b/block/io.c @@ -1318,10 +1318,11 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChi= ld *child, * potential fallback support, if we ever implement any read flags * to pass through to drivers. For now, there aren't any * passthrough flags. */ - assert(!(flags & ~(BDRV_REQ_NO_SERIALISING | BDRV_REQ_COPY_ON_READ))); + assert(!(flags & ~(BDRV_REQ_NO_SERIALISING | BDRV_REQ_COPY_ON_READ | + BDRV_REQ_SERIALISING))); =20 /* Handle Copy on Read and associated serialisation */ - if (flags & BDRV_REQ_COPY_ON_READ) { + if (flags & (BDRV_REQ_COPY_ON_READ | BDRV_REQ_SERIALISING)) { /* If we touch the same cluster it counts as an overlap. This * guarantees that allocating writes will be serialized and not ra= ce * with each other for the same cluster. For example, in copy-on-= read @@ -1330,9 +1331,6 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild= *child, mark_request_serialising(req, bdrv_get_cluster_size(bs)); } =20 - /* BDRV_REQ_SERIALISING is only for write operation */ - assert(!(flags & BDRV_REQ_SERIALISING)); - if (!(flags & BDRV_REQ_NO_SERIALISING)) { wait_serialising_requests(req); } @@ -3027,8 +3025,9 @@ static int coroutine_fn bdrv_co_copy_range_internal( tracked_request_begin(&req, src->bs, src_offset, bytes, BDRV_TRACKED_READ); =20 - /* BDRV_REQ_SERIALISING is only for write operation */ - assert(!(read_flags & BDRV_REQ_SERIALISING)); + if (read_flags & BDRV_REQ_SERIALISING) { + mark_request_serialising(&req, bdrv_get_cluster_size(src->bs)); + } if (!(read_flags & BDRV_REQ_NO_SERIALISING)) { wait_serialising_requests(&req); } --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390146599159.17542811870737; Mon, 1 Oct 2018 03:35:46 -0700 (PDT) Received: from localhost ([::1]:37181 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vYH-0000EM-AC for importer@patchew.org; Mon, 01 Oct 2018 06:35:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40388) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSN-0004qi-MX for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002LX-QQ for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:39650) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSK-0002JO-ED; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSE-0005Yj-Fx; Mon, 01 Oct 2018 13:29:30 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:13 +0300 Message-Id: <20181001102928.20533-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 03/18] block/backup: simplify backup_incremental_init_copy_bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Simplify backup_incremental_init_copy_bitmap using the function bdrv_dirty_bitmap_next_dirty_area. Note: move to job->len instead of bitmap size: it should not matter but less code. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 40 ++++++++++++---------------------------- 1 file changed, 12 insertions(+), 28 deletions(-) diff --git a/block/backup.c b/block/backup.c index 435414e964..fbe7ce19e1 100644 --- a/block/backup.c +++ b/block/backup.c @@ -406,43 +406,27 @@ static int coroutine_fn backup_run_incremental(Backup= BlockJob *job) /* init copy_bitmap from sync_bitmap */ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) { - BdrvDirtyBitmapIter *dbi; - int64_t offset; - int64_t end =3D DIV_ROUND_UP(bdrv_dirty_bitmap_size(job->sync_bitmap), - job->cluster_size); - - dbi =3D bdrv_dirty_iter_new(job->sync_bitmap); - while ((offset =3D bdrv_dirty_iter_next(dbi)) !=3D -1) { - int64_t cluster =3D offset / job->cluster_size; - int64_t next_cluster; - - offset +=3D bdrv_dirty_bitmap_granularity(job->sync_bitmap); - if (offset >=3D bdrv_dirty_bitmap_size(job->sync_bitmap)) { - hbitmap_set(job->copy_bitmap, cluster, end - cluster); - break; - } + uint64_t offset =3D 0; + uint64_t bytes =3D job->len; =20 - offset =3D bdrv_dirty_bitmap_next_zero(job->sync_bitmap, offset, - UINT64_MAX); - if (offset =3D=3D -1) { - hbitmap_set(job->copy_bitmap, cluster, end - cluster); - break; - } + while (bdrv_dirty_bitmap_next_dirty_area(job->sync_bitmap, + &offset, &bytes)) + { + uint64_t cluster =3D offset / job->cluster_size; + uint64_t last_cluster =3D (offset + bytes) / job->cluster_size; =20 - next_cluster =3D DIV_ROUND_UP(offset, job->cluster_size); - hbitmap_set(job->copy_bitmap, cluster, next_cluster - cluster); - if (next_cluster >=3D end) { + hbitmap_set(job->copy_bitmap, cluster, last_cluster - cluster + 1); + + offset =3D (last_cluster + 1) * job->cluster_size; + if (offset >=3D job->len) { break; } - - bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size); + bytes =3D job->len - offset; } =20 /* TODO job_progress_set_remaining() would make more sense */ job_progress_update(&job->common.job, job->len - hbitmap_count(job->copy_bitmap) * job->cluster_size); - - bdrv_dirty_iter_free(dbi); } =20 static int coroutine_fn backup_run(Job *job, Error **errp) --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538389947893788.064561126088; Mon, 1 Oct 2018 03:32:27 -0700 (PDT) Received: from localhost ([::1]:37167 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vUx-0006F2-2K for importer@patchew.org; Mon, 01 Oct 2018 06:32:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40384) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSN-0004qe-MQ for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002Kl-BC for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:39674) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSJ-0002JV-VP; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSE-0005Yj-Or; Mon, 01 Oct 2018 13:29:30 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:14 +0300 Message-Id: <20181001102928.20533-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 04/18] block/backup: move from HBitmap to BdrvDirtyBitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This is a necessary step to share copy-bitmap between backup job and special filter driver in further commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 73 ++++++++++++++++++++++++++++---------------------- 1 file changed, 41 insertions(+), 32 deletions(-) diff --git a/block/backup.c b/block/backup.c index fbe7ce19e1..45212d54c9 100644 --- a/block/backup.c +++ b/block/backup.c @@ -51,7 +51,7 @@ typedef struct BackupBlockJob { NotifierWithReturn before_write; QLIST_HEAD(, CowRequest) inflight_reqs; =20 - HBitmap *copy_bitmap; + BdrvDirtyBitmap *copy_bitmap; bool use_copy_range; int64_t copy_range_size; =20 @@ -114,7 +114,8 @@ static int coroutine_fn backup_cow_with_bounce_buffer(B= ackupBlockJob *job, int read_flags =3D is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; int write_flags =3D job->serialize_target_writes ? BDRV_REQ_SERIALISIN= G : 0; =20 - hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1); + assert(start % job->cluster_size =3D=3D 0); + bdrv_reset_dirty_bitmap(job->copy_bitmap, start, job->cluster_size); nbytes =3D MIN(job->cluster_size, job->len - start); if (!*bounce_buffer) { *bounce_buffer =3D blk_blockalign(blk, job->cluster_size); @@ -150,7 +151,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(B= ackupBlockJob *job, =20 return nbytes; fail: - hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1); + bdrv_set_dirty_bitmap(job->copy_bitmap, start, job->cluster_size); return ret; =20 } @@ -170,16 +171,17 @@ static int coroutine_fn backup_cow_with_offload(Backu= pBlockJob *job, int write_flags =3D job->serialize_target_writes ? BDRV_REQ_SERIALISIN= G : 0; =20 assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size)); + assert(QEMU_IS_ALIGNED(start, job->cluster_size)); nbytes =3D MIN(job->copy_range_size, end - start); nr_clusters =3D DIV_ROUND_UP(nbytes, job->cluster_size); - hbitmap_reset(job->copy_bitmap, start / job->cluster_size, - nr_clusters); + bdrv_reset_dirty_bitmap(job->copy_bitmap, start, + job->cluster_size * nr_clusters); ret =3D blk_co_copy_range(blk, start, job->target, start, nbytes, read_flags, write_flags); if (ret < 0) { trace_backup_do_cow_copy_range_fail(job, start, ret); - hbitmap_set(job->copy_bitmap, start / job->cluster_size, - nr_clusters); + bdrv_set_dirty_bitmap(job->copy_bitmap, start, + job->cluster_size * nr_clusters); return ret; } =20 @@ -207,7 +209,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *j= ob, cow_request_begin(&cow_request, job, start, end); =20 while (start < end) { - if (!hbitmap_get(job->copy_bitmap, start / job->cluster_size)) { + if (!bdrv_get_dirty_locked(NULL, job->copy_bitmap, start)) { trace_backup_do_cow_skip(job, start); start +=3D job->cluster_size; continue; /* already copied */ @@ -303,6 +305,11 @@ static void backup_clean(Job *job) assert(s->target); blk_unref(s->target); s->target =3D NULL; + + if (s->copy_bitmap) { + bdrv_release_dirty_bitmap(blk_bs(s->common.blk), s->copy_bitmap); + s->copy_bitmap =3D NULL; + } } =20 static void backup_attached_aio_context(BlockJob *job, AioContext *aio_con= text) @@ -315,7 +322,6 @@ static void backup_attached_aio_context(BlockJob *job, = AioContext *aio_context) void backup_do_checkpoint(BlockJob *job, Error **errp) { BackupBlockJob *backup_job =3D container_of(job, BackupBlockJob, commo= n); - int64_t len; =20 assert(block_job_driver(job) =3D=3D &backup_job_driver); =20 @@ -325,8 +331,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp) return; } =20 - len =3D DIV_ROUND_UP(backup_job->len, backup_job->cluster_size); - hbitmap_set(backup_job->copy_bitmap, 0, len); + bdrv_set_dirty_bitmap(backup_job->copy_bitmap, 0, backup_job->len); } =20 static void backup_drain(BlockJob *job) @@ -379,28 +384,30 @@ static bool coroutine_fn yield_and_check(BackupBlockJ= ob *job) =20 static int coroutine_fn backup_run_incremental(BackupBlockJob *job) { - int ret; + int ret =3D 0; bool error_is_read; - int64_t cluster; - HBitmapIter hbi; + int64_t offset; + BdrvDirtyBitmapIter *dbi =3D bdrv_dirty_iter_new(job->copy_bitmap); =20 - hbitmap_iter_init(&hbi, job->copy_bitmap, 0); - while ((cluster =3D hbitmap_iter_next(&hbi)) !=3D -1) { + while ((offset =3D bdrv_dirty_iter_next(dbi)) !=3D -1) { do { if (yield_and_check(job)) { - return 0; + goto out; } - ret =3D backup_do_cow(job, cluster * job->cluster_size, + ret =3D backup_do_cow(job, offset, job->cluster_size, &error_is_read, false); if (ret < 0 && backup_error_action(job, error_is_read, -ret) = =3D=3D BLOCK_ERROR_ACTION_REPORT) { - return ret; + goto out; } } while (ret < 0); } =20 - return 0; +out: + bdrv_dirty_iter_free(dbi); + + return ret; } =20 /* init copy_bitmap from sync_bitmap */ @@ -412,12 +419,9 @@ static void backup_incremental_init_copy_bitmap(Backup= BlockJob *job) while (bdrv_dirty_bitmap_next_dirty_area(job->sync_bitmap, &offset, &bytes)) { - uint64_t cluster =3D offset / job->cluster_size; - uint64_t last_cluster =3D (offset + bytes) / job->cluster_size; + bdrv_set_dirty_bitmap(job->copy_bitmap, offset, bytes); =20 - hbitmap_set(job->copy_bitmap, cluster, last_cluster - cluster + 1); - - offset =3D (last_cluster + 1) * job->cluster_size; + offset +=3D bytes; if (offset >=3D job->len) { break; } @@ -426,30 +430,29 @@ static void backup_incremental_init_copy_bitmap(Backu= pBlockJob *job) =20 /* TODO job_progress_set_remaining() would make more sense */ job_progress_update(&job->common.job, - job->len - hbitmap_count(job->copy_bitmap) * job->cluster_size); + job->len - bdrv_get_dirty_count(job->copy_bitmap)); } =20 static int coroutine_fn backup_run(Job *job, Error **errp) { BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); BlockDriverState *bs =3D blk_bs(s->common.blk); - int64_t offset, nb_clusters; + int64_t offset; int ret =3D 0; =20 QLIST_INIT(&s->inflight_reqs); qemu_co_rwlock_init(&s->flush_rwlock); =20 - nb_clusters =3D DIV_ROUND_UP(s->len, s->cluster_size); job_progress_set_remaining(job, s->len); =20 - s->copy_bitmap =3D hbitmap_alloc(nb_clusters, 0); + bdrv_disable_dirty_bitmap(s->copy_bitmap); + if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { backup_incremental_init_copy_bitmap(s); } else { - hbitmap_set(s->copy_bitmap, 0, nb_clusters); + bdrv_set_dirty_bitmap(s->copy_bitmap, 0, s->len); } =20 - s->before_write.notify =3D backup_before_write_notify; bdrv_add_before_write_notifier(bs, &s->before_write); =20 @@ -530,7 +533,6 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) /* wait until pending backup_do_cow() calls have completed */ qemu_co_rwlock_wrlock(&s->flush_rwlock); qemu_co_rwlock_unlock(&s->flush_rwlock); - hbitmap_free(s->copy_bitmap); =20 return ret; } @@ -681,6 +683,13 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, } else { job->cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster= _size); } + + job->copy_bitmap =3D bdrv_create_dirty_bitmap(bs, job->cluster_size, + NULL, errp); + if (!job->copy_bitmap) { + goto error; + } + job->use_copy_range =3D true; job->copy_range_size =3D MIN_NON_ZERO(blk_get_max_transfer(job->common= .blk), blk_get_max_transfer(job->target)); --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390512674862.5782189481956; Mon, 1 Oct 2018 03:41:52 -0700 (PDT) Received: from localhost ([::1]:37221 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6veB-00055W-Ih for importer@patchew.org; Mon, 01 Oct 2018 06:41:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40421) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSO-0004qj-Dv for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002Lf-S5 for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:40 -0400 Received: from relay.sw.ru ([185.231.240.75]:39670) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSK-0002JT-EF; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSF-0005Yj-1i; Mon, 01 Oct 2018 13:29:31 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:15 +0300 Message-Id: <20181001102928.20533-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 05/18] util/id: add block-bitmap subsystem X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add block-bitmap subsystem to generate bitmap names in the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/qemu/id.h | 1 + util/id.c | 1 + 2 files changed, 2 insertions(+) diff --git a/include/qemu/id.h b/include/qemu/id.h index 40c70103e4..013a560e01 100644 --- a/include/qemu/id.h +++ b/include/qemu/id.h @@ -4,6 +4,7 @@ typedef enum IdSubSystems { ID_QDEV, ID_BLOCK, + ID_BLOCK_BITMAP, ID_MAX /* last element, used as array size */ } IdSubSystems; =20 diff --git a/util/id.c b/util/id.c index 6141352955..beb4ee7f0b 100644 --- a/util/id.c +++ b/util/id.c @@ -34,6 +34,7 @@ bool id_wellformed(const char *id) static const char *const id_subsys_str[ID_MAX] =3D { [ID_QDEV] =3D "qdev", [ID_BLOCK] =3D "block", + [ID_BLOCK_BITMAP] =3D "block-bitmap", }; =20 /* --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390342921192.9951325543717; Mon, 1 Oct 2018 03:39:02 -0700 (PDT) Received: from localhost ([::1]:37201 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vbJ-0002ff-21 for importer@patchew.org; Mon, 01 Oct 2018 06:38:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40377) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSN-0004qZ-Kv for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002LD-OF for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:39680) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSK-0002JU-D1; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSF-0005Yj-Ex; Mon, 01 Oct 2018 13:29:31 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:16 +0300 Message-Id: <20181001102928.20533-7-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 06/18] block/backup: give a name to copy-bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The bitmap should be named to be shared with special filter node in further patches. Make it possible to set this name in qmp command. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 18 ++++++++++++++++-- include/block/block_int.h | 2 +- block/backup.c | 10 ++++++++-- block/replication.c | 2 +- blockdev.c | 12 ++++++++++-- 5 files changed, 36 insertions(+), 8 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index ac3b48ee54..c4774af18e 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1260,6 +1260,12 @@ # Must be present if sync is "incremental", must NOT be present # otherwise. (Since 2.4) # +# @x-copy-bitmap: the name for the copy-bitmap - the bitmap which drives t= he +# backup process. At backup start it corresponds to @bitma= p (or +# just all bits set for full backup) and at the finish it +# should be zero. If the bitmap already exists it will be = used +# as is. (Since 3.1) +# # @compress: true to compress data, if the target format supports it. # (default: false) (since 2.8) # @@ -1297,7 +1303,8 @@ '*bitmap': 'str', '*compress': 'bool', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', - '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } + '*auto-finalize': 'bool', '*auto-dismiss': 'bool', + '*x-copy-bitmap': 'str' } } =20 ## # @BlockdevBackup: @@ -1340,6 +1347,12 @@ # list without user intervention. # Defaults to true. (Since 2.12) # +# @x-copy-bitmap: the name for the copy-bitmap - the bitmap which drives t= he +# backup process. At backup start it corresponds to @bitma= p (or +# just all bits set for full backup) and at the finish it +# should be zero. If the bitmap already exists it will be = used +# as is. (Since 3.1) +# # Note: @on-source-error and @on-target-error only affect background # I/O. If an error occurs during a guest write request, the device's # rerror/werror actions will be used. @@ -1351,7 +1364,8 @@ 'sync': 'MirrorSyncMode', '*speed': 'int', '*compress': 'bool', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', - '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } + '*auto-finalize': 'bool', '*auto-dismiss': 'bool', + '*x-copy-bitmap': 'str' } } =20 ## # @blockdev-snapshot-sync: diff --git a/include/block/block_int.h b/include/block/block_int.h index 92ecbd866e..13f4a0f98e 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -1087,7 +1087,7 @@ BlockJob *backup_job_create(const char *job_id, Block= DriverState *bs, bool compress, BlockdevOnError on_source_error, BlockdevOnError on_target_error, - int creation_flags, + int creation_flags, const char *x_copy_bitmap, BlockCompletionFunc *cb, void *opaque, JobTxn *txn, Error **errp); =20 diff --git a/block/backup.c b/block/backup.c index 45212d54c9..ad143ea735 100644 --- a/block/backup.c +++ b/block/backup.c @@ -25,6 +25,7 @@ #include "sysemu/block-backend.h" #include "qemu/bitmap.h" #include "qemu/error-report.h" +#include "qemu/id.h" =20 #define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16) =20 @@ -559,7 +560,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, bool compress, BlockdevOnError on_source_error, BlockdevOnError on_target_error, - int creation_flags, + int creation_flags, const char *x_copy_bitmap, BlockCompletionFunc *cb, void *opaque, JobTxn *txn, Error **errp) { @@ -567,6 +568,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, BlockDriverInfo bdi; BackupBlockJob *job =3D NULL; int ret; + char *gen_bitmap_name =3D NULL; =20 assert(bs); assert(target); @@ -684,8 +686,12 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, job->cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster= _size); } =20 + if (!x_copy_bitmap) { + x_copy_bitmap =3D gen_bitmap_name =3D id_generate(ID_BLOCK_BITMAP); + } job->copy_bitmap =3D bdrv_create_dirty_bitmap(bs, job->cluster_size, - NULL, errp); + x_copy_bitmap, errp); + g_free(gen_bitmap_name); if (!job->copy_bitmap) { goto error; } diff --git a/block/replication.c b/block/replication.c index 0c2989d2cb..f4ff0bb034 100644 --- a/block/replication.c +++ b/block/replication.c @@ -548,7 +548,7 @@ static void replication_start(ReplicationState *rs, Rep= licationMode mode, job =3D backup_job_create(NULL, s->secondary_disk->bs, s->hidden_d= isk->bs, 0, MIRROR_SYNC_MODE_NONE, NULL, false, BLOCKDEV_ON_ERROR_REPORT, - BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, + BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, NU= LL, backup_job_completed, bs, NULL, &local_err= ); if (local_err) { error_propagate(errp, local_err); diff --git a/blockdev.c b/blockdev.c index a8755bd908..407e03d22a 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3407,6 +3407,9 @@ static BlockJob *do_drive_backup(DriveBackup *backup,= JobTxn *txn, if (!backup->has_compress) { backup->compress =3D false; } + if (!backup->has_x_copy_bitmap) { + backup->x_copy_bitmap =3D NULL; + } =20 bs =3D qmp_get_root_bs(backup->device, errp); if (!bs) { @@ -3511,7 +3514,8 @@ static BlockJob *do_drive_backup(DriveBackup *backup,= JobTxn *txn, job =3D backup_job_create(backup->job_id, bs, target_bs, backup->speed, backup->sync, bmap, backup->compress, backup->on_source_error, backup->on_target_err= or, - job_flags, NULL, NULL, txn, &local_err); + job_flags, backup->x_copy_bitmap, + NULL, NULL, txn, &local_err); bdrv_unref(target_bs); if (local_err !=3D NULL) { error_propagate(errp, local_err); @@ -3569,6 +3573,9 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, = JobTxn *txn, if (!backup->has_compress) { backup->compress =3D false; } + if (!backup->has_x_copy_bitmap) { + backup->x_copy_bitmap =3D NULL; + } =20 bs =3D bdrv_lookup_bs(backup->device, backup->device, errp); if (!bs) { @@ -3603,7 +3610,8 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, = JobTxn *txn, job =3D backup_job_create(backup->job_id, bs, target_bs, backup->speed, backup->sync, NULL, backup->compress, backup->on_source_error, backup->on_target_err= or, - job_flags, NULL, NULL, txn, &local_err); + job_flags, backup->x_copy_bitmap, + NULL, NULL, txn, &local_err); if (local_err !=3D NULL) { error_propagate(errp, local_err); } --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390155636132.83744198549368; Mon, 1 Oct 2018 03:35:55 -0700 (PDT) Received: from localhost ([::1]:37184 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vYQ-0000Lk-Ed for importer@patchew.org; Mon, 01 Oct 2018 06:35:54 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40630) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSi-0005BT-7V for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSd-0002w8-VI for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:00 -0400 Received: from relay.sw.ru ([185.231.240.75]:39658) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSd-0002JR-Lv; Mon, 01 Oct 2018 06:29:55 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSF-0005Yj-M9; Mon, 01 Oct 2018 13:29:31 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:17 +0300 Message-Id: <20181001102928.20533-8-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 07/18] block/backup: allow use existent copy-bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Allow use existent copy-bitmap to make it possible to synchronize with externally created fleecing-hook filter which will be introduced soon. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/block/backup.c b/block/backup.c index ad143ea735..11aa31a323 100644 --- a/block/backup.c +++ b/block/backup.c @@ -53,6 +53,7 @@ typedef struct BackupBlockJob { QLIST_HEAD(, CowRequest) inflight_reqs; =20 BdrvDirtyBitmap *copy_bitmap; + bool copy_bitmap_created; bool use_copy_range; int64_t copy_range_size; =20 @@ -307,7 +308,7 @@ static void backup_clean(Job *job) blk_unref(s->target); s->target =3D NULL; =20 - if (s->copy_bitmap) { + if (s->copy_bitmap_created) { bdrv_release_dirty_bitmap(blk_bs(s->common.blk), s->copy_bitmap); s->copy_bitmap =3D NULL; } @@ -686,11 +687,16 @@ BlockJob *backup_job_create(const char *job_id, Block= DriverState *bs, job->cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster= _size); } =20 - if (!x_copy_bitmap) { + if (x_copy_bitmap) { + job->copy_bitmap =3D bdrv_find_dirty_bitmap(bs, x_copy_bitmap); + } else { x_copy_bitmap =3D gen_bitmap_name =3D id_generate(ID_BLOCK_BITMAP); } - job->copy_bitmap =3D bdrv_create_dirty_bitmap(bs, job->cluster_size, - x_copy_bitmap, errp); + if (!job->copy_bitmap) { + job->copy_bitmap =3D bdrv_create_dirty_bitmap(bs, job->cluster_siz= e, + x_copy_bitmap, errp); + job->copy_bitmap_created =3D !!job->copy_bitmap; + } g_free(gen_bitmap_name); if (!job->copy_bitmap) { goto error; --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538389948318634.9246437641324; Mon, 1 Oct 2018 03:32:28 -0700 (PDT) Received: from localhost ([::1]:37165 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vUw-0006Dp-3U for importer@patchew.org; Mon, 01 Oct 2018 06:32:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40386) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSN-0004qg-MY for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002Lj-Ry for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:39662) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSK-0002JQ-EC; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSF-0005Yj-V5; Mon, 01 Oct 2018 13:29:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:18 +0300 Message-Id: <20181001102928.20533-9-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 08/18] block: allow serialized reads to intersect X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Otherwise, if we have serialized read-part in copy_range from backing file to its parent if CoW take place, this CoW's sub-reads will intersect with firstly created serialized read request. Anyway, reads should not influence on disk view, let's allow them to intersect. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/io.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/io.c b/block/io.c index b003b4d5bf..6d611bfe59 100644 --- a/block/io.c +++ b/block/io.c @@ -735,7 +735,8 @@ static bool coroutine_fn wait_serialising_requests(Bdrv= TrackedRequest *self) retry =3D false; qemu_co_mutex_lock(&bs->reqs_lock); QLIST_FOREACH(req, &bs->tracked_requests, list) { - if (req =3D=3D self || (!req->serialising && !self->serialisin= g)) { + if (req =3D=3D self || (!req->serialising && !self->serialisin= g) || + (self->type =3D=3D BDRV_TRACKED_READ && req->type =3D=3D s= elf->type)) { continue; } if (tracked_request_overlaps(req, self->overlap_offset, --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390925175368.1418096216196; Mon, 1 Oct 2018 03:48:45 -0700 (PDT) Received: from localhost ([::1]:37267 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vkq-0001t5-0H for importer@patchew.org; Mon, 01 Oct 2018 06:48:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40672) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSj-0005Cu-IW for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSi-0002xt-A3 for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:01 -0400 Received: from relay.sw.ru ([185.231.240.75]:39780) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSh-0002Ma-PO; Mon, 01 Oct 2018 06:30:00 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSG-0005Yj-DG; Mon, 01 Oct 2018 13:29:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:19 +0300 Message-Id: <20181001102928.20533-10-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 09/18] block: improve should_update_child X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" As it already said in the comment, we don't want to create loops in parent->child relations. So, when we try to append @to to @c, we should check that @c is not in @to children subtree, and we should check it recursively, not only the first level. The patch provides BFS-based search, to check the relations. This is needed for further fleecing-hook filter usage: we need to append it to source, when the hook is already a parent of target, and source may be in a backing chain of target (fleecing-scheme). So, on appending, the hook should not became a child (direct or through children subtree) of the target. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 32 +++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-) diff --git a/block.c b/block.c index c298ca6a19..7f605b0bf0 100644 --- a/block.c +++ b/block.c @@ -3432,7 +3432,7 @@ void bdrv_close_all(void) =20 static bool should_update_child(BdrvChild *c, BlockDriverState *to) { - BdrvChild *to_c; + GList *queue =3D NULL, *pos; =20 if (c->role->stay_at_node) { return false; @@ -3468,13 +3468,35 @@ static bool should_update_child(BdrvChild *c, Block= DriverState *to) * if A is a child of B, that means we cannot replace A by B there * because that would create a loop. Silently detaching A from B * is also not really an option. So overall just leaving A in - * place there is the most sensible choice. */ - QLIST_FOREACH(to_c, &to->children, next) { - if (to_c =3D=3D c) { - return false; + * place there is the most sensible choice. + * + * upd: If the child @c belongs to the @to's children, or children of = it's + * children and so on - this would create a loop to. To prevent it let= 's do + * a BFS search on @to children subtree. + */ + + pos =3D queue =3D g_list_append(queue, to); + while (pos) { + BlockDriverState *v =3D pos->data; + BdrvChild *c2; + + QLIST_FOREACH(c2, &v->children, next) { + if (c2 =3D=3D c) { + g_list_free(queue); + return false; + } + + if (g_list_find(queue, c2->bs)) { + continue; + } + + queue =3D g_list_append(queue, c2->bs); } + + pos =3D pos->next; } =20 + g_list_free(queue); return true; } =20 --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538389947829597.0988464214839; Mon, 1 Oct 2018 03:32:27 -0700 (PDT) Received: from localhost ([::1]:37166 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vUv-0006E4-4i for importer@patchew.org; Mon, 01 Oct 2018 06:32:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40387) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSN-0004qh-Mf for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002Kr-Ct for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:39684) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSJ-0002JX-VT; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSG-0005Yj-KG; Mon, 01 Oct 2018 13:29:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:20 +0300 Message-Id: <20181001102928.20533-11-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 10/18] iotests: handle -f argument correctly for qemu_io_silent X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Correctly rewrite default argument. After the patch, the function can be used for other (not only default test-chosen) image format. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 4e67fbbe96..bc0b8851bd 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -136,7 +136,12 @@ def qemu_io(*args): =20 def qemu_io_silent(*args): '''Run qemu-io and return the exit code, suppressing stdout''' - args =3D qemu_io_args + list(args) + if '-f' in qemu_io_args and '-f' in args: + ind =3D qemu_io_args.index('-f') + args =3D qemu_io_args[:ind] + qemu_io_args[ind+2:] + list(args) + else: + args =3D qemu_io_args + list(args) + exitcode =3D subprocess.call(args, stdout=3Dopen('/dev/null', 'w')) if exitcode < 0: sys.stderr.write('qemu-io received signal %i: %s\n' % --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 153839072146991.3578749522112; Mon, 1 Oct 2018 03:45:21 -0700 (PDT) Received: from localhost ([::1]:37241 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vhY-0007lI-2x for importer@patchew.org; Mon, 01 Oct 2018 06:45:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40669) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSj-0005Cp-Gx for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSi-0002xq-9j for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:01 -0400 Received: from relay.sw.ru ([185.231.240.75]:39776) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSh-0002MZ-PP; Mon, 01 Oct 2018 06:30:00 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSG-0005Yj-Sb; Mon, 01 Oct 2018 13:29:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:21 +0300 Message-Id: <20181001102928.20533-12-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 11/18] iotests: allow resume_drive by node name X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" After node graph changes, we may not be able to resume_drive by device name (backing files are not recursively searched). So, lets allow to resume by node-name. Set constant name for breakpoints, to avoid introducing extra parameters. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index bc0b8851bd..5cc47c85de 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -404,11 +404,11 @@ class VM(qtest.QEMUQtestMachine): self.pause_drive(drive, "write_aio") return self.qmp('human-monitor-command', - command_line=3D'qemu-io %s "break %s bp_%s"' % (drive,= event, drive)) + command_line=3D'qemu-io %s "break %s bp_0"' % (drive, = event)) =20 def resume_drive(self, drive): self.qmp('human-monitor-command', - command_line=3D'qemu-io %s "remove_break bp_%s"' % (dr= ive, drive)) + command_line=3D'qemu-io %s "remove_break bp_0"' % (dri= ve)) =20 def hmp_qemu_io(self, drive, cmd): '''Write to a given drive using an HMP command''' @@ -531,13 +531,14 @@ class QMPTestCase(unittest.TestCase): self.assertEqual(self.vm.flatten_qmp_object(json.loads(json_filena= me[5:])), self.vm.flatten_qmp_object(reference)) =20 - def cancel_and_wait(self, drive=3D'drive0', force=3DFalse, resume=3DFa= lse): + def cancel_and_wait(self, drive=3D'drive0', force=3DFalse, resume=3DFa= lse, + resume_node=3DNone): '''Cancel a block job and wait for it to finish, returning the eve= nt''' result =3D self.vm.qmp('block-job-cancel', device=3Ddrive, force= =3Dforce) self.assert_qmp(result, 'return', {}) =20 if resume: - self.vm.resume_drive(drive) + self.vm.resume_drive(resume_node or drive) =20 cancelled =3D False result =3D None --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390552806755.9987758700663; Mon, 1 Oct 2018 03:42:32 -0700 (PDT) Received: from localhost ([::1]:37224 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vep-0005cb-Lp for importer@patchew.org; Mon, 01 Oct 2018 06:42:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40675) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSj-0005Cw-Jw for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSi-0002xm-9c for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:01 -0400 Received: from relay.sw.ru ([185.231.240.75]:39772) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSh-0002MY-PO; Mon, 01 Oct 2018 06:30:00 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSH-0005Yj-2X; Mon, 01 Oct 2018 13:29:33 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:22 +0300 Message-Id: <20181001102928.20533-13-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 12/18] iotests: prepare 055 to graph changes during backup job X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Backup will append fleecing-hook node above source node, so, we can't resume by device name (because resume don't search recursively through backing chain). Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/055 | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests/055 b/tests/qemu-iotests/055 index 3437c11507..be5451e1c5 100755 --- a/tests/qemu-iotests/055 +++ b/tests/qemu-iotests/055 @@ -48,7 +48,8 @@ class TestSingleDrive(iotests.QMPTestCase): def setUp(self): qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(= image_len)) =20 - self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img) + self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img, + 'node-name=3Dsource') self.vm.add_drive(blockdev_target_img, interface=3D"none") if iotests.qemu_default_machine =3D=3D 'pc': self.vm.add_drive(None, 'media=3Dcdrom', 'ide') @@ -69,7 +70,7 @@ class TestSingleDrive(iotests.QMPTestCase): result =3D self.vm.qmp(cmd, device=3D'drive0', target=3Dtarget, sy= nc=3D'full') self.assert_qmp(result, 'return', {}) =20 - event =3D self.cancel_and_wait(resume=3DTrue) + event =3D self.cancel_and_wait(resume=3DTrue, resume_node=3D'sourc= e') self.assert_qmp(event, 'data/type', 'backup') =20 def test_cancel_drive_backup(self): @@ -87,7 +88,7 @@ class TestSingleDrive(iotests.QMPTestCase): self.assert_qmp(result, 'return', {}) =20 self.pause_job('drive0', wait=3DFalse) - self.vm.resume_drive('drive0') + self.vm.resume_drive('source') self.pause_wait('drive0') =20 result =3D self.vm.qmp('query-block-jobs') @@ -165,7 +166,8 @@ class TestSetSpeed(iotests.QMPTestCase): def setUp(self): qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(= image_len)) =20 - self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img) + self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img, + 'node-name=3Dsource') self.vm.add_drive(blockdev_target_img, interface=3D"none") self.vm.launch() =20 @@ -197,7 +199,7 @@ class TestSetSpeed(iotests.QMPTestCase): self.assert_qmp(result, 'return[0]/device', 'drive0') self.assert_qmp(result, 'return[0]/speed', 8 * 1024 * 1024) =20 - event =3D self.cancel_and_wait(resume=3DTrue) + event =3D self.cancel_and_wait(resume=3DTrue, resume_node=3D'sourc= e') self.assert_qmp(event, 'data/type', 'backup') =20 # Check setting speed option works @@ -210,7 +212,7 @@ class TestSetSpeed(iotests.QMPTestCase): self.assert_qmp(result, 'return[0]/device', 'drive0') self.assert_qmp(result, 'return[0]/speed', 4 * 1024 * 1024) =20 - event =3D self.cancel_and_wait(resume=3DTrue) + event =3D self.cancel_and_wait(resume=3DTrue, resume_node=3D'sourc= e') self.assert_qmp(event, 'data/type', 'backup') =20 def test_set_speed_drive_backup(self): @@ -236,7 +238,7 @@ class TestSetSpeed(iotests.QMPTestCase): result =3D self.vm.qmp('block-job-set-speed', device=3D'drive0', s= peed=3D-1) self.assert_qmp(result, 'error/class', 'GenericError') =20 - event =3D self.cancel_and_wait(resume=3DTrue) + event =3D self.cancel_and_wait(resume=3DTrue, resume_node=3D'sourc= e') self.assert_qmp(event, 'data/type', 'backup') =20 def test_set_speed_invalid_drive_backup(self): @@ -464,7 +466,8 @@ class TestDriveCompression(iotests.QMPTestCase): pass =20 def do_prepare_drives(self, fmt, args, attach_target): - self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img) + self.vm =3D iotests.VM().add_drive('blkdebug::' + test_img, + 'node-name=3Dsource') =20 qemu_img('create', '-f', fmt, blockdev_target_img, str(TestDriveCompression.image_len), *args) @@ -507,7 +510,7 @@ class TestDriveCompression(iotests.QMPTestCase): result =3D self.vm.qmp(cmd, device=3D'drive0', sync=3D'full', comp= ress=3DTrue, **args) self.assert_qmp(result, 'return', {}) =20 - event =3D self.cancel_and_wait(resume=3DTrue) + event =3D self.cancel_and_wait(resume=3DTrue, resume_node=3D'sourc= e') self.assert_qmp(event, 'data/type', 'backup') =20 self.vm.shutdown() @@ -532,7 +535,7 @@ class TestDriveCompression(iotests.QMPTestCase): self.assert_qmp(result, 'return', {}) =20 self.pause_job('drive0', wait=3DFalse) - self.vm.resume_drive('drive0') + self.vm.resume_drive('source') self.pause_wait('drive0') =20 result =3D self.vm.qmp('query-block-jobs') --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390700712492.0799899275635; Mon, 1 Oct 2018 03:45:00 -0700 (PDT) Received: from localhost ([::1]:37238 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vhB-0007Px-Gi for importer@patchew.org; Mon, 01 Oct 2018 06:44:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40381) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSN-0004qb-MJ for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002L3-Ey for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:39688) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSJ-0002Ja-VQ; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSH-0005Yj-9W; Mon, 01 Oct 2018 13:29:33 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:23 +0300 Message-Id: <20181001102928.20533-14-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 13/18] block: introduce new filter driver: fleecing-hook X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Fleecing-hook filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following: +-------+ | Guest | +---+---+ |r,w v +---+-----------+ target +---------------+ | Fleecing hook |---------->| target(qcow2) | +---+-----------+ CBW +---+-----------+ | | backing |r,w | v | +---+---------+ backing | | Active disk |<----------------+ +-------------+ r Target's backing may point to active disk (should be set up separately), which gives fleecing-scheme. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 22 +++- block/fleecing-hook.c | 298 ++++++++++++++++++++++++++++++++++++++++++ block/Makefile.objs | 2 + 3 files changed, 320 insertions(+), 2 deletions(-) create mode 100644 block/fleecing-hook.c diff --git a/qapi/block-core.json b/qapi/block-core.json index c4774af18e..13cf90eab6 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2628,7 +2628,8 @@ 'host_cdrom', 'host_device', 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', 'qco= w', 'qcow2', 'qed', 'quorum', 'raw', 'rbd', 'replication', 'sheepd= og', - 'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat', 'vxh= s' ] } + 'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat', 'vxh= s', + 'fleecing-hook'] } =20 ## # @BlockdevOptionsFile: @@ -2719,6 +2720,22 @@ { 'struct': 'BlockdevOptionsGenericFormat', 'data': { 'file': 'BlockdevRef' } } =20 +## +# @BlockdevOptionsFleecingHook: +# +# Driver specific block device options for image format that have no option +# besides their data source. +# +# @append-to: reference to or definition of the data source block d= evice +# @target: reference to or definition of the data source block devi= ce +# @copy-bitmap: name for the copy-bitmap of the process. May be shared T= ODO: normal description here +# +# Since: 2.9 +## + { 'struct': 'BlockdevOptionsFleecingHook', + 'data': { 'append-to': 'str', 'target': 'BlockdevRef', + '*copy-bitmap': 'str'} } + ## # @BlockdevOptionsLUKS: # @@ -3718,7 +3735,8 @@ 'vmdk': 'BlockdevOptionsGenericCOWFormat', 'vpc': 'BlockdevOptionsGenericFormat', 'vvfat': 'BlockdevOptionsVVFAT', - 'vxhs': 'BlockdevOptionsVxHS' + 'vxhs': 'BlockdevOptionsVxHS', + 'fleecing-hook': 'BlockdevOptionsFleecingHook' } } =20 ## diff --git a/block/fleecing-hook.c b/block/fleecing-hook.c new file mode 100644 index 0000000000..f4e2f3ce83 --- /dev/null +++ b/block/fleecing-hook.c @@ -0,0 +1,298 @@ +/* + * Fleecing Hook filter driver + * + * The driver performs Copy-Before-Write (CBW) operation: it is injected a= bove + * some node, and before each write it copies _old_ data to the target nod= e. + * + * Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved. + * + * Author: + * Sementsov-Ogievskiy Vladimir + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include "qemu/osdep.h" +#include "qemu/cutils.h" +#include "qapi/error.h" +#include "block/block_int.h" +#include "block/qdict.h" + +typedef struct BDRVFleecingHookState { + BdrvDirtyBitmap *cbw_bitmap; /* what should be copied to @target + on guest write. */ + BdrvChild *target; + bool cbw_bitmap_created; +} BDRVFleecingHookState; + +static coroutine_fn int fleecing_hook_co_preadv( + BlockDriverState *bs, uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags) +{ + /* Features to be implemented: + * F1. COR. save read data to fleecing target for fast access + * (to reduce reads). This possibly may be done with use of copy-o= n-read + * filter, but we need an ability to make COR requests optional: f= or + * example, if target is a ram-cache, and if it is full now, we sh= ould + * skip doing COR request, as it is actually not necessary. + * + * F2. Feature for guest: read from fleecing target if data is in ram-= cache + * and is unchanged + */ + + return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags); +} + +static coroutine_fn int fleecing_hook_cbw(BlockDriverState *bs, uint64_t o= ffset, + uint64_t bytes) +{ + int ret =3D 0; + BDRVFleecingHookState *s =3D bs->opaque; + uint64_t gran =3D bdrv_dirty_bitmap_granularity(s->cbw_bitmap); + uint64_t end =3D QEMU_ALIGN_UP(offset + bytes, gran); + uint64_t off =3D QEMU_ALIGN_DOWN(offset, gran), len; + size_t align =3D MAX(bdrv_opt_mem_align(bs->backing->bs), + bdrv_opt_mem_align(s->target->bs)); + struct iovec iov =3D { + .iov_base =3D qemu_memalign(align, end - off), + .iov_len =3D end - off + }; + QEMUIOVector qiov; + + qemu_iovec_init_external(&qiov, &iov, 1); + + /* Features to be implemented: + * F3. parallelize copying loop + * F4. detect zeros + * F5. use block_status ? + * F6. don't copy clusters which are already cached by COR [see F1] + */ + + len =3D end - off; + while (bdrv_dirty_bitmap_next_dirty_area(s->cbw_bitmap, &off, &len)) { + iov.iov_len =3D qiov.size =3D len; + + bdrv_reset_dirty_bitmap(s->cbw_bitmap, off, len); + + ret =3D bdrv_co_preadv(bs->backing, off, len, &qiov, + BDRV_REQ_NO_SERIALISING); + if (ret < 0) { + bdrv_set_dirty_bitmap(s->cbw_bitmap, off, len); + goto finish; + } + + ret =3D bdrv_co_pwritev(s->target, off, len, &qiov, BDRV_REQ_SERIA= LISING); + if (ret < 0) { + bdrv_set_dirty_bitmap(s->cbw_bitmap, off, len); + goto finish; + } + + off +=3D len; + if (off >=3D end) { + break; + } + len =3D end - off; + } + +finish: + qemu_vfree(iov.iov_base); + + return ret; +} + +static int coroutine_fn fleecing_hook_co_pdiscard(BlockDriverState *bs, + int64_t offset, int byte= s) +{ + int ret =3D fleecing_hook_cbw(bs, offset, bytes); + if (ret < 0) { + return ret; + } + + /* Features to be implemented: + * F7. possibility of lazy discard: just defer the discard after fleec= ing + * completion. If write (or new discard) occurs to the same area, = just + * drop deferred discard. + */ + + return bdrv_co_pdiscard(bs->backing, offset, bytes); +} + +static int coroutine_fn fleecing_hook_co_pwrite_zeroes(BlockDriverState *b= s, + int64_t offset, int bytes, BdrvRequestFlags flags) +{ + int ret =3D fleecing_hook_cbw(bs, offset, bytes); + if (ret < 0) { + /* F8. Additional option to break fleecing instead of breaking gue= st + * write here */ + return ret; + } + + return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags); +} + +static coroutine_fn int fleecing_hook_co_pwritev(BlockDriverState *bs, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, int f= lags) +{ + int ret =3D fleecing_hook_cbw(bs, offset, bytes); + if (ret < 0) { + return ret; + } + + return bdrv_co_pwritev(bs->backing, offset, bytes, qiov, flags); +} + +static int coroutine_fn fleecing_hook_co_flush(BlockDriverState *bs) +{ + if (!bs->backing) { + return 0; + } + + return bdrv_co_flush(bs->backing->bs); +} + +static void fleecing_hook_refresh_filename(BlockDriverState *bs, QDict *op= ts) +{ + if (bs->backing =3D=3D NULL) { + /* we can be here after failed bdrv_attach_child in + * bdrv_set_backing_hd */ + return; + } + bdrv_refresh_filename(bs->backing->bs); + pstrcpy(bs->exact_filename, sizeof(bs->exact_filename), + bs->backing->bs->filename); +} + +static void fleecing_hook_child_perm(BlockDriverState *bs, BdrvChild *c, + const BdrvChildRole *role, + BlockReopenQueue *reopen_queue, + uint64_t perm, uint64_t shared, + uint64_t *nperm, uint64_t *nshared) +{ + bdrv_filter_default_perms(bs, c, role, reopen_queue, perm, shared, npe= rm, + nshared); + + if (role =3D=3D &child_file) { + /* share write to target, to not interfere guest writes to it's di= sk + * which will be in target backing chain */ + *nshared =3D *nshared | BLK_PERM_WRITE; + } +} + +static int fleecing_hook_open(BlockDriverState *bs, QDict *options, int fl= ags, + Error **errp) +{ + BDRVFleecingHookState *s =3D bs->opaque; + Error *local_err =3D NULL; + const char *append_to, *copy_bitmap_name; + BlockDriverState *backing_bs; + + append_to =3D qdict_get_str(options, "append-to"); + qdict_del(options, "append-to"); + backing_bs =3D bdrv_lookup_bs(append_to, append_to, errp); + if (!backing_bs) { + return -EINVAL; + } + + bs->total_sectors =3D backing_bs->total_sectors; + + copy_bitmap_name =3D qdict_get_try_str(options, "copy-bitmap"); + if (copy_bitmap_name) { + qdict_del(options, "copy-bitmap"); + s->cbw_bitmap =3D bdrv_find_dirty_bitmap(backing_bs, copy_bitmap_n= ame); + } + + if (!s->cbw_bitmap) { + s->cbw_bitmap =3D bdrv_create_dirty_bitmap(bs, 65536, copy_bitmap_= name, + errp); + if (!s->cbw_bitmap) { + return -EINVAL; + } + s->cbw_bitmap_created =3D true; + } + + bdrv_disable_dirty_bitmap(s->cbw_bitmap); + bdrv_set_dirty_bitmap(s->cbw_bitmap, 0, bdrv_getlength(backing_bs)); + + s->target =3D bdrv_open_child(NULL, options, "target", bs, &child_file, + false, errp); + if (!s->target) { + return -EINVAL; + } + + bdrv_set_aio_context(bs, bdrv_get_aio_context(backing_bs)); + bdrv_set_aio_context(s->target->bs, bdrv_get_aio_context(backing_bs)); + + bdrv_drained_begin(backing_bs); + + bdrv_ref(bs); + bdrv_append(bs, backing_bs, &local_err); + + if (local_err) { + bdrv_unref(bs); + } + + bdrv_drained_end(backing_bs); + + if (local_err) { + bdrv_unref_child(bs, s->target); + error_propagate(errp, local_err); + return -EINVAL; + } + + return 0; +} + +static void fleecing_hook_close(BlockDriverState *bs) +{ + BDRVFleecingHookState *s =3D bs->opaque; + + if (s->cbw_bitmap && s->cbw_bitmap_created) { + bdrv_release_dirty_bitmap(bs, s->cbw_bitmap); + } + + if (s->target) { + bdrv_unref_child(bs, s->target); + } +} + +BlockDriver bdrv_fleecing_hook_filter =3D { + .format_name =3D "fleecing-hook", + .instance_size =3D sizeof(BDRVFleecingHookState), + + .bdrv_co_preadv =3D fleecing_hook_co_preadv, + .bdrv_co_pwritev =3D fleecing_hook_co_pwritev, + .bdrv_co_pwrite_zeroes =3D fleecing_hook_co_pwrite_zeroes, + .bdrv_co_pdiscard =3D fleecing_hook_co_pdiscard, + .bdrv_co_flush =3D fleecing_hook_co_flush, + + .bdrv_co_block_status =3D bdrv_co_block_status_from_backing, + + .bdrv_refresh_filename =3D fleecing_hook_refresh_filename, + + .bdrv_open =3D fleecing_hook_open, + .bdrv_close =3D fleecing_hook_close, + + .bdrv_child_perm =3D fleecing_hook_child_perm, + + .is_filter =3D true, +}; + +static void bdrv_fleecing_hook_init(void) +{ + bdrv_register(&bdrv_fleecing_hook_filter); +} + +block_init(bdrv_fleecing_hook_init); diff --git a/block/Makefile.objs b/block/Makefile.objs index c8337bf186..081720b14f 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -31,6 +31,8 @@ block-obj-y +=3D throttle.o copy-on-read.o =20 block-obj-y +=3D crypto.o =20 +block-obj-y +=3D fleecing-hook.o + common-obj-y +=3D stream.o =20 nfs.o-libs :=3D $(LIBNFS_LIBS) --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 153839014311475.49888518507271; Mon, 1 Oct 2018 03:35:43 -0700 (PDT) Received: from localhost ([::1]:37178 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vYE-0000CI-1f for importer@patchew.org; Mon, 01 Oct 2018 06:35:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40380) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSN-0004qc-MH for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSK-0002LN-PO for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:29:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:39694) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSK-0002Je-E1; Mon, 01 Oct 2018 06:29:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSH-0005Yj-KF; Mon, 01 Oct 2018 13:29:33 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:24 +0300 Message-Id: <20181001102928.20533-15-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 14/18] block/fleecing-hook: internal api X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add some functions to use fleecing-hook internally from backup job in further commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 9 ++++++++ block/fleecing-hook.c | 51 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) diff --git a/include/block/block.h b/include/block/block.h index 4edc1e8afa..018751b6ea 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -710,4 +710,13 @@ int coroutine_fn bdrv_co_copy_range(BdrvChild *src, ui= nt64_t src_offset, BdrvChild *dst, uint64_t dst_offset, uint64_t bytes, BdrvRequestFlags read_= flags, BdrvRequestFlags write_flags); + + +BlockDriverState *bdrv_fleecing_hook_append(BlockDriverState *source, + BlockDriverState *target, + const char *copy_bitmap_name, + Error **errp); +void bdrv_fleecing_hook_drop(BlockDriverState *hook); +uint64_t bdrv_fleecing_hook_progress(BlockDriverState *hook); + #endif diff --git a/block/fleecing-hook.c b/block/fleecing-hook.c index f4e2f3ce83..1471b985b2 100644 --- a/block/fleecing-hook.c +++ b/block/fleecing-hook.c @@ -34,6 +34,8 @@ typedef struct BDRVFleecingHookState { on guest write. */ BdrvChild *target; bool cbw_bitmap_created; + + uint64_t bytes_copied; } BDRVFleecingHookState; =20 static coroutine_fn int fleecing_hook_co_preadv( @@ -98,6 +100,7 @@ static coroutine_fn int fleecing_hook_cbw(BlockDriverSta= te *bs, uint64_t offset, goto finish; } =20 + s->bytes_copied +=3D len; off +=3D len; if (off >=3D end) { break; @@ -296,3 +299,51 @@ static void bdrv_fleecing_hook_init(void) } =20 block_init(bdrv_fleecing_hook_init); + +BlockDriverState *bdrv_fleecing_hook_append(BlockDriverState *source, + BlockDriverState *target, + const char *copy_bitmap_name, + Error **errp) +{ + QDict *opts =3D qdict_new(); + + qdict_put_str(opts, "driver", "fleecing-hook"); + qdict_put_str(opts, "append-to", bdrv_get_node_name(source)); + qdict_put_str(opts, "target", bdrv_get_node_name(target)); + if (copy_bitmap_name) { + qdict_put_str(opts, "copy-bitmap", copy_bitmap_name); + } + + /* set defaults like qmp_blockdev_add */ + qdict_put_str(opts, BDRV_OPT_CACHE_DIRECT, "off"); + qdict_put_str(opts, BDRV_OPT_CACHE_NO_FLUSH, "off"); + qdict_put_str(opts, BDRV_OPT_READ_ONLY, "off"); + + return bdrv_open(NULL, NULL, opts, 0, errp); +} + +void bdrv_fleecing_hook_drop(BlockDriverState *hook) +{ + AioContext *aio_context =3D bdrv_get_aio_context(hook); + + aio_context_acquire(aio_context); + + bdrv_drained_begin(hook); + + bdrv_child_try_set_perm(hook->backing, 0, BLK_PERM_ALL, &error_abort); + bdrv_replace_node(hook, backing_bs(hook), &error_abort); + bdrv_set_backing_hd(hook, NULL, &error_abort); + + bdrv_drained_end(hook); + + bdrv_unref(hook); + + aio_context_release(aio_context); +} + +uint64_t bdrv_fleecing_hook_progress(BlockDriverState *hook) +{ + BDRVFleecingHookState *s =3D hook->opaque; + + return s->bytes_copied; +} --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390525364593.1495874947492; Mon, 1 Oct 2018 03:42:05 -0700 (PDT) Received: from localhost ([::1]:37222 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6veO-0005F5-2k for importer@patchew.org; Mon, 01 Oct 2018 06:42:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40703) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSk-0005Dz-FK for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSj-0002yk-K9 for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:02 -0400 Received: from relay.sw.ru ([185.231.240.75]:39800) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSi-0002NR-Te; Mon, 01 Oct 2018 06:30:01 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSH-0005Yj-QY; Mon, 01 Oct 2018 13:29:34 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:25 +0300 Message-Id: <20181001102928.20533-16-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 15/18] qapi: add x-drop-fleecing qmp command X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To insert fleecing-hook, it should be just created (blockdev-add) with corresponding parameters. But it can't be properly removed by blockdev-del (we can't restore backing chain in fleecing-hook .close), So, let's add this expiremental api to drop the hook. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 17 +++++++++++++++++ blockdev.c | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/qapi/block-core.json b/qapi/block-core.json index 13cf90eab6..eda6fb6b7e 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3856,6 +3856,23 @@ ## { 'command': 'blockdev-del', 'data': { 'node-name': 'str' } } =20 +## +# @x-drop-fleecing: +# +# Deletes fleecing-hook filter from the top of the backing chain. +# +# @node-name: Name of the fleecing-hook node name. +# +# Since: 3.1 +# +# -> { "execute": "x-drop-fleecing", +# "arguments": { "node-name": "fleece0" } +# } +# <- { "return": {} } +# +## +{ 'command': 'x-drop-fleecing', 'data': { 'node-name': 'str' } } + ## # @BlockdevCreateOptionsFile: # diff --git a/blockdev.c b/blockdev.c index 407e03d22a..90c09d5cee 100644 --- a/blockdev.c +++ b/blockdev.c @@ -4250,6 +4250,43 @@ out: aio_context_release(aio_context); } =20 +void qmp_x_drop_fleecing(const char *node_name, Error **errp) +{ + AioContext *aio_context; + BlockDriverState *bs; + + bs =3D bdrv_find_node(node_name); + if (!bs) { + error_setg(errp, "Cannot find node %s", node_name); + return; + } + + if (!bdrv_has_blk(bs)) { + error_setg(errp, "Node %s is not inserted", node_name); + return; + } + + if (!bs->backing) { + error_setg(errp, "'%s' has no backing", node_name); + return; + } + + aio_context =3D bdrv_get_aio_context(bs); + aio_context_acquire(aio_context); + + bdrv_drained_begin(bs); + + bdrv_child_try_set_perm(bs->backing, 0, BLK_PERM_ALL, &error_abort); + bdrv_replace_node(bs, backing_bs(bs), &error_abort); + bdrv_set_backing_hd(bs, NULL, &error_abort); + + bdrv_drained_end(bs); + + qmp_blockdev_del(node_name, &error_abort); + + aio_context_release(aio_context); +} + static BdrvChild *bdrv_find_child(BlockDriverState *parent_bs, const char *child_name) { --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390374635576.4145689624431; Mon, 1 Oct 2018 03:39:34 -0700 (PDT) Received: from localhost ([::1]:37205 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vbx-0003CX-Dp for importer@patchew.org; Mon, 01 Oct 2018 06:39:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40631) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSi-0005BU-7S for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSe-0002wJ-1C for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:00 -0400 Received: from relay.sw.ru ([185.231.240.75]:39698) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSd-0002Jz-Oy; Mon, 01 Oct 2018 06:29:55 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSI-0005Yj-2v; Mon, 01 Oct 2018 13:29:34 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:26 +0300 Message-Id: <20181001102928.20533-17-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 16/18] iotests: test new fleecing-hook driver in context of 222 iotest X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/222 | 59 +++++++++++++++++++++++++++------- tests/qemu-iotests/222.out | 66 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 114 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 index 0ead56d574..3063dd0705 100644 --- a/tests/qemu-iotests/222 +++ b/tests/qemu-iotests/222 @@ -4,7 +4,9 @@ # point-in-time snapshot of a node that can be queried over NBD. # # Copyright (C) 2018 Red Hat, Inc. +# Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved. # John helped, too. +# And Vladimir, a bit. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -46,11 +48,40 @@ remainder =3D [("0xd5", "0x108000", "32k"), # Right-en= d of partial-left [1] ("0xdc", "32M", "32k"), # Left-end of partial-right [2] ("0xcd", "0x3ff0000", "64k")] # patterns[3] =20 -with iotests.FilePath('base.img') as base_img_path, \ - iotests.FilePath('fleece.img') as fleece_img_path, \ - iotests.FilePath('nbd.sock') as nbd_sock_path, \ - iotests.VM() as vm: +class Fleecing(): + def __init__(self, vm, src_node, tgt_node): + self.vm =3D vm + self.src_node =3D src_node + self.tgt_node =3D tgt_node =20 +class OldBackupFleecing(Fleecing): + def start(self): + log(self.vm.qmp("blockdev-backup", + device=3Dself.src_node, + target=3Dself.tgt_node, + sync=3D"none")) + + def stop(self): + log(self.vm.qmp('block-job-cancel', device=3Dself.src_node)) + log(self.vm.event_wait('BLOCK_JOB_CANCELLED'), + filters=3D[iotests.filter_qmp_event]) + + +class FleecingHookFleecing(Fleecing): + def start(self): + log(self.vm.qmp("blockdev-add", **{ + "driver": "fleecing-hook", + "node-name": "hook", + "append-to": self.src_node, + "target": self.tgt_node, + })) + + def stop(self): + log(self.vm.qmp('x-drop-fleecing', node_name=3D"hook")) + + +def do_test(base_img_path, fleece_img_path, nbd_sock_path, fleecing): + vm =3D iotests.VM() log('--- Setting up images ---') log('') =20 @@ -77,6 +108,7 @@ with iotests.FilePath('base.img') as base_img_path, \ =20 src_node =3D "drive0" tgt_node =3D "fleeceNode" + fl =3D fleecing(vm, src_node, tgt_node) =20 # create tgt_node backed by src_node log(vm.qmp("blockdev-add", **{ @@ -90,10 +122,7 @@ with iotests.FilePath('base.img') as base_img_path, \ })) =20 # Establish COW from source to fleecing node - log(vm.qmp("blockdev-backup", - device=3Dsrc_node, - target=3Dtgt_node, - sync=3D"none")) + fl.start() =20 log('') log('--- Setting up NBD Export ---') @@ -137,10 +166,8 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Cleanup ---') log('') =20 - log(vm.qmp('block-job-cancel', device=3Dsrc_node)) - log(vm.event_wait('BLOCK_JOB_CANCELLED'), - filters=3D[iotests.filter_qmp_event]) log(vm.qmp('nbd-server-stop')) + fl.stop() log(vm.qmp('blockdev-del', node_name=3Dtgt_node)) vm.shutdown() =20 @@ -155,3 +182,13 @@ with iotests.FilePath('base.img') as base_img_path, \ =20 log('') log('Done') + +def test(fleecing): + with iotests.FilePath('base.img') as base_img_path, \ + iotests.FilePath('fleece.img') as fleece_img_path, \ + iotests.FilePath('nbd.sock') as nbd_sock_path: + + do_test(base_img_path, fleece_img_path, nbd_sock_path, fleecing) + +test(OldBackupFleecing) +test(FleecingHookFleecing) diff --git a/tests/qemu-iotests/222.out b/tests/qemu-iotests/222.out index 48f336a02b..28c3623ee8 100644 --- a/tests/qemu-iotests/222.out +++ b/tests/qemu-iotests/222.out @@ -49,9 +49,75 @@ read -P0 0x3fe0000 64k =20 --- Cleanup --- =20 +{u'return': {}} {u'return': {}} {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u= 'device': u'drive0', u'type': u'backup', u'speed': 0, u'len': 67108864, u'o= ffset': 393216}, u'event': u'BLOCK_JOB_CANCELLED'} {u'return': {}} + +--- Confirming writes --- + +read -P0xab 0 64k +read -P0xad 0x00f8000 64k +read -P0x1d 0x2008000 64k +read -P0xea 0x3fe0000 64k +read -P0xd5 0x108000 32k +read -P0xdc 32M 32k +read -P0xcd 0x3ff0000 64k + +Done +--- Setting up images --- + +Done + +--- Launching VM --- + +Done + +--- Setting up Fleecing Graph --- + +{u'return': {}} +{u'return': {}} + +--- Setting up NBD Export --- + +{u'return': {}} +{u'return': {}} + +--- Sanity Check --- + +read -P0x5d 0 64k +read -P0xd5 1M 64k +read -P0xdc 32M 64k +read -P0xcd 0x3ff0000 64k +read -P0 0x00f8000 32k +read -P0 0x2010000 32k +read -P0 0x3fe0000 64k + +--- Testing COW --- + +write -P0xab 0 64k +{u'return': u''} +write -P0xad 0x00f8000 64k +{u'return': u''} +write -P0x1d 0x2008000 64k +{u'return': u''} +write -P0xea 0x3fe0000 64k +{u'return': u''} + +--- Verifying Data --- + +read -P0x5d 0 64k +read -P0xd5 1M 64k +read -P0xdc 32M 64k +read -P0xcd 0x3ff0000 64k +read -P0 0x00f8000 32k +read -P0 0x2010000 32k +read -P0 0x3fe0000 64k + +--- Cleanup --- + +{u'return': {}} +{u'return': {}} {u'return': {}} =20 --- Confirming writes --- --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390344382992.6032317598455; Mon, 1 Oct 2018 03:39:04 -0700 (PDT) Received: from localhost ([::1]:37204 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vbT-0002oX-3B for importer@patchew.org; Mon, 01 Oct 2018 06:39:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40628) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSi-0005BR-6A for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSd-0002wD-Vd for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:00 -0400 Received: from relay.sw.ru ([185.231.240.75]:39702) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSd-0002K3-Lu; Mon, 01 Oct 2018 06:29:55 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSI-0005Yj-Ax; Mon, 01 Oct 2018 13:29:34 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:27 +0300 Message-Id: <20181001102928.20533-18-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 17/18] block/backup: tiny refactor backup_job_create X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Move copy-bitmap find/create code. It's needed for the following commit, as we'll need copy_bitmap before actual block job creation. Do it in a separate commit to simplify review. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 95 +++++++++++++++++++++++++++----------------------- 1 file changed, 52 insertions(+), 43 deletions(-) diff --git a/block/backup.c b/block/backup.c index 11aa31a323..6cab54dea4 100644 --- a/block/backup.c +++ b/block/backup.c @@ -570,6 +570,9 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, BackupBlockJob *job =3D NULL; int ret; char *gen_bitmap_name =3D NULL; + int64_t cluster_size; + BdrvDirtyBitmap *copy_bitmap =3D NULL; + bool copy_bitmap_created =3D false; =20 assert(bs); assert(target); @@ -624,6 +627,48 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, return NULL; } =20 + /* If there is no backing file on the target, we cannot rely on COW if= our + * backup cluster size is smaller than the target cluster size. Even f= or + * targets with a backing file, try to avoid COW if possible. */ + ret =3D bdrv_get_info(target, &bdi); + if (ret =3D=3D -ENOTSUP && !target->backing) { + /* Cluster size is not defined */ + warn_report("The target block device doesn't provide " + "information about the block size and it doesn't have = a " + "backing file. The default block size of %u bytes is " + "used. If the actual block size of the target exceeds " + "this default, the backup may be unusable", + BACKUP_CLUSTER_SIZE_DEFAULT); + cluster_size =3D BACKUP_CLUSTER_SIZE_DEFAULT; + } else if (ret < 0 && !target->backing) { + error_setg_errno(errp, -ret, + "Couldn't determine the cluster size of the target image, " + "which has no backing file"); + error_append_hint(errp, + "Aborting, since this may create an unusable destination image= \n"); + return NULL; + } else if (ret < 0 && target->backing) { + /* Not fatal; just trudge on ahead. */ + cluster_size =3D BACKUP_CLUSTER_SIZE_DEFAULT; + } else { + cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size= ); + } + + if (x_copy_bitmap) { + copy_bitmap =3D bdrv_find_dirty_bitmap(bs, x_copy_bitmap); + } else { + x_copy_bitmap =3D gen_bitmap_name =3D id_generate(ID_BLOCK_BITMAP); + } + if (!copy_bitmap) { + copy_bitmap =3D bdrv_create_dirty_bitmap(bs, cluster_size, + x_copy_bitmap, errp); + copy_bitmap_created =3D !!copy_bitmap; + } + g_free(gen_bitmap_name); + if (!copy_bitmap) { + return NULL; + } + len =3D bdrv_getlength(bs); if (len < 0) { error_setg_errno(errp, -len, "unable to get length for '%s'", @@ -659,49 +704,9 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, =20 /* Detect image-fleecing (and similar) schemes */ job->serialize_target_writes =3D bdrv_chain_contains(target, bs); - - /* If there is no backing file on the target, we cannot rely on COW if= our - * backup cluster size is smaller than the target cluster size. Even f= or - * targets with a backing file, try to avoid COW if possible. */ - ret =3D bdrv_get_info(target, &bdi); - if (ret =3D=3D -ENOTSUP && !target->backing) { - /* Cluster size is not defined */ - warn_report("The target block device doesn't provide " - "information about the block size and it doesn't have = a " - "backing file. The default block size of %u bytes is " - "used. If the actual block size of the target exceeds " - "this default, the backup may be unusable", - BACKUP_CLUSTER_SIZE_DEFAULT); - job->cluster_size =3D BACKUP_CLUSTER_SIZE_DEFAULT; - } else if (ret < 0 && !target->backing) { - error_setg_errno(errp, -ret, - "Couldn't determine the cluster size of the target image, " - "which has no backing file"); - error_append_hint(errp, - "Aborting, since this may create an unusable destination image= \n"); - goto error; - } else if (ret < 0 && target->backing) { - /* Not fatal; just trudge on ahead. */ - job->cluster_size =3D BACKUP_CLUSTER_SIZE_DEFAULT; - } else { - job->cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster= _size); - } - - if (x_copy_bitmap) { - job->copy_bitmap =3D bdrv_find_dirty_bitmap(bs, x_copy_bitmap); - } else { - x_copy_bitmap =3D gen_bitmap_name =3D id_generate(ID_BLOCK_BITMAP); - } - if (!job->copy_bitmap) { - job->copy_bitmap =3D bdrv_create_dirty_bitmap(bs, job->cluster_siz= e, - x_copy_bitmap, errp); - job->copy_bitmap_created =3D !!job->copy_bitmap; - } - g_free(gen_bitmap_name); - if (!job->copy_bitmap) { - goto error; - } - + job->cluster_size =3D cluster_size; + job->copy_bitmap =3D copy_bitmap; + job->copy_bitmap_created =3D copy_bitmap_created; job->use_copy_range =3D true; job->copy_range_size =3D MIN_NON_ZERO(blk_get_max_transfer(job->common= .blk), blk_get_max_transfer(job->target)); @@ -717,6 +722,10 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, return &job->common; =20 error: + if (copy_bitmap_created) { + assert(!job || !job->copy_bitmap_created); + bdrv_release_dirty_bitmap(bs, copy_bitmap); + } if (sync_bitmap) { bdrv_reclaim_dirty_bitmap(bs, sync_bitmap, NULL); } --=20 2.18.0 From nobody Sat Feb 7 10:43:35 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1538390540552760.9140768280043; Mon, 1 Oct 2018 03:42:20 -0700 (PDT) Received: from localhost ([::1]:37223 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6ved-0005Qa-C5 for importer@patchew.org; Mon, 01 Oct 2018 06:42:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40632) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6vSi-0005BV-7U for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6vSe-0002wS-9Q for qemu-devel@nongnu.org; Mon, 01 Oct 2018 06:30:00 -0400 Received: from relay.sw.ru ([185.231.240.75]:39706) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g6vSd-0002K6-Sw; Mon, 01 Oct 2018 06:29:56 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1g6vSI-0005Yj-ID; Mon, 01 Oct 2018 13:29:34 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 1 Oct 2018 13:29:28 +0300 Message-Id: <20181001102928.20533-19-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20181001102928.20533-1-vsementsov@virtuozzo.com> References: <20181001102928.20533-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v3 18/18] block/backup: use fleecing-hook instead of write notifiers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com, jcody@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Drop write notifiers and use filter node instead. Changes: 1. copy-before-writes now handled by filter node, so, drop all is_write_notifier arguments. 2. we don't have intersecting requests, so their handling is dropped. Instead, synchronization works as follows: when backup or fleecing-hook starts copying of some area it firstly clears copy-bitmap bits, and nobody touches areas, not marked with dirty bits in copy-bitmap, so there no intersection. Also, read requests are marked serializing, to not interfer with guest writes and not read changed data from source (before reading we clear corresponding bit in copy-bitmap, so, this area is not more handled by fleecing-hook). 3. To sync with in-flight requests we no just drain hook node, we don't need rw-lock. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup.c | 142 ++++++++++++++++--------------------------------- 1 file changed, 45 insertions(+), 97 deletions(-) diff --git a/block/backup.c b/block/backup.c index 6cab54dea4..9c85b23d68 100644 --- a/block/backup.c +++ b/block/backup.c @@ -29,13 +29,6 @@ =20 #define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16) =20 -typedef struct CowRequest { - int64_t start_byte; - int64_t end_byte; - QLIST_ENTRY(CowRequest) list; - CoQueue wait_queue; /* coroutines blocked on this request */ -} CowRequest; - typedef struct BackupBlockJob { BlockJob common; BlockBackend *target; @@ -44,13 +37,10 @@ typedef struct BackupBlockJob { MirrorSyncMode sync_mode; BlockdevOnError on_source_error; BlockdevOnError on_target_error; - CoRwlock flush_rwlock; uint64_t len; uint64_t bytes_read; int64_t cluster_size; bool compress; - NotifierWithReturn before_write; - QLIST_HEAD(, CowRequest) inflight_reqs; =20 BdrvDirtyBitmap *copy_bitmap; bool copy_bitmap_created; @@ -58,53 +48,18 @@ typedef struct BackupBlockJob { int64_t copy_range_size; =20 bool serialize_target_writes; + + BlockDriverState *hook; + uint64_t fleecing_hook_progress; } BackupBlockJob; =20 static const BlockJobDriver backup_job_driver; =20 -/* See if in-flight requests overlap and wait for them to complete */ -static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job, - int64_t start, - int64_t end) -{ - CowRequest *req; - bool retry; - - do { - retry =3D false; - QLIST_FOREACH(req, &job->inflight_reqs, list) { - if (end > req->start_byte && start < req->end_byte) { - qemu_co_queue_wait(&req->wait_queue, NULL); - retry =3D true; - break; - } - } - } while (retry); -} - -/* Keep track of an in-flight request */ -static void cow_request_begin(CowRequest *req, BackupBlockJob *job, - int64_t start, int64_t end) -{ - req->start_byte =3D start; - req->end_byte =3D end; - qemu_co_queue_init(&req->wait_queue); - QLIST_INSERT_HEAD(&job->inflight_reqs, req, list); -} - -/* Forget about a completed request */ -static void cow_request_end(CowRequest *req) -{ - QLIST_REMOVE(req, list); - qemu_co_queue_restart_all(&req->wait_queue); -} - /* Copy range to target with a bounce buffer and return the bytes copied. = If * error occurred, return a negative error number */ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, int64_t start, int64_t end, - bool is_write_notifi= er, bool *error_is_read, void **bounce_buffer) { @@ -113,7 +68,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(Ba= ckupBlockJob *job, QEMUIOVector qiov; BlockBackend *blk =3D job->common.blk; int nbytes; - int read_flags =3D is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; + int read_flags =3D BDRV_REQ_SERIALISING; int write_flags =3D job->serialize_target_writes ? BDRV_REQ_SERIALISIN= G : 0; =20 assert(start % job->cluster_size =3D=3D 0); @@ -161,15 +116,13 @@ fail: /* Copy range to target and return the bytes copied. If error occurred, re= turn a * negative error number. */ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, - int64_t start, - int64_t end, - bool is_write_notifier) + int64_t start, int64_t end) { int ret; int nr_clusters; BlockBackend *blk =3D job->common.blk; int nbytes; - int read_flags =3D is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; + int read_flags =3D BDRV_REQ_SERIALISING; int write_flags =3D job->serialize_target_writes ? BDRV_REQ_SERIALISIN= G : 0; =20 assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size)); @@ -192,24 +145,18 @@ static int coroutine_fn backup_cow_with_offload(Backu= pBlockJob *job, =20 static int coroutine_fn backup_do_cow(BackupBlockJob *job, int64_t offset, uint64_t bytes, - bool *error_is_read, - bool is_write_notifier) + bool *error_is_read) { - CowRequest cow_request; int ret =3D 0; int64_t start, end; /* bytes */ void *bounce_buffer =3D NULL; - - qemu_co_rwlock_rdlock(&job->flush_rwlock); + uint64_t fleecing_hook_progress; =20 start =3D QEMU_ALIGN_DOWN(offset, job->cluster_size); end =3D QEMU_ALIGN_UP(bytes + offset, job->cluster_size); =20 trace_backup_do_cow_enter(job, start, offset, bytes); =20 - wait_for_overlapping_requests(job, start, end); - cow_request_begin(&cow_request, job, start, end); - while (start < end) { if (!bdrv_get_dirty_locked(NULL, job->copy_bitmap, start)) { trace_backup_do_cow_skip(job, start); @@ -220,13 +167,13 @@ static int coroutine_fn backup_do_cow(BackupBlockJob = *job, trace_backup_do_cow_process(job, start); =20 if (job->use_copy_range) { - ret =3D backup_cow_with_offload(job, start, end, is_write_noti= fier); + ret =3D backup_cow_with_offload(job, start, end); if (ret < 0) { job->use_copy_range =3D false; } } if (!job->use_copy_range) { - ret =3D backup_cow_with_bounce_buffer(job, start, end, is_writ= e_notifier, + ret =3D backup_cow_with_bounce_buffer(job, start, end, error_is_read, &bounce_buf= fer); } if (ret < 0) { @@ -238,7 +185,10 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *= job, */ start +=3D ret; job->bytes_read +=3D ret; - job_progress_update(&job->common.job, ret); + fleecing_hook_progress =3D bdrv_fleecing_hook_progress(job->hook); + job_progress_update(&job->common.job, ret + fleecing_hook_progress= - + job->fleecing_hook_progress); + job->fleecing_hook_progress =3D fleecing_hook_progress; ret =3D 0; } =20 @@ -246,29 +196,11 @@ static int coroutine_fn backup_do_cow(BackupBlockJob = *job, qemu_vfree(bounce_buffer); } =20 - cow_request_end(&cow_request); - trace_backup_do_cow_return(job, offset, bytes, ret); =20 - qemu_co_rwlock_unlock(&job->flush_rwlock); - return ret; } =20 -static int coroutine_fn backup_before_write_notify( - NotifierWithReturn *notifier, - void *opaque) -{ - BackupBlockJob *job =3D container_of(notifier, BackupBlockJob, before_= write); - BdrvTrackedRequest *req =3D opaque; - - assert(req->bs =3D=3D blk_bs(job->common.blk)); - assert(QEMU_IS_ALIGNED(req->offset, BDRV_SECTOR_SIZE)); - assert(QEMU_IS_ALIGNED(req->bytes, BDRV_SECTOR_SIZE)); - - return backup_do_cow(job, req->offset, req->bytes, NULL, true); -} - static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) { BdrvDirtyBitmap *bm; @@ -312,6 +244,8 @@ static void backup_clean(Job *job) bdrv_release_dirty_bitmap(blk_bs(s->common.blk), s->copy_bitmap); s->copy_bitmap =3D NULL; } + + bdrv_fleecing_hook_drop(s->hook); } =20 static void backup_attached_aio_context(BlockJob *job, AioContext *aio_con= text) @@ -396,8 +330,7 @@ static int coroutine_fn backup_run_incremental(BackupBl= ockJob *job) if (yield_and_check(job)) { goto out; } - ret =3D backup_do_cow(job, offset, - job->cluster_size, &error_is_read, false); + ret =3D backup_do_cow(job, offset, job->cluster_size, &error_i= s_read); if (ret < 0 && backup_error_action(job, error_is_read, -ret) = =3D=3D BLOCK_ERROR_ACTION_REPORT) { @@ -441,9 +374,7 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) BlockDriverState *bs =3D blk_bs(s->common.blk); int64_t offset; int ret =3D 0; - - QLIST_INIT(&s->inflight_reqs); - qemu_co_rwlock_init(&s->flush_rwlock); + uint64_t fleecing_hook_progress; =20 job_progress_set_remaining(job, s->len); =20 @@ -455,15 +386,12 @@ static int coroutine_fn backup_run(Job *job, Error **= errp) bdrv_set_dirty_bitmap(s->copy_bitmap, 0, s->len); } =20 - s->before_write.notify =3D backup_before_write_notify; - bdrv_add_before_write_notifier(bs, &s->before_write); - if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_NONE) { /* All bits are set in copy_bitmap to allow any cluster to be copi= ed. * This does not actually require them to be copied. */ while (!job_is_cancelled(job)) { - /* Yield until the job is cancelled. We just let our before_w= rite - * notify callback service CoW requests. */ + /* Yield until the job is cancelled. We just let our fleecing= -hook + * fileter driver service CbW requests. */ job_yield(job); } } else if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { @@ -514,7 +442,7 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) ret =3D alloced; } else { ret =3D backup_do_cow(s, offset, s->cluster_size, - &error_is_read, false); + &error_is_read); } if (ret < 0) { /* Depending on error action, fail now or retry cluster */ @@ -530,11 +458,13 @@ static int coroutine_fn backup_run(Job *job, Error **= errp) } } =20 - notifier_with_return_remove(&s->before_write); + /* wait pending CBW operations in fleecing hook */ + bdrv_drain(s->hook); =20 - /* wait until pending backup_do_cow() calls have completed */ - qemu_co_rwlock_wrlock(&s->flush_rwlock); - qemu_co_rwlock_unlock(&s->flush_rwlock); + fleecing_hook_progress =3D bdrv_fleecing_hook_progress(s->hook); + job_progress_update(job, ret + fleecing_hook_progress - + s->fleecing_hook_progress); + s->fleecing_hook_progress =3D fleecing_hook_progress; =20 return ret; } @@ -573,6 +503,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, int64_t cluster_size; BdrvDirtyBitmap *copy_bitmap =3D NULL; bool copy_bitmap_created =3D false; + BlockDriverState *hook; =20 assert(bs); assert(target); @@ -669,6 +600,19 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, return NULL; } =20 + /* bdrv_get_device_name will not help to find device name starting from + * @bs after fleecing hook append, so let's calculate job_id before. Do + * it in the same way like block_job_create + */ + if (job_id =3D=3D NULL && !(creation_flags & JOB_INTERNAL)) { + job_id =3D bdrv_get_device_name(bs); + } + + hook =3D bdrv_fleecing_hook_append(bs, target, x_copy_bitmap, errp); + if (!hook) { + return NULL; + } + len =3D bdrv_getlength(bs); if (len < 0) { error_setg_errno(errp, -len, "unable to get length for '%s'", @@ -718,6 +662,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, &error_abort); job->len =3D len; + job->hook =3D hook; =20 return &job->common; =20 @@ -733,6 +678,9 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, backup_clean(&job->common.job); job_early_fail(&job->common.job); } + if (hook) { + bdrv_fleecing_hook_drop(hook); + } =20 return NULL; } --=20 2.18.0