From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573827584; cv=none; d=zoho.com; s=zohoarc; b=HPRjGp5UNISBlwemFF9aKAzL5Y1zz/JmKHF9n+Q3kXhbHvoAfqpSkes1bM7fJ17r+G4TSpTN+YJUaWQVzMIy0JX71+0kCBB5g1MzLb/LxNN0MEWdTQAd3c7vxoSUq94oE86ooHyr9FkhXAsDbHf0US1520tKFsDY9KT1wv1HFY8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573827584; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9/Yz1yTb4IPw7H7YX1GI4BEIsw7181GDQE+jWcMBTHk=; b=JZYoxoiqI/Nh8oXHsfckM9yT3emFC63/O7uqsrl8h3Q/Y0pzLSfmziIc5L9nDCDXUs5+x3qr0f9gMl4uzOLfcGlkXYjXqfIwYKz7u/SF9CcUbDlz4ARRq4thUcbOcmSsWCsXthzUpm320cFd2eNnQHxUnegf7mMvEkai5QtDoHU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573827584093324.88637750022485; Fri, 15 Nov 2019 06:19:44 -0800 (PST) Received: from localhost ([::1]:39814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcRp-0008UM-CF for importer@patchew.org; Fri, 15 Nov 2019 09:19:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59262) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNf-000570-0w for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNd-0002AW-57 for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:22 -0500 Received: from relay.sw.ru ([185.231.240.75]:47420) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNa-0001rn-26; Fri, 15 Nov 2019 09:15:18 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN3-0006WW-Ui; Fri, 15 Nov 2019 17:14:46 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 01/24] block/block-copy: specialcase first copy_range request Date: Fri, 15 Nov 2019 17:14:21 +0300 Message-Id: <20191115141444.24155-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" In block_copy_do_copy we fallback to read+write if copy_range failed. In this case copy_size is larger than defined for buffered IO, and there is corresponding commit. Still, backup copies data cluster by cluster, and most of requests are limited to one cluster anyway, so the only source of this one bad-limited request is copy-before-write operation. Further patch will move backup to use block_copy directly, than for cases where copy_range is not supported, first request will be oversized in each backup. It's not good, let's change it now. Fix is simple: just limit first copy_range request like buffer-based request. If it succeed, set larger copy_range limit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 79798a1567..8602e2cae7 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -70,16 +70,19 @@ void block_copy_state_free(BlockCopyState *s) g_free(s); } =20 +static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *targ= et) +{ + return MIN_NON_ZERO(INT_MAX, + MIN_NON_ZERO(source->bs->bl.max_transfer, + target->bs->bl.max_transfer)); +} + BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, int64_t cluster_size, BdrvRequestFlags write_flags, Error *= *errp) { BlockCopyState *s; BdrvDirtyBitmap *copy_bitmap; - uint32_t max_transfer =3D - MIN_NON_ZERO(INT_MAX, - MIN_NON_ZERO(source->bs->bl.max_transfer, - target->bs->bl.max_transfer)); =20 copy_bitmap =3D bdrv_create_dirty_bitmap(source->bs, cluster_size, NUL= L, errp); @@ -99,7 +102,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, = BdrvChild *target, .mem =3D shres_create(BLOCK_COPY_MAX_MEM), }; =20 - if (max_transfer < cluster_size) { + if (block_copy_max_transfer(source, target) < cluster_size) { /* * copy_range does not respect max_transfer. We don't want to both= er * with requests smaller than block-copy cluster size, so fallback= to @@ -114,12 +117,11 @@ BlockCopyState *block_copy_state_new(BdrvChild *sourc= e, BdrvChild *target, s->copy_size =3D cluster_size; } else { /* - * copy_range does not respect max_transfer (it's a TODO), so we f= actor - * that in here. + * We enable copy-range, but keep small copy_size, until first + * successful copy_range (look at block_copy_do_copy). */ s->use_copy_range =3D true; - s->copy_size =3D MIN(MAX(cluster_size, BLOCK_COPY_MAX_COPY_RANGE), - QEMU_ALIGN_DOWN(max_transfer, cluster_size)); + s->copy_size =3D MAX(s->cluster_size, BLOCK_COPY_MAX_BUFFER); } =20 QLIST_INIT(&s->inflight_reqs); @@ -168,7 +170,21 @@ static int coroutine_fn block_copy_do_copy(BlockCopySt= ate *s, s->use_copy_range =3D false; s->copy_size =3D MAX(s->cluster_size, BLOCK_COPY_MAX_BUFFER); /* Fallback to read+write with allocated buffer */ - } else { + } else if (s->use_copy_range) { + /* + * Successful copy-range. Now increase copy_size. + * copy_range does not respect max_transfer (it's a TODO), so = we + * factor that in here. + * + * Note: we double-check s->use_copy_range for the case when + * parallel block-copy request unset it during previous + * bdrv_co_copy_range call. + */ + s->copy_size =3D + MIN(MAX(s->cluster_size, BLOCK_COPY_MAX_COPY_RANGE), + QEMU_ALIGN_DOWN(block_copy_max_transfer(s->source, + s->target), + s->cluster_size)); goto out; } } @@ -176,7 +192,10 @@ static int coroutine_fn block_copy_do_copy(BlockCopySt= ate *s, /* * In case of failed copy_range request above, we may proceed with buf= fered * request larger than BLOCK_COPY_MAX_BUFFER. Still, further requests = will - * be properly limited, so don't care too much. + * be properly limited, so don't care too much. Moreover the most poss= ible + * case (copy_range is unsupported for the configuration, so the very = first + * copy_range request fails) is handled by setting large copy_size only + * after first successful copy_range. */ =20 bounce_buffer =3D qemu_blockalign(s->source->bs, nbytes); --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828602; cv=none; d=zoho.com; s=zohoarc; b=ech5kvBiMbGYLIW3X+fm/9iXxUyM4hE3u6uWdxyQ+qpD5V9z1Myu/Bd4Kxmyeq7GR+JQzR5ZHmmm7TAcp97/7XWxMo5uQ9EeQVnk4sS2/NfY6LVvdMdzgav6zqE9q28toJ/oay4oSdKAlt03qHVUkruResTScFzrhNFgCPaXLMQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828602; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=s1gaZZlWfHqXs/hHHXZNSEALf74xhQ4S5ejZKozGpLw=; b=ZaSnWnqy+WfMms/i7PPcxvBkmxo10NrTe7VOKYRcYA129J5Nd5FuaroEoHrZ9pqV/DShosI+H1tkC9tYXJeQa49YLuTBklhe3UxnXITX0fqkbvnRsSW02MgJeLdsyoVrxXhWCWYMvDIdZHpG48RmSmHHTxWCsu0w0umV6nH6DMs= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828602991952.9894402380008; Fri, 15 Nov 2019 06:36:42 -0800 (PST) Received: from localhost ([::1]:40028 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVciH-00034G-P3 for importer@patchew.org; Fri, 15 Nov 2019 09:36:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59351) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNj-0005B9-Le for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNe-0002CI-KY for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:27 -0500 Received: from relay.sw.ru ([185.231.240.75]:47486) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNa-0001t4-VM; Fri, 15 Nov 2019 09:15:19 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN4-0006WW-3l; Fri, 15 Nov 2019 17:14:46 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 02/24] block/block-copy: use block_status Date: Fri, 15 Nov 2019 17:14:22 +0300 Message-Id: <20191115141444.24155-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Use bdrv_block_status_above to chose effective chunk size and to handle zeroes effectively. This substitutes checking for just being allocated or not, and drops old code path for it. Assistance by backup job is dropped too, as caching block-status information is more difficult than just caching is-allocated information in our dirty bitmap, and backup job is not good place for this caching anyway. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 67 +++++++++++++++++++++++++++++++++++++--------- block/trace-events | 1 + 2 files changed, 55 insertions(+), 13 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 8602e2cae7..74295d93d5 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -152,7 +152,7 @@ void block_copy_set_callbacks( */ static int coroutine_fn block_copy_do_copy(BlockCopyState *s, int64_t start, int64_t end, - bool *error_is_read) + bool zeroes, bool *error_is_rea= d) { int ret; int nbytes =3D MIN(end, s->len) - start; @@ -162,6 +162,18 @@ static int coroutine_fn block_copy_do_copy(BlockCopySt= ate *s, assert(QEMU_IS_ALIGNED(end, s->cluster_size)); assert(end < s->len || end =3D=3D QEMU_ALIGN_UP(s->len, s->cluster_siz= e)); =20 + if (zeroes) { + ret =3D bdrv_co_pwrite_zeroes(s->target, start, nbytes, s->write_f= lags & + ~BDRV_REQ_WRITE_COMPRESSED); + if (ret < 0) { + trace_block_copy_write_zeroes_fail(s, start, ret); + if (error_is_read) { + *error_is_read =3D false; + } + } + return ret; + } + if (s->use_copy_range) { ret =3D bdrv_co_copy_range(s->source, start, s->target, start, nby= tes, 0, s->write_flags); @@ -225,6 +237,34 @@ out: return ret; } =20 +static int block_copy_block_status(BlockCopyState *s, int64_t offset, + int64_t bytes, int64_t *pnum) +{ + int64_t num; + BlockDriverState *base; + int ret; + + if (s->skip_unallocated && s->source->bs->backing) { + base =3D s->source->bs->backing->bs; + } else { + base =3D NULL; + } + + ret =3D bdrv_block_status_above(s->source->bs, base, offset, bytes, &n= um, + NULL, NULL); + if (ret < 0 || num < s->cluster_size) { + num =3D s->cluster_size; + ret =3D BDRV_BLOCK_ALLOCATED | BDRV_BLOCK_DATA; + } else if (offset + num =3D=3D s->len) { + num =3D QEMU_ALIGN_UP(num, s->cluster_size); + } else { + num =3D QEMU_ALIGN_DOWN(num, s->cluster_size); + } + + *pnum =3D num; + return ret; +} + /* * Check if the cluster starting at offset is allocated or not. * return via pnum the number of contiguous clusters sharing this allocati= on. @@ -301,7 +341,6 @@ int coroutine_fn block_copy(BlockCopyState *s, { int ret =3D 0; int64_t end =3D bytes + start; /* bytes */ - int64_t status_bytes; BlockCopyInFlightReq req; =20 /* @@ -318,7 +357,7 @@ int coroutine_fn block_copy(BlockCopyState *s, block_copy_inflight_req_begin(s, &req, start, end); =20 while (start < end) { - int64_t next_zero, chunk_end; + int64_t next_zero, chunk_end, status_bytes; =20 if (!bdrv_dirty_bitmap_get(s->copy_bitmap, start)) { trace_block_copy_skip(s, start); @@ -336,23 +375,25 @@ int coroutine_fn block_copy(BlockCopyState *s, chunk_end =3D next_zero; } =20 - if (s->skip_unallocated) { - ret =3D block_copy_reset_unallocated(s, start, &status_bytes); - if (ret =3D=3D 0) { - trace_block_copy_skip_range(s, start, status_bytes); - start +=3D status_bytes; - continue; - } - /* Clamp to known allocated region */ - chunk_end =3D MIN(chunk_end, start + status_bytes); + ret =3D block_copy_block_status(s, start, chunk_end - start, + &status_bytes); + if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) { + bdrv_reset_dirty_bitmap(s->copy_bitmap, start, status_bytes); + s->progress_reset_callback(s->progress_opaque); + trace_block_copy_skip_range(s, start, status_bytes); + start +=3D status_bytes; + continue; } =20 + chunk_end =3D MIN(chunk_end, start + status_bytes); + trace_block_copy_process(s, start); =20 bdrv_reset_dirty_bitmap(s->copy_bitmap, start, chunk_end - start); =20 co_get_from_shres(s->mem, chunk_end - start); - ret =3D block_copy_do_copy(s, start, chunk_end, error_is_read); + ret =3D block_copy_do_copy(s, start, chunk_end, ret & BDRV_BLOCK_Z= ERO, + error_is_read); co_put_to_shres(s->mem, chunk_end - start); if (ret < 0) { bdrv_set_dirty_bitmap(s->copy_bitmap, start, chunk_end - start= ); diff --git a/block/trace-events b/block/trace-events index 6ba86decca..346537a1d2 100644 --- a/block/trace-events +++ b/block/trace-events @@ -48,6 +48,7 @@ block_copy_process(void *bcs, int64_t start) "bcs %p star= t %"PRId64 block_copy_copy_range_fail(void *bcs, int64_t start, int ret) "bcs %p star= t %"PRId64" ret %d" block_copy_read_fail(void *bcs, int64_t start, int ret) "bcs %p start %"PR= Id64" ret %d" block_copy_write_fail(void *bcs, int64_t start, int ret) "bcs %p start %"P= RId64" ret %d" +block_copy_write_zeroes_fail(void *bcs, int64_t start, int ret) "bcs %p st= art %"PRId64" ret %d" =20 # ../blockdev.c qmp_block_job_cancel(void *job) "job %p" --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573827854; cv=none; d=zoho.com; s=zohoarc; b=GjNlwpOylw0Yh9dZmb8KEcxi0Uh5oq/aAyNvZMRJpShpFjgjRObClDWWTCxaCUhLyvBOcokB10/18PzLjfOjrrNONAm+Vo01jbTCccu+cOktY3X1Nxv8Kp4DQ3AXTacTiTByurSUViRgnITMVCK5Aj8UPrp+G4i6JsgtT/BAlDI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573827854; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9tAXxvnQ8W/gq1FHA7QSxbbJv0X+qPLo3NUMGKuVAtc=; b=laQc21tFtrjeZR5M9J9kz3r8rMjQeMQOxGQknO9CTOYOO0JpDejUN/DBhNk8EP83d5aVhEr0HMG+HP5guvgeQg0t6B+o4YGYR+zpOEJa0ndBzDWhAi4hcf4BOXuPrCl1wikKgjjc/xcPTFYkM0r6mm9XK3raXkyAq83g9AqoSI4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573827854596845.1732739104641; Fri, 15 Nov 2019 06:24:14 -0800 (PST) Received: from localhost ([::1]:39852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcWD-0005AN-1U for importer@patchew.org; Fri, 15 Nov 2019 09:24:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:58981) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNK-0004cs-Bj for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNJ-0001xF-AB for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:02 -0500 Received: from relay.sw.ru ([185.231.240.75]:47466) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001s0-GP; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN4-0006WW-6x; Fri, 15 Nov 2019 17:14:46 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 03/24] block/block-copy: factor out block_copy_find_inflight_req Date: Fri, 15 Nov 2019 17:14:23 +0300 Message-Id: <20191115141444.24155-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Split block_copy_find_inflight_req to be used in seprate. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 74295d93d5..94e7e855ef 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -24,23 +24,30 @@ #define BLOCK_COPY_MAX_BUFFER (1 * MiB) #define BLOCK_COPY_MAX_MEM (128 * MiB) =20 +static BlockCopyInFlightReq *block_copy_find_inflight_req(BlockCopyState *= s, + int64_t start, + int64_t end) +{ + BlockCopyInFlightReq *req; + + QLIST_FOREACH(req, &s->inflight_reqs, list) { + if (end > req->start_byte && start < req->end_byte) { + return req; + } + } + + return NULL; +} + static void coroutine_fn block_copy_wait_inflight_reqs(BlockCopyState *s, int64_t start, int64_t end) { BlockCopyInFlightReq *req; - bool waited; - - do { - waited =3D false; - QLIST_FOREACH(req, &s->inflight_reqs, list) { - if (end > req->start_byte && start < req->end_byte) { - qemu_co_queue_wait(&req->wait_queue, NULL); - waited =3D true; - break; - } - } - } while (waited); + + while ((req =3D block_copy_find_inflight_req(s, start, end))) { + qemu_co_queue_wait(&req->wait_queue, NULL); + } } =20 static void block_copy_inflight_req_begin(BlockCopyState *s, --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828653; cv=none; d=zoho.com; s=zohoarc; b=TcZt43jEUyyUls9aAndBlX8IpNIjRORNa2Tdp9SqEcF6rYLVpF4eheBxHu6XCsQOTgI7/xG90X28a6AKOPGzAA5HwtqbwADEzGT1ZDgSxt8Cwb1hg+2+8dFqw6q2YNO/G8QTja7JPmpa4K0ixTbfhG6hllYCqC+OSAY37CWe6lI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828653; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=NZTUSZOsbbWgVFFF4KqwHD1Mm+T7hDElSQD5j+PqerA=; b=dbp1hDb0IcqhWicabCxF/VB+560G7ZlVGQ7lm4XmRvU3QD8KV2cRKGfGaCxi3Xw3NI6LE/UIeI7eTlgtVqLPzNpaGjmJqHP+IZ7ICm3TXBpkc5skRmdthZEM7FoAICyKZD20hFs/DH9GLW9hS01wGa+NbXfINqfqMEwd1qz+6b4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157382865341718.539441585197665; Fri, 15 Nov 2019 06:37:33 -0800 (PST) Received: from localhost ([::1]:40032 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcj6-0003pZ-8V for importer@patchew.org; Fri, 15 Nov 2019 09:37:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59345) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNj-0005B4-Kh for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNe-0002BQ-13 for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:26 -0500 Received: from relay.sw.ru ([185.231.240.75]:47412) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNa-0001rw-6b; Fri, 15 Nov 2019 09:15:18 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN4-0006WW-Am; Fri, 15 Nov 2019 17:14:46 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 04/24] block/block-copy: refactor interfaces to use bytes instead of end Date: Fri, 15 Nov 2019 17:14:24 +0300 Message-Id: <20191115141444.24155-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" We have a lot of "chunk_end - start" invocations, let's switch to bytes/cur_bytes scheme instead. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 4 +-- block/block-copy.c | 68 ++++++++++++++++++++------------------ 2 files changed, 37 insertions(+), 35 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 0a161724d7..7321b3d305 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -19,8 +19,8 @@ #include "qemu/co-shared-resource.h" =20 typedef struct BlockCopyInFlightReq { - int64_t start_byte; - int64_t end_byte; + int64_t start; + int64_t bytes; QLIST_ENTRY(BlockCopyInFlightReq) list; CoQueue wait_queue; /* coroutines blocked on this request */ } BlockCopyInFlightReq; diff --git a/block/block-copy.c b/block/block-copy.c index 94e7e855ef..cc273b6cb8 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -26,12 +26,12 @@ =20 static BlockCopyInFlightReq *block_copy_find_inflight_req(BlockCopyState *= s, int64_t start, - int64_t end) + int64_t bytes) { BlockCopyInFlightReq *req; =20 QLIST_FOREACH(req, &s->inflight_reqs, list) { - if (end > req->start_byte && start < req->end_byte) { + if (start + bytes > req->start && start < req->start + req->bytes)= { return req; } } @@ -41,21 +41,21 @@ static BlockCopyInFlightReq *block_copy_find_inflight_r= eq(BlockCopyState *s, =20 static void coroutine_fn block_copy_wait_inflight_reqs(BlockCopyState *s, int64_t start, - int64_t end) + int64_t bytes) { BlockCopyInFlightReq *req; =20 - while ((req =3D block_copy_find_inflight_req(s, start, end))) { + while ((req =3D block_copy_find_inflight_req(s, start, bytes))) { qemu_co_queue_wait(&req->wait_queue, NULL); } } =20 static void block_copy_inflight_req_begin(BlockCopyState *s, BlockCopyInFlightReq *req, - int64_t start, int64_t end) + int64_t start, int64_t bytes) { - req->start_byte =3D start; - req->end_byte =3D end; + req->start =3D start; + req->bytes =3D bytes; qemu_co_queue_init(&req->wait_queue); QLIST_INSERT_HEAD(&s->inflight_reqs, req, list); } @@ -150,24 +150,26 @@ void block_copy_set_callbacks( /* * block_copy_do_copy * - * Do copy of cluser-aligned chunk. @end is allowed to exceed s->len only = to - * cover last cluster when s->len is not aligned to clusters. + * Do copy of cluser-aligned chunk. Requested region is allowed to exceed = s->len + * only to cover last cluster when s->len is not aligned to clusters. * * No sync here: nor bitmap neighter intersecting requests handling, only = copy. * * Returns 0 on success. */ static int coroutine_fn block_copy_do_copy(BlockCopyState *s, - int64_t start, int64_t end, + int64_t start, int64_t bytes, bool zeroes, bool *error_is_rea= d) { int ret; - int nbytes =3D MIN(end, s->len) - start; + int nbytes =3D MIN(start + bytes, s->len) - start; void *bounce_buffer =3D NULL; =20 + assert(start >=3D 0 && bytes > 0 && INT64_MAX - start >=3D bytes); assert(QEMU_IS_ALIGNED(start, s->cluster_size)); - assert(QEMU_IS_ALIGNED(end, s->cluster_size)); - assert(end < s->len || end =3D=3D QEMU_ALIGN_UP(s->len, s->cluster_siz= e)); + assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); + assert(start + bytes <=3D s->len || + start + bytes =3D=3D QEMU_ALIGN_UP(s->len, s->cluster_size)); =20 if (zeroes) { ret =3D bdrv_co_pwrite_zeroes(s->target, start, nbytes, s->write_f= lags & @@ -347,7 +349,6 @@ int coroutine_fn block_copy(BlockCopyState *s, bool *error_is_read) { int ret =3D 0; - int64_t end =3D bytes + start; /* bytes */ BlockCopyInFlightReq req; =20 /* @@ -358,58 +359,59 @@ int coroutine_fn block_copy(BlockCopyState *s, bdrv_get_aio_context(s->target->bs)); =20 assert(QEMU_IS_ALIGNED(start, s->cluster_size)); - assert(QEMU_IS_ALIGNED(end, s->cluster_size)); + assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); =20 block_copy_wait_inflight_reqs(s, start, bytes); - block_copy_inflight_req_begin(s, &req, start, end); + block_copy_inflight_req_begin(s, &req, start, bytes); =20 - while (start < end) { - int64_t next_zero, chunk_end, status_bytes; + while (bytes) { + int64_t next_zero, cur_bytes, status_bytes; =20 if (!bdrv_dirty_bitmap_get(s->copy_bitmap, start)) { trace_block_copy_skip(s, start); start +=3D s->cluster_size; + bytes -=3D s->cluster_size; continue; /* already copied */ } =20 - chunk_end =3D MIN(end, start + s->copy_size); + cur_bytes =3D MIN(bytes, s->copy_size); =20 next_zero =3D bdrv_dirty_bitmap_next_zero(s->copy_bitmap, start, - chunk_end - start); + cur_bytes); if (next_zero >=3D 0) { assert(next_zero > start); /* start is dirty */ - assert(next_zero < chunk_end); /* no need to do MIN() */ - chunk_end =3D next_zero; + assert(next_zero < start + cur_bytes); /* no need to do MIN() = */ + cur_bytes =3D next_zero - start; } =20 - ret =3D block_copy_block_status(s, start, chunk_end - start, - &status_bytes); + ret =3D block_copy_block_status(s, start, cur_bytes, &status_bytes= ); if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) { bdrv_reset_dirty_bitmap(s->copy_bitmap, start, status_bytes); s->progress_reset_callback(s->progress_opaque); trace_block_copy_skip_range(s, start, status_bytes); start +=3D status_bytes; + bytes -=3D status_bytes; continue; } =20 - chunk_end =3D MIN(chunk_end, start + status_bytes); + cur_bytes =3D MIN(cur_bytes, status_bytes); =20 trace_block_copy_process(s, start); =20 - bdrv_reset_dirty_bitmap(s->copy_bitmap, start, chunk_end - start); + bdrv_reset_dirty_bitmap(s->copy_bitmap, start, cur_bytes); =20 - co_get_from_shres(s->mem, chunk_end - start); - ret =3D block_copy_do_copy(s, start, chunk_end, ret & BDRV_BLOCK_Z= ERO, + co_get_from_shres(s->mem, cur_bytes); + ret =3D block_copy_do_copy(s, start, cur_bytes, ret & BDRV_BLOCK_Z= ERO, error_is_read); - co_put_to_shres(s->mem, chunk_end - start); + co_put_to_shres(s->mem, cur_bytes); if (ret < 0) { - bdrv_set_dirty_bitmap(s->copy_bitmap, start, chunk_end - start= ); + bdrv_set_dirty_bitmap(s->copy_bitmap, start, cur_bytes); break; } =20 - s->progress_bytes_callback(chunk_end - start, s->progress_opaque); - start =3D chunk_end; - ret =3D 0; + s->progress_bytes_callback(cur_bytes, s->progress_opaque); + start +=3D cur_bytes; + bytes -=3D cur_bytes; } =20 block_copy_inflight_req_end(&req); --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828007; cv=none; d=zoho.com; s=zohoarc; b=mAa/dWeFd0p3QOg+j9eXx+NNfY9dF0Mp1LpKSeG6dXY/37VsFOYNmk+wMhvNn5w+uUDbzTxhifoomI+vG+cO24NrKU8p8m6f66lIiMCET8jmBScXtBKNfc7SWGMx0R/GCgb5R91kVFKRXzBf+rlj9sPQZTVsffcCxNw0L5AZxAA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828007; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=H3m5loof3BYDxAuY5TaMHCjPkK/vAW5MloVkq7NucpQ=; b=SYoKryDZGCmj3mwXjJCAbrsdIKwDOclWOVgqryNxShkoQfWtZ+hoo8uPwFsrLO2cRmjNPUFieB1V70d0/VelRmuUf5+NE7SkUkAeVBZEF8/OM/lZCbojpKfIH31Zs8k1AeYIsDspOWz0TwpYOlfKQMGOzjEmGjpspW/IsEkT2qU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828007638905.8735821938662; Fri, 15 Nov 2019 06:26:47 -0800 (PST) Received: from localhost ([::1]:39888 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcYg-00009n-0S for importer@patchew.org; Fri, 15 Nov 2019 09:26:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59347) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNj-0005B5-Le for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNe-0002Cr-WB for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:27 -0500 Received: from relay.sw.ru ([185.231.240.75]:47472) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNb-0001t5-18; Fri, 15 Nov 2019 09:15:19 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN4-0006WW-EH; Fri, 15 Nov 2019 17:14:46 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 05/24] block/block-copy: rename start to offset in interfaces Date: Fri, 15 Nov 2019 17:14:25 +0300 Message-Id: <20191115141444.24155-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" offset/bytes pair is more usual naming in block layer, let's use it. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 2 +- block/block-copy.c | 80 +++++++++++++++++++------------------- 2 files changed, 41 insertions(+), 41 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 7321b3d305..d96b097267 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -19,7 +19,7 @@ #include "qemu/co-shared-resource.h" =20 typedef struct BlockCopyInFlightReq { - int64_t start; + int64_t offset; int64_t bytes; QLIST_ENTRY(BlockCopyInFlightReq) list; CoQueue wait_queue; /* coroutines blocked on this request */ diff --git a/block/block-copy.c b/block/block-copy.c index cc273b6cb8..20068cd699 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -25,13 +25,13 @@ #define BLOCK_COPY_MAX_MEM (128 * MiB) =20 static BlockCopyInFlightReq *block_copy_find_inflight_req(BlockCopyState *= s, - int64_t start, + int64_t offset, int64_t bytes) { BlockCopyInFlightReq *req; =20 QLIST_FOREACH(req, &s->inflight_reqs, list) { - if (start + bytes > req->start && start < req->start + req->bytes)= { + if (offset + bytes > req->offset && offset < req->offset + req->by= tes) { return req; } } @@ -40,21 +40,21 @@ static BlockCopyInFlightReq *block_copy_find_inflight_r= eq(BlockCopyState *s, } =20 static void coroutine_fn block_copy_wait_inflight_reqs(BlockCopyState *s, - int64_t start, + int64_t offset, int64_t bytes) { BlockCopyInFlightReq *req; =20 - while ((req =3D block_copy_find_inflight_req(s, start, bytes))) { + while ((req =3D block_copy_find_inflight_req(s, offset, bytes))) { qemu_co_queue_wait(&req->wait_queue, NULL); } } =20 static void block_copy_inflight_req_begin(BlockCopyState *s, BlockCopyInFlightReq *req, - int64_t start, int64_t bytes) + int64_t offset, int64_t bytes) { - req->start =3D start; + req->offset =3D offset; req->bytes =3D bytes; qemu_co_queue_init(&req->wait_queue); QLIST_INSERT_HEAD(&s->inflight_reqs, req, list); @@ -158,24 +158,24 @@ void block_copy_set_callbacks( * Returns 0 on success. */ static int coroutine_fn block_copy_do_copy(BlockCopyState *s, - int64_t start, int64_t bytes, + int64_t offset, int64_t bytes, bool zeroes, bool *error_is_rea= d) { int ret; - int nbytes =3D MIN(start + bytes, s->len) - start; + int nbytes =3D MIN(offset + bytes, s->len) - offset; void *bounce_buffer =3D NULL; =20 - assert(start >=3D 0 && bytes > 0 && INT64_MAX - start >=3D bytes); - assert(QEMU_IS_ALIGNED(start, s->cluster_size)); + assert(offset >=3D 0 && bytes > 0 && INT64_MAX - offset >=3D bytes); + assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); - assert(start + bytes <=3D s->len || - start + bytes =3D=3D QEMU_ALIGN_UP(s->len, s->cluster_size)); + assert(offset + bytes <=3D s->len || + offset + bytes =3D=3D QEMU_ALIGN_UP(s->len, s->cluster_size)); =20 if (zeroes) { - ret =3D bdrv_co_pwrite_zeroes(s->target, start, nbytes, s->write_f= lags & + ret =3D bdrv_co_pwrite_zeroes(s->target, offset, nbytes, s->write_= flags & ~BDRV_REQ_WRITE_COMPRESSED); if (ret < 0) { - trace_block_copy_write_zeroes_fail(s, start, ret); + trace_block_copy_write_zeroes_fail(s, offset, ret); if (error_is_read) { *error_is_read =3D false; } @@ -184,10 +184,10 @@ static int coroutine_fn block_copy_do_copy(BlockCopyS= tate *s, } =20 if (s->use_copy_range) { - ret =3D bdrv_co_copy_range(s->source, start, s->target, start, nby= tes, + ret =3D bdrv_co_copy_range(s->source, offset, s->target, offset, n= bytes, 0, s->write_flags); if (ret < 0) { - trace_block_copy_copy_range_fail(s, start, ret); + trace_block_copy_copy_range_fail(s, offset, ret); s->use_copy_range =3D false; s->copy_size =3D MAX(s->cluster_size, BLOCK_COPY_MAX_BUFFER); /* Fallback to read+write with allocated buffer */ @@ -221,19 +221,19 @@ static int coroutine_fn block_copy_do_copy(BlockCopyS= tate *s, =20 bounce_buffer =3D qemu_blockalign(s->source->bs, nbytes); =20 - ret =3D bdrv_co_pread(s->source, start, nbytes, bounce_buffer, 0); + ret =3D bdrv_co_pread(s->source, offset, nbytes, bounce_buffer, 0); if (ret < 0) { - trace_block_copy_read_fail(s, start, ret); + trace_block_copy_read_fail(s, offset, ret); if (error_is_read) { *error_is_read =3D true; } goto out; } =20 - ret =3D bdrv_co_pwrite(s->target, start, nbytes, bounce_buffer, + ret =3D bdrv_co_pwrite(s->target, offset, nbytes, bounce_buffer, s->write_flags); if (ret < 0) { - trace_block_copy_write_fail(s, start, ret); + trace_block_copy_write_fail(s, offset, ret); if (error_is_read) { *error_is_read =3D false; } @@ -345,7 +345,7 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, } =20 int coroutine_fn block_copy(BlockCopyState *s, - int64_t start, uint64_t bytes, + int64_t offset, uint64_t bytes, bool *error_is_read) { int ret =3D 0; @@ -358,59 +358,59 @@ int coroutine_fn block_copy(BlockCopyState *s, assert(bdrv_get_aio_context(s->source->bs) =3D=3D bdrv_get_aio_context(s->target->bs)); =20 - assert(QEMU_IS_ALIGNED(start, s->cluster_size)); + assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); =20 - block_copy_wait_inflight_reqs(s, start, bytes); - block_copy_inflight_req_begin(s, &req, start, bytes); + block_copy_wait_inflight_reqs(s, offset, bytes); + block_copy_inflight_req_begin(s, &req, offset, bytes); =20 while (bytes) { int64_t next_zero, cur_bytes, status_bytes; =20 - if (!bdrv_dirty_bitmap_get(s->copy_bitmap, start)) { - trace_block_copy_skip(s, start); - start +=3D s->cluster_size; + if (!bdrv_dirty_bitmap_get(s->copy_bitmap, offset)) { + trace_block_copy_skip(s, offset); + offset +=3D s->cluster_size; bytes -=3D s->cluster_size; continue; /* already copied */ } =20 cur_bytes =3D MIN(bytes, s->copy_size); =20 - next_zero =3D bdrv_dirty_bitmap_next_zero(s->copy_bitmap, start, + next_zero =3D bdrv_dirty_bitmap_next_zero(s->copy_bitmap, offset, cur_bytes); if (next_zero >=3D 0) { - assert(next_zero > start); /* start is dirty */ - assert(next_zero < start + cur_bytes); /* no need to do MIN() = */ - cur_bytes =3D next_zero - start; + assert(next_zero > offset); /* offset is dirty */ + assert(next_zero < offset + cur_bytes); /* no need to do MIN()= */ + cur_bytes =3D next_zero - offset; } =20 - ret =3D block_copy_block_status(s, start, cur_bytes, &status_bytes= ); + ret =3D block_copy_block_status(s, offset, cur_bytes, &status_byte= s); if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) { - bdrv_reset_dirty_bitmap(s->copy_bitmap, start, status_bytes); + bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, status_bytes); s->progress_reset_callback(s->progress_opaque); - trace_block_copy_skip_range(s, start, status_bytes); - start +=3D status_bytes; + trace_block_copy_skip_range(s, offset, status_bytes); + offset +=3D status_bytes; bytes -=3D status_bytes; continue; } =20 cur_bytes =3D MIN(cur_bytes, status_bytes); =20 - trace_block_copy_process(s, start); + trace_block_copy_process(s, offset); =20 - bdrv_reset_dirty_bitmap(s->copy_bitmap, start, cur_bytes); + bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, cur_bytes); =20 co_get_from_shres(s->mem, cur_bytes); - ret =3D block_copy_do_copy(s, start, cur_bytes, ret & BDRV_BLOCK_Z= ERO, + ret =3D block_copy_do_copy(s, offset, cur_bytes, ret & BDRV_BLOCK_= ZERO, error_is_read); co_put_to_shres(s->mem, cur_bytes); if (ret < 0) { - bdrv_set_dirty_bitmap(s->copy_bitmap, start, cur_bytes); + bdrv_set_dirty_bitmap(s->copy_bitmap, offset, cur_bytes); break; } =20 s->progress_bytes_callback(cur_bytes, s->progress_opaque); - start +=3D cur_bytes; + offset +=3D cur_bytes; bytes -=3D cur_bytes; } =20 --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828507; cv=none; d=zoho.com; s=zohoarc; b=lRaXRLXSfMoYv5zpzisCnPMQgkTGVw01ohWpbOqIg7gYdlhEqgP0JoPKm/i0A1mkGAqxp3y7uIRPZKaocPrbBfmU8SvhwXMYcgZdSimbC2+Dv4wIghqG/zsIPnijmUgHyY/Dt/JvvK239ZCO/D1HaTb/ILSTfvusOKdG33TbkWI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828507; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tnpjhPxiZI4PXV/0cpXH+VXlfPLStKGCGGxTxwU4WG8=; b=igCMi1fw6gP5zGh/P+8+HB5iRPW+2kHoIrjqkVKOxVjqi9wWoJ8Z7oMqi+sVxl9ei71lFPSmHxevdf/aOQT9A5N0Z1FWcG3FqloAJ0/1sktpciqt0B2HkT9egsJlMJC+xM8jQLP3TaeoGoJepKykeNN3amWTSZFLIGeoUpK0la0= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828507418313.8240551857697; Fri, 15 Nov 2019 06:35:07 -0800 (PST) Received: from localhost ([::1]:40002 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcgj-0000nh-5d for importer@patchew.org; Fri, 15 Nov 2019 09:35:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59350) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNj-0005B8-Ke for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNe-0002CR-Pj for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:27 -0500 Received: from relay.sw.ru ([185.231.240.75]:47474) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNb-0001t6-0R; Fri, 15 Nov 2019 09:15:19 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN4-0006WW-II; Fri, 15 Nov 2019 17:14:46 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 06/24] block/block-copy: reduce intersecting request lock Date: Fri, 15 Nov 2019 17:14:26 +0300 Message-Id: <20191115141444.24155-7-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Currently, block_copy operation lock the whole requested region. But there is no reason to lock clusters, which are already copied, it will disturb other parallel block_copy requests for no reason. Let's instead do the following: Lock only sub-region, which we are going to operate on. Then, after copying all dirty sub-regions, we should wait for intersecting requests block-copy, if they failed, we should retry these new dirty clusters. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 116 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 95 insertions(+), 21 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 20068cd699..aca44b13fb 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -39,29 +39,62 @@ static BlockCopyInFlightReq *block_copy_find_inflight_r= eq(BlockCopyState *s, return NULL; } =20 -static void coroutine_fn block_copy_wait_inflight_reqs(BlockCopyState *s, - int64_t offset, - int64_t bytes) +/* + * If there are no intersecting requests return false. Otherwise, wait for= the + * first found intersecting request to finish and return true. + */ +static bool coroutine_fn block_copy_wait_one(BlockCopyState *s, int64_t st= art, + int64_t end) { - BlockCopyInFlightReq *req; + BlockCopyInFlightReq *req =3D block_copy_find_inflight_req(s, start, e= nd); =20 - while ((req =3D block_copy_find_inflight_req(s, offset, bytes))) { - qemu_co_queue_wait(&req->wait_queue, NULL); + if (!req) { + return false; } + + qemu_co_queue_wait(&req->wait_queue, NULL); + + return true; } =20 +/* Called only on full-dirty region */ static void block_copy_inflight_req_begin(BlockCopyState *s, BlockCopyInFlightReq *req, int64_t offset, int64_t bytes) { + assert(!block_copy_find_inflight_req(s, offset, bytes)); + + bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); + req->offset =3D offset; req->bytes =3D bytes; qemu_co_queue_init(&req->wait_queue); QLIST_INSERT_HEAD(&s->inflight_reqs, req, list); } =20 -static void coroutine_fn block_copy_inflight_req_end(BlockCopyInFlightReq = *req) +static void coroutine_fn block_copy_inflight_req_shrink(BlockCopyState *s, + BlockCopyInFlightReq *req, int64_t new_bytes) { + if (new_bytes =3D=3D req->bytes) { + return; + } + + assert(new_bytes > 0 && new_bytes < req->bytes); + + bdrv_set_dirty_bitmap(s->copy_bitmap, + req->offset + new_bytes, req->bytes - new_bytes); + + req->bytes =3D new_bytes; + qemu_co_queue_restart_all(&req->wait_queue); +} + +static void coroutine_fn block_copy_inflight_req_end(BlockCopyState *s, + BlockCopyInFlightReq = *req, + int ret) +{ + if (ret < 0) { + bdrv_set_dirty_bitmap(s->copy_bitmap, req->offset, req->bytes); + } QLIST_REMOVE(req, list); qemu_co_queue_restart_all(&req->wait_queue); } @@ -344,12 +377,19 @@ int64_t block_copy_reset_unallocated(BlockCopyState *= s, return ret; } =20 -int coroutine_fn block_copy(BlockCopyState *s, - int64_t offset, uint64_t bytes, - bool *error_is_read) +/* + * block_copy_dirty_clusters + * + * Copy dirty clusters in @start/@bytes range. + * Returns 1 if dirty clusters found and successfully copied, 0 if no dirty + * clusters found and -errno on failure. + */ +static int coroutine_fn block_copy_dirty_clusters(BlockCopyState *s, + int64_t offset, int64_t = bytes, + bool *error_is_read) { int ret =3D 0; - BlockCopyInFlightReq req; + bool found_dirty =3D false; =20 /* * block_copy() user is responsible for keeping source and target in s= ame @@ -361,10 +401,8 @@ int coroutine_fn block_copy(BlockCopyState *s, assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); =20 - block_copy_wait_inflight_reqs(s, offset, bytes); - block_copy_inflight_req_begin(s, &req, offset, bytes); - while (bytes) { + BlockCopyInFlightReq req; int64_t next_zero, cur_bytes, status_bytes; =20 if (!bdrv_dirty_bitmap_get(s->copy_bitmap, offset)) { @@ -374,6 +412,8 @@ int coroutine_fn block_copy(BlockCopyState *s, continue; /* already copied */ } =20 + found_dirty =3D true; + cur_bytes =3D MIN(bytes, s->copy_size); =20 next_zero =3D bdrv_dirty_bitmap_next_zero(s->copy_bitmap, offset, @@ -383,10 +423,12 @@ int coroutine_fn block_copy(BlockCopyState *s, assert(next_zero < offset + cur_bytes); /* no need to do MIN()= */ cur_bytes =3D next_zero - offset; } + block_copy_inflight_req_begin(s, &req, offset, cur_bytes); =20 ret =3D block_copy_block_status(s, offset, cur_bytes, &status_byte= s); + block_copy_inflight_req_shrink(s, &req, status_bytes); if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) { - bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, status_bytes); + block_copy_inflight_req_end(s, &req, 0); s->progress_reset_callback(s->progress_opaque); trace_block_copy_skip_range(s, offset, status_bytes); offset +=3D status_bytes; @@ -398,15 +440,13 @@ int coroutine_fn block_copy(BlockCopyState *s, =20 trace_block_copy_process(s, offset); =20 - bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, cur_bytes); - co_get_from_shres(s->mem, cur_bytes); ret =3D block_copy_do_copy(s, offset, cur_bytes, ret & BDRV_BLOCK_= ZERO, error_is_read); co_put_to_shres(s->mem, cur_bytes); + block_copy_inflight_req_end(s, &req, ret); if (ret < 0) { - bdrv_set_dirty_bitmap(s->copy_bitmap, offset, cur_bytes); - break; + return ret; } =20 s->progress_bytes_callback(cur_bytes, s->progress_opaque); @@ -414,7 +454,41 @@ int coroutine_fn block_copy(BlockCopyState *s, bytes -=3D cur_bytes; } =20 - block_copy_inflight_req_end(&req); + return found_dirty; +} =20 - return ret; +int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t byt= es, + bool *error_is_read) +{ + while (true) { + int ret =3D block_copy_dirty_clusters(s, start, bytes, error_is_re= ad); + + if (ret < 0) { + /* + * IO operation failed, which means the whole block_copy reque= st + * failed. + */ + return ret; + } + if (ret) { + /* + * Something was copied, which means that there were yield poi= nts + * and some new dirty bits may appered (due to failed parallel + * block-copy requests). + */ + continue; + } + + /* + * Here ret =3D=3D 0, which means that there is no dirty clusters = in + * requested region. + */ + + if (!block_copy_wait_one(s, start, bytes)) { + /* No dirty bits and nothing to wait: the whole request is don= e */ + break; + } + } + + return 0; } --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828169; cv=none; d=zoho.com; s=zohoarc; b=XjFNZAaoV+vfnASAiDBWR4JmC48ktge7petUnFxLhv4mN3WB8eud8zeVeP4sqNrhvoFBFzQort2OHYxLlWowSozsC8Jdfw4/rdDl3WSF0OEBYMo/Aja4FM+BW8IIVKb7MarFFeJK6TG6MMbVGoQdjmcbKFrijbjud7wnoWxrYNU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828169; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=JZFUHzmKoIkG/P8gty1nohpzVLbd69Vsa16/AjoXQFk=; b=eMcPTpNHqay9nXKWQq7n1biOUq6dOwDwfhf72daMqkrcECzwb9XN/79JCMoQF6gKlzVk4X475rooqj98iRizSYfjqhtPezstg0LeSnS0xTgKjsyvo5UZxIsNPrAjphWLszVK+KJqGDITjQz73ME3hOYK2WDmrgHqRBlvn9Pt3xw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828169547475.17111933315664; Fri, 15 Nov 2019 06:29:29 -0800 (PST) Received: from localhost ([::1]:39918 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcbG-0003Iy-TH for importer@patchew.org; Fri, 15 Nov 2019 09:29:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59348) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNj-0005B6-LW for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNf-0002Ck-01 for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:27 -0500 Received: from relay.sw.ru ([185.231.240.75]:47456) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNa-0001rx-6B; Fri, 15 Nov 2019 09:15:18 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN4-0006WW-Ma; Fri, 15 Nov 2019 17:14:46 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 07/24] block/block-copy: hide structure definitions Date: Fri, 15 Nov 2019 17:14:27 +0300 Message-Id: <20191115141444.24155-8-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Hide structure definitions and add explicit API instead, to keep an eye on the scope of the shared fields. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 57 +++------------------------------ block/backup-top.c | 6 ++-- block/backup.c | 27 ++++++++-------- block/block-copy.c | 64 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 86 insertions(+), 68 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index d96b097267..753fa663ac 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -18,61 +18,9 @@ #include "block/block.h" #include "qemu/co-shared-resource.h" =20 -typedef struct BlockCopyInFlightReq { - int64_t offset; - int64_t bytes; - QLIST_ENTRY(BlockCopyInFlightReq) list; - CoQueue wait_queue; /* coroutines blocked on this request */ -} BlockCopyInFlightReq; - typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque); typedef void (*ProgressResetCallbackFunc)(void *opaque); -typedef struct BlockCopyState { - /* - * BdrvChild objects are not owned or managed by block-copy. They are - * provided by block-copy user and user is responsible for appropriate - * permissions on these children. - */ - BdrvChild *source; - BdrvChild *target; - BdrvDirtyBitmap *copy_bitmap; - int64_t cluster_size; - bool use_copy_range; - int64_t copy_size; - uint64_t len; - QLIST_HEAD(, BlockCopyInFlightReq) inflight_reqs; - - BdrvRequestFlags write_flags; - - /* - * skip_unallocated: - * - * Used by sync=3Dtop jobs, which first scan the source node for unall= ocated - * areas and clear them in the copy_bitmap. During this process, the = bitmap - * is thus not fully initialized: It may still have bits set for areas= that - * are unallocated and should actually not be copied. - * - * This is indicated by skip_unallocated. - * - * In this case, block_copy() will query the source=E2=80=99s allocati= on status, - * skip unallocated regions, clear them in the copy_bitmap, and invoke - * block_copy_reset_unallocated() every time it does. - */ - bool skip_unallocated; - - /* progress_bytes_callback: called when some copying progress is done.= */ - ProgressBytesCallbackFunc progress_bytes_callback; - - /* - * progress_reset_callback: called when some bytes reset from copy_bit= map - * (see @skip_unallocated above). The callee is assumed to recalculate= how - * many bytes remain based on the dirty bit count of copy_bitmap. - */ - ProgressResetCallbackFunc progress_reset_callback; - void *progress_opaque; - - SharedResource *mem; -} BlockCopyState; +typedef struct BlockCopyState BlockCopyState; =20 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, int64_t cluster_size, @@ -93,4 +41,7 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t byt= es, bool *error_is_read); =20 +BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s); +void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip); + #endif /* BLOCK_COPY_H */ diff --git a/block/backup-top.c b/block/backup-top.c index 7cdb1f8eba..1026628b57 100644 --- a/block/backup-top.c +++ b/block/backup-top.c @@ -38,6 +38,7 @@ typedef struct BDRVBackupTopState { BlockCopyState *bcs; BdrvChild *target; bool active; + int64_t cluster_size; } BDRVBackupTopState; =20 static coroutine_fn int backup_top_co_preadv( @@ -51,8 +52,8 @@ static coroutine_fn int backup_top_cbw(BlockDriverState *= bs, uint64_t offset, uint64_t bytes) { BDRVBackupTopState *s =3D bs->opaque; - uint64_t end =3D QEMU_ALIGN_UP(offset + bytes, s->bcs->cluster_size); - uint64_t off =3D QEMU_ALIGN_DOWN(offset, s->bcs->cluster_size); + uint64_t end =3D QEMU_ALIGN_UP(offset + bytes, s->cluster_size); + uint64_t off =3D QEMU_ALIGN_DOWN(offset, s->cluster_size); =20 return block_copy(s->bcs, off, end - off, NULL); } @@ -227,6 +228,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverSta= te *source, goto failed_after_append; } =20 + state->cluster_size =3D cluster_size; state->bcs =3D block_copy_state_new(top->backing, state->target, cluster_size, write_flags, &local_er= r); if (local_err) { diff --git a/block/backup.c b/block/backup.c index cf62b1a38c..acab0d08da 100644 --- a/block/backup.c +++ b/block/backup.c @@ -48,6 +48,7 @@ typedef struct BackupBlockJob { int64_t cluster_size; =20 BlockCopyState *bcs; + BdrvDirtyBitmap *bcs_bitmap; } BackupBlockJob; =20 static const BlockJobDriver backup_job_driver; @@ -63,7 +64,7 @@ static void backup_progress_bytes_callback(int64_t bytes,= void *opaque) static void backup_progress_reset_callback(void *opaque) { BackupBlockJob *s =3D opaque; - uint64_t estimate =3D bdrv_get_dirty_count(s->bcs->copy_bitmap); + uint64_t estimate =3D bdrv_get_dirty_count(s->bcs_bitmap); =20 job_progress_set_remaining(&s->common.job, estimate); } @@ -111,8 +112,7 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob *= job, int ret) =20 if (ret < 0 && job->bitmap_mode =3D=3D BITMAP_SYNC_MODE_ALWAYS) { /* If we failed and synced, merge in the bits we didn't copy: */ - bdrv_dirty_bitmap_merge_internal(bm, job->bcs->copy_bitmap, - NULL, true); + bdrv_dirty_bitmap_merge_internal(bm, job->bcs_bitmap, NULL, true); } } =20 @@ -151,7 +151,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp) return; } =20 - bdrv_set_dirty_bitmap(backup_job->bcs->copy_bitmap, 0, backup_job->len= ); + bdrv_set_dirty_bitmap(backup_job->bcs_bitmap, 0, backup_job->len); } =20 static BlockErrorAction backup_error_action(BackupBlockJob *job, @@ -196,7 +196,7 @@ static int coroutine_fn backup_loop(BackupBlockJob *job) BdrvDirtyBitmapIter *bdbi; int ret =3D 0; =20 - bdbi =3D bdrv_dirty_iter_new(job->bcs->copy_bitmap); + bdbi =3D bdrv_dirty_iter_new(job->bcs_bitmap); while ((offset =3D bdrv_dirty_iter_next(bdbi)) !=3D -1) { do { if (yield_and_check(job)) { @@ -216,13 +216,13 @@ static int coroutine_fn backup_loop(BackupBlockJob *j= ob) return ret; } =20 -static void backup_init_copy_bitmap(BackupBlockJob *job) +static void backup_init_bcs_bitmap(BackupBlockJob *job) { bool ret; uint64_t estimate; =20 if (job->sync_mode =3D=3D MIRROR_SYNC_MODE_BITMAP) { - ret =3D bdrv_dirty_bitmap_merge_internal(job->bcs->copy_bitmap, + ret =3D bdrv_dirty_bitmap_merge_internal(job->bcs_bitmap, job->sync_bitmap, NULL, true); assert(ret); @@ -232,12 +232,12 @@ static void backup_init_copy_bitmap(BackupBlockJob *j= ob) * We can't hog the coroutine to initialize this thoroughly. * Set a flag and resume work when we are able to yield safely. */ - job->bcs->skip_unallocated =3D true; + block_copy_set_skip_unallocated(job->bcs, true); } - bdrv_set_dirty_bitmap(job->bcs->copy_bitmap, 0, job->len); + bdrv_set_dirty_bitmap(job->bcs_bitmap, 0, job->len); } =20 - estimate =3D bdrv_get_dirty_count(job->bcs->copy_bitmap); + estimate =3D bdrv_get_dirty_count(job->bcs_bitmap); job_progress_set_remaining(&job->common.job, estimate); } =20 @@ -246,7 +246,7 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); int ret =3D 0; =20 - backup_init_copy_bitmap(s); + backup_init_bcs_bitmap(s); =20 if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP) { int64_t offset =3D 0; @@ -265,12 +265,12 @@ static int coroutine_fn backup_run(Job *job, Error **= errp) =20 offset +=3D count; } - s->bcs->skip_unallocated =3D false; + block_copy_set_skip_unallocated(s->bcs, false); } =20 if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_NONE) { /* - * All bits are set in copy_bitmap to allow any cluster to be copi= ed. + * All bits are set in bcs_bitmap to allow any cluster to be copie= d. * This does not actually require them to be copied. */ while (!job_is_cancelled(job)) { @@ -458,6 +458,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, job->sync_bitmap =3D sync_bitmap; job->bitmap_mode =3D bitmap_mode; job->bcs =3D bcs; + job->bcs_bitmap =3D block_copy_dirty_bitmap(bcs); job->cluster_size =3D cluster_size; job->len =3D len; =20 diff --git a/block/block-copy.c b/block/block-copy.c index aca44b13fb..7e14e86a2d 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -24,6 +24,60 @@ #define BLOCK_COPY_MAX_BUFFER (1 * MiB) #define BLOCK_COPY_MAX_MEM (128 * MiB) =20 +typedef struct BlockCopyInFlightReq { + int64_t offset; + int64_t bytes; + QLIST_ENTRY(BlockCopyInFlightReq) list; + CoQueue wait_queue; /* coroutines blocked on this request */ +} BlockCopyInFlightReq; + +typedef struct BlockCopyState { + /* + * BdrvChild objects are not owned or managed by block-copy. They are + * provided by block-copy user and user is responsible for appropriate + * permissions on these children. + */ + BdrvChild *source; + BdrvChild *target; + BdrvDirtyBitmap *copy_bitmap; + int64_t cluster_size; + bool use_copy_range; + int64_t copy_size; + uint64_t len; + QLIST_HEAD(, BlockCopyInFlightReq) inflight_reqs; + + BdrvRequestFlags write_flags; + + /* + * skip_unallocated: + * + * Used by sync=3Dtop jobs, which first scan the source node for unall= ocated + * areas and clear them in the copy_bitmap. During this process, the = bitmap + * is thus not fully initialized: It may still have bits set for areas= that + * are unallocated and should actually not be copied. + * + * This is indicated by skip_unallocated. + * + * In this case, block_copy() will query the source=E2=80=99s allocati= on status, + * skip unallocated regions, clear them in the copy_bitmap, and invoke + * block_copy_reset_unallocated() every time it does. + */ + bool skip_unallocated; + + /* progress_bytes_callback: called when some copying progress is done.= */ + ProgressBytesCallbackFunc progress_bytes_callback; + + /* + * progress_reset_callback: called when some bytes reset from copy_bit= map + * (see @skip_unallocated above). The callee is assumed to recalculate= how + * many bytes remain based on the dirty bit count of copy_bitmap. + */ + ProgressResetCallbackFunc progress_reset_callback; + void *progress_opaque; + + SharedResource *mem; +} BlockCopyState; + static BlockCopyInFlightReq *block_copy_find_inflight_req(BlockCopyState *= s, int64_t offset, int64_t bytes) @@ -492,3 +546,13 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t= start, uint64_t bytes, =20 return 0; } + +BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s) +{ + return s->copy_bitmap; +} + +void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip) +{ + s->skip_unallocated =3D skip; +} --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573827854; cv=none; d=zoho.com; s=zohoarc; b=EThuCrmBKZ9uqVJLfGZyJ5foReQgYKWcRmWQh6fFjEMkN+BbbnlzJL5mRvI2bGemMtiX72U0gMzhqq4lGg1Wjwolf7SUOyUOevQKd3rtbLuUjaBb++MlhfBHLESJ/h36hsrkJv2Wne8pTizOxxWYS9zDHYj8FLf40sdkPclmcAc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573827854; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ij61h4BxvTvsOxYTJ1ErCeN5RhTvszB4EEtxBpWvYH0=; b=JsKoBIMiejPOUC1DptQptrtXaTmUwV1xr8iDl4AEOW8XcL8vHJDCF3lbvCKQDbCW/mpSMOuTFh3JjxgV0StdxRotKSuInoaHgQZOJb5V9sqp8guQ3VBapheBIZvfuHM0tsW6/ZyiSg9dB4SO2cowm6DwBNkz/fRgsDa4nWQ+ukc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573827854570490.78613641592267; Fri, 15 Nov 2019 06:24:14 -0800 (PST) Received: from localhost ([::1]:39850 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcWD-000595-0P for importer@patchew.org; Fri, 15 Nov 2019 09:24:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59284) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNf-000584-Cn for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNd-0002B3-KN for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:23 -0500 Received: from relay.sw.ru ([185.231.240.75]:47424) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNa-0001rp-5o; Fri, 15 Nov 2019 09:15:18 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN4-0006WW-PJ; Fri, 15 Nov 2019 17:14:46 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 08/24] block/block-copy: rename in-flight requests to tasks Date: Fri, 15 Nov 2019 17:14:28 +0300 Message-Id: <20191115141444.24155-9-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" We are going to use aio-task-pool API and extend in-flight request structure to be a successor of AioTask, so rename things appropriately. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 82 ++++++++++++++++++++++------------------------ 1 file changed, 40 insertions(+), 42 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 7e14e86a2d..b00cbfc841 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -24,12 +24,12 @@ #define BLOCK_COPY_MAX_BUFFER (1 * MiB) #define BLOCK_COPY_MAX_MEM (128 * MiB) =20 -typedef struct BlockCopyInFlightReq { +typedef struct BlockCopyTask { int64_t offset; int64_t bytes; - QLIST_ENTRY(BlockCopyInFlightReq) list; - CoQueue wait_queue; /* coroutines blocked on this request */ -} BlockCopyInFlightReq; + QLIST_ENTRY(BlockCopyTask) list; + CoQueue wait_queue; /* coroutines blocked on this task */ +} BlockCopyTask; =20 typedef struct BlockCopyState { /* @@ -44,7 +44,7 @@ typedef struct BlockCopyState { bool use_copy_range; int64_t copy_size; uint64_t len; - QLIST_HEAD(, BlockCopyInFlightReq) inflight_reqs; + QLIST_HEAD(, BlockCopyTask) tasks; =20 BdrvRequestFlags write_flags; =20 @@ -78,15 +78,14 @@ typedef struct BlockCopyState { SharedResource *mem; } BlockCopyState; =20 -static BlockCopyInFlightReq *block_copy_find_inflight_req(BlockCopyState *= s, - int64_t offset, - int64_t bytes) +static BlockCopyTask *block_copy_find_task(BlockCopyState *s, + int64_t offset, int64_t bytes) { - BlockCopyInFlightReq *req; + BlockCopyTask *t; =20 - QLIST_FOREACH(req, &s->inflight_reqs, list) { - if (offset + bytes > req->offset && offset < req->offset + req->by= tes) { - return req; + QLIST_FOREACH(t, &s->tasks, list) { + if (offset + bytes > t->offset && offset < t->offset + t->bytes) { + return t; } } =20 @@ -100,57 +99,56 @@ static BlockCopyInFlightReq *block_copy_find_inflight_= req(BlockCopyState *s, static bool coroutine_fn block_copy_wait_one(BlockCopyState *s, int64_t st= art, int64_t end) { - BlockCopyInFlightReq *req =3D block_copy_find_inflight_req(s, start, e= nd); + BlockCopyTask *task =3D block_copy_find_task(s, start, end); =20 - if (!req) { + if (!task) { return false; } =20 - qemu_co_queue_wait(&req->wait_queue, NULL); + qemu_co_queue_wait(&task->wait_queue, NULL); =20 return true; } =20 /* Called only on full-dirty region */ -static void block_copy_inflight_req_begin(BlockCopyState *s, - BlockCopyInFlightReq *req, - int64_t offset, int64_t bytes) +static void block_copy_task_begin(BlockCopyState *s, BlockCopyTask *task, + int64_t offset, int64_t bytes) { - assert(!block_copy_find_inflight_req(s, offset, bytes)); + assert(!block_copy_find_task(s, offset, bytes)); =20 bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); =20 - req->offset =3D offset; - req->bytes =3D bytes; - qemu_co_queue_init(&req->wait_queue); - QLIST_INSERT_HEAD(&s->inflight_reqs, req, list); + task->offset =3D offset; + task->bytes =3D bytes; + qemu_co_queue_init(&task->wait_queue); + QLIST_INSERT_HEAD(&s->tasks, task, list); } =20 -static void coroutine_fn block_copy_inflight_req_shrink(BlockCopyState *s, - BlockCopyInFlightReq *req, int64_t new_bytes) +static void coroutine_fn block_copy_task_shrink(BlockCopyState *s, + BlockCopyTask *task, + int64_t new_bytes) { - if (new_bytes =3D=3D req->bytes) { + if (new_bytes =3D=3D task->bytes) { return; } =20 - assert(new_bytes > 0 && new_bytes < req->bytes); + assert(new_bytes > 0 && new_bytes < task->bytes); =20 bdrv_set_dirty_bitmap(s->copy_bitmap, - req->offset + new_bytes, req->bytes - new_bytes); + task->offset + new_bytes, task->bytes - new_byte= s); =20 - req->bytes =3D new_bytes; - qemu_co_queue_restart_all(&req->wait_queue); + task->bytes =3D new_bytes; + qemu_co_queue_restart_all(&task->wait_queue); } =20 -static void coroutine_fn block_copy_inflight_req_end(BlockCopyState *s, - BlockCopyInFlightReq = *req, - int ret) +static void coroutine_fn block_copy_task_end(BlockCopyState *s, + BlockCopyTask *task, int ret) { if (ret < 0) { - bdrv_set_dirty_bitmap(s->copy_bitmap, req->offset, req->bytes); + bdrv_set_dirty_bitmap(s->copy_bitmap, task->offset, task->bytes); } - QLIST_REMOVE(req, list); - qemu_co_queue_restart_all(&req->wait_queue); + QLIST_REMOVE(task, list); + qemu_co_queue_restart_all(&task->wait_queue); } =20 void block_copy_state_free(BlockCopyState *s) @@ -218,7 +216,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source,= BdrvChild *target, s->copy_size =3D MAX(s->cluster_size, BLOCK_COPY_MAX_BUFFER); } =20 - QLIST_INIT(&s->inflight_reqs); + QLIST_INIT(&s->tasks); =20 return s; } @@ -456,7 +454,7 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); =20 while (bytes) { - BlockCopyInFlightReq req; + BlockCopyTask task; int64_t next_zero, cur_bytes, status_bytes; =20 if (!bdrv_dirty_bitmap_get(s->copy_bitmap, offset)) { @@ -477,12 +475,12 @@ static int coroutine_fn block_copy_dirty_clusters(Blo= ckCopyState *s, assert(next_zero < offset + cur_bytes); /* no need to do MIN()= */ cur_bytes =3D next_zero - offset; } - block_copy_inflight_req_begin(s, &req, offset, cur_bytes); + block_copy_task_begin(s, &task, offset, cur_bytes); =20 ret =3D block_copy_block_status(s, offset, cur_bytes, &status_byte= s); - block_copy_inflight_req_shrink(s, &req, status_bytes); + block_copy_task_shrink(s, &task, status_bytes); if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) { - block_copy_inflight_req_end(s, &req, 0); + block_copy_task_end(s, &task, 0); s->progress_reset_callback(s->progress_opaque); trace_block_copy_skip_range(s, offset, status_bytes); offset +=3D status_bytes; @@ -498,7 +496,7 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, ret =3D block_copy_do_copy(s, offset, cur_bytes, ret & BDRV_BLOCK_= ZERO, error_is_read); co_put_to_shres(s->mem, cur_bytes); - block_copy_inflight_req_end(s, &req, ret); + block_copy_task_end(s, &task, ret); if (ret < 0) { return ret; } --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828009; cv=none; d=zoho.com; s=zohoarc; b=hfqFj5/TbD9OvyUznUo6szO4uu5TVOMz3F8bAfvc3LEeeZUmcfmfUsFOoVVoMowNQpAOhklllgtyiOGCd6Ngv1rpkMd5+chispgThL1WWkt2EVy3v88Va777PAbsNZhK7KvfBU44BtoUgSNJ7a4Hd/uPD05b9/O78j8+Ht4pGCE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828009; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=f0k3aXBYuVfncm2uOTy/mHCnTPKlaYKIfE9W2eLmnoc=; b=nf2UCx6pcZr3N/TtDdeYUeXZ4rJfRV4j4cyypdm9eWMpOXh6gzGOdNO+rE0KAA63pm5TFVi8CaOo6qUE9THxEhL/Vx6pKD/TVPC55Oy5oB2jH5mMxJ5Z873IM/+wXKqp+SmZPo3nOFxfsSkYKPmtWFimrS5nWM1xhwY+3YrRvYM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157382800967139.60800658390349; Fri, 15 Nov 2019 06:26:49 -0800 (PST) Received: from localhost ([::1]:39890 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcYi-0000Dc-4n for importer@patchew.org; Fri, 15 Nov 2019 09:26:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59007) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNK-0004dL-RA for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNJ-0001xb-Jy for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:02 -0500 Received: from relay.sw.ru ([185.231.240.75]:47422) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001ro-Iz; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN4-0006WW-Sp; Fri, 15 Nov 2019 17:14:46 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 09/24] block/block-copy: alloc task on each iteration Date: Fri, 15 Nov 2019 17:14:29 +0300 Message-Id: <20191115141444.24155-10-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" We are going to use aio-task-pool API, so tasks will be handled in parallel. We need therefore separate allocated task on each iteration. Introduce this logic now. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index b00cbfc841..956b4b55e7 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -111,9 +111,11 @@ static bool coroutine_fn block_copy_wait_one(BlockCopy= State *s, int64_t start, } =20 /* Called only on full-dirty region */ -static void block_copy_task_begin(BlockCopyState *s, BlockCopyTask *task, - int64_t offset, int64_t bytes) +static BlockCopyTask *block_copy_task_create(BlockCopyState *s, + int64_t offset, int64_t bytes) { + BlockCopyTask *task =3D g_new(BlockCopyTask, 1); + assert(!block_copy_find_task(s, offset, bytes)); =20 bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); @@ -122,6 +124,8 @@ static void block_copy_task_begin(BlockCopyState *s, Bl= ockCopyTask *task, task->bytes =3D bytes; qemu_co_queue_init(&task->wait_queue); QLIST_INSERT_HEAD(&s->tasks, task, list); + + return task; } =20 static void coroutine_fn block_copy_task_shrink(BlockCopyState *s, @@ -454,7 +458,7 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); =20 while (bytes) { - BlockCopyTask task; + g_autofree BlockCopyTask *task =3D NULL; int64_t next_zero, cur_bytes, status_bytes; =20 if (!bdrv_dirty_bitmap_get(s->copy_bitmap, offset)) { @@ -475,12 +479,12 @@ static int coroutine_fn block_copy_dirty_clusters(Blo= ckCopyState *s, assert(next_zero < offset + cur_bytes); /* no need to do MIN()= */ cur_bytes =3D next_zero - offset; } - block_copy_task_begin(s, &task, offset, cur_bytes); + task =3D block_copy_task_create(s, offset, cur_bytes); =20 ret =3D block_copy_block_status(s, offset, cur_bytes, &status_byte= s); - block_copy_task_shrink(s, &task, status_bytes); + block_copy_task_shrink(s, task, status_bytes); if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) { - block_copy_task_end(s, &task, 0); + block_copy_task_end(s, task, 0); s->progress_reset_callback(s->progress_opaque); trace_block_copy_skip_range(s, offset, status_bytes); offset +=3D status_bytes; @@ -496,7 +500,7 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, ret =3D block_copy_do_copy(s, offset, cur_bytes, ret & BDRV_BLOCK_= ZERO, error_is_read); co_put_to_shres(s->mem, cur_bytes); - block_copy_task_end(s, &task, ret); + block_copy_task_end(s, task, ret); if (ret < 0) { return ret; } --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828370; cv=none; d=zoho.com; s=zohoarc; b=n6c5q7K0CCuNPYwx2H5EfL+mvqWFuqYKKb7ChlVNCmmyJutQaNTjNiUxzu98khi6Q42b5xMRuCCNnpgS6NZ/uto3pM6xDaTnCDNfd7iaP4886XtSv1kuKW8LcDpVOmZASjBh6HFy8jykuxt2zVbLixslkUYxNEYPoqY9le/Ys+0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828370; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yG9kccSQDuo4UkyFeD8yMdutnlmba30Aoy1MtR7cGCc=; b=iV90beOmSa+MFCAx/bGxifNyjcmt4qhPftTDIsLyjYuZzchG9fS07cq8ZXphR6pqvzcfkScVAHNd6QOvTjJQPsIpEtRl+0K49tvGTod6EiDJqzobSabPwI+rk89OiGtxSDikSzPQRitkbc/0yHgo+cZfr2d0lwn3c4c8tbCBX5o= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828370542287.46713273311696; Fri, 15 Nov 2019 06:32:50 -0800 (PST) Received: from localhost ([::1]:39966 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVceU-0006VZ-A4 for importer@patchew.org; Fri, 15 Nov 2019 09:32:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59289) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNf-00058a-KZ for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNe-0002Ba-30 for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:23 -0500 Received: from relay.sw.ru ([185.231.240.75]:47488) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNa-0001t8-WE; Fri, 15 Nov 2019 09:15:19 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN5-0006WW-1h; Fri, 15 Nov 2019 17:14:47 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 10/24] block/block-copy: add state pointer to BlockCopyTask Date: Fri, 15 Nov 2019 17:14:30 +0300 Message-Id: <20191115141444.24155-11-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" We are going to use aio-task-pool API, so we'll need state pointer in BlockCopyTask anyway. Add it now and use where possible. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 956b4b55e7..7652b4afc5 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -25,6 +25,7 @@ #define BLOCK_COPY_MAX_MEM (128 * MiB) =20 typedef struct BlockCopyTask { + BlockCopyState *s; int64_t offset; int64_t bytes; QLIST_ENTRY(BlockCopyTask) list; @@ -120,16 +121,18 @@ static BlockCopyTask *block_copy_task_create(BlockCop= yState *s, =20 bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); =20 - task->offset =3D offset; - task->bytes =3D bytes; + *task =3D (BlockCopyTask) { + .s =3D s, + .offset =3D offset, + .bytes =3D bytes, + }; qemu_co_queue_init(&task->wait_queue); QLIST_INSERT_HEAD(&s->tasks, task, list); =20 return task; } =20 -static void coroutine_fn block_copy_task_shrink(BlockCopyState *s, - BlockCopyTask *task, +static void coroutine_fn block_copy_task_shrink(BlockCopyTask *task, int64_t new_bytes) { if (new_bytes =3D=3D task->bytes) { @@ -138,18 +141,17 @@ static void coroutine_fn block_copy_task_shrink(Block= CopyState *s, =20 assert(new_bytes > 0 && new_bytes < task->bytes); =20 - bdrv_set_dirty_bitmap(s->copy_bitmap, + bdrv_set_dirty_bitmap(task->s->copy_bitmap, task->offset + new_bytes, task->bytes - new_byte= s); =20 task->bytes =3D new_bytes; qemu_co_queue_restart_all(&task->wait_queue); } =20 -static void coroutine_fn block_copy_task_end(BlockCopyState *s, - BlockCopyTask *task, int ret) +static void coroutine_fn block_copy_task_end(BlockCopyTask *task, int ret) { if (ret < 0) { - bdrv_set_dirty_bitmap(s->copy_bitmap, task->offset, task->bytes); + bdrv_set_dirty_bitmap(task->s->copy_bitmap, task->offset, task->by= tes); } QLIST_REMOVE(task, list); qemu_co_queue_restart_all(&task->wait_queue); @@ -482,9 +484,9 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, task =3D block_copy_task_create(s, offset, cur_bytes); =20 ret =3D block_copy_block_status(s, offset, cur_bytes, &status_byte= s); - block_copy_task_shrink(s, task, status_bytes); + block_copy_task_shrink(task, status_bytes); if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) { - block_copy_task_end(s, task, 0); + block_copy_task_end(task, 0); s->progress_reset_callback(s->progress_opaque); trace_block_copy_skip_range(s, offset, status_bytes); offset +=3D status_bytes; @@ -500,7 +502,7 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, ret =3D block_copy_do_copy(s, offset, cur_bytes, ret & BDRV_BLOCK_= ZERO, error_is_read); co_put_to_shres(s->mem, cur_bytes); - block_copy_task_end(s, task, ret); + block_copy_task_end(task, ret); if (ret < 0) { return ret; } --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573827797; cv=none; d=zoho.com; s=zohoarc; b=aMQO9BzAKH4S9jOC1Nktx99nZ3qZffv9cOVpBxuFDBMGX52FlrmtTkSMcvjupYsCU/G6fqkTkHPIsNoQow+YejEgsU2+gSuF1bOYIgHkMN1vXOf3C2nIEG1fNTZY7fa+PGHMBmStOf5b3EBrtEuVg3oNAzB5LlPg3VFxX+iemUA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573827797; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=rSxdweHd1WP/eeiIBNTMqSnZD2KQyJ2Ajr/SBk6h18E=; b=muhzBYtpONm/u/uA7FpoVlLqzoP8atoMHAC36gLNfVVUteTfyVV6Sx6QXJGcvBldgSObSjVrl+zlC6mxOGD59YNFHngIL8x53krWId/o/JTJGSyQ2PWPcLASvWtBHq0C1HBu8zjrfHUjGuzBsMFXgYMcl2Io2u7bi2rTHZWEeFE= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573827797875886.5962602251981; Fri, 15 Nov 2019 06:23:17 -0800 (PST) Received: from localhost ([::1]:39842 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcVI-0003jR-4v for importer@patchew.org; Fri, 15 Nov 2019 09:23:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59291) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNf-00058e-Li for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNe-0002Bs-7m for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:23 -0500 Received: from relay.sw.ru ([185.231.240.75]:47476) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNb-0001tB-0w; Fri, 15 Nov 2019 09:15:19 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN5-0006WW-78; Fri, 15 Nov 2019 17:14:47 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 11/24] block/block-copy: move task size initial calculation to _task_create Date: Fri, 15 Nov 2019 17:14:31 +0300 Message-Id: <20191115141444.24155-12-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Comment "Called only on full-dirty region" without corresponding assertion is a very unsafe thing. Adding assertion means call bdrv_dirty_bitmap_next_zero twice. Instead, let's move bdrv_dirty_bitmap_next_zero call to block_copy_task_create. It also allows to drop cur_bytes variable which partly duplicate task->bytes. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 7652b4afc5..0525a9fcd5 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -111,12 +111,23 @@ static bool coroutine_fn block_copy_wait_one(BlockCop= yState *s, int64_t start, return true; } =20 -/* Called only on full-dirty region */ static BlockCopyTask *block_copy_task_create(BlockCopyState *s, int64_t offset, int64_t bytes) { + int64_t next_zero; BlockCopyTask *task =3D g_new(BlockCopyTask, 1); =20 + assert(bdrv_dirty_bitmap_get(s->copy_bitmap, offset)); + + bytes =3D MIN(bytes, s->copy_size); + next_zero =3D bdrv_dirty_bitmap_next_zero(s->copy_bitmap, offset, byte= s); + if (next_zero >=3D 0) { + assert(next_zero > offset); /* offset is dirty */ + assert(next_zero < offset + bytes); /* no need to do MIN() */ + bytes =3D next_zero - offset; + } + + /* region is dirty, so no existent tasks possible in it */ assert(!block_copy_find_task(s, offset, bytes)); =20 bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); @@ -461,7 +472,7 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, =20 while (bytes) { g_autofree BlockCopyTask *task =3D NULL; - int64_t next_zero, cur_bytes, status_bytes; + int64_t status_bytes; =20 if (!bdrv_dirty_bitmap_get(s->copy_bitmap, offset)) { trace_block_copy_skip(s, offset); @@ -472,18 +483,9 @@ static int coroutine_fn block_copy_dirty_clusters(Bloc= kCopyState *s, =20 found_dirty =3D true; =20 - cur_bytes =3D MIN(bytes, s->copy_size); + task =3D block_copy_task_create(s, offset, bytes); =20 - next_zero =3D bdrv_dirty_bitmap_next_zero(s->copy_bitmap, offset, - cur_bytes); - if (next_zero >=3D 0) { - assert(next_zero > offset); /* offset is dirty */ - assert(next_zero < offset + cur_bytes); /* no need to do MIN()= */ - cur_bytes =3D next_zero - offset; - } - task =3D block_copy_task_create(s, offset, cur_bytes); - - ret =3D block_copy_block_status(s, offset, cur_bytes, &status_byte= s); + ret =3D block_copy_block_status(s, offset, task->bytes, &status_by= tes); block_copy_task_shrink(task, status_bytes); if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) { block_copy_task_end(task, 0); @@ -494,22 +496,20 @@ static int coroutine_fn block_copy_dirty_clusters(Blo= ckCopyState *s, continue; } =20 - cur_bytes =3D MIN(cur_bytes, status_bytes); - trace_block_copy_process(s, offset); =20 - co_get_from_shres(s->mem, cur_bytes); - ret =3D block_copy_do_copy(s, offset, cur_bytes, ret & BDRV_BLOCK_= ZERO, + co_get_from_shres(s->mem, task->bytes); + ret =3D block_copy_do_copy(s, offset, task->bytes, ret & BDRV_BLOC= K_ZERO, error_is_read); - co_put_to_shres(s->mem, cur_bytes); + co_put_to_shres(s->mem, task->bytes); block_copy_task_end(task, ret); if (ret < 0) { return ret; } =20 - s->progress_bytes_callback(cur_bytes, s->progress_opaque); - offset +=3D cur_bytes; - bytes -=3D cur_bytes; + s->progress_bytes_callback(task->bytes, s->progress_opaque); + offset +=3D task->bytes; + bytes -=3D task->bytes; } =20 return found_dirty; --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573827892; cv=none; d=zoho.com; s=zohoarc; b=HcSTjF/mtTqb10OiaCUu/N8v83gAHBn7uBlwFE7ifi4z0GCc4SQxy+0PvjSR1jdshDtgJYLVyN5wwe7IOS+FAGxC263x8DOni20xvTYIZegWwnoyacOa/8DH3upqxt+AkZWV93XgedprVRzbr8XW8Y5VTa9o50wNXavk9/UMlxs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573827892; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lpu3ZX0gd7zlGO66LLYnvvPom4F5yCpFX3joKL8An84=; b=G5OSY9FtyiZnN5G/N5ZHy7SKU19gfu3KAsIZzdkXUz3M5JEePJ+E06h/zsyauuNlykk8kUV4tY2Qyvs5u+EAe0epgKwrDm+RMNWtc0aclUYhFX4HZhMNqyqUw0ejvoR8EyI7yEqY2wpaub++uA7mkdABmnVajx8b9VM5VQzvXOc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157382789232363.19789613839771; Fri, 15 Nov 2019 06:24:52 -0800 (PST) Received: from localhost ([::1]:39866 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcWo-0006B7-QP for importer@patchew.org; Fri, 15 Nov 2019 09:24:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59000) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNK-0004dF-Oe for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNJ-0001xS-Gk for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:02 -0500 Received: from relay.sw.ru ([185.231.240.75]:47452) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001rs-J4; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcN5-0006WW-Ad; Fri, 15 Nov 2019 17:14:47 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 12/24] block/block-copy: move block_copy_task_create down Date: Fri, 15 Nov 2019 17:14:32 +0300 Message-Id: <20191115141444.24155-13-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Simple movement without any change. It's needed for the following patch, as this function will need to use some staff which is currently below it. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 64 +++++++++++++++++++++++----------------------- 1 file changed, 32 insertions(+), 32 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 0525a9fcd5..4e8c64a45d 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -111,38 +111,6 @@ static bool coroutine_fn block_copy_wait_one(BlockCopy= State *s, int64_t start, return true; } =20 -static BlockCopyTask *block_copy_task_create(BlockCopyState *s, - int64_t offset, int64_t bytes) -{ - int64_t next_zero; - BlockCopyTask *task =3D g_new(BlockCopyTask, 1); - - assert(bdrv_dirty_bitmap_get(s->copy_bitmap, offset)); - - bytes =3D MIN(bytes, s->copy_size); - next_zero =3D bdrv_dirty_bitmap_next_zero(s->copy_bitmap, offset, byte= s); - if (next_zero >=3D 0) { - assert(next_zero > offset); /* offset is dirty */ - assert(next_zero < offset + bytes); /* no need to do MIN() */ - bytes =3D next_zero - offset; - } - - /* region is dirty, so no existent tasks possible in it */ - assert(!block_copy_find_task(s, offset, bytes)); - - bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); - - *task =3D (BlockCopyTask) { - .s =3D s, - .offset =3D offset, - .bytes =3D bytes, - }; - qemu_co_queue_init(&task->wait_queue); - QLIST_INSERT_HEAD(&s->tasks, task, list); - - return task; -} - static void coroutine_fn block_copy_task_shrink(BlockCopyTask *task, int64_t new_bytes) { @@ -348,6 +316,38 @@ out: return ret; } =20 +static BlockCopyTask *block_copy_task_create(BlockCopyState *s, + int64_t offset, int64_t bytes) +{ + int64_t next_zero; + BlockCopyTask *task =3D g_new(BlockCopyTask, 1); + + assert(bdrv_dirty_bitmap_get(s->copy_bitmap, offset)); + + bytes =3D MIN(bytes, s->copy_size); + next_zero =3D bdrv_dirty_bitmap_next_zero(s->copy_bitmap, offset, byte= s); + if (next_zero >=3D 0) { + assert(next_zero > offset); /* offset is dirty */ + assert(next_zero < offset + bytes); /* no need to do MIN() */ + bytes =3D next_zero - offset; + } + + /* region is dirty, so no existent tasks possible in it */ + assert(!block_copy_find_task(s, offset, bytes)); + + bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); + + *task =3D (BlockCopyTask) { + .s =3D s, + .offset =3D offset, + .bytes =3D bytes, + }; + qemu_co_queue_init(&task->wait_queue); + QLIST_INSERT_HEAD(&s->tasks, task, list); + + return task; +} + static int block_copy_block_status(BlockCopyState *s, int64_t offset, int64_t bytes, int64_t *pnum) { --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828727; cv=none; d=zoho.com; s=zohoarc; b=PeDVG79AIXyDhG3BTmG1CCy1kQwrhwRVZAhavutBZpZDkfbLFt21MEWThYa1LJ1ZSYqW6yPkQ3cGGL4qepbHo+ZrSBapWG5X2vFo6eiKrlp4sIB1e83buGO9unf66vNRfICVBnQnuNS5DD+4hYpTHKKUwG8RoZZ+4KGH2g/25Ko= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828727; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tmsrFZY8PEdvOVWRV+y18SNGVN5wp6cxCggZetFFTE4=; b=joA83zvNM0L8MryOKjTnReXYsRn3xjc3Xbr37FHdp/knk/Tv3eHYaWawzBVcMqf2cm9IhJu5axxpF0XP/H/8EWaEKdmG7WA9tQQY7fsldXY1OrAEw/3CA2Bm3t0j8Ht0mO2TehIPbUGpbK0XVzEofYpHznGztZqyPkwFRbLoy00= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828727430327.5127040231864; Fri, 15 Nov 2019 06:38:47 -0800 (PST) Received: from localhost ([::1]:40050 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVckI-0005j3-Bs for importer@patchew.org; Fri, 15 Nov 2019 09:38:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59349) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNj-0005B7-Lk for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNe-0002CX-QI for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:27 -0500 Received: from relay.sw.ru ([185.231.240.75]:47490) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNb-0001tW-DY; Fri, 15 Nov 2019 09:15:19 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNA-0006WW-EX; Fri, 15 Nov 2019 17:14:52 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 13/24] block/block-copy: use aio-task-pool API Date: Fri, 15 Nov 2019 17:14:33 +0300 Message-Id: <20191115141444.24155-14-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Run block_copy iterations in parallel in aio tasks. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 102 +++++++++++++++++++++++++++++++++++++++------ 1 file changed, 90 insertions(+), 12 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 4e8c64a45d..b2e7825b53 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -19,15 +19,27 @@ #include "block/block-copy.h" #include "sysemu/block-backend.h" #include "qemu/units.h" +#include "qemu/coroutine.h" +#include "block/aio_task.h" =20 #define BLOCK_COPY_MAX_COPY_RANGE (16 * MiB) #define BLOCK_COPY_MAX_BUFFER (1 * MiB) #define BLOCK_COPY_MAX_MEM (128 * MiB) +#define BLOCK_COPY_MAX_WORKERS 64 + +typedef struct BlockCopyCallState { + bool failed; + bool error_is_read; +} BlockCopyCallState; =20 typedef struct BlockCopyTask { + AioTask task; + BlockCopyState *s; + BlockCopyCallState *call_state; int64_t offset; int64_t bytes; + bool zeroes; QLIST_ENTRY(BlockCopyTask) list; CoQueue wait_queue; /* coroutines blocked on this task */ } BlockCopyTask; @@ -217,6 +229,30 @@ void block_copy_set_callbacks( s->progress_opaque =3D progress_opaque; } =20 +/* Takes ownership on @task */ +static coroutine_fn int block_copy_task_run(AioTaskPool *pool, + BlockCopyTask *task) +{ + if (!pool) { + int ret =3D task->task.func(&task->task); + + g_free(task); + return ret; + } + + aio_task_pool_wait_slot(pool); + if (aio_task_pool_status(pool) < 0) { + co_put_to_shres(task->s->mem, task->bytes); + block_copy_task_end(task, -EAGAIN); + g_free(task); + return aio_task_pool_status(pool); + } + + aio_task_pool_start_task(pool, &task->task); + + return 0; +} + /* * block_copy_do_copy * @@ -316,8 +352,31 @@ out: return ret; } =20 +static coroutine_fn int block_copy_task_entry(AioTask *task) +{ + BlockCopyTask *t =3D container_of(task, BlockCopyTask, task); + bool error_is_read; + int ret; + + ret =3D block_copy_do_copy(t->s, t->offset, t->bytes, t->zeroes, + &error_is_read); + if (ret < 0 && !t->call_state->failed) { + t->call_state->failed =3D true; + t->call_state->error_is_read =3D error_is_read; + } else { + t->s->progress_bytes_callback(t->bytes, t->s->progress_opaque); + } + co_put_to_shres(t->s->mem, t->bytes); + block_copy_task_end(t, ret); + + return ret; +} + +/* Called only on full-dirty region */ static BlockCopyTask *block_copy_task_create(BlockCopyState *s, - int64_t offset, int64_t bytes) + BlockCopyCallState *call_stat= e, + int64_t offset, + int64_t bytes) { int64_t next_zero; BlockCopyTask *task =3D g_new(BlockCopyTask, 1); @@ -338,7 +397,9 @@ static BlockCopyTask *block_copy_task_create(BlockCopyS= tate *s, bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); =20 *task =3D (BlockCopyTask) { + .task.func =3D block_copy_task_entry, .s =3D s, + .call_state =3D call_state, .offset =3D offset, .bytes =3D bytes, }; @@ -459,6 +520,8 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, { int ret =3D 0; bool found_dirty =3D false; + AioTaskPool *aio =3D NULL; + BlockCopyCallState call_state =3D {false, false}; =20 /* * block_copy() user is responsible for keeping source and target in s= ame @@ -470,8 +533,8 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); =20 - while (bytes) { - g_autofree BlockCopyTask *task =3D NULL; + while (bytes && aio_task_pool_status(aio) =3D=3D 0) { + BlockCopyTask *task; int64_t status_bytes; =20 if (!bdrv_dirty_bitmap_get(s->copy_bitmap, offset)) { @@ -483,36 +546,51 @@ static int coroutine_fn block_copy_dirty_clusters(Blo= ckCopyState *s, =20 found_dirty =3D true; =20 - task =3D block_copy_task_create(s, offset, bytes); + task =3D block_copy_task_create(s, &call_state, offset, bytes); =20 ret =3D block_copy_block_status(s, offset, task->bytes, &status_by= tes); block_copy_task_shrink(task, status_bytes); if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) { block_copy_task_end(task, 0); + g_free(task); s->progress_reset_callback(s->progress_opaque); trace_block_copy_skip_range(s, offset, status_bytes); offset +=3D status_bytes; bytes -=3D status_bytes; continue; } + task->zeroes =3D ret & BDRV_BLOCK_ZERO; =20 trace_block_copy_process(s, offset); =20 co_get_from_shres(s->mem, task->bytes); - ret =3D block_copy_do_copy(s, offset, task->bytes, ret & BDRV_BLOC= K_ZERO, - error_is_read); - co_put_to_shres(s->mem, task->bytes); - block_copy_task_end(task, ret); - if (ret < 0) { - return ret; + + if (!aio && task->bytes !=3D bytes) { + aio =3D aio_task_pool_new(BLOCK_COPY_MAX_WORKERS); } =20 - s->progress_bytes_callback(task->bytes, s->progress_opaque); offset +=3D task->bytes; bytes -=3D task->bytes; + + ret =3D block_copy_task_run(aio, task); + if (ret < 0) { + goto out; + } + } + +out: + if (aio) { + aio_task_pool_wait_all(aio); + if (ret =3D=3D 0) { + ret =3D aio_task_pool_status(aio); + } + g_free(aio); + } + if (error_is_read && ret < 0) { + *error_is_read =3D call_state.error_is_read; } =20 - return found_dirty; + return ret < 0 ? ret : found_dirty; } =20 int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t byt= es, --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828527; cv=none; d=zoho.com; s=zohoarc; b=GHj4NVmdl9xFt/qrsz5Ta7Pww/hiV1nRFHqlGN+kBfJYR/06cTSuxX1ryxFiGbaWmVFjFt/gb/b1qxlEhSiRwmqr9MjnxPjdoZTrk5xTZs1OFJq4zRjPB8EJHkf4BwNXKo4j0DClhLoWFLNj0JMdt71fTrXNUCz1U62pAirtXIM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828527; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=stUc2214eD3jMtVkeUNiLxRKIAkQSoEi0bOP37XfktI=; b=HUu+vORCQzlTpFDxrSIV89QEafV7v+ocIRKzIwOnWXwqzHVAZV7E7zurdH2U7HZylqOPQJad74eldocMZ7SjCPaWBar7mKAuLHBbaSkkm+yzCkmXyKkAHNagRE65v+OH1nBCuDBvDa1FhH+ZszOWa1mfk3MdL0aiGCFOnDlvJ5M= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828527564782.4411414325333; Fri, 15 Nov 2019 06:35:27 -0800 (PST) Received: from localhost ([::1]:40006 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVch4-0001Kj-7f for importer@patchew.org; Fri, 15 Nov 2019 09:35:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59031) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNL-0004e8-9w for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNJ-0001xn-NR for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from relay.sw.ru ([185.231.240.75]:47442) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001rr-Db; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNB-0006WW-0n; Fri, 15 Nov 2019 17:14:53 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 14/24] block/block-copy: More explicit call_state Date: Fri, 15 Nov 2019 17:14:34 +0300 Message-Id: <20191115141444.24155-15-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Refactor common path to use BlockCopyCallState pointer as parameter, to prepare it for use in asynchronous block-copy (at least, we'll need to run block-copy in a coroutine, passing the whole parameters as one pointer). Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 50 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 38 insertions(+), 12 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index b2e7825b53..4b1a0cecbd 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -28,7 +28,15 @@ #define BLOCK_COPY_MAX_WORKERS 64 =20 typedef struct BlockCopyCallState { + /* IN parameters */ + BlockCopyState *s; + int64_t offset; + int64_t bytes; + + /* State */ bool failed; + + /* OUT parameters */ bool error_is_read; } BlockCopyCallState; =20 @@ -514,14 +522,16 @@ int64_t block_copy_reset_unallocated(BlockCopyState *= s, * Returns 1 if dirty clusters found and successfully copied, 0 if no dirty * clusters found and -errno on failure. */ -static int coroutine_fn block_copy_dirty_clusters(BlockCopyState *s, - int64_t offset, int64_t = bytes, - bool *error_is_read) +static int coroutine_fn +block_copy_dirty_clusters(BlockCopyCallState *call_state) { + BlockCopyState *s =3D call_state->s; + int64_t offset =3D call_state->offset; + int64_t bytes =3D call_state->bytes; + int ret =3D 0; bool found_dirty =3D false; AioTaskPool *aio =3D NULL; - BlockCopyCallState call_state =3D {false, false}; =20 /* * block_copy() user is responsible for keeping source and target in s= ame @@ -546,7 +556,7 @@ static int coroutine_fn block_copy_dirty_clusters(Block= CopyState *s, =20 found_dirty =3D true; =20 - task =3D block_copy_task_create(s, &call_state, offset, bytes); + task =3D block_copy_task_create(s, call_state, offset, bytes); =20 ret =3D block_copy_block_status(s, offset, task->bytes, &status_by= tes); block_copy_task_shrink(task, status_bytes); @@ -586,18 +596,14 @@ out: } g_free(aio); } - if (error_is_read && ret < 0) { - *error_is_read =3D call_state.error_is_read; - } =20 return ret < 0 ? ret : found_dirty; } =20 -int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t byt= es, - bool *error_is_read) +static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) { while (true) { - int ret =3D block_copy_dirty_clusters(s, start, bytes, error_is_re= ad); + int ret =3D block_copy_dirty_clusters(call_state); =20 if (ret < 0) { /* @@ -620,7 +626,9 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t = start, uint64_t bytes, * requested region. */ =20 - if (!block_copy_wait_one(s, start, bytes)) { + if (!block_copy_wait_one(call_state->s, call_state->offset, + call_state->bytes)) + { /* No dirty bits and nothing to wait: the whole request is don= e */ break; } @@ -629,6 +637,24 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t= start, uint64_t bytes, return 0; } =20 +int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t byt= es, + bool *error_is_read) +{ + BlockCopyCallState call_state =3D { + .s =3D s, + .offset =3D start, + .bytes =3D bytes, + }; + + int ret =3D block_copy_common(&call_state); + + if (error_is_read && ret < 0) { + *error_is_read =3D call_state.error_is_read; + } + + return ret; +} + BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s) { return s->copy_bitmap; --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828219; cv=none; d=zoho.com; s=zohoarc; b=ZuQVXLGIIf6omNrTS6HlMs9BE8FeSaWEnD3J5IDJYcY23A38oOLlCYAsf9GN+eKYBN+faSDi3XX/SX47RxfdVER2YpJKEV2FPdBrJfhlR6Z7L9iQNi9uaPeFNDggpUNNIvegPNTGSKsegbY1ZZazU3wcEjRZZ9gxnvw2RPuLIT4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828219; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AZv4KA6OooB05BQKGz/ro7a73hQrwf+O7MR2s0gAUCs=; b=JOcSpiOJZxfGW3rS8pseXouen5lOSkZ6X/hnKvYn5g92h2+bcMslB7FGtcQRibthFMNBUNVlEoszDK4Oct43EO1qQUEHkjmXwNHC5A6arIYDW6T1vG99yibJpCX1ymB7lDjSOAGLkgQOkQkVenGGVew0hA55MrALZac3gB6k+K4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828219133654.7066632539074; Fri, 15 Nov 2019 06:30:19 -0800 (PST) Received: from localhost ([::1]:39928 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcc5-0004Gv-GJ for importer@patchew.org; Fri, 15 Nov 2019 09:30:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59027) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNL-0004e0-9O for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNJ-0001y1-T9 for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from relay.sw.ru ([185.231.240.75]:47418) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001rz-GR; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNB-0006WW-8E; Fri, 15 Nov 2019 17:14:53 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 15/24] block/block-copy: implement block_copy_async Date: Fri, 15 Nov 2019 17:14:35 +0300 Message-Id: <20191115141444.24155-16-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" We'll need async block-copy invocation to use in backup directly. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 13 +++++++++++ block/block-copy.c | 48 +++++++++++++++++++++++++++++++++++--- 2 files changed, 58 insertions(+), 3 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 753fa663ac..407de7e520 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -20,7 +20,10 @@ =20 typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque); typedef void (*ProgressResetCallbackFunc)(void *opaque); +typedef void (*BlockCopyAsyncCallbackFunc)(int ret, bool error_is_read, + void *opaque); typedef struct BlockCopyState BlockCopyState; +typedef struct BlockCopyCallState BlockCopyCallState; =20 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, int64_t cluster_size, @@ -41,6 +44,16 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t byt= es, bool *error_is_read); =20 +/* + * Run block-copy in a coroutine, return state pointer. If finished early + * returns NULL (@cb is called anyway). + */ +BlockCopyCallState *block_copy_async(BlockCopyState *s, + int64_t offset, int64_t bytes, + bool ratelimit, int max_workers, + int64_t max_chunk, + BlockCopyAsyncCallbackFunc cb); + BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s); void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip); =20 diff --git a/block/block-copy.c b/block/block-copy.c index 4b1a0cecbd..0b41afd30d 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -32,9 +32,11 @@ typedef struct BlockCopyCallState { BlockCopyState *s; int64_t offset; int64_t bytes; + BlockCopyAsyncCallbackFunc cb; =20 /* State */ bool failed; + bool finished; =20 /* OUT parameters */ bool error_is_read; @@ -602,15 +604,17 @@ out: =20 static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) { + int ret =3D 0; + while (true) { - int ret =3D block_copy_dirty_clusters(call_state); + ret =3D block_copy_dirty_clusters(call_state); =20 if (ret < 0) { /* * IO operation failed, which means the whole block_copy reque= st * failed. */ - return ret; + break; } if (ret) { /* @@ -634,7 +638,14 @@ static int coroutine_fn block_copy_common(BlockCopyCal= lState *call_state) } } =20 - return 0; + if (call_state->cb) { + call_state->cb(ret, call_state->error_is_read, + call_state->s->progress_opaque); + } + + call_state->finished =3D true; + + return ret; } =20 int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t byt= es, @@ -655,6 +666,37 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t= start, uint64_t bytes, return ret; } =20 +static void coroutine_fn block_copy_async_co_entry(void *opaque) +{ + block_copy_common(opaque); +} + +BlockCopyCallState *block_copy_async(BlockCopyState *s, + int64_t offset, int64_t bytes, + bool ratelimit, int max_workers, + int64_t max_chunk, + BlockCopyAsyncCallbackFunc cb) +{ + BlockCopyCallState *call_state =3D g_new(BlockCopyCallState, 1); + Coroutine *co =3D qemu_coroutine_create(block_copy_async_co_entry, + call_state); + + *call_state =3D (BlockCopyCallState) { + .s =3D s, + .offset =3D offset, + .bytes =3D bytes, + .cb =3D cb, + }; + + qemu_coroutine_enter(co); + + if (call_state->finished) { + g_free(call_state); + return NULL; + } + + return call_state; +} BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s) { return s->copy_bitmap; --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573827684; cv=none; d=zoho.com; s=zohoarc; b=De/BVb0XC7yVfoBhP5Ym1E91vIL7n/Yp/OnsEYhGamBMvlIwrfqkVJc7Y1d3E2EkxAXF2bLJB1P0Q9F9MOCKLuhhYoDasaccMo8Ny70xaZ15KIPufh3ibgVU4PfyD4Ym5T+fCuidnzcCnfAj7hx9XokWHqGUCyTcBGIicIiDLaA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573827684; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=X7iSr2Fay9LlADuftKugGbnHLx7TSqoupFS7ksW1TNc=; b=E6GqexyzqiqJ6pA2JKsTvpBE0RxtuBI5m+vPjfrKXQuSZoG5rfN/i7HoH5Nkb9cFl5p2BPAisZnZRMNaF1nmh+UFVE0CwQ2+xFMbzV+StKrzxG9gQRVuZW6u5NqaGvUP7bAXrU1+YT+yfbx+xQoZBbDZ5TH4RrYvi9DqRrWNh6o= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573827684522703.5831538506205; Fri, 15 Nov 2019 06:21:24 -0800 (PST) Received: from localhost ([::1]:39823 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcTS-0001jb-Tj for importer@patchew.org; Fri, 15 Nov 2019 09:21:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:58986) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNK-0004cw-E9 for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNJ-0001x2-7C for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:02 -0500 Received: from relay.sw.ru ([185.231.240.75]:47428) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001rq-II; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNB-0006WW-EG; Fri, 15 Nov 2019 17:14:53 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 16/24] block/block-copy: add max_chunk and max_workers paramters Date: Fri, 15 Nov 2019 17:14:36 +0300 Message-Id: <20191115141444.24155-17-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" They will be used for backup. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 5 +++++ block/block-copy.c | 9 +++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 407de7e520..8fc27156b3 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -47,6 +47,11 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t s= tart, uint64_t bytes, /* * Run block-copy in a coroutine, return state pointer. If finished early * returns NULL (@cb is called anyway). + * + * @max_workers means maximum of parallel coroutines to execute sub-reques= ts, + * must be > 0. + * + * @max_chunk means maximum length for one IO operation. Zero means unlimi= ted. */ BlockCopyCallState *block_copy_async(BlockCopyState *s, int64_t offset, int64_t bytes, diff --git a/block/block-copy.c b/block/block-copy.c index 0b41afd30d..f28f3b8b6c 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -32,6 +32,8 @@ typedef struct BlockCopyCallState { BlockCopyState *s; int64_t offset; int64_t bytes; + int max_workers; + int64_t max_chunk; BlockCopyAsyncCallbackFunc cb; =20 /* State */ @@ -393,7 +395,7 @@ static BlockCopyTask *block_copy_task_create(BlockCopyS= tate *s, =20 assert(bdrv_dirty_bitmap_get(s->copy_bitmap, offset)); =20 - bytes =3D MIN(bytes, s->copy_size); + bytes =3D MIN(bytes, MIN_NON_ZERO(s->copy_size, call_state->max_chunk)= ); next_zero =3D bdrv_dirty_bitmap_next_zero(s->copy_bitmap, offset, byte= s); if (next_zero >=3D 0) { assert(next_zero > offset); /* offset is dirty */ @@ -578,7 +580,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_stat= e) co_get_from_shres(s->mem, task->bytes); =20 if (!aio && task->bytes !=3D bytes) { - aio =3D aio_task_pool_new(BLOCK_COPY_MAX_WORKERS); + aio =3D aio_task_pool_new(call_state->max_workers); } =20 offset +=3D task->bytes; @@ -655,6 +657,7 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t = start, uint64_t bytes, .s =3D s, .offset =3D start, .bytes =3D bytes, + .max_workers =3D BLOCK_COPY_MAX_WORKERS, }; =20 int ret =3D block_copy_common(&call_state); @@ -686,6 +689,8 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, .offset =3D offset, .bytes =3D bytes, .cb =3D cb, + .max_workers =3D max_workers ?: BLOCK_COPY_MAX_WORKERS, + .max_chunk =3D max_chunk, }; =20 qemu_coroutine_enter(co); --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828169; cv=none; d=zoho.com; s=zohoarc; b=ST2DCX0GaHVQ1VKzLQkLT/sPAuMqCOsCZ5yyzoS2rpi1xcAePk2qQoh0gzbo9+RaBw7NcVrGIX8HZsJHc9cdqxQ3HYmDdjnt2DgjOS2098W+nS9anDPWGRY9KCke42/NTcYFP4RMNocABjZPDCA+pyGHqP3bhBXKGdX91ZgO0g4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828169; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RhY6fqV0CRXwp3k6oAxQ6zrdC6UqegugEV14LMrcwh8=; b=mERZyk1oF7X+Sx7ip4h0aydX5N6i9RTCKLQ0UB0w2TCRRq85KfE08TEmDnF5+anu+qgA3wUl9jH8m8LpFfsx56fbSgHgBRBy2yQT+w2uR6rZJy3PHsjReE1CfMVdAYAmAlXCKTEiPcis0CQML2XLXgVduQ4JkPrqaPOJHWuS4UY= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15738281694781019.5542521040021; Fri, 15 Nov 2019 06:29:29 -0800 (PST) Received: from localhost ([::1]:39920 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcbH-0003K4-O3 for importer@patchew.org; Fri, 15 Nov 2019 09:29:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59040) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNL-0004eJ-HY for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNK-0001yD-0s for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from relay.sw.ru ([185.231.240.75]:47414) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001ry-J3; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNB-0006WW-NH; Fri, 15 Nov 2019 17:14:54 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 17/24] block/block-copy: add ratelimit to block-copy Date: Fri, 15 Nov 2019 17:14:37 +0300 Message-Id: <20191115141444.24155-18-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" We are going to directly use one async block-copy operation for backup job, so we need rate limitator. We want to maintain current backup behavior: only background copying is limited and copy-before-write operations only participate in limit calculation. Therefore we need one rate limitator for block-copy state and boolean flag for block-copy call state for actual limitation. Note, that we can't just calculate each chunk in limitator after successful copying: it will not restring starting a lot of async sub-requests which will exceed limit too much. Instead let's use the following scheme on sub-request creation: 1. If at the moment limit is not exceeded, create the request and account it immediately. 2. If at the moment limit is already exceeded, drop create sub-request and handle limit instead (by sleep). With this approach we'll never exceed the limit more than by one sub-request (which pretty much matches current backup behavior). Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 8 +++++++ block/block-copy.c | 44 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 8fc27156b3..3f9cdc5eb2 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -59,6 +59,14 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, int64_t max_chunk, BlockCopyAsyncCallbackFunc cb); =20 +/* + * Set speed limit for block-copy instance. All block-copy operations rela= ted to + * this BlockCopyState will participate in speed calculation, but only + * block_copy_async calls with @ratelimit=3Dtrue will be actually limited. + */ +void block_copy_set_speed(BlockCopyState *s, BlockCopyCallState *call_stat= e, + uint64_t speed); + BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s); void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip); =20 diff --git a/block/block-copy.c b/block/block-copy.c index f28f3b8b6c..091bc044de 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -26,6 +26,7 @@ #define BLOCK_COPY_MAX_BUFFER (1 * MiB) #define BLOCK_COPY_MAX_MEM (128 * MiB) #define BLOCK_COPY_MAX_WORKERS 64 +#define BLOCK_COPY_SLICE_TIME 100000000ULL /* ns */ =20 typedef struct BlockCopyCallState { /* IN parameters */ @@ -34,11 +35,13 @@ typedef struct BlockCopyCallState { int64_t bytes; int max_workers; int64_t max_chunk; + bool ratelimit; BlockCopyAsyncCallbackFunc cb; =20 /* State */ bool failed; bool finished; + QemuCoSleepState *sleep_state; =20 /* OUT parameters */ bool error_is_read; @@ -101,6 +104,9 @@ typedef struct BlockCopyState { void *progress_opaque; =20 SharedResource *mem; + + uint64_t speed; + RateLimit rate_limit; } BlockCopyState; =20 static BlockCopyTask *block_copy_find_task(BlockCopyState *s, @@ -575,6 +581,21 @@ block_copy_dirty_clusters(BlockCopyCallState *call_sta= te) } task->zeroes =3D ret & BDRV_BLOCK_ZERO; =20 + if (s->speed) { + if (call_state->ratelimit) { + uint64_t ns =3D ratelimit_calculate_delay(&s->rate_limit, = 0); + if (ns > 0) { + block_copy_task_end(task, -EAGAIN); + g_free(task); + qemu_co_sleep_ns_wakeable(QEMU_CLOCK_REALTIME, ns, + &call_state->sleep_state); + continue; + } + } + + ratelimit_calculate_delay(&s->rate_limit, task->bytes); + } + trace_block_copy_process(s, offset); =20 co_get_from_shres(s->mem, task->bytes); @@ -604,6 +625,13 @@ out: return ret < 0 ? ret : found_dirty; } =20 +static void block_copy_kick(BlockCopyCallState *call_state) +{ + if (call_state->sleep_state) { + qemu_co_sleep_wake(call_state->sleep_state); + } +} + static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) { int ret =3D 0; @@ -688,6 +716,7 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, .s =3D s, .offset =3D offset, .bytes =3D bytes, + .ratelimit =3D ratelimit, .cb =3D cb, .max_workers =3D max_workers ?: BLOCK_COPY_MAX_WORKERS, .max_chunk =3D max_chunk, @@ -711,3 +740,18 @@ void block_copy_set_skip_unallocated(BlockCopyState *s= , bool skip) { s->skip_unallocated =3D skip; } + +void block_copy_set_speed(BlockCopyState *s, BlockCopyCallState *call_stat= e, + uint64_t speed) +{ + uint64_t old_speed =3D s->speed; + + s->speed =3D speed; + if (speed > 0) { + ratelimit_set_speed(&s->rate_limit, speed, BLOCK_COPY_SLICE_TIME); + } + + if (call_state && old_speed && (speed > old_speed || speed =3D=3D 0)) { + block_copy_kick(call_state); + } +} --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828056; cv=none; d=zoho.com; s=zohoarc; b=nxKRA5o/pT1OvdPRXdBZil7fDg0shq40eS6tNCsefezju4u92mvwBLTXBGZOBHdA/+9YDmL1ySd9JT5ES3Qs6LSp1sgbrTXkZHziv00hQDuFV7mI8KDsxPR0Hi7e8OL4ZXTt+wY/obL3Uv9wHMrlZMf+yXSsjcHk+8QLYItqefQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828056; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=6ZiuHjIWRHsj7ex2qdwWmWkUcQ4TmQ0Y70SVdIl6uOY=; b=hUT21H2VWobxhTszksugdfAnBCi9C93LaDaQ11BGuMHFmEt+5I58oYgH3AymNOtRg9nMhrD0QYFs7O3Yjs7Ec7q53109zuO3TmrR0h8fX+0q82UXCh+5ybsDrbdlFmwPDn1tLXEL2UCwOS0EESCI6E8Yh8z5qiO8HMosGWBI2rA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157382805638130.686360406548943; Fri, 15 Nov 2019 06:27:36 -0800 (PST) Received: from localhost ([::1]:39896 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcZS-000184-Sw for importer@patchew.org; Fri, 15 Nov 2019 09:27:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59006) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNK-0004dJ-Qq for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNJ-0001xg-KQ for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:02 -0500 Received: from relay.sw.ru ([185.231.240.75]:47410) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001rv-IL; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNC-0006WW-5k; Fri, 15 Nov 2019 17:14:54 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 18/24] block/block-copy: add block_copy_cancel Date: Fri, 15 Nov 2019 17:14:38 +0300 Message-Id: <20191115141444.24155-19-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Add function to cancel running async block-copy call. It will be used in backup. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 7 +++++++ block/block-copy.c | 20 ++++++++++++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 3f9cdc5eb2..fbbee094e6 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -67,6 +67,13 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, void block_copy_set_speed(BlockCopyState *s, BlockCopyCallState *call_stat= e, uint64_t speed); =20 +/* + * Cancel running block-copy call. + * Cancel leaves block-copy state valid: dirty bits are correct and you ma= y use + * cancel + to emulate pause/resume. + */ +void block_copy_cancel(BlockCopyCallState *call_state); + BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s); void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip); =20 diff --git a/block/block-copy.c b/block/block-copy.c index 091bc044de..d11c744320 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -42,6 +42,8 @@ typedef struct BlockCopyCallState { bool failed; bool finished; QemuCoSleepState *sleep_state; + bool cancelled; + Coroutine *canceller; =20 /* OUT parameters */ bool error_is_read; @@ -553,7 +555,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_stat= e) assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); =20 - while (bytes && aio_task_pool_status(aio) =3D=3D 0) { + while (bytes && aio_task_pool_status(aio) =3D=3D 0 && !call_state->can= celled) { BlockCopyTask *task; int64_t status_bytes; =20 @@ -639,7 +641,7 @@ static int coroutine_fn block_copy_common(BlockCopyCall= State *call_state) while (true) { ret =3D block_copy_dirty_clusters(call_state); =20 - if (ret < 0) { + if (ret < 0 || call_state->cancelled) { /* * IO operation failed, which means the whole block_copy reque= st * failed. @@ -673,6 +675,11 @@ static int coroutine_fn block_copy_common(BlockCopyCal= lState *call_state) call_state->s->progress_opaque); } =20 + if (call_state->canceller) { + aio_co_wake(call_state->canceller); + call_state->canceller =3D NULL; + } + call_state->finished =3D true; =20 return ret; @@ -731,6 +738,15 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, =20 return call_state; } + +void block_copy_cancel(BlockCopyCallState *call_state) +{ + call_state->cancelled =3D true; + call_state->canceller =3D qemu_coroutine_self(); + block_copy_kick(call_state); + qemu_coroutine_yield(); +} + BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s) { return s->copy_bitmap; --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573827745; cv=none; d=zoho.com; s=zohoarc; b=QOxZR5YCFbwxDolE8yCXDgd3Yzp3DMbBERXgjdu5pUMG+ZOIEjLG8XGw7t83cTAuWzbEvV09KV3ccBLl+glZ3UYOWWzla2InX2hCe6DVxswp55QxkcQv23opTyMW2FQDOOSJBWSqq0PItGwDXRrWpLBtyaE182j/3AB7VqDaszU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573827745; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Qm6njz4etsgec6TTyzYEbienOTEpdqdjOdJxsm1RfXo=; b=OHX9npLIQrcHy+J6/G7wpyJAXloN4sfin/Z0BD8l5J/RJiO1AqcYreDr4sYIlE+aJ1HJQ/fL7GyNlWlWhVmxDt8QrroMQ32KYYdM0i15Xxvk136zelrhL8xVGY7kcV1seAPa0JJQj/iaDDm2M7ZbQ5XA2xD25OGM1bU6cymXNsc= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573827745110406.89612104827177; Fri, 15 Nov 2019 06:22:25 -0800 (PST) Received: from localhost ([::1]:39832 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcUO-0002ei-Bc for importer@patchew.org; Fri, 15 Nov 2019 09:22:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59013) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNK-0004dQ-Tr for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNJ-0001xu-QZ for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:02 -0500 Received: from relay.sw.ru ([185.231.240.75]:47416) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001ru-GU; Fri, 15 Nov 2019 09:14:59 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNC-0006WW-Dg; Fri, 15 Nov 2019 17:14:54 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 19/24] blockjob: add set_speed to BlockJobDriver Date: Fri, 15 Nov 2019 17:14:39 +0300 Message-Id: <20191115141444.24155-20-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" We are going to use async block-copy call in backup, so we'll need to passthrough setting backup speed to block-copy call. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/blockjob_int.h | 2 ++ blockjob.c | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h index e2824a36a8..6633d83da2 100644 --- a/include/block/blockjob_int.h +++ b/include/block/blockjob_int.h @@ -52,6 +52,8 @@ struct BlockJobDriver { * besides job->blk to the new AioContext. */ void (*attached_aio_context)(BlockJob *job, AioContext *new_context); + + void (*set_speed)(BlockJob *job, int64_t speed); }; =20 /** diff --git a/blockjob.c b/blockjob.c index c6e20e2fcd..3b827d420d 100644 --- a/blockjob.c +++ b/blockjob.c @@ -255,6 +255,7 @@ static bool job_timer_pending(Job *job) =20 void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) { + const BlockJobDriver *drv =3D block_job_driver(job); int64_t old_speed =3D job->speed; =20 if (job_apply_verb(&job->job, JOB_VERB_SET_SPEED, errp)) { @@ -268,6 +269,11 @@ void block_job_set_speed(BlockJob *job, int64_t speed,= Error **errp) ratelimit_set_speed(&job->limit, speed, BLOCK_JOB_SLICE_TIME); =20 job->speed =3D speed; + + if (drv->set_speed) { + drv->set_speed(job, speed); + } + if (speed && speed <=3D old_speed) { return; } --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573827529; cv=none; d=zoho.com; s=zohoarc; b=d/wyNxiada1eAZPOi9SzKKLk5A/1s218PjnMrgkilXBfNp17WqkZz3LTw0MHYVGFF8WOyPIxkbSJQdQ4K7xCJ/UgF7z6kwmKB1D03eduNBt/fyLtmWa9E1YM1/sl8VNT/mwH4kf1XZtnFPRK11C/7MkEL4Gshj6b2GpzIARg4mo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573827529; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VmpKtKJdEPlzc1FxCKenDA/IZph5oyixO3aqM4hdpLo=; b=Ftk3JI55foPbH3A+nwtxKwyMByM2SGGUu/UMK8BEZIIpf1E6Q0RqL9w3szhJJ0GPjIQacZuefU279g0ieVMBzVyeTCXPTnP/BaMZqk7AK7AAY+eFt3FQoaeo4vIbXIvIwsQK+snuPw08htDEmEV2q4cj+J8SxbRIji9QdLbBexs= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573827529262933.6530640909488; Fri, 15 Nov 2019 06:18:49 -0800 (PST) Received: from localhost ([::1]:39791 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcQv-0007SG-0P for importer@patchew.org; Fri, 15 Nov 2019 09:18:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:58975) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNK-0004ci-90 for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNJ-0001xA-9l for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:02 -0500 Received: from relay.sw.ru ([185.231.240.75]:47458) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001s1-K6; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNC-0006WW-Ir; Fri, 15 Nov 2019 17:14:54 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 20/24] job: call job_enter from job_user_pause Date: Fri, 15 Nov 2019 17:14:40 +0300 Message-Id: <20191115141444.24155-21-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" If main job coroutine called job_yield (while some background process is in progress), we should give it a chance to call job_pause_point(). It will be used in backup, when moved on async block-copy. Signed-off-by: Vladimir Sementsov-Ogievskiy --- job.c | 1 + 1 file changed, 1 insertion(+) diff --git a/job.c b/job.c index 04409b40aa..39c7baa436 100644 --- a/job.c +++ b/job.c @@ -557,6 +557,7 @@ void job_user_pause(Job *job, Error **errp) } job->user_paused =3D true; job_pause(job); + job_enter(job); } =20 bool job_user_paused(Job *job) --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573827964; cv=none; d=zoho.com; s=zohoarc; b=K82EkbedA3SvZRwwiphRGIdxvgYNLUuyMUsxj9pC6e8l+Ppu7AkEjSTlgVltTRl/DIqp8bn2X8eM8Iqv4ySsOFOwISTYwMx5sxFpDiR+hjdmG9+z49i7XRjNqQ5PUWAsDYvNncTKZQCVJ/+GXtktqTrhVfxlVFU34XhTVqJkQpM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573827964; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7DcwWjllKpcMrDBaDSf2ic2fyMbMMznA3g2TM3JMt3k=; b=EQyU6ChMjRR/ZHjTqNQmir2rN4qySmsZptcf/1EK1CqqeawrkiyWTk9iARWNNexH6L4x/AiWfIOH4W5eX4HH1+/dSoIge/G3uTUKCMydj5zDFi9Da/bOssgK7OfSOsHiZyyVyaXw9IVv+oEZ8cMmuBXwQP1fw/oTFhckTrUrWEA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573827964289648.5262851892521; Fri, 15 Nov 2019 06:26:04 -0800 (PST) Received: from localhost ([::1]:39882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcXy-0007Wt-Cu for importer@patchew.org; Fri, 15 Nov 2019 09:26:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59501) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcO9-0005kp-Ts for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:16:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNz-0002U8-Oc for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:53 -0500 Received: from relay.sw.ru ([185.231.240.75]:47554) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNd-0001wS-FB; Fri, 15 Nov 2019 09:15:22 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNC-0006WW-RY; Fri, 15 Nov 2019 17:14:55 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 21/24] backup: move to block-copy Date: Fri, 15 Nov 2019 17:14:41 +0300 Message-Id: <20191115141444.24155-22-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" This brings async request handling and block-status driven chunk sizes to backup out of the box, which improves backup performance. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 9 +- include/block/block_int.h | 7 + block/backup.c | 159 +++++++++++-------- block/replication.c | 1 + blockdev.c | 5 + tests/qemu-iotests/129 | 3 +- tests/qemu-iotests/185 | 3 +- tests/qemu-iotests/219 | 1 + tests/qemu-iotests/257 | 1 + tests/qemu-iotests/257.out | 306 ++++++++++++++++++------------------- 10 files changed, 276 insertions(+), 219 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index aa97ee2641..065098007f 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1455,6 +1455,12 @@ # above node specified by @drive. If this option is not= given, # a node name is autogenerated. (Since: 4.2) # +# @x-max-workers: maximum of parallel requests for static data backup. This +# doesn't influence copy-before-write operations. (Since: = 4.3) +# +# @x-max-chunk: maximum chunk length for static data backup. This doesn't +# influence copy-before-write operations. (Since: 4.3) +# # Note: @on-source-error and @on-target-error only affect background # I/O. If an error occurs during a guest write request, the device's # rerror/werror actions will be used. @@ -1469,7 +1475,8 @@ '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', '*auto-finalize': 'bool', '*auto-dismiss': 'bool', - '*filter-node-name': 'str' } } + '*filter-node-name': 'str', '*x-max-workers': 'int', + '*x-max-chunk': 'int64' } } =20 ## # @DriveBackup: diff --git a/include/block/block_int.h b/include/block/block_int.h index dd033d0b37..55772f1591 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -1193,6 +1193,11 @@ void mirror_start(const char *job_id, BlockDriverSta= te *bs, * @sync_mode: What parts of the disk image should be copied to the destin= ation. * @sync_bitmap: The dirty bitmap if sync_mode is 'bitmap' or 'incremental' * @bitmap_mode: The bitmap synchronization policy to use. + * @max_workers: The limit for parallel requests for main backup loop. + * Must be >=3D 1. + * @max_chunk: The limit for one IO operation length in main backup loop. + * Must be not less than job cluster size or zero. Zero means = no + * specific limit. * @on_source_error: The action to take upon error reading from the source. * @on_target_error: The action to take upon error writing to the target. * @creation_flags: Flags that control the behavior of the Job lifetime. @@ -1211,6 +1216,8 @@ BlockJob *backup_job_create(const char *job_id, Block= DriverState *bs, BitmapSyncMode bitmap_mode, bool compress, const char *filter_node_name, + int max_workers, + int64_t max_chunk, BlockdevOnError on_source_error, BlockdevOnError on_target_error, int creation_flags, diff --git a/block/backup.c b/block/backup.c index acab0d08da..606e4d8725 100644 --- a/block/backup.c +++ b/block/backup.c @@ -44,11 +44,16 @@ typedef struct BackupBlockJob { BlockdevOnError on_source_error; BlockdevOnError on_target_error; uint64_t len; - uint64_t bytes_read; int64_t cluster_size; + int max_workers; + int64_t max_chunk; =20 BlockCopyState *bcs; BdrvDirtyBitmap *bcs_bitmap; + + BlockCopyCallState *bcs_call; + int ret; + bool error_is_read; } BackupBlockJob; =20 static const BlockJobDriver backup_job_driver; @@ -57,7 +62,6 @@ static void backup_progress_bytes_callback(int64_t bytes,= void *opaque) { BackupBlockJob *s =3D opaque; =20 - s->bytes_read +=3D bytes; job_progress_update(&s->common.job, bytes); } =20 @@ -69,25 +73,6 @@ static void backup_progress_reset_callback(void *opaque) job_progress_set_remaining(&s->common.job, estimate); } =20 -static int coroutine_fn backup_do_cow(BackupBlockJob *job, - int64_t offset, uint64_t bytes, - bool *error_is_read) -{ - int ret =3D 0; - int64_t start, end; /* bytes */ - - start =3D QEMU_ALIGN_DOWN(offset, job->cluster_size); - end =3D QEMU_ALIGN_UP(bytes + offset, job->cluster_size); - - trace_backup_do_cow_enter(job, start, offset, bytes); - - ret =3D block_copy(job->bcs, start, end - start, error_is_read); - - trace_backup_do_cow_return(job, offset, bytes, ret); - - return ret; -} - static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) { BdrvDirtyBitmap *bm; @@ -166,54 +151,58 @@ static BlockErrorAction backup_error_action(BackupBlo= ckJob *job, } } =20 -static bool coroutine_fn yield_and_check(BackupBlockJob *job) +static void coroutine_fn backup_block_copy_callback(int ret, bool error_is= _read, + void *opaque) { - uint64_t delay_ns; - - if (job_is_cancelled(&job->common.job)) { - return true; - } - - /* - * We need to yield even for delay_ns =3D 0 so that bdrv_drain_all() c= an - * return. Without a yield, the VM would not reboot. - */ - delay_ns =3D block_job_ratelimit_get_delay(&job->common, job->bytes_re= ad); - job->bytes_read =3D 0; - job_sleep_ns(&job->common.job, delay_ns); - - if (job_is_cancelled(&job->common.job)) { - return true; - } + BackupBlockJob *s =3D opaque; =20 - return false; + s->bcs_call =3D NULL; + s->ret =3D ret; + s->error_is_read =3D error_is_read; + job_enter(&s->common.job); } =20 static int coroutine_fn backup_loop(BackupBlockJob *job) { - bool error_is_read; - int64_t offset; - BdrvDirtyBitmapIter *bdbi; - int ret =3D 0; + while (true) { /* retry loop */ + assert(!job->bcs_call); + job->bcs_call =3D block_copy_async(job->bcs, 0, + QEMU_ALIGN_UP(job->len, + job->cluster_size), + true, job->max_workers, job->max_= chunk, + backup_block_copy_callback); =20 - bdbi =3D bdrv_dirty_iter_new(job->bcs_bitmap); - while ((offset =3D bdrv_dirty_iter_next(bdbi)) !=3D -1) { - do { - if (yield_and_check(job)) { - goto out; + while (job->bcs_call && !job->common.job.cancelled) { + /* wait and handle pauses */ + + job_pause_point(&job->common.job); + + if (job->bcs_call && !job->common.job.cancelled) { + job_yield(&job->common.job); } - ret =3D backup_do_cow(job, offset, job->cluster_size, &error_i= s_read); - if (ret < 0 && backup_error_action(job, error_is_read, -ret) = =3D=3D - BLOCK_ERROR_ACTION_REPORT) - { - goto out; + } + + if (!job->bcs_call && job->ret =3D=3D 0) { + /* Success */ + return 0; + } + + if (job->common.job.cancelled) { + if (job->bcs_call) { + block_copy_cancel(job->bcs_call); } - } while (ret < 0); + return 0; + } + + if (!job->bcs_call && job->ret < 0 && + (backup_error_action(job, job->error_is_read, -job->ret) =3D= =3D + BLOCK_ERROR_ACTION_REPORT)) + { + return job->ret; + } } =20 - out: - bdrv_dirty_iter_free(bdbi); - return ret; + g_assert_not_reached(); } =20 static void backup_init_bcs_bitmap(BackupBlockJob *job) @@ -253,9 +242,14 @@ static int coroutine_fn backup_run(Job *job, Error **e= rrp) int64_t count; =20 for (offset =3D 0; offset < s->len; ) { - if (yield_and_check(s)) { - ret =3D -ECANCELED; - goto out; + if (job_is_cancelled(job)) { + return -ECANCELED; + } + + job_pause_point(job); + + if (job_is_cancelled(job)) { + return -ECANCELED; } =20 ret =3D block_copy_reset_unallocated(s->bcs, offset, &count); @@ -288,6 +282,25 @@ static int coroutine_fn backup_run(Job *job, Error **e= rrp) return ret; } =20 +static void coroutine_fn backup_pause(Job *job) +{ + BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); + + if (s->bcs_call) { + block_copy_cancel(s->bcs_call); + } +} + +static void coroutine_fn backup_set_speed(BlockJob *job, int64_t speed) +{ + BackupBlockJob *s =3D container_of(job, BackupBlockJob, common); + + if (s->bcs) { + /* In block_job_create we yet don't have bcs */ + block_copy_set_speed(s->bcs, s->bcs_call, speed); + } +} + static const BlockJobDriver backup_job_driver =3D { .job_driver =3D { .instance_size =3D sizeof(BackupBlockJob), @@ -298,7 +311,9 @@ static const BlockJobDriver backup_job_driver =3D { .commit =3D backup_commit, .abort =3D backup_abort, .clean =3D backup_clean, - } + .pause =3D backup_pause, + }, + .set_speed =3D backup_set_speed, }; =20 static int64_t backup_calculate_cluster_size(BlockDriverState *target, @@ -343,6 +358,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, BitmapSyncMode bitmap_mode, bool compress, const char *filter_node_name, + int max_workers, + int64_t max_chunk, BlockdevOnError on_source_error, BlockdevOnError on_target_error, int creation_flags, @@ -363,6 +380,16 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, assert(sync_mode !=3D MIRROR_SYNC_MODE_INCREMENTAL); assert(sync_bitmap || sync_mode !=3D MIRROR_SYNC_MODE_BITMAP); =20 + if (max_workers < 1) { + error_setg(errp, "At least one worker needed"); + return NULL; + } + + if (max_chunk < 0) { + error_setg(errp, "max-chunk is negative"); + return NULL; + } + if (bs =3D=3D target) { error_setg(errp, "Source and target cannot be the same"); return NULL; @@ -418,6 +445,11 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, if (cluster_size < 0) { goto error; } + if (max_chunk && max_chunk < cluster_size) { + error_setg(errp, "Required max-chunk (%" PRIi64") is less than bac= kup " + "cluster size (%" PRIi64 ")", max_chunk, cluster_size); + return NULL; + } =20 /* * If source is in backing chain of target assume that target is going= to be @@ -461,9 +493,12 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, job->bcs_bitmap =3D block_copy_dirty_bitmap(bcs); job->cluster_size =3D cluster_size; job->len =3D len; + job->max_workers =3D max_workers; + job->max_chunk =3D max_chunk; =20 block_copy_set_callbacks(bcs, backup_progress_bytes_callback, backup_progress_reset_callback, job); + block_copy_set_speed(bcs, NULL, speed); =20 /* Required permissions are already taken by backup-top target */ block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, diff --git a/block/replication.c b/block/replication.c index 99532ce521..6ca47b616f 100644 --- a/block/replication.c +++ b/block/replication.c @@ -544,6 +544,7 @@ static void replication_start(ReplicationState *rs, Rep= licationMode mode, s->backup_job =3D backup_job_create( NULL, s->secondary_disk->bs, s->hidden_dis= k->bs, 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, = NULL, + 0, 0, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, backup_job_completed, bs, NULL, &local_err= ); diff --git a/blockdev.c b/blockdev.c index 8e029e9c01..8b1dc6782d 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3502,6 +3502,9 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, if (!backup->has_compress) { backup->compress =3D false; } + if (!backup->has_x_max_workers) { + backup->x_max_workers =3D 64; + } =20 ret =3D bdrv_try_set_aio_context(target_bs, aio_context, errp); if (ret < 0) { @@ -3581,6 +3584,8 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, backup->sync, bmap, backup->bitmap_mode, backup->compress, backup->filter_node_name, + backup->x_max_workers, + backup->x_max_chunk, backup->on_source_error, backup->on_target_error, job_flags, NULL, NULL, txn, errp); diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index cd6b9e9ce7..4a4a3760de 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -66,7 +66,6 @@ class TestStopWithBlockJob(iotests.QMPTestCase): result =3D self.vm.qmp("stop") self.assert_qmp(result, 'return', {}) result =3D self.vm.qmp("query-block-jobs") - self.assert_qmp(result, 'return[0]/busy', True) self.assert_qmp(result, 'return[0]/ready', False) =20 def test_drive_mirror(self): @@ -77,7 +76,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def test_drive_backup(self): self.do_test_stop("drive-backup", device=3D"drive0", target=3Dself.target_img, - sync=3D"full") + sync=3D"full", speed=3D1024) =20 def test_block_commit(self): self.do_test_stop("block-commit", device=3D"drive0") diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185 index 454ff600cc..7cd79052d9 100755 --- a/tests/qemu-iotests/185 +++ b/tests/qemu-iotests/185 @@ -182,7 +182,8 @@ _send_qemu_cmd $h \ 'target': '$TEST_IMG.copy', 'format': '$IMGFMT', 'sync': 'full', - 'speed': 65536 } }" \ + 'speed': 65536, + 'x-max-chunk': 65536 } }" \ "return" =20 # If we don't sleep here 'quit' command races with disk I/O diff --git a/tests/qemu-iotests/219 b/tests/qemu-iotests/219 index e0c51662c0..e77ccf1b1b 100755 --- a/tests/qemu-iotests/219 +++ b/tests/qemu-iotests/219 @@ -223,6 +223,7 @@ with iotests.FilePath('disk.img') as disk_path, \ 'target': copy_path, 'sync': 'full', 'speed': 262144, + 'x-max-chunk': 65536, 'auto-finalize': auto_finalize, 'auto-dismiss': auto_dismiss, }) diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257 index a9828251cf..e86bd3d639 100755 --- a/tests/qemu-iotests/257 +++ b/tests/qemu-iotests/257 @@ -191,6 +191,7 @@ def blockdev_backup(vm, device, target, sync, **kwargs): target=3Dtarget, sync=3Dsync, filter_node_name=3D'backup-top', + x_max_workers=3D1, **kwargs) return result =20 diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out index 64dd460055..6997b56567 100644 --- a/tests/qemu-iotests/257.out +++ b/tests/qemu-iotests/257.out @@ -30,7 +30,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -78,7 +78,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -92,7 +92,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backu= p_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backu= p_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -205,7 +205,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -219,7 +219,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backu= p_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backu= p_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -290,7 +290,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -338,7 +338,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -354,7 +354,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backu= p_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backu= p_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"action": "report", "device": "backup_1", "operation": "read"}, = "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds= ": "SECS"}} {"data": {"device": "backup_1", "error": "Input/output error", "len": 3932= 16, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COM= PLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} @@ -416,7 +416,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -430,7 +430,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backu= p_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backu= p_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -501,7 +501,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -549,7 +549,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -563,7 +563,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backu= p_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "backu= p_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -676,7 +676,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -690,7 +690,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backu= p_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "filter-node-na= me": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "backu= p_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -761,7 +761,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -809,7 +809,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -823,7 +823,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "= backup_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "= backup_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -936,7 +936,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -950,7 +950,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -1021,7 +1021,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1069,7 +1069,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1085,7 +1085,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "= backup_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "= backup_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"action": "report", "device": "backup_1", "operation": "read"}, = "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds= ": "SECS"}} {"data": {"device": "backup_1", "error": "Input/output error", "len": 3932= 16, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COM= PLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} @@ -1147,7 +1147,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1161,7 +1161,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -1232,7 +1232,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1280,7 +1280,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1294,7 +1294,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "= backup_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "= backup_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -1407,7 +1407,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1421,7 +1421,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -1492,7 +1492,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1540,7 +1540,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1554,7 +1554,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "back= up_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "back= up_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -1667,7 +1667,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1681,7 +1681,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -1752,7 +1752,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1800,7 +1800,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1816,7 +1816,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "back= up_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "back= up_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"action": "report", "device": "backup_1", "operation": "read"}, = "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds= ": "SECS"}} {"data": {"device": "backup_1", "error": "Input/output error", "len": 3932= 16, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COM= PLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} @@ -1878,7 +1878,7 @@ expecting 13 dirty sectors; have 13. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -1892,7 +1892,7 @@ expecting 13 dirty sectors; have 13. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -1963,7 +1963,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2011,7 +2011,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2025,7 +2025,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "back= up_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "bitmap", "target": "back= up_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -2138,7 +2138,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2152,7 +2152,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -2223,7 +2223,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2271,7 +2271,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2285,7 +2285,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "ba= ckup_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "ba= ckup_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -2398,7 +2398,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2412,7 +2412,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -2483,7 +2483,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2531,7 +2531,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2547,7 +2547,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "ba= ckup_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "ba= ckup_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"action": "report", "device": "backup_1", "operation": "read"}, = "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds= ": "SECS"}} {"data": {"device": "backup_1", "error": "Input/output error", "len": 6710= 8864, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_= COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} @@ -2609,7 +2609,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2623,7 +2623,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -2694,7 +2694,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2742,7 +2742,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2756,7 +2756,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "ba= ckup_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "full", "target": "ba= ckup_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -2869,7 +2869,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -2883,7 +2883,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -2954,7 +2954,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3002,7 +3002,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3016,7 +3016,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup= _target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup= _target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -3129,7 +3129,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3143,7 +3143,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -3214,7 +3214,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3262,7 +3262,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3278,7 +3278,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup= _target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup= _target_1", "x-max-workers": 1}} {"return": {}} {"data": {"action": "report", "device": "backup_1", "operation": "read"}, = "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds= ": "SECS"}} {"data": {"device": "backup_1", "error": "Input/output error", "len": 6710= 8864, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_= COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} @@ -3340,7 +3340,7 @@ expecting 1014 dirty sectors; have 1014. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3354,7 +3354,7 @@ expecting 1014 dirty sectors; have 1014. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -3425,7 +3425,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3473,7 +3473,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3487,7 +3487,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup= _target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "full", "target": "backup= _target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -3600,7 +3600,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3614,7 +3614,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -3685,7 +3685,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3733,7 +3733,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3747,7 +3747,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "bac= kup_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "bac= kup_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -3860,7 +3860,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3874,7 +3874,7 @@ expecting 15 dirty sectors; have 15. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -3945,7 +3945,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -3993,7 +3993,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4009,7 +4009,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "bac= kup_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "bac= kup_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"action": "report", "device": "backup_1", "operation": "read"}, = "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds= ": "SECS"}} {"data": {"device": "backup_1", "error": "Input/output error", "len": 4587= 52, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COM= PLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} @@ -4071,7 +4071,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4085,7 +4085,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -4156,7 +4156,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4204,7 +4204,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4218,7 +4218,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "bac= kup_target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_1", "sync": "top", "target": "bac= kup_target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -4331,7 +4331,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4345,7 +4345,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "filter-no= de-name": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "= backup_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -4416,7 +4416,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4464,7 +4464,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4478,7 +4478,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_= target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_= target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -4591,7 +4591,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4605,7 +4605,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -4676,7 +4676,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4724,7 +4724,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4740,7 +4740,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_= target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_= target_1", "x-max-workers": 1}} {"return": {}} {"data": {"action": "report", "device": "backup_1", "operation": "read"}, = "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds= ": "SECS"}} {"data": {"device": "backup_1", "error": "Input/output error", "len": 4587= 52, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COM= PLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} @@ -4802,7 +4802,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4816,7 +4816,7 @@ expecting 14 dirty sectors; have 14. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -4887,7 +4887,7 @@ write -P0x76 0x3ff0000 0x10000 {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_0", "sync": "full", "target"= : "ref_target_0", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4935,7 +4935,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_1", "sync": "full", "target"= : "ref_target_1", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -4949,7 +4949,7 @@ expecting 6 dirty sectors; have 6. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_= target_1"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_1", "sync": "top", "target": "backup_= target_1", "x-max-workers": 1}} {"return": {}} =20 --- Write #2 --- @@ -5062,7 +5062,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "ref_backup_2", "sync": "full", "target"= : "ref_target_2", "x-max-workers": 1}} {"return": {}} {"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "= speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": = {"microseconds": "USECS", "seconds": "SECS"}} =20 @@ -5076,7 +5076,7 @@ expecting 12 dirty sectors; have 12. OK! {"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}} {"return": {}} {} -{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2"}} +{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitm= ap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "filter-node-n= ame": "backup-top", "job-id": "backup_2", "sync": "bitmap", "target": "back= up_target_2", "x-max-workers": 1}} {"return": {}} {"execute": "job-finalize", "arguments": {"id": "backup_2"}} {"return": {}} @@ -5139,155 +5139,155 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID= -fbackup2" =3D=3D> Identical, OK! =20 -- Sync mode incremental tests -- =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "incremental", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "incremental", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "must provide a valid bitmap n= ame for 'incremental' sync mode"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "incremental", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "incremental", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "must provide a valid bitmap n= ame for 'incremental' sync mode"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "incremental", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "incremental", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "must provide a valid bitmap n= ame for 'incremental' sync mode"}} =20 -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "api_job", "sync": "incremental", "targe= t": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "api_job", "sync": "incremental", "targe= t": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "must provide a valid bitmap n= ame for 'incremental' sync mode"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "incremental", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "incremental", "target": "backup_target", "x= -max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "incremental", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "incremental", "target": "backup_target", "x-max= -workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-= success' when using sync mode 'incremental'"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "incremental", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "incremental", "target": "backup_target", "x-max-= workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-= success' when using sync mode 'incremental'"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "incremental", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "incremental", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job= -id": "api_job", "sync": "incremental", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job= -id": "api_job", "sync": "incremental", "target": "backup_target", "x-max-w= orkers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-= success' when using sync mode 'incremental'"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-= id": "api_job", "sync": "incremental", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-= id": "api_job", "sync": "incremental", "target": "backup_target", "x-max-wo= rkers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-= success' when using sync mode 'incremental'"}} =20 -- Sync mode bitmap tests -- =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "bitmap", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "bitmap", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "must provide a valid bitmap n= ame for 'bitmap' sync mode"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "bitmap", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "bitmap", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "must provide a valid bitmap n= ame for 'bitmap' sync mode"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "bitmap", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "bitmap", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "must provide a valid bitmap n= ame for 'bitmap' sync mode"}} =20 -{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "= backup_target"}} +{"execute": "blockdev-backup", "arguments": {"device": "drive0", "filter-n= ode-name": "backup-top", "job-id": "api_job", "sync": "bitmap", "target": "= backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "must provide a valid bitmap n= ame for 'bitmap' sync mode"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-max-= workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "bitmap", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-max-work= ers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "bitmap", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "bitmap", "target": "backup_target", "x-max-worke= rs": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "bitmap", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "bitmap", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device"= : "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": = "bitmap", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device"= : "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": = "bitmap", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode must be give= n when providing a bitmap"}} =20 -- Sync mode full tests -- =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "full", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "full", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mo= de without a bitmap"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "full", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "full", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mo= de without a bitmap"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "full", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "full", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mo= de without a bitmap"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "full", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "full", "target": "backup_target", "x-max-wo= rkers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "full", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "full", "target": "backup_target", "x-max-worker= s": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "full", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "full", "target": "backup_target", "x-max-workers= ": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "full", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "full", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-= id": "api_job", "sync": "full", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-= id": "api_job", "sync": "full", "target": "backup_target", "x-max-workers":= 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode 'never' has = no meaningful effect when combined with sync mode 'full'"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device"= : "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": = "full", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device"= : "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": = "full", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode must be give= n when providing a bitmap"}} =20 -- Sync mode top tests -- =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "top", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "top", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mo= de without a bitmap"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "top", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "top", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mo= de without a bitmap"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "top", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "top", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mo= de without a bitmap"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "top", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "top", "target": "backup_target", "x-max-wor= kers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "top", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "top", "target": "backup_target", "x-max-workers= ": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "top", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "top", "target": "backup_target", "x-max-workers"= : 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "top", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "top", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-= id": "api_job", "sync": "top", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-= id": "api_job", "sync": "top", "target": "backup_target", "x-max-workers": = 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode 'never' has = no meaningful effect when combined with sync mode 'top'"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device"= : "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": = "top", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device"= : "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": = "top", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode must be give= n when providing a bitmap"}} =20 -- Sync mode none tests -- =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", = "device": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", = "sync": "none", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mo= de without a bitmap"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "dev= ice": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "syn= c": "none", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mo= de without a bitmap"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "devi= ce": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync= ": "none", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Cannot specify bitmap sync mo= de without a bitmap"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "on-success", "device": "drive0", "filter-node-name": "backup-top"= , "job-id": "api_job", "sync": "none", "target": "backup_target", "x-max-wo= rkers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "always", "device": "drive0", "filter-node-name": "backup-top", "j= ob-id": "api_job", "sync": "none", "target": "backup_target", "x-max-worker= s": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitma= p-mode": "never", "device": "drive0", "filter-node-name": "backup-top", "jo= b-id": "api_job", "sync": "none", "target": "backup_target", "x-max-workers= ": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "devic= e": "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync"= : "none", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not = be found"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", = "job-id": "api_job", "sync": "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "on-success", "device": "drive0", "filter-node-name": "backup-top", = "job-id": "api_job", "sync": "none", "target": "backup_target", "x-max-work= ers": 1}} {"error": {"class": "GenericError", "desc": "sync mode 'none' does not pro= duce meaningful bitmap outputs"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job= -id": "api_job", "sync": "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "always", "device": "drive0", "filter-node-name": "backup-top", "job= -id": "api_job", "sync": "none", "target": "backup_target", "x-max-workers"= : 1}} {"error": {"class": "GenericError", "desc": "sync mode 'none' does not pro= duce meaningful bitmap outputs"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-= id": "api_job", "sync": "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-= mode": "never", "device": "drive0", "filter-node-name": "backup-top", "job-= id": "api_job", "sync": "none", "target": "backup_target", "x-max-workers":= 1}} {"error": {"class": "GenericError", "desc": "sync mode 'none' does not pro= duce meaningful bitmap outputs"}} =20 -{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device"= : "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": = "none", "target": "backup_target"}} +{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device"= : "drive0", "filter-node-name": "backup-top", "job-id": "api_job", "sync": = "none", "target": "backup_target", "x-max-workers": 1}} {"error": {"class": "GenericError", "desc": "Bitmap sync mode must be give= n when providing a bitmap"}} =20 --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828818; cv=none; d=zoho.com; s=zohoarc; b=nTYGIlEPbsCpmeY0V7642AW/3vfSLsHOgPsIiNhTgqW93wTzhR+GUj4V34FICrv+s4LK/QMMts9+oQjuwJCSAOk7+zkhVT/0SZELqlAxJL6cIEGR9K2tUs3qsTfEzMC7jg3thMwxyXSlPd1gZQ5tR6TbbJwWGjmNw8CFACEdiXM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828818; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OWGtb3qwivIPKt37Js5cw/vetqxeFSoUaIxrucHGQ58=; b=UWT0PS01NSXpeEcr8DHw68Sd2+h3X5JdO5GE3F8Pyp+aSyWIYkwiaJbHwIDOCmdFC6yaTj+4HZZ1/n0wUf8XVuBPf1AGk4rhOfIR7u83mCUgYQZ6UYgYDAaMP4jRecfTCxNbo7Z4KZ+nuE9kOtcxAJECz5Fiv43NJZbXbyyNxL0= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828818421417.47405024758143; Fri, 15 Nov 2019 06:40:18 -0800 (PST) Received: from localhost ([::1]:40068 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcll-0007A9-9I for importer@patchew.org; Fri, 15 Nov 2019 09:40:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59399) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNo-0005Dg-5T for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNm-0002KQ-Ff for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:31 -0500 Received: from relay.sw.ru ([185.231.240.75]:47560) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNd-0001wW-Hk; Fri, 15 Nov 2019 09:15:21 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcND-0006WW-4C; Fri, 15 Nov 2019 17:14:55 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 22/24] python: add simplebench.py Date: Fri, 15 Nov 2019 17:14:42 +0300 Message-Id: <20191115141444.24155-23-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Add simple benchmark table creator. Signed-off-by: Vladimir Sementsov-Ogievskiy --- python/simplebench.py | 122 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 python/simplebench.py diff --git a/python/simplebench.py b/python/simplebench.py new file mode 100644 index 0000000000..b09c2cf40b --- /dev/null +++ b/python/simplebench.py @@ -0,0 +1,122 @@ +#!/usr/bin/env python +# +# Simple benchmarking framework +# +# Copyright (c) 2019 Virtuozzo International GmbH. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + + +def bench_one(test_func, test_env, test_case, count=3D5, initial_run=3DTru= e): + """Benchmark one test-case + + test_func -- benchmarking function with prototype + test_func(env, case), which takes test_env and test_case + arguments and returns {'seconds': int} (which is benchm= ark + result) on success and {'error': str} on error. Returned + dict may contain any other additional fields. + test_env -- test environment - opaque first argument for test_func + test_case -- test case - opaque second argument for test_func + count -- how many times to call test_func, to calculate average + initial_run -- do initial run of test_func, which don't get into result + + Returns dict with the following fields: + 'runs': list of test_func results + 'average': average seconds per run (exists only if at least one r= un + succeeded) + 'delta': maximum delta between test_func result and the average + (exists only if at least one run succeeded) + 'n-failed': number of failed runs (exists only if at least one run + failed) + """ + if initial_run: + test_func(test_env, test_case) + + runs =3D [test_func(test_env, test_case) for _ in range(count)] + + result =3D {'runs': runs} + + successed =3D [r for r in runs if ('seconds' in r)] + if successed: + avg =3D sum(r['seconds'] for r in successed) / len(successed) + result['average'] =3D avg + result['delta'] =3D max(abs(r['seconds'] - avg) for r in successed) + + if len(successed) < count: + result['n-failed'] =3D count - len(successed) + + return result + + +def ascii_one(result): + """Return ASCII representation of bench_one() returned dict.""" + if 'average' in result: + s =3D '{:.2f} +- {:.2f}'.format(result['average'], result['delta']) + if 'n-failed' in result: + s +=3D '\n({} failed)'.format(result['n-failed']) + return s + else: + return 'FAILED' + + +def bench(test_func, test_envs, test_cases, *args, **vargs): + """Fill benchmark table + + test_func -- benchmarking function, see bench_one for description + test_envs -- list of test environments, see bench_one + test_cases -- list of test cases, see bench_one + args, vargs -- additional arguments for bench_one + + Returns dict with the following fields: + 'envs': test_envs + 'cases': test_cases + 'tab': filled 2D array, where cell [i][j] is bench_one result for + test_cases[i] for test_envs[j] (i.e., rows are test cases= and + columns are test environments) + """ + tab =3D {} + results =3D { + 'envs': test_envs, + 'cases': test_cases, + 'tab': tab + } + n =3D 1 + n_tests =3D len(test_envs) * len(test_cases) + for env in test_envs: + for case in test_cases: + print('Testing {}/{}: {} :: {}'.format(n, n_tests, + env['id'], case['id'])) + if case['id'] not in tab: + tab[case['id']] =3D {} + tab[case['id']][env['id']] =3D bench_one(test_func, env, case, + *args, **vargs) + n +=3D 1 + + print('Done') + return results + + +def ascii(results): + """Return ASCII representation of bench() returned dict.""" + from tabulate import tabulate + + tab =3D [[""] + [c['id'] for c in results['envs']]] + for case in results['cases']: + row =3D [case['id']] + for env in results['envs']: + row.append(ascii_one(results['tab'][case['id']][env['id']])) + tab.append(row) + + return tabulate(tab) --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828398; cv=none; d=zoho.com; s=zohoarc; b=HSOh8oID+p+Kd429stwyplHdFOysCedJRPIWR5x57cBT9sB933V4px+jtDorLMqMofzl/56Dq7lLYE75Ox6in3Ew+M6IUZffdm8EqHfTfpvjB7axMclifLCbXEuctZ84vXUE004ibtNaqcYTQtQqKOd8LdT18QXPM06cCtNRf6k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828398; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=WIMBsaWMXQSb+hV882q/LB92YjjyG0rnRDAyebfhH1A=; b=DB8pYjpdp7vDjtpncsgkslh/2GkzVfzN++zQ25amN7bOWMLb0I6nGCnCjqslawaDoTu0YwyyhpVvxTFULB9hes7jinYho+QHK7DpufazKhqEty0WZgBMR5k9Fp1+ABQcJnRVAxnZtazyX5GIXcOEuhmlEJVnjE4bROEoCKZuLqs= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1573828398050224.4380573976082; Fri, 15 Nov 2019 06:33:18 -0800 (PST) Received: from localhost ([::1]:39974 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcew-00078L-Cs for importer@patchew.org; Fri, 15 Nov 2019 09:33:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59041) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNL-0004eR-Jx for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNK-0001yK-2E for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:03 -0500 Received: from relay.sw.ru ([185.231.240.75]:47464) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNG-0001rt-K4; Fri, 15 Nov 2019 09:14:58 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcND-0006WW-8I; Fri, 15 Nov 2019 17:14:55 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 23/24] python: add qemu/bench_block_job.py Date: Fri, 15 Nov 2019 17:14:43 +0300 Message-Id: <20191115141444.24155-24-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Add block-job benchmarking helper functions. Signed-off-by: Vladimir Sementsov-Ogievskiy --- python/qemu/bench_block_job.py | 114 +++++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100755 python/qemu/bench_block_job.py diff --git a/python/qemu/bench_block_job.py b/python/qemu/bench_block_job.py new file mode 100755 index 0000000000..6a72be9ba2 --- /dev/null +++ b/python/qemu/bench_block_job.py @@ -0,0 +1,114 @@ +#!/usr/bin/env python +# +# Benchmark block jobs +# +# Copyright (c) 2019 Virtuozzo International GmbH. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + + +import socket +import json +from .machine import QEMUMachine + + +def bench_block_job(cmd, cmd_args, qemu_args): + """Benchmark block-job + + cmd -- qmp command to run block-job (like blockdev-backup) + cmd_args -- dict of qmp command arguments + qemu_args -- list of Qemu command line arguments, including path to Qe= mu + binary + + Returns {'seconds': int} on success and {'error': str} on failure, dic= t may + contain addional 'vm-log' field. Return value is compatible with + simplebench lib. + """ + + vm =3D QEMUMachine(qemu_args[0], args=3Dqemu_args[1:]) + + try: + vm.launch() + except OSError as e: + return {'error': 'popen failed: ' + str(e)} + except socket.timeout: + return {'error': 'qemu failed: ' + str(vm.get_log())} + + try: + res =3D vm.qmp(cmd, **cmd_args) + if res !=3D {'return': {}}: + vm.shutdown() + return {'error': '"{}" command failed: {}'.format(cmd, str(res= ))} + + e =3D vm.event_wait('JOB_STATUS_CHANGE') + assert e['data']['status'] =3D=3D 'created' + start_ms =3D e['timestamp']['seconds'] * 1000000 + \ + e['timestamp']['microseconds'] + + e =3D vm.events_wait((('BLOCK_JOB_READY', None), + ('BLOCK_JOB_COMPLETED', None), + ('BLOCK_JOB_FAILED', None)), timeout=3DTrue) + if e['event'] not in ('BLOCK_JOB_READY', 'BLOCK_JOB_COMPLETED'): + vm.shutdown() + return {'error': 'block-job failed: ' + str(e), + 'vm-log': vm.get_log()} + end_ms =3D e['timestamp']['seconds'] * 1000000 + \ + e['timestamp']['microseconds'] + finally: + vm.shutdown() + + return {'seconds': (end_ms - start_ms) / 1000000.0} + + +# Bench backup or mirror +def bench_block_copy(qemu_binary, cmd, source, target): + """Helper to run bench_block_job() for mirror or backup""" + assert cmd in ('blockdev-backup', 'blockdev-mirror') + + source['node-name'] =3D 'source' + target['node-name'] =3D 'target' + + return bench_block_job(cmd, + {'job-id': 'job0', 'device': 'source', + 'target': 'target', 'sync': 'full'}, + [qemu_binary, + '-blockdev', json.dumps(source), + '-blockdev', json.dumps(target)]) + + +def drv_file(filename): + return {'driver': 'file', 'filename': filename, + 'cache': {'direct': True}, 'aio': 'native'} + + +def drv_nbd(host, port): + return {'driver': 'nbd', + 'server': {'type': 'inet', 'host': host, 'port': port}} + + +if __name__ =3D=3D '__main__': + import sys + + if len(sys.argv) < 4: + print('USAGE: {} ' + ' ' + ''.format(sys.argv[0])) + exit(1) + + res =3D bench_block_job(sys.argv[1], json.loads(sys.argv[2]), sys.argv= [3:]) + if 'seconds' in res: + print('{:.2f}'.format(res['seconds'])) + else: + print(res) --=20 2.21.0 From nobody Thu May 2 01:42:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1573828693; cv=none; d=zoho.com; s=zohoarc; b=BIsGSo5xmuAsj0f+1Ag03oJyAamOQU4mf2DHZsZWRZUh/267g142rRO/UwzvpU4YrOsjqw9HYq35EDIJAeZajzOyZlKkHnTUlbQ8baDzjZ8IvE0sctliZBdttB+t32TNt9vZ783yVcpiY93dj2n2CQlgefXst8vg5i+uRD5cmDc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573828693; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FZnvfYecUP1u1mwI/ggcmQEvpOcSSiEjdOJ3BqMX2jE=; b=bOBZ5RxPb/2QtNjNr7KITe9QrI4mJZHzUBdnBp5ZmyvQkZaBifK1bwZvsRnLgQz5XEbMvSWJaaZHeAjyGUAqUV8vgAw+5ijDTmXjx6imCKhr22T7eb9bDN0HqOT1njesLudJTfI8ikd8gEx2F2OdVjcLKWUsXNXTA0Bs5FU5x+4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157382869334818.89846631702858; Fri, 15 Nov 2019 06:38:13 -0800 (PST) Received: from localhost ([::1]:40042 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcjj-0004mq-Ou for importer@patchew.org; Fri, 15 Nov 2019 09:38:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59400) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iVcNo-0005Dq-8w for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iVcNm-0002LC-Sf for qemu-devel@nongnu.org; Fri, 15 Nov 2019 09:15:32 -0500 Received: from relay.sw.ru ([185.231.240.75]:47574) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iVcNd-0001wi-Ox; Fri, 15 Nov 2019 09:15:21 -0500 Received: from vovaso.qa.sw.ru ([10.94.3.0] helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1iVcNI-0006WW-CQ; Fri, 15 Nov 2019 17:15:00 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [RFC 24/24] python: benchmark new backup architecture Date: Fri, 15 Nov 2019 17:14:44 +0300 Message-Id: <20191115141444.24155-25-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com> References: <20191115141444.24155-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com, crosa@redhat.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Benchmark test for the series. This patch is RFC, it would be strange to commit it as is.. On the other hand I feel that we should commit some example to show usage of simplebench and bench_block_job. May be I should add some simple example to compare backup and mirror.. Any ideas? Anyway, this patch is here to show, how I produced the table in the series cover-letter. Note that all used images are 1000M files filled by ones, including images exported by two running NBD servers, and NBD servers are running as: qemu-nbd --persistent --nocache -p 10810 ones1000M-source qemu-nbd --persistent --nocache -p 10809 ones1000M-target Signed-off-by: Vladimir Sementsov-Ogievskiy --- python/bench-example.py | 93 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100755 python/bench-example.py diff --git a/python/bench-example.py b/python/bench-example.py new file mode 100755 index 0000000000..fb4c61c968 --- /dev/null +++ b/python/bench-example.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python3 + +import simplebench +from qemu.bench_block_job import bench_block_copy, drv_file, drv_nbd + + +def bench_func(env, case): + return bench_block_copy(env['qemu_binary'], env['cmd'], + case['source'], case['target']) + + +test_cases =3D [ + { + 'id': 'ssd -> ssd', + 'source': drv_file('/ssd/ones1000M-source'), + 'target': drv_file('/ssd/ones1000M-target') + }, + { + 'id': 'ssd -> hdd', + 'source': drv_file('/ssd/ones1000M-source'), + 'target': drv_file('/test-a/ones1000M-target') + }, + { + 'id': 'hdd -> hdd', + 'source': drv_file('/test-a/ones1000M-source'), + 'target': drv_file('/test-a/ones1000M-target') + } +] + +test_envs =3D [ + { + 'id': 'backup-old', + 'cmd': 'blockdev-backup', + 'qemu_binary': '/work/src/qemu/up-block-copy-block-status--before/= x86_64-softmmu/qemu-system-x86_64' + }, + { + 'id': 'backup-old(no CR)', + 'cmd': 'blockdev-backup', + 'qemu_binary': '/work/src/qemu/up-block-copy-block-status--before-= -no-copy-range/x86_64-softmmu/qemu-system-x86_64' + }, + { + 'id': 'backup-new', + 'cmd': 'blockdev-backup', + 'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-s= oftmmu/qemu-system-x86_64' + }, + { + 'id': 'backup-new(no CR)', + 'cmd': 'blockdev-backup', + 'qemu_binary': '/work/src/qemu/up-block-copy-block-status--no-copy= _range/x86_64-softmmu/qemu-system-x86_64' + }, + { + 'id': 'mirror', + 'cmd': 'blockdev-mirror', + 'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-s= oftmmu/qemu-system-x86_64' + } +] + +result =3D simplebench.bench(bench_func, test_envs, test_cases, count=3D3) +print(simplebench.ascii(result)) + +test_cases =3D [ + { + 'id': 'nbd -> ssd', + 'source': drv_nbd('172.16.24.200', '10810'), + 'target': drv_file('/ssd/ones1000M-target') + }, + { + 'id': 'ssd -> nbd', + 'source': drv_file('/ssd/ones1000M-target'), + 'target': drv_nbd('172.16.24.200', '10809') + }, +] + +test_envs =3D [ + { + 'id': 'backup-old', + 'cmd': 'blockdev-backup', + 'qemu_binary': '/work/src/qemu/up-block-copy-block-status--before/= x86_64-softmmu/qemu-system-x86_64' + }, + { + 'id': 'backup-new', + 'cmd': 'blockdev-backup', + 'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-s= oftmmu/qemu-system-x86_64' + }, + { + 'id': 'mirror', + 'cmd': 'blockdev-mirror', + 'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-s= oftmmu/qemu-system-x86_64' + } +] + +result =3D simplebench.bench(bench_func, test_envs, test_cases, count=3D2) +print(simplebench.ascii(result)) --=20 2.21.0