From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505326905021294.5823299246513; Wed, 13 Sep 2017 11:21:45 -0700 (PDT) Received: from localhost ([::1]:43992 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCIA-0000SP-39 for importer@patchew.org; Wed, 13 Sep 2017 14:21:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36390) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCG3-0007ah-P7 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:19:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCG2-0004bc-Ky for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:19:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50038) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCG0-0004a2-BN; Wed, 13 Sep 2017 14:19:28 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 78A5E81DE4; Wed, 13 Sep 2017 18:19:27 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5FD0C5D724; Wed, 13 Sep 2017 18:19:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 78A5E81DE4 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:18:53 +0200 Message-Id: <20170913181910.29688-2-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 13 Sep 2017 18:19:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 01/18] block: Add BdrvDeletedStatus X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Sometimes an operation may delete a BDS. It may then not be trivial to determine this because the BDS object itself cannot be accessed afterwards. With this patch, one can attach a BdrvDeletedStatus object to a BDS which can be used to safely query whether the BDS still exists even after it has been deleted. Signed-off-by: Max Reitz --- include/block/block_int.h | 12 ++++++++++++ block.c | 6 ++++++ 2 files changed, 18 insertions(+) diff --git a/include/block/block_int.h b/include/block/block_int.h index ba4c383393..eaeaad9428 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -498,6 +498,13 @@ typedef struct BdrvAioNotifier { QLIST_ENTRY(BdrvAioNotifier) list; } BdrvAioNotifier; =20 +typedef struct BdrvDeletedStatus { + /* Set to true by bdrv_delete() */ + bool deleted; + + QLIST_ENTRY(BdrvDeletedStatus) next; +} BdrvDeletedStatus; + struct BdrvChildRole { /* If true, bdrv_replace_node() doesn't change the node this BdrvChild * points to. */ @@ -706,6 +713,11 @@ struct BlockDriverState { =20 /* Only read/written by whoever has set active_flush_req to true. */ unsigned int flushed_gen; /* Flushed write generation */ + + /* When bdrv_delete() is invoked, it will walk through this list + * and set every entry's @deleted field to true. The entries will + * not be freed automatically. */ + QLIST_HEAD(, BdrvDeletedStatus) deleted_status; }; =20 struct BlockBackendRootState { diff --git a/block.c b/block.c index 6dd47e414e..0b55c5a41c 100644 --- a/block.c +++ b/block.c @@ -3246,10 +3246,16 @@ out: =20 static void bdrv_delete(BlockDriverState *bs) { + BdrvDeletedStatus *del_stat; + assert(!bs->job); assert(bdrv_op_blocker_is_empty(bs)); assert(!bs->refcnt); =20 + QLIST_FOREACH(del_stat, &bs->deleted_status, next) { + del_stat->deleted =3D true; + } + bdrv_close(bs); =20 /* remove from list, if necessary */ --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505326925549989.1388484064604; Wed, 13 Sep 2017 11:22:05 -0700 (PDT) Received: from localhost ([::1]:43994 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCIW-0000he-KS for importer@patchew.org; Wed, 13 Sep 2017 14:22:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36470) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCGE-0007jD-BK for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:19:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCGD-0004ia-3M for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:19:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47724) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCGA-0004gQ-CI; Wed, 13 Sep 2017 14:19:38 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7332CC04B924; Wed, 13 Sep 2017 18:19:37 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DB3BC60BE4; Wed, 13 Sep 2017 18:19:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 7332CC04B924 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:18:54 +0200 Message-Id: <20170913181910.29688-3-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 13 Sep 2017 18:19:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 02/18] block: BDS deletion during bdrv_drain_recurse X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Drainined a BDS child may lead to both the original BDS and/or its other children being deleted (e.g. if the original BDS represents a block job). We should prepare for this in both bdrv_drain_recurse() and bdrv_drained_begin() by monitoring whether the BDS we are about to drain still exists at all. Signed-off-by: Max Reitz --- block/io.c | 72 +++++++++++++++++++++++++++++++++++++++++++++-------------= ---- 1 file changed, 52 insertions(+), 20 deletions(-) diff --git a/block/io.c b/block/io.c index 4378ae4c7d..8ec1a564ad 100644 --- a/block/io.c +++ b/block/io.c @@ -182,33 +182,57 @@ static void bdrv_drain_invoke(BlockDriverState *bs) =20 static bool bdrv_drain_recurse(BlockDriverState *bs) { - BdrvChild *child, *tmp; + BdrvChild *child; bool waited; + struct BDSToDrain { + BlockDriverState *bs; + BdrvDeletedStatus del_stat; + QLIST_ENTRY(BDSToDrain) next; + }; + QLIST_HEAD(, BDSToDrain) bs_list =3D QLIST_HEAD_INITIALIZER(bs_list); + bool in_main_loop =3D + qemu_get_current_aio_context() =3D=3D qemu_get_aio_context(); =20 waited =3D BDRV_POLL_WHILE(bs, atomic_read(&bs->in_flight) > 0); =20 /* Ensure any pending metadata writes are submitted to bs->file. */ bdrv_drain_invoke(bs); =20 - QLIST_FOREACH_SAFE(child, &bs->children, next, tmp) { - BlockDriverState *bs =3D child->bs; - bool in_main_loop =3D - qemu_get_current_aio_context() =3D=3D qemu_get_aio_context(); - assert(bs->refcnt > 0); - if (in_main_loop) { - /* In case the recursive bdrv_drain_recurse processes a - * block_job_defer_to_main_loop BH and modifies the graph, - * let's hold a reference to bs until we are done. - * - * IOThread doesn't have such a BH, and it is not safe to call - * bdrv_unref without BQL, so skip doing it there. - */ - bdrv_ref(bs); - } - waited |=3D bdrv_drain_recurse(bs); - if (in_main_loop) { - bdrv_unref(bs); + /* Draining children may result in other children being removed and ma= ybe + * even deleted, so copy the children list first */ + QLIST_FOREACH(child, &bs->children, next) { + struct BDSToDrain *bs2d =3D g_new0(struct BDSToDrain, 1); + + bs2d->bs =3D child->bs; + QLIST_INSERT_HEAD(&bs->deleted_status, &bs2d->del_stat, next); + + QLIST_INSERT_HEAD(&bs_list, bs2d, next); + } + + while (!QLIST_EMPTY(&bs_list)) { + struct BDSToDrain *bs2d =3D QLIST_FIRST(&bs_list); + QLIST_REMOVE(bs2d, next); + + if (!bs2d->del_stat.deleted) { + QLIST_REMOVE(&bs2d->del_stat, next); + + if (in_main_loop) { + /* In case the recursive bdrv_drain_recurse processes a + * block_job_defer_to_main_loop BH and modifies the graph, + * let's hold a reference to the BDS until we are done. + * + * IOThread doesn't have such a BH, and it is not safe to = call + * bdrv_unref without BQL, so skip doing it there. + */ + bdrv_ref(bs2d->bs); + } + waited |=3D bdrv_drain_recurse(bs2d->bs); + if (in_main_loop) { + bdrv_unref(bs2d->bs); + } } + + g_free(bs2d); } =20 return waited; @@ -252,17 +276,25 @@ static void coroutine_fn bdrv_co_yield_to_drain(Block= DriverState *bs) =20 void bdrv_drained_begin(BlockDriverState *bs) { + BdrvDeletedStatus del_stat =3D { .deleted =3D false }; + if (qemu_in_coroutine()) { bdrv_co_yield_to_drain(bs); return; } =20 + QLIST_INSERT_HEAD(&bs->deleted_status, &del_stat, next); + if (atomic_fetch_inc(&bs->quiesce_counter) =3D=3D 0) { aio_disable_external(bdrv_get_aio_context(bs)); bdrv_parent_drained_begin(bs); } =20 - bdrv_drain_recurse(bs); + if (!del_stat.deleted) { + QLIST_REMOVE(&del_stat, next); + + bdrv_drain_recurse(bs); + } } =20 void bdrv_drained_end(BlockDriverState *bs) --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150532692898160.702104888948156; Wed, 13 Sep 2017 11:22:08 -0700 (PDT) Received: from localhost ([::1]:43995 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCIa-0000om-4j for importer@patchew.org; Wed, 13 Sep 2017 14:22:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36534) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCGN-0007ou-3r for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:19:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCGM-0004nc-5t for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:19:51 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39198) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCGJ-0004m2-S1; Wed, 13 Sep 2017 14:19:48 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9CB0F356D8; Wed, 13 Sep 2017 18:19:46 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 88A4C5C545; Wed, 13 Sep 2017 18:19:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 9CB0F356D8 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:18:55 +0200 Message-Id: <20170913181910.29688-4-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 13 Sep 2017 18:19:46 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 03/18] blockjob: Make drained_{begin, end} public X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When a block job decides to be represented as a BDS and track its associated child nodes itself instead of having the BlockJob object track them, it needs to implement the drained_begin/drained_end child operations. In order to do that, it has to be able to control drainage of the block job (i.e. to pause and resume it). Therefore, we need to make these operations public. Signed-off-by: Max Reitz Reviewed-by: Fam Zheng --- include/block/blockjob.h | 15 +++++++++++++++ blockjob.c | 20 ++++++++++++++------ 2 files changed, 29 insertions(+), 6 deletions(-) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 67c0968fa5..a59f316788 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -339,6 +339,21 @@ void block_job_ref(BlockJob *job); void block_job_unref(BlockJob *job); =20 /** + * block_job_drained_begin: + * + * Inhibit I/O requests initiated by the block job. + */ +void block_job_drained_begin(BlockJob *job); + +/** + * block_job_drained_end: + * + * Resume I/O after it has been paused through + * block_job_drained_begin(). + */ +void block_job_drained_end(BlockJob *job); + +/** * block_job_txn_unref: * * Release a reference that was previously acquired with block_job_txn_add= _job diff --git a/blockjob.c b/blockjob.c index 3a0c49137e..4312a121fa 100644 --- a/blockjob.c +++ b/blockjob.c @@ -217,21 +217,29 @@ static const BdrvChildRole child_job =3D { .stay_at_node =3D true, }; =20 -static void block_job_drained_begin(void *opaque) +void block_job_drained_begin(BlockJob *job) { - BlockJob *job =3D opaque; block_job_pause(job); } =20 -static void block_job_drained_end(void *opaque) +static void block_job_drained_begin_op(void *opaque) +{ + block_job_drained_begin(opaque); +} + +void block_job_drained_end(BlockJob *job) { - BlockJob *job =3D opaque; block_job_resume(job); } =20 +static void block_job_drained_end_op(void *opaque) +{ + block_job_drained_end(opaque); +} + static const BlockDevOps block_job_dev_ops =3D { - .drained_begin =3D block_job_drained_begin, - .drained_end =3D block_job_drained_end, + .drained_begin =3D block_job_drained_begin_op, + .drained_end =3D block_job_drained_end_op, }; =20 void block_job_remove_all_bdrv(BlockJob *job) --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327090990656.7551759555631; Wed, 13 Sep 2017 11:24:50 -0700 (PDT) Received: from localhost ([::1]:44014 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCLB-000314-88 for importer@patchew.org; Wed, 13 Sep 2017 14:24:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36583) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCGb-0007zB-Js for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCGa-0004wU-He for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:05 -0400 Received: from mx1.redhat.com ([209.132.183.28]:18800) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCGT-0004qQ-Q9; Wed, 13 Sep 2017 14:19:58 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D2C1D5D689; Wed, 13 Sep 2017 18:19:56 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B3CE960BE4; Wed, 13 Sep 2017 18:19:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D2C1D5D689 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:18:56 +0200 Message-Id: <20170913181910.29688-5-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 13 Sep 2017 18:19:56 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 04/18] block/mirror: Pull out mirror_perform() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When converting mirror's I/O to coroutines, we are going to need a point where these coroutines are created. mirror_perform() is going to be that point. Signed-off-by: Max Reitz Reviewed-by: Fam Zheng Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/mirror.c | 53 ++++++++++++++++++++++++++++++----------------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 6531652d73..4664b0516f 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -82,6 +82,12 @@ typedef struct MirrorOp { uint64_t bytes; } MirrorOp; =20 +typedef enum MirrorMethod { + MIRROR_METHOD_COPY, + MIRROR_METHOD_ZERO, + MIRROR_METHOD_DISCARD, +} MirrorMethod; + static BlockErrorAction mirror_error_action(MirrorBlockJob *s, bool read, int error) { @@ -324,6 +330,22 @@ static void mirror_do_zero_or_discard(MirrorBlockJob *= s, } } =20 +static unsigned mirror_perform(MirrorBlockJob *s, int64_t offset, + unsigned bytes, MirrorMethod mirror_method) +{ + switch (mirror_method) { + case MIRROR_METHOD_COPY: + return mirror_do_read(s, offset, bytes); + case MIRROR_METHOD_ZERO: + case MIRROR_METHOD_DISCARD: + mirror_do_zero_or_discard(s, offset, bytes, + mirror_method =3D=3D MIRROR_METHOD_DISCA= RD); + return bytes; + default: + abort(); + } +} + static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) { BlockDriverState *source =3D s->source; @@ -395,11 +417,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBl= ockJob *s) unsigned int io_bytes; int64_t io_bytes_acct; BlockDriverState *file; - enum MirrorMethod { - MIRROR_METHOD_COPY, - MIRROR_METHOD_ZERO, - MIRROR_METHOD_DISCARD - } mirror_method =3D MIRROR_METHOD_COPY; + MirrorMethod mirror_method =3D MIRROR_METHOD_COPY; =20 assert(!(offset % s->granularity)); ret =3D bdrv_get_block_status_above(source, NULL, @@ -439,22 +457,11 @@ static uint64_t coroutine_fn mirror_iteration(MirrorB= lockJob *s) } =20 io_bytes =3D mirror_clip_bytes(s, offset, io_bytes); - switch (mirror_method) { - case MIRROR_METHOD_COPY: - io_bytes =3D io_bytes_acct =3D mirror_do_read(s, offset, io_by= tes); - break; - case MIRROR_METHOD_ZERO: - case MIRROR_METHOD_DISCARD: - mirror_do_zero_or_discard(s, offset, io_bytes, - mirror_method =3D=3D MIRROR_METHOD_D= ISCARD); - if (write_zeroes_ok) { - io_bytes_acct =3D 0; - } else { - io_bytes_acct =3D io_bytes; - } - break; - default: - abort(); + io_bytes =3D mirror_perform(s, offset, io_bytes, mirror_method); + if (mirror_method !=3D MIRROR_METHOD_COPY && write_zeroes_ok) { + io_bytes_acct =3D 0; + } else { + io_bytes_acct =3D io_bytes; } assert(io_bytes); offset +=3D io_bytes; @@ -650,8 +657,8 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) continue; } =20 - mirror_do_zero_or_discard(s, sector_num * BDRV_SECTOR_SIZE, - nb_sectors * BDRV_SECTOR_SIZE, false= ); + mirror_perform(s, sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE, MIRROR_METHOD_ZE= RO); sector_num +=3D nb_sectors; } =20 --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327257934525.1340249415672; Wed, 13 Sep 2017 11:27:37 -0700 (PDT) Received: from localhost ([::1]:44031 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCNs-0005g5-TY for importer@patchew.org; Wed, 13 Sep 2017 14:27:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36676) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCGn-0008Do-IJ for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCGm-00057n-7y for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58496) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCGf-0004yt-1r; Wed, 13 Sep 2017 14:20:09 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1E1435D689; Wed, 13 Sep 2017 18:20:08 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 130495D6AE; Wed, 13 Sep 2017 18:19:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1E1435D689 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:18:57 +0200 Message-Id: <20170913181910.29688-6-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 13 Sep 2017 18:20:08 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 05/18] block/mirror: Convert to coroutines X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" In order to talk to the source BDS (and maybe in the future to the target BDS as well) directly, we need to convert our existing AIO requests into coroutine I/O requests. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf --- block/mirror.c | 134 +++++++++++++++++++++++++++++++++--------------------= ---- 1 file changed, 78 insertions(+), 56 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 4664b0516f..2b3297aa61 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -80,6 +80,9 @@ typedef struct MirrorOp { QEMUIOVector qiov; int64_t offset; uint64_t bytes; + + /* Set by mirror_co_read() before yielding for the first time */ + uint64_t bytes_copied; } MirrorOp; =20 typedef enum MirrorMethod { @@ -101,7 +104,7 @@ static BlockErrorAction mirror_error_action(MirrorBlock= Job *s, bool read, } } =20 -static void mirror_iteration_done(MirrorOp *op, int ret) +static void coroutine_fn mirror_iteration_done(MirrorOp *op, int ret) { MirrorBlockJob *s =3D op->s; struct iovec *iov; @@ -138,9 +141,8 @@ static void mirror_iteration_done(MirrorOp *op, int ret) } } =20 -static void mirror_write_complete(void *opaque, int ret) +static void coroutine_fn mirror_write_complete(MirrorOp *op, int ret) { - MirrorOp *op =3D opaque; MirrorBlockJob *s =3D op->s; =20 aio_context_acquire(blk_get_aio_context(s->common.blk)); @@ -158,9 +160,8 @@ static void mirror_write_complete(void *opaque, int ret) aio_context_release(blk_get_aio_context(s->common.blk)); } =20 -static void mirror_read_complete(void *opaque, int ret) +static void coroutine_fn mirror_read_complete(MirrorOp *op, int ret) { - MirrorOp *op =3D opaque; MirrorBlockJob *s =3D op->s; =20 aio_context_acquire(blk_get_aio_context(s->common.blk)); @@ -176,8 +177,11 @@ static void mirror_read_complete(void *opaque, int ret) =20 mirror_iteration_done(op, ret); } else { - blk_aio_pwritev(s->target, op->offset, &op->qiov, - 0, mirror_write_complete, op); + int ret; + + ret =3D blk_co_pwritev(s->target, op->offset, + op->qiov.size, &op->qiov, 0); + mirror_write_complete(op, ret); } aio_context_release(blk_get_aio_context(s->common.blk)); } @@ -242,53 +246,49 @@ static inline void mirror_wait_for_io(MirrorBlockJob = *s) * (new_end - offset) if tail is rounded up or down due to * alignment or buffer limit. */ -static uint64_t mirror_do_read(MirrorBlockJob *s, int64_t offset, - uint64_t bytes) +static void coroutine_fn mirror_co_read(void *opaque) { + MirrorOp *op =3D opaque; + MirrorBlockJob *s =3D op->s; BlockBackend *source =3D s->common.blk; int nb_chunks; uint64_t ret; - MirrorOp *op; uint64_t max_bytes; =20 max_bytes =3D s->granularity * s->max_iov; =20 /* We can only handle as much as buf_size at a time. */ - bytes =3D MIN(s->buf_size, MIN(max_bytes, bytes)); - assert(bytes); - assert(bytes < BDRV_REQUEST_MAX_BYTES); - ret =3D bytes; + op->bytes =3D MIN(s->buf_size, MIN(max_bytes, op->bytes)); + assert(op->bytes); + assert(op->bytes < BDRV_REQUEST_MAX_BYTES); + op->bytes_copied =3D op->bytes; =20 if (s->cow_bitmap) { - ret +=3D mirror_cow_align(s, &offset, &bytes); + op->bytes_copied +=3D mirror_cow_align(s, &op->offset, &op->bytes); } - assert(bytes <=3D s->buf_size); + /* Cannot exceed BDRV_REQUEST_MAX_BYTES + INT_MAX */ + assert(op->bytes_copied <=3D UINT_MAX); + assert(op->bytes <=3D s->buf_size); /* The offset is granularity-aligned because: * 1) Caller passes in aligned values; * 2) mirror_cow_align is used only when target cluster is larger. */ - assert(QEMU_IS_ALIGNED(offset, s->granularity)); + assert(QEMU_IS_ALIGNED(op->offset, s->granularity)); /* The range is sector-aligned, since bdrv_getlength() rounds up. */ - assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE)); - nb_chunks =3D DIV_ROUND_UP(bytes, s->granularity); + assert(QEMU_IS_ALIGNED(op->bytes, BDRV_SECTOR_SIZE)); + nb_chunks =3D DIV_ROUND_UP(op->bytes, s->granularity); =20 while (s->buf_free_count < nb_chunks) { - trace_mirror_yield_in_flight(s, offset, s->in_flight); + trace_mirror_yield_in_flight(s, op->offset, s->in_flight); mirror_wait_for_io(s); } =20 - /* Allocate a MirrorOp that is used as an AIO callback. */ - op =3D g_new(MirrorOp, 1); - op->s =3D s; - op->offset =3D offset; - op->bytes =3D bytes; - /* Now make a QEMUIOVector taking enough granularity-sized chunks * from s->buf_free. */ qemu_iovec_init(&op->qiov, nb_chunks); while (nb_chunks-- > 0) { MirrorBuffer *buf =3D QSIMPLEQ_FIRST(&s->buf_free); - size_t remaining =3D bytes - op->qiov.size; + size_t remaining =3D op->bytes - op->qiov.size; =20 QSIMPLEQ_REMOVE_HEAD(&s->buf_free, next); s->buf_free_count--; @@ -297,53 +297,75 @@ static uint64_t mirror_do_read(MirrorBlockJob *s, int= 64_t offset, =20 /* Copy the dirty cluster. */ s->in_flight++; - s->bytes_in_flight +=3D bytes; - trace_mirror_one_iteration(s, offset, bytes); + s->bytes_in_flight +=3D op->bytes; + trace_mirror_one_iteration(s, op->offset, op->bytes); =20 - blk_aio_preadv(source, offset, &op->qiov, 0, mirror_read_complete, op); - return ret; + ret =3D blk_co_preadv(source, op->offset, op->bytes, &op->qiov, 0); + mirror_read_complete(op, ret); } =20 -static void mirror_do_zero_or_discard(MirrorBlockJob *s, - int64_t offset, - uint64_t bytes, - bool is_discard) +static void coroutine_fn mirror_co_zero(void *opaque) { - MirrorOp *op; + MirrorOp *op =3D opaque; + int ret; =20 - /* Allocate a MirrorOp that is used as an AIO callback. The qiov is ze= roed - * so the freeing in mirror_iteration_done is nop. */ - op =3D g_new0(MirrorOp, 1); - op->s =3D s; - op->offset =3D offset; - op->bytes =3D bytes; + op->s->in_flight++; + op->s->bytes_in_flight +=3D op->bytes; =20 - s->in_flight++; - s->bytes_in_flight +=3D bytes; - if (is_discard) { - blk_aio_pdiscard(s->target, offset, - op->bytes, mirror_write_complete, op); - } else { - blk_aio_pwrite_zeroes(s->target, offset, - op->bytes, s->unmap ? BDRV_REQ_MAY_UNMAP : 0, - mirror_write_complete, op); - } + ret =3D blk_co_pwrite_zeroes(op->s->target, op->offset, op->bytes, + op->s->unmap ? BDRV_REQ_MAY_UNMAP : 0); + mirror_write_complete(op, ret); +} + +static void coroutine_fn mirror_co_discard(void *opaque) +{ + MirrorOp *op =3D opaque; + int ret; + + op->s->in_flight++; + op->s->bytes_in_flight +=3D op->bytes; + + ret =3D blk_co_pdiscard(op->s->target, op->offset, op->bytes); + mirror_write_complete(op, ret); } =20 static unsigned mirror_perform(MirrorBlockJob *s, int64_t offset, unsigned bytes, MirrorMethod mirror_method) { + MirrorOp *op; + Coroutine *co; + unsigned ret =3D bytes; + + op =3D g_new(MirrorOp, 1); + *op =3D (MirrorOp){ + .s =3D s, + .offset =3D offset, + .bytes =3D bytes, + }; + switch (mirror_method) { case MIRROR_METHOD_COPY: - return mirror_do_read(s, offset, bytes); + co =3D qemu_coroutine_create(mirror_co_read, op); + break; case MIRROR_METHOD_ZERO: + co =3D qemu_coroutine_create(mirror_co_zero, op); + break; case MIRROR_METHOD_DISCARD: - mirror_do_zero_or_discard(s, offset, bytes, - mirror_method =3D=3D MIRROR_METHOD_DISCA= RD); - return bytes; + co =3D qemu_coroutine_create(mirror_co_discard, op); + break; default: abort(); } + + qemu_coroutine_enter(co); + + if (mirror_method =3D=3D MIRROR_METHOD_COPY) { + /* Same assertion as in mirror_co_read() */ + assert(op->bytes_copied <=3D UINT_MAX); + ret =3D op->bytes_copied; + } + + return ret; } =20 static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327415431938.6482713158698; Wed, 13 Sep 2017 11:30:15 -0700 (PDT) Received: from localhost ([::1]:44046 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCQQ-000096-IU for importer@patchew.org; Wed, 13 Sep 2017 14:30:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36762) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCGw-0008LA-G5 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCGv-0005FA-7y for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41576) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCGo-0005Ac-KB; Wed, 13 Sep 2017 14:20:18 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B5B984ACA5; Wed, 13 Sep 2017 18:20:17 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5307060600; Wed, 13 Sep 2017 18:20:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B5B984ACA5 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:18:58 +0200 Message-Id: <20170913181910.29688-7-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 13 Sep 2017 18:20:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 06/18] block/mirror: Use CoQueue to wait on in-flight ops X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Attach a CoQueue to each in-flight operation so if we need to wait for any we can use it to wait instead of just blindly yielding and hoping for some operation to wake us. A later patch will use this infrastructure to allow requests accessing the same area of the virtual disk to specifically wait for each other. Signed-off-by: Max Reitz --- block/mirror.c | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 2b3297aa61..81253fbad1 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -13,6 +13,7 @@ =20 #include "qemu/osdep.h" #include "qemu/cutils.h" +#include "qemu/coroutine.h" #include "trace.h" #include "block/blockjob_int.h" #include "block/block_int.h" @@ -34,6 +35,8 @@ typedef struct MirrorBuffer { QSIMPLEQ_ENTRY(MirrorBuffer) next; } MirrorBuffer; =20 +typedef struct MirrorOp MirrorOp; + typedef struct MirrorBlockJob { BlockJob common; RateLimit limit; @@ -67,15 +70,15 @@ typedef struct MirrorBlockJob { unsigned long *in_flight_bitmap; int in_flight; int64_t bytes_in_flight; + QTAILQ_HEAD(MirrorOpList, MirrorOp) ops_in_flight; int ret; bool unmap; - bool waiting_for_io; int target_cluster_size; int max_iov; bool initial_zeroing_ongoing; } MirrorBlockJob; =20 -typedef struct MirrorOp { +struct MirrorOp { MirrorBlockJob *s; QEMUIOVector qiov; int64_t offset; @@ -83,7 +86,11 @@ typedef struct MirrorOp { =20 /* Set by mirror_co_read() before yielding for the first time */ uint64_t bytes_copied; -} MirrorOp; + + CoQueue waiting_requests; + + QTAILQ_ENTRY(MirrorOp) next; +}; =20 typedef enum MirrorMethod { MIRROR_METHOD_COPY, @@ -124,7 +131,9 @@ static void coroutine_fn mirror_iteration_done(MirrorOp= *op, int ret) =20 chunk_num =3D op->offset / s->granularity; nb_chunks =3D DIV_ROUND_UP(op->bytes, s->granularity); + bitmap_clear(s->in_flight_bitmap, chunk_num, nb_chunks); + QTAILQ_REMOVE(&s->ops_in_flight, op, next); if (ret >=3D 0) { if (s->cow_bitmap) { bitmap_set(s->cow_bitmap, chunk_num, nb_chunks); @@ -134,11 +143,9 @@ static void coroutine_fn mirror_iteration_done(MirrorO= p *op, int ret) } } qemu_iovec_destroy(&op->qiov); - g_free(op); =20 - if (s->waiting_for_io) { - qemu_coroutine_enter(s->common.co); - } + qemu_co_queue_restart_all(&op->waiting_requests); + g_free(op); } =20 static void coroutine_fn mirror_write_complete(MirrorOp *op, int ret) @@ -233,10 +240,11 @@ static int mirror_cow_align(MirrorBlockJob *s, int64_= t *offset, =20 static inline void mirror_wait_for_io(MirrorBlockJob *s) { - assert(!s->waiting_for_io); - s->waiting_for_io =3D true; - qemu_coroutine_yield(); - s->waiting_for_io =3D false; + MirrorOp *op; + + op =3D QTAILQ_FIRST(&s->ops_in_flight); + assert(op); + qemu_co_queue_wait(&op->waiting_requests, NULL); } =20 /* Submit async read while handling COW. @@ -342,6 +350,7 @@ static unsigned mirror_perform(MirrorBlockJob *s, int64= _t offset, .offset =3D offset, .bytes =3D bytes, }; + qemu_co_queue_init(&op->waiting_requests); =20 switch (mirror_method) { case MIRROR_METHOD_COPY: @@ -357,6 +366,7 @@ static unsigned mirror_perform(MirrorBlockJob *s, int64= _t offset, abort(); } =20 + QTAILQ_INSERT_TAIL(&s->ops_in_flight, op, next); qemu_coroutine_enter(co); =20 if (mirror_method =3D=3D MIRROR_METHOD_COPY) { @@ -1292,6 +1302,8 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, } } =20 + QTAILQ_INIT(&s->ops_in_flight); + trace_mirror_start(bs, s, opaque); block_job_start(&s->common); return; --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150532711443196.21535121276952; Wed, 13 Sep 2017 11:25:14 -0700 (PDT) Received: from localhost ([::1]:44015 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCLZ-0003MQ-Kp for importer@patchew.org; Wed, 13 Sep 2017 14:25:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36893) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCH7-0008VD-87 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCH5-0005Ma-EV for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46920) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCGy-0005Gc-JH; Wed, 13 Sep 2017 14:20:28 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9D6E5D556B; Wed, 13 Sep 2017 18:20:27 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 26B854D9E6; Wed, 13 Sep 2017 18:20:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 9D6E5D556B Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:18:59 +0200 Message-Id: <20170913181910.29688-8-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 13 Sep 2017 18:20:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 07/18] block/mirror: Wait for in-flight op conflicts X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch makes the mirror code differentiate between simply waiting for any operation to complete (mirror_wait_for_free_in_flight_slot()) and specifically waiting for all operations touching a certain range of the virtual disk to complete (mirror_wait_on_conflicts()). Signed-off-by: Max Reitz --- block/mirror.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++-------= ---- 1 file changed, 70 insertions(+), 15 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 81253fbad1..2ece38094d 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -14,6 +14,7 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" #include "qemu/coroutine.h" +#include "qemu/range.h" #include "trace.h" #include "block/blockjob_int.h" #include "block/block_int.h" @@ -111,6 +112,41 @@ static BlockErrorAction mirror_error_action(MirrorBloc= kJob *s, bool read, } } =20 +static void coroutine_fn mirror_wait_on_conflicts(MirrorOp *self, + MirrorBlockJob *s, + uint64_t offset, + uint64_t bytes) +{ + uint64_t self_start_chunk =3D offset / s->granularity; + uint64_t self_end_chunk =3D DIV_ROUND_UP(offset + bytes, s->granularit= y); + uint64_t self_nb_chunks =3D self_end_chunk - self_start_chunk; + + while (find_next_bit(s->in_flight_bitmap, self_end_chunk, + self_start_chunk) < self_end_chunk && + s->ret >=3D 0) + { + MirrorOp *op; + + QTAILQ_FOREACH(op, &s->ops_in_flight, next) { + uint64_t op_start_chunk =3D op->offset / s->granularity; + uint64_t op_nb_chunks =3D DIV_ROUND_UP(op->offset + op->bytes, + s->granularity) - + op_start_chunk; + + if (op =3D=3D self) { + continue; + } + + if (ranges_overlap(self_start_chunk, self_nb_chunks, + op_start_chunk, op_nb_chunks)) + { + qemu_co_queue_wait(&op->waiting_requests, NULL); + break; + } + } + } +} + static void coroutine_fn mirror_iteration_done(MirrorOp *op, int ret) { MirrorBlockJob *s =3D op->s; @@ -238,7 +274,7 @@ static int mirror_cow_align(MirrorBlockJob *s, int64_t = *offset, return ret; } =20 -static inline void mirror_wait_for_io(MirrorBlockJob *s) +static inline void mirror_wait_for_free_in_flight_slot(MirrorBlockJob *s) { MirrorOp *op; =20 @@ -287,7 +323,7 @@ static void coroutine_fn mirror_co_read(void *opaque) =20 while (s->buf_free_count < nb_chunks) { trace_mirror_yield_in_flight(s, op->offset, s->in_flight); - mirror_wait_for_io(s); + mirror_wait_for_free_in_flight_slot(s); } =20 /* Now make a QEMUIOVector taking enough granularity-sized chunks @@ -381,8 +417,9 @@ static unsigned mirror_perform(MirrorBlockJob *s, int64= _t offset, static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) { BlockDriverState *source =3D s->source; - int64_t offset, first_chunk; - uint64_t delay_ns =3D 0; + MirrorOp *pseudo_op; + int64_t offset; + uint64_t delay_ns =3D 0, ret =3D 0; /* At least the first dirty chunk is mirrored in one iteration. */ int nb_chunks =3D 1; int sectors_per_chunk =3D s->granularity >> BDRV_SECTOR_BITS; @@ -400,11 +437,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBl= ockJob *s) } bdrv_dirty_bitmap_unlock(s->dirty_bitmap); =20 - first_chunk =3D offset / s->granularity; - while (test_bit(first_chunk, s->in_flight_bitmap)) { - trace_mirror_yield_in_flight(s, offset, s->in_flight); - mirror_wait_for_io(s); - } + mirror_wait_on_conflicts(NULL, s, offset, 1); =20 block_job_pause_point(&s->common); =20 @@ -442,6 +475,20 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBl= ockJob *s) nb_chunks * sectors_per_chunk); bdrv_dirty_bitmap_unlock(s->dirty_bitmap); =20 + /* Before claiming an area in the in-flight bitmap, we have to + * create a MirrorOp for it so that conflicting requests can wait + * for it. mirror_perform() will create the real MirrorOps later, + * for now we just create a pseudo operation that will wake up all + * conflicting requests once all real operations have been + * launched. */ + pseudo_op =3D g_new(MirrorOp, 1); + *pseudo_op =3D (MirrorOp){ + .offset =3D offset, + .bytes =3D nb_chunks * s->granularity, + }; + qemu_co_queue_init(&pseudo_op->waiting_requests); + QTAILQ_INSERT_TAIL(&s->ops_in_flight, pseudo_op, next); + bitmap_set(s->in_flight_bitmap, offset / s->granularity, nb_chunks); while (nb_chunks > 0 && offset < s->bdev_length) { int64_t ret; @@ -481,11 +528,12 @@ static uint64_t coroutine_fn mirror_iteration(MirrorB= lockJob *s) =20 while (s->in_flight >=3D MAX_IN_FLIGHT) { trace_mirror_yield_in_flight(s, offset, s->in_flight); - mirror_wait_for_io(s); + mirror_wait_for_free_in_flight_slot(s); } =20 if (s->ret < 0) { - return 0; + ret =3D 0; + goto fail; } =20 io_bytes =3D mirror_clip_bytes(s, offset, io_bytes); @@ -502,7 +550,14 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBl= ockJob *s) delay_ns =3D ratelimit_calculate_delay(&s->limit, io_bytes_acc= t); } } - return delay_ns; + + ret =3D delay_ns; +fail: + QTAILQ_REMOVE(&s->ops_in_flight, pseudo_op, next); + qemu_co_queue_restart_all(&pseudo_op->waiting_requests); + g_free(pseudo_op); + + return ret; } =20 static void mirror_free_init(MirrorBlockJob *s) @@ -529,7 +584,7 @@ static void mirror_free_init(MirrorBlockJob *s) static void mirror_wait_for_all_io(MirrorBlockJob *s) { while (s->in_flight > 0) { - mirror_wait_for_io(s); + mirror_wait_for_free_in_flight_slot(s); } } =20 @@ -685,7 +740,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) if (s->in_flight >=3D MAX_IN_FLIGHT) { trace_mirror_yield(s, UINT64_MAX, s->buf_free_count, s->in_flight); - mirror_wait_for_io(s); + mirror_wait_for_free_in_flight_slot(s); continue; } =20 @@ -868,7 +923,7 @@ static void coroutine_fn mirror_run(void *opaque) (cnt =3D=3D 0 && s->in_flight > 0)) { trace_mirror_yield(s, cnt * BDRV_SECTOR_SIZE, s->buf_free_count, s->in_flight); - mirror_wait_for_io(s); + mirror_wait_for_free_in_flight_slot(s); continue; } else if (cnt !=3D 0) { delay_ns =3D mirror_iteration(s); --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327137494817.1218116797187; Wed, 13 Sep 2017 11:25:37 -0700 (PDT) Received: from localhost ([::1]:44018 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCLw-0003gs-Np for importer@patchew.org; Wed, 13 Sep 2017 14:25:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36905) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCH8-0008W0-1T for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCH6-0005Nj-Jw for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52700) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCH0-0005IO-Uz; Wed, 13 Sep 2017 14:20:31 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1312781DE6; Wed, 13 Sep 2017 18:20:30 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9759C61787; Wed, 13 Sep 2017 18:20:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1312781DE6 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:00 +0200 Message-Id: <20170913181910.29688-9-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 13 Sep 2017 18:20:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 08/18] block/mirror: Use source as a BdrvChild X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" With this, the mirror_top_bs is no longer just a technically required node in the BDS graph but actually represents the block job operation. Signed-off-by: Max Reitz --- block/mirror.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 2ece38094d..9df4157511 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -43,8 +43,8 @@ typedef struct MirrorBlockJob { RateLimit limit; BlockBackend *target; BlockDriverState *mirror_top_bs; - BlockDriverState *source; BlockDriverState *base; + BdrvChild *source; =20 /* The name of the graph node to replace */ char *replaces; @@ -294,7 +294,6 @@ static void coroutine_fn mirror_co_read(void *opaque) { MirrorOp *op =3D opaque; MirrorBlockJob *s =3D op->s; - BlockBackend *source =3D s->common.blk; int nb_chunks; uint64_t ret; uint64_t max_bytes; @@ -344,7 +343,7 @@ static void coroutine_fn mirror_co_read(void *opaque) s->bytes_in_flight +=3D op->bytes; trace_mirror_one_iteration(s, op->offset, op->bytes); =20 - ret =3D blk_co_preadv(source, op->offset, op->bytes, &op->qiov, 0); + ret =3D bdrv_co_preadv(s->source, op->offset, op->bytes, &op->qiov, 0); mirror_read_complete(op, ret); } =20 @@ -416,7 +415,7 @@ static unsigned mirror_perform(MirrorBlockJob *s, int64= _t offset, =20 static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) { - BlockDriverState *source =3D s->source; + BlockDriverState *source =3D s->source->bs; MirrorOp *pseudo_op; int64_t offset; uint64_t delay_ns =3D 0, ret =3D 0; @@ -597,7 +596,7 @@ static void mirror_exit(BlockJob *job, void *opaque) MirrorBlockJob *s =3D container_of(job, MirrorBlockJob, common); MirrorExitData *data =3D opaque; AioContext *replace_aio_context =3D NULL; - BlockDriverState *src =3D s->source; + BlockDriverState *src =3D s->source->bs; BlockDriverState *target_bs =3D blk_bs(s->target); BlockDriverState *mirror_top_bs =3D s->mirror_top_bs; Error *local_err =3D NULL; @@ -712,7 +711,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) { int64_t sector_num, end; BlockDriverState *base =3D s->base; - BlockDriverState *bs =3D s->source; + BlockDriverState *bs =3D s->source->bs; BlockDriverState *target_bs =3D blk_bs(s->target); int ret, n; int64_t count; @@ -802,7 +801,7 @@ static void coroutine_fn mirror_run(void *opaque) { MirrorBlockJob *s =3D opaque; MirrorExitData *data; - BlockDriverState *bs =3D s->source; + BlockDriverState *bs =3D s->source->bs; BlockDriverState *target_bs =3D blk_bs(s->target); bool need_drain =3D true; int64_t length; @@ -1284,7 +1283,7 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, /* The block job now has a reference to this node */ bdrv_unref(mirror_top_bs); =20 - s->source =3D bs; + s->source =3D mirror_top_bs->backing; s->mirror_top_bs =3D mirror_top_bs; =20 /* No resize for the target either; while the mirror is still running,= a @@ -1330,6 +1329,9 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, s->should_complete =3D true; } =20 + s->source =3D mirror_top_bs->backing; + s->mirror_top_bs =3D mirror_top_bs; + s->dirty_bitmap =3D bdrv_create_dirty_bitmap(bs, granularity, NULL, er= rp); if (!s->dirty_bitmap) { goto fail; --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327136676975.4848062967874; Wed, 13 Sep 2017 11:25:36 -0700 (PDT) Received: from localhost ([::1]:44017 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCLv-0003fm-Ms for importer@patchew.org; Wed, 13 Sep 2017 14:25:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37057) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCHN-0000Ec-RD for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCHM-0005ZG-NR for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:53 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42704) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCHC-0005RD-Jh; Wed, 13 Sep 2017 14:20:42 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AF6934A6E0; Wed, 13 Sep 2017 18:20:41 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 20C8460C17; Wed, 13 Sep 2017 18:20:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com AF6934A6E0 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:01 +0200 Message-Id: <20170913181910.29688-10-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 13 Sep 2017 18:20:41 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 09/18] block: Generalize should_update_child() rule X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Currently, bdrv_replace_node() refuses to create loops from one BDS to itself if the BDS to be replaced is the backing node of the BDS to replace it: Say there is a node A and a node B. Replacing B by A means making all references to B point to A. If B is a child of A (i.e. A has a reference to B), that would mean we would have to make this reference point to A itself -- so we'd create a loop. bdrv_replace_node() (through should_update_child()) refuses to do so if B is the backing node of A. There is no reason why we should create loops if B is not the backing node of A, though. The BDS graph should never contain loops, so we should always refuse to create them. If B is a child of A and B is to be replaced by A, we should simply leave B in place there because it is the most sensible choice. A more specific argument would be: Putting filter drivers into the BDS graph is basically the same as appending an overlay to a backing chain. But the main child BDS of a filter driver is not "backing" but "file", so restricting the no-loop rule to backing nodes would fail here. Signed-off-by: Max Reitz --- include/block/block_int.h | 2 ++ block.c | 44 ++++++++++++++++++++++++++++++++++---------- 2 files changed, 36 insertions(+), 10 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index eaeaad9428..fa8bbf1f8b 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -573,6 +573,8 @@ struct BdrvChild { QLIST_ENTRY(BdrvChild) next_parent; }; =20 +typedef QLIST_HEAD(BdrvChildList, BdrvChild) BdrvChildList; + /* * Note: the function bdrv_append() copies and swaps contents of * BlockDriverStates, so if you add new fields to this struct, please diff --git a/block.c b/block.c index 0b55c5a41c..1898b958c9 100644 --- a/block.c +++ b/block.c @@ -3134,16 +3134,39 @@ static bool should_update_child(BdrvChild *c, Block= DriverState *to) return false; } =20 - if (c->role =3D=3D &child_backing) { - /* If @from is a backing file of @to, ignore the child to avoid - * creating a loop. We only want to change the pointer of other - * parents. */ - QLIST_FOREACH(to_c, &to->children, next) { - if (to_c =3D=3D c) { - break; - } - } - if (to_c) { + /* If the child @c belongs to the BDS @to, replacing the current + * c->bs by @to would mean to create a loop. + * + * Such a case occurs when appending a BDS to a backing chain. + * For instance, imagine the following chain: + * + * guest device -> node A -> further backing chain... + * + * Now we create a new BDS B which we want to put on top of this + * chain, so we first attach A as its backing node: + * + * node B + * | + * v + * guest device -> node A -> further backing chain... + * + * Finally we want to replace A by B. When doing that, we want to + * replace all pointers to A by pointers to B -- except for the + * pointer from B because (1) that would create a loop, and (2) + * that pointer should simply stay intact: + * + * guest device -> node B + * | + * v + * node A -> further backing chain... + * + * In general, when replacing a node A (c->bs) by a node B (@to), + * if A is a child of B, that means we cannot replace A by B there + * because that would create a loop. Silently detaching A from B + * is also not really an option. So overall just leaving A in + * place there is the most sensible choice. */ + QLIST_FOREACH(to_c, &to->children, next) { + if (to_c =3D=3D c) { return false; } } @@ -3169,6 +3192,7 @@ void bdrv_replace_node(BlockDriverState *from, BlockD= riverState *to, =20 /* Put all parents into @list and calculate their cumulative permissio= ns */ QLIST_FOREACH_SAFE(c, &from->parents, next_parent, next) { + assert(c->bs =3D=3D from); if (!should_update_child(c, to)) { continue; } --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150532764234482.61121893106304; Wed, 13 Sep 2017 11:34:02 -0700 (PDT) Received: from localhost ([::1]:44070 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCU5-0003mU-JR for importer@patchew.org; Wed, 13 Sep 2017 14:34:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37181) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCHV-0000LU-7D for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCHS-0005fV-W3 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:01 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51504) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCHM-0005YV-BH; Wed, 13 Sep 2017 14:20:52 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6A118C047B88; Wed, 13 Sep 2017 18:20:51 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 202205D763; Wed, 13 Sep 2017 18:20:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6A118C047B88 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:02 +0200 Message-Id: <20170913181910.29688-11-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 13 Sep 2017 18:20:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 10/18] block/mirror: Make source the file child X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Regarding the source BDS, the mirror BDS is arguably a filter node. Therefore, the source BDS should be its "file" child. Signed-off-by: Max Reitz --- block/mirror.c | 127 ++++++++++++++++++++++++++++++++++-------= ---- block/qapi.c | 25 ++++++--- tests/qemu-iotests/141.out | 4 +- 3 files changed, 119 insertions(+), 37 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 9df4157511..05410c94ca 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -77,8 +77,16 @@ typedef struct MirrorBlockJob { int target_cluster_size; int max_iov; bool initial_zeroing_ongoing; + + /* Signals that we are no longer accessing source and target and the m= irror + * BDS should thus relinquish all permissions */ + bool exiting; } MirrorBlockJob; =20 +typedef struct MirrorBDSOpaque { + MirrorBlockJob *job; +} MirrorBDSOpaque; + struct MirrorOp { MirrorBlockJob *s; QEMUIOVector qiov; @@ -595,12 +603,15 @@ static void mirror_exit(BlockJob *job, void *opaque) { MirrorBlockJob *s =3D container_of(job, MirrorBlockJob, common); MirrorExitData *data =3D opaque; + MirrorBDSOpaque *bs_opaque =3D s->mirror_top_bs->opaque; AioContext *replace_aio_context =3D NULL; BlockDriverState *src =3D s->source->bs; BlockDriverState *target_bs =3D blk_bs(s->target); BlockDriverState *mirror_top_bs =3D s->mirror_top_bs; Error *local_err =3D NULL; =20 + s->exiting =3D true; + bdrv_release_dirty_bitmap(src, s->dirty_bitmap); =20 /* Make sure that the source BDS doesn't go away before we called @@ -622,7 +633,7 @@ static void mirror_exit(BlockJob *job, void *opaque) =20 /* We don't access the source any more. Dropping any WRITE/RESIZE is * required before it could become a backing file of target_bs. */ - bdrv_child_try_set_perm(mirror_top_bs->backing, 0, BLK_PERM_ALL, + bdrv_child_try_set_perm(mirror_top_bs->file, 0, BLK_PERM_ALL, &error_abort); if (s->backing_mode =3D=3D MIRROR_SOURCE_BACKING_CHAIN) { BlockDriverState *backing =3D s->is_none_mode ? src : s->base; @@ -673,12 +684,11 @@ static void mirror_exit(BlockJob *job, void *opaque) =20 /* Remove the mirror filter driver from the graph. Before this, get ri= d of * the blockers on the intermediate nodes so that the resulting state = is - * valid. Also give up permissions on mirror_top_bs->backing, which mi= ght + * valid. Also give up permissions on mirror_top_bs->file, which might * block the removal. */ block_job_remove_all_bdrv(job); - bdrv_child_try_set_perm(mirror_top_bs->backing, 0, BLK_PERM_ALL, - &error_abort); - bdrv_replace_node(mirror_top_bs, backing_bs(mirror_top_bs), &error_abo= rt); + bdrv_child_try_set_perm(mirror_top_bs->file, 0, BLK_PERM_ALL, &error_a= bort); + bdrv_replace_node(mirror_top_bs, mirror_top_bs->file->bs, &error_abort= ); =20 /* We just changed the BDS the job BB refers to (with either or both o= f the * bdrv_replace_node() calls), so switch the BB back so the cleanup do= es @@ -687,6 +697,7 @@ static void mirror_exit(BlockJob *job, void *opaque) blk_set_perm(job->blk, 0, BLK_PERM_ALL, &error_abort); blk_insert_bs(job->blk, mirror_top_bs, &error_abort); =20 + bs_opaque->job =3D NULL; block_job_completed(&s->common, data->ret); =20 g_free(data); @@ -1102,7 +1113,7 @@ static void mirror_drain(BlockJob *job) { MirrorBlockJob *s =3D container_of(job, MirrorBlockJob, common); =20 - /* Need to keep a reference in case blk_drain triggers execution + /* Need to keep a reference in case bdrv_drain triggers execution * of mirror_complete... */ if (s->target) { @@ -1135,44 +1146,88 @@ static const BlockJobDriver commit_active_job_drive= r =3D { .drain =3D mirror_drain, }; =20 +static void source_child_inherit_fmt_options(int *child_flags, + QDict *child_options, + int parent_flags, + QDict *parent_options) +{ + child_backing.inherit_options(child_flags, child_options, + parent_flags, parent_options); +} + +static char *source_child_get_parent_desc(BdrvChild *c) +{ + return child_backing.get_parent_desc(c); +} + +static void source_child_cb_drained_begin(BdrvChild *c) +{ + BlockDriverState *bs =3D c->opaque; + MirrorBDSOpaque *s =3D bs->opaque; + + if (s && s->job) { + block_job_drained_begin(&s->job->common); + } + bdrv_drained_begin(bs); +} + +static void source_child_cb_drained_end(BdrvChild *c) +{ + BlockDriverState *bs =3D c->opaque; + MirrorBDSOpaque *s =3D bs->opaque; + + if (s && s->job) { + block_job_drained_end(&s->job->common); + } + bdrv_drained_end(bs); +} + +static BdrvChildRole source_child_role =3D { + .inherit_options =3D source_child_inherit_fmt_options, + .get_parent_desc =3D source_child_get_parent_desc, + .drained_begin =3D source_child_cb_drained_begin, + .drained_end =3D source_child_cb_drained_end, +}; + static int coroutine_fn bdrv_mirror_top_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags); + return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags); } =20 static int coroutine_fn bdrv_mirror_top_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - return bdrv_co_pwritev(bs->backing, offset, bytes, qiov, flags); + return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags); } =20 static int coroutine_fn bdrv_mirror_top_flush(BlockDriverState *bs) { - return bdrv_co_flush(bs->backing->bs); + return bdrv_co_flush(bs->file->bs); } =20 static int coroutine_fn bdrv_mirror_top_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int bytes, BdrvRequestFlags flags) { - return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags); + return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags); } =20 static int coroutine_fn bdrv_mirror_top_pdiscard(BlockDriverState *bs, int64_t offset, int bytes) { - return bdrv_co_pdiscard(bs->backing->bs, offset, bytes); + return bdrv_co_pdiscard(bs->file->bs, offset, bytes); } =20 static void bdrv_mirror_top_refresh_filename(BlockDriverState *bs, QDict *= opts) { - bdrv_refresh_filename(bs->backing->bs); pstrcpy(bs->exact_filename, sizeof(bs->exact_filename), - bs->backing->bs->filename); + bs->file->bs->filename); } =20 static void bdrv_mirror_top_close(BlockDriverState *bs) { + bdrv_unref_child(bs, bs->file); + bs->file =3D NULL; } =20 static void bdrv_mirror_top_child_perm(BlockDriverState *bs, BdrvChild *c, @@ -1180,6 +1235,14 @@ static void bdrv_mirror_top_child_perm(BlockDriverSt= ate *bs, BdrvChild *c, uint64_t perm, uint64_t shared, uint64_t *nperm, uint64_t *nshared) { + MirrorBDSOpaque *s =3D bs->opaque; + + if (s->job && s->job->exiting) { + *nperm =3D 0; + *nshared =3D BLK_PERM_ALL; + return; + } + /* Must be able to forward guest writes to the real image */ *nperm =3D 0; if (perm & BLK_PERM_WRITE) { @@ -1190,7 +1253,7 @@ static void bdrv_mirror_top_child_perm(BlockDriverSta= te *bs, BdrvChild *c, } =20 /* Dummy node that provides consistent read to its users without requiring= it - * from its backing file and that allows writes on the backing file chain.= */ + * from its source file and that allows writes on the source file. */ static BlockDriver bdrv_mirror_top =3D { .format_name =3D "mirror_top", .bdrv_co_preadv =3D bdrv_mirror_top_preadv, @@ -1198,7 +1261,7 @@ static BlockDriver bdrv_mirror_top =3D { .bdrv_co_pwrite_zeroes =3D bdrv_mirror_top_pwrite_zeroes, .bdrv_co_pdiscard =3D bdrv_mirror_top_pdiscard, .bdrv_co_flush =3D bdrv_mirror_top_flush, - .bdrv_co_get_block_status =3D bdrv_co_get_block_status_from_backing, + .bdrv_co_get_block_status =3D bdrv_co_get_block_status_from_file, .bdrv_refresh_filename =3D bdrv_mirror_top_refresh_filename, .bdrv_close =3D bdrv_mirror_top_close, .bdrv_child_perm =3D bdrv_mirror_top_child_perm, @@ -1221,6 +1284,7 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, Error **errp) { MirrorBlockJob *s; + MirrorBDSOpaque *bs_opaque; BlockDriverState *mirror_top_bs; bool target_graph_mod; bool target_is_backing; @@ -1244,9 +1308,7 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, buf_size =3D DEFAULT_MIRROR_BUF_SIZE; } =20 - /* In the case of active commit, add dummy driver to provide consistent - * reads on the top, while disabling it in the intermediate nodes, and= make - * the backing chain writable. */ + /* Create mirror BDS */ mirror_top_bs =3D bdrv_new_open_driver(&bdrv_mirror_top, filter_node_n= ame, BDRV_O_RDWR, errp); if (mirror_top_bs =3D=3D NULL) { @@ -1256,14 +1318,19 @@ static void mirror_start_job(const char *job_id, Bl= ockDriverState *bs, mirror_top_bs->implicit =3D true; } mirror_top_bs->total_sectors =3D bs->total_sectors; + bs_opaque =3D g_new0(MirrorBDSOpaque, 1); + mirror_top_bs->opaque =3D bs_opaque; bdrv_set_aio_context(mirror_top_bs, bdrv_get_aio_context(bs)); =20 - /* bdrv_append takes ownership of the mirror_top_bs reference, need to= keep - * it alive until block_job_create() succeeds even if bs has no parent= . */ - bdrv_ref(mirror_top_bs); - bdrv_drained_begin(bs); - bdrv_append(mirror_top_bs, bs, &local_err); - bdrv_drained_end(bs); + /* Create reference for bdrv_attach_child() */ + bdrv_ref(bs); + mirror_top_bs->file =3D bdrv_attach_child(mirror_top_bs, bs, "file", + &source_child_role, &local_err= ); + if (!local_err) { + bdrv_drained_begin(bs); + bdrv_replace_node(bs, mirror_top_bs, &local_err); + bdrv_drained_end(bs); + } =20 if (local_err) { bdrv_unref(mirror_top_bs); @@ -1280,6 +1347,8 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, if (!s) { goto fail; } + bs_opaque->job =3D s; + /* The block job now has a reference to this node */ bdrv_unref(mirror_top_bs); =20 @@ -1329,7 +1398,7 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, s->should_complete =3D true; } =20 - s->source =3D mirror_top_bs->backing; + s->source =3D mirror_top_bs->file; s->mirror_top_bs =3D mirror_top_bs; =20 s->dirty_bitmap =3D bdrv_create_dirty_bitmap(bs, granularity, NULL, er= rp); @@ -1373,12 +1442,12 @@ fail: =20 g_free(s->replaces); blk_unref(s->target); - block_job_early_fail(&s->common); + bs_opaque->job =3D NULL; + block_job_unref(&s->common); } =20 - bdrv_child_try_set_perm(mirror_top_bs->backing, 0, BLK_PERM_ALL, - &error_abort); - bdrv_replace_node(mirror_top_bs, backing_bs(mirror_top_bs), &error_abo= rt); + bdrv_child_try_set_perm(mirror_top_bs->file, 0, BLK_PERM_ALL, &error_a= bort); + bdrv_replace_node(mirror_top_bs, mirror_top_bs->file->bs, &error_abort= ); =20 bdrv_unref(mirror_top_bs); } diff --git a/block/qapi.c b/block/qapi.c index 7fa2437923..ee792d0cbc 100644 --- a/block/qapi.c +++ b/block/qapi.c @@ -147,9 +147,13 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *= blk, =20 /* Skip automatically inserted nodes that the user isn't aware of = for * query-block (blk !=3D NULL), but not for query-named-block-node= s */ - while (blk && bs0->drv && bs0->implicit) { - bs0 =3D backing_bs(bs0); - assert(bs0); + while (blk && bs0 && bs0->drv && bs0->implicit) { + if (bs0->backing) { + bs0 =3D backing_bs(bs0); + } else { + assert(bs0->file); + bs0 =3D bs0->file->bs; + } } } =20 @@ -337,7 +341,12 @@ static void bdrv_query_info(BlockBackend *blk, BlockIn= fo **p_info, =20 /* Skip automatically inserted nodes that the user isn't aware of */ while (bs && bs->drv && bs->implicit) { - bs =3D backing_bs(bs); + if (bs->backing) { + bs =3D backing_bs(bs); + } else { + assert(bs->file); + bs =3D bs->file->bs; + } } =20 info->device =3D g_strdup(blk_name(blk)); @@ -466,8 +475,12 @@ static BlockStats *bdrv_query_bds_stats(BlockDriverSta= te *bs, * a BlockBackend-level command. Stay at the exact node for a node-lev= el * command. */ while (blk_level && bs->drv && bs->implicit) { - bs =3D backing_bs(bs); - assert(bs); + if (bs->backing) { + bs =3D backing_bs(bs); + } else { + assert(bs->file); + bs =3D bs->file->bs; + } } =20 if (bdrv_get_node_name(bs)[0]) { diff --git a/tests/qemu-iotests/141.out b/tests/qemu-iotests/141.out index 82e763b68d..8c4dd6d531 100644 --- a/tests/qemu-iotests/141.out +++ b/tests/qemu-iotests/141.out @@ -20,7 +20,7 @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=3DIMGFMT size=3D10485= 76 backing_file=3DTEST_DIR/t. Formatting 'TEST_DIR/o.IMGFMT', fmt=3DIMGFMT size=3D1048576 backing_file= =3DTEST_DIR/t.IMGFMT backing_fmt=3DIMGFMT {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "spe= ed": 0, "type": "mirror"}} {"return": {}} -{"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: node is = used as backing hd of 'NODE_NAME'"}} +{"error": {"class": "GenericError", "desc": "Block device drv0 is in use"}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 0, "offset": 0, = "speed": 0, "type": "mirror"}} {"return": {}} @@ -30,7 +30,7 @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=3DIMGFMT size=3D10485= 76 backing_file=3DTEST_DIR/t. {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "spe= ed": 0, "type": "commit"}} {"return": {}} -{"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: node is = used as backing hd of 'NODE_NAME'"}} +{"error": {"class": "GenericError", "desc": "Block device drv0 is in use"}} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 0, "offset": 0, = "speed": 0, "type": "commit"}} {"return": {}} --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327301478913.9526005409033; Wed, 13 Sep 2017 11:28:21 -0700 (PDT) Received: from localhost ([::1]:44034 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCOa-0006NG-LX for importer@patchew.org; Wed, 13 Sep 2017 14:28:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37328) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCHe-0000X2-Sz for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCHd-0005qx-QG for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:10 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52134) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCHX-0005k9-9A; Wed, 13 Sep 2017 14:21:03 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 61C33C047B73; Wed, 13 Sep 2017 18:21:02 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9B9525C6D2; Wed, 13 Sep 2017 18:20:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 61C33C047B73 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:03 +0200 Message-Id: <20170913181910.29688-12-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 13 Sep 2017 18:21:02 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 11/18] hbitmap: Add @advance param to hbitmap_iter_next() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This new parameter allows the caller to just query the next dirty position without moving the iterator. Signed-off-by: Max Reitz --- include/qemu/hbitmap.h | 4 +++- block/dirty-bitmap.c | 2 +- tests/test-hbitmap.c | 26 +++++++++++++------------- util/hbitmap.c | 10 +++++++--- 4 files changed, 24 insertions(+), 18 deletions(-) diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index d3a74a21fc..6a52575ad5 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -316,11 +316,13 @@ void hbitmap_free_meta(HBitmap *hb); /** * hbitmap_iter_next: * @hbi: HBitmapIter to operate on. + * @advance: If true, advance the iterator. Otherwise, the next call + * of this function will return the same result. * * Return the next bit that is set in @hbi's associated HBitmap, * or -1 if all remaining bits are zero. */ -int64_t hbitmap_iter_next(HBitmapIter *hbi); +int64_t hbitmap_iter_next(HBitmapIter *hbi, bool advance); =20 /** * hbitmap_iter_next_word: diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 30462d4f9a..aee57cf8c8 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -547,7 +547,7 @@ void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter) =20 int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter) { - return hbitmap_iter_next(&iter->hbi); + return hbitmap_iter_next(&iter->hbi, true); } =20 /* Called within bdrv_dirty_bitmap_lock..unlock */ diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c index 1acb353889..e6d4d563cb 100644 --- a/tests/test-hbitmap.c +++ b/tests/test-hbitmap.c @@ -46,7 +46,7 @@ static void hbitmap_test_check(TestHBitmapData *data, =20 i =3D first; for (;;) { - next =3D hbitmap_iter_next(&hbi); + next =3D hbitmap_iter_next(&hbi, true); if (next < 0) { next =3D data->size; } @@ -435,25 +435,25 @@ static void test_hbitmap_iter_granularity(TestHBitmap= Data *data, /* Note that hbitmap_test_check has to be invoked manually in this tes= t. */ hbitmap_test_init(data, 131072 << 7, 7); hbitmap_iter_init(&hbi, data->hb, 0); - g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); =20 hbitmap_test_set(data, ((L2 + L1 + 1) << 7) + 8, 8); hbitmap_iter_init(&hbi, data->hb, 0); - g_assert_cmpint(hbitmap_iter_next(&hbi), =3D=3D, (L2 + L1 + 1) << 7); - g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), =3D=3D, (L2 + L1 + 1) <= < 7); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); =20 hbitmap_iter_init(&hbi, data->hb, (L2 + L1 + 2) << 7); - g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); =20 hbitmap_test_set(data, (131072 << 7) - 8, 8); hbitmap_iter_init(&hbi, data->hb, 0); - g_assert_cmpint(hbitmap_iter_next(&hbi), =3D=3D, (L2 + L1 + 1) << 7); - g_assert_cmpint(hbitmap_iter_next(&hbi), =3D=3D, 131071 << 7); - g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), =3D=3D, (L2 + L1 + 1) <= < 7); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), =3D=3D, 131071 << 7); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); =20 hbitmap_iter_init(&hbi, data->hb, (L2 + L1 + 2) << 7); - g_assert_cmpint(hbitmap_iter_next(&hbi), =3D=3D, 131071 << 7); - g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), =3D=3D, 131071 << 7); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); } =20 static void hbitmap_test_set_boundary_bits(TestHBitmapData *data, ssize_t = diff) @@ -893,7 +893,7 @@ static void test_hbitmap_serialize_zeroes(TestHBitmapDa= ta *data, for (i =3D 0; i < num_positions; i++) { hbitmap_deserialize_zeroes(data->hb, positions[i], min_l1, true); hbitmap_iter_init(&iter, data->hb, 0); - next =3D hbitmap_iter_next(&iter); + next =3D hbitmap_iter_next(&iter, true); if (i =3D=3D num_positions - 1) { g_assert_cmpint(next, =3D=3D, -1); } else { @@ -919,10 +919,10 @@ static void test_hbitmap_iter_and_reset(TestHBitmapDa= ta *data, =20 hbitmap_iter_init(&hbi, data->hb, BITS_PER_LONG - 1); =20 - hbitmap_iter_next(&hbi); + hbitmap_iter_next(&hbi, true); =20 hbitmap_reset_all(data->hb); - hbitmap_iter_next(&hbi); + hbitmap_iter_next(&hbi, true); } =20 int main(int argc, char **argv) diff --git a/util/hbitmap.c b/util/hbitmap.c index 21535cc90b..96525983ce 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -141,7 +141,7 @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi) return cur; } =20 -int64_t hbitmap_iter_next(HBitmapIter *hbi) +int64_t hbitmap_iter_next(HBitmapIter *hbi, bool advance) { unsigned long cur =3D hbi->cur[HBITMAP_LEVELS - 1] & hbi->hb->levels[HBITMAP_LEVELS - 1][hbi->pos]; @@ -154,8 +154,12 @@ int64_t hbitmap_iter_next(HBitmapIter *hbi) } } =20 - /* The next call will resume work from the next bit. */ - hbi->cur[HBITMAP_LEVELS - 1] =3D cur & (cur - 1); + if (advance) { + /* The next call will resume work from the next bit. */ + hbi->cur[HBITMAP_LEVELS - 1] =3D cur & (cur - 1); + } else { + hbi->cur[HBITMAP_LEVELS - 1] =3D cur; + } item =3D ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur); =20 return item << hbi->granularity; --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327291866147.96756687304662; Wed, 13 Sep 2017 11:28:11 -0700 (PDT) Received: from localhost ([::1]:44033 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCOR-0006EP-37 for importer@patchew.org; Wed, 13 Sep 2017 14:28:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37474) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCHs-0000eH-IV for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCHr-000627-Ff for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52594) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCHh-0005uu-K7; Wed, 13 Sep 2017 14:21:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B7456C04B941; Wed, 13 Sep 2017 18:21:12 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 34E1A5EDE2; Wed, 13 Sep 2017 18:21:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B7456C04B941 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:04 +0200 Message-Id: <20170913181910.29688-13-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 13 Sep 2017 18:21:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 12/18] block/dirty-bitmap: Add bdrv_dirty_iter_next_area X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This new function allows to look for a consecutively dirty area in a dirty bitmap. Signed-off-by: Max Reitz --- include/block/dirty-bitmap.h | 2 ++ block/dirty-bitmap.c | 52 ++++++++++++++++++++++++++++++++++++++++= ++++ 2 files changed, 54 insertions(+) diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index a79a58d2c3..7654748700 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -90,6 +90,8 @@ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t cur_sector, int64_t nr_sectors= ); int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter); +bool bdrv_dirty_iter_next_area(BdrvDirtyBitmapIter *iter, uint64_t max_off= set, + uint64_t *offset, int *bytes); void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t sector_num); int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap); int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap); diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index aee57cf8c8..81b2f78016 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -550,6 +550,58 @@ int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter) return hbitmap_iter_next(&iter->hbi, true); } =20 +/** + * Return the next consecutively dirty area in the dirty bitmap + * belonging to the given iterator @iter. + * + * @max_offset: Maximum value that may be returned for + * *offset + *bytes + * @offset: Will contain the start offset of the next dirty area + * @bytes: Will contain the length of the next dirty area + * + * Returns: True if a dirty area could be found before max_offset + * (which means that *offset and *bytes then contain valid + * values), false otherwise. + */ +bool bdrv_dirty_iter_next_area(BdrvDirtyBitmapIter *iter, uint64_t max_off= set, + uint64_t *offset, int *bytes) +{ + uint32_t granularity =3D bdrv_dirty_bitmap_granularity(iter->bitmap); + uint64_t gran_max_offset; + int sector_gran =3D granularity >> BDRV_SECTOR_BITS; + int64_t ret; + int size; + + if (DIV_ROUND_UP(max_offset, BDRV_SECTOR_SIZE) =3D=3D iter->bitmap->si= ze) { + /* If max_offset points to the image end, round it up by the + * bitmap granularity */ + gran_max_offset =3D ROUND_UP(max_offset, granularity); + } else { + gran_max_offset =3D max_offset; + } + + ret =3D hbitmap_iter_next(&iter->hbi, false); + if (ret < 0 || (ret << BDRV_SECTOR_BITS) + granularity > gran_max_offs= et) { + return false; + } + + *offset =3D ret << BDRV_SECTOR_BITS; + size =3D 0; + + assert(granularity <=3D INT_MAX); + + do { + /* Advance iterator */ + ret =3D hbitmap_iter_next(&iter->hbi, true); + size +=3D granularity; + } while ((ret << BDRV_SECTOR_BITS) + granularity <=3D gran_max_offset = && + hbitmap_iter_next(&iter->hbi, false) =3D=3D ret + sector_gran= && + size <=3D INT_MAX - granularity); + + *bytes =3D MIN(size, max_offset - *offset); + return true; +} + /* Called within bdrv_dirty_bitmap_lock..unlock */ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t cur_sector, int64_t nr_sectors) --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327819246115.25406208220522; Wed, 13 Sep 2017 11:36:59 -0700 (PDT) Received: from localhost ([::1]:44088 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCWw-0006Iq-Dy for importer@patchew.org; Wed, 13 Sep 2017 14:36:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37549) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCI1-0000pK-N4 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCI0-00069F-R8 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53036) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCHv-00063o-4q; Wed, 13 Sep 2017 14:21:27 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 412C2C04B954; Wed, 13 Sep 2017 18:21:26 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0E0946179D; Wed, 13 Sep 2017 18:21:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 412C2C04B954 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:05 +0200 Message-Id: <20170913181910.29688-14-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 13 Sep 2017 18:21:26 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 13/18] block/mirror: Keep write perm for pending writes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The owner of the mirror BDS might retire its write permission; but there may still be pending mirror operations so the mirror BDS cannot necessarily retire its write permission for its child then. Signed-off-by: Max Reitz --- block/mirror.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 05410c94ca..612fab660e 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1236,6 +1236,7 @@ static void bdrv_mirror_top_child_perm(BlockDriverSta= te *bs, BdrvChild *c, uint64_t *nperm, uint64_t *nshared) { MirrorBDSOpaque *s =3D bs->opaque; + bool ops_in_flight =3D s->job && !QTAILQ_EMPTY(&s->job->ops_in_flight); =20 if (s->job && s->job->exiting) { *nperm =3D 0; @@ -1243,9 +1244,10 @@ static void bdrv_mirror_top_child_perm(BlockDriverSt= ate *bs, BdrvChild *c, return; } =20 - /* Must be able to forward guest writes to the real image */ + /* Must be able to forward both new and pending guest writes to + * the real image */ *nperm =3D 0; - if (perm & BLK_PERM_WRITE) { + if ((perm & BLK_PERM_WRITE) || ops_in_flight) { *nperm |=3D BLK_PERM_WRITE; } =20 --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327474254808.8034611403731; Wed, 13 Sep 2017 11:31:14 -0700 (PDT) Received: from localhost ([::1]:44055 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCRN-000195-Et for importer@patchew.org; Wed, 13 Sep 2017 14:31:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37635) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCIC-0000zr-Ey for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCIB-0006PQ-J4 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:21:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42848) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCI7-0006HL-Um; Wed, 13 Sep 2017 14:21:40 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0E5EBC058EAF; Wed, 13 Sep 2017 18:21:39 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8915C18787; Wed, 13 Sep 2017 18:21:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0E5EBC058EAF Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:06 +0200 Message-Id: <20170913181910.29688-15-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 13 Sep 2017 18:21:39 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 14/18] block/mirror: Distinguish active from passive ops X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Currently, the mirror block job only knows passive operations. But once we introduce active writes, we need to distinguish between the two; for example, mirror_wait_for_free_in_flight_slot() should wait for a passive operation because active writes will not use the same in-flight slots. Signed-off-by: Max Reitz --- block/mirror.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 612fab660e..8fea619a68 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -96,6 +96,7 @@ struct MirrorOp { /* Set by mirror_co_read() before yielding for the first time */ uint64_t bytes_copied; =20 + bool is_active_write; CoQueue waiting_requests; =20 QTAILQ_ENTRY(MirrorOp) next; @@ -286,9 +287,14 @@ static inline void mirror_wait_for_free_in_flight_slot= (MirrorBlockJob *s) { MirrorOp *op; =20 - op =3D QTAILQ_FIRST(&s->ops_in_flight); - assert(op); - qemu_co_queue_wait(&op->waiting_requests, NULL); + QTAILQ_FOREACH(op, &s->ops_in_flight, next) { + if (!op->is_active_write) { + /* Only non-active operations use up in-flight slots */ + qemu_co_queue_wait(&op->waiting_requests, NULL); + return; + } + } + abort(); } =20 /* Submit async read while handling COW. --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327449726291.5875766089373; Wed, 13 Sep 2017 11:30:49 -0700 (PDT) Received: from localhost ([::1]:44054 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCQy-0000mL-Uo for importer@patchew.org; Wed, 13 Sep 2017 14:30:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37787) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCIT-00019T-L2 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:22:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCIR-0006aX-HP for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:22:01 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47016) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCIK-0006W6-DT; Wed, 13 Sep 2017 14:21:52 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8764F356F4; Wed, 13 Sep 2017 18:21:51 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A817769FBC; Wed, 13 Sep 2017 18:21:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8764F356F4 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:07 +0200 Message-Id: <20170913181910.29688-16-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 13 Sep 2017 18:21:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 15/18] block/mirror: Add active mirroring X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch implements active synchronous mirroring. In active mode, the passive mechanism will still be in place and is used to copy all initially dirty clusters off the source disk; but every write request will write data both to the source and the target disk, so the source cannot be dirtied faster than data is mirrored to the target. Also, once the block job has converged (BLOCK_JOB_READY sent), source and target are guaranteed to stay in sync (unless an error occurs). Optionally, dirty data can be copied to the target disk on read operations, too. Active mode is completely optional and currently disabled at runtime. A later patch will add a way for users to enable it. Signed-off-by: Max Reitz --- qapi/block-core.json | 23 +++++++ block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++= ++-- 2 files changed, 205 insertions(+), 5 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index bb11815608..e072cfa67c 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -938,6 +938,29 @@ 'data': ['top', 'full', 'none', 'incremental'] } =20 ## +# @MirrorCopyMode: +# +# An enumeration whose values tell the mirror block job when to +# trigger writes to the target. +# +# @passive: copy data in background only. +# +# @active-write: when data is written to the source, write it +# (synchronously) to the target as well. In addition, +# data is copied in background just like in @passive +# mode. +# +# @active-read-write: write data to the target (synchronously) both +# when it is read from and written to the source. +# In addition, data is copied in background just +# like in @passive mode. +# +# Since: 2.11 +## +{ 'enum': 'MirrorCopyMode', + 'data': ['passive', 'active-write', 'active-read-write'] } + +## # @BlockJobType: # # Type of a block job. diff --git a/block/mirror.c b/block/mirror.c index 8fea619a68..c429aa77bb 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -54,8 +54,12 @@ typedef struct MirrorBlockJob { Error *replace_blocker; bool is_none_mode; BlockMirrorBackingMode backing_mode; + MirrorCopyMode copy_mode; BlockdevOnError on_source_error, on_target_error; bool synced; + /* Set when the target is synced (dirty bitmap is clean, nothing + * in flight) and the job is running in active mode */ + bool actively_synced; bool should_complete; int64_t granularity; size_t buf_size; @@ -77,6 +81,7 @@ typedef struct MirrorBlockJob { int target_cluster_size; int max_iov; bool initial_zeroing_ongoing; + int in_active_write_counter; =20 /* Signals that we are no longer accessing source and target and the m= irror * BDS should thus relinquish all permissions */ @@ -112,6 +117,7 @@ static BlockErrorAction mirror_error_action(MirrorBlock= Job *s, bool read, int error) { s->synced =3D false; + s->actively_synced =3D false; if (read) { return block_job_error_action(&s->common, s->on_source_error, true, error); @@ -283,13 +289,12 @@ static int mirror_cow_align(MirrorBlockJob *s, int64_= t *offset, return ret; } =20 -static inline void mirror_wait_for_free_in_flight_slot(MirrorBlockJob *s) +static inline void mirror_wait_for_any_operation(MirrorBlockJob *s, bool a= ctive) { MirrorOp *op; =20 QTAILQ_FOREACH(op, &s->ops_in_flight, next) { - if (!op->is_active_write) { - /* Only non-active operations use up in-flight slots */ + if (op->is_active_write =3D=3D active) { qemu_co_queue_wait(&op->waiting_requests, NULL); return; } @@ -297,6 +302,12 @@ static inline void mirror_wait_for_free_in_flight_slot= (MirrorBlockJob *s) abort(); } =20 +static inline void mirror_wait_for_free_in_flight_slot(MirrorBlockJob *s) +{ + /* Only non-active operations use up in-flight slots */ + mirror_wait_for_any_operation(s, false); +} + /* Submit async read while handling COW. * Returns: The number of bytes copied after and including offset, * excluding any bytes copied prior to offset due to alignment. @@ -861,6 +872,7 @@ static void coroutine_fn mirror_run(void *opaque) /* Report BLOCK_JOB_READY and wait for complete. */ block_job_event_ready(&s->common); s->synced =3D true; + s->actively_synced =3D true; while (!block_job_is_cancelled(&s->common) && !s->should_complete)= { block_job_yield(&s->common); } @@ -912,6 +924,12 @@ static void coroutine_fn mirror_run(void *opaque) int64_t cnt, delta; bool should_complete; =20 + /* Do not start passive operations while there are active + * writes in progress */ + while (s->in_active_write_counter) { + mirror_wait_for_any_operation(s, true); + } + if (s->ret < 0) { ret =3D s->ret; goto immediate_exit; @@ -961,6 +979,9 @@ static void coroutine_fn mirror_run(void *opaque) */ block_job_event_ready(&s->common); s->synced =3D true; + if (s->copy_mode !=3D MIRROR_COPY_MODE_PASSIVE) { + s->actively_synced =3D true; + } } =20 should_complete =3D s->should_complete || @@ -1195,16 +1216,171 @@ static BdrvChildRole source_child_role =3D { .drained_end =3D source_child_cb_drained_end, }; =20 +static void do_sync_target_write(MirrorBlockJob *job, uint64_t offset, + uint64_t bytes, QEMUIOVector *qiov, int f= lags) +{ + BdrvDirtyBitmapIter *iter; + QEMUIOVector target_qiov; + uint64_t dirty_offset; + int dirty_bytes; + + qemu_iovec_init(&target_qiov, qiov->niov); + + iter =3D bdrv_dirty_iter_new(job->dirty_bitmap, offset >> BDRV_SECTOR_= BITS); + + while (true) { + bool valid_area; + int ret; + + bdrv_dirty_bitmap_lock(job->dirty_bitmap); + valid_area =3D bdrv_dirty_iter_next_area(iter, offset + bytes, + &dirty_offset, &dirty_bytes= ); + bdrv_dirty_bitmap_unlock(job->dirty_bitmap); + if (!valid_area) { + break; + } + + job->common.len +=3D dirty_bytes; + + assert(dirty_offset - offset <=3D SIZE_MAX); + if (qiov) { + qemu_iovec_reset(&target_qiov); + qemu_iovec_concat(&target_qiov, qiov, + dirty_offset - offset, dirty_bytes); + } + + ret =3D blk_co_pwritev(job->target, dirty_offset, dirty_bytes, + qiov ? &target_qiov : NULL, flags); + if (ret >=3D 0) { + assert(dirty_offset % BDRV_SECTOR_SIZE =3D=3D 0); + assert(dirty_bytes % BDRV_SECTOR_SIZE =3D=3D 0); + bdrv_reset_dirty_bitmap(job->dirty_bitmap, + dirty_offset >> BDRV_SECTOR_BITS, + dirty_bytes >> BDRV_SECTOR_BITS); + + job->common.offset +=3D dirty_bytes; + } else { + BlockErrorAction action; + + action =3D mirror_error_action(job, false, -ret); + if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { + if (!job->ret) { + job->ret =3D ret; + } + break; + } + } + } + + bdrv_dirty_iter_free(iter); + qemu_iovec_destroy(&target_qiov); +} + +static MirrorOp *coroutine_fn active_write_prepare(MirrorBlockJob *s, + uint64_t offset, + uint64_t bytes) +{ + MirrorOp *op; + uint64_t start_chunk =3D offset / s->granularity; + uint64_t end_chunk =3D DIV_ROUND_UP(offset + bytes, s->granularity); + + op =3D g_new(MirrorOp, 1); + *op =3D (MirrorOp){ + .s =3D s, + .offset =3D offset, + .bytes =3D bytes, + .is_active_write =3D true, + }; + qemu_co_queue_init(&op->waiting_requests); + QTAILQ_INSERT_TAIL(&s->ops_in_flight, op, next); + + s->in_active_write_counter++; + + mirror_wait_on_conflicts(op, s, offset, bytes); + + bitmap_set(s->in_flight_bitmap, start_chunk, end_chunk - start_chunk); + + return op; +} + +static void coroutine_fn active_write_settle(MirrorOp *op) +{ + uint64_t start_chunk =3D op->offset / op->s->granularity; + uint64_t end_chunk =3D DIV_ROUND_UP(op->offset + op->bytes, + op->s->granularity); + + if (!--op->s->in_active_write_counter && op->s->actively_synced) { + /* Assert that we are back in sync once all active write + * operations are settled */ + assert(!bdrv_get_dirty_count(op->s->dirty_bitmap)); + } + bitmap_clear(op->s->in_flight_bitmap, start_chunk, end_chunk - start_c= hunk); + QTAILQ_REMOVE(&op->s->ops_in_flight, op, next); + qemu_co_queue_restart_all(&op->waiting_requests); + g_free(op); +} + static int coroutine_fn bdrv_mirror_top_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags); + MirrorOp *op =3D NULL; + MirrorBDSOpaque *s =3D bs->opaque; + int ret =3D 0; + bool copy_to_target; + + copy_to_target =3D s->job->ret >=3D 0 && + s->job->copy_mode =3D=3D MIRROR_COPY_MODE_ACTIVE_READ= _WRITE; + + if (copy_to_target) { + op =3D active_write_prepare(s->job, offset, bytes); + } + + ret =3D bdrv_co_preadv(bs->file, offset, bytes, qiov, flags); + if (ret < 0) { + goto out; + } + + if (copy_to_target) { + do_sync_target_write(s->job, offset, bytes, qiov, 0); + } + +out: + if (copy_to_target) { + active_write_settle(op); + } + return ret; } =20 static int coroutine_fn bdrv_mirror_top_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags); + MirrorOp *op =3D NULL; + MirrorBDSOpaque *s =3D bs->opaque; + int ret =3D 0; + bool copy_to_target; + + copy_to_target =3D s->job->ret >=3D 0 && + (s->job->copy_mode =3D=3D MIRROR_COPY_MODE_ACTIVE_WRI= TE || + s->job->copy_mode =3D=3D MIRROR_COPY_MODE_ACTIVE_REA= D_WRITE); + + if (copy_to_target) { + op =3D active_write_prepare(s->job, offset, bytes); + } + + ret =3D bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags); + if (ret < 0) { + goto out; + } + + if (copy_to_target) { + do_sync_target_write(s->job, offset, bytes, qiov, flags); + } + +out: + if (copy_to_target) { + active_write_settle(op); + } + return ret; } =20 static int coroutine_fn bdrv_mirror_top_flush(BlockDriverState *bs) @@ -1398,6 +1574,7 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, s->on_target_error =3D on_target_error; s->is_none_mode =3D is_none_mode; s->backing_mode =3D backing_mode; + s->copy_mode =3D MIRROR_COPY_MODE_PASSIVE; s->base =3D base; s->granularity =3D granularity; s->buf_size =3D ROUND_UP(buf_size, granularity); --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327657992597.075989575105; Wed, 13 Sep 2017 11:34:17 -0700 (PDT) Received: from localhost ([::1]:44071 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCUL-0003yD-2X for importer@patchew.org; Wed, 13 Sep 2017 14:34:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37893) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCIa-0001FP-E2 for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:22:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCIW-0006e5-8e for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:22:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46316) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCIS-0006ar-O8; Wed, 13 Sep 2017 14:22:01 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D5273C0587ED; Wed, 13 Sep 2017 18:21:59 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A43775D6AE; Wed, 13 Sep 2017 18:21:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D5273C0587ED Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:08 +0200 Message-Id: <20170913181910.29688-17-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 13 Sep 2017 18:21:59 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 16/18] block/mirror: Add copy mode QAPI interface X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch allows the user to specify whether to use active or only passive mode for mirror block jobs. Currently, this setting will remain constant for the duration of the entire block job. Signed-off-by: Max Reitz --- qapi/block-core.json | 11 +++++++++-- include/block/block_int.h | 4 +++- block/mirror.c | 11 ++++++----- blockdev.c | 9 ++++++++- 4 files changed, 26 insertions(+), 9 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index e072cfa67c..40204d367a 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1578,6 +1578,9 @@ # written. Both will result in identical contents. # Default is true. (Since 2.4) # +# @copy-mode: when to copy data to the destination; defaults to 'passive' +# (Since: 2.11) +# # Since: 1.3 ## { 'struct': 'DriveMirror', @@ -1587,7 +1590,7 @@ '*speed': 'int', '*granularity': 'uint32', '*buf-size': 'int', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', - '*unmap': 'bool' } } + '*unmap': 'bool', '*copy-mode': 'MirrorCopyMode' } } =20 ## # @BlockDirtyBitmap: @@ -1766,6 +1769,9 @@ # above @device. If this option is not given, a node na= me is # autogenerated. (Since: 2.9) # +# @copy-mode: when to copy data to the destination; defaults to 'passive' +# (Since: 2.11) +# # Returns: nothing on success. # # Since: 2.6 @@ -1786,7 +1792,8 @@ '*speed': 'int', '*granularity': 'uint32', '*buf-size': 'int', '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', - '*filter-node-name': 'str' } } + '*filter-node-name': 'str', + '*copy-mode': 'MirrorCopyMode' } } =20 ## # @block_set_io_throttle: diff --git a/include/block/block_int.h b/include/block/block_int.h index fa8bbf1f8b..517b2680ce 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -934,6 +934,7 @@ void commit_active_start(const char *job_id, BlockDrive= rState *bs, * @filter_node_name: The node name that should be assigned to the filter * driver that the mirror job inserts into the graph above @bs. NULL means= that * a node name should be autogenerated. + * @copy_mode: When to trigger writes to the target. * @errp: Error object. * * Start a mirroring operation on @bs. Clusters that are allocated @@ -947,7 +948,8 @@ void mirror_start(const char *job_id, BlockDriverState = *bs, MirrorSyncMode mode, BlockMirrorBackingMode backing_mode, BlockdevOnError on_source_error, BlockdevOnError on_target_error, - bool unmap, const char *filter_node_name, Error **errp); + bool unmap, const char *filter_node_name, + MirrorCopyMode copy_mode, Error **errp); =20 /* * backup_job_create: diff --git a/block/mirror.c b/block/mirror.c index c429aa77bb..8a67935cc4 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1464,7 +1464,7 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, const BlockJobDriver *driver, bool is_none_mode, BlockDriverState *base, bool auto_complete, const char *filter_node_n= ame, - bool is_mirror, + bool is_mirror, MirrorCopyMode copy_mode, Error **errp) { MirrorBlockJob *s; @@ -1574,7 +1574,7 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, s->on_target_error =3D on_target_error; s->is_none_mode =3D is_none_mode; s->backing_mode =3D backing_mode; - s->copy_mode =3D MIRROR_COPY_MODE_PASSIVE; + s->copy_mode =3D copy_mode; s->base =3D base; s->granularity =3D granularity; s->buf_size =3D ROUND_UP(buf_size, granularity); @@ -1643,7 +1643,8 @@ void mirror_start(const char *job_id, BlockDriverStat= e *bs, MirrorSyncMode mode, BlockMirrorBackingMode backing_mode, BlockdevOnError on_source_error, BlockdevOnError on_target_error, - bool unmap, const char *filter_node_name, Error **errp) + bool unmap, const char *filter_node_name, + MirrorCopyMode copy_mode, Error **errp) { bool is_none_mode; BlockDriverState *base; @@ -1658,7 +1659,7 @@ void mirror_start(const char *job_id, BlockDriverStat= e *bs, speed, granularity, buf_size, backing_mode, on_source_error, on_target_error, unmap, NULL, NULL, &mirror_job_driver, is_none_mode, base, false, - filter_node_name, true, errp); + filter_node_name, true, copy_mode, errp); } =20 void commit_active_start(const char *job_id, BlockDriverState *bs, @@ -1681,7 +1682,7 @@ void commit_active_start(const char *job_id, BlockDri= verState *bs, MIRROR_LEAVE_BACKING_CHAIN, on_error, on_error, true, cb, opaque, &commit_active_job_driver, false, base, auto_complete, - filter_node_name, false, &local_err); + filter_node_name, false, MIRROR_COPY_MODE_PASSIVE, &l= ocal_err); if (local_err) { error_propagate(errp, local_err); goto error_restore_flags; diff --git a/blockdev.c b/blockdev.c index 56a6b24a0b..7f9c215e98 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3408,6 +3408,7 @@ static void blockdev_mirror_common(const char *job_id= , BlockDriverState *bs, bool has_unmap, bool unmap, bool has_filter_node_name, const char *filter_node_name, + bool has_copy_mode, MirrorCopyMode copy= _mode, Error **errp) { =20 @@ -3432,6 +3433,9 @@ static void blockdev_mirror_common(const char *job_id= , BlockDriverState *bs, if (!has_filter_node_name) { filter_node_name =3D NULL; } + if (!has_copy_mode) { + copy_mode =3D MIRROR_COPY_MODE_PASSIVE; + } =20 if (granularity !=3D 0 && (granularity < 512 || granularity > 1048576 = * 64)) { error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "granularity", @@ -3462,7 +3466,7 @@ static void blockdev_mirror_common(const char *job_id= , BlockDriverState *bs, has_replaces ? replaces : NULL, speed, granularity, buf_size, sync, backing_mode, on_source_error, on_target_error, unmap, filter_node_name, - errp); + copy_mode, errp); } =20 void qmp_drive_mirror(DriveMirror *arg, Error **errp) @@ -3603,6 +3607,7 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp) arg->has_on_target_error, arg->on_target_error, arg->has_unmap, arg->unmap, false, NULL, + arg->has_copy_mode, arg->copy_mode, &local_err); bdrv_unref(target_bs); error_propagate(errp, local_err); @@ -3623,6 +3628,7 @@ void qmp_blockdev_mirror(bool has_job_id, const char = *job_id, BlockdevOnError on_target_error, bool has_filter_node_name, const char *filter_node_name, + bool has_copy_mode, MirrorCopyMode copy_mode, Error **errp) { BlockDriverState *bs; @@ -3655,6 +3661,7 @@ void qmp_blockdev_mirror(bool has_job_id, const char = *job_id, has_on_target_error, on_target_error, true, true, has_filter_node_name, filter_node_name, + has_copy_mode, copy_mode, &local_err); error_propagate(errp, local_err); =20 --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327804589283.0438023321916; Wed, 13 Sep 2017 11:36:44 -0700 (PDT) Received: from localhost ([::1]:44087 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCWg-00068A-Hy for importer@patchew.org; Wed, 13 Sep 2017 14:36:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37949) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCId-0001JG-4v for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:22:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCIb-0006hg-UX for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:22:11 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47404) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCIV-0006cd-7K; Wed, 13 Sep 2017 14:22:03 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 61CF7356D9; Wed, 13 Sep 2017 18:22:02 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D05265D6AE; Wed, 13 Sep 2017 18:22:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 61CF7356D9 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:09 +0200 Message-Id: <20170913181910.29688-18-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 13 Sep 2017 18:22:02 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 17/18] qemu-io: Add background write X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add a new parameter -B to qemu-io's write command. When used, qemu-io will not wait for the result of the operation and instead execute it in the background. Signed-off-by: Max Reitz --- qemu-io-cmds.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++++-= ---- 1 file changed, 77 insertions(+), 6 deletions(-) diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index 2811a89099..c635a248f5 100644 --- a/qemu-io-cmds.c +++ b/qemu-io-cmds.c @@ -481,6 +481,62 @@ static int do_pwrite(BlockBackend *blk, char *buf, int= 64_t offset, typedef struct { BlockBackend *blk; int64_t offset; + int bytes; + char *buf; + int flags; +} CoBackgroundWrite; + +static void coroutine_fn co_background_pwrite_entry(void *opaque) +{ + CoBackgroundWrite *data =3D opaque; + QEMUIOVector qiov; + int ret; + + qemu_iovec_init(&qiov, 1); + qemu_iovec_add(&qiov, data->buf, data->bytes); + + ret =3D blk_co_pwritev(data->blk, data->offset, data->bytes, &qiov, + data->flags); + + qemu_iovec_destroy(&qiov); + g_free(data->buf); + + if (ret < 0) { + Error *err; + error_setg_errno(&err, -ret, "Background write failed"); + error_report_err(err); + } +} + +/* Takes ownership of @buf */ +static int do_background_pwrite(BlockBackend *blk, char *buf, int64_t offs= et, + int64_t bytes, int flags) +{ + Coroutine *co; + CoBackgroundWrite *data; + + if (bytes > INT_MAX) { + return -ERANGE; + } + + data =3D g_new(CoBackgroundWrite, 1); + *data =3D (CoBackgroundWrite){ + .blk =3D blk, + .offset =3D offset, + .bytes =3D bytes, + .buf =3D buf, + .flags =3D flags, + }; + + co =3D qemu_coroutine_create(co_background_pwrite_entry, data); + bdrv_coroutine_enter(blk_bs(blk), co); + + return bytes; +} + +typedef struct { + BlockBackend *blk; + int64_t offset; int64_t bytes; int64_t *total; int flags; @@ -931,6 +987,7 @@ static void write_help(void) " Writes into a segment of the currently open file, using a buffer\n" " filled with a set pattern (0xcdcdcdcd).\n" " -b, -- write to the VM state rather than the virtual disk\n" +" -B, -- just start a background write, do not wait for the result\n" " -c, -- write compressed data with blk_write_compressed\n" " -f, -- use Force Unit Access semantics\n" " -p, -- ignored for backwards compatibility\n" @@ -951,7 +1008,7 @@ static const cmdinfo_t write_cmd =3D { .perm =3D BLK_PERM_WRITE, .argmin =3D 2, .argmax =3D -1, - .args =3D "[-bcCfquz] [-P pattern] off len", + .args =3D "[-bBcCfquz] [-P pattern] off len", .oneline =3D "writes a number of bytes at a specified offset", .help =3D write_help, }; @@ -961,6 +1018,7 @@ static int write_f(BlockBackend *blk, int argc, char *= *argv) struct timeval t1, t2; bool Cflag =3D false, qflag =3D false, bflag =3D false; bool Pflag =3D false, zflag =3D false, cflag =3D false; + bool background =3D false; int flags =3D 0; int c, cnt; char *buf =3D NULL; @@ -970,11 +1028,14 @@ static int write_f(BlockBackend *blk, int argc, char= **argv) int64_t total =3D 0; int pattern =3D 0xcd; =20 - while ((c =3D getopt(argc, argv, "bcCfpP:quz")) !=3D -1) { + while ((c =3D getopt(argc, argv, "bBcCfpP:quz")) !=3D -1) { switch (c) { case 'b': bflag =3D true; break; + case 'B': + background =3D true; + break; case 'c': cflag =3D true; break; @@ -1032,6 +1093,11 @@ static int write_f(BlockBackend *blk, int argc, char= **argv) return 0; } =20 + if (background && (bflag || cflag || zflag)) { + printf("-B cannot be specified together with -b, -c, or -z\n"); + return 0; + } + offset =3D cvtnum(argv[optind]); if (offset < 0) { print_cvtnum_err(offset, argv[optind]); @@ -1074,6 +1140,8 @@ static int write_f(BlockBackend *blk, int argc, char = **argv) cnt =3D do_co_pwrite_zeroes(blk, offset, count, flags, &total); } else if (cflag) { cnt =3D do_write_compressed(blk, buf, offset, count, &total); + } else if (background) { + cnt =3D do_background_pwrite(blk, buf, offset, count, flags); } else { cnt =3D do_pwrite(blk, buf, offset, count, flags, &total); } @@ -1088,12 +1156,15 @@ static int write_f(BlockBackend *blk, int argc, cha= r **argv) goto out; } =20 - /* Finally, report back -- -C gives a parsable format */ - t2 =3D tsub(t2, t1); - print_report("wrote", &t2, offset, count, total, cnt, Cflag); + if (!background) { + /* Finally, report back -- -C gives a parsable format */ + t2 =3D tsub(t2, t1); + print_report("wrote", &t2, offset, count, total, cnt, Cflag); + } =20 out: - if (!zflag) { + /* do_background_pwrite() takes ownership of the buffer */ + if (!zflag && !background) { qemu_io_free(buf); } =20 --=20 2.13.5 From nobody Sat May 4 05:51:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505327634536399.0107204527625; Wed, 13 Sep 2017 11:33:54 -0700 (PDT) Received: from localhost ([::1]:44069 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCTw-0003es-TJ for importer@patchew.org; Wed, 13 Sep 2017 14:33:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38017) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsCIm-0001T4-HX for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:22:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsCIl-0006rI-4K for qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:22:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52766) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dsCIe-0006lJ-Dt; Wed, 13 Sep 2017 14:22:12 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7E3A580F94; Wed, 13 Sep 2017 18:22:11 +0000 (UTC) Received: from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 74FC560600; Wed, 13 Sep 2017 18:22:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 7E3A580F94 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=mreitz@redhat.com From: Max Reitz To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 20:19:10 +0200 Message-Id: <20170913181910.29688-19-mreitz@redhat.com> In-Reply-To: <20170913181910.29688-1-mreitz@redhat.com> References: <20170913181910.29688-1-mreitz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 13 Sep 2017 18:22:11 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 18/18] iotests: Add test for active mirroring X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-devel@nongnu.org, Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Max Reitz --- tests/qemu-iotests/151 | 111 +++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/151.out | 5 ++ tests/qemu-iotests/group | 1 + 3 files changed, 117 insertions(+) create mode 100755 tests/qemu-iotests/151 create mode 100644 tests/qemu-iotests/151.out diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151 new file mode 100755 index 0000000000..49a60773f9 --- /dev/null +++ b/tests/qemu-iotests/151 @@ -0,0 +1,111 @@ +#!/usr/bin/env python +# +# Tests for active mirroring +# +# Copyright (C) 2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import os +import iotests +from iotests import qemu_img + +source_img =3D os.path.join(iotests.test_dir, 'source.' + iotests.imgfmt) +target_img =3D os.path.join(iotests.test_dir, 'target.' + iotests.imgfmt) + +class TestActiveMirror(iotests.QMPTestCase): + image_len =3D 128 * 1024 * 1024 # MB + potential_writes_in_flight =3D True + + def setUp(self): + qemu_img('create', '-f', iotests.imgfmt, source_img, '128M') + qemu_img('create', '-f', iotests.imgfmt, target_img, '128M') + + blk_source =3D {'node-name': 'source', + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', + 'filename': source_img}} + + blk_target =3D {'node-name': 'target', + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', + 'filename': target_img}} + + self.vm =3D iotests.VM() + self.vm.add_blockdev(self.qmp_to_opts(blk_source)) + self.vm.add_blockdev(self.qmp_to_opts(blk_target)) + self.vm.launch() + + def tearDown(self): + self.vm.shutdown() + + if not self.potential_writes_in_flight: + self.assertTrue(iotests.compare_images(source_img, target_img), + 'mirror target does not match source') + + os.remove(source_img) + os.remove(target_img) + + def doActiveIO(self, sync_source_and_target): + # Fill the source image + self.vm.hmp_qemu_io('source', + 'write -P 1 0 %i' % self.image_len); + + # Start some background requests + for offset in range(0, self.image_len, 1024 * 1024): + self.vm.hmp_qemu_io('source', 'write -B -P 2 %i 1M' % offset) + + # Start the block job + result =3D self.vm.qmp('blockdev-mirror', + job_id=3D'mirror', + filter_node_name=3D'mirror-node', + device=3D'source', + target=3D'target', + sync=3D'full', + copy_mode=3D'active-write') + self.assert_qmp(result, 'return', {}) + + # Start some more requests + for offset in range(0, self.image_len, 1024 * 1024): + self.vm.hmp_qemu_io('mirror-node', 'write -B -P 3 %i 1M' % off= set) + + # Wait for the READY event + self.wait_ready(drive=3D'mirror') + + # Now start some final requests; all of these (which land on + # the source) should be settled using the active mechanism. + # The mirror code itself asserts that the source BDS's dirty + # bitmap will stay clean between READY and COMPLETED. + for offset in range(0, self.image_len, 1024 * 1024): + self.vm.hmp_qemu_io('mirror-node', 'write -B -P 4 %i 1M' % off= set) + + if sync_source_and_target: + # If source and target should be in sync after the mirror, + # we have to flush before completion + self.vm.hmp_qemu_io('mirror-node', 'flush') + self.potential_writes_in_flight =3D False + + self.complete_and_wait(drive=3D'mirror', wait_ready=3DFalse) + + def testActiveIO(self): + self.doActiveIO(False) + + def testActiveIOFlushed(self): + self.doActiveIO(True) + + + +if __name__ =3D=3D '__main__': + iotests.main(supported_fmts=3D['qcow2', 'raw']) diff --git a/tests/qemu-iotests/151.out b/tests/qemu-iotests/151.out new file mode 100644 index 0000000000..fbc63e62f8 --- /dev/null +++ b/tests/qemu-iotests/151.out @@ -0,0 +1,5 @@ +.. +---------------------------------------------------------------------- +Ran 2 tests + +OK diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 94e764865a..c64adbe5bf 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -156,6 +156,7 @@ 148 rw auto quick 149 rw auto sudo 150 rw auto quick +151 rw auto 152 rw auto quick 153 rw auto quick 154 rw auto backing quick --=20 2.13.5