From nobody Fri May 3 01:35:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522321914075405.54921840503243; Thu, 29 Mar 2018 04:11:54 -0700 (PDT) Received: from localhost ([::1]:41662 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VTE-0002vu-QT for importer@patchew.org; Thu, 29 Mar 2018 07:11:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60630) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VR5-0001V4-Va for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1VR4-00037l-V0 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:39 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:57638 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1VQy-000359-GZ; Thu, 29 Mar 2018 07:09:32 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 94CEA4070212; Thu, 29 Mar 2018 11:09:31 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-85.pek2.redhat.com [10.72.12.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60FC42026DFD; Thu, 29 Mar 2018 11:09:28 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 19:09:07 +0800 Message-Id: <20180329110914.20888-2-famz@redhat.com> In-Reply-To: <20180329110914.20888-1-famz@redhat.com> References: <20180329110914.20888-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 29 Mar 2018 11:09:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 29 Mar 2018 11:09:31 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC PATCH 1/8] block: Introduce bdrv_co_map_range API X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" A little similar to bdrv_co_block_status but with the capability to do allocation depending on the BDRV_REQ_ALLOCATE flag, this API is the building block needed by copy offloading to work for format drivers. The idea is the format drivers return "where" to copy from/to in its underlying "file", then the block layer will drill down until it reaches a protocol layer that can do offloaded copying. Signed-off-by: Fam Zheng --- block/io.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ include/block/block.h | 8 +++++++- include/block/block_int.h | 21 +++++++++++++++++++++ 3 files changed, 72 insertions(+), 1 deletion(-) diff --git a/block/io.c b/block/io.c index bd9a19a9c4..1b4cfcacb1 100644 --- a/block/io.c +++ b/block/io.c @@ -2826,3 +2826,47 @@ void bdrv_unregister_buf(BlockDriverState *bs, void = *host) bdrv_unregister_buf(child->bs, host); } } + +int coroutine_fn bdrv_co_map_range(BdrvChild *child, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *= map, + BlockDriverState **file, + int flags) +{ + BlockDriverState *bs =3D child->bs; + + if (bs->encrypted) { + return -ENOTSUP; + } + while (bs && bs->drv && bs->drv->bdrv_co_map_range) { + int64_t cur_pnum, cur_map; + BlockDriverState *cur_file =3D NULL; + int r =3D bs->drv->bdrv_co_map_range(bs, offset, bytes, + &cur_pnum, &cur_map, &cur_file, + flags); + if (r < 0) { + return r; + } + offset =3D cur_map; + bytes =3D cur_pnum; + if (pnum) { + *pnum =3D cur_pnum; + } + if (map) { + *map =3D cur_map; + } + if (!(r & BDRV_BLOCK_OFFSET_VALID)) { + return -ENOTSUP; + } + if (file) { + *file =3D cur_file; + } + if (!cur_file) { + return -ENOTSUP; + } + if (cur_file =3D=3D bs) { + return r; + } + bs =3D cur_file; + } + return -ENOTSUP; +} diff --git a/include/block/block.h b/include/block/block.h index cdec3639a3..21d3e9db32 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -53,9 +53,10 @@ typedef enum { BDRV_REQ_NO_SERIALISING =3D 0x8, BDRV_REQ_FUA =3D 0x10, BDRV_REQ_WRITE_COMPRESSED =3D 0x20, + BDRV_REQ_ALLOCATE =3D 0x40, =20 /* Mask of valid flags */ - BDRV_REQ_MASK =3D 0x3f, + BDRV_REQ_MASK =3D 0x7f, } BdrvRequestFlags; =20 typedef struct BlockSizes { @@ -604,4 +605,9 @@ bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *= bs, const char *name, */ void bdrv_register_buf(BlockDriverState *bs, void *host, size_t size); void bdrv_unregister_buf(BlockDriverState *bs, void *host); + +int bdrv_co_map_range(BdrvChild *child, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *map, + BlockDriverState **file, + int flags); #endif diff --git a/include/block/block_int.h b/include/block/block_int.h index c4dd1d4bb8..3f3f6f3efd 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -475,6 +475,27 @@ struct BlockDriver { */ void (*bdrv_register_buf)(BlockDriverState *bs, void *host, size_t siz= e); void (*bdrv_unregister_buf)(BlockDriverState *bs, void *host); + + /** + * Get or create offset mappings for a virtual range. + * + * @offset: the offset of the beginning of the range. + * @bytes: the maximum bytes of the range to map. + * @pnum: at return, this will point to the number of bytes in the ret= urned + * mapping status + * @map: at return, this will point to the offset which the range maps= to + * @file: at return, this will point to the file where the data is map= ped. + * If it is set to bs, it means the mapping is terminal and the result= can + * be used for read/write and copy_range; if it is NULL, it means the = range + * doesn't map to a file, or no I/O to any file is necessary; otherwis= e, it + * means the query should be passed down to an underlying format/proto= col. + * @flags: flags for the request. BDRV_REQ_ALLOCATE means the range + * should be allocated if necessary. + */ + int (*bdrv_co_map_range)(BlockDriverState *bs, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *map, + BlockDriverState **file, + int flags); QLIST_ENTRY(BlockDriver) list; }; =20 --=20 2.14.3 From nobody Fri May 3 01:35:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522321927302391.64619463990164; Thu, 29 Mar 2018 04:12:07 -0700 (PDT) Received: from localhost ([::1]:41663 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VTS-00037c-Es for importer@patchew.org; Thu, 29 Mar 2018 07:12:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60624) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VR5-0001Uo-NG for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1VR4-00037b-IO for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:39 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:57644 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1VR1-00036F-QQ; Thu, 29 Mar 2018 07:09:35 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7CD024070203; Thu, 29 Mar 2018 11:09:35 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-85.pek2.redhat.com [10.72.12.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 352EB2026DFD; Thu, 29 Mar 2018 11:09:31 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 19:09:08 +0800 Message-Id: <20180329110914.20888-3-famz@redhat.com> In-Reply-To: <20180329110914.20888-1-famz@redhat.com> References: <20180329110914.20888-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 29 Mar 2018 11:09:35 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 29 Mar 2018 11:09:35 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC PATCH 2/8] qcow2: Implement bdrv_co_map_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Extract the common code from qcow2_co_block_status, then use it to implement qcow2_co_map_range. The major difference is the allocation logic, enabled by BDRV_REQ_ALLOCATE. Signed-off-by: Fam Zheng --- block/qcow2.c | 103 +++++++++++++++++++++++++++++++++++++++++-------------= ---- 1 file changed, 73 insertions(+), 30 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 486f3e83b7..86bbeb69db 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1755,6 +1755,73 @@ static int coroutine_fn qcow2_co_block_status(BlockD= riverState *bs, return status; } =20 +static int qcow2_handle_l2meta(BlockDriverState *bs, QCowL2Meta *l2meta) +{ + int ret =3D 0; + + while (l2meta !=3D NULL) { + QCowL2Meta *next; + + if (!ret) { + ret =3D qcow2_alloc_cluster_link_l2(bs, l2meta); + } + + /* Take the request off the list of running requests */ + if (l2meta->nb_clusters !=3D 0) { + QLIST_REMOVE(l2meta, next_in_flight); + } + + qemu_co_queue_restart_all(&l2meta->dependent_requests); + + next =3D l2meta->next; + g_free(l2meta); + l2meta =3D next; + } + return ret; +} + +static int coroutine_fn qcow2_co_map_range(BlockDriverState *bs, int64_t o= ffset, + int64_t bytes, int64_t *pnum, i= nt64_t *map, + BlockDriverState **file, + int flags) +{ + BDRVQcow2State *s =3D bs->opaque; + int ret; + uint64_t cluster_offset; + unsigned int cur_bytes =3D MIN(bytes, INT_MAX); + int offset_in_cluster; + QCowL2Meta *l2meta =3D NULL; + + if (!(flags & BDRV_REQ_ALLOCATE)) { + return qcow2_co_block_status(bs, true, offset, bytes, pnum, map, f= ile); + } + qemu_co_mutex_lock(&s->lock); + ret =3D qcow2_alloc_cluster_offset(bs, offset, &cur_bytes, + &cluster_offset, &l2meta); + if (ret < 0) { + goto out; + } + + offset_in_cluster =3D offset_into_cluster(s, offset); + ret =3D qcow2_pre_write_overlap_check(bs, 0, + cluster_offset + offset_in_cluster, + cur_bytes); + if (ret < 0) { + goto out; + } + ret =3D qcow2_handle_l2meta(bs, l2meta); + if (ret < 0) { + goto out; + } + *pnum =3D cur_bytes; + *map =3D cluster_offset + offset_in_cluster; + *file =3D bs->file->bs; + ret =3D BDRV_BLOCK_OFFSET_VALID | BDRV_BLOCK_DATA; +out: + qemu_co_mutex_unlock(&s->lock); + return ret; +} + static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t off= set, uint64_t bytes, QEMUIOVector *qiov, int flags) @@ -2041,24 +2108,10 @@ static coroutine_fn int qcow2_co_pwritev(BlockDrive= rState *bs, uint64_t offset, } } =20 - while (l2meta !=3D NULL) { - QCowL2Meta *next; - - ret =3D qcow2_alloc_cluster_link_l2(bs, l2meta); - if (ret < 0) { - goto fail; - } - - /* Take the request off the list of running requests */ - if (l2meta->nb_clusters !=3D 0) { - QLIST_REMOVE(l2meta, next_in_flight); - } - - qemu_co_queue_restart_all(&l2meta->dependent_requests); - - next =3D l2meta->next; - g_free(l2meta); - l2meta =3D next; + ret =3D qcow2_handle_l2meta(bs, l2meta); + l2meta =3D NULL; + if (ret) { + goto fail; } =20 bytes -=3D cur_bytes; @@ -2069,18 +2122,7 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriver= State *bs, uint64_t offset, ret =3D 0; =20 fail: - while (l2meta !=3D NULL) { - QCowL2Meta *next; - - if (l2meta->nb_clusters !=3D 0) { - QLIST_REMOVE(l2meta, next_in_flight); - } - qemu_co_queue_restart_all(&l2meta->dependent_requests); - - next =3D l2meta->next; - g_free(l2meta); - l2meta =3D next; - } + qcow2_handle_l2meta(bs, l2meta); =20 qemu_co_mutex_unlock(&s->lock); =20 @@ -4508,6 +4550,7 @@ BlockDriver bdrv_qcow2 =3D { .bdrv_co_create =3D qcow2_co_create, .bdrv_has_zero_init =3D bdrv_has_zero_init_1, .bdrv_co_block_status =3D qcow2_co_block_status, + .bdrv_co_map_range =3D qcow2_co_map_range, =20 .bdrv_co_preadv =3D qcow2_co_preadv, .bdrv_co_pwritev =3D qcow2_co_pwritev, --=20 2.14.3 From nobody Fri May 3 01:35:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 152232194259547.480295710141036; Thu, 29 Mar 2018 04:12:22 -0700 (PDT) Received: from localhost ([::1]:41664 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VTh-0003Jl-D2 for importer@patchew.org; Thu, 29 Mar 2018 07:12:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60690) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VRE-0001eb-VR for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1VR9-00039T-V2 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:48 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46460 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1VR7-00038a-0L; Thu, 29 Mar 2018 07:09:41 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A9C554276E3F; Thu, 29 Mar 2018 11:09:40 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-85.pek2.redhat.com [10.72.12.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id AA5122024CA4; Thu, 29 Mar 2018 11:09:36 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 19:09:09 +0800 Message-Id: <20180329110914.20888-4-famz@redhat.com> In-Reply-To: <20180329110914.20888-1-famz@redhat.com> References: <20180329110914.20888-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Thu, 29 Mar 2018 11:09:40 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Thu, 29 Mar 2018 11:09:40 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC PATCH 3/8] block: Introduce bdrv_co_copy_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This tries to do offloaded data copying in the same procotol (but possibly a different BDS), or fall back to a bounce buffer if not supported. Signed-off-by: Fam Zheng --- block/io.c | 148 ++++++++++++++++++++++++++++++++++++++++++= ++++ include/block/block.h | 4 ++ include/block/block_int.h | 14 +++++ 3 files changed, 166 insertions(+) diff --git a/block/io.c b/block/io.c index 1b4cfcacb1..64cd3a5170 100644 --- a/block/io.c +++ b/block/io.c @@ -2870,3 +2870,151 @@ int coroutine_fn bdrv_co_map_range(BdrvChild *child= , int64_t offset, } return -ENOTSUP; } + +typedef struct { + BdrvChild *child; + int64_t offset; + int64_t remaining; + BlockDriverState *file; + int64_t map; + int64_t pnum; + int status; +} BdrvCopyRangeState; + +static inline void bdrv_range_advance(BdrvCopyRangeState *s, int bytes) +{ + s->offset +=3D bytes; + s->remaining -=3D bytes; + assert(s->remaining >=3D 0); + s->map =3D 0; + s->pnum =3D MAX(0, s->pnum - bytes); + s->file =3D NULL; +} + +static int coroutine_fn +bdrv_co_copy_range_iteration(BdrvCopyRangeState *in, BdrvCopyRangeState *o= ut) +{ + int ret; + int bytes =3D MIN(in->remaining, out->remaining); + QEMUIOVector qiov; + BlockDriverState *in_bs =3D in->child->bs; + BlockDriverState *out_bs =3D out->child->bs; + uint8_t *buf; + + if (!in->pnum) { + in->status =3D bdrv_co_map_range(in->child, in->offset, in->remain= ing, + &in->pnum, &in->map, &in->file, 0); + } + if (in->status < 0 || !in->file || + in->offset + bytes > bdrv_getlength(in->file)) { + goto fallback; + } + + assert(in->pnum > 0); + bytes =3D MIN(bytes, in->pnum); + if (in->status & BDRV_BLOCK_ZERO) { + ret =3D bdrv_co_pwrite_zeroes(out->child, out->offset, in->pnum, 0= ); + if (ret) { + return ret; + } + bdrv_range_advance(in, bytes); + bdrv_range_advance(out, bytes); + return 0; + } + + if (!(in->status & BDRV_BLOCK_OFFSET_VALID)) { + goto fallback; + } + + out->status =3D bdrv_co_map_range(out->child, out->offset, out->remain= ing, + &out->pnum, &out->map, &out->file, + BDRV_REQ_ALLOCATE); + if (out->status < 0 || !out->file || + !(out->status & BDRV_BLOCK_OFFSET_VALID)) { + goto fallback; + } + + bytes =3D MIN(bytes, out->pnum); + + if (in->file->drv->bdrv_co_copy_range) { + if (!in->file->drv->bdrv_co_copy_range(in->file, in->map, + out->file, out->map, + bytes)) { + bdrv_range_advance(in, bytes); + bdrv_range_advance(out, bytes); + return 0; + } + } + /* At this point we could maximize bytes again */ + bytes =3D MIN(in->remaining, out->remaining); + +fallback: + buf =3D qemu_try_memalign(MAX(bdrv_opt_mem_align(in_bs), + bdrv_opt_mem_align(out_bs)), + bytes); + if (!buf) { + return -ENOMEM; + } + + qemu_iovec_init(&qiov, 1); + qemu_iovec_add(&qiov, buf, bytes); + + ret =3D bdrv_co_preadv(in->child, in->offset, bytes, &qiov, 0); + if (ret) { + goto out; + } + ret =3D bdrv_co_pwritev(out->child, out->offset, bytes, &qiov, 0); + if (!ret) { + bdrv_range_advance(in, bytes); + bdrv_range_advance(out, bytes); + } + +out: + qemu_vfree(buf); + qemu_iovec_destroy(&qiov); + return ret; +} + +int coroutine_fn bdrv_co_copy_range(BdrvChild *child_in, int64_t off_in, + BdrvChild *child_out, int64_t off_out, + int bytes) +{ + BdrvTrackedRequest in_req, out_req; + BlockDriverState *in_bs =3D child_in->bs; + BlockDriverState *out_bs =3D child_out->bs; + int ret =3D 0; + BdrvCopyRangeState in =3D (BdrvCopyRangeState) { + .child =3D child_in, + .offset =3D off_in, + .remaining =3D bytes, + }; + BdrvCopyRangeState out =3D (BdrvCopyRangeState) { + .child =3D child_out, + .offset =3D off_out, + .remaining =3D bytes, + }; + + bdrv_inc_in_flight(in_bs); + bdrv_inc_in_flight(out_bs); + tracked_request_begin(&in_req, in_bs, off_in, bytes, BDRV_TRACKED_READ= ); + tracked_request_begin(&out_req, out_bs, off_out, bytes, BDRV_TRACKED_W= RITE); + + wait_serialising_requests(&in_req); + wait_serialising_requests(&out_req); + + while (in.remaining > 0) { + ret =3D bdrv_co_copy_range_iteration(&in, &out); + assert(in.remaining =3D=3D out.remaining); + if (ret < 0) { + goto out; + } + } + assert(in.remaining =3D=3D 0); + assert(out.remaining =3D=3D 0); +out: + tracked_request_end(&in_req); + tracked_request_end(&out_req); + bdrv_dec_in_flight(in_bs); + bdrv_dec_in_flight(out_bs); + return ret; +} diff --git a/include/block/block.h b/include/block/block.h index 21d3e9db32..7d57e53cfe 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -610,4 +610,8 @@ int bdrv_co_map_range(BdrvChild *child, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file, int flags); +int bdrv_co_copy_range(BdrvChild *child_in, int64_t off_in, + BdrvChild *child_out, int64_t off_out, + int bytes); + #endif diff --git a/include/block/block_int.h b/include/block/block_int.h index 3f3f6f3efd..10a23b5754 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -496,6 +496,20 @@ struct BlockDriver { int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file, int flags); + /** + * Do offloaded copying of data range. Must be implemented together wi= th + * bdrv_co_map_range. + * + * @bs: where to copy data from + * @off_in: the offset to copy data from in @bs + * @out: where to write data to. This is guaranteed to be the same + * BlockDriver as @bs + * @off_out: the offset to copy data to in @out + * @bytes: the number of bytes to copy + */ + int (*bdrv_co_copy_range)(BlockDriverState *bs, int64_t off_in, + BlockDriverState *out, int64_t off_out, + int bytes); QLIST_ENTRY(BlockDriver) list; }; =20 --=20 2.14.3 From nobody Fri May 3 01:35:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522322074660763.5134545936047; Thu, 29 Mar 2018 04:14:34 -0700 (PDT) Received: from localhost ([::1]:41785 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VVp-0005P9-82 for importer@patchew.org; Thu, 29 Mar 2018 07:14:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60749) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VRK-0001he-UX for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1VRJ-0003H1-NQ for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:54 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:57652 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1VRB-00039m-Gd; Thu, 29 Mar 2018 07:09:45 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2C0A34070203; Thu, 29 Mar 2018 11:09:45 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-85.pek2.redhat.com [10.72.12.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4A4CB2024CA4; Thu, 29 Mar 2018 11:09:40 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 19:09:10 +0800 Message-Id: <20180329110914.20888-5-famz@redhat.com> In-Reply-To: <20180329110914.20888-1-famz@redhat.com> References: <20180329110914.20888-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 29 Mar 2018 11:09:45 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 29 Mar 2018 11:09:45 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC PATCH 4/8] file-posix: Implement bdrv_co_copy_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" With copy_file_range(2), we can implement the bdrv_co_copy_range semantics. Signed-off-by: Fam Zheng --- block/file-posix.c | 77 +++++++++++++++++++++++++++++++++++++++++++++= ++-- include/block/raw-aio.h | 10 +++++-- 2 files changed, 82 insertions(+), 5 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c index d7fb772c14..b13bc89423 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -100,6 +100,7 @@ #ifdef CONFIG_XFS #include #endif +#include =20 //#define DEBUG_BLOCK =20 @@ -185,6 +186,8 @@ typedef struct RawPosixAIOData { #define aio_ioctl_cmd aio_nbytes /* for QEMU_AIO_IOCTL */ off_t aio_offset; int aio_type; + int fd2; + off_t offset2; } RawPosixAIOData; =20 #if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) @@ -1421,6 +1424,41 @@ static ssize_t handle_aiocb_write_zeroes(RawPosixAIO= Data *aiocb) return -ENOTSUP; } =20 +#ifdef __NR_copy_file_range +#define HAS_COPY_FILE_RANGE +#endif + +#ifdef HAS_COPY_FILE_RANGE +static off_t copy_file_range(int in_fd, off_t *in_off, int out_fd, + off_t *out_off, size_t len, unsigned int flag= s) +{ + return syscall(__NR_copy_file_range, in_fd, in_off, out_fd, + out_off, len, flags); +} +#endif + +static ssize_t handle_aiocb_copy_range(RawPosixAIOData *aiocb) +{ +#ifndef HAS_COPY_FILE_RANGE + return -ENOTSUP; +#else + uint64_t bytes =3D aiocb->aio_nbytes; + off_t in_off =3D aiocb->aio_offset; + off_t out_off =3D aiocb->offset2; + + while (bytes) { + ssize_t ret =3D copy_file_range(aiocb->aio_fildes, &in_off, + aiocb->fd2, &out_off, + bytes, 0); + if (ret < 0) { + return -errno; + } + bytes -=3D ret; + } + return 0; +#endif +} + static ssize_t handle_aiocb_discard(RawPosixAIOData *aiocb) { int ret =3D -EOPNOTSUPP; @@ -1501,6 +1539,9 @@ static int aio_worker(void *arg) case QEMU_AIO_WRITE_ZEROES: ret =3D handle_aiocb_write_zeroes(aiocb); break; + case QEMU_AIO_COPY_RANGE: + ret =3D handle_aiocb_copy_range(aiocb); + break; default: fprintf(stderr, "invalid aio request (0x%x)\n", aiocb->aio_type); ret =3D -EINVAL; @@ -1511,9 +1552,10 @@ static int aio_worker(void *arg) return ret; } =20 -static int paio_submit_co(BlockDriverState *bs, int fd, - int64_t offset, QEMUIOVector *qiov, - int bytes, int type) +static int paio_submit_co_full(BlockDriverState *bs, int fd, + int64_t offset, int fd2, int64_t offset2, + QEMUIOVector *qiov, + int bytes, int type) { RawPosixAIOData *acb =3D g_new(RawPosixAIOData, 1); ThreadPool *pool; @@ -1521,6 +1563,8 @@ static int paio_submit_co(BlockDriverState *bs, int f= d, acb->bs =3D bs; acb->aio_type =3D type; acb->aio_fildes =3D fd; + acb->fd2 =3D fd2; + acb->offset2 =3D offset2; =20 acb->aio_nbytes =3D bytes; acb->aio_offset =3D offset; @@ -1536,6 +1580,13 @@ static int paio_submit_co(BlockDriverState *bs, int = fd, return thread_pool_submit_co(pool, aio_worker, acb); } =20 +static inline int paio_submit_co(BlockDriverState *bs, int fd, + int64_t offset, QEMUIOVector *qiov, + int bytes, int type) +{ + return paio_submit_co_full(bs, fd, offset, -1, 0, qiov, bytes, type); +} + static BlockAIOCB *paio_submit(BlockDriverState *bs, int fd, int64_t offset, QEMUIOVector *qiov, int bytes, BlockCompletionFunc *cb, void *opaque, int type) @@ -1605,6 +1656,22 @@ static int coroutine_fn raw_co_pwritev(BlockDriverSt= ate *bs, uint64_t offset, return raw_co_prw(bs, offset, bytes, qiov, QEMU_AIO_WRITE); } =20 +static int raw_co_copy_range(BlockDriverState *bs, int64_t off_in, + BlockDriverState *out, int64_t off_out, + int bytes) +{ + BDRVRawState *s =3D bs->opaque; + BDRVRawState *out_s; + + assert(out->drv->bdrv_co_copy_range =3D=3D raw_co_copy_range); + out_s =3D out->opaque; + if (fd_open(bs) < 0 || fd_open(out) < 0) { + return -EIO; + } + return paio_submit_co_full(bs, s->fd, off_in, out_s->fd, off_out, + NULL, bytes, QEMU_AIO_COPY_RANGE); +} + static void raw_aio_plug(BlockDriverState *bs) { #ifdef CONFIG_LINUX_AIO @@ -2321,6 +2388,7 @@ BlockDriver bdrv_file =3D { =20 .bdrv_co_preadv =3D raw_co_preadv, .bdrv_co_pwritev =3D raw_co_pwritev, + .bdrv_co_copy_range =3D raw_co_copy_range, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_aio_pdiscard =3D raw_aio_pdiscard, .bdrv_refresh_limits =3D raw_refresh_limits, @@ -2798,6 +2866,7 @@ static BlockDriver bdrv_host_device =3D { =20 .bdrv_co_preadv =3D raw_co_preadv, .bdrv_co_pwritev =3D raw_co_pwritev, + .bdrv_co_copy_range =3D raw_co_copy_range, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_aio_pdiscard =3D hdev_aio_pdiscard, .bdrv_refresh_limits =3D raw_refresh_limits, @@ -2920,6 +2989,7 @@ static BlockDriver bdrv_host_cdrom =3D { =20 .bdrv_co_preadv =3D raw_co_preadv, .bdrv_co_pwritev =3D raw_co_pwritev, + .bdrv_co_copy_range =3D raw_co_copy_range, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_refresh_limits =3D raw_refresh_limits, .bdrv_io_plug =3D raw_aio_plug, @@ -3050,6 +3120,7 @@ static BlockDriver bdrv_host_cdrom =3D { =20 .bdrv_co_preadv =3D raw_co_preadv, .bdrv_co_pwritev =3D raw_co_pwritev, + .bdrv_co_copy_range =3D raw_co_copy_range, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_refresh_limits =3D raw_refresh_limits, .bdrv_io_plug =3D raw_aio_plug, diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h index a4cdbbf1b7..324053020b 100644 --- a/include/block/raw-aio.h +++ b/include/block/raw-aio.h @@ -25,9 +25,15 @@ #define QEMU_AIO_FLUSH 0x0008 #define QEMU_AIO_DISCARD 0x0010 #define QEMU_AIO_WRITE_ZEROES 0x0020 +#define QEMU_AIO_COPY_RANGE 0x0040 #define QEMU_AIO_TYPE_MASK \ - (QEMU_AIO_READ|QEMU_AIO_WRITE|QEMU_AIO_IOCTL|QEMU_AIO_FLUSH| \ - QEMU_AIO_DISCARD|QEMU_AIO_WRITE_ZEROES) + (QEMU_AIO_READ | \ + QEMU_AIO_WRITE | \ + QEMU_AIO_IOCTL | \ + QEMU_AIO_FLUSH | \ + QEMU_AIO_DISCARD | \ + QEMU_AIO_WRITE_ZEROES | \ + QEMU_AIO_COPY_RANGE) =20 /* AIO flags */ #define QEMU_AIO_MISALIGNED 0x1000 --=20 2.14.3 From nobody Fri May 3 01:35:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522322091672854.4993805931921; Thu, 29 Mar 2018 04:14:51 -0700 (PDT) Received: from localhost ([::1]:41791 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VW6-0005fg-IH for importer@patchew.org; Thu, 29 Mar 2018 07:14:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60745) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VRK-0001hU-PM for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1VRJ-0003Gw-Ly for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:09:54 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:58464 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1VRF-0003Fs-Hc; Thu, 29 Mar 2018 07:09:49 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2DF7D40201A4; Thu, 29 Mar 2018 11:09:49 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-85.pek2.redhat.com [10.72.12.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id C02ED2024CA4; Thu, 29 Mar 2018 11:09:45 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 19:09:11 +0800 Message-Id: <20180329110914.20888-6-famz@redhat.com> In-Reply-To: <20180329110914.20888-1-famz@redhat.com> References: <20180329110914.20888-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 29 Mar 2018 11:09:49 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 29 Mar 2018 11:09:49 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC PATCH 5/8] file-posix: Implement bdrv_co_map_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Fam Zheng --- block/file-posix.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/block/file-posix.c b/block/file-posix.c index b13bc89423..4e615d937e 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -1656,6 +1656,17 @@ static int coroutine_fn raw_co_pwritev(BlockDriverSt= ate *bs, uint64_t offset, return raw_co_prw(bs, offset, bytes, qiov, QEMU_AIO_WRITE); } =20 +static int raw_co_map_range(BlockDriverState *bs, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *map, + BlockDriverState **file, + int flags) +{ + *file =3D bs; + *pnum =3D bytes; + *map =3D offset; + return BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID; +} + static int raw_co_copy_range(BlockDriverState *bs, int64_t off_in, BlockDriverState *out, int64_t off_out, int bytes) @@ -2389,6 +2400,7 @@ BlockDriver bdrv_file =3D { .bdrv_co_preadv =3D raw_co_preadv, .bdrv_co_pwritev =3D raw_co_pwritev, .bdrv_co_copy_range =3D raw_co_copy_range, + .bdrv_co_map_range =3D raw_co_map_range, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_aio_pdiscard =3D raw_aio_pdiscard, .bdrv_refresh_limits =3D raw_refresh_limits, @@ -2867,6 +2879,7 @@ static BlockDriver bdrv_host_device =3D { .bdrv_co_preadv =3D raw_co_preadv, .bdrv_co_pwritev =3D raw_co_pwritev, .bdrv_co_copy_range =3D raw_co_copy_range, + .bdrv_co_map_range =3D raw_co_map_range, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_aio_pdiscard =3D hdev_aio_pdiscard, .bdrv_refresh_limits =3D raw_refresh_limits, @@ -2990,6 +3003,7 @@ static BlockDriver bdrv_host_cdrom =3D { .bdrv_co_preadv =3D raw_co_preadv, .bdrv_co_pwritev =3D raw_co_pwritev, .bdrv_co_copy_range =3D raw_co_copy_range, + .bdrv_co_map_range =3D raw_co_map_range, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_refresh_limits =3D raw_refresh_limits, .bdrv_io_plug =3D raw_aio_plug, @@ -3121,6 +3135,7 @@ static BlockDriver bdrv_host_cdrom =3D { .bdrv_co_preadv =3D raw_co_preadv, .bdrv_co_pwritev =3D raw_co_pwritev, .bdrv_co_copy_range =3D raw_co_copy_range, + .bdrv_co_map_range =3D raw_co_map_range, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_refresh_limits =3D raw_refresh_limits, .bdrv_io_plug =3D raw_aio_plug, --=20 2.14.3 From nobody Fri May 3 01:35:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522322254126847.4633924887224; Thu, 29 Mar 2018 04:17:34 -0700 (PDT) Received: from localhost ([::1]:41990 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VYi-0007vF-3Q for importer@patchew.org; Thu, 29 Mar 2018 07:17:32 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60783) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VRQ-0001nM-MT for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:10:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1VRP-0003IE-NO for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:10:00 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:58466 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1VRJ-0003Gj-GC; Thu, 29 Mar 2018 07:09:53 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2E16240201A7; Thu, 29 Mar 2018 11:09:53 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-85.pek2.redhat.com [10.72.12.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id C22172024CA4; Thu, 29 Mar 2018 11:09:49 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 19:09:12 +0800 Message-Id: <20180329110914.20888-7-famz@redhat.com> In-Reply-To: <20180329110914.20888-1-famz@redhat.com> References: <20180329110914.20888-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 29 Mar 2018 11:09:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 29 Mar 2018 11:09:53 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC PATCH 6/8] raw: Implement raw_co_map_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Because raw can be seen as a "passthrough" format, its implementation of bdrv_co_map_range is exactly the same as bdrv_co_block_status. The BDRV_REQ_ALLOCATE flag can be ignored because no metadata update is necessary. Signed-off-by: Fam Zheng --- block/raw-format.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/block/raw-format.c b/block/raw-format.c index a378547c99..95e81af22d 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -263,6 +263,14 @@ static int coroutine_fn raw_co_block_status(BlockDrive= rState *bs, return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID; } =20 +static int coroutine_fn raw_co_map_range(BlockDriverState *bs, int64_t off= set, + int64_t bytes, int64_t *pnum, int= 64_t *map, + BlockDriverState **file, + int flags) +{ + return raw_co_block_status(bs, true, offset, bytes, pnum, map, file); +} + static int coroutine_fn raw_co_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int bytes, BdrvRequestFlags flags) @@ -498,6 +506,7 @@ BlockDriver bdrv_raw =3D { .bdrv_co_pwrite_zeroes =3D &raw_co_pwrite_zeroes, .bdrv_co_pdiscard =3D &raw_co_pdiscard, .bdrv_co_block_status =3D &raw_co_block_status, + .bdrv_co_map_range =3D &raw_co_map_range, .bdrv_truncate =3D &raw_truncate, .bdrv_getlength =3D &raw_getlength, .has_variable_length =3D true, --=20 2.14.3 From nobody Fri May 3 01:35:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522322235894649.1586201974886; Thu, 29 Mar 2018 04:17:15 -0700 (PDT) Received: from localhost ([::1]:41955 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VYP-0007gS-22 for importer@patchew.org; Thu, 29 Mar 2018 07:17:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60813) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VRU-0001rj-JE for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:10:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1VRT-0003J3-Jr for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:10:04 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:57656 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1VRN-0003Ha-Et; Thu, 29 Mar 2018 07:09:57 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 242A74070203; Thu, 29 Mar 2018 11:09:57 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-85.pek2.redhat.com [10.72.12.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id C4B1C2024CA4; Thu, 29 Mar 2018 11:09:53 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 19:09:13 +0800 Message-Id: <20180329110914.20888-8-famz@redhat.com> In-Reply-To: <20180329110914.20888-1-famz@redhat.com> References: <20180329110914.20888-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 29 Mar 2018 11:09:57 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 29 Mar 2018 11:09:57 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC PATCH 7/8] block-backend: Add blk_co_copy_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It's a BlockBackend wrapper of bdrv_co_copy_range. Signed-off-by: Fam Zheng --- block/block-backend.c | 8 ++++++++ include/sysemu/block-backend.h | 4 ++++ 2 files changed, 12 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index 681b240b12..54b0789282 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -2217,3 +2217,11 @@ void blk_unregister_buf(BlockBackend *blk, void *hos= t) { bdrv_unregister_buf(blk_bs(blk), host); } + +int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, + BlockBackend *blk_out, int64_t off_out, + int bytes) +{ + return bdrv_co_copy_range(blk_in->root, off_in, blk_out->root, off_out, + bytes); +} diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h index 92ab624fac..1fde14b461 100644 --- a/include/sysemu/block-backend.h +++ b/include/sysemu/block-backend.h @@ -232,4 +232,8 @@ void blk_set_force_allow_inactivate(BlockBackend *blk); void blk_register_buf(BlockBackend *blk, void *host, size_t size); void blk_unregister_buf(BlockBackend *blk, void *host); =20 +int blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, + BlockBackend *blk_out, int64_t off_out, + int bytes); + #endif --=20 2.14.3 From nobody Fri May 3 01:35:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1522322097524177.4541932145463; Thu, 29 Mar 2018 04:14:57 -0700 (PDT) Received: from localhost ([::1]:41793 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VWC-0005iU-5P for importer@patchew.org; Thu, 29 Mar 2018 07:14:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60823) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1VRV-0001tk-8g for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:10:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f1VRU-0003J9-74 for qemu-devel@nongnu.org; Thu, 29 Mar 2018 07:10:05 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46476 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f1VRR-0003IZ-H7; Thu, 29 Mar 2018 07:10:01 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2C39D4270947; Thu, 29 Mar 2018 11:10:01 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-85.pek2.redhat.com [10.72.12.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id B8E0E2026DFD; Thu, 29 Mar 2018 11:09:57 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 29 Mar 2018 19:09:14 +0800 Message-Id: <20180329110914.20888-9-famz@redhat.com> In-Reply-To: <20180329110914.20888-1-famz@redhat.com> References: <20180329110914.20888-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Thu, 29 Mar 2018 11:10:01 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Thu, 29 Mar 2018 11:10:01 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC PATCH 8/8] qemu-img: Convert with copy offloading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The new blk_co_copy_range interface offers a more efficient way in the case of network based storage. Make use of it to allow faster convert operation. Since copy offloading cannot do zero detection ('-S') and compression (-c), only try it when these options are not used. Signed-off-by: Fam Zheng --- qemu-img.c | 45 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 855fa52514..571b6e2e49 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -1544,6 +1544,7 @@ typedef struct ImgConvertState { bool compressed; bool target_has_backing; bool wr_in_order; + bool copy_range; int min_sparse; size_t cluster_sectors; size_t buf_sectors; @@ -1737,6 +1738,37 @@ static int coroutine_fn convert_co_write(ImgConvertS= tate *s, int64_t sector_num, return 0; } =20 +static int coroutine_fn convert_co_copy_range(ImgConvertState *s, int64_t = sector_num, + int nb_sectors) +{ + int n, ret; + + while (nb_sectors > 0) { + BlockBackend *blk; + int src_cur; + int64_t bs_sectors, src_cur_offset; + int64_t offset; + + convert_select_part(s, sector_num, &src_cur, &src_cur_offset); + offset =3D (sector_num - src_cur_offset) << BDRV_SECTOR_BITS; + blk =3D s->src[src_cur]; + bs_sectors =3D s->src_sectors[src_cur]; + + n =3D MIN(nb_sectors, bs_sectors - (sector_num - src_cur_offset)); + + ret =3D blk_co_copy_range(blk, offset, s->target, + sector_num << BDRV_SECTOR_BITS, + n << BDRV_SECTOR_BITS); + if (ret < 0) { + return ret; + } + + sector_num +=3D n; + nb_sectors -=3D n; + } + return 0; +} + static void coroutine_fn convert_co_do_copy(void *opaque) { ImgConvertState *s =3D opaque; @@ -1759,6 +1791,7 @@ static void coroutine_fn convert_co_do_copy(void *opa= que) int n; int64_t sector_num; enum ImgConvertBlockStatus status; + bool copy_range; =20 qemu_co_mutex_lock(&s->lock); if (s->ret !=3D -EINPROGRESS || s->sector_num >=3D s->total_sector= s) { @@ -1777,6 +1810,7 @@ static void coroutine_fn convert_co_do_copy(void *opa= que) if (!s->min_sparse && s->status =3D=3D BLK_ZERO) { n =3D MIN(n, s->buf_sectors); } + copy_range =3D s->copy_range && s->status =3D=3D BLK_DATA; /* increment global sector counter so that other coroutines can * already continue reading beyond this request */ s->sector_num +=3D n; @@ -1788,7 +1822,7 @@ static void coroutine_fn convert_co_do_copy(void *opa= que) s->allocated_sectors, 0); } =20 - if (status =3D=3D BLK_DATA) { + if (status =3D=3D BLK_DATA && !copy_range) { ret =3D convert_co_read(s, sector_num, n, buf); if (ret < 0) { error_report("error while reading sector %" PRId64 @@ -1810,7 +1844,11 @@ static void coroutine_fn convert_co_do_copy(void *op= aque) } =20 if (s->ret =3D=3D -EINPROGRESS) { - ret =3D convert_co_write(s, sector_num, n, buf, status); + if (copy_range) { + ret =3D convert_co_copy_range(s, sector_num, n); + } else { + ret =3D convert_co_write(s, sector_num, n, buf, status); + } if (ret < 0) { error_report("error while writing sector %" PRId64 ": %s", sector_num, strerror(-ret)); @@ -1933,6 +1971,7 @@ static int img_convert(int argc, char **argv) ImgConvertState s =3D (ImgConvertState) { /* Need at least 4k of zeros for sparse detection */ .min_sparse =3D 8, + .copy_range =3D true, .buf_sectors =3D IO_BUF_SIZE / BDRV_SECTOR_SIZE, .wr_in_order =3D true, .num_coroutines =3D 8, @@ -1973,6 +2012,7 @@ static int img_convert(int argc, char **argv) break; case 'c': s.compressed =3D true; + s.copy_range =3D false; break; case 'o': if (!is_valid_option_list(optarg)) { @@ -2014,6 +2054,7 @@ static int img_convert(int argc, char **argv) } =20 s.min_sparse =3D sval / BDRV_SECTOR_SIZE; + s.copy_range =3D false; break; } case 'p': --=20 2.14.3