From nobody Wed Oct 29 20:27:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525878064772887.7485358639833; Wed, 9 May 2018 08:01:04 -0700 (PDT) Received: from localhost ([::1]:56973 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQaS-0000wm-7T for importer@patchew.org; Wed, 09 May 2018 11:01:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56649) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQYF-0006pw-Mp for qemu-devel@nongnu.org; Wed, 09 May 2018 10:58:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQYE-0006Bs-CM for qemu-devel@nongnu.org; Wed, 09 May 2018 10:58:43 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:37284 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQY7-0005zU-Jo; Wed, 09 May 2018 10:58:35 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1952481A88A2; Wed, 9 May 2018 14:58:35 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id 061A4D74B9; Wed, 9 May 2018 14:58:30 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:07 +0800 Message-Id: <20180509145815.3330-2-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 09 May 2018 14:58:35 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 09 May 2018 14:58:35 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 1/9] block: Introduce API for copy offloading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Fam Zheng --- block/io.c | 91 +++++++++++++++++++++++++++++++++++++++++++= ++++ include/block/block.h | 4 +++ include/block/block_int.h | 30 ++++++++++++++++ 3 files changed, 125 insertions(+) diff --git a/block/io.c b/block/io.c index bd9a19a9c4..d274e9525f 100644 --- a/block/io.c +++ b/block/io.c @@ -2826,3 +2826,94 @@ void bdrv_unregister_buf(BlockDriverState *bs, void = *host) bdrv_unregister_buf(child->bs, host); } } + +static int bdrv_co_copy_range_internal(BdrvChild *src, + uint64_t src_offset, + BdrvChild *dst, + uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags fl= ags, + bool recurse_src) +{ + int ret; + + if (!src || !dst || !src->bs || !dst->bs) { + return -ENOMEDIUM; + } + ret =3D bdrv_check_byte_request(src->bs, src_offset, bytes); + if (ret) { + return ret; + } + + ret =3D bdrv_check_byte_request(dst->bs, dst_offset, bytes); + if (ret) { + return ret; + } + if (flags & BDRV_REQ_ZERO_WRITE) { + return bdrv_co_pwrite_zeroes(dst, dst_offset, bytes, flags); + } + + if (!src->bs->drv->bdrv_co_copy_range_from + || !dst->bs->drv->bdrv_co_copy_range_to + || src->bs->encrypted || dst->bs->encrypted) { + return -ENOTSUP; + } + if (recurse_src) { + return src->bs->drv->bdrv_co_copy_range_from(src->bs, + src, src_offset, + dst, dst_offset, + bytes, flags); + } else { + return dst->bs->drv->bdrv_co_copy_range_to(dst->bs, + src, src_offset, + dst, dst_offset, + bytes, flags); + } +} + +/* Copy range from @bs to @dst. */ +int bdrv_co_copy_range_from(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags) +{ + return bdrv_co_copy_range_internal(src, src_offset, dst, dst_offset, + bytes, flags, true); +} + +/* Copy range from @src to @bs. Should only be called by block drivers whe= n @bs + * is the leaf. */ +int bdrv_co_copy_range_to(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags) +{ + return bdrv_co_copy_range_internal(src, src_offset, dst, dst_offset, + bytes, flags, false); +} + +int bdrv_co_copy_range(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags) +{ + BdrvTrackedRequest src_req, dst_req; + BlockDriverState *src_bs =3D src->bs; + BlockDriverState *dst_bs =3D dst->bs; + int ret; + + bdrv_inc_in_flight(src_bs); + bdrv_inc_in_flight(dst_bs); + tracked_request_begin(&src_req, src_bs, src_offset, + bytes, BDRV_TRACKED_READ); + tracked_request_begin(&dst_req, dst_bs, dst_offset, + bytes, BDRV_TRACKED_WRITE); + + wait_serialising_requests(&src_req); + wait_serialising_requests(&dst_req); + ret =3D bdrv_co_copy_range_from(src, src_offset, + dst, dst_offset, + bytes, flags); + + tracked_request_end(&src_req); + tracked_request_end(&dst_req); + bdrv_dec_in_flight(src_bs); + bdrv_dec_in_flight(dst_bs); + return ret; +} diff --git a/include/block/block.h b/include/block/block.h index cdec3639a3..72ac011b2b 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -604,4 +604,8 @@ bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *= bs, const char *name, */ void bdrv_register_buf(BlockDriverState *bs, void *host, size_t size); void bdrv_unregister_buf(BlockDriverState *bs, void *host); + +int bdrv_co_copy_range(BdrvChild *bs, uint64_t offset, + BdrvChild *src, uint64_t src_offset, + uint64_t bytes, BdrvRequestFlags flags); #endif diff --git a/include/block/block_int.h b/include/block/block_int.h index c4dd1d4bb8..305c29b75e 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -206,6 +206,29 @@ struct BlockDriver { int coroutine_fn (*bdrv_co_pdiscard)(BlockDriverState *bs, int64_t offset, int bytes); =20 + /* Map [offset, offset + nbytes) range onto a child of @bs to copy fro= m, + * and invoke bdrv_co_copy_range_from(child, ...), or invoke + * bdrv_co_copy_range_to() if @bs is the leaf child to copy data from. + */ + int coroutine_fn (*bdrv_co_copy_range_from)(BlockDriverState *bs, + BdrvChild *src, + uint64_t offset, + BdrvChild *dst, + uint64_t dst_offset, + uint64_t bytes, BdrvReques= tFlags flags); + + /* Map [offset, offset + nbytes) range onto a child of bs to copy data= to, + * and invoke bdrv_co_copy_range_to(child, src, ...), or perform the c= opy + * operation if @bs is the leaf and @src has the same BlockDriver. Re= turn + * -ENOTSUP if @bs is the leaf but @src has a different BlockDriver. + */ + int coroutine_fn (*bdrv_co_copy_range_to)(BlockDriverState *bs, + BdrvChild *src, + uint64_t src_offset, + BdrvChild *dst, + uint64_t dst_offset, + uint64_t bytes, BdrvRequestF= lags flags); + /* * Building block for bdrv_block_status[_above] and * bdrv_is_allocated[_above]. The driver should answer only @@ -1090,4 +1113,11 @@ void bdrv_dec_in_flight(BlockDriverState *bs); =20 void blockdev_close_all_bdrv_states(void); =20 +int bdrv_co_copy_range_from(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags); +int bdrv_co_copy_range_to(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags); + #endif /* BLOCK_INT_H */ --=20 2.14.3 From nobody Wed Oct 29 20:27:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525878058269696.6056574727196; Wed, 9 May 2018 08:00:58 -0700 (PDT) Received: from localhost ([::1]:56971 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQaF-0000m2-C6 for importer@patchew.org; Wed, 09 May 2018 11:00:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56681) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQYH-0006rq-Tj for qemu-devel@nongnu.org; Wed, 09 May 2018 10:58:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQYH-0006It-42 for qemu-devel@nongnu.org; Wed, 09 May 2018 10:58:45 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45892 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQYC-00068g-G6; Wed, 09 May 2018 10:58:40 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 033E84059FE2; Wed, 9 May 2018 14:58:40 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id DCFD9D74BF; Wed, 9 May 2018 14:58:35 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:08 +0800 Message-Id: <20180509145815.3330-3-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:58:40 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:58:40 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 2/9] raw: Implement copy offloading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Just pass down to ->file. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/raw-format.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/block/raw-format.c b/block/raw-format.c index a378547c99..febddf00c0 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -482,6 +482,24 @@ static int raw_probe_geometry(BlockDriverState *bs, HD= Geometry *geo) return bdrv_probe_geometry(bs->file->bs, geo); } =20 +static int coroutine_fn raw_co_copy_range_from(BlockDriverState *bs, + BdrvChild *src, uint64_t sr= c_offset, + BdrvChild *dst, uint64_t ds= t_offset, + uint64_t bytes, BdrvRequest= Flags flags) +{ + return bdrv_co_copy_range_from(bs->file, src_offset, dst, dst_offset, + bytes, flags); +} + +static int coroutine_fn raw_co_copy_range_to(BlockDriverState *bs, + BdrvChild *src, uint64_t src_= offset, + BdrvChild *dst, uint64_t dst_= offset, + uint64_t bytes, BdrvRequestFl= ags flags) +{ + return bdrv_co_copy_range_to(src, src_offset, bs->file, dst_offset, by= tes, + flags); +} + BlockDriver bdrv_raw =3D { .format_name =3D "raw", .instance_size =3D sizeof(BDRVRawState), @@ -498,6 +516,8 @@ BlockDriver bdrv_raw =3D { .bdrv_co_pwrite_zeroes =3D &raw_co_pwrite_zeroes, .bdrv_co_pdiscard =3D &raw_co_pdiscard, .bdrv_co_block_status =3D &raw_co_block_status, + .bdrv_co_copy_range_from =3D &raw_co_copy_range_from, + .bdrv_co_copy_range_to =3D &raw_co_copy_range_to, .bdrv_truncate =3D &raw_truncate, .bdrv_getlength =3D &raw_getlength, .has_variable_length =3D true, --=20 2.14.3 From nobody Wed Oct 29 20:27:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 15258783831496.472642738093441; Wed, 9 May 2018 08:06:23 -0700 (PDT) Received: from localhost ([::1]:57008 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQfV-0006tk-3q for importer@patchew.org; Wed, 09 May 2018 11:06:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56728) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQYR-0006zH-Tg for qemu-devel@nongnu.org; Wed, 09 May 2018 10:58:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQYQ-0006at-O2 for qemu-devel@nongnu.org; Wed, 09 May 2018 10:58:55 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45898 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQYL-0006RK-JU; Wed, 09 May 2018 10:58:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1E0D54059FE2; Wed, 9 May 2018 14:58:49 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id DF26ED74BA; Wed, 9 May 2018 14:58:40 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:09 +0800 Message-Id: <20180509145815.3330-4-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:58:49 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:58:49 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 3/9] qcow2: Implement copy offloading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The two callbacks are implemented quite similarly to the read/write functions: bdrv_co_copy_range_from maps for read and calls into bs->file or bs->backing depending on the allocation status; bdrv_co_copy_range_to maps for write and calls into bs->file. Signed-off-by: Fam Zheng --- block/qcow2.c | 228 ++++++++++++++++++++++++++++++++++++++++++++++++++----= ---- 1 file changed, 198 insertions(+), 30 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 2f36e632f9..07e38b548a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1755,6 +1755,31 @@ static int coroutine_fn qcow2_co_block_status(BlockD= riverState *bs, return status; } =20 +static int qcow2_handle_l2meta(BlockDriverState *bs, QCowL2Meta *l2meta) +{ + int ret =3D 0; + + while (l2meta !=3D NULL) { + QCowL2Meta *next; + + if (!ret) { + ret =3D qcow2_alloc_cluster_link_l2(bs, l2meta); + } + + /* Take the request off the list of running requests */ + if (l2meta->nb_clusters !=3D 0) { + QLIST_REMOVE(l2meta, next_in_flight); + } + + qemu_co_queue_restart_all(&l2meta->dependent_requests); + + next =3D l2meta->next; + g_free(l2meta); + l2meta =3D next; + } + return ret; +} + static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t off= set, uint64_t bytes, QEMUIOVector *qiov, int flags) @@ -2041,24 +2066,10 @@ static coroutine_fn int qcow2_co_pwritev(BlockDrive= rState *bs, uint64_t offset, } } =20 - while (l2meta !=3D NULL) { - QCowL2Meta *next; - - ret =3D qcow2_alloc_cluster_link_l2(bs, l2meta); - if (ret < 0) { - goto fail; - } - - /* Take the request off the list of running requests */ - if (l2meta->nb_clusters !=3D 0) { - QLIST_REMOVE(l2meta, next_in_flight); - } - - qemu_co_queue_restart_all(&l2meta->dependent_requests); - - next =3D l2meta->next; - g_free(l2meta); - l2meta =3D next; + ret =3D qcow2_handle_l2meta(bs, l2meta); + l2meta =3D NULL; + if (ret) { + goto fail; } =20 bytes -=3D cur_bytes; @@ -2069,18 +2080,7 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriver= State *bs, uint64_t offset, ret =3D 0; =20 fail: - while (l2meta !=3D NULL) { - QCowL2Meta *next; - - if (l2meta->nb_clusters !=3D 0) { - QLIST_REMOVE(l2meta, next_in_flight); - } - qemu_co_queue_restart_all(&l2meta->dependent_requests); - - next =3D l2meta->next; - g_free(l2meta); - l2meta =3D next; - } + qcow2_handle_l2meta(bs, l2meta); =20 qemu_co_mutex_unlock(&s->lock); =20 @@ -3267,6 +3267,172 @@ static coroutine_fn int qcow2_co_pdiscard(BlockDriv= erState *bs, return ret; } =20 +static int qcow2_co_copy_range_from(BlockDriverState *bs, + BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags) +{ + BDRVQcow2State *s =3D bs->opaque; + int offset_in_cluster; + int ret; + unsigned int cur_bytes; /* number of bytes in current iteration */ + uint64_t cluster_offset =3D 0; + BdrvChild *child =3D NULL; + + assert(!bs->encrypted); + qemu_co_mutex_lock(&s->lock); + + while (bytes !=3D 0) { + + /* prepare next request */ + cur_bytes =3D MIN(bytes, INT_MAX); + + ret =3D qcow2_get_cluster_offset(bs, src_offset, &cur_bytes, &clus= ter_offset); + if (ret < 0) { + goto out; + } + + offset_in_cluster =3D offset_into_cluster(s, src_offset); + + switch (ret) { + case QCOW2_CLUSTER_UNALLOCATED: + if (bs->backing) { + child =3D bs->backing; + } else { + flags |=3D BDRV_REQ_ZERO_WRITE; + } + break; + + case QCOW2_CLUSTER_ZERO_PLAIN: + case QCOW2_CLUSTER_ZERO_ALLOC: + flags |=3D BDRV_REQ_ZERO_WRITE; + break; + + case QCOW2_CLUSTER_COMPRESSED: + ret =3D -ENOTSUP; + goto out; + break; + + case QCOW2_CLUSTER_NORMAL: + child =3D bs->file; + if ((cluster_offset & 511) !=3D 0) { + ret =3D -EIO; + goto out; + } + break; + + default: + abort(); + } + qemu_co_mutex_unlock(&s->lock); + ret =3D bdrv_co_copy_range_from(child, + cluster_offset + offset_in_cluster, + dst, dst_offset, + cur_bytes, flags); + qemu_co_mutex_lock(&s->lock); + if (ret < 0) { + goto out; + } + + bytes -=3D cur_bytes; + src_offset +=3D cur_bytes; + } + ret =3D 0; + +out: + qemu_co_mutex_unlock(&s->lock); + return ret; +} + +static int qcow2_co_copy_range_to(BlockDriverState *bs, + BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags) +{ + BDRVQcow2State *s =3D bs->opaque; + int offset_in_cluster; + int ret; + unsigned int cur_bytes; /* number of sectors in current iteration */ + uint64_t cluster_offset; + uint8_t *cluster_data =3D NULL; + QCowL2Meta *l2meta =3D NULL; + + assert(!bs->encrypted); + s->cluster_cache_offset =3D -1; /* disable compressed cache */ + + qemu_co_mutex_lock(&s->lock); + + while (bytes !=3D 0) { + + l2meta =3D NULL; + + offset_in_cluster =3D offset_into_cluster(s, dst_offset); + cur_bytes =3D MIN(bytes, INT_MAX); + + /* TODO: + * If src->bs =3D=3D dst->bs, we could simply copy by incrementing + * the refcnt, without copying user data. + * Or if src->bs =3D=3D dst->bs->backing->bs, we could copy by dis= carding. */ + ret =3D qcow2_alloc_cluster_offset(bs, dst_offset, &cur_bytes, + &cluster_offset, &l2meta); + if (ret < 0) { + goto fail; + } + + assert((cluster_offset & 511) =3D=3D 0); + + ret =3D qcow2_pre_write_overlap_check(bs, 0, + cluster_offset + offset_in_cluster, cur_bytes); + if (ret < 0) { + goto fail; + } + + qemu_co_mutex_unlock(&s->lock); + ret =3D bdrv_co_copy_range_to(src, src_offset, + bs->file, + cluster_offset + offset_in_cluster, + cur_bytes, flags); + qemu_co_mutex_lock(&s->lock); + if (ret < 0) { + goto fail; + } + + while (l2meta !=3D NULL) { + QCowL2Meta *next; + + ret =3D qcow2_alloc_cluster_link_l2(bs, l2meta); + if (ret < 0) { + goto fail; + } + + /* Take the request off the list of running requests */ + if (l2meta->nb_clusters !=3D 0) { + QLIST_REMOVE(l2meta, next_in_flight); + } + + qemu_co_queue_restart_all(&l2meta->dependent_requests); + + next =3D l2meta->next; + g_free(l2meta); + l2meta =3D next; + } + + bytes -=3D cur_bytes; + dst_offset +=3D cur_bytes; + } + ret =3D 0; + +fail: + qcow2_handle_l2meta(bs, l2meta); + + qemu_co_mutex_unlock(&s->lock); + + qemu_vfree(cluster_data); + trace_qcow2_writev_done_req(qemu_coroutine_self(), ret); + + return ret; +} + static int qcow2_truncate(BlockDriverState *bs, int64_t offset, PreallocMode prealloc, Error **errp) { @@ -4515,6 +4681,8 @@ BlockDriver bdrv_qcow2 =3D { =20 .bdrv_co_pwrite_zeroes =3D qcow2_co_pwrite_zeroes, .bdrv_co_pdiscard =3D qcow2_co_pdiscard, + .bdrv_co_copy_range_from =3D qcow2_co_copy_range_from, + .bdrv_co_copy_range_to =3D qcow2_co_copy_range_to, .bdrv_truncate =3D qcow2_truncate, .bdrv_co_pwritev_compressed =3D qcow2_co_pwritev_compressed, .bdrv_make_empty =3D qcow2_make_empty, --=20 2.14.3 From nobody Wed Oct 29 20:27:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525878248812777.1985786864964; Wed, 9 May 2018 08:04:08 -0700 (PDT) Received: from localhost ([::1]:56990 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQdT-0004qs-J8 for importer@patchew.org; Wed, 09 May 2018 11:04:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56759) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQYX-00074Z-7i for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQYU-0006h5-47 for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:01 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45906 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQYQ-0006aj-Sn; Wed, 09 May 2018 10:58:54 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5B4154059FE2; Wed, 9 May 2018 14:58:54 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id E1EADD74BA; Wed, 9 May 2018 14:58:49 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:10 +0800 Message-Id: <20180509145815.3330-5-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:58:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:58:54 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 4/9] file-posix: Implement bdrv_co_copy_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" With copy_file_range(2), we can implement the bdrv_co_copy_range semantics. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/file-posix.c | 94 +++++++++++++++++++++++++++++++++++++++++++++= ++-- include/block/raw-aio.h | 10 ++++-- 2 files changed, 99 insertions(+), 5 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c index 3794c0007a..138bf28747 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -59,6 +59,7 @@ #ifdef __linux__ #include #include +#include #include #include #include @@ -185,6 +186,8 @@ typedef struct RawPosixAIOData { #define aio_ioctl_cmd aio_nbytes /* for QEMU_AIO_IOCTL */ off_t aio_offset; int aio_type; + int aio_fd2; + off_t aio_offset2; } RawPosixAIOData; =20 #if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) @@ -1421,6 +1424,43 @@ static ssize_t handle_aiocb_write_zeroes(RawPosixAIO= Data *aiocb) return -ENOTSUP; } =20 +static off_t copy_file_range(int in_fd, off_t *in_off, int out_fd, + off_t *out_off, size_t len, unsigned int flag= s) +{ +#ifdef __NR_copy_file_range + return syscall(__NR_copy_file_range, in_fd, in_off, out_fd, + out_off, len, flags); +#else + errno =3D ENOSYS; + return -1; +#endif +} + +static ssize_t handle_aiocb_copy_range(RawPosixAIOData *aiocb) +{ + uint64_t bytes =3D aiocb->aio_nbytes; + off_t in_off =3D aiocb->aio_offset; + off_t out_off =3D aiocb->aio_offset2; + + while (bytes) { + ssize_t ret =3D copy_file_range(aiocb->aio_fildes, &in_off, + aiocb->aio_fd2, &out_off, + bytes, 0); + if (ret =3D=3D -EINTR) { + continue; + } + if (ret < 0) { + return -errno; + } + if (!ret) { + /* No progress (e.g. when beyond EOF), fall back to buffer I/O= . */ + return -ENOTSUP; + } + bytes -=3D ret; + } + return 0; +} + static ssize_t handle_aiocb_discard(RawPosixAIOData *aiocb) { int ret =3D -EOPNOTSUPP; @@ -1501,6 +1541,9 @@ static int aio_worker(void *arg) case QEMU_AIO_WRITE_ZEROES: ret =3D handle_aiocb_write_zeroes(aiocb); break; + case QEMU_AIO_COPY_RANGE: + ret =3D handle_aiocb_copy_range(aiocb); + break; default: fprintf(stderr, "invalid aio request (0x%x)\n", aiocb->aio_type); ret =3D -EINVAL; @@ -1511,9 +1554,10 @@ static int aio_worker(void *arg) return ret; } =20 -static int paio_submit_co(BlockDriverState *bs, int fd, - int64_t offset, QEMUIOVector *qiov, - int bytes, int type) +static int paio_submit_co_full(BlockDriverState *bs, int fd, + int64_t offset, int fd2, int64_t offset2, + QEMUIOVector *qiov, + int bytes, int type) { RawPosixAIOData *acb =3D g_new(RawPosixAIOData, 1); ThreadPool *pool; @@ -1521,6 +1565,8 @@ static int paio_submit_co(BlockDriverState *bs, int f= d, acb->bs =3D bs; acb->aio_type =3D type; acb->aio_fildes =3D fd; + acb->aio_fd2 =3D fd2; + acb->aio_offset2 =3D offset2; =20 acb->aio_nbytes =3D bytes; acb->aio_offset =3D offset; @@ -1536,6 +1582,13 @@ static int paio_submit_co(BlockDriverState *bs, int = fd, return thread_pool_submit_co(pool, aio_worker, acb); } =20 +static inline int paio_submit_co(BlockDriverState *bs, int fd, + int64_t offset, QEMUIOVector *qiov, + int bytes, int type) +{ + return paio_submit_co_full(bs, fd, offset, -1, 0, qiov, bytes, type); +} + static BlockAIOCB *paio_submit(BlockDriverState *bs, int fd, int64_t offset, QEMUIOVector *qiov, int bytes, BlockCompletionFunc *cb, void *opaque, int type) @@ -2312,6 +2365,35 @@ static void raw_abort_perm_update(BlockDriverState *= bs) raw_handle_perm_lock(bs, RAW_PL_ABORT, 0, 0, NULL); } =20 +static int coroutine_fn raw_co_copy_range_from(BlockDriverState *bs, + BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags) +{ + return bdrv_co_copy_range_to(src, src_offset, dst, dst_offset, bytes, = flags); +} + +static int coroutine_fn raw_co_copy_range_to(BlockDriverState *bs, + BdrvChild *src, uint64_t src_= offset, + BdrvChild *dst, uint64_t dst_= offset, + uint64_t bytes, BdrvRequestFl= ags flags) +{ + BDRVRawState *s =3D bs->opaque; + BDRVRawState *src_s; + + assert(dst->bs =3D=3D bs); + if (src->bs->drv->bdrv_co_copy_range_to !=3D raw_co_copy_range_to) { + return -ENOTSUP; + } + + src_s =3D src->opaque; + if (fd_open(bs) < 0 || fd_open(bs) < 0) { + return -EIO; + } + return paio_submit_co_full(bs, src_s->fd, src_offset, s->fd, dst_offse= t, + NULL, bytes, QEMU_AIO_COPY_RANGE); +} + BlockDriver bdrv_file =3D { .format_name =3D "file", .protocol_name =3D "file", @@ -2334,6 +2416,8 @@ BlockDriver bdrv_file =3D { .bdrv_co_pwritev =3D raw_co_pwritev, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_aio_pdiscard =3D raw_aio_pdiscard, + .bdrv_co_copy_range_from =3D raw_co_copy_range_from, + .bdrv_co_copy_range_to =3D raw_co_copy_range_to, .bdrv_refresh_limits =3D raw_refresh_limits, .bdrv_io_plug =3D raw_aio_plug, .bdrv_io_unplug =3D raw_aio_unplug, @@ -2811,6 +2895,10 @@ static BlockDriver bdrv_host_device =3D { .bdrv_co_pwritev =3D raw_co_pwritev, .bdrv_aio_flush =3D raw_aio_flush, .bdrv_aio_pdiscard =3D hdev_aio_pdiscard, + .bdrv_co_copy_range_from =3D raw_co_copy_range_from, + .bdrv_co_copy_range_to =3D raw_co_copy_range_to, + .bdrv_co_copy_range_from =3D raw_co_copy_range_from, + .bdrv_co_copy_range_to =3D raw_co_copy_range_to, .bdrv_refresh_limits =3D raw_refresh_limits, .bdrv_io_plug =3D raw_aio_plug, .bdrv_io_unplug =3D raw_aio_unplug, diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h index a4cdbbf1b7..324053020b 100644 --- a/include/block/raw-aio.h +++ b/include/block/raw-aio.h @@ -25,9 +25,15 @@ #define QEMU_AIO_FLUSH 0x0008 #define QEMU_AIO_DISCARD 0x0010 #define QEMU_AIO_WRITE_ZEROES 0x0020 +#define QEMU_AIO_COPY_RANGE 0x0040 #define QEMU_AIO_TYPE_MASK \ - (QEMU_AIO_READ|QEMU_AIO_WRITE|QEMU_AIO_IOCTL|QEMU_AIO_FLUSH| \ - QEMU_AIO_DISCARD|QEMU_AIO_WRITE_ZEROES) + (QEMU_AIO_READ | \ + QEMU_AIO_WRITE | \ + QEMU_AIO_IOCTL | \ + QEMU_AIO_FLUSH | \ + QEMU_AIO_DISCARD | \ + QEMU_AIO_WRITE_ZEROES | \ + QEMU_AIO_COPY_RANGE) =20 /* AIO flags */ #define QEMU_AIO_MISALIGNED 0x1000 --=20 2.14.3 From nobody Wed Oct 29 20:27:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525878250204609.8469179034253; Wed, 9 May 2018 08:04:10 -0700 (PDT) Received: from localhost ([::1]:56991 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQdV-0004sG-7S for importer@patchew.org; Wed, 09 May 2018 11:04:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56777) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQYZ-00076e-DD for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQYY-0006qQ-I6 for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:03 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47776 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQYV-0006kz-Rw; Wed, 09 May 2018 10:58:59 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5700140200A3; Wed, 9 May 2018 14:58:59 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0137ED74B9; Wed, 9 May 2018 14:58:54 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:11 +0800 Message-Id: <20180509145815.3330-6-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 09 May 2018 14:58:59 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 09 May 2018 14:58:59 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 5/9] iscsi: Query and save device designator when opening X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The device designator data returned in INQUIRY command will be useful to fill in source/target fields during copy offloading. Do this when connecting to the target and save the data for later use. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c | 41 +++++++++++++++++++++++++++++++++++++++++ include/scsi/constants.h | 2 ++ 2 files changed, 43 insertions(+) diff --git a/block/iscsi.c b/block/iscsi.c index d19ae0e398..5811ff1d51 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -68,6 +68,7 @@ typedef struct IscsiLun { QemuMutex mutex; struct scsi_inquiry_logical_block_provisioning lbp; struct scsi_inquiry_block_limits bl; + struct scsi_inquiry_device_designator *dd; unsigned char *zeroblock; /* The allocmap tracks which clusters (pages) on the iSCSI target are * allocated and which are not. In case a target returns zeros for @@ -1740,6 +1741,30 @@ static QemuOptsList runtime_opts =3D { }, }; =20 +static void iscsi_save_designator(IscsiLun *lun, + struct scsi_inquiry_device_identificatio= n *inq_di) +{ + struct scsi_inquiry_device_designator *desig, *copy =3D NULL; + + for (desig =3D inq_di->designators; desig; desig =3D desig->next) { + if (desig->association || + desig->designator_type > SCSI_DESIGNATOR_TYPE_NAA) { + continue; + } + /* NAA works better than T10 vendor ID based designator. */ + if (!copy || copy->designator_type < desig->designator_type) { + copy =3D desig; + } + } + if (copy) { + lun->dd =3D g_new(struct scsi_inquiry_device_designator, 1); + *lun->dd =3D *copy; + lun->dd->next =3D NULL; + lun->dd->designator =3D g_malloc(copy->designator_length); + memcpy(lun->dd->designator, copy->designator, copy->designator_len= gth); + } +} + static int iscsi_open(BlockDriverState *bs, QDict *options, int flags, Error **errp) { @@ -1922,6 +1947,7 @@ static int iscsi_open(BlockDriverState *bs, QDict *op= tions, int flags, struct scsi_task *inq_task; struct scsi_inquiry_logical_block_provisioning *inq_lbp; struct scsi_inquiry_block_limits *inq_bl; + struct scsi_inquiry_device_identification *inq_di; switch (inq_vpd->pages[i]) { case SCSI_INQUIRY_PAGECODE_LOGICAL_BLOCK_PROVISIONING: inq_task =3D iscsi_do_inquiry(iscsilun->iscsi, iscsilun->lun, = 1, @@ -1947,6 +1973,17 @@ static int iscsi_open(BlockDriverState *bs, QDict *o= ptions, int flags, sizeof(struct scsi_inquiry_block_limits)); scsi_free_scsi_task(inq_task); break; + case SCSI_INQUIRY_PAGECODE_DEVICE_IDENTIFICATION: + inq_task =3D iscsi_do_inquiry(iscsilun->iscsi, iscsilun->lun, = 1, + SCSI_INQUIRY_PAGECODE_DEVICE_IDENTIFIC= ATION, + (void **) &inq_di, errp); + if (inq_task =3D=3D NULL) { + ret =3D -EINVAL; + goto out; + } + iscsi_save_designator(iscsilun, inq_di); + scsi_free_scsi_task(inq_task); + break; default: break; } @@ -2003,6 +2040,10 @@ static void iscsi_close(BlockDriverState *bs) iscsi_logout_sync(iscsi); } iscsi_destroy_context(iscsi); + if (iscsilun->dd) { + g_free(iscsilun->dd->designator); + g_free(iscsilun->dd); + } g_free(iscsilun->zeroblock); iscsi_allocmap_free(iscsilun); qemu_mutex_destroy(&iscsilun->mutex); diff --git a/include/scsi/constants.h b/include/scsi/constants.h index a141dd71f8..54733b7110 100644 --- a/include/scsi/constants.h +++ b/include/scsi/constants.h @@ -311,4 +311,6 @@ #define MMC_PROFILE_HDDVD_RW_DL 0x005A #define MMC_PROFILE_INVALID 0xFFFF =20 +#define IDENT_DESCR_TGT_DESCR 0xE4 + #endif --=20 2.14.3 From nobody Wed Oct 29 20:27:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525878399739446.81501595935924; Wed, 9 May 2018 08:06:39 -0700 (PDT) Received: from localhost ([::1]:57009 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQfr-000789-1h for importer@patchew.org; Wed, 09 May 2018 11:06:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56846) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQYn-0007UP-7Z for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQYm-0007Hj-9N for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:17 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45916 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQYe-00072j-Pz; Wed, 09 May 2018 10:59:08 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4AD174059FEB; Wed, 9 May 2018 14:59:08 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0FF95D74B9; Wed, 9 May 2018 14:58:59 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:12 +0800 Message-Id: <20180509145815.3330-7-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:59:08 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:59:08 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 6/9] iscsi: Create and use iscsi_co_wait_for_task X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This loop is repeated a growing number times. Make a helper. Signed-off-by: Fam Zheng Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/iscsi.c | 54 +++++++++++++++++------------------------------------- 1 file changed, 17 insertions(+), 37 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index 5811ff1d51..32e755c021 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -556,6 +556,17 @@ static inline bool iscsi_allocmap_is_valid(IscsiLun *i= scsilun, offset / iscsilun->cluster_size) =3D=3D siz= e); } =20 +static void coroutine_fn iscsi_co_wait_for_task(IscsiTask *iTask, + IscsiLun *iscsilun) +{ + while (!iTask->complete) { + iscsi_set_events(iscsilun); + qemu_mutex_unlock(&iscsilun->mutex); + qemu_coroutine_yield(); + qemu_mutex_lock(&iscsilun->mutex); + } +} + static int coroutine_fn iscsi_co_writev_flags(BlockDriverState *bs, int64_t sector_num, int nb_sec= tors, QEMUIOVector *iov, int flags) @@ -617,12 +628,7 @@ retry: scsi_task_set_iov_out(iTask.task, (struct scsi_iovec *) iov->iov, iov->niov); #endif - while (!iTask.complete) { - iscsi_set_events(iscsilun); - qemu_mutex_unlock(&iscsilun->mutex); - qemu_coroutine_yield(); - qemu_mutex_lock(&iscsilun->mutex); - } + iscsi_co_wait_for_task(&iTask, iscsilun); =20 if (iTask.task !=3D NULL) { scsi_free_scsi_task(iTask.task); @@ -693,13 +699,7 @@ retry: ret =3D -ENOMEM; goto out_unlock; } - - while (!iTask.complete) { - iscsi_set_events(iscsilun); - qemu_mutex_unlock(&iscsilun->mutex); - qemu_coroutine_yield(); - qemu_mutex_lock(&iscsilun->mutex); - } + iscsi_co_wait_for_task(&iTask, iscsilun); =20 if (iTask.do_retry) { if (iTask.task !=3D NULL) { @@ -863,13 +863,8 @@ retry: #if LIBISCSI_API_VERSION < (20160603) scsi_task_set_iov_in(iTask.task, (struct scsi_iovec *) iov->iov, iov->= niov); #endif - while (!iTask.complete) { - iscsi_set_events(iscsilun); - qemu_mutex_unlock(&iscsilun->mutex); - qemu_coroutine_yield(); - qemu_mutex_lock(&iscsilun->mutex); - } =20 + iscsi_co_wait_for_task(&iTask, iscsilun); if (iTask.task !=3D NULL) { scsi_free_scsi_task(iTask.task); iTask.task =3D NULL; @@ -906,12 +901,7 @@ retry: return -ENOMEM; } =20 - while (!iTask.complete) { - iscsi_set_events(iscsilun); - qemu_mutex_unlock(&iscsilun->mutex); - qemu_coroutine_yield(); - qemu_mutex_lock(&iscsilun->mutex); - } + iscsi_co_wait_for_task(&iTask, iscsilun); =20 if (iTask.task !=3D NULL) { scsi_free_scsi_task(iTask.task); @@ -1143,12 +1133,7 @@ retry: goto out_unlock; } =20 - while (!iTask.complete) { - iscsi_set_events(iscsilun); - qemu_mutex_unlock(&iscsilun->mutex); - qemu_coroutine_yield(); - qemu_mutex_lock(&iscsilun->mutex); - } + iscsi_co_wait_for_task(&iTask, iscsilun); =20 if (iTask.task !=3D NULL) { scsi_free_scsi_task(iTask.task); @@ -1244,12 +1229,7 @@ retry: return -ENOMEM; } =20 - while (!iTask.complete) { - iscsi_set_events(iscsilun); - qemu_mutex_unlock(&iscsilun->mutex); - qemu_coroutine_yield(); - qemu_mutex_lock(&iscsilun->mutex); - } + iscsi_co_wait_for_task(&iTask, iscsilun); =20 if (iTask.status =3D=3D SCSI_STATUS_CHECK_CONDITION && iTask.task->sense.key =3D=3D SCSI_SENSE_ILLEGAL_REQUEST && --=20 2.14.3 From nobody Wed Oct 29 20:27:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525878501722574.6728277107642; Wed, 9 May 2018 08:08:21 -0700 (PDT) Received: from localhost ([::1]:57018 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQhY-0008Kn-Nb for importer@patchew.org; Wed, 09 May 2018 11:08:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56869) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQYo-0007Yd-NY for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQYn-0007KN-4R for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:18 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45926 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQYj-0007CN-EE; Wed, 09 May 2018 10:59:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ECA7E4059FE3; Wed, 9 May 2018 14:59:12 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF1C5D74BA; Wed, 9 May 2018 14:59:08 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:13 +0800 Message-Id: <20180509145815.3330-8-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:59:12 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 09 May 2018 14:59:12 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 7/9] iscsi: Implement copy offloading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Issue EXTENDED COPY (LID1) command to implement the copy_range API. The parameter data construction code is modified from libiscsi's iscsi-dd.c. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c | 228 +++++++++++++++++++++++++++++++++++++++++++= ++++ include/scsi/constants.h | 3 + 2 files changed, 231 insertions(+) diff --git a/block/iscsi.c b/block/iscsi.c index 32e755c021..b736c75db5 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -2205,6 +2205,230 @@ static void coroutine_fn iscsi_co_invalidate_cache(= BlockDriverState *bs, iscsi_allocmap_invalidate(iscsilun); } =20 +static int coroutine_fn iscsi_co_copy_range_from(BlockDriverState *bs, + BdrvChild *src, + uint64_t src_offset, + BdrvChild *dst, + uint64_t dst_offset, + uint64_t bytes, + BdrvRequestFlags flags) +{ + return bdrv_co_copy_range_to(src, src_offset, dst, dst_offset, bytes, = flags); +} + +static struct scsi_task *iscsi_xcopy_task(int param_len) +{ + struct scsi_task *task; + + task =3D g_new0(struct scsi_task, 1); + + task->cdb[0] =3D EXTENDED_COPY; + task->cdb[10] =3D (param_len >> 24) & 0xFF; + task->cdb[11] =3D (param_len >> 16) & 0xFF; + task->cdb[12] =3D (param_len >> 8) & 0xFF; + task->cdb[13] =3D param_len & 0xFF; + task->cdb_size =3D 16; + task->xfer_dir =3D SCSI_XFER_WRITE; + task->expxferlen =3D param_len; + + return task; +} + +static int iscsi_populate_target_desc(unsigned char *desc, IscsiLun *lun) +{ + struct scsi_inquiry_device_designator *dd =3D lun->dd; + + memset(desc, 0, 32); + desc[0] =3D IDENT_DESCR_TGT_DESCR; + desc[4] =3D dd->code_set; + desc[5] =3D (dd->designator_type & 0xF) + | ((dd->association & 3) << 4); + desc[7] =3D dd->designator_length; + memcpy(desc + 8, dd->designator, dd->designator_length); + + desc[28] =3D 0; + desc[29] =3D (lun->block_size >> 16) & 0xFF; + desc[30] =3D (lun->block_size >> 8) & 0xFF; + desc[31] =3D lun->block_size & 0xFF; + + return 32; +} + +static int iscsi_xcopy_desc_hdr(uint8_t *hdr, int dc, int cat, int src_ind= ex, + int dst_index) +{ + int desc_len =3D 28; + + hdr[0] =3D 0x02; /* BLK_TO_BLK_SEG_DESCR */ + hdr[1] =3D ((dc << 1) | cat) & 0xFF; + hdr[2] =3D (desc_len >> 8) & 0xFF; + /* don't account for the first 4 bytes in descriptor header*/ + hdr[3] =3D (desc_len - 4 /* SEG_DESC_SRC_INDEX_OFFSET */) & 0xFF; + hdr[4] =3D (src_index >> 8) & 0xFF; + hdr[5] =3D src_index & 0xFF; + hdr[6] =3D (dst_index >> 8) & 0xFF; + hdr[7] =3D dst_index & 0xFF; + + return desc_len; +} + +static int iscsi_xcopy_populate_desc(uint8_t *desc, int dc, int cat, + int src_index, int dst_index, int num= _blks, + uint64_t src_lba, uint64_t dst_lba) +{ + int desc_len =3D iscsi_xcopy_desc_hdr(desc, dc, cat, + src_index, dst_index); + + /* The caller should verify the request size */ + assert(num_blks < 65536); + desc[10] =3D (num_blks >> 8) & 0xFF; + desc[11] =3D num_blks & 0xFF; + desc[12] =3D (src_lba >> 56) & 0xFF; + desc[13] =3D (src_lba >> 48) & 0xFF; + desc[14] =3D (src_lba >> 40) & 0xFF; + desc[15] =3D (src_lba >> 32) & 0xFF; + desc[16] =3D (src_lba >> 24) & 0xFF; + desc[17] =3D (src_lba >> 16) & 0xFF; + desc[18] =3D (src_lba >> 8) & 0xFF; + desc[19] =3D src_lba & 0xFF; + desc[20] =3D (dst_lba >> 56) & 0xFF; + desc[21] =3D (dst_lba >> 48) & 0xFF; + desc[22] =3D (dst_lba >> 40) & 0xFF; + desc[23] =3D (dst_lba >> 32) & 0xFF; + desc[24] =3D (dst_lba >> 24) & 0xFF; + desc[25] =3D (dst_lba >> 16) & 0xFF; + desc[26] =3D (dst_lba >> 8) & 0xFF; + desc[27] =3D dst_lba & 0xFF; + + return desc_len; +} + +static void iscsi_xcopy_populate_header(unsigned char *buf, int list_id, i= nt str, + int list_id_usage, int prio, + int tgt_desc_len, + int seg_desc_len, int inline_data_= len) +{ + buf[0] =3D list_id; + buf[1] =3D ((str & 1) << 5) | ((list_id_usage & 3) << 3) | (prio & 7); + buf[2] =3D (tgt_desc_len >> 8) & 0xFF; + buf[3] =3D tgt_desc_len & 0xFF; + buf[8] =3D (seg_desc_len >> 24) & 0xFF; + buf[9] =3D (seg_desc_len >> 16) & 0xFF; + buf[10] =3D (seg_desc_len >> 8) & 0xFF; + buf[11] =3D seg_desc_len & 0xFF; + buf[12] =3D (inline_data_len >> 24) & 0xFF; + buf[13] =3D (inline_data_len >> 16) & 0xFF; + buf[14] =3D (inline_data_len >> 8) & 0xFF; + buf[15] =3D inline_data_len & 0xFF; +} + +static void iscsi_xcopy_data(struct iscsi_data *data, + IscsiLun *src, int64_t src_lba, + IscsiLun *dst, int64_t dst_lba, + uint16_t num_blocks) +{ + uint8_t *buf; + const int src_offset =3D XCOPY_DESC_OFFSET; + const int dst_offset =3D XCOPY_DESC_OFFSET + IDENT_DESCR_TGT_DESCR_SIZ= E; + const int seg_offset =3D dst_offset + IDENT_DESCR_TGT_DESCR_SIZE; + + data->size =3D XCOPY_DESC_OFFSET + + IDENT_DESCR_TGT_DESCR_SIZE * 2 + + XCOPY_BLK2BLK_SEG_DESC_SIZE; + data->data =3D g_malloc0(data->size); + buf =3D data->data; + + /* Initialise the parameter list header */ + iscsi_xcopy_populate_header(buf, 1, 0, 2 /* LIST_ID_USAGE_DISCARD */, + 0, 2 * IDENT_DESCR_TGT_DESCR_SIZE, + XCOPY_BLK2BLK_SEG_DESC_SIZE, + 0); + + /* Initialise CSCD list with one src + one dst descriptor */ + iscsi_populate_target_desc(&buf[src_offset], src); + iscsi_populate_target_desc(&buf[dst_offset], dst); + + /* Initialise one segment descriptor */ + iscsi_xcopy_populate_desc(&buf[seg_offset], 0, 0, 0, 1, num_blocks, + src_lba, dst_lba); +} + +static int coroutine_fn iscsi_co_copy_range_to(BlockDriverState *bs, + BdrvChild *src, + uint64_t src_offset, + BdrvChild *dst, + uint64_t dst_offset, + uint64_t bytes, + BdrvRequestFlags flags) +{ + IscsiLun *dst_lun =3D dst->bs->opaque; + IscsiLun *src_lun; + struct IscsiTask iscsi_task; + struct iscsi_data data; + int r =3D 0; + int block_size; + + if (src->bs->drv->bdrv_co_copy_range_to !=3D iscsi_co_copy_range_to) { + return -ENOTSUP; + } + src_lun =3D src->bs->opaque; + + if (!src_lun->dd || !dst_lun->dd) { + return -ENOTSUP; + } + if (!is_byte_request_lun_aligned(dst_offset, bytes, dst_lun)) { + return -ENOTSUP; + } + if (!is_byte_request_lun_aligned(src_offset, bytes, src_lun)) { + return -ENOTSUP; + } + if (dst_lun->block_size !=3D src_lun->block_size || + !dst_lun->block_size) { + return -ENOTSUP; + } + + block_size =3D dst_lun->block_size; + if (bytes / block_size > 65535) { + return -ENOTSUP; + } + + iscsi_xcopy_data(&data, + src_lun, src_offset / block_size, + dst_lun, dst_offset / block_size, + bytes / block_size); + + iscsi_co_init_iscsitask(dst_lun, &iscsi_task); + + qemu_mutex_lock(&dst_lun->mutex); + iscsi_task.task =3D iscsi_xcopy_task(data.size); +retry: + if (iscsi_scsi_command_async(dst_lun->iscsi, dst_lun->lun, + iscsi_task.task, iscsi_co_generic_cb, + &data, + &iscsi_task) !=3D 0) { + r =3D -EIO; + goto out_unlock; + } + + iscsi_co_wait_for_task(&iscsi_task, dst_lun); + + if (iscsi_task.do_retry) { + iscsi_task.complete =3D 0; + goto retry; + } + + if (iscsi_task.status !=3D SCSI_STATUS_GOOD) { + r =3D iscsi_task.err_code; + goto out_unlock; + } + +out_unlock: + g_free(iscsi_task.task); + qemu_mutex_unlock(&dst_lun->mutex); + g_free(iscsi_task.err_str); + return r; +} + static QemuOptsList iscsi_create_opts =3D { .name =3D "iscsi-create-opts", .head =3D QTAILQ_HEAD_INITIALIZER(iscsi_create_opts.head), @@ -2239,6 +2463,8 @@ static BlockDriver bdrv_iscsi =3D { =20 .bdrv_co_block_status =3D iscsi_co_block_status, .bdrv_co_pdiscard =3D iscsi_co_pdiscard, + .bdrv_co_copy_range_from =3D iscsi_co_copy_range_from, + .bdrv_co_copy_range_to =3D iscsi_co_copy_range_to, .bdrv_co_pwrite_zeroes =3D iscsi_co_pwrite_zeroes, .bdrv_co_readv =3D iscsi_co_readv, .bdrv_co_writev_flags =3D iscsi_co_writev_flags, @@ -2274,6 +2500,8 @@ static BlockDriver bdrv_iser =3D { =20 .bdrv_co_block_status =3D iscsi_co_block_status, .bdrv_co_pdiscard =3D iscsi_co_pdiscard, + .bdrv_co_copy_range_from =3D iscsi_co_copy_range_from, + .bdrv_co_copy_range_to =3D iscsi_co_copy_range_to, .bdrv_co_pwrite_zeroes =3D iscsi_co_pwrite_zeroes, .bdrv_co_readv =3D iscsi_co_readv, .bdrv_co_writev_flags =3D iscsi_co_writev_flags, diff --git a/include/scsi/constants.h b/include/scsi/constants.h index 54733b7110..b53ff8adcb 100644 --- a/include/scsi/constants.h +++ b/include/scsi/constants.h @@ -311,6 +311,9 @@ #define MMC_PROFILE_HDDVD_RW_DL 0x005A #define MMC_PROFILE_INVALID 0xFFFF =20 +#define XCOPY_DESC_OFFSET 16 #define IDENT_DESCR_TGT_DESCR 0xE4 +#define IDENT_DESCR_TGT_DESCR_SIZE 32 +#define XCOPY_BLK2BLK_SEG_DESC_SIZE 28 =20 #endif --=20 2.14.3 From nobody Wed Oct 29 20:27:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525878096038810.6807612104473; Wed, 9 May 2018 08:01:36 -0700 (PDT) Received: from localhost ([::1]:56976 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQb1-0001Lr-9w for importer@patchew.org; Wed, 09 May 2018 11:01:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56908) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQYt-0007o4-TQ for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQYt-0007WT-5n for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:23 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47814 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQYp-0007Nb-3n; Wed, 09 May 2018 10:59:19 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 67BAC40200A0; Wed, 9 May 2018 14:59:18 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA3AFD74B9; Wed, 9 May 2018 14:59:13 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:14 +0800 Message-Id: <20180509145815.3330-9-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 09 May 2018 14:59:18 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 09 May 2018 14:59:18 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 8/9] block-backend: Add blk_co_copy_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It's a BlockBackend wrapper of the BDS interface. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 18 ++++++++++++++++++ include/sysemu/block-backend.h | 4 ++++ 2 files changed, 22 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index 681b240b12..5562ec4238 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -2217,3 +2217,21 @@ void blk_unregister_buf(BlockBackend *blk, void *hos= t) { bdrv_unregister_buf(blk_bs(blk), host); } + +int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, + BlockBackend *blk_out, int64_t off_out, + int bytes, BdrvRequestFlags flags) +{ + int r; + r =3D blk_check_byte_request(blk_in, off_in, bytes); + if (r) { + return r; + } + r =3D blk_check_byte_request(blk_out, off_out, bytes); + if (r) { + return r; + } + return bdrv_co_copy_range(blk_in->root, off_in, + blk_out->root, off_out, + bytes, flags); +} diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h index 92ab624fac..6f043b6b51 100644 --- a/include/sysemu/block-backend.h +++ b/include/sysemu/block-backend.h @@ -232,4 +232,8 @@ void blk_set_force_allow_inactivate(BlockBackend *blk); void blk_register_buf(BlockBackend *blk, void *host, size_t size); void blk_unregister_buf(BlockBackend *blk, void *host); =20 +int blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, + BlockBackend *blk_out, int64_t off_out, + int bytes, BdrvRequestFlags flags); + #endif --=20 2.14.3 From nobody Wed Oct 29 20:27:44 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525878279319696.2975852469408; Wed, 9 May 2018 08:04:39 -0700 (PDT) Received: from localhost ([::1]:56992 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQdy-0005D4-7H for importer@patchew.org; Wed, 09 May 2018 11:04:38 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56961) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQZ7-0008SX-J7 for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQZ6-0007wj-2N for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:37 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:37318 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQYy-0007gD-0P; Wed, 09 May 2018 10:59:28 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 60B8B81A88A2; Wed, 9 May 2018 14:59:27 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id 14A8CD74BA; Wed, 9 May 2018 14:59:18 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:15 +0800 Message-Id: <20180509145815.3330-10-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 09 May 2018 14:59:27 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 09 May 2018 14:59:27 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 9/9] qemu-img: Convert with copy offloading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The new blk_co_copy_range interface offers a more efficient way in the case of network based storage. Make use of it to allow faster convert operation. Since copy offloading cannot do zero detection ('-S') and compression (-c), only try it when these options are not used. Update 178.out.qcow2 as we are now a little more efficient on the actual file size. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- qemu-img.c | 50 ++++++++++++++++++++++++++++++++++++= ++-- tests/qemu-iotests/178.out.qcow2 | 8 +++---- 2 files changed, 52 insertions(+), 6 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index ea62d2d61e..9c3678ebf0 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -1544,6 +1544,7 @@ typedef struct ImgConvertState { bool compressed; bool target_has_backing; bool wr_in_order; + bool copy_range; int min_sparse; size_t cluster_sectors; size_t buf_sectors; @@ -1737,6 +1738,37 @@ static int coroutine_fn convert_co_write(ImgConvertS= tate *s, int64_t sector_num, return 0; } =20 +static int coroutine_fn convert_co_copy_range(ImgConvertState *s, int64_t = sector_num, + int nb_sectors) +{ + int n, ret; + + while (nb_sectors > 0) { + BlockBackend *blk; + int src_cur; + int64_t bs_sectors, src_cur_offset; + int64_t offset; + + convert_select_part(s, sector_num, &src_cur, &src_cur_offset); + offset =3D (sector_num - src_cur_offset) << BDRV_SECTOR_BITS; + blk =3D s->src[src_cur]; + bs_sectors =3D s->src_sectors[src_cur]; + + n =3D MIN(nb_sectors, bs_sectors - (sector_num - src_cur_offset)); + + ret =3D blk_co_copy_range(blk, offset, s->target, + sector_num << BDRV_SECTOR_BITS, + n << BDRV_SECTOR_BITS, 0); + if (ret < 0) { + return ret; + } + + sector_num +=3D n; + nb_sectors -=3D n; + } + return 0; +} + static void coroutine_fn convert_co_do_copy(void *opaque) { ImgConvertState *s =3D opaque; @@ -1759,6 +1791,7 @@ static void coroutine_fn convert_co_do_copy(void *opa= que) int n; int64_t sector_num; enum ImgConvertBlockStatus status; + bool copy_range; =20 qemu_co_mutex_lock(&s->lock); if (s->ret !=3D -EINPROGRESS || s->sector_num >=3D s->total_sector= s) { @@ -1788,7 +1821,9 @@ static void coroutine_fn convert_co_do_copy(void *opa= que) s->allocated_sectors, 0); } =20 - if (status =3D=3D BLK_DATA) { +retry: + copy_range =3D s->copy_range && s->status =3D=3D BLK_DATA; + if (status =3D=3D BLK_DATA && !copy_range) { ret =3D convert_co_read(s, sector_num, n, buf); if (ret < 0) { error_report("error while reading sector %" PRId64 @@ -1810,7 +1845,15 @@ static void coroutine_fn convert_co_do_copy(void *op= aque) } =20 if (s->ret =3D=3D -EINPROGRESS) { - ret =3D convert_co_write(s, sector_num, n, buf, status); + if (copy_range) { + ret =3D convert_co_copy_range(s, sector_num, n); + if (ret) { + s->copy_range =3D false; + goto retry; + } + } else { + ret =3D convert_co_write(s, sector_num, n, buf, status); + } if (ret < 0) { error_report("error while writing sector %" PRId64 ": %s", sector_num, strerror(-ret)); @@ -1933,6 +1976,7 @@ static int img_convert(int argc, char **argv) ImgConvertState s =3D (ImgConvertState) { /* Need at least 4k of zeros for sparse detection */ .min_sparse =3D 8, + .copy_range =3D true, .buf_sectors =3D IO_BUF_SIZE / BDRV_SECTOR_SIZE, .wr_in_order =3D true, .num_coroutines =3D 8, @@ -1973,6 +2017,7 @@ static int img_convert(int argc, char **argv) break; case 'c': s.compressed =3D true; + s.copy_range =3D false; break; case 'o': if (!is_valid_option_list(optarg)) { @@ -2014,6 +2059,7 @@ static int img_convert(int argc, char **argv) } =20 s.min_sparse =3D sval / BDRV_SECTOR_SIZE; + s.copy_range =3D false; break; } case 'p': diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.= qcow2 index d42d4a4597..3e72893451 100644 --- a/tests/qemu-iotests/178.out.qcow2 +++ b/tests/qemu-iotests/178.out.qcow2 @@ -54,7 +54,7 @@ wrote 64512/64512 bytes at offset 134217728 required size: 589824 fully allocated size: 1074135040 =20 -converted image file size in bytes: 524288 +converted image file size in bytes: 523264 =20 =3D=3D qcow2 input image with internal snapshot (human) =3D=3D =20 @@ -107,7 +107,7 @@ wrote 64512/64512 bytes at offset 134217728 required size: 589824 fully allocated size: 1074135040 =20 -converted image file size in bytes: 524288 +converted image file size in bytes: 523264 =20 =3D=3D raw input image and a backing file (human) =3D=3D =20 @@ -187,7 +187,7 @@ wrote 64512/64512 bytes at offset 134217728 "fully-allocated": 1074135040 } =20 -converted image file size in bytes: 524288 +converted image file size in bytes: 523264 =20 =3D=3D qcow2 input image with internal snapshot (json) =3D=3D =20 @@ -254,7 +254,7 @@ wrote 64512/64512 bytes at offset 134217728 "fully-allocated": 1074135040 } =20 -converted image file size in bytes: 524288 +converted image file size in bytes: 523264 =20 =3D=3D raw input image and a backing file (json) =3D=3D =20 --=20 2.14.3