From nobody Sat Nov 1 22:28:56 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1527734216119442.8413406373212; Wed, 30 May 2018 19:36:56 -0700 (PDT) Received: from localhost ([::1]:41637 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fODSQ-0004J3-S7 for importer@patchew.org; Wed, 30 May 2018 22:36:54 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43304) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fODQf-0003MB-Pb for qemu-devel@nongnu.org; Wed, 30 May 2018 22:35:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fODQf-0001k0-0J for qemu-devel@nongnu.org; Wed, 30 May 2018 22:35:05 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:38366 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fODQd-0001jA-1E; Wed, 30 May 2018 22:35:03 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 992C44067F19; Thu, 31 May 2018 02:35:02 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-47.pek2.redhat.com [10.72.12.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id E42F42023450; Thu, 31 May 2018 02:34:58 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 31 May 2018 10:34:44 +0800 Message-Id: <20180531023445.5923-2-famz@redhat.com> In-Reply-To: <20180531023445.5923-1-famz@redhat.com> References: <20180531023445.5923-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Thu, 31 May 2018 02:35:02 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Thu, 31 May 2018 02:35:02 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH 1/2] block: Honour BDRV_REQ_NO_SERIALISING in copy range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Jeff Cody , Max Reitz , Stefan Hajnoczi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This semantics is needed by drive-backup so implement it before using this API there. Signed-off-by: Fam Zheng --- block/io.c | 6 ++++-- include/block/block.h | 5 +++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/block/io.c b/block/io.c index b7beaeeb9f..f55e0c39fe 100644 --- a/block/io.c +++ b/block/io.c @@ -2920,8 +2920,10 @@ int coroutine_fn bdrv_co_copy_range(BdrvChild *src, = uint64_t src_offset, tracked_request_begin(&dst_req, dst_bs, dst_offset, bytes, BDRV_TRACKED_WRITE); =20 - wait_serialising_requests(&src_req); - wait_serialising_requests(&dst_req); + if (!(flags & BDRV_REQ_NO_SERIALISING)) { + wait_serialising_requests(&src_req); + wait_serialising_requests(&dst_req); + } ret =3D bdrv_co_copy_range_from(src, src_offset, dst, dst_offset, bytes, flags); diff --git a/include/block/block.h b/include/block/block.h index 6cc6c7e699..6d3d156927 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -630,13 +630,14 @@ void bdrv_unregister_buf(BlockDriverState *bs, void *= host); * @dst: Destination child to copy data to * @dst_offset: offset in @dst image to write data * @bytes: number of bytes to copy - * @flags: request flags. Must be one of: - * 0 - actually read data from src; + * @flags: request flags. Supported flags: * BDRV_REQ_ZERO_WRITE - treat the @src range as zero data and do = zero * write on @dst as if bdrv_co_pwrite_zeroes= is * called. Used to simplify caller code, or * during BlockDriver.bdrv_co_copy_range_fro= m() * recursion. + * BDRV_REQ_NO_SERIALISING - do not serialize with other overlappi= ng + * requests currently in flight. * * Returns: 0 if succeeded; negative error code if failed. **/ --=20 2.14.3 From nobody Sat Nov 1 22:28:56 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1527734231896696.5494804848018; Wed, 30 May 2018 19:37:11 -0700 (PDT) Received: from localhost ([::1]:41639 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fODSe-0004UR-Qv for importer@patchew.org; Wed, 30 May 2018 22:37:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43344) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fODQp-0003TI-7C for qemu-devel@nongnu.org; Wed, 30 May 2018 22:35:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fODQj-0001lB-LX for qemu-devel@nongnu.org; Wed, 30 May 2018 22:35:15 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:52834 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fODQg-0001kN-Sg; Wed, 30 May 2018 22:35:06 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7275284257; Thu, 31 May 2018 02:35:06 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-47.pek2.redhat.com [10.72.12.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id 388942023451; Thu, 31 May 2018 02:35:02 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Thu, 31 May 2018 10:34:45 +0800 Message-Id: <20180531023445.5923-3-famz@redhat.com> In-Reply-To: <20180531023445.5923-1-famz@redhat.com> References: <20180531023445.5923-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 31 May 2018 02:35:06 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 31 May 2018 02:35:06 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH 2/2] backup: Use copy offloading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Jeff Cody , Max Reitz , Stefan Hajnoczi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The implementation is similar to the 'qemu-img convert'. In the beginning of the job, offloaded copy is attempted. If it fails, further I/O will go through the existing bounce buffer code path. Signed-off-by: Fam Zheng --- block/backup.c | 93 +++++++++++++++++++++++++++++++++++---------------= ---- block/trace-events | 1 + 2 files changed, 62 insertions(+), 32 deletions(-) diff --git a/block/backup.c b/block/backup.c index 4e228e959b..ab189693f4 100644 --- a/block/backup.c +++ b/block/backup.c @@ -45,6 +45,8 @@ typedef struct BackupBlockJob { QLIST_HEAD(, CowRequest) inflight_reqs; =20 HBitmap *copy_bitmap; + bool use_copy_range; + int64_t copy_range_size; } BackupBlockJob; =20 static const BlockJobDriver backup_job_driver; @@ -111,49 +113,70 @@ static int coroutine_fn backup_do_cow(BackupBlockJob = *job, cow_request_begin(&cow_request, job, start, end); =20 for (; start < end; start +=3D job->cluster_size) { +retry: if (!hbitmap_get(job->copy_bitmap, start / job->cluster_size)) { trace_backup_do_cow_skip(job, start); continue; /* already copied */ } - hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1); =20 trace_backup_do_cow_process(job, start); =20 - n =3D MIN(job->cluster_size, job->len - start); - - if (!bounce_buffer) { - bounce_buffer =3D blk_blockalign(blk, job->cluster_size); - } - iov.iov_base =3D bounce_buffer; - iov.iov_len =3D n; - qemu_iovec_init_external(&bounce_qiov, &iov, 1); - - ret =3D blk_co_preadv(blk, start, bounce_qiov.size, &bounce_qiov, - is_write_notifier ? BDRV_REQ_NO_SERIALISING : = 0); - if (ret < 0) { - trace_backup_do_cow_read_fail(job, start, ret); - if (error_is_read) { - *error_is_read =3D true; + if (!job->use_copy_range) { + hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1); + n =3D MIN(job->cluster_size, job->len - start); + if (!bounce_buffer) { + bounce_buffer =3D blk_blockalign(blk, job->cluster_size); } - hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1); - goto out; - } + iov.iov_base =3D bounce_buffer; + iov.iov_len =3D n; + qemu_iovec_init_external(&bounce_qiov, &iov, 1); =20 - if (buffer_is_zero(iov.iov_base, iov.iov_len)) { - ret =3D blk_co_pwrite_zeroes(job->target, start, - bounce_qiov.size, BDRV_REQ_MAY_UNMA= P); + ret =3D blk_co_preadv(blk, start, bounce_qiov.size, &bounce_qi= ov, + is_write_notifier ? BDRV_REQ_NO_SERIALISIN= G : 0); + if (ret < 0) { + trace_backup_do_cow_read_fail(job, start, ret); + if (error_is_read) { + *error_is_read =3D true; + } + hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1= ); + goto out; + } + + if (buffer_is_zero(iov.iov_base, iov.iov_len)) { + ret =3D blk_co_pwrite_zeroes(job->target, start, + bounce_qiov.size, BDRV_REQ_MAY_= UNMAP); + } else { + ret =3D blk_co_pwritev(job->target, start, + bounce_qiov.size, &bounce_qiov, + job->compress ? BDRV_REQ_WRITE_COMPRE= SSED : 0); + } + if (ret < 0) { + trace_backup_do_cow_write_fail(job, start, ret); + if (error_is_read) { + *error_is_read =3D false; + } + hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1= ); + goto out; + } } else { - ret =3D blk_co_pwritev(job->target, start, - bounce_qiov.size, &bounce_qiov, - job->compress ? BDRV_REQ_WRITE_COMPRESSED= : 0); - } - if (ret < 0) { - trace_backup_do_cow_write_fail(job, start, ret); - if (error_is_read) { - *error_is_read =3D false; + int nr_clusters; + + assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size= )); + n =3D MIN(job->copy_range_size, job->len - start); + n =3D MIN(bytes, n); + nr_clusters =3D DIV_ROUND_UP(n, job->cluster_size); + hbitmap_reset(job->copy_bitmap, start / job->cluster_size, + nr_clusters); + ret =3D blk_co_copy_range(blk, start, job->target, start, n, + is_write_notifier ? BDRV_REQ_NO_SERIAL= ISING : 0); + if (ret < 0) { + job->use_copy_range =3D false; + trace_backup_do_cow_copy_range_fail(job, start, ret); + hbitmap_set(job->copy_bitmap, start / job->cluster_size, + nr_clusters); + /* Retry with bounce buffer. */ + goto retry; } - hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1); - goto out; } =20 /* Publish progress, guest I/O counts as progress too. Note that = the @@ -665,6 +688,12 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, } else { job->cluster_size =3D MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster= _size); } + job->use_copy_range =3D true; + job->copy_range_size =3D MIN_NON_ZERO(blk_get_max_transfer(job->common= .blk), + blk_get_max_transfer(job->target)); + job->copy_range_size =3D MAX(job->cluster_size, + QEMU_ALIGN_UP(job->copy_range_size, + job->cluster_size)); =20 /* Required permissions are already taken with target's blk_new() */ block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, diff --git a/block/trace-events b/block/trace-events index 2d59b53fd3..c35287b48a 100644 --- a/block/trace-events +++ b/block/trace-events @@ -42,6 +42,7 @@ backup_do_cow_skip(void *job, int64_t start) "job %p star= t %"PRId64 backup_do_cow_process(void *job, int64_t start) "job %p start %"PRId64 backup_do_cow_read_fail(void *job, int64_t start, int ret) "job %p start %= "PRId64" ret %d" backup_do_cow_write_fail(void *job, int64_t start, int ret) "job %p start = %"PRId64" ret %d" +backup_do_cow_copy_range_fail(void *job, int64_t start, int ret) "job %p s= tart %"PRId64" ret %d" =20 # blockdev.c qmp_block_job_cancel(void *job) "job %p" --=20 2.14.3