From nobody Sun Oct 5 21:01:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1499289402314322.2772290815011; Wed, 5 Jul 2017 14:16:42 -0700 (PDT) Received: from localhost ([::1]:48093 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dSrf6-00032P-TE for importer@patchew.org; Wed, 05 Jul 2017 17:16:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38250) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dSrYH-0005JD-3E for qemu-devel@nongnu.org; Wed, 05 Jul 2017 17:09:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dSrYF-0002sy-7a for qemu-devel@nongnu.org; Wed, 05 Jul 2017 17:09:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36914) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dSrYB-0002rG-4T; Wed, 05 Jul 2017 17:09:31 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2C63561B83; Wed, 5 Jul 2017 21:09:30 +0000 (UTC) Received: from red.redhat.com (ovpn-120-36.rdu2.redhat.com [10.10.120.36]) by smtp.corp.redhat.com (Postfix) with ESMTP id 033F1600CC; Wed, 5 Jul 2017 21:09:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 2C63561B83 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 2C63561B83 From: Eric Blake To: qemu-devel@nongnu.org Date: Wed, 5 Jul 2017 16:08:42 -0500 Message-Id: <20170705210842.960-22-eblake@redhat.com> In-Reply-To: <20170705210842.960-1-eblake@redhat.com> References: <20170705210842.960-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 05 Jul 2017 21:09:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 21/21] block: Make bdrv_is_allocated_above() byte-based X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Fam Zheng , qemu-block@nongnu.org, Wen Congyang , jcody@redhat.com, Max Reitz , Stefan Hajnoczi , jsnow@redhat.com, Xie Changlong Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We are gradually moving away from sector-based interfaces, towards byte-based. In the common case, allocation is unlikely to ever use values that are not naturally sector-aligned, but it is possible that byte-based values will let us be more precise about allocation at the end of an unaligned file that can do byte-based access. Changing the signature of the function to use int64_t *pnum ensures that the compiler enforces that all callers are updated. For now, the io.c layer still assert()s that all callers are sector-aligned, but that can be relaxed when a later patch implements byte-based block status. Therefore, for the most part this patch is just the addition of scaling at the callers followed by inverse scaling at bdrv_is_allocated(). But some code, particularly stream_run(), gets a lot simpler because it no longer has to mess with sectors. For ease of review, bdrv_is_allocated() was tackled separately. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Xie Changlong [replication= part] Reviewed-by: Jeff Cody --- v3-v4: no change v2: tweak function comments, favor bdrv_getlength() over ->total_sectors --- include/block/block.h | 2 +- block/commit.c | 20 ++++++++------------ block/io.c | 42 ++++++++++++++++++++---------------------- block/mirror.c | 5 ++++- block/replication.c | 17 ++++++++++++----- block/stream.c | 21 +++++++++------------ qemu-img.c | 10 +++++++--- 7 files changed, 61 insertions(+), 56 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index d3e01fb..f0fdbe8 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -430,7 +430,7 @@ int64_t bdrv_get_block_status_above(BlockDriverState *b= s, int bdrv_is_allocated(BlockDriverState *bs, int64_t offset, int64_t bytes, int64_t *pnum); int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, - int64_t sector_num, int nb_sectors, int *pnum); + int64_t offset, int64_t bytes, int64_t *pnum); bool bdrv_is_read_only(BlockDriverState *bs); bool bdrv_is_writable(BlockDriverState *bs); diff --git a/block/commit.c b/block/commit.c index 241aa95..774a8a5 100644 --- a/block/commit.c +++ b/block/commit.c @@ -146,7 +146,7 @@ static void coroutine_fn commit_run(void *opaque) int64_t offset; uint64_t delay_ns =3D 0; int ret =3D 0; - int n =3D 0; /* sectors */ + int64_t n =3D 0; /* bytes */ void *buf =3D NULL; int bytes_written =3D 0; int64_t base_len; @@ -171,7 +171,7 @@ static void coroutine_fn commit_run(void *opaque) buf =3D blk_blockalign(s->top, COMMIT_BUFFER_SIZE); - for (offset =3D 0; offset < s->common.len; offset +=3D n * BDRV_SECTOR= _SIZE) { + for (offset =3D 0; offset < s->common.len; offset +=3D n) { bool copy; /* Note that even when no rate limit is applied we need to yield @@ -183,15 +183,12 @@ static void coroutine_fn commit_run(void *opaque) } /* Copy if allocated above the base */ ret =3D bdrv_is_allocated_above(blk_bs(s->top), blk_bs(s->base), - offset / BDRV_SECTOR_SIZE, - COMMIT_BUFFER_SIZE / BDRV_SECTOR_SIZ= E, - &n); + offset, COMMIT_BUFFER_SIZE, &n); copy =3D (ret =3D=3D 1); - trace_commit_one_iteration(s, offset, n * BDRV_SECTOR_SIZE, ret); + trace_commit_one_iteration(s, offset, n, ret); if (copy) { - ret =3D commit_populate(s->top, s->base, offset, - n * BDRV_SECTOR_SIZE, buf); - bytes_written +=3D n * BDRV_SECTOR_SIZE; + ret =3D commit_populate(s->top, s->base, offset, n, buf); + bytes_written +=3D n; } if (ret < 0) { BlockErrorAction action =3D @@ -204,11 +201,10 @@ static void coroutine_fn commit_run(void *opaque) } } /* Publish progress */ - s->common.offset +=3D n * BDRV_SECTOR_SIZE; + s->common.offset +=3D n; if (copy && s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->limit, - n * BDRV_SECTOR_SIZE); + delay_ns =3D ratelimit_calculate_delay(&s->limit, n); } } diff --git a/block/io.c b/block/io.c index fb8d1c7..569c503 100644 --- a/block/io.c +++ b/block/io.c @@ -1931,54 +1931,52 @@ int coroutine_fn bdrv_is_allocated(BlockDriverState= *bs, int64_t offset, /* * Given an image chain: ... -> [BASE] -> [INTER1] -> [INTER2] -> [TOP] * - * Return true if the given sector is allocated in any image between - * BASE and TOP (inclusive). BASE can be NULL to check if the given - * sector is allocated in any image of the chain. Return false otherwise, + * Return true if the (prefix of the) given range is allocated in any image + * between BASE and TOP (inclusive). BASE can be NULL to check if the giv= en + * offset is allocated in any image of the chain. Return false otherwise, * or negative errno on failure. * - * 'pnum' is set to the number of sectors (including and immediately follo= wing - * the specified sector) that are known to be in the same - * allocated/unallocated state. + * 'pnum' is set to the number of bytes (including and immediately + * following the specified offset) that are known to be in the same + * allocated/unallocated state. Note that a subsequent call starting + * at 'offset + *pnum' may return the same allocation status (in other + * words, the result is not necessarily the maximum possible range); + * but 'pnum' will only be 0 when end of file is reached. * */ int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, - int64_t sector_num, - int nb_sectors, int *pnum) + int64_t offset, int64_t bytes, int64_t *pnum) { BlockDriverState *intermediate; - int ret, n =3D nb_sectors; + int ret; + int64_t n =3D bytes; intermediate =3D top; while (intermediate && intermediate !=3D base) { int64_t pnum_inter; int64_t size_inter; - int psectors_inter; - ret =3D bdrv_is_allocated(intermediate, sector_num * BDRV_SECTOR_S= IZE, - nb_sectors * BDRV_SECTOR_SIZE, - &pnum_inter); + ret =3D bdrv_is_allocated(intermediate, offset, bytes, &pnum_inter= ); if (ret < 0) { return ret; } - assert(pnum_inter < INT_MAX * BDRV_SECTOR_SIZE); - psectors_inter =3D pnum_inter >> BDRV_SECTOR_BITS; if (ret) { - *pnum =3D psectors_inter; + *pnum =3D pnum_inter; return 1; } /* - * [sector_num, nb_sectors] is unallocated on top but intermediate - * might have [sector_num+x, nb_sectors-x] allocated. + * [offset, bytes] is unallocated on top but intermediate + * might have [offset+x, bytes-x] allocated. */ - size_inter =3D bdrv_nb_sectors(intermediate); + size_inter =3D bdrv_getlength(intermediate); if (size_inter < 0) { return size_inter; } - if (n > psectors_inter && - (intermediate =3D=3D top || sector_num + psectors_inter < size= _inter)) { - n =3D psectors_inter; + if (n > pnum_inter && + (intermediate =3D=3D top || offset + pnum_inter < size_inter))= { + n =3D pnum_inter; } intermediate =3D backing_bs(intermediate); diff --git a/block/mirror.c b/block/mirror.c index f54a8d7..c717f60 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -621,6 +621,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) BlockDriverState *bs =3D s->source; BlockDriverState *target_bs =3D blk_bs(s->target); int ret, n; + int64_t count; end =3D s->bdev_length / BDRV_SECTOR_SIZE; @@ -670,11 +671,13 @@ static int coroutine_fn mirror_dirty_init(MirrorBlock= Job *s) return 0; } - ret =3D bdrv_is_allocated_above(bs, base, sector_num, nb_sectors, = &n); + ret =3D bdrv_is_allocated_above(bs, base, sector_num * BDRV_SECTOR= _SIZE, + nb_sectors * BDRV_SECTOR_SIZE, &coun= t); if (ret < 0) { return ret; } + n =3D DIV_ROUND_UP(count, BDRV_SECTOR_SIZE); assert(n > 0); if (ret =3D=3D 1) { bdrv_set_dirty_bitmap(s->dirty_bitmap, sector_num, n); diff --git a/block/replication.c b/block/replication.c index 8f3aba7..bf4462c 100644 --- a/block/replication.c +++ b/block/replication.c @@ -264,7 +264,8 @@ static coroutine_fn int replication_co_writev(BlockDriv= erState *bs, BdrvChild *top =3D bs->file; BdrvChild *base =3D s->secondary_disk; BdrvChild *target; - int ret, n; + int ret; + int64_t n; ret =3D replication_get_io_status(s); if (ret < 0) { @@ -283,14 +284,20 @@ static coroutine_fn int replication_co_writev(BlockDr= iverState *bs, */ qemu_iovec_init(&hd_qiov, qiov->niov); while (remaining_sectors > 0) { - ret =3D bdrv_is_allocated_above(top->bs, base->bs, sector_num, - remaining_sectors, &n); + int64_t count; + + ret =3D bdrv_is_allocated_above(top->bs, base->bs, + sector_num * BDRV_SECTOR_SIZE, + remaining_sectors * BDRV_SECTOR_SIZE, + &count); if (ret < 0) { goto out1; } + assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); + n =3D count >> BDRV_SECTOR_BITS; qemu_iovec_reset(&hd_qiov); - qemu_iovec_concat(&hd_qiov, qiov, bytes_done, n * BDRV_SECTOR_SIZE= ); + qemu_iovec_concat(&hd_qiov, qiov, bytes_done, count); target =3D ret ? top : base; ret =3D bdrv_co_writev(target, sector_num, n, &hd_qiov); @@ -300,7 +307,7 @@ static coroutine_fn int replication_co_writev(BlockDriv= erState *bs, remaining_sectors -=3D n; sector_num +=3D n; - bytes_done +=3D n * BDRV_SECTOR_SIZE; + bytes_done +=3D count; } out1: diff --git a/block/stream.c b/block/stream.c index e5f2a08..e6f7234 100644 --- a/block/stream.c +++ b/block/stream.c @@ -111,7 +111,7 @@ static void coroutine_fn stream_run(void *opaque) uint64_t delay_ns =3D 0; int error =3D 0; int ret =3D 0; - int n =3D 0; /* sectors */ + int64_t n =3D 0; /* bytes */ void *buf; if (!bs->backing) { @@ -135,9 +135,8 @@ static void coroutine_fn stream_run(void *opaque) bdrv_enable_copy_on_read(bs); } - for ( ; offset < s->common.len; offset +=3D n * BDRV_SECTOR_SIZE) { + for ( ; offset < s->common.len; offset +=3D n) { bool copy; - int64_t count =3D 0; /* Note that even when no rate limit is applied we need to yield * with no pending I/O here so that bdrv_drain_all() returns. @@ -149,26 +148,25 @@ static void coroutine_fn stream_run(void *opaque) copy =3D false; - ret =3D bdrv_is_allocated(bs, offset, STREAM_BUFFER_SIZE, &count); - n =3D DIV_ROUND_UP(count, BDRV_SECTOR_SIZE); + ret =3D bdrv_is_allocated(bs, offset, STREAM_BUFFER_SIZE, &n); if (ret =3D=3D 1) { /* Allocated in the top, no need to copy. */ } else if (ret >=3D 0) { /* Copy if allocated in the intermediate images. Limit to the * known-unallocated area [offset, offset+n*BDRV_SECTOR_SIZE).= */ ret =3D bdrv_is_allocated_above(backing_bs(bs), base, - offset / BDRV_SECTOR_SIZE, n, &n= ); + offset, n, &n); /* Finish early if end of backing file has been reached */ if (ret =3D=3D 0 && n =3D=3D 0) { - n =3D (s->common.len - offset) / BDRV_SECTOR_SIZE; + n =3D s->common.len - offset; } copy =3D (ret =3D=3D 1); } - trace_stream_one_iteration(s, offset, n * BDRV_SECTOR_SIZE, ret); + trace_stream_one_iteration(s, offset, n, ret); if (copy) { - ret =3D stream_populate(blk, offset, n * BDRV_SECTOR_SIZE, buf= ); + ret =3D stream_populate(blk, offset, n, buf); } if (ret < 0) { BlockErrorAction action =3D @@ -187,10 +185,9 @@ static void coroutine_fn stream_run(void *opaque) ret =3D 0; /* Publish progress */ - s->common.offset +=3D n * BDRV_SECTOR_SIZE; + s->common.offset +=3D n; if (copy && s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->limit, - n * BDRV_SECTOR_SIZE); + delay_ns =3D ratelimit_calculate_delay(&s->limit, n); } } diff --git a/qemu-img.c b/qemu-img.c index 5271b41..960f42a 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -1477,12 +1477,16 @@ static int img_compare(int argc, char **argv) } for (;;) { + int64_t count; + nb_sectors =3D sectors_to_process(total_sectors_over, sector_n= um); if (nb_sectors <=3D 0) { break; } - ret =3D bdrv_is_allocated_above(blk_bs(blk_over), NULL, sector= _num, - nb_sectors, &pnum); + ret =3D bdrv_is_allocated_above(blk_bs(blk_over), NULL, + sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE, + &count); if (ret < 0) { ret =3D 3; error_report("Sector allocation test failed for %s", @@ -1490,7 +1494,7 @@ static int img_compare(int argc, char **argv) goto out; } - nb_sectors =3D pnum; + nb_sectors =3D DIV_ROUND_UP(count, BDRV_SECTOR_SIZE); if (ret) { ret =3D check_empty_sectors(blk_over, sector_num, nb_secto= rs, filename_over, buf1, quiet); --=20 2.9.4