From nobody Wed Apr 16 07:38:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1499449606788954.2115394193879; Fri, 7 Jul 2017 10:46:46 -0700 (PDT) Received: from localhost ([::1]:57905 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dTXL3-0004jI-GL for importer@patchew.org; Fri, 07 Jul 2017 13:46:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42203) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dTWlc-0005U0-0m for qemu-devel@nongnu.org; Fri, 07 Jul 2017 13:10:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dTWlX-0008Kz-Uu for qemu-devel@nongnu.org; Fri, 07 Jul 2017 13:10:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56866) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dTWlT-0008DC-9g; Fri, 07 Jul 2017 13:09:59 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5EA8D74864; Fri, 7 Jul 2017 17:09:58 +0000 (UTC) Received: from noname.redhat.com (ovpn-117-34.ams2.redhat.com [10.36.117.34]) by smtp.corp.redhat.com (Postfix) with ESMTP id 73F8561F23; Fri, 7 Jul 2017 17:09:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5EA8D74864 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=kwolf@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 5EA8D74864 From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 7 Jul 2017 19:07:55 +0200 Message-Id: <1499447335-6125-41-git-send-email-kwolf@redhat.com> In-Reply-To: <1499447335-6125-1-git-send-email-kwolf@redhat.com> References: <1499447335-6125-1-git-send-email-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 07 Jul 2017 17:09:58 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 040/100] block: Make bdrv_is_allocated_above() byte-based X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. In the common case, allocation is unlikely to ever use values that are not naturally sector-aligned, but it is possible that byte-based values will let us be more precise about allocation at the end of an unaligned file that can do byte-based access. Changing the signature of the function to use int64_t *pnum ensures that the compiler enforces that all callers are updated. For now, the io.c layer still assert()s that all callers are sector-aligned, but that can be relaxed when a later patch implements byte-based block status. Therefore, for the most part this patch is just the addition of scaling at the callers followed by inverse scaling at bdrv_is_allocated(). But some code, particularly stream_run(), gets a lot simpler because it no longer has to mess with sectors. Leave comments where we can further simplify by switching to byte-based iterations, once later patches eliminate the need for sector-aligned operations. For ease of review, bdrv_is_allocated() was tackled separately. Signed-off-by: Eric Blake Signed-off-by: Kevin Wolf --- block/commit.c | 20 ++++++++------------ block/io.c | 38 ++++++++++++++++++-------------------- block/mirror.c | 8 +++++++- block/replication.c | 17 ++++++++++++----- block/stream.c | 23 +++++++++-------------- include/block/block.h | 2 +- qemu-img.c | 13 ++++++++++--- 7 files changed, 65 insertions(+), 56 deletions(-) diff --git a/block/commit.c b/block/commit.c index 241aa95..774a8a5 100644 --- a/block/commit.c +++ b/block/commit.c @@ -146,7 +146,7 @@ static void coroutine_fn commit_run(void *opaque) int64_t offset; uint64_t delay_ns =3D 0; int ret =3D 0; - int n =3D 0; /* sectors */ + int64_t n =3D 0; /* bytes */ void *buf =3D NULL; int bytes_written =3D 0; int64_t base_len; @@ -171,7 +171,7 @@ static void coroutine_fn commit_run(void *opaque) =20 buf =3D blk_blockalign(s->top, COMMIT_BUFFER_SIZE); =20 - for (offset =3D 0; offset < s->common.len; offset +=3D n * BDRV_SECTOR= _SIZE) { + for (offset =3D 0; offset < s->common.len; offset +=3D n) { bool copy; =20 /* Note that even when no rate limit is applied we need to yield @@ -183,15 +183,12 @@ static void coroutine_fn commit_run(void *opaque) } /* Copy if allocated above the base */ ret =3D bdrv_is_allocated_above(blk_bs(s->top), blk_bs(s->base), - offset / BDRV_SECTOR_SIZE, - COMMIT_BUFFER_SIZE / BDRV_SECTOR_SIZ= E, - &n); + offset, COMMIT_BUFFER_SIZE, &n); copy =3D (ret =3D=3D 1); - trace_commit_one_iteration(s, offset, n * BDRV_SECTOR_SIZE, ret); + trace_commit_one_iteration(s, offset, n, ret); if (copy) { - ret =3D commit_populate(s->top, s->base, offset, - n * BDRV_SECTOR_SIZE, buf); - bytes_written +=3D n * BDRV_SECTOR_SIZE; + ret =3D commit_populate(s->top, s->base, offset, n, buf); + bytes_written +=3D n; } if (ret < 0) { BlockErrorAction action =3D @@ -204,11 +201,10 @@ static void coroutine_fn commit_run(void *opaque) } } /* Publish progress */ - s->common.offset +=3D n * BDRV_SECTOR_SIZE; + s->common.offset +=3D n; =20 if (copy && s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->limit, - n * BDRV_SECTOR_SIZE); + delay_ns =3D ratelimit_calculate_delay(&s->limit, n); } } =20 diff --git a/block/io.c b/block/io.c index aad7fd3..23170a5 100644 --- a/block/io.c +++ b/block/io.c @@ -1926,50 +1926,48 @@ int coroutine_fn bdrv_is_allocated(BlockDriverState= *bs, int64_t offset, /* * Given an image chain: ... -> [BASE] -> [INTER1] -> [INTER2] -> [TOP] * - * Return true if the given sector is allocated in any image between - * BASE and TOP (inclusive). BASE can be NULL to check if the given - * sector is allocated in any image of the chain. Return false otherwise, + * Return true if (a prefix of) the given range is allocated in any image + * between BASE and TOP (inclusive). BASE can be NULL to check if the giv= en + * offset is allocated in any image of the chain. Return false otherwise, * or negative errno on failure. * - * 'pnum' is set to the number of sectors (including and immediately follo= wing - * the specified sector) that are known to be in the same - * allocated/unallocated state. + * 'pnum' is set to the number of bytes (including and immediately + * following the specified offset) that are known to be in the same + * allocated/unallocated state. Note that a subsequent call starting + * at 'offset + *pnum' may return the same allocation status (in other + * words, the result is not necessarily the maximum possible range); + * but 'pnum' will only be 0 when end of file is reached. * */ int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, - int64_t sector_num, - int nb_sectors, int *pnum) + int64_t offset, int64_t bytes, int64_t *pnum) { BlockDriverState *intermediate; - int ret, n =3D nb_sectors; + int ret; + int64_t n =3D bytes; =20 intermediate =3D top; while (intermediate && intermediate !=3D base) { int64_t pnum_inter; int64_t size_inter; - int psectors_inter; =20 - ret =3D bdrv_is_allocated(intermediate, sector_num * BDRV_SECTOR_S= IZE, - nb_sectors * BDRV_SECTOR_SIZE, - &pnum_inter); + ret =3D bdrv_is_allocated(intermediate, offset, bytes, &pnum_inter= ); if (ret < 0) { return ret; } - assert(pnum_inter < INT_MAX * BDRV_SECTOR_SIZE); - psectors_inter =3D pnum_inter >> BDRV_SECTOR_BITS; if (ret) { - *pnum =3D psectors_inter; + *pnum =3D pnum_inter; return 1; } =20 - size_inter =3D bdrv_nb_sectors(intermediate); + size_inter =3D bdrv_getlength(intermediate); if (size_inter < 0) { return size_inter; } - if (n > psectors_inter && - (intermediate =3D=3D top || sector_num + psectors_inter < size= _inter)) { - n =3D psectors_inter; + if (n > pnum_inter && + (intermediate =3D=3D top || offset + pnum_inter < size_inter))= { + n =3D pnum_inter; } =20 intermediate =3D backing_bs(intermediate); diff --git a/block/mirror.c b/block/mirror.c index b33f4bb..eaf0fe7 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -621,6 +621,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) BlockDriverState *bs =3D s->source; BlockDriverState *target_bs =3D blk_bs(s->target); int ret, n; + int64_t count; =20 end =3D s->bdev_length / BDRV_SECTOR_SIZE; =20 @@ -670,11 +671,16 @@ static int coroutine_fn mirror_dirty_init(MirrorBlock= Job *s) return 0; } =20 - ret =3D bdrv_is_allocated_above(bs, base, sector_num, nb_sectors, = &n); + ret =3D bdrv_is_allocated_above(bs, base, sector_num * BDRV_SECTOR= _SIZE, + nb_sectors * BDRV_SECTOR_SIZE, &coun= t); if (ret < 0) { return ret; } =20 + /* TODO: Relax this once bdrv_is_allocated_above and dirty + * bitmaps no longer require sector alignment. */ + assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); + n =3D count >> BDRV_SECTOR_BITS; assert(n > 0); if (ret =3D=3D 1) { bdrv_set_dirty_bitmap(s->dirty_bitmap, sector_num, n); diff --git a/block/replication.c b/block/replication.c index 8f3aba7..bf4462c 100644 --- a/block/replication.c +++ b/block/replication.c @@ -264,7 +264,8 @@ static coroutine_fn int replication_co_writev(BlockDriv= erState *bs, BdrvChild *top =3D bs->file; BdrvChild *base =3D s->secondary_disk; BdrvChild *target; - int ret, n; + int ret; + int64_t n; =20 ret =3D replication_get_io_status(s); if (ret < 0) { @@ -283,14 +284,20 @@ static coroutine_fn int replication_co_writev(BlockDr= iverState *bs, */ qemu_iovec_init(&hd_qiov, qiov->niov); while (remaining_sectors > 0) { - ret =3D bdrv_is_allocated_above(top->bs, base->bs, sector_num, - remaining_sectors, &n); + int64_t count; + + ret =3D bdrv_is_allocated_above(top->bs, base->bs, + sector_num * BDRV_SECTOR_SIZE, + remaining_sectors * BDRV_SECTOR_SIZE, + &count); if (ret < 0) { goto out1; } =20 + assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); + n =3D count >> BDRV_SECTOR_BITS; qemu_iovec_reset(&hd_qiov); - qemu_iovec_concat(&hd_qiov, qiov, bytes_done, n * BDRV_SECTOR_SIZE= ); + qemu_iovec_concat(&hd_qiov, qiov, bytes_done, count); =20 target =3D ret ? top : base; ret =3D bdrv_co_writev(target, sector_num, n, &hd_qiov); @@ -300,7 +307,7 @@ static coroutine_fn int replication_co_writev(BlockDriv= erState *bs, =20 remaining_sectors -=3D n; sector_num +=3D n; - bytes_done +=3D n * BDRV_SECTOR_SIZE; + bytes_done +=3D count; } =20 out1: diff --git a/block/stream.c b/block/stream.c index df9679c..e6f7234 100644 --- a/block/stream.c +++ b/block/stream.c @@ -111,7 +111,7 @@ static void coroutine_fn stream_run(void *opaque) uint64_t delay_ns =3D 0; int error =3D 0; int ret =3D 0; - int n =3D 0; /* sectors */ + int64_t n =3D 0; /* bytes */ void *buf; =20 if (!bs->backing) { @@ -135,9 +135,8 @@ static void coroutine_fn stream_run(void *opaque) bdrv_enable_copy_on_read(bs); } =20 - for ( ; offset < s->common.len; offset +=3D n * BDRV_SECTOR_SIZE) { + for ( ; offset < s->common.len; offset +=3D n) { bool copy; - int64_t count =3D 0; =20 /* Note that even when no rate limit is applied we need to yield * with no pending I/O here so that bdrv_drain_all() returns. @@ -149,28 +148,25 @@ static void coroutine_fn stream_run(void *opaque) =20 copy =3D false; =20 - ret =3D bdrv_is_allocated(bs, offset, STREAM_BUFFER_SIZE, &count); - /* TODO relax this once bdrv_is_allocated does not enforce sectors= */ - assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); - n =3D count >> BDRV_SECTOR_BITS; + ret =3D bdrv_is_allocated(bs, offset, STREAM_BUFFER_SIZE, &n); if (ret =3D=3D 1) { /* Allocated in the top, no need to copy. */ } else if (ret >=3D 0) { /* Copy if allocated in the intermediate images. Limit to the * known-unallocated area [offset, offset+n*BDRV_SECTOR_SIZE).= */ ret =3D bdrv_is_allocated_above(backing_bs(bs), base, - offset / BDRV_SECTOR_SIZE, n, &n= ); + offset, n, &n); =20 /* Finish early if end of backing file has been reached */ if (ret =3D=3D 0 && n =3D=3D 0) { - n =3D (s->common.len - offset) / BDRV_SECTOR_SIZE; + n =3D s->common.len - offset; } =20 copy =3D (ret =3D=3D 1); } - trace_stream_one_iteration(s, offset, n * BDRV_SECTOR_SIZE, ret); + trace_stream_one_iteration(s, offset, n, ret); if (copy) { - ret =3D stream_populate(blk, offset, n * BDRV_SECTOR_SIZE, buf= ); + ret =3D stream_populate(blk, offset, n, buf); } if (ret < 0) { BlockErrorAction action =3D @@ -189,10 +185,9 @@ static void coroutine_fn stream_run(void *opaque) ret =3D 0; =20 /* Publish progress */ - s->common.offset +=3D n * BDRV_SECTOR_SIZE; + s->common.offset +=3D n; if (copy && s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->limit, - n * BDRV_SECTOR_SIZE); + delay_ns =3D ratelimit_calculate_delay(&s->limit, n); } } =20 diff --git a/include/block/block.h b/include/block/block.h index 8d16cc1..4a27252 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -430,7 +430,7 @@ int64_t bdrv_get_block_status_above(BlockDriverState *b= s, int bdrv_is_allocated(BlockDriverState *bs, int64_t offset, int64_t bytes, int64_t *pnum); int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, - int64_t sector_num, int nb_sectors, int *pnum); + int64_t offset, int64_t bytes, int64_t *pnum); =20 bool bdrv_is_read_only(BlockDriverState *bs); bool bdrv_is_writable(BlockDriverState *bs); diff --git a/qemu-img.c b/qemu-img.c index 86e2433..f7ffb79 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -1508,12 +1508,16 @@ static int img_compare(int argc, char **argv) } =20 for (;;) { + int64_t count; + nb_sectors =3D sectors_to_process(total_sectors_over, sector_n= um); if (nb_sectors <=3D 0) { break; } - ret =3D bdrv_is_allocated_above(blk_bs(blk_over), NULL, sector= _num, - nb_sectors, &pnum); + ret =3D bdrv_is_allocated_above(blk_bs(blk_over), NULL, + sector_num * BDRV_SECTOR_SIZE, + nb_sectors * BDRV_SECTOR_SIZE, + &count); if (ret < 0) { ret =3D 3; error_report("Sector allocation test failed for %s", @@ -1521,7 +1525,10 @@ static int img_compare(int argc, char **argv) goto out; =20 } - nb_sectors =3D pnum; + /* TODO relax this once bdrv_is_allocated_above does not enfor= ce + * sector alignment */ + assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); + nb_sectors =3D count >> BDRV_SECTOR_BITS; if (ret) { ret =3D check_empty_sectors(blk_over, sector_num, nb_secto= rs, filename_over, buf1, quiet); --=20 1.8.3.1