From nobody Thu May 2 08:33:37 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498831953686351.38947135166427; Fri, 30 Jun 2017 07:12:33 -0700 (PDT) Received: from localhost ([::1]:44727 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQwen-0005PX-3F for importer@patchew.org; Fri, 30 Jun 2017 10:12:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57065) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQwdH-0004W9-MA for qemu-devel@nongnu.org; Fri, 30 Jun 2017 10:10:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQwdG-0001Zx-JW for qemu-devel@nongnu.org; Fri, 30 Jun 2017 10:10:51 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50302) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQwdG-0001ZK-BP for qemu-devel@nongnu.org; Fri, 30 Jun 2017 10:10:50 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6B9B5C0587E3; Fri, 30 Jun 2017 14:10:49 +0000 (UTC) Received: from lemon.redhat.com (ovpn-12-32.pek2.redhat.com [10.72.12.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3BA4F177F3; Fri, 30 Jun 2017 14:10:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6B9B5C0587E3 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=famz@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 6B9B5C0587E3 From: Fam Zheng To: qemu-devel@nongnu.org Date: Fri, 30 Jun 2017 22:10:39 +0800 Message-Id: <20170630141040.15353-2-famz@redhat.com> In-Reply-To: <20170630141040.15353-1-famz@redhat.com> References: <20170630141040.15353-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 30 Jun 2017 14:10:49 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 1/2] block: Add BDRV_BLOCK_EOF to bdrv_get_block_status() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Fam Zheng Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Just as the block layer already sets BDRV_BLOCK_ALLOCATED as a shortcut for subsequent operations, there are also some optimizations that are made easier if we can quickly tell that *pnum will advance us to the end of a file, via a new BDRV_BLOCK_EOF which gets set by the block layer. This just plumbs up the new bit; subsequent patches will make use of it. Signed-off-by: Eric Blake Message-Id: <20170505021500.19315-2-eblake@redhat.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Fam Zheng --- block/io.c | 15 +++++++++++---- include/block/block.h | 2 ++ 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/block/io.c b/block/io.c index 9bba730..7e6abef 100644 --- a/block/io.c +++ b/block/io.c @@ -1711,15 +1711,16 @@ typedef struct BdrvCoGetBlockStatusData { * Drivers not implementing the functionality are assumed to not support * backing files, hence all their sectors are reported as allocated. * - * If 'sector_num' is beyond the end of the disk image the return value is= 0 - * and 'pnum' is set to 0. + * If 'sector_num' is beyond the end of the disk image the return value is + * BDRV_BLOCK_EOF and 'pnum' is set to 0. * * 'pnum' is set to the number of sectors (including and immediately follo= wing * the specified sector) that are known to be in the same * allocated/unallocated state. * * 'nb_sectors' is the max value 'pnum' should be set to. If nb_sectors g= oes - * beyond the end of the disk image it will be clamped. + * beyond the end of the disk image it will be clamped; if 'pnum' is set to + * the end of the image, then the returned value will include BDRV_BLOCK_E= OF. * * If returned value is positive and BDRV_BLOCK_OFFSET_VALID bit is set, '= file' * points to the BDS which the sector range is allocated in. @@ -1740,7 +1741,7 @@ static int64_t coroutine_fn bdrv_co_get_block_status(= BlockDriverState *bs, =20 if (sector_num >=3D total_sectors) { *pnum =3D 0; - return 0; + return BDRV_BLOCK_EOF; } =20 n =3D total_sectors - sector_num; @@ -1751,6 +1752,9 @@ static int64_t coroutine_fn bdrv_co_get_block_status(= BlockDriverState *bs, if (!bs->drv->bdrv_co_get_block_status) { *pnum =3D nb_sectors; ret =3D BDRV_BLOCK_DATA | BDRV_BLOCK_ALLOCATED; + if (sector_num + nb_sectors =3D=3D total_sectors) { + ret |=3D BDRV_BLOCK_EOF; + } if (bs->drv->protocol_name) { ret |=3D BDRV_BLOCK_OFFSET_VALID | (sector_num * BDRV_SECTOR_S= IZE); } @@ -1814,6 +1818,9 @@ static int64_t coroutine_fn bdrv_co_get_block_status(= BlockDriverState *bs, =20 out: bdrv_dec_in_flight(bs); + if (ret >=3D 0 && sector_num + *pnum =3D=3D total_sectors) { + ret |=3D BDRV_BLOCK_EOF; + } return ret; } =20 diff --git a/include/block/block.h b/include/block/block.h index 85e4be7..4c149ad 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -129,6 +129,7 @@ typedef struct HDGeometry { * BDRV_BLOCK_OFFSET_VALID: an associated offset exists for accessing raw = data * BDRV_BLOCK_ALLOCATED: the content of the block is determined by this * layer (short for DATA || ZERO), set by block layer + * BDRV_BLOCK_EOF: the returned pnum covers through end of file for this l= ayer * * Internal flag: * BDRV_BLOCK_RAW: used internally to indicate that the request was @@ -157,6 +158,7 @@ typedef struct HDGeometry { #define BDRV_BLOCK_OFFSET_VALID 0x04 #define BDRV_BLOCK_RAW 0x08 #define BDRV_BLOCK_ALLOCATED 0x10 +#define BDRV_BLOCK_EOF 0x20 #define BDRV_BLOCK_OFFSET_MASK BDRV_SECTOR_MASK =20 typedef QSIMPLEQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry) BlockReopen= Queue; --=20 2.9.4 From nobody Thu May 2 08:33:37 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498831951355329.9453122246183; Fri, 30 Jun 2017 07:12:31 -0700 (PDT) Received: from localhost ([::1]:44728 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQweo-0005QS-U6 for importer@patchew.org; Fri, 30 Jun 2017 10:12:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57079) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQwdK-0004XW-CV for qemu-devel@nongnu.org; Fri, 30 Jun 2017 10:10:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQwdI-0001aq-Jg for qemu-devel@nongnu.org; Fri, 30 Jun 2017 10:10:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59954) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQwdI-0001aU-BU for qemu-devel@nongnu.org; Fri, 30 Jun 2017 10:10:52 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 801CAC07DE80; Fri, 30 Jun 2017 14:10:51 +0000 (UTC) Received: from lemon.redhat.com (ovpn-12-32.pek2.redhat.com [10.72.12.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 03A368275C; Fri, 30 Jun 2017 14:10:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 801CAC07DE80 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=famz@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 801CAC07DE80 From: Fam Zheng To: qemu-devel@nongnu.org Date: Fri, 30 Jun 2017 22:10:40 +0800 Message-Id: <20170630141040.15353-3-famz@redhat.com> In-Reply-To: <20170630141040.15353-1-famz@redhat.com> References: <20170630141040.15353-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 30 Jun 2017 14:10:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 2/2] block: Exploit BDRV_BLOCK_EOF for larger zero blocks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Fam Zheng Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake When we have a BDS with unallocated clusters, but asking the status of its underlying bs->file or backing layer encounters an end-of-file condition, we know that the rest of the unallocated area will read as zeroes. However, pre-patch, this required two separate calls to bdrv_get_block_status(), as the first call stops at the point where the underlying file ends. Thanks to BDRV_BLOCK_EOF, we can now widen the results of the primary status if the secondary status already includes BDRV_BLOCK_ZERO. In turn, this fixes a TODO mentioned in iotest 154, where we can now see that all sectors in a partial cluster at the end of a file read as zero when coupling the shorter backing file's status along with our knowledge that the remaining sectors came from an unallocated cluster. Also, note that the loop in bdrv_co_get_block_status_above() had an inefficent exit: in cases where the active layer sets BDRV_BLOCK_ZERO but does NOT set BDRV_BLOCK_ALLOCATED (namely, where we know we read zeroes merely because our unallocated clusters lie beyond the backing file's shorter length), we still ended up probing the backing layer even though we already had a good answer. Signed-off-by: Eric Blake Message-Id: <20170505021500.19315-3-eblake@redhat.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Fam Zheng --- block/io.c | 27 ++++++++++++++++++++++----- tests/qemu-iotests/154 | 4 ---- tests/qemu-iotests/154.out | 12 ++++++------ 3 files changed, 28 insertions(+), 15 deletions(-) diff --git a/block/io.c b/block/io.c index 7e6abef..2de7c77 100644 --- a/block/io.c +++ b/block/io.c @@ -1803,10 +1803,13 @@ static int64_t coroutine_fn bdrv_co_get_block_statu= s(BlockDriverState *bs, /* Ignore errors. This is just providing extra information, it * is useful but not necessary. */ - if (!file_pnum) { - /* !file_pnum indicates an offset at or beyond the EOF; it= is - * perfectly valid for the format block driver to point to= such - * offsets, so catch it and mark everything as zero */ + if (ret2 & BDRV_BLOCK_EOF && + (!file_pnum || ret2 & BDRV_BLOCK_ZERO)) { + /* + * It is valid for the format block driver to read + * beyond the end of the underlying file's current + * size; such areas read as zero. + */ ret |=3D BDRV_BLOCK_ZERO; } else { /* Limit request to the range reported by the protocol dri= ver */ @@ -1833,16 +1836,30 @@ static int64_t coroutine_fn bdrv_co_get_block_statu= s_above(BlockDriverState *bs, { BlockDriverState *p; int64_t ret =3D 0; + bool first =3D true; =20 assert(bs !=3D base); for (p =3D bs; p !=3D base; p =3D backing_bs(p)) { ret =3D bdrv_co_get_block_status(p, sector_num, nb_sectors, pnum, = file); - if (ret < 0 || ret & BDRV_BLOCK_ALLOCATED) { + if (ret < 0) { + break; + } + if (ret & BDRV_BLOCK_ZERO && ret & BDRV_BLOCK_EOF && !first) { + /* + * Reading beyond the end of the file continues to read + * zeroes, but we can only widen the result to the + * unallocated length we learned from an earlier + * iteration. + */ + *pnum =3D nb_sectors; + } + if (ret & (BDRV_BLOCK_ZERO | BDRV_BLOCK_DATA)) { break; } /* [sector_num, pnum] unallocated on this layer, which could be on= ly * the first part of [sector_num, nb_sectors]. */ nb_sectors =3D MIN(nb_sectors, *pnum); + first =3D false; } return ret; } diff --git a/tests/qemu-iotests/154 b/tests/qemu-iotests/154 index dd8a426..fde03b0 100755 --- a/tests/qemu-iotests/154 +++ b/tests/qemu-iotests/154 @@ -334,8 +334,6 @@ $QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qe= mu_io $QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qemu_img_map =20 # Repeat with backing file holding unallocated cluster. -# TODO: Note that this forces an allocation, because we aren't yet able to -# quickly detect that reads beyond EOF of the backing file are always zero CLUSTER_SIZE=3D2048 TEST_IMG=3D"$TEST_IMG.base" _make_test_img $((size + 1= 024)) =20 # Write at the front: sector-wise, the request is: @@ -371,8 +369,6 @@ $QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qe= mu_io $QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qemu_img_map =20 # Repeat with backing file holding zero'd cluster -# TODO: Note that this forces an allocation, because we aren't yet able to -# quickly detect that reads beyond EOF of the backing file are always zero $QEMU_IO -c "write -z $size 512" "$TEST_IMG.base" | _filter_qemu_io =20 # Write at the front: sector-wise, the request is: diff --git a/tests/qemu-iotests/154.out b/tests/qemu-iotests/154.out index d8485ee..fa36733 100644 --- a/tests/qemu-iotests/154.out +++ b/tests/qemu-iotests/154.out @@ -310,19 +310,19 @@ wrote 512/512 bytes at offset 134217728 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB [{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, -{ "start": 134217728, "length": 2048, "depth": 0, "zero": false, "data": t= rue, "offset": OFFSET}] +{ "start": 134217728, "length": 2048, "depth": 0, "zero": true, "data": fa= lse}] Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base wrote 512/512 bytes at offset 134219264 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB [{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, -{ "start": 134217728, "length": 2048, "depth": 0, "zero": false, "data": t= rue, "offset": OFFSET}] +{ "start": 134217728, "length": 2048, "depth": 0, "zero": true, "data": fa= lse}] Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base wrote 1024/1024 bytes at offset 134218240 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB [{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, -{ "start": 134217728, "length": 2048, "depth": 0, "zero": false, "data": t= rue, "offset": OFFSET}] +{ "start": 134217728, "length": 2048, "depth": 0, "zero": true, "data": fa= lse}] Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base wrote 2048/2048 bytes at offset 134217728 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -336,19 +336,19 @@ wrote 512/512 bytes at offset 134217728 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB [{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, -{ "start": 134217728, "length": 2048, "depth": 0, "zero": false, "data": t= rue, "offset": OFFSET}] +{ "start": 134217728, "length": 2048, "depth": 0, "zero": true, "data": fa= lse}] Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base wrote 512/512 bytes at offset 134219264 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB [{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, -{ "start": 134217728, "length": 2048, "depth": 0, "zero": false, "data": t= rue, "offset": OFFSET}] +{ "start": 134217728, "length": 2048, "depth": 0, "zero": true, "data": fa= lse}] Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base wrote 1024/1024 bytes at offset 134218240 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 2048/2048 bytes allocated at offset 128 MiB [{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, -{ "start": 134217728, "length": 2048, "depth": 0, "zero": false, "data": t= rue, "offset": OFFSET}] +{ "start": 134217728, "length": 2048, "depth": 0, "zero": true, "data": fa= lse}] Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base wrote 2048/2048 bytes at offset 134217728 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) --=20 2.9.4