From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873578125497.55291312751433; Mon, 10 Apr 2017 18:19:38 -0700 (PDT) Received: from localhost ([::1]:36688 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkT3-0006Zz-5J for importer@patchew.org; Mon, 10 Apr 2017 21:19:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46412) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkR6-0005H9-2t for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkR4-0005q2-FS for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36998) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkQx-0005mQ-M9; Mon, 10 Apr 2017 21:17:27 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A7C8C80F90; Tue, 11 Apr 2017 01:17:26 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id F104881868; Tue, 11 Apr 2017 01:17:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A7C8C80F90 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com A7C8C80F90 From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:06 -0500 Message-Id: <20170411011718.9152-2-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 11 Apr 2017 01:17:26 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 01/13] qcow2: Unallocate unmapped zero clusters if no backing file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" 'qemu-img map' already coalesces information about unallocated clusters (those with status 0) and pure zero clusters (those with status BDRV_BLOCK_ZERO and no offset). Furthermore, all qcow2 images with no backing file already report all unallocated clusters (in the preallocation sense of clusters with no offset) as BDRV_BLOCK_ZERO, regardless of whether the QCOW_OFLAG_ZERO was set in that L2 entry (QCOW_OFLAG_ZERO also implies a return of BDRV_BLOCK_ALLOCATED, but we intentionally do not expose that bit to external users), thanks to generic block layer code in bdrv_co_get_block_status(). So, for an image with no backing file, having bdrv_pwrite_zeroes mark clusters as unallocated (defer to backing file) rather than reads-as-zero (regardless of backing file) makes no difference to normal behavior, but may potentially allow for fewer writes to the L2 table of a freshly-created image where the L2 table is initially written to all-zeroes (although I don't actually know if we skip an L2 update and flush when re-writing the same contents as already present). Furthermore, this matches the behavior of discard_single_l2(), in favoring an unallocated cluster over a zero cluster when full discard is requested. Meanwhile, version 2 qcow2 files (compat=3D0.10) lack support for an explicit zero cluster. This minor tweak therefore allows us to turn write zeroes with unmap into an actual unallocation on those files, where they used to return -ENOTSUP and cause an allocation due to the fallback to explicitly written zeroes. Note that technically, we _could_ write a cluster as unallocated rather than zero if a backing file exists but the backing file also reads as zero, but that's more expensive to determine, so this optimization is limited to qcow2 without a backing file. Also note that this patch unmaps a compressed cluster when there is no backing file, even when BDRV_REQ_MAY_UNMAP was not set, but it is unlikely to have compressed clusters in a fully preallocated image (the point of compression is to reduce space requirements), so the side effect of unmapping a cluster in that case is not deemed to be a problem. Finally, note that this change can create a subtle difference if a backing file is later added with 'qemu-img rebase -u'; pre-patch, a v3 file (compat=3D1.1) will have a cluster that still reads as 0 (the cluster is not allocated in the sense of preallocation, but is provided from the layer), while post-patch the cluster will now defer to the backing file - but it's already an unsupported action if you are modifying guest-visible data by messing with backing chains ;) Signed-off-by: Eric Blake --- v9: new patch --- block/qcow2-cluster.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 100398c..12f44b2 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1579,7 +1579,8 @@ static int zero_single_l2(BlockDriverState *bs, uint6= 4_t offset, /* Update L2 entries */ qcow2_cache_entry_mark_dirty(bs, s->l2_table_cache, l2_table); if (old_offset & QCOW_OFLAG_COMPRESSED || flags & BDRV_REQ_MAY_UNM= AP) { - l2_table[l2_index + i] =3D cpu_to_be64(QCOW_OFLAG_ZERO); + l2_table[l2_index + i] =3D bs->backing + ? cpu_to_be64(QCOW_OFLAG_ZERO) : 0; qcow2_free_any_clusters(bs, old_offset, 1, QCOW2_DISCARD_REQUE= ST); } else { l2_table[l2_index + i] |=3D cpu_to_be64(QCOW_OFLAG_ZERO); @@ -1598,8 +1599,11 @@ int qcow2_zero_clusters(BlockDriverState *bs, uint64= _t offset, int nb_sectors, uint64_t nb_clusters; int ret; - /* The zero flag is only supported by version 3 and newer */ - if (s->qcow_version < 3) { + /* The zero flag is only supported by version 3 and newer; we + * require the use of that flag if there is a backing file or if + * we are not allowed to unmap. */ + if (s->qcow_version < 3 && + (bs->backing || !(flags & BDRV_REQ_MAY_UNMAP))) { return -ENOTSUP; } --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873608665118.05924338918499; Mon, 10 Apr 2017 18:20:08 -0700 (PDT) Received: from localhost ([::1]:36690 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkTX-0006y3-Dr for importer@patchew.org; Mon, 10 Apr 2017 21:20:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46416) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkR6-0005HI-AI for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkR4-0005px-E9 for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36620) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkQy-0005n9-P5; Mon, 10 Apr 2017 21:17:28 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C482C80461; Tue, 11 Apr 2017 01:17:27 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id E355C78DF2; Tue, 11 Apr 2017 01:17:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C482C80461 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C482C80461 From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:07 -0500 Message-Id: <20170411011718.9152-3-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 11 Apr 2017 01:17:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 02/13] iotests: Add test 179 to cover write zeroes with unmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" No tests were covering write zeroes with unmap. Additionally, I wanted to prove that my patch to optimize write zeroes for compat=3D0.10 images actually had an impact; for that, run: ./check -qcow2 -o compat=3D0.10 179 Writing the test to work correctly for both old and new qcow2 images is a bit tricky: 'qemu-img map' works for showing whether a cluster is assigned an offset (in the preallocation sense), but older images have to write literal zeroes where newer images can set the zero flag. Thankfully, 'qemu-io alloc' hides the difference, by instead reporting whether a cluster's content comes from the current layer (regardless of whether it was due to a cluster allocation or a flag). Signed-off-by: Eric Blake --- tests/qemu-iotests/179 | 78 ++++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/179.out | 22 +++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 101 insertions(+) create mode 100755 tests/qemu-iotests/179 create mode 100644 tests/qemu-iotests/179.out diff --git a/tests/qemu-iotests/179 b/tests/qemu-iotests/179 new file mode 100755 index 0000000..338a45d --- /dev/null +++ b/tests/qemu-iotests/179 @@ -0,0 +1,78 @@ +#!/bin/bash +# +# Test case for write zeroes with unmap +# +# Copyright (C) 2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=3Deblake@redhat.com + +seq=3D"$(basename $0)" +echo "QA output created by $seq" + +here=3D"$PWD" +status=3D1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +echo +echo '=3D=3D=3D Testing write zeroes with unmap =3D=3D=3D' +echo + +TEST_IMG=3D"$TEST_IMG.base" _make_test_img 16M +_make_test_img -b "$TEST_IMG.base" + +# Aligned writes should not allocate clusters, if unmap is requested +# and there is no backing file. +$QEMU_IO -c "write -z -u 1M 1M" "$TEST_IMG.base" | _filter_qemu_io + +# Unmap can even clear previously-allocated clusters. +$QEMU_IO -c "write 3M 1M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z -u 3M 1M" "$TEST_IMG.base" | _filter_qemu_io + +# Up to now, the entire image should still be unallocated. +$QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qemu_img_map + +# But not requesting unmap must result in allocation (whether a cluster +# allocation in compat=3D0.10 or a flag allocation in compat=3D1.1). +$QEMU_IO -c "write -z 5M 1M" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "alloc 5M $((1024*1024 / 512))" "$TEST_IMG.base" | _filter_qem= u_io + +# Presence of a backing file overrides permission to unmap. Again, +# compat=3D0.10 images allocate, while compat=3D1.1 images set zero flag. +$QEMU_IO -c "write -z -u 7M 1M" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "alloc 7M $((1024 * 1024 / 512))" "$TEST_IMG" | _filter_qemu_io + +# Final check that images are still sane. +TEST_IMG=3D"$TEST_IMG.base" _check_test_img +_check_test_img + +# success, all done +echo '*** done' +status=3D0 diff --git a/tests/qemu-iotests/179.out b/tests/qemu-iotests/179.out new file mode 100644 index 0000000..fc97b19 --- /dev/null +++ b/tests/qemu-iotests/179.out @@ -0,0 +1,22 @@ +QA output created by 179 + +=3D=3D=3D Testing write zeroes with unmap =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D16777216 +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D16777216 backing_file= =3DTEST_DIR/t.IMGFMT.base +wrote 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1048576/1048576 bytes at offset 3145728 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1048576/1048576 bytes at offset 3145728 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 16777216, "depth": 1, "zero": true, "data": false= }] +wrote 1048576/1048576 bytes at offset 5242880 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +2048/2048 sectors allocated at offset 5 MiB +wrote 1048576/1048576 bytes at offset 7340032 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +2048/2048 sectors allocated at offset 7 MiB +No errors were found on the image. +No errors were found on the image. +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 43142dd..cfc8823 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -169,3 +169,4 @@ 174 auto 175 auto quick 176 rw auto backing +179 rw auto quick --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873944470827.0454163576421; Mon, 10 Apr 2017 18:25:44 -0700 (PDT) Received: from localhost ([::1]:36727 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkYx-0003Q1-Cz for importer@patchew.org; Mon, 10 Apr 2017 21:25:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46414) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkR6-0005HD-65 for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkR4-0005pq-EP for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33100) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkQz-0005nf-NW; Mon, 10 Apr 2017 21:17:29 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C6374C059727; Tue, 11 Apr 2017 01:17:28 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0CBF178DF2; Tue, 11 Apr 2017 01:17:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C6374C059727 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C6374C059727 From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:08 -0500 Message-Id: <20170411011718.9152-4-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 11 Apr 2017 01:17:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 03/13] qemu-io: Switch 'alloc' command to byte-based length X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" For the 'alloc' command, accepting an offset in bytes but a length in sectors, and reporting output in sectors, is confusing. Do everything in bytes, and adjust the expected output accordingly. Signed-off-by: Eric Blake Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- qemu-io-cmds.c | 30 ++++++++++++++++++------------ tests/qemu-iotests/019.out | 8 ++++---- tests/qemu-iotests/179 | 4 ++-- tests/qemu-iotests/179.out | 4 ++-- tests/qemu-iotests/common.pattern | 2 +- 5 files changed, 27 insertions(+), 21 deletions(-) diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index 9e023a4..df7297f 100644 --- a/qemu-io-cmds.c +++ b/qemu-io-cmds.c @@ -1760,7 +1760,7 @@ out: static int alloc_f(BlockBackend *blk, int argc, char **argv) { BlockDriverState *bs =3D blk_bs(blk); - int64_t offset, sector_num, nb_sectors, remaining; + int64_t offset, sector_num, nb_sectors, remaining, bytes; char s1[64]; int num, ret; int64_t sum_alloc; @@ -1776,18 +1776,24 @@ static int alloc_f(BlockBackend *blk, int argc, cha= r **argv) } if (argc =3D=3D 3) { - nb_sectors =3D cvtnum(argv[2]); - if (nb_sectors < 0) { - print_cvtnum_err(nb_sectors, argv[2]); + bytes =3D cvtnum(argv[2]); + if (bytes < 0) { + print_cvtnum_err(bytes, argv[2]); return 0; - } else if (nb_sectors > INT_MAX) { - printf("length argument cannot exceed %d, given %s\n", - INT_MAX, argv[2]); + } else if (bytes > INT_MAX * BDRV_SECTOR_SIZE) { + printf("length argument cannot exceed %llu, given %s\n", + INT_MAX * BDRV_SECTOR_SIZE, argv[2]); return 0; } } else { - nb_sectors =3D 1; + bytes =3D BDRV_SECTOR_SIZE; } + if (bytes & 0x1ff) { + printf("bytes %" PRId64 " is not sector aligned\n", + bytes); + return 0; + } + nb_sectors =3D bytes >> BDRV_SECTOR_BITS; remaining =3D nb_sectors; sum_alloc =3D 0; @@ -1811,8 +1817,8 @@ static int alloc_f(BlockBackend *blk, int argc, char = **argv) cvtstr(offset, s1, sizeof(s1)); - printf("%"PRId64"/%"PRId64" sectors allocated at offset %s\n", - sum_alloc, nb_sectors, s1); + printf("%"PRId64"/%"PRId64" bytes allocated at offset %s\n", + sum_alloc << BDRV_SECTOR_BITS, nb_sectors << BDRV_SECTOR_BITS, = s1); return 0; } @@ -1822,8 +1828,8 @@ static const cmdinfo_t alloc_cmd =3D { .argmin =3D 1, .argmax =3D 2, .cfunc =3D alloc_f, - .args =3D "off [sectors]", - .oneline =3D "checks if a sector is present in the file", + .args =3D "offset [bytes]", + .oneline =3D "checks if offset is allocated in the file", }; diff --git a/tests/qemu-iotests/019.out b/tests/qemu-iotests/019.out index 0124264..17a7c03 100644 --- a/tests/qemu-iotests/019.out +++ b/tests/qemu-iotests/019.out @@ -542,8 +542,8 @@ Testing conversion with -B TEST_DIR/t.IMGFMT.base Checking if backing clusters are allocated when they shouldn't -0/128 sectors allocated at offset 1 MiB -0/128 sectors allocated at offset 4.001 GiB +0/65536 bytes allocated at offset 1 MiB +0/65536 bytes allocated at offset 4.001 GiB Reading =3D=3D=3D IO: pattern 42 @@ -1086,8 +1086,8 @@ Testing conversion with -o backing_file=3DTEST_DIR/t.= IMGFMT.base Checking if backing clusters are allocated when they shouldn't -0/128 sectors allocated at offset 1 MiB -0/128 sectors allocated at offset 4.001 GiB +0/65536 bytes allocated at offset 1 MiB +0/65536 bytes allocated at offset 4.001 GiB Reading =3D=3D=3D IO: pattern 42 diff --git a/tests/qemu-iotests/179 b/tests/qemu-iotests/179 index 338a45d..44541e1 100755 --- a/tests/qemu-iotests/179 +++ b/tests/qemu-iotests/179 @@ -62,12 +62,12 @@ $QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qem= u_img_map # But not requesting unmap must result in allocation (whether a cluster # allocation in compat=3D0.10 or a flag allocation in compat=3D1.1). $QEMU_IO -c "write -z 5M 1M" "$TEST_IMG.base" | _filter_qemu_io -$QEMU_IO -c "alloc 5M $((1024*1024 / 512))" "$TEST_IMG.base" | _filter_qem= u_io +$QEMU_IO -c "alloc 5M 1M" "$TEST_IMG.base" | _filter_qemu_io # Presence of a backing file overrides permission to unmap. Again, # compat=3D0.10 images allocate, while compat=3D1.1 images set zero flag. $QEMU_IO -c "write -z -u 7M 1M" "$TEST_IMG" | _filter_qemu_io -$QEMU_IO -c "alloc 7M $((1024 * 1024 / 512))" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "alloc 7M 1M" "$TEST_IMG" | _filter_qemu_io # Final check that images are still sane. TEST_IMG=3D"$TEST_IMG.base" _check_test_img diff --git a/tests/qemu-iotests/179.out b/tests/qemu-iotests/179.out index fc97b19..18ecf0f 100644 --- a/tests/qemu-iotests/179.out +++ b/tests/qemu-iotests/179.out @@ -13,10 +13,10 @@ wrote 1048576/1048576 bytes at offset 3145728 [{ "start": 0, "length": 16777216, "depth": 1, "zero": true, "data": false= }] wrote 1048576/1048576 bytes at offset 5242880 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -2048/2048 sectors allocated at offset 5 MiB +1048576/1048576 bytes allocated at offset 5 MiB wrote 1048576/1048576 bytes at offset 7340032 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -2048/2048 sectors allocated at offset 7 MiB +1048576/1048576 bytes allocated at offset 7 MiB No errors were found on the image. No errors were found on the image. *** done diff --git a/tests/qemu-iotests/common.pattern b/tests/qemu-iotests/common.= pattern index ddfbca1..34f4a8d 100644 --- a/tests/qemu-iotests/common.pattern +++ b/tests/qemu-iotests/common.pattern @@ -18,7 +18,7 @@ function do_is_allocated() { local start=3D$1 - local size=3D$(( $2 / 512)) + local size=3D$2 local step=3D$3 local count=3D$4 --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873589486422.64848705151496; Mon, 10 Apr 2017 18:19:49 -0700 (PDT) Received: from localhost ([::1]:36689 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkTE-0006k1-EV for importer@patchew.org; Mon, 10 Apr 2017 21:19:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46400) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkR5-0005Gu-Ne for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkR4-0005pn-E3 for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:35 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46678) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR0-0005nr-O4; Mon, 10 Apr 2017 21:17:30 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C64744621C; Tue, 11 Apr 2017 01:17:29 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0D9CC78DF2; Tue, 11 Apr 2017 01:17:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C64744621C Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C64744621C From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:09 -0500 Message-Id: <20170411011718.9152-5-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 11 Apr 2017 01:17:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 04/13] qemu-io: Switch 'map' output to byte-based reporting X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Mixing byte offset and sector allocation counts is a bit confusing. Also, reporting n/m sectors, where m decreases according to the remaining size of the file, isn't really adding any useful information. Update the output to use byte counts, and adjust the affected tests (./check -qcow2 102, ./check -vpc 146). Signed-off-by: Eric Blake --- qemu-io-cmds.c | 5 ++--- tests/qemu-iotests/102.out | 4 ++-- tests/qemu-iotests/146.out | 30 +++++++++++++++--------------- 3 files changed, 19 insertions(+), 20 deletions(-) diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index df7297f..ed79c30 100644 --- a/qemu-io-cmds.c +++ b/qemu-io-cmds.c @@ -1895,9 +1895,8 @@ static int map_f(BlockBackend *blk, int argc, char **= argv) retstr =3D ret ? " allocated" : "not allocated"; cvtstr(offset << 9ULL, s1, sizeof(s1)); - printf("[% 24" PRId64 "] % 8" PRId64 "/% 8" PRId64 " sectors %s " - "at offset %s (%d)\n", - offset << 9ULL, num, nb_sectors, retstr, s1, ret); + printf("[% 24" PRId64 "] % 16" PRId64 " bytes %s at offset %s (%d)= \n", + offset << 9ULL, num << 9ULL, retstr, s1, ret); offset +=3D num; nb_sectors -=3D num; diff --git a/tests/qemu-iotests/102.out b/tests/qemu-iotests/102.out index eecde16..0e0ae4c 100644 --- a/tests/qemu-iotests/102.out +++ b/tests/qemu-iotests/102.out @@ -6,7 +6,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D65536 wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Image resized. -[ 0] 128/ 128 sectors allocated at offs= et 0 bytes (1) +[ 0] 65536 bytes allocated at offset = 0 bytes (1) Offset Length Mapped to File =3D=3D=3D Testing map on an image file truncated outside of qemu =3D=3D=3D @@ -17,5 +17,5 @@ wrote 65536/65536 bytes at offset 0 Image resized. QEMU X.Y.Z monitor - type 'help' for more information (qemu) qemu-io drv0 map -[ 0] 128/ 128 sectors allocated at offs= et 0 bytes (1) +[ 0] 65536 bytes allocated at offset = 0 bytes (1) *** done diff --git a/tests/qemu-iotests/146.out b/tests/qemu-iotests/146.out index 4f334d8..d4e2f62 100644 --- a/tests/qemu-iotests/146.out +++ b/tests/qemu-iotests/146.out @@ -2,39 +2,39 @@ QA output created by 146 =3D=3D=3D Testing VPC Autodetect =3D=3D=3D -[ 0] 266334240/ 266334240 sectors not allocated at = offset 0 bytes (0) +[ 0] 136363130880 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Testing VPC with current_size force =3D=3D=3D -[ 0] 266338304/ 266338304 sectors not allocated at = offset 0 bytes (0) +[ 0] 136365211648 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Testing VPC with chs force =3D=3D=3D -[ 0] 266334240/ 266334240 sectors not allocated at = offset 0 bytes (0) +[ 0] 136363130880 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Testing Hyper-V Autodetect =3D=3D=3D -[ 0] 266338304/ 266338304 sectors not allocated at = offset 0 bytes (0) +[ 0] 136365211648 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Testing Hyper-V with current_size force =3D=3D=3D -[ 0] 266338304/ 266338304 sectors not allocated at = offset 0 bytes (0) +[ 0] 136365211648 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Testing Hyper-V with chs force =3D=3D=3D -[ 0] 266334240/ 266334240 sectors not allocated at = offset 0 bytes (0) +[ 0] 136363130880 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Testing d2v Autodetect =3D=3D=3D -[ 0] 514560/ 514560 sectors allocated at offs= et 0 bytes (1) +[ 0] 263454720 bytes allocated at offset = 0 bytes (1) =3D=3D=3D Testing d2v with current_size force =3D=3D=3D -[ 0] 514560/ 514560 sectors allocated at offs= et 0 bytes (1) +[ 0] 263454720 bytes allocated at offset = 0 bytes (1) =3D=3D=3D Testing d2v with chs force =3D=3D=3D -[ 0] 514560/ 514560 sectors allocated at offs= et 0 bytes (1) +[ 0] 263454720 bytes allocated at offset = 0 bytes (1) =3D=3D=3D Testing Image create, default =3D=3D=3D @@ -42,15 +42,15 @@ Formatting 'TEST_DIR/IMGFMT-create-test.IMGFMT', fmt=3D= IMGFMT size=3D4294967296 =3D=3D=3D Read created image, default opts =3D=3D=3D=3D -[ 0] 8389584/ 8389584 sectors not allocated at offs= et 0 bytes (0) +[ 0] 4295467008 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Read created image, force_size_calc=3Dchs =3D=3D=3D=3D -[ 0] 8389584/ 8389584 sectors not allocated at offs= et 0 bytes (0) +[ 0] 4295467008 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Read created image, force_size_calc=3Dcurrent_size =3D=3D=3D=3D -[ 0] 8389584/ 8389584 sectors not allocated at offs= et 0 bytes (0) +[ 0] 4295467008 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Testing Image create, force_size =3D=3D=3D @@ -58,13 +58,13 @@ Formatting 'TEST_DIR/IMGFMT-create-test.IMGFMT', fmt=3D= IMGFMT size=3D4294967296 forc =3D=3D=3D Read created image, default opts =3D=3D=3D=3D -[ 0] 8388608/ 8388608 sectors not allocated at offs= et 0 bytes (0) +[ 0] 4294967296 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Read created image, force_size_calc=3Dchs =3D=3D=3D=3D -[ 0] 8388608/ 8388608 sectors not allocated at offs= et 0 bytes (0) +[ 0] 4294967296 bytes not allocated at offset = 0 bytes (0) =3D=3D=3D Read created image, force_size_calc=3Dcurrent_size =3D=3D=3D=3D -[ 0] 8388608/ 8388608 sectors not allocated at offs= et 0 bytes (0) +[ 0] 4294967296 bytes not allocated at offset = 0 bytes (0) *** done --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873728836917.3269303736763; Mon, 10 Apr 2017 18:22:08 -0700 (PDT) Received: from localhost ([::1]:36708 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkVT-0000Eo-OA for importer@patchew.org; Mon, 10 Apr 2017 21:22:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46485) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkR9-0005J8-8P for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkR6-0005rZ-P4 for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59510) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR1-0005oH-MG; Mon, 10 Apr 2017 21:17:31 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B97C78124F; Tue, 11 Apr 2017 01:17:30 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0A07178DF2; Tue, 11 Apr 2017 01:17:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B97C78124F Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B97C78124F From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:10 -0500 Message-Id: <20170411011718.9152-6-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 11 Apr 2017 01:17:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 05/13] qcow2: Optimize write zero of unaligned tail cluster X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly read as zero, and therefore should not penalize our logic for widening a partial cluster into writing the whole cluster as zero. Update test 154 to cover the new scenarios, using two images of intentionally differing length. While at it, fix the test to gracefully skip when run as ./check -qcow2 -o compat=3D0.10 154 since the older format lacks zero clusters already required earlier in the test. Signed-off-by: Eric Blake --- block/qcow2.c | 7 +++ tests/qemu-iotests/154 | 111 +++++++++++++++++++++++++++++++++++++++++= +++- tests/qemu-iotests/154.out | 82 +++++++++++++++++++++++++++++++++ 3 files changed, 198 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 5c1573c..8038793 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2449,6 +2449,10 @@ static bool is_zero_sectors(BlockDriverState *bs, in= t64_t start, BlockDriverState *file; int64_t res; + if (start + count > bs->total_sectors) { + count =3D bs->total_sectors - start; + } + if (!count) { return true; } @@ -2467,6 +2471,9 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockD= riverState *bs, uint32_t tail =3D (offset + count) % s->cluster_size; trace_qcow2_pwrite_zeroes_start_req(qemu_coroutine_self(), offset, cou= nt); + if (offset + count =3D=3D bs->total_sectors * BDRV_SECTOR_SIZE) { + tail =3D 0; + } if (head || tail) { int64_t cl_start =3D (offset - head) >> BDRV_SECTOR_BITS; diff --git a/tests/qemu-iotests/154 b/tests/qemu-iotests/154 index 7ca7219..66128f5 100755 --- a/tests/qemu-iotests/154 +++ b/tests/qemu-iotests/154 @@ -2,7 +2,7 @@ # # qcow2 specific bdrv_pwrite_zeroes tests with backing files (complements = 034) # -# Copyright (C) 2016 Red Hat, Inc. +# Copyright (C) 2016-2017 Red Hat, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -42,7 +42,10 @@ _supported_proto file _supported_os Linux CLUSTER_SIZE=3D4k -size=3D128M +size=3D$((128 * 1024 * 1024)) + +# This test requires zero clusters, added in v3 images +_unsupported_imgopts compat=3D0.10 echo echo =3D=3D backing file contains zeros =3D=3D @@ -299,6 +302,110 @@ $QEMU_IO -c "read -P 0 75k 1k" "$TEST_IMG" | _filter_= qemu_io $QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qemu_img_map +echo +echo =3D=3D unaligned image tail cluster, no allocation needed =3D=3D + +CLUSTER_SIZE=3D1024 TEST_IMG=3D"$TEST_IMG.base" _make_test_img $((size + 1= 024)) +_make_test_img -b "$TEST_IMG.base" $((size + 2048)) + +# With no backing file, write to all or part of unallocated partial cluster + +# Backing file: 128m: ... | 00 -- +$QEMU_IO -c "write -z $size 512" "$TEST_IMG.base" | _filter_qemu_io + +# Backing file: 128m: ... | -- 00 +$QEMU_IO -c "write -z $size 512" "$TEST_IMG.base" | _filter_qemu_io + +# Backing file: 128m: ... | 00 00 +$QEMU_IO -c "write -z $size 1024" "$TEST_IMG.base" | _filter_qemu_io + +# No offset should be allocated, although the cluster itself is considered +# allocated due to the zero flag +$QEMU_IO -c "alloc $size 1024" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IMG map --output=3Djson "$TEST_IMG.base" | _filter_qemu_img_map + +# With backing file that reads as zero, write to all or part of entire clu= ster + +# Backing file: 128m: ... | 00 00 +# Active layer: 128m: ... | 00 00 00 00 +$QEMU_IO -c "write -z $size 2048" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qemu_img_map + +# Backing file: 128m: ... | 00 00 +# Active layer: 128m: ... | 00 00 -- -- +$QEMU_IO -c "write -z $((size)) 1024" "$TEST_IMG" | _filter_qemu_io + +# Backing file: 128m: ... | 00 00 +# Active layer: 128m: ... | -- -- 00 00 +$QEMU_IO -c "write -z $((size + 1024)) 1024" "$TEST_IMG" | _filter_qemu_io + +# Backing file: 128m: ... | 00 00 +# Active layer: 128m: ... | -- 00 00 -- +$QEMU_IO -c "write -z $((size + 512)) 1024" "$TEST_IMG" | _filter_qemu_io + +# No offset should be allocated, although the cluster itself is considered +# allocated due to the zero flag +$QEMU_IO -c "alloc $size 2048" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qemu_img_map + +# Preallocated cluster is not freed when unmap is not requested + +# Backing file: 128m: ... | XX -- =3D> ... | XX 00 +$QEMU_IO -c "write $((size)) 512" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z $((size + 512)) 512" "$TEST_IMG.base" | _filter_qemu= _io + +# Backing file: 128m: ... | XX 00 =3D> ... | 00 00 +$QEMU_IO -c "write -z $((size)) 1024" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IMG map --output=3Djson "$TEST_IMG.base" | _filter_qemu_img_map + +echo +echo =3D=3D unaligned image tail cluster, allocation required =3D=3D + +CLUSTER_SIZE=3D512 TEST_IMG=3D"$TEST_IMG.base" _make_test_img $((size + 10= 24)) +_make_test_img -b "$TEST_IMG.base" $((size + 2048)) + +# Write beyond backing file must COW +# Backing file: 128m: ... | XX -- +# Active layer: 128m: ... | -- -- 00 -- + +$QEMU_IO -c "write -P 1 $((size)) 512" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -z $((size + 1024)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 1 $((size)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 $((size + 512)) 1536" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qemu_img_map + +CLUSTER_SIZE=3D512 TEST_IMG=3D"$TEST_IMG.base" _make_test_img $((size + 10= 24)) +_make_test_img -b "$TEST_IMG.base" $((size + 2048)) + +# Writes at boundaries of (partial) cluster must not lose mid-cluster data +# Backing file: 128m: ... | -- XX +# Active layer: 128m: ... | 00 -- -- 00 + +$QEMU_IO -c "write -P 1 $((size + 512)) 512" "$TEST_IMG.base" | _filter_qe= mu_io +$QEMU_IO -c "write -z $((size)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 $((size)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 1 $((size + 512)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 $((size + 1024)) 1024" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z $((size + 1536)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 $((size)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 1 $((size + 512)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 $((size + 1024)) 1024" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qemu_img_map + +CLUSTER_SIZE=3D512 TEST_IMG=3D"$TEST_IMG.base" _make_test_img $((size + 10= 24)) +_make_test_img -b "$TEST_IMG.base" $((size + 2048)) + +# Partial write must not lose data +# Backing file: 128m: ... | -- -- +# Active layer: 128m: ... | -- -- XX 00 + +$QEMU_IO -c "write -P 1 $((size + 1024)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "write -z $((size + 1536)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 $((size)) 1024" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 1 $((size + 1024)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c "read -P 0 $((size + 1536)) 512" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=3Djson "$TEST_IMG" | _filter_qemu_img_map + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/154.out b/tests/qemu-iotests/154.out index da9eabd..d6e76f4 100644 --- a/tests/qemu-iotests/154.out +++ b/tests/qemu-iotests/154.out @@ -282,4 +282,86 @@ read 1024/1024 bytes at offset 76800 { "start": 69632, "length": 4096, "depth": 0, "zero": true, "data": false}, { "start": 73728, "length": 4096, "depth": 0, "zero": false, "data": true,= "offset": 24576}, { "start": 77824, "length": 134139904, "depth": 1, "zero": true, "data": f= alse}] + +=3D=3D unaligned image tail cluster, no allocation needed =3D=3D +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D134218752 +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base +wrote 512/512 bytes at offset 134217728 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 512/512 bytes at offset 134217728 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 134217728 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +1024/1024 bytes allocated at offset 128 MiB +[{ "start": 0, "length": 134218752, "depth": 0, "zero": true, "data": fals= e}] +wrote 2048/2048 bytes at offset 134217728 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, +{ "start": 134217728, "length": 2048, "depth": 0, "zero": true, "data": fa= lse}] +wrote 1024/1024 bytes at offset 134217728 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 134218752 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 134218240 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +2048/2048 bytes allocated at offset 128 MiB +[{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, +{ "start": 134217728, "length": 2048, "depth": 0, "zero": true, "data": fa= lse}] +wrote 512/512 bytes at offset 134217728 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 512/512 bytes at offset 134218240 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1024/1024 bytes at offset 134217728 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 134218752, "depth": 0, "zero": true, "data": fals= e}] + +=3D=3D unaligned image tail cluster, allocation required =3D=3D +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D134218752 +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base +wrote 512/512 bytes at offset 134217728 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 512/512 bytes at offset 134218752 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 512/512 bytes at offset 134217728 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1536/1536 bytes at offset 134218240 +1.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, +{ "start": 134217728, "length": 2048, "depth": 0, "zero": false, "data": t= rue, "offset": 20480}] +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D134218752 +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base +wrote 512/512 bytes at offset 134218240 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 512/512 bytes at offset 134217728 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 512/512 bytes at offset 134217728 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 512/512 bytes at offset 134218240 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 134218752 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 512/512 bytes at offset 134219264 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 512/512 bytes at offset 134217728 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 512/512 bytes at offset 134218240 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 134218752 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, +{ "start": 134217728, "length": 2048, "depth": 0, "zero": false, "data": t= rue, "offset": 20480}] +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D134218752 +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134219776 backing_file= =3DTEST_DIR/t.IMGFMT.base +wrote 512/512 bytes at offset 134218752 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 512/512 bytes at offset 134219264 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 134217728 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 512/512 bytes at offset 134218752 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 512/512 bytes at offset 134219264 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 134217728, "depth": 1, "zero": true, "data": fals= e}, +{ "start": 134217728, "length": 2048, "depth": 0, "zero": false, "data": t= rue, "offset": 20480}] *** done --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873800909936.8132638872355; Mon, 10 Apr 2017 18:23:20 -0700 (PDT) Received: from localhost ([::1]:36713 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkWd-0001J6-Up for importer@patchew.org; Mon, 10 Apr 2017 21:23:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46417) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkR6-0005HK-Af for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkR5-0005qM-4X for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40230) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR2-0005oe-LO; Mon, 10 Apr 2017 21:17:32 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A9D6A4AEA3; Tue, 11 Apr 2017 01:17:31 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id EE47278DF2; Tue, 11 Apr 2017 01:17:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A9D6A4AEA3 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com A9D6A4AEA3 From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:11 -0500 Message-Id: <20170411011718.9152-7-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 11 Apr 2017 01:17:31 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 06/13] qcow2: Assert that cluster operations are aligned X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We already audited (in commit 0c1bd469) that qcow2_discard_clusters() is only passed cluster-aligned start values; but we can further tighten the assertion that the only unaligned end value is at EOF. Recent commits have taken advantage of an unaligned tail cluster, for both discard and write zeroes. Signed-off-by: Eric Blake --- v9: rebase to master, by asserting that only tail cluster is unaligned v7, v8: only earlier half of series submitted for 2.9 v6: avoid assertion on non-cluster-aligned image, use s->cluster_sectors to avoid a shift, drop R-b v5: no change v4: new patch --- block/qcow2-cluster.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 12f44b2..362a855 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1519,11 +1519,10 @@ int qcow2_discard_clusters(BlockDriverState *bs, ui= nt64_t offset, end_offset =3D offset + (nb_sectors << BDRV_SECTOR_BITS); - /* The caller must cluster-align start; round end down except at EOF */ + /* Caller must pass aligned values, except at image end */ assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); - if (end_offset !=3D bs->total_sectors * BDRV_SECTOR_SIZE) { - end_offset =3D start_of_cluster(s, end_offset); - } + assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) || + end_offset =3D=3D bs->total_sectors << BDRV_SECTOR_BITS); nb_clusters =3D size_to_clusters(s, end_offset - offset); @@ -1596,9 +1595,17 @@ int qcow2_zero_clusters(BlockDriverState *bs, uint64= _t offset, int nb_sectors, int flags) { BDRVQcow2State *s =3D bs->opaque; + uint64_t end_offset; uint64_t nb_clusters; int ret; + end_offset =3D offset + (nb_sectors << BDRV_SECTOR_BITS); + + /* Caller must pass aligned values, except at image end */ + assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); + assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) || + end_offset =3D=3D bs->total_sectors << BDRV_SECTOR_BITS); + /* The zero flag is only supported by version 3 and newer; we * require the use of that flag if there is a backing file or if * we are not allowed to unmap. */ --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491874092095989.3125666674246; Mon, 10 Apr 2017 18:28:12 -0700 (PDT) Received: from localhost ([::1]:36739 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkbK-0005ci-Tt for importer@patchew.org; Mon, 10 Apr 2017 21:28:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46535) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkRB-0005MA-Ab for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkR9-0005sr-It for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59528) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR3-0005ou-IC; Mon, 10 Apr 2017 21:17:33 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9B67D8124F; Tue, 11 Apr 2017 01:17:32 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id E281678DF2; Tue, 11 Apr 2017 01:17:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 9B67D8124F Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 9B67D8124F From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:12 -0500 Message-Id: <20170411011718.9152-8-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 11 Apr 2017 01:17:32 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 07/13] qcow2: Discard/zero clusters by byte count X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Passing a byte offset, but sector count, when we ultimately want to operate on cluster granularity, is madness. Clean up the external interfaces to take both offset and count as bytes, while still keeping the assertion added previously that the caller must align the values to a cluster. Then rename things to make sure backports don't get confused by changed units: instead of qcow2_discard_clusters() and qcow2_zero_clusters(), we now have qcow2_cluster_discard() and qcow2_cluster_zeroize(). The internal functions still operate on clusters at a time, and return an int for number of cleared clusters; but on an image with 2M clusters, a single L2 table holds 256k entries that each represent a 2M cluster, totalling well over INT_MAX bytes if we ever had a request for that many bytes at once. All our callers currently limit themselves to 32-bit bytes (and therefore fewer clusters), but by making this function 64-bit clean, we have one less place to clean up if we later improve the block layer to support 64-bit bytes through all operations (with the block layer auto-fragmenting on behalf of more-limited drivers), rather than the current state where some interfaces are artificially limited to INT_MAX at a time. Signed-off-by: Eric Blake --- v9: rebase to earlier changes, drop R-b v7, v8: only earlier half of series submitted for 2.9 v6: rebase due to context v5: s/count/byte/ to make the units obvious, and rework the math to ensure no 32-bit integer overflow on large clusters v4: improve function names, split assertion additions into earlier patch [no v3 or v2] v1: https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg00339.html --- block/qcow2.h | 9 +++++---- block/qcow2-cluster.c | 39 ++++++++++++++++++++++----------------- block/qcow2-snapshot.c | 7 +++---- block/qcow2.c | 21 +++++++++------------ 4 files changed, 39 insertions(+), 37 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index f8aeb08..808104c 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -544,10 +544,11 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockD= riverState *bs, int compressed_size); int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m); -int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset, - int nb_sectors, enum qcow2_discard_type type, bool full_discard); -int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sect= ors, - int flags); +int qcow2_cluster_discard(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, enum qcow2_discard_type type, + bool full_discard); +int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, int flags); int qcow2_expand_zero_clusters(BlockDriverState *bs, BlockDriverAmendStatusCB *status_cb, diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 362a855..595e6e3 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1509,16 +1509,16 @@ static int discard_single_l2(BlockDriverState *bs, = uint64_t offset, return nb_clusters; } -int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset, - int nb_sectors, enum qcow2_discard_type type, bool full_discard) +int qcow2_cluster_discard(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, enum qcow2_discard_type type, + bool full_discard) { BDRVQcow2State *s =3D bs->opaque; - uint64_t end_offset; + uint64_t end_offset =3D offset + bytes; uint64_t nb_clusters; + int64_t cleared; int ret; - end_offset =3D offset + (nb_sectors << BDRV_SECTOR_BITS); - /* Caller must pass aligned values, except at image end */ assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) || @@ -1530,13 +1530,15 @@ int qcow2_discard_clusters(BlockDriverState *bs, ui= nt64_t offset, /* Each L2 table is handled by its own loop iteration */ while (nb_clusters > 0) { - ret =3D discard_single_l2(bs, offset, nb_clusters, type, full_disc= ard); - if (ret < 0) { + cleared =3D discard_single_l2(bs, offset, nb_clusters, type, + full_discard); + if (cleared < 0) { + ret =3D cleared; goto fail; } - nb_clusters -=3D ret; - offset +=3D (ret * s->cluster_size); + nb_clusters -=3D cleared; + offset +=3D (cleared * s->cluster_size); } ret =3D 0; @@ -1591,20 +1593,22 @@ static int zero_single_l2(BlockDriverState *bs, uin= t64_t offset, return nb_clusters; } -int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sect= ors, - int flags) +int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, int flags) { BDRVQcow2State *s =3D bs->opaque; uint64_t end_offset; uint64_t nb_clusters; + int64_t cleared; int ret; - end_offset =3D offset + (nb_sectors << BDRV_SECTOR_BITS); + end_offset =3D offset + bytes; /* Caller must pass aligned values, except at image end */ assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) || end_offset =3D=3D bs->total_sectors << BDRV_SECTOR_BITS); + assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); /* The zero flag is only supported by version 3 and newer; we * require the use of that flag if there is a backing file or if @@ -1615,18 +1619,19 @@ int qcow2_zero_clusters(BlockDriverState *bs, uint6= 4_t offset, int nb_sectors, } /* Each L2 table is handled by its own loop iteration */ - nb_clusters =3D size_to_clusters(s, nb_sectors << BDRV_SECTOR_BITS); + nb_clusters =3D size_to_clusters(s, bytes); s->cache_discards =3D true; while (nb_clusters > 0) { - ret =3D zero_single_l2(bs, offset, nb_clusters, flags); - if (ret < 0) { + cleared =3D zero_single_l2(bs, offset, nb_clusters, flags); + if (cleared < 0) { + ret =3D cleared; goto fail; } - nb_clusters -=3D ret; - offset +=3D (ret * s->cluster_size); + nb_clusters -=3D cleared; + offset +=3D (cleared * s->cluster_size); } ret =3D 0; diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 0324243..44243e0 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -440,10 +440,9 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSn= apshotInfo *sn_info) /* The VM state isn't needed any more in the active L1 table; in fact,= it * hurts by causing expensive COW for the next snapshot. */ - qcow2_discard_clusters(bs, qcow2_vm_state_offset(s), - align_offset(sn->vm_state_size, s->cluster_size) - >> BDRV_SECTOR_BITS, - QCOW2_DISCARD_NEVER, false); + qcow2_cluster_discard(bs, qcow2_vm_state_offset(s), + align_offset(sn->vm_state_size, s->cluster_size), + QCOW2_DISCARD_NEVER, false); #ifdef DEBUG_ALLOC { diff --git a/block/qcow2.c b/block/qcow2.c index 8038793..4d34610 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2508,7 +2508,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockD= riverState *bs, trace_qcow2_pwrite_zeroes(qemu_coroutine_self(), offset, count); /* Whatever is left can use real zero clusters */ - ret =3D qcow2_zero_clusters(bs, offset, count >> BDRV_SECTOR_BITS, fla= gs); + ret =3D qcow2_cluster_zeroize(bs, offset, count, flags); qemu_co_mutex_unlock(&s->lock); return ret; @@ -2531,8 +2531,8 @@ static coroutine_fn int qcow2_co_pdiscard(BlockDriver= State *bs, } qemu_co_mutex_lock(&s->lock); - ret =3D qcow2_discard_clusters(bs, offset, count >> BDRV_SECTOR_BITS, - QCOW2_DISCARD_REQUEST, false); + ret =3D qcow2_cluster_discard(bs, offset, count, QCOW2_DISCARD_REQUEST, + false); qemu_co_mutex_unlock(&s->lock); return ret; } @@ -2839,9 +2839,8 @@ fail: static int qcow2_make_empty(BlockDriverState *bs) { BDRVQcow2State *s =3D bs->opaque; - uint64_t start_sector; - int sector_step =3D (QEMU_ALIGN_DOWN(INT_MAX, s->cluster_size) / - BDRV_SECTOR_SIZE); + uint64_t offset, end_offset; + int step =3D QEMU_ALIGN_DOWN(INT_MAX, s->cluster_size); int l1_clusters, ret =3D 0; l1_clusters =3D DIV_ROUND_UP(s->l1_size, s->cluster_size / sizeof(uint= 64_t)); @@ -2858,18 +2857,16 @@ static int qcow2_make_empty(BlockDriverState *bs) /* This fallback code simply discards every active cluster; this is sl= ow, * but works in all cases */ - for (start_sector =3D 0; start_sector < bs->total_sectors; - start_sector +=3D sector_step) + end_offset =3D bs->total_sectors * BDRV_SECTOR_SIZE; + for (offset =3D 0; offset < end_offset; offset +=3D step) { /* As this function is generally used after committing an external * snapshot, QCOW2_DISCARD_SNAPSHOT seems appropriate. Also, the * default action for this kind of discard is to pass the discard, * which will ideally result in an actually smaller image file, as * is probably desired. */ - ret =3D qcow2_discard_clusters(bs, start_sector * BDRV_SECTOR_SIZE, - MIN(sector_step, - bs->total_sectors - start_sector), - QCOW2_DISCARD_SNAPSHOT, true); + ret =3D qcow2_cluster_discard(bs, offset, MIN(step, end_offset - o= ffset), + QCOW2_DISCARD_SNAPSHOT, true); if (ret < 0) { break; } --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873905190873.9677790065155; Mon, 10 Apr 2017 18:25:05 -0700 (PDT) Received: from localhost ([::1]:36720 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkYK-0002oD-7k for importer@patchew.org; Mon, 10 Apr 2017 21:25:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46560) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkRC-0005Mx-0z for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkRA-0005tV-UI for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36812) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR4-0005pQ-EV; Mon, 10 Apr 2017 21:17:34 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8B5144E35B; Tue, 11 Apr 2017 01:17:33 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id D104B78DF2; Tue, 11 Apr 2017 01:17:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8B5144E35B Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 8B5144E35B From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:13 -0500 Message-Id: <20170411011718.9152-9-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 11 Apr 2017 01:17:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 08/13] blkdebug: Sanity check block layer guarantees X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Commits 04ed95f4 and 1a62d0ac updated the block layer to auto-fragment any I/O to fit within device boundaries. Additionally, when using a minimum alignment of 4k, we want to ensure the block layer does proper read-modify-write rather than requesting I/O on a slice of a sector. Let's enforce that the contract is obeyed when using blkdebug. For now, blkdebug only allows alignment overrides, and just inherits other limits from whatever device it is wrapping, but a future patch will further enhance things. Signed-off-by: Eric Blake Reviewed-by: Kevin Wolf Reviewed-by: Max Reitz --- v5-v9: no change v4: no change v3: rebase to byte-based interfaces v2: new patch --- block/blkdebug.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/block/blkdebug.c b/block/blkdebug.c index c795ae9..14d3fc5 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -431,6 +431,13 @@ blkdebug_co_preadv(BlockDriverState *bs, uint64_t offs= et, uint64_t bytes, BDRVBlkdebugState *s =3D bs->opaque; BlkdebugRule *rule =3D NULL; + /* Sanity check block layer guarantees */ + assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment)); + assert(QEMU_IS_ALIGNED(bytes, bs->bl.request_alignment)); + if (bs->bl.max_transfer) { + assert(bytes <=3D bs->bl.max_transfer); + } + QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) { uint64_t inject_offset =3D rule->options.inject.offset; @@ -455,6 +462,13 @@ blkdebug_co_pwritev(BlockDriverState *bs, uint64_t off= set, uint64_t bytes, BDRVBlkdebugState *s =3D bs->opaque; BlkdebugRule *rule =3D NULL; + /* Sanity check block layer guarantees */ + assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment)); + assert(QEMU_IS_ALIGNED(bytes, bs->bl.request_alignment)); + if (bs->bl.max_transfer) { + assert(bytes <=3D bs->bl.max_transfer); + } + QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) { uint64_t inject_offset =3D rule->options.inject.offset; --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873761943187.73609612776886; Mon, 10 Apr 2017 18:22:41 -0700 (PDT) Received: from localhost ([::1]:36710 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkW0-0000kc-W4 for importer@patchew.org; Mon, 10 Apr 2017 21:22:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46521) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkRA-0005LQ-OW for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkR9-0005sm-Gl for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37070) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR5-0005qG-E1; Mon, 10 Apr 2017 21:17:35 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7CC7880F75; Tue, 11 Apr 2017 01:17:34 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id C324E78DF2; Tue, 11 Apr 2017 01:17:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 7CC7880F75 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 7CC7880F75 From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:14 -0500 Message-Id: <20170411011718.9152-10-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 11 Apr 2017 01:17:34 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 09/13] blkdebug: Refactor error injection X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Rather than repeat the logic at each caller of checking if a Rule exists that warrants an error injection, fold that logic into inject_error(); and rename it to rule_check() for legibility. This will help the next patch, which adds two more callers that need to check rules for the potential of injecting errors. Signed-off-by: Eric Blake Reviewed-by: Max Reitz --- v9: no change v7-v8: not submitted (earlier half of series sent for 2.9) v6: new patch --- block/blkdebug.c | 74 +++++++++++++++++++++++++---------------------------= ---- 1 file changed, 33 insertions(+), 41 deletions(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index 14d3fc5..db58db0 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -405,11 +405,30 @@ out: return ret; } -static int inject_error(BlockDriverState *bs, BlkdebugRule *rule) +static int rule_check(BlockDriverState *bs, uint64_t offset, uint64_t byte= s) { BDRVBlkdebugState *s =3D bs->opaque; - int error =3D rule->options.inject.error; - bool immediately =3D rule->options.inject.immediately; + BlkdebugRule *rule =3D NULL; + int error; + bool immediately; + + QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) { + uint64_t inject_offset =3D rule->options.inject.offset; + + if (inject_offset =3D=3D -1 || + (bytes && inject_offset >=3D offset && + inject_offset < offset + bytes)) + { + break; + } + } + + if (!rule || !rule->options.inject.error) { + return 0; + } + + immediately =3D rule->options.inject.immediately; + error =3D rule->options.inject.error; if (rule->options.inject.once) { QSIMPLEQ_REMOVE(&s->active_rules, rule, BlkdebugRule, active_next); @@ -428,8 +447,7 @@ static int coroutine_fn blkdebug_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - BDRVBlkdebugState *s =3D bs->opaque; - BlkdebugRule *rule =3D NULL; + int err; /* Sanity check block layer guarantees */ assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment)); @@ -438,18 +456,9 @@ blkdebug_co_preadv(BlockDriverState *bs, uint64_t offs= et, uint64_t bytes, assert(bytes <=3D bs->bl.max_transfer); } - QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) { - uint64_t inject_offset =3D rule->options.inject.offset; - - if (inject_offset =3D=3D -1 || - (inject_offset >=3D offset && inject_offset < offset + bytes)) - { - break; - } - } - - if (rule && rule->options.inject.error) { - return inject_error(bs, rule); + err =3D rule_check(bs, offset, bytes); + if (err) { + return err; } return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags); @@ -459,8 +468,7 @@ static int coroutine_fn blkdebug_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - BDRVBlkdebugState *s =3D bs->opaque; - BlkdebugRule *rule =3D NULL; + int err; /* Sanity check block layer guarantees */ assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment)); @@ -469,18 +477,9 @@ blkdebug_co_pwritev(BlockDriverState *bs, uint64_t off= set, uint64_t bytes, assert(bytes <=3D bs->bl.max_transfer); } - QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) { - uint64_t inject_offset =3D rule->options.inject.offset; - - if (inject_offset =3D=3D -1 || - (inject_offset >=3D offset && inject_offset < offset + bytes)) - { - break; - } - } - - if (rule && rule->options.inject.error) { - return inject_error(bs, rule); + err =3D rule_check(bs, offset, bytes); + if (err) { + return err; } return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags); @@ -488,17 +487,10 @@ blkdebug_co_pwritev(BlockDriverState *bs, uint64_t of= fset, uint64_t bytes, static int blkdebug_co_flush(BlockDriverState *bs) { - BDRVBlkdebugState *s =3D bs->opaque; - BlkdebugRule *rule =3D NULL; + int err =3D rule_check(bs, 0, 0); - QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) { - if (rule->options.inject.offset =3D=3D -1) { - break; - } - } - - if (rule && rule->options.inject.error) { - return inject_error(bs, rule); + if (err) { + return err; } return bdrv_co_flush(bs->file->bs); --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873877460749.0640463346829; Mon, 10 Apr 2017 18:24:37 -0700 (PDT) Received: from localhost ([::1]:36719 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkXs-0002R0-3z for importer@patchew.org; Mon, 10 Apr 2017 21:24:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46582) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkRC-0005Nw-Rb for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkRB-0005tj-EQ for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55188) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR6-0005qp-Ey; Mon, 10 Apr 2017 21:17:36 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6F5967E9EB; Tue, 11 Apr 2017 01:17:35 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id B376A78DF2; Tue, 11 Apr 2017 01:17:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6F5967E9EB Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 6F5967E9EB From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:15 -0500 Message-Id: <20170411011718.9152-11-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 11 Apr 2017 01:17:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 10/13] blkdebug: Add pass-through write_zero and discard support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" In order to test the effects of artificial geometry constraints on operations like write zero or discard, we first need blkdebug to manage these actions. It also allows us to inject errors on those operations, just like we can for read/write/flush. We can also test the contract promised by the block layer; namely, if a device has specified limits on alignment or maximum size, then those limits must be obeyed (for now, the blkdebug driver merely inherits limits from whatever it is wrapping, but the next patch will further enhance it to allow specific limit overrides). This patch intentionally refuses to service requests smaller than the requested alignments; this is because an upcoming patch adds a qemu-iotest to prove that the block layer is correctly handling fragmentation, but the test only works if there is a way to tell the difference at artificial alignment boundaries when blkdebug is using a larger-than-default alignment. If we let the blkdebug layer always defer to the underlying layer, which potentially has a smaller granularity, the iotest will be thwarted. Tested by setting up an NBD server with export 'foo', then invoking: $ ./qemu-io qemu-io> open -o driver=3Dblkdebug blkdebug::nbd://localhost:10809/foo qemu-io> d 0 15M qemu-io> w -z 0 15M Pre-patch, the server never sees the discard (it was silently eaten by the block layer); post-patch it is passed across the wire. Likewise, pre-patch the write is always passed with NBD_WRITE (with 15M of zeroes on the wire), while post-patch it can utilize NBD_WRITE_ZEROES (for less traffic). Signed-off-by: Eric Blake Reviewed-by: Max Reitz --- v9: no change v7-v8: not submitted (earlier half of series sent for 2.9) v6: tighten check of unaligned requests, rebase on rule check refactoring, drop R-b v5: include 2017 copyright v4: correct error injection to respect byte range, tweak formatting v3: rebase to byte-based read/write, improve docs on why no partial write zero passthrough v2: new patch --- block/blkdebug.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 74 insertions(+) diff --git a/block/blkdebug.c b/block/blkdebug.c index db58db0..62d113c 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -1,6 +1,7 @@ /* * Block protocol for I/O error injection * + * Copyright (C) 2016-2017 Red Hat, Inc. * Copyright (c) 2010 Kevin Wolf * * Permission is hereby granted, free of charge, to any person obtaining a= copy @@ -382,6 +383,11 @@ static int blkdebug_open(BlockDriverState *bs, QDict *= options, int flags, goto out; } + bs->supported_write_flags =3D BDRV_REQ_FUA & + bs->file->bs->supported_write_flags; + bs->supported_zero_flags =3D (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & + bs->file->bs->supported_zero_flags; + /* Set request alignment */ align =3D qemu_opt_get_size(opts, "align", 0); if (align < INT_MAX && is_power_of_2(align)) { @@ -496,6 +502,72 @@ static int blkdebug_co_flush(BlockDriverState *bs) return bdrv_co_flush(bs->file->bs); } +static int coroutine_fn blkdebug_co_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, int coun= t, + BdrvRequestFlags flags) +{ + uint32_t align =3D MAX(bs->bl.request_alignment, + bs->bl.pwrite_zeroes_alignment); + int err; + + /* Only pass through requests that are larger than requested + * preferred alignment (so that we test the fallback to writes on + * unaligned portions), and check that the block layer never hands + * us anything unaligned that crosses an alignment boundary. */ + if (count < align) { + assert(QEMU_IS_ALIGNED(offset, align) || + QEMU_IS_ALIGNED(offset + count, align) || + DIV_ROUND_UP(offset, align) =3D=3D + DIV_ROUND_UP(offset + count, align)); + return -ENOTSUP; + } + assert(QEMU_IS_ALIGNED(offset, align)); + assert(QEMU_IS_ALIGNED(count, align)); + if (bs->bl.max_pwrite_zeroes) { + assert(count <=3D bs->bl.max_pwrite_zeroes); + } + + err =3D rule_check(bs, offset, count); + if (err) { + return err; + } + + return bdrv_co_pwrite_zeroes(bs->file, offset, count, flags); +} + +static int coroutine_fn blkdebug_co_pdiscard(BlockDriverState *bs, + int64_t offset, int count) +{ + uint32_t align =3D bs->bl.pdiscard_alignment; + int err; + + /* Only pass through requests that are larger than requested + * minimum alignment, and ensure that unaligned requests do not + * cross optimum discard boundaries. */ + if (count < bs->bl.request_alignment) { + assert(QEMU_IS_ALIGNED(offset, align) || + QEMU_IS_ALIGNED(offset + count, align) || + DIV_ROUND_UP(offset, align) =3D=3D + DIV_ROUND_UP(offset + count, align)); + return -ENOTSUP; + } + assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment)); + assert(QEMU_IS_ALIGNED(count, bs->bl.request_alignment)); + if (align && count >=3D align) { + assert(QEMU_IS_ALIGNED(offset, align)); + assert(QEMU_IS_ALIGNED(count, align)); + } + if (bs->bl.max_pdiscard) { + assert(count <=3D bs->bl.max_pdiscard); + } + + err =3D rule_check(bs, offset, count); + if (err) { + return err; + } + + return bdrv_co_pdiscard(bs->file->bs, offset, count); +} static void blkdebug_close(BlockDriverState *bs) { @@ -750,6 +822,8 @@ static BlockDriver bdrv_blkdebug =3D { .bdrv_co_preadv =3D blkdebug_co_preadv, .bdrv_co_pwritev =3D blkdebug_co_pwritev, .bdrv_co_flush_to_disk =3D blkdebug_co_flush, + .bdrv_co_pwrite_zeroes =3D blkdebug_co_pwrite_zeroes, + .bdrv_co_pdiscard =3D blkdebug_co_pdiscard, .bdrv_debug_event =3D blkdebug_debug_event, .bdrv_debug_breakpoint =3D blkdebug_debug_breakpoint, --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149187373174236.69313706660944; Mon, 10 Apr 2017 18:22:11 -0700 (PDT) Received: from localhost ([::1]:36709 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkVW-0000HQ-P3 for importer@patchew.org; Mon, 10 Apr 2017 21:22:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46564) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkRC-0005N0-2a for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkRA-0005tP-T3 for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59540) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR7-0005rR-9r; Mon, 10 Apr 2017 21:17:37 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5BDA57E9C2; Tue, 11 Apr 2017 01:17:36 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id A38F878DF2; Tue, 11 Apr 2017 01:17:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5BDA57E9C2 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 5BDA57E9C2 From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:16 -0500 Message-Id: <20170411011718.9152-12-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 11 Apr 2017 01:17:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 11/13] blkdebug: Simplify override logic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Rather than store into a local variable, then copy to the struct if the value is valid, then reporting errors otherwise, it is simpler to just store into the struct and report errors if the value is invalid. This however requires that the struct store a 64-bit number, rather than a narrower type. Move the errno assignment into a label that will be reused from more places in the next patch. Signed-off-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Kevin Wolf --- v9: no change v7-v8: not submitted (earlier half of series sent for 2.9) v6: tweak error message, but R-b kept v5: no change v4: fix typo in commit message, move errno assignment v3: new patch --- block/blkdebug.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index 62d113c..a1501eb 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -38,7 +38,7 @@ typedef struct BDRVBlkdebugState { int state; int new_state; - int align; + uint64_t align; /* For blkdebug_refresh_filename() */ char *config_file; @@ -353,7 +353,6 @@ static int blkdebug_open(BlockDriverState *bs, QDict *o= ptions, int flags, BDRVBlkdebugState *s =3D bs->opaque; QemuOpts *opts; Error *local_err =3D NULL; - uint64_t align; int ret; opts =3D qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); @@ -389,12 +388,10 @@ static int blkdebug_open(BlockDriverState *bs, QDict = *options, int flags, bs->file->bs->supported_zero_flags; /* Set request alignment */ - align =3D qemu_opt_get_size(opts, "align", 0); - if (align < INT_MAX && is_power_of_2(align)) { - s->align =3D align; - } else if (align) { - error_setg(errp, "Invalid alignment"); - ret =3D -EINVAL; + s->align =3D qemu_opt_get_size(opts, "align", 0); + if (s->align && (s->align >=3D INT_MAX || !is_power_of_2(s->align))) { + error_setg(errp, "Cannot meet constraints with align %" PRIu64, + s->align); goto fail_unref; } @@ -402,6 +399,7 @@ static int blkdebug_open(BlockDriverState *bs, QDict *o= ptions, int flags, goto out; fail_unref: + ret =3D -EINVAL; bdrv_unref_child(bs, bs->file); out: if (ret < 0) { --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491873874676921.3826045739557; Mon, 10 Apr 2017 18:24:34 -0700 (PDT) Received: from localhost ([::1]:36718 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkXp-0002Ni-Gs for importer@patchew.org; Mon, 10 Apr 2017 21:24:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46598) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkRE-0005P0-3w for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkRC-0005uE-JN for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33156) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR8-0005ry-C1; Mon, 10 Apr 2017 21:17:38 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6A3A4C059730; Tue, 11 Apr 2017 01:17:37 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id 946C178DF2; Tue, 11 Apr 2017 01:17:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6A3A4C059730 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 6A3A4C059730 From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:17 -0500 Message-Id: <20170411011718.9152-13-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 11 Apr 2017 01:17:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 12/13] blkdebug: Add ability to override unmap geometries X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Markus Armbruster , qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make it easier to simulate various unusual hardware setups (for example, recent commits 3482b9b and b8d0a98 affect the Dell Equallogic iSCSI with its 15M preferred and maximum unmap and write zero sizing, or b2f95fe deals with the Linux loopback block device having a max_transfer of 64k), by allowing blkdebug to wrap any other device with further restrictions on various alignments. Signed-off-by: Eric Blake Reviewed-by: Max Reitz --- v9: rebase to master (dropped #optional tags) v7-v8: not submitted (earlier half of series sent for 2.9) v6: more tweaks in docs and error messages v5: tweak docs regarding max-transfer minimum v4: relax 512 byte minimum now that blkdebug is byte-based, fix doc typo v3: improve legibility of bounds checking, improve docs v2: new patch --- qapi/block-core.json | 33 ++++++++++++++++-- block/blkdebug.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++= +++- 2 files changed, 125 insertions(+), 4 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 033457c..38edada 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2428,8 +2428,33 @@ # # @config: filename of the configuration file # -# @align: required alignment for requests in bytes, -# must be power of 2, or 0 for default +# @align: required alignment for requests in bytes, must be +# positive power of 2, or 0 for default +# +# @max-transfer: maximum size for I/O transfers in bytes, must be +# positive multiple of @align and of the underlying +# file's request alignment (but need not be a power of +# 2), or 0 for default (since 2.10) +# +# @opt-write-zero: preferred alignment for write zero requests in bytes, +# must be positive multiple of @align and of the +# underlying file's request alignment (but need not be a +# power of 2), or 0 for default (since 2.10) +# +# @max-write-zero: maximum size for write zero requests in bytes, must be +# positive multiple of @align, of @opt-write-zero, and of +# the underlying file's request alignment (but need not +# be a power of 2), or 0 for default (since 2.10) +# +# @opt-discard: preferred alignment for discard requests in bytes, must +# be positive multiple of @align and of the underlying +# file's request alignment (but need not be a power of +# 2), or 0 for default (since 2.10) +# +# @max-discard: maximum size for discard requests in bytes, must be +# positive multiple of @align, of @opt-discard, and of +# the underlying file's request alignment (but need not +# be a power of 2), or 0 for default (since 2.10) # # @inject-error: array of error injection descriptions # @@ -2440,7 +2465,9 @@ { 'struct': 'BlockdevOptionsBlkdebug', 'data': { 'image': 'BlockdevRef', '*config': 'str', - '*align': 'int', + '*align': 'int', '*max-transfer': 'int32', + '*opt-write-zero': 'int32', '*max-write-zero': 'int32', + '*opt-discard': 'int32', '*max-discard': 'int32', '*inject-error': ['BlkdebugInjectErrorOptions'], '*set-state': ['BlkdebugSetStateOptions'] } } diff --git a/block/blkdebug.c b/block/blkdebug.c index a1501eb..4ccdceb 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -39,6 +39,11 @@ typedef struct BDRVBlkdebugState { int state; int new_state; uint64_t align; + uint64_t max_transfer; + uint64_t opt_write_zero; + uint64_t max_write_zero; + uint64_t opt_discard; + uint64_t max_discard; /* For blkdebug_refresh_filename() */ char *config_file; @@ -343,6 +348,31 @@ static QemuOptsList runtime_opts =3D { .type =3D QEMU_OPT_SIZE, .help =3D "Required alignment in bytes", }, + { + .name =3D "max-transfer", + .type =3D QEMU_OPT_SIZE, + .help =3D "Maximum transfer size in bytes", + }, + { + .name =3D "opt-write-zero", + .type =3D QEMU_OPT_SIZE, + .help =3D "Optimum write zero alignment in bytes", + }, + { + .name =3D "max-write-zero", + .type =3D QEMU_OPT_SIZE, + .help =3D "Maximum write zero size in bytes", + }, + { + .name =3D "opt-discard", + .type =3D QEMU_OPT_SIZE, + .help =3D "Optimum discard alignment in bytes", + }, + { + .name =3D "max-discard", + .type =3D QEMU_OPT_SIZE, + .help =3D "Maximum discard size in bytes", + }, { /* end of list */ } }, }; @@ -354,6 +384,7 @@ static int blkdebug_open(BlockDriverState *bs, QDict *o= ptions, int flags, QemuOpts *opts; Error *local_err =3D NULL; int ret; + uint64_t align; opts =3D qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); qemu_opts_absorb_qdict(opts, options, &local_err); @@ -387,13 +418,61 @@ static int blkdebug_open(BlockDriverState *bs, QDict = *options, int flags, bs->supported_zero_flags =3D (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & bs->file->bs->supported_zero_flags; - /* Set request alignment */ + /* Set alignment overrides */ s->align =3D qemu_opt_get_size(opts, "align", 0); if (s->align && (s->align >=3D INT_MAX || !is_power_of_2(s->align))) { error_setg(errp, "Cannot meet constraints with align %" PRIu64, s->align); goto fail_unref; } + align =3D MAX(s->align, bs->file->bs->bl.request_alignment); + + s->max_transfer =3D qemu_opt_get_size(opts, "max-transfer", 0); + if (s->max_transfer && + (s->max_transfer >=3D INT_MAX || + !QEMU_IS_ALIGNED(s->max_transfer, align))) { + error_setg(errp, "Cannot meet constraints with max-transfer %" PRI= u64, + s->max_transfer); + goto fail_unref; + } + + s->opt_write_zero =3D qemu_opt_get_size(opts, "opt-write-zero", 0); + if (s->opt_write_zero && + (s->opt_write_zero >=3D INT_MAX || + !QEMU_IS_ALIGNED(s->opt_write_zero, align))) { + error_setg(errp, "Cannot meet constraints with opt-write-zero %" P= RIu64, + s->opt_write_zero); + goto fail_unref; + } + + s->max_write_zero =3D qemu_opt_get_size(opts, "max-write-zero", 0); + if (s->max_write_zero && + (s->max_write_zero >=3D INT_MAX || + !QEMU_IS_ALIGNED(s->max_write_zero, + MAX(s->opt_write_zero, align)))) { + error_setg(errp, "Cannot meet constraints with max-write-zero %" P= RIu64, + s->max_write_zero); + goto fail_unref; + } + + s->opt_discard =3D qemu_opt_get_size(opts, "opt-discard", 0); + if (s->opt_discard && + (s->opt_discard >=3D INT_MAX || + !QEMU_IS_ALIGNED(s->opt_discard, align))) { + error_setg(errp, "Cannot meet constraints with opt-discard %" PRIu= 64, + s->opt_discard); + goto fail_unref; + } + + s->max_discard =3D qemu_opt_get_size(opts, "max-discard", 0); + if (s->max_discard && + (s->max_discard >=3D INT_MAX || + !QEMU_IS_ALIGNED(s->max_discard, + MAX(s->opt_discard, align)))) { + error_setg(errp, "Cannot meet constraints with max-discard %" PRIu= 64, + s->max_discard); + goto fail_unref; + } ret =3D 0; goto out; @@ -793,6 +872,21 @@ static void blkdebug_refresh_limits(BlockDriverState *= bs, Error **errp) if (s->align) { bs->bl.request_alignment =3D s->align; } + if (s->max_transfer) { + bs->bl.max_transfer =3D s->max_transfer; + } + if (s->opt_write_zero) { + bs->bl.pwrite_zeroes_alignment =3D s->opt_write_zero; + } + if (s->max_write_zero) { + bs->bl.max_pwrite_zeroes =3D s->max_write_zero; + } + if (s->opt_discard) { + bs->bl.pdiscard_alignment =3D s->opt_discard; + } + if (s->max_discard) { + bs->bl.max_pdiscard =3D s->max_discard; + } } static int blkdebug_reopen_prepare(BDRVReopenState *reopen_state, --=20 2.9.3 From nobody Sat May 4 19:27:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1491874048900380.5362887888207; Mon, 10 Apr 2017 18:27:28 -0700 (PDT) Received: from localhost ([::1]:36736 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkad-0004yA-RQ for importer@patchew.org; Mon, 10 Apr 2017 21:27:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46625) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxkRG-0005Qu-5L for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxkRE-0005uk-5o for qemu-devel@nongnu.org; Mon, 10 Apr 2017 21:17:46 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46734) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cxkR9-0005sJ-E1; Mon, 10 Apr 2017 21:17:39 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5D9E015567; Tue, 11 Apr 2017 01:17:38 +0000 (UTC) Received: from red.redhat.com (ovpn-122-58.rdu2.redhat.com [10.10.122.58]) by smtp.corp.redhat.com (Postfix) with ESMTP id A2B4478DF2; Tue, 11 Apr 2017 01:17:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5D9E015567 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 5D9E015567 From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 10 Apr 2017 20:17:18 -0500 Message-Id: <20170411011718.9152-14-eblake@redhat.com> In-Reply-To: <20170411011718.9152-1-eblake@redhat.com> References: <20170411011718.9152-1-eblake@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 11 Apr 2017 01:17:38 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v9 13/13] tests: Add coverage for recent block geometry fixes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-block@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Use blkdebug's new geometry constraints to emulate setups that have caused recent regression fixes: write zeroes asserting when running through a loopback block device with max-transfer smaller than cluster size, and discard rounding away portions of requests not aligned to preferred boundaries. Also, add coverage that the block layer is honoring max transfer limits. For now, a single iotest performs all actions, with the idea that we can add future blkdebug constraint test cases in the same file; but it can be split into multiple iotests if we find reason to run one portion of the test in more setups than what are possible in the other. For reference, the final portion of the test (checking whether discard passes as much as possible to the lowest layers of the stack) works as follows: qemu-io: discard 30M at 80000001, passed to blkdebug blkdebug: discard 511 bytes at 80000001, -ENOTSUP (smaller than blkdebug's 512 align) blkdebug: discard 14371328 bytes at 80000512, passed to qcow2 qcow2: discard 739840 bytes at 80000512, -ENOTSUP (smaller than qcow2's 1M align) qcow2: discard 13M bytes at 77M, succeeds blkdebug: discard 15M bytes at 90M, passed to qcow2 qcow2: discard 15M bytes at 90M, succeeds blkdebug: discard 1356800 bytes at 105M, passed to qcow2 qcow2: discard 1M at 105M, succeeds qcow2: discard 308224 bytes at 106M, -ENOTSUP (smaller than qcow2's 1M align) blkdebug: discard 1 byte at 111457280, -ENOTSUP (smaller than blkdebug's 512 align) Signed-off-by: Eric Blake Reviewed-by: Max Reitz --- [yes, I added tests 177 and 179 out of order during this series. Oh well; I'm tired of renumbering this one.] v9: no change v7-v8: not submitted (earlier half of series sent for 2.9) v6: rebase to master by renumbering s/175/177/ v5: rebase to master by renumbering s/173/175/ v4: clean up some comments, nicer backing file creation, more commit message v3: make comments tied more to test at hand, rather than the particular hardware that led to the earlier patches being tested v2: new patch --- tests/qemu-iotests/177 | 114 +++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/177.out | 49 +++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 164 insertions(+) create mode 100755 tests/qemu-iotests/177 create mode 100644 tests/qemu-iotests/177.out diff --git a/tests/qemu-iotests/177 b/tests/qemu-iotests/177 new file mode 100755 index 0000000..e4ddec7 --- /dev/null +++ b/tests/qemu-iotests/177 @@ -0,0 +1,114 @@ +#!/bin/bash +# +# Test corner cases with unusual block geometries +# +# Copyright (C) 2016-2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=3Deblake@redhat.com + +seq=3D`basename $0` +echo "QA output created by $seq" + +here=3D`pwd` +status=3D1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file + +CLUSTER_SIZE=3D1M +size=3D128M +options=3Ddriver=3Dblkdebug,image.driver=3Dqcow2 + +echo +echo "=3D=3D setting up files =3D=3D" + +TEST_IMG=3D"$TEST_IMG.base" _make_test_img $size +$QEMU_IO -c "write -P 11 0 $size" "$TEST_IMG.base" | _filter_qemu_io +_make_test_img -b "$TEST_IMG.base" +$QEMU_IO -c "write -P 22 0 $size" "$TEST_IMG" | _filter_qemu_io + +# Limited to 64k max-transfer +echo +echo "=3D=3D constrained alignment and max-transfer =3D=3D" +limits=3Dalign=3D4k,max-transfer=3D64k +$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \ + -c "write -P 33 1000 128k" -c "read -P 33 1000 128k" | _filter_qe= mu_io + +echo +echo "=3D=3D write zero with constrained max-transfer =3D=3D" +limits=3Dalign=3D512,max-transfer=3D64k,opt-write-zero=3D$CLUSTER_SIZE +$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \ + -c "write -z 8003584 2093056" | _filter_qemu_io + +# non-power-of-2 write-zero/discard alignments +echo +echo "=3D=3D non-power-of-2 write zeroes limits =3D=3D" + +limits=3Dalign=3D512,opt-write-zero=3D15M,max-write-zero=3D15M,opt-discard= =3D15M,max-discard=3D15M +$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \ + -c "write -z 32M 32M" | _filter_qemu_io + +echo +echo "=3D=3D non-power-of-2 discard limits =3D=3D" + +limits=3Dalign=3D512,opt-write-zero=3D15M,max-write-zero=3D15M,opt-discard= =3D15M,max-discard=3D15M +$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \ + -c "discard 80000001 30M" | _filter_qemu_io + +echo +echo "=3D=3D verify image content =3D=3D" + +function verify_io() +{ + if ($QEMU_IMG info -f "$IMGFMT" "$TEST_IMG" | + grep "compat: 0.10" > /dev/null); then + # For v2 images, discarded clusters are read from the backing file + discarded=3D11 + else + # Discarded clusters are zeroed for v3 or later + discarded=3D0 + fi + + echo read -P 22 0 1000 + echo read -P 33 1000 128k + echo read -P 22 132072 7871512 + echo read -P 0 8003584 2093056 + echo read -P 22 10096640 23457792 + echo read -P 0 32M 32M + echo read -P 22 64M 13M + echo read -P $discarded 77M 29M + echo read -P 22 106M 22M +} + +verify_io | $QEMU_IO "$TEST_IMG" | _filter_qemu_io + +_check_test_img + +# success, all done +echo "*** done" +status=3D0 diff --git a/tests/qemu-iotests/177.out b/tests/qemu-iotests/177.out new file mode 100644 index 0000000..e887542 --- /dev/null +++ b/tests/qemu-iotests/177.out @@ -0,0 +1,49 @@ +QA output created by 177 + +=3D=3D setting up files =3D=3D +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D134217728 +wrote 134217728/134217728 bytes at offset 0 +128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134217728 backing_file= =3DTEST_DIR/t.IMGFMT.base +wrote 134217728/134217728 bytes at offset 0 +128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=3D=3D constrained alignment and max-transfer =3D=3D +wrote 131072/131072 bytes at offset 1000 +128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 131072/131072 bytes at offset 1000 +128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=3D=3D write zero with constrained max-transfer =3D=3D +wrote 2093056/2093056 bytes at offset 8003584 +1.996 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=3D=3D non-power-of-2 write zeroes limits =3D=3D +wrote 33554432/33554432 bytes at offset 33554432 +32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=3D=3D non-power-of-2 discard limits =3D=3D +discard 31457280/31457280 bytes at offset 80000001 +30 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=3D=3D verify image content =3D=3D +read 1000/1000 bytes at offset 0 +1000 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 131072/131072 bytes at offset 1000 +128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 7871512/7871512 bytes at offset 132072 +7.507 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2093056/2093056 bytes at offset 8003584 +1.996 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 23457792/23457792 bytes at offset 10096640 +22.371 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 33554432/33554432 bytes at offset 33554432 +32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 13631488/13631488 bytes at offset 67108864 +13 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 30408704/30408704 bytes at offset 80740352 +29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 23068672/23068672 bytes at offset 111149056 +22 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +No errors were found on the image. +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index cfc8823..e7f987e 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -169,4 +169,5 @@ 174 auto 175 auto quick 176 rw auto backing +177 rw auto quick 179 rw auto quick --=20 2.9.3