From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305567674950.5628495511598; Fri, 6 Oct 2017 08:59:27 -0700 (PDT) Received: from localhost ([::1]:45613 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V21-0003yN-Qc for importer@patchew.org; Fri, 06 Oct 2017 11:59:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55262) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UxQ-0008Tp-9V for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UxP-0002NK-HU for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39360) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxJ-0002C6-As; Fri, 06 Oct 2017 11:54:29 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 387C3CD254; Fri, 6 Oct 2017 15:54:28 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 480A067585; Fri, 6 Oct 2017 15:54:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 387C3CD254 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:29 +0200 Message-Id: <20171006155422.10135-2-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 06 Oct 2017 15:54:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 01/54] block: Typo fix in copy_on_readv() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Signed-off-by: Eric Blake Signed-off-by: Kevin Wolf --- block/io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/io.c b/block/io.c index 4378ae4c7d..d633b0f851 100644 --- a/block/io.c +++ b/block/io.c @@ -954,7 +954,7 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(BdrvCh= ild *child, /* FIXME We cannot require callers to have write permissions when all = they * are doing is a read request. If we did things right, write permissi= ons * would be obtained anyway, but internally by the copy-on-read code. = As - * long as it is implemented here rather than in a separat filter driv= er, + * long as it is implemented here rather than in a separate filter dri= ver, * the copy-on-read code doesn't have its own BdrvChild, however, for = which * it could request permissions. Therefore we have to bypass the permi= ssion * system for the moment. */ --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305440377409.75128759789015; Fri, 6 Oct 2017 08:57:20 -0700 (PDT) Received: from localhost ([::1]:45604 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uzx-0002Cz-Je for importer@patchew.org; Fri, 06 Oct 2017 11:57:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55268) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UxQ-0008Tv-Es for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UxP-0002Nm-NL for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56398) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxK-0002EF-HO; Fri, 06 Oct 2017 11:54:30 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 755233E2A4; Fri, 6 Oct 2017 15:54:29 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 885A667594; Fri, 6 Oct 2017 15:54:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 755233E2A4 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:30 +0200 Message-Id: <20171006155422.10135-3-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 06 Oct 2017 15:54:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 02/54] block: Make bdrv_img_create() size selection easier to read X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake All callers of bdrv_img_create() pass in a size, or -1 to read the size from the backing file. We then set that size as the QemuOpt default, which means we will reuse that default rather than the final parameter to qemu_opt_get_size() several lines later. But it is rather confusing to read subsequent checks of 'size =3D=3D -1' when it looks (without seeing the full context) like size defaults to 0; it also doesn't help that a size of 0 is valid (for some formats). Rework the logic to make things more legible. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block.c b/block.c index 5c65fac672..528cda7b2c 100644 --- a/block.c +++ b/block.c @@ -4488,7 +4488,7 @@ void bdrv_img_create(const char *filename, const char= *fmt, =20 /* The size for the image must always be specified, unless we have a b= acking * file and we have not been forbidden from opening it. */ - size =3D qemu_opt_get_size(opts, BLOCK_OPT_SIZE, 0); + size =3D qemu_opt_get_size(opts, BLOCK_OPT_SIZE, img_size); if (backing_file && !(flags & BDRV_O_NO_BACKING)) { BlockDriverState *bs; char *full_backing =3D g_new0(char, PATH_MAX); --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305611756354.1759205630069; Fri, 6 Oct 2017 09:00:11 -0700 (PDT) Received: from localhost ([::1]:45615 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V2l-0004pj-KN for importer@patchew.org; Fri, 06 Oct 2017 12:00:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55326) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UxR-0008Vr-SC for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UxQ-0002P4-AX for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56444) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxL-0002G7-N0; Fri, 06 Oct 2017 11:54:31 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B4791267CD; Fri, 6 Oct 2017 15:54:30 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id C770967585; Fri, 6 Oct 2017 15:54:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B4791267CD Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:31 +0200 Message-Id: <20171006155422.10135-4-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 06 Oct 2017 15:54:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 03/54] hbitmap: Rename serialization_granularity to serialization_align X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake The only client of hbitmap_serialization_granularity() is dirty-bitmap's bdrv_dirty_bitmap_serialization_align(). Keeping the two names consistent is worthwhile, and the shorter name is more representative of what the function returns (the required alignment to be used for start/count of other serialization functions, where violating the alignment causes assertion failures). Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- include/qemu/hbitmap.h | 8 ++++---- block/dirty-bitmap.c | 2 +- tests/test-hbitmap.c | 10 +++++----- util/hbitmap.c | 8 ++++---- 4 files changed, 14 insertions(+), 14 deletions(-) diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index d3a74a21fc..81e78043d1 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -159,16 +159,16 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item); bool hbitmap_is_serializable(const HBitmap *hb); =20 /** - * hbitmap_serialization_granularity: + * hbitmap_serialization_align: * @hb: HBitmap to operate on. * - * Granularity of serialization chunks, used by other serialization functi= ons. - * For every chunk: + * Required alignment of serialization chunks, used by other serialization + * functions. For every chunk: * 1. Chunk start should be aligned to this granularity. * 2. Chunk size should be aligned too, except for last chunk (for which * start + count =3D=3D hb->size) */ -uint64_t hbitmap_serialization_granularity(const HBitmap *hb); +uint64_t hbitmap_serialization_align(const HBitmap *hb); =20 /** * hbitmap_serialization_size: diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 30462d4f9a..0490ca3aff 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -617,7 +617,7 @@ uint64_t bdrv_dirty_bitmap_serialization_size(const Bdr= vDirtyBitmap *bitmap, =20 uint64_t bdrv_dirty_bitmap_serialization_align(const BdrvDirtyBitmap *bitm= ap) { - return hbitmap_serialization_granularity(bitmap->bitmap); + return hbitmap_serialization_align(bitmap->bitmap); } =20 void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap, diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c index 1acb353889..af41642346 100644 --- a/tests/test-hbitmap.c +++ b/tests/test-hbitmap.c @@ -738,15 +738,15 @@ static void test_hbitmap_meta_one(TestHBitmapData *da= ta, const void *unused) } } =20 -static void test_hbitmap_serialize_granularity(TestHBitmapData *data, - const void *unused) +static void test_hbitmap_serialize_align(TestHBitmapData *data, + const void *unused) { int r; =20 hbitmap_test_init(data, L3 * 2, 3); g_assert(hbitmap_is_serializable(data->hb)); =20 - r =3D hbitmap_serialization_granularity(data->hb); + r =3D hbitmap_serialization_align(data->hb); g_assert_cmpint(r, =3D=3D, 64 << 3); } =20 @@ -974,8 +974,8 @@ int main(int argc, char **argv) hbitmap_test_add("/hbitmap/meta/word", test_hbitmap_meta_word); hbitmap_test_add("/hbitmap/meta/sector", test_hbitmap_meta_sector); =20 - hbitmap_test_add("/hbitmap/serialize/granularity", - test_hbitmap_serialize_granularity); + hbitmap_test_add("/hbitmap/serialize/align", + test_hbitmap_serialize_align); hbitmap_test_add("/hbitmap/serialize/basic", test_hbitmap_serialize_basic); hbitmap_test_add("/hbitmap/serialize/part", diff --git a/util/hbitmap.c b/util/hbitmap.c index 21535cc90b..2f9d0fdbd0 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -413,14 +413,14 @@ bool hbitmap_is_serializable(const HBitmap *hb) { /* Every serialized chunk must be aligned to 64 bits so that endianness * requirements can be fulfilled on both 64 bit and 32 bit hosts. - * We have hbitmap_serialization_granularity() which converts this + * We have hbitmap_serialization_align() which converts this * alignment requirement from bitmap bits to items covered (e.g. secto= rs). * That value is: * 64 << hb->granularity * Since this value must not exceed UINT64_MAX, hb->granularity must be * less than 58 (=3D=3D 64 - 6, where 6 is ld(64), i.e. 1 << 6 =3D=3D = 64). * - * In order for hbitmap_serialization_granularity() to always return a + * In order for hbitmap_serialization_align() to always return a * meaningful value, bitmaps that are to be serialized must have a * granularity of less than 58. */ =20 @@ -437,7 +437,7 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item) return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] & bit) != =3D 0; } =20 -uint64_t hbitmap_serialization_granularity(const HBitmap *hb) +uint64_t hbitmap_serialization_align(const HBitmap *hb) { assert(hbitmap_is_serializable(hb)); =20 @@ -454,7 +454,7 @@ static void serialization_chunk(const HBitmap *hb, unsigned long **first_el, uint64_t *el_cou= nt) { uint64_t last =3D start + count - 1; - uint64_t gran =3D hbitmap_serialization_granularity(hb); + uint64_t gran =3D hbitmap_serialization_align(hb); =20 assert((start & (gran - 1)) =3D=3D 0); assert((last >> hb->granularity) < hb->size); --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305416862452.42890057494685; Fri, 6 Oct 2017 08:56:56 -0700 (PDT) Received: from localhost ([::1]:45602 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UzP-0001kh-Un for importer@patchew.org; Fri, 06 Oct 2017 11:56:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55265) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UxQ-0008Tt-Da for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UxP-0002NZ-Ib for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51788) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxN-0002I4-1X; Fri, 06 Oct 2017 11:54:33 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F42157C82F; Fri, 6 Oct 2017 15:54:31 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 121AB67585; Fri, 6 Oct 2017 15:54:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com F42157C82F Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:32 +0200 Message-Id: <20171006155422.10135-5-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 06 Oct 2017 15:54:32 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 04/54] qcow2: Ensure bitmap serialization is aligned X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake When subdividing a bitmap serialization, the code in hbitmap.c enforces that start/count parameters are aligned (except that count can end early at end-of-bitmap). We exposed this required alignment through bdrv_dirty_bitmap_serialization_align(), but forgot to actually check that we comply with it. Fortunately, qcow2 is never dividing bitmap serialization smaller than one cluster (which is a minimum of 512 bytes); so we are always compliant with the serialization alignment (which insists that we partition at least 64 bits per chunk) because we are doing at least 4k bits per chunk. Still, it's safer to add an assertion (for the unlikely case that we'd ever support a cluster smaller than 512 bytes, or if the hbitmap implementation changes what it considers to be aligned), rather than leaving bdrv_dirty_bitmap_serialization_align() without a caller. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/qcow2-bitmap.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 14f41d0427..1fc375b40a 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -274,10 +274,13 @@ static int free_bitmap_clusters(BlockDriverState *bs,= Qcow2BitmapTable *tb) static uint64_t sectors_covered_by_bitmap_cluster(const BDRVQcow2State *s, const BdrvDirtyBitmap *b= itmap) { - uint32_t sector_granularity =3D + uint64_t sector_granularity =3D bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS; + uint64_t sbc =3D sector_granularity * (s->cluster_size << 3); =20 - return (uint64_t)sector_granularity * (s->cluster_size << 3); + assert(QEMU_IS_ALIGNED(sbc, + bdrv_dirty_bitmap_serialization_align(bitmap))); + return sbc; } =20 /* load_bitmap_data --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305421314685.4886425943009; Fri, 6 Oct 2017 08:57:01 -0700 (PDT) Received: from localhost ([::1]:45603 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UzU-0001pm-Ev for importer@patchew.org; Fri, 06 Oct 2017 11:56:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55360) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UxS-000052-Ok for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UxR-0002Rn-MK for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:38 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51846) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxO-0002Jv-7t; Fri, 06 Oct 2017 11:54:34 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3F16EC0587F5; Fri, 6 Oct 2017 15:54:33 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 521D8675A1; Fri, 6 Oct 2017 15:54:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 3F16EC0587F5 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:33 +0200 Message-Id: <20171006155422.10135-6-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 06 Oct 2017 15:54:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 05/54] dirty-bitmap: Drop unused functions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We had several functions that no one is currently using, and which use sector-based interfaces. I'm trying to convert towards byte-based interfaces, so it's easier to just drop the unused functions: bdrv_dirty_bitmap_get_meta bdrv_dirty_bitmap_get_meta_locked bdrv_dirty_bitmap_reset_meta bdrv_dirty_bitmap_meta_granularity Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- include/block/dirty-bitmap.h | 10 ---------- block/dirty-bitmap.c | 44 ----------------------------------------= ---- 2 files changed, 54 deletions(-) diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index a79a58d2c3..8fd842eac9 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -34,7 +34,6 @@ void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap); BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs); uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs); uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirtyBitmap *bitmap); -uint32_t bdrv_dirty_bitmap_meta_granularity(BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap); bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap); const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap); @@ -44,15 +43,6 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap, int64_t cur_sector, int64_t nr_sectors); void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap, int64_t cur_sector, int64_t nr_sectors); -int bdrv_dirty_bitmap_get_meta(BlockDriverState *bs, - BdrvDirtyBitmap *bitmap, int64_t sector, - int nb_sectors); -int bdrv_dirty_bitmap_get_meta_locked(BlockDriverState *bs, - BdrvDirtyBitmap *bitmap, int64_t sec= tor, - int nb_sectors); -void bdrv_dirty_bitmap_reset_meta(BlockDriverState *bs, - BdrvDirtyBitmap *bitmap, int64_t sector, - int nb_sectors); BdrvDirtyBitmapIter *bdrv_dirty_meta_iter_new(BdrvDirtyBitmap *bitmap); BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap, uint64_t first_sector); diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 0490ca3aff..42a55e4a4b 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -173,45 +173,6 @@ void bdrv_release_meta_dirty_bitmap(BdrvDirtyBitmap *b= itmap) qemu_mutex_unlock(bitmap->mutex); } =20 -int bdrv_dirty_bitmap_get_meta_locked(BlockDriverState *bs, - BdrvDirtyBitmap *bitmap, int64_t sec= tor, - int nb_sectors) -{ - uint64_t i; - int sectors_per_bit =3D 1 << hbitmap_granularity(bitmap->meta); - - /* To optimize: we can make hbitmap to internally check the range in a - * coarse level, or at least do it word by word. */ - for (i =3D sector; i < sector + nb_sectors; i +=3D sectors_per_bit) { - if (hbitmap_get(bitmap->meta, i)) { - return true; - } - } - return false; -} - -int bdrv_dirty_bitmap_get_meta(BlockDriverState *bs, - BdrvDirtyBitmap *bitmap, int64_t sector, - int nb_sectors) -{ - bool dirty; - - qemu_mutex_lock(bitmap->mutex); - dirty =3D bdrv_dirty_bitmap_get_meta_locked(bs, bitmap, sector, nb_sec= tors); - qemu_mutex_unlock(bitmap->mutex); - - return dirty; -} - -void bdrv_dirty_bitmap_reset_meta(BlockDriverState *bs, - BdrvDirtyBitmap *bitmap, int64_t sector, - int nb_sectors) -{ - qemu_mutex_lock(bitmap->mutex); - hbitmap_reset(bitmap->meta, sector, nb_sectors); - qemu_mutex_unlock(bitmap->mutex); -} - int64_t bdrv_dirty_bitmap_size(const BdrvDirtyBitmap *bitmap) { return bitmap->size; @@ -511,11 +472,6 @@ uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirty= Bitmap *bitmap) return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap); } =20 -uint32_t bdrv_dirty_bitmap_meta_granularity(BdrvDirtyBitmap *bitmap) -{ - return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->meta); -} - BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap, uint64_t first_sector) { --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305780794825.0311410009119; Fri, 6 Oct 2017 09:03:00 -0700 (PDT) Received: from localhost ([::1]:45633 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V5U-0007GW-Ew for importer@patchew.org; Fri, 06 Oct 2017 12:02:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55386) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UxT-00006l-V3 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UxS-0002U1-TH for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51898) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxP-0002Mm-NS; Fri, 06 Oct 2017 11:54:35 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AEFE3C0587EB; Fri, 6 Oct 2017 15:54:34 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 91478675AA; Fri, 6 Oct 2017 15:54:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com AEFE3C0587EB Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:34 +0200 Message-Id: <20171006155422.10135-7-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 06 Oct 2017 15:54:34 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 06/54] dirty-bitmap: Avoid size query failure during truncate X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We've previously fixed several places where we failed to account for possible errors from bdrv_nb_sectors(). Fix another one by making bdrv_dirty_bitmap_truncate() take the new size from the caller instead of querying itself; then adjust the sole caller bdrv_truncate() to pass the size just determined by a successful resize, or to reuse the size given to the original truncate operation when refresh_total_sectors() was not able to confirm the actual size (the two sizes can potentially differ according to rounding constraints), thus avoiding sizing the bitmaps to -1. This also fixes a bug where not all failure paths in bdrv_truncate() would set errp. Note that bdrv_truncate() is still a bit awkward. We may want to revisit it later and clean up things to better guarantee that a resize attempt either fails cleanly up front, or cannot fail after guest-visible changes have been made (if temporary changes are made, then they need to be cleanly rolled back). But that is a task for another day; for now, the goal is the bare minimum fix to ensure that just bdrv_dirty_bitmap_truncate() cannot fail. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- include/block/dirty-bitmap.h | 2 +- block.c | 16 +++++++++++----- block/dirty-bitmap.c | 6 +++--- 3 files changed, 15 insertions(+), 9 deletions(-) diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 8fd842eac9..7a27590047 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -83,7 +83,7 @@ int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter); void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t sector_num); int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap); int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap); -void bdrv_dirty_bitmap_truncate(BlockDriverState *bs); +void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, int64_t bytes); bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap); bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap); diff --git a/block.c b/block.c index 528cda7b2c..ef5af81f66 100644 --- a/block.c +++ b/block.c @@ -3545,12 +3545,18 @@ int bdrv_truncate(BdrvChild *child, int64_t offset,= PreallocMode prealloc, assert(!(bs->open_flags & BDRV_O_INACTIVE)); =20 ret =3D drv->bdrv_truncate(bs, offset, prealloc, errp); - if (ret =3D=3D 0) { - ret =3D refresh_total_sectors(bs, offset >> BDRV_SECTOR_BITS); - bdrv_dirty_bitmap_truncate(bs); - bdrv_parent_cb_resize(bs); - atomic_inc(&bs->write_gen); + if (ret < 0) { + return ret; + } + ret =3D refresh_total_sectors(bs, offset >> BDRV_SECTOR_BITS); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not refresh total sector count= "); + } else { + offset =3D bs->total_sectors * BDRV_SECTOR_SIZE; } + bdrv_dirty_bitmap_truncate(bs, offset); + bdrv_parent_cb_resize(bs); + atomic_inc(&bs->write_gen); return ret; } =20 diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 42a55e4a4b..ee164fb518 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -1,7 +1,7 @@ /* * Block Dirty Bitmap * - * Copyright (c) 2016 Red Hat. Inc + * Copyright (c) 2016-2017 Red Hat. Inc * * Permission is hereby granted, free of charge, to any person obtaining a= copy * of this software and associated documentation files (the "Software"), t= o deal @@ -302,10 +302,10 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDrive= rState *bs, * Truncates _all_ bitmaps attached to a BDS. * Called with BQL taken. */ -void bdrv_dirty_bitmap_truncate(BlockDriverState *bs) +void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, int64_t bytes) { BdrvDirtyBitmap *bitmap; - uint64_t size =3D bdrv_nb_sectors(bs); + int64_t size =3D DIV_ROUND_UP(bytes, BDRV_SECTOR_SIZE); =20 bdrv_dirty_bitmaps_lock(bs); QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) { --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305589715618.6261007091339; Fri, 6 Oct 2017 08:59:49 -0700 (PDT) Received: from localhost ([::1]:45614 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V2H-0004FC-Uk for importer@patchew.org; Fri, 06 Oct 2017 11:59:38 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55417) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UxY-0000Cb-SZ for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UxU-0002Vw-3i for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56648) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxR-0002PC-0E; Fri, 06 Oct 2017 11:54:37 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EF22E1E2F8; Fri, 6 Oct 2017 15:54:35 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0D9E267598; Fri, 6 Oct 2017 15:54:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com EF22E1E2F8 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:35 +0200 Message-Id: <20171006155422.10135-8-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 06 Oct 2017 15:54:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 07/54] dirty-bitmap: Change bdrv_dirty_bitmap_size() to report bytes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We're already reporting bytes for bdrv_dirty_bitmap_granularity(); mixing bytes and sectors in our return values is a recipe for confusion. A later cleanup will convert dirty bitmap internals to be entirely byte-based, but in the meantime, we should report the bitmap size in bytes. The only external caller in qcow2-bitmap.c is temporarily more verbose (because it is still using sector-based math), but will later be switched to track progress by bytes instead of sectors. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/dirty-bitmap.c | 2 +- block/qcow2-bitmap.c | 14 ++++++++------ 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index ee164fb518..755555af32 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -175,7 +175,7 @@ void bdrv_release_meta_dirty_bitmap(BdrvDirtyBitmap *bi= tmap) =20 int64_t bdrv_dirty_bitmap_size(const BdrvDirtyBitmap *bitmap) { - return bitmap->size; + return bitmap->size * BDRV_SECTOR_SIZE; } =20 const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 1fc375b40a..9c185b5202 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -295,10 +295,11 @@ static int load_bitmap_data(BlockDriverState *bs, BDRVQcow2State *s =3D bs->opaque; uint64_t sector, sbc; uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); + uint64_t bm_sectors =3D DIV_ROUND_UP(bm_size, BDRV_SECTOR_SIZE); uint8_t *buf =3D NULL; uint64_t i, tab_size =3D size_to_clusters(s, - bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size)); + bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_sectors= )); =20 if (tab_size !=3D bitmap_table_size || tab_size > BME_MAX_TABLE_SIZE) { return -EINVAL; @@ -307,7 +308,7 @@ static int load_bitmap_data(BlockDriverState *bs, buf =3D g_malloc(s->cluster_size); sbc =3D sectors_covered_by_bitmap_cluster(s, bitmap); for (i =3D 0, sector =3D 0; i < tab_size; ++i, sector +=3D sbc) { - uint64_t count =3D MIN(bm_size - sector, sbc); + uint64_t count =3D MIN(bm_sectors - sector, sbc); uint64_t entry =3D bitmap_table[i]; uint64_t offset =3D entry & BME_TABLE_ENTRY_OFFSET_MASK; =20 @@ -1077,13 +1078,14 @@ static uint64_t *store_bitmap_data(BlockDriverState= *bs, int64_t sector; uint64_t sbc; uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); + uint64_t bm_sectors =3D DIV_ROUND_UP(bm_size, BDRV_SECTOR_SIZE); const char *bm_name =3D bdrv_dirty_bitmap_name(bitmap); uint8_t *buf =3D NULL; BdrvDirtyBitmapIter *dbi; uint64_t *tb; uint64_t tb_size =3D size_to_clusters(s, - bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size)); + bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_sectors= )); =20 if (tb_size > BME_MAX_TABLE_SIZE || tb_size * s->cluster_size > BME_MAX_PHYS_SIZE) @@ -1101,7 +1103,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, dbi =3D bdrv_dirty_iter_new(bitmap, 0); buf =3D g_malloc(s->cluster_size); sbc =3D sectors_covered_by_bitmap_cluster(s, bitmap); - assert(DIV_ROUND_UP(bm_size, sbc) =3D=3D tb_size); + assert(DIV_ROUND_UP(bm_sectors, sbc) =3D=3D tb_size); =20 while ((sector =3D bdrv_dirty_iter_next(dbi)) !=3D -1) { uint64_t cluster =3D sector / sbc; @@ -1109,7 +1111,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, int64_t off; =20 sector =3D cluster * sbc; - end =3D MIN(bm_size, sector + sbc); + end =3D MIN(bm_sectors, sector + sbc); write_size =3D bdrv_dirty_bitmap_serialization_size(bitmap, sector, end - sec= tor); assert(write_size <=3D s->cluster_size); @@ -1141,7 +1143,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, goto fail; } =20 - if (end >=3D bm_size) { + if (end >=3D bm_sectors) { break; } =20 --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305912061970.499901438016; Fri, 6 Oct 2017 09:05:12 -0700 (PDT) Received: from localhost ([::1]:45643 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V7U-0000W2-40 for importer@patchew.org; Fri, 06 Oct 2017 12:05:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55449) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxa-0000E5-4o for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UxZ-0002dL-4i for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:46 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44928) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxS-0002SI-G8; Fri, 06 Oct 2017 11:54:38 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6A4A4550B0; Fri, 6 Oct 2017 15:54:37 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4D4B267585; Fri, 6 Oct 2017 15:54:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6A4A4550B0 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:36 +0200 Message-Id: <20171006155422.10135-9-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 06 Oct 2017 15:54:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 08/54] dirty-bitmap: Track bitmap size by bytes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We are still using an internal hbitmap that tracks a size in sectors, with the granularity scaled down accordingly, because it lets us use a shortcut for our iterators which are currently sector-based. But there's no reason we can't track the dirty bitmap size in bytes, since it is (mostly) an internal-only variable (remember, the size is how many bytes are covered by the bitmap, not how many bytes the bitmap occupies). A later cleanup will convert dirty bitmap internals to be entirely byte-based, eliminating the intermediate sector rounding added here; and technically, since bdrv_getlength() already rounds up to sectors, our use of DIV_ROUND_UP is more for theoretical completeness than for any actual rounding. Use is_power_of_2() while at it, instead of open-coding that. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/dirty-bitmap.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 755555af32..6d32554b4e 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -42,7 +42,7 @@ struct BdrvDirtyBitmap { HBitmap *meta; /* Meta dirty bitmap */ BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status = */ char *name; /* Optional non-empty unique ID */ - int64_t size; /* Size of the bitmap (Number of sectors) = */ + int64_t size; /* Size of the bitmap, in bytes */ bool disabled; /* Bitmap is disabled. It ignores all writ= es to the device */ int active_iterators; /* How many iterators are active */ @@ -115,17 +115,14 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriver= State *bs, { int64_t bitmap_size; BdrvDirtyBitmap *bitmap; - uint32_t sector_granularity; =20 - assert((granularity & (granularity - 1)) =3D=3D 0); + assert(is_power_of_2(granularity) && granularity >=3D BDRV_SECTOR_SIZE= ); =20 if (name && bdrv_find_dirty_bitmap(bs, name)) { error_setg(errp, "Bitmap already exists: %s", name); return NULL; } - sector_granularity =3D granularity >> BDRV_SECTOR_BITS; - assert(sector_granularity); - bitmap_size =3D bdrv_nb_sectors(bs); + bitmap_size =3D bdrv_getlength(bs); if (bitmap_size < 0) { error_setg_errno(errp, -bitmap_size, "could not get length of devi= ce"); errno =3D -bitmap_size; @@ -133,7 +130,12 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverS= tate *bs, } bitmap =3D g_new0(BdrvDirtyBitmap, 1); bitmap->mutex =3D &bs->dirty_bitmap_mutex; - bitmap->bitmap =3D hbitmap_alloc(bitmap_size, ctz32(sector_granularity= )); + /* + * TODO - let hbitmap track full granularity. For now, it is tracking + * only sector granularity, as a shortcut for our iterators. + */ + bitmap->bitmap =3D hbitmap_alloc(DIV_ROUND_UP(bitmap_size, BDRV_SECTOR= _SIZE), + ctz32(granularity) - BDRV_SECTOR_BITS); bitmap->size =3D bitmap_size; bitmap->name =3D g_strdup(name); bitmap->disabled =3D false; @@ -175,7 +177,7 @@ void bdrv_release_meta_dirty_bitmap(BdrvDirtyBitmap *bi= tmap) =20 int64_t bdrv_dirty_bitmap_size(const BdrvDirtyBitmap *bitmap) { - return bitmap->size * BDRV_SECTOR_SIZE; + return bitmap->size; } =20 const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap) @@ -305,14 +307,13 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDrive= rState *bs, void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, int64_t bytes) { BdrvDirtyBitmap *bitmap; - int64_t size =3D DIV_ROUND_UP(bytes, BDRV_SECTOR_SIZE); =20 bdrv_dirty_bitmaps_lock(bs); QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) { assert(!bdrv_dirty_bitmap_frozen(bitmap)); assert(!bitmap->active_iterators); - hbitmap_truncate(bitmap->bitmap, size); - bitmap->size =3D size; + hbitmap_truncate(bitmap->bitmap, DIV_ROUND_UP(bytes, BDRV_SECTOR_S= IZE)); + bitmap->size =3D bytes; } bdrv_dirty_bitmaps_unlock(bs); } @@ -549,7 +550,8 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, H= Bitmap **out) hbitmap_reset_all(bitmap->bitmap); } else { HBitmap *backup =3D bitmap->bitmap; - bitmap->bitmap =3D hbitmap_alloc(bitmap->size, + bitmap->bitmap =3D hbitmap_alloc(DIV_ROUND_UP(bitmap->size, + BDRV_SECTOR_SIZE), hbitmap_granularity(backup)); *out =3D backup; } --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150730608154770.63613873015424; Fri, 6 Oct 2017 09:08:01 -0700 (PDT) Received: from localhost ([::1]:45663 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VA8-0002qH-CF for importer@patchew.org; Fri, 06 Oct 2017 12:07:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55481) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxc-0000F8-32 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxa-0002fq-L2 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:48 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49586) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxV-0002Wh-Gl; Fri, 06 Oct 2017 11:54:41 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6C3E385547; Fri, 6 Oct 2017 15:54:40 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id B0EAD67588; Fri, 6 Oct 2017 15:54:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6C3E385547 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:37 +0200 Message-Id: <20171006155422.10135-10-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 06 Oct 2017 15:54:40 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 09/54] dirty-bitmap: Change bdrv_dirty_bitmap_*serialize*() to take bytes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Right now, the dirty-bitmap code exposes the fact that we use a scale of sector granularity in the underlying hbitmap to anything that wants to serialize a dirty bitmap. It's nicer to uniformly expose bytes as our dirty-bitmap interface, matching the previous change to bitmap size. The only caller to serialization is currently qcow2-cluster.c, which becomes a bit more verbose because it is still tracking sectors for other reasons, but a later patch will fix that to more uniformly use byte offsets everywhere. Likewise, within dirty-bitmap, we have to add more assertions that we are not truncating incorrectly, which can go away once the internal hbitmap is byte-based rather than sector-based. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- include/block/dirty-bitmap.h | 14 +++++++------- block/dirty-bitmap.c | 37 ++++++++++++++++++++++++------------- block/qcow2-bitmap.c | 22 ++++++++++++++-------- 3 files changed, 45 insertions(+), 28 deletions(-) diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 7a27590047..5f34a1a3c7 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -49,19 +49,19 @@ BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitma= p *bitmap, void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter); =20 uint64_t bdrv_dirty_bitmap_serialization_size(const BdrvDirtyBitmap *bitma= p, - uint64_t start, uint64_t cou= nt); + uint64_t offset, uint64_t by= tes); uint64_t bdrv_dirty_bitmap_serialization_align(const BdrvDirtyBitmap *bitm= ap); void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap, - uint8_t *buf, uint64_t start, - uint64_t count); + uint8_t *buf, uint64_t offset, + uint64_t bytes); void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap, - uint8_t *buf, uint64_t start, - uint64_t count, bool finish); + uint8_t *buf, uint64_t offset, + uint64_t bytes, bool finish); void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap, - uint64_t start, uint64_t count, + uint64_t offset, uint64_t bytes, bool finish); void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap, - uint64_t start, uint64_t count, + uint64_t offset, uint64_t bytes, bool finish); void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap); =20 diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 6d32554b4e..555c736024 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -568,42 +568,53 @@ void bdrv_undo_clear_dirty_bitmap(BdrvDirtyBitmap *bi= tmap, HBitmap *in) } =20 uint64_t bdrv_dirty_bitmap_serialization_size(const BdrvDirtyBitmap *bitma= p, - uint64_t start, uint64_t cou= nt) + uint64_t offset, uint64_t by= tes) { - return hbitmap_serialization_size(bitmap->bitmap, start, count); + assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); + return hbitmap_serialization_size(bitmap->bitmap, + offset >> BDRV_SECTOR_BITS, + bytes >> BDRV_SECTOR_BITS); } =20 uint64_t bdrv_dirty_bitmap_serialization_align(const BdrvDirtyBitmap *bitm= ap) { - return hbitmap_serialization_align(bitmap->bitmap); + return hbitmap_serialization_align(bitmap->bitmap) * BDRV_SECTOR_SIZE; } =20 void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap, - uint8_t *buf, uint64_t start, - uint64_t count) + uint8_t *buf, uint64_t offset, + uint64_t bytes) { - hbitmap_serialize_part(bitmap->bitmap, buf, start, count); + assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); + hbitmap_serialize_part(bitmap->bitmap, buf, offset >> BDRV_SECTOR_BITS, + bytes >> BDRV_SECTOR_BITS); } =20 void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap, - uint8_t *buf, uint64_t start, - uint64_t count, bool finish) + uint8_t *buf, uint64_t offset, + uint64_t bytes, bool finish) { - hbitmap_deserialize_part(bitmap->bitmap, buf, start, count, finish); + assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); + hbitmap_deserialize_part(bitmap->bitmap, buf, offset >> BDRV_SECTOR_BI= TS, + bytes >> BDRV_SECTOR_BITS, finish); } =20 void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap, - uint64_t start, uint64_t count, + uint64_t offset, uint64_t bytes, bool finish) { - hbitmap_deserialize_zeroes(bitmap->bitmap, start, count, finish); + assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); + hbitmap_deserialize_zeroes(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, + bytes >> BDRV_SECTOR_BITS, finish); } =20 void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap, - uint64_t start, uint64_t count, + uint64_t offset, uint64_t bytes, bool finish) { - hbitmap_deserialize_ones(bitmap->bitmap, start, count, finish); + assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); + hbitmap_deserialize_ones(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, + bytes >> BDRV_SECTOR_BITS, finish); } =20 void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 9c185b5202..9e799996e6 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -278,7 +278,7 @@ static uint64_t sectors_covered_by_bitmap_cluster(const= BDRVQcow2State *s, bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS; uint64_t sbc =3D sector_granularity * (s->cluster_size << 3); =20 - assert(QEMU_IS_ALIGNED(sbc, + assert(QEMU_IS_ALIGNED(sbc << BDRV_SECTOR_BITS, bdrv_dirty_bitmap_serialization_align(bitmap))); return sbc; } @@ -299,7 +299,7 @@ static int load_bitmap_data(BlockDriverState *bs, uint8_t *buf =3D NULL; uint64_t i, tab_size =3D size_to_clusters(s, - bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_sectors= )); + bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size)); =20 if (tab_size !=3D bitmap_table_size || tab_size > BME_MAX_TABLE_SIZE) { return -EINVAL; @@ -316,7 +316,9 @@ static int load_bitmap_data(BlockDriverState *bs, =20 if (offset =3D=3D 0) { if (entry & BME_TABLE_ENTRY_FLAG_ALL_ONES) { - bdrv_dirty_bitmap_deserialize_ones(bitmap, sector, count, + bdrv_dirty_bitmap_deserialize_ones(bitmap, + sector * BDRV_SECTOR_SI= ZE, + count * BDRV_SECTOR_SIZ= E, false); } else { /* No need to deserialize zeros because the dirty bitmap is @@ -327,7 +329,9 @@ static int load_bitmap_data(BlockDriverState *bs, if (ret < 0) { goto finish; } - bdrv_dirty_bitmap_deserialize_part(bitmap, buf, sector, count, + bdrv_dirty_bitmap_deserialize_part(bitmap, buf, + sector * BDRV_SECTOR_SIZE, + count * BDRV_SECTOR_SIZE, false); } } @@ -1085,7 +1089,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, uint64_t *tb; uint64_t tb_size =3D size_to_clusters(s, - bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_sectors= )); + bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size)); =20 if (tb_size > BME_MAX_TABLE_SIZE || tb_size * s->cluster_size > BME_MAX_PHYS_SIZE) @@ -1112,8 +1116,8 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, =20 sector =3D cluster * sbc; end =3D MIN(bm_sectors, sector + sbc); - write_size =3D - bdrv_dirty_bitmap_serialization_size(bitmap, sector, end - sec= tor); + write_size =3D bdrv_dirty_bitmap_serialization_size(bitmap, + sector * BDRV_SECTOR_SIZE, (end - sector) * BDRV_SECTOR_SIZE); assert(write_size <=3D s->cluster_size); =20 off =3D qcow2_alloc_clusters(bs, s->cluster_size); @@ -1125,7 +1129,9 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, } tb[cluster] =3D off; =20 - bdrv_dirty_bitmap_serialize_part(bitmap, buf, sector, end - sector= ); + bdrv_dirty_bitmap_serialize_part(bitmap, buf, + sector * BDRV_SECTOR_SIZE, + (end - sector) * BDRV_SECTOR_SIZE= ); if (write_size < s->cluster_size) { memset(buf + write_size, 0, s->cluster_size - write_size); } --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305952053700.0297704707998; Fri, 6 Oct 2017 09:05:52 -0700 (PDT) Received: from localhost ([::1]:45653 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V8F-0001GJ-Gf for importer@patchew.org; Fri, 06 Oct 2017 12:05:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55466) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxb-0000F4-5A for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxa-0002ex-7O for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56922) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxX-0002Ze-My; Fri, 06 Oct 2017 11:54:43 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AD046356F0; Fri, 6 Oct 2017 15:54:42 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id B69B267588; Fri, 6 Oct 2017 15:54:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com AD046356F0 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:38 +0200 Message-Id: <20171006155422.10135-11-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 06 Oct 2017 15:54:42 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 10/54] qcow2: Switch sectors_covered_by_bitmap_cluster() to byte-based X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Change the qcow2 bitmap helper function sectors_covered_by_bitmap_cluster(), renaming it to bytes_covered_by_bitmap_cluster() in the process. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/qcow2-bitmap.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 9e799996e6..8324468b16 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -269,18 +269,16 @@ static int free_bitmap_clusters(BlockDriverState *bs,= Qcow2BitmapTable *tb) return 0; } =20 -/* This function returns the number of disk sectors covered by a single qc= ow2 - * cluster of bitmap data. */ -static uint64_t sectors_covered_by_bitmap_cluster(const BDRVQcow2State *s, - const BdrvDirtyBitmap *b= itmap) +/* Return the disk size covered by a single qcow2 cluster of bitmap data. = */ +static uint64_t bytes_covered_by_bitmap_cluster(const BDRVQcow2State *s, + const BdrvDirtyBitmap *bit= map) { - uint64_t sector_granularity =3D - bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS; - uint64_t sbc =3D sector_granularity * (s->cluster_size << 3); + uint64_t granularity =3D bdrv_dirty_bitmap_granularity(bitmap); + uint64_t limit =3D granularity * (s->cluster_size << 3); =20 - assert(QEMU_IS_ALIGNED(sbc << BDRV_SECTOR_BITS, + assert(QEMU_IS_ALIGNED(limit, bdrv_dirty_bitmap_serialization_align(bitmap))); - return sbc; + return limit; } =20 /* load_bitmap_data @@ -293,7 +291,7 @@ static int load_bitmap_data(BlockDriverState *bs, { int ret =3D 0; BDRVQcow2State *s =3D bs->opaque; - uint64_t sector, sbc; + uint64_t sector, limit, sbc; uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); uint64_t bm_sectors =3D DIV_ROUND_UP(bm_size, BDRV_SECTOR_SIZE); uint8_t *buf =3D NULL; @@ -306,7 +304,8 @@ static int load_bitmap_data(BlockDriverState *bs, } =20 buf =3D g_malloc(s->cluster_size); - sbc =3D sectors_covered_by_bitmap_cluster(s, bitmap); + limit =3D bytes_covered_by_bitmap_cluster(s, bitmap); + sbc =3D limit >> BDRV_SECTOR_BITS; for (i =3D 0, sector =3D 0; i < tab_size; ++i, sector +=3D sbc) { uint64_t count =3D MIN(bm_sectors - sector, sbc); uint64_t entry =3D bitmap_table[i]; @@ -1080,7 +1079,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, int ret; BDRVQcow2State *s =3D bs->opaque; int64_t sector; - uint64_t sbc; + uint64_t limit, sbc; uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); uint64_t bm_sectors =3D DIV_ROUND_UP(bm_size, BDRV_SECTOR_SIZE); const char *bm_name =3D bdrv_dirty_bitmap_name(bitmap); @@ -1106,8 +1105,9 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, =20 dbi =3D bdrv_dirty_iter_new(bitmap, 0); buf =3D g_malloc(s->cluster_size); - sbc =3D sectors_covered_by_bitmap_cluster(s, bitmap); - assert(DIV_ROUND_UP(bm_sectors, sbc) =3D=3D tb_size); + limit =3D bytes_covered_by_bitmap_cluster(s, bitmap); + sbc =3D limit >> BDRV_SECTOR_BITS; + assert(DIV_ROUND_UP(bm_size, limit) =3D=3D tb_size); =20 while ((sector =3D bdrv_dirty_iter_next(dbi)) !=3D -1) { uint64_t cluster =3D sector / sbc; --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306134663228.4496362932731; Fri, 6 Oct 2017 09:08:54 -0700 (PDT) Received: from localhost ([::1]:45667 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VB4-0003d1-NV for importer@patchew.org; Fri, 06 Oct 2017 12:08:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55523) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxe-0000H8-HB for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxd-0002kp-As for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:50 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39940) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UxZ-0002cq-Jy; Fri, 06 Oct 2017 11:54:45 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 857EE4E028; Fri, 6 Oct 2017 15:54:44 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B14267588; Fri, 6 Oct 2017 15:54:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 857EE4E028 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:39 +0200 Message-Id: <20171006155422.10135-12-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 06 Oct 2017 15:54:44 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 11/54] dirty-bitmap: Set iterator start by offset, not sector X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake All callers to bdrv_dirty_iter_new() passed 0 for their initial starting point, drop that parameter. Most callers to bdrv_set_dirty_iter() were scaling a byte offset to a sector number; the exception qcow2-bitmap will be converted later to use byte rather than sector iteration. Move the scaling to occur internally to dirty bitmap code instead, so that callers now pass in bytes. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- include/block/dirty-bitmap.h | 5 ++--- block/backup.c | 5 ++--- block/dirty-bitmap.c | 9 ++++----- block/mirror.c | 4 ++-- block/qcow2-bitmap.c | 4 ++-- 5 files changed, 12 insertions(+), 15 deletions(-) diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 5f34a1a3c7..757fc4c5b8 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -44,8 +44,7 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap, void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap, int64_t cur_sector, int64_t nr_sectors); BdrvDirtyBitmapIter *bdrv_dirty_meta_iter_new(BdrvDirtyBitmap *bitmap); -BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap, - uint64_t first_sector); +BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap); void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter); =20 uint64_t bdrv_dirty_bitmap_serialization_size(const BdrvDirtyBitmap *bitma= p, @@ -80,7 +79,7 @@ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t cur_sector, int64_t nr_sectors= ); int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter); -void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t sector_num); +void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t offset); int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap); int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, int64_t bytes); diff --git a/block/backup.c b/block/backup.c index 517c300a49..ac9c018717 100644 --- a/block/backup.c +++ b/block/backup.c @@ -372,7 +372,7 @@ static int coroutine_fn backup_run_incremental(BackupBl= ockJob *job) =20 granularity =3D bdrv_dirty_bitmap_granularity(job->sync_bitmap); clusters_per_iter =3D MAX((granularity / job->cluster_size), 1); - dbi =3D bdrv_dirty_iter_new(job->sync_bitmap, 0); + dbi =3D bdrv_dirty_iter_new(job->sync_bitmap); =20 /* Find the next dirty sector(s) */ while ((offset =3D bdrv_dirty_iter_next(dbi) * BDRV_SECTOR_SIZE) >=3D = 0) { @@ -403,8 +403,7 @@ static int coroutine_fn backup_run_incremental(BackupBl= ockJob *job) /* If the bitmap granularity is smaller than the backup granularit= y, * we need to advance the iterator pointer to the next cluster. */ if (granularity < job->cluster_size) { - bdrv_set_dirty_iter(dbi, - cluster * job->cluster_size / BDRV_SECTOR_= SIZE); + bdrv_set_dirty_iter(dbi, cluster * job->cluster_size); } =20 last_cluster =3D cluster - 1; diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 555c736024..84509476ba 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -473,11 +473,10 @@ uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirt= yBitmap *bitmap) return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap); } =20 -BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap, - uint64_t first_sector) +BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap) { BdrvDirtyBitmapIter *iter =3D g_new(BdrvDirtyBitmapIter, 1); - hbitmap_iter_init(&iter->hbi, bitmap->bitmap, first_sector); + hbitmap_iter_init(&iter->hbi, bitmap->bitmap, 0); iter->bitmap =3D bitmap; bitmap->active_iterators++; return iter; @@ -645,9 +644,9 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_s= ector, /** * Advance a BdrvDirtyBitmapIter to an arbitrary offset. */ -void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *iter, int64_t sector_num) +void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *iter, int64_t offset) { - hbitmap_iter_init(&iter->hbi, iter->hbi.hb, sector_num); + hbitmap_iter_init(&iter->hbi, iter->hbi.hb, offset >> BDRV_SECTOR_BITS= ); } =20 int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap) diff --git a/block/mirror.c b/block/mirror.c index 6f5cb9f26c..0c705e0b4f 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -373,7 +373,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlo= ckJob *s) next_dirty =3D bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; if (next_dirty > next_offset || next_dirty < 0) { /* The bitmap iterator's cache is stale, refresh it */ - bdrv_set_dirty_iter(s->dbi, next_offset >> BDRV_SECTOR_BITS); + bdrv_set_dirty_iter(s->dbi, next_offset); next_dirty =3D bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; } assert(next_dirty =3D=3D next_offset); @@ -796,7 +796,7 @@ static void coroutine_fn mirror_run(void *opaque) } =20 assert(!s->dbi); - s->dbi =3D bdrv_dirty_iter_new(s->dirty_bitmap, 0); + s->dbi =3D bdrv_dirty_iter_new(s->dirty_bitmap); for (;;) { uint64_t delay_ns =3D 0; int64_t cnt, delta; diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 8324468b16..90756cf561 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -1103,7 +1103,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, return NULL; } =20 - dbi =3D bdrv_dirty_iter_new(bitmap, 0); + dbi =3D bdrv_dirty_iter_new(bitmap); buf =3D g_malloc(s->cluster_size); limit =3D bytes_covered_by_bitmap_cluster(s, bitmap); sbc =3D limit >> BDRV_SECTOR_BITS; @@ -1153,7 +1153,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, break; } =20 - bdrv_set_dirty_iter(dbi, end); + bdrv_set_dirty_iter(dbi, end * BDRV_SECTOR_SIZE); } =20 *bitmap_table_size =3D tb_size; --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150730578461632.13315284526311; Fri, 6 Oct 2017 09:03:04 -0700 (PDT) Received: from localhost ([::1]:45632 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V5L-00079g-Id for importer@patchew.org; Fri, 06 Oct 2017 12:02:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55562) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxj-0000Ll-9Z for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxe-0002lp-BH for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53944) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxb-0002gI-FB; Fri, 06 Oct 2017 11:54:47 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 688A5883C6; Fri, 6 Oct 2017 15:54:46 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6C9267588; Fri, 6 Oct 2017 15:54:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 688A5883C6 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:40 +0200 Message-Id: <20171006155422.10135-13-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 06 Oct 2017 15:54:46 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 12/54] dirty-bitmap: Change bdrv_dirty_iter_next() to report byte offset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Thanks to recent cleanups, most callers were scaling a return value of sectors into bytes (the exception, in qcow2-bitmap, will be converted to byte-based iteration later). Update the interface to do the scaling internally instead. In qcow2-bitmap, the code was specifically checking for an error return of -1. To avoid a regression, we either have to make sure we continue to return -1 (rather than a scaled -512) on error, or we have to fix the caller to treat all negative values as error rather than just one magic value. It's easy enough to make both changes at the same time, even though either one in isolation would work. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/backup.c | 2 +- block/dirty-bitmap.c | 3 ++- block/mirror.c | 8 ++++---- block/qcow2-bitmap.c | 2 +- 4 files changed, 8 insertions(+), 7 deletions(-) diff --git a/block/backup.c b/block/backup.c index ac9c018717..06ddbfd03d 100644 --- a/block/backup.c +++ b/block/backup.c @@ -375,7 +375,7 @@ static int coroutine_fn backup_run_incremental(BackupBl= ockJob *job) dbi =3D bdrv_dirty_iter_new(job->sync_bitmap); =20 /* Find the next dirty sector(s) */ - while ((offset =3D bdrv_dirty_iter_next(dbi) * BDRV_SECTOR_SIZE) >=3D = 0) { + while ((offset =3D bdrv_dirty_iter_next(dbi)) >=3D 0) { cluster =3D offset / job->cluster_size; =20 /* Fake progress updates for any clusters we skipped */ diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 84509476ba..e451916187 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -503,7 +503,8 @@ void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter) =20 int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter) { - return hbitmap_iter_next(&iter->hbi); + int64_t ret =3D hbitmap_iter_next(&iter->hbi); + return ret < 0 ? -1 : ret * BDRV_SECTOR_SIZE; } =20 /* Called within bdrv_dirty_bitmap_lock..unlock */ diff --git a/block/mirror.c b/block/mirror.c index 0c705e0b4f..de0a02778c 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -336,10 +336,10 @@ static uint64_t coroutine_fn mirror_iteration(MirrorB= lockJob *s) int max_io_bytes =3D MAX(s->buf_size / MAX_IN_FLIGHT, MAX_IO_BYTES); =20 bdrv_dirty_bitmap_lock(s->dirty_bitmap); - offset =3D bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; + offset =3D bdrv_dirty_iter_next(s->dbi); if (offset < 0) { bdrv_set_dirty_iter(s->dbi, 0); - offset =3D bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; + offset =3D bdrv_dirty_iter_next(s->dbi); trace_mirror_restart_iter(s, bdrv_get_dirty_count(s->dirty_bitmap)= * BDRV_SECTOR_SIZE); assert(offset >=3D 0); @@ -370,11 +370,11 @@ static uint64_t coroutine_fn mirror_iteration(MirrorB= lockJob *s) break; } =20 - next_dirty =3D bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; + next_dirty =3D bdrv_dirty_iter_next(s->dbi); if (next_dirty > next_offset || next_dirty < 0) { /* The bitmap iterator's cache is stale, refresh it */ bdrv_set_dirty_iter(s->dbi, next_offset); - next_dirty =3D bdrv_dirty_iter_next(s->dbi) * BDRV_SECTOR_SIZE; + next_dirty =3D bdrv_dirty_iter_next(s->dbi); } assert(next_dirty =3D=3D next_offset); nb_chunks++; diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 90756cf561..2d8dcba3e8 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -1109,7 +1109,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, sbc =3D limit >> BDRV_SECTOR_BITS; assert(DIV_ROUND_UP(bm_size, limit) =3D=3D tb_size); =20 - while ((sector =3D bdrv_dirty_iter_next(dbi)) !=3D -1) { + while ((sector =3D bdrv_dirty_iter_next(dbi) >> BDRV_SECTOR_BITS) >=3D= 0) { uint64_t cluster =3D sector / sbc; uint64_t end, write_size; int64_t off; --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306231241880.773707233525; Fri, 6 Oct 2017 09:10:31 -0700 (PDT) Received: from localhost ([::1]:45676 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VCj-00050I-2y for importer@patchew.org; Fri, 06 Oct 2017 12:10:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55692) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxn-0000Oe-Fn for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxm-0002y3-1I for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40038) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxc-0002iU-Qn; Fri, 06 Oct 2017 11:54:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C35553A272; Fri, 6 Oct 2017 15:54:47 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9243675A1; Fri, 6 Oct 2017 15:54:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C35553A272 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:41 +0200 Message-Id: <20171006155422.10135-14-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 06 Oct 2017 15:54:47 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 13/54] dirty-bitmap: Change bdrv_get_dirty_count() to report bytes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Thanks to recent cleanups, all callers were scaling a return value of sectors into bytes; do the scaling internally instead. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/dirty-bitmap.c | 4 ++-- block/mirror.c | 16 ++++++---------- migration/block.c | 2 +- 3 files changed, 9 insertions(+), 13 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index e451916187..8322e23f0d 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -423,7 +423,7 @@ BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDrive= rState *bs) QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) { BlockDirtyInfo *info =3D g_new0(BlockDirtyInfo, 1); BlockDirtyInfoList *entry =3D g_new0(BlockDirtyInfoList, 1); - info->count =3D bdrv_get_dirty_count(bm) << BDRV_SECTOR_BITS; + info->count =3D bdrv_get_dirty_count(bm); info->granularity =3D bdrv_dirty_bitmap_granularity(bm); info->has_name =3D !!bm->name; info->name =3D g_strdup(bm->name); @@ -652,7 +652,7 @@ void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *iter, int= 64_t offset) =20 int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap) { - return hbitmap_count(bitmap->bitmap); + return hbitmap_count(bitmap->bitmap) << BDRV_SECTOR_BITS; } =20 int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap) diff --git a/block/mirror.c b/block/mirror.c index de0a02778c..c5212d2c8d 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -340,8 +340,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlo= ckJob *s) if (offset < 0) { bdrv_set_dirty_iter(s->dbi, 0); offset =3D bdrv_dirty_iter_next(s->dbi); - trace_mirror_restart_iter(s, bdrv_get_dirty_count(s->dirty_bitmap)= * - BDRV_SECTOR_SIZE); + trace_mirror_restart_iter(s, bdrv_get_dirty_count(s->dirty_bitmap)= ); assert(offset >=3D 0); } bdrv_dirty_bitmap_unlock(s->dirty_bitmap); @@ -811,11 +810,10 @@ static void coroutine_fn mirror_run(void *opaque) =20 cnt =3D bdrv_get_dirty_count(s->dirty_bitmap); /* s->common.offset contains the number of bytes already processed= so - * far, cnt is the number of dirty sectors remaining and + * far, cnt is the number of dirty bytes remaining and * s->bytes_in_flight is the number of bytes currently being * processed; together those are the current total operation lengt= h */ - s->common.len =3D s->common.offset + s->bytes_in_flight + - cnt * BDRV_SECTOR_SIZE; + s->common.len =3D s->common.offset + s->bytes_in_flight + cnt; =20 /* Note that even when no rate limit is applied we need to yield * periodically with no pending I/O so that bdrv_drain_all() retur= ns. @@ -827,8 +825,7 @@ static void coroutine_fn mirror_run(void *opaque) s->common.iostatus =3D=3D BLOCK_DEVICE_IO_STATUS_OK) { if (s->in_flight >=3D MAX_IN_FLIGHT || s->buf_free_count =3D= =3D 0 || (cnt =3D=3D 0 && s->in_flight > 0)) { - trace_mirror_yield(s, cnt * BDRV_SECTOR_SIZE, - s->buf_free_count, s->in_flight); + trace_mirror_yield(s, cnt, s->buf_free_count, s->in_flight= ); mirror_wait_for_io(s); continue; } else if (cnt !=3D 0) { @@ -869,7 +866,7 @@ static void coroutine_fn mirror_run(void *opaque) * whether to switch to target check one last time if I/O has * come in the meanwhile, and if not flush the data to disk. */ - trace_mirror_before_drain(s, cnt * BDRV_SECTOR_SIZE); + trace_mirror_before_drain(s, cnt); =20 bdrv_drained_begin(bs); cnt =3D bdrv_get_dirty_count(s->dirty_bitmap); @@ -888,8 +885,7 @@ static void coroutine_fn mirror_run(void *opaque) } =20 ret =3D 0; - trace_mirror_before_sleep(s, cnt * BDRV_SECTOR_SIZE, - s->synced, delay_ns); + trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); if (!s->synced) { block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, delay_ns); if (block_job_is_cancelled(&s->common)) { diff --git a/migration/block.c b/migration/block.c index 606ad4db92..cbcadce168 100644 --- a/migration/block.c +++ b/migration/block.c @@ -672,7 +672,7 @@ static int64_t get_remaining_dirty(void) aio_context_release(blk_get_aio_context(bmds->blk)); } =20 - return dirty << BDRV_SECTOR_BITS; + return dirty; } =20 =20 --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306607071986.8582244588285; Fri, 6 Oct 2017 09:16:47 -0700 (PDT) Received: from localhost ([::1]:45708 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VIn-0002Gf-4e for importer@patchew.org; Fri, 06 Oct 2017 12:16:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55701) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxn-0000P7-Ic for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxm-0002yS-5N for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49922) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxe-0002l6-6K; Fri, 06 Oct 2017 11:54:50 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 205D47EAB7; Fri, 6 Oct 2017 15:54:49 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2102867598; Fri, 6 Oct 2017 15:54:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 205D47EAB7 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:42 +0200 Message-Id: <20171006155422.10135-15-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 06 Oct 2017 15:54:49 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 14/54] dirty-bitmap: Change bdrv_get_dirty_locked() to take bytes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Half the callers were already scaling bytes to sectors; the other half can eventually be simplified to use byte iteration. Both callers were already using the result as a bool, so make that explicit. Making the change also makes it easier for a future dirty-bitmap patch to offload scaling over to the internal hbitmap. Remember, asking whether a byte is dirty is effectively asking whether the entire granularity containing the byte is dirty, since we only track dirtiness by granularity. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Juan Quintela Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- include/block/dirty-bitmap.h | 4 ++-- block/dirty-bitmap.c | 8 ++++---- block/mirror.c | 3 +-- migration/block.c | 3 ++- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 757fc4c5b8..9e39537e4b 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -72,8 +72,8 @@ void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *b= itmap, /* Functions that require manual locking. */ void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); void bdrv_dirty_bitmap_unlock(BdrvDirtyBitmap *bitmap); -int bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, - int64_t sector); +bool bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, + int64_t offset); void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t cur_sector, int64_t nr_sectors); void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 8322e23f0d..ad559c62b1 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -438,13 +438,13 @@ BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDri= verState *bs) } =20 /* Called within bdrv_dirty_bitmap_lock..unlock */ -int bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, - int64_t sector) +bool bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, + int64_t offset) { if (bitmap) { - return hbitmap_get(bitmap->bitmap, sector); + return hbitmap_get(bitmap->bitmap, offset >> BDRV_SECTOR_BITS); } else { - return 0; + return false; } } =20 diff --git a/block/mirror.c b/block/mirror.c index c5212d2c8d..545dfc50aa 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -361,8 +361,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlo= ckJob *s) int64_t next_offset =3D offset + nb_chunks * s->granularity; int64_t next_chunk =3D next_offset / s->granularity; if (next_offset >=3D s->bdev_length || - !bdrv_get_dirty_locked(source, s->dirty_bitmap, - next_offset >> BDRV_SECTOR_BITS)) { + !bdrv_get_dirty_locked(source, s->dirty_bitmap, next_offset)) { break; } if (test_bit(next_chunk, s->in_flight_bitmap)) { diff --git a/migration/block.c b/migration/block.c index cbcadce168..f5bcf5505c 100644 --- a/migration/block.c +++ b/migration/block.c @@ -535,7 +535,8 @@ static int mig_save_device_dirty(QEMUFile *f, BlkMigDev= State *bmds, blk_mig_unlock(); } bdrv_dirty_bitmap_lock(bmds->dirty_bitmap); - if (bdrv_get_dirty_locked(bs, bmds->dirty_bitmap, sector)) { + if (bdrv_get_dirty_locked(bs, bmds->dirty_bitmap, + sector * BDRV_SECTOR_SIZE)) { if (total_sectors - sector < BDRV_SECTORS_PER_DIRTY_CHUNK) { nr_sectors =3D total_sectors - sector; } else { --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306394423972.4496482515244; Fri, 6 Oct 2017 09:13:14 -0700 (PDT) Received: from localhost ([::1]:45689 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VFB-0007PX-Er for importer@patchew.org; Fri, 06 Oct 2017 12:12:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55717) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxn-0000QJ-V0 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxm-0002ya-5q for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36190) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxf-0002mV-C7; Fri, 06 Oct 2017 11:54:51 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 57FB5C04B333; Fri, 6 Oct 2017 15:54:50 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B09D67598; Fri, 6 Oct 2017 15:54:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 57FB5C04B333 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:43 +0200 Message-Id: <20171006155422.10135-16-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:54:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 15/54] dirty-bitmap: Change bdrv_[re]set_dirty_bitmap() to use bytes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Some of the callers were already scaling bytes to sectors; others can be easily converted to pass byte offsets, all in our shift towards a consistent byte interface everywhere. Making the change will also make it easier to write the hold-out callers to use byte rather than sectors for their iterations; it also makes it easier for a future dirty-bitmap patch to offload scaling over to the internal hbitmap. Although all callers happen to pass sector-aligned values, make the internal scaling robust to any sub-sector requests. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- include/block/dirty-bitmap.h | 8 ++++---- block/dirty-bitmap.c | 22 ++++++++++++++-------- block/mirror.c | 16 ++++++++-------- migration/block.c | 7 +++++-- 4 files changed, 31 insertions(+), 22 deletions(-) diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h index 9e39537e4b..3579a7597c 100644 --- a/include/block/dirty-bitmap.h +++ b/include/block/dirty-bitmap.h @@ -40,9 +40,9 @@ const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap = *bitmap); int64_t bdrv_dirty_bitmap_size(const BdrvDirtyBitmap *bitmap); DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap); void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap, - int64_t cur_sector, int64_t nr_sectors); + int64_t offset, int64_t bytes); void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap, - int64_t cur_sector, int64_t nr_sectors); + int64_t offset, int64_t bytes); BdrvDirtyBitmapIter *bdrv_dirty_meta_iter_new(BdrvDirtyBitmap *bitmap); BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap); void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter); @@ -75,9 +75,9 @@ void bdrv_dirty_bitmap_unlock(BdrvDirtyBitmap *bitmap); bool bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, int64_t offset); void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, - int64_t cur_sector, int64_t nr_sectors); + int64_t offset, int64_t bytes); void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, - int64_t cur_sector, int64_t nr_sectors= ); + int64_t offset, int64_t bytes); int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter); void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t offset); int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap); diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index ad559c62b1..117837b3cc 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -509,35 +509,41 @@ int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *ite= r) =20 /* Called within bdrv_dirty_bitmap_lock..unlock */ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, - int64_t cur_sector, int64_t nr_sectors) + int64_t offset, int64_t bytes) { + int64_t end_sector =3D DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE); + assert(bdrv_dirty_bitmap_enabled(bitmap)); assert(!bdrv_dirty_bitmap_readonly(bitmap)); - hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors); + hbitmap_set(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, + end_sector - (offset >> BDRV_SECTOR_BITS)); } =20 void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap, - int64_t cur_sector, int64_t nr_sectors) + int64_t offset, int64_t bytes) { bdrv_dirty_bitmap_lock(bitmap); - bdrv_set_dirty_bitmap_locked(bitmap, cur_sector, nr_sectors); + bdrv_set_dirty_bitmap_locked(bitmap, offset, bytes); bdrv_dirty_bitmap_unlock(bitmap); } =20 /* Called within bdrv_dirty_bitmap_lock..unlock */ void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, - int64_t cur_sector, int64_t nr_sectors) + int64_t offset, int64_t bytes) { + int64_t end_sector =3D DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE); + assert(bdrv_dirty_bitmap_enabled(bitmap)); assert(!bdrv_dirty_bitmap_readonly(bitmap)); - hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors); + hbitmap_reset(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, + end_sector - (offset >> BDRV_SECTOR_BITS)); } =20 void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap, - int64_t cur_sector, int64_t nr_sectors) + int64_t offset, int64_t bytes) { bdrv_dirty_bitmap_lock(bitmap); - bdrv_reset_dirty_bitmap_locked(bitmap, cur_sector, nr_sectors); + bdrv_reset_dirty_bitmap_locked(bitmap, offset, bytes); bdrv_dirty_bitmap_unlock(bitmap); } =20 diff --git a/block/mirror.c b/block/mirror.c index 545dfc50aa..5c0f79f56a 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -141,8 +141,7 @@ static void mirror_write_complete(void *opaque, int ret) if (ret < 0) { BlockErrorAction action; =20 - bdrv_set_dirty_bitmap(s->dirty_bitmap, op->offset >> BDRV_SECTOR_B= ITS, - op->bytes >> BDRV_SECTOR_BITS); + bdrv_set_dirty_bitmap(s->dirty_bitmap, op->offset, op->bytes); action =3D mirror_error_action(s, false, -ret); if (action =3D=3D BLOCK_ERROR_ACTION_REPORT && s->ret >=3D 0) { s->ret =3D ret; @@ -161,8 +160,7 @@ static void mirror_read_complete(void *opaque, int ret) if (ret < 0) { BlockErrorAction action; =20 - bdrv_set_dirty_bitmap(s->dirty_bitmap, op->offset >> BDRV_SECTOR_B= ITS, - op->bytes >> BDRV_SECTOR_BITS); + bdrv_set_dirty_bitmap(s->dirty_bitmap, op->offset, op->bytes); action =3D mirror_error_action(s, true, -ret); if (action =3D=3D BLOCK_ERROR_ACTION_REPORT && s->ret >=3D 0) { s->ret =3D ret; @@ -382,8 +380,8 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlo= ckJob *s) * calling bdrv_get_block_status_above could yield - if some blocks are * marked dirty in this window, we need to know. */ - bdrv_reset_dirty_bitmap_locked(s->dirty_bitmap, offset >> BDRV_SECTOR_= BITS, - nb_chunks * sectors_per_chunk); + bdrv_reset_dirty_bitmap_locked(s->dirty_bitmap, offset, + nb_chunks * s->granularity); bdrv_dirty_bitmap_unlock(s->dirty_bitmap); =20 bitmap_set(s->in_flight_bitmap, offset / s->granularity, nb_chunks); @@ -625,7 +623,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) =20 if (base =3D=3D NULL && !bdrv_has_zero_init(target_bs)) { if (!bdrv_can_write_zeroes_with_unmap(target_bs)) { - bdrv_set_dirty_bitmap(s->dirty_bitmap, 0, end); + bdrv_set_dirty_bitmap(s->dirty_bitmap, 0, s->bdev_length); return 0; } =20 @@ -681,7 +679,9 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) n =3D count >> BDRV_SECTOR_BITS; assert(n > 0); if (ret =3D=3D 1) { - bdrv_set_dirty_bitmap(s->dirty_bitmap, sector_num, n); + bdrv_set_dirty_bitmap(s->dirty_bitmap, + sector_num * BDRV_SECTOR_SIZE, + n * BDRV_SECTOR_SIZE); } sector_num +=3D n; } diff --git a/migration/block.c b/migration/block.c index f5bcf5505c..3282809583 100644 --- a/migration/block.c +++ b/migration/block.c @@ -334,7 +334,8 @@ static int mig_save_device_bulk(QEMUFile *f, BlkMigDevS= tate *bmds) blk->aiocb =3D blk_aio_preadv(bb, cur_sector * BDRV_SECTOR_SIZE, &blk-= >qiov, 0, blk_mig_read_cb, blk); =20 - bdrv_reset_dirty_bitmap(bmds->dirty_bitmap, cur_sector, nr_sectors); + bdrv_reset_dirty_bitmap(bmds->dirty_bitmap, cur_sector * BDRV_SECTOR_S= IZE, + nr_sectors * BDRV_SECTOR_SIZE); aio_context_release(blk_get_aio_context(bmds->blk)); qemu_mutex_unlock_iothread(); =20 @@ -542,7 +543,9 @@ static int mig_save_device_dirty(QEMUFile *f, BlkMigDev= State *bmds, } else { nr_sectors =3D BDRV_SECTORS_PER_DIRTY_CHUNK; } - bdrv_reset_dirty_bitmap_locked(bmds->dirty_bitmap, sector, nr_= sectors); + bdrv_reset_dirty_bitmap_locked(bmds->dirty_bitmap, + sector * BDRV_SECTOR_SIZE, + nr_sectors * BDRV_SECTOR_SIZE); bdrv_dirty_bitmap_unlock(bmds->dirty_bitmap); =20 blk =3D g_new(BlkMigBlock, 1); --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306453858996.9177648785472; Fri, 6 Oct 2017 09:14:13 -0700 (PDT) Received: from localhost ([::1]:45693 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VGJ-0008Nf-43 for importer@patchew.org; Fri, 06 Oct 2017 12:14:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55703) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxn-0000P8-Im for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxm-0002yH-2H for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45564) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxg-0002oP-KR; Fri, 06 Oct 2017 11:54:52 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 99820550B0; Fri, 6 Oct 2017 15:54:51 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id AA2F467598; Fri, 6 Oct 2017 15:54:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 99820550B0 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:44 +0200 Message-Id: <20171006155422.10135-17-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 06 Oct 2017 15:54:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 16/54] mirror: Switch mirror_dirty_init() to byte-based iteration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Now that we have adjusted the majority of the calls this function makes to be byte-based, it is easier to read the code if it makes passes over the image using bytes rather than sectors. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/mirror.c | 38 ++++++++++++++------------------------ 1 file changed, 14 insertions(+), 24 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 5c0f79f56a..459b80f8f3 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -612,15 +612,13 @@ static void mirror_throttle(MirrorBlockJob *s) =20 static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s) { - int64_t sector_num, end; + int64_t offset; BlockDriverState *base =3D s->base; BlockDriverState *bs =3D s->source; BlockDriverState *target_bs =3D blk_bs(s->target); - int ret, n; + int ret; int64_t count; =20 - end =3D s->bdev_length / BDRV_SECTOR_SIZE; - if (base =3D=3D NULL && !bdrv_has_zero_init(target_bs)) { if (!bdrv_can_write_zeroes_with_unmap(target_bs)) { bdrv_set_dirty_bitmap(s->dirty_bitmap, 0, s->bdev_length); @@ -628,9 +626,9 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) } =20 s->initial_zeroing_ongoing =3D true; - for (sector_num =3D 0; sector_num < end; ) { - int nb_sectors =3D MIN(end - sector_num, - QEMU_ALIGN_DOWN(INT_MAX, s->granularity) >> BDRV_SECTOR_BI= TS); + for (offset =3D 0; offset < s->bdev_length; ) { + int bytes =3D MIN(s->bdev_length - offset, + QEMU_ALIGN_DOWN(INT_MAX, s->granularity)); =20 mirror_throttle(s); =20 @@ -646,9 +644,8 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) continue; } =20 - mirror_do_zero_or_discard(s, sector_num * BDRV_SECTOR_SIZE, - nb_sectors * BDRV_SECTOR_SIZE, false= ); - sector_num +=3D nb_sectors; + mirror_do_zero_or_discard(s, offset, bytes, false); + offset +=3D bytes; } =20 mirror_wait_for_all_io(s); @@ -656,10 +653,10 @@ static int coroutine_fn mirror_dirty_init(MirrorBlock= Job *s) } =20 /* First part, loop on the sectors and initialize the dirty bitmap. */ - for (sector_num =3D 0; sector_num < end; ) { + for (offset =3D 0; offset < s->bdev_length; ) { /* Just to make sure we are not exceeding int limit. */ - int nb_sectors =3D MIN(INT_MAX >> BDRV_SECTOR_BITS, - end - sector_num); + int bytes =3D MIN(s->bdev_length - offset, + QEMU_ALIGN_DOWN(INT_MAX, s->granularity)); =20 mirror_throttle(s); =20 @@ -667,23 +664,16 @@ static int coroutine_fn mirror_dirty_init(MirrorBlock= Job *s) return 0; } =20 - ret =3D bdrv_is_allocated_above(bs, base, sector_num * BDRV_SECTOR= _SIZE, - nb_sectors * BDRV_SECTOR_SIZE, &coun= t); + ret =3D bdrv_is_allocated_above(bs, base, offset, bytes, &count); if (ret < 0) { return ret; } =20 - /* TODO: Relax this once bdrv_is_allocated_above and dirty - * bitmaps no longer require sector alignment. */ - assert(QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)); - n =3D count >> BDRV_SECTOR_BITS; - assert(n > 0); + assert(count); if (ret =3D=3D 1) { - bdrv_set_dirty_bitmap(s->dirty_bitmap, - sector_num * BDRV_SECTOR_SIZE, - n * BDRV_SECTOR_SIZE); + bdrv_set_dirty_bitmap(s->dirty_bitmap, offset, count); } - sector_num +=3D n; + offset +=3D count; } return 0; } --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306304372524.8621476213037; Fri, 6 Oct 2017 09:11:44 -0700 (PDT) Received: from localhost ([::1]:45684 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VDt-00069V-EA for importer@patchew.org; Fri, 06 Oct 2017 12:11:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55682) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxn-0000OF-52 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxm-0002y1-1M for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:54:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52696) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxi-0002pu-1v; Fri, 06 Oct 2017 11:54:54 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D81247C837; Fri, 6 Oct 2017 15:54:52 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id E9F74675BC; Fri, 6 Oct 2017 15:54:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D81247C837 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:45 +0200 Message-Id: <20171006155422.10135-18-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 06 Oct 2017 15:54:53 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 17/54] qcow2: Switch qcow2_measure() to byte-based iteration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake This is new code, but it is easier to read if it makes passes over the image using bytes rather than sectors (and will get easier in the future when bdrv_get_block_status is converted to byte-based). Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/qcow2.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 970006fc1d..b8da8ca105 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3673,20 +3673,19 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *op= ts, BlockDriverState *in_bs, */ required =3D virtual_size; } else { - int cluster_sectors =3D cluster_size / BDRV_SECTOR_SIZE; - int64_t sector_num; + int64_t offset; int pnum =3D 0; =20 - for (sector_num =3D 0; - sector_num < ssize / BDRV_SECTOR_SIZE; - sector_num +=3D pnum) { - int nb_sectors =3D MIN(ssize / BDRV_SECTOR_SIZE - sector_n= um, - BDRV_REQUEST_MAX_SECTORS); + for (offset =3D 0; offset < ssize; + offset +=3D pnum * BDRV_SECTOR_SIZE) { + int nb_sectors =3D MIN(ssize - offset, + BDRV_REQUEST_MAX_BYTES) / BDRV_SECTOR= _SIZE; BlockDriverState *file; int64_t ret; =20 ret =3D bdrv_get_block_status_above(in_bs, NULL, - sector_num, nb_sectors, + offset >> BDRV_SECTOR_BI= TS, + nb_sectors, &pnum, &file); if (ret < 0) { error_setg_errno(&local_err, -ret, @@ -3699,12 +3698,11 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *op= ts, BlockDriverState *in_bs, } else if ((ret & (BDRV_BLOCK_DATA | BDRV_BLOCK_ALLOCATED)= ) =3D=3D (BDRV_BLOCK_DATA | BDRV_BLOCK_ALLOCATED)) { /* Extend pnum to end of cluster for next iteration */ - pnum =3D ROUND_UP(sector_num + pnum, cluster_sectors) - - sector_num; + pnum =3D (ROUND_UP(offset + pnum * BDRV_SECTOR_SIZE, + cluster_size) - offset) >> BDRV_SECTOR_BI= TS; =20 /* Count clusters we've seen */ - required +=3D (sector_num % cluster_sectors + pnum) * - BDRV_SECTOR_SIZE; + required +=3D offset % cluster_size + pnum * BDRV_SECT= OR_SIZE; } } } --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305612151198.87234184708882; Fri, 6 Oct 2017 09:00:12 -0700 (PDT) Received: from localhost ([::1]:45616 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V2p-0004tR-8p for importer@patchew.org; Fri, 06 Oct 2017 12:00:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55868) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxt-0000YI-Ur for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxq-00039U-TJ for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:06 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45692) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxj-0002rj-AY; Fri, 06 Oct 2017 11:54:55 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 49CBF13AAE; Fri, 6 Oct 2017 15:54:54 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 349B567598; Fri, 6 Oct 2017 15:54:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 49CBF13AAE Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:46 +0200 Message-Id: <20171006155422.10135-19-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 06 Oct 2017 15:54:54 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 18/54] qcow2: Switch load_bitmap_data() to byte-based iteration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Now that we have adjusted the majority of the calls this function makes to be byte-based, it is easier to read the code if it makes passes over the image using bytes rather than sectors. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/qcow2-bitmap.c | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 2d8dcba3e8..02512a21f2 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -291,9 +291,8 @@ static int load_bitmap_data(BlockDriverState *bs, { int ret =3D 0; BDRVQcow2State *s =3D bs->opaque; - uint64_t sector, limit, sbc; + uint64_t offset, limit; uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); - uint64_t bm_sectors =3D DIV_ROUND_UP(bm_size, BDRV_SECTOR_SIZE); uint8_t *buf =3D NULL; uint64_t i, tab_size =3D size_to_clusters(s, @@ -305,32 +304,27 @@ static int load_bitmap_data(BlockDriverState *bs, =20 buf =3D g_malloc(s->cluster_size); limit =3D bytes_covered_by_bitmap_cluster(s, bitmap); - sbc =3D limit >> BDRV_SECTOR_BITS; - for (i =3D 0, sector =3D 0; i < tab_size; ++i, sector +=3D sbc) { - uint64_t count =3D MIN(bm_sectors - sector, sbc); + for (i =3D 0, offset =3D 0; i < tab_size; ++i, offset +=3D limit) { + uint64_t count =3D MIN(bm_size - offset, limit); uint64_t entry =3D bitmap_table[i]; - uint64_t offset =3D entry & BME_TABLE_ENTRY_OFFSET_MASK; + uint64_t data_offset =3D entry & BME_TABLE_ENTRY_OFFSET_MASK; =20 assert(check_table_entry(entry, s->cluster_size) =3D=3D 0); =20 - if (offset =3D=3D 0) { + if (data_offset =3D=3D 0) { if (entry & BME_TABLE_ENTRY_FLAG_ALL_ONES) { - bdrv_dirty_bitmap_deserialize_ones(bitmap, - sector * BDRV_SECTOR_SI= ZE, - count * BDRV_SECTOR_SIZ= E, + bdrv_dirty_bitmap_deserialize_ones(bitmap, offset, count, false); } else { /* No need to deserialize zeros because the dirty bitmap is * already cleared */ } } else { - ret =3D bdrv_pread(bs->file, offset, buf, s->cluster_size); + ret =3D bdrv_pread(bs->file, data_offset, buf, s->cluster_size= ); if (ret < 0) { goto finish; } - bdrv_dirty_bitmap_deserialize_part(bitmap, buf, - sector * BDRV_SECTOR_SIZE, - count * BDRV_SECTOR_SIZE, + bdrv_dirty_bitmap_deserialize_part(bitmap, buf, offset, count, false); } } --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305465122236.21841416214636; Fri, 6 Oct 2017 08:57:45 -0700 (PDT) Received: from localhost ([::1]:45605 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V0H-0002TE-69 for importer@patchew.org; Fri, 06 Oct 2017 11:57:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55781) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxq-0000TP-30 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxo-00034E-NJ for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:02 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36614) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxl-0002ve-2R; Fri, 06 Oct 2017 11:54:57 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0EDA8C047B74; Fri, 6 Oct 2017 15:54:56 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BFB367598; Fri, 6 Oct 2017 15:54:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0EDA8C047B74 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:47 +0200 Message-Id: <20171006155422.10135-20-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:54:56 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 19/54] qcow2: Switch store_bitmap_data() to byte-based iteration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Now that we have adjusted the majority of the calls this function makes to be byte-based, it is easier to read the code if it makes passes over the image using bytes rather than sectors. iotests 165 was rather weak - on a default 64k-cluster image, where bitmap granularity also defaults to 64k bytes, a single cluster of the bitmap table thus covers (64*1024*8) bits which each cover 64k bytes, or 32G of image space. But the test only uses a 1G image, so it cannot trigger any more than one loop of the code in store_bitmap_data(); and it was writing to the first cluster. In order to test that we are properly aligning which portions of the bitmap are being written to the file, we really want to test a case where the first dirty bit returned by bdrv_dirty_iter_next() is not aligned to the start of a cluster, which we can do by modifying the test to write data that doesn't happen to fall in the first cluster of the image. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/qcow2-bitmap.c | 31 ++++++++++++++++--------------- tests/qemu-iotests/165 | 2 +- 2 files changed, 17 insertions(+), 16 deletions(-) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 02512a21f2..f45e46cfbd 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -1072,10 +1072,9 @@ static uint64_t *store_bitmap_data(BlockDriverState = *bs, { int ret; BDRVQcow2State *s =3D bs->opaque; - int64_t sector; - uint64_t limit, sbc; + int64_t offset; + uint64_t limit; uint64_t bm_size =3D bdrv_dirty_bitmap_size(bitmap); - uint64_t bm_sectors =3D DIV_ROUND_UP(bm_size, BDRV_SECTOR_SIZE); const char *bm_name =3D bdrv_dirty_bitmap_name(bitmap); uint8_t *buf =3D NULL; BdrvDirtyBitmapIter *dbi; @@ -1100,18 +1099,22 @@ static uint64_t *store_bitmap_data(BlockDriverState= *bs, dbi =3D bdrv_dirty_iter_new(bitmap); buf =3D g_malloc(s->cluster_size); limit =3D bytes_covered_by_bitmap_cluster(s, bitmap); - sbc =3D limit >> BDRV_SECTOR_BITS; assert(DIV_ROUND_UP(bm_size, limit) =3D=3D tb_size); =20 - while ((sector =3D bdrv_dirty_iter_next(dbi) >> BDRV_SECTOR_BITS) >=3D= 0) { - uint64_t cluster =3D sector / sbc; + while ((offset =3D bdrv_dirty_iter_next(dbi)) >=3D 0) { + uint64_t cluster =3D offset / limit; uint64_t end, write_size; int64_t off; =20 - sector =3D cluster * sbc; - end =3D MIN(bm_sectors, sector + sbc); - write_size =3D bdrv_dirty_bitmap_serialization_size(bitmap, - sector * BDRV_SECTOR_SIZE, (end - sector) * BDRV_SECTOR_SIZE); + /* + * We found the first dirty offset, but want to write out the + * entire cluster of the bitmap that includes that offset, + * including any leading zero bits. + */ + offset =3D QEMU_ALIGN_DOWN(offset, limit); + end =3D MIN(bm_size, offset + limit); + write_size =3D bdrv_dirty_bitmap_serialization_size(bitmap, offset, + end - offset); assert(write_size <=3D s->cluster_size); =20 off =3D qcow2_alloc_clusters(bs, s->cluster_size); @@ -1123,9 +1126,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *= bs, } tb[cluster] =3D off; =20 - bdrv_dirty_bitmap_serialize_part(bitmap, buf, - sector * BDRV_SECTOR_SIZE, - (end - sector) * BDRV_SECTOR_SIZE= ); + bdrv_dirty_bitmap_serialize_part(bitmap, buf, offset, end - offset= ); if (write_size < s->cluster_size) { memset(buf + write_size, 0, s->cluster_size - write_size); } @@ -1143,11 +1144,11 @@ static uint64_t *store_bitmap_data(BlockDriverState= *bs, goto fail; } =20 - if (end >=3D bm_sectors) { + if (end >=3D bm_size) { break; } =20 - bdrv_set_dirty_iter(dbi, end * BDRV_SECTOR_SIZE); + bdrv_set_dirty_iter(dbi, end); } =20 *bitmap_table_size =3D tb_size; diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165 index 74d7b79a0b..a3932db3de 100755 --- a/tests/qemu-iotests/165 +++ b/tests/qemu-iotests/165 @@ -27,7 +27,7 @@ disk =3D os.path.join(iotests.test_dir, 'disk') disk_size =3D 0x40000000 # 1G =20 # regions for qemu_io: (start, count) in bytes -regions1 =3D ((0, 0x100000), +regions1 =3D ((0x0fff00, 0x10000), (0x200000, 0x100000)) =20 regions2 =3D ((0x10000000, 0x20000), --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150730594671892.2231720733347; Fri, 6 Oct 2017 09:05:46 -0700 (PDT) Received: from localhost ([::1]:45649 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V87-00016q-Rc for importer@patchew.org; Fri, 06 Oct 2017 12:05:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55960) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxy-0000d6-4G for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxu-0003Ey-5S for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:10 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36676) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxm-0002xf-BZ; Fri, 06 Oct 2017 11:54:58 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4C56EC047B74; Fri, 6 Oct 2017 15:54:57 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5F39267585; Fri, 6 Oct 2017 15:54:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 4C56EC047B74 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:48 +0200 Message-Id: <20171006155422.10135-21-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:54:57 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 20/54] dirty-bitmap: Switch bdrv_set_dirty() to bytes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Both callers already had bytes available, but were scaling to sectors. Move the scaling to internal code. In the case of bdrv_aligned_pwritev(), we are now passing the exact offset rather than a rounded sector-aligned value, but that's okay as long as dirty bitmap widens start/bytes to granularity boundaries. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- include/block/block_int.h | 2 +- block/dirty-bitmap.c | 7 ++++--- block/io.c | 6 ++---- 3 files changed, 7 insertions(+), 8 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index 99abe2ce74..79366b94b5 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -1028,7 +1028,7 @@ void blk_dev_eject_request(BlockBackend *blk, bool fo= rce); bool blk_dev_is_tray_open(BlockBackend *blk); bool blk_dev_is_medium_locked(BlockBackend *blk); =20 -void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector, int64_t nr_s= ect); +void bdrv_set_dirty(BlockDriverState *bs, int64_t offset, int64_t bytes); bool bdrv_requests_pending(BlockDriverState *bs); =20 void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out); diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 117837b3cc..58a3f330a9 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -628,10 +628,10 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBi= tmap *bitmap) hbitmap_deserialize_finish(bitmap->bitmap); } =20 -void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector, - int64_t nr_sectors) +void bdrv_set_dirty(BlockDriverState *bs, int64_t offset, int64_t bytes) { BdrvDirtyBitmap *bitmap; + int64_t end_sector =3D DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE); =20 if (QLIST_EMPTY(&bs->dirty_bitmaps)) { return; @@ -643,7 +643,8 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_s= ector, continue; } assert(!bdrv_dirty_bitmap_readonly(bitmap)); - hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors); + hbitmap_set(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, + end_sector - (offset >> BDRV_SECTOR_BITS)); } bdrv_dirty_bitmaps_unlock(bs); } diff --git a/block/io.c b/block/io.c index d633b0f851..e0f904583f 100644 --- a/block/io.c +++ b/block/io.c @@ -1334,7 +1334,6 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChil= d *child, bool waited; int ret; =20 - int64_t start_sector =3D offset >> BDRV_SECTOR_BITS; int64_t end_sector =3D DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE); uint64_t bytes_remaining =3D bytes; int max_transfer; @@ -1409,7 +1408,7 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChil= d *child, bdrv_debug_event(bs, BLKDBG_PWRITEV_DONE); =20 atomic_inc(&bs->write_gen); - bdrv_set_dirty(bs, start_sector, end_sector - start_sector); + bdrv_set_dirty(bs, offset, bytes); =20 stat64_max(&bs->wr_highest_offset, offset + bytes); =20 @@ -2438,8 +2437,7 @@ int coroutine_fn bdrv_co_pdiscard(BlockDriverState *b= s, int64_t offset, ret =3D 0; out: atomic_inc(&bs->write_gen); - bdrv_set_dirty(bs, req.offset >> BDRV_SECTOR_BITS, - req.bytes >> BDRV_SECTOR_BITS); + bdrv_set_dirty(bs, req.offset, req.bytes); tracked_request_end(&req); bdrv_dec_in_flight(bs); return ret; --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306761911334.5792232041896; Fri, 6 Oct 2017 09:19:21 -0700 (PDT) Received: from localhost ([::1]:45722 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VL6-0004Cv-Pl for importer@patchew.org; Fri, 06 Oct 2017 12:19:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55911) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxw-0000aw-5S for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxv-0003Fw-1N for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52964) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxn-00030x-KI; Fri, 06 Oct 2017 11:54:59 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8B0FC7C839; Fri, 6 Oct 2017 15:54:58 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9E3C667585; Fri, 6 Oct 2017 15:54:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8B0FC7C839 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:49 +0200 Message-Id: <20171006155422.10135-22-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 06 Oct 2017 15:54:58 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 21/54] dirty-bitmap: Convert internal hbitmap size/granularity X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Now that all callers are using byte-based interfaces, there's no reason for our internal hbitmap to remain with sector-based granularity. It also simplifies our internal scaling, since we already know that hbitmap widens requests out to granularity boundaries. Signed-off-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/dirty-bitmap.c | 62 +++++++++++++++---------------------------------= ---- 1 file changed, 18 insertions(+), 44 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 58a3f330a9..bd04e991b1 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -38,7 +38,7 @@ */ struct BdrvDirtyBitmap { QemuMutex *mutex; - HBitmap *bitmap; /* Dirty sector bitmap implementation */ + HBitmap *bitmap; /* Dirty bitmap implementation */ HBitmap *meta; /* Meta dirty bitmap */ BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status = */ char *name; /* Optional non-empty unique ID */ @@ -130,12 +130,7 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverS= tate *bs, } bitmap =3D g_new0(BdrvDirtyBitmap, 1); bitmap->mutex =3D &bs->dirty_bitmap_mutex; - /* - * TODO - let hbitmap track full granularity. For now, it is tracking - * only sector granularity, as a shortcut for our iterators. - */ - bitmap->bitmap =3D hbitmap_alloc(DIV_ROUND_UP(bitmap_size, BDRV_SECTOR= _SIZE), - ctz32(granularity) - BDRV_SECTOR_BITS); + bitmap->bitmap =3D hbitmap_alloc(bitmap_size, ctz32(granularity)); bitmap->size =3D bitmap_size; bitmap->name =3D g_strdup(name); bitmap->disabled =3D false; @@ -312,7 +307,7 @@ void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, i= nt64_t bytes) QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) { assert(!bdrv_dirty_bitmap_frozen(bitmap)); assert(!bitmap->active_iterators); - hbitmap_truncate(bitmap->bitmap, DIV_ROUND_UP(bytes, BDRV_SECTOR_S= IZE)); + hbitmap_truncate(bitmap->bitmap, bytes); bitmap->size =3D bytes; } bdrv_dirty_bitmaps_unlock(bs); @@ -442,7 +437,7 @@ bool bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDi= rtyBitmap *bitmap, int64_t offset) { if (bitmap) { - return hbitmap_get(bitmap->bitmap, offset >> BDRV_SECTOR_BITS); + return hbitmap_get(bitmap->bitmap, offset); } else { return false; } @@ -470,7 +465,7 @@ uint32_t bdrv_get_default_bitmap_granularity(BlockDrive= rState *bs) =20 uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirtyBitmap *bitmap) { - return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap); + return 1U << hbitmap_granularity(bitmap->bitmap); } =20 BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap) @@ -503,20 +498,16 @@ void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter) =20 int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter) { - int64_t ret =3D hbitmap_iter_next(&iter->hbi); - return ret < 0 ? -1 : ret * BDRV_SECTOR_SIZE; + return hbitmap_iter_next(&iter->hbi); } =20 /* Called within bdrv_dirty_bitmap_lock..unlock */ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t offset, int64_t bytes) { - int64_t end_sector =3D DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE); - assert(bdrv_dirty_bitmap_enabled(bitmap)); assert(!bdrv_dirty_bitmap_readonly(bitmap)); - hbitmap_set(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, - end_sector - (offset >> BDRV_SECTOR_BITS)); + hbitmap_set(bitmap->bitmap, offset, bytes); } =20 void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap, @@ -531,12 +522,9 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap, void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, int64_t offset, int64_t bytes) { - int64_t end_sector =3D DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE); - assert(bdrv_dirty_bitmap_enabled(bitmap)); assert(!bdrv_dirty_bitmap_readonly(bitmap)); - hbitmap_reset(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, - end_sector - (offset >> BDRV_SECTOR_BITS)); + hbitmap_reset(bitmap->bitmap, offset, bytes); } =20 void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap, @@ -556,8 +544,7 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, H= Bitmap **out) hbitmap_reset_all(bitmap->bitmap); } else { HBitmap *backup =3D bitmap->bitmap; - bitmap->bitmap =3D hbitmap_alloc(DIV_ROUND_UP(bitmap->size, - BDRV_SECTOR_SIZE), + bitmap->bitmap =3D hbitmap_alloc(bitmap->size, hbitmap_granularity(backup)); *out =3D backup; } @@ -576,51 +563,40 @@ void bdrv_undo_clear_dirty_bitmap(BdrvDirtyBitmap *bi= tmap, HBitmap *in) uint64_t bdrv_dirty_bitmap_serialization_size(const BdrvDirtyBitmap *bitma= p, uint64_t offset, uint64_t by= tes) { - assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); - return hbitmap_serialization_size(bitmap->bitmap, - offset >> BDRV_SECTOR_BITS, - bytes >> BDRV_SECTOR_BITS); + return hbitmap_serialization_size(bitmap->bitmap, offset, bytes); } =20 uint64_t bdrv_dirty_bitmap_serialization_align(const BdrvDirtyBitmap *bitm= ap) { - return hbitmap_serialization_align(bitmap->bitmap) * BDRV_SECTOR_SIZE; + return hbitmap_serialization_align(bitmap->bitmap); } =20 void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap, uint8_t *buf, uint64_t offset, uint64_t bytes) { - assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); - hbitmap_serialize_part(bitmap->bitmap, buf, offset >> BDRV_SECTOR_BITS, - bytes >> BDRV_SECTOR_BITS); + hbitmap_serialize_part(bitmap->bitmap, buf, offset, bytes); } =20 void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap, uint8_t *buf, uint64_t offset, uint64_t bytes, bool finish) { - assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); - hbitmap_deserialize_part(bitmap->bitmap, buf, offset >> BDRV_SECTOR_BI= TS, - bytes >> BDRV_SECTOR_BITS, finish); + hbitmap_deserialize_part(bitmap->bitmap, buf, offset, bytes, finish); } =20 void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap, uint64_t offset, uint64_t bytes, bool finish) { - assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); - hbitmap_deserialize_zeroes(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, - bytes >> BDRV_SECTOR_BITS, finish); + hbitmap_deserialize_zeroes(bitmap->bitmap, offset, bytes, finish); } =20 void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap, uint64_t offset, uint64_t bytes, bool finish) { - assert(QEMU_IS_ALIGNED(offset | bytes, BDRV_SECTOR_SIZE)); - hbitmap_deserialize_ones(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, - bytes >> BDRV_SECTOR_BITS, finish); + hbitmap_deserialize_ones(bitmap->bitmap, offset, bytes, finish); } =20 void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap) @@ -631,7 +607,6 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitm= ap *bitmap) void bdrv_set_dirty(BlockDriverState *bs, int64_t offset, int64_t bytes) { BdrvDirtyBitmap *bitmap; - int64_t end_sector =3D DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE); =20 if (QLIST_EMPTY(&bs->dirty_bitmaps)) { return; @@ -643,8 +618,7 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t offse= t, int64_t bytes) continue; } assert(!bdrv_dirty_bitmap_readonly(bitmap)); - hbitmap_set(bitmap->bitmap, offset >> BDRV_SECTOR_BITS, - end_sector - (offset >> BDRV_SECTOR_BITS)); + hbitmap_set(bitmap->bitmap, offset, bytes); } bdrv_dirty_bitmaps_unlock(bs); } @@ -654,12 +628,12 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t off= set, int64_t bytes) */ void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *iter, int64_t offset) { - hbitmap_iter_init(&iter->hbi, iter->hbi.hb, offset >> BDRV_SECTOR_BITS= ); + hbitmap_iter_init(&iter->hbi, iter->hbi.hb, offset); } =20 int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap) { - return hbitmap_count(bitmap->bitmap) << BDRV_SECTOR_BITS; + return hbitmap_count(bitmap->bitmap); } =20 int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap) --=20 2.13.6 From nobody Sat Apr 27 19:30:39 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507305788650282.721114028864; Fri, 6 Oct 2017 09:03:08 -0700 (PDT) Received: from localhost ([::1]:45634 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V5Z-0007KY-Tr for importer@patchew.org; Fri, 06 Oct 2017 12:03:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55913) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxw-0000ax-5t for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxv-0003HY-EF for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36792) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxo-00033E-SQ; Fri, 06 Oct 2017 11:55:01 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CD5D9C047B70; Fri, 6 Oct 2017 15:54:59 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD05E67585; Fri, 6 Oct 2017 15:54:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com CD5D9C047B70 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:50 +0200 Message-Id: <20171006155422.10135-23-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:54:59 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 22/54] hw/block/onenand: Remove dead code block X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Thomas Huth The condition of the for-loop makes sure that b is always smaller than s->blocks, so the "if (b >=3D s->blocks)" statement is completely superfluous here. Buglink: https://bugs.launchpad.net/qemu/+bug/1715007 Signed-off-by: Thomas Huth Reviewed-by: Laurent Vivier Signed-off-by: Kevin Wolf --- hw/block/onenand.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/hw/block/onenand.c b/hw/block/onenand.c index 30e40f3914..de65c9ebb9 100644 --- a/hw/block/onenand.c +++ b/hw/block/onenand.c @@ -520,10 +520,6 @@ static void onenand_command(OneNANDState *s) s->intstatus |=3D ONEN_INT; =20 for (b =3D 0; b < s->blocks; b ++) { - if (b >=3D s->blocks) { - s->status |=3D ONEN_ERR_CMD; - break; - } if (s->blockwp[b] =3D=3D ONEN_LOCK_LOCKTIGHTEN) break; =20 --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150730602233097.09803359343152; Fri, 6 Oct 2017 09:07:02 -0700 (PDT) Received: from localhost ([::1]:45655 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0V9L-0002AL-Fi for importer@patchew.org; Fri, 06 Oct 2017 12:06:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55948) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxx-0000cP-EV for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxv-0003Im-SM for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:09 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36946) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxq-00035z-38; Fri, 06 Oct 2017 11:55:02 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 175EAC047B93; Fri, 6 Oct 2017 15:55:01 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 292AF67585; Fri, 6 Oct 2017 15:54:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 175EAC047B93 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:51 +0200 Message-Id: <20171006155422.10135-24-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:55:01 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 23/54] qemu-iotests: remove dead code X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini This includes shell function, shell variables and command line options (randomize.awk does not exist). Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 28 ----------------- tests/qemu-iotests/common | 23 -------------- tests/qemu-iotests/common.config | 26 --------------- tests/qemu-iotests/common.rc | 68 ------------------------------------= ---- 4 files changed, 145 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 4583a0c269..041780b09f 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -65,7 +65,6 @@ then export SOCKET_SCM_HELPER=3D"$build_iotests/socket_scm_helper" fi =20 -# if ./qemu exists, it should be prioritized and will be chosen by common.= config if [[ -z "$QEMU_PROG" && ! -x './qemu' ]] then arch=3D$(uname -m 2> /dev/null) @@ -140,12 +139,6 @@ _timestamp() =20 _wrapup() { - # for hangcheck ... - # remove files that were used by hangcheck - # - [ -f "${TEST_DIR}"/check.pid ] && rm -rf "${TEST_DIR}"/check.pid - [ -f "${TEST_DIR}"/check.sts ] && rm -rf "${TEST_DIR}"/check.sts - if $showme then : @@ -201,24 +194,6 @@ END { if (NR > 0) { =20 trap "_wrapup; exit \$status" 0 1 2 3 15 =20 -# for hangcheck ... -# Save pid of check in a well known place, so that hangcheck can be sure it -# has the right pid (getting the pid from ps output is not reliable enough= ). -# -rm -rf "${TEST_DIR}"/check.pid -echo $$ > "${TEST_DIR}"/check.pid - -# for hangcheck ... -# Save the status of check in a well known place, so that hangcheck can be -# sure to know where check is up to (getting test number from ps output is -# not reliable enough since the trace stuff has been introduced). -# -rm -rf "${TEST_DIR}"/check.sts -echo "preamble" > "${TEST_DIR}"/check.sts - -# don't leave old full output behind on a clean run -rm -f check.full - [ -f $TIMESTAMP_FILE ] || touch $TIMESTAMP_FILE =20 FULL_IMGFMT_DETAILS=3D`_full_imgfmt_details` @@ -276,9 +251,6 @@ do fi rm -f core $seq.notrun =20 - # for hangcheck ... - echo "$seq" > "${TEST_DIR}"/check.sts - start=3D`_wallclock` $timestamp && printf %s " [$(date "+%T")]" =20 diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common index d34c11c056..867918895b 100644 --- a/tests/qemu-iotests/common +++ b/tests/qemu-iotests/common @@ -19,17 +19,6 @@ # common procedures for QA scripts # =20 -_setenvironment() -{ - MSGVERB=3D"text:action" - export MSGVERB -} - -rm -f "$OUTPUT_DIR/$iam.out" -_setenvironment - -check=3D${check-true} - diff=3D"diff -u" verbose=3Dfalse debug=3Dfalse @@ -40,7 +29,6 @@ showme=3Dfalse sortme=3Dfalse expunge=3Dtrue have_test_arg=3Dfalse -randomize=3Dfalse cachemode=3Dfalse rm -f $tmp.list $tmp.tmp $tmp.sed =20 @@ -170,7 +158,6 @@ other options -n show me, do not run tests -o options -o options to pass to qemu-img create/convert -T output timestamps - -r randomize test order -c mode cache mode =20 testlist options @@ -327,11 +314,6 @@ testlist options cachemode=3Dtrue xpand=3Dfalse ;; - -r) # randomize test order - randomize=3Dtrue - xpand=3Dfalse - ;; - -T) # turn on timestamp output timestamp=3Dtrue xpand=3Dfalse @@ -445,11 +427,6 @@ fi list=3D`sort $tmp.list` rm -f $tmp.list $tmp.tmp $tmp.sed =20 -if $randomize -then - list=3D`echo $list | awk -f randomize.awk` -fi - [ "$QEMU" =3D "" ] && _fatal "qemu not found" [ "$QEMU_IMG" =3D "" ] && _fatal "qemu-img not found" [ "$QEMU_IO" =3D "" ] && _fatal "qemu-io not found" diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.c= onfig index e0883a0c65..b599c72211 100644 --- a/tests/qemu-iotests/common.config +++ b/tests/qemu-iotests/common.config @@ -15,33 +15,14 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . # -# -# setup and check for config parameters, and in particular -# -# EMAIL - email of the script runner. -# TEST_DIR - scratch test directory -# -# - These can be added to $HOST_CONFIG_DIR (witch default to ./config) -# below or a separate local configuration file can be used (using -# the HOST_OPTIONS variable). -# - This script is shared by the stress test system and the auto-qa -# system (includes both regression test and benchmark components). -# - this script shouldn't make any assertions about filesystem -# validity or mountedness. -# - # all tests should use a common language setting to prevent golden # output mismatches. export LANG=3DC =20 PATH=3D".:$PATH" =20 -HOST=3D`hostname -s 2> /dev/null` HOSTOS=3D`uname -s` =20 -EMAIL=3Droot@localhost # where auto-qa will send its status messages -export HOST_OPTIONS=3D${HOST_OPTIONS:=3Dlocal.config} -export CHECK_OPTIONS=3D${CHECK_OPTIONS:=3D"-g auto"} export PWD=3D`pwd` =20 export _QEMU_HANDLE=3D0 @@ -78,11 +59,6 @@ _fatal() export AWK_PROG=3D"`set_prog_path awk`" [ "$AWK_PROG" =3D "" ] && _fatal "awk not found" =20 -export SED_PROG=3D"`set_prog_path sed`" -[ "$SED_PROG" =3D "" ] && _fatal "sed not found" - -export PS_ALL_FLAGS=3D"-ef" - if [ -z "$QEMU_PROG" ]; then export QEMU_PROG=3D"`set_prog_path qemu`" fi @@ -198,8 +174,6 @@ fi =20 export QEMU_DEFAULT_MACHINE=3D"$default_machine" =20 -[ -f /etc/qemu-iotest.config ] && . /etc/qemu-iotest.config - if [ -z "$TEST_DIR" ]; then TEST_DIR=3D`pwd`/scratch fi diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 8d486dbeb4..5938d5145f 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -293,51 +293,6 @@ _img_info() done } =20 -_get_pids_by_name() -{ - if [ $# -ne 1 ] - then - echo "Usage: _get_pids_by_name process-name" 1>&2 - exit 1 - fi - - # Algorithm ... all ps(1) variants have a time of the form MM:SS or - # HH:MM:SS before the psargs field, use this as the search anchor. - # - # Matches with $1 (process-name) occur if the first psarg is $1 - # or ends in /$1 ... the matching uses sed's regular expressions, - # so passing a regex into $1 will work. - - ps $PS_ALL_FLAGS \ - | sed -n \ - -e 's/$/ /' \ - -e 's/[ ][ ]*/ /g' \ - -e 's/^ //' \ - -e 's/^[^ ]* //' \ - -e "/[0-9]:[0-9][0-9] *[^ ]*\/$1 /s/ .*//p" \ - -e "/[0-9]:[0-9][0-9] *$1 /s/ .*//p" -} - -# fqdn for localhost -# -_get_fqdn() -{ - host=3D`hostname` - $NSLOOKUP_PROG $host | $AWK_PROG '{ if ($1 =3D=3D "Name:") print $2 }' -} - -# check if run as root -# -_need_to_be_root() -{ - id=3D`id | $SED_PROG -e 's/(.*//' -e 's/.*=3D//'` - if [ "$id" -ne 0 ] - then - echo "Arrgh ... you need to be root (not uid=3D$id) to run this te= st" - exit 1 - fi -} - # bail out, setting up .notrun file # _notrun() @@ -491,28 +446,5 @@ _full_platform_details() echo "$os/$platform $host $kernel" } =20 -_link_out_file() -{ - if [ -z "$1" ]; then - echo Error must pass \$seq. - exit - fi - rm -f $1 - if [ "`uname`" =3D=3D "IRIX64" ] || [ "`uname`" =3D=3D "IRIX" ]; then - ln -s $1.irix $1 - elif [ "`uname`" =3D=3D "Linux" ]; then - ln -s $1.linux $1 - else - echo Error test $seq does not run on the operating system: `uname` - exit - fi -} - -_die() -{ - echo $@ - exit 1 -} - # make sure this script returns success true --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306928175581.6743134531764; Fri, 6 Oct 2017 09:22:08 -0700 (PDT) Received: from localhost ([::1]:45742 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VNt-00071G-1G for importer@patchew.org; Fri, 06 Oct 2017 12:21:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55994) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxz-0000fF-KY for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxy-0003Q9-Tk for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:11 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57876) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxr-0003AP-PX; Fri, 06 Oct 2017 11:55:03 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B6E2D285C1; Fri, 6 Oct 2017 15:55:02 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 694CA67585; Fri, 6 Oct 2017 15:55:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B6E2D285C1 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:52 +0200 Message-Id: <20171006155422.10135-25-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 06 Oct 2017 15:55:02 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 24/54] qemu-iotests: get rid of AWK_PROG X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 4 ++-- tests/qemu-iotests/common | 2 +- tests/qemu-iotests/common.config | 3 --- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 041780b09f..67da67bd54 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -128,7 +128,7 @@ tmp=3D"${TEST_DIR}"/$$ =20 _wallclock() { - date "+%H %M %S" | $AWK_PROG '{ print $1*3600 + $2*60 + $3 }' + date "+%H %M %S" | awk '{ print $1*3600 + $2*60 + $3 }' } =20 _timestamp() @@ -147,7 +147,7 @@ _wrapup() if [ -f $TIMESTAMP_FILE -a -f $tmp.time ] then cat $TIMESTAMP_FILE $tmp.time \ - | $AWK_PROG ' + | awk ' { t[$1] =3D $2 } END { if (NR > 0) { for (i in t) print i " " t[i] diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common index 867918895b..130f647a4d 100644 --- a/tests/qemu-iotests/common +++ b/tests/qemu-iotests/common @@ -366,7 +366,7 @@ testlist options if $xpand then have_test_arg=3Dtrue - $AWK_PROG Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306205567321.77981341617874; Fri, 6 Oct 2017 09:10:05 -0700 (PDT) Received: from localhost ([::1]:45670 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VC4-0004TY-Rn for importer@patchew.org; Fri, 06 Oct 2017 12:09:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55990) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uxz-0000er-8F for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uxy-0003Ow-AW for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:11 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50534) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxv-0003GB-M0; Fri, 06 Oct 2017 11:55:07 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 73BB87EA92; Fri, 6 Oct 2017 15:55:06 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 175AD67585; Fri, 6 Oct 2017 15:55:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 73BB87EA92 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:53 +0200 Message-Id: <20171006155422.10135-26-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 06 Oct 2017 15:55:06 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 25/54] qemu-iotests: move "check" code out of common.rc X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini Some functions in common.rc are never used by the tests. Move them out of that file and into common, which is already included only by "check". Code that actually *is* common to "check" and tests can be placed in common.config. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/common | 25 ++++++++++++++++++++++++- tests/qemu-iotests/common.config | 12 ++++++++++++ tests/qemu-iotests/common.rc | 40 ------------------------------------= ---- 3 files changed, 36 insertions(+), 41 deletions(-) diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common index 130f647a4d..2e98e64d5c 100644 --- a/tests/qemu-iotests/common +++ b/tests/qemu-iotests/common @@ -19,6 +19,24 @@ # common procedures for QA scripts # =20 +_full_imgfmt_details() +{ + if [ -n "$IMGOPTS" ]; then + echo "$IMGFMT ($IMGOPTS)" + else + echo "$IMGFMT" + fi +} + +_full_platform_details() +{ + os=3D`uname -s` + host=3D`hostname -s` + kernel=3D`uname -r` + platform=3D`uname -m` + echo "$os/$platform $host $kernel" +} + diff=3D"diff -u" verbose=3Dfalse debug=3Dfalse @@ -404,7 +422,12 @@ if [ "$IMGOPTSSYNTAX" !=3D "true" ]; then fi =20 # Set default options for qemu-img create -o if they were not specified -_set_default_imgopts +if [ "$IMGFMT" =3D=3D "qcow2" ] && ! (echo "$IMGOPTS" | grep "compat=3D" >= /dev/null); then + IMGOPTS=3D$(_optstr_add "$IMGOPTS" "compat=3D1.1") +fi +if [ "$IMGFMT" =3D=3D "luks" ] && ! (echo "$IMGOPTS" | grep "iter-time=3D"= > /dev/null); then + IMGOPTS=3D$(_optstr_add "$IMGOPTS" "iter-time=3D10") +fi =20 if [ -s $tmp.list ] then diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.c= onfig index 0f571d46eb..91da65f3dc 100644 --- a/tests/qemu-iotests/common.config +++ b/tests/qemu-iotests/common.config @@ -27,6 +27,9 @@ export PWD=3D`pwd` =20 export _QEMU_HANDLE=3D0 =20 +# make sure we have a standard umask +umask 022 + # $1 =3D prog to look for, $2* =3D default pathnames if not found in $PATH set_prog_path() { @@ -49,6 +52,15 @@ set_prog_path() return 1 } =20 +_optstr_add() +{ + if [ -n "$1" ]; then + echo "$1,$2" + else + echo "$2" + fi +} + _fatal() { echo "$*" diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 5938d5145f..6f6e03366f 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -50,9 +50,6 @@ then fi fi =20 -# make sure we have a standard umask -umask 022 - if [ "$IMGOPTSSYNTAX" =3D "true" ]; then DRIVER=3D"driver=3D$IMGFMT" if [ "$IMGFMT" =3D "luks" ]; then @@ -94,25 +91,6 @@ else fi ORIG_TEST_IMG=3D"$TEST_IMG" =20 -_optstr_add() -{ - if [ -n "$1" ]; then - echo "$1,$2" - else - echo "$2" - fi -} - -_set_default_imgopts() -{ - if [ "$IMGFMT" =3D=3D "qcow2" ] && ! (echo "$IMGOPTS" | grep "compat= =3D" > /dev/null); then - IMGOPTS=3D$(_optstr_add "$IMGOPTS" "compat=3D1.1") - fi - if [ "$IMGFMT" =3D=3D "luks" ] && ! (echo "$IMGOPTS" | grep "iter-time= =3D" > /dev/null); then - IMGOPTS=3D$(_optstr_add "$IMGOPTS" "iter-time=3D10") - fi -} - _use_sample_img() { SAMPLE_IMG_FILE=3D"${1%\.bz2}" @@ -428,23 +406,5 @@ _require_command() [ -x "$c" ] || _notrun "$1 utility required, skipped this test" } =20 -_full_imgfmt_details() -{ - if [ -n "$IMGOPTS" ]; then - echo "$IMGFMT ($IMGOPTS)" - else - echo "$IMGFMT" - fi -} - -_full_platform_details() -{ - os=3D`uname -s` - host=3D`hostname -s` - kernel=3D`uname -r` - platform=3D`uname -m` - echo "$os/$platform $host $kernel" -} - # make sure this script returns success true --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306347584459.85602713504886; Fri, 6 Oct 2017 09:12:27 -0700 (PDT) Received: from localhost ([::1]:45687 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VEa-0006n2-Pc for importer@patchew.org; Fri, 06 Oct 2017 12:12:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56090) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uy5-0000om-RU for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uy4-0003Wi-FK for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58118) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxx-0003KV-An; Fri, 06 Oct 2017 11:55:09 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 416173E2A9; Fri, 6 Oct 2017 15:55:08 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id C540E67585; Fri, 6 Oct 2017 15:55:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 416173E2A9 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:54 +0200 Message-Id: <20171006155422.10135-27-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 06 Oct 2017 15:55:08 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 26/54] qemu-iotests: cleanup and fix search for programs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini Instead of ./check failing when a binary is missing, we try each test case now and each one fails with tons of test case diffs. Also, all the variables were initialized by "check" prior to "common" being sourced, and then (uselessly) checked for emptiness again in "check". Centralize the search for programs in "common" (which will soon be one with "check"), including the "realpath" invocation which can be done just once in "check" rather than in the tests. For qnio_server, move the detection to "common", simplifying set_prog_path to stop handling the unused second argument, and embedding the "realpath" pass. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 41 --------------------- tests/qemu-iotests/common | 77 ++++++++++++++++++++++++++++++++++++= +--- tests/qemu-iotests/common.config | 61 +------------------------------ 3 files changed, 73 insertions(+), 106 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 67da67bd54..03871d0681 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -60,47 +60,6 @@ fi =20 build_root=3D"$build_iotests/../.." =20 -if [ -x "$build_iotests/socket_scm_helper" ] -then - export SOCKET_SCM_HELPER=3D"$build_iotests/socket_scm_helper" -fi - -if [[ -z "$QEMU_PROG" && ! -x './qemu' ]] -then - arch=3D$(uname -m 2> /dev/null) - - if [[ -n $arch && -x "$build_root/$arch-softmmu/qemu-system-$arch" ]] - then - export QEMU_PROG=3D"$build_root/$arch-softmmu/qemu-system-$arch" - else - pushd "$build_root" > /dev/null - for binary in *-softmmu/qemu-system-* - do - if [ -x "$binary" ] - then - export QEMU_PROG=3D"$build_root/$binary" - break - fi - done - popd > /dev/null - fi -fi - -if [[ -z $QEMU_IMG_PROG && -x "$build_root/qemu-img" && ! -x './qemu-img' = ]] -then - export QEMU_IMG_PROG=3D"$build_root/qemu-img" -fi - -if [[ -z $QEMU_IO_PROG && -x "$build_root/qemu-io" && ! -x './qemu-io' ]] -then - export QEMU_IO_PROG=3D"$build_root/qemu-io" -fi - -if [[ -z $QEMU_NBD_PROG && -x "$build_root/qemu-nbd" && ! -x './qemu-nbd' = ]] -then - export QEMU_NBD_PROG=3D"$build_root/qemu-nbd" -fi - # we need common.env if ! . "$build_iotests/common.env" then diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common index 2e98e64d5c..abacc24114 100644 --- a/tests/qemu-iotests/common +++ b/tests/qemu-iotests/common @@ -37,6 +37,17 @@ _full_platform_details() echo "$os/$platform $host $kernel" } =20 +# $1 =3D prog to look for +set_prog_path() +{ + p=3D`command -v $1 2> /dev/null` + if [ -n "$p" -a -x "$p" ]; then + realpath -- "$(type -p "$p")" + else + return 1 + fi +} + diff=3D"diff -u" verbose=3Dfalse debug=3Dfalse @@ -450,10 +461,66 @@ fi list=3D`sort $tmp.list` rm -f $tmp.list $tmp.tmp $tmp.sed =20 -[ "$QEMU" =3D "" ] && _fatal "qemu not found" -[ "$QEMU_IMG" =3D "" ] && _fatal "qemu-img not found" -[ "$QEMU_IO" =3D "" ] && _fatal "qemu-io not found" +if [ -z "$QEMU_PROG" ] +then + if [ -x "$build_iotests/qemu" ]; then + export QEMU_PROG=3D"$build_iotests/qemu" + elif [ -x "$build_root/$arch-softmmu/qemu-system-$arch" ]; then + export QEMU_PROG=3D"$build_root/$arch-softmmu/qemu-system-$arch" + else + pushd "$build_root" > /dev/null + for binary in *-softmmu/qemu-system-* + do + if [ -x "$binary" ] + then + export QEMU_PROG=3D"$build_root/$binary" + break + fi + done + popd > /dev/null + [ "$QEMU_PROG" =3D "" ] && _init_error "qemu not found" + fi +fi +export QEMU_PROG=3D$(realpath -- "$(type -p "$QEMU_PROG")") + +if [ -z "$QEMU_IMG_PROG" ]; then + if [ -x "$build_iotests/qemu-img" ]; then + export QEMU_IMG_PROG=3D"$build_iotests/qemu-img" + elif [ -x "$build_root/qemu-img" ]; then + export QEMU_IMG_PROG=3D"$build_root/qemu-img" + else + _init_error "qemu-img not found" + fi +fi +export QEMU_IMG_PROG=3D$(realpath -- "$(type -p "$QEMU_IMG_PROG")") + +if [ -z "$QEMU_IO_PROG" ]; then + if [ -x "$build_iotests/qemu-io" ]; then + export QEMU_IO_PROG=3D"$build_iotests/qemu-io" + elif [ -x "$build_root/qemu-io" ]; then + export QEMU_IO_PROG=3D"$build_root/qemu-io" + else + _init_error "qemu-io not found" + fi +fi +export QEMU_IO_PROG=3D$(realpath -- "$(type -p "$QEMU_IO_PROG")") =20 -if [ "$IMGPROTO" =3D "nbd" ] ; then - [ "$QEMU_NBD" =3D "" ] && _fatal "qemu-nbd not found" +if [ -z $QEMU_NBD_PROG ]; then + if [ -x "$build_iotests/qemu-nbd" ]; then + export QEMU_NBD_PROG=3D"$build_iotests/qemu-nbd" + elif [ -x "$build_root/qemu-nbd" ]; then + export QEMU_NBD_PROG=3D"$build_root/qemu-nbd" + else + _init_error "qemu-nbd not found" + fi +fi +export QEMU_NBD_PROG=3D$(realpath -- "$(type -p "$QEMU_NBD_PROG")") + +if [ -z "$QEMU_VXHS_PROG" ]; then + export QEMU_VXHS_PROG=3D"`set_prog_path qnio_server`" +fi + +if [ -x "$build_iotests/socket_scm_helper" ] +then + export SOCKET_SCM_HELPER=3D"$build_iotests/socket_scm_helper" fi diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.c= onfig index 91da65f3dc..c97c63983f 100644 --- a/tests/qemu-iotests/common.config +++ b/tests/qemu-iotests/common.config @@ -22,6 +22,7 @@ export LANG=3DC PATH=3D".:$PATH" =20 HOSTOS=3D`uname -s` +arch=3D`uname -m` =20 export PWD=3D`pwd` =20 @@ -30,28 +31,6 @@ export _QEMU_HANDLE=3D0 # make sure we have a standard umask umask 022 =20 -# $1 =3D prog to look for, $2* =3D default pathnames if not found in $PATH -set_prog_path() -{ - p=3D`command -v $1 2> /dev/null` - if [ -n "$p" -a -x "$p" ]; then - echo $p - return 0 - fi - p=3D$1 - - shift - for f; do - if [ -x $f ]; then - echo $f - return 0 - fi - done - - echo "" - return 1 -} - _optstr_add() { if [ -n "$1" ]; then @@ -61,44 +40,6 @@ _optstr_add() fi } =20 -_fatal() -{ - echo "$*" - status=3D1 - exit 1 -} - -if [ -z "$QEMU_PROG" ]; then - export QEMU_PROG=3D"`set_prog_path qemu`" -fi - -if [ -z "$QEMU_IMG_PROG" ]; then - export QEMU_IMG_PROG=3D"`set_prog_path qemu-img`" -fi - -if [ -z "$QEMU_IO_PROG" ]; then - export QEMU_IO_PROG=3D"`set_prog_path qemu-io`" -fi - -if [ -z "$QEMU_NBD_PROG" ]; then - export QEMU_NBD_PROG=3D"`set_prog_path qemu-nbd`" -fi - -if [ -z "$QEMU_VXHS_PROG" ]; then - export QEMU_VXHS_PROG=3D"`set_prog_path qnio_server`" -fi - -export QEMU_PROG=3D$(realpath -- "$(type -p "$QEMU_PROG")") -export QEMU_IMG_PROG=3D$(realpath -- "$(type -p "$QEMU_IMG_PROG")") -export QEMU_IO_PROG=3D$(realpath -- "$(type -p "$QEMU_IO_PROG")") -export QEMU_NBD_PROG=3D$(realpath -- "$(type -p "$QEMU_NBD_PROG")") - -# This program is not built as part of qemu but (possibly) provided by the -# system, so it may not be present at all -if [ -n "$QEMU_VXHS_PROG" ]; then - export QEMU_VXHS_PROG=3D$(realpath -- "$(type -p "$QEMU_VXHS_PROG")") -fi - _qemu_wrapper() { ( --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306802819530.3420371810231; Fri, 6 Oct 2017 09:20:02 -0700 (PDT) Received: from localhost ([::1]:45725 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VLo-0004rb-SX for importer@patchew.org; Fri, 06 Oct 2017 12:19:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56125) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uy7-0000qy-6V for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uy5-0003Xs-Mp for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:19 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53544) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uxy-0003Nu-GD; Fri, 06 Oct 2017 11:55:10 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 815CB7C842; Fri, 6 Oct 2017 15:55:09 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9064367598; Fri, 6 Oct 2017 15:55:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 815CB7C842 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:55 +0200 Message-Id: <20171006155422.10135-28-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 06 Oct 2017 15:55:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 27/54] qemu-iotests: limit non-_PROG-suffixed variables to common.rc X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini These are never used by "check", with one exception that does not need $QEMU_OPTIONS. Keep them in common.rc, which will be soon included only by the tests. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/039.out | 10 +++--- tests/qemu-iotests/061.out | 4 +-- tests/qemu-iotests/137.out | 2 +- tests/qemu-iotests/common.config | 69 ++----------------------------------= ---- tests/qemu-iotests/common.rc | 65 +++++++++++++++++++++++++++++++++++++ 5 files changed, 75 insertions(+), 75 deletions(-) diff --git a/tests/qemu-iotests/039.out b/tests/qemu-iotests/039.out index c6e0ac2da3..724d7b2508 100644 --- a/tests/qemu-iotests/039.out +++ b/tests/qemu-iotests/039.out @@ -11,7 +11,7 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134217728 wrote 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -./common.config: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D = "y" ]; then +./common.rc: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D "y" = ]; then exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitcode=3D99= "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; else exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; @@ -50,7 +50,7 @@ read 512/512 bytes at offset 0 Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134217728 wrote 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -./common.config: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D = "y" ]; then +./common.rc: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D "y" = ]; then exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitcode=3D99= "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; else exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; @@ -68,7 +68,7 @@ incompatible_features 0x0 Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134217728 wrote 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -./common.config: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D = "y" ]; then +./common.rc: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D "y" = ]; then exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitcode=3D99= "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; else exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; @@ -91,7 +91,7 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134217728 wrote 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -./common.config: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D = "y" ]; then +./common.rc: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D "y" = ]; then exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitcode=3D99= "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; else exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; @@ -105,7 +105,7 @@ Data may be corrupted, or further writes to the image m= ay corrupt it. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D134217728 wrote 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -./common.config: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D = "y" ]; then +./common.rc: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D "y" = ]; then exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitcode=3D99= "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; else exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index a431b7f305..942485de99 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -57,7 +57,7 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -./common.config: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D = "y" ]; then +./common.rc: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D "y" = ]; then exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitcode=3D99= "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; else exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; @@ -219,7 +219,7 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -./common.config: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D = "y" ]; then +./common.rc: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D "y" = ]; then exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitcode=3D99= "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; else exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; diff --git a/tests/qemu-iotests/137.out b/tests/qemu-iotests/137.out index c0e753483b..05efd74d17 100644 --- a/tests/qemu-iotests/137.out +++ b/tests/qemu-iotests/137.out @@ -31,7 +31,7 @@ Cache clean interval too big Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are an= y of the following: none, constant, cached, all wrote 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -./common.config: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D = "y" ]; then +./common.rc: Killed ( if [ "${VALGRIND_QEMU}" =3D=3D "y" = ]; then exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitcode=3D99= "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; else exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@"; diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.c= onfig index c97c63983f..c7f0a7a7e0 100644 --- a/tests/qemu-iotests/common.config +++ b/tests/qemu-iotests/common.config @@ -40,71 +40,6 @@ _optstr_add() fi } =20 -_qemu_wrapper() -{ - ( - if [ -n "${QEMU_NEED_PID}" ]; then - echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid" - fi - exec "$QEMU_PROG" $QEMU_OPTIONS "$@" - ) -} - -_qemu_img_wrapper() -{ - (exec "$QEMU_IMG_PROG" $QEMU_IMG_OPTIONS "$@") -} - -_qemu_io_wrapper() -{ - local VALGRIND_LOGFILE=3D"${TEST_DIR}"/$$.valgrind - local QEMU_IO_ARGS=3D"$QEMU_IO_OPTIONS" - if [ "$IMGOPTSSYNTAX" =3D "true" ]; then - QEMU_IO_ARGS=3D"--image-opts $QEMU_IO_ARGS" - if [ -n "$IMGKEYSECRET" ]; then - QEMU_IO_ARGS=3D"--object secret,id=3Dkeysec0,data=3D$IMGKEYSEC= RET $QEMU_IO_ARGS" - fi - fi - local RETVAL - ( - if [ "${VALGRIND_QEMU}" =3D=3D "y" ]; then - exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitc= ode=3D99 "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" - else - exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" - fi - ) - RETVAL=3D$? - if [ "${VALGRIND_QEMU}" =3D=3D "y" ]; then - if [ $RETVAL =3D=3D 99 ]; then - cat "${VALGRIND_LOGFILE}" - fi - rm -f "${VALGRIND_LOGFILE}" - fi - (exit $RETVAL) -} - -_qemu_nbd_wrapper() -{ - ( - echo $BASHPID > "${QEMU_TEST_DIR}/qemu-nbd.pid" - exec "$QEMU_NBD_PROG" $QEMU_NBD_OPTIONS "$@" - ) -} - -_qemu_vxhs_wrapper() -{ - ( - echo $BASHPID > "${TEST_DIR}/qemu-vxhs.pid" - exec "$QEMU_VXHS_PROG" $QEMU_VXHS_OPTIONS "$@" - ) -} - -export QEMU=3D_qemu_wrapper -export QEMU_IMG=3D_qemu_img_wrapper -export QEMU_IO=3D_qemu_io_wrapper -export QEMU_NBD=3D_qemu_nbd_wrapper -export QEMU_VXHS=3D_qemu_vxhs_wrapper - QEMU_IMG_EXTRA_ARGS=3D if [ "$IMGOPTSSYNTAX" =3D "true" ]; then QEMU_IMG_EXTRA_ARGS=3D"--image-opts $QEMU_IMG_EXTRA_ARGS" @@ -115,8 +50,8 @@ fi export QEMU_IMG_EXTRA_ARGS =20 =20 -default_machine=3D$($QEMU -machine help | sed -n '/(default)/ s/ .*//p') -default_alias_machine=3D$($QEMU -machine help | \ +default_machine=3D$($QEMU_PROG -machine help | sed -n '/(default)/ s/ .*//= p') +default_alias_machine=3D$($QEMU_PROG -machine help | \ sed -n "/(alias of $default_machine)/ { s/ .*//p; q; }") if [[ "$default_alias_machine" ]]; then default_machine=3D"$default_alias_machine" diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 6f6e03366f..e7f74b4dbd 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -50,6 +50,71 @@ then fi fi =20 +_qemu_wrapper() +{ + ( + if [ -n "${QEMU_NEED_PID}" ]; then + echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid" + fi + exec "$QEMU_PROG" $QEMU_OPTIONS "$@" + ) +} + +_qemu_img_wrapper() +{ + (exec "$QEMU_IMG_PROG" $QEMU_IMG_OPTIONS "$@") +} + +_qemu_io_wrapper() +{ + local VALGRIND_LOGFILE=3D"${TEST_DIR}"/$$.valgrind + local QEMU_IO_ARGS=3D"$QEMU_IO_OPTIONS" + if [ "$IMGOPTSSYNTAX" =3D "true" ]; then + QEMU_IO_ARGS=3D"--image-opts $QEMU_IO_ARGS" + if [ -n "$IMGKEYSECRET" ]; then + QEMU_IO_ARGS=3D"--object secret,id=3Dkeysec0,data=3D$IMGKEYSEC= RET $QEMU_IO_ARGS" + fi + fi + local RETVAL + ( + if [ "${VALGRIND_QEMU}" =3D=3D "y" ]; then + exec valgrind --log-file=3D"${VALGRIND_LOGFILE}" --error-exitc= ode=3D99 "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" + else + exec "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" + fi + ) + RETVAL=3D$? + if [ "${VALGRIND_QEMU}" =3D=3D "y" ]; then + if [ $RETVAL =3D=3D 99 ]; then + cat "${VALGRIND_LOGFILE}" + fi + rm -f "${VALGRIND_LOGFILE}" + fi + (exit $RETVAL) +} + +_qemu_nbd_wrapper() +{ + ( + echo $BASHPID > "${QEMU_TEST_DIR}/qemu-nbd.pid" + exec "$QEMU_NBD_PROG" $QEMU_NBD_OPTIONS "$@" + ) +} + +_qemu_vxhs_wrapper() +{ + ( + echo $BASHPID > "${TEST_DIR}/qemu-vxhs.pid" + exec "$QEMU_VXHS_PROG" $QEMU_VXHS_OPTIONS "$@" + ) +} + +export QEMU=3D_qemu_wrapper +export QEMU_IMG=3D_qemu_img_wrapper +export QEMU_IO=3D_qemu_io_wrapper +export QEMU_NBD=3D_qemu_nbd_wrapper +export QEMU_VXHS=3D_qemu_vxhs_wrapper + if [ "$IMGOPTSSYNTAX" =3D "true" ]; then DRIVER=3D"driver=3D$IMGFMT" if [ "$IMGFMT" =3D "luks" ]; then --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15073061511128.443807248484177; Fri, 6 Oct 2017 09:09:11 -0700 (PDT) Received: from localhost ([::1]:45669 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VBT-0003wX-BA for importer@patchew.org; Fri, 06 Oct 2017 12:09:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56241) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyC-0000xq-Vf for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uy7-0003Zs-4l for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:25 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36222) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uy0-0003Rl-Kp; Fri, 06 Oct 2017 11:55:12 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A0B17272CA; Fri, 6 Oct 2017 15:55:11 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id B4010675BC; Fri, 6 Oct 2017 15:55:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A0B17272CA Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:56 +0200 Message-Id: <20171006155422.10135-29-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Fri, 06 Oct 2017 15:55:11 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 28/54] qemu-iotests: do not include common.rc in "check" X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini It only provides functions used by the test programs. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 6 ------ tests/qemu-iotests/common.rc | 13 +++++-------- 2 files changed, 5 insertions(+), 14 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 03871d0681..e39680ade2 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -72,12 +72,6 @@ then _init_error "failed to source common.config" fi =20 -# we need common.rc -if ! . "$source_iotests/common.rc" -then - _init_error "failed to source common.rc" -fi - # we need common . "$source_iotests/common" =20 diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index e7f74b4dbd..20f6821a69 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -40,14 +40,11 @@ poke_file() printf "$3" | dd "of=3D$1" bs=3D1 "seek=3D$2" conv=3Dnotrunc &>/dev/nu= ll } =20 -# we need common.config -if [ "$iam" !=3D "check" ] -then - if ! . ./common.config - then - echo "$iam: failed to source common.config" - exit 1 - fi + +if ! . ./common.config + then + echo "$iam: failed to source common.config" + exit 1 fi =20 _qemu_wrapper() --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306497898122.61557530978871; Fri, 6 Oct 2017 09:14:57 -0700 (PDT) Received: from localhost ([::1]:45696 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VH1-0000ad-0O for importer@patchew.org; Fri, 06 Oct 2017 12:14:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56174) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uy8-0000sl-S6 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uy7-0003b0-LZ for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49990) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uy1-0003Th-V8; Fri, 06 Oct 2017 11:55:14 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E0BAA81DE3; Fri, 6 Oct 2017 15:55:12 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id F260767585; Fri, 6 Oct 2017 15:55:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E0BAA81DE3 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:57 +0200 Message-Id: <20171006155422.10135-30-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Fri, 06 Oct 2017 15:55:13 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 29/54] qemu-iotests: disintegrate more parts of common.config X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini Split "check" parts from tests part. For the directory setup, the actual computation of directories goes in "check", while the sanity checks go in the tests. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/common | 24 ++++++++++++++++++++ tests/qemu-iotests/common.config | 49 ------------------------------------= ---- tests/qemu-iotests/common.qemu | 1 + tests/qemu-iotests/common.rc | 25 ++++++++++++++++++++ 4 files changed, 50 insertions(+), 49 deletions(-) diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common index abacc24114..ee313af92f 100644 --- a/tests/qemu-iotests/common +++ b/tests/qemu-iotests/common @@ -48,6 +48,14 @@ set_prog_path() fi } =20 +if [ -z "$TEST_DIR" ]; then + TEST_DIR=3D`pwd`/scratch +fi + +if [ ! -e "$TEST_DIR" ]; then + mkdir "$TEST_DIR" +fi + diff=3D"diff -u" verbose=3Dfalse debug=3Dfalse @@ -440,6 +448,13 @@ if [ "$IMGFMT" =3D=3D "luks" ] && ! (echo "$IMGOPTS" |= grep "iter-time=3D" > /dev/null IMGOPTS=3D$(_optstr_add "$IMGOPTS" "iter-time=3D10") fi =20 +if [ -z "$SAMPLE_IMG_DIR" ]; then + SAMPLE_IMG_DIR=3D"$source_iotests/sample_images" +fi + +export TEST_DIR +export SAMPLE_IMG_DIR + if [ -s $tmp.list ] then # found some valid test numbers ... this is good @@ -524,3 +539,12 @@ if [ -x "$build_iotests/socket_scm_helper" ] then export SOCKET_SCM_HELPER=3D"$build_iotests/socket_scm_helper" fi + +default_machine=3D$($QEMU_PROG -machine help | sed -n '/(default)/ s/ .*//= p') +default_alias_machine=3D$($QEMU_PROG -machine help | \ + sed -n "/(alias of $default_machine)/ { s/ .*//p; q; }") +if [[ "$default_alias_machine" ]]; then + default_machine=3D"$default_alias_machine" +fi + +export QEMU_DEFAULT_MACHINE=3D"$default_machine" diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.c= onfig index c7f0a7a7e0..cdcda54546 100644 --- a/tests/qemu-iotests/common.config +++ b/tests/qemu-iotests/common.config @@ -26,8 +26,6 @@ arch=3D`uname -m` =20 export PWD=3D`pwd` =20 -export _QEMU_HANDLE=3D0 - # make sure we have a standard umask umask 022 =20 @@ -40,52 +38,5 @@ _optstr_add() fi } =20 -QEMU_IMG_EXTRA_ARGS=3D -if [ "$IMGOPTSSYNTAX" =3D "true" ]; then - QEMU_IMG_EXTRA_ARGS=3D"--image-opts $QEMU_IMG_EXTRA_ARGS" - if [ -n "$IMGKEYSECRET" ]; then - QEMU_IMG_EXTRA_ARGS=3D"--object secret,id=3Dkeysec0,data=3D$IMGKEY= SECRET $QEMU_IMG_EXTRA_ARGS" - fi -fi -export QEMU_IMG_EXTRA_ARGS - - -default_machine=3D$($QEMU_PROG -machine help | sed -n '/(default)/ s/ .*//= p') -default_alias_machine=3D$($QEMU_PROG -machine help | \ - sed -n "/(alias of $default_machine)/ { s/ .*//p; q; }") -if [[ "$default_alias_machine" ]]; then - default_machine=3D"$default_alias_machine" -fi - -export QEMU_DEFAULT_MACHINE=3D"$default_machine" - -if [ -z "$TEST_DIR" ]; then - TEST_DIR=3D`pwd`/scratch -fi - -QEMU_TEST_DIR=3D"${TEST_DIR}" - -if [ ! -e "$TEST_DIR" ]; then - mkdir "$TEST_DIR" -fi - -if [ ! -d "$TEST_DIR" ]; then - echo "common.config: Error: \$TEST_DIR ($TEST_DIR) is not a directory" - exit 1 -fi - -export TEST_DIR - -if [ -z "$SAMPLE_IMG_DIR" ]; then - SAMPLE_IMG_DIR=3D"$source_iotests/sample_images" -fi - -if [ ! -d "$SAMPLE_IMG_DIR" ]; then - echo "common.config: Error: \$SAMPLE_IMG_DIR ($SAMPLE_IMG_DIR) is not = a directory" - exit 1 -fi - -export SAMPLE_IMG_DIR - # make sure this script returns success true diff --git a/tests/qemu-iotests/common.qemu b/tests/qemu-iotests/common.qemu index 7645f1dc72..9f9aecc9df 100644 --- a/tests/qemu-iotests/common.qemu +++ b/tests/qemu-iotests/common.qemu @@ -31,6 +31,7 @@ QEMU_FIFO_IN=3D"${QEMU_TEST_DIR}/qmp-in-$$" QEMU_FIFO_OUT=3D"${QEMU_TEST_DIR}/qmp-out-$$" =20 QEMU_HANDLE=3D0 +export _QEMU_HANDLE=3D0 =20 # If bash version is >=3D 4.1, these will be overwritten and dynamic # file descriptor values assigned. diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 20f6821a69..f4dc0104e6 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -114,6 +114,10 @@ export QEMU_VXHS=3D_qemu_vxhs_wrapper =20 if [ "$IMGOPTSSYNTAX" =3D "true" ]; then DRIVER=3D"driver=3D$IMGFMT" + QEMU_IMG_EXTRA_ARGS=3D"--image-opts $QEMU_IMG_EXTRA_ARGS" + if [ -n "$IMGKEYSECRET" ]; then + QEMU_IMG_EXTRA_ARGS=3D"--object secret,id=3Dkeysec0,data=3D$IMGKEY= SECRET $QEMU_IMG_EXTRA_ARGS" + fi if [ "$IMGFMT" =3D "luks" ]; then DRIVER=3D"$DRIVER,key-secret=3Dkeysec0" fi @@ -133,6 +137,7 @@ if [ "$IMGOPTSSYNTAX" =3D "true" ]; then TEST_IMG=3D"$DRIVER,file.driver=3D$IMGPROTO,file.filename=3D$TEST_= DIR/t.$IMGFMT" fi else + QEMU_IMG_EXTRA_ARGS=3D if [ "$IMGPROTO" =3D "file" ]; then TEST_IMG=3D$TEST_DIR/t.$IMGFMT elif [ "$IMGPROTO" =3D "nbd" ]; then @@ -153,6 +158,26 @@ else fi ORIG_TEST_IMG=3D"$TEST_IMG" =20 +if [ -z "$TEST_DIR" ]; then + TEST_DIR=3D`pwd`/scratch +fi + +QEMU_TEST_DIR=3D"${TEST_DIR}" + +if [ ! -e "$TEST_DIR" ]; then + mkdir "$TEST_DIR" +fi + +if [ ! -d "$TEST_DIR" ]; then + echo "common.config: Error: \$TEST_DIR ($TEST_DIR) is not a directory" + exit 1 +fi + +if [ ! -d "$SAMPLE_IMG_DIR" ]; then + echo "common.config: Error: \$SAMPLE_IMG_DIR ($SAMPLE_IMG_DIR) is not = a directory" + exit 1 +fi + _use_sample_img() { SAMPLE_IMG_FILE=3D"${1%\.bz2}" --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307103860942.6997298484375; Fri, 6 Oct 2017 09:25:03 -0700 (PDT) Received: from localhost ([::1]:45759 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VQl-0001Mr-3d for importer@patchew.org; Fri, 06 Oct 2017 12:24:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56147) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uy7-0000rk-Vf for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uy7-0003Zz-4n for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36296) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uy3-0003Ux-5v; Fri, 06 Oct 2017 11:55:15 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2BBDA5F7AE; Fri, 6 Oct 2017 15:55:14 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D78167598; Fri, 6 Oct 2017 15:55:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 2BBDA5F7AE Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:58 +0200 Message-Id: <20171006155422.10135-31-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Fri, 06 Oct 2017 15:55:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 30/54] qemu-iotests: fix uninitialized variable X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini The variable is used in "common" but defined only after the file is sourced. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 2 -- tests/qemu-iotests/common | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index e39680ade2..2d2ef687ad 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -77,8 +77,6 @@ fi =20 TIMESTAMP_FILE=3Dcheck.time-$IMGPROTO-$IMGFMT =20 -tmp=3D"${TEST_DIR}"/$$ - _wallclock() { date "+%H %M %S" | awk '{ print $1*3600 + $2*60 + $3 }' diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common index ee313af92f..365d3c4349 100644 --- a/tests/qemu-iotests/common +++ b/tests/qemu-iotests/common @@ -67,6 +67,8 @@ sortme=3Dfalse expunge=3Dtrue have_test_arg=3Dfalse cachemode=3Dfalse + +tmp=3D"${TEST_DIR}"/$$ rm -f $tmp.list $tmp.tmp $tmp.sed =20 export IMGFMT=3Draw --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306657263184.93445707598914; Fri, 6 Oct 2017 09:17:37 -0700 (PDT) Received: from localhost ([::1]:45714 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VJa-0002vq-Fw for importer@patchew.org; Fri, 06 Oct 2017 12:17:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56143) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uy7-0000rh-Sy for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uy7-0003a4-53 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:19 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58476) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uy4-0003WB-Ii; Fri, 06 Oct 2017 11:55:16 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 87F983E2A2; Fri, 6 Oct 2017 15:55:15 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7DF2B67585; Fri, 6 Oct 2017 15:55:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 87F983E2A2 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:53:59 +0200 Message-Id: <20171006155422.10135-32-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 06 Oct 2017 15:55:15 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 31/54] qemu-iotests: get rid of $iam X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini The variable is almost unused, and one of the two uses is actually uninitialized. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 5 +---- tests/qemu-iotests/common.rc | 2 +- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 2d2ef687ad..010bf48e92 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -30,12 +30,9 @@ interrupt=3Dtrue # by default don't output timestamps timestamp=3D${TIMESTAMP:=3Dfalse} =20 -# generic initialization -iam=3Dcheck - _init_error() { - echo "$iam: $1" >&2 + echo "check: $1" >&2 exit 1 } =20 diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index f4dc0104e6..0e8a33c696 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -43,7 +43,7 @@ poke_file() =20 if ! . ./common.config then - echo "$iam: failed to source common.config" + echo "$0: failed to source common.config" exit 1 fi =20 --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306984329900.790213565087; Fri, 6 Oct 2017 09:23:04 -0700 (PDT) Received: from localhost ([::1]:45744 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VOr-00085I-6k for importer@patchew.org; Fri, 06 Oct 2017 12:22:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56295) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyG-00012D-V7 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyC-0003iM-W5 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49169) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uy6-0003XT-0m; Fri, 06 Oct 2017 11:55:18 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F2E691E33C; Fri, 6 Oct 2017 15:55:16 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id D9CA367594; Fri, 6 Oct 2017 15:55:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com F2E691E33C Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:00 +0200 Message-Id: <20171006155422.10135-33-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 06 Oct 2017 15:55:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 32/54] qemu-iotests: merge "check" and "common" X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini "check" is full of qemu-iotests--specific details. Separating it from "common" does not make much sense anymore. Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 533 ++++++++++++++++++++++++++++++++++++++++++= +- tests/qemu-iotests/common | 552 ------------------------------------------= ---- 2 files changed, 531 insertions(+), 554 deletions(-) delete mode 100644 tests/qemu-iotests/common diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 010bf48e92..176cb8e937 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -69,8 +69,537 @@ then _init_error "failed to source common.config" fi =20 -# we need common -. "$source_iotests/common" +_full_imgfmt_details() +{ + if [ -n "$IMGOPTS" ]; then + echo "$IMGFMT ($IMGOPTS)" + else + echo "$IMGFMT" + fi +} + +_full_platform_details() +{ + os=3D`uname -s` + host=3D`hostname -s` + kernel=3D`uname -r` + platform=3D`uname -m` + echo "$os/$platform $host $kernel" +} + +# $1 =3D prog to look for +set_prog_path() +{ + p=3D`command -v $1 2> /dev/null` + if [ -n "$p" -a -x "$p" ]; then + realpath -- "$(type -p "$p")" + else + return 1 + fi +} + +if [ -z "$TEST_DIR" ]; then + TEST_DIR=3D`pwd`/scratch +fi + +if [ ! -e "$TEST_DIR" ]; then + mkdir "$TEST_DIR" +fi + +diff=3D"diff -u" +verbose=3Dfalse +debug=3Dfalse +group=3Dfalse +xgroup=3Dfalse +imgopts=3Dfalse +showme=3Dfalse +sortme=3Dfalse +expunge=3Dtrue +have_test_arg=3Dfalse +cachemode=3Dfalse + +tmp=3D"${TEST_DIR}"/$$ +rm -f $tmp.list $tmp.tmp $tmp.sed + +export IMGFMT=3Draw +export IMGFMT_GENERIC=3Dtrue +export IMGPROTO=3Dfile +export IMGOPTS=3D"" +export CACHEMODE=3D"writeback" +export QEMU_IO_OPTIONS=3D"" +export QEMU_IO_OPTIONS_NO_FMT=3D"" +export CACHEMODE_IS_DEFAULT=3Dtrue +export QEMU_OPTIONS=3D"-nodefaults -machine accel=3Dqtest" +export VALGRIND_QEMU=3D +export IMGKEYSECRET=3D +export IMGOPTSSYNTAX=3Dfalse + +for r +do + + if $group + then + # arg after -g + group_list=3D`sed -n <"$source_iotests/group" -e 's/$/ /' -e "/^[0= -9][0-9][0-9].* $r /"'{ +s/ .*//p +}'` + if [ -z "$group_list" ] + then + echo "Group \"$r\" is empty or not defined?" + exit 1 + fi + [ ! -s $tmp.list ] && touch $tmp.list + for t in $group_list + do + if grep -s "^$t\$" $tmp.list >/dev/null + then + : + else + echo "$t" >>$tmp.list + fi + done + group=3Dfalse + continue + + elif $xgroup + then + # arg after -x + # Populate $tmp.list with all tests + awk '/^[0-9]{3,}/ {print $1}' "${source_iotests}/group" > $tmp.lis= t 2>/dev/null + group_list=3D`sed -n <"$source_iotests/group" -e 's/$/ /' -e "/^[0= -9][0-9][0-9].* $r /"'{ +s/ .*//p +}'` + if [ -z "$group_list" ] + then + echo "Group \"$r\" is empty or not defined?" + exit 1 + fi + numsed=3D0 + rm -f $tmp.sed + for t in $group_list + do + if [ $numsed -gt 100 ] + then + sed -f $tmp.sed <$tmp.list >$tmp.tmp + mv $tmp.tmp $tmp.list + numsed=3D0 + rm -f $tmp.sed + fi + echo "/^$t\$/d" >>$tmp.sed + numsed=3D`expr $numsed + 1` + done + sed -f $tmp.sed <$tmp.list >$tmp.tmp + mv $tmp.tmp $tmp.list + xgroup=3Dfalse + continue + + elif $imgopts + then + IMGOPTS=3D"$r" + imgopts=3Dfalse + continue + elif $cachemode + then + CACHEMODE=3D"$r" + CACHEMODE_IS_DEFAULT=3Dfalse + cachemode=3Dfalse + continue + fi + + xpand=3Dtrue + case "$r" + in + + -\? | -h | --help) # usage + echo "Usage: $0 [options] [testlist]"' + +common options + -v verbose + -d debug + +image format options + -raw test raw (default) + -bochs test bochs + -cloop test cloop + -parallels test parallels + -qcow test qcow + -qcow2 test qcow2 + -qed test qed + -vdi test vdi + -vpc test vpc + -vhdx test vhdx + -vmdk test vmdk + -luks test luks + +image protocol options + -file test file (default) + -rbd test rbd + -sheepdog test sheepdog + -nbd test nbd + -ssh test ssh + -nfs test nfs + -vxhs test vxhs + +other options + -xdiff graphical mode diff + -nocache use O_DIRECT on backing file + -misalign misalign memory allocations + -n show me, do not run tests + -o options -o options to pass to qemu-img create/convert + -T output timestamps + -c mode cache mode + +testlist options + -g group[,group...] include tests from these groups + -x group[,group...] exclude tests from these groups + NNN include test NNN + NNN-NNN include test range (eg. 012-021) +' + exit 0 + ;; + + -raw) + IMGFMT=3Draw + xpand=3Dfalse + ;; + + -bochs) + IMGFMT=3Dbochs + IMGFMT_GENERIC=3Dfalse + xpand=3Dfalse + ;; + + -cloop) + IMGFMT=3Dcloop + IMGFMT_GENERIC=3Dfalse + xpand=3Dfalse + ;; + + -parallels) + IMGFMT=3Dparallels + IMGFMT_GENERIC=3Dfalse + xpand=3Dfalse + ;; + + -qcow) + IMGFMT=3Dqcow + xpand=3Dfalse + ;; + + -qcow2) + IMGFMT=3Dqcow2 + xpand=3Dfalse + ;; + + -luks) + IMGOPTSSYNTAX=3Dtrue + IMGFMT=3Dluks + IMGKEYSECRET=3D123456 + xpand=3Dfalse + ;; + + -qed) + IMGFMT=3Dqed + xpand=3Dfalse + ;; + + -vdi) + IMGFMT=3Dvdi + xpand=3Dfalse + ;; + + -vmdk) + IMGFMT=3Dvmdk + xpand=3Dfalse + ;; + + -vpc) + IMGFMT=3Dvpc + xpand=3Dfalse + ;; + + -vhdx) + IMGFMT=3Dvhdx + xpand=3Dfalse + ;; + + -file) + IMGPROTO=3Dfile + xpand=3Dfalse + ;; + + -rbd) + IMGPROTO=3Drbd + xpand=3Dfalse + ;; + + -sheepdog) + IMGPROTO=3Dsheepdog + xpand=3Dfalse + ;; + + -nbd) + IMGPROTO=3Dnbd + xpand=3Dfalse + ;; + + -vxhs) + IMGPROTO=3Dvxhs + xpand=3Dfalse + ;; + + -ssh) + IMGPROTO=3Dssh + xpand=3Dfalse + ;; + + -nfs) + IMGPROTO=3Dnfs + xpand=3Dfalse + ;; + + -nocache) + CACHEMODE=3D"none" + CACHEMODE_IS_DEFAULT=3Dfalse + xpand=3Dfalse + ;; + + -misalign) + QEMU_IO_OPTIONS=3D"$QEMU_IO_OPTIONS --misalign" + xpand=3Dfalse + ;; + + -valgrind) + VALGRIND_QEMU=3D'y' + xpand=3Dfalse + ;; + + -g) # -g group ... pick from group file + group=3Dtrue + xpand=3Dfalse + ;; + + -xdiff) # graphical diff mode + xpand=3Dfalse + + if [ ! -z "$DISPLAY" ] + then + command -v xdiff >/dev/null 2>&1 && diff=3Dxdiff + command -v gdiff >/dev/null 2>&1 && diff=3Dgdiff + command -v tkdiff >/dev/null 2>&1 && diff=3Dtkdiff + command -v xxdiff >/dev/null 2>&1 && diff=3Dxxdiff + fi + ;; + + -n) # show me, don't do it + showme=3Dtrue + xpand=3Dfalse + ;; + -o) + imgopts=3Dtrue + xpand=3Dfalse + ;; + -c) + cachemode=3Dtrue + xpand=3Dfalse + ;; + -T) # turn on timestamp output + timestamp=3Dtrue + xpand=3Dfalse + ;; + + -v) + verbose=3Dtrue + xpand=3Dfalse + ;; + -d) + debug=3Dtrue + xpand=3Dfalse + ;; + -x) # -x group ... exclude from group file + xgroup=3Dtrue + xpand=3Dfalse + ;; + '[0-9][0-9][0-9] [0-9][0-9][0-9][0-9]') + echo "No tests?" + status=3D1 + exit $status + ;; + + [0-9]*-[0-9]*) + eval `echo $r | sed -e 's/^/start=3D/' -e 's/-/ end=3D/'` + ;; + + [0-9]*-) + eval `echo $r | sed -e 's/^/start=3D/' -e 's/-//'` + end=3D`echo [0-9][0-9][0-9] [0-9][0-9][0-9][0-9] | sed -e 's/\= [0-9]//g' -e 's/ *$//' -e 's/.* //'` + if [ -z "$end" ] + then + echo "No tests in range \"$r\"?" + status=3D1 + exit $status + fi + ;; + + *) + start=3D$r + end=3D$r + ;; + + esac + + # get rid of leading 0s as can be interpreted as octal + start=3D`echo $start | sed 's/^0*//'` + end=3D`echo $end | sed 's/^0*//'` + + if $xpand + then + have_test_arg=3Dtrue + awk /dev/null + then + # in group file ... OK + echo $id >>$tmp.list + else + if [ -f expunged ] && $expunge && egrep "^$id([ ]|= \$)" expunged >/dev/null + then + # expunged ... will be reported, but not run, later + echo $id >>$tmp.list + else + # oops + if [ "$start" =3D=3D "$end" -a "$id" =3D=3D "$end" ] + then + echo "$id - unknown test" + exit 1 + else + echo "$id - unknown test, ignored" + fi + fi + fi + done || exit 1 + fi + +done + +# Set qemu-io cache mode with $CACHEMODE we have +QEMU_IO_OPTIONS=3D"$QEMU_IO_OPTIONS --cache $CACHEMODE" + +QEMU_IO_OPTIONS_NO_FMT=3D"$QEMU_IO_OPTIONS" +if [ "$IMGOPTSSYNTAX" !=3D "true" ]; then + QEMU_IO_OPTIONS=3D"$QEMU_IO_OPTIONS -f $IMGFMT" +fi + +# Set default options for qemu-img create -o if they were not specified +if [ "$IMGFMT" =3D=3D "qcow2" ] && ! (echo "$IMGOPTS" | grep "compat=3D" >= /dev/null); then + IMGOPTS=3D$(_optstr_add "$IMGOPTS" "compat=3D1.1") +fi +if [ "$IMGFMT" =3D=3D "luks" ] && ! (echo "$IMGOPTS" | grep "iter-time=3D"= > /dev/null); then + IMGOPTS=3D$(_optstr_add "$IMGOPTS" "iter-time=3D10") +fi + +if [ -z "$SAMPLE_IMG_DIR" ]; then + SAMPLE_IMG_DIR=3D"$source_iotests/sample_images" +fi + +export TEST_DIR +export SAMPLE_IMG_DIR + +if [ -s $tmp.list ] +then + # found some valid test numbers ... this is good + : +else + if $have_test_arg + then + # had test numbers, but none in group file ... do nothing + touch $tmp.list + else + # no test numbers, do everything from group file + sed -n -e '/^[0-9][0-9][0-9]*/s/[ ].*//p' <"$source_iotest= s/group" >$tmp.list + fi +fi + +# should be sort -n, but this did not work for Linux when this +# was ported from IRIX +# +list=3D`sort $tmp.list` +rm -f $tmp.list $tmp.tmp $tmp.sed + +if [ -z "$QEMU_PROG" ] +then + if [ -x "$build_iotests/qemu" ]; then + export QEMU_PROG=3D"$build_iotests/qemu" + elif [ -x "$build_root/$arch-softmmu/qemu-system-$arch" ]; then + export QEMU_PROG=3D"$build_root/$arch-softmmu/qemu-system-$arch" + else + pushd "$build_root" > /dev/null + for binary in *-softmmu/qemu-system-* + do + if [ -x "$binary" ] + then + export QEMU_PROG=3D"$build_root/$binary" + break + fi + done + popd > /dev/null + [ "$QEMU_PROG" =3D "" ] && _init_error "qemu not found" + fi +fi +export QEMU_PROG=3D$(realpath -- "$(type -p "$QEMU_PROG")") + +if [ -z "$QEMU_IMG_PROG" ]; then + if [ -x "$build_iotests/qemu-img" ]; then + export QEMU_IMG_PROG=3D"$build_iotests/qemu-img" + elif [ -x "$build_root/qemu-img" ]; then + export QEMU_IMG_PROG=3D"$build_root/qemu-img" + else + _init_error "qemu-img not found" + fi +fi +export QEMU_IMG_PROG=3D$(realpath -- "$(type -p "$QEMU_IMG_PROG")") + +if [ -z "$QEMU_IO_PROG" ]; then + if [ -x "$build_iotests/qemu-io" ]; then + export QEMU_IO_PROG=3D"$build_iotests/qemu-io" + elif [ -x "$build_root/qemu-io" ]; then + export QEMU_IO_PROG=3D"$build_root/qemu-io" + else + _init_error "qemu-io not found" + fi +fi +export QEMU_IO_PROG=3D$(realpath -- "$(type -p "$QEMU_IO_PROG")") + +if [ -z $QEMU_NBD_PROG ]; then + if [ -x "$build_iotests/qemu-nbd" ]; then + export QEMU_NBD_PROG=3D"$build_iotests/qemu-nbd" + elif [ -x "$build_root/qemu-nbd" ]; then + export QEMU_NBD_PROG=3D"$build_root/qemu-nbd" + else + _init_error "qemu-nbd not found" + fi +fi +export QEMU_NBD_PROG=3D$(realpath -- "$(type -p "$QEMU_NBD_PROG")") + +if [ -z "$QEMU_VXHS_PROG" ]; then + export QEMU_VXHS_PROG=3D"`set_prog_path qnio_server`" +fi + +if [ -x "$build_iotests/socket_scm_helper" ] +then + export SOCKET_SCM_HELPER=3D"$build_iotests/socket_scm_helper" +fi + +default_machine=3D$($QEMU_PROG -machine help | sed -n '/(default)/ s/ .*//= p') +default_alias_machine=3D$($QEMU_PROG -machine help | \ + sed -n "/(alias of $default_machine)/ { s/ .*//p; q; }") +if [[ "$default_alias_machine" ]]; then + default_machine=3D"$default_alias_machine" +fi + +export QEMU_DEFAULT_MACHINE=3D"$default_machine" =20 TIMESTAMP_FILE=3Dcheck.time-$IMGPROTO-$IMGFMT =20 diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common deleted file mode 100644 index 365d3c4349..0000000000 --- a/tests/qemu-iotests/common +++ /dev/null @@ -1,552 +0,0 @@ -#!/bin/bash -# -# Copyright (C) 2009 Red Hat, Inc. -# Copyright (c) 2000-2005 Silicon Graphics, Inc. All Rights Reserved. -# -# This program is free software; you can redistribute it and/or -# modify it under the terms of the GNU General Public License as -# published by the Free Software Foundation. -# -# This program is distributed in the hope that it would be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . -# -# -# common procedures for QA scripts -# - -_full_imgfmt_details() -{ - if [ -n "$IMGOPTS" ]; then - echo "$IMGFMT ($IMGOPTS)" - else - echo "$IMGFMT" - fi -} - -_full_platform_details() -{ - os=3D`uname -s` - host=3D`hostname -s` - kernel=3D`uname -r` - platform=3D`uname -m` - echo "$os/$platform $host $kernel" -} - -# $1 =3D prog to look for -set_prog_path() -{ - p=3D`command -v $1 2> /dev/null` - if [ -n "$p" -a -x "$p" ]; then - realpath -- "$(type -p "$p")" - else - return 1 - fi -} - -if [ -z "$TEST_DIR" ]; then - TEST_DIR=3D`pwd`/scratch -fi - -if [ ! -e "$TEST_DIR" ]; then - mkdir "$TEST_DIR" -fi - -diff=3D"diff -u" -verbose=3Dfalse -debug=3Dfalse -group=3Dfalse -xgroup=3Dfalse -imgopts=3Dfalse -showme=3Dfalse -sortme=3Dfalse -expunge=3Dtrue -have_test_arg=3Dfalse -cachemode=3Dfalse - -tmp=3D"${TEST_DIR}"/$$ -rm -f $tmp.list $tmp.tmp $tmp.sed - -export IMGFMT=3Draw -export IMGFMT_GENERIC=3Dtrue -export IMGPROTO=3Dfile -export IMGOPTS=3D"" -export CACHEMODE=3D"writeback" -export QEMU_IO_OPTIONS=3D"" -export QEMU_IO_OPTIONS_NO_FMT=3D"" -export CACHEMODE_IS_DEFAULT=3Dtrue -export QEMU_OPTIONS=3D"-nodefaults -machine accel=3Dqtest" -export VALGRIND_QEMU=3D -export IMGKEYSECRET=3D -export IMGOPTSSYNTAX=3Dfalse - -for r -do - - if $group - then - # arg after -g - group_list=3D`sed -n <"$source_iotests/group" -e 's/$/ /' -e "/^[0= -9][0-9][0-9].* $r /"'{ -s/ .*//p -}'` - if [ -z "$group_list" ] - then - echo "Group \"$r\" is empty or not defined?" - exit 1 - fi - [ ! -s $tmp.list ] && touch $tmp.list - for t in $group_list - do - if grep -s "^$t\$" $tmp.list >/dev/null - then - : - else - echo "$t" >>$tmp.list - fi - done - group=3Dfalse - continue - - elif $xgroup - then - # arg after -x - # Populate $tmp.list with all tests - awk '/^[0-9]{3,}/ {print $1}' "${source_iotests}/group" > $tmp.lis= t 2>/dev/null - group_list=3D`sed -n <"$source_iotests/group" -e 's/$/ /' -e "/^[0= -9][0-9][0-9].* $r /"'{ -s/ .*//p -}'` - if [ -z "$group_list" ] - then - echo "Group \"$r\" is empty or not defined?" - exit 1 - fi - numsed=3D0 - rm -f $tmp.sed - for t in $group_list - do - if [ $numsed -gt 100 ] - then - sed -f $tmp.sed <$tmp.list >$tmp.tmp - mv $tmp.tmp $tmp.list - numsed=3D0 - rm -f $tmp.sed - fi - echo "/^$t\$/d" >>$tmp.sed - numsed=3D`expr $numsed + 1` - done - sed -f $tmp.sed <$tmp.list >$tmp.tmp - mv $tmp.tmp $tmp.list - xgroup=3Dfalse - continue - - elif $imgopts - then - IMGOPTS=3D"$r" - imgopts=3Dfalse - continue - elif $cachemode - then - CACHEMODE=3D"$r" - CACHEMODE_IS_DEFAULT=3Dfalse - cachemode=3Dfalse - continue - fi - - xpand=3Dtrue - case "$r" - in - - -\? | -h | --help) # usage - echo "Usage: $0 [options] [testlist]"' - -common options - -v verbose - -d debug - -image format options - -raw test raw (default) - -bochs test bochs - -cloop test cloop - -parallels test parallels - -qcow test qcow - -qcow2 test qcow2 - -qed test qed - -vdi test vdi - -vpc test vpc - -vhdx test vhdx - -vmdk test vmdk - -luks test luks - -image protocol options - -file test file (default) - -rbd test rbd - -sheepdog test sheepdog - -nbd test nbd - -ssh test ssh - -nfs test nfs - -vxhs test vxhs - -other options - -xdiff graphical mode diff - -nocache use O_DIRECT on backing file - -misalign misalign memory allocations - -n show me, do not run tests - -o options -o options to pass to qemu-img create/convert - -T output timestamps - -c mode cache mode - -testlist options - -g group[,group...] include tests from these groups - -x group[,group...] exclude tests from these groups - NNN include test NNN - NNN-NNN include test range (eg. 012-021) -' - exit 0 - ;; - - -raw) - IMGFMT=3Draw - xpand=3Dfalse - ;; - - -bochs) - IMGFMT=3Dbochs - IMGFMT_GENERIC=3Dfalse - xpand=3Dfalse - ;; - - -cloop) - IMGFMT=3Dcloop - IMGFMT_GENERIC=3Dfalse - xpand=3Dfalse - ;; - - -parallels) - IMGFMT=3Dparallels - IMGFMT_GENERIC=3Dfalse - xpand=3Dfalse - ;; - - -qcow) - IMGFMT=3Dqcow - xpand=3Dfalse - ;; - - -qcow2) - IMGFMT=3Dqcow2 - xpand=3Dfalse - ;; - - -luks) - IMGOPTSSYNTAX=3Dtrue - IMGFMT=3Dluks - IMGKEYSECRET=3D123456 - xpand=3Dfalse - ;; - - -qed) - IMGFMT=3Dqed - xpand=3Dfalse - ;; - - -vdi) - IMGFMT=3Dvdi - xpand=3Dfalse - ;; - - -vmdk) - IMGFMT=3Dvmdk - xpand=3Dfalse - ;; - - -vpc) - IMGFMT=3Dvpc - xpand=3Dfalse - ;; - - -vhdx) - IMGFMT=3Dvhdx - xpand=3Dfalse - ;; - - -file) - IMGPROTO=3Dfile - xpand=3Dfalse - ;; - - -rbd) - IMGPROTO=3Drbd - xpand=3Dfalse - ;; - - -sheepdog) - IMGPROTO=3Dsheepdog - xpand=3Dfalse - ;; - - -nbd) - IMGPROTO=3Dnbd - xpand=3Dfalse - ;; - - -vxhs) - IMGPROTO=3Dvxhs - xpand=3Dfalse - ;; - - -ssh) - IMGPROTO=3Dssh - xpand=3Dfalse - ;; - - -nfs) - IMGPROTO=3Dnfs - xpand=3Dfalse - ;; - - -nocache) - CACHEMODE=3D"none" - CACHEMODE_IS_DEFAULT=3Dfalse - xpand=3Dfalse - ;; - - -misalign) - QEMU_IO_OPTIONS=3D"$QEMU_IO_OPTIONS --misalign" - xpand=3Dfalse - ;; - - -valgrind) - VALGRIND_QEMU=3D'y' - xpand=3Dfalse - ;; - - -g) # -g group ... pick from group file - group=3Dtrue - xpand=3Dfalse - ;; - - -xdiff) # graphical diff mode - xpand=3Dfalse - - if [ ! -z "$DISPLAY" ] - then - command -v xdiff >/dev/null 2>&1 && diff=3Dxdiff - command -v gdiff >/dev/null 2>&1 && diff=3Dgdiff - command -v tkdiff >/dev/null 2>&1 && diff=3Dtkdiff - command -v xxdiff >/dev/null 2>&1 && diff=3Dxxdiff - fi - ;; - - -n) # show me, don't do it - showme=3Dtrue - xpand=3Dfalse - ;; - -o) - imgopts=3Dtrue - xpand=3Dfalse - ;; - -c) - cachemode=3Dtrue - xpand=3Dfalse - ;; - -T) # turn on timestamp output - timestamp=3Dtrue - xpand=3Dfalse - ;; - - -v) - verbose=3Dtrue - xpand=3Dfalse - ;; - -d) - debug=3Dtrue - xpand=3Dfalse - ;; - -x) # -x group ... exclude from group file - xgroup=3Dtrue - xpand=3Dfalse - ;; - '[0-9][0-9][0-9] [0-9][0-9][0-9][0-9]') - echo "No tests?" - status=3D1 - exit $status - ;; - - [0-9]*-[0-9]*) - eval `echo $r | sed -e 's/^/start=3D/' -e 's/-/ end=3D/'` - ;; - - [0-9]*-) - eval `echo $r | sed -e 's/^/start=3D/' -e 's/-//'` - end=3D`echo [0-9][0-9][0-9] [0-9][0-9][0-9][0-9] | sed -e 's/\= [0-9]//g' -e 's/ *$//' -e 's/.* //'` - if [ -z "$end" ] - then - echo "No tests in range \"$r\"?" - status=3D1 - exit $status - fi - ;; - - *) - start=3D$r - end=3D$r - ;; - - esac - - # get rid of leading 0s as can be interpreted as octal - start=3D`echo $start | sed 's/^0*//'` - end=3D`echo $end | sed 's/^0*//'` - - if $xpand - then - have_test_arg=3Dtrue - awk /dev/null - then - # in group file ... OK - echo $id >>$tmp.list - else - if [ -f expunged ] && $expunge && egrep "^$id([ ]|= \$)" expunged >/dev/null - then - # expunged ... will be reported, but not run, later - echo $id >>$tmp.list - else - # oops - if [ "$start" =3D=3D "$end" -a "$id" =3D=3D "$end" ] - then - echo "$id - unknown test" - exit 1 - else - echo "$id - unknown test, ignored" - fi - fi - fi - done || exit 1 - fi - -done - -# Set qemu-io cache mode with $CACHEMODE we have -QEMU_IO_OPTIONS=3D"$QEMU_IO_OPTIONS --cache $CACHEMODE" - -QEMU_IO_OPTIONS_NO_FMT=3D"$QEMU_IO_OPTIONS" -if [ "$IMGOPTSSYNTAX" !=3D "true" ]; then - QEMU_IO_OPTIONS=3D"$QEMU_IO_OPTIONS -f $IMGFMT" -fi - -# Set default options for qemu-img create -o if they were not specified -if [ "$IMGFMT" =3D=3D "qcow2" ] && ! (echo "$IMGOPTS" | grep "compat=3D" >= /dev/null); then - IMGOPTS=3D$(_optstr_add "$IMGOPTS" "compat=3D1.1") -fi -if [ "$IMGFMT" =3D=3D "luks" ] && ! (echo "$IMGOPTS" | grep "iter-time=3D"= > /dev/null); then - IMGOPTS=3D$(_optstr_add "$IMGOPTS" "iter-time=3D10") -fi - -if [ -z "$SAMPLE_IMG_DIR" ]; then - SAMPLE_IMG_DIR=3D"$source_iotests/sample_images" -fi - -export TEST_DIR -export SAMPLE_IMG_DIR - -if [ -s $tmp.list ] -then - # found some valid test numbers ... this is good - : -else - if $have_test_arg - then - # had test numbers, but none in group file ... do nothing - touch $tmp.list - else - # no test numbers, do everything from group file - sed -n -e '/^[0-9][0-9][0-9]*/s/[ ].*//p' <"$source_iotest= s/group" >$tmp.list - fi -fi - -# should be sort -n, but this did not work for Linux when this -# was ported from IRIX -# -list=3D`sort $tmp.list` -rm -f $tmp.list $tmp.tmp $tmp.sed - -if [ -z "$QEMU_PROG" ] -then - if [ -x "$build_iotests/qemu" ]; then - export QEMU_PROG=3D"$build_iotests/qemu" - elif [ -x "$build_root/$arch-softmmu/qemu-system-$arch" ]; then - export QEMU_PROG=3D"$build_root/$arch-softmmu/qemu-system-$arch" - else - pushd "$build_root" > /dev/null - for binary in *-softmmu/qemu-system-* - do - if [ -x "$binary" ] - then - export QEMU_PROG=3D"$build_root/$binary" - break - fi - done - popd > /dev/null - [ "$QEMU_PROG" =3D "" ] && _init_error "qemu not found" - fi -fi -export QEMU_PROG=3D$(realpath -- "$(type -p "$QEMU_PROG")") - -if [ -z "$QEMU_IMG_PROG" ]; then - if [ -x "$build_iotests/qemu-img" ]; then - export QEMU_IMG_PROG=3D"$build_iotests/qemu-img" - elif [ -x "$build_root/qemu-img" ]; then - export QEMU_IMG_PROG=3D"$build_root/qemu-img" - else - _init_error "qemu-img not found" - fi -fi -export QEMU_IMG_PROG=3D$(realpath -- "$(type -p "$QEMU_IMG_PROG")") - -if [ -z "$QEMU_IO_PROG" ]; then - if [ -x "$build_iotests/qemu-io" ]; then - export QEMU_IO_PROG=3D"$build_iotests/qemu-io" - elif [ -x "$build_root/qemu-io" ]; then - export QEMU_IO_PROG=3D"$build_root/qemu-io" - else - _init_error "qemu-io not found" - fi -fi -export QEMU_IO_PROG=3D$(realpath -- "$(type -p "$QEMU_IO_PROG")") - -if [ -z $QEMU_NBD_PROG ]; then - if [ -x "$build_iotests/qemu-nbd" ]; then - export QEMU_NBD_PROG=3D"$build_iotests/qemu-nbd" - elif [ -x "$build_root/qemu-nbd" ]; then - export QEMU_NBD_PROG=3D"$build_root/qemu-nbd" - else - _init_error "qemu-nbd not found" - fi -fi -export QEMU_NBD_PROG=3D$(realpath -- "$(type -p "$QEMU_NBD_PROG")") - -if [ -z "$QEMU_VXHS_PROG" ]; then - export QEMU_VXHS_PROG=3D"`set_prog_path qnio_server`" -fi - -if [ -x "$build_iotests/socket_scm_helper" ] -then - export SOCKET_SCM_HELPER=3D"$build_iotests/socket_scm_helper" -fi - -default_machine=3D$($QEMU_PROG -machine help | sed -n '/(default)/ s/ .*//= p') -default_alias_machine=3D$($QEMU_PROG -machine help | \ - sed -n "/(alias of $default_machine)/ { s/ .*//p; q; }") -if [[ "$default_alias_machine" ]]; then - default_machine=3D"$default_alias_machine" -fi - -export QEMU_DEFAULT_MACHINE=3D"$default_machine" --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307259509538.410612348079; Fri, 6 Oct 2017 09:27:39 -0700 (PDT) Received: from localhost ([::1]:45779 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VTE-0003k6-IR for importer@patchew.org; Fri, 06 Oct 2017 12:27:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56267) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyF-00010Y-6u for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyA-0003fL-Tx for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:27 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37850) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uy7-0003Zp-NV; Fri, 06 Oct 2017 11:55:19 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A1465C047B70; Fri, 6 Oct 2017 15:55:18 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5138167588; Fri, 6 Oct 2017 15:55:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A1465C047B70 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:01 +0200 Message-Id: <20171006155422.10135-34-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:55:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 33/54] block: Introduce BdrvChildRole.update_filename X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" There is no good reason for bdrv_drop_intermediate() to know the active layer above the subchain it is operating on - even more so, because the assumption that there is a single active layer above it is not generally true. In order to prepare removal of the active parameter, use a BdrvChildRole callback to update the backing file string in the overlay image instead of directly calling bdrv_change_backing_file(). Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- include/block/block_int.h | 6 ++++++ block.c | 33 ++++++++++++++++++++++++++++----- 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index 79366b94b5..7e8a206239 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -544,6 +544,12 @@ struct BdrvChildRole { =20 void (*attach)(BdrvChild *child); void (*detach)(BdrvChild *child); + + /* Notifies the parent that the filename of its child has changed (e.g. + * because the direct child was removed from the backing chain), so th= at it + * can update its reference. */ + int (*update_filename)(BdrvChild *child, BlockDriverState *new_base, + const char *filename, Error **errp); }; =20 extern const BdrvChildRole child_file; diff --git a/block.c b/block.c index ef5af81f66..ab354ba82a 100644 --- a/block.c +++ b/block.c @@ -981,6 +981,21 @@ static void bdrv_backing_options(int *child_flags, QDi= ct *child_options, *child_flags =3D flags; } =20 +static int bdrv_backing_update_filename(BdrvChild *c, BlockDriverState *ba= se, + const char *filename, Error **errp) +{ + BlockDriverState *parent =3D c->opaque; + int ret; + + ret =3D bdrv_change_backing_file(parent, filename, + base->drv ? base->drv->format_name : ""= ); + if (ret < 0) { + error_setg_errno(errp, ret, "Could not update backing file link"); + } + + return ret; +} + const BdrvChildRole child_backing =3D { .get_parent_desc =3D bdrv_child_get_parent_desc, .attach =3D bdrv_backing_attach, @@ -989,6 +1004,7 @@ const BdrvChildRole child_backing =3D { .drained_begin =3D bdrv_child_cb_drained_begin, .drained_end =3D bdrv_child_cb_drained_end, .inactivate =3D bdrv_child_cb_inactivate, + .update_filename =3D bdrv_backing_update_filename, }; =20 static int bdrv_open_flags(BlockDriverState *bs, int flags) @@ -3470,6 +3486,8 @@ int bdrv_drop_intermediate(BlockDriverState *active, = BlockDriverState *top, Error *local_err =3D NULL; int ret =3D -EIO; =20 + bdrv_ref(top); + if (!top->drv || !base->drv) { goto exit; } @@ -3494,11 +3512,15 @@ int bdrv_drop_intermediate(BlockDriverState *active= , BlockDriverState *top, } =20 /* success - we can delete the intermediate states, and link top->base= */ - backing_file_str =3D backing_file_str ? backing_file_str : base->filen= ame; - ret =3D bdrv_change_backing_file(new_top_bs, backing_file_str, - base->drv ? base->drv->format_name : ""= ); - if (ret) { - goto exit; + if (new_top_bs->backing->role->update_filename) { + backing_file_str =3D backing_file_str ? backing_file_str : base->f= ilename; + ret =3D new_top_bs->backing->role->update_filename(new_top_bs->bac= king, + base, backing_fil= e_str, + &local_err); + if (ret < 0) { + bdrv_set_backing_hd(new_top_bs, top, &error_abort); + goto exit; + } } =20 bdrv_set_backing_hd(new_top_bs, base, &local_err); @@ -3510,6 +3532,7 @@ int bdrv_drop_intermediate(BlockDriverState *active, = BlockDriverState *top, =20 ret =3D 0; exit: + bdrv_unref(top); return ret; } =20 --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306819479223.81570102081332; Fri, 6 Oct 2017 09:20:19 -0700 (PDT) Received: from localhost ([::1]:45726 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VMC-0005Ef-MV for importer@patchew.org; Fri, 06 Oct 2017 12:20:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56304) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyH-00012x-J6 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyF-0003lw-8H for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41342) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uy8-0003cL-Ux; Fri, 06 Oct 2017 11:55:21 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E03DD3A272; Fri, 6 Oct 2017 15:55:19 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id F293867585; Fri, 6 Oct 2017 15:55:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E03DD3A272 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:02 +0200 Message-Id: <20171006155422.10135-35-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 06 Oct 2017 15:55:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 34/54] commit: Support multiple roots above top node X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This changes the commit block job to support operation in a graph where there is more than a single active layer that references the top node. This involves inserting the commit filter node not only on the path between the given active node and the top node, but between the top node and all of its parents. On completion, bdrv_drop_intermediate() must consider all parents for updating the backing file link. These parents may be backing files themselves and as such read-only; reopen them temporarily if necessary. Previously this was achieved by the bdrv_reopen() calls in the commit block job that made overlay_bs read-write for the whole duration of the block job, even though write access is only needed on completion. Now that we consider all parents, overlay_bs is meaningless. It is left in place in this commit, but we'll remove it soon. Signed-off-by: Kevin Wolf --- block.c | 68 ++++++++++++++++++++++++++++++++++--------------------= ---- block/commit.c | 2 +- 2 files changed, 41 insertions(+), 29 deletions(-) diff --git a/block.c b/block.c index ab354ba82a..1b098d4d09 100644 --- a/block.c +++ b/block.c @@ -985,14 +985,26 @@ static int bdrv_backing_update_filename(BdrvChild *c,= BlockDriverState *base, const char *filename, Error **errp) { BlockDriverState *parent =3D c->opaque; + int orig_flags =3D bdrv_get_flags(parent); int ret; =20 + if (!(orig_flags & BDRV_O_RDWR)) { + ret =3D bdrv_reopen(parent, orig_flags | BDRV_O_RDWR, errp); + if (ret < 0) { + return ret; + } + } + ret =3D bdrv_change_backing_file(parent, filename, base->drv ? base->drv->format_name : ""= ); if (ret < 0) { error_setg_errno(errp, ret, "Could not update backing file link"); } =20 + if (!(orig_flags & BDRV_O_RDWR)) { + bdrv_reopen(parent, orig_flags, NULL); + } + return ret; } =20 @@ -3482,7 +3494,7 @@ BlockDriverState *bdrv_find_base(BlockDriverState *bs) int bdrv_drop_intermediate(BlockDriverState *active, BlockDriverState *top, BlockDriverState *base, const char *backing_fil= e_str) { - BlockDriverState *new_top_bs =3D NULL; + BdrvChild *c, *next; Error *local_err =3D NULL; int ret =3D -EIO; =20 @@ -3492,42 +3504,42 @@ int bdrv_drop_intermediate(BlockDriverState *active= , BlockDriverState *top, goto exit; } =20 - new_top_bs =3D bdrv_find_overlay(active, top); - - if (new_top_bs =3D=3D NULL) { - /* we could not find the image above 'top', this is an error */ - goto exit; - } - - /* special case of new_top_bs->backing->bs already pointing to base - = nothing - * to do, no intermediate images */ - if (backing_bs(new_top_bs) =3D=3D base) { - ret =3D 0; - goto exit; - } - /* Make sure that base is in the backing chain of top */ if (!bdrv_chain_contains(top, base)) { goto exit; } =20 /* success - we can delete the intermediate states, and link top->base= */ - if (new_top_bs->backing->role->update_filename) { - backing_file_str =3D backing_file_str ? backing_file_str : base->f= ilename; - ret =3D new_top_bs->backing->role->update_filename(new_top_bs->bac= king, - base, backing_fil= e_str, - &local_err); - if (ret < 0) { - bdrv_set_backing_hd(new_top_bs, top, &error_abort); + backing_file_str =3D backing_file_str ? backing_file_str : base->filen= ame; + + QLIST_FOREACH_SAFE(c, &top->parents, next_parent, next) { + /* Check whether we are allowed to switch c from top to base */ + GSList *ignore_children =3D g_slist_prepend(NULL, c); + bdrv_check_update_perm(base, NULL, c->perm, c->shared_perm, + ignore_children, &local_err); + if (local_err) { + ret =3D -EPERM; + error_report_err(local_err); goto exit; } - } + g_slist_free(ignore_children); =20 - bdrv_set_backing_hd(new_top_bs, base, &local_err); - if (local_err) { - ret =3D -EPERM; - error_report_err(local_err); - goto exit; + /* If so, update the backing file path in the image file */ + if (c->role->update_filename) { + ret =3D c->role->update_filename(c, base, backing_file_str, + &local_err); + if (ret < 0) { + bdrv_abort_perm_update(base); + error_report_err(local_err); + goto exit; + } + } + + /* Do the actual switch in the in-memory graph. + * Completes bdrv_check_update_perm() transaction internally. */ + bdrv_ref(base); + bdrv_replace_child(c, base); + bdrv_unref(top); } =20 ret =3D 0; diff --git a/block/commit.c b/block/commit.c index 8f0e83578a..610e1cd8f5 100644 --- a/block/commit.c +++ b/block/commit.c @@ -350,7 +350,7 @@ void commit_start(const char *job_id, BlockDriverState = *bs, error_propagate(errp, local_err); goto fail; } - bdrv_set_backing_hd(overlay_bs, commit_top_bs, &local_err); + bdrv_replace_node(top, commit_top_bs, &local_err); if (local_err) { bdrv_unref(commit_top_bs); commit_top_bs =3D NULL; --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306313190285.7856235873368; Fri, 6 Oct 2017 09:11:53 -0700 (PDT) Received: from localhost ([::1]:45686 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VE6-0006Js-DQ for importer@patchew.org; Fri, 06 Oct 2017 12:11:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56345) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyJ-00014H-1J for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyH-0003qb-P2 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46980) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyA-0003dy-6G; Fri, 06 Oct 2017 11:55:22 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2B00A13AAE; Fri, 6 Oct 2017 15:55:21 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3E450675AA; Fri, 6 Oct 2017 15:55:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 2B00A13AAE Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:03 +0200 Message-Id: <20171006155422.10135-36-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 06 Oct 2017 15:55:21 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 35/54] qemu-iotests: Allow QMP pretty printing in common.qemu X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" QMP responses to certain commands can become quite long, which doesn't only make reading them hard, but also means that the maximum line length in patch emails can be exceeded. Allow tests to switch to QMP pretty printing, which results in more, but shorter lines. We also need to make sure to keep indentation in the response for this to work as expected. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- tests/qemu-iotests/common.qemu | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/common.qemu b/tests/qemu-iotests/common.qemu index 9f9aecc9df..7b3052dc79 100644 --- a/tests/qemu-iotests/common.qemu +++ b/tests/qemu-iotests/common.qemu @@ -56,13 +56,13 @@ function _timed_wait_for() shift =20 QEMU_STATUS[$h]=3D0 - while read -t ${QEMU_COMM_TIMEOUT} resp <&${QEMU_OUT[$h]} + while IFS=3D read -t ${QEMU_COMM_TIMEOUT} resp <&${QEMU_OUT[$h]} do if [ -z "${silent}" ]; then echo "${resp}" | _filter_testdir | _filter_qemu \ | _filter_qemu_io | _filter_qmp | _filter_hmp fi - grep -q "${*}" < <(echo ${resp}) + grep -q "${*}" < <(echo "${resp}") if [ $? -eq 0 ]; then return fi @@ -130,6 +130,7 @@ function _send_qemu_cmd() # $qemu_comm_method: set this variable to 'monitor' (case insensitive) # to use the QEMU HMP monitor for communication. # Otherwise, the default of QMP is used. +# $qmp_pretty: Set this variable to 'y' to enable QMP pretty printing. # $keep_stderr: Set this variable to 'y' to keep QEMU's stderr output on s= tderr. # If this variable is empty, stderr will be redirected to st= dout. # Returns: @@ -146,7 +147,11 @@ function _launch_qemu() comm=3D"-monitor stdio" else local qemu_comm_method=3D"qmp" - comm=3D"-monitor none -qmp stdio" + if [ "$qmp_pretty" =3D "y" ]; then + comm=3D"-monitor none -qmp-pretty stdio" + else + comm=3D"-monitor none -qmp stdio" + fi fi =20 fifo_out=3D${QEMU_FIFO_OUT}_${_QEMU_HANDLE} @@ -193,6 +198,9 @@ function _launch_qemu() then # Don't print response, since it has version information in it silent=3Dyes _timed_wait_for ${_QEMU_HANDLE} "capabilities" + if [ "$qmp_pretty" =3D "y" ]; then + silent=3Dyes _timed_wait_for ${_QEMU_HANDLE} "^}" + fi fi QEMU_HANDLE=3D${_QEMU_HANDLE} let _QEMU_HANDLE++ --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307469011816.5102097819369; Fri, 6 Oct 2017 09:31:09 -0700 (PDT) Received: from localhost ([::1]:45802 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VWc-0006sI-5B for importer@patchew.org; Fri, 06 Oct 2017 12:30:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56498) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyQ-0001AZ-RQ for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyM-0003zp-S0 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:38 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38004) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyB-0003fg-Qf; Fri, 06 Oct 2017 11:55:24 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C7184C04B954; Fri, 6 Oct 2017 15:55:22 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7D26867594; Fri, 6 Oct 2017 15:55:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C7184C04B954 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:04 +0200 Message-Id: <20171006155422.10135-37-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:55:22 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 36/54] qemu-iotests: Test commit block job where top has two parents X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Kevin Wolf --- tests/qemu-iotests/191 | 153 +++++++++ tests/qemu-iotests/191.out | 827 +++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/group | 1 + 3 files changed, 981 insertions(+) create mode 100755 tests/qemu-iotests/191 create mode 100644 tests/qemu-iotests/191.out diff --git a/tests/qemu-iotests/191 b/tests/qemu-iotests/191 new file mode 100755 index 0000000000..ac2b88fd78 --- /dev/null +++ b/tests/qemu-iotests/191 @@ -0,0 +1,153 @@ +#!/bin/bash +# +# Test commit block job where top has two parents +# +# Copyright (C) 2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=3Dkwolf@redhat.com + +seq=3D`basename $0` +echo "QA output created by $seq" + +here=3D`pwd` +status=3D1 # failure is the default! + +MIG_SOCKET=3D"${TEST_DIR}/migrate" + +_cleanup() +{ + rm -f "${TEST_IMG}.mid" + rm -f "${TEST_IMG}.ovl2" + rm -f "${TEST_IMG}.ovl3" + _cleanup_test_img + _cleanup_qemu +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +. ./common.qemu + +_supported_fmt qcow2 +_unsupported_imgopts compat=3D0.10 +_supported_proto file +_supported_os Linux + +size=3D64M + +echo +echo =3D=3D=3D Preparing and starting VM =3D=3D=3D +echo + +TEST_IMG=3D"${TEST_IMG}.base" _make_test_img $size +TEST_IMG=3D"${TEST_IMG}.mid" _make_test_img -b "${TEST_IMG}.base" +_make_test_img -b "${TEST_IMG}.mid" +TEST_IMG=3D"${TEST_IMG}.ovl2" _make_test_img -b "${TEST_IMG}.mid" + +$QEMU_IO -c 'write -P 0x55 1M 64k' "${TEST_IMG}.mid" | _filter_qemu_io + +qemu_comm_method=3D"qmp" +qmp_pretty=3D"y" + +_launch_qemu \ + -blockdev "driver=3D${IMGFMT},file.driver=3Dfile,file.filename=3D${TES= T_IMG}.base,node-name=3Dbase" \ + -blockdev "driver=3D${IMGFMT},file.driver=3Dfile,file.filename=3D${TES= T_IMG}.mid,node-name=3Dmid,backing=3Dbase" \ + -blockdev "driver=3D${IMGFMT},file.driver=3Dfile,file.filename=3D${TES= T_IMG},node-name=3Dtop,backing=3Dmid" \ + -blockdev "driver=3D${IMGFMT},file.driver=3Dfile,file.filename=3D${TES= T_IMG}.ovl2,node-name=3Dtop2,backing=3Dmid" +h=3D$QEMU_HANDLE +_send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" '^}' + +echo +echo =3D=3D=3D Perform commit job =3D=3D=3D +echo + +_send_qemu_cmd $h \ + "{ 'execute': 'block-commit', + 'arguments': { 'job-id': 'commit0', + 'device': 'top', + 'base':'$TEST_IMG.base', + 'top': '$TEST_IMG.mid' } }" \ + "BLOCK_JOB_COMPLETED" +_send_qemu_cmd $h "" "^}" + +echo +echo =3D=3D=3D Check that both top and top2 point to base now =3D=3D=3D +echo + +_send_qemu_cmd $h "{ 'execute': 'query-named-block-nodes' }" "^}" | + _filter_generated_node_ids + +_send_qemu_cmd $h "{ 'execute': 'quit' }" "^}" +wait=3D1 _cleanup_qemu + +_img_info +TEST_IMG=3D"$TEST_IMG.ovl2" _img_info + + +echo +echo =3D=3D=3D Preparing and starting VM with -drive =3D=3D=3D +echo + +TEST_IMG=3D"${TEST_IMG}.base" _make_test_img $size +TEST_IMG=3D"${TEST_IMG}.mid" _make_test_img -b "${TEST_IMG}.base" +_make_test_img -b "${TEST_IMG}.mid" +TEST_IMG=3D"${TEST_IMG}.ovl2" _make_test_img -b "${TEST_IMG}.mid" +TEST_IMG=3D"${TEST_IMG}.ovl3" _make_test_img -b "${TEST_IMG}.ovl2" + +$QEMU_IO -c 'write -P 0x55 1M 64k' "${TEST_IMG}.mid" | _filter_qemu_io + +qemu_comm_method=3D"qmp" +qmp_pretty=3D"y" + +_launch_qemu \ + -drive "driver=3D${IMGFMT},file=3D${TEST_IMG},node-name=3Dtop,backing.= node-name=3Dmid" \ + -drive "driver=3D${IMGFMT},file=3D${TEST_IMG}.ovl3,node-name=3Dtop2,ba= cking.backing=3Dmid" +h=3D$QEMU_HANDLE +_send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" '^}' + +echo +echo =3D=3D=3D Perform commit job =3D=3D=3D +echo + +_send_qemu_cmd $h \ + "{ 'execute': 'block-commit', + 'arguments': { 'job-id': 'commit0', + 'device': 'top', + 'base':'$TEST_IMG.base', + 'top': '$TEST_IMG.mid' } }" \ + "BLOCK_JOB_COMPLETED" +_send_qemu_cmd $h "" "^}" + +echo +echo =3D=3D=3D Check that both top and top2 point to base now =3D=3D=3D +echo + +_send_qemu_cmd $h "{ 'execute': 'query-named-block-nodes' }" "^}" | + _filter_generated_node_ids + +_send_qemu_cmd $h "{ 'execute': 'quit' }" "^}" +wait=3D1 _cleanup_qemu + +_img_info +TEST_IMG=3D"$TEST_IMG.ovl2" _img_info + +# success, all done +echo "*** done" +rm -f $seq.full +status=3D0 diff --git a/tests/qemu-iotests/191.out b/tests/qemu-iotests/191.out new file mode 100644 index 0000000000..7bfcd2d5d8 --- /dev/null +++ b/tests/qemu-iotests/191.out @@ -0,0 +1,827 @@ +QA output created by 191 + +=3D=3D=3D Preparing and starting VM =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D67108864 +Formatting 'TEST_DIR/t.IMGFMT.mid', fmt=3DIMGFMT size=3D67108864 backing_f= ile=3DTEST_DIR/t.IMGFMT.base +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 backing_file= =3DTEST_DIR/t.IMGFMT.mid +Formatting 'TEST_DIR/t.IMGFMT.ovl2', fmt=3DIMGFMT size=3D67108864 backing_= file=3DTEST_DIR/t.IMGFMT.mid +wrote 65536/65536 bytes at offset 1048576 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +{ + "return": { + } +} + +=3D=3D=3D Perform commit job =3D=3D=3D + +{ + "return": { + } +} +{ + "timestamp": { + "seconds": TIMESTAMP, + "microseconds": TIMESTAMP + }, + "event": "BLOCK_JOB_COMPLETED", + "data": { + "device": "commit0", + "len": 67108864, + "offset": 67108864, + "speed": 0, + "type": "commit" + } +} + +=3D=3D=3D Check that both top and top2 point to base now =3D=3D=3D + +{ + "return": [ + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "backing-image": { + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.base", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 397312, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "dirty-flag": false + }, + "backing-filename-format": "qcow2", + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.ovl2", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 200704, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "full-backing-filename": "TEST_DIR/t.qcow2.base", + "backing-filename": "TEST_DIR/t.qcow2.base", + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "top2", + "backing_file_depth": 1, + "drv": "qcow2", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "backing_file": "TEST_DIR/t.qcow2.base", + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.ovl2", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 197120, + "filename": "TEST_DIR/t.qcow2.ovl2", + "format": "file", + "actual-size": 200704, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "NODE_NAME", + "backing_file_depth": 0, + "drv": "file", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.ovl2", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "backing-image": { + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.base", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 397312, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "dirty-flag": false + }, + "backing-filename-format": "qcow2", + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 200704, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "full-backing-filename": "TEST_DIR/t.qcow2.base", + "backing-filename": "TEST_DIR/t.qcow2.base", + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "top", + "backing_file_depth": 1, + "drv": "qcow2", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "backing_file": "TEST_DIR/t.qcow2.base", + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 197120, + "filename": "TEST_DIR/t.qcow2", + "format": "file", + "actual-size": 200704, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "NODE_NAME", + "backing_file_depth": 0, + "drv": "file", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "backing-image": { + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.base", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 397312, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "dirty-flag": false + }, + "backing-filename-format": "qcow2", + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.mid", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 397312, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "full-backing-filename": "TEST_DIR/t.qcow2.base", + "backing-filename": "TEST_DIR/t.qcow2.base", + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "mid", + "backing_file_depth": 1, + "drv": "qcow2", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "backing_file": "TEST_DIR/t.qcow2.base", + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.mid", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 393216, + "filename": "TEST_DIR/t.qcow2.mid", + "format": "file", + "actual-size": 397312, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "NODE_NAME", + "backing_file_depth": 0, + "drv": "file", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.mid", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.base", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 397312, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "base", + "backing_file_depth": 0, + "drv": "qcow2", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.base", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 393216, + "filename": "TEST_DIR/t.qcow2.base", + "format": "file", + "actual-size": 397312, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "NODE_NAME", + "backing_file_depth": 0, + "drv": "file", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.base", + "encryption_key_missing": false + } + ] +} +{ + "return": { + } +} +{ + "timestamp": { + "seconds": TIMESTAMP, + "microseconds": TIMESTAMP + }, + "event": "SHUTDOWN", + "data": { + "guest": false + } +} +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +backing file: TEST_DIR/t.IMGFMT.base +backing file format: IMGFMT +image: TEST_DIR/t.IMGFMT.ovl2 +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +backing file: TEST_DIR/t.IMGFMT.base +backing file format: IMGFMT + +=3D=3D=3D Preparing and starting VM with -drive =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D67108864 +Formatting 'TEST_DIR/t.IMGFMT.mid', fmt=3DIMGFMT size=3D67108864 backing_f= ile=3DTEST_DIR/t.IMGFMT.base +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 backing_file= =3DTEST_DIR/t.IMGFMT.mid +Formatting 'TEST_DIR/t.IMGFMT.ovl2', fmt=3DIMGFMT size=3D67108864 backing_= file=3DTEST_DIR/t.IMGFMT.mid +Formatting 'TEST_DIR/t.IMGFMT.ovl3', fmt=3DIMGFMT size=3D67108864 backing_= file=3DTEST_DIR/t.IMGFMT.ovl2 +wrote 65536/65536 bytes at offset 1048576 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +{ + "return": { + } +} + +=3D=3D=3D Perform commit job =3D=3D=3D + +{ + "return": { + } +} +{ + "timestamp": { + "seconds": TIMESTAMP, + "microseconds": TIMESTAMP + }, + "event": "BLOCK_JOB_COMPLETED", + "data": { + "device": "commit0", + "len": 67108864, + "offset": 67108864, + "speed": 0, + "type": "commit" + } +} + +=3D=3D=3D Check that both top and top2 point to base now =3D=3D=3D + +{ + "return": [ + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "backing-image": { + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.base", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 397312, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "dirty-flag": false + }, + "backing-filename-format": "qcow2", + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.ovl2", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 200704, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "full-backing-filename": "TEST_DIR/t.qcow2.base", + "backing-filename": "TEST_DIR/t.qcow2.base", + "dirty-flag": false + }, + "iops_wr": 0, + "ro": true, + "node-name": "NODE_NAME", + "backing_file_depth": 1, + "drv": "qcow2", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "backing_file": "TEST_DIR/t.qcow2.base", + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.ovl2", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 197120, + "filename": "TEST_DIR/t.qcow2.ovl2", + "format": "file", + "actual-size": 200704, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": true, + "node-name": "NODE_NAME", + "backing_file_depth": 0, + "drv": "file", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.ovl2", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "backing-image": { + "backing-image": { + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.base", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 397312, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "dirty-flag": false + }, + "backing-filename-format": "qcow2", + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.ovl2", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 200704, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "full-backing-filename": "TEST_DIR/t.qcow2.base", + "backing-filename": "TEST_DIR/t.qcow2.base", + "dirty-flag": false + }, + "backing-filename-format": "qcow2", + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.ovl3", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 200704, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "full-backing-filename": "TEST_DIR/t.qcow2.ovl2", + "backing-filename": "TEST_DIR/t.qcow2.ovl2", + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "top2", + "backing_file_depth": 2, + "drv": "qcow2", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "backing_file": "TEST_DIR/t.qcow2.ovl2", + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.ovl3", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 197120, + "filename": "TEST_DIR/t.qcow2.ovl3", + "format": "file", + "actual-size": 200704, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "NODE_NAME", + "backing_file_depth": 0, + "drv": "file", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.ovl3", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.base", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 397312, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": true, + "node-name": "NODE_NAME", + "backing_file_depth": 0, + "drv": "qcow2", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.base", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 393216, + "filename": "TEST_DIR/t.qcow2.base", + "format": "file", + "actual-size": 397312, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": true, + "node-name": "NODE_NAME", + "backing_file_depth": 0, + "drv": "file", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2.base", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "backing-image": { + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2.base", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 397312, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "dirty-flag": false + }, + "backing-filename-format": "qcow2", + "virtual-size": 67108864, + "filename": "TEST_DIR/t.qcow2", + "cluster-size": 65536, + "format": "qcow2", + "actual-size": 200704, + "format-specific": { + "type": "qcow2", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + } + }, + "full-backing-filename": "TEST_DIR/t.qcow2.base", + "backing-filename": "TEST_DIR/t.qcow2.base", + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "top", + "backing_file_depth": 1, + "drv": "qcow2", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "backing_file": "TEST_DIR/t.qcow2.base", + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2", + "encryption_key_missing": false + }, + { + "iops_rd": 0, + "detect_zeroes": "off", + "image": { + "virtual-size": 197120, + "filename": "TEST_DIR/t.qcow2", + "format": "file", + "actual-size": 200704, + "dirty-flag": false + }, + "iops_wr": 0, + "ro": false, + "node-name": "NODE_NAME", + "backing_file_depth": 0, + "drv": "file", + "iops": 0, + "bps_wr": 0, + "write_threshold": 0, + "encrypted": false, + "bps": 0, + "bps_rd": 0, + "cache": { + "no-flush": false, + "direct": false, + "writeback": true + }, + "file": "TEST_DIR/t.qcow2", + "encryption_key_missing": false + } + ] +} +{ + "return": { + } +} +{ + "timestamp": { + "seconds": TIMESTAMP, + "microseconds": TIMESTAMP + }, + "event": "SHUTDOWN", + "data": { + "guest": false + } +} +image: TEST_DIR/t.IMGFMT +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +backing file: TEST_DIR/t.IMGFMT.base +backing file format: IMGFMT +image: TEST_DIR/t.IMGFMT.ovl2 +file format: IMGFMT +virtual size: 64M (67108864 bytes) +cluster_size: 65536 +backing file: TEST_DIR/t.IMGFMT.base +backing file format: IMGFMT +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index cdccee319e..595f4fd416 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -188,6 +188,7 @@ 188 rw auto quick 189 rw auto 190 rw auto quick +191 rw auto 192 rw auto quick 194 rw auto migration quick 195 rw auto quick --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306531124200.74491953141353; Fri, 6 Oct 2017 09:15:31 -0700 (PDT) Received: from localhost ([::1]:45698 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VHY-000146-CE for importer@patchew.org; Fri, 06 Oct 2017 12:15:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56412) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyL-000158-Ek for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyJ-0003uz-Tc for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:33 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53000) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyD-0003hd-7u; Fri, 06 Oct 2017 11:55:25 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3AFE8C0587EB; Fri, 6 Oct 2017 15:55:24 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 242BC67598; Fri, 6 Oct 2017 15:55:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 3AFE8C0587EB Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:05 +0200 Message-Id: <20171006155422.10135-38-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 06 Oct 2017 15:55:24 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 37/54] commit: Remove overlay_bs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We don't need to make any assumptions about the graph layout above the top node of the commit operation any more. Remove the use of bdrv_find_overlay() and related variables from the commit job code. bdrv_drop_intermediate() doesn't use the 'active' parameter any more, so we can just drop it. The overlay node was previously added to the block job to get a BLK_PERM_GRAPH_MOD. We really need to respect those permissions in bdrv_drop_intermediate() now, but as long as we haven't figured out yet how BLK_PERM_GRAPH_MOD is actually supposed to work, just leave a TODO comment there. With this change, it is now possible to perform another block job on an overlay node without conflicts. qemu-iotests 030 is changed accordingly. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- include/block/block.h | 3 +-- block.c | 6 +++-- block/commit.c | 62 ++++++++++++----------------------------------= ---- tests/qemu-iotests/030 | 4 ---- 4 files changed, 20 insertions(+), 55 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index 3c3af462e4..d5c2731a03 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -315,8 +315,7 @@ int bdrv_commit(BlockDriverState *bs); int bdrv_change_backing_file(BlockDriverState *bs, const char *backing_file, const char *backing_fmt); void bdrv_register(BlockDriver *bdrv); -int bdrv_drop_intermediate(BlockDriverState *active, BlockDriverState *top, - BlockDriverState *base, +int bdrv_drop_intermediate(BlockDriverState *top, BlockDriverState *base, const char *backing_file_str); BlockDriverState *bdrv_find_overlay(BlockDriverState *active, BlockDriverState *bs); diff --git a/block.c b/block.c index 1b098d4d09..46eb1728da 100644 --- a/block.c +++ b/block.c @@ -3491,8 +3491,8 @@ BlockDriverState *bdrv_find_base(BlockDriverState *bs) * if active =3D=3D top, that is considered an error * */ -int bdrv_drop_intermediate(BlockDriverState *active, BlockDriverState *top, - BlockDriverState *base, const char *backing_fil= e_str) +int bdrv_drop_intermediate(BlockDriverState *top, BlockDriverState *base, + const char *backing_file_str) { BdrvChild *c, *next; Error *local_err =3D NULL; @@ -3510,6 +3510,8 @@ int bdrv_drop_intermediate(BlockDriverState *active, = BlockDriverState *top, } =20 /* success - we can delete the intermediate states, and link top->base= */ + /* TODO Check graph modification op blockers (BLK_PERM_GRAPH_MOD) once + * we've figured out how they should work. */ backing_file_str =3D backing_file_str ? backing_file_str : base->filen= ame; =20 QLIST_FOREACH_SAFE(c, &top->parents, next_parent, next) { diff --git a/block/commit.c b/block/commit.c index 610e1cd8f5..5036eec434 100644 --- a/block/commit.c +++ b/block/commit.c @@ -36,13 +36,11 @@ enum { typedef struct CommitBlockJob { BlockJob common; RateLimit limit; - BlockDriverState *active; BlockDriverState *commit_top_bs; BlockBackend *top; BlockBackend *base; BlockdevOnError on_error; int base_flags; - int orig_overlay_flags; char *backing_file_str; } CommitBlockJob; =20 @@ -81,18 +79,15 @@ static void commit_complete(BlockJob *job, void *opaque) { CommitBlockJob *s =3D container_of(job, CommitBlockJob, common); CommitCompleteData *data =3D opaque; - BlockDriverState *active =3D s->active; BlockDriverState *top =3D blk_bs(s->top); BlockDriverState *base =3D blk_bs(s->base); - BlockDriverState *overlay_bs =3D bdrv_find_overlay(active, s->commit_t= op_bs); + BlockDriverState *commit_top_bs =3D s->commit_top_bs; int ret =3D data->ret; bool remove_commit_top_bs =3D false; =20 - /* Make sure overlay_bs and top stay around until bdrv_set_backing_hd(= ) */ + /* Make sure commit_top_bs and top stay around until bdrv_replace_node= () */ bdrv_ref(top); - if (overlay_bs) { - bdrv_ref(overlay_bs); - } + bdrv_ref(commit_top_bs); =20 /* Remove base node parent that still uses BLK_PERM_WRITE/RESIZE before * the normal backing chain can be restored. */ @@ -100,9 +95,9 @@ static void commit_complete(BlockJob *job, void *opaque) =20 if (!block_job_is_cancelled(&s->common) && ret =3D=3D 0) { /* success */ - ret =3D bdrv_drop_intermediate(active, s->commit_top_bs, base, + ret =3D bdrv_drop_intermediate(s->commit_top_bs, base, s->backing_file_str); - } else if (overlay_bs) { + } else { /* XXX Can (or should) we somehow keep 'consistent read' blocked e= ven * after the failed/cancelled commit job is gone? If we already wr= ote * something to base, the intermediate images aren't valid any mor= e. */ @@ -115,9 +110,6 @@ static void commit_complete(BlockJob *job, void *opaque) if (s->base_flags !=3D bdrv_get_flags(base)) { bdrv_reopen(base, s->base_flags, NULL); } - if (overlay_bs && s->orig_overlay_flags !=3D bdrv_get_flags(overlay_bs= )) { - bdrv_reopen(overlay_bs, s->orig_overlay_flags, NULL); - } g_free(s->backing_file_str); blk_unref(s->top); =20 @@ -134,10 +126,13 @@ static void commit_complete(BlockJob *job, void *opaq= ue) * filter driver from the backing chain. Do this as the final step so = that * the 'consistent read' permission can be granted. */ if (remove_commit_top_bs) { - bdrv_set_backing_hd(overlay_bs, top, &error_abort); + bdrv_child_try_set_perm(commit_top_bs->backing, 0, BLK_PERM_ALL, + &error_abort); + bdrv_replace_node(commit_top_bs, backing_bs(commit_top_bs), + &error_abort); } =20 - bdrv_unref(overlay_bs); + bdrv_unref(commit_top_bs); bdrv_unref(top); } =20 @@ -283,10 +278,8 @@ void commit_start(const char *job_id, BlockDriverState= *bs, { CommitBlockJob *s; BlockReopenQueue *reopen_queue =3D NULL; - int orig_overlay_flags; int orig_base_flags; BlockDriverState *iter; - BlockDriverState *overlay_bs; BlockDriverState *commit_top_bs =3D NULL; Error *local_err =3D NULL; int ret; @@ -297,31 +290,19 @@ void commit_start(const char *job_id, BlockDriverStat= e *bs, return; } =20 - overlay_bs =3D bdrv_find_overlay(bs, top); - - if (overlay_bs =3D=3D NULL) { - error_setg(errp, "Could not find overlay image for %s:", top->file= name); - return; - } - s =3D block_job_create(job_id, &commit_job_driver, bs, 0, BLK_PERM_ALL, speed, BLOCK_JOB_DEFAULT, NULL, NULL, errp); if (!s) { return; } =20 - orig_base_flags =3D bdrv_get_flags(base); - orig_overlay_flags =3D bdrv_get_flags(overlay_bs); - - /* convert base & overlay_bs to r/w, if necessary */ + /* convert base to r/w, if necessary */ + orig_base_flags =3D bdrv_get_flags(base); if (!(orig_base_flags & BDRV_O_RDWR)) { reopen_queue =3D bdrv_reopen_queue(reopen_queue, base, NULL, orig_base_flags | BDRV_O_RDWR); } - if (!(orig_overlay_flags & BDRV_O_RDWR)) { - reopen_queue =3D bdrv_reopen_queue(reopen_queue, overlay_bs, NULL, - orig_overlay_flags | BDRV_O_RDWR); - } + if (reopen_queue) { bdrv_reopen_multiple(bdrv_get_aio_context(bs), reopen_queue, &loca= l_err); if (local_err !=3D NULL) { @@ -382,14 +363,6 @@ void commit_start(const char *job_id, BlockDriverState= *bs, goto fail; } =20 - /* overlay_bs must be blocked because it needs to be modified to - * update the backing image string. */ - ret =3D block_job_add_bdrv(&s->common, "overlay of top", overlay_bs, - BLK_PERM_GRAPH_MOD, BLK_PERM_ALL, errp); - if (ret < 0) { - goto fail; - } - s->base =3D blk_new(BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE | BLK_PERM_RESIZE, @@ -408,13 +381,8 @@ void commit_start(const char *job_id, BlockDriverState= *bs, goto fail; } =20 - s->active =3D bs; - - s->base_flags =3D orig_base_flags; - s->orig_overlay_flags =3D orig_overlay_flags; - + s->base_flags =3D orig_base_flags; s->backing_file_str =3D g_strdup(backing_file_str); - s->on_error =3D on_error; =20 trace_commit_start(bs, base, top, s); @@ -429,7 +397,7 @@ fail: blk_unref(s->top); } if (commit_top_bs) { - bdrv_set_backing_hd(overlay_bs, top, &error_abort); + bdrv_replace_node(commit_top_bs, top, &error_abort); } block_job_early_fail(&s->common); } diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index d745cb4cde..18838948fa 100755 --- a/tests/qemu-iotests/030 +++ b/tests/qemu-iotests/030 @@ -287,10 +287,6 @@ class TestParallelOps(iotests.QMPTestCase): result =3D self.vm.qmp('block-stream', device=3D'node6', base=3Dse= lf.imgs[4], job_id=3D'stream-node6-v2') self.assert_qmp(result, 'error/class', 'GenericError') =20 - # This fails because block-commit needs to block node6, the overla= y of the 'top' image - result =3D self.vm.qmp('block-stream', device=3D'node7', base=3Dse= lf.imgs[5], job_id=3D'stream-node6-v3') - self.assert_qmp(result, 'error/class', 'GenericError') - # This fails because block-commit currently blocks the active laye= r even if it's not used result =3D self.vm.qmp('block-stream', device=3D'drive0', base=3Ds= elf.imgs[5], job_id=3D'stream-drive0') self.assert_qmp(result, 'error/class', 'GenericError') --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307148251339.22915185802094; Fri, 6 Oct 2017 09:25:48 -0700 (PDT) Received: from localhost ([::1]:45764 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VRL-0001sK-CV for importer@patchew.org; Fri, 06 Oct 2017 12:25:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56393) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyK-000157-Do for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyJ-0003ty-Ca for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:32 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50436) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyE-0003k0-EN; Fri, 06 Oct 2017 11:55:26 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 74D947AE97; Fri, 6 Oct 2017 15:55:25 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 86FBA67598; Fri, 6 Oct 2017 15:55:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 74D947AE97 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:06 +0200 Message-Id: <20171006155422.10135-39-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Fri, 06 Oct 2017 15:55:25 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 38/54] qemu-io: Add -C for opening with copy-on-read X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Make it easier to enable copy-on-read during iotests, by exposing a new bool option to main and open. Signed-off-by: Eric Blake Reviewed-by: Jeff Cody Reviewed-by: Kevin Wolf Reviewed-by: John Snow Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- qemu-io.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/qemu-io.c b/qemu-io.c index 265445ad89..c70bde3eb1 100644 --- a/qemu-io.c +++ b/qemu-io.c @@ -102,6 +102,7 @@ static void open_help(void) " Opens a file for subsequent use by all of the other qemu-io commands.\n" " -r, -- open file read-only\n" " -s, -- use snapshot file\n" +" -C, -- use copy-on-read\n" " -n, -- disable host cache, short for -t none\n" " -U, -- force shared permissions\n" " -k, -- use kernel AIO implementation (on Linux only)\n" @@ -120,7 +121,7 @@ static const cmdinfo_t open_cmd =3D { .argmin =3D 1, .argmax =3D -1, .flags =3D CMD_NOFILE_OK, - .args =3D "[-rsnkU] [-t cache] [-d discard] [-o options] [path]", + .args =3D "[-rsCnkU] [-t cache] [-d discard] [-o options] [path]= ", .oneline =3D "open the file specified by path", .help =3D open_help, }; @@ -145,7 +146,7 @@ static int open_f(BlockBackend *blk, int argc, char **a= rgv) QDict *opts; bool force_share =3D false; =20 - while ((c =3D getopt(argc, argv, "snro:kt:d:U")) !=3D -1) { + while ((c =3D getopt(argc, argv, "snCro:kt:d:U")) !=3D -1) { switch (c) { case 's': flags |=3D BDRV_O_SNAPSHOT; @@ -154,6 +155,9 @@ static int open_f(BlockBackend *blk, int argc, char **a= rgv) flags |=3D BDRV_O_NOCACHE; writethrough =3D false; break; + case 'C': + flags |=3D BDRV_O_COPY_ON_READ; + break; case 'r': readonly =3D 1; break; @@ -251,6 +255,7 @@ static void usage(const char *name) " -r, --read-only export read-only\n" " -s, --snapshot use snapshot file\n" " -n, --nocache disable host cache, short for -t none\n" +" -C, --copy-on-read enable copy-on-read\n" " -m, --misalign misalign allocations for O_DIRECT\n" " -k, --native-aio use kernel AIO implementation (on Linux only)\n" " -t, --cache=3DMODE use the given cache mode for the image\n" @@ -439,7 +444,7 @@ static QemuOptsList file_opts =3D { int main(int argc, char **argv) { int readonly =3D 0; - const char *sopt =3D "hVc:d:f:rsnmkt:T:U"; + const char *sopt =3D "hVc:d:f:rsnCmkt:T:U"; const struct option lopt[] =3D { { "help", no_argument, NULL, 'h' }, { "version", no_argument, NULL, 'V' }, @@ -448,6 +453,7 @@ int main(int argc, char **argv) { "read-only", no_argument, NULL, 'r' }, { "snapshot", no_argument, NULL, 's' }, { "nocache", no_argument, NULL, 'n' }, + { "copy-on-read", no_argument, NULL, 'C' }, { "misalign", no_argument, NULL, 'm' }, { "native-aio", no_argument, NULL, 'k' }, { "discard", required_argument, NULL, 'd' }, @@ -492,6 +498,9 @@ int main(int argc, char **argv) flags |=3D BDRV_O_NOCACHE; writethrough =3D false; break; + case 'C': + flags |=3D BDRV_O_COPY_ON_READ; + break; case 'd': if (bdrv_parse_discard_flags(optarg, &flags) < 0) { error_report("Invalid discard option: %s", optarg); --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306976294171.8223832291236; Fri, 6 Oct 2017 09:22:56 -0700 (PDT) Received: from localhost ([::1]:45745 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VOp-00085m-GE for importer@patchew.org; Fri, 06 Oct 2017 12:22:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56390) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyK-000156-Ch for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyJ-0003uH-IK for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:32 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50484) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyF-0003m3-V4; Fri, 06 Oct 2017 11:55:28 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D58F67AE97; Fri, 6 Oct 2017 15:55:26 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id C6B3D67588; Fri, 6 Oct 2017 15:55:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D58F67AE97 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:07 +0200 Message-Id: <20171006155422.10135-40-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Fri, 06 Oct 2017 15:55:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 39/54] block: Uniform handling of 0-length bdrv_get_block_status() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Handle a 0-length block status request up front, with a uniform return value claiming the area is not allocated. Most callers don't pass a length of 0 to bdrv_get_block_status() and friends; but it definitely happens with a 0-length read when copy-on-read is enabled. While we could audit all callers to ensure that they never make a 0-length request, and then assert that fact, it was just as easy to fix things to always report success (as long as the callers are careful to not go into an infinite loop). However, we had inconsistent behavior on whether the status is reported as allocated or defers to the backing layer, depending on what callbacks the driver implements, and possibly wasting quite a few CPU cycles to get to that answer. Consistently reporting unallocated up front doesn't really hurt anything, and makes it easier both for callers (0-length requests now have well-defined behavior) and for drivers (drivers don't have to deal with 0-length requests). Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- block/io.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/block/io.c b/block/io.c index e0f904583f..94f74703b7 100644 --- a/block/io.c +++ b/block/io.c @@ -1777,6 +1777,10 @@ static int64_t coroutine_fn bdrv_co_get_block_status= (BlockDriverState *bs, *pnum =3D 0; return BDRV_BLOCK_EOF; } + if (!nb_sectors) { + *pnum =3D 0; + return 0; + } =20 n =3D total_sectors - sector_num; if (n < nb_sectors) { --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306692166784.2450878803661; Fri, 6 Oct 2017 09:18:12 -0700 (PDT) Received: from localhost ([::1]:45718 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VK6-0003NF-NN for importer@patchew.org; Fri, 06 Oct 2017 12:18:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56480) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyQ-00019f-52 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyM-0003zY-Mq for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:38 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36808) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyH-0003o2-4M; Fri, 06 Oct 2017 11:55:29 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 223842578F; Fri, 6 Oct 2017 15:55:28 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3338067585; Fri, 6 Oct 2017 15:55:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 223842578F Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:08 +0200 Message-Id: <20171006155422.10135-41-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Fri, 06 Oct 2017 15:55:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 40/54] iotests: Restore stty settings on completion X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Executing qemu with a terminal as stdin will temporarily alter stty settings on that terminal (for example, disabling echo), because of how we run both the monitor and any multiplexing with guest input. Normally, qemu restores the original settings on exit; but if an iotest triggers qemu to abort in the middle, we can be left with the altered terminal setup. This can make life very annoying when debugging an iotest failure (not everyone remembers the trick of blind-typing 'stty sane' without echo, and some people prefer terminal settings that are slightly different than the defaults picked by 'stty sane'). It is possible to avoid qemu corrupting the terminal by not passing a terminal to qemu's stdin in the first place (as in, use './check ... Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 176cb8e937..e6b6ff7a04 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -134,6 +134,13 @@ export VALGRIND_QEMU=3D export IMGKEYSECRET=3D export IMGOPTSSYNTAX=3Dfalse =20 +# Save current tty settings, since an aborting qemu call may leave things +# screwed up +STTY_RESTORE=3D +if test -t 0; then + STTY_RESTORE=3D$(stty -g) +fi + for r do =20 @@ -664,6 +671,9 @@ END { if (NR > 0) { needwrap=3Dfalse fi =20 + if test -n "$STTY_RESTORE"; then + stty $STTY_RESTORE + fi rm -f "${TEST_DIR}"/*.out "${TEST_DIR}"/*.err "${TEST_DIR}"/*.time rm -f "${TEST_DIR}"/check.pid "${TEST_DIR}"/check.sts rm -f $tmp.* --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307298696252.35855434409507; Fri, 6 Oct 2017 09:28:18 -0700 (PDT) Received: from localhost ([::1]:45781 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VTy-0004TE-Vd for importer@patchew.org; Fri, 06 Oct 2017 12:28:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56511) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyR-0001BG-Jy for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyN-000409-1S for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47278) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyI-0003qn-EG; Fri, 06 Oct 2017 11:55:30 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 60CEF1299; Fri, 6 Oct 2017 15:55:29 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7395C67585; Fri, 6 Oct 2017 15:55:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 60CEF1299 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:09 +0200 Message-Id: <20171006155422.10135-42-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 06 Oct 2017 15:55:29 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 41/54] block: Add blkdebug hook for copy-on-read X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Make it possible to inject errors on writes performed during a read operation due to copy-on-read semantics. Signed-off-by: Eric Blake Reviewed-by: Jeff Cody Reviewed-by: Kevin Wolf Reviewed-by: John Snow Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- qapi/block-core.json | 5 ++++- block/io.c | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 750bb0c77c..ab96e348e6 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2538,6 +2538,8 @@ # # @l1_shrink_free_l2_clusters: discard the l2 tables. (since 2.11) # +# @cor_write: a write due to copy-on-read (since 2.11) +# # Since: 2.9 ## { 'enum': 'BlkdebugEvent', 'prefix': 'BLKDBG', @@ -2555,7 +2557,8 @@ 'flush_to_disk', 'pwritev_rmw_head', 'pwritev_rmw_after_head', 'pwritev_rmw_tail', 'pwritev_rmw_after_tail', 'pwritev', 'pwritev_zero', 'pwritev_done', 'empty_image_prepare', - 'l1_shrink_write_table', 'l1_shrink_free_l2_clusters' ] } + 'l1_shrink_write_table', 'l1_shrink_free_l2_clusters', + 'cor_write'] } =20 ## # @BlkdebugInjectErrorOptions: diff --git a/block/io.c b/block/io.c index 94f74703b7..a5598ed869 100644 --- a/block/io.c +++ b/block/io.c @@ -983,6 +983,7 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(BdrvCh= ild *child, goto err; } =20 + bdrv_debug_event(bs, BLKDBG_COR_WRITE); if (drv->bdrv_co_pwrite_zeroes && buffer_is_zero(bounce_buffer, iov.iov_len)) { /* FIXME: Should we (perhaps conditionally) be setting --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507306863371283.31331147228764; Fri, 6 Oct 2017 09:21:03 -0700 (PDT) Received: from localhost ([::1]:45734 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VMx-0006Fm-GB for importer@patchew.org; Fri, 06 Oct 2017 12:20:59 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56518) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyS-0001C1-21 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyQ-00045X-4W for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50672) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyJ-0003tJ-OE; Fri, 06 Oct 2017 11:55:31 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AA916796E4; Fri, 6 Oct 2017 15:55:30 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id B313667585; Fri, 6 Oct 2017 15:55:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com AA916796E4 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:10 +0200 Message-Id: <20171006155422.10135-43-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Fri, 06 Oct 2017 15:55:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 42/54] block: Perform copy-on-read in loop X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Improve our braindead copy-on-read implementation. Pre-patch, we have multiple issues: - we create a bounce buffer and perform a write for the entire request, even if the active image already has 99% of the clusters occupied, and really only needs to copy-on-read the remaining 1% of the clusters - our bounce buffer was as large as the read request, and can needlessly exhaust our memory by using double the memory of the request size (the original request plus our bounce buffer), rather than a capped maximum overhead beyond the original - if a driver has a max_transfer limit, we are bypassing the normal code in bdrv_aligned_preadv() that fragments to that limit, and instead attempt to read the entire buffer from the driver in one go, which some drivers may assert on - a client can request a large request of nearly 2G such that rounding the request out to cluster boundaries results in a byte count larger than 2G. While this cannot exceed 32 bits, it DOES have some follow-on problems: -- the call to bdrv_driver_pread() can assert for exceeding BDRV_REQUEST_MAX_BYTES, if the driver is old and lacks .bdrv_co_preadv -- if the buffer is all zeroes, the subsequent call to bdrv_co_do_pwrite_zeroes is a no-op due to a negative size, which means we did not actually copy on read Fix all of these issues by breaking up the action into a loop, where each iteration is capped to sane limits. Also, querying the allocation status allows us to optimize: when data is already present in the active layer, we don't need to bounce. Note that the code has a telling comment that copy-on-read should probably be a filter driver rather than a bolt-on hack in io.c; but that remains a task for another day. CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake Reviewed-by: Kevin Wolf Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- block/io.c | 120 +++++++++++++++++++++++++++++++++++++++++----------------= ---- 1 file changed, 82 insertions(+), 38 deletions(-) diff --git a/block/io.c b/block/io.c index a5598ed869..8e419070b5 100644 --- a/block/io.c +++ b/block/io.c @@ -34,6 +34,9 @@ =20 #define NOT_DONE 0x7fffffff /* used while emulated sync operation in progr= ess */ =20 +/* Maximum bounce buffer for copy-on-read and write zeroes, in bytes */ +#define MAX_BOUNCE_BUFFER (32768 << BDRV_SECTOR_BITS) + static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int bytes, BdrvRequestFlags flags); =20 @@ -945,11 +948,14 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(Bdrv= Child *child, =20 BlockDriver *drv =3D bs->drv; struct iovec iov; - QEMUIOVector bounce_qiov; + QEMUIOVector local_qiov; int64_t cluster_offset; unsigned int cluster_bytes; size_t skip_bytes; int ret; + int max_transfer =3D MIN_NON_ZERO(bs->bl.max_transfer, + BDRV_REQUEST_MAX_BYTES); + unsigned int progress =3D 0; =20 /* FIXME We cannot require callers to have write permissions when all = they * are doing is a read request. If we did things right, write permissi= ons @@ -961,53 +967,95 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(Bdrv= Child *child, // assert(child->perm & (BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE)); =20 /* Cover entire cluster so no additional backing file I/O is required = when - * allocating cluster in the image file. + * allocating cluster in the image file. Note that this value may exc= eed + * BDRV_REQUEST_MAX_BYTES (even when the original read did not), which + * is one reason we loop rather than doing it all at once. */ bdrv_round_to_clusters(bs, offset, bytes, &cluster_offset, &cluster_by= tes); + skip_bytes =3D offset - cluster_offset; =20 trace_bdrv_co_do_copy_on_readv(bs, offset, bytes, cluster_offset, cluster_bytes); =20 - iov.iov_len =3D cluster_bytes; - iov.iov_base =3D bounce_buffer =3D qemu_try_blockalign(bs, iov.iov_len= ); + bounce_buffer =3D qemu_try_blockalign(bs, + MIN(MIN(max_transfer, cluster_byte= s), + MAX_BOUNCE_BUFFER)); if (bounce_buffer =3D=3D NULL) { ret =3D -ENOMEM; goto err; } =20 - qemu_iovec_init_external(&bounce_qiov, &iov, 1); + while (cluster_bytes) { + int64_t pnum; =20 - ret =3D bdrv_driver_preadv(bs, cluster_offset, cluster_bytes, - &bounce_qiov, 0); - if (ret < 0) { - goto err; - } + ret =3D bdrv_is_allocated(bs, cluster_offset, + MIN(cluster_bytes, max_transfer), &pnum); + if (ret < 0) { + /* Safe to treat errors in querying allocation as if + * unallocated; we'll probably fail again soon on the + * read, but at least that will set a decent errno. + */ + pnum =3D MIN(cluster_bytes, max_transfer); + } =20 - bdrv_debug_event(bs, BLKDBG_COR_WRITE); - if (drv->bdrv_co_pwrite_zeroes && - buffer_is_zero(bounce_buffer, iov.iov_len)) { - /* FIXME: Should we (perhaps conditionally) be setting - * BDRV_REQ_MAY_UNMAP, if it will allow for a sparser copy - * that still correctly reads as zero? */ - ret =3D bdrv_co_do_pwrite_zeroes(bs, cluster_offset, cluster_bytes= , 0); - } else { - /* This does not change the data on the disk, it is not necessary - * to flush even in cache=3Dwritethrough mode. - */ - ret =3D bdrv_driver_pwritev(bs, cluster_offset, cluster_bytes, - &bounce_qiov, 0); - } + assert(skip_bytes < pnum); =20 - if (ret < 0) { - /* It might be okay to ignore write errors for guest requests. If= this - * is a deliberate copy-on-read then we don't want to ignore the e= rror. - * Simply report it in all cases. - */ - goto err; - } + if (ret <=3D 0) { + /* Must copy-on-read; use the bounce buffer */ + iov.iov_base =3D bounce_buffer; + iov.iov_len =3D pnum =3D MIN(pnum, MAX_BOUNCE_BUFFER); + qemu_iovec_init_external(&local_qiov, &iov, 1); =20 - skip_bytes =3D offset - cluster_offset; - qemu_iovec_from_buf(qiov, 0, bounce_buffer + skip_bytes, bytes); + ret =3D bdrv_driver_preadv(bs, cluster_offset, pnum, + &local_qiov, 0); + if (ret < 0) { + goto err; + } + + bdrv_debug_event(bs, BLKDBG_COR_WRITE); + if (drv->bdrv_co_pwrite_zeroes && + buffer_is_zero(bounce_buffer, pnum)) { + /* FIXME: Should we (perhaps conditionally) be setting + * BDRV_REQ_MAY_UNMAP, if it will allow for a sparser copy + * that still correctly reads as zero? */ + ret =3D bdrv_co_do_pwrite_zeroes(bs, cluster_offset, pnum,= 0); + } else { + /* This does not change the data on the disk, it is not + * necessary to flush even in cache=3Dwritethrough mode. + */ + ret =3D bdrv_driver_pwritev(bs, cluster_offset, pnum, + &local_qiov, 0); + } + + if (ret < 0) { + /* It might be okay to ignore write errors for guest + * requests. If this is a deliberate copy-on-read + * then we don't want to ignore the error. Simply + * report it in all cases. + */ + goto err; + } + + qemu_iovec_from_buf(qiov, progress, bounce_buffer + skip_bytes, + pnum - skip_bytes); + } else { + /* Read directly into the destination */ + qemu_iovec_init(&local_qiov, qiov->niov); + qemu_iovec_concat(&local_qiov, qiov, progress, pnum - skip_byt= es); + ret =3D bdrv_driver_preadv(bs, offset + progress, local_qiov.s= ize, + &local_qiov, 0); + qemu_iovec_destroy(&local_qiov); + if (ret < 0) { + goto err; + } + } + + cluster_offset +=3D pnum; + cluster_bytes -=3D pnum; + progress +=3D pnum - skip_bytes; + skip_bytes =3D 0; + } + ret =3D 0; =20 err: qemu_vfree(bounce_buffer); @@ -1213,9 +1261,6 @@ int coroutine_fn bdrv_co_readv(BdrvChild *child, int6= 4_t sector_num, return bdrv_co_do_readv(child, sector_num, nb_sectors, qiov, 0); } =20 -/* Maximum buffer for write zeroes fallback, in bytes */ -#define MAX_WRITE_ZEROES_BOUNCE_BUFFER (32768 << BDRV_SECTOR_BITS) - static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int bytes, BdrvRequestFlags flags) { @@ -1230,8 +1275,7 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(Bloc= kDriverState *bs, int max_write_zeroes =3D MIN_NON_ZERO(bs->bl.max_pwrite_zeroes, INT_MA= X); int alignment =3D MAX(bs->bl.pwrite_zeroes_alignment, bs->bl.request_alignment); - int max_transfer =3D MIN_NON_ZERO(bs->bl.max_transfer, - MAX_WRITE_ZEROES_BOUNCE_BUFFER); + int max_transfer =3D MIN_NON_ZERO(bs->bl.max_transfer, MAX_BOUNCE_BUFF= ER); =20 assert(alignment % bs->bl.request_alignment =3D=3D 0); head =3D offset % alignment; --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307642707491.227401393828; Fri, 6 Oct 2017 09:34:02 -0700 (PDT) Received: from localhost ([::1]:45811 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VZM-0000ip-RK for importer@patchew.org; Fri, 06 Oct 2017 12:33:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56575) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyT-0001FE-S2 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyS-000492-2u for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55114) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyK-0003vj-Ur; Fri, 06 Oct 2017 11:55:33 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E9DE8883C3; Fri, 6 Oct 2017 15:55:31 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 08FE567585; Fri, 6 Oct 2017 15:55:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E9DE8883C3 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:11 +0200 Message-Id: <20171006155422.10135-44-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 06 Oct 2017 15:55:32 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 43/54] iotests: Add test 197 for covering copy-on-read X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake Add a test for qcow2 copy-on-read behavior, including exposure for the just-fixed bugs. The copy-on-read behavior is always to a qcow2 image, but the test is careful to allow running with most image protocol/format combos as the backing file being copied from (luks being the exception, as it is harder to pass the right secret to all the right places). In fact, for './check nbd', this appears to be the first time we've had a qcow2 image wrapping NBD, requiring an additional line in _filter_img_create to match the similar line in _filter_img_info. Invoking blkdebug to prove we don't write too much took some effort to get working; and it requires that $TEST_WRAP (based on $TEST_DIR) not be subject to word splitting. We may decide later to have the entire iotests suite use relative rather than absolute names, to avoid problems inherited by the absolute name of $PWD or $TEST_DIR, at which point the sanity check in this commit could be simplified. This test requires at least 2G of consecutive memory to succeed; as such, it is prone to spurious failures, particularly on 32-bit machines under load. This situation is detected and triggers an early exit to skip the test, rather than a failure. To manually provoke this setup on a beefier machine, I used: $ (ulimit -S -v 1000000; ./check -qcow2 197) Signed-off-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/197 | 109 +++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/197.out | 26 ++++++++++ tests/qemu-iotests/common.filter | 1 + tests/qemu-iotests/group | 1 + 4 files changed, 137 insertions(+) create mode 100755 tests/qemu-iotests/197 create mode 100644 tests/qemu-iotests/197.out diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 new file mode 100755 index 0000000000..887eb4f496 --- /dev/null +++ b/tests/qemu-iotests/197 @@ -0,0 +1,109 @@ +#!/bin/bash +# +# Test case for copy-on-read into qcow2 +# +# Copyright (C) 2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=3Deblake@redhat.com + +seq=3D"$(basename $0)" +echo "QA output created by $seq" + +here=3D"$PWD" +status=3D1 # failure is the default! + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +TEST_WRAP=3D"$TEST_DIR/t.wrap.qcow2" +BLKDBG_CONF=3D"$TEST_DIR/blkdebug.conf" + +# Sanity check: our use of blkdebug fails if $TEST_DIR contains spaces +# or other problems +case "$TEST_DIR" in + *[^-_a-zA-Z0-9/]*) + _notrun "Suspicious TEST_DIR=3D'$TEST_DIR', cowardly refusing to r= un" ;; +esac + +_cleanup() +{ + _cleanup_test_img + rm -f "$BLKDBG_CONF" +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# Test is supported for any backing file; but we force qcow2 for our wrapp= er. +_supported_fmt generic +_supported_proto generic +_supported_os Linux +# LUKS support may be possible, but it complicates things. +_unsupported_fmt luks + +echo +echo '=3D=3D=3D Copy-on-read =3D=3D=3D' +echo + +# Prep the images +_make_test_img 4G +$QEMU_IO -c "write -P 55 3G 1k" "$TEST_IMG" | _filter_qemu_io +IMGPROTO=3Dfile IMGFMT=3Dqcow2 IMGOPTS=3D TEST_IMG_FILE=3D"$TEST_WRAP" \ + _make_test_img -F "$IMGFMT" -b "$TEST_IMG" | _filter_img_create +$QEMU_IO -f qcow2 -c "write -z -u 1M 64k" "$TEST_WRAP" | _filter_qemu_io + +# Ensure that a read of two clusters, but where one is already allocated, +# does not re-write the allocated cluster +cat > "$BLKDBG_CONF" <&1 | _filter_qemu_io) +case $output in + *allocate*) + _notrun "Insufficent memory to run test" ;; + *) printf '%s\n' "$output" ;; +esac +$QEMU_IO -f qcow2 -C -c "read -P 0 $((3*1024*1024*1024 + 1024)) 1k" \ + "$TEST_WRAP" | _filter_qemu_io + +# Copy-on-read is incompatible with read-only +$QEMU_IO -f qcow2 -C -r "$TEST_WRAP" 2>&1 | _filter_testdir + +# Break the backing chain, and show that images are identical, and that +# we properly copied over explicit zeros. +$QEMU_IMG rebase -u -b "" -f qcow2 "$TEST_WRAP" +$QEMU_IO -f qcow2 -c map "$TEST_WRAP" +_check_test_img +$QEMU_IMG compare -f $IMGFMT -F qcow2 "$TEST_IMG" "$TEST_WRAP" + +# success, all done +echo '*** done' +status=3D0 diff --git a/tests/qemu-iotests/197.out b/tests/qemu-iotests/197.out new file mode 100644 index 0000000000..52b4137d7b --- /dev/null +++ b/tests/qemu-iotests/197.out @@ -0,0 +1,26 @@ +QA output created by 197 + +=3D=3D=3D Copy-on-read =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D4294967296 +wrote 1024/1024 bytes at offset 3221225472 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.wrap.IMGFMT', fmt=3DIMGFMT size=3D4294967296 backin= g_file=3DTEST_DIR/t.IMGFMT backing_fmt=3DIMGFMT +wrote 65536/65536 bytes at offset 1048576 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 131072/131072 bytes at offset 1048576 +128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 0/0 bytes at offset 0 +0 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2147483136/2147483136 bytes at offset 1024 +2 GiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 3221226496 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +can't open device TEST_DIR/t.wrap.qcow2: Can't use copy-on-read on read-on= ly device +2 GiB (0x80010000) bytes allocated at offset 0 bytes (0x0) +1023.938 MiB (0x3fff0000) bytes not allocated at offset 2 GiB (0x80010000) +64 KiB (0x10000) bytes allocated at offset 3 GiB (0xc0000000) +1023.938 MiB (0x3fff0000) bytes not allocated at offset 3 GiB (0xc0010000) +No errors were found on the image. +Images are identical. +*** done diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.f= ilter index 9d5442ecd9..227b37e941 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -111,6 +111,7 @@ _filter_img_create() sed -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ -e "s#$IMGFMT#IMGFMT#g" \ + -e 's#nbd:127.0.0.1:10810#TEST_DIR/t.IMGFMT#g' \ -e "s# encryption=3Doff##g" \ -e "s# cluster_size=3D[0-9]\\+##g" \ -e "s# table_size=3D[0-9]\\+##g" \ diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 595f4fd416..83da427c0a 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -192,3 +192,4 @@ 192 rw auto quick 194 rw auto migration quick 195 rw auto quick +197 rw auto quick --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307028669697.2159129080671; Fri, 6 Oct 2017 09:23:48 -0700 (PDT) Received: from localhost ([::1]:45750 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VPT-0000Db-Fo for importer@patchew.org; Fri, 06 Oct 2017 12:23:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56555) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyT-0001E3-Ao for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyS-00048y-3U for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53140) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyM-0003yP-Ek; Fri, 06 Oct 2017 11:55:34 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6E434C058EA1; Fri, 6 Oct 2017 15:55:33 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4801267594; Fri, 6 Oct 2017 15:55:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6E434C058EA1 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:12 +0200 Message-Id: <20171006155422.10135-45-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 06 Oct 2017 15:55:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 44/54] block: use 1 MB bounce buffers for crypto instead of 16KB X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Daniel P. Berrange" Using 16KB bounce buffers creates a significant performance penalty for I/O to encrypted volumes on storage which high I/O latency (rotating rust & network drives), because it triggers lots of fairly small I/O operations. On tests with rotating rust, and cache=3Dnone|directsync, write speed increased from 2MiB/s to 32MiB/s, on a par with that achieved by the in-kernel luks driver. With other cache modes the in-kernel driver is still notably faster because it is able to report completion of the I/O request before any encryption is done, while the in-QEMU driver must encrypt the data before completion. Signed-off-by: Daniel P. Berrange Message-id: 20170927125340.12360-2-berrange@redhat.com Reviewed-by: Eric Blake Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- block/crypto.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index 58ef6f2f52..684cabeaf8 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -379,7 +379,11 @@ static void block_crypto_close(BlockDriverState *bs) } =20 =20 -#define BLOCK_CRYPTO_MAX_SECTORS 32 +/* + * 1 MB bounce buffer gives good performance / memory tradeoff + * when using cache=3Dnone|directsync. + */ +#define BLOCK_CRYPTO_MAX_IO_SIZE (1024 * 1024) =20 static coroutine_fn int block_crypto_co_readv(BlockDriverState *bs, int64_t sector_num, @@ -396,12 +400,11 @@ block_crypto_co_readv(BlockDriverState *bs, int64_t s= ector_num, =20 qemu_iovec_init(&hd_qiov, qiov->niov); =20 - /* Bounce buffer so we have a linear mem region for - * entire sector. XXX optimize so we avoid bounce - * buffer in case that qiov->niov =3D=3D 1 + /* Bounce buffer because we don't wish to expose cipher text + * in qiov which points to guest memory. */ cipher_data =3D - qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_SECTORS * 5= 12, + qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE, qiov->size)); if (cipher_data =3D=3D NULL) { ret =3D -ENOMEM; @@ -411,8 +414,8 @@ block_crypto_co_readv(BlockDriverState *bs, int64_t sec= tor_num, while (remaining_sectors) { cur_nr_sectors =3D remaining_sectors; =20 - if (cur_nr_sectors > BLOCK_CRYPTO_MAX_SECTORS) { - cur_nr_sectors =3D BLOCK_CRYPTO_MAX_SECTORS; + if (cur_nr_sectors > (BLOCK_CRYPTO_MAX_IO_SIZE / 512)) { + cur_nr_sectors =3D (BLOCK_CRYPTO_MAX_IO_SIZE / 512); } =20 qemu_iovec_reset(&hd_qiov); @@ -464,12 +467,11 @@ block_crypto_co_writev(BlockDriverState *bs, int64_t = sector_num, =20 qemu_iovec_init(&hd_qiov, qiov->niov); =20 - /* Bounce buffer so we have a linear mem region for - * entire sector. XXX optimize so we avoid bounce - * buffer in case that qiov->niov =3D=3D 1 + /* Bounce buffer because we're not permitted to touch + * contents of qiov - it points to guest memory. */ cipher_data =3D - qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_SECTORS * 5= 12, + qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE, qiov->size)); if (cipher_data =3D=3D NULL) { ret =3D -ENOMEM; @@ -479,8 +481,8 @@ block_crypto_co_writev(BlockDriverState *bs, int64_t se= ctor_num, while (remaining_sectors) { cur_nr_sectors =3D remaining_sectors; =20 - if (cur_nr_sectors > BLOCK_CRYPTO_MAX_SECTORS) { - cur_nr_sectors =3D BLOCK_CRYPTO_MAX_SECTORS; + if (cur_nr_sectors > (BLOCK_CRYPTO_MAX_IO_SIZE / 512)) { + cur_nr_sectors =3D (BLOCK_CRYPTO_MAX_IO_SIZE / 512); } =20 qemu_iovec_to_buf(qiov, bytes_done, --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307616265167.68790981432778; Fri, 6 Oct 2017 09:33:36 -0700 (PDT) Received: from localhost ([::1]:45809 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VYv-0000MO-AJ for importer@patchew.org; Fri, 06 Oct 2017 12:33:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56603) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyU-0001GL-RC for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyT-0004BT-I3 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47492) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyO-00041W-Op; Fri, 06 Oct 2017 11:55:36 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B9FFB1299; Fri, 6 Oct 2017 15:55:35 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id C16B867585; Fri, 6 Oct 2017 15:55:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B9FFB1299 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:13 +0200 Message-Id: <20171006155422.10135-46-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 06 Oct 2017 15:55:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 45/54] crypto: expose encryption sector size in APIs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Daniel P. Berrange" While current encryption schemes all have a fixed sector size of 512 bytes, this is not guaranteed to be the case in future. Expose the sector size in the APIs so the block layer can remove assumptions about fixed 512 byte sectors. Reviewed-by: Max Reitz Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange Message-id: 20170927125340.12360-3-berrange@redhat.com Signed-off-by: Max Reitz --- crypto/blockpriv.h | 1 + include/crypto/block.h | 15 +++++++++++++++ crypto/block-luks.c | 6 ++++-- crypto/block-qcow.c | 1 + crypto/block.c | 6 ++++++ 5 files changed, 27 insertions(+), 2 deletions(-) diff --git a/crypto/blockpriv.h b/crypto/blockpriv.h index 0edb810e22..d227522d88 100644 --- a/crypto/blockpriv.h +++ b/crypto/blockpriv.h @@ -36,6 +36,7 @@ struct QCryptoBlock { QCryptoHashAlgorithm kdfhash; size_t niv; uint64_t payload_offset; /* In bytes */ + uint64_t sector_size; /* In bytes */ }; =20 struct QCryptoBlockDriver { diff --git a/include/crypto/block.h b/include/crypto/block.h index f0e543bee1..13232b2472 100644 --- a/include/crypto/block.h +++ b/include/crypto/block.h @@ -241,6 +241,21 @@ QCryptoHashAlgorithm qcrypto_block_get_kdf_hash(QCrypt= oBlock *block); uint64_t qcrypto_block_get_payload_offset(QCryptoBlock *block); =20 /** + * qcrypto_block_get_sector_size: + * @block: the block encryption object + * + * Get the size of sectors used for payload encryption. A new + * IV is used at the start of each sector. The encryption + * sector size is not required to match the sector size of the + * underlying storage. For example LUKS will always use a 512 + * byte sector size, even if the volume is on a disk with 4k + * sectors. + * + * Returns: the sector in bytes + */ +uint64_t qcrypto_block_get_sector_size(QCryptoBlock *block); + +/** * qcrypto_block_free: * @block: the block encryption object * diff --git a/crypto/block-luks.c b/crypto/block-luks.c index 36bc856084..a9062bb0f2 100644 --- a/crypto/block-luks.c +++ b/crypto/block-luks.c @@ -846,8 +846,9 @@ qcrypto_block_luks_open(QCryptoBlock *block, } } =20 + block->sector_size =3D QCRYPTO_BLOCK_LUKS_SECTOR_SIZE; block->payload_offset =3D luks->header.payload_offset * - QCRYPTO_BLOCK_LUKS_SECTOR_SIZE; + block->sector_size; =20 luks->cipher_alg =3D cipheralg; luks->cipher_mode =3D ciphermode; @@ -1240,8 +1241,9 @@ qcrypto_block_luks_create(QCryptoBlock *block, QCRYPTO_BLOCK_LUKS_SECTOR_SIZE)) * QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS); =20 + block->sector_size =3D QCRYPTO_BLOCK_LUKS_SECTOR_SIZE; block->payload_offset =3D luks->header.payload_offset * - QCRYPTO_BLOCK_LUKS_SECTOR_SIZE; + block->sector_size; =20 /* Reserve header space to match payload offset */ initfunc(block, block->payload_offset, opaque, &local_err); diff --git a/crypto/block-qcow.c b/crypto/block-qcow.c index a456fe338b..4dd594a9ba 100644 --- a/crypto/block-qcow.c +++ b/crypto/block-qcow.c @@ -80,6 +80,7 @@ qcrypto_block_qcow_init(QCryptoBlock *block, goto fail; } =20 + block->sector_size =3D QCRYPTO_BLOCK_QCOW_SECTOR_SIZE; block->payload_offset =3D 0; =20 return 0; diff --git a/crypto/block.c b/crypto/block.c index c382393d9a..a7a9ad240e 100644 --- a/crypto/block.c +++ b/crypto/block.c @@ -170,6 +170,12 @@ uint64_t qcrypto_block_get_payload_offset(QCryptoBlock= *block) } =20 =20 +uint64_t qcrypto_block_get_sector_size(QCryptoBlock *block) +{ + return block->sector_size; +} + + void qcrypto_block_free(QCryptoBlock *block) { if (!block) { --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307441703232.16038532330322; Fri, 6 Oct 2017 09:30:41 -0700 (PDT) Received: from localhost ([::1]:45798 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VWE-0006Wh-UC for importer@patchew.org; Fri, 06 Oct 2017 12:30:34 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56595) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyU-0001Fx-GB for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyT-0004BF-Ea for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55000) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyQ-00044y-Ag; Fri, 06 Oct 2017 11:55:38 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 40C4E7C844; Fri, 6 Oct 2017 15:55:37 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 173A667585; Fri, 6 Oct 2017 15:55:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 40C4E7C844 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:14 +0200 Message-Id: <20171006155422.10135-47-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 06 Oct 2017 15:55:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 46/54] block: fix data type casting for crypto payload offset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Daniel P. Berrange" The crypto APIs report the offset of the data payload as an uint64_t type, but the block driver is casting to size_t or ssize_t which will potentially truncate. Most of the block APIs use int64_t for offsets meanwhile, so even if using uint64_t in the crypto block driver we are still at risk of truncation. Change the block crypto driver to use uint64_t, but add asserts that the value is less than INT64_MAX. Reviewed-by: Max Reitz Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange Message-id: 20170927125340.12360-4-berrange@redhat.com Signed-off-by: Max Reitz --- block/crypto.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index 684cabeaf8..61f5d77bc0 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -364,8 +364,9 @@ static int block_crypto_truncate(BlockDriverState *bs, = int64_t offset, PreallocMode prealloc, Error **errp) { BlockCrypto *crypto =3D bs->opaque; - size_t payload_offset =3D + uint64_t payload_offset =3D qcrypto_block_get_payload_offset(crypto->block); + assert(payload_offset < (INT64_MAX - offset)); =20 offset +=3D payload_offset; =20 @@ -395,8 +396,9 @@ block_crypto_co_readv(BlockDriverState *bs, int64_t sec= tor_num, uint8_t *cipher_data =3D NULL; QEMUIOVector hd_qiov; int ret =3D 0; - size_t payload_offset =3D + uint64_t payload_offset =3D qcrypto_block_get_payload_offset(crypto->block) / 512; + assert(payload_offset < (INT64_MAX / 512)); =20 qemu_iovec_init(&hd_qiov, qiov->niov); =20 @@ -462,8 +464,9 @@ block_crypto_co_writev(BlockDriverState *bs, int64_t se= ctor_num, uint8_t *cipher_data =3D NULL; QEMUIOVector hd_qiov; int ret =3D 0; - size_t payload_offset =3D + uint64_t payload_offset =3D qcrypto_block_get_payload_offset(crypto->block) / 512; + assert(payload_offset < (INT64_MAX / 512)); =20 qemu_iovec_init(&hd_qiov, qiov->niov); =20 @@ -524,7 +527,9 @@ static int64_t block_crypto_getlength(BlockDriverState = *bs) BlockCrypto *crypto =3D bs->opaque; int64_t len =3D bdrv_getlength(bs->file->bs); =20 - ssize_t offset =3D qcrypto_block_get_payload_offset(crypto->block); + uint64_t offset =3D qcrypto_block_get_payload_offset(crypto->block); + assert(offset < INT64_MAX); + assert(offset < len); =20 len -=3D offset; =20 --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307802193567.9664149775392; Fri, 6 Oct 2017 09:36:42 -0700 (PDT) Received: from localhost ([::1]:45833 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Vc3-00030l-4z for importer@patchew.org; Fri, 06 Oct 2017 12:36:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56632) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0UyW-0001IN-Gl for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0UyV-0004EA-Bz for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52124) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyR-00047E-N1; Fri, 06 Oct 2017 11:55:39 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A38D58B135; Fri, 6 Oct 2017 15:55:38 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 926B6675BC; Fri, 6 Oct 2017 15:55:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A38D58B135 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:15 +0200 Message-Id: <20171006155422.10135-48-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 06 Oct 2017 15:55:38 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 47/54] block: convert crypto driver to bdrv_co_preadv|pwritev X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Daniel P. Berrange" Make the crypto driver implement the bdrv_co_preadv|pwritev callbacks, and also use bdrv_co_preadv|pwritev for I/O with the protocol driver beneath. This replaces sector based I/O with byte based I/O, and allows us to stop assuming the physical sector size matches the encryption sector size. Signed-off-by: Daniel P. Berrange Message-id: 20170927125340.12360-5-berrange@redhat.com Reviewed-by: Eric Blake Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- block/crypto.c | 106 +++++++++++++++++++++++++++++------------------------= ---- 1 file changed, 54 insertions(+), 52 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index 61f5d77bc0..965c173b01 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -387,18 +387,23 @@ static void block_crypto_close(BlockDriverState *bs) #define BLOCK_CRYPTO_MAX_IO_SIZE (1024 * 1024) =20 static coroutine_fn int -block_crypto_co_readv(BlockDriverState *bs, int64_t sector_num, - int remaining_sectors, QEMUIOVector *qiov) +block_crypto_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t byt= es, + QEMUIOVector *qiov, int flags) { BlockCrypto *crypto =3D bs->opaque; - int cur_nr_sectors; /* number of sectors in current iteration */ + uint64_t cur_bytes; /* number of bytes in current iteration */ uint64_t bytes_done =3D 0; uint8_t *cipher_data =3D NULL; QEMUIOVector hd_qiov; int ret =3D 0; - uint64_t payload_offset =3D - qcrypto_block_get_payload_offset(crypto->block) / 512; - assert(payload_offset < (INT64_MAX / 512)); + uint64_t sector_size =3D qcrypto_block_get_sector_size(crypto->block); + uint64_t payload_offset =3D qcrypto_block_get_payload_offset(crypto->b= lock); + uint64_t sector_num =3D offset / sector_size; + + assert(!flags); + assert(payload_offset < INT64_MAX); + assert(QEMU_IS_ALIGNED(offset, sector_size)); + assert(QEMU_IS_ALIGNED(bytes, sector_size)); =20 qemu_iovec_init(&hd_qiov, qiov->niov); =20 @@ -413,37 +418,29 @@ block_crypto_co_readv(BlockDriverState *bs, int64_t s= ector_num, goto cleanup; } =20 - while (remaining_sectors) { - cur_nr_sectors =3D remaining_sectors; - - if (cur_nr_sectors > (BLOCK_CRYPTO_MAX_IO_SIZE / 512)) { - cur_nr_sectors =3D (BLOCK_CRYPTO_MAX_IO_SIZE / 512); - } + while (bytes) { + cur_bytes =3D MIN(bytes, BLOCK_CRYPTO_MAX_IO_SIZE); =20 qemu_iovec_reset(&hd_qiov); - qemu_iovec_add(&hd_qiov, cipher_data, cur_nr_sectors * 512); + qemu_iovec_add(&hd_qiov, cipher_data, cur_bytes); =20 - ret =3D bdrv_co_readv(bs->file, - payload_offset + sector_num, - cur_nr_sectors, &hd_qiov); + ret =3D bdrv_co_preadv(bs->file, payload_offset + offset + bytes_d= one, + cur_bytes, &hd_qiov, 0); if (ret < 0) { goto cleanup; } =20 - if (qcrypto_block_decrypt(crypto->block, - sector_num, - cipher_data, cur_nr_sectors * 512, - NULL) < 0) { + if (qcrypto_block_decrypt(crypto->block, sector_num, cipher_data, + cur_bytes, NULL) < 0) { ret =3D -EIO; goto cleanup; } =20 - qemu_iovec_from_buf(qiov, bytes_done, - cipher_data, cur_nr_sectors * 512); + qemu_iovec_from_buf(qiov, bytes_done, cipher_data, cur_bytes); =20 - remaining_sectors -=3D cur_nr_sectors; - sector_num +=3D cur_nr_sectors; - bytes_done +=3D cur_nr_sectors * 512; + sector_num +=3D cur_bytes / sector_size; + bytes -=3D cur_bytes; + bytes_done +=3D cur_bytes; } =20 cleanup: @@ -455,18 +452,23 @@ block_crypto_co_readv(BlockDriverState *bs, int64_t s= ector_num, =20 =20 static coroutine_fn int -block_crypto_co_writev(BlockDriverState *bs, int64_t sector_num, - int remaining_sectors, QEMUIOVector *qiov) +block_crypto_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t by= tes, + QEMUIOVector *qiov, int flags) { BlockCrypto *crypto =3D bs->opaque; - int cur_nr_sectors; /* number of sectors in current iteration */ + uint64_t cur_bytes; /* number of bytes in current iteration */ uint64_t bytes_done =3D 0; uint8_t *cipher_data =3D NULL; QEMUIOVector hd_qiov; int ret =3D 0; - uint64_t payload_offset =3D - qcrypto_block_get_payload_offset(crypto->block) / 512; - assert(payload_offset < (INT64_MAX / 512)); + uint64_t sector_size =3D qcrypto_block_get_sector_size(crypto->block); + uint64_t payload_offset =3D qcrypto_block_get_payload_offset(crypto->b= lock); + uint64_t sector_num =3D offset / sector_size; + + assert(!flags); + assert(payload_offset < INT64_MAX); + assert(QEMU_IS_ALIGNED(offset, sector_size)); + assert(QEMU_IS_ALIGNED(bytes, sector_size)); =20 qemu_iovec_init(&hd_qiov, qiov->niov); =20 @@ -481,37 +483,29 @@ block_crypto_co_writev(BlockDriverState *bs, int64_t = sector_num, goto cleanup; } =20 - while (remaining_sectors) { - cur_nr_sectors =3D remaining_sectors; + while (bytes) { + cur_bytes =3D MIN(bytes, BLOCK_CRYPTO_MAX_IO_SIZE); =20 - if (cur_nr_sectors > (BLOCK_CRYPTO_MAX_IO_SIZE / 512)) { - cur_nr_sectors =3D (BLOCK_CRYPTO_MAX_IO_SIZE / 512); - } - - qemu_iovec_to_buf(qiov, bytes_done, - cipher_data, cur_nr_sectors * 512); + qemu_iovec_to_buf(qiov, bytes_done, cipher_data, cur_bytes); =20 - if (qcrypto_block_encrypt(crypto->block, - sector_num, - cipher_data, cur_nr_sectors * 512, - NULL) < 0) { + if (qcrypto_block_encrypt(crypto->block, sector_num, cipher_data, + cur_bytes, NULL) < 0) { ret =3D -EIO; goto cleanup; } =20 qemu_iovec_reset(&hd_qiov); - qemu_iovec_add(&hd_qiov, cipher_data, cur_nr_sectors * 512); + qemu_iovec_add(&hd_qiov, cipher_data, cur_bytes); =20 - ret =3D bdrv_co_writev(bs->file, - payload_offset + sector_num, - cur_nr_sectors, &hd_qiov); + ret =3D bdrv_co_pwritev(bs->file, payload_offset + offset + bytes_= done, + cur_bytes, &hd_qiov, 0); if (ret < 0) { goto cleanup; } =20 - remaining_sectors -=3D cur_nr_sectors; - sector_num +=3D cur_nr_sectors; - bytes_done +=3D cur_nr_sectors * 512; + sector_num +=3D cur_bytes / sector_size; + bytes -=3D cur_bytes; + bytes_done +=3D cur_bytes; } =20 cleanup: @@ -521,6 +515,13 @@ block_crypto_co_writev(BlockDriverState *bs, int64_t s= ector_num, return ret; } =20 +static void block_crypto_refresh_limits(BlockDriverState *bs, Error **errp) +{ + BlockCrypto *crypto =3D bs->opaque; + uint64_t sector_size =3D qcrypto_block_get_sector_size(crypto->block); + bs->bl.request_alignment =3D sector_size; /* No sub-sector I/O */ +} + =20 static int64_t block_crypto_getlength(BlockDriverState *bs) { @@ -620,8 +621,9 @@ BlockDriver bdrv_crypto_luks =3D { .bdrv_truncate =3D block_crypto_truncate, .create_opts =3D &block_crypto_create_opts_luks, =20 - .bdrv_co_readv =3D block_crypto_co_readv, - .bdrv_co_writev =3D block_crypto_co_writev, + .bdrv_refresh_limits =3D block_crypto_refresh_limits, + .bdrv_co_preadv =3D block_crypto_co_preadv, + .bdrv_co_pwritev =3D block_crypto_co_pwritev, .bdrv_getlength =3D block_crypto_getlength, .bdrv_get_info =3D block_crypto_get_info_luks, .bdrv_get_specific_info =3D block_crypto_get_specific_info_luks, --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507308151171452.9654821498858; Fri, 6 Oct 2017 09:42:31 -0700 (PDT) Received: from localhost ([::1]:45861 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VhW-00082T-8Q for importer@patchew.org; Fri, 06 Oct 2017 12:42:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56802) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uyg-0001TD-6i for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uye-0004Ru-60 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37238) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyW-0004FM-NH; Fri, 06 Oct 2017 11:55:44 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A89C74D4A9; Fri, 6 Oct 2017 15:55:43 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id F13E667585; Fri, 6 Oct 2017 15:55:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com A89C74D4A9 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:16 +0200 Message-Id: <20171006155422.10135-49-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Fri, 06 Oct 2017 15:55:43 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 48/54] block: convert qcrypto_block_encrypt|decrypt to take bytes offset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Daniel P. Berrange" Instead of sector offset, take the bytes offset when encrypting or decrypting data. Signed-off-by: Daniel P. Berrange Message-id: 20170927125340.12360-6-berrange@redhat.com Reviewed-by: Eric Blake Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- crypto/blockpriv.h | 4 ++-- include/crypto/block.h | 14 ++++++++------ block/crypto.c | 12 ++++-------- block/qcow.c | 11 +++++++---- block/qcow2-cluster.c | 8 +++----- block/qcow2.c | 4 ++-- crypto/block-luks.c | 12 ++++++++---- crypto/block-qcow.c | 12 ++++++++---- crypto/block.c | 20 ++++++++++++++------ 9 files changed, 56 insertions(+), 41 deletions(-) diff --git a/crypto/blockpriv.h b/crypto/blockpriv.h index d227522d88..41840abcec 100644 --- a/crypto/blockpriv.h +++ b/crypto/blockpriv.h @@ -82,7 +82,7 @@ int qcrypto_block_decrypt_helper(QCryptoCipher *cipher, size_t niv, QCryptoIVGen *ivgen, int sectorsize, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp); @@ -91,7 +91,7 @@ int qcrypto_block_encrypt_helper(QCryptoCipher *cipher, size_t niv, QCryptoIVGen *ivgen, int sectorsize, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp); diff --git a/include/crypto/block.h b/include/crypto/block.h index 13232b2472..cd18f46d56 100644 --- a/include/crypto/block.h +++ b/include/crypto/block.h @@ -161,18 +161,19 @@ QCryptoBlockInfo *qcrypto_block_get_info(QCryptoBlock= *block, /** * @qcrypto_block_decrypt: * @block: the block encryption object - * @startsector: the sector from which @buf was read + * @offset: the position at which @iov was read * @buf: the buffer to decrypt * @len: the length of @buf in bytes * @errp: pointer to a NULL-initialized error object * * Decrypt @len bytes of cipher text in @buf, writing - * plain text back into @buf + * plain text back into @buf. @len and @offset must be + * a multiple of the encryption format sector size. * * Returns 0 on success, -1 on failure */ int qcrypto_block_decrypt(QCryptoBlock *block, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp); @@ -180,18 +181,19 @@ int qcrypto_block_decrypt(QCryptoBlock *block, /** * @qcrypto_block_encrypt: * @block: the block encryption object - * @startsector: the sector to which @buf will be written + * @offset: the position at which @iov will be written * @buf: the buffer to decrypt * @len: the length of @buf in bytes * @errp: pointer to a NULL-initialized error object * * Encrypt @len bytes of plain text in @buf, writing - * cipher text back into @buf + * cipher text back into @buf. @len and @offset must be + * a multiple of the encryption format sector size. * * Returns 0 on success, -1 on failure */ int qcrypto_block_encrypt(QCryptoBlock *block, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp); diff --git a/block/crypto.c b/block/crypto.c index 965c173b01..edf53d49d1 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -398,7 +398,6 @@ block_crypto_co_preadv(BlockDriverState *bs, uint64_t o= ffset, uint64_t bytes, int ret =3D 0; uint64_t sector_size =3D qcrypto_block_get_sector_size(crypto->block); uint64_t payload_offset =3D qcrypto_block_get_payload_offset(crypto->b= lock); - uint64_t sector_num =3D offset / sector_size; =20 assert(!flags); assert(payload_offset < INT64_MAX); @@ -430,15 +429,14 @@ block_crypto_co_preadv(BlockDriverState *bs, uint64_t= offset, uint64_t bytes, goto cleanup; } =20 - if (qcrypto_block_decrypt(crypto->block, sector_num, cipher_data, - cur_bytes, NULL) < 0) { + if (qcrypto_block_decrypt(crypto->block, offset + bytes_done, + cipher_data, cur_bytes, NULL) < 0) { ret =3D -EIO; goto cleanup; } =20 qemu_iovec_from_buf(qiov, bytes_done, cipher_data, cur_bytes); =20 - sector_num +=3D cur_bytes / sector_size; bytes -=3D cur_bytes; bytes_done +=3D cur_bytes; } @@ -463,7 +461,6 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t = offset, uint64_t bytes, int ret =3D 0; uint64_t sector_size =3D qcrypto_block_get_sector_size(crypto->block); uint64_t payload_offset =3D qcrypto_block_get_payload_offset(crypto->b= lock); - uint64_t sector_num =3D offset / sector_size; =20 assert(!flags); assert(payload_offset < INT64_MAX); @@ -488,8 +485,8 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t = offset, uint64_t bytes, =20 qemu_iovec_to_buf(qiov, bytes_done, cipher_data, cur_bytes); =20 - if (qcrypto_block_encrypt(crypto->block, sector_num, cipher_data, - cur_bytes, NULL) < 0) { + if (qcrypto_block_encrypt(crypto->block, offset + bytes_done, + cipher_data, cur_bytes, NULL) < 0) { ret =3D -EIO; goto cleanup; } @@ -503,7 +500,6 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t = offset, uint64_t bytes, goto cleanup; } =20 - sector_num +=3D cur_bytes / sector_size; bytes -=3D cur_bytes; bytes_done +=3D cur_bytes; } diff --git a/block/qcow.c b/block/qcow.c index f450b00cfc..9569deeaf0 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -478,7 +478,9 @@ static int get_cluster_offset(BlockDriverState *bs, for(i =3D 0; i < s->cluster_sectors; i++) { if (i < n_start || i >=3D n_end) { memset(s->cluster_data, 0x00, 512); - if (qcrypto_block_encrypt(s->crypto, start_sec= t + i, + if (qcrypto_block_encrypt(s->crypto, + (start_sect + i) * + BDRV_SECTOR_SIZE, s->cluster_data, BDRV_SECTOR_SIZE, NULL) < 0) { @@ -668,7 +670,8 @@ static coroutine_fn int qcow_co_readv(BlockDriverState = *bs, int64_t sector_num, } if (bs->encrypted) { assert(s->crypto); - if (qcrypto_block_decrypt(s->crypto, sector_num, buf, + if (qcrypto_block_decrypt(s->crypto, + sector_num * BDRV_SECTOR_SIZE, b= uf, n * BDRV_SECTOR_SIZE, NULL) < 0)= { ret =3D -EIO; break; @@ -740,8 +743,8 @@ static coroutine_fn int qcow_co_writev(BlockDriverState= *bs, int64_t sector_num, } if (bs->encrypted) { assert(s->crypto); - if (qcrypto_block_encrypt(s->crypto, sector_num, buf, - n * BDRV_SECTOR_SIZE, NULL) < 0) { + if (qcrypto_block_encrypt(s->crypto, sector_num * BDRV_SECTOR_= SIZE, + buf, n * BDRV_SECTOR_SIZE, NULL) < 0= ) { ret =3D -EIO; break; } diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index d2518d1893..0e5aec81cb 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -446,15 +446,13 @@ static bool coroutine_fn do_perform_cow_encrypt(Block= DriverState *bs, { if (bytes && bs->encrypted) { BDRVQcow2State *s =3D bs->opaque; - int64_t sector =3D (s->crypt_physical_offset ? + int64_t offset =3D (s->crypt_physical_offset ? (cluster_offset + offset_in_cluster) : - (src_cluster_offset + offset_in_cluster)) - >> BDRV_SECTOR_BITS; + (src_cluster_offset + offset_in_cluster)); assert((offset_in_cluster & ~BDRV_SECTOR_MASK) =3D=3D 0); assert((bytes & ~BDRV_SECTOR_MASK) =3D=3D 0); assert(s->crypto); - if (qcrypto_block_encrypt(s->crypto, sector, buffer, - bytes, NULL) < 0) { + if (qcrypto_block_encrypt(s->crypto, offset, buffer, bytes, NULL) = < 0) { return false; } } diff --git a/block/qcow2.c b/block/qcow2.c index b8da8ca105..33597394b5 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1811,7 +1811,7 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverSt= ate *bs, uint64_t offset, if (qcrypto_block_decrypt(s->crypto, (s->crypt_physical_offset ? cluster_offset + offset_in_clus= ter : - offset) >> BDRV_SECTOR_BITS, + offset), cluster_data, cur_bytes, NULL) < 0) { @@ -1946,7 +1946,7 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverS= tate *bs, uint64_t offset, if (qcrypto_block_encrypt(s->crypto, (s->crypt_physical_offset ? cluster_offset + offset_in_cluster : - offset) >> BDRV_SECTOR_BITS, + offset), cluster_data, cur_bytes, NULL) < 0) { ret =3D -EIO; diff --git a/crypto/block-luks.c b/crypto/block-luks.c index a9062bb0f2..d418ac30b8 100644 --- a/crypto/block-luks.c +++ b/crypto/block-luks.c @@ -1399,29 +1399,33 @@ static void qcrypto_block_luks_cleanup(QCryptoBlock= *block) =20 static int qcrypto_block_luks_decrypt(QCryptoBlock *block, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp) { + assert(QEMU_IS_ALIGNED(offset, QCRYPTO_BLOCK_LUKS_SECTOR_SIZE)); + assert(QEMU_IS_ALIGNED(len, QCRYPTO_BLOCK_LUKS_SECTOR_SIZE)); return qcrypto_block_decrypt_helper(block->cipher, block->niv, block->ivgen, QCRYPTO_BLOCK_LUKS_SECTOR_SIZE, - startsector, buf, len, errp); + offset, buf, len, errp); } =20 =20 static int qcrypto_block_luks_encrypt(QCryptoBlock *block, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp) { + assert(QEMU_IS_ALIGNED(offset, QCRYPTO_BLOCK_LUKS_SECTOR_SIZE)); + assert(QEMU_IS_ALIGNED(len, QCRYPTO_BLOCK_LUKS_SECTOR_SIZE)); return qcrypto_block_encrypt_helper(block->cipher, block->niv, block->ivgen, QCRYPTO_BLOCK_LUKS_SECTOR_SIZE, - startsector, buf, len, errp); + offset, buf, len, errp); } =20 =20 diff --git a/crypto/block-qcow.c b/crypto/block-qcow.c index 4dd594a9ba..8817d6aaa7 100644 --- a/crypto/block-qcow.c +++ b/crypto/block-qcow.c @@ -143,29 +143,33 @@ qcrypto_block_qcow_cleanup(QCryptoBlock *block) =20 static int qcrypto_block_qcow_decrypt(QCryptoBlock *block, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp) { + assert(QEMU_IS_ALIGNED(offset, QCRYPTO_BLOCK_QCOW_SECTOR_SIZE)); + assert(QEMU_IS_ALIGNED(len, QCRYPTO_BLOCK_QCOW_SECTOR_SIZE)); return qcrypto_block_decrypt_helper(block->cipher, block->niv, block->ivgen, QCRYPTO_BLOCK_QCOW_SECTOR_SIZE, - startsector, buf, len, errp); + offset, buf, len, errp); } =20 =20 static int qcrypto_block_qcow_encrypt(QCryptoBlock *block, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp) { + assert(QEMU_IS_ALIGNED(offset, QCRYPTO_BLOCK_QCOW_SECTOR_SIZE)); + assert(QEMU_IS_ALIGNED(len, QCRYPTO_BLOCK_QCOW_SECTOR_SIZE)); return qcrypto_block_encrypt_helper(block->cipher, block->niv, block->ivgen, QCRYPTO_BLOCK_QCOW_SECTOR_SIZE, - startsector, buf, len, errp); + offset, buf, len, errp); } =20 =20 diff --git a/crypto/block.c b/crypto/block.c index a7a9ad240e..f206d5eea8 100644 --- a/crypto/block.c +++ b/crypto/block.c @@ -127,22 +127,22 @@ QCryptoBlockInfo *qcrypto_block_get_info(QCryptoBlock= *block, =20 =20 int qcrypto_block_decrypt(QCryptoBlock *block, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp) { - return block->driver->decrypt(block, startsector, buf, len, errp); + return block->driver->decrypt(block, offset, buf, len, errp); } =20 =20 int qcrypto_block_encrypt(QCryptoBlock *block, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp) { - return block->driver->encrypt(block, startsector, buf, len, errp); + return block->driver->encrypt(block, offset, buf, len, errp); } =20 =20 @@ -194,13 +194,17 @@ int qcrypto_block_decrypt_helper(QCryptoCipher *ciphe= r, size_t niv, QCryptoIVGen *ivgen, int sectorsize, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp) { uint8_t *iv; int ret =3D -1; + uint64_t startsector =3D offset / sectorsize; + + assert(QEMU_IS_ALIGNED(offset, sectorsize)); + assert(QEMU_IS_ALIGNED(len, sectorsize)); =20 iv =3D niv ? g_new0(uint8_t, niv) : NULL; =20 @@ -243,13 +247,17 @@ int qcrypto_block_encrypt_helper(QCryptoCipher *ciphe= r, size_t niv, QCryptoIVGen *ivgen, int sectorsize, - uint64_t startsector, + uint64_t offset, uint8_t *buf, size_t len, Error **errp) { uint8_t *iv; int ret =3D -1; + uint64_t startsector =3D offset / sectorsize; + + assert(QEMU_IS_ALIGNED(offset, sectorsize)); + assert(QEMU_IS_ALIGNED(len, sectorsize)); =20 iv =3D niv ? g_new0(uint8_t, niv) : NULL; =20 --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 150730737182577.89488446884673; Fri, 6 Oct 2017 09:29:31 -0700 (PDT) Received: from localhost ([::1]:45788 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VUu-0005HB-Ku for importer@patchew.org; Fri, 06 Oct 2017 12:29:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56815) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uyh-0001U5-21 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uyd-0004Pa-5c for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39074) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyX-0004Gw-V0; Fri, 06 Oct 2017 11:55:46 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E6352C047B93; Fri, 6 Oct 2017 15:55:44 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 04C1167585; Fri, 6 Oct 2017 15:55:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E6352C047B93 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:17 +0200 Message-Id: <20171006155422.10135-50-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:55:45 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 49/54] block: support passthrough of BDRV_REQ_FUA in crypto driver X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Daniel P. Berrange" The BDRV_REQ_FUA flag can trivially be allowed in the crypt driver as a passthrough to the underlying block driver. Reviewed-by: Max Reitz Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange Message-id: 20170927125340.12360-7-berrange@redhat.com Signed-off-by: Max Reitz --- block/crypto.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index edf53d49d1..60ddf8623e 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -279,6 +279,9 @@ static int block_crypto_open_generic(QCryptoBlockFormat= format, return -EINVAL; } =20 + bs->supported_write_flags =3D BDRV_REQ_FUA & + bs->file->bs->supported_write_flags; + opts =3D qemu_opts_create(opts_spec, NULL, 0, &error_abort); qemu_opts_absorb_qdict(opts, options, &local_err); if (local_err) { @@ -462,7 +465,7 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t = offset, uint64_t bytes, uint64_t sector_size =3D qcrypto_block_get_sector_size(crypto->block); uint64_t payload_offset =3D qcrypto_block_get_payload_offset(crypto->b= lock); =20 - assert(!flags); + assert(!(flags & ~BDRV_REQ_FUA)); assert(payload_offset < INT64_MAX); assert(QEMU_IS_ALIGNED(offset, sector_size)); assert(QEMU_IS_ALIGNED(bytes, sector_size)); @@ -495,7 +498,7 @@ block_crypto_co_pwritev(BlockDriverState *bs, uint64_t = offset, uint64_t bytes, qemu_iovec_add(&hd_qiov, cipher_data, cur_bytes); =20 ret =3D bdrv_co_pwritev(bs->file, payload_offset + offset + bytes_= done, - cur_bytes, &hd_qiov, 0); + cur_bytes, &hd_qiov, flags); if (ret < 0) { goto cleanup; } --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307213452196.9477118504068; Fri, 6 Oct 2017 09:26:53 -0700 (PDT) Received: from localhost ([::1]:45772 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VSR-0002uQ-Ln for importer@patchew.org; Fri, 06 Oct 2017 12:26:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56750) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uyd-0001Qu-TY for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uyd-0004PO-40 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:51 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52734) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0UyZ-0004JC-Lu; Fri, 06 Oct 2017 11:55:47 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8C8CA7EAB2; Fri, 6 Oct 2017 15:55:46 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4550E67585; Fri, 6 Oct 2017 15:55:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8C8CA7EAB2 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:18 +0200 Message-Id: <20171006155422.10135-51-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 06 Oct 2017 15:55:46 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 50/54] block/mirror: check backing in bdrv_mirror_top_refresh_filename X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Backing may be zero after failed bdrv_attach_child in bdrv_set_backing_hd, which leads to SIGSEGV. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20170928120300.58164-1-vsementsov@virtuozzo.com Reviewed-by: John Snow Signed-off-by: Max Reitz --- block/mirror.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/block/mirror.c b/block/mirror.c index 459b80f8f3..3b6f0c5772 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1058,6 +1058,11 @@ static int coroutine_fn bdrv_mirror_top_pdiscard(Blo= ckDriverState *bs, =20 static void bdrv_mirror_top_refresh_filename(BlockDriverState *bs, QDict *= opts) { + if (bs->backing =3D=3D NULL) { + /* we can be here after failed bdrv_attach_child in + * bdrv_set_backing_hd */ + return; + } bdrv_refresh_filename(bs->backing->bs); pstrcpy(bs->exact_filename, sizeof(bs->exact_filename), bs->backing->bs->filename); --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307968314588.6323368409453; Fri, 6 Oct 2017 09:39:28 -0700 (PDT) Received: from localhost ([::1]:45842 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Vej-0005KK-FK for importer@patchew.org; Fri, 06 Oct 2017 12:39:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56759) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uye-0001RS-74 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uyd-0004Ps-AK for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:52 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52798) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uyb-0004L1-1b; Fri, 06 Oct 2017 11:55:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E1DEA7EA9B; Fri, 6 Oct 2017 15:55:47 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id D928C67585; Fri, 6 Oct 2017 15:55:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E1DEA7EA9B Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:19 +0200 Message-Id: <20171006155422.10135-52-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 06 Oct 2017 15:55:48 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 51/54] iotests: Fix 195 if IMGFMT is part of TEST_DIR X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz do_run_qemu() in iotest 195 first applies _filter_imgfmt when printing qemu's command line and _filter_testdir only afterwards. Therefore, if the image format is part of the test directory path, _filter_testdir will no longer apply and the actual output will differ from the reference output even in case of success. For example, TEST_DIR might be "/tmp/test-qcow2", in which case _filter_imgfmt first transforms this to "/tmp/test-IMGFMT" which is no longer recognized as the TEST_DIR by _filter_testdir. Fix this by not applying _filter_imgfmt in do_run_qemu() but in run_qemu() instead, and only after _filter_testdir. Signed-off-by: Max Reitz Message-id: 20170927211334.3988-1-mreitz@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- tests/qemu-iotests/195 | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/195 b/tests/qemu-iotests/195 index 05a239cbf5..e7a403ded2 100755 --- a/tests/qemu-iotests/195 +++ b/tests/qemu-iotests/195 @@ -44,15 +44,16 @@ _supported_os Linux =20 function do_run_qemu() { - echo Testing: "$@" | _filter_imgfmt + echo Testing: "$@" $QEMU -nographic -qmp-pretty stdio -serial none "$@" echo } =20 function run_qemu() { - do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qemu | _filter_qmp \ - | _filter_qemu_io | _filter_generated_node_ids + do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_imgfmt | _filter_qem= u \ + | _filter_qmp | _filter_qemu_io \ + | _filter_generated_node_ids } =20 size=3D64M --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507308019754400.0881620234161; Fri, 6 Oct 2017 09:40:19 -0700 (PDT) Received: from localhost ([::1]:45845 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VfZ-000696-0A for importer@patchew.org; Fri, 06 Oct 2017 12:40:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56885) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uym-0001bY-Pe for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:56:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uyi-0004a6-TO for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:56:00 -0400 Received: from mx1.redhat.com ([209.132.183.28]:19813) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uyc-0004MS-4i; Fri, 06 Oct 2017 11:55:50 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 29A233DBC2; Fri, 6 Oct 2017 15:55:49 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3B54467585; Fri, 6 Oct 2017 15:55:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 29A233DBC2 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:20 +0200 Message-Id: <20171006155422.10135-53-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 06 Oct 2017 15:55:49 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 52/54] qcow2: fix return error code in qcow2_truncate() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Pavel Butsykin Signed-off-by: Pavel Butsykin Reviewed-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Max Reitz Message-id: 20170929121613.25997-2-pbutsykin@virtuozzo.com Signed-off-by: Max Reitz --- block/qcow2.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 33597394b5..960b3ab977 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3167,7 +3167,7 @@ static int qcow2_truncate(BlockDriverState *bs, int64= _t offset, if (old_file_size < 0) { error_setg_errno(errp, -old_file_size, "Failed to inquire current file length"); - return ret; + return old_file_size; } =20 nb_new_data_clusters =3D DIV_ROUND_UP(offset - old_length, @@ -3196,7 +3196,7 @@ static int qcow2_truncate(BlockDriverState *bs, int64= _t offset, if (allocation_start < 0) { error_setg_errno(errp, -allocation_start, "Failed to resize refcount structures"); - return -allocation_start; + return allocation_start; } =20 clusters_allocated =3D qcow2_alloc_clusters_at(bs, allocation_star= t, --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307166515841.7913578707327; Fri, 6 Oct 2017 09:26:06 -0700 (PDT) Received: from localhost ([::1]:45769 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VRd-0002C9-OF for importer@patchew.org; Fri, 06 Oct 2017 12:25:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56839) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uyh-0001Up-TI for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:56:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uyg-0004Wh-R8 for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42330) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uyd-0004Oz-FK; Fri, 06 Oct 2017 11:55:51 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6EC4C5F2976; Fri, 6 Oct 2017 15:55:50 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8070267585; Fri, 6 Oct 2017 15:55:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6EC4C5F2976 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:21 +0200 Message-Id: <20171006155422.10135-54-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 06 Oct 2017 15:55:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 53/54] qcow2: truncate the tail of the image file after shrinking the image X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Pavel Butsykin Now after shrinking the image, at the end of the image file, there might be= a tail that probably will never be used. So we can find the last used cluster= and cut the tail. Signed-off-by: Pavel Butsykin Reviewed-by: John Snow Message-id: 20170929121613.25997-3-pbutsykin@virtuozzo.com Signed-off-by: Max Reitz --- block/qcow2.h | 1 + block/qcow2-refcount.c | 22 ++++++++++++++++++++++ block/qcow2.c | 23 +++++++++++++++++++++++ 3 files changed, 46 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index 5a289a81e2..782a206ecb 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -597,6 +597,7 @@ int qcow2_change_refcount_order(BlockDriverState *bs, i= nt refcount_order, BlockDriverAmendStatusCB *status_cb, void *cb_opaque, Error **errp); int qcow2_shrink_reftable(BlockDriverState *bs); +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size); =20 /* qcow2-cluster.c functions */ int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size, diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 88d5a3f1ad..aa3fd6cf17 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -3181,3 +3181,25 @@ out: g_free(reftable_tmp); return ret; } + +int64_t qcow2_get_last_cluster(BlockDriverState *bs, int64_t size) +{ + BDRVQcow2State *s =3D bs->opaque; + int64_t i; + + for (i =3D size_to_clusters(s, size) - 1; i >=3D 0; i--) { + uint64_t refcount; + int ret =3D qcow2_get_refcount(bs, i, &refcount); + if (ret < 0) { + fprintf(stderr, "Can't get refcount for cluster %" PRId64 ": %= s\n", + i, strerror(-ret)); + return ret; + } + if (refcount > 0) { + return i; + } + } + qcow2_signal_corruption(bs, true, -1, -1, + "There are no references in the refcount table= ."); + return -EIO; +} diff --git a/block/qcow2.c b/block/qcow2.c index 960b3ab977..f63d1831f8 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3107,6 +3107,7 @@ static int qcow2_truncate(BlockDriverState *bs, int64= _t offset, new_l1_size =3D size_to_l1(s, offset); =20 if (offset < old_length) { + int64_t last_cluster, old_file_size; if (prealloc !=3D PREALLOC_MODE_OFF) { error_setg(errp, "Preallocation can't be used for shrinking an image= "); @@ -3135,6 +3136,28 @@ static int qcow2_truncate(BlockDriverState *bs, int6= 4_t offset, "Failed to discard unused refblocks"); return ret; } + + old_file_size =3D bdrv_getlength(bs->file->bs); + if (old_file_size < 0) { + error_setg_errno(errp, -old_file_size, + "Failed to inquire current file length"); + return old_file_size; + } + last_cluster =3D qcow2_get_last_cluster(bs, old_file_size); + if (last_cluster < 0) { + error_setg_errno(errp, -last_cluster, + "Failed to find the last cluster"); + return last_cluster; + } + if ((last_cluster + 1) * s->cluster_size < old_file_size) { + ret =3D bdrv_truncate(bs->file, (last_cluster + 1) * s->cluste= r_size, + PREALLOC_MODE_OFF, NULL); + if (ret < 0) { + warn_report("Failed to truncate the tail of the image: %s", + strerror(-ret)); + ret =3D 0; + } + } } else { ret =3D qcow2_grow_l1_table(bs, new_l1_size, true); if (ret < 0) { --=20 2.13.6 From nobody Sat Apr 27 19:30:40 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507307868518878.1330902943146; Fri, 6 Oct 2017 09:37:48 -0700 (PDT) Received: from localhost ([::1]:45836 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0VdA-0003yA-OQ for importer@patchew.org; Fri, 06 Oct 2017 12:37:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56868) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e0Uyk-0001YE-TP for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:56:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e0Uyh-0004X8-0H for qemu-devel@nongnu.org; Fri, 06 Oct 2017 11:55:58 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39448) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e0Uye-0004S9-Sj; Fri, 06 Oct 2017 11:55:52 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D3FB3C047B96; Fri, 6 Oct 2017 15:55:51 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.118.31]) by smtp.corp.redhat.com (Postfix) with ESMTP id C0B7A67585; Fri, 6 Oct 2017 15:55:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D3FB3C047B96 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=kwolf@redhat.com From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 6 Oct 2017 17:54:22 +0200 Message-Id: <20171006155422.10135-55-kwolf@redhat.com> In-Reply-To: <20171006155422.10135-1-kwolf@redhat.com> References: <20171006155422.10135-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 06 Oct 2017 15:55:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 54/54] block/mirror: check backing in bdrv_mirror_top_flush X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Backing may be zero after failed bdrv_append in mirror_start_job, which leads to SIGSEGV. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20170929152255.5431-1-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- block/mirror.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/block/mirror.c b/block/mirror.c index 3b6f0c5772..153758ca9f 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1041,6 +1041,10 @@ static int coroutine_fn bdrv_mirror_top_pwritev(Bloc= kDriverState *bs, =20 static int coroutine_fn bdrv_mirror_top_flush(BlockDriverState *bs) { + if (bs->backing =3D=3D NULL) { + /* we can be here after failed bdrv_append in mirror_start_job */ + return 0; + } return bdrv_co_flush(bs->backing->bs); } =20 --=20 2.13.6