From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201757344400.5956861306472; Wed, 20 Sep 2023 02:22:37 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOV-0007AN-Sy; Wed, 20 Sep 2023 05:21:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOU-00079r-Gb; Wed, 20 Sep 2023 05:21:14 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOS-0001fj-El; Wed, 20 Sep 2023 05:21:13 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKv-0028Y8-27; Wed, 20 Sep 2023 11:20:58 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 01/22] parallels: fix formatting in bdrv_parallels initialization Date: Wed, 20 Sep 2023 11:20:47 +0200 Message-Id: <20230920092108.258898-2-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201759631100013 Content-Type: text/plain; charset="utf-8" Old code is ugly and contains tabulations. There are no functional changes in this patch. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 36 +++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 48c32d6821..2ebd8e1301 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -1249,23 +1249,25 @@ static void parallels_close(BlockDriverState *bs) } =20 static BlockDriver bdrv_parallels =3D { - .format_name =3D "parallels", - .instance_size =3D sizeof(BDRVParallelsState), - .bdrv_probe =3D parallels_probe, - .bdrv_open =3D parallels_open, - .bdrv_close =3D parallels_close, - .bdrv_child_perm =3D bdrv_default_perms, - .bdrv_co_block_status =3D parallels_co_block_status, - .bdrv_has_zero_init =3D bdrv_has_zero_init_1, - .bdrv_co_flush_to_os =3D parallels_co_flush_to_os, - .bdrv_co_readv =3D parallels_co_readv, - .bdrv_co_writev =3D parallels_co_writev, - .is_format =3D true, - .supports_backing =3D true, - .bdrv_co_create =3D parallels_co_create, - .bdrv_co_create_opts =3D parallels_co_create_opts, - .bdrv_co_check =3D parallels_co_check, - .create_opts =3D ¶llels_create_opts, + .format_name =3D "parallels", + .instance_size =3D sizeof(BDRVParallelsState), + .create_opts =3D ¶llels_create_opts, + .is_format =3D true, + .supports_backing =3D true, + + .bdrv_has_zero_init =3D bdrv_has_zero_init_1, + + .bdrv_probe =3D parallels_probe, + .bdrv_open =3D parallels_open, + .bdrv_close =3D parallels_close, + .bdrv_child_perm =3D bdrv_default_perms, + .bdrv_co_block_status =3D parallels_co_block_status, + .bdrv_co_flush_to_os =3D parallels_co_flush_to_os, + .bdrv_co_readv =3D parallels_co_readv, + .bdrv_co_writev =3D parallels_co_writev, + .bdrv_co_create =3D parallels_co_create, + .bdrv_co_create_opts =3D parallels_co_create_opts, + .bdrv_co_check =3D parallels_co_check, }; =20 static void bdrv_parallels_init(void) --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201770221937.9551202185983; Wed, 20 Sep 2023 02:22:50 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOV-0007AM-Rg; Wed, 20 Sep 2023 05:21:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOU-00079s-Gc; Wed, 20 Sep 2023 05:21:14 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOS-0001fi-E3; Wed, 20 Sep 2023 05:21:13 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKv-0028Y8-3D; Wed, 20 Sep 2023 11:20:58 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 02/22] parallels: mark driver as supporting CBT Date: Wed, 20 Sep 2023 11:20:48 +0200 Message-Id: <20230920092108.258898-3-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201770785100001 Content-Type: text/plain; charset="utf-8" Parallels driver indeed support Parallels Dirty Bitmap Feature in read-only mode. The patch adds bdrv_supports_persistent_dirty_bitmap() callback which always return 1 to indicate that. This will allow to copy CBT from Parallels image with qemu-img. Note: read-write support is signalled through bdrv_co_can_store_new_dirty_bitmap() and is different. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/block/parallels.c b/block/parallels.c index 2ebd8e1301..428f72de1c 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -1248,6 +1248,11 @@ static void parallels_close(BlockDriverState *bs) error_free(s->migration_blocker); } =20 +static bool parallels_is_support_dirty_bitmaps(BlockDriverState *bs) +{ + return 1; +} + static BlockDriver bdrv_parallels =3D { .format_name =3D "parallels", .instance_size =3D sizeof(BDRVParallelsState), @@ -1256,6 +1261,7 @@ static BlockDriver bdrv_parallels =3D { .supports_backing =3D true, =20 .bdrv_has_zero_init =3D bdrv_has_zero_init_1, + .bdrv_supports_persistent_dirty_bitmap =3D parallels_is_support_dirty_= bitmaps, =20 .bdrv_probe =3D parallels_probe, .bdrv_open =3D parallels_open, --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201804185227.310366738457; Wed, 20 Sep 2023 02:23:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOa-0007FH-Oc; Wed, 20 Sep 2023 05:21:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOX-0007CB-MA; Wed, 20 Sep 2023 05:21:17 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOS-0001fl-FD; Wed, 20 Sep 2023 05:21:17 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKw-0028Y8-17; Wed, 20 Sep 2023 11:20:58 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 03/22] parallels: fix memory leak in parallels_open() Date: Wed, 20 Sep 2023 11:20:49 +0200 Message-Id: <20230920092108.258898-4-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_PASS=-0.001, T_SPF_HELO_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201805266100005 Content-Type: text/plain; charset="utf-8" We should free opts allocated through qemu_opts_create() at the end. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/parallels.c b/block/parallels.c index 428f72de1c..af7be427c9 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -1217,6 +1217,7 @@ fail_format: fail_options: ret =3D -EINVAL; fail: + qemu_opts_del(opts); /* * "s" object was allocated by g_malloc0 so we can safely * try to free its fields even they were not allocated. --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201750202964.8154636320635; Wed, 20 Sep 2023 02:22:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOY-0007D8-Pu; Wed, 20 Sep 2023 05:21:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOV-0007AO-Un; Wed, 20 Sep 2023 05:21:15 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOS-0001fq-KP; Wed, 20 Sep 2023 05:21:15 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKw-0028Y8-2L; Wed, 20 Sep 2023 11:20:59 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 04/22] parallels: invent parallels_opts_prealloc() helper to parse prealloc opts Date: Wed, 20 Sep 2023 11:20:50 +0200 Message-Id: <20230920092108.258898-5-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201751336100001 Content-Type: text/plain; charset="utf-8" This patch creates above mentioned helper and moves its usage to the beginning of parallels_open(). This simplifies parallels_open() a bit. The patch also ensures that we store prealloc_size on block driver state always in sectors. This makes code cleaner and avoids wrong opinion at the assignment that the value is in bytes. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 72 +++++++++++++++++++++++++++++------------------ 1 file changed, 44 insertions(+), 28 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index af7be427c9..ae006e7fc7 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -1025,6 +1025,44 @@ static int parallels_update_header(BlockDriverState = *bs) return bdrv_pwrite_sync(bs->file, 0, size, s->header, 0); } =20 + +static int parallels_opts_prealloc(BlockDriverState *bs, QDict *options, + Error **errp) +{ + int err; + char *buf; + int64_t bytes; + BDRVParallelsState *s =3D bs->opaque; + Error *local_err =3D NULL; + QemuOpts *opts =3D qemu_opts_create(¶llels_runtime_opts, NULL, 0, = errp); + if (!opts) { + return -ENOMEM; + } + + err =3D -EINVAL; + if (!qemu_opts_absorb_qdict(opts, options, errp)) { + goto done; + } + + bytes =3D qemu_opt_get_size_del(opts, PARALLELS_OPT_PREALLOC_SIZE, 0); + s->prealloc_size =3D bytes >> BDRV_SECTOR_BITS; + buf =3D qemu_opt_get_del(opts, PARALLELS_OPT_PREALLOC_MODE); + /* prealloc_mode can be downgraded later during allocate_clusters */ + s->prealloc_mode =3D qapi_enum_parse(&prealloc_mode_lookup, buf, + PRL_PREALLOC_MODE_FALLOCATE, + &local_err); + g_free(buf); + if (local_err !=3D NULL) { + error_propagate(errp, local_err); + goto done; + } + err =3D 0; + +done: + qemu_opts_del(opts); + return err; +} + static int parallels_open(BlockDriverState *bs, QDict *options, int flags, Error **errp) { @@ -1033,11 +1071,13 @@ static int parallels_open(BlockDriverState *bs, QDi= ct *options, int flags, int ret, size, i; int64_t file_nb_sectors, sector; uint32_t data_start; - QemuOpts *opts =3D NULL; - Error *local_err =3D NULL; - char *buf; bool data_off_is_correct; =20 + ret =3D parallels_opts_prealloc(bs, options, errp); + if (ret < 0) { + return ret; + } + ret =3D bdrv_open_file_child(NULL, options, "file", bs, errp); if (ret < 0) { return ret; @@ -1078,6 +1118,7 @@ static int parallels_open(BlockDriverState *bs, QDict= *options, int flags, ret =3D -EFBIG; goto fail; } + s->prealloc_size =3D MAX(s->tracks, s->prealloc_size); s->cluster_size =3D s->tracks << BDRV_SECTOR_BITS; =20 s->bat_size =3D le32_to_cpu(ph.bat_entries); @@ -1117,29 +1158,6 @@ static int parallels_open(BlockDriverState *bs, QDic= t *options, int flags, s->header_size =3D size; } =20 - opts =3D qemu_opts_create(¶llels_runtime_opts, NULL, 0, errp); - if (!opts) { - goto fail_options; - } - - if (!qemu_opts_absorb_qdict(opts, options, errp)) { - goto fail_options; - } - - s->prealloc_size =3D - qemu_opt_get_size_del(opts, PARALLELS_OPT_PREALLOC_SIZE, 0); - s->prealloc_size =3D MAX(s->tracks, s->prealloc_size >> BDRV_SECTOR_BI= TS); - buf =3D qemu_opt_get_del(opts, PARALLELS_OPT_PREALLOC_MODE); - /* prealloc_mode can be downgraded later during allocate_clusters */ - s->prealloc_mode =3D qapi_enum_parse(&prealloc_mode_lookup, buf, - PRL_PREALLOC_MODE_FALLOCATE, - &local_err); - g_free(buf); - if (local_err !=3D NULL) { - error_propagate(errp, local_err); - goto fail_options; - } - if (ph.ext_off) { if (flags & BDRV_O_RDWR) { /* @@ -1214,10 +1232,8 @@ static int parallels_open(BlockDriverState *bs, QDic= t *options, int flags, =20 fail_format: error_setg(errp, "Image not in Parallels format"); -fail_options: ret =3D -EINVAL; fail: - qemu_opts_del(opts); /* * "s" object was allocated by g_malloc0 so we can safely * try to free its fields even they were not allocated. --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16952018036491001.2152266888875; Wed, 20 Sep 2023 02:23:23 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOY-0007D9-RA; Wed, 20 Sep 2023 05:21:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOX-0007Be-6w; Wed, 20 Sep 2023 05:21:17 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOU-0001ft-Jz; Wed, 20 Sep 2023 05:21:16 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKx-0028Y8-0D; Wed, 20 Sep 2023 11:20:59 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 05/22] parallels: return earler in fail_format branch in parallels_open() Date: Wed, 20 Sep 2023 11:20:51 +0200 Message-Id: <20230920092108.258898-6-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201804440100003 Content-Type: text/plain; charset="utf-8" We do not need to perform any deallocation/cleanup if wrong format is detected. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/parallels.c b/block/parallels.c index ae006e7fc7..12f38cf70b 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -1232,7 +1232,8 @@ static int parallels_open(BlockDriverState *bs, QDict= *options, int flags, =20 fail_format: error_setg(errp, "Image not in Parallels format"); - ret =3D -EINVAL; + return -EINVAL; + fail: /* * "s" object was allocated by g_malloc0 so we can safely --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201758466493.03357705348424; Wed, 20 Sep 2023 02:22:38 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOb-0007FW-1z; Wed, 20 Sep 2023 05:21:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOX-0007CH-QF; Wed, 20 Sep 2023 05:21:17 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOU-0001g3-Ju; Wed, 20 Sep 2023 05:21:17 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKx-0028Y8-1R; Wed, 20 Sep 2023 11:20:59 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 06/22] parallels: return earlier from parallels_open() function on error Date: Wed, 20 Sep 2023 11:20:52 +0200 Message-Id: <20230920092108.258898-7-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201759454100011 Content-Type: text/plain; charset="utf-8" At the beginning of the function we can return immediately until we really allocate s->header. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 12f38cf70b..bd26c8db63 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -1090,7 +1090,7 @@ static int parallels_open(BlockDriverState *bs, QDict= *options, int flags, =20 ret =3D bdrv_pread(bs->file, 0, sizeof(ph), &ph, 0); if (ret < 0) { - goto fail; + return ret; } =20 bs->total_sectors =3D le64_to_cpu(ph.nb_sectors); @@ -1110,13 +1110,11 @@ static int parallels_open(BlockDriverState *bs, QDi= ct *options, int flags, s->tracks =3D le32_to_cpu(ph.tracks); if (s->tracks =3D=3D 0) { error_setg(errp, "Invalid image: Zero sectors per track"); - ret =3D -EINVAL; - goto fail; + return -EINVAL; } if (s->tracks > INT32_MAX/513) { error_setg(errp, "Invalid image: Too big cluster"); - ret =3D -EFBIG; - goto fail; + return -EFBIG; } s->prealloc_size =3D MAX(s->tracks, s->prealloc_size); s->cluster_size =3D s->tracks << BDRV_SECTOR_BITS; @@ -1124,16 +1122,14 @@ static int parallels_open(BlockDriverState *bs, QDi= ct *options, int flags, s->bat_size =3D le32_to_cpu(ph.bat_entries); if (s->bat_size > INT_MAX / sizeof(uint32_t)) { error_setg(errp, "Catalog too large"); - ret =3D -EFBIG; - goto fail; + return -EFBIG; } =20 size =3D bat_entry_off(s->bat_size); s->header_size =3D ROUND_UP(size, bdrv_opt_mem_align(bs->file->bs)); s->header =3D qemu_try_blockalign(bs->file->bs, s->header_size); if (s->header =3D=3D NULL) { - ret =3D -ENOMEM; - goto fail; + return -ENOMEM; } =20 ret =3D bdrv_pread(bs->file, 0, s->header_size, s->header, 0); --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201757286972.3686063186481; Wed, 20 Sep 2023 02:22:37 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOa-0007Em-F7; Wed, 20 Sep 2023 05:21:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOY-0007Ct-GN; Wed, 20 Sep 2023 05:21:18 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOU-0001g6-Ki; Wed, 20 Sep 2023 05:21:18 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKx-0028Y8-2X; Wed, 20 Sep 2023 11:21:00 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 07/22] parallels: refactor path when we need to re-check image in parallels_open Date: Wed, 20 Sep 2023 11:20:53 +0200 Message-Id: <20230920092108.258898-8-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201757745100001 Content-Type: text/plain; charset="utf-8" More conditions follows thus the check should be more scalable. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index bd26c8db63..af3b4894d7 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -1071,7 +1071,7 @@ static int parallels_open(BlockDriverState *bs, QDict= *options, int flags, int ret, size, i; int64_t file_nb_sectors, sector; uint32_t data_start; - bool data_off_is_correct; + bool need_check =3D false; =20 ret =3D parallels_opts_prealloc(bs, options, errp); if (ret < 0) { @@ -1139,11 +1139,14 @@ static int parallels_open(BlockDriverState *bs, QDi= ct *options, int flags, s->bat_bitmap =3D (uint32_t *)(s->header + 1); =20 if (le32_to_cpu(ph.inuse) =3D=3D HEADER_INUSE_MAGIC) { - s->header_unclean =3D true; + need_check =3D s->header_unclean =3D true; + } + + { + bool ok =3D parallels_test_data_off(s, file_nb_sectors, &data_star= t); + need_check =3D need_check || !ok; } =20 - data_off_is_correct =3D parallels_test_data_off(s, file_nb_sectors, - &data_start); s->data_start =3D data_start; s->data_end =3D s->data_start; if (s->data_end < (s->header_size >> BDRV_SECTOR_BITS)) { @@ -1200,6 +1203,7 @@ static int parallels_open(BlockDriverState *bs, QDict= *options, int flags, s->data_end =3D sector + s->tracks; } } + need_check =3D need_check || s->data_end > file_nb_sectors; =20 /* * We don't repair the image here if it's opened for checks. Also we d= on't @@ -1209,12 +1213,8 @@ static int parallels_open(BlockDriverState *bs, QDic= t *options, int flags, return 0; } =20 - /* - * Repair the image if it's dirty or - * out-of-image corruption was detected. - */ - if (s->data_end > file_nb_sectors || s->header_unclean - || !data_off_is_correct) { + /* Repair the image if corruption was detected. */ + if (need_check) { BdrvCheckResult res; ret =3D bdrv_check(bs, &res, BDRV_FIX_ERRORS | BDRV_FIX_LEAKS); if (ret < 0) { @@ -1223,7 +1223,6 @@ static int parallels_open(BlockDriverState *bs, QDict= *options, int flags, goto fail; } } - return 0; =20 fail_format: --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201693745937.4457244771562; Wed, 20 Sep 2023 02:21:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOX-0007CQ-Vy; Wed, 20 Sep 2023 05:21:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOX-0007Bf-7S; Wed, 20 Sep 2023 05:21:17 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOU-0001gI-Jo; Wed, 20 Sep 2023 05:21:16 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKy-0028Y8-0P; Wed, 20 Sep 2023 11:21:00 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 08/22] parallels: create mark_used() helper which sets bit in used bitmap Date: Wed, 20 Sep 2023 11:20:54 +0200 Message-Id: <20230920092108.258898-9-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201696454100003 Content-Type: text/plain; charset="utf-8" This functionality is used twice already and next patch will add more code with it. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 34 +++++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index af3b4894d7..66c86d87e3 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -178,6 +178,21 @@ static void parallels_set_bat_entry(BDRVParallelsState= *s, bitmap_set(s->bat_dirty_bmap, bat_entry_off(index) / s->bat_dirty_bloc= k, 1); } =20 +static int mark_used(BlockDriverState *bs, + unsigned long *bitmap, uint32_t bitmap_size, int64_t = off) +{ + BDRVParallelsState *s =3D bs->opaque; + uint32_t cluster_index =3D host_cluster_index(s, off); + if (cluster_index >=3D bitmap_size) { + return -E2BIG; + } + if (test_bit(cluster_index, bitmap)) { + return -EBUSY; + } + bitmap_set(bitmap, cluster_index, 1); + return 0; +} + static int64_t coroutine_fn GRAPH_RDLOCK allocate_clusters(BlockDriverState *bs, int64_t sector_num, int nb_sectors, int *pnum) @@ -621,7 +636,7 @@ parallels_check_duplicate(BlockDriverState *bs, BdrvChe= ckResult *res, BDRVParallelsState *s =3D bs->opaque; int64_t host_off, host_sector, guest_sector; unsigned long *bitmap; - uint32_t i, bitmap_size, cluster_index, bat_entry; + uint32_t i, bitmap_size, bat_entry; int n, ret =3D 0; uint64_t *buf =3D NULL; bool fixed =3D false; @@ -655,10 +670,9 @@ parallels_check_duplicate(BlockDriverState *bs, BdrvCh= eckResult *res, continue; } =20 - cluster_index =3D host_cluster_index(s, host_off); - assert(cluster_index < bitmap_size); - if (!test_bit(cluster_index, bitmap)) { - bitmap_set(bitmap, cluster_index, 1); + ret =3D mark_used(bs, bitmap, bitmap_size, host_off); + assert(ret !=3D -E2BIG); + if (ret =3D=3D 0) { continue; } =20 @@ -713,11 +727,13 @@ parallels_check_duplicate(BlockDriverState *bs, BdrvC= heckResult *res, * consistent for the new allocated clusters too. * * Note, clusters allocated outside the current image are not - * considered, and the bitmap size doesn't change. + * considered, and the bitmap size doesn't change. This specifical= ly + * means that -E2BIG is OK. */ - cluster_index =3D host_cluster_index(s, host_off); - if (cluster_index < bitmap_size) { - bitmap_set(bitmap, cluster_index, 1); + ret =3D mark_used(bs, bitmap, bitmap_size, host_off); + if (ret =3D=3D -EBUSY) { + res->check_errors++; + goto out_repair_bat; } =20 fixed =3D true; --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201857816942.4754375323888; Wed, 20 Sep 2023 02:24:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOZ-0007Dj-Mp; Wed, 20 Sep 2023 05:21:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOY-0007Cs-FI; Wed, 20 Sep 2023 05:21:18 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOU-0001gZ-Kc; Wed, 20 Sep 2023 05:21:18 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKy-0028Y8-1Z; Wed, 20 Sep 2023 11:21:00 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 09/22] tests: ensure that image validation will not cure the corruption Date: Wed, 20 Sep 2023 11:20:55 +0200 Message-Id: <20230920092108.258898-10-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201859691100003 Content-Type: text/plain; charset="utf-8" Since commit cfce1091d55322789582480798a891cbaf66924e Author: Alexander Ivanov Date: Tue Jul 18 12:44:29 2023 +0200 parallels: Image repairing in parallels_open() there is a potential pit fall with calling qemu-io -c "read" The image is opened in read-write mode and thus could be potentially repaired. This could ruin testing process. The patch forces read-only opening for reads. In that case repairing is impossible. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- tests/qemu-iotests/tests/parallels-checks | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/tests/parallels-checks b/tests/qemu-iotests= /tests/parallels-checks index a7a1b357b5..5917ee079d 100755 --- a/tests/qemu-iotests/tests/parallels-checks +++ b/tests/qemu-iotests/tests/parallels-checks @@ -91,7 +91,7 @@ file_size=3D`stat --printf=3D"%s" "$TEST_IMG"` echo "file size: $file_size" =20 echo "=3D=3D check last cluster =3D=3D" -{ $QEMU_IO -c "read -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" "$TEST_IMG"; = } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -r -c "read -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" "$TEST_IMG= "; } 2>&1 | _filter_qemu_io | _filter_testdir =20 # Clear image _make_test_img $SIZE @@ -105,19 +105,20 @@ echo "=3D=3D write another pattern to second cluster = =3D=3D" { $QEMU_IO -c "write -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2= >&1 | _filter_qemu_io | _filter_testdir =20 echo "=3D=3D check second cluster =3D=3D" -{ $QEMU_IO -c "read -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>= &1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -r -c "read -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; }= 2>&1 | _filter_qemu_io | _filter_testdir + =20 echo "=3D=3D corrupt image =3D=3D" poke_file "$TEST_IMG" "$(($BAT_OFFSET + 4))" "\x01\x00\x00\x00" =20 echo "=3D=3D check second cluster =3D=3D" -{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>= &1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -r -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; }= 2>&1 | _filter_qemu_io | _filter_testdir =20 echo "=3D=3D repair image =3D=3D" _check_test_img -r all =20 echo "=3D=3D check second cluster =3D=3D" -{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>= &1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -r -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; }= 2>&1 | _filter_qemu_io | _filter_testdir =20 echo "=3D=3D check first cluster on host =3D=3D" printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1` --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201862697136.01653447434444; Wed, 20 Sep 2023 02:24:22 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitPU-0008CC-Nk; Wed, 20 Sep 2023 05:22:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPN-0007yq-O5; Wed, 20 Sep 2023 05:22:09 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPL-0001sm-EO; Wed, 20 Sep 2023 05:22:09 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKy-0028Y8-2e; Wed, 20 Sep 2023 11:21:01 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 10/22] parallels: fix broken parallels_check_data_off() Date: Wed, 20 Sep 2023 11:20:56 +0200 Message-Id: <20230920092108.258898-11-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201863131100001 Content-Type: text/plain; charset="utf-8" Once we have repaired data_off field in the header we should update s->data_start which is calculated on the base of it. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/parallels.c b/block/parallels.c index 66c86d87e3..2b5f2b54a0 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -531,6 +531,7 @@ parallels_check_data_off(BlockDriverState *bs, BdrvChec= kResult *res, res->corruptions++; if (fix & BDRV_FIX_ERRORS) { s->header->data_off =3D cpu_to_le32(data_off); + s->data_start =3D data_off; res->corruptions_fixed++; } =20 --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201899977897.742017753693; Wed, 20 Sep 2023 02:24:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitOd-0007HT-BC; Wed, 20 Sep 2023 05:21:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOc-0007H4-IZ; Wed, 20 Sep 2023 05:21:22 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOb-0001ji-3C; Wed, 20 Sep 2023 05:21:22 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKz-0028Y8-0e; Wed, 20 Sep 2023 11:21:01 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 11/22] parallels: add test which will validate data_off fixes through repair Date: Wed, 20 Sep 2023 11:20:57 +0200 Message-Id: <20230920092108.258898-12-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201900516100001 Content-Type: text/plain; charset="utf-8" We have only check through self-repair and that proven to be not enough. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- tests/qemu-iotests/tests/parallels-checks | 17 +++++++++++++++++ tests/qemu-iotests/tests/parallels-checks.out | 18 ++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/tests/qemu-iotests/tests/parallels-checks b/tests/qemu-iotests= /tests/parallels-checks index 5917ee079d..f4ca50295e 100755 --- a/tests/qemu-iotests/tests/parallels-checks +++ b/tests/qemu-iotests/tests/parallels-checks @@ -140,6 +140,23 @@ poke_file "$TEST_IMG" "$DATA_OFF_OFFSET" "\xff\xff\xff= \xff" echo "=3D=3D check first cluster =3D=3D" { $QEMU_IO -c "read -P 0x55 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter= _qemu_io | _filter_testdir =20 +# Clear image +_make_test_img $SIZE + +echo "=3D=3D TEST DATA_OFF THROUGH REPAIR =3D=3D" + +echo "=3D=3D write pattern to first cluster =3D=3D" +{ $QEMU_IO -c "write -P 0x55 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filte= r_qemu_io | _filter_testdir + +echo "=3D=3D spoil data_off field =3D=3D" +poke_file "$TEST_IMG" "$DATA_OFF_OFFSET" "\xff\xff\xff\xff" + +echo "=3D=3D repair image =3D=3D" +_check_test_img -r all + +echo "=3D=3D check first cluster =3D=3D" +{ $QEMU_IO -r -c "read -P 0x55 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _fil= ter_qemu_io | _filter_testdir + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/tests/parallels-checks.out b/tests/qemu-iot= ests/tests/parallels-checks.out index 98a3a7f55e..74a5e29260 100644 --- a/tests/qemu-iotests/tests/parallels-checks.out +++ b/tests/qemu-iotests/tests/parallels-checks.out @@ -72,4 +72,22 @@ wrote 1048576/1048576 bytes at offset 0 Repairing data_off field has incorrect value read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D4194304 +=3D=3D TEST DATA_OFF THROUGH REPAIR =3D=3D +=3D=3D write pattern to first cluster =3D=3D +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D spoil data_off field =3D=3D +=3D=3D repair image =3D=3D +Repairing data_off field has incorrect value +The following inconsistencies were found and repaired: + + 0 leaked clusters + 1 corruptions + +Double checking the fixed image now... +No errors were found on the image. +=3D=3D check first cluster =3D=3D +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201881750408.273998321587; Wed, 20 Sep 2023 02:24:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitPD-0007uR-Dz; Wed, 20 Sep 2023 05:21:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPB-0007t9-Lf; Wed, 20 Sep 2023 05:21:57 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitP9-0001rd-Vv; Wed, 20 Sep 2023 05:21:57 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKz-0028Y8-1l; Wed, 20 Sep 2023 11:21:01 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 12/22] parallels: collect bitmap of used clusters at open Date: Wed, 20 Sep 2023 11:20:58 +0200 Message-Id: <20230920092108.258898-13-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201883133100001 Content-Type: text/plain; charset="utf-8" If the operation is failed, we need to check image consistency if the problem is not about memory allocation. Bitmap adjustments in allocate_cluster are not performed yet. They worth to be separate. This was proven useful during debug of this series. Kept as is for future bissecting. It should be specifically noted that used bitmap must be recalculated if data_off has been fixed during image consistency check. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++ block/parallels.h | 3 ++ 2 files changed, 76 insertions(+) diff --git a/block/parallels.c b/block/parallels.c index 2b5f2b54a0..c41511398f 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -193,6 +193,58 @@ static int mark_used(BlockDriverState *bs, return 0; } =20 +/* + * Collect used bitmap. The image can contain errors, we should fill the + * bitmap anyway, as much as we can. This information will be used for + * error resolution. + */ +static int parallels_fill_used_bitmap(BlockDriverState *bs) +{ + BDRVParallelsState *s =3D bs->opaque; + int64_t payload_bytes; + uint32_t i; + int err =3D 0; + + payload_bytes =3D bdrv_co_getlength(bs->file->bs); + if (payload_bytes < 0) { + return payload_bytes; + } + payload_bytes -=3D s->data_start * BDRV_SECTOR_SIZE; + if (payload_bytes < 0) { + return -EINVAL; + } + + s->used_bmap_size =3D DIV_ROUND_UP(payload_bytes, s->cluster_size); + if (s->used_bmap_size =3D=3D 0) { + return 0; + } + s->used_bmap =3D bitmap_try_new(s->used_bmap_size); + if (s->used_bmap =3D=3D NULL) { + return -ENOMEM; + } + + for (i =3D 0; i < s->bat_size; i++) { + int err2; + int64_t host_off =3D bat2sect(s, i) << BDRV_SECTOR_BITS; + if (host_off =3D=3D 0) { + continue; + } + + err2 =3D mark_used(bs, s->used_bmap, s->used_bmap_size, host_off); + if (err2 < 0 && err =3D=3D 0) { + err =3D err2; + } + } + return err; +} + +static void parallels_free_used_bitmap(BlockDriverState *bs) +{ + BDRVParallelsState *s =3D bs->opaque; + s->used_bmap_size =3D 0; + g_free(s->used_bmap); +} + static int64_t coroutine_fn GRAPH_RDLOCK allocate_clusters(BlockDriverState *bs, int64_t sector_num, int nb_sectors, int *pnum) @@ -530,8 +582,17 @@ parallels_check_data_off(BlockDriverState *bs, BdrvChe= ckResult *res, =20 res->corruptions++; if (fix & BDRV_FIX_ERRORS) { + int err; s->header->data_off =3D cpu_to_le32(data_off); s->data_start =3D data_off; + + parallels_free_used_bitmap(bs); + err =3D parallels_fill_used_bitmap(bs); + if (err =3D=3D -ENOMEM) { + res->check_errors++; + return err; + } + res->corruptions_fixed++; } =20 @@ -1222,6 +1283,14 @@ static int parallels_open(BlockDriverState *bs, QDic= t *options, int flags, } need_check =3D need_check || s->data_end > file_nb_sectors; =20 + if (!need_check) { + ret =3D parallels_fill_used_bitmap(bs); + if (ret =3D=3D -ENOMEM) { + goto fail; + } + need_check =3D need_check || ret < 0; /* These are correctable err= ors */ + } + /* * We don't repair the image here if it's opened for checks. Also we d= on't * want to change inactive images and can't change readonly images. @@ -1251,6 +1320,8 @@ fail: * "s" object was allocated by g_malloc0 so we can safely * try to free its fields even they were not allocated. */ + parallels_free_used_bitmap(bs); + error_free(s->migration_blocker); g_free(s->bat_dirty_bmap); qemu_vfree(s->header); @@ -1271,6 +1342,8 @@ static void parallels_close(BlockDriverState *bs) PREALLOC_MODE_OFF, 0, NULL); } =20 + parallels_free_used_bitmap(bs); + g_free(s->bat_dirty_bmap); qemu_vfree(s->header); =20 diff --git a/block/parallels.h b/block/parallels.h index 4e53e9572d..6b199443cf 100644 --- a/block/parallels.h +++ b/block/parallels.h @@ -72,6 +72,9 @@ typedef struct BDRVParallelsState { unsigned long *bat_dirty_bmap; unsigned int bat_dirty_block; =20 + unsigned long *used_bmap; + unsigned long used_bmap_size; + uint32_t *bat_bitmap; unsigned int bat_size; =20 --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201895227449.43076713114397; Wed, 20 Sep 2023 02:24:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitQ5-0002L5-B3; Wed, 20 Sep 2023 05:22:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitQ3-0001yB-7E; Wed, 20 Sep 2023 05:22:51 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitQ1-0001xF-Ee; Wed, 20 Sep 2023 05:22:50 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitKz-0028Y8-2r; Wed, 20 Sep 2023 11:21:02 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 13/22] tests: fix broken deduplication check in parallels format test Date: Wed, 20 Sep 2023 11:20:59 +0200 Message-Id: <20230920092108.258898-14-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201895494100001 Content-Type: text/plain; charset="utf-8" Original check is broken as supposed reading from 2 different clusters results in read from the same file offset twice. This is definitely wrong. We should be sure that * the content of both clusters is correct after repair * clusters are at the different offsets after repair In order to check the latter we write some content into the first one and validate that fact. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- tests/qemu-iotests/tests/parallels-checks | 14 ++++++++++---- tests/qemu-iotests/tests/parallels-checks.out | 16 ++++++++++++---- 2 files changed, 22 insertions(+), 8 deletions(-) diff --git a/tests/qemu-iotests/tests/parallels-checks b/tests/qemu-iotests= /tests/parallels-checks index f4ca50295e..df99558486 100755 --- a/tests/qemu-iotests/tests/parallels-checks +++ b/tests/qemu-iotests/tests/parallels-checks @@ -117,14 +117,20 @@ echo "=3D=3D check second cluster =3D=3D" echo "=3D=3D repair image =3D=3D" _check_test_img -r all =20 +echo "=3D=3D check the first cluster =3D=3D" +{ $QEMU_IO -r -c "read -P 0x11 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _fil= ter_qemu_io | _filter_testdir + echo "=3D=3D check second cluster =3D=3D" { $QEMU_IO -r -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; }= 2>&1 | _filter_qemu_io | _filter_testdir =20 -echo "=3D=3D check first cluster on host =3D=3D" -printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1` +echo "=3D=3D write another pattern to the first clusters =3D=3D" +{ $QEMU_IO -c "write -P 0x66 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filte= r_qemu_io | _filter_testdir + +echo "=3D=3D check the first cluster =3D=3D" +{ $QEMU_IO -r -c "read -P 0x66 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _fil= ter_qemu_io | _filter_testdir =20 -echo "=3D=3D check second cluster on host =3D=3D" -printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1` +echo "=3D=3D check the second cluster (deduplicated) =3D=3D" +{ $QEMU_IO -r -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; }= 2>&1 | _filter_qemu_io | _filter_testdir =20 # Clear image _make_test_img $SIZE diff --git a/tests/qemu-iotests/tests/parallels-checks.out b/tests/qemu-iot= ests/tests/parallels-checks.out index 74a5e29260..1325d2b611 100644 --- a/tests/qemu-iotests/tests/parallels-checks.out +++ b/tests/qemu-iotests/tests/parallels-checks.out @@ -55,13 +55,21 @@ The following inconsistencies were found and repaired: =20 Double checking the fixed image now... No errors were found on the image. +=3D=3D check the first cluster =3D=3D +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) =3D=3D check second cluster =3D=3D read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -=3D=3D check first cluster on host =3D=3D -content: 0x11 -=3D=3D check second cluster on host =3D=3D -content: 0x11 +=3D=3D write another pattern to the first clusters =3D=3D +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check the first cluster =3D=3D +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check the second cluster (deduplicated) =3D=3D +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D4194304 =3D=3D TEST DATA_OFF CHECK =3D=3D =3D=3D write pattern to first cluster =3D=3D --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16952018044527.319522696882814; Wed, 20 Sep 2023 02:23:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitQA-0003G9-VN; Wed, 20 Sep 2023 05:22:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitQA-0003EV-7A; Wed, 20 Sep 2023 05:22:58 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitQ8-00021A-DM; Wed, 20 Sep 2023 05:22:57 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitL0-0028Y8-0k; Wed, 20 Sep 2023 11:21:02 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 14/22] tests: test self-cure of parallels image with duplicated clusters Date: Wed, 20 Sep 2023 11:21:00 +0200 Message-Id: <20230920092108.258898-15-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201805967100009 Content-Type: text/plain; charset="utf-8" The test is quite similar with the original one for duplicated clusters. There is the only difference in the operation which should fix the image. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- tests/qemu-iotests/tests/parallels-checks | 36 +++++++++++++++++++ tests/qemu-iotests/tests/parallels-checks.out | 31 ++++++++++++++++ 2 files changed, 67 insertions(+) diff --git a/tests/qemu-iotests/tests/parallels-checks b/tests/qemu-iotests= /tests/parallels-checks index df99558486..b281246a42 100755 --- a/tests/qemu-iotests/tests/parallels-checks +++ b/tests/qemu-iotests/tests/parallels-checks @@ -135,6 +135,42 @@ echo "=3D=3D check the second cluster (deduplicated) = =3D=3D" # Clear image _make_test_img $SIZE =20 +echo "=3D=3D TEST DUPLICATION SELF-CURE =3D=3D" + +echo "=3D=3D write pattern to whole image =3D=3D" +{ $QEMU_IO -c "write -P 0x11 0 $SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_i= o | _filter_testdir + +echo "=3D=3D write another pattern to second cluster =3D=3D" +{ $QEMU_IO -c "write -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2= >&1 | _filter_qemu_io | _filter_testdir + +echo "=3D=3D check second cluster =3D=3D" +{ $QEMU_IO -r -c "read -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; }= 2>&1 | _filter_qemu_io | _filter_testdir + + +echo "=3D=3D corrupt image =3D=3D" +poke_file "$TEST_IMG" "$(($BAT_OFFSET + 4))" "\x01\x00\x00\x00" + +echo "=3D=3D check second cluster =3D=3D" +{ $QEMU_IO -r -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; }= 2>&1 | _filter_qemu_io | _filter_testdir + +echo "=3D=3D check the first cluster with self-repair =3D=3D" +{ $QEMU_IO -c "read -P 0x11 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter= _qemu_io | _filter_testdir + +echo "=3D=3D check second cluster =3D=3D" +{ $QEMU_IO -r -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; }= 2>&1 | _filter_qemu_io | _filter_testdir + +echo "=3D=3D write another pattern to the first clusters =3D=3D" +{ $QEMU_IO -c "write -P 0x66 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filte= r_qemu_io | _filter_testdir + +echo "=3D=3D check the first cluster =3D=3D" +{ $QEMU_IO -r -c "read -P 0x66 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _fil= ter_qemu_io | _filter_testdir + +echo "=3D=3D check the second cluster (deduplicated) =3D=3D" +{ $QEMU_IO -r -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; }= 2>&1 | _filter_qemu_io | _filter_testdir + +# Clear image +_make_test_img $SIZE + echo "=3D=3D TEST DATA_OFF CHECK =3D=3D" =20 echo "=3D=3D write pattern to first cluster =3D=3D" diff --git a/tests/qemu-iotests/tests/parallels-checks.out b/tests/qemu-iot= ests/tests/parallels-checks.out index 1325d2b611..9793423111 100644 --- a/tests/qemu-iotests/tests/parallels-checks.out +++ b/tests/qemu-iotests/tests/parallels-checks.out @@ -71,6 +71,37 @@ read 1048576/1048576 bytes at offset 0 read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D4194304 +=3D=3D TEST DUPLICATION SELF-CURE =3D=3D +=3D=3D write pattern to whole image =3D=3D +wrote 4194304/4194304 bytes at offset 0 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D write another pattern to second cluster =3D=3D +wrote 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check second cluster =3D=3D +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D corrupt image =3D=3D +=3D=3D check second cluster =3D=3D +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check the first cluster with self-repair =3D=3D +Repairing duplicate offset in BAT entry 1 +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check second cluster =3D=3D +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D write another pattern to the first clusters =3D=3D +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check the first cluster =3D=3D +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check the second cluster (deduplicated) =3D=3D +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D4194304 =3D=3D TEST DATA_OFF CHECK =3D=3D =3D=3D write pattern to first cluster =3D=3D wrote 1048576/1048576 bytes at offset 0 --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201841573135.07173996195684; Wed, 20 Sep 2023 02:24:01 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitP5-0007Yy-Cn; Wed, 20 Sep 2023 05:21:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitP3-0007YE-96; Wed, 20 Sep 2023 05:21:49 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitP1-0001qk-PX; Wed, 20 Sep 2023 05:21:49 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitL0-0028Y8-1q; Wed, 20 Sep 2023 11:21:03 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 15/22] parallels: accept multiple clusters in mark_used() Date: Wed, 20 Sep 2023 11:21:01 +0200 Message-Id: <20230920092108.258898-16-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201842450100003 Content-Type: text/plain; charset="utf-8" This would be useful in the next patch in allocate_clusters(). This change would not imply serious performance drawbacks as usually image is full of data or are at the end of the bitmap. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index c41511398f..b6505fcc5b 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -178,18 +178,20 @@ static void parallels_set_bat_entry(BDRVParallelsStat= e *s, bitmap_set(s->bat_dirty_bmap, bat_entry_off(index) / s->bat_dirty_bloc= k, 1); } =20 -static int mark_used(BlockDriverState *bs, - unsigned long *bitmap, uint32_t bitmap_size, int64_t = off) +static int mark_used(BlockDriverState *bs, unsigned long *bitmap, + uint32_t bitmap_size, int64_t off, uint32_t count) { BDRVParallelsState *s =3D bs->opaque; uint32_t cluster_index =3D host_cluster_index(s, off); - if (cluster_index >=3D bitmap_size) { + unsigned long next_used; + if (cluster_index + count > bitmap_size) { return -E2BIG; } - if (test_bit(cluster_index, bitmap)) { + next_used =3D find_next_bit(bitmap, bitmap_size, cluster_index); + if (next_used < cluster_index + count) { return -EBUSY; } - bitmap_set(bitmap, cluster_index, 1); + bitmap_set(bitmap, cluster_index, count); return 0; } =20 @@ -230,7 +232,7 @@ static int parallels_fill_used_bitmap(BlockDriverState = *bs) continue; } =20 - err2 =3D mark_used(bs, s->used_bmap, s->used_bmap_size, host_off); + err2 =3D mark_used(bs, s->used_bmap, s->used_bmap_size, host_off, = 1); if (err2 < 0 && err =3D=3D 0) { err =3D err2; } @@ -732,7 +734,7 @@ parallels_check_duplicate(BlockDriverState *bs, BdrvChe= ckResult *res, continue; } =20 - ret =3D mark_used(bs, bitmap, bitmap_size, host_off); + ret =3D mark_used(bs, bitmap, bitmap_size, host_off, 1); assert(ret !=3D -E2BIG); if (ret =3D=3D 0) { continue; @@ -792,7 +794,7 @@ parallels_check_duplicate(BlockDriverState *bs, BdrvChe= ckResult *res, * considered, and the bitmap size doesn't change. This specifical= ly * means that -E2BIG is OK. */ - ret =3D mark_used(bs, bitmap, bitmap_size, host_off); + ret =3D mark_used(bs, bitmap, bitmap_size, host_off, 1); if (ret =3D=3D -EBUSY) { res->check_errors++; goto out_repair_bat; --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201762923188.60420139069868; Wed, 20 Sep 2023 02:22:42 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitPe-000074-9S; Wed, 20 Sep 2023 05:22:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPc-0008W3-Jv; Wed, 20 Sep 2023 05:22:24 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPa-0001tg-2b; Wed, 20 Sep 2023 05:22:23 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitL0-0028Y8-2w; Wed, 20 Sep 2023 11:21:03 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 16/22] parallels: update used bitmap in allocate_cluster Date: Wed, 20 Sep 2023 11:21:02 +0200 Message-Id: <20230920092108.258898-17-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201764435100003 Content-Type: text/plain; charset="utf-8" We should extend the bitmap if the file is extended and set the bit in the image used bitmap once the cluster is allocated. Sanity check at that moment also looks like a good idea. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/block/parallels.c b/block/parallels.c index b6505fcc5b..3beb18e44f 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -282,6 +282,8 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_= num, return len; } if (s->data_end + space > (len >> BDRV_SECTOR_BITS)) { + uint32_t new_usedsize; + space +=3D s->prealloc_size; /* * We require the expanded size to read back as zero. If the @@ -305,6 +307,12 @@ allocate_clusters(BlockDriverState *bs, int64_t sector= _num, if (ret < 0) { return ret; } + + new_usedsize =3D s->used_bmap_size + + (space << BDRV_SECTOR_BITS) / s->cluster_size; + s->used_bmap =3D bitmap_zero_extend(s->used_bmap, s->used_bmap_siz= e, + new_usedsize); + s->used_bmap_size =3D new_usedsize; } =20 /* @@ -336,6 +344,12 @@ allocate_clusters(BlockDriverState *bs, int64_t sector= _num, } } =20 + ret =3D mark_used(bs, s->used_bmap, s->used_bmap_size, + s->data_end << BDRV_SECTOR_BITS, to_allocate); + if (ret < 0) { + /* Image consistency is broken. Alarm! */ + return ret; + } for (i =3D 0; i < to_allocate; i++) { parallels_set_bat_entry(s, idx + i, s->data_end / s->off_multiplie= r); s->data_end +=3D s->tracks; --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201832926583.2156905059039; Wed, 20 Sep 2023 02:23:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitPm-0000JM-TN; Wed, 20 Sep 2023 05:22:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPl-0000Ic-8s; Wed, 20 Sep 2023 05:22:33 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPj-0001tA-Mq; Wed, 20 Sep 2023 05:22:33 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitL1-0028Y8-0o; Wed, 20 Sep 2023 11:21:03 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 17/22] parallels: naive implementation of allocate_clusters with used bitmap Date: Wed, 20 Sep 2023 11:21:03 +0200 Message-Id: <20230920092108.258898-18-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201834403100003 Content-Type: text/plain; charset="utf-8" The access to the bitmap is not optimized completely. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 51 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 39 insertions(+), 12 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 3beb18e44f..6a5bff4fcb 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -253,7 +253,7 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_= num, { int ret =3D 0; BDRVParallelsState *s =3D bs->opaque; - int64_t pos, space, idx, to_allocate, i, len; + int64_t i, pos, idx, to_allocate, first_free, host_off; =20 pos =3D block_status(s, sector_num, nb_sectors, pnum); if (pos > 0) { @@ -276,15 +276,13 @@ allocate_clusters(BlockDriverState *bs, int64_t secto= r_num, */ assert(idx < s->bat_size && idx + to_allocate <=3D s->bat_size); =20 - space =3D to_allocate * s->tracks; - len =3D bdrv_co_getlength(bs->file->bs); - if (len < 0) { - return len; - } - if (s->data_end + space > (len >> BDRV_SECTOR_BITS)) { + first_free =3D find_first_zero_bit(s->used_bmap, s->used_bmap_size); + if (first_free =3D=3D s->used_bmap_size) { uint32_t new_usedsize; + int64_t space =3D to_allocate * s->tracks + s->prealloc_size; + + host_off =3D s->data_end * BDRV_SECTOR_SIZE; =20 - space +=3D s->prealloc_size; /* * We require the expanded size to read back as zero. If the * user permitted truncation, we try that; but if it fails, we @@ -313,6 +311,32 @@ allocate_clusters(BlockDriverState *bs, int64_t sector= _num, s->used_bmap =3D bitmap_zero_extend(s->used_bmap, s->used_bmap_siz= e, new_usedsize); s->used_bmap_size =3D new_usedsize; + } else { + int64_t next_used; + next_used =3D find_next_bit(s->used_bmap, s->used_bmap_size, first= _free); + + /* Not enough continuous clusters in the middle, adjust the size */ + if (next_used - first_free < to_allocate) { + to_allocate =3D next_used - first_free; + *pnum =3D (idx + to_allocate) * s->tracks - sector_num; + } + + host_off =3D s->data_start * BDRV_SECTOR_SIZE; + host_off +=3D first_free * s->cluster_size; + + /* + * No need to preallocate if we are using tail area from the above + * branch. In the other case we are likely re-using hole. Prealloc= ate + * the space if required by the prealloc_mode. + */ + if (s->prealloc_mode =3D=3D PRL_PREALLOC_MODE_FALLOCATE && + host_off < s->data_end * BDRV_SECTOR_SIZE) { + ret =3D bdrv_co_pwrite_zeroes(bs->file, host_off, + s->cluster_size * to_allocate, 0); + if (ret < 0) { + return ret; + } + } } =20 /* @@ -344,15 +368,18 @@ allocate_clusters(BlockDriverState *bs, int64_t secto= r_num, } } =20 - ret =3D mark_used(bs, s->used_bmap, s->used_bmap_size, - s->data_end << BDRV_SECTOR_BITS, to_allocate); + ret =3D mark_used(bs, s->used_bmap, s->used_bmap_size, host_off, to_al= locate); if (ret < 0) { /* Image consistency is broken. Alarm! */ return ret; } for (i =3D 0; i < to_allocate; i++) { - parallels_set_bat_entry(s, idx + i, s->data_end / s->off_multiplie= r); - s->data_end +=3D s->tracks; + parallels_set_bat_entry(s, idx + i, + host_off / BDRV_SECTOR_SIZE / s->off_multiplier); + host_off +=3D s->cluster_size; + } + if (host_off > s->data_end * BDRV_SECTOR_SIZE) { + s->data_end =3D host_off / BDRV_SECTOR_SIZE; } =20 return bat2sect(s, idx) + sector_num % s->tracks; --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16952017573141014.8681670052256; Wed, 20 Sep 2023 02:22:37 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitPI-0007vm-M9; Wed, 20 Sep 2023 05:22:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPF-0007v3-Gp; Wed, 20 Sep 2023 05:22:01 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPE-0001s4-3j; Wed, 20 Sep 2023 05:22:01 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitL1-0028Y8-1v; Wed, 20 Sep 2023 11:21:04 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 18/22] parallels: improve readability of allocate_clusters Date: Wed, 20 Sep 2023 11:21:04 +0200 Message-Id: <20230920092108.258898-19-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201757864100003 Content-Type: text/plain; charset="utf-8" Replace 'space' representing the amount of data to preallocate with 'bytes'. Rationale: * 'space' at each place is converted to bytes * the unit is more close to the variable name Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 6a5bff4fcb..d9d36c514b 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -279,7 +279,8 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_= num, first_free =3D find_first_zero_bit(s->used_bmap, s->used_bmap_size); if (first_free =3D=3D s->used_bmap_size) { uint32_t new_usedsize; - int64_t space =3D to_allocate * s->tracks + s->prealloc_size; + int64_t bytes =3D to_allocate * s->cluster_size; + bytes +=3D s->prealloc_size * BDRV_SECTOR_SIZE; =20 host_off =3D s->data_end * BDRV_SECTOR_SIZE; =20 @@ -289,8 +290,7 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_= num, * force the safer-but-slower fallocate. */ if (s->prealloc_mode =3D=3D PRL_PREALLOC_MODE_TRUNCATE) { - ret =3D bdrv_co_truncate(bs->file, - (s->data_end + space) << BDRV_SECTOR_BI= TS, + ret =3D bdrv_co_truncate(bs->file, host_off + bytes, false, PREALLOC_MODE_OFF, BDRV_REQ_ZERO_WRITE, NULL); if (ret =3D=3D -ENOTSUP) { @@ -298,16 +298,13 @@ allocate_clusters(BlockDriverState *bs, int64_t secto= r_num, } } if (s->prealloc_mode =3D=3D PRL_PREALLOC_MODE_FALLOCATE) { - ret =3D bdrv_co_pwrite_zeroes(bs->file, - s->data_end << BDRV_SECTOR_BITS, - space << BDRV_SECTOR_BITS, 0); + ret =3D bdrv_co_pwrite_zeroes(bs->file, host_off, bytes, 0); } if (ret < 0) { return ret; } =20 - new_usedsize =3D s->used_bmap_size + - (space << BDRV_SECTOR_BITS) / s->cluster_size; + new_usedsize =3D s->used_bmap_size + bytes / s->cluster_size; s->used_bmap =3D bitmap_zero_extend(s->used_bmap, s->used_bmap_siz= e, new_usedsize); s->used_bmap_size =3D new_usedsize; --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16952019074931011.0409137114285; Wed, 20 Sep 2023 02:25:07 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitP1-0007XU-Lw; Wed, 20 Sep 2023 05:21:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOz-0007P4-Fg; Wed, 20 Sep 2023 05:21:45 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitOy-0001k2-0s; Wed, 20 Sep 2023 05:21:45 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitL1-0028Y8-31; Wed, 20 Sep 2023 11:21:04 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 19/22] parallels: naive implementation of parallels_co_pdiscard Date: Wed, 20 Sep 2023 11:21:05 +0200 Message-Id: <20230920092108.258898-20-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201909442100003 Content-Type: text/plain; charset="utf-8" * Discarding with backing stores is not supported by the format. * There is no buffering/queueing of the discard operation. * Only operations aligned to the cluster are supported. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/block/parallels.c b/block/parallels.c index d9d36c514b..1ef23f6669 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -537,6 +537,51 @@ parallels_co_readv(BlockDriverState *bs, int64_t secto= r_num, int nb_sectors, return ret; } =20 + +static int coroutine_fn +parallels_co_pdiscard(BlockDriverState *bs, int64_t offset, int64_t bytes) +{ + int ret =3D 0; + uint32_t cluster, count; + BDRVParallelsState *s =3D bs->opaque; + + /* + * The image does not support ZERO mark inside the BAT, which means th= at + * stale data could be exposed from the backing file. + */ + if (bs->backing) { + return -ENOTSUP; + } + + if (!QEMU_IS_ALIGNED(offset, s->cluster_size)) { + return -ENOTSUP; + } else if (!QEMU_IS_ALIGNED(bytes, s->cluster_size)) { + return -ENOTSUP; + } + + cluster =3D offset / s->cluster_size; + count =3D bytes / s->cluster_size; + + qemu_co_mutex_lock(&s->lock); + for (; count > 0; cluster++, count--) { + int64_t host_off =3D bat2sect(s, cluster) << BDRV_SECTOR_BITS; + if (host_off =3D=3D 0) { + continue; + } + + ret =3D bdrv_co_pdiscard(bs->file, host_off, s->cluster_size); + if (ret < 0) { + goto done; + } + + parallels_set_bat_entry(s, cluster, 0); + bitmap_clear(s->used_bmap, host_cluster_index(s, host_off), 1); + } +done: + qemu_co_mutex_unlock(&s->lock); + return ret; +} + static void parallels_check_unclean(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix) @@ -1417,6 +1462,7 @@ static BlockDriver bdrv_parallels =3D { .bdrv_co_create =3D parallels_co_create, .bdrv_co_create_opts =3D parallels_co_create_opts, .bdrv_co_check =3D parallels_co_check, + .bdrv_co_pdiscard =3D parallels_co_pdiscard, }; =20 static void bdrv_parallels_init(void) --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201840786540.2683137429364; Wed, 20 Sep 2023 02:24:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitPg-0000B0-Nw; Wed, 20 Sep 2023 05:22:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPf-00007x-28; Wed, 20 Sep 2023 05:22:27 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPd-0001u5-AM; Wed, 20 Sep 2023 05:22:26 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitL2-0028Y8-0t; Wed, 20 Sep 2023 11:21:04 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 20/22] tests: extend test 131 to cover availability of the discard operation Date: Wed, 20 Sep 2023 11:21:06 +0200 Message-Id: <20230920092108.258898-21-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201842456100004 Content-Type: text/plain; charset="utf-8" This patch contains test which minimally tests discard and new cluster allocation logic. The following checks are added: * write 2 clusters, discard the first allocated * write another cluster, check that the hole is filled * write 2 clusters, discard the first allocated, write 1 cluster at non-aligned to cluster offset (2 new clusters should be allocated) Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- tests/qemu-iotests/131 | 31 +++++++++++++++++++++++++++++++ tests/qemu-iotests/131.out | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/tests/qemu-iotests/131 b/tests/qemu-iotests/131 index 304bbb3f61..324008b3f6 100755 --- a/tests/qemu-iotests/131 +++ b/tests/qemu-iotests/131 @@ -74,6 +74,37 @@ poke_file "$TEST_IMG" "$inuse_offset" "\x59\x6e\x6f\x74" echo "=3D=3D read corrupted image with repairing =3D=3D" { $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>= &1 | _filter_qemu_io | _filter_testdir =20 +echo "=3D=3D check discard =3D=3D" + +# Clear image +_make_test_img $size + +{ $QEMU_IO -c "write -P 0x11 0 $CLUSTER_DBL_SIZE" "$TEST_IMG"; } 2>&1 | _f= ilter_qemu_io | _filter_testdir +{ $QEMU_IMG map "$TEST_IMG"; } 2>&1 | _filter_qemu_img_map +{ $QEMU_IO -c "discard 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu= _io | _filter_testdir +{ $QEMU_IMG map "$TEST_IMG"; } 2>&1 | _filter_qemu_img_map +{ $QEMU_IO -c "read -P 0 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qe= mu_io | _filter_testdir + +echo "=3D=3D check simple allocation over the discarded hole =3D=3D" + +{ $QEMU_IO -c "write -P 0x11 $CLUSTER_DBL_SIZE $CLUSTER_SIZE" "$TEST_IMG";= } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IMG map "$TEST_IMG"; } 2>&1 | _filter_qemu_img_map +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_DBL_SIZE $CLUSTER_SIZE" "$TEST_IMG"; = } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "=3D=3D check more complex allocation over the discard hole =3D=3D" + +# Clear image +_make_test_img $size + +{ $QEMU_IO -c "write -P 0x11 $CLUSTER_DBL_SIZE $CLUSTER_DBL_SIZE" "$TEST_I= MG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "discard $CLUSTER_DBL_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&= 1 | _filter_qemu_io | _filter_testdir +# There is 1 cluster hole. Fill it fully and allocate 1 cluster at the end +{ $QEMU_IO -c "write -P 0x12 $CLUSTER_HALF_SIZE $CLUSTER_SIZE" "$TEST_IMG"= ; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IMG map "$TEST_IMG"; } 2>&1 | _filter_qemu_img_map +{ $QEMU_IO -c "read -P 0x12 $CLUSTER_HALF_SIZE $CLUSTER_SIZE" "$TEST_IMG";= } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0 0 $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filt= er_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0 $((CLUSTER_SIZE + CLUSTER_HALF_SIZE)) $CLUSTER_HA= LF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + echo "=3D=3D allocate with backing =3D=3D" # Verify that allocating clusters works fine even when there is a backing = image. # Regression test for a bug where we would pass a buffer read from the bac= king diff --git a/tests/qemu-iotests/131.out b/tests/qemu-iotests/131.out index d2904578df..27df91ca97 100644 --- a/tests/qemu-iotests/131.out +++ b/tests/qemu-iotests/131.out @@ -26,6 +26,44 @@ read 524288/524288 bytes at offset 0 Repairing image was not closed correctly read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check discard =3D=3D +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +wrote 2097152/2097152 bytes at offset 0 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0 0x200000 TEST_DIR/t.IMGFMT +discard 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0x100000 0x100000 TEST_DIR/t.IMGFMT +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check simple allocation over the discarded hole =3D=3D +wrote 1048576/1048576 bytes at offset 2097152 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0x100000 0x100000 TEST_DIR/t.IMGFMT +0x200000 0x100000 TEST_DIR/t.IMGFMT +read 1048576/1048576 bytes at offset 2097152 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check more complex allocation over the discard hole =3D=3D +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +wrote 2097152/2097152 bytes at offset 2097152 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +discard 1048576/1048576 bytes at offset 2097152 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1048576/1048576 bytes at offset 524288 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0 0x100000 TEST_DIR/t.IMGFMT +0x100000 0x100000 TEST_DIR/t.IMGFMT +0x300000 0x100000 TEST_DIR/t.IMGFMT +read 1048576/1048576 bytes at offset 524288 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 524288/524288 bytes at offset 0 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 524288/524288 bytes at offset 1572864 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) =3D=3D allocate with backing =3D=3D Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D67108864 --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201785946771.0572365314325; Wed, 20 Sep 2023 02:23:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitPT-00086W-UU; Wed, 20 Sep 2023 05:22:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPK-0007xt-68; Wed, 20 Sep 2023 05:22:08 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitPG-0001sY-RH; Wed, 20 Sep 2023 05:22:05 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitL2-0028Y8-20; Wed, 20 Sep 2023 11:21:05 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 21/22] parallels: naive implementation of parallels_co_pwrite_zeroes Date: Wed, 20 Sep 2023 11:21:07 +0200 Message-Id: <20230920092108.258898-22-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201786824100001 Content-Type: text/plain; charset="utf-8" The zero flag is missed in the Parallels format specification. We can resort to discard if we have no backing file. Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- block/parallels.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/block/parallels.c b/block/parallels.c index 1ef23f6669..a6d64d0d47 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -582,6 +582,19 @@ done: return ret; } =20 +static int coroutine_fn +parallels_co_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int64_t b= ytes, + BdrvRequestFlags flags) +{ + /* + * The zero flag is missed in the Parallels format specification. We c= an + * resort to discard if we have no backing file (this condition is che= cked + * inside parallels_co_pdiscard(). + */ + return parallels_co_pdiscard(bs, offset, bytes); +} + + static void parallels_check_unclean(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix) @@ -1463,6 +1476,7 @@ static BlockDriver bdrv_parallels =3D { .bdrv_co_create_opts =3D parallels_co_create_opts, .bdrv_co_check =3D parallels_co_check, .bdrv_co_pdiscard =3D parallels_co_pdiscard, + .bdrv_co_pwrite_zeroes =3D parallels_co_pwrite_zeroes, }; =20 static void bdrv_parallels_init(void) --=20 2.34.1 From nobody Wed May 15 20:09:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1695201827632718.2713315355912; Wed, 20 Sep 2023 02:23:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qitPA-0007o7-2V; Wed, 20 Sep 2023 05:21:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitP8-0007gh-Jn; Wed, 20 Sep 2023 05:21:54 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qitP4-0001r5-Lj; Wed, 20 Sep 2023 05:21:54 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qitL2-0028Y8-36; Wed, 20 Sep 2023 11:21:05 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Alexander Ivanov Subject: [PULL 22/22] tests: extend test 131 to cover availability of the write-zeroes Date: Wed, 20 Sep 2023 11:21:08 +0200 Message-Id: <20230920092108.258898-23-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920092108.258898-1-den@openvz.org> References: <20230920092108.258898-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1695201828954100001 Content-Type: text/plain; charset="utf-8" This patch contains test which minimally tests write-zeroes on top of working discard. The following checks are added: * write 2 clusters, write-zero to the first allocated cluster * write 2 cluster, write-zero to the half the first allocated cluster Signed-off-by: Denis V. Lunev Reviewed-by: Alexander Ivanov --- tests/qemu-iotests/131 | 21 +++++++++++++++++++++ tests/qemu-iotests/131.out | 22 ++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/tests/qemu-iotests/131 b/tests/qemu-iotests/131 index 324008b3f6..3119100e78 100755 --- a/tests/qemu-iotests/131 +++ b/tests/qemu-iotests/131 @@ -105,6 +105,27 @@ _make_test_img $size { $QEMU_IO -c "read -P 0 0 $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filt= er_qemu_io | _filter_testdir { $QEMU_IO -c "read -P 0 $((CLUSTER_SIZE + CLUSTER_HALF_SIZE)) $CLUSTER_HA= LF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir =20 +echo "=3D=3D check write-zeroes =3D=3D" + +# Clear image +_make_test_img $size + +{ $QEMU_IO -c "write -P 0x11 0 $CLUSTER_DBL_SIZE" "$TEST_IMG"; } 2>&1 | _f= ilter_qemu_io | _filter_testdir +{ $QEMU_IO -c "write -z 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qem= u_io | _filter_testdir +{ $QEMU_IMG map "$TEST_IMG"; } 2>&1 | _filter_qemu_img_map +{ $QEMU_IO -c "read -P 0 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qe= mu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>= &1 | _filter_qemu_io | _filter_testdir + +echo "=3D=3D check cluster-partial write-zeroes =3D=3D" + +# Clear image +_make_test_img $size + +{ $QEMU_IO -c "write -P 0x11 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filte= r_qemu_io | _filter_testdir +{ $QEMU_IO -c "write -z 0 $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filte= r_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0 0 $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filt= er_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_HALF_SIZE" "$TEST_= IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + echo "=3D=3D allocate with backing =3D=3D" # Verify that allocating clusters works fine even when there is a backing = image. # Regression test for a bug where we would pass a buffer read from the bac= king diff --git a/tests/qemu-iotests/131.out b/tests/qemu-iotests/131.out index 27df91ca97..86a2d2a49b 100644 --- a/tests/qemu-iotests/131.out +++ b/tests/qemu-iotests/131.out @@ -64,6 +64,28 @@ read 524288/524288 bytes at offset 0 512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 524288/524288 bytes at offset 1572864 512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check write-zeroes =3D=3D +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +wrote 2097152/2097152 bytes at offset 0 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0x100000 0x100000 TEST_DIR/t.IMGFMT +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +=3D=3D check cluster-partial write-zeroes =3D=3D +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 524288/524288 bytes at offset 0 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 524288/524288 bytes at offset 0 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 524288/524288 bytes at offset 524288 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) =3D=3D allocate with backing =3D=3D Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D67108864 --=20 2.34.1