From nobody Thu Mar 28 20:29:27 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1575284310; cv=none; d=zohomail.com; s=zohoarc; b=W0559HHD5fdHLJrvTOwWinV3eGdRqZ2Uy9Mucx2F0Jb4gYGP+jPi9lIMBOmkyod9Z+TbGYiixvd4xerJC+gIeHNMzddRw92H2Yzj3qfFMdBjf3ctiDtxfFF52Xqychp9Yr/9Cxm6xdy3AJnwV8TA8LDda4v1RaKEwwWJBP8Yi7c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575284310; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=2JeyXMlWsa5MkpR4UV14vYBfUUvoyGxJ3vX92LkuhlE=; b=GDfsbJYjoLr1fwMfh8e5R8ZJl9TkWecDK/CKO5iLD479KVOALAURG5unPuyMpHAHc8cBfBUM/a9aa9wK+3hRgnI3cxGyLHwq2pmz6Q0Sf5lL6/CIJbf9Pq4EhvMWW6wyzDCzzWb5UxgkaEuJdD3p9qEdSGCWLVlNRq5NdM0pUc0= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1575284310681423.97185147936807; Mon, 2 Dec 2019 02:58:30 -0800 (PST) Received: from localhost ([::1]:33894 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ibjPR-0000fy-D4 for importer@patchew.org; Mon, 02 Dec 2019 05:58:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38322) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ibjOB-0007fS-KF for qemu-devel@nongnu.org; Mon, 02 Dec 2019 05:57:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ibjOA-0001Wt-0w for qemu-devel@nongnu.org; Mon, 02 Dec 2019 05:57:11 -0500 Received: from relay.sw.ru ([185.231.240.75]:40866) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ibjO9-0001Vl-Pm; Mon, 02 Dec 2019 05:57:09 -0500 Received: from dhcp-172-16-25-136.sw.ru ([172.16.25.136] helo=localhost.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1ibjO2-0003e4-JW; Mon, 02 Dec 2019 13:57:02 +0300 From: Andrey Shinkevich To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH v11 1/3] block: introduce compress filter driver Date: Mon, 2 Dec 2019 13:57:00 +0300 Message-Id: <1575284222-548831-2-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1575284222-548831-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1575284222-548831-1-git-send-email-andrey.shinkevich@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Allow writing all the data compressed through the filter driver. The written data will be aligned by the cluster size. Based on the QEMU current implementation, that data can be written to unallocated clusters only. May be used for a backup job. Suggested-by: Max Reitz Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/Makefile.objs | 1 + block/filter-compress.c | 168 ++++++++++++++++++++++++++++++++++++++++++++= ++++ qapi/block-core.json | 10 +-- 3 files changed, 175 insertions(+), 4 deletions(-) create mode 100644 block/filter-compress.c diff --git a/block/Makefile.objs b/block/Makefile.objs index e394fe0..330529b 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -43,6 +43,7 @@ block-obj-y +=3D crypto.o =20 block-obj-y +=3D aio_task.o block-obj-y +=3D backup-top.o +block-obj-y +=3D filter-compress.o =20 common-obj-y +=3D stream.o =20 diff --git a/block/filter-compress.c b/block/filter-compress.c new file mode 100644 index 0000000..3e89249 --- /dev/null +++ b/block/filter-compress.c @@ -0,0 +1,168 @@ +/* + * Compress filter block driver + * + * Copyright (c) 2019 Virtuozzo International GmbH + * + * Author: + * Andrey Shinkevich + * (based on block/copy-on-read.c by Max Reitz) + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 or + * (at your option) any later version of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, see . + */ + +#include "qemu/osdep.h" +#include "block/block_int.h" +#include "qemu/module.h" +#include "qapi/error.h" + + +static int compress_open(BlockDriverState *bs, QDict *options, int flags, + Error **errp) +{ + bs->file =3D bdrv_open_child(NULL, options, "file", bs, &child_file, f= alse, + errp); + if (!bs->file) { + return -EINVAL; + } + + if (!bs->file->bs->drv || !block_driver_can_compress(bs->file->bs->drv= )) { + error_setg(errp, + "Compression is not supported for underlying format: %s= ", + bdrv_get_format_name(bs->file->bs)); + + return -ENOTSUP; + } + + bs->supported_write_flags =3D BDRV_REQ_WRITE_UNCHANGED | + (BDRV_REQ_FUA & bs->file->bs->supported_write_flags); + + bs->supported_zero_flags =3D BDRV_REQ_WRITE_UNCHANGED | + ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & + bs->file->bs->supported_zero_flags); + + return 0; +} + + +static int64_t compress_getlength(BlockDriverState *bs) +{ + return bdrv_getlength(bs->file->bs); +} + + +static int coroutine_fn compress_co_preadv_part(BlockDriverState *bs, + uint64_t offset, uint64_t = bytes, + QEMUIOVector *qiov, + size_t qiov_offset, + int flags) +{ + return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset, + flags); +} + + +static int coroutine_fn compress_co_pwritev_part(BlockDriverState *bs, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, + size_t qiov_offset, int f= lags) +{ + return bdrv_co_pwritev_part(bs->file, offset, bytes, qiov, qiov_offset, + flags | BDRV_REQ_WRITE_COMPRESSED); +} + + +static int coroutine_fn compress_co_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, int byte= s, + BdrvRequestFlags flags) +{ + return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags); +} + + +static int coroutine_fn compress_co_pdiscard(BlockDriverState *bs, + int64_t offset, int bytes) +{ + return bdrv_co_pdiscard(bs->file, offset, bytes); +} + + +static void compress_refresh_limits(BlockDriverState *bs, Error **errp) +{ + BlockDriverInfo bdi; + int ret; + + if (!bs->file) { + return; + } + + ret =3D bdrv_get_info(bs->file->bs, &bdi); + if (ret < 0 || bdi.cluster_size =3D=3D 0) { + return; + } + + bs->bl.request_alignment =3D bdi.cluster_size; +} + + +static void compress_eject(BlockDriverState *bs, bool eject_flag) +{ + bdrv_eject(bs->file->bs, eject_flag); +} + + +static void compress_lock_medium(BlockDriverState *bs, bool locked) +{ + bdrv_lock_medium(bs->file->bs, locked); +} + + +static bool compress_recurse_is_first_non_filter(BlockDriverState *bs, + BlockDriverState *candida= te) +{ + return bdrv_recurse_is_first_non_filter(bs->file->bs, candidate); +} + + +static BlockDriver bdrv_compress =3D { + .format_name =3D "compress", + + .bdrv_open =3D compress_open, + .bdrv_child_perm =3D bdrv_filter_default_perms, + + .bdrv_getlength =3D compress_getlength, + + .bdrv_co_preadv_part =3D compress_co_preadv_part, + .bdrv_co_pwritev_part =3D compress_co_pwritev_part, + .bdrv_co_pwrite_zeroes =3D compress_co_pwrite_zeroes, + .bdrv_co_pdiscard =3D compress_co_pdiscard, + .bdrv_refresh_limits =3D compress_refresh_limits, + + .bdrv_eject =3D compress_eject, + .bdrv_lock_medium =3D compress_lock_medium, + + .bdrv_co_block_status =3D bdrv_co_block_status_from_back= ing, + + .bdrv_recurse_is_first_non_filter =3D compress_recurse_is_first_non_= filter, + + .has_variable_length =3D true, + .is_filter =3D true, +}; + +static void bdrv_compress_init(void) +{ + bdrv_register(&bdrv_compress); +} + +block_init(bdrv_compress_init); diff --git a/qapi/block-core.json b/qapi/block-core.json index 0cf68fe..93ee04e 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2884,15 +2884,16 @@ # @copy-on-read: Since 3.0 # @blklogwrites: Since 3.0 # @blkreplay: Since 4.2 +# @compress: Since 5.0 # # Since: 2.9 ## { 'enum': 'BlockdevDriver', 'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs', - 'cloop', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', 'gluste= r', - 'host_cdrom', 'host_device', 'http', 'https', 'iscsi', 'luks', - 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', 'qco= w', - 'qcow2', 'qed', 'quorum', 'raw', 'rbd', + 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ft= ps', + 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscs= i', + 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallel= s', + 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd', { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' }, 'sheepdog', 'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat', 'vxh= s' ] } @@ -4044,6 +4045,7 @@ 'blkreplay': 'BlockdevOptionsBlkreplay', 'bochs': 'BlockdevOptionsGenericFormat', 'cloop': 'BlockdevOptionsGenericFormat', + 'compress': 'BlockdevOptionsGenericFormat', 'copy-on-read':'BlockdevOptionsGenericFormat', 'dmg': 'BlockdevOptionsGenericFormat', 'file': 'BlockdevOptionsFile', --=20 1.8.3.1 From nobody Thu Mar 28 20:29:27 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1575284306; cv=none; d=zohomail.com; s=zohoarc; b=NrjQ2roQNZDLXnm2gkkd3qLPdMNc6tUfaej451bOVTnGWfKENime/4ySAuNT8OXkvo95byPMhPTgr+KA2Ho8/6WtOp0lOaW3aQoN3A3QdysI2De/5eWaCoTaRdGu9gBLtRcxjkMDIFc9QxpRcwrSdHoMHB65TRhk1yoA4EsBF08= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575284306; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=qWFXhcTjO2N05toANN86TuF1YA2S54nCLhLqvxxEIls=; b=BkjSHq0Fao4180EbjNHWpu9tuWSvmaeok6KUh1BvGInWy4g4XyEpUgnX0Tmrxp6yAxLXr5zwf+2z9hekvE2d7NF4XHF1izabrMQFzavQQ/uLzT1Ga6DW8jevhpxm76t/t6674lMRTp4p6l+zFfPsHQkNazdfAU8XJuUvq3grEw4= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1575284306630590.3133291335221; Mon, 2 Dec 2019 02:58:26 -0800 (PST) Received: from localhost ([::1]:33892 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ibjPM-0000WK-Fg for importer@patchew.org; Mon, 02 Dec 2019 05:58:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38315) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ibjOB-0007ew-57 for qemu-devel@nongnu.org; Mon, 02 Dec 2019 05:57:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ibjO9-0001WW-SW for qemu-devel@nongnu.org; Mon, 02 Dec 2019 05:57:11 -0500 Received: from relay.sw.ru ([185.231.240.75]:40856) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ibjO9-0001Vn-Kq; Mon, 02 Dec 2019 05:57:09 -0500 Received: from dhcp-172-16-25-136.sw.ru ([172.16.25.136] helo=localhost.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1ibjO2-0003e4-MT; Mon, 02 Dec 2019 13:57:02 +0300 From: Andrey Shinkevich To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH v11 2/3] qcow2: Allow writing compressed data of multiple clusters Date: Mon, 2 Dec 2019 13:57:01 +0300 Message-Id: <1575284222-548831-3-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1575284222-548831-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1575284222-548831-1-git-send-email-andrey.shinkevich@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" QEMU currently supports writing compressed data of the size equal to one cluster. This patch allows writing QCOW2 compressed data that exceed one cluster. Now, we split buffered data into separate clusters and write them compressed using the block/aio_task API. Suggested-by: Pavel Butsykin Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2.c | 102 ++++++++++++++++++++++++++++++++++++++++++------------= ---- 1 file changed, 75 insertions(+), 27 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 7c18721..0e03a1a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4222,10 +4222,8 @@ fail: return ret; } =20 -/* XXX: put compressed sectors first, then all the cluster aligned - tables to avoid losing bytes in alignment */ static coroutine_fn int -qcow2_co_pwritev_compressed_part(BlockDriverState *bs, +qcow2_co_pwritev_compressed_task(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, size_t qiov_offset) { @@ -4235,32 +4233,11 @@ qcow2_co_pwritev_compressed_part(BlockDriverState *= bs, uint8_t *buf, *out_buf; uint64_t cluster_offset; =20 - if (has_data_file(bs)) { - return -ENOTSUP; - } - - if (bytes =3D=3D 0) { - /* align end of file to a sector boundary to ease reading with - sector based I/Os */ - int64_t len =3D bdrv_getlength(bs->file->bs); - if (len < 0) { - return len; - } - return bdrv_co_truncate(bs->file, len, false, PREALLOC_MODE_OFF, N= ULL); - } - - if (offset_into_cluster(s, offset)) { - return -EINVAL; - } + assert(bytes =3D=3D s->cluster_size || (bytes < s->cluster_size && + (offset + bytes =3D=3D bs->total_sectors << BDRV_SECTOR_BITS))); =20 buf =3D qemu_blockalign(bs, s->cluster_size); - if (bytes !=3D s->cluster_size) { - if (bytes > s->cluster_size || - offset + bytes !=3D bs->total_sectors << BDRV_SECTOR_BITS) - { - qemu_vfree(buf); - return -EINVAL; - } + if (bytes < s->cluster_size) { /* Zero-pad last write if image size is not cluster aligned */ memset(buf + bytes, 0, s->cluster_size - bytes); } @@ -4309,6 +4286,77 @@ fail: return ret; } =20 +static coroutine_fn int qcow2_co_pwritev_compressed_task_entry(AioTask *ta= sk) +{ + Qcow2AioTask *t =3D container_of(task, Qcow2AioTask, task); + + assert(!t->cluster_type && !t->l2meta); + + return qcow2_co_pwritev_compressed_task(t->bs, t->offset, t->bytes, t-= >qiov, + t->qiov_offset); +} + +/* + * XXX: put compressed sectors first, then all the cluster aligned + * tables to avoid losing bytes in alignment + */ +static coroutine_fn int +qcow2_co_pwritev_compressed_part(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, size_t qiov_offset) +{ + BDRVQcow2State *s =3D bs->opaque; + AioTaskPool *aio =3D NULL; + int ret =3D 0; + + if (has_data_file(bs)) { + return -ENOTSUP; + } + + if (bytes =3D=3D 0) { + /* + * align end of file to a sector boundary to ease reading with + * sector based I/Os + */ + int64_t len =3D bdrv_getlength(bs->file->bs); + if (len < 0) { + return len; + } + return bdrv_co_truncate(bs->file, len, false, PREALLOC_MODE_OFF, N= ULL); + } + + if (offset_into_cluster(s, offset)) { + return -EINVAL; + } + + while (bytes && aio_task_pool_status(aio) =3D=3D 0) { + uint64_t chunk_size =3D MIN(bytes, s->cluster_size); + + if (!aio && chunk_size !=3D bytes) { + aio =3D aio_task_pool_new(QCOW2_MAX_WORKERS); + } + + ret =3D qcow2_add_task(bs, aio, qcow2_co_pwritev_compressed_task_e= ntry, + 0, 0, offset, chunk_size, qiov, qiov_offset, = NULL); + if (ret < 0) { + break; + } + qiov_offset +=3D chunk_size; + offset +=3D chunk_size; + bytes -=3D chunk_size; + } + + if (aio) { + aio_task_pool_wait_all(aio); + if (ret =3D=3D 0) { + ret =3D aio_task_pool_status(aio); + } + g_free(aio); + } + + return ret; +} + static int coroutine_fn qcow2_co_preadv_compressed(BlockDriverState *bs, uint64_t file_cluster_offset, --=20 1.8.3.1 From nobody Thu Mar 28 20:29:27 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1575284390; cv=none; d=zohomail.com; s=zohoarc; b=gJDrFiMJ4Ovvjuhid89oWS0AfqrHTzRMzv6ClM2MQobzsEaN6foHmjR3r9olLzcRPo1AKkmKPmrBTIN+UO2Cl8kRYXyDErcxI2mHutX9orPJE1sUC/gu+7Ft3BgT8NT2Z7DLA3sX8g9NFDSwl05GBJLD4lrIENYg9p+S1eZKmnk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575284390; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=J1CRUS/V0RzBt1yJXe3Fk65ahC0SoaMjchaNc8EA/mI=; b=GrVkENBXDFXl63+9IyE5IrEAg3h8TAfqxXA9tJQZXzABup2MLSL4MfoTR8X7jG6ZGOREmg+nWt1ylJ6TE92RTTG25NB6VYRrV2PH3Mr2W8tWAGQKMuRnpK8FLiK7CFbWFrkmfZ6ZEB96SRQuExZPdyGT9LojRkMQQrLWp3P/sOw= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1575284390838639.1999159206807; Mon, 2 Dec 2019 02:59:50 -0800 (PST) Received: from localhost ([::1]:33938 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ibjQj-0002df-N5 for importer@patchew.org; Mon, 02 Dec 2019 05:59:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38314) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ibjOB-0007es-7I for qemu-devel@nongnu.org; Mon, 02 Dec 2019 05:57:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ibjO9-0001Wj-TV for qemu-devel@nongnu.org; Mon, 02 Dec 2019 05:57:11 -0500 Received: from relay.sw.ru ([185.231.240.75]:40854) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ibjO9-0001Vo-Lo; Mon, 02 Dec 2019 05:57:09 -0500 Received: from dhcp-172-16-25-136.sw.ru ([172.16.25.136] helo=localhost.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.3) (envelope-from ) id 1ibjO2-0003e4-Sn; Mon, 02 Dec 2019 13:57:02 +0300 From: Andrey Shinkevich To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH v11 3/3] tests/qemu-iotests: add case to write compressed data of multiple clusters Date: Mon, 2 Dec 2019 13:57:02 +0300 Message-Id: <1575284222-548831-4-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1575284222-548831-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1575284222-548831-1-git-send-email-andrey.shinkevich@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add the case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. The test case involves the compress filter driver showing a sample usage of that. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/214 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/214.out | 14 ++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214 index 21ec8a2..0b6ea0b 100755 --- a/tests/qemu-iotests/214 +++ b/tests/qemu-iotests/214 @@ -89,6 +89,49 @@ _check_test_img -r all $QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir $QEMU_IO -c "read -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir =20 +echo +echo "=3D=3D=3D Write compressed data of multiple clusters =3D=3D=3D" +echo +cluster_size=3D0x10000 +_make_test_img 2M -o cluster_size=3D$cluster_size + +echo "Write uncompressed data:" +let data_size=3D"8 * $cluster_size" +$QEMU_IO -c "write -P 0xaa 0 $data_size" "$TEST_IMG" \ + 2>&1 | _filter_qemu_io | _filter_testdir +sizeA=3D$($QEMU_IMG info --output=3Djson "$TEST_IMG" | + sed -n '/"actual-size":/ s/[^0-9]//gp') + +_make_test_img 2M -o cluster_size=3D$cluster_size +echo "Write compressed data:" +let data_size=3D"3 * $cluster_size + $cluster_size / 2" +# Set compress on. That will align the written data +# by the cluster size and will write them compressed. +QEMU_IO_OPTIONS=3D$QEMU_IO_OPTIONS_NO_FMT \ +$QEMU_IO -c "write -P 0xbb 0 $data_size" --image-opts \ + "driver=3Dcompress,file.driver=3D$IMGFMT,file.file.driver=3Dfile,= file.file.filename=3D$TEST_IMG" \ + 2>&1 | _filter_qemu_io | _filter_testdir + +let offset=3D"4 * $cluster_size + $cluster_size / 4" +QEMU_IO_OPTIONS=3D$QEMU_IO_OPTIONS_NO_FMT \ +$QEMU_IO -c "write -P 0xcc $offset $data_size" "json:{\ + 'driver': 'compress', + 'file': {'driver': '$IMGFMT', + 'file': {'driver': 'file', + 'filename': '$TEST_IMG'}}}" | \ + _filter_qemu_io | _filter_testdir + +sizeB=3D$($QEMU_IMG info --output=3Djson "$TEST_IMG" | + sed -n '/"actual-size":/ s/[^0-9]//gp') + +if [ $sizeA -le $sizeB ] +then + echo "Compression ERROR" +fi + +$QEMU_IMG check --output=3Djson "$TEST_IMG" | + sed -n 's/,$//; /"compressed-clusters":/ s/^ *//p' + # success, all done echo '*** done' rm -f $seq.full diff --git a/tests/qemu-iotests/214.out b/tests/qemu-iotests/214.out index 0fcd8dc..9fc6728 100644 --- a/tests/qemu-iotests/214.out +++ b/tests/qemu-iotests/214.out @@ -32,4 +32,18 @@ read 4194304/4194304 bytes at offset 0 4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 4194304/4194304 bytes at offset 4194304 4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=3D=3D=3D Write compressed data of multiple clusters =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D2097152 +Write uncompressed data: +wrote 524288/524288 bytes at offset 0 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D2097152 +Write compressed data: +wrote 229376/229376 bytes at offset 0 +224 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 229376/229376 bytes at offset 278528 +224 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +"compressed-clusters": 8 *** done --=20 1.8.3.1