From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1554219751425834.294169113381; Tue, 2 Apr 2019 08:42:31 -0700 (PDT) Received: from localhost ([127.0.0.1]:38116 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLYP-0005E0-Co for importer@patchew.org; Tue, 02 Apr 2019 11:42:25 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49570) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTm-0001eX-6y for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTk-0000Vb-Mu for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:38 -0400 Received: from relay.sw.ru ([185.231.240.75]:43480) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTk-0000Ti-AJ; Tue, 02 Apr 2019 11:37:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTf-0001s1-F7; Tue, 02 Apr 2019 18:37:31 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:21 +0300 Message-Id: <20190402153730.54145-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 01/10] tests/perf: Test qemu-img convert from raw to encrypted qcow2 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/perf/block/qcow2/convert-to-encrypted | 48 +++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100755 tests/perf/block/qcow2/convert-to-encrypted diff --git a/tests/perf/block/qcow2/convert-to-encrypted b/tests/perf/block= /qcow2/convert-to-encrypted new file mode 100755 index 0000000000..7a6b7b1cab --- /dev/null +++ b/tests/perf/block/qcow2/convert-to-encrypted @@ -0,0 +1,48 @@ +#!/bin/bash +# +# Test qemu-img convert from raw to encrypted qcow2 +# +# Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +if [ "$#" -lt 2 ]; then + echo "Usage: $0 SOURCE_FILE DESTINATION_FILE [additional qemu-img conv= ert parameters]" + exit 1 +fi + +ROOT_DIR=3D"$( cd "$( dirname "${BASH_SOURCE[0]}" )/../../../.." >/dev/nul= l 2>&1 && pwd )" +QEMU_IMG=3D"$ROOT_DIR/qemu-img" +QEMU_IO=3D"$ROOT_DIR/qemu-io" + +size=3D1G + +src=3D"$1" +shift + +dst=3D"$1" +shift + +( +# create source +$QEMU_IMG create -f raw "$src" $size +$QEMU_IO -f raw -c "write -P 0xa 0 $size" "$src" + +# create target +$QEMU_IMG create -f qcow2 --object secret,id=3Dsec0,data=3Dtest -o encrypt= .format=3Dluks,encrypt.key-secret=3Dsec0 "$dst" $size +) > /dev/null + +# test with additional parameters left in $@ +/usr/bin/time -f %e $QEMU_IMG convert "$@" -f raw --object secret,id=3Dsec= 0,data=3Dtest --target-image-opts -n "$src" "driver=3Dqcow2,file.filename= =3D$dst,encrypt.key-secret=3Dsec0" --=20 2.18.0 From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1554219791145429.2315225977476; Tue, 2 Apr 2019 08:43:11 -0700 (PDT) Received: from localhost ([127.0.0.1]:38295 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLZ4-0005jY-0c for importer@patchew.org; Tue, 02 Apr 2019 11:43:06 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49555) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTl-0001eR-TT for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTk-0000V2-L9 for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:37 -0400 Received: from relay.sw.ru ([185.231.240.75]:43478) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTk-0000Td-9W; Tue, 02 Apr 2019 11:37:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTf-0001s1-NC; Tue, 02 Apr 2019 18:37:31 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:22 +0300 Message-Id: <20190402153730.54145-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 02/10] qcow2.h: add missing include X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" qcow2.h depends on block_int.h. Compilation isn't broken currently only due to block_int.h always included before qcow2.h. Though, it seems better to directly include block_int.h in qcow2.h. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 1 + block/qcow2-bitmap.c | 1 - block/qcow2-cache.c | 1 - block/qcow2-cluster.c | 1 - block/qcow2-refcount.c | 1 - block/qcow2-snapshot.c | 1 - block/qcow2.c | 1 - 7 files changed, 1 insertion(+), 6 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index fdee297f33..ecb026e19a 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -28,6 +28,7 @@ #include "crypto/block.h" #include "qemu/coroutine.h" #include "qemu/units.h" +#include "block/block_int.h" =20 //#define DEBUG_ALLOC //#define DEBUG_ALLOC2 diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index e53a1609d7..337c0ac680 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -29,7 +29,6 @@ #include "qapi/error.h" #include "qemu/cutils.h" =20 -#include "block/block_int.h" #include "qcow2.h" =20 /* NOTICE: BME here means Bitmaps Extension and used as a namespace for diff --git a/block/qcow2-cache.c b/block/qcow2-cache.c index df02e7b20a..b33bcbc984 100644 --- a/block/qcow2-cache.c +++ b/block/qcow2-cache.c @@ -23,7 +23,6 @@ */ =20 #include "qemu/osdep.h" -#include "block/block_int.h" #include "qemu-common.h" #include "qcow2.h" #include "trace.h" diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 974a4e8656..c4965479f6 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -27,7 +27,6 @@ =20 #include "qapi/error.h" #include "qemu-common.h" -#include "block/block_int.h" #include "qcow2.h" #include "qemu/bswap.h" #include "trace.h" diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index e0fe322500..ff27b6afd3 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -25,7 +25,6 @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "qemu-common.h" -#include "block/block_int.h" #include "qcow2.h" #include "qemu/range.h" #include "qemu/bswap.h" diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index a6ffae89a6..d0e7fa9311 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -24,7 +24,6 @@ =20 #include "qemu/osdep.h" #include "qapi/error.h" -#include "block/block_int.h" #include "qcow2.h" #include "qemu/bswap.h" #include "qemu/error-report.h" diff --git a/block/qcow2.c b/block/qcow2.c index d507ee0686..7c68a5b5f9 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -27,7 +27,6 @@ #define ZLIB_CONST #include =20 -#include "block/block_int.h" #include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" --=20 2.18.0 From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1554220056109968.461845097011; Tue, 2 Apr 2019 08:47:36 -0700 (PDT) Received: from localhost ([127.0.0.1]:39485 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLdK-0000WH-Qv for importer@patchew.org; Tue, 02 Apr 2019 11:47:30 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49608) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTn-0001eY-2w for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTk-0000Vt-RH for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:43510) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTk-0000Tf-91; Tue, 02 Apr 2019 11:37:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTf-0001s1-TC; Tue, 02 Apr 2019 18:37:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:23 +0300 Message-Id: <20190402153730.54145-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 03/10] qcow2: add separate file for threaded data processing functions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Move compression-on-threads to separate file. Encryption will be in it too. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 7 ++ block/qcow2-threads.c | 201 ++++++++++++++++++++++++++++++++++++++++++ block/qcow2.c | 169 ----------------------------------- block/Makefile.objs | 2 +- 4 files changed, 209 insertions(+), 170 deletions(-) create mode 100644 block/qcow2-threads.c diff --git a/block/qcow2.h b/block/qcow2.h index ecb026e19a..61079f7ee8 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -735,4 +735,11 @@ void qcow2_remove_persistent_dirty_bitmap(BlockDriverS= tate *bs, const char *name, Error **errp); =20 +ssize_t coroutine_fn +qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size); +ssize_t coroutine_fn +qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size); + #endif diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c new file mode 100644 index 0000000000..dea9ac431c --- /dev/null +++ b/block/qcow2-threads.c @@ -0,0 +1,201 @@ +/* + * Threaded data processing for Qcow2: compression, encryption + * + * Copyright (c) 2004-2006 Fabrice Bellard + * Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved. + * + * Permission is hereby granted, free of charge, to any person obtaining a= copy + * of this software and associated documentation files (the "Software"), t= o deal + * in the Software without restriction, including without limitation the r= ights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or se= ll + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included= in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS= OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OT= HER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING= FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS = IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" + +#define ZLIB_CONST +#include + +#include "qcow2.h" +#include "block/thread-pool.h" + +#define MAX_COMPRESS_THREADS 4 + +typedef ssize_t (*Qcow2CompressFunc)(void *dest, size_t dest_size, + const void *src, size_t src_size); +typedef struct Qcow2CompressData { + void *dest; + size_t dest_size; + const void *src; + size_t src_size; + ssize_t ret; + + Qcow2CompressFunc func; +} Qcow2CompressData; + +/* + * qcow2_compress() + * + * @dest - destination buffer, @dest_size bytes + * @src - source buffer, @src_size bytes + * + * Returns: compressed size on success + * -1 destination buffer is not enough to store compressed data + * -2 on any other error + */ +static ssize_t qcow2_compress(void *dest, size_t dest_size, + const void *src, size_t src_size) +{ + ssize_t ret; + z_stream strm; + + /* best compression, small window, no zlib header */ + memset(&strm, 0, sizeof(strm)); + ret =3D deflateInit2(&strm, Z_DEFAULT_COMPRESSION, Z_DEFLATED, + -12, 9, Z_DEFAULT_STRATEGY); + if (ret !=3D Z_OK) { + return -2; + } + + /* + * strm.next_in is not const in old zlib versions, such as those used = on + * OpenBSD/NetBSD, so cast the const away + */ + strm.avail_in =3D src_size; + strm.next_in =3D (void *) src; + strm.avail_out =3D dest_size; + strm.next_out =3D dest; + + ret =3D deflate(&strm, Z_FINISH); + if (ret =3D=3D Z_STREAM_END) { + ret =3D dest_size - strm.avail_out; + } else { + ret =3D (ret =3D=3D Z_OK ? -1 : -2); + } + + deflateEnd(&strm); + + return ret; +} + +/* + * qcow2_decompress() + * + * Decompress some data (not more than @src_size bytes) to produce exactly + * @dest_size bytes. + * + * @dest - destination buffer, @dest_size bytes + * @src - source buffer, @src_size bytes + * + * Returns: 0 on success + * -1 on fail + */ +static ssize_t qcow2_decompress(void *dest, size_t dest_size, + const void *src, size_t src_size) +{ + int ret =3D 0; + z_stream strm; + + memset(&strm, 0, sizeof(strm)); + strm.avail_in =3D src_size; + strm.next_in =3D (void *) src; + strm.avail_out =3D dest_size; + strm.next_out =3D dest; + + ret =3D inflateInit2(&strm, -12); + if (ret !=3D Z_OK) { + return -1; + } + + ret =3D inflate(&strm, Z_FINISH); + if ((ret !=3D Z_STREAM_END && ret !=3D Z_BUF_ERROR) || strm.avail_out = !=3D 0) { + /* + * We approve Z_BUF_ERROR because we need @dest buffer to be fille= d, but + * @src buffer may be processed partly (because in qcow2 we know s= ize of + * compressed data with precision of one sector) + */ + ret =3D -1; + } + + inflateEnd(&strm); + + return ret; +} + +static int qcow2_compress_pool_func(void *opaque) +{ + Qcow2CompressData *data =3D opaque; + + data->ret =3D data->func(data->dest, data->dest_size, + data->src, data->src_size); + + return 0; +} + +static void qcow2_compress_complete(void *opaque, int ret) +{ + qemu_coroutine_enter(opaque); +} + +static ssize_t coroutine_fn +qcow2_co_do_compress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size, Qcow2CompressFunc f= unc) +{ + BDRVQcow2State *s =3D bs->opaque; + BlockAIOCB *acb; + ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); + Qcow2CompressData arg =3D { + .dest =3D dest, + .dest_size =3D dest_size, + .src =3D src, + .src_size =3D src_size, + .func =3D func, + }; + + while (s->nb_compress_threads >=3D MAX_COMPRESS_THREADS) { + qemu_co_queue_wait(&s->compress_wait_queue, NULL); + } + + s->nb_compress_threads++; + acb =3D thread_pool_submit_aio(pool, qcow2_compress_pool_func, &arg, + qcow2_compress_complete, + qemu_coroutine_self()); + + if (!acb) { + s->nb_compress_threads--; + return -EINVAL; + } + qemu_coroutine_yield(); + s->nb_compress_threads--; + qemu_co_queue_next(&s->compress_wait_queue); + + return arg.ret; +} + +ssize_t coroutine_fn +qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size) +{ + return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, + qcow2_compress); +} + +ssize_t coroutine_fn +qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size) +{ + return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, + qcow2_decompress); +} diff --git a/block/qcow2.c b/block/qcow2.c index 7c68a5b5f9..0d1d39c3e6 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -24,9 +24,6 @@ =20 #include "qemu/osdep.h" =20 -#define ZLIB_CONST -#include - #include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" @@ -43,7 +40,6 @@ #include "qapi/qobject-input-visitor.h" #include "qapi/qapi-visit-block-core.h" #include "crypto.h" -#include "block/thread-pool.h" =20 /* Differences with QCOW: @@ -3914,171 +3910,6 @@ fail: return ret; } =20 -/* - * qcow2_compress() - * - * @dest - destination buffer, @dest_size bytes - * @src - source buffer, @src_size bytes - * - * Returns: compressed size on success - * -1 destination buffer is not enough to store compressed data - * -2 on any other error - */ -static ssize_t qcow2_compress(void *dest, size_t dest_size, - const void *src, size_t src_size) -{ - ssize_t ret; - z_stream strm; - - /* best compression, small window, no zlib header */ - memset(&strm, 0, sizeof(strm)); - ret =3D deflateInit2(&strm, Z_DEFAULT_COMPRESSION, Z_DEFLATED, - -12, 9, Z_DEFAULT_STRATEGY); - if (ret !=3D Z_OK) { - return -2; - } - - /* strm.next_in is not const in old zlib versions, such as those used = on - * OpenBSD/NetBSD, so cast the const away */ - strm.avail_in =3D src_size; - strm.next_in =3D (void *) src; - strm.avail_out =3D dest_size; - strm.next_out =3D dest; - - ret =3D deflate(&strm, Z_FINISH); - if (ret =3D=3D Z_STREAM_END) { - ret =3D dest_size - strm.avail_out; - } else { - ret =3D (ret =3D=3D Z_OK ? -1 : -2); - } - - deflateEnd(&strm); - - return ret; -} - -/* - * qcow2_decompress() - * - * Decompress some data (not more than @src_size bytes) to produce exactly - * @dest_size bytes. - * - * @dest - destination buffer, @dest_size bytes - * @src - source buffer, @src_size bytes - * - * Returns: 0 on success - * -1 on fail - */ -static ssize_t qcow2_decompress(void *dest, size_t dest_size, - const void *src, size_t src_size) -{ - int ret =3D 0; - z_stream strm; - - memset(&strm, 0, sizeof(strm)); - strm.avail_in =3D src_size; - strm.next_in =3D (void *) src; - strm.avail_out =3D dest_size; - strm.next_out =3D dest; - - ret =3D inflateInit2(&strm, -12); - if (ret !=3D Z_OK) { - return -1; - } - - ret =3D inflate(&strm, Z_FINISH); - if ((ret !=3D Z_STREAM_END && ret !=3D Z_BUF_ERROR) || strm.avail_out = !=3D 0) { - /* We approve Z_BUF_ERROR because we need @dest buffer to be fille= d, but - * @src buffer may be processed partly (because in qcow2 we know s= ize of - * compressed data with precision of one sector) */ - ret =3D -1; - } - - inflateEnd(&strm); - - return ret; -} - -#define MAX_COMPRESS_THREADS 4 - -typedef ssize_t (*Qcow2CompressFunc)(void *dest, size_t dest_size, - const void *src, size_t src_size); -typedef struct Qcow2CompressData { - void *dest; - size_t dest_size; - const void *src; - size_t src_size; - ssize_t ret; - - Qcow2CompressFunc func; -} Qcow2CompressData; - -static int qcow2_compress_pool_func(void *opaque) -{ - Qcow2CompressData *data =3D opaque; - - data->ret =3D data->func(data->dest, data->dest_size, - data->src, data->src_size); - - return 0; -} - -static void qcow2_compress_complete(void *opaque, int ret) -{ - qemu_coroutine_enter(opaque); -} - -static ssize_t coroutine_fn -qcow2_co_do_compress(BlockDriverState *bs, void *dest, size_t dest_size, - const void *src, size_t src_size, Qcow2CompressFunc f= unc) -{ - BDRVQcow2State *s =3D bs->opaque; - BlockAIOCB *acb; - ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); - Qcow2CompressData arg =3D { - .dest =3D dest, - .dest_size =3D dest_size, - .src =3D src, - .src_size =3D src_size, - .func =3D func, - }; - - while (s->nb_compress_threads >=3D MAX_COMPRESS_THREADS) { - qemu_co_queue_wait(&s->compress_wait_queue, NULL); - } - - s->nb_compress_threads++; - acb =3D thread_pool_submit_aio(pool, qcow2_compress_pool_func, &arg, - qcow2_compress_complete, - qemu_coroutine_self()); - - if (!acb) { - s->nb_compress_threads--; - return -EINVAL; - } - qemu_coroutine_yield(); - s->nb_compress_threads--; - qemu_co_queue_next(&s->compress_wait_queue); - - return arg.ret; -} - -static ssize_t coroutine_fn -qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size, - const void *src, size_t src_size) -{ - return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, - qcow2_compress); -} - -static ssize_t coroutine_fn -qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size, - const void *src, size_t src_size) -{ - return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, - qcow2_decompress); -} - /* XXX: put compressed sectors first, then all the cluster aligned tables to avoid losing bytes in alignment */ static coroutine_fn int diff --git a/block/Makefile.objs b/block/Makefile.objs index 7a81892a52..ae11605c9f 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -6,7 +6,7 @@ block-obj-$(CONFIG_BOCHS) +=3D bochs.o block-obj-$(CONFIG_VVFAT) +=3D vvfat.o block-obj-$(CONFIG_DMG) +=3D dmg.o =20 -block-obj-y +=3D qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o= qcow2-cache.o qcow2-bitmap.o +block-obj-y +=3D qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o= qcow2-cache.o qcow2-bitmap.o qcow2-threads.o block-obj-$(CONFIG_QED) +=3D qed.o qed-l2-cache.o qed-table.o qed-cluster.o block-obj-$(CONFIG_QED) +=3D qed-check.o block-obj-y +=3D vhdx.o vhdx-endian.o vhdx-log.o --=20 2.18.0 From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155421959054295.2157475835694; Tue, 2 Apr 2019 08:39:50 -0700 (PDT) Received: from localhost ([127.0.0.1]:37284 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLVj-0002rR-JF for importer@patchew.org; Tue, 02 Apr 2019 11:39:39 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49564) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTm-0001eU-2w for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTk-0000VS-NC for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:37 -0400 Received: from relay.sw.ru ([185.231.240.75]:43496) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTk-0000Tk-9t; Tue, 02 Apr 2019 11:37:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTg-0001s1-3J; Tue, 02 Apr 2019 18:37:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:24 +0300 Message-Id: <20190402153730.54145-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 04/10] qcow2-threads: use thread_pool_submit_co X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Use thread_pool_submit_co, instead of reinventing it here. Note, that thread_pool_submit_aio() never returns NULL, so checking it was an extra thing. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-threads.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c index dea9ac431c..20b2616529 100644 --- a/block/qcow2-threads.c +++ b/block/qcow2-threads.c @@ -144,17 +144,11 @@ static int qcow2_compress_pool_func(void *opaque) return 0; } =20 -static void qcow2_compress_complete(void *opaque, int ret) -{ - qemu_coroutine_enter(opaque); -} - static ssize_t coroutine_fn qcow2_co_do_compress(BlockDriverState *bs, void *dest, size_t dest_size, const void *src, size_t src_size, Qcow2CompressFunc f= unc) { BDRVQcow2State *s =3D bs->opaque; - BlockAIOCB *acb; ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); Qcow2CompressData arg =3D { .dest =3D dest, @@ -169,16 +163,9 @@ qcow2_co_do_compress(BlockDriverState *bs, void *dest,= size_t dest_size, } =20 s->nb_compress_threads++; - acb =3D thread_pool_submit_aio(pool, qcow2_compress_pool_func, &arg, - qcow2_compress_complete, - qemu_coroutine_self()); - - if (!acb) { - s->nb_compress_threads--; - return -EINVAL; - } - qemu_coroutine_yield(); + thread_pool_submit_co(pool, qcow2_compress_pool_func, &arg); s->nb_compress_threads--; + qemu_co_queue_next(&s->compress_wait_queue); =20 return arg.ret; --=20 2.18.0 From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1554219590436370.5997293504415; Tue, 2 Apr 2019 08:39:50 -0700 (PDT) Received: from localhost ([127.0.0.1]:37288 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLVl-0002rq-91 for importer@patchew.org; Tue, 02 Apr 2019 11:39:41 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49566) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTm-0001eV-3y for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTk-0000Vi-OG for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:38 -0400 Received: from relay.sw.ru ([185.231.240.75]:43492) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTk-0000Tl-Cx; Tue, 02 Apr 2019 11:37:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTg-0001s1-7g; Tue, 02 Apr 2019 18:37:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:25 +0300 Message-Id: <20190402153730.54145-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 05/10] qcow2-threads: qcow2_co_do_compress: protect queuing by mutex X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Drop dependence on AioContext lock. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Paolo Bonzini Reviewed-by: Max Reitz --- block/qcow2-threads.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c index 20b2616529..156e0667be 100644 --- a/block/qcow2-threads.c +++ b/block/qcow2-threads.c @@ -158,15 +158,19 @@ qcow2_co_do_compress(BlockDriverState *bs, void *dest= , size_t dest_size, .func =3D func, }; =20 + qemu_co_mutex_lock(&s->lock); while (s->nb_compress_threads >=3D MAX_COMPRESS_THREADS) { - qemu_co_queue_wait(&s->compress_wait_queue, NULL); + qemu_co_queue_wait(&s->compress_wait_queue, &s->lock); } - s->nb_compress_threads++; + qemu_co_mutex_unlock(&s->lock); + thread_pool_submit_co(pool, qcow2_compress_pool_func, &arg); - s->nb_compress_threads--; =20 + qemu_co_mutex_lock(&s->lock); + s->nb_compress_threads--; qemu_co_queue_next(&s->compress_wait_queue); + qemu_co_mutex_unlock(&s->lock); =20 return arg.ret; } --=20 2.18.0 From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1554219890832889.3581865124946; Tue, 2 Apr 2019 08:44:50 -0700 (PDT) Received: from localhost ([127.0.0.1]:38779 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLaf-00073P-E7 for importer@patchew.org; Tue, 02 Apr 2019 11:44:45 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49554) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTl-0001eQ-TO for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTk-0000VG-MD for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:37 -0400 Received: from relay.sw.ru ([185.231.240.75]:43502) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTk-0000Te-8y; Tue, 02 Apr 2019 11:37:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTg-0001s1-H0; Tue, 02 Apr 2019 18:37:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:26 +0300 Message-Id: <20190402153730.54145-7-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 06/10] qcow2-threads: split out generic path X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Move generic part out of qcow2_co_do_compress, to reuse it for encryption and rename things that would be shared with encryption path. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 4 ++-- block/qcow2-threads.c | 47 ++++++++++++++++++++++++++++--------------- block/qcow2.c | 2 +- 3 files changed, 34 insertions(+), 19 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 61079f7ee8..3089e9a68b 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -347,8 +347,8 @@ typedef struct BDRVQcow2State { char *image_backing_format; char *image_data_file; =20 - CoQueue compress_wait_queue; - int nb_compress_threads; + CoQueue thread_task_queue; + int nb_threads; =20 BdrvChild *data_file; } BDRVQcow2State; diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c index 156e0667be..e3066075da 100644 --- a/block/qcow2-threads.c +++ b/block/qcow2-threads.c @@ -31,7 +31,36 @@ #include "qcow2.h" #include "block/thread-pool.h" =20 -#define MAX_COMPRESS_THREADS 4 +#define QCOW2_MAX_THREADS 4 + +static int coroutine_fn +qcow2_co_process(BlockDriverState *bs, ThreadPoolFunc *func, void *arg) +{ + int ret; + BDRVQcow2State *s =3D bs->opaque; + ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); + + qemu_co_mutex_lock(&s->lock); + while (s->nb_threads >=3D QCOW2_MAX_THREADS) { + qemu_co_queue_wait(&s->thread_task_queue, &s->lock); + } + s->nb_threads++; + qemu_co_mutex_unlock(&s->lock); + + ret =3D thread_pool_submit_co(pool, func, arg); + + qemu_co_mutex_lock(&s->lock); + s->nb_threads--; + qemu_co_queue_next(&s->thread_task_queue); + qemu_co_mutex_unlock(&s->lock); + + return ret; +} + + +/* + * Compression + */ =20 typedef ssize_t (*Qcow2CompressFunc)(void *dest, size_t dest_size, const void *src, size_t src_size); @@ -148,8 +177,6 @@ static ssize_t coroutine_fn qcow2_co_do_compress(BlockDriverState *bs, void *dest, size_t dest_size, const void *src, size_t src_size, Qcow2CompressFunc f= unc) { - BDRVQcow2State *s =3D bs->opaque; - ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); Qcow2CompressData arg =3D { .dest =3D dest, .dest_size =3D dest_size, @@ -158,19 +185,7 @@ qcow2_co_do_compress(BlockDriverState *bs, void *dest,= size_t dest_size, .func =3D func, }; =20 - qemu_co_mutex_lock(&s->lock); - while (s->nb_compress_threads >=3D MAX_COMPRESS_THREADS) { - qemu_co_queue_wait(&s->compress_wait_queue, &s->lock); - } - s->nb_compress_threads++; - qemu_co_mutex_unlock(&s->lock); - - thread_pool_submit_co(pool, qcow2_compress_pool_func, &arg); - - qemu_co_mutex_lock(&s->lock); - s->nb_compress_threads--; - qemu_co_queue_next(&s->compress_wait_queue); - qemu_co_mutex_unlock(&s->lock); + qcow2_co_process(bs, qcow2_compress_pool_func, &arg); =20 return arg.ret; } diff --git a/block/qcow2.c b/block/qcow2.c index 0d1d39c3e6..46e8e39da5 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1694,7 +1694,7 @@ static int coroutine_fn qcow2_do_open(BlockDriverStat= e *bs, QDict *options, } #endif =20 - qemu_co_queue_init(&s->compress_wait_queue); + qemu_co_queue_init(&s->thread_task_queue); =20 return ret; =20 --=20 2.18.0 From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1554219920317895.7523278973904; Tue, 2 Apr 2019 08:45:20 -0700 (PDT) Received: from localhost ([127.0.0.1]:38943 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLbB-0007UZ-2q for importer@patchew.org; Tue, 02 Apr 2019 11:45:17 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49561) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTm-0001eT-0k for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTk-0000VF-MC for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:37 -0400 Received: from relay.sw.ru ([185.231.240.75]:43490) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTk-0000Th-Aj; Tue, 02 Apr 2019 11:37:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTg-0001s1-Ml; Tue, 02 Apr 2019 18:37:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:27 +0300 Message-Id: <20190402153730.54145-8-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 07/10] qcow2: qcow2_co_preadv: improve locking X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Background: decryption will be done in threads, to take benefit of it, we should move it out of the lock first. But let's go further: it turns out, that for locking around switch cases we have only two variants: when we just do memset(0) not releasing the lock (it is useless) and when we actually can handle the whole case out of the lock. So, refactor the whole thing to reduce locked code region and make it clean. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.c | 46 ++++++++++++++++++++++------------------------ 1 file changed, 22 insertions(+), 24 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 46e8e39da5..fcf92a7eb6 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1983,6 +1983,7 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverSt= ate *bs, uint64_t offset, =20 ret =3D qcow2_get_cluster_offset(bs, offset, &cur_bytes, &cluster_= offset); if (ret < 0) { + qemu_co_mutex_unlock(&s->lock); goto fail; } =20 @@ -1991,39 +1992,36 @@ static coroutine_fn int qcow2_co_preadv(BlockDriver= State *bs, uint64_t offset, qemu_iovec_reset(&hd_qiov); qemu_iovec_concat(&hd_qiov, qiov, bytes_done, cur_bytes); =20 + if (ret =3D=3D QCOW2_CLUSTER_ZERO_PLAIN || + ret =3D=3D QCOW2_CLUSTER_ZERO_ALLOC || + (ret =3D=3D QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) + { + qemu_iovec_memset(&hd_qiov, 0, 0, cur_bytes); + + bytes -=3D cur_bytes; + offset +=3D cur_bytes; + bytes_done +=3D cur_bytes; + continue; + } + + qemu_co_mutex_unlock(&s->lock); + switch (ret) { case QCOW2_CLUSTER_UNALLOCATED: - - if (bs->backing) { - BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO); - qemu_co_mutex_unlock(&s->lock); - ret =3D bdrv_co_preadv(bs->backing, offset, cur_bytes, - &hd_qiov, 0); - qemu_co_mutex_lock(&s->lock); - if (ret < 0) { - goto fail; - } - } else { - /* Note: in this case, no need to wait */ - qemu_iovec_memset(&hd_qiov, 0, 0, cur_bytes); + BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO); + ret =3D bdrv_co_preadv(bs->backing, offset, cur_bytes, &hd_qio= v, 0); + if (ret < 0) { + goto fail; } break; =20 - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: - qemu_iovec_memset(&hd_qiov, 0, 0, cur_bytes); - break; - case QCOW2_CLUSTER_COMPRESSED: - qemu_co_mutex_unlock(&s->lock); ret =3D qcow2_co_preadv_compressed(bs, cluster_offset, offset, cur_bytes, &hd_qiov); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { goto fail; } - break; =20 case QCOW2_CLUSTER_NORMAL: @@ -2056,11 +2054,9 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverS= tate *bs, uint64_t offset, } =20 BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); - qemu_co_mutex_unlock(&s->lock); ret =3D bdrv_co_preadv(s->data_file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { goto fail; } @@ -2091,12 +2087,14 @@ static coroutine_fn int qcow2_co_preadv(BlockDriver= State *bs, uint64_t offset, bytes -=3D cur_bytes; offset +=3D cur_bytes; bytes_done +=3D cur_bytes; + + qemu_co_mutex_lock(&s->lock); } ret =3D 0; =20 -fail: qemu_co_mutex_unlock(&s->lock); =20 +fail: qemu_iovec_destroy(&hd_qiov); qemu_vfree(cluster_data); =20 --=20 2.18.0 From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1554219751606228.34353074965497; Tue, 2 Apr 2019 08:42:31 -0700 (PDT) Received: from localhost ([127.0.0.1]:38126 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLYR-0005F9-Kt for importer@patchew.org; Tue, 02 Apr 2019 11:42:27 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49567) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTm-0001eW-3r for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTk-0000Vo-Qh for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:38 -0400 Received: from relay.sw.ru ([185.231.240.75]:43482) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTk-0000Tg-DA; Tue, 02 Apr 2019 11:37:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTg-0001s1-Sw; Tue, 02 Apr 2019 18:37:32 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:28 +0300 Message-Id: <20190402153730.54145-9-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 08/10] qcow2: qcow2_co_preadv: skip using hd_qiov when possible X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" qemu_iovec_memset has @offset parameter, so using hd_qiov for it is not needed. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index fcf92a7eb6..b44b455ad1 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1987,16 +1987,11 @@ static coroutine_fn int qcow2_co_preadv(BlockDriver= State *bs, uint64_t offset, goto fail; } =20 - offset_in_cluster =3D offset_into_cluster(s, offset); - - qemu_iovec_reset(&hd_qiov); - qemu_iovec_concat(&hd_qiov, qiov, bytes_done, cur_bytes); - if (ret =3D=3D QCOW2_CLUSTER_ZERO_PLAIN || ret =3D=3D QCOW2_CLUSTER_ZERO_ALLOC || (ret =3D=3D QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) { - qemu_iovec_memset(&hd_qiov, 0, 0, cur_bytes); + qemu_iovec_memset(qiov, bytes_done, 0, cur_bytes); =20 bytes -=3D cur_bytes; offset +=3D cur_bytes; @@ -2004,6 +1999,11 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverS= tate *bs, uint64_t offset, continue; } =20 + offset_in_cluster =3D offset_into_cluster(s, offset); + + qemu_iovec_reset(&hd_qiov); + qemu_iovec_concat(&hd_qiov, qiov, bytes_done, cur_bytes); + qemu_co_mutex_unlock(&s->lock); =20 switch (ret) { --=20 2.18.0 From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155421959048716.015852516781138; Tue, 2 Apr 2019 08:39:50 -0700 (PDT) Received: from localhost ([127.0.0.1]:37298 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLVl-0002ty-77 for importer@patchew.org; Tue, 02 Apr 2019 11:39:41 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49559) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTl-0001eS-V5 for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTk-0000VV-NG for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:37 -0400 Received: from relay.sw.ru ([185.231.240.75]:43506) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTk-0000Tm-99; Tue, 02 Apr 2019 11:37:36 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTh-0001s1-1j; Tue, 02 Apr 2019 18:37:33 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:29 +0300 Message-Id: <20190402153730.54145-10-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 09/10] qcow2: bdrv_co_pwritev: move encryption code out of the lock X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Encryption will be done in threads, to take benefit of it, we should move it out of the lock first. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.c | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index b44b455ad1..9bd5e8e48c 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2175,11 +2175,20 @@ static coroutine_fn int qcow2_co_pwritev(BlockDrive= rState *bs, uint64_t offset, ret =3D qcow2_alloc_cluster_offset(bs, offset, &cur_bytes, &cluster_offset, &l2meta); if (ret < 0) { - goto fail; + goto out_locked; } =20 assert((cluster_offset & 511) =3D=3D 0); =20 + ret =3D qcow2_pre_write_overlap_check(bs, 0, + cluster_offset + offset_in_clu= ster, + cur_bytes, true); + if (ret < 0) { + goto out_locked; + } + + qemu_co_mutex_unlock(&s->lock); + qemu_iovec_reset(&hd_qiov); qemu_iovec_concat(&hd_qiov, qiov, bytes_done, cur_bytes); =20 @@ -2191,7 +2200,7 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverS= tate *bs, uint64_t offset, * s->cluster_size); if (cluster_data =3D=3D NULL) { ret =3D -ENOMEM; - goto fail; + goto out_unlocked; } } =20 @@ -2206,40 +2215,34 @@ static coroutine_fn int qcow2_co_pwritev(BlockDrive= rState *bs, uint64_t offset, cluster_data, cur_bytes, NULL) < 0) { ret =3D -EIO; - goto fail; + goto out_unlocked; } =20 qemu_iovec_reset(&hd_qiov); qemu_iovec_add(&hd_qiov, cluster_data, cur_bytes); } =20 - ret =3D qcow2_pre_write_overlap_check(bs, 0, - cluster_offset + offset_in_cluster, cur_bytes, true); - if (ret < 0) { - goto fail; - } - /* If we need to do COW, check if it's possible to merge the * writing of the guest data together with that of the COW regions. * If it's not possible (or not necessary) then write the * guest data now. */ if (!merge_cow(offset, cur_bytes, &hd_qiov, l2meta)) { - qemu_co_mutex_unlock(&s->lock); BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); trace_qcow2_writev_data(qemu_coroutine_self(), cluster_offset + offset_in_cluster); ret =3D bdrv_co_pwritev(s->data_file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { - goto fail; + goto out_unlocked; } } =20 + qemu_co_mutex_lock(&s->lock); + ret =3D qcow2_handle_l2meta(bs, &l2meta, true); if (ret) { - goto fail; + goto out_locked; } =20 bytes -=3D cur_bytes; @@ -2248,8 +2251,12 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriver= State *bs, uint64_t offset, trace_qcow2_writev_done_part(qemu_coroutine_self(), cur_bytes); } ret =3D 0; + goto out_locked; =20 -fail: +out_unlocked: + qemu_co_mutex_lock(&s->lock); + +out_locked: qcow2_handle_l2meta(bs, &l2meta, false); =20 qemu_co_mutex_unlock(&s->lock); --=20 2.18.0 From nobody Sat May 4 10:52:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1554220122172989.1804705893899; Tue, 2 Apr 2019 08:48:42 -0700 (PDT) Received: from localhost ([127.0.0.1]:39733 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLeL-0001D3-6g for importer@patchew.org; Tue, 02 Apr 2019 11:48:33 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49732) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hBLTs-0001jT-Fx for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hBLTq-0000ZI-LN for qemu-devel@nongnu.org; Tue, 02 Apr 2019 11:37:44 -0400 Received: from relay.sw.ru ([185.231.240.75]:43536) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hBLTo-0000XE-7n; Tue, 02 Apr 2019 11:37:40 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hBLTh-0001s1-9M; Tue, 02 Apr 2019 18:37:33 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 2 Apr 2019 18:37:30 +0300 Message-Id: <20190402153730.54145-11-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190402153730.54145-1-vsementsov@virtuozzo.com> References: <20190402153730.54145-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 10/10] qcow2: do encryption in threads X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Do encryption/decryption in threads, like it is already done for compression. This improves asynchronous encrypted io. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 8 ++++++ block/qcow2-cluster.c | 7 ++--- block/qcow2-threads.c | 65 +++++++++++++++++++++++++++++++++++++++++-- block/qcow2.c | 22 +++++---------- 4 files changed, 81 insertions(+), 21 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 3089e9a68b..5169c0d8ab 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -264,6 +264,8 @@ typedef struct Qcow2BitmapHeaderExt { uint64_t bitmap_directory_offset; } QEMU_PACKED Qcow2BitmapHeaderExt; =20 +#define QCOW2_MAX_THREADS 4 + typedef struct BDRVQcow2State { int cluster_bits; int cluster_size; @@ -741,5 +743,11 @@ qcow2_co_compress(BlockDriverState *bs, void *dest, si= ze_t dest_size, ssize_t coroutine_fn qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size, const void *src, size_t src_size); +int coroutine_fn +qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len); +int coroutine_fn +qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len); =20 #endif diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index c4965479f6..e6624c6e4e 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -471,13 +471,12 @@ static bool coroutine_fn do_perform_cow_encrypt(Block= DriverState *bs, { if (bytes && bs->encrypted) { BDRVQcow2State *s =3D bs->opaque; - int64_t offset =3D (s->crypt_physical_offset ? - (cluster_offset + offset_in_cluster) : - (src_cluster_offset + offset_in_cluster)); assert((offset_in_cluster & ~BDRV_SECTOR_MASK) =3D=3D 0); assert((bytes & ~BDRV_SECTOR_MASK) =3D=3D 0); assert(s->crypto); - if (qcrypto_block_encrypt(s->crypto, offset, buffer, bytes, NULL) = < 0) { + if (qcow2_co_encrypt(bs, cluster_offset, + src_cluster_offset + offset_in_cluster, + buffer, bytes) < 0) { return false; } } diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c index e3066075da..0e20723a4b 100644 --- a/block/qcow2-threads.c +++ b/block/qcow2-threads.c @@ -30,8 +30,7 @@ =20 #include "qcow2.h" #include "block/thread-pool.h" - -#define QCOW2_MAX_THREADS 4 +#include "crypto.h" =20 static int coroutine_fn qcow2_co_process(BlockDriverState *bs, ThreadPoolFunc *func, void *arg) @@ -205,3 +204,65 @@ qcow2_co_decompress(BlockDriverState *bs, void *dest, = size_t dest_size, return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, qcow2_decompress); } + + +/* + * Cryptography + */ + +/* + * Qcow2EncDecFunc: common prototype of qcrypto_block_encrypt() and + * qcrypto_block_decrypt() functions. + */ +typedef int (*Qcow2EncDecFunc)(QCryptoBlock *block, uint64_t offset, + uint8_t *buf, size_t len, Error **errp); + +typedef struct Qcow2EncDecData { + QCryptoBlock *block; + uint64_t offset; + uint8_t *buf; + size_t len; + + Qcow2EncDecFunc func; +} Qcow2EncDecData; + +static int qcow2_encdec_pool_func(void *opaque) +{ + Qcow2EncDecData *data =3D opaque; + + return data->func(data->block, data->offset, data->buf, data->len, NUL= L); +} + +static int coroutine_fn +qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc = func) +{ + BDRVQcow2State *s =3D bs->opaque; + Qcow2EncDecData arg =3D { + .block =3D s->crypto, + .offset =3D s->crypt_physical_offset ? + file_cluster_offset + offset_into_cluster(s, offset)= : + offset, + .buf =3D buf, + .len =3D len, + .func =3D func, + }; + + return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg); +} + +int coroutine_fn +qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len) +{ + return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len, + qcrypto_block_encrypt); +} + +int coroutine_fn +qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len) +{ + return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len, + qcrypto_block_decrypt); +} diff --git a/block/qcow2.c b/block/qcow2.c index 9bd5e8e48c..ba9725bcd5 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -297,7 +297,7 @@ static int qcow2_read_extensions(BlockDriverState *bs, = uint64_t start_offset, } s->crypto =3D qcrypto_block_open(s->crypto_opts, "encrypt.", qcow2_crypto_hdr_read_func, - bs, cflags, 1, errp); + bs, cflags, QCOW2_MAX_THREADS, = errp); if (!s->crypto) { return -EINVAL; } @@ -1539,7 +1539,8 @@ static int coroutine_fn qcow2_do_open(BlockDriverStat= e *bs, QDict *options, cflags |=3D QCRYPTO_BLOCK_OPEN_NO_IO; } s->crypto =3D qcrypto_block_open(s->crypto_opts, "encrypt.", - NULL, NULL, cflags, 1, errp); + NULL, NULL, cflags, + QCOW2_MAX_THREADS, errp); if (!s->crypto) { ret =3D -EINVAL; goto fail; @@ -2064,13 +2065,8 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverS= tate *bs, uint64_t offset, assert(s->crypto); assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - if (qcrypto_block_decrypt(s->crypto, - (s->crypt_physical_offset ? - cluster_offset + offset_in_clus= ter : - offset), - cluster_data, - cur_bytes, - NULL) < 0) { + if (qcow2_co_decrypt(bs, cluster_offset, offset, + cluster_data, cur_bytes) < 0) { ret =3D -EIO; goto fail; } @@ -2208,12 +2204,8 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriver= State *bs, uint64_t offset, QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); qemu_iovec_to_buf(&hd_qiov, 0, cluster_data, hd_qiov.size); =20 - if (qcrypto_block_encrypt(s->crypto, - (s->crypt_physical_offset ? - cluster_offset + offset_in_cluster : - offset), - cluster_data, - cur_bytes, NULL) < 0) { + if (qcow2_co_encrypt(bs, cluster_offset, offset, + cluster_data, cur_bytes) < 0) { ret =3D -EIO; goto out_unlocked; } --=20 2.18.0