From nobody Mon Apr 29 02:04:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1557152997; cv=none; d=zoho.com; s=zohoarc; b=gp788gxZCK8TE7jd4G3EaaC6LzviBXpcc7jvfQqLn2MfEGBSDh1ZZc2VvH1h33XkM3S/OflbXWGz2lN7ehTfqBbAwI/ghCnpD300AXrb9j2yl4kGb3MlWEseYmlRtm9JfSjd0kgr+aQqfEv7+v0sbm4g2iF5KIU4X6JudWDc+zM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557152997; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=UBCLztCL7hE21oxGzZ8oS21k1eeUCfdFqUPi8dPRH2o=; b=UN4R4TKRw0ezNswMtG41jZ24IifTZNKeyfLnc8kgUt6qYdfkynjN+lTs9KBS8waIm5x3MFtzgxBUrcE6CehpJAIItAuC42uk+XwogWdQRTfUY49Th+2y1pHAZaoOSPiSoQeII9oE40QhlrEApzmMiJrIFGWEaHSNU82mLp5X2a8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155715299762782.64282389583502; Mon, 6 May 2019 07:29:57 -0700 (PDT) Received: from localhost ([127.0.0.1]:57354 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNecu-0007uM-Jp for importer@patchew.org; Mon, 06 May 2019 10:29:56 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50262) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNeaq-0006PB-E5 for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hNeao-0000d5-UD for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:48 -0400 Received: from relay.sw.ru ([185.231.240.75]:45584) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hNeao-0000bf-HT; Mon, 06 May 2019 10:27:46 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hNeaj-000444-RG; Mon, 06 May 2019 17:27:41 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 6 May 2019 17:27:34 +0300 Message-Id: <20190506142741.41731-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190506142741.41731-1-vsementsov@virtuozzo.com> References: <20190506142741.41731-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 1/8] qcow2.h: add missing include X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" qcow2.h depends on block_int.h. Compilation isn't broken currently only due to block_int.h always included before qcow2.h. Though, it seems better to directly include block_int.h in qcow2.h. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 1 + block/qcow2-bitmap.c | 1 - block/qcow2-cache.c | 1 - block/qcow2-cluster.c | 1 - block/qcow2-refcount.c | 1 - block/qcow2-snapshot.c | 1 - block/qcow2.c | 1 - 7 files changed, 1 insertion(+), 6 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index fdee297f33..ecb026e19a 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -28,6 +28,7 @@ #include "crypto/block.h" #include "qemu/coroutine.h" #include "qemu/units.h" +#include "block/block_int.h" =20 //#define DEBUG_ALLOC //#define DEBUG_ALLOC2 diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index e53a1609d7..337c0ac680 100644 --- a/block/qcow2-bitmap.c +++ b/block/qcow2-bitmap.c @@ -29,7 +29,6 @@ #include "qapi/error.h" #include "qemu/cutils.h" =20 -#include "block/block_int.h" #include "qcow2.h" =20 /* NOTICE: BME here means Bitmaps Extension and used as a namespace for diff --git a/block/qcow2-cache.c b/block/qcow2-cache.c index df02e7b20a..b33bcbc984 100644 --- a/block/qcow2-cache.c +++ b/block/qcow2-cache.c @@ -23,7 +23,6 @@ */ =20 #include "qemu/osdep.h" -#include "block/block_int.h" #include "qemu-common.h" #include "qcow2.h" #include "trace.h" diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 974a4e8656..c4965479f6 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -27,7 +27,6 @@ =20 #include "qapi/error.h" #include "qemu-common.h" -#include "block/block_int.h" #include "qcow2.h" #include "qemu/bswap.h" #include "trace.h" diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index e0fe322500..ff27b6afd3 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -25,7 +25,6 @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "qemu-common.h" -#include "block/block_int.h" #include "qcow2.h" #include "qemu/range.h" #include "qemu/bswap.h" diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index a6ffae89a6..d0e7fa9311 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -24,7 +24,6 @@ =20 #include "qemu/osdep.h" #include "qapi/error.h" -#include "block/block_int.h" #include "qcow2.h" #include "qemu/bswap.h" #include "qemu/error-report.h" diff --git a/block/qcow2.c b/block/qcow2.c index a520d116ef..ab231cbfe7 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -27,7 +27,6 @@ #define ZLIB_CONST #include =20 -#include "block/block_int.h" #include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" --=20 2.18.0 From nobody Mon Apr 29 02:04:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1557153404; cv=none; d=zoho.com; s=zohoarc; b=gs4tZskh0aeXJP57qaJp9QIFgEyHOaSPaO9PVT2WvtGgJG8yw12bpHA+O1q5eQltb2wKsmi7OraDvmxSyf8At8ofoLW/HUD59+KY+7YBEFbMSe5WLtNbtbsZkjaEP8mim7KXq4T3OvSePfZhVyBUANpOs8ITjO7TaXNLAagfNeI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557153404; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=/ZdlldhpMTO+6tMB6KifIT0cy2eb8amI6eVMN+EoIt0=; b=IzEqmgXzMbIWdlWUSNmdYwCRJGbADjxtwH6JZSON1IAdkmmNePxoYVuQU3d/d9kd+1pK8+QcUO8M8FrFu5OXTFcQhjjPxJ77SqQQvQWePpdAqSW5Y8A5TucQ992+60PFbSFsxITg5cCBd+o5BC/xtN8B3VK7v7j/PtHTlgaw+DM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155715340412026.413999739669975; Mon, 6 May 2019 07:36:44 -0700 (PDT) Received: from localhost ([127.0.0.1]:57480 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNejK-00058P-1z for importer@patchew.org; Mon, 06 May 2019 10:36:34 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50286) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNear-0006PY-1r for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hNeao-0000d4-Tz for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:49 -0400 Received: from relay.sw.ru ([185.231.240.75]:45588) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hNeao-0000bb-El; Mon, 06 May 2019 10:27:46 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hNeak-000444-8E; Mon, 06 May 2019 17:27:42 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 6 May 2019 17:27:35 +0300 Message-Id: <20190506142741.41731-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190506142741.41731-1-vsementsov@virtuozzo.com> References: <20190506142741.41731-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 2/8] qcow2: add separate file for threaded data processing functions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Move compression-on-threads to separate file. Encryption will be in it too. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 7 ++ block/qcow2-threads.c | 201 ++++++++++++++++++++++++++++++++++++++++++ block/qcow2.c | 169 ----------------------------------- block/Makefile.objs | 2 +- 4 files changed, 209 insertions(+), 170 deletions(-) create mode 100644 block/qcow2-threads.c diff --git a/block/qcow2.h b/block/qcow2.h index ecb026e19a..61079f7ee8 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -735,4 +735,11 @@ void qcow2_remove_persistent_dirty_bitmap(BlockDriverS= tate *bs, const char *name, Error **errp); =20 +ssize_t coroutine_fn +qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size); +ssize_t coroutine_fn +qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size); + #endif diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c new file mode 100644 index 0000000000..85b8034fd2 --- /dev/null +++ b/block/qcow2-threads.c @@ -0,0 +1,201 @@ +/* + * Threaded data processing for Qcow2: compression, encryption + * + * Copyright (c) 2004-2006 Fabrice Bellard + * Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved. + * + * Permission is hereby granted, free of charge, to any person obtaining a= copy + * of this software and associated documentation files (the "Software"), t= o deal + * in the Software without restriction, including without limitation the r= ights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or se= ll + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included= in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS= OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OT= HER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING= FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS = IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" + +#define ZLIB_CONST +#include + +#include "qcow2.h" +#include "block/thread-pool.h" + +#define MAX_COMPRESS_THREADS 4 + +typedef ssize_t (*Qcow2CompressFunc)(void *dest, size_t dest_size, + const void *src, size_t src_size); +typedef struct Qcow2CompressData { + void *dest; + size_t dest_size; + const void *src; + size_t src_size; + ssize_t ret; + + Qcow2CompressFunc func; +} Qcow2CompressData; + +/* + * qcow2_compress() + * + * @dest - destination buffer, @dest_size bytes + * @src - source buffer, @src_size bytes + * + * Returns: compressed size on success + * -ENOMEM destination buffer is not enough to store compressed d= ata + * -EIO on any other error + */ +static ssize_t qcow2_compress(void *dest, size_t dest_size, + const void *src, size_t src_size) +{ + ssize_t ret; + z_stream strm; + + /* best compression, small window, no zlib header */ + memset(&strm, 0, sizeof(strm)); + ret =3D deflateInit2(&strm, Z_DEFAULT_COMPRESSION, Z_DEFLATED, + -12, 9, Z_DEFAULT_STRATEGY); + if (ret !=3D Z_OK) { + return -EIO; + } + + /* + * strm.next_in is not const in old zlib versions, such as those used = on + * OpenBSD/NetBSD, so cast the const away + */ + strm.avail_in =3D src_size; + strm.next_in =3D (void *) src; + strm.avail_out =3D dest_size; + strm.next_out =3D dest; + + ret =3D deflate(&strm, Z_FINISH); + if (ret =3D=3D Z_STREAM_END) { + ret =3D dest_size - strm.avail_out; + } else { + ret =3D (ret =3D=3D Z_OK ? -ENOMEM : -EIO); + } + + deflateEnd(&strm); + + return ret; +} + +/* + * qcow2_decompress() + * + * Decompress some data (not more than @src_size bytes) to produce exactly + * @dest_size bytes. + * + * @dest - destination buffer, @dest_size bytes + * @src - source buffer, @src_size bytes + * + * Returns: 0 on success + * -1 on fail + */ +static ssize_t qcow2_decompress(void *dest, size_t dest_size, + const void *src, size_t src_size) +{ + int ret =3D 0; + z_stream strm; + + memset(&strm, 0, sizeof(strm)); + strm.avail_in =3D src_size; + strm.next_in =3D (void *) src; + strm.avail_out =3D dest_size; + strm.next_out =3D dest; + + ret =3D inflateInit2(&strm, -12); + if (ret !=3D Z_OK) { + return -1; + } + + ret =3D inflate(&strm, Z_FINISH); + if ((ret !=3D Z_STREAM_END && ret !=3D Z_BUF_ERROR) || strm.avail_out = !=3D 0) { + /* + * We approve Z_BUF_ERROR because we need @dest buffer to be fille= d, but + * @src buffer may be processed partly (because in qcow2 we know s= ize of + * compressed data with precision of one sector) + */ + ret =3D -1; + } + + inflateEnd(&strm); + + return ret; +} + +static int qcow2_compress_pool_func(void *opaque) +{ + Qcow2CompressData *data =3D opaque; + + data->ret =3D data->func(data->dest, data->dest_size, + data->src, data->src_size); + + return 0; +} + +static void qcow2_compress_complete(void *opaque, int ret) +{ + qemu_coroutine_enter(opaque); +} + +static ssize_t coroutine_fn +qcow2_co_do_compress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size, Qcow2CompressFunc f= unc) +{ + BDRVQcow2State *s =3D bs->opaque; + BlockAIOCB *acb; + ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); + Qcow2CompressData arg =3D { + .dest =3D dest, + .dest_size =3D dest_size, + .src =3D src, + .src_size =3D src_size, + .func =3D func, + }; + + while (s->nb_compress_threads >=3D MAX_COMPRESS_THREADS) { + qemu_co_queue_wait(&s->compress_wait_queue, NULL); + } + + s->nb_compress_threads++; + acb =3D thread_pool_submit_aio(pool, qcow2_compress_pool_func, &arg, + qcow2_compress_complete, + qemu_coroutine_self()); + + if (!acb) { + s->nb_compress_threads--; + return -EINVAL; + } + qemu_coroutine_yield(); + s->nb_compress_threads--; + qemu_co_queue_next(&s->compress_wait_queue); + + return arg.ret; +} + +ssize_t coroutine_fn +qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size) +{ + return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, + qcow2_compress); +} + +ssize_t coroutine_fn +qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size, + const void *src, size_t src_size) +{ + return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, + qcow2_decompress); +} diff --git a/block/qcow2.c b/block/qcow2.c index ab231cbfe7..facf8ab993 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -24,9 +24,6 @@ =20 #include "qemu/osdep.h" =20 -#define ZLIB_CONST -#include - #include "block/qdict.h" #include "sysemu/block-backend.h" #include "qemu/module.h" @@ -43,7 +40,6 @@ #include "qapi/qobject-input-visitor.h" #include "qapi/qapi-visit-block-core.h" #include "crypto.h" -#include "block/thread-pool.h" =20 /* Differences with QCOW: @@ -3921,171 +3917,6 @@ fail: return ret; } =20 -/* - * qcow2_compress() - * - * @dest - destination buffer, @dest_size bytes - * @src - source buffer, @src_size bytes - * - * Returns: compressed size on success - * -ENOMEM destination buffer is not enough to store compressed d= ata - * -EIO on any other error - */ -static ssize_t qcow2_compress(void *dest, size_t dest_size, - const void *src, size_t src_size) -{ - ssize_t ret; - z_stream strm; - - /* best compression, small window, no zlib header */ - memset(&strm, 0, sizeof(strm)); - ret =3D deflateInit2(&strm, Z_DEFAULT_COMPRESSION, Z_DEFLATED, - -12, 9, Z_DEFAULT_STRATEGY); - if (ret !=3D Z_OK) { - return -EIO; - } - - /* strm.next_in is not const in old zlib versions, such as those used = on - * OpenBSD/NetBSD, so cast the const away */ - strm.avail_in =3D src_size; - strm.next_in =3D (void *) src; - strm.avail_out =3D dest_size; - strm.next_out =3D dest; - - ret =3D deflate(&strm, Z_FINISH); - if (ret =3D=3D Z_STREAM_END) { - ret =3D dest_size - strm.avail_out; - } else { - ret =3D (ret =3D=3D Z_OK ? -ENOMEM : -EIO); - } - - deflateEnd(&strm); - - return ret; -} - -/* - * qcow2_decompress() - * - * Decompress some data (not more than @src_size bytes) to produce exactly - * @dest_size bytes. - * - * @dest - destination buffer, @dest_size bytes - * @src - source buffer, @src_size bytes - * - * Returns: 0 on success - * -1 on fail - */ -static ssize_t qcow2_decompress(void *dest, size_t dest_size, - const void *src, size_t src_size) -{ - int ret =3D 0; - z_stream strm; - - memset(&strm, 0, sizeof(strm)); - strm.avail_in =3D src_size; - strm.next_in =3D (void *) src; - strm.avail_out =3D dest_size; - strm.next_out =3D dest; - - ret =3D inflateInit2(&strm, -12); - if (ret !=3D Z_OK) { - return -1; - } - - ret =3D inflate(&strm, Z_FINISH); - if ((ret !=3D Z_STREAM_END && ret !=3D Z_BUF_ERROR) || strm.avail_out = !=3D 0) { - /* We approve Z_BUF_ERROR because we need @dest buffer to be fille= d, but - * @src buffer may be processed partly (because in qcow2 we know s= ize of - * compressed data with precision of one sector) */ - ret =3D -1; - } - - inflateEnd(&strm); - - return ret; -} - -#define MAX_COMPRESS_THREADS 4 - -typedef ssize_t (*Qcow2CompressFunc)(void *dest, size_t dest_size, - const void *src, size_t src_size); -typedef struct Qcow2CompressData { - void *dest; - size_t dest_size; - const void *src; - size_t src_size; - ssize_t ret; - - Qcow2CompressFunc func; -} Qcow2CompressData; - -static int qcow2_compress_pool_func(void *opaque) -{ - Qcow2CompressData *data =3D opaque; - - data->ret =3D data->func(data->dest, data->dest_size, - data->src, data->src_size); - - return 0; -} - -static void qcow2_compress_complete(void *opaque, int ret) -{ - qemu_coroutine_enter(opaque); -} - -static ssize_t coroutine_fn -qcow2_co_do_compress(BlockDriverState *bs, void *dest, size_t dest_size, - const void *src, size_t src_size, Qcow2CompressFunc f= unc) -{ - BDRVQcow2State *s =3D bs->opaque; - BlockAIOCB *acb; - ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); - Qcow2CompressData arg =3D { - .dest =3D dest, - .dest_size =3D dest_size, - .src =3D src, - .src_size =3D src_size, - .func =3D func, - }; - - while (s->nb_compress_threads >=3D MAX_COMPRESS_THREADS) { - qemu_co_queue_wait(&s->compress_wait_queue, NULL); - } - - s->nb_compress_threads++; - acb =3D thread_pool_submit_aio(pool, qcow2_compress_pool_func, &arg, - qcow2_compress_complete, - qemu_coroutine_self()); - - if (!acb) { - s->nb_compress_threads--; - return -EINVAL; - } - qemu_coroutine_yield(); - s->nb_compress_threads--; - qemu_co_queue_next(&s->compress_wait_queue); - - return arg.ret; -} - -static ssize_t coroutine_fn -qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size, - const void *src, size_t src_size) -{ - return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, - qcow2_compress); -} - -static ssize_t coroutine_fn -qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size, - const void *src, size_t src_size) -{ - return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, - qcow2_decompress); -} - /* XXX: put compressed sectors first, then all the cluster aligned tables to avoid losing bytes in alignment */ static coroutine_fn int diff --git a/block/Makefile.objs b/block/Makefile.objs index 7a81892a52..ae11605c9f 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -6,7 +6,7 @@ block-obj-$(CONFIG_BOCHS) +=3D bochs.o block-obj-$(CONFIG_VVFAT) +=3D vvfat.o block-obj-$(CONFIG_DMG) +=3D dmg.o =20 -block-obj-y +=3D qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o= qcow2-cache.o qcow2-bitmap.o +block-obj-y +=3D qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o= qcow2-cache.o qcow2-bitmap.o qcow2-threads.o block-obj-$(CONFIG_QED) +=3D qed.o qed-l2-cache.o qed-table.o qed-cluster.o block-obj-$(CONFIG_QED) +=3D qed-check.o block-obj-y +=3D vhdx.o vhdx-endian.o vhdx-log.o --=20 2.18.0 From nobody Mon Apr 29 02:04:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1557152987; cv=none; d=zoho.com; s=zohoarc; b=XnFM6mOuL78dHwsORr2bdKIci6EIt/TDP85RMUmr1ehKnI0Zle4K3Jnp8ZeJn/SZso19xdRHBupjzpHCcxg5iQL/hXyFMcGkRDppSnb36MFimX7l/bAMuBOE9BaxT5+SGiaoNi1dxSBGHNhJaKZiuVAyoVQGCB1BYVfAeI1NyKs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557152987; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=UXrT4Mk9q7Yd1gt5eSkeaHwmRGwCArGRs+oAylaM69c=; b=YSYhI35giMigiegVMstt5EQUFMmw6ZGORNClzFthaIx3FBENJkbyqk0n5qW1N4LVr6b03/7N3oot13ARU12Vuv1ORkkGOSTix+D6926kby9dhG40R4x4h84Z798Hn/p5SvDyerX5iYVHFonC+4nhvbsNcy3yoScAUv7NZy0yX60= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557152986870513.4509983651782; Mon, 6 May 2019 07:29:46 -0700 (PDT) Received: from localhost ([127.0.0.1]:57352 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNece-0007ea-Fd for importer@patchew.org; Mon, 06 May 2019 10:29:40 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50239) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNeap-0006On-RF for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hNeao-0000cY-PB for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:47 -0400 Received: from relay.sw.ru ([185.231.240.75]:45582) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hNeao-0000bZ-G2; Mon, 06 May 2019 10:27:46 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hNeak-000444-Ji; Mon, 06 May 2019 17:27:42 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 6 May 2019 17:27:36 +0300 Message-Id: <20190506142741.41731-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190506142741.41731-1-vsementsov@virtuozzo.com> References: <20190506142741.41731-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 3/8] qcow2-threads: use thread_pool_submit_co X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Use thread_pool_submit_co, instead of reinventing it here. Note, that thread_pool_submit_aio() never returns NULL, so checking it was an extra thing. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-threads.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c index 85b8034fd2..5a39fdac69 100644 --- a/block/qcow2-threads.c +++ b/block/qcow2-threads.c @@ -144,17 +144,11 @@ static int qcow2_compress_pool_func(void *opaque) return 0; } =20 -static void qcow2_compress_complete(void *opaque, int ret) -{ - qemu_coroutine_enter(opaque); -} - static ssize_t coroutine_fn qcow2_co_do_compress(BlockDriverState *bs, void *dest, size_t dest_size, const void *src, size_t src_size, Qcow2CompressFunc f= unc) { BDRVQcow2State *s =3D bs->opaque; - BlockAIOCB *acb; ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); Qcow2CompressData arg =3D { .dest =3D dest, @@ -169,16 +163,9 @@ qcow2_co_do_compress(BlockDriverState *bs, void *dest,= size_t dest_size, } =20 s->nb_compress_threads++; - acb =3D thread_pool_submit_aio(pool, qcow2_compress_pool_func, &arg, - qcow2_compress_complete, - qemu_coroutine_self()); - - if (!acb) { - s->nb_compress_threads--; - return -EINVAL; - } - qemu_coroutine_yield(); + thread_pool_submit_co(pool, qcow2_compress_pool_func, &arg); s->nb_compress_threads--; + qemu_co_queue_next(&s->compress_wait_queue); =20 return arg.ret; --=20 2.18.0 From nobody Mon Apr 29 02:04:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1557152987; cv=none; d=zoho.com; s=zohoarc; b=idy4vgZNUpT4UssI9+FF36r+M8cfv0J58aPtukhXHC9CIUUJI2pHgLw6E5pZPVFzpQvF/PMiWo97G/CNY84mLRArMPpOt4nWW396s3825Wf95Ggy6erHsRbn/gzRCFJN3/oxwBHWlkCJ9fe2EqksrHSZblcPcwAVQTx+Njxe30A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557152987; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=3hQekHs0tbIfhzrYvCT7nIn8S8k8REZKGVJk60Rwnno=; b=j4XrMbz0hYfYcnwW4o7JOfJM3oRozD26DcA6xckwq06HAtwLkV+Qwd0YlA42vRKOe2lvI/iCdUGi8/RQz3j19wpZFwWW0E4incLqwBpAZZtqmWrT8p9aEzCwi64nlSIREEZe6m14GEjWUm8kioGdbTVrhz2/Om7bNVcZTN++2e8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557152986878187.52086835620798; Mon, 6 May 2019 07:29:46 -0700 (PDT) Received: from localhost ([127.0.0.1]:57350 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNece-0007eN-Mi for importer@patchew.org; Mon, 06 May 2019 10:29:40 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50229) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNeap-0006Of-Nt for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hNeao-0000cM-Ms for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:47 -0400 Received: from relay.sw.ru ([185.231.240.75]:45606) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hNeao-0000bd-Fe; Mon, 06 May 2019 10:27:46 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hNeak-000444-P9; Mon, 06 May 2019 17:27:42 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 6 May 2019 17:27:37 +0300 Message-Id: <20190506142741.41731-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190506142741.41731-1-vsementsov@virtuozzo.com> References: <20190506142741.41731-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 4/8] qcow2-threads: qcow2_co_do_compress: protect queuing by mutex X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Drop dependence on AioContext lock. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Paolo Bonzini Reviewed-by: Max Reitz --- block/qcow2-threads.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c index 5a39fdac69..50a9fdeec5 100644 --- a/block/qcow2-threads.c +++ b/block/qcow2-threads.c @@ -158,15 +158,19 @@ qcow2_co_do_compress(BlockDriverState *bs, void *dest= , size_t dest_size, .func =3D func, }; =20 + qemu_co_mutex_lock(&s->lock); while (s->nb_compress_threads >=3D MAX_COMPRESS_THREADS) { - qemu_co_queue_wait(&s->compress_wait_queue, NULL); + qemu_co_queue_wait(&s->compress_wait_queue, &s->lock); } - s->nb_compress_threads++; + qemu_co_mutex_unlock(&s->lock); + thread_pool_submit_co(pool, qcow2_compress_pool_func, &arg); - s->nb_compress_threads--; =20 + qemu_co_mutex_lock(&s->lock); + s->nb_compress_threads--; qemu_co_queue_next(&s->compress_wait_queue); + qemu_co_mutex_unlock(&s->lock); =20 return arg.ret; } --=20 2.18.0 From nobody Mon Apr 29 02:04:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1557153095; cv=none; d=zoho.com; s=zohoarc; b=mVQ8BMQjz1qpnVwJ0I6LNASmqDZFdAJpKpiVfcaoIb5p5qMSQbfzCgBaL7RHteAz0Xs3zOAfqCshnfSZ5bp5fBGQvYf41THkVjsMTCgY+rMxNLtA1FXFZv38QG9vjhw7RwG8n6ky1vOMeJpJtm9p5rUksUrH3DS8mx+BpgrTBsQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557153095; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=aDT/7VDp2B1OZ82XCNo61VEHbKELFn10SXkEx9aMlWQ=; b=NqQggCMuVDBTIWtiGZ9agwSAPZ54zQE57XjLQNYYYlT1gVyYTXdtQFaZxsfLmXBHjOI7WeGDQVziGLV5ldT/94iChwufflJMU0faaQ4L8UCppNRHzRKKTPKf7AEMeJWFsamuNlDg2ngJiHwZq+5mEvJZqtf58Cd4Je7K+dkdJfM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557153095497680.2964985248693; Mon, 6 May 2019 07:31:35 -0700 (PDT) Received: from localhost ([127.0.0.1]:57403 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNeeR-00010N-BZ for importer@patchew.org; Mon, 06 May 2019 10:31:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50249) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNeaq-0006Ow-3j for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hNeao-0000cg-Qg for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:48 -0400 Received: from relay.sw.ru ([185.231.240.75]:45596) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hNeao-0000bg-Go; Mon, 06 May 2019 10:27:46 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hNeak-000444-W9; Mon, 06 May 2019 17:27:43 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 6 May 2019 17:27:38 +0300 Message-Id: <20190506142741.41731-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190506142741.41731-1-vsementsov@virtuozzo.com> References: <20190506142741.41731-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 5/8] qcow2-threads: split out generic path X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Move generic part out of qcow2_co_do_compress, to reuse it for encryption and rename things that would be shared with encryption path. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 4 ++-- block/qcow2-threads.c | 47 ++++++++++++++++++++++++++++--------------- block/qcow2.c | 2 +- 3 files changed, 34 insertions(+), 19 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 61079f7ee8..3089e9a68b 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -347,8 +347,8 @@ typedef struct BDRVQcow2State { char *image_backing_format; char *image_data_file; =20 - CoQueue compress_wait_queue; - int nb_compress_threads; + CoQueue thread_task_queue; + int nb_threads; =20 BdrvChild *data_file; } BDRVQcow2State; diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c index 50a9fdeec5..650aa2ed19 100644 --- a/block/qcow2-threads.c +++ b/block/qcow2-threads.c @@ -31,7 +31,36 @@ #include "qcow2.h" #include "block/thread-pool.h" =20 -#define MAX_COMPRESS_THREADS 4 +#define QCOW2_MAX_THREADS 4 + +static int coroutine_fn +qcow2_co_process(BlockDriverState *bs, ThreadPoolFunc *func, void *arg) +{ + int ret; + BDRVQcow2State *s =3D bs->opaque; + ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); + + qemu_co_mutex_lock(&s->lock); + while (s->nb_threads >=3D QCOW2_MAX_THREADS) { + qemu_co_queue_wait(&s->thread_task_queue, &s->lock); + } + s->nb_threads++; + qemu_co_mutex_unlock(&s->lock); + + ret =3D thread_pool_submit_co(pool, func, arg); + + qemu_co_mutex_lock(&s->lock); + s->nb_threads--; + qemu_co_queue_next(&s->thread_task_queue); + qemu_co_mutex_unlock(&s->lock); + + return ret; +} + + +/* + * Compression + */ =20 typedef ssize_t (*Qcow2CompressFunc)(void *dest, size_t dest_size, const void *src, size_t src_size); @@ -148,8 +177,6 @@ static ssize_t coroutine_fn qcow2_co_do_compress(BlockDriverState *bs, void *dest, size_t dest_size, const void *src, size_t src_size, Qcow2CompressFunc f= unc) { - BDRVQcow2State *s =3D bs->opaque; - ThreadPool *pool =3D aio_get_thread_pool(bdrv_get_aio_context(bs)); Qcow2CompressData arg =3D { .dest =3D dest, .dest_size =3D dest_size, @@ -158,19 +185,7 @@ qcow2_co_do_compress(BlockDriverState *bs, void *dest,= size_t dest_size, .func =3D func, }; =20 - qemu_co_mutex_lock(&s->lock); - while (s->nb_compress_threads >=3D MAX_COMPRESS_THREADS) { - qemu_co_queue_wait(&s->compress_wait_queue, &s->lock); - } - s->nb_compress_threads++; - qemu_co_mutex_unlock(&s->lock); - - thread_pool_submit_co(pool, qcow2_compress_pool_func, &arg); - - qemu_co_mutex_lock(&s->lock); - s->nb_compress_threads--; - qemu_co_queue_next(&s->compress_wait_queue); - qemu_co_mutex_unlock(&s->lock); + qcow2_co_process(bs, qcow2_compress_pool_func, &arg); =20 return arg.ret; } diff --git a/block/qcow2.c b/block/qcow2.c index facf8ab993..7970d0992a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1694,7 +1694,7 @@ static int coroutine_fn qcow2_do_open(BlockDriverStat= e *bs, QDict *options, } #endif =20 - qemu_co_queue_init(&s->compress_wait_queue); + qemu_co_queue_init(&s->thread_task_queue); =20 return ret; =20 --=20 2.18.0 From nobody Mon Apr 29 02:04:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1557153279; cv=none; d=zoho.com; s=zohoarc; b=FaF/1QUa8LsRkR2MQ1hLQn/cBX4K4Zwj0kcS1zRblBc0E+Lfgty4nUEYzYfd9mCXGwmHIeo7TPauL2H71BLa2M7dt9hMbIMlDHEijeQm4YUXD6xmq9rmgoIqZwx5tmx5Ce58wSfjbkzpJXT+z8WbHaiTRCia+IVwAPWH4qGw4F0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557153279; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=nz7S/WOXcZEXLhnmNyXbCS0IhYwG1n8UC9BPIT4zmjw=; b=JZgwDtQl5ptWvuytsW8Nca1DYKNiafz2j/juOYIlvJxaiZi8QLV6IvD5OwrdbNZjpiSipUC9NcdnpNR27hEOJ9Q/aMc8IhKBpOgwq8BdCdhVXXRoisD11qK+tbc5rhGzToQYki2VoccZxtNdnZCRyJE2SpI+67105dnS9Hivv58= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557153279749794.4125267480208; Mon, 6 May 2019 07:34:39 -0700 (PDT) Received: from localhost ([127.0.0.1]:57433 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNehK-0003Op-La for importer@patchew.org; Mon, 06 May 2019 10:34:30 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50252) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNeaq-0006Oz-4n for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hNeao-0000cm-Qj for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:48 -0400 Received: from relay.sw.ru ([185.231.240.75]:45586) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hNeao-0000be-GZ; Mon, 06 May 2019 10:27:46 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hNeal-000444-71; Mon, 06 May 2019 17:27:43 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 6 May 2019 17:27:39 +0300 Message-Id: <20190506142741.41731-7-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190506142741.41731-1-vsementsov@virtuozzo.com> References: <20190506142741.41731-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 6/8] qcow2: qcow2_co_preadv: improve locking X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Background: decryption will be done in threads, to take benefit of it, we should move it out of the lock first. But let's go further: it turns out, that only qcow2_get_cluster_offset() needs locking, so reduce locking to it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia --- block/qcow2.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 7970d0992a..5954696245 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1970,8 +1970,6 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverSt= ate *bs, uint64_t offset, =20 qemu_iovec_init(&hd_qiov, qiov->niov); =20 - qemu_co_mutex_lock(&s->lock); - while (bytes !=3D 0) { =20 /* prepare next request */ @@ -1981,7 +1979,9 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverSt= ate *bs, uint64_t offset, QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); } =20 + qemu_co_mutex_lock(&s->lock); ret =3D qcow2_get_cluster_offset(bs, offset, &cur_bytes, &cluster_= offset); + qemu_co_mutex_unlock(&s->lock); if (ret < 0) { goto fail; } @@ -1996,10 +1996,8 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverS= tate *bs, uint64_t offset, =20 if (bs->backing) { BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO); - qemu_co_mutex_unlock(&s->lock); ret =3D bdrv_co_preadv(bs->backing, offset, cur_bytes, &hd_qiov, 0); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { goto fail; } @@ -2015,11 +2013,9 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverS= tate *bs, uint64_t offset, break; =20 case QCOW2_CLUSTER_COMPRESSED: - qemu_co_mutex_unlock(&s->lock); ret =3D qcow2_co_preadv_compressed(bs, cluster_offset, offset, cur_bytes, &hd_qiov); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { goto fail; } @@ -2056,11 +2052,9 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverS= tate *bs, uint64_t offset, } =20 BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); - qemu_co_mutex_unlock(&s->lock); ret =3D bdrv_co_preadv(s->data_file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { goto fail; } @@ -2095,8 +2089,6 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverSt= ate *bs, uint64_t offset, ret =3D 0; =20 fail: - qemu_co_mutex_unlock(&s->lock); - qemu_iovec_destroy(&hd_qiov); qemu_vfree(cluster_data); =20 --=20 2.18.0 From nobody Mon Apr 29 02:04:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1557153147; cv=none; d=zoho.com; s=zohoarc; b=WNWQxq07JqKTzZwrBKuJAPbzWP/XIVQKz0sKN3ICwMe4hPWDDLH1jdIFr0ctpkZfxU6JIHmKV08GDpJECyywoTsjtjrp1/eb2sZKiyjfyFtRyAPMqnXBHceD00GQsTqTqJI2iF1FZjqquqksmE9jNTN7ke4qLMfidBGYMZlfX60= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557153147; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=cp6STxjxOjOf3YuApFn+x0fmeNdxzTukaHB3cbzAPdM=; b=g5XvMVwT2P9bzcczjy0OBnB9SpoNskkcqn6zcHCrk0rldToXh6WNDOaESUkzDMjY9Idy9jrT9BkpOLzSbPQ7cEA5/YDIRvyM2aY3lnuieDor8cjO14i3SfOeGniyE2P4ckwCJqkrjKi0jxgSI+HvEJQ01ukjYHWjqqm41BA9Wl8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557153147226281.525550730795; Mon, 6 May 2019 07:32:27 -0700 (PDT) Received: from localhost ([127.0.0.1]:57408 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNefF-0001mz-7O for importer@patchew.org; Mon, 06 May 2019 10:32:21 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50255) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNeaq-0006P4-9P for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hNeao-0000cz-TL for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:48 -0400 Received: from relay.sw.ru ([185.231.240.75]:45590) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hNeao-0000ba-IL; Mon, 06 May 2019 10:27:46 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hNeal-000444-DI; Mon, 06 May 2019 17:27:43 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 6 May 2019 17:27:40 +0300 Message-Id: <20190506142741.41731-8-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190506142741.41731-1-vsementsov@virtuozzo.com> References: <20190506142741.41731-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 7/8] qcow2: bdrv_co_pwritev: move encryption code out of the lock X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Encryption will be done in threads, to take benefit of it, we should move it out of the lock first. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.c | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 5954696245..cba2328005 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2169,11 +2169,20 @@ static coroutine_fn int qcow2_co_pwritev(BlockDrive= rState *bs, uint64_t offset, ret =3D qcow2_alloc_cluster_offset(bs, offset, &cur_bytes, &cluster_offset, &l2meta); if (ret < 0) { - goto fail; + goto out_locked; } =20 assert((cluster_offset & 511) =3D=3D 0); =20 + ret =3D qcow2_pre_write_overlap_check(bs, 0, + cluster_offset + offset_in_clu= ster, + cur_bytes, true); + if (ret < 0) { + goto out_locked; + } + + qemu_co_mutex_unlock(&s->lock); + qemu_iovec_reset(&hd_qiov); qemu_iovec_concat(&hd_qiov, qiov, bytes_done, cur_bytes); =20 @@ -2185,7 +2194,7 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverS= tate *bs, uint64_t offset, * s->cluster_size); if (cluster_data =3D=3D NULL) { ret =3D -ENOMEM; - goto fail; + goto out_unlocked; } } =20 @@ -2200,40 +2209,34 @@ static coroutine_fn int qcow2_co_pwritev(BlockDrive= rState *bs, uint64_t offset, cluster_data, cur_bytes, NULL) < 0) { ret =3D -EIO; - goto fail; + goto out_unlocked; } =20 qemu_iovec_reset(&hd_qiov); qemu_iovec_add(&hd_qiov, cluster_data, cur_bytes); } =20 - ret =3D qcow2_pre_write_overlap_check(bs, 0, - cluster_offset + offset_in_cluster, cur_bytes, true); - if (ret < 0) { - goto fail; - } - /* If we need to do COW, check if it's possible to merge the * writing of the guest data together with that of the COW regions. * If it's not possible (or not necessary) then write the * guest data now. */ if (!merge_cow(offset, cur_bytes, &hd_qiov, l2meta)) { - qemu_co_mutex_unlock(&s->lock); BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); trace_qcow2_writev_data(qemu_coroutine_self(), cluster_offset + offset_in_cluster); ret =3D bdrv_co_pwritev(s->data_file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { - goto fail; + goto out_unlocked; } } =20 + qemu_co_mutex_lock(&s->lock); + ret =3D qcow2_handle_l2meta(bs, &l2meta, true); if (ret) { - goto fail; + goto out_locked; } =20 bytes -=3D cur_bytes; @@ -2242,8 +2245,12 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriver= State *bs, uint64_t offset, trace_qcow2_writev_done_part(qemu_coroutine_self(), cur_bytes); } ret =3D 0; + goto out_locked; =20 -fail: +out_unlocked: + qemu_co_mutex_lock(&s->lock); + +out_locked: qcow2_handle_l2meta(bs, &l2meta, false); =20 qemu_co_mutex_unlock(&s->lock); --=20 2.18.0 From nobody Mon Apr 29 02:04:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1557153274; cv=none; d=zoho.com; s=zohoarc; b=ZzsqQKHUyUl87jCVdEHJwyjj1VxfuoIwSebGKT0Dcblx7lZtI44E5NPft7RNy8ym9bbO8J9D8YCHm1pVAple6yg8+3QYxhqKemevlVynSt00zSy3B+5thzDESXmhgkcgAZDjkcOwjMCJCJ9e5LllizX0fyVjReS+dCIZOYOpxxw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1557153274; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=z9X9MroHPnMsrTghqwGhUijX3ca6JMShYqYk/46yNTc=; b=Isywcjs+5fQTShD3ROvlQ83Bxqq8o4ta424rZ1kyv+CmIfjyN2BcryKbUgtTH4ajDPjWlqntIkYMN5a9XgmHvzYNCu+fnM0DC3vrlW8ZpBOcUq4zTykqca0+Rn50mWUgjQqr8EFRUU0Im9rTbD078460jCWdlg3lOco/P5he9Xg= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1557153274820754.5626598233966; Mon, 6 May 2019 07:34:34 -0700 (PDT) Received: from localhost ([127.0.0.1]:57431 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNehJ-0003Nz-Jb for importer@patchew.org; Mon, 06 May 2019 10:34:29 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50256) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hNeaq-0006P5-9W for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hNeao-0000cS-OC for qemu-devel@nongnu.org; Mon, 06 May 2019 10:27:48 -0400 Received: from relay.sw.ru ([185.231.240.75]:45614) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hNeao-0000bh-Ek; Mon, 06 May 2019 10:27:46 -0400 Received: from [10.28.8.145] (helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hNeal-000444-Kd; Mon, 06 May 2019 17:27:43 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Mon, 6 May 2019 17:27:41 +0300 Message-Id: <20190506142741.41731-9-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190506142741.41731-1-vsementsov@virtuozzo.com> References: <20190506142741.41731-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v6 8/8] qcow2: do encryption in threads X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, berto@igalia.com, mreitz@redhat.com, den@openvz.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Do encryption/decryption in threads, like it is already done for compression. This improves asynchronous encrypted io. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 8 ++++++ block/qcow2-cluster.c | 7 ++--- block/qcow2-threads.c | 65 +++++++++++++++++++++++++++++++++++++++++-- block/qcow2.c | 22 +++++---------- 4 files changed, 81 insertions(+), 21 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 3089e9a68b..5169c0d8ab 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -264,6 +264,8 @@ typedef struct Qcow2BitmapHeaderExt { uint64_t bitmap_directory_offset; } QEMU_PACKED Qcow2BitmapHeaderExt; =20 +#define QCOW2_MAX_THREADS 4 + typedef struct BDRVQcow2State { int cluster_bits; int cluster_size; @@ -741,5 +743,11 @@ qcow2_co_compress(BlockDriverState *bs, void *dest, si= ze_t dest_size, ssize_t coroutine_fn qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size, const void *src, size_t src_size); +int coroutine_fn +qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len); +int coroutine_fn +qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len); =20 #endif diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index c4965479f6..e6624c6e4e 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -471,13 +471,12 @@ static bool coroutine_fn do_perform_cow_encrypt(Block= DriverState *bs, { if (bytes && bs->encrypted) { BDRVQcow2State *s =3D bs->opaque; - int64_t offset =3D (s->crypt_physical_offset ? - (cluster_offset + offset_in_cluster) : - (src_cluster_offset + offset_in_cluster)); assert((offset_in_cluster & ~BDRV_SECTOR_MASK) =3D=3D 0); assert((bytes & ~BDRV_SECTOR_MASK) =3D=3D 0); assert(s->crypto); - if (qcrypto_block_encrypt(s->crypto, offset, buffer, bytes, NULL) = < 0) { + if (qcow2_co_encrypt(bs, cluster_offset, + src_cluster_offset + offset_in_cluster, + buffer, bytes) < 0) { return false; } } diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c index 650aa2ed19..3b1e63fe41 100644 --- a/block/qcow2-threads.c +++ b/block/qcow2-threads.c @@ -30,8 +30,7 @@ =20 #include "qcow2.h" #include "block/thread-pool.h" - -#define QCOW2_MAX_THREADS 4 +#include "crypto.h" =20 static int coroutine_fn qcow2_co_process(BlockDriverState *bs, ThreadPoolFunc *func, void *arg) @@ -205,3 +204,65 @@ qcow2_co_decompress(BlockDriverState *bs, void *dest, = size_t dest_size, return qcow2_co_do_compress(bs, dest, dest_size, src, src_size, qcow2_decompress); } + + +/* + * Cryptography + */ + +/* + * Qcow2EncDecFunc: common prototype of qcrypto_block_encrypt() and + * qcrypto_block_decrypt() functions. + */ +typedef int (*Qcow2EncDecFunc)(QCryptoBlock *block, uint64_t offset, + uint8_t *buf, size_t len, Error **errp); + +typedef struct Qcow2EncDecData { + QCryptoBlock *block; + uint64_t offset; + uint8_t *buf; + size_t len; + + Qcow2EncDecFunc func; +} Qcow2EncDecData; + +static int qcow2_encdec_pool_func(void *opaque) +{ + Qcow2EncDecData *data =3D opaque; + + return data->func(data->block, data->offset, data->buf, data->len, NUL= L); +} + +static int coroutine_fn +qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc = func) +{ + BDRVQcow2State *s =3D bs->opaque; + Qcow2EncDecData arg =3D { + .block =3D s->crypto, + .offset =3D s->crypt_physical_offset ? + file_cluster_offset + offset_into_cluster(s, offset)= : + offset, + .buf =3D buf, + .len =3D len, + .func =3D func, + }; + + return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg); +} + +int coroutine_fn +qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len) +{ + return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len, + qcrypto_block_encrypt); +} + +int coroutine_fn +qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset, + uint64_t offset, void *buf, size_t len) +{ + return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len, + qcrypto_block_decrypt); +} diff --git a/block/qcow2.c b/block/qcow2.c index cba2328005..dee3b8cca3 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -297,7 +297,7 @@ static int qcow2_read_extensions(BlockDriverState *bs, = uint64_t start_offset, } s->crypto =3D qcrypto_block_open(s->crypto_opts, "encrypt.", qcow2_crypto_hdr_read_func, - bs, cflags, 1, errp); + bs, cflags, QCOW2_MAX_THREADS, = errp); if (!s->crypto) { return -EINVAL; } @@ -1539,7 +1539,8 @@ static int coroutine_fn qcow2_do_open(BlockDriverStat= e *bs, QDict *options, cflags |=3D QCRYPTO_BLOCK_OPEN_NO_IO; } s->crypto =3D qcrypto_block_open(s->crypto_opts, "encrypt.", - NULL, NULL, cflags, 1, errp); + NULL, NULL, cflags, + QCOW2_MAX_THREADS, errp); if (!s->crypto) { ret =3D -EINVAL; goto fail; @@ -2062,13 +2063,8 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverS= tate *bs, uint64_t offset, assert(s->crypto); assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - if (qcrypto_block_decrypt(s->crypto, - (s->crypt_physical_offset ? - cluster_offset + offset_in_clus= ter : - offset), - cluster_data, - cur_bytes, - NULL) < 0) { + if (qcow2_co_decrypt(bs, cluster_offset, offset, + cluster_data, cur_bytes) < 0) { ret =3D -EIO; goto fail; } @@ -2202,12 +2198,8 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriver= State *bs, uint64_t offset, QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); qemu_iovec_to_buf(&hd_qiov, 0, cluster_data, hd_qiov.size); =20 - if (qcrypto_block_encrypt(s->crypto, - (s->crypt_physical_offset ? - cluster_offset + offset_in_cluster : - offset), - cluster_data, - cur_bytes, NULL) < 0) { + if (qcow2_co_encrypt(bs, cluster_offset, offset, + cluster_data, cur_bytes) < 0) { ret =3D -EIO; goto out_unlocked; } --=20 2.18.0