From nobody Sun Feb 8 14:52:54 2026 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1506516961251696.3065228984666; Wed, 27 Sep 2017 05:56:01 -0700 (PDT) Received: from localhost ([::1]:54703 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dxBsP-0005Tt-Ph for importer@patchew.org; Wed, 27 Sep 2017 08:55:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49930) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dxBqi-0004I2-Ac for qemu-devel@nongnu.org; Wed, 27 Sep 2017 08:54:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dxBqe-0001Qr-Cf for qemu-devel@nongnu.org; Wed, 27 Sep 2017 08:54:00 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52442) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dxBqa-0001MO-Qo; Wed, 27 Sep 2017 08:53:53 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BC13FC04D2E4; Wed, 27 Sep 2017 12:53:51 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.42.22.189]) by smtp.corp.redhat.com (Postfix) with ESMTP id 245927F760; Wed, 27 Sep 2017 12:53:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com BC13FC04D2E4 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=berrange@redhat.com From: "Daniel P. Berrange" To: qemu-devel@nongnu.org Date: Wed, 27 Sep 2017 13:53:35 +0100 Message-Id: <20170927125340.12360-2-berrange@redhat.com> In-Reply-To: <20170927125340.12360-1-berrange@redhat.com> References: <20170927125340.12360-1-berrange@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 27 Sep 2017 12:53:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 1/6] block: use 1 MB bounce buffers for crypto instead of 16KB X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-block@nongnu.org, Stefan Hajnoczi , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Using 16KB bounce buffers creates a significant performance penalty for I/O to encrypted volumes on storage which high I/O latency (rotating rust & network drives), because it triggers lots of fairly small I/O operations. On tests with rotating rust, and cache=3Dnone|directsync, write speed increased from 2MiB/s to 32MiB/s, on a par with that achieved by the in-kernel luks driver. With other cache modes the in-kernel driver is still notably faster because it is able to report completion of the I/O request before any encryption is done, while the in-QEMU driver must encrypt the data before completion. Signed-off-by: Daniel P. Berrange Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/crypto.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index 58ef6f2f52..684cabeaf8 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -379,7 +379,11 @@ static void block_crypto_close(BlockDriverState *bs) } =20 =20 -#define BLOCK_CRYPTO_MAX_SECTORS 32 +/* + * 1 MB bounce buffer gives good performance / memory tradeoff + * when using cache=3Dnone|directsync. + */ +#define BLOCK_CRYPTO_MAX_IO_SIZE (1024 * 1024) =20 static coroutine_fn int block_crypto_co_readv(BlockDriverState *bs, int64_t sector_num, @@ -396,12 +400,11 @@ block_crypto_co_readv(BlockDriverState *bs, int64_t s= ector_num, =20 qemu_iovec_init(&hd_qiov, qiov->niov); =20 - /* Bounce buffer so we have a linear mem region for - * entire sector. XXX optimize so we avoid bounce - * buffer in case that qiov->niov =3D=3D 1 + /* Bounce buffer because we don't wish to expose cipher text + * in qiov which points to guest memory. */ cipher_data =3D - qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_SECTORS * 5= 12, + qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE, qiov->size)); if (cipher_data =3D=3D NULL) { ret =3D -ENOMEM; @@ -411,8 +414,8 @@ block_crypto_co_readv(BlockDriverState *bs, int64_t sec= tor_num, while (remaining_sectors) { cur_nr_sectors =3D remaining_sectors; =20 - if (cur_nr_sectors > BLOCK_CRYPTO_MAX_SECTORS) { - cur_nr_sectors =3D BLOCK_CRYPTO_MAX_SECTORS; + if (cur_nr_sectors > (BLOCK_CRYPTO_MAX_IO_SIZE / 512)) { + cur_nr_sectors =3D (BLOCK_CRYPTO_MAX_IO_SIZE / 512); } =20 qemu_iovec_reset(&hd_qiov); @@ -464,12 +467,11 @@ block_crypto_co_writev(BlockDriverState *bs, int64_t = sector_num, =20 qemu_iovec_init(&hd_qiov, qiov->niov); =20 - /* Bounce buffer so we have a linear mem region for - * entire sector. XXX optimize so we avoid bounce - * buffer in case that qiov->niov =3D=3D 1 + /* Bounce buffer because we're not permitted to touch + * contents of qiov - it points to guest memory. */ cipher_data =3D - qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_SECTORS * 5= 12, + qemu_try_blockalign(bs->file->bs, MIN(BLOCK_CRYPTO_MAX_IO_SIZE, qiov->size)); if (cipher_data =3D=3D NULL) { ret =3D -ENOMEM; @@ -479,8 +481,8 @@ block_crypto_co_writev(BlockDriverState *bs, int64_t se= ctor_num, while (remaining_sectors) { cur_nr_sectors =3D remaining_sectors; =20 - if (cur_nr_sectors > BLOCK_CRYPTO_MAX_SECTORS) { - cur_nr_sectors =3D BLOCK_CRYPTO_MAX_SECTORS; + if (cur_nr_sectors > (BLOCK_CRYPTO_MAX_IO_SIZE / 512)) { + cur_nr_sectors =3D (BLOCK_CRYPTO_MAX_IO_SIZE / 512); } =20 qemu_iovec_to_buf(qiov, bytes_done, --=20 2.13.5