From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 152639957164047.52026628186161; Tue, 15 May 2018 08:52:51 -0700 (PDT) Received: from localhost ([::1]:44488 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcFu-0007pY-Oh for importer@patchew.org; Tue, 15 May 2018 11:52:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38640) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4O-0006cI-Ag for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4J-0001M1-NV for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:56 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47534 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc49-0001CU-Ag; Tue, 15 May 2018 11:40:41 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DC64B7CBBA; Tue, 15 May 2018 15:40:40 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3A1C12024CBA; Tue, 15 May 2018 15:40:40 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:39:57 +0200 Message-Id: <20180515154033.19899-2-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:40:40 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:40:40 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 01/37] block-backend: simplify blk_get_aio_context X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Daniel Henrique Barboza blk_get_aio_context verifies if BlockDriverState bs is not NULL, return bdrv_get_aio_context(bs) if true or qemu_get_aio_context() otherwise. However, bdrv_get_aio_context from block.c already does this verification itself, also returning qemu_get_aio_context() if bs is NULL: AioContext *bdrv_get_aio_context(BlockDriverState *bs) { return bs ? bs->aio_context : qemu_get_aio_context(); } This patch simplifies blk_get_aio_context to simply call bdrv_get_aio_context instead of replicating the same logic. Signed-off-by: Daniel Henrique Barboza Reviewed-by: Darren Kenny Signed-off-by: Kevin Wolf --- block/block-backend.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index 681b240b12..89f47b00ea 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -1865,13 +1865,7 @@ void blk_op_unblock_all(BlockBackend *blk, Error *re= ason) =20 AioContext *blk_get_aio_context(BlockBackend *blk) { - BlockDriverState *bs =3D blk_bs(blk); - - if (bs) { - return bdrv_get_aio_context(bs); - } else { - return qemu_get_aio_context(); - } + return bdrv_get_aio_context(blk_bs(blk)); } =20 static AioContext *blk_aiocb_get_aio_context(BlockAIOCB *acb) --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15263992883981009.8527476575861; Tue, 15 May 2018 08:48:08 -0700 (PDT) Received: from localhost ([::1]:44451 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcBL-0003wB-J0 for importer@patchew.org; Tue, 15 May 2018 11:48:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38496) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4G-0006Sg-VN for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4F-0001HL-Dp for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:48 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47536 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4A-0001D8-7c; Tue, 15 May 2018 11:40:42 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C3F937CBBA; Tue, 15 May 2018 15:40:41 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 22ADC200BC1E; Tue, 15 May 2018 15:40:41 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:39:58 +0200 Message-Id: <20180515154033.19899-3-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:40:41 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:40:41 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 02/37] block: Support byte-based aio callbacks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Add new sector-based aio callbacks for read and write, to match the fact that bdrv_aio_pdiscard is already byte-based. Ideally, drivers should be converted to use coroutine callbacks rather than aio; but that is not quite as trivial (and if we were to do that conversion, the null-aio driver would disappear), so for the short term, converting the signature but keeping things with aio is easier. However, we CAN declare that a driver that uses the byte-based aio interfaces now defaults to byte-based operations, and must explicitly provide a refresh_limits override to stick with larger alignments (making the alignment issues more obvious directly in the drivers touched in the next few patches). Once all drivers are converted, the sector-based aio callbacks will be removed; in the meantime, a FIXME comment is added due to a slight inefficiency that will be touched up as part of that later cleanup. Simplify some instances of 'bs->drv' into 'drv' while touching this, since the local variable already exists to reduce typing. Signed-off-by: Eric Blake Signed-off-by: Kevin Wolf --- include/block/block_int.h | 6 ++++++ block/io.c | 38 +++++++++++++++++++++++++++++--------- 2 files changed, 35 insertions(+), 9 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index c4dd1d4bb8..e772e3502b 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -144,9 +144,15 @@ struct BlockDriver { BlockAIOCB *(*bdrv_aio_readv)(BlockDriverState *bs, int64_t sector_num, QEMUIOVector *qiov, int nb_sectors, BlockCompletionFunc *cb, void *opaque); + BlockAIOCB *(*bdrv_aio_preadv)(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags, + BlockCompletionFunc *cb, void *opaque); BlockAIOCB *(*bdrv_aio_writev)(BlockDriverState *bs, int64_t sector_num, QEMUIOVector *qiov, int nb_sectors, BlockCompletionFunc *cb, void *opaque); + BlockAIOCB *(*bdrv_aio_pwritev)(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags, + BlockCompletionFunc *cb, void *opaque); BlockAIOCB *(*bdrv_aio_flush)(BlockDriverState *bs, BlockCompletionFunc *cb, void *opaque); BlockAIOCB *(*bdrv_aio_pdiscard)(BlockDriverState *bs, diff --git a/block/io.c b/block/io.c index bd9a19a9c4..407bc25df4 100644 --- a/block/io.c +++ b/block/io.c @@ -92,7 +92,8 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **er= rp) } =20 /* Default alignment based on whether driver has byte interface */ - bs->bl.request_alignment =3D drv->bdrv_co_preadv ? 1 : 512; + bs->bl.request_alignment =3D (drv->bdrv_co_preadv || + drv->bdrv_aio_preadv) ? 1 : 512; =20 /* Take some limits from the children as a default */ if (bs->file) { @@ -924,12 +925,15 @@ static int coroutine_fn bdrv_driver_preadv(BlockDrive= rState *bs, return drv->bdrv_co_preadv(bs, offset, bytes, qiov, flags); } =20 + /* FIXME - no need to calculate these if .bdrv_aio_preadv exists */ sector_num =3D offset >> BDRV_SECTOR_BITS; nb_sectors =3D bytes >> BDRV_SECTOR_BITS; =20 - assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - assert((bytes >> BDRV_SECTOR_BITS) <=3D BDRV_REQUEST_MAX_SECTORS); + if (!drv->bdrv_aio_preadv) { + assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); + assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); + assert((bytes >> BDRV_SECTOR_BITS) <=3D BDRV_REQUEST_MAX_SECTORS); + } =20 if (drv->bdrv_co_readv) { return drv->bdrv_co_readv(bs, sector_num, nb_sectors, qiov); @@ -939,8 +943,13 @@ static int coroutine_fn bdrv_driver_preadv(BlockDriver= State *bs, .coroutine =3D qemu_coroutine_self(), }; =20 - acb =3D bs->drv->bdrv_aio_readv(bs, sector_num, qiov, nb_sectors, + if (drv->bdrv_aio_preadv) { + acb =3D drv->bdrv_aio_preadv(bs, offset, bytes, qiov, flags, + bdrv_co_io_em_complete, &co); + } else { + acb =3D drv->bdrv_aio_readv(bs, sector_num, qiov, nb_sectors, bdrv_co_io_em_complete, &co); + } if (acb =3D=3D NULL) { return -EIO; } else { @@ -972,12 +981,15 @@ static int coroutine_fn bdrv_driver_pwritev(BlockDriv= erState *bs, goto emulate_flags; } =20 + /* FIXME - no need to calculate these if .bdrv_aio_pwritev exists */ sector_num =3D offset >> BDRV_SECTOR_BITS; nb_sectors =3D bytes >> BDRV_SECTOR_BITS; =20 - assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - assert((bytes >> BDRV_SECTOR_BITS) <=3D BDRV_REQUEST_MAX_SECTORS); + if (!drv->bdrv_aio_pwritev) { + assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); + assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); + assert((bytes >> BDRV_SECTOR_BITS) <=3D BDRV_REQUEST_MAX_SECTORS); + } =20 if (drv->bdrv_co_writev_flags) { ret =3D drv->bdrv_co_writev_flags(bs, sector_num, nb_sectors, qiov, @@ -992,8 +1004,16 @@ static int coroutine_fn bdrv_driver_pwritev(BlockDriv= erState *bs, .coroutine =3D qemu_coroutine_self(), }; =20 - acb =3D bs->drv->bdrv_aio_writev(bs, sector_num, qiov, nb_sectors, + if (drv->bdrv_aio_pwritev) { + acb =3D drv->bdrv_aio_pwritev(bs, offset, bytes, qiov, + flags & bs->supported_write_flags, + bdrv_co_io_em_complete, &co); + flags &=3D ~bs->supported_write_flags; + } else { + assert(!bs->supported_write_flags); + acb =3D drv->bdrv_aio_writev(bs, sector_num, qiov, nb_sectors, bdrv_co_io_em_complete, &co); + } if (acb =3D=3D NULL) { ret =3D -EIO; } else { --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399006217982.6093866718988; Tue, 15 May 2018 08:43:26 -0700 (PDT) Received: from localhost ([::1]:44326 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc6d-0008GZ-0V for importer@patchew.org; Tue, 15 May 2018 11:43:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38530) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4I-0006Tl-1u for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4F-0001HF-CZ for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:50 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:50318 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4B-0001Dp-4b; Tue, 15 May 2018 11:40:43 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AC7044021BA3; Tue, 15 May 2018 15:40:42 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B77F2024CBA; Tue, 15 May 2018 15:40:41 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:39:59 +0200 Message-Id: <20180515154033.19899-4-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 15 May 2018 15:40:42 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 15 May 2018 15:40:42 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 03/37] file-win32: Switch to byte-based callbacks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Make the change for the last few sector-based callbacks in the file-win32 driver. Note that the driver was already using byte-based calls for performing actual I/O, so this just gets rid of a round trip of scaling; however, as I don't know if Windows is tolerant of non-sector AIO operations, I went with the conservative approach of modifying .bdrv_refresh_limits to override the block layer defaults back to the pre-patch value of 512. Signed-off-by: Eric Blake Signed-off-by: Kevin Wolf --- include/block/raw-aio.h | 2 +- block/file-win32.c | 47 +++++++++++++++++++++++++++++----------------= -- block/win32-aio.c | 5 ++--- 3 files changed, 32 insertions(+), 22 deletions(-) diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h index a4cdbbf1b7..9e47b8a629 100644 --- a/include/block/raw-aio.h +++ b/include/block/raw-aio.h @@ -57,7 +57,7 @@ void win32_aio_cleanup(QEMUWin32AIOState *aio); int win32_aio_attach(QEMUWin32AIOState *aio, HANDLE hfile); BlockAIOCB *win32_aio_submit(BlockDriverState *bs, QEMUWin32AIOState *aio, HANDLE hfile, - int64_t sector_num, QEMUIOVector *qiov, int nb_sectors, + uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, BlockCompletionFunc *cb, void *opaque, int type); void win32_aio_detach_aio_context(QEMUWin32AIOState *aio, AioContext *old_context); diff --git a/block/file-win32.c b/block/file-win32.c index 2e2f746bb1..3c67db4336 100644 --- a/block/file-win32.c +++ b/block/file-win32.c @@ -251,7 +251,11 @@ static void raw_probe_alignment(BlockDriverState *bs, = Error **errp) &dg.Geometry.BytesPerSector, &freeClusters, &totalClusters); bs->bl.request_alignment =3D dg.Geometry.BytesPerSector; + return; } + + /* XXX Does Windows support AIO on less than 512-byte alignment? */ + bs->bl.request_alignment =3D 512; } =20 static void raw_parse_flags(int flags, bool use_aio, int *access_flags, @@ -410,32 +414,32 @@ fail: return ret; } =20 -static BlockAIOCB *raw_aio_readv(BlockDriverState *bs, - int64_t sector_num, QEMUIOVector *qiov, int nb_se= ctors, - BlockCompletionFunc *cb, void *opaque) +static BlockAIOCB *raw_aio_preadv(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags, + BlockCompletionFunc *cb, void *opaque) { BDRVRawState *s =3D bs->opaque; if (s->aio) { - return win32_aio_submit(bs, s->aio, s->hfile, sector_num, qiov, - nb_sectors, cb, opaque, QEMU_AIO_READ); + return win32_aio_submit(bs, s->aio, s->hfile, offset, bytes, qiov, + cb, opaque, QEMU_AIO_READ); } else { - return paio_submit(bs, s->hfile, sector_num << BDRV_SECTOR_BITS, q= iov, - nb_sectors << BDRV_SECTOR_BITS, + return paio_submit(bs, s->hfile, offset, qiov, bytes, cb, opaque, QEMU_AIO_READ); } } =20 -static BlockAIOCB *raw_aio_writev(BlockDriverState *bs, - int64_t sector_num, QEMUIOVector *qiov, int nb_s= ectors, - BlockCompletionFunc *cb, void *opaque) +static BlockAIOCB *raw_aio_pwritev(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags, + BlockCompletionFunc *cb, void *opaque) { BDRVRawState *s =3D bs->opaque; if (s->aio) { - return win32_aio_submit(bs, s->aio, s->hfile, sector_num, qiov, - nb_sectors, cb, opaque, QEMU_AIO_WRITE); + return win32_aio_submit(bs, s->aio, s->hfile, offset, bytes, qiov, + cb, opaque, QEMU_AIO_WRITE); } else { - return paio_submit(bs, s->hfile, sector_num << BDRV_SECTOR_BITS, q= iov, - nb_sectors << BDRV_SECTOR_BITS, + return paio_submit(bs, s->hfile, offset, qiov, bytes, cb, opaque, QEMU_AIO_WRITE); } } @@ -632,8 +636,8 @@ BlockDriver bdrv_file =3D { .bdrv_co_create_opts =3D raw_co_create_opts, .bdrv_has_zero_init =3D bdrv_has_zero_init_1, =20 - .bdrv_aio_readv =3D raw_aio_readv, - .bdrv_aio_writev =3D raw_aio_writev, + .bdrv_aio_preadv =3D raw_aio_preadv, + .bdrv_aio_pwritev =3D raw_aio_pwritev, .bdrv_aio_flush =3D raw_aio_flush, =20 .bdrv_truncate =3D raw_truncate, @@ -708,6 +712,12 @@ static void hdev_parse_filename(const char *filename, = QDict *options, bdrv_parse_filename_strip_prefix(filename, "host_device:", options); } =20 +static void hdev_refresh_limits(BlockDriverState *bs, Error **errp) +{ + /* XXX Does Windows support AIO on less than 512-byte alignment? */ + bs->bl.request_alignment =3D 512; +} + static int hdev_open(BlockDriverState *bs, QDict *options, int flags, Error **errp) { @@ -793,9 +803,10 @@ static BlockDriver bdrv_host_device =3D { .bdrv_probe_device =3D hdev_probe_device, .bdrv_file_open =3D hdev_open, .bdrv_close =3D raw_close, + .bdrv_refresh_limits =3D hdev_refresh_limits, =20 - .bdrv_aio_readv =3D raw_aio_readv, - .bdrv_aio_writev =3D raw_aio_writev, + .bdrv_aio_preadv =3D raw_aio_preadv, + .bdrv_aio_pwritev =3D raw_aio_pwritev, .bdrv_aio_flush =3D raw_aio_flush, =20 .bdrv_detach_aio_context =3D raw_detach_aio_context, diff --git a/block/win32-aio.c b/block/win32-aio.c index 3be8f458fa..9cd355d42f 100644 --- a/block/win32-aio.c +++ b/block/win32-aio.c @@ -112,15 +112,14 @@ static const AIOCBInfo win32_aiocb_info =3D { =20 BlockAIOCB *win32_aio_submit(BlockDriverState *bs, QEMUWin32AIOState *aio, HANDLE hfile, - int64_t sector_num, QEMUIOVector *qiov, int nb_sectors, + uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, BlockCompletionFunc *cb, void *opaque, int type) { struct QEMUWin32AIOCB *waiocb; - uint64_t offset =3D sector_num * 512; DWORD rc; =20 waiocb =3D qemu_aio_get(&win32_aiocb_info, bs, cb, opaque); - waiocb->nbytes =3D nb_sectors * 512; + waiocb->nbytes =3D bytes; waiocb->qiov =3D qiov; waiocb->is_read =3D (type =3D=3D QEMU_AIO_READ); =20 --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399565345643.4105844758913; Tue, 15 May 2018 08:52:45 -0700 (PDT) Received: from localhost ([::1]:44487 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcFo-0007mm-J0 for importer@patchew.org; Tue, 15 May 2018 11:52:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38585) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4K-0006XP-8j for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4F-0001H0-5J for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:52 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:50324 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4C-0001EO-1f; Tue, 15 May 2018 11:40:44 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9464B4021BB0; Tue, 15 May 2018 15:40:43 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id E7D982024CBA; Tue, 15 May 2018 15:40:42 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:00 +0200 Message-Id: <20180515154033.19899-5-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 15 May 2018 15:40:43 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 15 May 2018 15:40:43 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 04/37] null: Switch to byte-based read/write X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Make the change for the last few sector-based callbacks in the null-co and null-aio drivers. Note that since the null driver does nothing on writes, it trivially supports the BDRV_REQ_FUA flag (all writes have already landed to the same bit-bucket without needing an extra flush call). Also, since the null driver does just as well with byte-based requests, we can now avoid cycles wasted on read-modify-write by taking advantage of the block layer now defaulting the alignment to 1 instead of 512. Signed-off-by: Eric Blake Signed-off-by: Kevin Wolf --- block/null.c | 45 +++++++++++++++++++++++---------------------- 1 file changed, 23 insertions(+), 22 deletions(-) diff --git a/block/null.c b/block/null.c index 3944550f67..5d610fdfba 100644 --- a/block/null.c +++ b/block/null.c @@ -93,6 +93,7 @@ static int null_file_open(BlockDriverState *bs, QDict *op= tions, int flags, } s->read_zeroes =3D qemu_opt_get_bool(opts, NULL_OPT_ZEROES, false); qemu_opts_del(opts); + bs->supported_write_flags =3D BDRV_REQ_FUA; return ret; } =20 @@ -116,22 +117,22 @@ static coroutine_fn int null_co_common(BlockDriverSta= te *bs) return 0; } =20 -static coroutine_fn int null_co_readv(BlockDriverState *bs, - int64_t sector_num, int nb_sectors, - QEMUIOVector *qiov) +static coroutine_fn int null_co_preadv(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags) { BDRVNullState *s =3D bs->opaque; =20 if (s->read_zeroes) { - qemu_iovec_memset(qiov, 0, 0, nb_sectors * BDRV_SECTOR_SIZE); + qemu_iovec_memset(qiov, 0, 0, bytes); } =20 return null_co_common(bs); } =20 -static coroutine_fn int null_co_writev(BlockDriverState *bs, - int64_t sector_num, int nb_sectors, - QEMUIOVector *qiov) +static coroutine_fn int null_co_pwritev(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags) { return null_co_common(bs); } @@ -186,26 +187,26 @@ static inline BlockAIOCB *null_aio_common(BlockDriver= State *bs, return &acb->common; } =20 -static BlockAIOCB *null_aio_readv(BlockDriverState *bs, - int64_t sector_num, QEMUIOVector *qiov, - int nb_sectors, - BlockCompletionFunc *cb, - void *opaque) +static BlockAIOCB *null_aio_preadv(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags, + BlockCompletionFunc *cb, + void *opaque) { BDRVNullState *s =3D bs->opaque; =20 if (s->read_zeroes) { - qemu_iovec_memset(qiov, 0, 0, nb_sectors * BDRV_SECTOR_SIZE); + qemu_iovec_memset(qiov, 0, 0, bytes); } =20 return null_aio_common(bs, cb, opaque); } =20 -static BlockAIOCB *null_aio_writev(BlockDriverState *bs, - int64_t sector_num, QEMUIOVector *qiov, - int nb_sectors, - BlockCompletionFunc *cb, - void *opaque) +static BlockAIOCB *null_aio_pwritev(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags, + BlockCompletionFunc *cb, + void *opaque) { return null_aio_common(bs, cb, opaque); } @@ -265,8 +266,8 @@ static BlockDriver bdrv_null_co =3D { .bdrv_close =3D null_close, .bdrv_getlength =3D null_getlength, =20 - .bdrv_co_readv =3D null_co_readv, - .bdrv_co_writev =3D null_co_writev, + .bdrv_co_preadv =3D null_co_preadv, + .bdrv_co_pwritev =3D null_co_pwritev, .bdrv_co_flush_to_disk =3D null_co_flush, .bdrv_reopen_prepare =3D null_reopen_prepare, =20 @@ -285,8 +286,8 @@ static BlockDriver bdrv_null_aio =3D { .bdrv_close =3D null_close, .bdrv_getlength =3D null_getlength, =20 - .bdrv_aio_readv =3D null_aio_readv, - .bdrv_aio_writev =3D null_aio_writev, + .bdrv_aio_preadv =3D null_aio_preadv, + .bdrv_aio_pwritev =3D null_aio_pwritev, .bdrv_aio_flush =3D null_aio_flush, .bdrv_reopen_prepare =3D null_reopen_prepare, =20 --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399002262373.54022225786196; Tue, 15 May 2018 08:43:22 -0700 (PDT) Received: from localhost ([::1]:44327 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc6j-0008MH-HL for importer@patchew.org; Tue, 15 May 2018 11:43:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38489) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4G-0006SR-Qe for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4F-0001HV-GM for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:48 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:50326 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4C-0001Eu-Uv; Tue, 15 May 2018 11:40:45 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7B9894021BA3; Tue, 15 May 2018 15:40:44 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id CF500200BC1E; Tue, 15 May 2018 15:40:43 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:01 +0200 Message-Id: <20180515154033.19899-6-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 15 May 2018 15:40:44 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 15 May 2018 15:40:44 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 05/37] rbd: Switch to byte-based callbacks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Make the change for the last few sector-based callbacks in the rbd driver. Note that the driver was already using byte-based calls for performing actual I/O, so this just gets rid of a round trip of scaling; however, as I don't know if RBD is tolerant of non-sector AIO operations, I went with the conservate approach of adding .bdrv_refresh_limits to override the block layer defaults back to the pre-patch value of 512. Signed-off-by: Eric Blake Signed-off-by: Kevin Wolf --- block/rbd.c | 40 ++++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index a14b42fcde..a16431e267 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -231,6 +231,13 @@ done: } =20 =20 +static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp) +{ + /* XXX Does RBD support AIO on less than 512-byte alignment? */ + bs->bl.request_alignment =3D 512; +} + + static int qemu_rbd_set_auth(rados_t cluster, const char *secretid, Error **errp) { @@ -899,27 +906,23 @@ failed: return NULL; } =20 -static BlockAIOCB *qemu_rbd_aio_readv(BlockDriverState *bs, - int64_t sector_num, - QEMUIOVector *qiov, - int nb_sectors, - BlockCompletionFunc *cb, - void *opaque) +static BlockAIOCB *qemu_rbd_aio_preadv(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags, + BlockCompletionFunc *cb, + void *opaque) { - return rbd_start_aio(bs, sector_num << BDRV_SECTOR_BITS, qiov, - (int64_t) nb_sectors << BDRV_SECTOR_BITS, cb, opa= que, + return rbd_start_aio(bs, offset, qiov, bytes, cb, opaque, RBD_AIO_READ); } =20 -static BlockAIOCB *qemu_rbd_aio_writev(BlockDriverState *bs, - int64_t sector_num, - QEMUIOVector *qiov, - int nb_sectors, - BlockCompletionFunc *cb, - void *opaque) +static BlockAIOCB *qemu_rbd_aio_pwritev(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags, + BlockCompletionFunc *cb, + void *opaque) { - return rbd_start_aio(bs, sector_num << BDRV_SECTOR_BITS, qiov, - (int64_t) nb_sectors << BDRV_SECTOR_BITS, cb, opa= que, + return rbd_start_aio(bs, offset, qiov, bytes, cb, opaque, RBD_AIO_WRITE); } =20 @@ -1158,6 +1161,7 @@ static BlockDriver bdrv_rbd =3D { .format_name =3D "rbd", .instance_size =3D sizeof(BDRVRBDState), .bdrv_parse_filename =3D qemu_rbd_parse_filename, + .bdrv_refresh_limits =3D qemu_rbd_refresh_limits, .bdrv_file_open =3D qemu_rbd_open, .bdrv_close =3D qemu_rbd_close, .bdrv_reopen_prepare =3D qemu_rbd_reopen_prepare, @@ -1170,8 +1174,8 @@ static BlockDriver bdrv_rbd =3D { .bdrv_truncate =3D qemu_rbd_truncate, .protocol_name =3D "rbd", =20 - .bdrv_aio_readv =3D qemu_rbd_aio_readv, - .bdrv_aio_writev =3D qemu_rbd_aio_writev, + .bdrv_aio_preadv =3D qemu_rbd_aio_preadv, + .bdrv_aio_pwritev =3D qemu_rbd_aio_pwritev, =20 #ifdef LIBRBD_SUPPORTS_AIO_FLUSH .bdrv_aio_flush =3D qemu_rbd_aio_flush, --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399395006237.71699689671402; Tue, 15 May 2018 08:49:55 -0700 (PDT) Received: from localhost ([::1]:44456 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcCx-0005C8-4b for importer@patchew.org; Tue, 15 May 2018 11:49:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38601) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4K-0006Xp-QS for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4G-0001If-T3 for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:52 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34222 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4D-0001FJ-R4; Tue, 15 May 2018 11:40:45 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 63577BB40C; Tue, 15 May 2018 15:40:45 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id B6A632026E0E; Tue, 15 May 2018 15:40:44 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:02 +0200 Message-Id: <20180515154033.19899-7-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:45 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:45 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 06/37] vxhs: Switch to byte-based callbacks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Make the change for the last few sector-based callbacks in the vxhs driver. Note that the driver was already using byte-based calls for performing actual I/O, so this just gets rid of a round trip of scaling; however, as I don't know if VxHS is tolerant of non-sector AIO operations, I went with the conservative approach of adding .bdrv_refresh_limits to override the block layer defaults back to the pre-patch value of 512. Signed-off-by: Eric Blake Signed-off-by: Kevin Wolf --- block/vxhs.c | 43 ++++++++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 21 deletions(-) diff --git a/block/vxhs.c b/block/vxhs.c index 55ae1a666e..339e23218d 100644 --- a/block/vxhs.c +++ b/block/vxhs.c @@ -216,6 +216,12 @@ static void vxhs_parse_filename(const char *filename, = QDict *options, } } =20 +static void vxhs_refresh_limits(BlockDriverState *bs, Error **errp) +{ + /* XXX Does VXHS support AIO on less than 512-byte alignment? */ + bs->bl.request_alignment =3D 512; +} + static int vxhs_init_and_ref(void) { if (vxhs_ref++ =3D=3D 0) { @@ -424,21 +430,17 @@ static const AIOCBInfo vxhs_aiocb_info =3D { * and is passed to QNIO. When QNIO completes the work, * it will be passed back through the callback. */ -static BlockAIOCB *vxhs_aio_rw(BlockDriverState *bs, int64_t sector_num, - QEMUIOVector *qiov, int nb_sectors, +static BlockAIOCB *vxhs_aio_rw(BlockDriverState *bs, uint64_t offset, + QEMUIOVector *qiov, uint64_t size, BlockCompletionFunc *cb, void *opaque, VDISKAIOCmd iodir) { VXHSAIOCB *acb =3D NULL; BDRVVXHSState *s =3D bs->opaque; - size_t size; - uint64_t offset; int iio_flags =3D 0; int ret =3D 0; void *dev_handle =3D s->vdisk_hostinfo.dev_handle; =20 - offset =3D sector_num * BDRV_SECTOR_SIZE; - size =3D nb_sectors * BDRV_SECTOR_SIZE; acb =3D qemu_aio_get(&vxhs_aiocb_info, bs, cb, opaque); =20 /* @@ -451,11 +453,11 @@ static BlockAIOCB *vxhs_aio_rw(BlockDriverState *bs, = int64_t sector_num, switch (iodir) { case VDISK_AIO_WRITE: ret =3D iio_writev(dev_handle, acb, qiov->iov, qiov->niov, - offset, (uint64_t)size, iio_flags); + offset, size, iio_flags); break; case VDISK_AIO_READ: ret =3D iio_readv(dev_handle, acb, qiov->iov, qiov->niov, - offset, (uint64_t)size, iio_flags); + offset, size, iio_flags); break; default: trace_vxhs_aio_rw_invalid(iodir); @@ -474,22 +476,20 @@ errout: return NULL; } =20 -static BlockAIOCB *vxhs_aio_readv(BlockDriverState *bs, - int64_t sector_num, QEMUIOVector *qiov, - int nb_sectors, +static BlockAIOCB *vxhs_aio_preadv(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags, BlockCompletionFunc *cb, void *opaque) { - return vxhs_aio_rw(bs, sector_num, qiov, nb_sectors, cb, - opaque, VDISK_AIO_READ); + return vxhs_aio_rw(bs, offset, qiov, bytes, cb, opaque, VDISK_AIO_READ= ); } =20 -static BlockAIOCB *vxhs_aio_writev(BlockDriverState *bs, - int64_t sector_num, QEMUIOVector *qiov, - int nb_sectors, - BlockCompletionFunc *cb, void *opaque) +static BlockAIOCB *vxhs_aio_pwritev(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags, + BlockCompletionFunc *cb, void *opaque) { - return vxhs_aio_rw(bs, sector_num, qiov, nb_sectors, - cb, opaque, VDISK_AIO_WRITE); + return vxhs_aio_rw(bs, offset, qiov, bytes, cb, opaque, VDISK_AIO_WRIT= E); } =20 static void vxhs_close(BlockDriverState *bs) @@ -561,10 +561,11 @@ static BlockDriver bdrv_vxhs =3D { .instance_size =3D sizeof(BDRVVXHSState), .bdrv_file_open =3D vxhs_open, .bdrv_parse_filename =3D vxhs_parse_filename, + .bdrv_refresh_limits =3D vxhs_refresh_limits, .bdrv_close =3D vxhs_close, .bdrv_getlength =3D vxhs_getlength, - .bdrv_aio_readv =3D vxhs_aio_readv, - .bdrv_aio_writev =3D vxhs_aio_writev, + .bdrv_aio_preadv =3D vxhs_aio_preadv, + .bdrv_aio_pwritev =3D vxhs_aio_pwritev, }; =20 static void bdrv_vxhs_init(void) --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399500981347.4743570308739; Tue, 15 May 2018 08:51:40 -0700 (PDT) Received: from localhost ([::1]:44483 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcEm-0006tU-4t for importer@patchew.org; Tue, 15 May 2018 11:51:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38635) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4N-0006bm-Q6 for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4H-0001JM-Ly for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:55 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:39510 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4E-0001GQ-NB; Tue, 15 May 2018 11:40:46 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4C4454201AE0; Tue, 15 May 2018 15:40:46 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9E6A52024CBA; Tue, 15 May 2018 15:40:45 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:03 +0200 Message-Id: <20180515154033.19899-8-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 15 May 2018 15:40:46 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 15 May 2018 15:40:46 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 07/37] block: Drop last of the sector-based aio callbacks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Now that all drivers with aio callbacks are using the byte-based interfaces, we can remove the sector-based versions. Signed-off-by: Eric Blake Signed-off-by: Kevin Wolf --- include/block/block_int.h | 6 ---- block/io.c | 84 ++++++++++++++++++++-----------------------= ---- 2 files changed, 36 insertions(+), 54 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index e772e3502b..0bba7ed024 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -141,15 +141,9 @@ struct BlockDriver { void (*bdrv_refresh_filename)(BlockDriverState *bs, QDict *options); =20 /* aio */ - BlockAIOCB *(*bdrv_aio_readv)(BlockDriverState *bs, - int64_t sector_num, QEMUIOVector *qiov, int nb_sectors, - BlockCompletionFunc *cb, void *opaque); BlockAIOCB *(*bdrv_aio_preadv)(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags, BlockCompletionFunc *cb, void *opaque); - BlockAIOCB *(*bdrv_aio_writev)(BlockDriverState *bs, - int64_t sector_num, QEMUIOVector *qiov, int nb_sectors, - BlockCompletionFunc *cb, void *opaque); BlockAIOCB *(*bdrv_aio_pwritev)(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags, BlockCompletionFunc *cb, void *opaque); diff --git a/block/io.c b/block/io.c index 407bc25df4..6b110b207a 100644 --- a/block/io.c +++ b/block/io.c @@ -925,31 +925,14 @@ static int coroutine_fn bdrv_driver_preadv(BlockDrive= rState *bs, return drv->bdrv_co_preadv(bs, offset, bytes, qiov, flags); } =20 - /* FIXME - no need to calculate these if .bdrv_aio_preadv exists */ - sector_num =3D offset >> BDRV_SECTOR_BITS; - nb_sectors =3D bytes >> BDRV_SECTOR_BITS; - - if (!drv->bdrv_aio_preadv) { - assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - assert((bytes >> BDRV_SECTOR_BITS) <=3D BDRV_REQUEST_MAX_SECTORS); - } - - if (drv->bdrv_co_readv) { - return drv->bdrv_co_readv(bs, sector_num, nb_sectors, qiov); - } else { + if (drv->bdrv_aio_preadv) { BlockAIOCB *acb; CoroutineIOCompletion co =3D { .coroutine =3D qemu_coroutine_self(), }; =20 - if (drv->bdrv_aio_preadv) { - acb =3D drv->bdrv_aio_preadv(bs, offset, bytes, qiov, flags, - bdrv_co_io_em_complete, &co); - } else { - acb =3D drv->bdrv_aio_readv(bs, sector_num, qiov, nb_sectors, - bdrv_co_io_em_complete, &co); - } + acb =3D drv->bdrv_aio_preadv(bs, offset, bytes, qiov, flags, + bdrv_co_io_em_complete, &co); if (acb =3D=3D NULL) { return -EIO; } else { @@ -957,6 +940,16 @@ static int coroutine_fn bdrv_driver_preadv(BlockDriver= State *bs, return co.ret; } } + + sector_num =3D offset >> BDRV_SECTOR_BITS; + nb_sectors =3D bytes >> BDRV_SECTOR_BITS; + + assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); + assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); + assert((bytes >> BDRV_SECTOR_BITS) <=3D BDRV_REQUEST_MAX_SECTORS); + assert(drv->bdrv_co_readv); + + return drv->bdrv_co_readv(bs, sector_num, nb_sectors, qiov); } =20 static int coroutine_fn bdrv_driver_pwritev(BlockDriverState *bs, @@ -981,45 +974,40 @@ static int coroutine_fn bdrv_driver_pwritev(BlockDriv= erState *bs, goto emulate_flags; } =20 - /* FIXME - no need to calculate these if .bdrv_aio_pwritev exists */ - sector_num =3D offset >> BDRV_SECTOR_BITS; - nb_sectors =3D bytes >> BDRV_SECTOR_BITS; - - if (!drv->bdrv_aio_pwritev) { - assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); - assert((bytes >> BDRV_SECTOR_BITS) <=3D BDRV_REQUEST_MAX_SECTORS); - } - - if (drv->bdrv_co_writev_flags) { - ret =3D drv->bdrv_co_writev_flags(bs, sector_num, nb_sectors, qiov, - flags & bs->supported_write_flags); - flags &=3D ~bs->supported_write_flags; - } else if (drv->bdrv_co_writev) { - assert(!bs->supported_write_flags); - ret =3D drv->bdrv_co_writev(bs, sector_num, nb_sectors, qiov); - } else { + if (drv->bdrv_aio_pwritev) { BlockAIOCB *acb; CoroutineIOCompletion co =3D { .coroutine =3D qemu_coroutine_self(), }; =20 - if (drv->bdrv_aio_pwritev) { - acb =3D drv->bdrv_aio_pwritev(bs, offset, bytes, qiov, - flags & bs->supported_write_flags, - bdrv_co_io_em_complete, &co); - flags &=3D ~bs->supported_write_flags; - } else { - assert(!bs->supported_write_flags); - acb =3D drv->bdrv_aio_writev(bs, sector_num, qiov, nb_sectors, - bdrv_co_io_em_complete, &co); - } + acb =3D drv->bdrv_aio_pwritev(bs, offset, bytes, qiov, + flags & bs->supported_write_flags, + bdrv_co_io_em_complete, &co); + flags &=3D ~bs->supported_write_flags; if (acb =3D=3D NULL) { ret =3D -EIO; } else { qemu_coroutine_yield(); ret =3D co.ret; } + goto emulate_flags; + } + + sector_num =3D offset >> BDRV_SECTOR_BITS; + nb_sectors =3D bytes >> BDRV_SECTOR_BITS; + + assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); + assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); + assert((bytes >> BDRV_SECTOR_BITS) <=3D BDRV_REQUEST_MAX_SECTORS); + + if (drv->bdrv_co_writev_flags) { + ret =3D drv->bdrv_co_writev_flags(bs, sector_num, nb_sectors, qiov, + flags & bs->supported_write_flags); + flags &=3D ~bs->supported_write_flags; + } else { + assert(drv->bdrv_co_writev); + assert(!bs->supported_write_flags); + ret =3D drv->bdrv_co_writev(bs, sector_num, nb_sectors, qiov); } =20 emulate_flags: --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399928279390.02812614320555; Tue, 15 May 2018 08:58:48 -0700 (PDT) Received: from localhost ([::1]:44525 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcLf-0004ya-76 for importer@patchew.org; Tue, 15 May 2018 11:58:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38644) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4O-0006cP-Ci for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4J-0001Ld-Dr for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:56 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:39512 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4F-0001HB-LJ; Tue, 15 May 2018 11:40:47 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 342BE4201AE0; Tue, 15 May 2018 15:40:47 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 86A3B2024CBA; Tue, 15 May 2018 15:40:46 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:04 +0200 Message-Id: <20180515154033.19899-9-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 15 May 2018 15:40:47 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 15 May 2018 15:40:47 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 08/37] block: Merge .bdrv_co_writev{, _flags} in drivers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Eric Blake We have too many driver callback interfaces; simplify the mess somewhat by merging the flags parameter of .bdrv_co_writev_flags() into .bdrv_co_writev(). Note that as long as a driver doesn't set .supported_write_flags, the flags argument will be 0 and behavior is identical. Also note that the public function bdrv_co_writev() still lacks a flags argument; so the driver signature is thus intentionally slightly different. But that's not the end of the world, nor the first time that the driver interface differs slightly from the public interface. Ideally, we should be rewriting all of these drivers to use modern byte-based interfaces. But that's a more invasive patch to write and audit, compared to the simplification done here. Signed-off-by: Eric Blake Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: Kevin Wolf --- include/block/block_int.h | 2 -- block/gluster.c | 4 +++- block/io.c | 13 ++++--------- block/iscsi.c | 8 ++++---- block/parallels.c | 4 +++- block/qcow.c | 6 ++++-- block/qed.c | 3 ++- block/replication.c | 4 +++- block/sheepdog.c | 4 +++- block/ssh.c | 4 +++- block/vhdx.c | 4 +++- 11 files changed, 32 insertions(+), 24 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index 0bba7ed024..e3d6219f4e 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -174,8 +174,6 @@ struct BlockDriver { int coroutine_fn (*bdrv_co_preadv)(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags); int coroutine_fn (*bdrv_co_writev)(BlockDriverState *bs, - int64_t sector_num, int nb_sectors, QEMUIOVector *qiov); - int coroutine_fn (*bdrv_co_writev_flags)(BlockDriverState *bs, int64_t sector_num, int nb_sectors, QEMUIOVector *qiov, int flags); /** * @offset: position in bytes to write at diff --git a/block/gluster.c b/block/gluster.c index 55be566f6d..9900b6420c 100644 --- a/block/gluster.c +++ b/block/gluster.c @@ -1194,8 +1194,10 @@ static coroutine_fn int qemu_gluster_co_readv(BlockD= riverState *bs, static coroutine_fn int qemu_gluster_co_writev(BlockDriverState *bs, int64_t sector_num, int nb_sectors, - QEMUIOVector *qiov) + QEMUIOVector *qiov, + int flags) { + assert(!flags); return qemu_gluster_co_rw(bs, sector_num, nb_sectors, qiov, 1); } =20 diff --git a/block/io.c b/block/io.c index 6b110b207a..4fad5ac2fe 100644 --- a/block/io.c +++ b/block/io.c @@ -1000,15 +1000,10 @@ static int coroutine_fn bdrv_driver_pwritev(BlockDr= iverState *bs, assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); assert((bytes >> BDRV_SECTOR_BITS) <=3D BDRV_REQUEST_MAX_SECTORS); =20 - if (drv->bdrv_co_writev_flags) { - ret =3D drv->bdrv_co_writev_flags(bs, sector_num, nb_sectors, qiov, - flags & bs->supported_write_flags); - flags &=3D ~bs->supported_write_flags; - } else { - assert(drv->bdrv_co_writev); - assert(!bs->supported_write_flags); - ret =3D drv->bdrv_co_writev(bs, sector_num, nb_sectors, qiov); - } + assert(drv->bdrv_co_writev); + ret =3D drv->bdrv_co_writev(bs, sector_num, nb_sectors, qiov, + flags & bs->supported_write_flags); + flags &=3D ~bs->supported_write_flags; =20 emulate_flags: if (ret =3D=3D 0 && (flags & BDRV_REQ_FUA)) { diff --git a/block/iscsi.c b/block/iscsi.c index d19ae0e398..3fd7203916 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -556,8 +556,8 @@ static inline bool iscsi_allocmap_is_valid(IscsiLun *is= csilun, } =20 static int coroutine_fn -iscsi_co_writev_flags(BlockDriverState *bs, int64_t sector_num, int nb_sec= tors, - QEMUIOVector *iov, int flags) +iscsi_co_writev(BlockDriverState *bs, int64_t sector_num, int nb_sectors, + QEMUIOVector *iov, int flags) { IscsiLun *iscsilun =3D bs->opaque; struct IscsiTask iTask; @@ -2220,7 +2220,7 @@ static BlockDriver bdrv_iscsi =3D { .bdrv_co_pdiscard =3D iscsi_co_pdiscard, .bdrv_co_pwrite_zeroes =3D iscsi_co_pwrite_zeroes, .bdrv_co_readv =3D iscsi_co_readv, - .bdrv_co_writev_flags =3D iscsi_co_writev_flags, + .bdrv_co_writev =3D iscsi_co_writev, .bdrv_co_flush_to_disk =3D iscsi_co_flush, =20 #ifdef __linux__ @@ -2255,7 +2255,7 @@ static BlockDriver bdrv_iser =3D { .bdrv_co_pdiscard =3D iscsi_co_pdiscard, .bdrv_co_pwrite_zeroes =3D iscsi_co_pwrite_zeroes, .bdrv_co_readv =3D iscsi_co_readv, - .bdrv_co_writev_flags =3D iscsi_co_writev_flags, + .bdrv_co_writev =3D iscsi_co_writev, .bdrv_co_flush_to_disk =3D iscsi_co_flush, =20 #ifdef __linux__ diff --git a/block/parallels.c b/block/parallels.c index 045810d00f..6e9c37f44e 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -311,13 +311,15 @@ static int coroutine_fn parallels_co_block_status(Blo= ckDriverState *bs, } =20 static coroutine_fn int parallels_co_writev(BlockDriverState *bs, - int64_t sector_num, int nb_sectors, QEMUIOVector *qiov) + int64_t sector_num, int nb_sec= tors, + QEMUIOVector *qiov, int flags) { BDRVParallelsState *s =3D bs->opaque; uint64_t bytes_done =3D 0; QEMUIOVector hd_qiov; int ret =3D 0; =20 + assert(!flags); qemu_iovec_init(&hd_qiov, qiov->niov); =20 while (nb_sectors > 0) { diff --git a/block/qcow.c b/block/qcow.c index 4b2f7db74c..3ba2ca25ea 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -720,7 +720,8 @@ static coroutine_fn int qcow_co_readv(BlockDriverState = *bs, int64_t sector_num, } =20 static coroutine_fn int qcow_co_writev(BlockDriverState *bs, int64_t secto= r_num, - int nb_sectors, QEMUIOVector *qiov) + int nb_sectors, QEMUIOVector *qiov, + int flags) { BDRVQcowState *s =3D bs->opaque; int index_in_cluster; @@ -731,6 +732,7 @@ static coroutine_fn int qcow_co_writev(BlockDriverState= *bs, int64_t sector_num, uint8_t *buf; void *orig_buf; =20 + assert(!flags); s->cluster_cache_offset =3D -1; /* disable compressed cache */ =20 /* We must always copy the iov when encrypting, so we @@ -1110,7 +1112,7 @@ qcow_co_pwritev_compressed(BlockDriverState *bs, uint= 64_t offset, if (ret !=3D Z_STREAM_END || out_len >=3D s->cluster_size) { /* could not compress: write normal cluster */ ret =3D qcow_co_writev(bs, offset >> BDRV_SECTOR_BITS, - bytes >> BDRV_SECTOR_BITS, qiov); + bytes >> BDRV_SECTOR_BITS, qiov, 0); if (ret < 0) { goto fail; } diff --git a/block/qed.c b/block/qed.c index 1db8eaf241..65cfe92393 100644 --- a/block/qed.c +++ b/block/qed.c @@ -1437,8 +1437,9 @@ static int coroutine_fn bdrv_qed_co_readv(BlockDriver= State *bs, =20 static int coroutine_fn bdrv_qed_co_writev(BlockDriverState *bs, int64_t sector_num, int nb_sect= ors, - QEMUIOVector *qiov) + QEMUIOVector *qiov, int flags) { + assert(!flags); return qed_co_request(bs, sector_num, qiov, nb_sectors, QED_AIOCB_WRIT= E); } =20 diff --git a/block/replication.c b/block/replication.c index 6c0c7186d9..48148b884a 100644 --- a/block/replication.c +++ b/block/replication.c @@ -260,7 +260,8 @@ out: static coroutine_fn int replication_co_writev(BlockDriverState *bs, int64_t sector_num, int remaining_sectors, - QEMUIOVector *qiov) + QEMUIOVector *qiov, + int flags) { BDRVReplicationState *s =3D bs->opaque; QEMUIOVector hd_qiov; @@ -271,6 +272,7 @@ static coroutine_fn int replication_co_writev(BlockDriv= erState *bs, int ret; int64_t n; =20 + assert(!flags); ret =3D replication_get_io_status(s); if (ret < 0) { goto out; diff --git a/block/sheepdog.c b/block/sheepdog.c index fed2a04797..4237132419 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -2614,13 +2614,15 @@ static void sd_aio_complete(SheepdogAIOCB *acb) } =20 static coroutine_fn int sd_co_writev(BlockDriverState *bs, int64_t sector_= num, - int nb_sectors, QEMUIOVector *qiov) + int nb_sectors, QEMUIOVector *qiov, + int flags) { SheepdogAIOCB acb; int ret; int64_t offset =3D (sector_num + nb_sectors) * BDRV_SECTOR_SIZE; BDRVSheepdogState *s =3D bs->opaque; =20 + assert(!flags); if (offset > s->inode.vdi_size) { ret =3D sd_truncate(bs, offset, PREALLOC_MODE_OFF, NULL); if (ret < 0) { diff --git a/block/ssh.c b/block/ssh.c index 412a1bfc17..4c4fa3ccfc 100644 --- a/block/ssh.c +++ b/block/ssh.c @@ -1164,11 +1164,13 @@ static int ssh_write(BDRVSSHState *s, BlockDriverSt= ate *bs, =20 static coroutine_fn int ssh_co_writev(BlockDriverState *bs, int64_t sector_num, - int nb_sectors, QEMUIOVector *qiov) + int nb_sectors, QEMUIOVector *qiov, + int flags) { BDRVSSHState *s =3D bs->opaque; int ret; =20 + assert(!flags); qemu_co_mutex_lock(&s->lock); ret =3D ssh_write(s, bs, sector_num * BDRV_SECTOR_SIZE, nb_sectors * BDRV_SECTOR_SIZE, qiov); diff --git a/block/vhdx.c b/block/vhdx.c index c3a4220a35..0b1e21c750 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -1226,7 +1226,8 @@ int vhdx_user_visible_write(BlockDriverState *bs, BDR= VVHDXState *s) } =20 static coroutine_fn int vhdx_co_writev(BlockDriverState *bs, int64_t secto= r_num, - int nb_sectors, QEMUIOVector *qiov) + int nb_sectors, QEMUIOVector *qiov, + int flags) { int ret =3D -ENOTSUP; BDRVVHDXState *s =3D bs->opaque; @@ -1242,6 +1243,7 @@ static coroutine_fn int vhdx_co_writev(BlockDriverSta= te *bs, int64_t sector_num, uint64_t bat_prior_offset =3D 0; bool bat_update =3D false; =20 + assert(!flags); qemu_iovec_init(&hd_qiov, qiov->niov); =20 qemu_co_mutex_lock(&s->lock); --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526400556002292.23811033095205; Tue, 15 May 2018 09:09:16 -0700 (PDT) Received: from localhost ([::1]:44723 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcVn-00051s-3z for importer@patchew.org; Tue, 15 May 2018 12:09:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38673) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4P-0006dl-Ed for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4J-0001LO-BI for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:57 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:40540 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4G-0001ID-Gt; Tue, 15 May 2018 11:40:48 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1C72681FE142; Tue, 15 May 2018 15:40:48 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6F304200BC1E; Tue, 15 May 2018 15:40:47 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:05 +0200 Message-Id: <20180515154033.19899-10-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 15 May 2018 15:40:48 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 15 May 2018 15:40:48 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 09/37] hmp: Allow using a qdev id in block_set_io_throttle X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia The QMP version of this command can take a qdev ID since 7a9877a02635, but the HMP version is still using the deprecated block device name so there's no way to refer to a block device added like this: -blockdev node-name=3Ddisk0,driver=3Dqcow2,file.driver=3Dfile,file.filena= me=3Dhd.qcow2 -device virtio-blk-pci,id=3Dvirtio-blk-pci0,drive=3Ddisk0 This patch works around this problem by using the specified name as a qdev ID if the block device name is not found. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- hmp.c | 14 ++++++++++++-- hmp-commands.hx | 3 ++- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/hmp.c b/hmp.c index 898e25f3e1..bdb340605c 100644 --- a/hmp.c +++ b/hmp.c @@ -1789,9 +1789,8 @@ void hmp_change(Monitor *mon, const QDict *qdict) void hmp_block_set_io_throttle(Monitor *mon, const QDict *qdict) { Error *err =3D NULL; + char *device =3D (char *) qdict_get_str(qdict, "device"); BlockIOThrottle throttle =3D { - .has_device =3D true, - .device =3D (char *) qdict_get_str(qdict, "device"), .bps =3D qdict_get_int(qdict, "bps"), .bps_rd =3D qdict_get_int(qdict, "bps_rd"), .bps_wr =3D qdict_get_int(qdict, "bps_wr"), @@ -1800,6 +1799,17 @@ void hmp_block_set_io_throttle(Monitor *mon, const Q= Dict *qdict) .iops_wr =3D qdict_get_int(qdict, "iops_wr"), }; =20 + /* qmp_block_set_io_throttle has separate parameters for the + * (deprecated) block device name and the qdev ID but the HMP + * version has only one, so we must decide which one to pass. */ + if (blk_by_name(device)) { + throttle.has_device =3D true; + throttle.device =3D device; + } else { + throttle.has_id =3D true; + throttle.id =3D device; + } + qmp_block_set_io_throttle(&throttle, &err); hmp_handle_error(mon, &err); } diff --git a/hmp-commands.hx b/hmp-commands.hx index 35d862a5d2..227f7eee88 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -1647,7 +1647,8 @@ ETEXI STEXI @item block_set_io_throttle @var{device} @var{bps} @var{bps_rd} @var{bps_w= r} @var{iops} @var{iops_rd} @var{iops_wr} @findex block_set_io_throttle -Change I/O throttle limits for a block drive to @var{bps} @var{bps_rd} @va= r{bps_wr} @var{iops} @var{iops_rd} @var{iops_wr} +Change I/O throttle limits for a block drive to @var{bps} @var{bps_rd} @va= r{bps_wr} @var{iops} @var{iops_rd} @var{iops_wr}. +@var{device} can be a block device name, a qdev ID or a QOM path. ETEXI =20 { --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399758578719.2753109023848; Tue, 15 May 2018 08:55:58 -0700 (PDT) Received: from localhost ([::1]:44510 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcIr-000212-9Y for importer@patchew.org; Tue, 15 May 2018 11:55:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38638) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4O-0006c5-5U for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4J-0001MB-Pi for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:56 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34236 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4H-0001It-Do; Tue, 15 May 2018 11:40:49 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 050C4BB409; Tue, 15 May 2018 15:40:49 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 57E832024CBB; Tue, 15 May 2018 15:40:48 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:06 +0200 Message-Id: <20180515154033.19899-11-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:49 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:49 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 10/37] blockjob: expose error string via query X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow When we've reached the concluded state, we need to expose the error state if applicable. Add the new field. This should be sufficient for determining if a job completed successfully or not after concluding; if we want to discriminate based on how it failed more mechanically, we can always add an explicit return code enumeration later. I didn't bother to make it only show up if we are in the concluded state; I don't think it's necessary. Cc: qemu-stable@nongnu.org Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia Signed-off-by: Kevin Wolf --- qapi/block-core.json | 6 +++++- blockjob.c | 2 ++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 21c3470234..17ffd44cce 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1172,6 +1172,9 @@ # @auto-dismiss: Job will dismiss itself when CONCLUDED, moving to the NULL # state and disappearing from the query list. (since 2.12) # +# @error: Error information if the job did not complete successfully. +# Not set if the job completed successfully. (since 2.12.1) +# # Since: 1.1 ## { 'struct': 'BlockJobInfo', @@ -1179,7 +1182,8 @@ 'offset': 'int', 'busy': 'bool', 'paused': 'bool', 'speed': 'in= t', 'io-status': 'BlockDeviceIoStatus', 'ready': 'bool', 'status': 'BlockJobStatus', - 'auto-finalize': 'bool', 'auto-dismiss': 'bool' } } + 'auto-finalize': 'bool', 'auto-dismiss': 'bool', + '*error': 'str' } } =20 ## # @query-block-jobs: diff --git a/blockjob.c b/blockjob.c index dfffad921a..0b9b336d19 100644 --- a/blockjob.c +++ b/blockjob.c @@ -831,6 +831,8 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **er= rp) info->status =3D job->status; info->auto_finalize =3D job->auto_finalize; info->auto_dismiss =3D job->auto_dismiss; + info->has_error =3D job->ret !=3D 0; + info->error =3D job->ret ? g_strdup(strerror(-job->ret)) : NULL; return info; } =20 --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399757564416.53239892862985; Tue, 15 May 2018 08:55:57 -0700 (PDT) Received: from localhost ([::1]:44513 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcIu-000244-Pd for importer@patchew.org; Tue, 15 May 2018 11:55:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38729) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4Q-0006fs-Qw for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4P-0001SN-KS for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:58 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47562 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4I-0001Jm-Ak; Tue, 15 May 2018 11:40:50 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E23EE7D843; Tue, 15 May 2018 15:40:49 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 408B7200BC1E; Tue, 15 May 2018 15:40:49 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:07 +0200 Message-Id: <20180515154033.19899-12-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:40:49 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:40:49 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 11/37] blockjob: Fix assertion in block_job_finalize() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Every job gets a non-NULL job->txn on creation, but it doesn't necessarily keep it until it is decommissioned: Finalising a job removes it from its transaction. Therefore, calling 'blockdev-job-finalize' a second time on an already concluded job causes an assertion failure. Remove job->txn from the assertion in block_job_finalize() to fix this. block_job_do_finalize() still has the same assertion, but if a job is already removed from its transaction, block_job_apply_verb() will already error out before we run into that assertion. Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: John Snow --- blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blockjob.c b/blockjob.c index 0b9b336d19..0ebc5ab178 100644 --- a/blockjob.c +++ b/blockjob.c @@ -702,7 +702,7 @@ void block_job_complete(BlockJob *job, Error **errp) =20 void block_job_finalize(BlockJob *job, Error **errp) { - assert(job && job->id && job->txn); + assert(job && job->id); if (block_job_apply_verb(job, BLOCK_JOB_VERB_FINALIZE, errp)) { return; } --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 152640034774049.99773046576456; Tue, 15 May 2018 09:05:47 -0700 (PDT) Received: from localhost ([::1]:44626 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcSQ-0002A5-Sf for importer@patchew.org; Tue, 15 May 2018 12:05:46 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38807) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4S-0006iH-PL for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4Q-0001Ta-Mu for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:00 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34238 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4J-0001KV-96; Tue, 15 May 2018 11:40:51 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CE4BFBB409; Tue, 15 May 2018 15:40:50 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 292752026E0E; Tue, 15 May 2018 15:40:50 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:08 +0200 Message-Id: <20180515154033.19899-13-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:50 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:50 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 12/37] blockjob: Wrappers for progress counter access X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Block job drivers are not expected to mess with the internals of the BlockJob object, so provide wrapper functions for one of the cases where they still do it: Updating the progress counter. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/block/blockjob.h | 19 +++++++++++++++++++ block/backup.c | 22 +++++++++++++--------- block/commit.c | 16 ++++++++-------- block/mirror.c | 11 +++++------ block/stream.c | 14 ++++++++------ blockjob.c | 10 ++++++++++ 6 files changed, 63 insertions(+), 29 deletions(-) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index fc645dac68..a2cc52233b 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -278,6 +278,25 @@ void block_job_finalize(BlockJob *job, Error **errp); void block_job_dismiss(BlockJob **job, Error **errp); =20 /** + * block_job_progress_update: + * @job: The job that has made progress + * @done: How much progress the job made + * + * Updates the progress counter of the job. + */ +void block_job_progress_update(BlockJob *job, uint64_t done); + +/** + * block_job_progress_set_remaining: + * @job: The job whose expected progress end value is set + * @remaining: Expected end value of the progress counter of the job + * + * Sets the expected end value of the progress counter of a job so that a + * completion percentage can be calculated when the progress is updated. + */ +void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining); + +/** * block_job_query: * @job: The job to get information about. * diff --git a/block/backup.c b/block/backup.c index 453cd62c24..5d95805472 100644 --- a/block/backup.c +++ b/block/backup.c @@ -39,6 +39,7 @@ typedef struct BackupBlockJob { BlockdevOnError on_source_error; BlockdevOnError on_target_error; CoRwlock flush_rwlock; + uint64_t len; uint64_t bytes_read; int64_t cluster_size; bool compress; @@ -118,7 +119,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *j= ob, =20 trace_backup_do_cow_process(job, start); =20 - n =3D MIN(job->cluster_size, job->common.len - start); + n =3D MIN(job->cluster_size, job->len - start); =20 if (!bounce_buffer) { bounce_buffer =3D blk_blockalign(blk, job->cluster_size); @@ -159,7 +160,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *j= ob, * offset field is an opaque progress value, it is not a disk offs= et. */ job->bytes_read +=3D n; - job->common.offset +=3D n; + block_job_progress_update(&job->common, n); } =20 out: @@ -261,7 +262,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp) return; } =20 - len =3D DIV_ROUND_UP(backup_job->common.len, backup_job->cluster_size); + len =3D DIV_ROUND_UP(backup_job->len, backup_job->cluster_size); hbitmap_set(backup_job->copy_bitmap, 0, len); } =20 @@ -420,8 +421,9 @@ static void backup_incremental_init_copy_bitmap(BackupB= lockJob *job) bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size); } =20 - job->common.offset =3D job->common.len - - hbitmap_count(job->copy_bitmap) * job->cluster_si= ze; + /* TODO block_job_progress_set_remaining() would make more sense */ + block_job_progress_update(&job->common, + job->len - hbitmap_count(job->copy_bitmap) * job->cluster_size); =20 bdrv_dirty_iter_free(dbi); } @@ -437,7 +439,9 @@ static void coroutine_fn backup_run(void *opaque) QLIST_INIT(&job->inflight_reqs); qemu_co_rwlock_init(&job->flush_rwlock); =20 - nb_clusters =3D DIV_ROUND_UP(job->common.len, job->cluster_size); + nb_clusters =3D DIV_ROUND_UP(job->len, job->cluster_size); + block_job_progress_set_remaining(&job->common, job->len); + job->copy_bitmap =3D hbitmap_alloc(nb_clusters, 0); if (job->sync_mode =3D=3D MIRROR_SYNC_MODE_INCREMENTAL) { backup_incremental_init_copy_bitmap(job); @@ -461,7 +465,7 @@ static void coroutine_fn backup_run(void *opaque) ret =3D backup_run_incremental(job); } else { /* Both FULL and TOP SYNC_MODE's require copying.. */ - for (offset =3D 0; offset < job->common.len; + for (offset =3D 0; offset < job->len; offset +=3D job->cluster_size) { bool error_is_read; int alloced =3D 0; @@ -620,7 +624,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, goto error; } =20 - /* job->common.len is fixed, so we can't allow resize */ + /* job->len is fixed, so we can't allow resize */ job =3D block_job_create(job_id, &backup_job_driver, txn, bs, BLK_PERM_CONSISTENT_READ, BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE | @@ -676,7 +680,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, /* Required permissions are already taken with target's blk_new() */ block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, &error_abort); - job->common.len =3D len; + job->len =3D len; =20 return &job->common; =20 diff --git a/block/commit.c b/block/commit.c index 1432baeef4..50b191c980 100644 --- a/block/commit.c +++ b/block/commit.c @@ -146,21 +146,21 @@ static void coroutine_fn commit_run(void *opaque) int64_t n =3D 0; /* bytes */ void *buf =3D NULL; int bytes_written =3D 0; - int64_t base_len; + int64_t len, base_len; =20 - ret =3D s->common.len =3D blk_getlength(s->top); - - if (s->common.len < 0) { + ret =3D len =3D blk_getlength(s->top); + if (len < 0) { goto out; } + block_job_progress_set_remaining(&s->common, len); =20 ret =3D base_len =3D blk_getlength(s->base); if (base_len < 0) { goto out; } =20 - if (base_len < s->common.len) { - ret =3D blk_truncate(s->base, s->common.len, PREALLOC_MODE_OFF, NU= LL); + if (base_len < len) { + ret =3D blk_truncate(s->base, len, PREALLOC_MODE_OFF, NULL); if (ret) { goto out; } @@ -168,7 +168,7 @@ static void coroutine_fn commit_run(void *opaque) =20 buf =3D blk_blockalign(s->top, COMMIT_BUFFER_SIZE); =20 - for (offset =3D 0; offset < s->common.len; offset +=3D n) { + for (offset =3D 0; offset < len; offset +=3D n) { bool copy; =20 /* Note that even when no rate limit is applied we need to yield @@ -198,7 +198,7 @@ static void coroutine_fn commit_run(void *opaque) } } /* Publish progress */ - s->common.offset +=3D n; + block_job_progress_update(&s->common, n); =20 if (copy && s->common.speed) { delay_ns =3D ratelimit_calculate_delay(&s->limit, n); diff --git a/block/mirror.c b/block/mirror.c index 99da9c0858..56a7ce2f55 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -121,7 +121,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret) bitmap_set(s->cow_bitmap, chunk_num, nb_chunks); } if (!s->initial_zeroing_ongoing) { - s->common.offset +=3D op->bytes; + block_job_progress_update(&s->common, op->bytes); } } qemu_iovec_destroy(&op->qiov); @@ -792,11 +792,10 @@ static void coroutine_fn mirror_run(void *opaque) block_job_pause_point(&s->common); =20 cnt =3D bdrv_get_dirty_count(s->dirty_bitmap); - /* s->common.offset contains the number of bytes already processed= so - * far, cnt is the number of dirty bytes remaining and - * s->bytes_in_flight is the number of bytes currently being - * processed; together those are the current total operation lengt= h */ - s->common.len =3D s->common.offset + s->bytes_in_flight + cnt; + /* cnt is the number of dirty bytes remaining and s->bytes_in_flig= ht is + * the number of bytes currently being processed; together those a= re + * the current remaining operation length */ + block_job_progress_set_remaining(&s->common, s->bytes_in_flight + = cnt); =20 /* Note that even when no rate limit is applied we need to yield * periodically with no pending I/O so that bdrv_drain_all() retur= ns. diff --git a/block/stream.c b/block/stream.c index 1a85708fcf..8369852bda 100644 --- a/block/stream.c +++ b/block/stream.c @@ -107,6 +107,7 @@ static void coroutine_fn stream_run(void *opaque) BlockBackend *blk =3D s->common.blk; BlockDriverState *bs =3D blk_bs(blk); BlockDriverState *base =3D s->base; + int64_t len; int64_t offset =3D 0; uint64_t delay_ns =3D 0; int error =3D 0; @@ -118,11 +119,12 @@ static void coroutine_fn stream_run(void *opaque) goto out; } =20 - s->common.len =3D bdrv_getlength(bs); - if (s->common.len < 0) { - ret =3D s->common.len; + len =3D bdrv_getlength(bs); + if (len < 0) { + ret =3D len; goto out; } + block_job_progress_set_remaining(&s->common, len); =20 buf =3D qemu_blockalign(bs, STREAM_BUFFER_SIZE); =20 @@ -135,7 +137,7 @@ static void coroutine_fn stream_run(void *opaque) bdrv_enable_copy_on_read(bs); } =20 - for ( ; offset < s->common.len; offset +=3D n) { + for ( ; offset < len; offset +=3D n) { bool copy; =20 /* Note that even when no rate limit is applied we need to yield @@ -159,7 +161,7 @@ static void coroutine_fn stream_run(void *opaque) =20 /* Finish early if end of backing file has been reached */ if (ret =3D=3D 0 && n =3D=3D 0) { - n =3D s->common.len - offset; + n =3D len - offset; } =20 copy =3D (ret =3D=3D 1); @@ -185,7 +187,7 @@ static void coroutine_fn stream_run(void *opaque) ret =3D 0; =20 /* Publish progress */ - s->common.offset +=3D n; + block_job_progress_update(&s->common, n); if (copy && s->common.speed) { delay_ns =3D ratelimit_calculate_delay(&s->limit, n); } else { diff --git a/blockjob.c b/blockjob.c index 0ebc5ab178..ebc26a5245 100644 --- a/blockjob.c +++ b/blockjob.c @@ -810,6 +810,16 @@ int block_job_complete_sync(BlockJob *job, Error **err= p) return block_job_finish_sync(job, &block_job_complete, errp); } =20 +void block_job_progress_update(BlockJob *job, uint64_t done) +{ + job->offset +=3D done; +} + +void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining) +{ + job->len =3D job->offset + remaining; +} + BlockJobInfo *block_job_query(BlockJob *job, Error **errp) { BlockJobInfo *info; --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526400142676947.2573369389373; Tue, 15 May 2018 09:02:22 -0700 (PDT) Received: from localhost ([::1]:44559 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcP7-0007v5-Pr for importer@patchew.org; Tue, 15 May 2018 12:02:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38768) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4R-0006gy-VF for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4Q-0001T7-BJ for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:59 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:40542 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4K-0001Me-7u; Tue, 15 May 2018 11:40:52 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B2AFF81536A0; Tue, 15 May 2018 15:40:51 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 11D072024CBA; Tue, 15 May 2018 15:40:50 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:09 +0200 Message-Id: <20180515154033.19899-14-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 15 May 2018 15:40:51 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 15 May 2018 15:40:51 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 13/37] blockjob: Move RateLimit to BlockJob X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Every block job has a RateLimit, and they all do the exact same thing with it, so it should be common infrastructure. Move the struct field for a start. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/block/blockjob.h | 4 ++++ block/backup.c | 5 ++--- block/commit.c | 5 ++--- block/mirror.c | 6 +++--- block/stream.c | 5 ++--- 5 files changed, 13 insertions(+), 12 deletions(-) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index a2cc52233b..22bf418209 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -27,6 +27,7 @@ #define BLOCKJOB_H =20 #include "block/block.h" +#include "qemu/ratelimit.h" =20 typedef struct BlockJobDriver BlockJobDriver; typedef struct BlockJobTxn BlockJobTxn; @@ -118,6 +119,9 @@ typedef struct BlockJob { /** Speed that was set with @block_job_set_speed. */ int64_t speed; =20 + /** Rate limiting data structure for implementing @speed. */ + RateLimit limit; + /** The completion function that will be called when the job completes= . */ BlockCompletionFunc *cb; =20 diff --git a/block/backup.c b/block/backup.c index 5d95805472..7585c4391e 100644 --- a/block/backup.c +++ b/block/backup.c @@ -35,7 +35,6 @@ typedef struct BackupBlockJob { /* bitmap for sync=3Dincremental */ BdrvDirtyBitmap *sync_bitmap; MirrorSyncMode sync_mode; - RateLimit limit; BlockdevOnError on_source_error; BlockdevOnError on_target_error; CoRwlock flush_rwlock; @@ -199,7 +198,7 @@ static void backup_set_speed(BlockJob *job, int64_t spe= ed, Error **errp) error_setg(errp, QERR_INVALID_PARAMETER, "speed"); return; } - ratelimit_set_speed(&s->limit, speed, SLICE_TIME); + ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); } =20 static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) @@ -346,7 +345,7 @@ static bool coroutine_fn yield_and_check(BackupBlockJob= *job) * (without, VM does not reboot) */ if (job->common.speed) { - uint64_t delay_ns =3D ratelimit_calculate_delay(&job->limit, + uint64_t delay_ns =3D ratelimit_calculate_delay(&job->common.limit, job->bytes_read); job->bytes_read =3D 0; block_job_sleep_ns(&job->common, delay_ns); diff --git a/block/commit.c b/block/commit.c index 50b191c980..beec5d0ad6 100644 --- a/block/commit.c +++ b/block/commit.c @@ -35,7 +35,6 @@ enum { =20 typedef struct CommitBlockJob { BlockJob common; - RateLimit limit; BlockDriverState *commit_top_bs; BlockBackend *top; BlockBackend *base; @@ -201,7 +200,7 @@ static void coroutine_fn commit_run(void *opaque) block_job_progress_update(&s->common, n); =20 if (copy && s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->limit, n); + delay_ns =3D ratelimit_calculate_delay(&s->common.limit, n); } else { delay_ns =3D 0; } @@ -225,7 +224,7 @@ static void commit_set_speed(BlockJob *job, int64_t spe= ed, Error **errp) error_setg(errp, QERR_INVALID_PARAMETER, "speed"); return; } - ratelimit_set_speed(&s->limit, speed, SLICE_TIME); + ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); } =20 static const BlockJobDriver commit_job_driver =3D { diff --git a/block/mirror.c b/block/mirror.c index 56a7ce2f55..702c139368 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -36,7 +36,6 @@ typedef struct MirrorBuffer { =20 typedef struct MirrorBlockJob { BlockJob common; - RateLimit limit; BlockBackend *target; BlockDriverState *mirror_top_bs; BlockDriverState *source; @@ -450,7 +449,8 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlo= ckJob *s) offset +=3D io_bytes; nb_chunks -=3D DIV_ROUND_UP(io_bytes, s->granularity); if (s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->limit, io_bytes_acc= t); + delay_ns =3D ratelimit_calculate_delay(&s->common.limit, + io_bytes_acct); } } return delay_ns; @@ -916,7 +916,7 @@ static void mirror_set_speed(BlockJob *job, int64_t spe= ed, Error **errp) error_setg(errp, QERR_INVALID_PARAMETER, "speed"); return; } - ratelimit_set_speed(&s->limit, speed, SLICE_TIME); + ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); } =20 static void mirror_complete(BlockJob *job, Error **errp) diff --git a/block/stream.c b/block/stream.c index 8369852bda..a1d4768c2e 100644 --- a/block/stream.c +++ b/block/stream.c @@ -33,7 +33,6 @@ enum { =20 typedef struct StreamBlockJob { BlockJob common; - RateLimit limit; BlockDriverState *base; BlockdevOnError on_error; char *backing_file_str; @@ -189,7 +188,7 @@ static void coroutine_fn stream_run(void *opaque) /* Publish progress */ block_job_progress_update(&s->common, n); if (copy && s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->limit, n); + delay_ns =3D ratelimit_calculate_delay(&s->common.limit, n); } else { delay_ns =3D 0; } @@ -219,7 +218,7 @@ static void stream_set_speed(BlockJob *job, int64_t spe= ed, Error **errp) error_setg(errp, QERR_INVALID_PARAMETER, "speed"); return; } - ratelimit_set_speed(&s->limit, speed, SLICE_TIME); + ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); } =20 static const BlockJobDriver stream_job_driver =3D { --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399937477665.4889622459366; Tue, 15 May 2018 08:58:57 -0700 (PDT) Received: from localhost ([::1]:44527 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcLo-00057u-Il for importer@patchew.org; Tue, 15 May 2018 11:58:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38811) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4S-0006iX-Tc for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4R-0001U4-0A for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:00 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34244 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4L-0001OC-1a; Tue, 15 May 2018 11:40:53 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9B5F4BB409; Tue, 15 May 2018 15:40:52 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id EDE2B200BC1E; Tue, 15 May 2018 15:40:51 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:10 +0200 Message-Id: <20180515154033.19899-15-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:52 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 14/37] blockjob: Implement block_job_set_speed() centrally X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" All block job drivers support .set_speed and all of them duplicate the same code to implement it. Move that code to blockjob.c and remove the now useless callback. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/block/blockjob.h | 2 ++ include/block/blockjob_int.h | 3 --- block/backup.c | 13 ------------- block/commit.c | 14 -------------- block/mirror.c | 26 ++++++-------------------- block/stream.c | 14 -------------- blockjob.c | 12 ++++-------- 7 files changed, 12 insertions(+), 72 deletions(-) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 22bf418209..82f52f4b14 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -29,6 +29,8 @@ #include "block/block.h" #include "qemu/ratelimit.h" =20 +#define BLOCK_JOB_SLICE_TIME 100000000ULL /* ns */ + typedef struct BlockJobDriver BlockJobDriver; typedef struct BlockJobTxn BlockJobTxn; =20 diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h index d5a515de9b..ad510d5a0c 100644 --- a/include/block/blockjob_int.h +++ b/include/block/blockjob_int.h @@ -41,9 +41,6 @@ struct BlockJobDriver { /** String describing the operation, part of query-block-jobs QMP API = */ BlockJobType job_type; =20 - /** Optional callback for job types that support setting a speed limit= */ - void (*set_speed)(BlockJob *job, int64_t speed, Error **errp); - /** Mandatory: Entrypoint for the Coroutine. */ CoroutineEntry *start; =20 diff --git a/block/backup.c b/block/backup.c index 7585c4391e..8468fd9f94 100644 --- a/block/backup.c +++ b/block/backup.c @@ -27,7 +27,6 @@ #include "qemu/error-report.h" =20 #define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16) -#define SLICE_TIME 100000000ULL /* ns */ =20 typedef struct BackupBlockJob { BlockJob common; @@ -190,17 +189,6 @@ static int coroutine_fn backup_before_write_notify( return backup_do_cow(job, req->offset, req->bytes, NULL, true); } =20 -static void backup_set_speed(BlockJob *job, int64_t speed, Error **errp) -{ - BackupBlockJob *s =3D container_of(job, BackupBlockJob, common); - - if (speed < 0) { - error_setg(errp, QERR_INVALID_PARAMETER, "speed"); - return; - } - ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); -} - static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) { BdrvDirtyBitmap *bm; @@ -540,7 +528,6 @@ static const BlockJobDriver backup_job_driver =3D { .instance_size =3D sizeof(BackupBlockJob), .job_type =3D BLOCK_JOB_TYPE_BACKUP, .start =3D backup_run, - .set_speed =3D backup_set_speed, .commit =3D backup_commit, .abort =3D backup_abort, .clean =3D backup_clean, diff --git a/block/commit.c b/block/commit.c index beec5d0ad6..46cbeaec3e 100644 --- a/block/commit.c +++ b/block/commit.c @@ -31,8 +31,6 @@ enum { COMMIT_BUFFER_SIZE =3D 512 * 1024, /* in bytes */ }; =20 -#define SLICE_TIME 100000000ULL /* ns */ - typedef struct CommitBlockJob { BlockJob common; BlockDriverState *commit_top_bs; @@ -216,21 +214,9 @@ out: block_job_defer_to_main_loop(&s->common, commit_complete, data); } =20 -static void commit_set_speed(BlockJob *job, int64_t speed, Error **errp) -{ - CommitBlockJob *s =3D container_of(job, CommitBlockJob, common); - - if (speed < 0) { - error_setg(errp, QERR_INVALID_PARAMETER, "speed"); - return; - } - ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); -} - static const BlockJobDriver commit_job_driver =3D { .instance_size =3D sizeof(CommitBlockJob), .job_type =3D BLOCK_JOB_TYPE_COMMIT, - .set_speed =3D commit_set_speed, .start =3D commit_run, }; =20 diff --git a/block/mirror.c b/block/mirror.c index 702c139368..6955d68804 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -22,7 +22,6 @@ #include "qemu/ratelimit.h" #include "qemu/bitmap.h" =20 -#define SLICE_TIME 100000000ULL /* ns */ #define MAX_IN_FLIGHT 16 #define MAX_IO_BYTES (1 << 20) /* 1 Mb */ #define DEFAULT_MIRROR_BUF_SIZE (MAX_IN_FLIGHT * MAX_IO_BYTES) @@ -596,7 +595,7 @@ static void mirror_throttle(MirrorBlockJob *s) { int64_t now =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); =20 - if (now - s->last_pause_ns > SLICE_TIME) { + if (now - s->last_pause_ns > BLOCK_JOB_SLICE_TIME) { s->last_pause_ns =3D now; block_job_sleep_ns(&s->common, 0); } else { @@ -799,11 +798,10 @@ static void coroutine_fn mirror_run(void *opaque) =20 /* Note that even when no rate limit is applied we need to yield * periodically with no pending I/O so that bdrv_drain_all() retur= ns. - * We do so every SLICE_TIME nanoseconds, or when there is an erro= r, - * or when the source is clean, whichever comes first. - */ + * We do so every BLKOCK_JOB_SLICE_TIME nanoseconds, or when there= is + * an error, or when the source is clean, whichever comes first. */ delta =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - s->last_pause_n= s; - if (delta < SLICE_TIME && + if (delta < BLOCK_JOB_SLICE_TIME && s->common.iostatus =3D=3D BLOCK_DEVICE_IO_STATUS_OK) { if (s->in_flight >=3D MAX_IN_FLIGHT || s->buf_free_count =3D= =3D 0 || (cnt =3D=3D 0 && s->in_flight > 0)) { @@ -869,7 +867,8 @@ static void coroutine_fn mirror_run(void *opaque) ret =3D 0; =20 if (s->synced && !should_complete) { - delay_ns =3D (s->in_flight =3D=3D 0 && cnt =3D=3D 0 ? SLICE_TI= ME : 0); + delay_ns =3D (s->in_flight =3D=3D 0 && + cnt =3D=3D 0 ? BLOCK_JOB_SLICE_TIME : 0); } trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); block_job_sleep_ns(&s->common, delay_ns); @@ -908,17 +907,6 @@ immediate_exit: block_job_defer_to_main_loop(&s->common, mirror_exit, data); } =20 -static void mirror_set_speed(BlockJob *job, int64_t speed, Error **errp) -{ - MirrorBlockJob *s =3D container_of(job, MirrorBlockJob, common); - - if (speed < 0) { - error_setg(errp, QERR_INVALID_PARAMETER, "speed"); - return; - } - ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); -} - static void mirror_complete(BlockJob *job, Error **errp) { MirrorBlockJob *s =3D container_of(job, MirrorBlockJob, common); @@ -1003,7 +991,6 @@ static void mirror_drain(BlockJob *job) static const BlockJobDriver mirror_job_driver =3D { .instance_size =3D sizeof(MirrorBlockJob), .job_type =3D BLOCK_JOB_TYPE_MIRROR, - .set_speed =3D mirror_set_speed, .start =3D mirror_run, .complete =3D mirror_complete, .pause =3D mirror_pause, @@ -1014,7 +1001,6 @@ static const BlockJobDriver mirror_job_driver =3D { static const BlockJobDriver commit_active_job_driver =3D { .instance_size =3D sizeof(MirrorBlockJob), .job_type =3D BLOCK_JOB_TYPE_COMMIT, - .set_speed =3D mirror_set_speed, .start =3D mirror_run, .complete =3D mirror_complete, .pause =3D mirror_pause, diff --git a/block/stream.c b/block/stream.c index a1d4768c2e..797d7c4f21 100644 --- a/block/stream.c +++ b/block/stream.c @@ -29,8 +29,6 @@ enum { STREAM_BUFFER_SIZE =3D 512 * 1024, /* in bytes */ }; =20 -#define SLICE_TIME 100000000ULL /* ns */ - typedef struct StreamBlockJob { BlockJob common; BlockDriverState *base; @@ -210,21 +208,9 @@ out: block_job_defer_to_main_loop(&s->common, stream_complete, data); } =20 -static void stream_set_speed(BlockJob *job, int64_t speed, Error **errp) -{ - StreamBlockJob *s =3D container_of(job, StreamBlockJob, common); - - if (speed < 0) { - error_setg(errp, QERR_INVALID_PARAMETER, "speed"); - return; - } - ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); -} - static const BlockJobDriver stream_job_driver =3D { .instance_size =3D sizeof(StreamBlockJob), .job_type =3D BLOCK_JOB_TYPE_STREAM, - .set_speed =3D stream_set_speed, .start =3D stream_run, }; =20 diff --git a/blockjob.c b/blockjob.c index ebc26a5245..2f4b768338 100644 --- a/blockjob.c +++ b/blockjob.c @@ -659,22 +659,18 @@ static bool block_job_timer_pending(BlockJob *job) =20 void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) { - Error *local_err =3D NULL; int64_t old_speed =3D job->speed; =20 - if (!job->driver->set_speed) { - error_setg(errp, QERR_UNSUPPORTED); - return; - } if (block_job_apply_verb(job, BLOCK_JOB_VERB_SET_SPEED, errp)) { return; } - job->driver->set_speed(job, speed, &local_err); - if (local_err) { - error_propagate(errp, local_err); + if (speed < 0) { + error_setg(errp, QERR_INVALID_PARAMETER, "speed"); return; } =20 + ratelimit_set_speed(&job->limit, speed, BLOCK_JOB_SLICE_TIME); + job->speed =3D speed; if (speed && speed <=3D old_speed) { return; --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399758627516.085111692284; Tue, 15 May 2018 08:55:58 -0700 (PDT) Received: from localhost ([::1]:44512 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcIr-00022j-IA for importer@patchew.org; Tue, 15 May 2018 11:55:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38743) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4R-0006gF-4u for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4P-0001Sh-Sz for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:59 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47564 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4L-0001P8-VL; Tue, 15 May 2018 11:40:54 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 828E17D843; Tue, 15 May 2018 15:40:53 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id D5EA52026E0E; Tue, 15 May 2018 15:40:52 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:11 +0200 Message-Id: <20180515154033.19899-16-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:40:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:40:53 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 15/37] blockjob: Introduce block_job_ratelimit_get_delay() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This gets us rid of more direct accesses to BlockJob fields from the job drivers. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/block/blockjob_int.h | 8 ++++++++ block/backup.c | 18 +++++++----------- block/commit.c | 4 ++-- block/mirror.c | 5 +---- block/stream.c | 4 ++-- blockjob.c | 9 +++++++++ 6 files changed, 29 insertions(+), 19 deletions(-) diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h index ad510d5a0c..62ec964d09 100644 --- a/include/block/blockjob_int.h +++ b/include/block/blockjob_int.h @@ -166,6 +166,14 @@ void block_job_sleep_ns(BlockJob *job, int64_t ns); void block_job_yield(BlockJob *job); =20 /** + * block_job_ratelimit_get_delay: + * + * Calculate and return delay for the next request in ns. See the document= ation + * of ratelimit_calculate_delay() for details. + */ +int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n); + +/** * block_job_early_fail: * @bs: The block device. * diff --git a/block/backup.c b/block/backup.c index 8468fd9f94..cfdb6ecdf5 100644 --- a/block/backup.c +++ b/block/backup.c @@ -325,21 +325,17 @@ static void backup_complete(BlockJob *job, void *opaq= ue) =20 static bool coroutine_fn yield_and_check(BackupBlockJob *job) { + uint64_t delay_ns; + if (block_job_is_cancelled(&job->common)) { return true; } =20 - /* we need to yield so that bdrv_drain_all() returns. - * (without, VM does not reboot) - */ - if (job->common.speed) { - uint64_t delay_ns =3D ratelimit_calculate_delay(&job->common.limit, - job->bytes_read); - job->bytes_read =3D 0; - block_job_sleep_ns(&job->common, delay_ns); - } else { - block_job_sleep_ns(&job->common, 0); - } + /* We need to yield even for delay_ns =3D 0 so that bdrv_drain_all() c= an + * return. Without a yield, the VM would not reboot. */ + delay_ns =3D block_job_ratelimit_get_delay(&job->common, job->bytes_re= ad); + job->bytes_read =3D 0; + block_job_sleep_ns(&job->common, delay_ns); =20 if (block_job_is_cancelled(&job->common)) { return true; diff --git a/block/commit.c b/block/commit.c index 46cbeaec3e..ba5df6aa0a 100644 --- a/block/commit.c +++ b/block/commit.c @@ -197,8 +197,8 @@ static void coroutine_fn commit_run(void *opaque) /* Publish progress */ block_job_progress_update(&s->common, n); =20 - if (copy && s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->common.limit, n); + if (copy) { + delay_ns =3D block_job_ratelimit_get_delay(&s->common, n); } else { delay_ns =3D 0; } diff --git a/block/mirror.c b/block/mirror.c index 6955d68804..6aa38db114 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -447,10 +447,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBl= ockJob *s) assert(io_bytes); offset +=3D io_bytes; nb_chunks -=3D DIV_ROUND_UP(io_bytes, s->granularity); - if (s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->common.limit, - io_bytes_acct); - } + delay_ns =3D block_job_ratelimit_get_delay(&s->common, io_bytes_ac= ct); } return delay_ns; } diff --git a/block/stream.c b/block/stream.c index 797d7c4f21..df9660d2fc 100644 --- a/block/stream.c +++ b/block/stream.c @@ -185,8 +185,8 @@ static void coroutine_fn stream_run(void *opaque) =20 /* Publish progress */ block_job_progress_update(&s->common, n); - if (copy && s->common.speed) { - delay_ns =3D ratelimit_calculate_delay(&s->common.limit, n); + if (copy) { + delay_ns =3D block_job_ratelimit_get_delay(&s->common, n); } else { delay_ns =3D 0; } diff --git a/blockjob.c b/blockjob.c index 2f4b768338..04416f11cd 100644 --- a/blockjob.c +++ b/blockjob.c @@ -680,6 +680,15 @@ void block_job_set_speed(BlockJob *job, int64_t speed,= Error **errp) block_job_enter_cond(job, block_job_timer_pending); } =20 +int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n) +{ + if (!job->speed) { + return 0; + } + + return ratelimit_calculate_delay(&job->limit, n); +} + void block_job_complete(BlockJob *job, Error **errp) { /* Should not be reachable via external interface for internal jobs */ --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399929279721.5927211528382; Tue, 15 May 2018 08:58:49 -0700 (PDT) Received: from localhost ([::1]:44526 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcLg-00050v-HH for importer@patchew.org; Tue, 15 May 2018 11:58:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38751) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4R-0006gV-B5 for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4Q-0001Sz-7G for qemu-devel@nongnu.org; Tue, 15 May 2018 11:40:59 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36646 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4M-0001Pf-S2; Tue, 15 May 2018 11:40:54 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6B67640201A4; Tue, 15 May 2018 15:40:54 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id BE17F2024CBA; Tue, 15 May 2018 15:40:53 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:12 +0200 Message-Id: <20180515154033.19899-17-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:54 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 16/37] blockjob: Add block_job_driver() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The backup block job directly accesses the driver field in BlockJob. Add a wrapper for getting it. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/block/blockjob.h | 7 +++++++ block/backup.c | 8 +++++--- blockjob.c | 5 +++++ 3 files changed, 17 insertions(+), 3 deletions(-) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 82f52f4b14..0f56f723de 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -452,4 +452,11 @@ void block_job_txn_add_job(BlockJobTxn *txn, BlockJob = *job); */ bool block_job_is_internal(BlockJob *job); =20 +/** + * block_job_driver: + * + * Returns the driver associated with a block job. + */ +const BlockJobDriver *block_job_driver(BlockJob *job); + #endif diff --git a/block/backup.c b/block/backup.c index cfdb6ecdf5..e14d99560d 100644 --- a/block/backup.c +++ b/block/backup.c @@ -47,6 +47,8 @@ typedef struct BackupBlockJob { HBitmap *copy_bitmap; } BackupBlockJob; =20 +static const BlockJobDriver backup_job_driver; + /* See if in-flight requests overlap and wait for them to complete */ static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job, int64_t start, @@ -241,7 +243,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp) BackupBlockJob *backup_job =3D container_of(job, BackupBlockJob, commo= n); int64_t len; =20 - assert(job->driver->job_type =3D=3D BLOCK_JOB_TYPE_BACKUP); + assert(block_job_driver(job) =3D=3D &backup_job_driver); =20 if (backup_job->sync_mode !=3D MIRROR_SYNC_MODE_NONE) { error_setg(errp, "The backup job only supports block checkpoint in" @@ -259,7 +261,7 @@ void backup_wait_for_overlapping_requests(BlockJob *job= , int64_t offset, BackupBlockJob *backup_job =3D container_of(job, BackupBlockJob, commo= n); int64_t start, end; =20 - assert(job->driver->job_type =3D=3D BLOCK_JOB_TYPE_BACKUP); + assert(block_job_driver(job) =3D=3D &backup_job_driver); =20 start =3D QEMU_ALIGN_DOWN(offset, backup_job->cluster_size); end =3D QEMU_ALIGN_UP(offset + bytes, backup_job->cluster_size); @@ -272,7 +274,7 @@ void backup_cow_request_begin(CowRequest *req, BlockJob= *job, BackupBlockJob *backup_job =3D container_of(job, BackupBlockJob, commo= n); int64_t start, end; =20 - assert(job->driver->job_type =3D=3D BLOCK_JOB_TYPE_BACKUP); + assert(block_job_driver(job) =3D=3D &backup_job_driver); =20 start =3D QEMU_ALIGN_DOWN(offset, backup_job->cluster_size); end =3D QEMU_ALIGN_UP(offset + bytes, backup_job->cluster_size); diff --git a/blockjob.c b/blockjob.c index 04416f11cd..36c5fdeb2f 100644 --- a/blockjob.c +++ b/blockjob.c @@ -359,6 +359,11 @@ static bool block_job_started(BlockJob *job) return job->co; } =20 +const BlockJobDriver *block_job_driver(BlockJob *job) +{ + return job->driver; +} + /** * All jobs must allow a pause point before entering their job proper. This * ensures that jobs can be paused prior to being started, then resumed la= ter. --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526400753979814.541919476825; Tue, 15 May 2018 09:12:33 -0700 (PDT) Received: from localhost ([::1]:44766 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcYy-0007hE-VH for importer@patchew.org; Tue, 15 May 2018 12:12:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38890) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4V-0006oG-Ib for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4T-0001X0-Eg for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:03 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34250 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4N-0001QC-PJ; Tue, 15 May 2018 11:40:55 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 54428BB40C; Tue, 15 May 2018 15:40:55 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id A68932024CBA; Tue, 15 May 2018 15:40:54 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:13 +0200 Message-Id: <20180515154033.19899-18-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:55 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:55 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 17/37] iotests: Split 214 off of 122 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Commit abd3622cc03cf41ed542126a540385f30a4c0175 added a case to 122 regarding how the qcow2 driver handles an incorrect compressed data length value. This does not really fit into 122, as that file is supposed to contain qemu-img convert test cases, which this case is not. So this patch splits it off into its own file; maybe we will even get more qcow2-only compression tests in the future. Also, that test case does not work with refcount_bits=3D1, so mark that option as unsupported. Signed-off-by: Max Reitz Message-id: 20180406164108.26118-1-mreitz@redhat.com Reviewed-by: Eric Blake Signed-off-by: Alberto Garcia Signed-off-by: Max Reitz --- tests/qemu-iotests/122 | 47 ---------------------- tests/qemu-iotests/122.out | 33 ---------------- tests/qemu-iotests/214 | 97 ++++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/214.out | 35 +++++++++++++++++ tests/qemu-iotests/group | 1 + 5 files changed, 133 insertions(+), 80 deletions(-) create mode 100755 tests/qemu-iotests/214 create mode 100644 tests/qemu-iotests/214.out diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122 index 6cf4fcb866..45b359c2ba 100755 --- a/tests/qemu-iotests/122 +++ b/tests/qemu-iotests/122 @@ -130,53 +130,6 @@ $QEMU_IO -c "read -P 0 1024k 1022k" "$TEST_IMG" 2>&= 1 | _filter_qemu_io | _fil =20 =20 echo -echo "=3D=3D=3D Corrupted size field in compressed cluster descriptor =3D= =3D=3D" -echo -# Create an empty image and fill half of it with compressed data. -# The L2 entries of the two compressed clusters are located at -# 0x800000 and 0x800008, their original values are 0x4008000000a00000 -# and 0x4008000000a00802 (5 sectors for compressed data each). -_make_test_img 8M -o cluster_size=3D2M -$QEMU_IO -c "write -c -P 0x11 0 2M" -c "write -c -P 0x11 2M 2M" "$TEST_IMG= " \ - 2>&1 | _filter_qemu_io | _filter_testdir - -# Reduce size of compressed data to 4 sectors: this corrupts the image. -poke_file "$TEST_IMG" $((0x800000)) "\x40\x06" -$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fil= ter_testdir - -# 'qemu-img check' however doesn't see anything wrong because it -# doesn't try to decompress the data and the refcounts are consistent. -# TODO: update qemu-img so this can be detected. -_check_test_img - -# Increase size of compressed data to the maximum (8192 sectors). -# This makes QEMU read more data (8192 sectors instead of 5, host -# addresses [0xa00000, 0xdfffff]), but the decompression algorithm -# stops once we have enough to restore the uncompressed cluster, so -# the rest of the data is ignored. -poke_file "$TEST_IMG" $((0x800000)) "\x7f\xfe" -# Do it also for the second compressed cluster (L2 entry at 0x800008). -# In this case the compressed data would span 3 host clusters -# (host addresses: [0xa00802, 0xe00801]) -poke_file "$TEST_IMG" $((0x800008)) "\x7f\xfe" - -# Here the image is too small so we're asking QEMU to read beyond the -# end of the image. -$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir -# But if we grow the image we won't be reading beyond its end anymore. -$QEMU_IO -c "write -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir -$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir - -# The refcount data is however wrong because due to the increased size -# of the compressed data it now reaches the following host clusters. -# This can be repaired by qemu-img check by increasing the refcount of -# those clusters. -# TODO: update qemu-img to correct the compressed cluster size instead. -_check_test_img -r all -$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir -$QEMU_IO -c "read -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir - -echo echo "=3D=3D=3D Full allocation with -S 0 =3D=3D=3D" echo =20 diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out index a6b7fe007e..47d8656db8 100644 --- a/tests/qemu-iotests/122.out +++ b/tests/qemu-iotests/122.out @@ -99,39 +99,6 @@ read 1024/1024 bytes at offset 1047552 read 1046528/1046528 bytes at offset 1048576 1022 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) =20 -=3D=3D=3D Corrupted size field in compressed cluster descriptor =3D=3D=3D - -Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D8388608 -wrote 2097152/2097152 bytes at offset 0 -2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -wrote 2097152/2097152 bytes at offset 2097152 -2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -read failed: Input/output error -No errors were found on the image. -read 4194304/4194304 bytes at offset 0 -4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -wrote 4194304/4194304 bytes at offset 4194304 -4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -read 4194304/4194304 bytes at offset 0 -4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -ERROR cluster 6 refcount=3D1 reference=3D3 -ERROR cluster 7 refcount=3D1 reference=3D2 -Repairing cluster 6 refcount=3D1 reference=3D3 -Repairing cluster 7 refcount=3D1 reference=3D2 -Repairing OFLAG_COPIED data cluster: l2_entry=3D8000000000c00000 refcount= =3D3 -Repairing OFLAG_COPIED data cluster: l2_entry=3D8000000000e00000 refcount= =3D2 -The following inconsistencies were found and repaired: - - 0 leaked clusters - 4 corruptions - -Double checking the fixed image now... -No errors were found on the image. -read 4194304/4194304 bytes at offset 0 -4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -read 4194304/4194304 bytes at offset 4194304 -4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) - =3D=3D=3D Full allocation with -S 0 =3D=3D=3D =20 Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108864 diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214 new file mode 100755 index 0000000000..c46ca2a6dd --- /dev/null +++ b/tests/qemu-iotests/214 @@ -0,0 +1,97 @@ +#!/bin/bash +# +# Test qcow2 image compression +# +# Copyright (C) 2018 Igalia, S.L. +# Author: Alberto Garcia +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +seq=3D$(basename "$0") +echo "QA output created by $seq" + +here=3D$PWD +status=3D1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +# Repairing the corrupted image requires qemu-img check to store a +# refcount up to 3, which requires at least two refcount bits. +_unsupported_imgopts 'refcount_bits=3D1[^0-9]' + + +echo +echo "=3D=3D=3D Corrupted size field in compressed cluster descriptor =3D= =3D=3D" +echo +# Create an empty image and fill half of it with compressed data. +# The L2 entries of the two compressed clusters are located at +# 0x800000 and 0x800008, their original values are 0x4008000000a00000 +# and 0x4008000000a00802 (5 sectors for compressed data each). +_make_test_img 8M -o cluster_size=3D2M +$QEMU_IO -c "write -c -P 0x11 0 2M" -c "write -c -P 0x11 2M 2M" "$TEST_IMG= " \ + 2>&1 | _filter_qemu_io | _filter_testdir + +# Reduce size of compressed data to 4 sectors: this corrupts the image. +poke_file "$TEST_IMG" $((0x800000)) "\x40\x06" +$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fil= ter_testdir + +# 'qemu-img check' however doesn't see anything wrong because it +# doesn't try to decompress the data and the refcounts are consistent. +# TODO: update qemu-img so this can be detected. +_check_test_img + +# Increase size of compressed data to the maximum (8192 sectors). +# This makes QEMU read more data (8192 sectors instead of 5, host +# addresses [0xa00000, 0xdfffff]), but the decompression algorithm +# stops once we have enough to restore the uncompressed cluster, so +# the rest of the data is ignored. +poke_file "$TEST_IMG" $((0x800000)) "\x7f\xfe" +# Do it also for the second compressed cluster (L2 entry at 0x800008). +# In this case the compressed data would span 3 host clusters +# (host addresses: [0xa00802, 0xe00801]) +poke_file "$TEST_IMG" $((0x800008)) "\x7f\xfe" + +# Here the image is too small so we're asking QEMU to read beyond the +# end of the image. +$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir +# But if we grow the image we won't be reading beyond its end anymore. +$QEMU_IO -c "write -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir +$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir + +# The refcount data is however wrong because due to the increased size +# of the compressed data it now reaches the following host clusters. +# This can be repaired by qemu-img check by increasing the refcount of +# those clusters. +# TODO: update qemu-img to correct the compressed cluster size instead. +_check_test_img -r all +$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir +$QEMU_IO -c "read -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fi= lter_testdir + +# success, all done +echo '*** done' +rm -f $seq.full +status=3D0 diff --git a/tests/qemu-iotests/214.out b/tests/qemu-iotests/214.out new file mode 100644 index 0000000000..0fcd8dc051 --- /dev/null +++ b/tests/qemu-iotests/214.out @@ -0,0 +1,35 @@ +QA output created by 214 + +=3D=3D=3D Corrupted size field in compressed cluster descriptor =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D8388608 +wrote 2097152/2097152 bytes at offset 0 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2097152/2097152 bytes at offset 2097152 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read failed: Input/output error +No errors were found on the image. +read 4194304/4194304 bytes at offset 0 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4194304/4194304 bytes at offset 4194304 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4194304/4194304 bytes at offset 0 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +ERROR cluster 6 refcount=3D1 reference=3D3 +ERROR cluster 7 refcount=3D1 reference=3D2 +Repairing cluster 6 refcount=3D1 reference=3D3 +Repairing cluster 7 refcount=3D1 reference=3D2 +Repairing OFLAG_COPIED data cluster: l2_entry=3D8000000000c00000 refcount= =3D3 +Repairing OFLAG_COPIED data cluster: l2_entry=3D8000000000e00000 refcount= =3D2 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 4 corruptions + +Double checking the fixed image now... +No errors were found on the image. +read 4194304/4194304 bytes at offset 0 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4194304/4194304 bytes at offset 4194304 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 5daef24020..aed024af05 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -212,4 +212,5 @@ 211 rw auto quick 212 rw auto quick 213 rw auto quick +214 rw auto 218 rw auto quick --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 152640095116878.74101658650682; Tue, 15 May 2018 09:15:51 -0700 (PDT) Received: from localhost ([::1]:44847 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcc8-00020Z-3n for importer@patchew.org; Tue, 15 May 2018 12:15:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38935) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4Z-0006pG-9r for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4U-0001Yb-Jk for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:07 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34252 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4O-0001RH-OM; Tue, 15 May 2018 11:40:56 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3CE65BB40C; Tue, 15 May 2018 15:40:56 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8F8F5200BC1E; Tue, 15 May 2018 15:40:55 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:14 +0200 Message-Id: <20180515154033.19899-19-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:56 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:40:56 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 18/37] Fix error message about compressed clusters with OFLAG_COPIED X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia Compressed clusters are not supposed to have the COPIED bit set. "qemu-img check" detects that and prints an error message reporting the number of the affected host cluster. This doesn't make much sense because compressed clusters are not aligned to host clusters, so it would be better to report the offset instead. Plus, the calculation is wrong and it uses the raw L2 entry as if it was simply an offset. This patch fixes the error message and reports the offset of the compressed cluster. Signed-off-by: Alberto Garcia Message-id: 0f687957feb72e80c740403191a47e607c2463fe.1523376013.git.berto@i= galia.com Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 6b8b63514a..2dc23005b7 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1577,9 +1577,9 @@ static int check_refcounts_l2(BlockDriverState *bs, B= drvCheckResult *res, case QCOW2_CLUSTER_COMPRESSED: /* Compressed clusters don't have QCOW_OFLAG_COPIED */ if (l2_entry & QCOW_OFLAG_COPIED) { - fprintf(stderr, "ERROR: cluster %" PRId64 ": " + fprintf(stderr, "ERROR: coffset=3D0x%" PRIx64 ": " "copied flag must never be set for compressed " - "clusters\n", l2_entry >> s->cluster_bits); + "clusters\n", l2_entry & s->cluster_offset_mask); l2_entry &=3D ~QCOW_OFLAG_COPIED; res->corruptions++; } --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399279701474.78586235779755; Tue, 15 May 2018 08:47:59 -0700 (PDT) Received: from localhost ([::1]:44447 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcBA-0003p8-5t for importer@patchew.org; Tue, 15 May 2018 11:47:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38815) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4S-0006ib-UZ for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4S-0001VG-2R for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:00 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36650 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4P-0001S8-KQ; Tue, 15 May 2018 11:40:57 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2580D40201A4; Tue, 15 May 2018 15:40:57 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 789632026E0E; Tue, 15 May 2018 15:40:56 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:15 +0200 Message-Id: <20180515154033.19899-20-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:57 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:57 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 19/37] specs/qcow2: Clarify that compressed clusters have the COPIED bit reset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia Compressed clusters are not supposed to have the COPIED bit set, but this is not made explicit in the specs, so let's document it. Signed-off-by: Alberto Garcia Message-id: 74552e1d6e858d3159cb0c0e188e80bc9248e337.1523376013.git.berto@i= galia.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- docs/interop/qcow2.txt | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index feb711fb6a..8e1547ded2 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -400,10 +400,10 @@ L2 table entry: 62: 0 for standard clusters 1 for compressed clusters =20 - 63: 0 for a cluster that is unused or requires COW, 1 if i= ts - refcount is exactly one. This information is only accu= rate - in L2 tables that are reachable from the active L1 - table. + 63: 0 for clusters that are unused, compressed or require = COW. + 1 for standard clusters whose refcount is exactly one. + This information is only accurate in L2 tables + that are reachable from the active L1 table. =20 Standard Cluster Descriptor: =20 --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526400140324397.3195520264851; Tue, 15 May 2018 09:02:20 -0700 (PDT) Received: from localhost ([::1]:44557 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcP4-0007sZ-UI for importer@patchew.org; Tue, 15 May 2018 12:02:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38933) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4Z-0006pE-8u for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4U-0001YC-A4 for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:07 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36652 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4Q-0001T3-Hk; Tue, 15 May 2018 11:40:58 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0DE6440201A4; Tue, 15 May 2018 15:40:58 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60C442024CBA; Tue, 15 May 2018 15:40:57 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:16 +0200 Message-Id: <20180515154033.19899-21-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:58 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:58 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 20/37] qcow2: Give the refcount cache the minimum possible size by default X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia The L2 and refcount caches have default sizes that can be overridden using the l2-cache-size and refcount-cache-size (an additional parameter named cache-size sets the combined size of both caches). Unless forced by one of the aforementioned parameters, QEMU will set the unspecified sizes so that the L2 cache is 4 times larger than the refcount cache. This is based on the premise that the refcount metadata needs to be only a fourth of the L2 metadata to cover the same amount of disk space. This is incorrect for two reasons: a) The amount of disk covered by an L2 table depends solely on the cluster size, but in the case of a refcount block it depends on the cluster size *and* the width of each refcount entry. The 4/1 ratio is only valid with 16-bit entries (the default). b) When we talk about disk space and L2 tables we are talking about guest space (L2 tables map guest clusters to host clusters), whereas refcount blocks are used for host clusters (including L1/L2 tables and the refcount blocks themselves). On a fully populated (and uncompressed) qcow2 file, image size > virtual size so there are more refcount entries than L2 entries. Problem (a) could be fixed by adjusting the algorithm to take into account the refcount entry width. Problem (b) could be fixed by increasing a bit the refcount cache size to account for the clusters used for qcow2 metadata. However this patch takes a completely different approach and instead of keeping a ratio between both cache sizes it assigns as much as possible to the L2 cache and the remainder to the refcount cache. The reason is that L2 tables are used for every single I/O request from the guest and the effect of increasing the cache is significant and clearly measurable. Refcount blocks are however only used for cluster allocation and internal snapshots and in practice are accessed sequentially in most cases, so the effect of increasing the cache is negligible (even when doing random writes from the guest). So, make the refcount cache as small as possible unless the user explicitly asks for a larger one. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Message-id: 9695182c2eb11b77cb319689a1ebaa4e7c9d6591.1523968389.git.berto@i= galia.com Signed-off-by: Max Reitz --- block/qcow2.h | 4 ---- block/qcow2.c | 31 +++++++++++++++++++------------ tests/qemu-iotests/137.out | 2 +- 3 files changed, 20 insertions(+), 17 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index adf5c3950f..01b5250415 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -77,10 +77,6 @@ #define DEFAULT_L2_CACHE_CLUSTERS 8 /* clusters */ #define DEFAULT_L2_CACHE_BYTE_SIZE 1048576 /* bytes */ =20 -/* The refblock cache needs only a fourth of the L2 cache size to cover as= many - * clusters */ -#define DEFAULT_L2_REFCOUNT_SIZE_RATIO 4 - #define DEFAULT_CLUSTER_SIZE 65536 =20 =20 diff --git a/block/qcow2.c b/block/qcow2.c index 2f36e632f9..6d532470a8 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -802,23 +802,30 @@ static void read_cache_sizes(BlockDriverState *bs, Qe= muOpts *opts, } else if (refcount_cache_size_set) { *l2_cache_size =3D combined_cache_size - *refcount_cache_size; } else { - *refcount_cache_size =3D combined_cache_size - / (DEFAULT_L2_REFCOUNT_SIZE_RATIO + 1); - *l2_cache_size =3D combined_cache_size - *refcount_cache_size; + uint64_t virtual_disk_size =3D bs->total_sectors * BDRV_SECTOR= _SIZE; + uint64_t max_l2_cache =3D virtual_disk_size / (s->cluster_size= / 8); + uint64_t min_refcount_cache =3D + (uint64_t) MIN_REFCOUNT_CACHE_SIZE * s->cluster_size; + + /* Assign as much memory as possible to the L2 cache, and + * use the remainder for the refcount cache */ + if (combined_cache_size >=3D max_l2_cache + min_refcount_cache= ) { + *l2_cache_size =3D max_l2_cache; + *refcount_cache_size =3D combined_cache_size - *l2_cache_s= ize; + } else { + *refcount_cache_size =3D + MIN(combined_cache_size, min_refcount_cache); + *l2_cache_size =3D combined_cache_size - *refcount_cache_s= ize; + } } } else { - if (!l2_cache_size_set && !refcount_cache_size_set) { + if (!l2_cache_size_set) { *l2_cache_size =3D MAX(DEFAULT_L2_CACHE_BYTE_SIZE, (uint64_t)DEFAULT_L2_CACHE_CLUSTERS * s->cluster_size); - *refcount_cache_size =3D *l2_cache_size - / DEFAULT_L2_REFCOUNT_SIZE_RATIO; - } else if (!l2_cache_size_set) { - *l2_cache_size =3D *refcount_cache_size - * DEFAULT_L2_REFCOUNT_SIZE_RATIO; - } else if (!refcount_cache_size_set) { - *refcount_cache_size =3D *l2_cache_size - / DEFAULT_L2_REFCOUNT_SIZE_RATIO; + } + if (!refcount_cache_size_set) { + *refcount_cache_size =3D MIN_REFCOUNT_CACHE_SIZE * s->cluster_= size; } } =20 diff --git a/tests/qemu-iotests/137.out b/tests/qemu-iotests/137.out index e28e1eadba..96724a6c33 100644 --- a/tests/qemu-iotests/137.out +++ b/tests/qemu-iotests/137.out @@ -22,7 +22,7 @@ refcount-cache-size may not exceed cache-size L2 cache size too big L2 cache entry size must be a power of two between 512 and the cluster siz= e (65536) L2 cache entry size must be a power of two between 512 and the cluster siz= e (65536) -L2 cache size too big +Refcount cache size too big Conflicting values for qcow2 options 'overlap-check' ('constant') and 'ove= rlap-check.template' ('all') Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are an= y of the following: none, constant, cached, all Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are an= y of the following: none, constant, cached, all --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399653800331.6319060596644; Tue, 15 May 2018 08:54:13 -0700 (PDT) Received: from localhost ([::1]:44491 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcHE-0000UD-HG for importer@patchew.org; Tue, 15 May 2018 11:54:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38929) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4Z-0006pC-4D for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4U-0001YK-AT for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:07 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36656 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4R-0001UD-EQ; Tue, 15 May 2018 11:40:59 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EAF9640201A4; Tue, 15 May 2018 15:40:58 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 495A42024CBA; Tue, 15 May 2018 15:40:58 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:17 +0200 Message-Id: <20180515154033.19899-22-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:58 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:58 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 21/37] docs: Document the new default sizes of the qcow2 caches X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Alberto Garcia We have just reduced the refcount cache size to the minimum unless the user explicitly requests a larger one, so we have to update the documentation to reflect this change. Signed-off-by: Alberto Garcia Message-id: c5f0bde23558dd9d33b21fffc76ac9953cc19c56.1523968389.git.berto@i= galia.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- docs/qcow2-cache.txt | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt index 170191a242..8a09a5cc5f 100644 --- a/docs/qcow2-cache.txt +++ b/docs/qcow2-cache.txt @@ -116,31 +116,30 @@ There are three options available, and all of them ta= ke bytes: "refcount-cache-size": maximum size of the refcount block cache "cache-size": maximum size of both caches combined =20 -There are two things that need to be taken into account: +There are a few things that need to be taken into account: =20 - Both caches must have a size that is a multiple of the cluster size (or the cache entry size: see "Using smaller cache sizes" below). =20 - - If you only set one of the options above, QEMU will automatically - adjust the others so that the L2 cache is 4 times bigger than the - refcount cache. + - The default L2 cache size is 8 clusters or 1MB (whichever is more), + and the minimum is 2 clusters (or 2 cache entries, see below). =20 -This means that these options are equivalent: + - The default (and minimum) refcount cache size is 4 clusters. =20 - -drive file=3Dhd.qcow2,l2-cache-size=3D2097152 - -drive file=3Dhd.qcow2,refcount-cache-size=3D524288 - -drive file=3Dhd.qcow2,cache-size=3D2621440 + - If only "cache-size" is specified then QEMU will assign as much + memory as possible to the L2 cache before increasing the refcount + cache size. =20 -The reason for this 1/4 ratio is to ensure that both caches cover the -same amount of disk space. Note however that this is only valid with -the default value of refcount_bits (16). If you are using a different -value you might want to calculate both cache sizes yourself since QEMU -will always use the same 1/4 ratio. +Unlike L2 tables, refcount blocks are not used during normal I/O but +only during allocations and internal snapshots. In most cases they are +accessed sequentially (even during random guest I/O) so increasing the +refcount cache size won't have any measurable effect in performance +(this can change if you are using internal snapshots, so you may want +to think about increasing the cache size if you use them heavily). =20 -It's also worth mentioning that there's no strict need for both caches -to cover the same amount of disk space. The refcount cache is used -much less often than the L2 cache, so it's perfectly reasonable to -keep it small. +Before QEMU 2.12 the refcount cache had a default size of 1/4 of the +L2 cache size. This resulted in unnecessarily large caches, so now the +refcount cache is as small as possible unless overridden by the user. =20 =20 Using smaller cache entries --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526401181040359.3093867850373; Tue, 15 May 2018 09:19:41 -0700 (PDT) Received: from localhost ([::1]:44959 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcfo-0004jc-2C for importer@patchew.org; Tue, 15 May 2018 12:19:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38941) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4Z-0006pI-BO for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4V-0001Z9-5w for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:07 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36658 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4S-0001V7-9j; Tue, 15 May 2018 11:41:00 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D2C5C40201A4; Tue, 15 May 2018 15:40:59 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 320EE200BC1E; Tue, 15 May 2018 15:40:59 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:18 +0200 Message-Id: <20180515154033.19899-23-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:59 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:40:59 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 22/37] iotests: Add failure matching to common.qemu X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Currently, common.qemu only allows to match for results indicating success. The only way to fail is by provoking a timeout. However, sometimes we do have a defined failure output and can match for that, which saves us from having to wait for the timeout in case of failure. Because failure can sometimes just result in a _notrun in the test, it is actually important to care about being able to fail quickly. Also, sometimes we simply do not get any specific output in case of success. The only way to handle this currently would be to define an error message as the string to look for, which means that actual success results in a timeout. This is really bad because it unnecessarily slows down a succeeding test. Therefore, this patch adds a new parameter $success_or_failure to _timed_wait_for and _send_qemu_cmd. Setting this to a non-empty string makes both commands expect two match parameters: If the first matches, the function succeeds. If the second matches, the function fails. Signed-off-by: Max Reitz Message-id: 20180406151731.4285-2-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/common.qemu | 58 +++++++++++++++++++++++++++++++++++++-= ---- 1 file changed, 51 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/common.qemu b/tests/qemu-iotests/common.qemu index 85f66b852c..f285484951 100644 --- a/tests/qemu-iotests/common.qemu +++ b/tests/qemu-iotests/common.qemu @@ -52,11 +52,29 @@ _in_fd=3D4 # response is not echoed out. # If $mismatch_only is set, only non-matching responses will # be echoed. +# +# If $success_or_failure is set, the meaning of the arguments is +# changed as follows: +# $2: A string to search for in the response; if found, this indicates +# success and ${QEMU_STATUS[$1]} is set to 0. +# $3: A string to search for in the response; if found, this indicates +# failure and the test is either aborted (if $qemu_error_no_exit +# is not set) or ${QEMU_STATUS[$1]} is set to -1 (otherwise). function _timed_wait_for() { local h=3D${1} shift =20 + if [ -z "${success_or_failure}" ]; then + success_match=3D${*} + failure_match=3D + else + success_match=3D${1} + failure_match=3D${2} + fi + + timeout=3Dyes + QEMU_STATUS[$h]=3D0 while IFS=3D read -t ${QEMU_COMM_TIMEOUT} resp <&${QEMU_OUT[$h]} do @@ -64,10 +82,18 @@ function _timed_wait_for() echo "${resp}" | _filter_testdir | _filter_qemu \ | _filter_qemu_io | _filter_qmp | _filter_hmp fi - grep -q "${*}" < <(echo "${resp}") + if [ -n "${failure_match}" ]; then + grep -q "${failure_match}" < <(echo "${resp}") + if [ $? -eq 0 ]; then + timeout=3D + break + fi + fi + grep -q "${success_match}" < <(echo "${resp}") if [ $? -eq 0 ]; then return - elif [ -z "${silent}" ] && [ -n "${mismatch_only}" ]; then + fi + if [ -z "${silent}" ] && [ -n "${mismatch_only}" ]; then echo "${resp}" | _filter_testdir | _filter_qemu \ | _filter_qemu_io | _filter_qmp | _filter_hmp fi @@ -75,8 +101,12 @@ function _timed_wait_for() done QEMU_STATUS[$h]=3D-1 if [ -z "${qemu_error_no_exit}" ]; then - echo "Timeout waiting for ${*} on handle ${h}" - exit 1 # Timeout means the test failed + if [ -n "${timeout}" ]; then + echo "Timeout waiting for ${success_match} on handle ${h}" + else + echo "Wrong response matching ${failure_match} on handle ${h}" + fi + exit 1 # Timeout or wrong match mean the test failed fi } =20 @@ -96,6 +126,11 @@ function _timed_wait_for() # If $qemu_error_no_exit is set, then even if the expected response # is not seen, we will not exit. $QEMU_STATUS[$1] will be set it -1 in # that case. +# +# If $success_or_failure is set, then the last two strings are the +# strings the response will be scanned for. The first of the two +# indicates success, the latter indicates failure. Failure is handled +# like a timeout. function _send_qemu_cmd() { local h=3D${1} @@ -109,14 +144,23 @@ function _send_qemu_cmd() use_error=3D"no" fi # This array element extraction is done to accommodate pathnames with = spaces - cmd=3D${@: 1:${#@}-1} - shift $(($# - 1)) + if [ -z "${success_or_failure}" ]; then + cmd=3D${@: 1:${#@}-1} + shift $(($# - 1)) + else + cmd=3D${@: 1:${#@}-2} + shift $(($# - 2)) + fi =20 while [ ${count} -gt 0 ] do echo "${cmd}" >&${QEMU_IN[${h}]} if [ -n "${1}" ]; then - qemu_error_no_exit=3D${use_error} _timed_wait_for ${h} "${1}" + if [ -z "${success_or_failure}" ]; then + qemu_error_no_exit=3D${use_error} _timed_wait_for ${h} "${= 1}" + else + qemu_error_no_exit=3D${use_error} _timed_wait_for ${h} "${= 1}" "${2}" + fi if [ ${QEMU_STATUS[$h]} -eq 0 ]; then return fi --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399473765749.389547064874; Tue, 15 May 2018 08:51:13 -0700 (PDT) Received: from localhost ([::1]:44476 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcEL-0006Xm-1u for importer@patchew.org; Tue, 15 May 2018 11:51:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38999) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4a-0006rD-RN for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4Z-0001fk-Sb for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:08 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36662 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4T-0001WS-5v; Tue, 15 May 2018 11:41:01 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BB4BF40201A6; Tue, 15 May 2018 15:41:00 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1A4322026E0E; Tue, 15 May 2018 15:40:59 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:19 +0200 Message-Id: <20180515154033.19899-24-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:00 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:00 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 23/37] iotests: Skip 181 and 201 without userfaultfd X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz userfaultfd support depends on the host kernel, so it may not be available. If so, 181 and 201 should be skipped. Signed-off-by: Max Reitz Message-id: 20180406151731.4285-3-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/181 | 13 +++++++++++++ tests/qemu-iotests/201 | 13 +++++++++++++ 2 files changed, 26 insertions(+) diff --git a/tests/qemu-iotests/181 b/tests/qemu-iotests/181 index 5e767c6195..e02979378d 100755 --- a/tests/qemu-iotests/181 +++ b/tests/qemu-iotests/181 @@ -96,6 +96,19 @@ echo # Enable postcopy-ram capability both on source and destination silent=3Dyes _send_qemu_cmd $dest 'migrate_set_capability postcopy-ram on' "(qemu)" + +qemu_error_no_exit=3Dyes success_or_failure=3Dyes \ + _send_qemu_cmd $dest '' "(qemu)" "Postcopy is not supported" +if [ ${QEMU_STATUS[$dest]} -lt 0 ]; then + _send_qemu_cmd $dest '' "(qemu)" + + _send_qemu_cmd $src 'quit' "" + _send_qemu_cmd $dest 'quit' "" + wait=3D1 _cleanup_qemu + + _notrun 'Postcopy is not supported' +fi + _send_qemu_cmd $src 'migrate_set_speed 4k' "(qemu)" _send_qemu_cmd $src 'migrate_set_capability postcopy-ram on' "(qemu)" _send_qemu_cmd $src "migrate -d unix:${MIG_SOCKET}" "(qemu)" diff --git a/tests/qemu-iotests/201 b/tests/qemu-iotests/201 index 11f640f5df..c1a1e00077 100755 --- a/tests/qemu-iotests/201 +++ b/tests/qemu-iotests/201 @@ -82,6 +82,19 @@ echo =20 silent=3Dyes _send_qemu_cmd $dest 'migrate_set_capability postcopy-ram on' "(qemu)" + +qemu_error_no_exit=3Dyes success_or_failure=3Dyes \ + _send_qemu_cmd $dest '' "(qemu)" "Postcopy is not supported" +if [ ${QEMU_STATUS[$dest]} -lt 0 ]; then + _send_qemu_cmd $dest '' "(qemu)" + + _send_qemu_cmd $src 'quit' "" + _send_qemu_cmd $dest 'quit' "" + wait=3D1 _cleanup_qemu + + _notrun 'Postcopy is not supported' +fi + _send_qemu_cmd $src 'migrate_set_capability postcopy-ram on' "(qemu)" _send_qemu_cmd $src "migrate -d unix:${MIG_SOCKET}" "(qemu)" =20 --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 152640060257893.74629973571007; Tue, 15 May 2018 09:10:02 -0700 (PDT) Received: from localhost ([::1]:44726 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcWX-0005de-JJ for importer@patchew.org; Tue, 15 May 2018 12:10:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39179) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4k-00073O-6z for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4h-0001pr-CS for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:18 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36664 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4U-0001XN-37; Tue, 15 May 2018 11:41:02 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A36E740201A4; Tue, 15 May 2018 15:41:01 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 01C812024CBA; Tue, 15 May 2018 15:41:00 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:20 +0200 Message-Id: <20180515154033.19899-25-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:01 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:01 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 24/37] block: Add COR filter driver X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz This adds a simple copy-on-read filter driver. It relies on the already existing COR functionality in the central block layer code, which may be moved here once we no longer need it there. Signed-off-by: Max Reitz Message-id: 20180421132929.21610-2-mreitz@redhat.com Reviewed-by: Alberto Garcia Reviewed-by: Kevin Wolf Signed-off-by: Max Reitz --- qapi/block-core.json | 5 +- block/copy-on-read.c | 171 +++++++++++++++++++++++++++++++++++++++++++++++= ++++ block/Makefile.objs | 2 +- 3 files changed, 176 insertions(+), 2 deletions(-) create mode 100644 block/copy-on-read.c diff --git a/qapi/block-core.json b/qapi/block-core.json index 17ffd44cce..55728cb823 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2510,11 +2510,12 @@ # @vxhs: Since 2.10 # @throttle: Since 2.11 # @nvme: Since 2.12 +# @copy-on-read: Since 2.13 # # Since: 2.9 ## { 'enum': 'BlockdevDriver', - 'data': [ 'blkdebug', 'blkverify', 'bochs', 'cloop', + 'data': [ 'blkdebug', 'blkverify', 'bochs', 'cloop', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', 'qcow', 'qcow2', '= qed', @@ -3531,6 +3532,7 @@ 'blkverify': 'BlockdevOptionsBlkverify', 'bochs': 'BlockdevOptionsGenericFormat', 'cloop': 'BlockdevOptionsGenericFormat', + 'copy-on-read':'BlockdevOptionsGenericFormat', 'dmg': 'BlockdevOptionsGenericFormat', 'file': 'BlockdevOptionsFile', 'ftp': 'BlockdevOptionsCurlFtp', @@ -4058,6 +4060,7 @@ 'blkverify': 'BlockdevCreateNotSupported', 'bochs': 'BlockdevCreateNotSupported', 'cloop': 'BlockdevCreateNotSupported', + 'copy-on-read': 'BlockdevCreateNotSupported', 'dmg': 'BlockdevCreateNotSupported', 'file': 'BlockdevCreateOptionsFile', 'ftp': 'BlockdevCreateNotSupported', diff --git a/block/copy-on-read.c b/block/copy-on-read.c new file mode 100644 index 0000000000..823ec751c4 --- /dev/null +++ b/block/copy-on-read.c @@ -0,0 +1,171 @@ +/* + * Copy-on-read filter block driver + * + * Copyright (c) 2018 Red Hat, Inc. + * + * Author: + * Max Reitz + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 or + * (at your option) version 3 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, see . + */ + +#include "qemu/osdep.h" +#include "block/block_int.h" + + +static int cor_open(BlockDriverState *bs, QDict *options, int flags, + Error **errp) +{ + bs->file =3D bdrv_open_child(NULL, options, "file", bs, &child_file, f= alse, + errp); + if (!bs->file) { + return -EINVAL; + } + + bs->supported_write_flags =3D BDRV_REQ_FUA & + bs->file->bs->supported_write_flags; + + bs->supported_zero_flags =3D (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & + bs->file->bs->supported_zero_flags; + + return 0; +} + + +static void cor_close(BlockDriverState *bs) +{ +} + + +#define PERM_PASSTHROUGH (BLK_PERM_CONSISTENT_READ \ + | BLK_PERM_WRITE \ + | BLK_PERM_RESIZE) +#define PERM_UNCHANGED (BLK_PERM_ALL & ~PERM_PASSTHROUGH) + +static void cor_child_perm(BlockDriverState *bs, BdrvChild *c, + const BdrvChildRole *role, + BlockReopenQueue *reopen_queue, + uint64_t perm, uint64_t shared, + uint64_t *nperm, uint64_t *nshared) +{ + if (c =3D=3D NULL) { + *nperm =3D (perm & PERM_PASSTHROUGH) | BLK_PERM_WRITE_UNCHANGED; + *nshared =3D (shared & PERM_PASSTHROUGH) | PERM_UNCHANGED; + return; + } + + *nperm =3D (perm & PERM_PASSTHROUGH) | + (c->perm & PERM_UNCHANGED); + *nshared =3D (shared & PERM_PASSTHROUGH) | + (c->shared_perm & PERM_UNCHANGED); +} + + +static int64_t cor_getlength(BlockDriverState *bs) +{ + return bdrv_getlength(bs->file->bs); +} + + +static int cor_truncate(BlockDriverState *bs, int64_t offset, + PreallocMode prealloc, Error **errp) +{ + return bdrv_truncate(bs->file, offset, prealloc, errp); +} + + +static int coroutine_fn cor_co_preadv(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags) +{ + return bdrv_co_preadv(bs->file, offset, bytes, qiov, + flags | BDRV_REQ_COPY_ON_READ); +} + + +static int coroutine_fn cor_co_pwritev(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags) +{ + + return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags); +} + + +static int coroutine_fn cor_co_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, int bytes, + BdrvRequestFlags flags) +{ + return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags); +} + + +static int coroutine_fn cor_co_pdiscard(BlockDriverState *bs, + int64_t offset, int bytes) +{ + return bdrv_co_pdiscard(bs->file->bs, offset, bytes); +} + + +static void cor_eject(BlockDriverState *bs, bool eject_flag) +{ + bdrv_eject(bs->file->bs, eject_flag); +} + + +static void cor_lock_medium(BlockDriverState *bs, bool locked) +{ + bdrv_lock_medium(bs->file->bs, locked); +} + + +static bool cor_recurse_is_first_non_filter(BlockDriverState *bs, + BlockDriverState *candidate) +{ + return bdrv_recurse_is_first_non_filter(bs->file->bs, candidate); +} + + +BlockDriver bdrv_copy_on_read =3D { + .format_name =3D "copy-on-read", + + .bdrv_open =3D cor_open, + .bdrv_close =3D cor_close, + .bdrv_child_perm =3D cor_child_perm, + + .bdrv_getlength =3D cor_getlength, + .bdrv_truncate =3D cor_truncate, + + .bdrv_co_preadv =3D cor_co_preadv, + .bdrv_co_pwritev =3D cor_co_pwritev, + .bdrv_co_pwrite_zeroes =3D cor_co_pwrite_zeroes, + .bdrv_co_pdiscard =3D cor_co_pdiscard, + + .bdrv_eject =3D cor_eject, + .bdrv_lock_medium =3D cor_lock_medium, + + .bdrv_co_block_status =3D bdrv_co_block_status_from_file, + + .bdrv_recurse_is_first_non_filter =3D cor_recurse_is_first_non_filte= r, + + .has_variable_length =3D true, + .is_filter =3D true, +}; + +static void bdrv_copy_on_read_init(void) +{ + bdrv_register(&bdrv_copy_on_read); +} + +block_init(bdrv_copy_on_read_init); diff --git a/block/Makefile.objs b/block/Makefile.objs index d644bac60a..899bfb5e2c 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -26,7 +26,7 @@ block-obj-y +=3D accounting.o dirty-bitmap.o block-obj-y +=3D write-threshold.o block-obj-y +=3D backup.o block-obj-$(CONFIG_REPLICATION) +=3D replication.o -block-obj-y +=3D throttle.o +block-obj-y +=3D throttle.o copy-on-read.o =20 block-obj-y +=3D crypto.o =20 --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526399909715764.8973068785231; Tue, 15 May 2018 08:58:29 -0700 (PDT) Received: from localhost ([::1]:44524 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcLM-0004jp-OK for importer@patchew.org; Tue, 15 May 2018 11:58:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39151) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4i-000714-BF for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4e-0001mv-Cz for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:16 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34268 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4V-0001Yf-0K; Tue, 15 May 2018 11:41:03 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8B8BCBB40F; Tue, 15 May 2018 15:41:02 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id DE4FB2024CBA; Tue, 15 May 2018 15:41:01 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:21 +0200 Message-Id: <20180515154033.19899-26-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:41:02 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:41:02 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 25/37] block: BLK_PERM_WRITE includes ..._UNCHANGED X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Currently we never actually check whether the WRITE_UNCHANGED permission has been taken for unchanging writes. But the one check that is commented out checks both WRITE and WRITE_UNCHANGED; and considering that WRITE_UNCHANGED is already documented as being weaker than WRITE, we should probably explicitly document WRITE to include WRITE_UNCHANGED. Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi Reviewed-by: Alberto Garcia Message-id: 20180421132929.21610-3-mreitz@redhat.com Reviewed-by: Kevin Wolf Signed-off-by: Max Reitz --- include/block/block.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/block/block.h b/include/block/block.h index cdec3639a3..397b5e8d44 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -205,6 +205,9 @@ enum { * This permission (which is weaker than BLK_PERM_WRITE) is both enoug= h and * required for writes to the block node when the caller promises that * the visible disk content doesn't change. + * + * As the BLK_PERM_WRITE permission is strictly stronger, either is + * sufficient to perform an unchanging write. */ BLK_PERM_WRITE_UNCHANGED =3D 0x04, =20 --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526400383372344.74375923424145; Tue, 15 May 2018 09:06:23 -0700 (PDT) Received: from localhost ([::1]:44630 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcT0-0002iV-FJ for importer@patchew.org; Tue, 15 May 2018 12:06:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39027) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4b-0006s3-N9 for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4a-0001ge-Ly for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:09 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36666 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4V-0001a9-SD; Tue, 15 May 2018 11:41:03 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 738F640201A6; Tue, 15 May 2018 15:41:03 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id C6BA2200BC1E; Tue, 15 May 2018 15:41:02 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:22 +0200 Message-Id: <20180515154033.19899-27-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:03 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:03 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 26/37] block: Add BDRV_REQ_WRITE_UNCHANGED flag X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz This flag signifies that a write request will not change the visible disk content. With this flag set, it is sufficient to have the BLK_PERM_WRITE_UNCHANGED permission instead of BLK_PERM_WRITE. Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi Reviewed-by: Alberto Garcia Message-id: 20180421132929.21610-4-mreitz@redhat.com Reviewed-by: Kevin Wolf Signed-off-by: Max Reitz --- include/block/block.h | 6 +++++- block/io.c | 6 +++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index 397b5e8d44..3894edda9d 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -54,8 +54,12 @@ typedef enum { BDRV_REQ_FUA =3D 0x10, BDRV_REQ_WRITE_COMPRESSED =3D 0x20, =20 + /* Signifies that this write request will not change the visible disk + * content. */ + BDRV_REQ_WRITE_UNCHANGED =3D 0x40, + /* Mask of valid flags */ - BDRV_REQ_MASK =3D 0x3f, + BDRV_REQ_MASK =3D 0x7f, } BdrvRequestFlags; =20 typedef struct BlockSizes { diff --git a/block/io.c b/block/io.c index 4fad5ac2fe..9e8449e795 100644 --- a/block/io.c +++ b/block/io.c @@ -1504,7 +1504,11 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChi= ld *child, assert(!waited || !req->serialising); assert(req->overlap_offset <=3D offset); assert(offset + bytes <=3D req->overlap_offset + req->overlap_bytes); - assert(child->perm & BLK_PERM_WRITE); + if (flags & BDRV_REQ_WRITE_UNCHANGED) { + assert(child->perm & (BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE)); + } else { + assert(child->perm & BLK_PERM_WRITE); + } assert(end_sector <=3D bs->total_sectors || child->perm & BLK_PERM_RES= IZE); =20 ret =3D notifier_with_return_list_notify(&bs->before_write_notifiers, = req); --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15264001428941001.1702248575722; Tue, 15 May 2018 09:02:22 -0700 (PDT) Received: from localhost ([::1]:44558 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcP8-0007uo-2o for importer@patchew.org; Tue, 15 May 2018 12:02:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39069) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4e-0006vx-FR for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4a-0001gR-Jf for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:12 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34274 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4X-0001c7-Qc; Tue, 15 May 2018 11:41:05 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 594A9BB409; Tue, 15 May 2018 15:41:05 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id AC1C82024CBA; Tue, 15 May 2018 15:41:04 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:23 +0200 Message-Id: <20180515154033.19899-28-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:41:05 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:41:05 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 27/37] block: Set BDRV_REQ_WRITE_UNCHANGED for COR writes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi Reviewed-by: Alberto Garcia Message-id: 20180421132929.21610-5-mreitz@redhat.com Reviewed-by: Kevin Wolf Signed-off-by: Max Reitz --- block/io.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/block/io.c b/block/io.c index 9e8449e795..ca96b487eb 100644 --- a/block/io.c +++ b/block/io.c @@ -1118,13 +1118,15 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(Bd= rvChild *child, /* FIXME: Should we (perhaps conditionally) be setting * BDRV_REQ_MAY_UNMAP, if it will allow for a sparser copy * that still correctly reads as zero? */ - ret =3D bdrv_co_do_pwrite_zeroes(bs, cluster_offset, pnum,= 0); + ret =3D bdrv_co_do_pwrite_zeroes(bs, cluster_offset, pnum, + BDRV_REQ_WRITE_UNCHANGED); } else { /* This does not change the data on the disk, it is not * necessary to flush even in cache=3Dwritethrough mode. */ ret =3D bdrv_driver_pwritev(bs, cluster_offset, pnum, - &local_qiov, 0); + &local_qiov, + BDRV_REQ_WRITE_UNCHANGED); } =20 if (ret < 0) { --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526401384808807.1415455555231; Tue, 15 May 2018 09:23:04 -0700 (PDT) Received: from localhost ([::1]:45078 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcj5-0007Pn-J2 for importer@patchew.org; Tue, 15 May 2018 12:22:59 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39104) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4f-0006xq-PI for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4b-0001i6-Nu for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:13 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36672 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4Y-0001cs-MN; Tue, 15 May 2018 11:41:06 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 43AC040201A4; Tue, 15 May 2018 15:41:06 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 953032024CBA; Tue, 15 May 2018 15:41:05 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:24 +0200 Message-Id: <20180515154033.19899-29-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:06 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:06 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 28/37] block/quorum: Support BDRV_REQ_WRITE_UNCHANGED X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz We just need to forward it to quorum's children (except in case of a rewrite because of corruption), but for that we first have to support flags in child requests at all. Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi Reviewed-by: Alberto Garcia Message-id: 20180421132929.21610-6-mreitz@redhat.com Reviewed-by: Kevin Wolf Signed-off-by: Max Reitz --- block/quorum.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/block/quorum.c b/block/quorum.c index a5051da56e..e448d7e384 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -115,6 +115,7 @@ struct QuorumAIOCB { /* Request metadata */ uint64_t offset; uint64_t bytes; + int flags; =20 QEMUIOVector *qiov; /* calling IOV */ =20 @@ -157,7 +158,8 @@ static bool quorum_64bits_compare(QuorumVoteValue *a, Q= uorumVoteValue *b) static QuorumAIOCB *quorum_aio_get(BlockDriverState *bs, QEMUIOVector *qiov, uint64_t offset, - uint64_t bytes) + uint64_t bytes, + int flags) { BDRVQuorumState *s =3D bs->opaque; QuorumAIOCB *acb =3D g_new(QuorumAIOCB, 1); @@ -168,6 +170,7 @@ static QuorumAIOCB *quorum_aio_get(BlockDriverState *bs, .bs =3D bs, .offset =3D offset, .bytes =3D bytes, + .flags =3D flags, .qiov =3D qiov, .votes.compare =3D quorum_sha256_compare, .votes.vote_list =3D QLIST_HEAD_INITIALIZER(acb.votes.vote_list= ), @@ -271,9 +274,11 @@ static void quorum_rewrite_entry(void *opaque) BDRVQuorumState *s =3D acb->bs->opaque; =20 /* Ignore any errors, it's just a correction attempt for already - * corrupted data. */ + * corrupted data. + * Mask out BDRV_REQ_WRITE_UNCHANGED because this overwrites the + * area with different data from the other children. */ bdrv_co_pwritev(s->children[co->idx], acb->offset, acb->bytes, - acb->qiov, 0); + acb->qiov, acb->flags & ~BDRV_REQ_WRITE_UNCHANGED); =20 /* Wake up the caller after the last rewrite */ acb->rewrite_count--; @@ -673,7 +678,7 @@ static int quorum_co_preadv(BlockDriverState *bs, uint6= 4_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { BDRVQuorumState *s =3D bs->opaque; - QuorumAIOCB *acb =3D quorum_aio_get(bs, qiov, offset, bytes); + QuorumAIOCB *acb =3D quorum_aio_get(bs, qiov, offset, bytes, flags); int ret; =20 acb->is_read =3D true; @@ -699,7 +704,7 @@ static void write_quorum_entry(void *opaque) =20 sacb->bs =3D s->children[i]->bs; sacb->ret =3D bdrv_co_pwritev(s->children[i], acb->offset, acb->bytes, - acb->qiov, 0); + acb->qiov, acb->flags); if (sacb->ret =3D=3D 0) { acb->success_count++; } else { @@ -719,7 +724,7 @@ static int quorum_co_pwritev(BlockDriverState *bs, uint= 64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { BDRVQuorumState *s =3D bs->opaque; - QuorumAIOCB *acb =3D quorum_aio_get(bs, qiov, offset, bytes); + QuorumAIOCB *acb =3D quorum_aio_get(bs, qiov, offset, bytes, flags); int i, ret; =20 for (i =3D 0; i < s->num_children; i++) { @@ -961,6 +966,8 @@ static int quorum_open(BlockDriverState *bs, QDict *opt= ions, int flags, } s->next_child_index =3D s->num_children; =20 + bs->supported_write_flags =3D BDRV_REQ_WRITE_UNCHANGED; + g_free(opened); goto exit; =20 --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 152640056484816.89258288819724; Tue, 15 May 2018 09:09:24 -0700 (PDT) Received: from localhost ([::1]:44725 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcVv-00058b-Vn for importer@patchew.org; Tue, 15 May 2018 12:09:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39207) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4l-00074y-6k for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4g-0001ol-2J for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:19 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34280 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4Z-0001en-LQ; Tue, 15 May 2018 11:41:07 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2DC56BB409; Tue, 15 May 2018 15:41:07 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F6CF2026E0E; Tue, 15 May 2018 15:41:06 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:25 +0200 Message-Id: <20180515154033.19899-30-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:41:07 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:41:07 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 29/37] block: Support BDRV_REQ_WRITE_UNCHANGED in filters X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Update the rest of the filter drivers to support BDRV_REQ_WRITE_UNCHANGED. They already forward write request flags to their children, so we just have to announce support for it. This patch does not cover the replication driver because that currently does not support flags at all, and because it just grabs the WRITE permission for its children when it can, so we should be fine just submitting the incoming WRITE_UNCHANGED requests as normal writes. It also does not cover format drivers for similar reasons. They all use bdrv_format_default_perms() as their .bdrv_child_perm() implementation so they just always grab the WRITE permission for their file children whenever possible. In addition, it often would be difficult to ascertain whether incoming unchanging writes end up as unchanging writes in their files. So we just leave them as normal potentially changing writes. Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi Reviewed-by: Alberto Garcia Message-id: 20180421132929.21610-7-mreitz@redhat.com Reviewed-by: Kevin Wolf Signed-off-by: Max Reitz --- block/blkdebug.c | 9 +++++---- block/blkreplay.c | 3 +++ block/blkverify.c | 3 +++ block/copy-on-read.c | 10 ++++++---- block/mirror.c | 2 ++ block/raw-format.c | 9 +++++---- block/throttle.c | 6 ++++-- 7 files changed, 28 insertions(+), 14 deletions(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index 053372c22e..526af2a808 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -398,10 +398,11 @@ static int blkdebug_open(BlockDriverState *bs, QDict = *options, int flags, goto out; } =20 - bs->supported_write_flags =3D BDRV_REQ_FUA & - bs->file->bs->supported_write_flags; - bs->supported_zero_flags =3D (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & - bs->file->bs->supported_zero_flags; + bs->supported_write_flags =3D BDRV_REQ_WRITE_UNCHANGED | + (BDRV_REQ_FUA & bs->file->bs->supported_write_flags); + bs->supported_zero_flags =3D BDRV_REQ_WRITE_UNCHANGED | + ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & + bs->file->bs->supported_zero_flags); ret =3D -EINVAL; =20 /* Set alignment overrides */ diff --git a/block/blkreplay.c b/block/blkreplay.c index fe5a9b4a98..b016dbeee7 100755 --- a/block/blkreplay.c +++ b/block/blkreplay.c @@ -35,6 +35,9 @@ static int blkreplay_open(BlockDriverState *bs, QDict *op= tions, int flags, goto fail; } =20 + bs->supported_write_flags =3D BDRV_REQ_WRITE_UNCHANGED; + bs->supported_zero_flags =3D BDRV_REQ_WRITE_UNCHANGED; + ret =3D 0; fail: return ret; diff --git a/block/blkverify.c b/block/blkverify.c index 754cc9e857..da97ee5927 100644 --- a/block/blkverify.c +++ b/block/blkverify.c @@ -141,6 +141,9 @@ static int blkverify_open(BlockDriverState *bs, QDict *= options, int flags, goto fail; } =20 + bs->supported_write_flags =3D BDRV_REQ_WRITE_UNCHANGED; + bs->supported_zero_flags =3D BDRV_REQ_WRITE_UNCHANGED; + ret =3D 0; fail: qemu_opts_del(opts); diff --git a/block/copy-on-read.c b/block/copy-on-read.c index 823ec751c4..6a97208888 100644 --- a/block/copy-on-read.c +++ b/block/copy-on-read.c @@ -33,11 +33,13 @@ static int cor_open(BlockDriverState *bs, QDict *option= s, int flags, return -EINVAL; } =20 - bs->supported_write_flags =3D BDRV_REQ_FUA & - bs->file->bs->supported_write_flags; + bs->supported_write_flags =3D BDRV_REQ_WRITE_UNCHANGED | + (BDRV_REQ_FUA & + bs->file->bs->supported_write_flags); =20 - bs->supported_zero_flags =3D (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & - bs->file->bs->supported_zero_flags; + bs->supported_zero_flags =3D BDRV_REQ_WRITE_UNCHANGED | + ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & + bs->file->bs->supported_zero_flags); =20 return 0; } diff --git a/block/mirror.c b/block/mirror.c index 6aa38db114..a4197bb975 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1134,6 +1134,8 @@ static void mirror_start_job(const char *job_id, Bloc= kDriverState *bs, mirror_top_bs->implicit =3D true; } mirror_top_bs->total_sectors =3D bs->total_sectors; + mirror_top_bs->supported_write_flags =3D BDRV_REQ_WRITE_UNCHANGED; + mirror_top_bs->supported_zero_flags =3D BDRV_REQ_WRITE_UNCHANGED; bdrv_set_aio_context(mirror_top_bs, bdrv_get_aio_context(bs)); =20 /* bdrv_append takes ownership of the mirror_top_bs reference, need to= keep diff --git a/block/raw-format.c b/block/raw-format.c index a378547c99..fe33693a2d 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -415,10 +415,11 @@ static int raw_open(BlockDriverState *bs, QDict *opti= ons, int flags, } =20 bs->sg =3D bs->file->bs->sg; - bs->supported_write_flags =3D BDRV_REQ_FUA & - bs->file->bs->supported_write_flags; - bs->supported_zero_flags =3D (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & - bs->file->bs->supported_zero_flags; + bs->supported_write_flags =3D BDRV_REQ_WRITE_UNCHANGED | + (BDRV_REQ_FUA & bs->file->bs->supported_write_flags); + bs->supported_zero_flags =3D BDRV_REQ_WRITE_UNCHANGED | + ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & + bs->file->bs->supported_zero_flags); =20 if (bs->probed && !bdrv_is_read_only(bs)) { fprintf(stderr, diff --git a/block/throttle.c b/block/throttle.c index 95ed06acd8..e298827f95 100644 --- a/block/throttle.c +++ b/block/throttle.c @@ -81,8 +81,10 @@ static int throttle_open(BlockDriverState *bs, QDict *op= tions, if (!bs->file) { return -EINVAL; } - bs->supported_write_flags =3D bs->file->bs->supported_write_flags; - bs->supported_zero_flags =3D bs->file->bs->supported_zero_flags; + bs->supported_write_flags =3D bs->file->bs->supported_write_flags | + BDRV_REQ_WRITE_UNCHANGED; + bs->supported_zero_flags =3D bs->file->bs->supported_zero_flags | + BDRV_REQ_WRITE_UNCHANGED; =20 return throttle_configure_tgm(bs, tgm, options, errp); } --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526401710113164.78682487467245; Tue, 15 May 2018 09:28:30 -0700 (PDT) Received: from localhost ([::1]:45735 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcoK-0003ob-2M for importer@patchew.org; Tue, 15 May 2018 12:28:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39212) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4l-000756-AW for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4k-0001tG-CP for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:19 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36676 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4a-0001fz-H4; Tue, 15 May 2018 11:41:08 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 16E6640201A4; Tue, 15 May 2018 15:41:08 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 696E52024CBA; Tue, 15 May 2018 15:41:07 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:26 +0200 Message-Id: <20180515154033.19899-31-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:08 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:08 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 30/37] iotests: Clean up wrap image in 197 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi Reviewed-by: Alberto Garcia Message-id: 20180421132929.21610-8-mreitz@redhat.com Reviewed-by: Kevin Wolf Signed-off-by: Max Reitz --- tests/qemu-iotests/197 | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 index 5e869fe2b7..3ae4975eec 100755 --- a/tests/qemu-iotests/197 +++ b/tests/qemu-iotests/197 @@ -44,6 +44,7 @@ esac _cleanup() { _cleanup_test_img + rm -f "$TEST_WRAP" rm -f "$BLKDBG_CONF" } trap "_cleanup; exit \$status" 0 1 2 3 15 --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526401546824998.7053456818542; Tue, 15 May 2018 09:25:46 -0700 (PDT) Received: from localhost ([::1]:45450 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcll-0001Wa-RT for importer@patchew.org; Tue, 15 May 2018 12:25:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39176) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4k-00073K-6L for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4f-0001nZ-5h for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:18 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36678 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4b-0001gx-Dj; Tue, 15 May 2018 11:41:09 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F3F4440201A4; Tue, 15 May 2018 15:41:08 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5344C2024CBA; Tue, 15 May 2018 15:41:08 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:27 +0200 Message-Id: <20180515154033.19899-32-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:09 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:09 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 31/37] iotests: Copy 197 for COR filter driver X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz iotest 197 tests copy-on-read using the (now old) copy-on-read flag. Copy it to 215 and modify it to use the COR filter driver instead. Signed-off-by: Max Reitz Message-id: 20180421132929.21610-9-mreitz@redhat.com Reviewed-by: Kevin Wolf Signed-off-by: Max Reitz --- tests/qemu-iotests/215 | 120 +++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/215.out | 26 ++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 147 insertions(+) create mode 100755 tests/qemu-iotests/215 create mode 100644 tests/qemu-iotests/215.out diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215 new file mode 100755 index 0000000000..2e616ed659 --- /dev/null +++ b/tests/qemu-iotests/215 @@ -0,0 +1,120 @@ +#!/bin/bash +# +# Test case for copy-on-read into qcow2, using the COR filter driver +# +# Copyright (C) 2018 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +seq=3D"$(basename $0)" +echo "QA output created by $seq" + +here=3D"$PWD" +status=3D1 # failure is the default! + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +TEST_WRAP=3D"$TEST_DIR/t.wrap.qcow2" +BLKDBG_CONF=3D"$TEST_DIR/blkdebug.conf" + +# Sanity check: our use of blkdebug fails if $TEST_DIR contains spaces +# or other problems +case "$TEST_DIR" in + *[^-_a-zA-Z0-9/]*) + _notrun "Suspicious TEST_DIR=3D'$TEST_DIR', cowardly refusing to r= un" ;; +esac + +_cleanup() +{ + _cleanup_test_img + rm -f "$TEST_WRAP" + rm -f "$BLKDBG_CONF" +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# Test is supported for any backing file; but we force qcow2 for our wrapp= er. +_supported_fmt generic +_supported_proto generic +_supported_os Linux +# LUKS support may be possible, but it complicates things. +_unsupported_fmt luks + +echo +echo '=3D=3D=3D Copy-on-read =3D=3D=3D' +echo + +# Prep the images +# VPC rounds image sizes to a specific geometry, force a specific size. +if [ "$IMGFMT" =3D "vpc" ]; then + IMGOPTS=3D$(_optstr_add "$IMGOPTS" "force_size") +fi +_make_test_img 4G +$QEMU_IO -c "write -P 55 3G 1k" "$TEST_IMG" | _filter_qemu_io +IMGPROTO=3Dfile IMGFMT=3Dqcow2 IMGOPTS=3D TEST_IMG_FILE=3D"$TEST_WRAP" \ + _make_test_img -F "$IMGFMT" -b "$TEST_IMG" | _filter_img_create +$QEMU_IO -f qcow2 -c "write -z -u 1M 64k" "$TEST_WRAP" | _filter_qemu_io + +# Ensure that a read of two clusters, but where one is already allocated, +# does not re-write the allocated cluster +cat > "$BLKDBG_CONF" <&1 | _filter_qemu_io) +case $output in + *allocate*) + _notrun "Insufficent memory to run test" ;; + *) printf '%s\n' "$output" ;; +esac +$QEMU_IO \ + -c "open -o driver=3Dcopy-on-read,file.driver=3Dqcow2 $TEST_WRAP" \ + -c "read -P 0 $((3*1024*1024*1024 + 1024)) 1k" \ + | _filter_qemu_io + +# Copy-on-read is incompatible with read-only +$QEMU_IO \ + -c "open -r -o driver=3Dcopy-on-read,file.driver=3Dqcow2 $TEST_WRAP" \ + 2>&1 | _filter_testdir + +# Break the backing chain, and show that images are identical, and that +# we properly copied over explicit zeros. +$QEMU_IMG rebase -u -b "" -f qcow2 "$TEST_WRAP" +$QEMU_IO -f qcow2 -c map "$TEST_WRAP" +_check_test_img +$QEMU_IMG compare -f $IMGFMT -F qcow2 "$TEST_IMG" "$TEST_WRAP" + +# success, all done +echo '*** done' +status=3D0 diff --git a/tests/qemu-iotests/215.out b/tests/qemu-iotests/215.out new file mode 100644 index 0000000000..70b0f5fb19 --- /dev/null +++ b/tests/qemu-iotests/215.out @@ -0,0 +1,26 @@ +QA output created by 215 + +=3D=3D=3D Copy-on-read =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D4294967296 +wrote 1024/1024 bytes at offset 3221225472 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.wrap.IMGFMT', fmt=3DIMGFMT size=3D4294967296 backin= g_file=3DTEST_DIR/t.IMGFMT backing_fmt=3DIMGFMT +wrote 65536/65536 bytes at offset 1048576 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 131072/131072 bytes at offset 1048576 +128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 0/0 bytes at offset 0 +0 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2147483136/2147483136 bytes at offset 1024 +2 GiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1024/1024 bytes at offset 3221226496 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +can't open device TEST_DIR/t.wrap.qcow2: Block node is read-only +2 GiB (0x80010000) bytes allocated at offset 0 bytes (0x0) +1023.938 MiB (0x3fff0000) bytes not allocated at offset 2 GiB (0x80010000) +64 KiB (0x10000) bytes allocated at offset 3 GiB (0xc0000000) +1023.938 MiB (0x3fff0000) bytes not allocated at offset 3 GiB (0xc0010000) +No errors were found on the image. +Images are identical. +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index aed024af05..b59bcea640 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -213,4 +213,5 @@ 212 rw auto quick 213 rw auto quick 214 rw auto +215 rw auto quick 218 rw auto quick --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526402041657668.7521276294465; Tue, 15 May 2018 09:34:01 -0700 (PDT) Received: from localhost ([::1]:45979 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIctk-00081n-Jw for importer@patchew.org; Tue, 15 May 2018 12:34:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39280) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4o-00078j-5C for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4m-0001uz-Fs for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:22 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47574 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4c-0001ia-A2; Tue, 15 May 2018 11:41:10 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DD7F37D843; Tue, 15 May 2018 15:41:09 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3BAF6200BC1E; Tue, 15 May 2018 15:41:09 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:28 +0200 Message-Id: <20180515154033.19899-33-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:41:09 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 15 May 2018 15:41:09 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 32/37] iotests: Add test for COR across nodes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz COR across nodes (that is, you have some filter node between the actually COR target and the node that performs the COR) cannot reliably work together with the permission system when there is no explicit COR node that can request the WRITE_UNCHANGED permission for its child. This is because COR (currently) sneaks its requests by the usual permission checks, so it can work without a WRITE* permission; but if there is a filter node in between, that will re-issue the request, which then passes through the usual check -- and if nobody has requested a WRITE_UNCHANGED permission, that check will fail. There is no real direct fix apart from hoping that there is someone who has requested that permission; in case of just the qemu-io HMP command (and no guest device), however, that is not the case. The real real fix is to implement the copy-on-read flag through an implicitly added COR node. Such a node can request the necessary permissions as shown in this test. Signed-off-by: Max Reitz Message-id: 20180421132929.21610-10-mreitz@redhat.com Reviewed-by: Kevin Wolf Signed-off-by: Max Reitz --- tests/qemu-iotests/216 | 115 +++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/216.out | 28 +++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 144 insertions(+) create mode 100755 tests/qemu-iotests/216 create mode 100644 tests/qemu-iotests/216.out diff --git a/tests/qemu-iotests/216 b/tests/qemu-iotests/216 new file mode 100755 index 0000000000..ca9b47a7fd --- /dev/null +++ b/tests/qemu-iotests/216 @@ -0,0 +1,115 @@ +#!/usr/bin/env python +# +# Copy-on-read tests using a COR filter node +# +# Copyright (C) 2018 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# Creator/Owner: Max Reitz + +import iotests +from iotests import log, qemu_img_pipe, qemu_io, filter_qemu_io + +# Need backing file support +iotests.verify_image_format(supported_fmts=3D['qcow2', 'qcow', 'qed', 'vmd= k']) +iotests.verify_platform(['linux']) + +log('') +log('=3D=3D=3D Copy-on-read across nodes =3D=3D=3D') +log('') + +# The old copy-on-read mechanism without a filter node cannot request +# WRITE_UNCHANGED permissions for its child. Therefore it just tries +# to sneak its write by the usual permission system and holds its +# fingers crossed. However, that sneaking does not work so well when +# there is a filter node in the way: That will receive the write +# request and re-issue a new one to its child, which this time is a +# proper write request that will make the permission system cough -- +# unless there is someone at the top (like a guest device) that has +# requested write permissions. +# +# A COR filter node, however, can request the proper permissions for +# its child and therefore is not hit by this issue. + +with iotests.FilePath('base.img') as base_img_path, \ + iotests.FilePath('top.img') as top_img_path, \ + iotests.VM() as vm: + + log('--- Setting up images ---') + log('') + + qemu_img_pipe('create', '-f', iotests.imgfmt, base_img_path, '64M') + + log(filter_qemu_io(qemu_io(base_img_path, '-c', 'write -P 1 0M 1M'))) + + qemu_img_pipe('create', '-f', iotests.imgfmt, '-b', base_img_path, + top_img_path) + + log(filter_qemu_io(qemu_io(top_img_path, '-c', 'write -P 2 1M 1M'))) + + log('') + log('--- Doing COR ---') + log('') + + # Compare with e.g. the following: + # vm.add_drive_raw('if=3Dnone,node-name=3Dnode0,copy-on-read=3Don,dr= iver=3Draw,' \ + # 'file.driver=3D%s,file.file.filename=3D%s' % + # (iotests.imgfmt, top_img_path)) + # (Remove the blockdev-add instead.) + # ((Not tested here because it hits an assertion in the permission + # system.)) + + vm.launch() + + log(vm.qmp('blockdev-add', + node_name=3D'node0', + driver=3D'copy-on-read', + file=3D{ + 'driver': 'raw', + 'file': { + 'driver': 'copy-on-read', + 'file': { + 'driver': 'raw', + 'file': { + 'driver': iotests.imgfmt, + 'file': { + 'driver': 'file', + 'filename': top_img_path + }, + 'backing': { + 'driver': iotests.imgfmt, + 'file': { + 'driver': 'file', + 'filename': base_img_path + } + } + } + } + } + })) + + # Trigger COR + log(vm.qmp('human-monitor-command', + command_line=3D'qemu-io node0 "read 0 64M"')) + + vm.shutdown() + + log('') + log('--- Checking COR result ---') + log('') + + log(filter_qemu_io(qemu_io(base_img_path, '-c', 'discard 0 64M'))) + log(filter_qemu_io(qemu_io(top_img_path, '-c', 'read -P 1 0M 1M'))) + log(filter_qemu_io(qemu_io(top_img_path, '-c', 'read -P 2 1M 1M'))) diff --git a/tests/qemu-iotests/216.out b/tests/qemu-iotests/216.out new file mode 100644 index 0000000000..d3fc590d29 --- /dev/null +++ b/tests/qemu-iotests/216.out @@ -0,0 +1,28 @@ + +=3D=3D=3D Copy-on-read across nodes =3D=3D=3D + +--- Setting up images --- + +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +wrote 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + + +--- Doing COR --- + +{u'return': {}} +{u'return': u''} + +--- Checking COR result --- + +discard 67108864/67108864 bytes at offset 0 +64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index b59bcea640..cc8cd8cc8e 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -214,4 +214,5 @@ 213 rw auto quick 214 rw auto 215 rw auto quick +216 rw auto quick 218 rw auto quick --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526400802175188.0111094999795; Tue, 15 May 2018 09:13:22 -0700 (PDT) Received: from localhost ([::1]:44768 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcZl-0008IS-7T for importer@patchew.org; Tue, 15 May 2018 12:13:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39224) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4l-00075V-JZ for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4k-0001tC-CP for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:19 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36682 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4d-0001jv-7w; Tue, 15 May 2018 11:41:11 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C57EE40201A4; Tue, 15 May 2018 15:41:10 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 242842026E0E; Tue, 15 May 2018 15:41:10 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:29 +0200 Message-Id: <20180515154033.19899-34-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:10 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:10 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 33/37] qemu-img: Check post-truncation size X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Some block drivers (iscsi and file-posix when dealing with device files) do not actually support truncation, even though they provide a .bdrv_truncate() method and will happily return success when providing a new size that does not exceed the current size. This is because these drivers expect the user to resize the image outside of qemu and then provide qemu with that information through the block_resize command (compare cb1b83e740384b4e0d950f3d7c81c02b8ce86c2e). Of course, anyone using qemu-img resize will find that behavior useless. So we should check the actual size of the image after the supposedly successful truncation took place, emit an error if nothing changed and emit a warning if the target size was not met. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=3D1523065 Signed-off-by: Max Reitz Message-id: 20180421163957.29872-1-mreitz@redhat.com Reviewed-by: Stefan Hajnoczi Signed-off-by: Max Reitz --- qemu-img.c | 39 +++++++++++++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 4 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index ea62d2d61e..62b29e7feb 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -3381,7 +3381,7 @@ static int img_resize(int argc, char **argv) Error *err =3D NULL; int c, ret, relative; const char *filename, *fmt, *size; - int64_t n, total_size, current_size; + int64_t n, total_size, current_size, new_size; bool quiet =3D false; BlockBackend *blk =3D NULL; PreallocMode prealloc =3D PREALLOC_MODE_OFF; @@ -3557,11 +3557,42 @@ static int img_resize(int argc, char **argv) } =20 ret =3D blk_truncate(blk, total_size, prealloc, &err); - if (!ret) { - qprintf(quiet, "Image resized.\n"); - } else { + if (ret < 0) { error_report_err(err); + goto out; + } + + new_size =3D blk_getlength(blk); + if (new_size < 0) { + error_report("Failed to verify truncated image length: %s", + strerror(-new_size)); + ret =3D -1; + goto out; } + + /* Some block drivers implement a truncation method, but only so + * the user can cause qemu to refresh the image's size from disk. + * The idea is that the user resizes the image outside of qemu and + * then invokes block_resize to inform qemu about it. + * (This includes iscsi and file-posix for device files.) + * Of course, that is not the behavior someone invoking + * qemu-img resize would find useful, so we catch that behavior + * here and tell the user. */ + if (new_size !=3D total_size && new_size =3D=3D current_size) { + error_report("Image was not resized; resizing may not be supported= " + "for this image"); + ret =3D -1; + goto out; + } + + if (new_size !=3D total_size) { + warn_report("Image should have been resized to %" PRIi64 + " bytes, but was resized to %" PRIi64 " bytes", + total_size, new_size); + } + + qprintf(quiet, "Image resized.\n"); + out: blk_unref(blk); if (ret) { --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526401857058839.7773050912557; Tue, 15 May 2018 09:30:57 -0700 (PDT) Received: from localhost ([::1]:45794 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcqm-0005t8-7B for importer@patchew.org; Tue, 15 May 2018 12:30:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39217) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4l-00075H-Dh for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4k-0001t3-Bn for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:19 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36684 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4e-0001li-4X; Tue, 15 May 2018 11:41:12 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AD09440201A4; Tue, 15 May 2018 15:41:11 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0CA2C2024CBA; Tue, 15 May 2018 15:41:10 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:30 +0200 Message-Id: <20180515154033.19899-35-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:11 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 15 May 2018 15:41:11 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 34/37] block: Document BDRV_REQ_WRITE_UNCHANGED support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Add BDRV_REQ_WRITE_UNCHANGED to the list of flags honored during pwrite and pwrite_zeroes, and also add a note on when you absolutely need to support it. Signed-off-by: Max Reitz Message-id: 20180502140359.18222-1-mreitz@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- include/block/block_int.h | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index e3d6219f4e..76b589da57 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -656,10 +656,24 @@ struct BlockDriverState { /* I/O Limits */ BlockLimits bl; =20 - /* Flags honored during pwrite (so far: BDRV_REQ_FUA) */ + /* Flags honored during pwrite (so far: BDRV_REQ_FUA, + * BDRV_REQ_WRITE_UNCHANGED). + * If a driver does not support BDRV_REQ_WRITE_UNCHANGED, those + * writes will be issued as normal writes without the flag set. + * This is important to note for drivers that do not explicitly + * request a WRITE permission for their children and instead take + * the same permissions as their parent did (this is commonly what + * block filters do). Such drivers have to be aware that the + * parent may have taken a WRITE_UNCHANGED permission only and is + * issuing such requests. Drivers either must make sure that + * these requests do not result in plain WRITE accesses (usually + * by supporting BDRV_REQ_WRITE_UNCHANGED, and then forwarding + * every incoming write request as-is, including potentially that + * flag), or they have to explicitly take the WRITE permission for + * their children. */ unsigned int supported_write_flags; /* Flags honored during pwrite_zeroes (so far: BDRV_REQ_FUA, - * BDRV_REQ_MAY_UNMAP) */ + * BDRV_REQ_MAY_UNMAP, BDRV_REQ_WRITE_UNCHANGED) */ unsigned int supported_zero_flags; =20 /* the following member gives a name to every node on the bs graph. */ --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526400773619702.445495746312; Tue, 15 May 2018 09:12:53 -0700 (PDT) Received: from localhost ([::1]:44767 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcZI-0007xJ-N0 for importer@patchew.org; Tue, 15 May 2018 12:12:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39257) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4m-00076v-IP for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4l-0001uF-Fx for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:20 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34286 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4f-0001nO-1g; Tue, 15 May 2018 11:41:13 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 94D33BB40C; Tue, 15 May 2018 15:41:12 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id E87622024CBA; Tue, 15 May 2018 15:41:11 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:31 +0200 Message-Id: <20180515154033.19899-36-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:41:12 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 15 May 2018 15:41:12 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 35/37] qemu-io: Use purely string blockdev options X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Currently, qemu-io only uses string-valued blockdev options (as all are converted directly from QemuOpts) -- with one exception: -U adds the force-share option as a boolean. This in itself is already a bit questionable, but a real issue is that it also assumes the value already existing in the options QDict would be a boolean, which is wrong. That has the following effect: $ ./qemu-io -r -U --image-opts \ driver=3Dfile,filename=3D/dev/null,force-share=3Doff [1] 15200 segmentation fault (core dumped) ./qemu-io -r -U --image-opts driver=3Dfile,filename=3D/dev/null,force-share=3Doff Since @opts is converted from QemuOpts, the value must be a string, and we have to compare it as such. Consequently, it makes sense to also set it as a string instead of a boolean. Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Message-id: 20180502202051.15493-2-mreitz@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- qemu-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/qemu-io.c b/qemu-io.c index 72fee0d8b7..73c638ff8b 100644 --- a/qemu-io.c +++ b/qemu-io.c @@ -95,12 +95,12 @@ static int openfile(char *name, int flags, bool writeth= rough, bool force_share, opts =3D qdict_new(); } if (qdict_haskey(opts, BDRV_OPT_FORCE_SHARE) - && !qdict_get_bool(opts, BDRV_OPT_FORCE_SHARE)) { + && strcmp(qdict_get_str(opts, BDRV_OPT_FORCE_SHARE), "on")) { error_report("-U conflicts with image options"); qobject_unref(opts); return 1; } - qdict_put_bool(opts, BDRV_OPT_FORCE_SHARE, true); + qdict_put_str(opts, BDRV_OPT_FORCE_SHARE, "on"); } qemuio_blk =3D blk_new_open(name, NULL, opts, flags, &local_err); if (!qemuio_blk) { --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526400952520427.2617212364006; Tue, 15 May 2018 09:15:52 -0700 (PDT) Received: from localhost ([::1]:44851 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcc9-00027n-LW for importer@patchew.org; Tue, 15 May 2018 12:15:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39325) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4q-0007Ay-Al for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4p-0001xZ-De for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:24 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:50346 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4g-0001ot-C3; Tue, 15 May 2018 11:41:14 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ECF9D4021BA3; Tue, 15 May 2018 15:41:13 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id CFCF0200BC1E; Tue, 15 May 2018 15:41:12 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:32 +0200 Message-Id: <20180515154033.19899-37-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 15 May 2018 15:41:13 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 15 May 2018 15:41:13 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 36/37] qemu-img: Use only string options in img_open_opts X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz img_open_opts() takes a QemuOpts and converts them to a QDict, so all values therein are strings. Then it may try to call qdict_get_bool(), however, which will fail with a segmentation fault every time: $ ./qemu-img info -U --image-opts \ driver=3Dfile,filename=3D/dev/null,force-share=3Doff [1] 27869 segmentation fault (core dumped) ./qemu-img info -U --image-opts driver=3Dfile,filename=3D/dev/null,force-share=3Doff Fix this by using qdict_get_str() and comparing the value as a string. Also, when adding a force-share value to the QDict, add it as a string so it fits the rest of the dict. Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Message-id: 20180502202051.15493-3-mreitz@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- qemu-img.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 62b29e7feb..60e45ec103 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -277,12 +277,12 @@ static BlockBackend *img_open_opts(const char *optstr, options =3D qemu_opts_to_qdict(opts, NULL); if (force_share) { if (qdict_haskey(options, BDRV_OPT_FORCE_SHARE) - && !qdict_get_bool(options, BDRV_OPT_FORCE_SHARE)) { + && strcmp(qdict_get_str(options, BDRV_OPT_FORCE_SHARE), "on"))= { error_report("--force-share/-U conflicts with image options"); qobject_unref(options); return NULL; } - qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true); + qdict_put_str(options, BDRV_OPT_FORCE_SHARE, "on"); } blk =3D blk_new_open(NULL, NULL, options, flags, &local_err); if (!blk) { --=20 2.13.6 From nobody Sat Apr 20 09:40:33 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1526400115273367.6475861063957; Tue, 15 May 2018 09:01:55 -0700 (PDT) Received: from localhost ([::1]:44554 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIcOW-0007QV-RQ for importer@patchew.org; Tue, 15 May 2018 12:01:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39327) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIc4q-0007B0-C9 for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIc4p-0001xR-Cz for qemu-devel@nongnu.org; Tue, 15 May 2018 11:41:24 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:40588 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIc4h-0001pT-A1; Tue, 15 May 2018 11:41:15 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D5A9E81FE152; Tue, 15 May 2018 15:41:14 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-164.ams2.redhat.com [10.36.117.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 33EA82026E0E; Tue, 15 May 2018 15:41:14 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Tue, 15 May 2018 17:40:33 +0200 Message-Id: <20180515154033.19899-38-kwolf@redhat.com> In-Reply-To: <20180515154033.19899-1-kwolf@redhat.com> References: <20180515154033.19899-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 15 May 2018 15:41:14 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 15 May 2018 15:41:14 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 37/37] iotests: Add test for -U/force-share conflicts X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Max Reitz Signed-off-by: Max Reitz Message-id: 20180502202051.15493-4-mreitz@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- tests/qemu-iotests/153 | 17 +++++++++++++++++ tests/qemu-iotests/153.out | 16 ++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153 index a0fd815483..ec508c758f 100755 --- a/tests/qemu-iotests/153 +++ b/tests/qemu-iotests/153 @@ -242,6 +242,23 @@ _run_cmd $QEMU_IO "${TEST_IMG}" -c 'write 0 512' =20 _cleanup_qemu =20 +echo +echo "=3D=3D Detecting -U and force-share conflicts =3D=3D" + +echo +echo 'No conflict:' +$QEMU_IMG info -U --image-opts driver=3Dnull-co,force-share=3Don +echo +echo 'Conflict:' +$QEMU_IMG info -U --image-opts driver=3Dnull-co,force-share=3Doff + +echo +echo 'No conflict:' +$QEMU_IO -c 'open -r -U -o driver=3Dnull-co,force-share=3Don' +echo +echo 'Conflict:' +$QEMU_IO -c 'open -r -U -o driver=3Dnull-co,force-share=3Doff' + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/153.out b/tests/qemu-iotests/153.out index bb721cb747..2510762ba1 100644 --- a/tests/qemu-iotests/153.out +++ b/tests/qemu-iotests/153.out @@ -399,4 +399,20 @@ Is another process using the image? Closing the other =20 _qemu_io_wrapper TEST_DIR/t.qcow2 -c write 0 512 + +=3D=3D Detecting -U and force-share conflicts =3D=3D + +No conflict: +image: null-co:// +file format: null-co +virtual size: 1.0G (1073741824 bytes) +disk size: unavailable + +Conflict: +qemu-img: --force-share/-U conflicts with image options + +No conflict: + +Conflict: +-U conflicts with image options *** done --=20 2.13.6