From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1745013345; cv=none; d=zohomail.com; s=zohoarc; b=YnNPekZCtOcVupHsYDs/GnDDVXl8+PgvMIfe0Gtfnnsl3bWtksUqorw1ki33X5QCKkYzh2smWSlfRpOT9mSvovH8LCBbeZBHngBhkuMEKMlmOp2jPArHRFXbxA++P19QGWGyeX6nu5DVbBxI/0DCj0haChHB4RuizFCw2wR7rQM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1745013345; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5vh+totLyjjjuU0FdW+Y4coOji7NsokPbvcFSdbSw4w=; b=cepFi7AvityJNZg2xbh3f66ms8nKOR592GkwkhkuWEJwwRF2YFIe+Pk35SdqFMjWiPk0KmdsTT2D8D1R42a6rz1HuM2vMgMaTOka4Rw+E4fRfLTRK+6ysGQ8C6W9up5R4BCwmcZhLUFXrhobvyfFBlcwsqxvJOIXTqCL/0pANtM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1745013345969623.3786105444175; Fri, 18 Apr 2025 14:55:45 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5tfX-0003Ix-FJ; Fri, 18 Apr 2025 17:54:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5tfT-0003Hj-Fp for qemu-devel@nongnu.org; Fri, 18 Apr 2025 17:54:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5tfP-00037Y-Pa for qemu-devel@nongnu.org; Fri, 18 Apr 2025 17:54:39 -0400 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-120-1yCGPxkeOC6JLhO6ckBNDg-1; Fri, 18 Apr 2025 17:54:30 -0400 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E1AB41956048; Fri, 18 Apr 2025 21:54:27 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 721FA180177D; Fri, 18 Apr 2025 21:54:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1745013273; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5vh+totLyjjjuU0FdW+Y4coOji7NsokPbvcFSdbSw4w=; b=IVdJUEUzswy4LrLSJvnbKCQUTQnXXaa9l7sD+Hn3ItP2YDvGEo1MpSv9Gj8cq8qFPiPhKT g31UHFxaYJBPpODTB1zC2X15/RyfdR/CjHfCuXx076WBqnppOSsz6PXh8UuvzkTLK7ZpfN 758w4lgoK206zNs9ETcFMPaBZi3H1W0= X-MC-Unique: 1yCGPxkeOC6JLhO6ckBNDg-1 X-Mimecast-MFC-AGG-ID: 1yCGPxkeOC6JLhO6ckBNDg_1745013268 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@redhat.com, Kevin Wolf , Hanna Reitz , John Snow , Vladimir Sementsov-Ogievskiy , Fam Zheng , Ronnie Sahlberg , Paolo Bonzini , Peter Lieven , "Denis V. Lunev" , Alberto Garcia , Ilya Dryomov , Stefan Weil , integration@gluster.org (open list:GLUSTER) Subject: [PATCH v2.5 01/11] block: Expand block status mode from bool to flags Date: Fri, 18 Apr 2025 16:52:46 -0500 Message-ID: <20250418215412.404558-3-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1745013349184019000 Content-Type: text/plain; charset="utf-8" This patch is purely mechanical, changing bool want_zero into an unsigned int for bitwise-or of flags. As of this patch, all implementations are unchanged (the old want_zero=3D=3Dtrue is now mode=3D=3DBDRV_WANT_PRECISE which is a superset of BDRV_WANT_ZERO); but the callers in io.c that used to pass want_zero=3D=3Dfalse are now prepared for future driver changes that can now distinguish bewteen BDRV_WANT_ZERO vs. BDRV_WANT_ALLOCATED. The next patch will actually change the file-posix driver along those lines, now that we have more-specific hints. As for the background why this patch is useful: right now, the file-posix driver recognizes that if allocation is being queried, the entire image can be reported as allocated (there is no backing file to refer to) - but this throws away information on whether the entire image reads as zero (trivially true if lseek(SEEK_HOLE) at offset 0 returns -ENXIO, a bit more complicated to prove if the raw file was created with 'qemu-img create' since we intentionally allocate a small chunk of all-zero data to help with alignment probing). Later patches will add a generic algorithm for seeing if an entire file reads as zeroes. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- In response to Stefan's ask for what it would look like as a bitmask of flags instead of an enum. Only the first two patches of the series change. block/coroutines.h | 4 +-- include/block/block-common.h | 11 +++++++ include/block/block_int-common.h | 27 +++++++++-------- include/block/block_int-io.h | 4 +-- block/io.c | 51 ++++++++++++++++---------------- block/blkdebug.c | 6 ++-- block/copy-before-write.c | 4 +-- block/file-posix.c | 4 +-- block/gluster.c | 4 +-- block/iscsi.c | 6 ++-- block/nbd.c | 4 +-- block/null.c | 6 ++-- block/parallels.c | 6 ++-- block/qcow.c | 2 +- block/qcow2.c | 6 ++-- block/qed.c | 6 ++-- block/quorum.c | 4 +-- block/raw-format.c | 4 +-- block/rbd.c | 6 ++-- block/snapshot-access.c | 4 +-- block/vdi.c | 4 +-- block/vmdk.c | 2 +- block/vpc.c | 2 +- block/vvfat.c | 6 ++-- tests/unit/test-block-iothread.c | 2 +- 25 files changed, 99 insertions(+), 86 deletions(-) diff --git a/block/coroutines.h b/block/coroutines.h index 79e5efbf752..892646bb7aa 100644 --- a/block/coroutines.h +++ b/block/coroutines.h @@ -47,7 +47,7 @@ int coroutine_fn GRAPH_RDLOCK bdrv_co_common_block_status_above(BlockDriverState *bs, BlockDriverState *base, bool include_base, - bool want_zero, + unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, @@ -78,7 +78,7 @@ int co_wrapper_mixed_bdrv_rdlock bdrv_common_block_status_above(BlockDriverState *bs, BlockDriverState *base, bool include_base, - bool want_zero, + unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, diff --git a/include/block/block-common.h b/include/block/block-common.h index 0b831ef87b1..c8c626daeaa 100644 --- a/include/block/block-common.h +++ b/include/block/block-common.h @@ -333,6 +333,17 @@ typedef enum { #define BDRV_BLOCK_RECURSE 0x40 #define BDRV_BLOCK_COMPRESSED 0x80 +/* + * Block status hints: the bitwise-or of these flags emphasize what + * the caller hopes to learn, and some drivers may be able to give + * faster answers by doing less work when the hint permits. + */ +#define BDRV_WANT_ZERO BDRV_BLOCK_ZERO +#define BDRV_WANT_OFFSET_VALID BDRV_BLOCK_OFFSET_VALID +#define BDRV_WANT_ALLOCATED BDRV_BLOCK_ALLOCATED +#define BDRV_WANT_PRECISE (BDRV_WANT_ZERO | BDRV_WANT_OFFSET_VALID |= \ + BDRV_WANT_OFFSET_VALID) + typedef QTAILQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry) BlockReopenQu= eue; typedef struct BDRVReopenState { diff --git a/include/block/block_int-common.h b/include/block/block_int-com= mon.h index ebb4e56a503..a9c0daa2a4d 100644 --- a/include/block/block_int-common.h +++ b/include/block/block_int-common.h @@ -608,15 +608,16 @@ struct BlockDriver { * according to the current layer, and should only need to set * BDRV_BLOCK_DATA, BDRV_BLOCK_ZERO, BDRV_BLOCK_OFFSET_VALID, * and/or BDRV_BLOCK_RAW; if the current layer defers to a backing - * layer, the result should be 0 (and not BDRV_BLOCK_ZERO). See - * block.h for the overall meaning of the bits. As a hint, the - * flag want_zero is true if the caller cares more about precise - * mappings (favor accurate _OFFSET_VALID/_ZERO) or false for - * overall allocation (favor larger *pnum, perhaps by reporting - * _DATA instead of _ZERO). The block layer guarantees input - * clamped to bdrv_getlength() and aligned to request_alignment, - * as well as non-NULL pnum, map, and file; in turn, the driver - * must return an error or set pnum to an aligned non-zero value. + * layer, the result should be 0 (and not BDRV_BLOCK_ZERO). The + * caller will synthesize BDRV_BLOCK_ALLOCATED based on the + * non-zero results. See block.h for the overall meaning of the + * bits. As a hint, the flags in @mode may include a bitwise-or + * of BDRV_WANT_ALLOCATED, BDRV_WANT_OFFSET_VALID, or + * BDRV_WANT_ZERO based on what the caller is looking for in the + * results. The block layer guarantees input clamped to + * bdrv_getlength() and aligned to request_alignment, as well as + * non-NULL pnum, map, and file; in turn, the driver must return + * an error or set pnum to an aligned non-zero value. * * Note that @bytes is just a hint on how big of a region the * caller wants to inspect. It is not a limit on *pnum. @@ -628,8 +629,8 @@ struct BlockDriver { * to clamping *pnum for return to its caller. */ int coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_block_status)( - BlockDriverState *bs, - bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum, + BlockDriverState *bs, unsigned int mode, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file); /* @@ -653,8 +654,8 @@ struct BlockDriver { QEMUIOVector *qiov, size_t qiov_offset); int coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_snapshot_block_status)( - BlockDriverState *bs, bool want_zero, int64_t offset, int64_t byte= s, - int64_t *pnum, int64_t *map, BlockDriverState **file); + BlockDriverState *bs, unsigned int mode, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **fil= e); int coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_pdiscard_snapshot)( BlockDriverState *bs, int64_t offset, int64_t bytes); diff --git a/include/block/block_int-io.h b/include/block/block_int-io.h index 4a7cf2b4fdc..4f94eb3c5a2 100644 --- a/include/block/block_int-io.h +++ b/include/block/block_int-io.h @@ -38,8 +38,8 @@ int coroutine_fn GRAPH_RDLOCK bdrv_co_preadv_snapshot(BdrvChild *child, int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset); int coroutine_fn GRAPH_RDLOCK bdrv_co_snapshot_block_status( - BlockDriverState *bs, bool want_zero, int64_t offset, int64_t bytes, - int64_t *pnum, int64_t *map, BlockDriverState **file); + BlockDriverState *bs, unsigned int mode, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file); int coroutine_fn GRAPH_RDLOCK bdrv_co_pdiscard_snapshot(BlockDriverState *= bs, int64_t offset, int64_t bytes); diff --git a/block/io.c b/block/io.c index 1ba8d1aeea1..f625bf2ac8b 100644 --- a/block/io.c +++ b/block/io.c @@ -2364,10 +2364,8 @@ int bdrv_flush_all(void) * Drivers not implementing the functionality are assumed to not support * backing files, hence all their sectors are reported as allocated. * - * If 'want_zero' is true, the caller is querying for mapping - * purposes, with a focus on valid BDRV_BLOCK_OFFSET_VALID, _DATA, and - * _ZERO where possible; otherwise, the result favors larger 'pnum', - * with a focus on accurate BDRV_BLOCK_ALLOCATED. + * 'mode' serves as a hint as to which results are favored; see enum + * BlockStatusMode for details of the supported modes. * * If 'offset' is beyond the end of the disk image the return value is * BDRV_BLOCK_EOF and 'pnum' is set to 0. @@ -2387,7 +2385,7 @@ int bdrv_flush_all(void) * set to the host mapping and BDS corresponding to the guest offset. */ static int coroutine_fn GRAPH_RDLOCK -bdrv_co_do_block_status(BlockDriverState *bs, bool want_zero, +bdrv_co_do_block_status(BlockDriverState *bs, unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **fi= le) { @@ -2476,7 +2474,7 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool wa= nt_zero, local_file =3D bs; local_map =3D aligned_offset; } else { - ret =3D bs->drv->bdrv_co_block_status(bs, want_zero, aligned_o= ffset, + ret =3D bs->drv->bdrv_co_block_status(bs, mode, aligned_offset, aligned_bytes, pnum, &loca= l_map, &local_file); @@ -2488,10 +2486,10 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool = want_zero, * the cache requires an RCU update, so double check here to a= void * such an update if possible. * - * Check want_zero, because we only want to update the cache w= hen we + * Check mode, because we only want to update the cache when we * have accurate information about what is zero and what is da= ta. */ - if (want_zero && + if (mode =3D=3D BDRV_WANT_PRECISE && ret =3D=3D (BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID) && QLIST_EMPTY(&bs->children)) { @@ -2548,7 +2546,7 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool wa= nt_zero, if (ret & BDRV_BLOCK_RAW) { assert(ret & BDRV_BLOCK_OFFSET_VALID && local_file); - ret =3D bdrv_co_do_block_status(local_file, want_zero, local_map, + ret =3D bdrv_co_do_block_status(local_file, mode, local_map, *pnum, pnum, &local_map, &local_file= ); goto out; } @@ -2560,7 +2558,7 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool wa= nt_zero, if (!cow_bs) { ret |=3D BDRV_BLOCK_ZERO; - } else if (want_zero) { + } else if (mode =3D=3D BDRV_WANT_PRECISE) { int64_t size2 =3D bdrv_co_getlength(cow_bs); if (size2 >=3D 0 && offset >=3D size2) { @@ -2569,14 +2567,14 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool = want_zero, } } - if (want_zero && ret & BDRV_BLOCK_RECURSE && + if (mode =3D=3D BDRV_WANT_PRECISE && ret & BDRV_BLOCK_RECURSE && local_file && local_file !=3D bs && (ret & BDRV_BLOCK_DATA) && !(ret & BDRV_BLOCK_ZERO) && (ret & BDRV_BLOCK_OFFSET_VALID)) { int64_t file_pnum; int ret2; - ret2 =3D bdrv_co_do_block_status(local_file, want_zero, local_map, + ret2 =3D bdrv_co_do_block_status(local_file, mode, local_map, *pnum, &file_pnum, NULL, NULL); if (ret2 >=3D 0) { /* Ignore errors. This is just providing extra information, it @@ -2627,7 +2625,7 @@ int coroutine_fn bdrv_co_common_block_status_above(BlockDriverState *bs, BlockDriverState *base, bool include_base, - bool want_zero, + unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, @@ -2654,7 +2652,7 @@ bdrv_co_common_block_status_above(BlockDriverState *b= s, return 0; } - ret =3D bdrv_co_do_block_status(bs, want_zero, offset, bytes, pnum, + ret =3D bdrv_co_do_block_status(bs, mode, offset, bytes, pnum, map, file); ++*depth; if (ret < 0 || *pnum =3D=3D 0 || ret & BDRV_BLOCK_ALLOCATED || bs =3D= =3D base) { @@ -2671,7 +2669,7 @@ bdrv_co_common_block_status_above(BlockDriverState *b= s, for (p =3D bdrv_filter_or_cow_bs(bs); include_base || p !=3D base; p =3D bdrv_filter_or_cow_bs(p)) { - ret =3D bdrv_co_do_block_status(p, want_zero, offset, bytes, pnum, + ret =3D bdrv_co_do_block_status(p, mode, offset, bytes, pnum, map, file); ++*depth; if (ret < 0) { @@ -2734,7 +2732,8 @@ int coroutine_fn bdrv_co_block_status_above(BlockDriv= erState *bs, BlockDriverState **file) { IO_CODE(); - return bdrv_co_common_block_status_above(bs, base, false, true, offset, + return bdrv_co_common_block_status_above(bs, base, false, + BDRV_WANT_PRECISE, offset, bytes, pnum, map, file, NULL); } @@ -2765,8 +2764,9 @@ int coroutine_fn bdrv_co_is_zero_fast(BlockDriverStat= e *bs, int64_t offset, return 1; } - ret =3D bdrv_co_common_block_status_above(bs, NULL, false, false, offs= et, - bytes, &pnum, NULL, NULL, NULL= ); + ret =3D bdrv_co_common_block_status_above(bs, NULL, false, BDRV_WANT_Z= ERO, + offset, bytes, &pnum, NULL, NU= LL, + NULL); if (ret < 0) { return ret; @@ -2782,9 +2782,9 @@ int coroutine_fn bdrv_co_is_allocated(BlockDriverStat= e *bs, int64_t offset, int64_t dummy; IO_CODE(); - ret =3D bdrv_co_common_block_status_above(bs, bs, true, false, offset, - bytes, pnum ? pnum : &dummy, N= ULL, - NULL, NULL); + ret =3D bdrv_co_common_block_status_above(bs, bs, true, BDRV_WANT_ALLO= CATED, + offset, bytes, pnum ? pnum : &= dummy, + NULL, NULL, NULL); if (ret < 0) { return ret; } @@ -2817,7 +2817,8 @@ int coroutine_fn bdrv_co_is_allocated_above(BlockDriv= erState *bs, int ret; IO_CODE(); - ret =3D bdrv_co_common_block_status_above(bs, base, include_base, fals= e, + ret =3D bdrv_co_common_block_status_above(bs, base, include_base, + BDRV_WANT_ALLOCATED, offset, bytes, pnum, NULL, NUL= L, &depth); if (ret < 0) { @@ -3709,8 +3710,8 @@ bdrv_co_preadv_snapshot(BdrvChild *child, int64_t off= set, int64_t bytes, } int coroutine_fn -bdrv_co_snapshot_block_status(BlockDriverState *bs, - bool want_zero, int64_t offset, int64_t byte= s, +bdrv_co_snapshot_block_status(BlockDriverState *bs, unsigned int mode, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { @@ -3728,7 +3729,7 @@ bdrv_co_snapshot_block_status(BlockDriverState *bs, } bdrv_inc_in_flight(bs); - ret =3D drv->bdrv_co_snapshot_block_status(bs, want_zero, offset, byte= s, + ret =3D drv->bdrv_co_snapshot_block_status(bs, mode, offset, bytes, pnum, map, file); bdrv_dec_in_flight(bs); diff --git a/block/blkdebug.c b/block/blkdebug.c index 1c1967f8e0a..c54aee0c84b 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -751,9 +751,9 @@ blkdebug_co_pdiscard(BlockDriverState *bs, int64_t offs= et, int64_t bytes) } static int coroutine_fn GRAPH_RDLOCK -blkdebug_co_block_status(BlockDriverState *bs, bool want_zero, int64_t off= set, - int64_t bytes, int64_t *pnum, int64_t *map, - BlockDriverState **file) +blkdebug_co_block_status(BlockDriverState *bs, unsigned int mode, + int64_t offset, int64_t bytes, int64_t *pnum, + int64_t *map, BlockDriverState **file) { int err; diff --git a/block/copy-before-write.c b/block/copy-before-write.c index fd470f5f926..2badb3a8856 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -291,8 +291,8 @@ cbw_co_preadv_snapshot(BlockDriverState *bs, int64_t of= fset, int64_t bytes, } static int coroutine_fn GRAPH_RDLOCK -cbw_co_snapshot_block_status(BlockDriverState *bs, - bool want_zero, int64_t offset, int64_t bytes, +cbw_co_snapshot_block_status(BlockDriverState *bs, unsigned int mode, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { diff --git a/block/file-posix.c b/block/file-posix.c index 56d1972d156..91deb5bf5af 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -3201,7 +3201,7 @@ static int find_allocation(BlockDriverState *bs, off_= t start, * well exceed it. */ static int coroutine_fn raw_co_block_status(BlockDriverState *bs, - bool want_zero, + unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, @@ -3217,7 +3217,7 @@ static int coroutine_fn raw_co_block_status(BlockDriv= erState *bs, return ret; } - if (!want_zero) { + if (mode !=3D BDRV_WANT_PRECISE) { *pnum =3D bytes; *map =3D offset; *file =3D bs; diff --git a/block/gluster.c b/block/gluster.c index c6d25ae7335..8197b0ecefa 100644 --- a/block/gluster.c +++ b/block/gluster.c @@ -1465,7 +1465,7 @@ exit: * (Based on raw_co_block_status() from file-posix.c.) */ static int coroutine_fn qemu_gluster_co_block_status(BlockDriverState *bs, - bool want_zero, + unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, @@ -1482,7 +1482,7 @@ static int coroutine_fn qemu_gluster_co_block_status(= BlockDriverState *bs, return ret; } - if (!want_zero) { + if (mode !=3D BDRV_WANT_PRECISE) { *pnum =3D bytes; *map =3D offset; *file =3D bs; diff --git a/block/iscsi.c b/block/iscsi.c index 2f0f4dac097..15b96ee8800 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -694,9 +694,9 @@ out_unlock: static int coroutine_fn iscsi_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t offs= et, - int64_t bytes, int64_t *pnum, - int64_t *map, + unsigned int mode, + int64_t offset, int64_t byte= s, + int64_t *pnum, int64_t *map, BlockDriverState **file) { IscsiLun *iscsilun =3D bs->opaque; diff --git a/block/nbd.c b/block/nbd.c index 887841bc813..d5a2b21c6d1 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -1397,8 +1397,8 @@ nbd_client_co_pdiscard(BlockDriverState *bs, int64_t = offset, int64_t bytes) } static int coroutine_fn GRAPH_RDLOCK nbd_client_co_block_status( - BlockDriverState *bs, bool want_zero, int64_t offset, int64_t byte= s, - int64_t *pnum, int64_t *map, BlockDriverState **file) + BlockDriverState *bs, unsigned int mode, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **fil= e) { int ret, request_ret; NBDExtent64 extent =3D { 0 }; diff --git a/block/null.c b/block/null.c index dc0b1fdbd9b..4e448d593d7 100644 --- a/block/null.c +++ b/block/null.c @@ -227,9 +227,9 @@ static int null_reopen_prepare(BDRVReopenState *reopen_= state, } static int coroutine_fn null_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t offse= t, - int64_t bytes, int64_t *pnum, - int64_t *map, + unsigned int mode, + int64_t offset, int64_t bytes, + int64_t *pnum, int64_t *map, BlockDriverState **file) { BDRVNullState *s =3D bs->opaque; diff --git a/block/parallels.c b/block/parallels.c index 347ca127f34..3a375e2a8ab 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -416,9 +416,9 @@ parallels_co_flush_to_os(BlockDriverState *bs) } static int coroutine_fn GRAPH_RDLOCK -parallels_co_block_status(BlockDriverState *bs, bool want_zero, int64_t of= fset, - int64_t bytes, int64_t *pnum, int64_t *map, - BlockDriverState **file) +parallels_co_block_status(BlockDriverState *bs, unsigned int mode, + int64_t offset, int64_t bytes, int64_t *pnum, + int64_t *map, BlockDriverState **file) { BDRVParallelsState *s =3D bs->opaque; int count; diff --git a/block/qcow.c b/block/qcow.c index da8ad4d2430..8a3e7591a92 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -530,7 +530,7 @@ get_cluster_offset(BlockDriverState *bs, uint64_t offse= t, int allocate, } static int coroutine_fn GRAPH_RDLOCK -qcow_co_block_status(BlockDriverState *bs, bool want_zero, +qcow_co_block_status(BlockDriverState *bs, unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { diff --git a/block/qcow2.c b/block/qcow2.c index 7774e7f0909..66fba89b414 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2141,9 +2141,9 @@ static void qcow2_join_options(QDict *options, QDict = *old_options) } static int coroutine_fn GRAPH_RDLOCK -qcow2_co_block_status(BlockDriverState *bs, bool want_zero, int64_t offset, - int64_t count, int64_t *pnum, int64_t *map, - BlockDriverState **file) +qcow2_co_block_status(BlockDriverState *bs, unsigned int mode, + int64_t offset, int64_t count, int64_t *pnum, + int64_t *map, BlockDriverState **file) { BDRVQcow2State *s =3D bs->opaque; uint64_t host_offset; diff --git a/block/qed.c b/block/qed.c index ac24449ffb3..4a36fb39294 100644 --- a/block/qed.c +++ b/block/qed.c @@ -833,9 +833,9 @@ fail: } static int coroutine_fn GRAPH_RDLOCK -bdrv_qed_co_block_status(BlockDriverState *bs, bool want_zero, int64_t pos, - int64_t bytes, int64_t *pnum, int64_t *map, - BlockDriverState **file) +bdrv_qed_co_block_status(BlockDriverState *bs, unsigned int mode, + int64_t pos, int64_t bytes, int64_t *pnum, + int64_t *map, BlockDriverState **file) { BDRVQEDState *s =3D bs->opaque; size_t len =3D MIN(bytes, SIZE_MAX); diff --git a/block/quorum.c b/block/quorum.c index 30747a6df93..ed8ce801ee3 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -1226,7 +1226,7 @@ static void quorum_child_perm(BlockDriverState *bs, B= drvChild *c, * region contains zeroes, and BDRV_BLOCK_DATA otherwise. */ static int coroutine_fn GRAPH_RDLOCK -quorum_co_block_status(BlockDriverState *bs, bool want_zero, +quorum_co_block_status(BlockDriverState *bs, unsigned int mode, int64_t offset, int64_t count, int64_t *pnum, int64_t *map, BlockDriverState **fil= e) { @@ -1238,7 +1238,7 @@ quorum_co_block_status(BlockDriverState *bs, bool wan= t_zero, for (i =3D 0; i < s->num_children; i++) { int64_t bytes; ret =3D bdrv_co_common_block_status_above(s->children[i]->bs, NULL= , false, - want_zero, offset, count, + mode, offset, count, &bytes, NULL, NULL, NULL); if (ret < 0) { quorum_report_bad(QUORUM_OP_TYPE_READ, offset, count, diff --git a/block/raw-format.c b/block/raw-format.c index e08526e2eca..df16ac1ea25 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -283,8 +283,8 @@ fail: } static int coroutine_fn GRAPH_RDLOCK -raw_co_block_status(BlockDriverState *bs, bool want_zero, int64_t offset, - int64_t bytes, int64_t *pnum, int64_t *map, +raw_co_block_status(BlockDriverState *bs, unsigned int mode, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t = *map, BlockDriverState **file) { BDRVRawState *s =3D bs->opaque; diff --git a/block/rbd.c b/block/rbd.c index af984fb7db4..4f3d42a8e7f 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -1504,9 +1504,9 @@ static int qemu_rbd_diff_iterate_cb(uint64_t offs, si= ze_t len, } static int coroutine_fn qemu_rbd_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t o= ffset, - int64_t bytes, int64_t *p= num, - int64_t *map, + unsigned int mode, + int64_t offset, int64_t b= ytes, + int64_t *pnum, int64_t *m= ap, BlockDriverState **file) { BDRVRBDState *s =3D bs->opaque; diff --git a/block/snapshot-access.c b/block/snapshot-access.c index 71ac83c01f0..17ed2402db8 100644 --- a/block/snapshot-access.c +++ b/block/snapshot-access.c @@ -41,11 +41,11 @@ snapshot_access_co_preadv_part(BlockDriverState *bs, static int coroutine_fn GRAPH_RDLOCK snapshot_access_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t offset, + unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { - return bdrv_co_snapshot_block_status(bs->file->bs, want_zero, offset, + return bdrv_co_snapshot_block_status(bs->file->bs, mode, offset, bytes, pnum, map, file); } diff --git a/block/vdi.c b/block/vdi.c index a2da6ecab01..3ddc62a5690 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -523,8 +523,8 @@ static int vdi_reopen_prepare(BDRVReopenState *state, } static int coroutine_fn GRAPH_RDLOCK -vdi_co_block_status(BlockDriverState *bs, bool want_zero, int64_t offset, - int64_t bytes, int64_t *pnum, int64_t *map, +vdi_co_block_status(BlockDriverState *bs, unsigned int mode, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t = *map, BlockDriverState **file) { BDRVVdiState *s =3D (BDRVVdiState *)bs->opaque; diff --git a/block/vmdk.c b/block/vmdk.c index 2adec499122..9c7ab037e14 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1777,7 +1777,7 @@ static inline uint64_t vmdk_find_offset_in_cluster(Vm= dkExtent *extent, } static int coroutine_fn GRAPH_RDLOCK -vmdk_co_block_status(BlockDriverState *bs, bool want_zero, +vmdk_co_block_status(BlockDriverState *bs, unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { diff --git a/block/vpc.c b/block/vpc.c index 0309e319f60..801ff5793f8 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -726,7 +726,7 @@ fail: } static int coroutine_fn GRAPH_RDLOCK -vpc_co_block_status(BlockDriverState *bs, bool want_zero, +vpc_co_block_status(BlockDriverState *bs, unsigned int mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) diff --git a/block/vvfat.c b/block/vvfat.c index 91d69b3cc83..814796d9185 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -3134,9 +3134,9 @@ vvfat_co_pwritev(BlockDriverState *bs, int64_t offset= , int64_t bytes, } static int coroutine_fn vvfat_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t offs= et, - int64_t bytes, int64_t *n, - int64_t *map, + unsigned int mode, + int64_t offset, int64_t byte= s, + int64_t *n, int64_t *map, BlockDriverState **file) { *n =3D bytes; diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothr= ead.c index 2b358eaaa82..e26b3be5939 100644 --- a/tests/unit/test-block-iothread.c +++ b/tests/unit/test-block-iothread.c @@ -63,7 +63,7 @@ bdrv_test_co_truncate(BlockDriverState *bs, int64_t offse= t, bool exact, } static int coroutine_fn bdrv_test_co_block_status(BlockDriverState *bs, - bool want_zero, + unsigned int mode, int64_t offset, int64_t = count, int64_t *pnum, int64_t *= map, BlockDriverState **file) --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915375; cv=none; d=zohomail.com; s=zohoarc; b=CHE8je6zQnEuQDTfPHJ9UEp9e+L/jvC3rkTr2INkscc5sIBHfH92WmHixTE/y1q2SDRcOZNQJyXVkZu7L0uDYiF+wzDkA18N+Udo0mmeI/98RWW+0X9AAE6ECURHDSa4bS8s5Lq/gbzlybd/bEav0kb3rDaOtZib1oWt8FuNi38= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915375; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SEEczVSQ8XxMy3TXTJCROAGAwA6Jyx1FD/YAkL3ooOk=; b=XacoT/kdZ5/TiFWUh5GD9hlqLUC5UU3yMb2VmFrplxbzFuR8eRlUuG+4llG+LVLD364w+vWdjQTdkVUpM2G1GKxgRgE1tb3HdPwJmPx8cSZccmZHxrmuPtWHt1WF2pIbBxlgbNJXU42pAb02Nd0irtKweaEIaKfkPnd6Len1Euo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915375159305.98111952737963; Thu, 17 Apr 2025 11:42:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UC4-0004TN-RN; Thu, 17 Apr 2025 14:42:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBb-00044p-Kr for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBX-0002td-Da for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:07 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-655-LKl-i6VDOgCmc-aodh99Wg-1; Thu, 17 Apr 2025 14:41:57 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 74D1419560A5; Thu, 17 Apr 2025 18:41:50 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CC8ED1956095; Thu, 17 Apr 2025 18:41:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915321; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SEEczVSQ8XxMy3TXTJCROAGAwA6Jyx1FD/YAkL3ooOk=; b=VH75kUm9KtXA45m5DaRbppKeve152FcKn8hFmnhPJn1FTzCITD9fZpaNJF9DIDWIzflXgs GBHGNIxfggzbr46uLXlMhtxcfsGW0l5rq7CM7L+1RQta5KdfIW49Y9s+RIxgr6aN0CIGm/ qqSc6NaS61LplGAbEvBORHIqCApTwgs= X-MC-Unique: LKl-i6VDOgCmc-aodh99Wg-1 X-Mimecast-MFC-AGG-ID: LKl-i6VDOgCmc-aodh99Wg_1744915316 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, Kevin Wolf , Hanna Reitz , John Snow , Stefan Hajnoczi , Fam Zheng , Ronnie Sahlberg , Paolo Bonzini , Peter Lieven , "Denis V. Lunev" , Alberto Garcia , Ilya Dryomov , Stefan Weil , integration@gluster.org (open list:GLUSTER) Subject: [PATCH v2 01/11] block: Expand block status mode from bool to enum Date: Thu, 17 Apr 2025 13:39:06 -0500 Message-ID: <20250417184133.105746-14-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915377540019000 Content-Type: text/plain; charset="utf-8" This patch is purely mechanical, changing bool want_zero into a new enum BlockStatusMode. As of this patch, all implementations are unchanged (the old want_zero=3D=3Dtrue is now mode=3D=3DBDRV_BSTAT_PRECISE), but the callers in io.c are set up so that future patches will be able to differente between whether the caller cares more about allocation or about reads-as-zero, for driver implementations that will actually want to behave differently for those more-specific hints. As for the background why this patch is useful: right now, the file-posix driver recognizes that if allocation is being queried, the entire image can be reported as allocated (there is no backing file to refer to) - but this throws away information on whether the entire image reads as zero (trivially true if lseek(SEEK_HOLE) at offset 0 returns -ENXIO, a bit more complicated to prove if the raw file was created with 'qemu-img create' since we intentionally allocate a small chunk of all-zero data to help with alignment probing). The next patches will add a generic algorithm for seeing if an entire file reads as zeroes, as well as tweak the file-posix driver to react to the new hints. Signed-off-by: Eric Blake --- block/coroutines.h | 4 +-- include/block/block-common.h | 26 ++++++++++++++++ include/block/block_int-common.h | 25 +++++++++------- include/block/block_int-io.h | 4 +-- block/io.c | 51 ++++++++++++++++---------------- block/blkdebug.c | 6 ++-- block/copy-before-write.c | 4 +-- block/file-posix.c | 4 +-- block/gluster.c | 4 +-- block/iscsi.c | 6 ++-- block/nbd.c | 4 +-- block/null.c | 6 ++-- block/parallels.c | 6 ++-- block/qcow.c | 2 +- block/qcow2.c | 6 ++-- block/qed.c | 6 ++-- block/quorum.c | 4 +-- block/raw-format.c | 4 +-- block/rbd.c | 6 ++-- block/snapshot-access.c | 4 +-- block/vdi.c | 4 +-- block/vmdk.c | 2 +- block/vpc.c | 2 +- block/vvfat.c | 6 ++-- tests/unit/test-block-iothread.c | 2 +- 25 files changed, 114 insertions(+), 84 deletions(-) diff --git a/block/coroutines.h b/block/coroutines.h index 79e5efbf752..c8323aa67e6 100644 --- a/block/coroutines.h +++ b/block/coroutines.h @@ -47,7 +47,7 @@ int coroutine_fn GRAPH_RDLOCK bdrv_co_common_block_status_above(BlockDriverState *bs, BlockDriverState *base, bool include_base, - bool want_zero, + enum BlockStatusMode mode, int64_t offset, int64_t bytes, int64_t *pnum, @@ -78,7 +78,7 @@ int co_wrapper_mixed_bdrv_rdlock bdrv_common_block_status_above(BlockDriverState *bs, BlockDriverState *base, bool include_base, - bool want_zero, + enum BlockStatusMode mode, int64_t offset, int64_t bytes, int64_t *pnum, diff --git a/include/block/block-common.h b/include/block/block-common.h index 0b831ef87b1..619e75b9c8d 100644 --- a/include/block/block-common.h +++ b/include/block/block-common.h @@ -508,6 +508,32 @@ enum BdrvChildRoleBits { | BDRV_CHILD_PRIMARY, }; +/* Modes for block status calls */ +enum BlockStatusMode { + /* + * Status should be as accurate as possible: _OFFSET_VALID + * and_OFFSET_ZERO should each be set where efficiently possible, + * extents may be smaller, and iteration through the entire block + * device may take more calls. + */ + BDRV_BSTAT_PRECISE, + + /* + * The caller is primarily concerned about overall allocation: + * favor larger *pnum, perhaps by coalescing extents and reporting + * _DATA instead of _ZERO, and without needing to read data or + * bothering with _OFFSET_VALID. + */ + BDRV_BSTAT_ALLOCATED, + + /* + * The caller is primarily concerned about whether the device + * reads as zero: favor a result of _ZERO, even if it requires + * reading a few sectors to verify, without needing _OFFSET_VALID. + */ + BDRV_BSTAT_ZERO, +}; + /* Mask of BdrvChildRoleBits values */ typedef unsigned int BdrvChildRole; diff --git a/include/block/block_int-common.h b/include/block/block_int-com= mon.h index ebb4e56a503..1fd94b2b568 100644 --- a/include/block/block_int-common.h +++ b/include/block/block_int-common.h @@ -610,13 +610,16 @@ struct BlockDriver { * and/or BDRV_BLOCK_RAW; if the current layer defers to a backing * layer, the result should be 0 (and not BDRV_BLOCK_ZERO). See * block.h for the overall meaning of the bits. As a hint, the - * flag want_zero is true if the caller cares more about precise - * mappings (favor accurate _OFFSET_VALID/_ZERO) or false for - * overall allocation (favor larger *pnum, perhaps by reporting - * _DATA instead of _ZERO). The block layer guarantees input - * clamped to bdrv_getlength() and aligned to request_alignment, - * as well as non-NULL pnum, map, and file; in turn, the driver - * must return an error or set pnum to an aligned non-zero value. + * flag @mode is BDRV_BSTAT_PRECISE if the caller cares more about + * precise mappings (favor accurate _OFFSET_VALID/_ZERO), + * BDRV_BSTAT_ALLOCATED for overall allocation (favor larger + * *pnum, perhaps by reporting _DATA instead of _ZERO), or + * BDRV_BSTAT_ZERO for overall reads-as-zero (favor _ZERO, even if + * it requires reading a few sectors to verify). The block layer + * guarantees input clamped to bdrv_getlength() and aligned to + * request_alignment, as well as non-NULL pnum, map, and file; in + * turn, the driver must return an error or set pnum to an aligned + * non-zero value. * * Note that @bytes is just a hint on how big of a region the * caller wants to inspect. It is not a limit on *pnum. @@ -628,8 +631,8 @@ struct BlockDriver { * to clamping *pnum for return to its caller. */ int coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_block_status)( - BlockDriverState *bs, - bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum, + BlockDriverState *bs, enum BlockStatusMode mode, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file); /* @@ -653,8 +656,8 @@ struct BlockDriver { QEMUIOVector *qiov, size_t qiov_offset); int coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_snapshot_block_status)( - BlockDriverState *bs, bool want_zero, int64_t offset, int64_t byte= s, - int64_t *pnum, int64_t *map, BlockDriverState **file); + BlockDriverState *bs, enum BlockStatusMode mode, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **fil= e); int coroutine_fn GRAPH_RDLOCK_PTR (*bdrv_co_pdiscard_snapshot)( BlockDriverState *bs, int64_t offset, int64_t bytes); diff --git a/include/block/block_int-io.h b/include/block/block_int-io.h index 4a7cf2b4fdc..e019e81fa0c 100644 --- a/include/block/block_int-io.h +++ b/include/block/block_int-io.h @@ -38,8 +38,8 @@ int coroutine_fn GRAPH_RDLOCK bdrv_co_preadv_snapshot(BdrvChild *child, int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset); int coroutine_fn GRAPH_RDLOCK bdrv_co_snapshot_block_status( - BlockDriverState *bs, bool want_zero, int64_t offset, int64_t bytes, - int64_t *pnum, int64_t *map, BlockDriverState **file); + BlockDriverState *bs, enum BlockStatusMode mode, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file); int coroutine_fn GRAPH_RDLOCK bdrv_co_pdiscard_snapshot(BlockDriverState *= bs, int64_t offset, int64_t bytes); diff --git a/block/io.c b/block/io.c index 1ba8d1aeea1..73c96084e62 100644 --- a/block/io.c +++ b/block/io.c @@ -2364,10 +2364,8 @@ int bdrv_flush_all(void) * Drivers not implementing the functionality are assumed to not support * backing files, hence all their sectors are reported as allocated. * - * If 'want_zero' is true, the caller is querying for mapping - * purposes, with a focus on valid BDRV_BLOCK_OFFSET_VALID, _DATA, and - * _ZERO where possible; otherwise, the result favors larger 'pnum', - * with a focus on accurate BDRV_BLOCK_ALLOCATED. + * 'mode' serves as a hint as to which results are favored; see enum + * BlockStatusMode for details of the supported modes. * * If 'offset' is beyond the end of the disk image the return value is * BDRV_BLOCK_EOF and 'pnum' is set to 0. @@ -2387,7 +2385,7 @@ int bdrv_flush_all(void) * set to the host mapping and BDS corresponding to the guest offset. */ static int coroutine_fn GRAPH_RDLOCK -bdrv_co_do_block_status(BlockDriverState *bs, bool want_zero, +bdrv_co_do_block_status(BlockDriverState *bs, enum BlockStatusMode mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **fi= le) { @@ -2476,7 +2474,7 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool wa= nt_zero, local_file =3D bs; local_map =3D aligned_offset; } else { - ret =3D bs->drv->bdrv_co_block_status(bs, want_zero, aligned_o= ffset, + ret =3D bs->drv->bdrv_co_block_status(bs, mode, aligned_offset, aligned_bytes, pnum, &loca= l_map, &local_file); @@ -2488,10 +2486,10 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool = want_zero, * the cache requires an RCU update, so double check here to a= void * such an update if possible. * - * Check want_zero, because we only want to update the cache w= hen we + * Check mode, because we only want to update the cache when we * have accurate information about what is zero and what is da= ta. */ - if (want_zero && + if (mode =3D=3D BDRV_BSTAT_PRECISE && ret =3D=3D (BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID) && QLIST_EMPTY(&bs->children)) { @@ -2548,7 +2546,7 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool wa= nt_zero, if (ret & BDRV_BLOCK_RAW) { assert(ret & BDRV_BLOCK_OFFSET_VALID && local_file); - ret =3D bdrv_co_do_block_status(local_file, want_zero, local_map, + ret =3D bdrv_co_do_block_status(local_file, mode, local_map, *pnum, pnum, &local_map, &local_file= ); goto out; } @@ -2560,7 +2558,7 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool wa= nt_zero, if (!cow_bs) { ret |=3D BDRV_BLOCK_ZERO; - } else if (want_zero) { + } else if (mode =3D=3D BDRV_BSTAT_PRECISE) { int64_t size2 =3D bdrv_co_getlength(cow_bs); if (size2 >=3D 0 && offset >=3D size2) { @@ -2569,14 +2567,14 @@ bdrv_co_do_block_status(BlockDriverState *bs, bool = want_zero, } } - if (want_zero && ret & BDRV_BLOCK_RECURSE && + if (mode =3D=3D BDRV_BSTAT_PRECISE && ret & BDRV_BLOCK_RECURSE && local_file && local_file !=3D bs && (ret & BDRV_BLOCK_DATA) && !(ret & BDRV_BLOCK_ZERO) && (ret & BDRV_BLOCK_OFFSET_VALID)) { int64_t file_pnum; int ret2; - ret2 =3D bdrv_co_do_block_status(local_file, want_zero, local_map, + ret2 =3D bdrv_co_do_block_status(local_file, mode, local_map, *pnum, &file_pnum, NULL, NULL); if (ret2 >=3D 0) { /* Ignore errors. This is just providing extra information, it @@ -2627,7 +2625,7 @@ int coroutine_fn bdrv_co_common_block_status_above(BlockDriverState *bs, BlockDriverState *base, bool include_base, - bool want_zero, + enum BlockStatusMode mode, int64_t offset, int64_t bytes, int64_t *pnum, @@ -2654,7 +2652,7 @@ bdrv_co_common_block_status_above(BlockDriverState *b= s, return 0; } - ret =3D bdrv_co_do_block_status(bs, want_zero, offset, bytes, pnum, + ret =3D bdrv_co_do_block_status(bs, mode, offset, bytes, pnum, map, file); ++*depth; if (ret < 0 || *pnum =3D=3D 0 || ret & BDRV_BLOCK_ALLOCATED || bs =3D= =3D base) { @@ -2671,7 +2669,7 @@ bdrv_co_common_block_status_above(BlockDriverState *b= s, for (p =3D bdrv_filter_or_cow_bs(bs); include_base || p !=3D base; p =3D bdrv_filter_or_cow_bs(p)) { - ret =3D bdrv_co_do_block_status(p, want_zero, offset, bytes, pnum, + ret =3D bdrv_co_do_block_status(p, mode, offset, bytes, pnum, map, file); ++*depth; if (ret < 0) { @@ -2734,7 +2732,8 @@ int coroutine_fn bdrv_co_block_status_above(BlockDriv= erState *bs, BlockDriverState **file) { IO_CODE(); - return bdrv_co_common_block_status_above(bs, base, false, true, offset, + return bdrv_co_common_block_status_above(bs, base, false, + BDRV_BSTAT_PRECISE, offset, bytes, pnum, map, file, NULL); } @@ -2765,8 +2764,9 @@ int coroutine_fn bdrv_co_is_zero_fast(BlockDriverStat= e *bs, int64_t offset, return 1; } - ret =3D bdrv_co_common_block_status_above(bs, NULL, false, false, offs= et, - bytes, &pnum, NULL, NULL, NULL= ); + ret =3D bdrv_co_common_block_status_above(bs, NULL, false, BDRV_BSTAT_= ZERO, + offset, bytes, &pnum, NULL, NU= LL, + NULL); if (ret < 0) { return ret; @@ -2782,9 +2782,9 @@ int coroutine_fn bdrv_co_is_allocated(BlockDriverStat= e *bs, int64_t offset, int64_t dummy; IO_CODE(); - ret =3D bdrv_co_common_block_status_above(bs, bs, true, false, offset, - bytes, pnum ? pnum : &dummy, N= ULL, - NULL, NULL); + ret =3D bdrv_co_common_block_status_above(bs, bs, true, BDRV_BSTAT_ALL= OCATED, + offset, bytes, pnum ? pnum : &= dummy, + NULL, NULL, NULL); if (ret < 0) { return ret; } @@ -2817,7 +2817,8 @@ int coroutine_fn bdrv_co_is_allocated_above(BlockDriv= erState *bs, int ret; IO_CODE(); - ret =3D bdrv_co_common_block_status_above(bs, base, include_base, fals= e, + ret =3D bdrv_co_common_block_status_above(bs, base, include_base, + BDRV_BSTAT_ALLOCATED, offset, bytes, pnum, NULL, NUL= L, &depth); if (ret < 0) { @@ -3709,8 +3710,8 @@ bdrv_co_preadv_snapshot(BdrvChild *child, int64_t off= set, int64_t bytes, } int coroutine_fn -bdrv_co_snapshot_block_status(BlockDriverState *bs, - bool want_zero, int64_t offset, int64_t byte= s, +bdrv_co_snapshot_block_status(BlockDriverState *bs, enum BlockStatusMode m= ode, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { @@ -3728,7 +3729,7 @@ bdrv_co_snapshot_block_status(BlockDriverState *bs, } bdrv_inc_in_flight(bs); - ret =3D drv->bdrv_co_snapshot_block_status(bs, want_zero, offset, byte= s, + ret =3D drv->bdrv_co_snapshot_block_status(bs, mode, offset, bytes, pnum, map, file); bdrv_dec_in_flight(bs); diff --git a/block/blkdebug.c b/block/blkdebug.c index 1c1967f8e0a..f3eba9e6f27 100644 --- a/block/blkdebug.c +++ b/block/blkdebug.c @@ -751,9 +751,9 @@ blkdebug_co_pdiscard(BlockDriverState *bs, int64_t offs= et, int64_t bytes) } static int coroutine_fn GRAPH_RDLOCK -blkdebug_co_block_status(BlockDriverState *bs, bool want_zero, int64_t off= set, - int64_t bytes, int64_t *pnum, int64_t *map, - BlockDriverState **file) +blkdebug_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, + int64_t offset, int64_t bytes, int64_t *pnum, + int64_t *map, BlockDriverState **file) { int err; diff --git a/block/copy-before-write.c b/block/copy-before-write.c index fd470f5f926..e75e6925e50 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -291,8 +291,8 @@ cbw_co_preadv_snapshot(BlockDriverState *bs, int64_t of= fset, int64_t bytes, } static int coroutine_fn GRAPH_RDLOCK -cbw_co_snapshot_block_status(BlockDriverState *bs, - bool want_zero, int64_t offset, int64_t bytes, +cbw_co_snapshot_block_status(BlockDriverState *bs, enum BlockStatusMode mo= de, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { diff --git a/block/file-posix.c b/block/file-posix.c index 56d1972d156..30129ec9b37 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -3201,7 +3201,7 @@ static int find_allocation(BlockDriverState *bs, off_= t start, * well exceed it. */ static int coroutine_fn raw_co_block_status(BlockDriverState *bs, - bool want_zero, + enum BlockStatusMode mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, @@ -3217,7 +3217,7 @@ static int coroutine_fn raw_co_block_status(BlockDriv= erState *bs, return ret; } - if (!want_zero) { + if (mode !=3D BDRV_BSTAT_PRECISE) { *pnum =3D bytes; *map =3D offset; *file =3D bs; diff --git a/block/gluster.c b/block/gluster.c index c6d25ae7335..f5ee3cdcc1f 100644 --- a/block/gluster.c +++ b/block/gluster.c @@ -1465,7 +1465,7 @@ exit: * (Based on raw_co_block_status() from file-posix.c.) */ static int coroutine_fn qemu_gluster_co_block_status(BlockDriverState *bs, - bool want_zero, + enum BlockStatusMode = mode, int64_t offset, int64_t bytes, int64_t *pnum, @@ -1482,7 +1482,7 @@ static int coroutine_fn qemu_gluster_co_block_status(= BlockDriverState *bs, return ret; } - if (!want_zero) { + if (mode !=3D BDRV_BSTAT_PRECISE) { *pnum =3D bytes; *map =3D offset; *file =3D bs; diff --git a/block/iscsi.c b/block/iscsi.c index 2f0f4dac097..c7b425597f0 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -694,9 +694,9 @@ out_unlock: static int coroutine_fn iscsi_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t offs= et, - int64_t bytes, int64_t *pnum, - int64_t *map, + enum BlockStatusMode mode, + int64_t offset, int64_t byte= s, + int64_t *pnum, int64_t *map, BlockDriverState **file) { IscsiLun *iscsilun =3D bs->opaque; diff --git a/block/nbd.c b/block/nbd.c index 887841bc813..591dedde62e 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -1397,8 +1397,8 @@ nbd_client_co_pdiscard(BlockDriverState *bs, int64_t = offset, int64_t bytes) } static int coroutine_fn GRAPH_RDLOCK nbd_client_co_block_status( - BlockDriverState *bs, bool want_zero, int64_t offset, int64_t byte= s, - int64_t *pnum, int64_t *map, BlockDriverState **file) + BlockDriverState *bs, enum BlockStatusMode mode, int64_t offset, + int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **fil= e) { int ret, request_ret; NBDExtent64 extent =3D { 0 }; diff --git a/block/null.c b/block/null.c index dc0b1fdbd9b..66470787cfd 100644 --- a/block/null.c +++ b/block/null.c @@ -227,9 +227,9 @@ static int null_reopen_prepare(BDRVReopenState *reopen_= state, } static int coroutine_fn null_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t offse= t, - int64_t bytes, int64_t *pnum, - int64_t *map, + enum BlockStatusMode mode, + int64_t offset, int64_t bytes, + int64_t *pnum, int64_t *map, BlockDriverState **file) { BDRVNullState *s =3D bs->opaque; diff --git a/block/parallels.c b/block/parallels.c index 347ca127f34..93b42b47239 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -416,9 +416,9 @@ parallels_co_flush_to_os(BlockDriverState *bs) } static int coroutine_fn GRAPH_RDLOCK -parallels_co_block_status(BlockDriverState *bs, bool want_zero, int64_t of= fset, - int64_t bytes, int64_t *pnum, int64_t *map, - BlockDriverState **file) +parallels_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, + int64_t offset, int64_t bytes, int64_t *pnum, + int64_t *map, BlockDriverState **file) { BDRVParallelsState *s =3D bs->opaque; int count; diff --git a/block/qcow.c b/block/qcow.c index da8ad4d2430..de7fb42c51c 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -530,7 +530,7 @@ get_cluster_offset(BlockDriverState *bs, uint64_t offse= t, int allocate, } static int coroutine_fn GRAPH_RDLOCK -qcow_co_block_status(BlockDriverState *bs, bool want_zero, +qcow_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { diff --git a/block/qcow2.c b/block/qcow2.c index 7774e7f0909..14fa1c00df1 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2141,9 +2141,9 @@ static void qcow2_join_options(QDict *options, QDict = *old_options) } static int coroutine_fn GRAPH_RDLOCK -qcow2_co_block_status(BlockDriverState *bs, bool want_zero, int64_t offset, - int64_t count, int64_t *pnum, int64_t *map, - BlockDriverState **file) +qcow2_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, + int64_t offset, int64_t count, int64_t *pnum, + int64_t *map, BlockDriverState **file) { BDRVQcow2State *s =3D bs->opaque; uint64_t host_offset; diff --git a/block/qed.c b/block/qed.c index ac24449ffb3..6e57859d05f 100644 --- a/block/qed.c +++ b/block/qed.c @@ -833,9 +833,9 @@ fail: } static int coroutine_fn GRAPH_RDLOCK -bdrv_qed_co_block_status(BlockDriverState *bs, bool want_zero, int64_t pos, - int64_t bytes, int64_t *pnum, int64_t *map, - BlockDriverState **file) +bdrv_qed_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, + int64_t pos, int64_t bytes, int64_t *pnum, + int64_t *map, BlockDriverState **file) { BDRVQEDState *s =3D bs->opaque; size_t len =3D MIN(bytes, SIZE_MAX); diff --git a/block/quorum.c b/block/quorum.c index 30747a6df93..97091136fcb 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -1226,7 +1226,7 @@ static void quorum_child_perm(BlockDriverState *bs, B= drvChild *c, * region contains zeroes, and BDRV_BLOCK_DATA otherwise. */ static int coroutine_fn GRAPH_RDLOCK -quorum_co_block_status(BlockDriverState *bs, bool want_zero, +quorum_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, int64_t offset, int64_t count, int64_t *pnum, int64_t *map, BlockDriverState **fil= e) { @@ -1238,7 +1238,7 @@ quorum_co_block_status(BlockDriverState *bs, bool wan= t_zero, for (i =3D 0; i < s->num_children; i++) { int64_t bytes; ret =3D bdrv_co_common_block_status_above(s->children[i]->bs, NULL= , false, - want_zero, offset, count, + mode, offset, count, &bytes, NULL, NULL, NULL); if (ret < 0) { quorum_report_bad(QUORUM_OP_TYPE_READ, offset, count, diff --git a/block/raw-format.c b/block/raw-format.c index e08526e2eca..0ff5367123b 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -283,8 +283,8 @@ fail: } static int coroutine_fn GRAPH_RDLOCK -raw_co_block_status(BlockDriverState *bs, bool want_zero, int64_t offset, - int64_t bytes, int64_t *pnum, int64_t *map, +raw_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t = *map, BlockDriverState **file) { BDRVRawState *s =3D bs->opaque; diff --git a/block/rbd.c b/block/rbd.c index af984fb7db4..abcdd5e4e76 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -1504,9 +1504,9 @@ static int qemu_rbd_diff_iterate_cb(uint64_t offs, si= ze_t len, } static int coroutine_fn qemu_rbd_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t o= ffset, - int64_t bytes, int64_t *p= num, - int64_t *map, + enum BlockStatusMode mode, + int64_t offset, int64_t b= ytes, + int64_t *pnum, int64_t *m= ap, BlockDriverState **file) { BDRVRBDState *s =3D bs->opaque; diff --git a/block/snapshot-access.c b/block/snapshot-access.c index 71ac83c01f0..1d91b6df5d7 100644 --- a/block/snapshot-access.c +++ b/block/snapshot-access.c @@ -41,11 +41,11 @@ snapshot_access_co_preadv_part(BlockDriverState *bs, static int coroutine_fn GRAPH_RDLOCK snapshot_access_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t offset, + enum BlockStatusMode mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { - return bdrv_co_snapshot_block_status(bs->file->bs, want_zero, offset, + return bdrv_co_snapshot_block_status(bs->file->bs, mode, offset, bytes, pnum, map, file); } diff --git a/block/vdi.c b/block/vdi.c index a2da6ecab01..9a9d402c946 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -523,8 +523,8 @@ static int vdi_reopen_prepare(BDRVReopenState *state, } static int coroutine_fn GRAPH_RDLOCK -vdi_co_block_status(BlockDriverState *bs, bool want_zero, int64_t offset, - int64_t bytes, int64_t *pnum, int64_t *map, +vdi_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, + int64_t offset, int64_t bytes, int64_t *pnum, int64_t = *map, BlockDriverState **file) { BDRVVdiState *s =3D (BDRVVdiState *)bs->opaque; diff --git a/block/vmdk.c b/block/vmdk.c index 2adec499122..6e2fd8d16ef 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1777,7 +1777,7 @@ static inline uint64_t vmdk_find_offset_in_cluster(Vm= dkExtent *extent, } static int coroutine_fn GRAPH_RDLOCK -vmdk_co_block_status(BlockDriverState *bs, bool want_zero, +vmdk_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) { diff --git a/block/vpc.c b/block/vpc.c index 0309e319f60..4eac32fd1c4 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -726,7 +726,7 @@ fail: } static int coroutine_fn GRAPH_RDLOCK -vpc_co_block_status(BlockDriverState *bs, bool want_zero, +vpc_co_block_status(BlockDriverState *bs, enum BlockStatusMode mode, int64_t offset, int64_t bytes, int64_t *pnum, int64_t *map, BlockDriverState **file) diff --git a/block/vvfat.c b/block/vvfat.c index 91d69b3cc83..336679cac12 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -3134,9 +3134,9 @@ vvfat_co_pwritev(BlockDriverState *bs, int64_t offset= , int64_t bytes, } static int coroutine_fn vvfat_co_block_status(BlockDriverState *bs, - bool want_zero, int64_t offs= et, - int64_t bytes, int64_t *n, - int64_t *map, + enum BlockStatusMode mode, + int64_t offset, int64_t byte= s, + int64_t *n, int64_t *map, BlockDriverState **file) { *n =3D bytes; diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothr= ead.c index 2b358eaaa82..8189b32fd52 100644 --- a/tests/unit/test-block-iothread.c +++ b/tests/unit/test-block-iothread.c @@ -63,7 +63,7 @@ bdrv_test_co_truncate(BlockDriverState *bs, int64_t offse= t, bool exact, } static int coroutine_fn bdrv_test_co_block_status(BlockDriverState *bs, - bool want_zero, + enum BlockStatusMode mod= e, int64_t offset, int64_t = count, int64_t *pnum, int64_t *= map, BlockDriverState **file) --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915352; cv=none; d=zohomail.com; s=zohoarc; b=a3tOkaDhhi4ZB7ZzJYskOCgN3XoIZVILL19jNX81Pc3jjU2xnabdy6itdAsj/wbGLhJ6yIt7Kfa66+6PRGPA/tYnOk5O1X3uT5IvhNc9U3PhYHxfiwrOOmyU0xsGLeUeY/VSQt4GK/GHOcPJHuy3gus90YHpfxO2um/Cn05aJ7M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915352; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=NfqoMro0GIz5LImw8hVnGM/j3vZZ3MGIlfImJYmZAIM=; b=P9C0rfkkt/dI2uwhXLOEAUPdEddeILAdRy9H4KshzqCnIpB4UfeviOgMaLj85gExtNTC5WQJT8dN4ODZjbQ0jEkBLzQqgWQcF/ye5V/ZCwSObWGsNd8xuV8SsfRdxX9jldiex84Fm2Qfwhq+l7x1WsctyxQIdwxJaHiXJGIkyr8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 174491535232963.12371123843877; Thu, 17 Apr 2025 11:42:32 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UBX-00042v-UN; Thu, 17 Apr 2025 14:42:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBV-00041x-1t for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBT-0002qt-Ln for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:00 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-250-3uGYGDo4MVKNvXSZO8uZ_g-1; Thu, 17 Apr 2025 14:41:55 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5F54719560AF; Thu, 17 Apr 2025 18:41:54 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E0B331955E96; Thu, 17 Apr 2025 18:41:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915318; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NfqoMro0GIz5LImw8hVnGM/j3vZZ3MGIlfImJYmZAIM=; b=CqFifwEyjG4VAJ2OAsoybQf6NK1BDollG5K05Ivx4qFAILS7hxsi3MinS5lwbgjoLCOXss TE544kSXj0gbe7UqrXvZEfye8KteAs6VcyGvzMl0qYGGbrHBBbYHjSjJaQp3n2qKuz+EnP qDCUoS2c1l6jj6hpGhezRyQxCY5Yucs= X-MC-Unique: 3uGYGDo4MVKNvXSZO8uZ_g-1 X-Mimecast-MFC-AGG-ID: 3uGYGDo4MVKNvXSZO8uZ_g_1744915314 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, Kevin Wolf , Hanna Reitz Subject: [PATCH v2 02/11] file-posix: Handle zero block status hint better Date: Thu, 17 Apr 2025 13:39:07 -0500 Message-ID: <20250417184133.105746-15-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915353140019000 Content-Type: text/plain; charset="utf-8" Although the previous patch to change 'bool want_zero' into an enum made no semantic change, it is now time to differentiate. When the caller specifically wants to know what parts of the file read as zero, we need to use lseek and actually reporting holes, rather than short-circuiting and advertising full allocation. This change will be utilized in later patches to let mirroring optimize for the case when the destination already reads as zeroes. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/file-posix.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/file-posix.c b/block/file-posix.c index 30129ec9b37..8f3e7a8304f 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -3217,7 +3217,8 @@ static int coroutine_fn raw_co_block_status(BlockDriv= erState *bs, return ret; } - if (mode !=3D BDRV_BSTAT_PRECISE) { + if (mode =3D=3D BDRV_BSTAT_ALLOCATED) { + /* There is no backing file - all bytes are allocated in this file= . */ *pnum =3D bytes; *map =3D offset; *file =3D bs; --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915353; cv=none; d=zohomail.com; s=zohoarc; b=bm751vQjPSNnkbmfTE9huKk/wXV6hH7HO2fasMSuMWSyaaozOUouJTTRKk8i2UXXP6HO+RQlASMNl9Mxb9LpGTzkopr20uoQLEvQdcBpvG5E3AFIoXhlxlLyvkvJ1HzQLhTkKCny2wu5eZWBdoNlMuAqZqtCZI69fJb5q25co4A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915353; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=csqy0lFVvHPMOE3Y+BWOfgNJuQqp7DKuYy228mUVHtk=; b=QawxULHrIawlIepRnJD9fS/JFVjnMYrNbQBU2zopjLSY/0hRiU85mjLFokH/2IYroNEeFiXsASOo2UrZeioWMvicor6ab/pmKskSYw26n+Xq2NdzxsZNWQ8pFu1re8Uyx4nOxWR5mTv1MHHJxLROR2qo3/HkLrB6h76g7g6fiQI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915353501996.8466584420714; Thu, 17 Apr 2025 11:42:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UBZ-00043y-Uh; Thu, 17 Apr 2025 14:42:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBY-000436-Et for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBW-0002tg-P1 for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:04 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-563--3VsHMIcOe2j8rvNlqrr-A-1; Thu, 17 Apr 2025 14:41:57 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CAB81195608B; Thu, 17 Apr 2025 18:41:56 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A59F81955E96; Thu, 17 Apr 2025 18:41:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915321; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=csqy0lFVvHPMOE3Y+BWOfgNJuQqp7DKuYy228mUVHtk=; b=W1PmD6xhti+LISvhoukRv0/ZLVnd5gTTuaiCaa4154rKRoCs5zd4Fma7G+w2qtRuybVcS9 f8N6z4IHywrgWET9UL43oGMtsDyAIP9056ry9tDp61yIeIpz4QFEhxIKOZDeBcN0A1DOUT FKdgmxjPctzFRRRujgY8RlN7SfQ6sb4= X-MC-Unique: -3VsHMIcOe2j8rvNlqrr-A-1 X-Mimecast-MFC-AGG-ID: -3VsHMIcOe2j8rvNlqrr-A_1744915316 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, Stefan Hajnoczi , Fam Zheng , Kevin Wolf , Hanna Reitz Subject: [PATCH v2 03/11] block: Let bdrv_co_is_zero_fast consolidate adjacent extents Date: Thu, 17 Apr 2025 13:39:08 -0500 Message-ID: <20250417184133.105746-16-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915355520019100 Content-Type: text/plain; charset="utf-8" Some BDS drivers have a cap on how much block status they can supply in one query (for example, NBD talking to an older server cannot inspect more than 4G per query; and qcow2 tends to cap its answers rather than cross a cluster boundary of an L1 table). Although the existing callers of bdrv_co_is_zero_fast are not passing in that large of a 'bytes' parameter, an upcoming caller wants to query the entire image at once, and will thus benefit from being able to treat adjacent zero regions in a coalesced manner, rather than claiming the region is non-zero merely because pnum was truncated and didn't match the incoming bytes. While refactoring this into a loop, note that there is no need to assign pnum prior to calling bdrv_co_common_block_status_above() (it is guaranteed to be assigned deeper in the callstack). Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/io.c | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/block/io.c b/block/io.c index 73c96084e62..6ef78070915 100644 --- a/block/io.c +++ b/block/io.c @@ -2757,22 +2757,25 @@ int coroutine_fn bdrv_co_is_zero_fast(BlockDriverSt= ate *bs, int64_t offset, int64_t bytes) { int ret; - int64_t pnum =3D bytes; + int64_t pnum; IO_CODE(); - if (!bytes) { - return 1; + while (bytes) { + ret =3D bdrv_co_common_block_status_above(bs, NULL, false, + BDRV_BSTAT_ZERO, offset, b= ytes, + &pnum, NULL, NULL, NULL); + + if (ret < 0) { + return ret; + } + if (!(ret & BDRV_BLOCK_ZERO)) { + return 0; + } + offset +=3D pnum; + bytes -=3D pnum; } - ret =3D bdrv_co_common_block_status_above(bs, NULL, false, BDRV_BSTAT_= ZERO, - offset, bytes, &pnum, NULL, NU= LL, - NULL); - - if (ret < 0) { - return ret; - } - - return (pnum =3D=3D bytes) && (ret & BDRV_BLOCK_ZERO); + return 1; } int coroutine_fn bdrv_co_is_allocated(BlockDriverState *bs, int64_t offset, --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915353; cv=none; d=zohomail.com; s=zohoarc; b=h8diy3CMQMdMNvQSuR93ftUuR47RkCZOn0DbjvaPejsX2QUEzrMM0lPwdne+AOn0bSPEURu74ccvwdTqVO22lMDqXdFWGYCStWc9qRnPOWem8pBcxblgKJ0xfhofWzZCCbYMtYVWQsv7mF3K6NWaWMKNXS7K7vKUWNxyiWV/0GA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915353; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dakir4L3nktKnN6Yme7bev8PnDnH2GQk65lKvpaop5E=; b=GvDqN1l5JlOelcsCRLzVCQMDNWD/8enBvEYvogRuBKWNJGFuE4maX+iHKF2pYpO31diQsUTVHkE/fkbho66xbx5dJqeblbbfK8RgP2xI2E4BUqgaKdDvuw0kRpyCWc1XKi1R/M750EZg5B3Pa8k0QD+I7b1W87is9ybeaX4b1Ks= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915353339244.69100256066451; Thu, 17 Apr 2025 11:42:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UBf-00045E-G5; Thu, 17 Apr 2025 14:42:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBa-000440-3h for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBX-0002uG-P4 for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:05 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-512-ft3T3zqCMUGyzLSKHzjHYA-1; Thu, 17 Apr 2025 14:42:00 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 09D341956086; Thu, 17 Apr 2025 18:41:59 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1ED651956095; Thu, 17 Apr 2025 18:41:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915323; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dakir4L3nktKnN6Yme7bev8PnDnH2GQk65lKvpaop5E=; b=a6jzrTOpAmJvhrdxJeppuu7FqLE8OkvXAmLOadwuIByrxRyzvEBizLxbfhW4RpcAaFDUzY S9V+0IbBmS9A1kLkPJ59v04qjXKkfP4LmcJ800mFpPPx3j9ItcWN3SQzeSyibHePEkoShq z+HhTUY0JKNLblhOmwsPql/OxnpqyMY= X-MC-Unique: ft3T3zqCMUGyzLSKHzjHYA-1 X-Mimecast-MFC-AGG-ID: ft3T3zqCMUGyzLSKHzjHYA_1744915319 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, Stefan Hajnoczi , Fam Zheng , Kevin Wolf , Hanna Reitz Subject: [PATCH v2 04/11] block: Add new bdrv_co_is_all_zeroes() function Date: Thu, 17 Apr 2025 13:39:09 -0500 Message-ID: <20250417184133.105746-17-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915355105019000 Content-Type: text/plain; charset="utf-8" There are some optimizations that require knowing if an image starts out as reading all zeroes, such as making blockdev-mirror faster by skipping the copying of source zeroes to the destination. The existing bdrv_co_is_zero_fast() is a good building block for answering this question, but it tends to give an answer of 0 for a file we just created via QMP 'blockdev-create' or similar (such as 'qemu-img create -f raw'). Why? Because file-posix.c insists on allocating a tiny header to any file rather than leaving it 100% sparse, due to some filesystems that are unable to answer alignment probes on a hole. But teaching file-posix.c to read the tiny header doesn't scale - the problem of a small header is also visible when libvirt sets up an NBD client to a just-created file on a migration destination host. So, we need a wrapper function that handles a bit more complexity in a common manner for all block devices - when the BDS is mostly a hole, but has a small non-hole header, it is still worth the time to read that header and check if it reads as all zeroes before giving up and returning a pessimistic answer. Signed-off-by: Eric Blake --- include/block/block-io.h | 2 ++ block/io.c | 58 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) diff --git a/include/block/block-io.h b/include/block/block-io.h index b49e0537dd4..b99cc98d265 100644 --- a/include/block/block-io.h +++ b/include/block/block-io.h @@ -161,6 +161,8 @@ bdrv_is_allocated_above(BlockDriverState *bs, BlockDriv= erState *base, int coroutine_fn GRAPH_RDLOCK bdrv_co_is_zero_fast(BlockDriverState *bs, int64_t offset, int64_t bytes); +int coroutine_fn GRAPH_RDLOCK +bdrv_co_is_all_zeroes(BlockDriverState *bs); int GRAPH_RDLOCK bdrv_apply_auto_read_only(BlockDriverState *bs, const char *errmsg, diff --git a/block/io.c b/block/io.c index 6ef78070915..dc1341e4029 100644 --- a/block/io.c +++ b/block/io.c @@ -2778,6 +2778,64 @@ int coroutine_fn bdrv_co_is_zero_fast(BlockDriverSta= te *bs, int64_t offset, return 1; } +/* + * Check @bs (and its backing chain) to see if the entire image is known + * to read as zeroes. + * Return 1 if that is the case, 0 otherwise and -errno on error. + * This test is meant to be fast rather than accurate so returning 0 + * does not guarantee non-zero data; however, it can report 1 in more + * cases than bdrv_co_is_zero_fast. + */ +int coroutine_fn bdrv_co_is_all_zeroes(BlockDriverState *bs) +{ + int ret; + int64_t pnum, bytes; + char *buf; + QEMUIOVector local_qiov; + IO_CODE(); + + bytes =3D bdrv_co_getlength(bs); + if (bytes < 0) { + return bytes; + } + + /* First probe - see if the entire image reads as zero */ + ret =3D bdrv_co_common_block_status_above(bs, NULL, false, BDRV_BSTAT_= ZERO, + 0, bytes, &pnum, NULL, NULL, + NULL); + if (ret < 0) { + return ret; + } + if (ret & BDRV_BLOCK_ZERO) { + return bdrv_co_is_zero_fast(bs, pnum, bytes - pnum); + } + + /* + * Because of the way 'blockdev-create' works, raw files tend to + * be created with a non-sparse region at the front to make + * alignment probing easier. If the block starts with only a + * small allocated region, it is still worth the effort to see if + * the rest of the image is still sparse, coupled with manually + * reading the first region to see if it reads zero after all. + */ + if (pnum > qemu_real_host_page_size()) { + return 0; + } + ret =3D bdrv_co_is_zero_fast(bs, pnum, bytes - pnum); + if (ret <=3D 0) { + return ret; + } + /* Only the head of the image is unknown, and it's small. Read it. */ + buf =3D qemu_blockalign(bs, pnum); + qemu_iovec_init_buf(&local_qiov, buf, pnum); + ret =3D bdrv_driver_preadv(bs, 0, pnum, &local_qiov, 0, 0); + if (ret >=3D 0) { + ret =3D buffer_is_zero(buf, pnum); + } + qemu_vfree(buf); + return ret; +} + int coroutine_fn bdrv_co_is_allocated(BlockDriverState *bs, int64_t offset, int64_t bytes, int64_t *pnum) { --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915393; cv=none; d=zohomail.com; s=zohoarc; b=JUMKzs3j5yirI+f5r68OYtg+n5VoAkW1oO6mbsGPP2U2dWV0+EOjCyeE0p9ETtu0rYT1NHuNpzLkPYq4GYw/0NkKBNEj4n1VBTZiE8e0Jyr3ZT0sqlxAxzIyqzPksuVsS+ZOCIvibDRW0vHM7L7Irl9dBBcYNzdeMpUVuZaff44= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915393; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fMxEsLdatiMy8ojQLd9WbC0vJ5vopuEUrFLmu1I773Y=; b=flhxKelT8Oq2NZmY0kz4Zncv0+R/05PDJNCBp68AMaM1GEUK+zr7/z1RfsVyTgg4WehikU3y99UoIb3a6+zvsv3q1tp9TRyITe0eyswE3IQA1EGhTnKT88ORn/P5HA9Oh6UxdtGsiq+vAtWOW18LIDhHaXfmf+uPzghEbt20FSE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915393602351.42267714698255; Thu, 17 Apr 2025 11:43:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UC7-0004bh-7T; Thu, 17 Apr 2025 14:42:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBc-00045R-Ka for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBa-0002vP-R0 for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:08 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-552-0n2XLTv_MLW9FfyygBG6BQ-1; Thu, 17 Apr 2025 14:42:01 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B7D1D1800EC9; Thu, 17 Apr 2025 18:42:00 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 673D21956095; Thu, 17 Apr 2025 18:41:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915326; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fMxEsLdatiMy8ojQLd9WbC0vJ5vopuEUrFLmu1I773Y=; b=JelI35uQvoSAKpup710FErrVChTo0jQ3WbMZQYOym1Ysf/vHzrK3Fd9v/2yGZs7wrJ2KwC NS3H49Y7Rq9GcaTq1upH4g7c0+yXk7vSxjP08sj4KewzUOdbReS0W73btE5bz/5Vpe65dI Tg52QxZ6cKMWFLP51PxsVN85xz+mvAk= X-MC-Unique: 0n2XLTv_MLW9FfyygBG6BQ-1 X-Mimecast-MFC-AGG-ID: 0n2XLTv_MLW9FfyygBG6BQ_1744915320 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, Kevin Wolf , Hanna Reitz Subject: [PATCH v2 05/11] iotests: Improve iotest 194 to mirror data Date: Thu, 17 Apr 2025 13:39:10 -0500 Message-ID: <20250417184133.105746-18-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915395170019000 Content-Type: text/plain; charset="utf-8" Mirroring a completely sparse image to a sparse destination should be practically instantaneous. It isn't yet, but the test will be more realistic if it has some non-zero to mirror as well as the holes. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/194 | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194 index c0ce82dd257..d0b9c084f5f 100755 --- a/tests/qemu-iotests/194 +++ b/tests/qemu-iotests/194 @@ -34,6 +34,7 @@ with iotests.FilePath('source.img') as source_img_path, \ img_size =3D '1G' iotests.qemu_img_create('-f', iotests.imgfmt, source_img_path, img_siz= e) + iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write 512M 1M', source_im= g_path) iotests.qemu_img_create('-f', iotests.imgfmt, dest_img_path, img_size) iotests.log('Launching VMs...') --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915420; cv=none; d=zohomail.com; s=zohoarc; b=gH6RbPCmOvBZ+ZGOPXw5nJBQOdezBCE7wvseALnKsR5TYLV4VvMhhDzwLOeFpNtCb46Jhh+4qrOLVEG0OJ1uUh6I0aiPlnM52HsgGT2tn6peufZKgT1q/Ddist5MBgjXCzGF88zkdLIusj4hxyUehwpJEi4jdMdbGBqJz/oM7JI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915420; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=BHT8GdipCgk2LgZCaGH26+jIXGEa1CkOwW0RiwZNGNU=; b=l9EH2/RDADi/9Yqhg2dZNs/H7fBDlW/zy89dcPzUAG5Z2E3p0AH9+0pK1ESFJw+QcwbjX6luC+hQc7I6H1dfL2dnyRmA1e0XsmKwffvBweGYdvrzriRUfrI/cO/PBGPDh9Nd7YRGm3qMjFtX5ZcbkDcHN+aGRA/eoFa/l6pOTkc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915420540400.30004223660376; Thu, 17 Apr 2025 11:43:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UC3-0004Nj-D1; Thu, 17 Apr 2025 14:42:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBf-00045k-52 for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBd-0002vu-3m for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:10 -0400 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-562-rOYk1QwQOzWYPlGsInXv-Q-1; Thu, 17 Apr 2025 14:42:03 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 960B21955DCE; Thu, 17 Apr 2025 18:42:02 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 237511956095; Thu, 17 Apr 2025 18:42:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915327; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BHT8GdipCgk2LgZCaGH26+jIXGEa1CkOwW0RiwZNGNU=; b=cEFXNEosDiWnRcTauMuuYGByY6bYhz1E1LfTWmU0YUKGzCtpHAvejaXu0mgWfv3o6+54b6 gY+qg+d6WGXFWsyqVs5BzJ3UB7CnJFhWEt1xauHHrO0wuG9XMK4EqvfrhZqO4JiVkyWtv/ R7ZxSqGKD073yp8lbFH0D++JMNJ3DDY= X-MC-Unique: rOYk1QwQOzWYPlGsInXv-Q-1 X-Mimecast-MFC-AGG-ID: rOYk1QwQOzWYPlGsInXv-Q_1744915322 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, John Snow , Kevin Wolf , Hanna Reitz Subject: [PATCH v2 06/11] mirror: Minor refactoring Date: Thu, 17 Apr 2025 13:39:11 -0500 Message-ID: <20250417184133.105746-19-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915421318019000 Content-Type: text/plain; charset="utf-8" Commit 5791ba52 (v9.2) pre-initialized ret in mirror_dirty_init to silence a false positive compiler warning, even though in all code paths where ret is used, it was guaranteed to be reassigned beforehand. But since the function returns -errno, and -1 is not always the right errno, it's better to initialize to -EIO. An upcoming patch wants to track two bitmaps in do_sync_target_write(); this will be easier if the current variables related to the dirty bitmap are renamed. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/mirror.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index a53582f17bb..34c6c5252e1 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -841,7 +841,7 @@ static int coroutine_fn GRAPH_UNLOCKED mirror_dirty_ini= t(MirrorBlockJob *s) int64_t offset; BlockDriverState *bs; BlockDriverState *target_bs =3D blk_bs(s->target); - int ret =3D -1; + int ret =3D -EIO; int64_t count; bdrv_graph_co_rdlock(); @@ -1341,7 +1341,7 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMetho= d method, { int ret; size_t qiov_offset =3D 0; - int64_t bitmap_offset, bitmap_end; + int64_t dirty_bitmap_offset, dirty_bitmap_end; if (!QEMU_IS_ALIGNED(offset, job->granularity) && bdrv_dirty_bitmap_get(job->dirty_bitmap, offset)) @@ -1388,11 +1388,11 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMet= hod method, * Tails are either clean or shrunk, so for bitmap resetting * we safely align the range down. */ - bitmap_offset =3D QEMU_ALIGN_UP(offset, job->granularity); - bitmap_end =3D QEMU_ALIGN_DOWN(offset + bytes, job->granularity); - if (bitmap_offset < bitmap_end) { - bdrv_reset_dirty_bitmap(job->dirty_bitmap, bitmap_offset, - bitmap_end - bitmap_offset); + dirty_bitmap_offset =3D QEMU_ALIGN_UP(offset, job->granularity); + dirty_bitmap_end =3D QEMU_ALIGN_DOWN(offset + bytes, job->granularity); + if (dirty_bitmap_offset < dirty_bitmap_end) { + bdrv_reset_dirty_bitmap(job->dirty_bitmap, dirty_bitmap_offset, + dirty_bitmap_end - dirty_bitmap_offset); } job_progress_increase_remaining(&job->common.job, bytes); @@ -1430,10 +1430,10 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMet= hod method, * at function start, and they must be still dirty, as we've locked * the region for in-flight op. */ - bitmap_offset =3D QEMU_ALIGN_DOWN(offset, job->granularity); - bitmap_end =3D QEMU_ALIGN_UP(offset + bytes, job->granularity); - bdrv_set_dirty_bitmap(job->dirty_bitmap, bitmap_offset, - bitmap_end - bitmap_offset); + dirty_bitmap_offset =3D QEMU_ALIGN_DOWN(offset, job->granularity); + dirty_bitmap_end =3D QEMU_ALIGN_UP(offset + bytes, job->granularit= y); + bdrv_set_dirty_bitmap(job->dirty_bitmap, dirty_bitmap_offset, + dirty_bitmap_end - dirty_bitmap_offset); qatomic_set(&job->actively_synced, false); action =3D mirror_error_action(job, false, -ret); --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915379; cv=none; d=zohomail.com; s=zohoarc; b=UksWtncnW5JabwsWMzlDeKCmUuDUHy5r+jXe+qNpA7XDG8esL2sNqn7lYWv0jtVsLwE75z8kH04TTYVkqeujINT4oiwAcutoEppOpkanH0E8LQp3r4peG0j2RJhIxc65veVXOyImCoqWyKhAo1/fxbD4rcHuoVI4xsnz7bexEHY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915379; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=wxuP466kXuSDE0lxOwUGNvb3Egct/43T4tpjip7S2+A=; b=fjhWrtZ+S8+u8jh2ER9qbT9STeR9b73G6oK4ie/13z7FpNxYRH36nx1YQl9Ab4MYUuvHzrOpncc4FfbK88YZBY1ms0YpiZ8Znjix/vVQpISS+ogiUyULiWAZPgzEdHqwKc1aZ0l8NLpa0HX8MR3zPs9IBPPSY/y7hlKCCaREajQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915379046516.6470103155427; Thu, 17 Apr 2025 11:42:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UC8-0004de-Mm; Thu, 17 Apr 2025 14:42:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBg-00046X-GL for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBe-0002wf-Rn for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:12 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-695-Dbq1GMjJNqmBoKMnwLiOdw-1; Thu, 17 Apr 2025 14:42:05 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7B968180098C; Thu, 17 Apr 2025 18:42:04 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 095F51956095; Thu, 17 Apr 2025 18:42:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915328; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wxuP466kXuSDE0lxOwUGNvb3Egct/43T4tpjip7S2+A=; b=Tf+Ecg+04Jx9EVbSvf82c0EYK9eCqxje5Ks1fmNPsnTa9ctFRG1J396f4IAe2ptDRb6d0F LcEsQL+jLXONOqJj+InXR2zcuyXaKzWXHV2cKJo6h1ilieOWyU5lEMqtDrYWon8YuOxZLQ wSwCfiApG53WD74AnG2Sun0155FURW4= X-MC-Unique: Dbq1GMjJNqmBoKMnwLiOdw-1 X-Mimecast-MFC-AGG-ID: Dbq1GMjJNqmBoKMnwLiOdw_1744915324 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, John Snow , Kevin Wolf , Hanna Reitz Subject: [PATCH v2 07/11] mirror: Skip pre-zeroing destination if it is already zero Date: Thu, 17 Apr 2025 13:39:12 -0500 Message-ID: <20250417184133.105746-20-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915381391019100 Content-Type: text/plain; charset="utf-8" When doing a sync=3Dfull mirroring, QMP drive-mirror requests full zeroing if it did not just create the destination, and blockdev-mirror requests full zeroing unconditionally. This is because during a full sync, we must ensure that the portions of the disk that are not otherwise touched by the source still read as zero upon completion. However, in mirror_dirty_init(), we were blindly assuming that if the destination allows punching holes, we should pre-zero the entire image; and if it does not allow punching holes, then treat the entire source as dirty rather than mirroring just the allocated portions of the source. Without the ability to punch holes, this results in the destination file being fully allocated; and even when punching holes is supported, it causes duplicate I/O to the portions of the destination corresponding to chunks of the source that are allocated but read as zero. Smarter is to avoid the pre-zeroing pass over the destination if it can be proved the destination already reads as zero. Note that a later patch will then further improve things to skip writing to the destination for parts of the image where the source is zero; but even with just this patch, it is possible to see a difference for any BDS that can quickly report that it already reads as zero. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/mirror.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/block/mirror.c b/block/mirror.c index 34c6c5252e1..234e3a55e60 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -849,13 +849,23 @@ static int coroutine_fn GRAPH_UNLOCKED mirror_dirty_i= nit(MirrorBlockJob *s) bdrv_graph_co_rdunlock(); if (s->zero_target) { + offset =3D 0; + bdrv_graph_co_rdlock(); + ret =3D bdrv_co_is_all_zeroes(target_bs); + bdrv_graph_co_rdunlock(); + if (ret < 0) { + return ret; + } + if (ret > 0) { + offset =3D s->bdev_length; + } if (!bdrv_can_write_zeroes_with_unmap(target_bs)) { bdrv_set_dirty_bitmap(s->dirty_bitmap, 0, s->bdev_length); return 0; } s->initial_zeroing_ongoing =3D true; - for (offset =3D 0; offset < s->bdev_length; ) { + while (offset < s->bdev_length) { int bytes =3D MIN(s->bdev_length - offset, QEMU_ALIGN_DOWN(INT_MAX, s->granularity)); --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915462; cv=none; d=zohomail.com; s=zohoarc; b=MurxSouzfG0LtajxBRcq/CIE8RNPqQ1j/A3DnF4XEopucu0LA1EkqXMOXpX6tp15ZjOJWjw8yNAgIV9Pcbo5fOzlG1TK2wuUfxtmsyuvBAROYv/Hc//yZ5Ujf2kBvhBy3GxbqPPNgg2ckdsbW7ePS75dOzbqsV0NBfvDEtqXE5o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915462; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=d1vRfeDREQD1hncpOR7GfsoXjDQ1SKmrSCAmi+0dkSQ=; b=MO4mPwRMpnb5VaysJYu2PastebkqnztlMXKOvJbZm5O0EAHnvft0MVDWotMwOMGSTqWM9FPpUuaqMH40vh2L/YGDW5va0OpUxtdKgTuCQg6yiU6Cw+uSBvbrIs1LH30DPaK5QDfh8XTUTS7GVQ+LldWAKoraGjQiilnOU/zYLUs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915462634128.57170747518683; Thu, 17 Apr 2025 11:44:22 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UCv-0006Yr-01; Thu, 17 Apr 2025 14:43:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UCm-0006Cd-BV for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:43:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UCk-0003B3-1c for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:43:20 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-441-iALaUuEqNaq8yDNwYNMO8w-1; Thu, 17 Apr 2025 14:42:07 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5C9AA1800370; Thu, 17 Apr 2025 18:42:06 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DC77F1956095; Thu, 17 Apr 2025 18:42:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915397; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d1vRfeDREQD1hncpOR7GfsoXjDQ1SKmrSCAmi+0dkSQ=; b=BbunRuGMuosTH0BhwX0dmWLZCmcyvmFtatD5mn4I3X0bxZz9VnTQiMmFkfRZJm5FE55Kqb slZh6ILMEtNx2by69SpxM1e/3csu4qorBiYErslTVk1VvUNOA3aJes/tpwtgGY00DqlN/j STuoR8vaKHMJh27EpvuavTcbqihdH14= X-MC-Unique: iALaUuEqNaq8yDNwYNMO8w-1 X-Mimecast-MFC-AGG-ID: iALaUuEqNaq8yDNwYNMO8w_1744915326 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, John Snow , Kevin Wolf , Hanna Reitz Subject: [PATCH v2 08/11] mirror: Skip writing zeroes when target is already zero Date: Thu, 17 Apr 2025 13:39:13 -0500 Message-ID: <20250417184133.105746-21-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915463710019100 Content-Type: text/plain; charset="utf-8" When mirroring, the goal is to ensure that the destination reads the same as the source; this goal is met whether the destination is sparse or fully-allocated. However, if the destination cannot efficiently write zeroes, then any time the mirror operation wants to copy zeroes from the source to the destination (either during the background over sparse regions when doing a full mirror, or in the foreground when the guest actively writes zeroes), we were causing the destination to fully allocate that portion of the disk, even if it already read as zeroes. The effect is especially pronounced when the source is a raw file. That's because when the source is a qcow2 file, the dirty bitmap only visits the portions of the source that are allocated, which tend to be non-zero. But when the source is a raw file, bdrv_co_is_allocated_above() reports the entire file as allocated so mirror_dirty_init sets the entire dirty bitmap, and it is only later during mirror_iteration that we change to consulting the more precise bdrv_co_block_status_above() to learn where the source reads as zero. Remember that since a mirror operation can write a cluster more than once (every time the guest changes the source, the destination is also changed to keep up), we can't take the shortcut of relying on s->zero_target (which is static for the life of the job) in mirror_co_zero() to see if the destination is already zero, because that information may be stale. Any solution we use must be dynamic in the face of the guest writing or discarding a cluster while the mirror has been ongoing. We could just teach mirror_co_zero() to do a block_status() probe of the destination, and skip the zeroes if the destination already reads as zero, but we know from past experience that extra block_status() calls are not always cheap (tmpfs, anyone?), especially when they are random access rather than linear. Use of block_status() of the source by the background task in a linear fashion is not our bottleneck (it's a background task, after all); but since mirroring can be done while the source is actively being changed, we don't want a slow block_status() of the destination to occur on the hot path of the guest trying to do random-access writes to the source. So this patch takes a slightly different approach: any time we have to transfer the full image, we know that mirror_dirty_init() is _already_ doing a pre-zero pass over the entire destination. Therefore, if we track which clusters of the destination are zero at any given moment, we don't have to do a block_status() call on the destination, but can instead just refer to the zero bitmap associated with the job. With this patch, if I create a raw sparse destination file, connect it with QMP 'blockdev-add' while leaving it at the default "discard": "ignore", then run QMP 'blockdev-mirror' with "sync": "full", the destination remains sparse rather than fully allocated. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/mirror.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 65 insertions(+), 5 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 234e3a55e60..4770d87abf6 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -73,6 +73,7 @@ typedef struct MirrorBlockJob { size_t buf_size; int64_t bdev_length; unsigned long *cow_bitmap; + unsigned long *zero_bitmap; BdrvDirtyBitmap *dirty_bitmap; BdrvDirtyBitmapIter *dbi; uint8_t *buf; @@ -408,15 +409,33 @@ static void coroutine_fn mirror_co_read(void *opaque) static void coroutine_fn mirror_co_zero(void *opaque) { MirrorOp *op =3D opaque; - int ret; + bool write_needed =3D true; + int ret =3D 0; op->s->in_flight++; op->s->bytes_in_flight +=3D op->bytes; *op->bytes_handled =3D op->bytes; op->is_in_flight =3D true; - ret =3D blk_co_pwrite_zeroes(op->s->target, op->offset, op->bytes, - op->s->unmap ? BDRV_REQ_MAY_UNMAP : 0); + if (op->s->zero_bitmap) { + unsigned long end =3D DIV_ROUND_UP(op->offset + op->bytes, + op->s->granularity); + assert(QEMU_IS_ALIGNED(op->offset, op->s->granularity)); + assert(QEMU_IS_ALIGNED(op->bytes, op->s->granularity) || + op->offset + op->bytes =3D=3D op->s->bdev_length); + if (find_next_zero_bit(op->s->zero_bitmap, end, + op->offset / op->s->granularity) =3D=3D end= ) { + write_needed =3D false; + } + } + if (write_needed) { + ret =3D blk_co_pwrite_zeroes(op->s->target, op->offset, op->bytes, + op->s->unmap ? BDRV_REQ_MAY_UNMAP : 0); + } + if (ret >=3D 0 && op->s->zero_bitmap) { + bitmap_set(op->s->zero_bitmap, op->offset / op->s->granularity, + DIV_ROUND_UP(op->bytes, op->s->granularity)); + } mirror_write_complete(op, ret); } @@ -441,6 +460,9 @@ static unsigned mirror_perform(MirrorBlockJob *s, int64= _t offset, Coroutine *co; int64_t bytes_handled =3D -1; + assert(QEMU_IS_ALIGNED(offset, s->granularity)); + assert(QEMU_IS_ALIGNED(bytes, s->granularity) || + offset + bytes =3D=3D s->bdev_length); op =3D g_new(MirrorOp, 1); *op =3D (MirrorOp){ .s =3D s, @@ -452,12 +474,21 @@ static unsigned mirror_perform(MirrorBlockJob *s, int= 64_t offset, switch (mirror_method) { case MIRROR_METHOD_COPY: + if (s->zero_bitmap) { + bitmap_clear(s->zero_bitmap, offset / s->granularity, + DIV_ROUND_UP(bytes, s->granularity)); + } co =3D qemu_coroutine_create(mirror_co_read, op); break; case MIRROR_METHOD_ZERO: + /* s->zero_bitmap handled in mirror_co_zero */ co =3D qemu_coroutine_create(mirror_co_zero, op); break; case MIRROR_METHOD_DISCARD: + if (s->zero_bitmap) { + bitmap_clear(s->zero_bitmap, offset / s->granularity, + DIV_ROUND_UP(bytes, s->granularity)); + } co =3D qemu_coroutine_create(mirror_co_discard, op); break; default: @@ -849,6 +880,8 @@ static int coroutine_fn GRAPH_UNLOCKED mirror_dirty_ini= t(MirrorBlockJob *s) bdrv_graph_co_rdunlock(); if (s->zero_target) { + int64_t bitmap_length =3D DIV_ROUND_UP(s->bdev_length, s->granular= ity); + offset =3D 0; bdrv_graph_co_rdlock(); ret =3D bdrv_co_is_all_zeroes(target_bs); @@ -856,7 +889,9 @@ static int coroutine_fn GRAPH_UNLOCKED mirror_dirty_ini= t(MirrorBlockJob *s) if (ret < 0) { return ret; } + s->zero_bitmap =3D bitmap_new(bitmap_length); if (ret > 0) { + bitmap_set(s->zero_bitmap, 0, bitmap_length); offset =3D s->bdev_length; } if (!bdrv_can_write_zeroes_with_unmap(target_bs)) { @@ -1173,6 +1208,7 @@ immediate_exit: assert(s->in_flight =3D=3D 0); qemu_vfree(s->buf); g_free(s->cow_bitmap); + g_free(s->zero_bitmap); g_free(s->in_flight_bitmap); bdrv_dirty_iter_free(s->dbi); @@ -1352,6 +1388,7 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMetho= d method, int ret; size_t qiov_offset =3D 0; int64_t dirty_bitmap_offset, dirty_bitmap_end; + int64_t zero_bitmap_offset, zero_bitmap_end; if (!QEMU_IS_ALIGNED(offset, job->granularity) && bdrv_dirty_bitmap_get(job->dirty_bitmap, offset)) @@ -1395,8 +1432,9 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMetho= d method, } /* - * Tails are either clean or shrunk, so for bitmap resetting - * we safely align the range down. + * Tails are either clean or shrunk, so for dirty bitmap resetting + * we safely align the range narrower. But for zero bitmap, round + * range wider for checking or clearing, and narrower for setting. */ dirty_bitmap_offset =3D QEMU_ALIGN_UP(offset, job->granularity); dirty_bitmap_end =3D QEMU_ALIGN_DOWN(offset + bytes, job->granularity); @@ -1404,22 +1442,44 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMet= hod method, bdrv_reset_dirty_bitmap(job->dirty_bitmap, dirty_bitmap_offset, dirty_bitmap_end - dirty_bitmap_offset); } + zero_bitmap_offset =3D offset / job->granularity; + zero_bitmap_end =3D DIV_ROUND_UP(offset + bytes, job->granularity); job_progress_increase_remaining(&job->common.job, bytes); job->active_write_bytes_in_flight +=3D bytes; switch (method) { case MIRROR_METHOD_COPY: + if (job->zero_bitmap) { + bitmap_clear(job->zero_bitmap, zero_bitmap_offset, + zero_bitmap_end - zero_bitmap_offset); + } ret =3D blk_co_pwritev_part(job->target, offset, bytes, qiov, qiov_offset, flags); break; case MIRROR_METHOD_ZERO: + if (job->zero_bitmap) { + if (find_next_zero_bit(job->zero_bitmap, zero_bitmap_end, + zero_bitmap_offset) =3D=3D zero_bitmap_= end) { + ret =3D 0; + break; + } + } assert(!qiov); ret =3D blk_co_pwrite_zeroes(job->target, offset, bytes, flags); + if (job->zero_bitmap && ret >=3D 0) { + bitmap_set(job->zero_bitmap, dirty_bitmap_offset / job->granul= arity, + (dirty_bitmap_end - dirty_bitmap_offset) / + job->granularity); + } break; case MIRROR_METHOD_DISCARD: + if (job->zero_bitmap) { + bitmap_clear(job->zero_bitmap, zero_bitmap_offset, + zero_bitmap_end - zero_bitmap_offset); + } assert(!qiov); ret =3D blk_co_pdiscard(job->target, offset, bytes); break; --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915420; cv=none; d=zohomail.com; s=zohoarc; b=iVqPxrHIDiZAGjDDPzMR/Ua8+LWNIhuehtfZu3jR3YnWK08kDEBEpoe9RAU2VWSuT0wAzAm5F+2BWlYCp47LUBnynGTFFupCY6hyKkG1XDPgwQ5R0AdOwTPvyXjanqqvKR58/Hz/f2S25LfOCD20oQ0QvEGitzvpPPcGSRdWcCQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915420; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SXatnc1b9eqCASVVPcev/TVCCApUnlZ5cjVc5du+s4c=; b=haWsFNd6ZHZEoUV510qf6L8iTgEjkWXU+3QNc/X5g6Ik5WYmg2ziLF/0P9KubuLMXw5dyC+AJaEpa4pvjNL2SJtoxgVHDICh3RNkJYFkHDZHFvJ8zbbQGT87c0YdF4V6xR1dXDAdrZCeUvvDpgGV7hbOdvQAodoxjUTlqRenODs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915420494474.585224362081; Thu, 17 Apr 2025 11:43:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UC5-0004V4-97; Thu, 17 Apr 2025 14:42:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBm-0004Av-9q for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBk-0002yw-5Q for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:17 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-447-MlAgc0OXP_OQVKYlexQUgw-1; Thu, 17 Apr 2025 14:42:11 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C1F0E19560AE; Thu, 17 Apr 2025 18:42:09 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BFE1C1956095; Thu, 17 Apr 2025 18:42:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915335; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SXatnc1b9eqCASVVPcev/TVCCApUnlZ5cjVc5du+s4c=; b=a76FVBYRd8ZUq2fJMJfYdZ4EesIJOYCPCnDEw6Kp1xzHOFftny0XktwknC3bgZCbpd8wQR LR6zxGE3Wiq/OlEIjXFxkGXCj01BDXuJTuXj7Q0oYbINh8oYmSDI3iITYprvD1j4Ywk0+h 1JtOErvbpfyN0QeMVwFhwSgMVQBv7gY= X-MC-Unique: MlAgc0OXP_OQVKYlexQUgw-1 X-Mimecast-MFC-AGG-ID: MlAgc0OXP_OQVKYlexQUgw_1744915330 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, Andrey Drobyshev , Alexander Ivanov , Alberto Garcia , Kevin Wolf , Hanna Reitz Subject: [PATCH v2 09/11] iotests/common.rc: add disk_usage function Date: Thu, 17 Apr 2025 13:39:14 -0500 Message-ID: <20250417184133.105746-22-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915421289019000 Content-Type: text/plain; charset="utf-8" From: Andrey Drobyshev Move the definition from iotests/250 to common.rc. This is used to detect real disk usage of sparse files. In particular, we want to use it for checking subclusters-based discards. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov Reviewed-by: Alberto Garcia Message-ID: <20240913163942.423050-6-andrey.drobyshev@virtuozzo.com> Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/common.rc | 6 ++++++ tests/qemu-iotests/250 | 5 ----- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 95c12577dd4..237f746af88 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -140,6 +140,12 @@ _optstr_add() fi } +# report real disk usage for sparse files +disk_usage() +{ + du --block-size=3D1 "$1" | awk '{print $1}' +} + # Set the variables to the empty string to turn Valgrind off # for specific processes, e.g. # $ VALGRIND_QEMU_IO=3D ./check -qcow2 -valgrind 015 diff --git a/tests/qemu-iotests/250 b/tests/qemu-iotests/250 index af48f83abac..c0a0dbc0ff1 100755 --- a/tests/qemu-iotests/250 +++ b/tests/qemu-iotests/250 @@ -52,11 +52,6 @@ _unsupported_imgopts data_file # bdrv_co_truncate(bs->file) call in qcow2_co_truncate(), which might succ= eed # anyway. -disk_usage() -{ - du --block-size=3D1 $1 | awk '{print $1}' -} - size=3D2100M _make_test_img -o "cluster_size=3D1M,preallocation=3Dmetadata" $size --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915365; cv=none; d=zohomail.com; s=zohoarc; b=Sb2+ptZbMuChdOyRlEKA1FcDmVIYsP7NjIb5coNpG1tR7UhVNY3/3af1DKMxq4LTNIxy2D2Gw45OdwMpJnk8+Zf0aAebQakhGbasqzW1a0r6W/zab/t4jkVpsz77kjfofREJflR/WXAS3yG8DV8tigzCoB0UPAKmaKaub/ryk7w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915365; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ql4bxEbP2+dUcmQSrh9zv04ppkyak1jsM4syv7TsdI8=; b=auKRjQhjk2pDoLjv8z9G3gpWwfZVeRMbcQqvNmXkyp6vkWLTMczwjhAyLok7UUR5H5JOGugEzfgzIyCgB91CAtmQT3YXiv5aacx8ZDq1GMfA79Es7FgGdFWTl1Ve35tjJYTuyms8hdsxtlIvArrvFEjNoi/h1ea88D6Mz81UagU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915365772668.7423231165394; Thu, 17 Apr 2025 11:42:45 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UCC-0004ph-02; Thu, 17 Apr 2025 14:42:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBn-0004D9-Jr for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBk-0002z9-QV for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:19 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-133-HuR3-47_PSCYEelb_kaiLg-1; Thu, 17 Apr 2025 14:42:12 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 807BC195608A; Thu, 17 Apr 2025 18:42:11 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 39CE51956095; Thu, 17 Apr 2025 18:42:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915335; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ql4bxEbP2+dUcmQSrh9zv04ppkyak1jsM4syv7TsdI8=; b=jUcgRSKVdADflukf131TVZf4u8pqx0tPSqMMz2uJhiv+XttH6PS1YcmQCdtwojfn6BbWXR GF49942JOAZX+NPTnxMgpn++DUgBrkcdNhZGae70pzlexAljO3OZYO5Pg6eMxnLoNtI6Hr pNOH1/awWqYvoCTk1cIOijUOgXkeCGA= X-MC-Unique: HuR3-47_PSCYEelb_kaiLg-1 X-Mimecast-MFC-AGG-ID: HuR3-47_PSCYEelb_kaiLg_1744915331 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, Kevin Wolf , Hanna Reitz Subject: [PATCH v2 10/11] tests: Add iotest mirror-sparse for recent patches Date: Thu, 17 Apr 2025 13:39:15 -0500 Message-ID: <20250417184133.105746-23-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915367226019000 Content-Type: text/plain; charset="utf-8" Prove that blockdev-mirror can now result in sparse raw destination files, regardless of whether the source is raw or qcow2. By making this a separate test, it was possible to test effects of individual patches for the various pieces that all have to work together for a sparse mirror to be successful. Note that ./check -file produces different job lengths than ./check -qcow2 (the test uses a filter to normalize); that's because when deciding how much of the image to be mirrored, the code looks at how much of the source image was allocated (for qcow2, this is only the written clusters; for raw, it is the entire file). But the important part is that the destination file ends up smaller than 3M, rather than the 20M it used to be before this patch series. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/tests/mirror-sparse | 109 +++++++++++++++ tests/qemu-iotests/tests/mirror-sparse.out | 153 +++++++++++++++++++++ 2 files changed, 262 insertions(+) create mode 100755 tests/qemu-iotests/tests/mirror-sparse create mode 100644 tests/qemu-iotests/tests/mirror-sparse.out diff --git a/tests/qemu-iotests/tests/mirror-sparse b/tests/qemu-iotests/te= sts/mirror-sparse new file mode 100755 index 00000000000..ef68542a122 --- /dev/null +++ b/tests/qemu-iotests/tests/mirror-sparse @@ -0,0 +1,109 @@ +#!/usr/bin/env bash +# group: rw auto quick +# +# Test blockdev-mirror with raw sparse destination +# +# Copyright (C) 2025 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +seq=3D"$(basename $0)" +echo "QA output created by $seq" + +status=3D1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + _cleanup_qemu +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +cd .. +. ./common.rc +. ./common.filter +. ./common.qemu + +_supported_fmt qcow2 raw # Format of the source. dst is always raw file +_supported_proto file +_supported_os Linux + +filter_len() { + sed -e 's/"len": [0-9]*/"len": LEN/g' \ + -e 's/"offset": [0-9]*/"offset": OFFSET/g' +} + +echo +echo "=3D=3D=3D Initial image setup =3D=3D=3D" +echo + +TEST_IMG=3D"$TEST_IMG.base" _make_test_img 20M +$QEMU_IO -c 'w 8M 2M' -f $IMGFMT "$TEST_IMG.base" | _filter_qemu_io + +_launch_qemu -machine q35 \ + -blockdev '{"driver":"file", "cache":{"direct":true, "no-flush":false}, + "filename":"'"$TEST_IMG.base"'", "node-name":"src-file"}' \ + -blockdev '{"driver":"'$IMGFMT'", "node-name":"src", "file":"src-file"= }' +h1=3D$QEMU_HANDLE +_send_qemu_cmd $h1 '{"execute": "qmp_capabilities"}' 'return' + +# Each of these combinations should result in a sparse destination; +# the destination should only be fully allocated if pre-allocated +for creation in external blockdev-create; do +for discard in ignore unmap; do + +echo +echo "=3D=3D=3D Destination with $creation creation and discard=3D$discard= =3D=3D=3D" +echo + +rm -f $TEST_IMG +if test $creation =3D external; then + truncate --size=3D20M $TEST_IMG +else + _send_qemu_cmd $h1 '{"execute": "blockdev-create", "arguments": + {"options": {"driver":"file", "filename":"'$TEST_IMG'", + "size":'$((20*1024*1024))'}, "job-id":"job1"}}' 'concluded' + _send_qemu_cmd $h1 '{"execute": "job-dismiss", "arguments": + {"id": "job1"}}' 'return' +fi +_send_qemu_cmd $h1 '{"execute": "blockdev-add", "arguments": + {"node-name": "dst", "driver":"file", + "filename":"'$TEST_IMG'", "aio":"threads", + "auto-read-only":true, "discard":"'$discard'"}}' 're= turn' +_send_qemu_cmd $h1 '{"execute":"blockdev-mirror", "arguments": + {"sync":"full", "device":"src", "target":"dst", + "job-id":"job2"}}' 'return' +_timed_wait_for $h1 '"ready"' +_send_qemu_cmd $h1 '{"execute": "job-complete", "arguments": + {"id":"job2"}}' 'return' | filter_len +_send_qemu_cmd $h1 '{"execute": "blockdev-del", "arguments": + {"node-name": "dst"}}' 'return' | filter_len +$QEMU_IMG compare -U -f $IMGFMT -F raw $TEST_IMG.base $TEST_IMG +if test $(disk_usage $TEST_IMG) -lt $((3*1024*1024)); then + echo "destination image is sparse" +else + echo "destination not sparse as expected" +fi + +done +done + +_send_qemu_cmd $h1 '{"execute":"quit"}' '' + +# success, all done +echo '*** done' +rm -f $seq.full +status=3D0 diff --git a/tests/qemu-iotests/tests/mirror-sparse.out b/tests/qemu-iotest= s/tests/mirror-sparse.out new file mode 100644 index 00000000000..ee569b92868 --- /dev/null +++ b/tests/qemu-iotests/tests/mirror-sparse.out @@ -0,0 +1,153 @@ +QA output created by mirror-sparse + +=3D=3D=3D Initial image setup =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D20971520 +wrote 2097152/2097152 bytes at offset 8388608 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +{"execute": "qmp_capabilities"} +{"return": {}} + +=3D=3D=3D Destination with external creation and discard=3Dignore =3D=3D= =3D + +{"execute": "blockdev-add", "arguments": + {"node-name": "dst", "driver":"file", + "filename":"TEST_DIR/t.IMGFMT", "aio":"threads", + "auto-read-only":true, "discard":"ignore"}} +{"return": {}} +{"execute":"blockdev-mirror", "arguments": + {"sync":"full", "device":"src", "target":"dst", + "job-id":"job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job2"}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job2"}} +{"execute": "job-complete", "arguments": + {"id":"job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_READY", "data": {"device": "job2", "len": LEN, "offset": OFFSE= T, "speed": 0, "type": "mirror"}} +{"return": {}} +{"execute": "blockdev-del", "arguments": + {"node-name": "dst"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_COMPLETED", "data": {"device": "job2", "len": LEN, "offset": O= FFSET, "speed": 0, "type": "mirror"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job2"}} +{"return": {}} +Images are identical. +destination image is sparse + +=3D=3D=3D Destination with external creation and discard=3Dunmap =3D=3D=3D + +{"execute": "blockdev-add", "arguments": + {"node-name": "dst", "driver":"file", + "filename":"TEST_DIR/t.IMGFMT", "aio":"threads", + "auto-read-only":true, "discard":"unmap"}} +{"return": {}} +{"execute":"blockdev-mirror", "arguments": + {"sync":"full", "device":"src", "target":"dst", + "job-id":"job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job2"}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job2"}} +{"execute": "job-complete", "arguments": + {"id":"job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_READY", "data": {"device": "job2", "len": LEN, "offset": OFFSE= T, "speed": 0, "type": "mirror"}} +{"return": {}} +{"execute": "blockdev-del", "arguments": + {"node-name": "dst"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_COMPLETED", "data": {"device": "job2", "len": LEN, "offset": O= FFSET, "speed": 0, "type": "mirror"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job2"}} +{"return": {}} +Images are identical. +destination image is sparse + +=3D=3D=3D Destination with blockdev-create creation and discard=3Dignore = =3D=3D=3D + +{"execute": "blockdev-create", "arguments": + {"options": {"driver":"file", "filename":"TEST_DIR/t.IMGFMT", + "size":20971520}, "job-id":"job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job1"}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job1"}} +{"execute": "job-dismiss", "arguments": + {"id": "job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job1"}} +{"return": {}} +{"execute": "blockdev-add", "arguments": + {"node-name": "dst", "driver":"file", + "filename":"TEST_DIR/t.IMGFMT", "aio":"threads", + "auto-read-only":true, "discard":"ignore"}} +{"return": {}} +{"execute":"blockdev-mirror", "arguments": + {"sync":"full", "device":"src", "target":"dst", + "job-id":"job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job2"}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job2"}} +{"execute": "job-complete", "arguments": + {"id":"job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_READY", "data": {"device": "job2", "len": LEN, "offset": OFFSE= T, "speed": 0, "type": "mirror"}} +{"return": {}} +{"execute": "blockdev-del", "arguments": + {"node-name": "dst"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_COMPLETED", "data": {"device": "job2", "len": LEN, "offset": O= FFSET, "speed": 0, "type": "mirror"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job2"}} +{"return": {}} +Images are identical. +destination image is sparse + +=3D=3D=3D Destination with blockdev-create creation and discard=3Dunmap = =3D=3D=3D + +{"execute": "blockdev-create", "arguments": + {"options": {"driver":"file", "filename":"TEST_DIR/t.IMGFMT", + "size":20971520}, "job-id":"job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job1"}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job1"}} +{"execute": "job-dismiss", "arguments": + {"id": "job1"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job1"}} +{"return": {}} +{"execute": "blockdev-add", "arguments": + {"node-name": "dst", "driver":"file", + "filename":"TEST_DIR/t.IMGFMT", "aio":"threads", + "auto-read-only":true, "discard":"unmap"}} +{"return": {}} +{"execute":"blockdev-mirror", "arguments": + {"sync":"full", "device":"src", "target":"dst", + "job-id":"job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job2"}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job2"}} +{"execute": "job-complete", "arguments": + {"id":"job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_READY", "data": {"device": "job2", "len": LEN, "offset": OFFSE= T, "speed": 0, "type": "mirror"}} +{"return": {}} +{"execute": "blockdev-del", "arguments": + {"node-name": "dst"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_COMPLETED", "data": {"device": "job2", "len": LEN, "offset": O= FFSET, "speed": 0, "type": "mirror"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job2"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job2"}} +{"return": {}} +Images are identical. +destination image is sparse +{"execute":"quit"} +*** done --=20 2.49.0 From nobody Sun Oct 5 13:58:16 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1744915398; cv=none; d=zohomail.com; s=zohoarc; b=h1dAW8aMjCmtUkakzgTTfLEhb4ft1S7RVeW0AVBPeqlHkVyQYKiyfv042AJ4S0lmBfEhbnKJXTpYANmEWqK9F4f+sYj5ph0QmvEUOGNFhJV5LwrdFlgusJLkUqBI7pXp/veA6YdNTKRlQDs/BtS4ASxggsnwIabaTKcVdM1hIiw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744915398; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9TSvZOan1GP0VtSPKXM3CrFSicyuh3mnQKzM0bDOKOE=; b=Y5aW6yqiyAWklCAnW/OLqp0FwvIWx5zGemBPyqN0aMFnxuNol6dS+o0kCDyk/u4VkqbtlV9z5Slm72gZX6oImDu+RX7ZZGvtYL3VNlS+4Y+4j4Ih8oE/NRiw9OaYEa++Ei7doN5pdaWwTI/N+4zFh+qUJ6sPUvUgAuBIzVHBW/k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744915398408622.1342182949419; Thu, 17 Apr 2025 11:43:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5UCA-0004jz-Dx; Thu, 17 Apr 2025 14:42:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBr-0004E3-L7 for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5UBp-00030l-NE for qemu-devel@nongnu.org; Thu, 17 Apr 2025 14:42:23 -0400 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-12-TpUChdHxOAGvLvui-XKwrw-1; Thu, 17 Apr 2025 14:42:14 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9DF7119560AB; Thu, 17 Apr 2025 18:42:13 +0000 (UTC) Received: from green.redhat.com (unknown [10.2.16.121]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id F3F9C1956095; Thu, 17 Apr 2025 18:42:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744915340; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9TSvZOan1GP0VtSPKXM3CrFSicyuh3mnQKzM0bDOKOE=; b=IOzIGX23uYZeouvqSfKL9z7OlxsxxKhw3US8vRIK2W4hfe/q8HCEGQTLTynXRppA5gloPT kjHgdaSzPZ4NFqtDvLfv6zMqsNvmJn10zxqUDNczXj3UUp1P1wouC0Sk6rcM6Tx8oFTL5W AYFq3iZqz3llJOG3J5VVqDchDoAhN1g= X-MC-Unique: TpUChdHxOAGvLvui-XKwrw-1 X-Mimecast-MFC-AGG-ID: TpUChdHxOAGvLvui-XKwrw_1744915333 From: Eric Blake To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, vsementsov@yandex-team.ru, John Snow , Kevin Wolf , Hanna Reitz , Markus Armbruster Subject: [PATCH v2 11/11] mirror: Allow QMP override to declare target already zero Date: Thu, 17 Apr 2025 13:39:16 -0500 Message-ID: <20250417184133.105746-24-eblake@redhat.com> In-Reply-To: <20250417184133.105746-13-eblake@redhat.com> References: <20250417184133.105746-13-eblake@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1744915401840019100 Content-Type: text/plain; charset="utf-8" QEMU's attempts to learn whether a destination file starts life with all zero contents are just a heuristic. There may be cases where the caller is aware of information that QEMU cannot learn quickly, in which case telling QEMU what to assume about the destination can make the mirror operation faster. Given our existing example of "qemu-img convert --target-is-zero", it is time to expose this override in QMP for blockdev-mirror as well. Signed-off-by: Eric Blake Acked-by: Markus Armbruster Reviewed-by: Stefan Hajnoczi --- qapi/block-core.json | 8 +++++++- include/block/block_int-global-state.h | 3 ++- block/mirror.c | 23 +++++++++++++++-------- blockdev.c | 18 +++++++++++------- tests/unit/test-block-iothread.c | 2 +- 5 files changed, 36 insertions(+), 18 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index b1937780e19..7f70ec6d3cb 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2538,6 +2538,11 @@ # disappear from the query list without user intervention. # Defaults to true. (Since 3.1) # +# @target-is-zero: Assume the destination reads as all zeroes before +# the mirror started. Setting this to true can speed up the +# mirror. Setting this to true when the destination is not +# actually all zero can corrupt the destination. (Since 10.1) +# # Since: 2.6 # # .. qmp-example:: @@ -2557,7 +2562,8 @@ '*on-target-error': 'BlockdevOnError', '*filter-node-name': 'str', '*copy-mode': 'MirrorCopyMode', - '*auto-finalize': 'bool', '*auto-dismiss': 'bool' }, + '*auto-finalize': 'bool', '*auto-dismiss': 'bool', + '*target-is-zero': 'bool'}, 'allow-preconfig': true } ## diff --git a/include/block/block_int-global-state.h b/include/block/block_i= nt-global-state.h index eb2d92a2261..a2b96f90d44 100644 --- a/include/block/block_int-global-state.h +++ b/include/block/block_int-global-state.h @@ -140,6 +140,7 @@ BlockJob *commit_active_start(const char *job_id, Block= DriverState *bs, * @mode: Whether to collapse all images in the chain to the target. * @backing_mode: How to establish the target's backing chain after comple= tion. * @zero_target: Whether the target should be explicitly zero-initialized + * @target_is_zero: Whether the target already is zero-initialized * @on_source_error: The action to take upon error reading from the source. * @on_target_error: The action to take upon error writing to the target. * @unmap: Whether to unmap target where source sectors only contain zeroe= s. @@ -159,7 +160,7 @@ void mirror_start(const char *job_id, BlockDriverState = *bs, int creation_flags, int64_t speed, uint32_t granularity, int64_t buf_size, MirrorSyncMode mode, BlockMirrorBackingMode backing_mode, - bool zero_target, + bool zero_target, bool target_is_zero, BlockdevOnError on_source_error, BlockdevOnError on_target_error, bool unmap, const char *filter_node_name, diff --git a/block/mirror.c b/block/mirror.c index 4770d87abf6..22cca5d22aa 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -55,6 +55,8 @@ typedef struct MirrorBlockJob { BlockMirrorBackingMode backing_mode; /* Whether the target image requires explicit zero-initialization */ bool zero_target; + /* Whether the target should be assumed to be already zero initialized= */ + bool target_is_zero; /* * To be accesssed with atomics. Written only under the BQL (required = by the * current implementation of mirror_change()). @@ -883,9 +885,13 @@ static int coroutine_fn GRAPH_UNLOCKED mirror_dirty_in= it(MirrorBlockJob *s) int64_t bitmap_length =3D DIV_ROUND_UP(s->bdev_length, s->granular= ity); offset =3D 0; - bdrv_graph_co_rdlock(); - ret =3D bdrv_co_is_all_zeroes(target_bs); - bdrv_graph_co_rdunlock(); + if (s->target_is_zero) { + ret =3D 1; + } else { + bdrv_graph_co_rdlock(); + ret =3D bdrv_co_is_all_zeroes(target_bs); + bdrv_graph_co_rdunlock(); + } if (ret < 0) { return ret; } @@ -1782,7 +1788,7 @@ static BlockJob *mirror_start_job( const char *replaces, int64_t speed, uint32_t granularity, int64_t buf_size, BlockMirrorBackingMode backing_mode, - bool zero_target, + bool zero_target, bool target_is_zero, BlockdevOnError on_source_error, BlockdevOnError on_target_error, bool unmap, @@ -1951,6 +1957,7 @@ static BlockJob *mirror_start_job( s->is_none_mode =3D is_none_mode; s->backing_mode =3D backing_mode; s->zero_target =3D zero_target; + s->target_is_zero =3D target_is_zero; qatomic_set(&s->copy_mode, copy_mode); s->base =3D base; s->base_overlay =3D bdrv_find_overlay(bs, base); @@ -2079,7 +2086,7 @@ void mirror_start(const char *job_id, BlockDriverStat= e *bs, int creation_flags, int64_t speed, uint32_t granularity, int64_t buf_size, MirrorSyncMode mode, BlockMirrorBackingMode backing_mode, - bool zero_target, + bool zero_target, bool target_is_zero, BlockdevOnError on_source_error, BlockdevOnError on_target_error, bool unmap, const char *filter_node_name, @@ -2104,8 +2111,8 @@ void mirror_start(const char *job_id, BlockDriverStat= e *bs, mirror_start_job(job_id, bs, creation_flags, target, replaces, speed, granularity, buf_size, backing_mode, zero_targ= et, - on_source_error, on_target_error, unmap, NULL, NULL, - &mirror_job_driver, is_none_mode, base, false, + target_is_zero, on_source_error, on_target_error, unm= ap, + NULL, NULL, &mirror_job_driver, is_none_mode, base, f= alse, filter_node_name, true, copy_mode, false, errp); } @@ -2131,7 +2138,7 @@ BlockJob *commit_active_start(const char *job_id, Blo= ckDriverState *bs, job =3D mirror_start_job( job_id, bs, creation_flags, base, NULL, speed, 0, 0, - MIRROR_LEAVE_BACKING_CHAIN, false, + MIRROR_LEAVE_BACKING_CHAIN, false, false, on_error, on_error, true, cb, opaque, &commit_active_job_driver, false, base, auto_complete, filter_node_name, false, MIRROR_COPY_MODE_BACKGROUND, diff --git a/blockdev.c b/blockdev.c index 1d1f27cfff6..6f5373991c8 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2798,7 +2798,7 @@ static void blockdev_mirror_common(const char *job_id= , BlockDriverState *bs, const char *replaces, enum MirrorSyncMode sync, BlockMirrorBackingMode backing_mode, - bool zero_target, + bool zero_target, bool target_is_zero, bool has_speed, int64_t speed, bool has_granularity, uint32_t granular= ity, bool has_buf_size, int64_t buf_size, @@ -2909,11 +2909,10 @@ static void blockdev_mirror_common(const char *job_= id, BlockDriverState *bs, /* pass the node name to replace to mirror start since it's loose coup= ling * and will allow to check whether the node still exist at mirror comp= letion */ - mirror_start(job_id, bs, target, - replaces, job_flags, + mirror_start(job_id, bs, target, replaces, job_flags, speed, granularity, buf_size, sync, backing_mode, zero_ta= rget, - on_source_error, on_target_error, unmap, filter_node_name, - copy_mode, errp); + target_is_zero, on_source_error, on_target_error, unmap, + filter_node_name, copy_mode, errp); } void qmp_drive_mirror(DriveMirror *arg, Error **errp) @@ -2928,6 +2927,7 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp) int64_t size; const char *format =3D arg->format; bool zero_target; + bool target_is_zero; int ret; bs =3D qmp_get_root_bs(arg->device, errp); @@ -3044,6 +3044,8 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp) zero_target =3D (arg->sync =3D=3D MIRROR_SYNC_MODE_FULL && (arg->mode =3D=3D NEW_IMAGE_MODE_EXISTING || !bdrv_has_zero_init(target_bs))); + target_is_zero =3D (arg->mode =3D=3D NEW_IMAGE_MODE_ABSOLUTE_PATHS && + bdrv_has_zero_init(target_bs)); bdrv_graph_rdunlock_main_loop(); @@ -3055,7 +3057,7 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp) blockdev_mirror_common(arg->job_id, bs, target_bs, arg->replaces, arg->sync, - backing_mode, zero_target, + backing_mode, zero_target, target_is_zero, arg->has_speed, arg->speed, arg->has_granularity, arg->granularity, arg->has_buf_size, arg->buf_size, @@ -3085,6 +3087,7 @@ void qmp_blockdev_mirror(const char *job_id, bool has_copy_mode, MirrorCopyMode copy_mode, bool has_auto_finalize, bool auto_finalize, bool has_auto_dismiss, bool auto_dismiss, + bool has_target_is_zero, bool target_is_zero, Error **errp) { BlockDriverState *bs; @@ -3115,7 +3118,8 @@ void qmp_blockdev_mirror(const char *job_id, blockdev_mirror_common(job_id, bs, target_bs, replaces, sync, backing_mode, - zero_target, has_speed, speed, + zero_target, has_target_is_zero && target_is_ze= ro, + has_speed, speed, has_granularity, granularity, has_buf_size, buf_size, has_on_source_error, on_source_error, diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothr= ead.c index 8189b32fd52..ffc878d401e 100644 --- a/tests/unit/test-block-iothread.c +++ b/tests/unit/test-block-iothread.c @@ -755,7 +755,7 @@ static void test_propagate_mirror(void) /* Start a mirror job */ mirror_start("job0", src, target, NULL, JOB_DEFAULT, 0, 0, 0, - MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN, false, + MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN, false, = false, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, false, "filter_node", MIRROR_COPY_MODE_BACKGROUND, &error_abort); --=20 2.49.0