From nobody Sat Nov 23 18:16:36 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17299603670571016.9060554079027; Sat, 26 Oct 2024 09:32:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t4jh9-0007wP-Iu; Sat, 26 Oct 2024 12:31:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jgi-0007s3-3q; Sat, 26 Oct 2024 12:30:52 -0400 Received: from icts-p-cavuit-4.kulnet.kuleuven.be ([134.58.240.134]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jge-0003bc-99; Sat, 26 Oct 2024 12:30:51 -0400 Received: from icts-p-ceifnet-smtps-1.kuleuven.be (icts-p-ceifnet-smtps.service.icts.svcd [IPv6:2a02:2c40:0:51:135:242:ac11:35]) by icts-p-cavuit-4.kulnet.kuleuven.be (Postfix) with ESMTP id D28AF106; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) Received: from flutterbat.volkihar.be (236.58-240-81.adsl-dyn.isp.belgacom.be [81.240.58.236]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by icts-p-ceifnet-smtps-1.kuleuven.be (Postfix) with ESMTPSA id B5DADD4F17F51; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) Received: by flutterbat.volkihar.be (Postfix, from userid 1000) id 743004B91C6; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) X-KULeuven-Envelope-From: libvirt-e6954efa@volkihar.be X-KULeuven-Scanned: Found to be clean X-KULeuven-ID: D28AF106.AF92C X-KULeuven-Information: Katholieke Universiteit Leuven BCmilterd-Mark-Subject: no BCmilterd-Errors: BCmilterd-Report: X-CAV-Cluster: smtps X-Kuleuven: This mail passed the K.U.Leuven mailcluster From: Vincent Vanlaer To: qemu-devel@nongnu.org Cc: John Snow , Vladimir Sementsov-Ogievskiy , Kevin Wolf , qemu-block@nongnu.org, Hanna Reitz , Vincent Vanlaer Subject: [PATCH v4 1/5] block: get type of block allocation in commit_run Date: Sat, 26 Oct 2024 18:30:05 +0200 Message-ID: <20241026163010.2865002-2-libvirt-e6954efa@volkihar.be> X-Mailer: git-send-email 2.44.1 In-Reply-To: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> References: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=134.58.240.134; envelope-from=libvirt-e6954efa@volkihar.be; helo=icts-p-cavuit-4.kulnet.kuleuven.be X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1729960368628116600 Content-Type: text/plain; charset="utf-8" bdrv_co_common_block_status_above not only returns whether the block is allocated, but also if it contains zeroes. Signed-off-by: Vincent Vanlaer Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/commit.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/block/commit.c b/block/commit.c index 7c3fdcb0ca..8dee25b313 100644 --- a/block/commit.c +++ b/block/commit.c @@ -15,6 +15,8 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" #include "trace.h" +#include "block/block-common.h" +#include "block/coroutines.h" #include "block/block_int.h" #include "block/blockjob_int.h" #include "qapi/error.h" @@ -167,9 +169,13 @@ static int coroutine_fn commit_run(Job *job, Error **e= rrp) break; } /* Copy if allocated above the base */ - ret =3D blk_co_is_allocated_above(s->top, s->base_overlay, true, - offset, COMMIT_BUFFER_SIZE, &n); - copy =3D (ret > 0); + WITH_GRAPH_RDLOCK_GUARD() { + ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), + s->base_overlay, true, true, offset, COMMIT_BUFFER_SIZE, + &n, NULL, NULL, NULL); + } + + copy =3D (ret >=3D 0 && ret & BDRV_BLOCK_ALLOCATED); trace_commit_one_iteration(s, offset, n, ret); if (copy) { assert(n < SIZE_MAX); --=20 2.44.1 From nobody Sat Nov 23 18:16:36 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1729960367157620.152412821306; Sat, 26 Oct 2024 09:32:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t4jh9-0007wK-FQ; Sat, 26 Oct 2024 12:31:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jgj-0007sj-Sr; Sat, 26 Oct 2024 12:31:01 -0400 Received: from icts-p-cavuit-1.kulnet.kuleuven.be ([2a02:2c40:0:c0::25:132]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jgg-0003bd-8u; Sat, 26 Oct 2024 12:30:52 -0400 Received: from icts-p-ceifnet-smtps-0.kuleuven.be (icts-p-ceifnet-smtps.service.icts.svcd [IPv6:2a02:2c40:0:51:139:242:ac11:49]) by icts-p-cavuit-1.kulnet.kuleuven.be (Postfix) with ESMTP id E58C420082; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) Received: from flutterbat.volkihar.be (236.58-240-81.adsl-dyn.isp.belgacom.be [81.240.58.236]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by icts-p-ceifnet-smtps-0.kuleuven.be (Postfix) with ESMTPSA id C7A18D4F17F5F; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) Received: by flutterbat.volkihar.be (Postfix, from userid 1000) id 8883A4B91EC; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) X-KULeuven-Envelope-From: libvirt-e6954efa@volkihar.be X-KULeuven-Scanned: Found to be clean X-KULeuven-ID: E58C420082.AFC06 X-KULeuven-Information: Katholieke Universiteit Leuven BCmilterd-Mark-Subject: no BCmilterd-Errors: BCmilterd-Report: X-CAV-Cluster: smtps X-Kuleuven: This mail passed the K.U.Leuven mailcluster From: Vincent Vanlaer To: qemu-devel@nongnu.org Cc: John Snow , Vladimir Sementsov-Ogievskiy , Kevin Wolf , qemu-block@nongnu.org, Hanna Reitz , Vincent Vanlaer Subject: [PATCH v4 2/5] block: move commit_run loop to separate function Date: Sat, 26 Oct 2024 18:30:06 +0200 Message-ID: <20241026163010.2865002-3-libvirt-e6954efa@volkihar.be> X-Mailer: git-send-email 2.44.1 In-Reply-To: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> References: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:2c40:0:c0::25:132; envelope-from=libvirt-e6954efa@volkihar.be; helo=icts-p-cavuit-1.kulnet.kuleuven.be X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1729960368339116600 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vincent Vanlaer Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/commit.c | 89 +++++++++++++++++++++++++++++--------------------- 1 file changed, 52 insertions(+), 37 deletions(-) diff --git a/block/commit.c b/block/commit.c index 8dee25b313..078e54f51f 100644 --- a/block/commit.c +++ b/block/commit.c @@ -128,6 +128,55 @@ static void commit_clean(Job *job) blk_unref(s->top); } =20 +static int commit_iteration(CommitBlockJob *s, int64_t offset, + int64_t *n, void *buf) +{ + int ret =3D 0; + bool copy; + bool error_in_source =3D true; + + /* Copy if allocated above the base */ + WITH_GRAPH_RDLOCK_GUARD() { + ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), + s->base_overlay, true, true, offset, COMMIT_BUFFER_SIZE, + n, NULL, NULL, NULL); + } + + copy =3D (ret >=3D 0 && ret & BDRV_BLOCK_ALLOCATED); + trace_commit_one_iteration(s, offset, *n, ret); + if (copy) { + assert(*n < SIZE_MAX); + + ret =3D blk_co_pread(s->top, offset, *n, buf, 0); + if (ret >=3D 0) { + ret =3D blk_co_pwrite(s->base, offset, *n, buf, 0); + if (ret < 0) { + error_in_source =3D false; + } + } + } + if (ret < 0) { + BlockErrorAction action =3D block_job_error_action(&s->common, + s->on_error, + error_in_source, + -ret); + if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { + return ret; + } else { + *n =3D 0; + return 0; + } + } + /* Publish progress */ + job_progress_update(&s->common.job, *n); + + if (copy) { + block_job_ratelimit_processed_bytes(&s->common, *n); + } + + return 0; +} + static int coroutine_fn commit_run(Job *job, Error **errp) { CommitBlockJob *s =3D container_of(job, CommitBlockJob, common.job); @@ -158,9 +207,6 @@ static int coroutine_fn commit_run(Job *job, Error **er= rp) buf =3D blk_blockalign(s->top, COMMIT_BUFFER_SIZE); =20 for (offset =3D 0; offset < len; offset +=3D n) { - bool copy; - bool error_in_source =3D true; - /* Note that even when no rate limit is applied we need to yield * with no pending I/O here so that bdrv_drain_all() returns. */ @@ -168,42 +214,11 @@ static int coroutine_fn commit_run(Job *job, Error **= errp) if (job_is_cancelled(&s->common.job)) { break; } - /* Copy if allocated above the base */ - WITH_GRAPH_RDLOCK_GUARD() { - ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), - s->base_overlay, true, true, offset, COMMIT_BUFFER_SIZE, - &n, NULL, NULL, NULL); - } =20 - copy =3D (ret >=3D 0 && ret & BDRV_BLOCK_ALLOCATED); - trace_commit_one_iteration(s, offset, n, ret); - if (copy) { - assert(n < SIZE_MAX); - - ret =3D blk_co_pread(s->top, offset, n, buf, 0); - if (ret >=3D 0) { - ret =3D blk_co_pwrite(s->base, offset, n, buf, 0); - if (ret < 0) { - error_in_source =3D false; - } - } - } - if (ret < 0) { - BlockErrorAction action =3D - block_job_error_action(&s->common, s->on_error, - error_in_source, -ret); - if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { - return ret; - } else { - n =3D 0; - continue; - } - } - /* Publish progress */ - job_progress_update(&s->common.job, n); + ret =3D commit_iteration(s, offset, &n, buf); =20 - if (copy) { - block_job_ratelimit_processed_bytes(&s->common, n); + if (ret < 0) { + return ret; } } =20 --=20 2.44.1 From nobody Sat Nov 23 18:16:36 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 172996036719084.31780658513458; Sat, 26 Oct 2024 09:32:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t4jhA-0007wt-Ux; Sat, 26 Oct 2024 12:31:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jgj-0007sh-SC; Sat, 26 Oct 2024 12:31:01 -0400 Received: from icts-p-cavuit-3.kulnet.kuleuven.be ([2a02:2c40:0:c0::25:133]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jge-0003c3-BZ; Sat, 26 Oct 2024 12:30:52 -0400 Received: from icts-p-ceifnet-smtps-1.kuleuven.be (icts-p-ceifnet-smtps.service.icts.svcd [IPv6:2a02:2c40:0:51:135:242:ac11:35]) by icts-p-cavuit-3.kulnet.kuleuven.be (Postfix) with ESMTP id EF25620066; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) Received: from flutterbat.volkihar.be (236.58-240-81.adsl-dyn.isp.belgacom.be [81.240.58.236]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by icts-p-ceifnet-smtps-1.kuleuven.be (Postfix) with ESMTPSA id D3D00D4F2584F; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) Received: by flutterbat.volkihar.be (Postfix, from userid 1000) id 9C8004B91EE; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) X-KULeuven-Envelope-From: libvirt-e6954efa@volkihar.be X-KULeuven-Scanned: Found to be clean X-KULeuven-ID: EF25620066.ADDCD X-KULeuven-Information: Katholieke Universiteit Leuven BCmilterd-Mark-Subject: no BCmilterd-Errors: BCmilterd-Report: X-CAV-Cluster: smtps X-Kuleuven: This mail passed the K.U.Leuven mailcluster From: Vincent Vanlaer To: qemu-devel@nongnu.org Cc: John Snow , Vladimir Sementsov-Ogievskiy , Kevin Wolf , qemu-block@nongnu.org, Hanna Reitz , Vincent Vanlaer Subject: [PATCH v4 3/5] block: refactor error handling of commit_iteration Date: Sat, 26 Oct 2024 18:30:07 +0200 Message-ID: <20241026163010.2865002-4-libvirt-e6954efa@volkihar.be> X-Mailer: git-send-email 2.44.1 In-Reply-To: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> References: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:2c40:0:c0::25:133; envelope-from=libvirt-e6954efa@volkihar.be; helo=icts-p-cavuit-3.kulnet.kuleuven.be X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1729960368638116600 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vincent Vanlaer Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/commit.c | 61 ++++++++++++++++++++++++++++---------------------- 1 file changed, 34 insertions(+), 27 deletions(-) diff --git a/block/commit.c b/block/commit.c index 078e54f51f..5c24c8b80a 100644 --- a/block/commit.c +++ b/block/commit.c @@ -129,51 +129,58 @@ static void commit_clean(Job *job) } =20 static int commit_iteration(CommitBlockJob *s, int64_t offset, - int64_t *n, void *buf) + int64_t *requested_bytes, void *buf) { + int64_t bytes =3D *requested_bytes; int ret =3D 0; - bool copy; bool error_in_source =3D true; =20 /* Copy if allocated above the base */ WITH_GRAPH_RDLOCK_GUARD() { ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), s->base_overlay, true, true, offset, COMMIT_BUFFER_SIZE, - n, NULL, NULL, NULL); + &bytes, NULL, NULL, NULL); } =20 - copy =3D (ret >=3D 0 && ret & BDRV_BLOCK_ALLOCATED); - trace_commit_one_iteration(s, offset, *n, ret); - if (copy) { - assert(*n < SIZE_MAX); + trace_commit_one_iteration(s, offset, bytes, ret); =20 - ret =3D blk_co_pread(s->top, offset, *n, buf, 0); - if (ret >=3D 0) { - ret =3D blk_co_pwrite(s->base, offset, *n, buf, 0); - if (ret < 0) { - error_in_source =3D false; - } - } - } if (ret < 0) { - BlockErrorAction action =3D block_job_error_action(&s->common, - s->on_error, - error_in_source, - -ret); - if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { - return ret; - } else { - *n =3D 0; - return 0; + goto fail; + } + + if (ret & BDRV_BLOCK_ALLOCATED) { + assert(bytes < SIZE_MAX); + + ret =3D blk_co_pread(s->top, offset, bytes, buf, 0); + if (ret < 0) { + goto fail; } + + ret =3D blk_co_pwrite(s->base, offset, bytes, buf, 0); + if (ret < 0) { + error_in_source =3D false; + goto fail; + } + + block_job_ratelimit_processed_bytes(&s->common, bytes); } + /* Publish progress */ - job_progress_update(&s->common.job, *n); =20 - if (copy) { - block_job_ratelimit_processed_bytes(&s->common, *n); + job_progress_update(&s->common.job, bytes); + + *requested_bytes =3D bytes; + + return 0; +fail:; + BlockErrorAction action =3D block_job_error_action(&s->common, s->on_e= rror, + error_in_source, -ret= ); + if (action =3D=3D BLOCK_ERROR_ACTION_REPORT) { + return ret; } =20 + *requested_bytes =3D 0; + return 0; } =20 --=20 2.44.1 From nobody Sat Nov 23 18:16:36 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1729960361443656.5460959787321; Sat, 26 Oct 2024 09:32:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t4jgv-0007u3-Ji; Sat, 26 Oct 2024 12:31:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jgg-0007rz-BP; Sat, 26 Oct 2024 12:30:51 -0400 Received: from icts-p-cavuit-4.kulnet.kuleuven.be ([2a02:2c40:0:c0::25:134]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jge-0003be-05; Sat, 26 Oct 2024 12:30:49 -0400 Received: from icts-p-ceifnet-smtps-0.kuleuven.be (icts-p-ceifnet-smtps.service.icts.svcd [IPv6:2a02:2c40:0:51:139:242:ac11:49]) by icts-p-cavuit-4.kulnet.kuleuven.be (Postfix) with ESMTP id 1347A120; Sat, 26 Oct 2024 18:30:38 +0200 (CEST) Received: from flutterbat.volkihar.be (236.58-240-81.adsl-dyn.isp.belgacom.be [81.240.58.236]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by icts-p-ceifnet-smtps-0.kuleuven.be (Postfix) with ESMTPSA id ED8DBD4F17F51; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) Received: by flutterbat.volkihar.be (Postfix, from userid 1000) id B07114B91F0; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) X-KULeuven-Envelope-From: libvirt-e6954efa@volkihar.be X-KULeuven-Scanned: Found to be clean X-KULeuven-ID: 1347A120.A0028 X-KULeuven-Information: Katholieke Universiteit Leuven BCmilterd-Mark-Subject: no BCmilterd-Errors: BCmilterd-Report: X-CAV-Cluster: smtps X-Kuleuven: This mail passed the K.U.Leuven mailcluster From: Vincent Vanlaer To: qemu-devel@nongnu.org Cc: John Snow , Vladimir Sementsov-Ogievskiy , Kevin Wolf , qemu-block@nongnu.org, Hanna Reitz , Vincent Vanlaer Subject: [PATCH v4 4/5] block: allow commit to unmap zero blocks Date: Sat, 26 Oct 2024 18:30:08 +0200 Message-ID: <20241026163010.2865002-5-libvirt-e6954efa@volkihar.be> X-Mailer: git-send-email 2.44.1 In-Reply-To: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> References: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:2c40:0:c0::25:134; envelope-from=libvirt-e6954efa@volkihar.be; helo=icts-p-cavuit-4.kulnet.kuleuven.be X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1729960363401116600 Content-Type: text/plain; charset="utf-8" Non-active block commits do not discard blocks only containing zeros, causing images to lose sparseness after the commit. This commit fixes that by writing zero blocks using blk_co_pwrite_zeroes rather than writing them out as any other arbitrary data. Signed-off-by: Vincent Vanlaer Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/commit.c | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/block/commit.c b/block/commit.c index 5c24c8b80a..ca1a367b41 100644 --- a/block/commit.c +++ b/block/commit.c @@ -149,19 +149,39 @@ static int commit_iteration(CommitBlockJob *s, int64_= t offset, } =20 if (ret & BDRV_BLOCK_ALLOCATED) { - assert(bytes < SIZE_MAX); + if (ret & BDRV_BLOCK_ZERO) { + /* + * If the top (sub)clusters are smaller than the base + * (sub)clusters, this will not unmap unless the underlying de= vice + * does some tracking of these requests. Ideally, we would find + * the maximal extent of the zero clusters. + */ + ret =3D blk_co_pwrite_zeroes(s->base, offset, bytes, + BDRV_REQ_MAY_UNMAP); + if (ret < 0) { + error_in_source =3D false; + goto fail; + } + } else { + assert(bytes < SIZE_MAX); =20 - ret =3D blk_co_pread(s->top, offset, bytes, buf, 0); - if (ret < 0) { - goto fail; - } + ret =3D blk_co_pread(s->top, offset, bytes, buf, 0); + if (ret < 0) { + goto fail; + } =20 - ret =3D blk_co_pwrite(s->base, offset, bytes, buf, 0); - if (ret < 0) { - error_in_source =3D false; - goto fail; + ret =3D blk_co_pwrite(s->base, offset, bytes, buf, 0); + if (ret < 0) { + error_in_source =3D false; + goto fail; + } } =20 + /* + * Whether zeroes actually end up on disk depends on the details of + * the underlying driver. Therefore, this might rate limit more th= an + * is necessary. + */ block_job_ratelimit_processed_bytes(&s->common, bytes); } =20 --=20 2.44.1 From nobody Sat Nov 23 18:16:36 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1729960367231834.6058768472304; Sat, 26 Oct 2024 09:32:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t4jhC-0007xJ-7f; Sat, 26 Oct 2024 12:31:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jgj-0007si-S0; Sat, 26 Oct 2024 12:31:01 -0400 Received: from icts-p-cavuit-3.kulnet.kuleuven.be ([2a02:2c40:0:c0::25:133]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4jge-0003c4-CN; Sat, 26 Oct 2024 12:30:53 -0400 Received: from icts-p-ceifnet-smtps-1.kuleuven.be (icts-p-ceifnet-smtps.service.icts.svcd [IPv6:2a02:2c40:0:51:135:242:ac11:35]) by icts-p-cavuit-3.kulnet.kuleuven.be (Postfix) with ESMTP id 315BA200BC; Sat, 26 Oct 2024 18:30:38 +0200 (CEST) Received: from flutterbat.volkihar.be (236.58-240-81.adsl-dyn.isp.belgacom.be [81.240.58.236]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by icts-p-ceifnet-smtps-1.kuleuven.be (Postfix) with ESMTPSA id 0E73ED4F2584F; Sat, 26 Oct 2024 18:30:38 +0200 (CEST) Received: by flutterbat.volkihar.be (Postfix, from userid 1000) id C703B4B91F2; Sat, 26 Oct 2024 18:30:37 +0200 (CEST) X-KULeuven-Envelope-From: libvirt-e6954efa@volkihar.be X-KULeuven-Scanned: Found to be clean X-KULeuven-ID: 315BA200BC.ADDE3 X-KULeuven-Information: Katholieke Universiteit Leuven BCmilterd-Mark-Subject: no BCmilterd-Errors: BCmilterd-Report: X-CAV-Cluster: smtps X-Kuleuven: This mail passed the K.U.Leuven mailcluster From: Vincent Vanlaer To: qemu-devel@nongnu.org Cc: John Snow , Vladimir Sementsov-Ogievskiy , Kevin Wolf , qemu-block@nongnu.org, Hanna Reitz , Vincent Vanlaer Subject: [PATCH v4 5/5] block: add test non-active commit with zeroed data Date: Sat, 26 Oct 2024 18:30:09 +0200 Message-ID: <20241026163010.2865002-6-libvirt-e6954efa@volkihar.be> X-Mailer: git-send-email 2.44.1 In-Reply-To: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> References: <20241026163010.2865002-1-libvirt-e6954efa@volkihar.be> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:2c40:0:c0::25:133; envelope-from=libvirt-e6954efa@volkihar.be; helo=icts-p-cavuit-3.kulnet.kuleuven.be X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1729960368734116600 Content-Type: text/plain; charset="utf-8" Signed-off-by: Vincent Vanlaer Tested-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/tests/commit-zero-blocks | 96 +++++++++++++++++++ .../qemu-iotests/tests/commit-zero-blocks.out | 54 +++++++++++ 2 files changed, 150 insertions(+) create mode 100755 tests/qemu-iotests/tests/commit-zero-blocks create mode 100644 tests/qemu-iotests/tests/commit-zero-blocks.out diff --git a/tests/qemu-iotests/tests/commit-zero-blocks b/tests/qemu-iotes= ts/tests/commit-zero-blocks new file mode 100755 index 0000000000..de00273e72 --- /dev/null +++ b/tests/qemu-iotests/tests/commit-zero-blocks @@ -0,0 +1,96 @@ +#!/usr/bin/env bash +# group: rw quick +# +# Test for commit of discarded blocks +# +# This tests committing a live snapshot where some of the blocks that +# are present in the base image are discarded in the intermediate image. +# This intends to check that these blocks are also discarded in the base +# image after the commit. +# +# Copyright (C) 2024 Vincent Vanlaer. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# creator +owner=3Dlibvirt-e6954efa@volkihar.be + +seq=3D`basename $0` +echo "QA output created by $seq" + +status=3D1 # failure is the default! + +_cleanup() +{ + _cleanup_qemu + _rm_test_img "${TEST_IMG}.base" + _rm_test_img "${TEST_IMG}.mid" + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +cd .. +. ./common.rc +. ./common.filter +. ./common.qemu + +_supported_fmt qcow2 +_supported_proto file + +size=3D"1M" + +TEST_IMG=3D"$TEST_IMG.base" _make_test_img $size +TEST_IMG=3D"$TEST_IMG.mid" _make_test_img -b "$TEST_IMG.base" -F $IMGFMT $= size +_make_test_img -b "${TEST_IMG}.mid" -F $IMGFMT $size + +$QEMU_IO -c "write -P 0x01 64k 128k" "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "discard 64k 64k" "$TEST_IMG.mid" | _filter_qemu_io + +echo +echo "=3D=3D=3D Base image info before commit =3D=3D=3D" +TEST_IMG=3D"${TEST_IMG}.base" _img_info | _filter_img_info +$QEMU_IMG map --output=3Djson "$TEST_IMG.base" | _filter_qemu_img_map + +echo +echo "=3D=3D=3D Middle image info before commit =3D=3D=3D" +TEST_IMG=3D"${TEST_IMG}.mid" _img_info | _filter_img_info +$QEMU_IMG map --output=3Djson "$TEST_IMG.mid" | _filter_qemu_img_map + +echo +echo =3D=3D=3D Running QEMU Live Commit Test =3D=3D=3D +echo + +qemu_comm_method=3D"qmp" +_launch_qemu -drive file=3D"${TEST_IMG}",if=3Dvirtio,id=3Dtest +h=3D$QEMU_HANDLE + +_send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" "return" + +_send_qemu_cmd $h "{ 'execute': 'block-commit', + 'arguments': { 'device': 'test', + 'top': '"${TEST_IMG}.mid"', + 'base': '"${TEST_IMG}.base"'} }" '"status= ": "null"' + +_cleanup_qemu + +echo +echo "=3D=3D=3D Base image info after commit =3D=3D=3D" +TEST_IMG=3D"${TEST_IMG}.base" _img_info | _filter_img_info +$QEMU_IMG map --output=3Djson "$TEST_IMG.base" | _filter_qemu_img_map + +# success, all done +echo "*** done" +rm -f $seq.full +status=3D0 diff --git a/tests/qemu-iotests/tests/commit-zero-blocks.out b/tests/qemu-i= otests/tests/commit-zero-blocks.out new file mode 100644 index 0000000000..85bdc46aaf --- /dev/null +++ b/tests/qemu-iotests/tests/commit-zero-blocks.out @@ -0,0 +1,54 @@ +QA output created by commit-zero-blocks +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D1048576 +Formatting 'TEST_DIR/t.IMGFMT.mid', fmt=3DIMGFMT size=3D1048576 backing_fi= le=3DTEST_DIR/t.IMGFMT.base backing_fmt=3DIMGFMT +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1048576 backing_file= =3DTEST_DIR/t.IMGFMT.mid backing_fmt=3DIMGFMT +wrote 131072/131072 bytes at offset 65536 +128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +discard 65536/65536 bytes at offset 65536 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=3D=3D=3D Base image info before commit =3D=3D=3D +image: TEST_DIR/t.IMGFMT.base +file format: IMGFMT +virtual size: 1 MiB (1048576 bytes) +[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": true= , "data": false, "compressed": false}, +{ "start": 65536, "length": 131072, "depth": 0, "present": true, "zero": f= alse, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 196608, "length": 851968, "depth": 0, "present": false, "zero":= true, "data": false, "compressed": false}] + +=3D=3D=3D Middle image info before commit =3D=3D=3D +image: TEST_DIR/t.IMGFMT.mid +file format: IMGFMT +virtual size: 1 MiB (1048576 bytes) +backing file: TEST_DIR/t.IMGFMT.base +backing file format: IMGFMT +[{ "start": 0, "length": 65536, "depth": 1, "present": false, "zero": true= , "data": false, "compressed": false}, +{ "start": 65536, "length": 65536, "depth": 0, "present": true, "zero": tr= ue, "data": false, "compressed": false}, +{ "start": 131072, "length": 65536, "depth": 1, "present": true, "zero": f= alse, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 196608, "length": 851968, "depth": 1, "present": false, "zero":= true, "data": false, "compressed": false}] + +=3D=3D=3D Running QEMU Live Commit Test =3D=3D=3D + +{ 'execute': 'qmp_capabilities' } +{"return": {}} +{ 'execute': 'block-commit', + 'arguments': { 'device': 'test', + 'top': 'TEST_DIR/t.IMGFMT.mid', + 'base': 'TEST_DIR/t.IMGFMT.base'} } +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "test"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "test"}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "test"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "test"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "BLOCK_JOB_COMPLETED", "data": {"device": "test", "len": 1048576, "offset= ": 1048576, "speed": 0, "type": "commit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "test"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event"= : "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "test"}} + +=3D=3D=3D Base image info after commit =3D=3D=3D +image: TEST_DIR/t.IMGFMT.base +file format: IMGFMT +virtual size: 1 MiB (1048576 bytes) +[{ "start": 0, "length": 65536, "depth": 0, "present": false, "zero": true= , "data": false, "compressed": false}, +{ "start": 65536, "length": 65536, "depth": 0, "present": true, "zero": tr= ue, "data": false, "compressed": false}, +{ "start": 131072, "length": 65536, "depth": 0, "present": true, "zero": f= alse, "data": true, "compressed": false, "offset": OFFSET}, +{ "start": 196608, "length": 851968, "depth": 0, "present": false, "zero":= true, "data": false, "compressed": false}] +*** done --=20 2.44.1