From nobody Sat Nov 15 20:51:44 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1747100223; cv=none; d=zohomail.com; s=zohoarc; b=U9PC0c14BEiKrqIAAuAPq9621w3K8hFcRmrT8X+CUQpAznUDpgCFcLHpEPV5LQvf2jkuVrO4UdTX4aLU2ZaZ3YPWJ7DwlmAqWx5fJsUh3hK6TJEzYNCIn4AI0HKTFrorDLgYnsUoin3CUWeec/9+s3xhpmfIzkldL+R0TLHDQ+Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1747100223; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=o76L16+85oXwxINpy+kttS3ujUGzFeqh/Z5rK6Yd30A=; b=nqLfqmbcEw4ScLdmFLVv+9W+Zy2gn1AOx8sK0VRmIcMR5gNLNlJWvgBNFymtkUuemFMGodXYGV8MqUkbx9J+x9uksVvms5J2/zHzg1pegLb5RfAEP7gxy+9BaAMUOx+NofewCPwuSJJTEYQSr/nuFTvnPkhvM21qWDy8SF5m9jc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1747100223011906.5920028817487; Mon, 12 May 2025 18:37:03 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uEeYm-0002If-SP; Mon, 12 May 2025 21:35:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYi-0002F1-Cy; Mon, 12 May 2025 21:35:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYf-00021T-Vp; Mon, 12 May 2025 21:35:52 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=debian.fritz.box) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1uEeUe-00DboQ-0k; Tue, 13 May 2025 03:35:36 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=o76L16+85oXwxINpy+kttS3ujUGzFeqh/Z5rK6Yd30A=; b=sYq6qqZJtCIi mob/IlvSIwTcwVRxDbI2ZBcogj/VFPV27TEIVL2GpW/Ybg/MQlnE9Ax5z0j2VN46VokneLb9CKyAc oowjJ6K0Wxolize8ytDSwdaQBcHv6UeOAJ04jmD9y0up9yyzig0I2J2FR1uEeDRUm0ypk3YB0ehb6 EoWpmjAEgC7+ZRiva5VJVTwqK8xp/rBrgPwPu4RG/mQmt0wQOOCbGIwtyZAjzK7L04qdsjKEZwPMV mbMYbajDft+Zmd4psP+DzcFw7Oau67H3zkgDJDmihiNEYgWWqw36+lDzrwHj/znYK/H41EetFVolQ MKeaLjmQhLfL+hU0zs/4Wg==; From: Andrey Zhadchenko To: qemu-block@nongnu.org, vsementsov@yandex-team.ru, eblake@redhat.com Cc: jsnow@redhat.com, kwolf@redhat.com, hreitz@redhat.com, qemu-devel@nongnu.org, andrey.drobyshev@virtuozzo.com, den@virtuozzo.com Subject: [PATCH 1/4] hbitmap: drop meta bitmap leftovers Date: Tue, 13 May 2025 03:32:35 +0200 Message-ID: <20250513013238.1213539-2-andrey.zhadchenko@virtuozzo.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> References: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=andrey.zhadchenko@virtuozzo.com; helo=relay.virtuozzo.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @virtuozzo.com) X-ZM-MESSAGEID: 1747100224055116600 Content-Type: text/plain; charset="utf-8" API to manipulate meta bitmap was removed with commit 0c88f1970c76 Signed-off-by: Andrey Zhadchenko Reviewed-by: Eric Blake --- util/hbitmap.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/util/hbitmap.c b/util/hbitmap.c index d9a1dabc63..16674f33e4 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -84,9 +84,6 @@ struct HBitmap { */ int granularity; =20 - /* A meta dirty bitmap to track the dirtiness of bits in this HBitmap.= */ - HBitmap *meta; - /* A number of progressively less coarse bitmaps (i.e. level 0 is the * coarsest). Each bit in level N represents a word in level N+1 that * has a set bit, except the last level where each bit represents the @@ -480,10 +477,7 @@ void hbitmap_set(HBitmap *hb, uint64_t start, uint64_t= count) n =3D last - first + 1; =20 hb->count +=3D n - hb_count_between(hb, first, last); - if (hb_set_between(hb, HBITMAP_LEVELS - 1, first, last) && - hb->meta) { - hbitmap_set(hb->meta, start, count); - } + hb_set_between(hb, HBITMAP_LEVELS - 1, first, last); } =20 /* Resetting works the other way round: propagate up if the new @@ -577,10 +571,7 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64= _t count) assert(last < hb->size); =20 hb->count -=3D hb_count_between(hb, first, last); - if (hb_reset_between(hb, HBITMAP_LEVELS - 1, first, last) && - hb->meta) { - hbitmap_set(hb->meta, start, count); - } + hb_reset_between(hb, HBITMAP_LEVELS - 1, first, last); } =20 void hbitmap_reset_all(HBitmap *hb) @@ -784,7 +775,6 @@ void hbitmap_deserialize_finish(HBitmap *bitmap) void hbitmap_free(HBitmap *hb) { unsigned i; - assert(!hb->meta); for (i =3D HBITMAP_LEVELS; i-- > 0; ) { g_free(hb->levels[i]); } @@ -868,9 +858,6 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size) (size - old) * sizeof(*hb->levels[i])); } } - if (hb->meta) { - hbitmap_truncate(hb->meta, hb->size << hb->granularity); - } } =20 /** --=20 2.43.0 From nobody Sat Nov 15 20:51:44 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1747100220; cv=none; d=zohomail.com; s=zohoarc; b=c5CMx+AZd6MBACpQNI7fFJ+XR72xbE6QiT2N9nZNHyADxRHRcvWl8bDZpVCT0xo/dNYwXAMDgHdZSuk7WE6V+WdzvES9XxfCC9a0q9Ts4ZXuVmJ+nJsyiztzxfX586UxZxeq0CFu6gNBQlSqF22pB0CQAqb7ULcJGU9RddUjDgY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1747100220; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=+L+BU/R0UiYOQwIozPYQoIx70UFXZ5v2qWVT4AX4P+k=; b=dJp+zIETRnZ8gx6Rf/XyNL1tbWflTSyropHRzLEFBsLBl5aM8nToB1vGiaCBomN+vKVRmQ4uPSvgvPX7URYZ/iFzXHr+YHiIIvt0oJSaygV2nHgTWOEiDzbLCrEZjhYkMwq9eDskS309Zx/jzZvWxQlbzAQTOcx3ZGZtlZNJ9O0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1747100220600764.2784192641878; Mon, 12 May 2025 18:37:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uEeYk-0002Gp-RK; Mon, 12 May 2025 21:35:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYi-0002Ez-AG; Mon, 12 May 2025 21:35:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYf-00021S-VD; Mon, 12 May 2025 21:35:52 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=debian.fritz.box) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1uEeUe-00DboQ-1z; Tue, 13 May 2025 03:35:37 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=+L+BU/R0UiYOQwIozPYQoIx70UFXZ5v2qWVT4AX4P+k=; b=NunU+OlPGWG5 LSJYo929eb6VBFhzL7W4jiA4pDRDuawj0nwHUzXmz3DnJCRiZeEygk48Vbr4wPBkVDREAp99fbI7Y jSffTuBvXystnAvZNmKSzmF2ROG0Ml+Yc50XL42OMsqK7s/Kuc3f+H0uJZas78Ah0B4BrPTjub96w ZKPp41DFHCI4bX3ntF2pomPzzJOxb4QvLWx4/lJUxGUS/cxq502G+o610dyFaiuQog4jTPEczQ6rq VuHde+tHVLYaFu/gfLQ8JuCWHfnjd3ryyxGk3iM8v9N4HZN8PS1Y2k+7NLlBNaBVubGI003i3ndy2 Ho6SVyZPDsSPUleyxG2yFg==; From: Andrey Zhadchenko To: qemu-block@nongnu.org, vsementsov@yandex-team.ru, eblake@redhat.com Cc: jsnow@redhat.com, kwolf@redhat.com, hreitz@redhat.com, qemu-devel@nongnu.org, andrey.drobyshev@virtuozzo.com, den@virtuozzo.com Subject: [PATCH 2/4] hbitmap: introduce hbitmap_reverse() Date: Tue, 13 May 2025 03:32:36 +0200 Message-ID: <20250513013238.1213539-3-andrey.zhadchenko@virtuozzo.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> References: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=andrey.zhadchenko@virtuozzo.com; helo=relay.virtuozzo.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @virtuozzo.com) X-ZM-MESSAGEID: 1747100221936019000 Content-Type: text/plain; charset="utf-8" and bdrv_dirty_bitmap_reverse() helper Signed-off-by: Andrey Zhadchenko --- block/dirty-bitmap.c | 9 +++++++++ include/block/block_int-io.h | 1 + include/qemu/hbitmap.h | 8 ++++++++ util/hbitmap.c | 15 +++++++++++++++ 4 files changed, 33 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 13a1979755..c7f453fdb9 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -888,3 +888,12 @@ void bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap = *dest, } } } + +void bdrv_dirty_bitmap_reverse(BdrvDirtyBitmap *bitmap) +{ + assert(!bdrv_dirty_bitmap_readonly(bitmap)); + assert(!bdrv_dirty_bitmap_inconsistent(bitmap)); + bdrv_dirty_bitmaps_lock(bitmap->bs); + hbitmap_reverse(bitmap->bitmap); + bdrv_dirty_bitmaps_unlock(bitmap->bs); +} diff --git a/include/block/block_int-io.h b/include/block/block_int-io.h index 4a7cf2b4fd..093613e7d1 100644 --- a/include/block/block_int-io.h +++ b/include/block/block_int-io.h @@ -109,6 +109,7 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, H= Bitmap **out); void bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, HBitmap **backup, bool lock); +void bdrv_dirty_bitmap_reverse(BdrvDirtyBitmap *bitmap); =20 void bdrv_inc_in_flight(BlockDriverState *bs); void bdrv_dec_in_flight(BlockDriverState *bs); diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index 8136e33674..dbdc9aa2d4 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -350,4 +350,12 @@ bool hbitmap_status(const HBitmap *hb, int64_t start, = int64_t count, */ int64_t hbitmap_iter_next(HBitmapIter *hbi); =20 +/** + * hbitmap_reverse: + * @bitmap: The HBitmap to operate on + * + * Reverse the bits in the bitmap. + */ +void hbitmap_reverse(HBitmap *bitmap); + #endif diff --git a/util/hbitmap.c b/util/hbitmap.c index 16674f33e4..b99c4b1eec 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -940,3 +940,18 @@ char *hbitmap_sha256(const HBitmap *bitmap, Error **er= rp) =20 return hash; } + +void hbitmap_reverse(HBitmap *bitmap) +{ + int64_t pnum, pos =3D 0; + int64_t size =3D bitmap->orig_size; + + while (pos < size) { + if (hbitmap_status(bitmap, pos, size - pos, &pnum)) { + hbitmap_reset(bitmap, pos, pnum); + } else { + hbitmap_set(bitmap, pos, pnum); + } + pos +=3D pnum; + } +} --=20 2.43.0 From nobody Sat Nov 15 20:51:44 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1747100186; cv=none; d=zohomail.com; s=zohoarc; b=oCAmci2HxkmWNi/iNVnxgRUqpXSms7zBzIQhSin25e2tLLMnopD7Ju3PJsOxvW7qkkUAxzrzz2ZQJ4dsSZyF9Y0GqAjWRf0WhLO6bE0ThWaeN2nWyc29YbHyO7G+V5FDjOvzdpOv4v0W5dwintV3fuEUzOi+/KJ3xZSa844qI7g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1747100186; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=M37sAYz/UezG1DeFTg2QB3ntIIKeW+/F83Bp66GbxIc=; b=LtNgi+Hu3T69cfBsP9BU4G0RjnyPdneLnS7KD4twKh+37yuXRaM70Sjz72KNVlhR8XWpkGBbInL+nDaHg6ipPlBM85oGL4ThULdXx2u1P0fVon2qXKfI2MySM/7oVYdp1CbjD6oHcG4FZQAltleAsHllSK/LXRx1l7EzCV9D/Dg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1747100186591154.03289512666913; Mon, 12 May 2025 18:36:26 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uEeYl-0002Gy-AZ; Mon, 12 May 2025 21:35:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYi-0002FB-Fi; Mon, 12 May 2025 21:35:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYg-00021W-2P; Mon, 12 May 2025 21:35:52 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=debian.fritz.box) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1uEeUe-00DboQ-3A; Tue, 13 May 2025 03:35:37 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=M37sAYz/UezG1DeFTg2QB3ntIIKeW+/F83Bp66GbxIc=; b=jUFHjnULNk8S m+RbMFS0hpJriwh63ow/z2GI7mKAswBT8JrAwQG7ga9KJz3B2ov0E43SPC+UrIHXyToeY0s2UKDDN hn1UALkDcu1FXifNGx+ji+uD+IzNc4EaxczW4zW9lrWtIxHOhs/1DJUYXl+O7K7H9CtEIU/SN97x3 F5AUoH1gCaTZ6ZGnb5plP2z1ivgfAsuKAUhmqkbI7ptxqb9SAHNerf9puxdpQV2VPEeFNfq3yNe8M ECi1l1vNu2wx6/ty7QLMB8peIQCjTeWogr+P9+EQe7/pnNq3Vsk/C+be21gtHBM51tNYp9K7kJD7D l1GYVgsamdT4Vku9T0FHNw==; From: Andrey Zhadchenko To: qemu-block@nongnu.org, vsementsov@yandex-team.ru, eblake@redhat.com Cc: jsnow@redhat.com, kwolf@redhat.com, hreitz@redhat.com, qemu-devel@nongnu.org, andrey.drobyshev@virtuozzo.com, den@virtuozzo.com Subject: [PATCH 3/4] block/copy-before-write: reverse access bitmap Date: Tue, 13 May 2025 03:32:37 +0200 Message-ID: <20250513013238.1213539-4-andrey.zhadchenko@virtuozzo.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> References: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=andrey.zhadchenko@virtuozzo.com; helo=relay.virtuozzo.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @virtuozzo.com) X-ZM-MESSAGEID: 1747100188343019000 Content-Type: text/plain; charset="utf-8" HBitmaps allow us to search set bits pretty fast. On the contrary, when searching zeroes, we may be forced to fully traverse the lower level. When we run blockdev-backup with mode=3Dfull on top of snapshot filter + cbw filter, the job fills copy bitmap by calling block_status() with range (X, virtual_size). The problem is that we check for zeroes in this whole range. We also hit the worst case here, as access bitmap is fully set and we need to scan the entire lowest level. After scanning the full bitmap we actually ask the block status of original image, which may return significantly lower amount of empty clusters. Beacuse of this, the backup job 'hangs' on block copy initializaiton for a long time with 100% CPU. Example copy bitmap buildup time for image with clu_size=3D65536 and preallocated metadata size 10T 11T blockdev-backup 52s 57s cbw + snap 325s 413s cbw + snap + patch 55s 61s To fix it, reverse the access bitmap in cbw filter: rather set it when the user is not allowed to read the cluster. Update qemu-iotest 257: now access bitmap have count 0 instead of the image size 67108864 Signed-off-by: Andrey Zhadchenko --- block/copy-before-write.c | 17 ++++++++++------- tests/qemu-iotests/257.out | 28 ++++++++++++++-------------- 2 files changed, 24 insertions(+), 21 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index fd470f5f92..5f5b3e7515 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -53,7 +53,7 @@ typedef struct BDRVCopyBeforeWriteState { CoMutex lock; =20 /* - * @access_bitmap: represents areas allowed for reading by fleecing us= er. + * @access_bitmap: represents areas disallowed for reading by fleecing= user. * Reading from non-dirty areas leads to -EACCES. */ BdrvDirtyBitmap *access_bitmap; @@ -220,7 +220,7 @@ cbw_snapshot_read_lock(BlockDriverState *bs, int64_t of= fset, int64_t bytes, return NULL; } =20 - if (bdrv_dirty_bitmap_next_zero(s->access_bitmap, offset, bytes) !=3D = -1) { + if (bdrv_dirty_bitmap_next_dirty(s->access_bitmap, offset, bytes) !=3D= -1) { g_free(req); return NULL; } @@ -338,8 +338,8 @@ cbw_co_pdiscard_snapshot(BlockDriverState *bs, int64_t = offset, int64_t bytes) aligned_bytes =3D aligned_end - aligned_offset; =20 WITH_QEMU_LOCK_GUARD(&s->lock) { - bdrv_reset_dirty_bitmap(s->access_bitmap, aligned_offset, - aligned_bytes); + bdrv_set_dirty_bitmap(s->access_bitmap, aligned_offset, + aligned_bytes); } =20 block_copy_reset(s->bcs, aligned_offset, aligned_bytes); @@ -501,9 +501,12 @@ static int cbw_open(BlockDriverState *bs, QDict *optio= ns, int flags, return -EINVAL; } bdrv_disable_dirty_bitmap(s->access_bitmap); - bdrv_dirty_bitmap_merge_internal(s->access_bitmap, - block_copy_dirty_bitmap(s->bcs), NULL, - true); + if (bitmap) { + bdrv_dirty_bitmap_merge_internal(s->access_bitmap, + block_copy_dirty_bitmap(s->bcs), = NULL, + true); + bdrv_dirty_bitmap_reverse(s->access_bitmap); + } =20 qemu_co_mutex_init(&s->lock); QLIST_INIT(&s->frozen_read_reqs); diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out index c33dd7f3a9..55efb418e6 100644 --- a/tests/qemu-iotests/257.out +++ b/tests/qemu-iotests/257.out @@ -109,7 +109,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -585,7 +585,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -854,7 +854,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -1330,7 +1330,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -1599,7 +1599,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -2075,7 +2075,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -2344,7 +2344,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -2820,7 +2820,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -3089,7 +3089,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -3565,7 +3565,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -3834,7 +3834,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -4310,7 +4310,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -4579,7 +4579,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -5055,7 +5055,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false --=20 2.43.0 From nobody Sat Nov 15 20:51:44 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1747100223; cv=none; d=zohomail.com; s=zohoarc; b=XrzHiW80VqeJt8BhrcjqO4a8QmebRe2ZvxIc2bggCYDbGVw3OY/YJxqTz9Mqh1ckNNNMctzFgcQlU5a2bkGU/5AFIbf04lZhZ8bNZy/XXAkDDaw4gAjlqDAA5ldsVGVeZjn7olJoe+lvvJknpbaXJGkLKpHC3gdZ3/TLt7wETtk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1747100223; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=t1AoeW5+yWy2YGfiL3VknobGzbBoiHSM2roimpSm4Xc=; b=AbPvP7KloLMXSZKt7tVn394jipAZezb6r4/na47cOyv3JoS6zceFEl3DnkXugDMsaOLguYUrEkm66f8Cjirf0+b7/baTIsIcTNNtADX5IcqjIIhZ+ZGpA4QWE6PPqVh66m/2t2h48S4LYS8lvRqz1qvhEZwN+vUGCsoF6L9cYZs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1747100223576922.1725224164477; Mon, 12 May 2025 18:37:03 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uEeYk-0002Go-Q8; Mon, 12 May 2025 21:35:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYi-0002Ey-8J; Mon, 12 May 2025 21:35:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYg-00021Z-6k; Mon, 12 May 2025 21:35:51 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=debian.fritz.box) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1uEeUf-00DboQ-18; Tue, 13 May 2025 03:35:38 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=t1AoeW5+yWy2YGfiL3VknobGzbBoiHSM2roimpSm4Xc=; b=FEhJgPZGgukQ aiUOT9l8c0zq61uLDwXRtl9hLV9N3DbJUjrsmvBvsApfN/5uH3iL7etV3cQN+SNAXTO79NrpVh4vP buUMblYHQ5Q6V4h29HS5GHgQiNVGc27MDNPsdu+IS1QSK5oNxmX7PQRZpP/nQ7N9h1P2m3xhJYFMh B3J4lzy8zDOBeewsDb75+JQa8FDBKa9BYgiB44gYEJPcgHgLydm6q1VJ27PSqR8HahsWq6jSRgt4C x4vJwKtPQ+RajpHMCFjrMDRN5MlJwxeoUSrggjN1KcfngrIcer84+MYzZT/jMtOBxQij48Q2Djvtc JWAHFojyFavTWkCY9W38pQ==; From: Andrey Zhadchenko To: qemu-block@nongnu.org, vsementsov@yandex-team.ru, eblake@redhat.com Cc: jsnow@redhat.com, kwolf@redhat.com, hreitz@redhat.com, qemu-devel@nongnu.org, andrey.drobyshev@virtuozzo.com, den@virtuozzo.com Subject: [PATCH 4/4] block/copy-before-write: report partial block status to snapshot Date: Tue, 13 May 2025 03:32:38 +0200 Message-ID: <20250513013238.1213539-5-andrey.zhadchenko@virtuozzo.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> References: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=andrey.zhadchenko@virtuozzo.com; helo=relay.virtuozzo.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @virtuozzo.com) X-ZM-MESSAGEID: 1747100225741116600 Content-Type: text/plain; charset="utf-8" until the non-accessible area Signed-off-by: Andrey Zhadchenko --- block/copy-before-write.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 5f5b3e7515..81d7f40b13 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -207,10 +207,11 @@ static int coroutine_fn GRAPH_RDLOCK cbw_co_flush(Blo= ckDriverState *bs) */ static BlockReq * coroutine_fn GRAPH_RDLOCK cbw_snapshot_read_lock(BlockDriverState *bs, int64_t offset, int64_t bytes, - int64_t *pnum, BdrvChild **file) + int64_t *pnum, BdrvChild **file, bool query) { BDRVCopyBeforeWriteState *s =3D bs->opaque; BlockReq *req =3D g_new(BlockReq, 1); + int64_t next_dirty; bool done; =20 QEMU_LOCK_GUARD(&s->lock); @@ -220,9 +221,13 @@ cbw_snapshot_read_lock(BlockDriverState *bs, int64_t o= ffset, int64_t bytes, return NULL; } =20 - if (bdrv_dirty_bitmap_next_dirty(s->access_bitmap, offset, bytes) !=3D= -1) { - g_free(req); - return NULL; + next_dirty =3D bdrv_dirty_bitmap_next_dirty(s->access_bitmap, offset, = bytes); + if (next_dirty !=3D -1) { + if (!query || next_dirty =3D=3D offset) { + g_free(req); + return NULL; + } + bytes =3D offset + bytes - next_dirty; } =20 done =3D bdrv_dirty_bitmap_status(s->done_bitmap, offset, bytes, pnum); @@ -270,7 +275,8 @@ cbw_co_preadv_snapshot(BlockDriverState *bs, int64_t of= fset, int64_t bytes, while (bytes) { int64_t cur_bytes; =20 - req =3D cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &fil= e); + req =3D cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &fil= e, + false); if (!req) { return -EACCES; } @@ -302,7 +308,7 @@ cbw_co_snapshot_block_status(BlockDriverState *bs, int64_t cur_bytes; BdrvChild *child; =20 - req =3D cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &child); + req =3D cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &child, = true); if (!req) { return -EACCES; } --=20 2.43.0