From nobody Sat Nov 15 22:35:15 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1747100186; cv=none; d=zohomail.com; s=zohoarc; b=oCAmci2HxkmWNi/iNVnxgRUqpXSms7zBzIQhSin25e2tLLMnopD7Ju3PJsOxvW7qkkUAxzrzz2ZQJ4dsSZyF9Y0GqAjWRf0WhLO6bE0ThWaeN2nWyc29YbHyO7G+V5FDjOvzdpOv4v0W5dwintV3fuEUzOi+/KJ3xZSa844qI7g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1747100186; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=M37sAYz/UezG1DeFTg2QB3ntIIKeW+/F83Bp66GbxIc=; b=LtNgi+Hu3T69cfBsP9BU4G0RjnyPdneLnS7KD4twKh+37yuXRaM70Sjz72KNVlhR8XWpkGBbInL+nDaHg6ipPlBM85oGL4ThULdXx2u1P0fVon2qXKfI2MySM/7oVYdp1CbjD6oHcG4FZQAltleAsHllSK/LXRx1l7EzCV9D/Dg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1747100186591154.03289512666913; Mon, 12 May 2025 18:36:26 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uEeYl-0002Gy-AZ; Mon, 12 May 2025 21:35:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYi-0002FB-Fi; Mon, 12 May 2025 21:35:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEeYg-00021W-2P; Mon, 12 May 2025 21:35:52 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=debian.fritz.box) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1uEeUe-00DboQ-3A; Tue, 13 May 2025 03:35:37 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=M37sAYz/UezG1DeFTg2QB3ntIIKeW+/F83Bp66GbxIc=; b=jUFHjnULNk8S m+RbMFS0hpJriwh63ow/z2GI7mKAswBT8JrAwQG7ga9KJz3B2ov0E43SPC+UrIHXyToeY0s2UKDDN hn1UALkDcu1FXifNGx+ji+uD+IzNc4EaxczW4zW9lrWtIxHOhs/1DJUYXl+O7K7H9CtEIU/SN97x3 F5AUoH1gCaTZ6ZGnb5plP2z1ivgfAsuKAUhmqkbI7ptxqb9SAHNerf9puxdpQV2VPEeFNfq3yNe8M ECi1l1vNu2wx6/ty7QLMB8peIQCjTeWogr+P9+EQe7/pnNq3Vsk/C+be21gtHBM51tNYp9K7kJD7D l1GYVgsamdT4Vku9T0FHNw==; From: Andrey Zhadchenko To: qemu-block@nongnu.org, vsementsov@yandex-team.ru, eblake@redhat.com Cc: jsnow@redhat.com, kwolf@redhat.com, hreitz@redhat.com, qemu-devel@nongnu.org, andrey.drobyshev@virtuozzo.com, den@virtuozzo.com Subject: [PATCH 3/4] block/copy-before-write: reverse access bitmap Date: Tue, 13 May 2025 03:32:37 +0200 Message-ID: <20250513013238.1213539-4-andrey.zhadchenko@virtuozzo.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> References: <20250513013238.1213539-1-andrey.zhadchenko@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=130.117.225.111; envelope-from=andrey.zhadchenko@virtuozzo.com; helo=relay.virtuozzo.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @virtuozzo.com) X-ZM-MESSAGEID: 1747100188343019000 Content-Type: text/plain; charset="utf-8" HBitmaps allow us to search set bits pretty fast. On the contrary, when searching zeroes, we may be forced to fully traverse the lower level. When we run blockdev-backup with mode=3Dfull on top of snapshot filter + cbw filter, the job fills copy bitmap by calling block_status() with range (X, virtual_size). The problem is that we check for zeroes in this whole range. We also hit the worst case here, as access bitmap is fully set and we need to scan the entire lowest level. After scanning the full bitmap we actually ask the block status of original image, which may return significantly lower amount of empty clusters. Beacuse of this, the backup job 'hangs' on block copy initializaiton for a long time with 100% CPU. Example copy bitmap buildup time for image with clu_size=3D65536 and preallocated metadata size 10T 11T blockdev-backup 52s 57s cbw + snap 325s 413s cbw + snap + patch 55s 61s To fix it, reverse the access bitmap in cbw filter: rather set it when the user is not allowed to read the cluster. Update qemu-iotest 257: now access bitmap have count 0 instead of the image size 67108864 Signed-off-by: Andrey Zhadchenko --- block/copy-before-write.c | 17 ++++++++++------- tests/qemu-iotests/257.out | 28 ++++++++++++++-------------- 2 files changed, 24 insertions(+), 21 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index fd470f5f92..5f5b3e7515 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -53,7 +53,7 @@ typedef struct BDRVCopyBeforeWriteState { CoMutex lock; =20 /* - * @access_bitmap: represents areas allowed for reading by fleecing us= er. + * @access_bitmap: represents areas disallowed for reading by fleecing= user. * Reading from non-dirty areas leads to -EACCES. */ BdrvDirtyBitmap *access_bitmap; @@ -220,7 +220,7 @@ cbw_snapshot_read_lock(BlockDriverState *bs, int64_t of= fset, int64_t bytes, return NULL; } =20 - if (bdrv_dirty_bitmap_next_zero(s->access_bitmap, offset, bytes) !=3D = -1) { + if (bdrv_dirty_bitmap_next_dirty(s->access_bitmap, offset, bytes) !=3D= -1) { g_free(req); return NULL; } @@ -338,8 +338,8 @@ cbw_co_pdiscard_snapshot(BlockDriverState *bs, int64_t = offset, int64_t bytes) aligned_bytes =3D aligned_end - aligned_offset; =20 WITH_QEMU_LOCK_GUARD(&s->lock) { - bdrv_reset_dirty_bitmap(s->access_bitmap, aligned_offset, - aligned_bytes); + bdrv_set_dirty_bitmap(s->access_bitmap, aligned_offset, + aligned_bytes); } =20 block_copy_reset(s->bcs, aligned_offset, aligned_bytes); @@ -501,9 +501,12 @@ static int cbw_open(BlockDriverState *bs, QDict *optio= ns, int flags, return -EINVAL; } bdrv_disable_dirty_bitmap(s->access_bitmap); - bdrv_dirty_bitmap_merge_internal(s->access_bitmap, - block_copy_dirty_bitmap(s->bcs), NULL, - true); + if (bitmap) { + bdrv_dirty_bitmap_merge_internal(s->access_bitmap, + block_copy_dirty_bitmap(s->bcs), = NULL, + true); + bdrv_dirty_bitmap_reverse(s->access_bitmap); + } =20 qemu_co_mutex_init(&s->lock); QLIST_INIT(&s->frozen_read_reqs); diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out index c33dd7f3a9..55efb418e6 100644 --- a/tests/qemu-iotests/257.out +++ b/tests/qemu-iotests/257.out @@ -109,7 +109,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -585,7 +585,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -854,7 +854,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -1330,7 +1330,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -1599,7 +1599,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -2075,7 +2075,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -2344,7 +2344,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -2820,7 +2820,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -3089,7 +3089,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -3565,7 +3565,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -3834,7 +3834,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -4310,7 +4310,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -4579,7 +4579,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false @@ -5055,7 +5055,7 @@ write -P0x67 0x3fe0000 0x20000 "backup-top": [ { "busy": false, - "count": 67108864, + "count": 0, "granularity": 65536, "persistent": false, "recording": false --=20 2.43.0