From nobody Fri May 17 01:32:30 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1597138594; cv=none; d=zohomail.com; s=zohoarc; b=QTx2JT0K1h3bBHP31MmmKSqr4NDZwOdTvkVmSUN/OARcku9BHcDn/lr28M/qqxdctvXkdLErMX5M/hm/+/VI9+4z7BFw1382VZCTWTBpo6LLxFQlXQ4hRAe1cisoCaEie/zTl89UE7co9Yxopg4mMQ/IJEdMmRZGYsXEbdBCE34= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1597138594; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QcQViF1bFKP2ex+fLMMInijtnVjDUfdtfdB75Zbi7ic=; b=U7i4RsI0CH0M9N7fa1SHXFw8/H9oIhzYFWlkPqvNx58ng9JdOnPkbOSPFJPT4NU5MhxyuJxrUtS/0y2iyjgogw/Phfwnstp22qeBA7ZTcKAwaGADvAnCvBB2ZMUxEAcLtSpfZH8jsN2VKNlzzuQICWFy5ZQB/KEJRS7vr2Q90Hs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1597138594538184.85134536405644; Tue, 11 Aug 2020 02:36:34 -0700 (PDT) Received: from localhost ([::1]:46618 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k5Qht-0001KU-77 for importer@patchew.org; Tue, 11 Aug 2020 05:36:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k5Qge-000094-0p for qemu-devel@nongnu.org; Tue, 11 Aug 2020 05:35:16 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:57421 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k5Qgb-0001Dn-Hw for qemu-devel@nongnu.org; Tue, 11 Aug 2020 05:35:15 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-333-ig_l7z5kPIqEA0AGDM-sPg-1; Tue, 11 Aug 2020 05:35:10 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5B41779EC0; Tue, 11 Aug 2020 09:35:09 +0000 (UTC) Received: from localhost (ovpn-113-152.ams2.redhat.com [10.36.113.152]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EDAC35D992; Tue, 11 Aug 2020 09:35:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597138512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QcQViF1bFKP2ex+fLMMInijtnVjDUfdtfdB75Zbi7ic=; b=QjMY3adoT8aEAKI94X1enDPMZxlmjPAxJdGwxBfZc/dgysN0zh97Hbk0Pe/TNPCHaQavl/ HzoL9j8jn3lPL9wiPJpaxoQ4dq64EZiO/IAX3STTFsVY7lAo2cWjbs51ncCbih2JITGdfa ivzNOoc1dvib5nIOxR51yZs69d3oXxQ= X-MC-Unique: ig_l7z5kPIqEA0AGDM-sPg-1 From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 1/2] block/block-copy: always align copied region to cluster size Date: Tue, 11 Aug 2020 11:35:04 +0200 Message-Id: <20200811093505.972894-2-mreitz@redhat.com> In-Reply-To: <20200811093505.972894-1-mreitz@redhat.com> References: <20200811093505.972894-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=mreitz@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/11 04:41:55 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Stefan Reiter Since commit 42ac214406e0 (block/block-copy: refactor task creation) block_copy_task_create calculates the area to be copied via bdrv_dirty_bitmap_next_dirty_area, but that can return an unaligned byte count if the image's last cluster end is not aligned to the bitmap's granularity. Always ALIGN_UP the resulting bytes value to satisfy block_copy_do_copy, which requires the 'bytes' parameter to be aligned to cluster size. Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Stefan Reiter Message-Id: <20200810095523.15071-1-s.reiter@proxmox.com> Signed-off-by: Max Reitz --- block/block-copy.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/block-copy.c b/block/block-copy.c index f7428a7c08..a30b9097ef 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -142,6 +142,9 @@ static BlockCopyTask *block_copy_task_create(BlockCopyS= tate *s, return NULL; } =20 + assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); + bytes =3D QEMU_ALIGN_UP(bytes, s->cluster_size); + /* region is dirty, so no existent tasks possible in it */ assert(!find_conflicting_task(s, offset, bytes)); =20 --=20 2.26.2 From nobody Fri May 17 01:32:30 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1597138654; cv=none; d=zohomail.com; s=zohoarc; b=TFB9gAm18PnKRYu8GOdrGLv3bgMVxJlN1s1kbTYDmprusKXWJQYrfYADJDOdp2AFuZCyQA95mKu0bVcDogSYCN9zFtZsfo8A7SKqwyPsGmd8lj9Xsq7DEIGwSBTgFmAfa7TIQBSJnBGr3qIH7yCas68vhFh0IPytFxXN0iOZEGc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1597138654; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QneWKlrP0albiRJHLibfn0RC27lZqr/kUeBMHqn6H6g=; b=Aw1o//Gzd2wxfI8HkvVmeUydo+FGcAiwTq+4zmZ8f5vP5h6wF2yLCI4frSQzX68dYH+nb6Qnujfwj9l7fmVobJnnXLKxC4GGoDn+ir8Kw8Rc9lnF0PTlXWSadeIo+Y8en8p1MS2NzOSn3GAqqEZA5Qap9wj1j2dkIQfVlsGz/UU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1597138654697625.8506664956838; Tue, 11 Aug 2020 02:37:34 -0700 (PDT) Received: from localhost ([::1]:49930 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k5Qir-0002hX-1q for importer@patchew.org; Tue, 11 Aug 2020 05:37:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56730) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k5Qgh-0000Dq-EO for qemu-devel@nongnu.org; Tue, 11 Aug 2020 05:35:19 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:46322 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k5Qgf-0001Fh-Cw for qemu-devel@nongnu.org; Tue, 11 Aug 2020 05:35:19 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-254-T1JDczgYOraBS_Xch3BklQ-1; Tue, 11 Aug 2020 05:35:12 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8F24A800479; Tue, 11 Aug 2020 09:35:11 +0000 (UTC) Received: from localhost (ovpn-113-152.ams2.redhat.com [10.36.113.152]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2F1AB5DA33; Tue, 11 Aug 2020 09:35:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597138516; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QneWKlrP0albiRJHLibfn0RC27lZqr/kUeBMHqn6H6g=; b=KQoQrFx9WE5gnuTTsrd7YHOia085vQNPQokLYWOfBO0qCPPum6IJVxPVOwO0ZqHPiOrVHJ TOXHDcwPhCKqzuUQSIaYfSCoi5tFXZFOnMRQnlkfFimpCk1r/5CKV0VHqBdXj73aazdf65 tnnvldXRoplXVA+cMiHUTvKOsS3OTWk= X-MC-Unique: T1JDczgYOraBS_Xch3BklQ-1 From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 2/2] iotests: add test for unaligned granularity bitmap backup Date: Tue, 11 Aug 2020 11:35:05 +0200 Message-Id: <20200811093505.972894-3-mreitz@redhat.com> In-Reply-To: <20200811093505.972894-1-mreitz@redhat.com> References: <20200811093505.972894-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=mreitz@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/11 02:18:06 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Stefan Reiter Start a VM with a 4097 byte image attached, add a 4096 byte granularity dirty bitmap, mark it dirty, and then do a backup. This used to run into an assert and fail, check that it works as expected and also check the created image to ensure that misaligned backups in general work correctly. Signed-off-by: Stefan Reiter Message-Id: <20200810095523.15071-2-s.reiter@proxmox.com> [mreitz: Drop bitmap, and do not write past the image's end] Signed-off-by: Max Reitz --- tests/qemu-iotests/304 | 60 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/304.out | 2 ++ tests/qemu-iotests/group | 1 + 3 files changed, 63 insertions(+) create mode 100755 tests/qemu-iotests/304 create mode 100644 tests/qemu-iotests/304.out diff --git a/tests/qemu-iotests/304 b/tests/qemu-iotests/304 new file mode 100755 index 0000000000..aaf9e14617 --- /dev/null +++ b/tests/qemu-iotests/304 @@ -0,0 +1,60 @@ +#!/usr/bin/env python3 +# +# Tests dirty-bitmap backup with unaligned bitmap granularity +# +# Copyright (c) 2020 Proxmox Server Solutions +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# owner=3Ds.reiter@proxmox.com + +import iotests +from iotests import qemu_img_create, qemu_img_log, file_path + +iotests.script_initialize(supported_fmts=3D['qcow2'], + supported_protocols=3D['file']) + +test_img =3D file_path('test.qcow2') +target_img =3D file_path('target.qcow2') + +# unaligned by one byte +image_len =3D 4097 +bitmap_granularity =3D 4096 + +qemu_img_create('-f', iotests.imgfmt, test_img, str(image_len)) + +# create VM +vm =3D iotests.VM().add_drive(test_img) +vm.launch() + +# write to the entire image +vm.hmp_qemu_io('drive0', 'write -P0x16 0 4096'); +vm.hmp_qemu_io('drive0', 'write -P0x17 4096 1'); + +# do backup and wait for completion +vm.qmp('drive-backup', **{ + 'device': 'drive0', + 'sync': 'full', + 'target': target_img +}) + +event =3D vm.event_wait(name=3D'BLOCK_JOB_COMPLETED', + match=3D{'data': {'device': 'drive0'}}, + timeout=3D5.0) + +# shutdown to sync images +vm.shutdown() + +# backup succeeded, check if image is correct +qemu_img_log('compare', test_img, target_img) diff --git a/tests/qemu-iotests/304.out b/tests/qemu-iotests/304.out new file mode 100644 index 0000000000..381cc056f7 --- /dev/null +++ b/tests/qemu-iotests/304.out @@ -0,0 +1,2 @@ +Images are identical. + diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 025ed5238d..7f76066640 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -309,3 +309,4 @@ 299 auto quick 301 backing quick 302 quick +304 rw quick --=20 2.26.2