From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598018997; cv=none; d=zohomail.com; s=zohoarc; b=iQwL2E/GqzqHtjGPdG43hBK3iodU0FpH/LLM5U/r74jTgKcIdTqGLBEO7KVyOBq+MVtCzOxxPrB8wPij1xAcjd61AEAqSduYkzI9rQ53L45lwVyevbkLINX/v7p0P05Mcb9K5KplJfI3Cfqz0AdN0MWj+d84fUsLYc7TzVO/Ltw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598018997; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=emZF2mYnnEZmfx/We7TVxfW8bQGfU8OWig/qmvfgpYU=; b=bQIwOoUuGHwcCs+o2FPtOfttMwMmEkRZ9bxCS6Xqp+Qb/3wWiISaeUn0+WCq8Gf3zdgpOUNO5oZBFyT1xHxaCpyWCPF6td+Bh1ar7nlKWj4Fd0cco1KxOpKosKkmm4JmorHJPDFiVj4F+RByC9ANdz5PXo8ySrtFiQSp1BZX2Rg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598018997509360.7974984832065; Fri, 21 Aug 2020 07:09:57 -0700 (PDT) Received: from localhost ([::1]:38154 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97jw-0000tb-9L for importer@patchew.org; Fri, 21 Aug 2020 10:09:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54402) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97iu-00075F-8b for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:52 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:32250 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97ir-0004Vw-CG for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:51 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-290-a0TGXhipO16Jiv12XfvLXQ-1; Fri, 21 Aug 2020 10:08:31 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D00211074666; Fri, 21 Aug 2020 14:08:29 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B8797A40B; Fri, 21 Aug 2020 14:08:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018928; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=emZF2mYnnEZmfx/We7TVxfW8bQGfU8OWig/qmvfgpYU=; b=fzQ560FDCZGRVubhjKn10ZxTV8kGggu+mEAuCvuE9JEr0ez2sm5ckg+zhXtgrBcvUgraF5 eWFQqQ2Mkt2QJuv81m+3/pJyZ+1vzc5CAim1Ewt7AavcFOCiG2yzZZRKRDmjjFu4N/VBfj iLwkVwwRTYp/ZRZQWF7GQwlpPq8Qqss= X-MC-Unique: a0TGXhipO16Jiv12XfvLXQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 01/14] iotests: add test for QCOW2 header dump Date: Fri, 21 Aug 2020 09:08:13 -0500 Message-Id: <20200821140826.194322-2-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/20 23:41:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich The simple script creates a QCOW2 image and fills it with some data. Two bitmaps are created as well. Then the script reads the image header with extensions from the disk by running the script qcow2.py and dumps the information to the output. Other entities, such as snapshots, may be added to the test later. Suggested-by: Eric Blake Signed-off-by: Andrey Shinkevich Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-2-git-send-email-andrey.shinkevich@virtuozzo= .com> Signed-off-by: Eric Blake --- tests/qemu-iotests/303 | 60 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/303.out | 60 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 121 insertions(+) create mode 100755 tests/qemu-iotests/303 create mode 100644 tests/qemu-iotests/303.out diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303 new file mode 100755 index 000000000000..e9accdc7bc92 --- /dev/null +++ b/tests/qemu-iotests/303 @@ -0,0 +1,60 @@ +#!/usr/bin/env python3 +# +# Test for dumping of qcow2 image metadata +# +# Copyright (c) 2020 Virtuozzo International GmbH +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import iotests +import subprocess +from iotests import qemu_img_create, qemu_io, file_path, log, filter_qemu_= io + +iotests.script_initialize(supported_fmts=3D['qcow2']) + +disk =3D file_path('disk') +chunk =3D 1024 * 1024 + + +def create_bitmap(bitmap_number, disabled): + granularity =3D 1 << (14 + bitmap_number) + bitmap_name =3D 'bitmap-' + str(bitmap_number) + args =3D ['bitmap', '--add', '-g', f'{granularity}', '-f', iotests.img= fmt, + disk, bitmap_name] + if disabled: + args.append('--disable') + + iotests.qemu_img_pipe(*args) + + +def write_to_disk(offset, size): + write =3D f'write {offset} {size}' + log(qemu_io('-c', write, disk), filters=3D[filter_qemu_io]) + + +def add_bitmap(num, begin, end, disabled): + log(f'Add bitmap {num}') + create_bitmap(num, disabled) + for i in range(begin, end): + write_to_disk((i) * chunk, chunk) + log('') + + +qemu_img_create('-f', iotests.imgfmt, disk, '10M') + +add_bitmap(1, 0, 6, False) +add_bitmap(2, 6, 8, True) +dump =3D ['qcow2.py', disk, 'dump-header'] +subprocess.run(dump) diff --git a/tests/qemu-iotests/303.out b/tests/qemu-iotests/303.out new file mode 100644 index 000000000000..8d7973ccc201 --- /dev/null +++ b/tests/qemu-iotests/303.out @@ -0,0 +1,60 @@ +Add bitmap 1 +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +wrote 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +wrote 1048576/1048576 bytes at offset 2097152 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +wrote 1048576/1048576 bytes at offset 3145728 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +wrote 1048576/1048576 bytes at offset 4194304 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +wrote 1048576/1048576 bytes at offset 5242880 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + + +Add bitmap 2 +wrote 1048576/1048576 bytes at offset 6291456 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +wrote 1048576/1048576 bytes at offset 7340032 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + + +magic 0x514649fb +version 3 +backing_file_offset 0x0 +backing_file_size 0x0 +cluster_bits 16 +size 10485760 +crypt_method 0 +l1_size 1 +l1_table_offset 0x30000 +refcount_table_offset 0x10000 +refcount_table_clusters 1 +nb_snapshots 0 +snapshot_offset 0x0 +incompatible_features [] +compatible_features [] +autoclear_features [0] +refcount_order 4 +header_length 112 + +Header extension: +magic 0x6803f857 (Feature table) +length 336 +data + +Header extension: +magic 0x23852875 (Bitmaps) +length 24 +nb_bitmaps 2 +reserved32 0 +bitmap_directory_size 0x40 +bitmap_directory_offset 0x9d0000 + diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 7f76066640a6..ecff2621cddc 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -309,4 +309,5 @@ 299 auto quick 301 backing quick 302 quick +303 rw quick 304 rw quick --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019089; cv=none; d=zohomail.com; s=zohoarc; b=iD59e3hLZtqlCSlfg3LFbhE/tybII8XXC4/jl0EMgeU00kjnGHv9d5lguoWbkAQZ9pjd0FRmHhfidaO4oTApoJRJvurNfWQ0xGxaezvVamE2aRKnMisVtOPzp8AYbtxfJJRVsxi/qcW0fw8r0dOrW5Dkg/8BnAplxN8sION/BtA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019089; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tGA1OwgdJtJFHMQOYUPRrEoY0FpOkOV+pPbvZU7ogLM=; b=kzviETUSQYUzUH7+HPWYaHcN9gVNh3Y6HvkNpAxIsV+/YsnF7FAHGynKfWTlkV31Z8LsUTAqwcD3yWuFJVJY6w5qgnhUMjCWBWTjttvbnZW117+lCC81yuOUtwCUGbcyoldBxN/huUFQk64R6h6qS7BgbfkwTy26HwuoKokBtJw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019088999932.9410369657764; Fri, 21 Aug 2020 07:11:28 -0700 (PDT) Received: from localhost ([::1]:44950 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97lP-0003h2-Jb for importer@patchew.org; Fri, 21 Aug 2020 10:11:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54406) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97iu-000763-IC for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:48554) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97is-0004W9-Qu for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:52 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-220-nJXqKfJPOEueYHCDMsOJvw-1; Fri, 21 Aug 2020 10:08:31 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6B26080EDA4; Fri, 21 Aug 2020 14:08:30 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 033BA7AEC9; Fri, 21 Aug 2020 14:08:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tGA1OwgdJtJFHMQOYUPRrEoY0FpOkOV+pPbvZU7ogLM=; b=JmaVq/yjvEPVIS3hXX5rc2+9UQcW70m3AfHPIWuPcmjerhio30hPhsfssq0+VgDt0P2taB Pztudt4yNt89pIbbH8YJcl/vDVstqizXQQGIlLHmcsYnUyhTkltm8eY4fwOlxSUNkup5w/ hFFoJDiJtcx5yfmBpUOqQA/iKni5oGc= X-MC-Unique: nJXqKfJPOEueYHCDMsOJvw-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 02/14] qcow2_format.py: make printable data an extension class member Date: Fri, 21 Aug 2020 09:08:14 -0500 Message-Id: <20200821140826.194322-3-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 02:43:55 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich Let us differ binary data type from string one for the extension data variable and keep the string as the QcowHeaderExtension class member. Signed-off-by: Andrey Shinkevich Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-3-git-send-email-andrey.shinkevich@virtuozzo= .com> Signed-off-by: Eric Blake --- tests/qemu-iotests/qcow2_format.py | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index cc432e7ae06c..2f3681bb5f7c 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -165,6 +165,13 @@ class QcowHeaderExtension(Qcow2Struct): self.data =3D fd.read(padded) assert self.data is not None + data_str =3D self.data[:self.length] + if all(c in string.printable.encode('ascii') for c in data_str): + data_str =3D f"'{ data_str.decode('ascii') }'" + else: + data_str =3D '' + self.data_str =3D data_str + if self.magic =3D=3D QCOW2_EXT_MAGIC_BITMAPS: self.obj =3D Qcow2BitmapExt(data=3Dself.data) else: @@ -174,12 +181,7 @@ class QcowHeaderExtension(Qcow2Struct): super().dump() if self.obj is None: - data =3D self.data[:self.length] - if all(c in string.printable.encode('ascii') for c in data): - data =3D f"'{ data.decode('ascii') }'" - else: - data =3D '' - print(f'{"data":<25} {data}') + print(f'{"data":<25} {self.data_str}') else: self.obj.dump() --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019240; cv=none; d=zohomail.com; s=zohoarc; b=cBXYgHrkPZ9CbySbmP1KeDi8S7sQNd4NFjC7vtKRXMFQ2siQgsay0zZP9Qwi7usHHf627+C5OpBfSwGer0M4J+A5zpODVgLncF3VIK26pq3pXHRPnRtynIko6/HcbbbqsWJ0za1SwzWzzwvzP0Tn3I56vUf/wznbTrphpIsKDno= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019240; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XOclbXnZE4kY6hOS/0TCAiAkVWRtcrn3Emkrekd17sQ=; b=CS1c1oDnuvUCwSKMZQql+C9VAEkWm50P6k0cbWRi3jDa6JPRWsJhN4I2a1rBEkjlLNXpqX5sTKfPhBP+HS5dcibBAxtenSmaSbWdXvq55Uq//NGph+AFXFlmXVaeMqOBOrV8LkbzCrMvs0SIggkiz/QG+nA0qTLtdlAl+BGGfps= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019240096773.4252608046804; Fri, 21 Aug 2020 07:14:00 -0700 (PDT) Received: from localhost ([::1]:57166 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97np-0000HG-Qk for importer@patchew.org; Fri, 21 Aug 2020 10:13:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54438) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97iv-00079z-Pk for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:53 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:55937 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97it-0004WG-To for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:53 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-398-1pq0FeeePj2s11Vrpbwwpg-1; Fri, 21 Aug 2020 10:08:32 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0984A18A2245; Fri, 21 Aug 2020 14:08:31 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 930A67A40B; Fri, 21 Aug 2020 14:08:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XOclbXnZE4kY6hOS/0TCAiAkVWRtcrn3Emkrekd17sQ=; b=A6lJs0ojhyPQG0XpaxJ9tml4K5+NFSVdsCeNbwP8y99n1lbvo316esKNBwCPFXPaDrsXSi TATB9bRXGhB/UEMihslvvfgkXUvC+RxbZ1K47Kv7OXsjbXpygenJGkdRzD/wIDLvR87T6Y ycSg39AAHajEwGpHPPhj9ZjckEtm9+A= X-MC-Unique: 1pq0FeeePj2s11Vrpbwwpg-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 03/14] qcow2_format.py: change Qcow2BitmapExt initialization method Date: Fri, 21 Aug 2020 09:08:15 -0500 Message-Id: <20200821140826.194322-4-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 01:00:15 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich There are two ways to initialize a class derived from Qcow2Struct: 1. Pass a block of binary data to the constructor. 2. Pass the file descriptor to allow reading the file from constructor. Let's change the Qcow2BitmapExt initialization method from 1 to 2 to support a scattered reading in the initialization chain. The implementation comes with the patch that follows. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-4-git-send-email-andrey.shinkevich@virtuozzo= .com> Signed-off-by: Eric Blake --- tests/qemu-iotests/qcow2_format.py | 34 ++++++++++++++++++------------ 1 file changed, 21 insertions(+), 13 deletions(-) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index 2f3681bb5f7c..d4a997453758 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -113,6 +113,11 @@ class Qcow2BitmapExt(Qcow2Struct): ('u64', '{:#x}', 'bitmap_directory_offset') ) + def __init__(self, fd): + super().__init__(fd=3Dfd) + tail =3D struct.calcsize(self.fmt) % 8 + if tail: + fd.seek(8 - tail, 1) QCOW2_EXT_MAGIC_BITMAPS =3D 0x23852875 @@ -161,21 +166,24 @@ class QcowHeaderExtension(Qcow2Struct): else: assert all(v is None for v in (magic, length, data)) super().__init__(fd=3Dfd) - padded =3D (self.length + 7) & ~7 - self.data =3D fd.read(padded) - assert self.data is not None + if self.magic =3D=3D QCOW2_EXT_MAGIC_BITMAPS: + self.obj =3D Qcow2BitmapExt(fd=3Dfd) + self.data =3D None + else: + padded =3D (self.length + 7) & ~7 + self.data =3D fd.read(padded) + assert self.data is not None + self.obj =3D None - data_str =3D self.data[:self.length] - if all(c in string.printable.encode('ascii') for c in data_str): - data_str =3D f"'{ data_str.decode('ascii') }'" - else: - data_str =3D '' - self.data_str =3D data_str + if self.data is not None: + data_str =3D self.data[:self.length] + if all(c in string.printable.encode( + 'ascii') for c in data_str): + data_str =3D f"'{ data_str.decode('ascii') }'" + else: + data_str =3D '' + self.data_str =3D data_str - if self.magic =3D=3D QCOW2_EXT_MAGIC_BITMAPS: - self.obj =3D Qcow2BitmapExt(data=3Dself.data) - else: - self.obj =3D None def dump(self): super().dump() --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598018977; cv=none; d=zohomail.com; s=zohoarc; b=In4HS+E4wTytOhvKUiROqbrdfSiR+5Dj6UT9vCERYGW3ty8fC+qxPf6/IlB6epZl0LjAVgGbt1YKut+3KhhJlbsYr24NCSo4wtnbztrFKRVbwI7po+y+9nZ/xHITfU8+7h/OGpzCM/OQ3BRAoHcLDsh+2Z93DsRaMSdxUnSqVuw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598018977; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=iSUl++kbgd9oEPboS2uSlV/VLy8nf/lhIZru0Q/wXH0=; b=lZ/S8684WcQRJHNkGp6LbmWbsfOSCU73UYfzeatzzoKbJ5w4JsZcy+x0F6Inl/q4wkj/fI4Ni7VLi4jfA9w6IJkDRvTboIcdfgRPX6UbKW2tUfYv25pSrkz0hY+FJAzM8JoWllkoZRk5l2ydhzA0H6+Fa04G5l3oekMMivHCC5Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159801897795792.32626337324143; Fri, 21 Aug 2020 07:09:37 -0700 (PDT) Received: from localhost ([::1]:35868 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97jc-0008Oj-Qm for importer@patchew.org; Fri, 21 Aug 2020 10:09:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54286) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97ih-0006gl-W1 for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:40 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:57901 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97ig-0004Ul-3v for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:39 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-298-RZKeFV5hPbaC-CtbQTFpCQ-1; Fri, 21 Aug 2020 10:08:32 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A2EE780EFBD; Fri, 21 Aug 2020 14:08:31 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3A5F47A40B; Fri, 21 Aug 2020 14:08:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018916; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iSUl++kbgd9oEPboS2uSlV/VLy8nf/lhIZru0Q/wXH0=; b=a9Abb/Ni0SYRNSqjCRHxYeZk/fOoYb2UXdMVamQf/EFew5BOdx5SkA9WB9CWUVf7AmzfF4 NfV8PBErvEe0NcTz8ROslzvrmr5MKYm9UXX2eENHUJjEr3//SXCiGUY8vayTcgAzLtybnK TRo2ocUeeKUZeYbaIRi6hPET+yZ6mRM= X-MC-Unique: RZKeFV5hPbaC-CtbQTFpCQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 04/14] qcow2_format.py: dump bitmap flags in human readable way. Date: Fri, 21 Aug 2020 09:08:16 -0500 Message-Id: <20200821140826.194322-5-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/20 23:41:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich Introduce the class BitmapFlags that parses a bitmap flags mask. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-5-git-send-email-andrey.shinkevich@virtuozzo= .com> Signed-off-by: Eric Blake --- tests/qemu-iotests/qcow2_format.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index d4a997453758..b4473442c9d4 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -40,6 +40,22 @@ class Flags64(Qcow2Field): return str(bits) +class BitmapFlags(Qcow2Field): + + flags =3D { + 0x1: 'in-use', + 0x2: 'auto' + } + + def __str__(self): + bits =3D [] + for bit in range(64): + flag =3D self.value & (1 << bit) + if flag: + bits.append(self.flags.get(flag, f'bit-{bit}')) + return f'{self.value:#x} ({bits})' + + class Enum(Qcow2Field): def __str__(self): --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019284; cv=none; d=zohomail.com; s=zohoarc; b=S4WxOllpGEsXWeMQAQCCh2++WHXYVyH974dkvrjWiFg2zoh2duH4hlf4HEO1/3rf4tVrI7RvjYsqPIeox/UWG1gWh//J9Bm1tbwjfEpcR5tTIu8eP8211laD1JrIoQBiX3Xd7DkcGXCHqMXNM5+uMrNh6S32lDQO7cUE5XOtsNQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019284; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Ft+sZwk9gcDvF38VuUsUZ21PepdX21tQNatQ9qp1XgI=; b=IFsumx36Hvg8Y4Wm2qduxoqoRZI1nR63Gd8A8eTbwGtxZxn3UVvHHXLeb17Y5/1JlqGblFfQWBDvayyWD0szHMbMQFOaAD4MpWpe/imI8cVtYfxLB471sfMHWIximsZ3h5eNVH4gS7hJDknA7GoEZLY8eXb4YoFxW8OktWwIT04= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019284386871.6397761106787; Fri, 21 Aug 2020 07:14:44 -0700 (PDT) Received: from localhost ([::1]:32968 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97oZ-0001u5-46 for importer@patchew.org; Fri, 21 Aug 2020 10:14:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54520) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97iz-0007KF-Br for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:57 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:46210 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97ix-0004Xg-8e for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:57 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-405-wIc2WR9CPPOCtde7RYdzLA-1; Fri, 21 Aug 2020 10:08:33 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 44A441019633; Fri, 21 Aug 2020 14:08:32 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id CE8617A40B; Fri, 21 Aug 2020 14:08:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018934; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ft+sZwk9gcDvF38VuUsUZ21PepdX21tQNatQ9qp1XgI=; b=Z6krHWy8fYMAbsQHxiLgJPLB2q3Tk78AzNFoOPAvDji4V8/esFUSj9h9/fdpAjNhtrWDmX 4//T62zHVmVBdb26axNsNDKdzslRRAzon/DKnzOLyWtETthctp0/Xza9p0aI2AKx08iwue IJhkPeCFoNy6SBusW66a6zjFBdEgTRA= X-MC-Unique: wIc2WR9CPPOCtde7RYdzLA-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 05/14] qcow2_format.py: Dump bitmap directory information Date: Fri, 21 Aug 2020 09:08:17 -0500 Message-Id: <20200821140826.194322-6-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 05:15:56 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich Read and dump entries from the bitmap directory of QCOW2 image. Header extension: magic 0x23852875 (Bitmaps) ... Bitmap name bitmap-1 bitmap_table_offset 0xf0000 bitmap_table_size 1 flags 0x2 (['auto']) type 1 granularity_bits 16 name_size 8 extra_data_size 0 Suggested-by: Kevin Wolf Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-6-git-send-email-andrey.shinkevich@virtuozzo= .com> Signed-off-by: Eric Blake --- tests/qemu-iotests/303.out | 18 ++++++++++++ tests/qemu-iotests/qcow2_format.py | 47 ++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/tests/qemu-iotests/303.out b/tests/qemu-iotests/303.out index 8d7973ccc201..038ba93a87d7 100644 --- a/tests/qemu-iotests/303.out +++ b/tests/qemu-iotests/303.out @@ -58,3 +58,21 @@ reserved32 0 bitmap_directory_size 0x40 bitmap_directory_offset 0x9d0000 +Bitmap name bitmap-1 +bitmap_table_offset 0x9b0000 +bitmap_table_size 1 +flags 0x2 (['auto']) +type 1 +granularity_bits 15 +name_size 8 +extra_data_size 0 + +Bitmap name bitmap-2 +bitmap_table_offset 0x9c0000 +bitmap_table_size 1 +flags 0x0 ([]) +type 1 +granularity_bits 16 +name_size 8 +extra_data_size 0 + diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index b4473442c9d4..05a8aa98f72c 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -134,6 +134,53 @@ class Qcow2BitmapExt(Qcow2Struct): tail =3D struct.calcsize(self.fmt) % 8 if tail: fd.seek(8 - tail, 1) + position =3D fd.tell() + self.read_bitmap_directory(fd) + fd.seek(position) + + def read_bitmap_directory(self, fd): + fd.seek(self.bitmap_directory_offset) + self.bitmap_directory =3D \ + [Qcow2BitmapDirEntry(fd) for _ in range(self.nb_bitmaps)] + + def dump(self): + super().dump() + for entry in self.bitmap_directory: + print() + entry.dump() + + +class Qcow2BitmapDirEntry(Qcow2Struct): + + fields =3D ( + ('u64', '{:#x}', 'bitmap_table_offset'), + ('u32', '{}', 'bitmap_table_size'), + ('u32', BitmapFlags, 'flags'), + ('u8', '{}', 'type'), + ('u8', '{}', 'granularity_bits'), + ('u16', '{}', 'name_size'), + ('u32', '{}', 'extra_data_size') + ) + + def __init__(self, fd): + super().__init__(fd=3Dfd) + # Seek relative to the current position in the file + fd.seek(self.extra_data_size, 1) + bitmap_name =3D fd.read(self.name_size) + self.name =3D bitmap_name.decode('ascii') + # Move position to the end of the entry in the directory + entry_raw_size =3D self.bitmap_dir_entry_raw_size() + padding =3D ((entry_raw_size + 7) & ~7) - entry_raw_size + fd.seek(padding, 1) + + def bitmap_dir_entry_raw_size(self): + return struct.calcsize(self.fmt) + self.name_size + \ + self.extra_data_size + + def dump(self): + print(f'{"Bitmap name":<25} {self.name}') + super(Qcow2BitmapDirEntry, self).dump() + QCOW2_EXT_MAGIC_BITMAPS =3D 0x23852875 --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019089; cv=none; d=zohomail.com; s=zohoarc; b=e93pPj+mTYwQnlNyj2K0T7vK8VkkUzST22E0h6iHRKQIc/SjCrj6nho/jtmRyXTUVft++BOFtFdkgF/Bj5IWS5eANiQQeLrpIlbLYuwm09LNNeVRUQrPaurmwhueeW9VV9nslZ74L20HBiaHqkW9BoU0YQJ4MqRWjRsKc3VhueA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019089; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8ITtCRSoAJ/tFdQGx3yupOIQKm7eEYRXmBMA6aBINiA=; b=kFsvOGcDCq8F2Kl12A66dDSVuNrtHWzJp+kOCvfzJVmP7nrrSmW9omTYoX9sIAxJZIZ8DCk6l/jeAxrPXlUbV4hSAD4JixB/aiR5i1ryJpcBolueCAq9bCeXWmoSDdiibM9Yx6cWE86S3JsdUNxmjSTNu9w1PvorOUO0pkNJxf0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019089614864.7219828042098; Fri, 21 Aug 2020 07:11:29 -0700 (PDT) Received: from localhost ([::1]:45006 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97lQ-0003iO-7B for importer@patchew.org; Fri, 21 Aug 2020 10:11:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97iy-0007HY-Eo for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:24287) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97iw-0004XR-HS for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:56 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-576-QRQJ3Gu2MpGGZwHc164rgA-1; Fri, 21 Aug 2020 10:08:37 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C375A85C70A; Fri, 21 Aug 2020 14:08:35 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6CAC87AEC9; Fri, 21 Aug 2020 14:08:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018933; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8ITtCRSoAJ/tFdQGx3yupOIQKm7eEYRXmBMA6aBINiA=; b=UfUEOe5oXikZxY+GL9tCjcxAyE1ZJrEzEueF3MubAfZF39wFG3gMmohhVOFD7IlF1aH4Xt B8SFH33cA5sKQCnLnkqGFwl/lo2B2pMGPZ4rFWzfczTJR1jDPCb0izh9OAOvtfriI8XurQ MZqX4ogWXgkzTwqZBTYNYWXt1QjeQu4= X-MC-Unique: QRQJ3Gu2MpGGZwHc164rgA-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 06/14] qcow2_format.py: pass cluster size to substructures Date: Fri, 21 Aug 2020 09:08:18 -0500 Message-Id: <20200821140826.194322-7-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 02:43:55 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich The cluster size of an image is the QcowHeader class member and may be obtained by dependent extension structures such as Qcow2BitmapExt for further bitmap table details print. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-7-git-send-email-andrey.shinkevich@virtuozzo= .com> Signed-off-by: Eric Blake --- tests/qemu-iotests/qcow2_format.py | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index 05a8aa98f72c..ca0d3501e0a2 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -129,19 +129,21 @@ class Qcow2BitmapExt(Qcow2Struct): ('u64', '{:#x}', 'bitmap_directory_offset') ) - def __init__(self, fd): + def __init__(self, fd, cluster_size): super().__init__(fd=3Dfd) tail =3D struct.calcsize(self.fmt) % 8 if tail: fd.seek(8 - tail, 1) position =3D fd.tell() + self.cluster_size =3D cluster_size self.read_bitmap_directory(fd) fd.seek(position) def read_bitmap_directory(self, fd): fd.seek(self.bitmap_directory_offset) self.bitmap_directory =3D \ - [Qcow2BitmapDirEntry(fd) for _ in range(self.nb_bitmaps)] + [Qcow2BitmapDirEntry(fd, cluster_size=3Dself.cluster_size) + for _ in range(self.nb_bitmaps)] def dump(self): super().dump() @@ -162,8 +164,9 @@ class Qcow2BitmapDirEntry(Qcow2Struct): ('u32', '{}', 'extra_data_size') ) - def __init__(self, fd): + def __init__(self, fd, cluster_size): super().__init__(fd=3Dfd) + self.cluster_size =3D cluster_size # Seek relative to the current position in the file fd.seek(self.extra_data_size, 1) bitmap_name =3D fd.read(self.name_size) @@ -203,11 +206,13 @@ class QcowHeaderExtension(Qcow2Struct): # then padding to next multiply of 8 ) - def __init__(self, magic=3DNone, length=3DNone, data=3DNone, fd=3DNone= ): + def __init__(self, magic=3DNone, length=3DNone, data=3DNone, fd=3DNone, + cluster_size=3DNone): """ Support both loading from fd and creation from user data. For fd-based creation current position in a file will be used to r= ead the data. + The cluster_size value may be obtained by dependent structures. This should be somehow refactored and functionality should be move= d to superclass (to allow creation of any qcow2 struct), but then, fiel= ds @@ -230,7 +235,7 @@ class QcowHeaderExtension(Qcow2Struct): assert all(v is None for v in (magic, length, data)) super().__init__(fd=3Dfd) if self.magic =3D=3D QCOW2_EXT_MAGIC_BITMAPS: - self.obj =3D Qcow2BitmapExt(fd=3Dfd) + self.obj =3D Qcow2BitmapExt(fd=3Dfd, cluster_size=3Dcluste= r_size) self.data =3D None else: padded =3D (self.length + 7) & ~7 @@ -319,7 +324,7 @@ class QcowHeader(Qcow2Struct): end =3D self.cluster_size while fd.tell() < end: - ext =3D QcowHeaderExtension(fd=3Dfd) + ext =3D QcowHeaderExtension(fd=3Dfd, cluster_size=3Dself.clust= er_size) if ext.magic =3D=3D 0: break else: --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019052; cv=none; d=zohomail.com; s=zohoarc; b=CpQ03vXXxIB+APLAfMVwvFIl4z6loxoHIGDKXki38hx/o8z2C9+FwlPneMMmrRej+UAC1KWAvV/MpywSPfiZpzZcrq+GUA3uKO3k0Vsjz7cBSlIXLRBdAtyQ3xWpLzFWrHkSZoEtGLM3HBoetSgIgU9bnXQQJlIeKdDUpuT9IPk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019052; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dFTJI66gXbHFJpiFeUt37MLvYmD4RYoxQkVSz55LzJM=; b=Tt3Se8WQWSgqIkAllwSZn76cW1TiVovXEXyyKesySPIgZ10izQxH/HQsJlzByqGkgNDtobcAF+wVCtjsZcQTI5BVnszEWgQ9Q3hOpsLFWkBCrDYCKug5hkmFjGJpDfjw75a2+W9kAHqUnrnDDGbQgzv1NS4Q/gU/eWM08Zf5/IA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019052592531.043160957153; Fri, 21 Aug 2020 07:10:52 -0700 (PDT) Received: from localhost ([::1]:41878 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97kp-0002Oh-Aq for importer@patchew.org; Fri, 21 Aug 2020 10:10:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54544) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97j0-0007Md-42 for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:58 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:21958 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97ix-0004Xd-5j for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:57 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-467-co64W7f9MSOUk9jMUFP85Q-1; Fri, 21 Aug 2020 10:08:37 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 65B1B101962B; Fri, 21 Aug 2020 14:08:36 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id ED4D07AEC9; Fri, 21 Aug 2020 14:08:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018934; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dFTJI66gXbHFJpiFeUt37MLvYmD4RYoxQkVSz55LzJM=; b=CQJ2hNUcIdQ79whHcWqgGE46c5FfKLQ23u9LgJ1LKumdPqKvRzkvoqJClDRZV34uAPYpaA PQYO6I6anwl++yclR3lqWgGXFWfuWA6jmvtJX7WuPkDBokHKfK16sClFadLj6RW0ry69DB 75kZoP12wayCvvPUKdu6cO8ixHdM3yQ= X-MC-Unique: co64W7f9MSOUk9jMUFP85Q-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 07/14] qcow2_format.py: Dump bitmap table serialized entries Date: Fri, 21 Aug 2020 09:08:19 -0500 Message-Id: <20200821140826.194322-8-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 05:15:56 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich Add bitmap table information to the QCOW2 metadata dump. Bitmap name bitmap-1 ... Bitmap table type size offset 0 serialized 65536 10092544 1 all-zeroes 0 0 2 all-zeroes 0 0 Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-8-git-send-email-andrey.shinkevich@virtuozzo= .com> Signed-off-by: Eric Blake --- tests/qemu-iotests/303.out | 4 +++ tests/qemu-iotests/qcow2_format.py | 50 ++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/tests/qemu-iotests/303.out b/tests/qemu-iotests/303.out index 038ba93a87d7..70828e05f11f 100644 --- a/tests/qemu-iotests/303.out +++ b/tests/qemu-iotests/303.out @@ -66,6 +66,8 @@ type 1 granularity_bits 15 name_size 8 extra_data_size 0 +Bitmap table type size offset +0 serialized 65536 10092544 Bitmap name bitmap-2 bitmap_table_offset 0x9c0000 @@ -75,4 +77,6 @@ type 1 granularity_bits 16 name_size 8 extra_data_size 0 +Bitmap table type size offset +0 all-zeroes 0 0 diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index ca0d3501e0a2..574249bc463c 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -175,6 +175,10 @@ class Qcow2BitmapDirEntry(Qcow2Struct): entry_raw_size =3D self.bitmap_dir_entry_raw_size() padding =3D ((entry_raw_size + 7) & ~7) - entry_raw_size fd.seek(padding, 1) + self.bitmap_table =3D Qcow2BitmapTable(fd=3Dfd, + offset=3Dself.bitmap_table_of= fset, + nb_entries=3Dself.bitmap_tabl= e_size, + cluster_size=3Dself.cluster_s= ize) def bitmap_dir_entry_raw_size(self): return struct.calcsize(self.fmt) + self.name_size + \ @@ -183,6 +187,52 @@ class Qcow2BitmapDirEntry(Qcow2Struct): def dump(self): print(f'{"Bitmap name":<25} {self.name}') super(Qcow2BitmapDirEntry, self).dump() + self.bitmap_table.dump() + + +class Qcow2BitmapTableEntry(Qcow2Struct): + + fields =3D ( + ('u64', '{}', 'entry'), + ) + + BME_TABLE_ENTRY_RESERVED_MASK =3D 0xff000000000001fe + BME_TABLE_ENTRY_OFFSET_MASK =3D 0x00fffffffffffe00 + BME_TABLE_ENTRY_FLAG_ALL_ONES =3D 1 + + def __init__(self, fd): + super().__init__(fd=3Dfd) + self.reserved =3D self.entry & self.BME_TABLE_ENTRY_RESERVED_MASK + self.offset =3D self.entry & self.BME_TABLE_ENTRY_OFFSET_MASK + if self.offset: + if self.entry & self.BME_TABLE_ENTRY_FLAG_ALL_ONES: + self.type =3D 'invalid' + else: + self.type =3D 'serialized' + elif self.entry & self.BME_TABLE_ENTRY_FLAG_ALL_ONES: + self.type =3D 'all-ones' + else: + self.type =3D 'all-zeroes' + + +class Qcow2BitmapTable: + + def __init__(self, fd, offset, nb_entries, cluster_size): + self.cluster_size =3D cluster_size + position =3D fd.tell() + fd.seek(offset) + self.entries =3D [Qcow2BitmapTableEntry(fd) for _ in range(nb_entr= ies)] + fd.seek(position) + + def dump(self): + bitmap_table =3D enumerate(self.entries) + print(f'{"Bitmap table":<14} {"type":<15} {"size":<12} {"offset"}') + for i, entry in bitmap_table: + if entry.type =3D=3D 'serialized': + size =3D self.cluster_size + else: + size =3D 0 + print(f'{i:<14} {entry.type:<15} {size:<12} {entry.offset}') QCOW2_EXT_MAGIC_BITMAPS =3D 0x23852875 --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019487; cv=none; d=zohomail.com; s=zohoarc; b=jGsNeHGZxLTZIZtnviO+9k3d/2PfXRI1EgZz2Xq/CEnEolVwBWcMPEaKFgZJUImuBkc2XJACqesw62HVvPPRlfzNnrRIab61wiTW6L6J24XgvQyx9l/NKrB5m0/xbBw7QBTONnsTVfVX7PGIWL8Is/qKCYif4VU5r0f8pZ544Qk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019487; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OKTiWydU/nqR2BLF9AhL3+Bno7cgn33+guhRB4DEWZc=; b=I8OJVlE/DdK5vExKDSSWD1chEUU2az/vPqVLZD40Q8c2Gbd6nq/YJ1+51CuyzlwnEMrjUGuvgZf0Pkv/EVIEcEABxfMqCjSa3cHqV7vPKFOknVUbNY/W+/5Q8AvmIzluYq5LAyZwQQiGiISFb3wq96glSuhes/8rbH48xneaVfQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019487889125.83970542985753; Fri, 21 Aug 2020 07:18:07 -0700 (PDT) Received: from localhost ([::1]:50214 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97rq-0000Yt-Fp for importer@patchew.org; Fri, 21 Aug 2020 10:18:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54582) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97j1-0007RV-Uf for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:59 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:57049) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97iz-0004Yt-Mr for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:59 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-17-uv-TMDEWNaePRDHmLWbE_w-1; Fri, 21 Aug 2020 10:08:38 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0465364081; Fri, 21 Aug 2020 14:08:37 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8F49D7AEC9; Fri, 21 Aug 2020 14:08:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018937; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OKTiWydU/nqR2BLF9AhL3+Bno7cgn33+guhRB4DEWZc=; b=DDRR6x00S3QsrpqYm/qRWHo+8kXWEPQ3i4b2hQmsyqRH0Vxq6l6gnGk1rcfmpD9RZ9A348 erNB0FLDIf7MHdzH6dfMxJTlEW5JSQWMFetB45TxRSDlJXb3wXHNacOfDLOewQWRAR8uJA dKRKAKbW4uaGxkzzpNYBUwiHL1ItqTQ= X-MC-Unique: uv-TMDEWNaePRDHmLWbE_w-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 08/14] qcow2.py: Introduce '-j' key to dump in JSON format Date: Fri, 21 Aug 2020 09:08:20 -0500 Message-Id: <20200821140826.194322-9-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 05:15:56 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich Add the command key to the qcow2.py arguments list to dump QCOW2 metadata in JSON format. Here is the suggested way to do that. The implementation of the dump in JSON format is in the patch that follows. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-9-git-send-email-andrey.shinkevich@virtuozzo= .com> Signed-off-by: Eric Blake --- tests/qemu-iotests/qcow2.py | 18 ++++++++++++++---- tests/qemu-iotests/qcow2_format.py | 4 ++-- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index 0910e6ac0705..77ca59cc663d 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -26,16 +26,19 @@ from qcow2_format import ( ) +is_json =3D False + + def cmd_dump_header(fd): h =3D QcowHeader(fd) - h.dump() + h.dump(is_json) print() - h.dump_extensions() + h.dump_extensions(is_json) def cmd_dump_header_exts(fd): h =3D QcowHeader(fd) - h.dump_extensions() + h.dump_extensions(is_json) def cmd_set_header(fd, name, value): @@ -151,11 +154,14 @@ def main(filename, cmd, args): def usage(): - print("Usage: %s [, ...]" % sys.argv[0]) + print("Usage: %s [, ...] [, ...]" % sys.argv[0]) print("") print("Supported commands:") for name, handler, num_args, desc in cmds: print(" %-20s - %s" % (name, desc)) + print("") + print("Supported keys:") + print(" %-20s - %s" % ('-j', 'Dump in JSON format')) if __name__ =3D=3D '__main__': @@ -163,4 +169,8 @@ if __name__ =3D=3D '__main__': usage() sys.exit(1) + is_json =3D '-j' in sys.argv + if is_json: + sys.argv.remove('-j') + main(sys.argv[1], sys.argv[2], sys.argv[3:]) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index 574249bc463c..de0adcbf9db0 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -109,7 +109,7 @@ class Qcow2Struct(metaclass=3DQcow2StructMeta): self.__dict__ =3D dict((field[2], values[i]) for i, field in enumerate(self.fields)) - def dump(self): + def dump(self, is_json=3DFalse): for f in self.fields: value =3D self.__dict__[f[2]] if isinstance(f[1], str): @@ -408,7 +408,7 @@ class QcowHeader(Qcow2Struct): buf =3D buf[0:header_bytes-1] fd.write(buf) - def dump_extensions(self): + def dump_extensions(self, is_json=3DFalse): for ex in self.extensions: print('Header extension:') ex.dump() --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019373; cv=none; d=zohomail.com; s=zohoarc; b=O3URNo00+Bx0cpgIWwxWEtg6niJc+hK2JDJfJAqgm2Jm+C2lIQL+BeJ0nsqtA3ElR/q1z3b5HQPkCZnR2zUXhlLgMnWjODgrl+McRHGedXx3ERSj+3xtVZUl5c2K7fHFttm7gty0MmX0XwZJpTwTPOnPd9ITSOczYqfPntN2IBQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019373; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bb4fvTMSCItB9jUwQLJKgqwZGCl+g+KP57d0VMWcbPo=; b=PyFU5VhJheCxFglllqqR60ZClccu6v/Sy5kVXbgRTguJLtU+Gdlxq5ikzol6eCi7R8Zk/ISUQNw1a8bKQlP8lfOrK7Ith+WUuudI1BSEsI9ADoBH5n0Tj/nLX+olKEm7N6UjIHG0T0lsbkp9v/1SKJotsWgQYgpdKH6iOxgWGds= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019373446810.5716740840095; Fri, 21 Aug 2020 07:16:13 -0700 (PDT) Received: from localhost ([::1]:41296 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97pz-0005LK-Tz for importer@patchew.org; Fri, 21 Aug 2020 10:16:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54558) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97j0-0007OT-RF for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:58 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:51482 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97iy-0004YK-G8 for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:58 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-69-FSEnJA6-NSu7NVfGIRUuPg-1; Fri, 21 Aug 2020 10:08:38 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9D69781F03E; Fri, 21 Aug 2020 14:08:37 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2E1067AEC9; Fri, 21 Aug 2020 14:08:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018935; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bb4fvTMSCItB9jUwQLJKgqwZGCl+g+KP57d0VMWcbPo=; b=JjwoGtLbAlU82lLk6X0OwibPL7l9ROUwSR5PRbvjCeoch9ve6I/B2HVGri64SmLwk8UkCy 6pg1jUqpfp1GuWjmpnSygHa668dpkYzhJi3oSxICcc3OBixhoLmPGvRgc6USJIauTl1hMr 7uerL2ol7FuEMyMA3a9sEWsllnTw/9c= X-MC-Unique: FSEnJA6-NSu7NVfGIRUuPg-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 09/14] qcow2_format.py: collect fields to dump in JSON format Date: Fri, 21 Aug 2020 09:08:21 -0500 Message-Id: <20200821140826.194322-10-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 05:15:56 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich As __dict__ is being extended with class members we do not want to print, add the to_json() method to classes that returns a json-dumpable object with desired fields and their values. Extend it in subclass when necessary to print the final dictionary in the JSON output which follows. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Message-Id: <1596742557-320265-10-git-send-email-andrey.shinkevich@virtuozz= o.com> Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Eric Blake --- tests/qemu-iotests/qcow2_format.py | 36 ++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index de0adcbf9db0..5a298b2f1357 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -119,6 +119,9 @@ class Qcow2Struct(metaclass=3DQcow2StructMeta): print('{:<25} {}'.format(f[2], value_str)) + def to_json(self): + return dict((f[2], self.__dict__[f[2]]) for f in self.fields) + class Qcow2BitmapExt(Qcow2Struct): @@ -151,6 +154,11 @@ class Qcow2BitmapExt(Qcow2Struct): print() entry.dump() + def to_json(self): + fields_dict =3D super().to_json() + fields_dict['bitmap_directory'] =3D self.bitmap_directory + return fields_dict + class Qcow2BitmapDirEntry(Qcow2Struct): @@ -189,6 +197,14 @@ class Qcow2BitmapDirEntry(Qcow2Struct): super(Qcow2BitmapDirEntry, self).dump() self.bitmap_table.dump() + def to_json(self): + # Put the name ahead of the dict + return { + 'name': self.name, + **super().to_json(), + 'bitmap_table': self.bitmap_table + } + class Qcow2BitmapTableEntry(Qcow2Struct): @@ -214,6 +230,10 @@ class Qcow2BitmapTableEntry(Qcow2Struct): else: self.type =3D 'all-zeroes' + def to_json(self): + return {'type': self.type, 'offset': self.offset, + 'reserved': self.reserved} + class Qcow2BitmapTable: @@ -234,6 +254,9 @@ class Qcow2BitmapTable: size =3D 0 print(f'{i:<14} {entry.type:<15} {size:<12} {entry.offset}') + def to_json(self): + return self.entries + QCOW2_EXT_MAGIC_BITMAPS =3D 0x23852875 @@ -249,6 +272,9 @@ class QcowHeaderExtension(Qcow2Struct): 0x44415441: 'Data file' } + def to_json(self): + return self.mapping.get(self.value, "") + fields =3D ( ('u32', Magic, 'magic'), ('u32', '{}', 'length') @@ -311,6 +337,16 @@ class QcowHeaderExtension(Qcow2Struct): else: self.obj.dump() + def to_json(self): + # Put the name ahead of the dict + res =3D {'name': self.Magic(self.magic), **super().to_json()} + if self.obj is not None: + res['data'] =3D self.obj + else: + res['data_str'] =3D self.data_str + + return res + @classmethod def create(cls, magic, data): return QcowHeaderExtension(magic, len(data), data) --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598018983; cv=none; d=zohomail.com; s=zohoarc; b=Wlo1rwIBM8D54vxYpREhSZvWUjHIQ57z4Nm6uSPtwpWWfX8M4duue4CNxlKMWxwgOn4I8WlYxeS8UtIaeKiPeUBbZoEkzkSjiJh7tel7lWEWGNSdLonfnOc0HcWgDoei6n5Csrk/4WPOQBUKD/89/knUE0DFwUB9m5GO5dgb6zU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598018983; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0uHSGE1QSAmI5bNa26JHPi6ph20kehqML5uG+ym6vxA=; b=PrCWREMuD3iCYbIUKqCNa+0wW3uW62Ch3qkJ763l4X7jl4A4vxvG2tU/cxByl9HuCIML3KvJY5tfT8FAKoLXHK5vRpLFvjCEHAoHzNefXVl16yMtsNEVNrdis8eIJIMK97387Vw4I326LGei0/RPrB8zBoS1va77+ckJ/NhF9Sc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159801898316889.63119878635666; Fri, 21 Aug 2020 07:09:43 -0700 (PDT) Received: from localhost ([::1]:36454 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97jh-0000Cv-Si for importer@patchew.org; Fri, 21 Aug 2020 10:09:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54338) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97in-0006qp-3x for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:45026) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97ik-0004VD-Hd for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:44 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-94-fsbBm6SbNNeRUDZQMUsUKQ-1; Fri, 21 Aug 2020 10:08:39 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 394C018A2252; Fri, 21 Aug 2020 14:08:38 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id C3A2D7AECA; Fri, 21 Aug 2020 14:08:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0uHSGE1QSAmI5bNa26JHPi6ph20kehqML5uG+ym6vxA=; b=S6SEspwUlmLNpCIdOfpUg1sTP2wEYk3GQETkLZB2WzRARM7SI3qnwlOjxUY+O4YPvs9Lkd DVJ7MvfYsTXsGCKLIcv5PiHXxkkvaY7PTKiQ/FPK+hGr/+SlY/fZKdZp+123ppTxrAhulL 6sN1NLyARL9XSdEGEnzV9jFtty/TKu8= X-MC-Unique: fsbBm6SbNNeRUDZQMUsUKQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 10/14] qcow2_format.py: support dumping metadata in JSON format Date: Fri, 21 Aug 2020 09:08:22 -0500 Message-Id: <20200821140826.194322-11-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 02:43:55 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich Implementation of dumping QCOW2 image metadata. The sample output: { "Header_extensions": [ { "name": "Feature table", "magic": 1745090647, "length": 192, "data_str": "" }, { "name": "Bitmaps", "magic": 595929205, "length": 24, "data": { "nb_bitmaps": 2, "reserved32": 0, "bitmap_directory_size": 64, "bitmap_directory_offset": 1048576, "bitmap_directory": [ { "name": "bitmap-1", "bitmap_table_offset": 589824, "bitmap_table_size": 1, "flags": 2, "type": 1, "granularity_bits": 15, "name_size": 8, "extra_data_size": 0, "bitmap_table": [ { "type": "serialized", "offset": 655360 }, ... Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-11-git-send-email-andrey.shinkevich@virtuozz= o.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/qcow2_format.py | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index 5a298b2f1357..8adc9959e10b 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -19,6 +19,15 @@ import struct import string +import json + + +class ComplexEncoder(json.JSONEncoder): + def default(self, obj): + if hasattr(obj, 'to_json'): + return obj.to_json() + else: + return json.JSONEncoder.default(self, obj) class Qcow2Field: @@ -110,6 +119,10 @@ class Qcow2Struct(metaclass=3DQcow2StructMeta): for i, field in enumerate(self.fields)) def dump(self, is_json=3DFalse): + if is_json: + print(json.dumps(self.to_json(), indent=3D4, cls=3DComplexEnco= der)) + return + for f in self.fields: value =3D self.__dict__[f[2]] if isinstance(f[1], str): @@ -445,6 +458,10 @@ class QcowHeader(Qcow2Struct): fd.write(buf) def dump_extensions(self, is_json=3DFalse): + if is_json: + print(json.dumps(self.extensions, indent=3D4, cls=3DComplexEnc= oder)) + return + for ex in self.extensions: print('Header extension:') ex.dump() --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019188; cv=none; d=zohomail.com; s=zohoarc; b=htqISpl+1GjW5A9x+4T07TygX8wHQ5VegVldaGOqt2HXi1AKmRJKExT3fD6/sDvljylYpzk/xSZVz7bwvbIho56Rr7Gg1gCWOmG2rQr/YK4QsAsxWUmXG3Nz7/8a3Cz9huoiI1eEjOk3c2B+Jt7nbqGqMkuIP0pHFyf4aNvvEmw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019188; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=O6byl7PYJtPbf4q/rlmARLiYl1eSQpVb4EhYzY2wJOU=; b=eVefzXuLJtdvlDPpm5QD/oDMVqrP9X1TSkSk+UGeI7NquFFQPAlr8bBVpVUXKVnBjFEsCPDZG75jxYfwoxcjgApeQEHbRGt5MuIb/oFhNS8P2HPAy+udx3WH5uk049spLni7HfDj9xt624llRziSF6zGvzFtMqPSyIEQ4y3ZqvU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15980191883011019.7429878994267; Fri, 21 Aug 2020 07:13:08 -0700 (PDT) Received: from localhost ([::1]:52424 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97n0-0006nq-TO for importer@patchew.org; Fri, 21 Aug 2020 10:13:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54572) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97j1-0007Qo-Kl for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:59 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:29935 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97iz-0004Yf-5L for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:59 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-566-m_OJ_mQwMSSjK5RV4ZVlTw-1; Fri, 21 Aug 2020 10:08:39 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D0834101963B; Fri, 21 Aug 2020 14:08:38 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 66E097AECA; Fri, 21 Aug 2020 14:08:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018936; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O6byl7PYJtPbf4q/rlmARLiYl1eSQpVb4EhYzY2wJOU=; b=DdM1LeoVnX6JDGwWRbXj9FdpWDEaF1/M75uwqK7A4eGDZEoAEtLBRibS/uD5dCtEyuICp2 NpFDXJJ2zpfMTpy4QEldj+KTFx0fK4ucNlDRaVRfVyRyxrfLGRopS51ZoVD7jaxMoub7cG pGVUNwPRZaaw8wHumNMcvqAPO5MHYT8= X-MC-Unique: m_OJ_mQwMSSjK5RV4ZVlTw-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 11/14] iotests: dump QCOW2 header in JSON in #303 Date: Fri, 21 Aug 2020 09:08:23 -0500 Message-Id: <20200821140826.194322-12-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/20 23:41:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Andrey Shinkevich , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich Extend the test case #303 by dumping QCOW2 image metadata in JSON format. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <1596742557-320265-12-git-send-email-andrey.shinkevich@virtuozz= o.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/303 | 3 ++ tests/qemu-iotests/303.out | 76 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+) diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303 index e9accdc7bc92..6c2177448348 100755 --- a/tests/qemu-iotests/303 +++ b/tests/qemu-iotests/303 @@ -58,3 +58,6 @@ add_bitmap(1, 0, 6, False) add_bitmap(2, 6, 8, True) dump =3D ['qcow2.py', disk, 'dump-header'] subprocess.run(dump) +# Dump the metadata in JSON format +dump.append('-j') +subprocess.run(dump) diff --git a/tests/qemu-iotests/303.out b/tests/qemu-iotests/303.out index 70828e05f11f..7fa1edef0d89 100644 --- a/tests/qemu-iotests/303.out +++ b/tests/qemu-iotests/303.out @@ -80,3 +80,79 @@ extra_data_size 0 Bitmap table type size offset 0 all-zeroes 0 0 +{ + "magic": 1363560955, + "version": 3, + "backing_file_offset": 0, + "backing_file_size": 0, + "cluster_bits": 16, + "size": 10485760, + "crypt_method": 0, + "l1_size": 1, + "l1_table_offset": 196608, + "refcount_table_offset": 65536, + "refcount_table_clusters": 1, + "nb_snapshots": 0, + "snapshot_offset": 0, + "incompatible_features": 0, + "compatible_features": 0, + "autoclear_features": 1, + "refcount_order": 4, + "header_length": 112 +} + +[ + { + "name": "Feature table", + "magic": 1745090647, + "length": 336, + "data_str": "" + }, + { + "name": "Bitmaps", + "magic": 595929205, + "length": 24, + "data": { + "nb_bitmaps": 2, + "reserved32": 0, + "bitmap_directory_size": 64, + "bitmap_directory_offset": 10289152, + "bitmap_directory": [ + { + "name": "bitmap-1", + "bitmap_table_offset": 10158080, + "bitmap_table_size": 1, + "flags": 2, + "type": 1, + "granularity_bits": 15, + "name_size": 8, + "extra_data_size": 0, + "bitmap_table": [ + { + "type": "serialized", + "offset": 10092544, + "reserved": 0 + } + ] + }, + { + "name": "bitmap-2", + "bitmap_table_offset": 10223616, + "bitmap_table_size": 1, + "flags": 0, + "type": 1, + "granularity_bits": 16, + "name_size": 8, + "extra_data_size": 0, + "bitmap_table": [ + { + "type": "all-zeroes", + "offset": 0, + "reserved": 0 + } + ] + } + ] + } + } +] --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019191; cv=none; d=zohomail.com; s=zohoarc; b=CEAInykTAZ9OgzuUGY4PZyDHfa+u4I8UzoyUpgvq2eegbNlah5nSQ0e3dPC6Y6FleqJ0FYuiJ/vu3OpzE4Id0M6apRXq/MubfKdNDZte3LZUhQLL9wa/uhx691rBrEvQPYDUOLevfSxLy4mLQBSLb6QmzKMNmgC89q4iCdNBaiw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019191; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mP0yQS1S+Jw+c7RTKmW1Fy96BiPBJqEi1uJiEFIMJkw=; b=GTf2ghAUyHVXssJkGtjvhfSADxROd9XVb/Z0G71VJZlf0HlVtowSH5AEUoD0OZ/+cz/q0ou689fp+MYrFHkZZOCofgGShUGvsirG7s1FEZvOG4JFQvGJkyEpF5RduEnPl6gKelHzdknb/Nw49POm4uB3dzpnjR8HoZrB/YfzHH8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019191327572.5335500787733; Fri, 21 Aug 2020 07:13:11 -0700 (PDT) Received: from localhost ([::1]:52658 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97n4-0006tk-07 for importer@patchew.org; Fri, 21 Aug 2020 10:13:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54418) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97iu-00076f-Pt for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:52 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:49192 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97iq-0004Vr-WE for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:08:52 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-255-_5XPMJ4wPS2rfQ7W4_NbHg-1; Fri, 21 Aug 2020 10:08:44 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C68EC8797DD; Fri, 21 Aug 2020 14:08:42 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1742774E2A; Fri, 21 Aug 2020 14:08:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018928; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mP0yQS1S+Jw+c7RTKmW1Fy96BiPBJqEi1uJiEFIMJkw=; b=WdJaTzrXZsKZOrcxQkxHHMkvcfgmj++rvNFLX2hc4+KH7vuVWWHPTuOe6d9oTx4Ue4f1Cq MMFplpvxI92125aN5qRZ1jzyN+Js+F2R3Ac1PCugeHz0l1KinuZmg56uCIHhla4Ur5tw1/ 4PzyRE/gaoSzgHTOpyGLJ3q3odWD63Y= X-MC-Unique: _5XPMJ4wPS2rfQ7W4_NbHg-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 12/14] migration: Add block-bitmap-mapping parameter Date: Fri, 21 Aug 2020 09:08:24 -0500 Message-Id: <20200821140826.194322-13-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.003 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=eblake@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 04:55:47 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Vladimir Sementsov-Ogievskiy , "open list:Dirty Bitmaps" , Juan Quintela , Markus Armbruster , "Dr. David Alan Gilbert" , Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Max Reitz This migration parameter allows mapping block node names and bitmap names to aliases for the purpose of block dirty bitmap migration. This way, management tools can use different node and bitmap names on the source and destination and pass the mapping of how bitmaps are to be transferred to qemu (on the source, the destination, or even both with arbitrary aliases in the migration stream). While touching this code, fix a bug where bitmap names longer than 255 bytes would fail an assertion in qemu_put_counted_string(). Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz Message-Id: <20200820150725.68687-2-mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Eric Blake --- qapi/migration.json | 104 ++++++++- migration/migration.h | 3 + migration/block-dirty-bitmap.c | 410 ++++++++++++++++++++++++++++----- migration/migration.c | 30 +++ monitor/hmp-cmds.c | 30 +++ 5 files changed, 521 insertions(+), 56 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index ea53b23dca90..5f6b06172cab 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -508,6 +508,44 @@ 'data': [ 'none', 'zlib', { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] } +## +# @BitmapMigrationBitmapAlias: +# +# @name: The name of the bitmap. +# +# @alias: An alias name for migration (for example the bitmap name on +# the opposite site). +# +# Since: 5.2 +## +{ 'struct': 'BitmapMigrationBitmapAlias', + 'data': { + 'name': 'str', + 'alias': 'str' + } } + +## +# @BitmapMigrationNodeAlias: +# +# Maps a block node name and the bitmaps it has to aliases for dirty +# bitmap migration. +# +# @node-name: A block node name. +# +# @alias: An alias block node name for migration (for example the +# node name on the opposite site). +# +# @bitmaps: Mappings for the bitmaps on this node. +# +# Since: 5.2 +## +{ 'struct': 'BitmapMigrationNodeAlias', + 'data': { + 'node-name': 'str', + 'alias': 'str', + 'bitmaps': [ 'BitmapMigrationBitmapAlias' ] + } } + ## # @MigrationParameter: # @@ -642,6 +680,25 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @block-bitmap-mapping: Maps block nodes and bitmaps on them to +# aliases for the purpose of dirty bitmap migration. Such +# aliases may for example be the corresponding names on the +# opposite site. +# The mapping must be one-to-one, but not necessarily +# complete: On the source, unmapped bitmaps and all bitmaps +# on unmapped nodes will be ignored. On the destination, +# encountering an unmapped alias in the incoming migration +# stream will result in a report, and all further bitmap +# migration data will then be discarded. +# Note that the destination does not know about bitmaps it +# does not receive, so there is no limitation or requirement +# regarding the number of bitmaps received, or how they are +# named, or on which nodes they are placed. +# By default (when this parameter has never been set), bitmap +# names are mapped to themselves. Nodes are mapped to their +# block device name if there is one, and to their node name +# otherwise. (Since 5.2) +# # Since: 2.4 ## { 'enum': 'MigrationParameter', @@ -656,7 +713,8 @@ 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', - 'multifd-zlib-level' ,'multifd-zstd-level' ] } + 'multifd-zlib-level' ,'multifd-zstd-level', + 'block-bitmap-mapping' ] } ## # @MigrateSetParameters: @@ -782,6 +840,25 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @block-bitmap-mapping: Maps block nodes and bitmaps on them to +# aliases for the purpose of dirty bitmap migration. Such +# aliases may for example be the corresponding names on the +# opposite site. +# The mapping must be one-to-one, but not necessarily +# complete: On the source, unmapped bitmaps and all bitmaps +# on unmapped nodes will be ignored. On the destination, +# encountering an unmapped alias in the incoming migration +# stream will result in a report, and all further bitmap +# migration data will then be discarded. +# Note that the destination does not know about bitmaps it +# does not receive, so there is no limitation or requirement +# regarding the number of bitmaps received, or how they are +# named, or on which nodes they are placed. +# By default (when this parameter has never been set), bitmap +# names are mapped to themselves. Nodes are mapped to their +# block device name if there is one, and to their node name +# otherwise. (Since 5.2) +# # Since: 2.4 ## # TODO either fuse back into MigrationParameters, or make @@ -812,7 +889,8 @@ '*max-cpu-throttle': 'int', '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'int', - '*multifd-zstd-level': 'int' } } + '*multifd-zstd-level': 'int', + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } ## # @migrate-set-parameters: @@ -958,6 +1036,25 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @block-bitmap-mapping: Maps block nodes and bitmaps on them to +# aliases for the purpose of dirty bitmap migration. Such +# aliases may for example be the corresponding names on the +# opposite site. +# The mapping must be one-to-one, but not necessarily +# complete: On the source, unmapped bitmaps and all bitmaps +# on unmapped nodes will be ignored. On the destination, +# encountering an unmapped alias in the incoming migration +# stream will result in a report, and all further bitmap +# migration data will then be discarded. +# Note that the destination does not know about bitmaps it +# does not receive, so there is no limitation or requirement +# regarding the number of bitmaps received, or how they are +# named, or on which nodes they are placed. +# By default (when this parameter has never been set), bitmap +# names are mapped to themselves. Nodes are mapped to their +# block device name if there is one, and to their node name +# otherwise. (Since 5.2) +# # Since: 2.4 ## { 'struct': 'MigrationParameters', @@ -986,7 +1083,8 @@ '*max-cpu-throttle': 'uint8', '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', - '*multifd-zstd-level': 'uint8' } } + '*multifd-zstd-level': 'uint8', + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } ## # @query-migrate-parameters: diff --git a/migration/migration.h b/migration/migration.h index 6c6a931d0dc2..2ed55c4aefc3 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -337,6 +337,9 @@ void migrate_send_rp_resume_ack(MigrationIncomingState = *mis, uint32_t value); void dirty_bitmap_mig_before_vm_start(void); void dirty_bitmap_mig_cancel_outgoing(void); void dirty_bitmap_mig_cancel_incoming(void); +bool check_dirty_bitmap_mig_alias_map(const BitmapMigrationNodeAliasList *= bbm, + Error **errp); + void migrate_add_address(SocketAddress *address); int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque); diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 784330ebe130..549e14daba4f 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -29,10 +29,10 @@ * * # Header (shared for different chunk types) * 1, 2 or 4 bytes: flags (see qemu_{put,put}_flags) - * [ 1 byte: node name size ] \ flags & DEVICE_NAME - * [ n bytes: node name ] / - * [ 1 byte: bitmap name size ] \ flags & BITMAP_NAME - * [ n bytes: bitmap name ] / + * [ 1 byte: node alias size ] \ flags & DEVICE_NAME + * [ n bytes: node alias ] / + * [ 1 byte: bitmap alias size ] \ flags & BITMAP_NAME + * [ n bytes: bitmap alias ] / * * # Start of bitmap migration (flags & START) * header @@ -72,7 +72,9 @@ #include "migration/register.h" #include "qemu/hbitmap.h" #include "qemu/cutils.h" +#include "qemu/id.h" #include "qapi/error.h" +#include "qapi/qapi-commands-migration.h" #include "trace.h" #define CHUNK_SIZE (1 << 10) @@ -104,7 +106,8 @@ typedef struct SaveBitmapState { /* Written during setup phase. */ BlockDriverState *bs; - const char *node_name; + char *node_alias; + char *bitmap_alias; BdrvDirtyBitmap *bitmap; uint64_t total_sectors; uint64_t sectors_per_chunk; @@ -138,8 +141,9 @@ typedef struct LoadBitmapState { /* State of the dirty bitmap migration (DBM) during load process */ typedef struct DBMLoadState { uint32_t flags; - char node_name[256]; - char bitmap_name[256]; + char node_alias[256]; + char bitmap_alias[256]; + char bitmap_name[BDRV_BITMAP_MAX_NAME_SIZE + 1]; BlockDriverState *bs; BdrvDirtyBitmap *bitmap; @@ -165,6 +169,188 @@ typedef struct DBMState { static DBMState dbm_state; +/* For hash tables that map node/bitmap names to aliases */ +typedef struct AliasMapInnerNode { + char *string; + GHashTable *subtree; +} AliasMapInnerNode; + +static void free_alias_map_inner_node(void *amin_ptr) +{ + AliasMapInnerNode *amin =3D amin_ptr; + + g_free(amin->string); + g_hash_table_unref(amin->subtree); + g_free(amin); +} + +/** + * Construct an alias map based on the given QMP structure. + * + * (Note that we cannot store such maps in the MigrationParameters + * object, because that struct is defined by the QAPI schema, which + * makes it basically impossible to have dicts with arbitrary keys. + * Therefore, we instead have to construct these maps when migration + * starts.) + * + * @bbm is the block_bitmap_mapping from the migration parameters. + * + * If @name_to_alias is true, the returned hash table will map node + * and bitmap names to their respective aliases (for outgoing + * migration). + * + * If @name_to_alias is false, the returned hash table will map node + * and bitmap aliases to their respective names (for incoming + * migration). + * + * The hash table maps node names/aliases to AliasMapInnerNode + * objects, whose .string is the respective node alias/name, and whose + * .subtree table maps bitmap names/aliases to the respective bitmap + * alias/name. + */ +static GHashTable *construct_alias_map(const BitmapMigrationNodeAliasList = *bbm, + bool name_to_alias, + Error **errp) +{ + GHashTable *alias_map; + size_t max_node_name_len =3D sizeof_field(BlockDriverState, node_name)= - 1; + + alias_map =3D g_hash_table_new_full(g_str_hash, g_str_equal, + g_free, free_alias_map_inner_node); + + for (; bbm; bbm =3D bbm->next) { + const BitmapMigrationNodeAlias *bmna =3D bbm->value; + const BitmapMigrationBitmapAliasList *bmbal; + AliasMapInnerNode *amin; + GHashTable *bitmaps_map; + const char *node_map_from, *node_map_to; + + if (!id_wellformed(bmna->alias)) { + error_setg(errp, "The node alias '%s' is not well-formed", + bmna->alias); + goto fail; + } + + if (strlen(bmna->alias) > UINT8_MAX) { + error_setg(errp, "The node alias '%s' is longer than %u bytes", + bmna->alias, UINT8_MAX); + goto fail; + } + + if (strlen(bmna->node_name) > max_node_name_len) { + error_setg(errp, "The node name '%s' is longer than %zu bytes", + bmna->node_name, max_node_name_len); + goto fail; + } + + if (name_to_alias) { + if (g_hash_table_contains(alias_map, bmna->node_name)) { + error_setg(errp, "The node name '%s' is mapped twice", + bmna->node_name); + goto fail; + } + + node_map_from =3D bmna->node_name; + node_map_to =3D bmna->alias; + } else { + if (g_hash_table_contains(alias_map, bmna->alias)) { + error_setg(errp, "The node alias '%s' is used twice", + bmna->alias); + goto fail; + } + + node_map_from =3D bmna->alias; + node_map_to =3D bmna->node_name; + } + + bitmaps_map =3D g_hash_table_new_full(g_str_hash, g_str_equal, + g_free, g_free); + + amin =3D g_new(AliasMapInnerNode, 1); + *amin =3D (AliasMapInnerNode){ + .string =3D g_strdup(node_map_to), + .subtree =3D bitmaps_map, + }; + + g_hash_table_insert(alias_map, g_strdup(node_map_from), amin); + + for (bmbal =3D bmna->bitmaps; bmbal; bmbal =3D bmbal->next) { + const BitmapMigrationBitmapAlias *bmba =3D bmbal->value; + const char *bmap_map_from, *bmap_map_to; + + if (strlen(bmba->alias) > UINT8_MAX) { + error_setg(errp, + "The bitmap alias '%s' is longer than %u bytes", + bmba->alias, UINT8_MAX); + goto fail; + } + + if (strlen(bmba->name) > BDRV_BITMAP_MAX_NAME_SIZE) { + error_setg(errp, "The bitmap name '%s' is longer than %d b= ytes", + bmba->name, BDRV_BITMAP_MAX_NAME_SIZE); + goto fail; + } + + if (name_to_alias) { + bmap_map_from =3D bmba->name; + bmap_map_to =3D bmba->alias; + + if (g_hash_table_contains(bitmaps_map, bmba->name)) { + error_setg(errp, "The bitmap '%s'/'%s' is mapped twice= ", + bmna->node_name, bmba->name); + goto fail; + } + } else { + bmap_map_from =3D bmba->alias; + bmap_map_to =3D bmba->name; + + if (g_hash_table_contains(bitmaps_map, bmba->alias)) { + error_setg(errp, "The bitmap alias '%s'/'%s' is used t= wice", + bmna->alias, bmba->alias); + goto fail; + } + } + + g_hash_table_insert(bitmaps_map, + g_strdup(bmap_map_from), g_strdup(bmap_map= _to)); + } + } + + return alias_map; + +fail: + g_hash_table_destroy(alias_map); + return NULL; +} + +/** + * Run construct_alias_map() in both directions to check whether @bbm + * is valid. + * (This function is to be used by migration/migration.c to validate + * the user-specified block-bitmap-mapping migration parameter.) + * + * Returns true if and only if the mapping is valid. + */ +bool check_dirty_bitmap_mig_alias_map(const BitmapMigrationNodeAliasList *= bbm, + Error **errp) +{ + GHashTable *alias_map; + + alias_map =3D construct_alias_map(bbm, true, errp); + if (!alias_map) { + return false; + } + g_hash_table_destroy(alias_map); + + alias_map =3D construct_alias_map(bbm, false, errp); + if (!alias_map) { + return false; + } + g_hash_table_destroy(alias_map); + + return true; +} + static uint32_t qemu_get_bitmap_flags(QEMUFile *f) { uint8_t flags =3D qemu_get_byte(f); @@ -207,11 +393,11 @@ static void send_bitmap_header(QEMUFile *f, DBMSaveSt= ate *s, qemu_put_bitmap_flags(f, flags); if (flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) { - qemu_put_counted_string(f, dbms->node_name); + qemu_put_counted_string(f, dbms->node_alias); } if (flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) { - qemu_put_counted_string(f, bdrv_dirty_bitmap_name(bitmap)); + qemu_put_counted_string(f, dbms->bitmap_alias); } } @@ -282,18 +468,25 @@ static void dirty_bitmap_do_save_cleanup(DBMSaveState= *s) QSIMPLEQ_REMOVE_HEAD(&s->dbms_list, entry); bdrv_dirty_bitmap_set_busy(dbms->bitmap, false); bdrv_unref(dbms->bs); + g_free(dbms->node_alias); + g_free(dbms->bitmap_alias); g_free(dbms); } } /* Called with iothread lock taken. */ static int add_bitmaps_to_list(DBMSaveState *s, BlockDriverState *bs, - const char *bs_name) + const char *bs_name, GHashTable *alias_map) { BdrvDirtyBitmap *bitmap; SaveBitmapState *dbms; + GHashTable *bitmap_aliases; + const char *node_alias, *bitmap_name, *bitmap_alias; Error *local_err =3D NULL; + /* When an alias map is given, @bs_name must be @bs's node name */ + assert(!alias_map || !strcmp(bs_name, bdrv_get_node_name(bs))); + FOR_EACH_DIRTY_BITMAP(bs, bitmap) { if (bdrv_dirty_bitmap_name(bitmap)) { break; @@ -303,21 +496,39 @@ static int add_bitmaps_to_list(DBMSaveState *s, Block= DriverState *bs, return 0; } + bitmap_name =3D bdrv_dirty_bitmap_name(bitmap); + if (!bs_name || strcmp(bs_name, "") =3D=3D 0) { error_report("Bitmap '%s' in unnamed node can't be migrated", - bdrv_dirty_bitmap_name(bitmap)); + bitmap_name); return -1; } - if (bs_name[0] =3D=3D '#') { + if (alias_map) { + const AliasMapInnerNode *amin =3D g_hash_table_lookup(alias_map, b= s_name); + + if (!amin) { + /* Skip bitmaps on nodes with no alias */ + return 0; + } + + node_alias =3D amin->string; + bitmap_aliases =3D amin->subtree; + } else { + node_alias =3D bs_name; + bitmap_aliases =3D NULL; + } + + if (node_alias[0] =3D=3D '#') { error_report("Bitmap '%s' in a node with auto-generated " "name '%s' can't be migrated", - bdrv_dirty_bitmap_name(bitmap), bs_name); + bitmap_name, node_alias); return -1; } FOR_EACH_DIRTY_BITMAP(bs, bitmap) { - if (!bdrv_dirty_bitmap_name(bitmap)) { + bitmap_name =3D bdrv_dirty_bitmap_name(bitmap); + if (!bitmap_name) { continue; } @@ -326,12 +537,29 @@ static int add_bitmaps_to_list(DBMSaveState *s, Block= DriverState *bs, return -1; } + if (bitmap_aliases) { + bitmap_alias =3D g_hash_table_lookup(bitmap_aliases, bitmap_na= me); + if (!bitmap_alias) { + /* Skip bitmaps with no alias */ + continue; + } + } else { + if (strlen(bitmap_name) > UINT8_MAX) { + error_report("Cannot migrate bitmap '%s' on node '%s': " + "Name is longer than %u bytes", + bitmap_name, bs_name, UINT8_MAX); + return -1; + } + bitmap_alias =3D bitmap_name; + } + bdrv_ref(bs); bdrv_dirty_bitmap_set_busy(bitmap, true); dbms =3D g_new0(SaveBitmapState, 1); dbms->bs =3D bs; - dbms->node_name =3D bs_name; + dbms->node_alias =3D g_strdup(node_alias); + dbms->bitmap_alias =3D g_strdup(bitmap_alias); dbms->bitmap =3D bitmap; dbms->total_sectors =3D bdrv_nb_sectors(bs); dbms->sectors_per_chunk =3D CHUNK_SIZE * 8 * @@ -356,43 +584,52 @@ static int init_dirty_bitmap_migration(DBMSaveState *= s) SaveBitmapState *dbms; GHashTable *handled_by_blk =3D g_hash_table_new(NULL, NULL); BlockBackend *blk; + const MigrationParameters *mig_params =3D &migrate_get_current()->para= meters; + GHashTable *alias_map =3D NULL; + + if (mig_params->has_block_bitmap_mapping) { + alias_map =3D construct_alias_map(mig_params->block_bitmap_mapping= , true, + &error_abort); + } s->bulk_completed =3D false; s->prev_bs =3D NULL; s->prev_bitmap =3D NULL; s->no_bitmaps =3D false; - /* - * Use blockdevice name for direct (or filtered) children of named blo= ck - * backends. - */ - for (blk =3D blk_next(NULL); blk; blk =3D blk_next(blk)) { - const char *name =3D blk_name(blk); + if (!alias_map) { + /* + * Use blockdevice name for direct (or filtered) children of named= block + * backends. + */ + for (blk =3D blk_next(NULL); blk; blk =3D blk_next(blk)) { + const char *name =3D blk_name(blk); - if (!name || strcmp(name, "") =3D=3D 0) { - continue; - } + if (!name || strcmp(name, "") =3D=3D 0) { + continue; + } - bs =3D blk_bs(blk); + bs =3D blk_bs(blk); - /* Skip filters without bitmaps */ - while (bs && bs->drv && bs->drv->is_filter && - !bdrv_has_named_bitmaps(bs)) - { - if (bs->backing) { - bs =3D bs->backing->bs; - } else if (bs->file) { - bs =3D bs->file->bs; - } else { - bs =3D NULL; + /* Skip filters without bitmaps */ + while (bs && bs->drv && bs->drv->is_filter && + !bdrv_has_named_bitmaps(bs)) + { + if (bs->backing) { + bs =3D bs->backing->bs; + } else if (bs->file) { + bs =3D bs->file->bs; + } else { + bs =3D NULL; + } } - } - if (bs && bs->drv && !bs->drv->is_filter) { - if (add_bitmaps_to_list(s, bs, name)) { - goto fail; + if (bs && bs->drv && !bs->drv->is_filter) { + if (add_bitmaps_to_list(s, bs, name, NULL)) { + goto fail; + } + g_hash_table_add(handled_by_blk, bs); } - g_hash_table_add(handled_by_blk, bs); } } @@ -401,7 +638,7 @@ static int init_dirty_bitmap_migration(DBMSaveState *s) continue; } - if (add_bitmaps_to_list(s, bs, bdrv_get_node_name(bs))) { + if (add_bitmaps_to_list(s, bs, bdrv_get_node_name(bs), alias_map))= { goto fail; } } @@ -416,11 +653,17 @@ static int init_dirty_bitmap_migration(DBMSaveState *= s) } g_hash_table_destroy(handled_by_blk); + if (alias_map) { + g_hash_table_destroy(alias_map); + } return 0; fail: g_hash_table_destroy(handled_by_blk); + if (alias_map) { + g_hash_table_destroy(alias_map); + } dirty_bitmap_do_save_cleanup(s); return -1; @@ -770,8 +1013,10 @@ static int dirty_bitmap_load_bits(QEMUFile *f, DBMLoa= dState *s) return 0; } -static int dirty_bitmap_load_header(QEMUFile *f, DBMLoadState *s) +static int dirty_bitmap_load_header(QEMUFile *f, DBMLoadState *s, + GHashTable *alias_map) { + GHashTable *bitmap_alias_map =3D NULL; Error *local_err =3D NULL; bool nothing; s->flags =3D qemu_get_bitmap_flags(f); @@ -780,28 +1025,75 @@ static int dirty_bitmap_load_header(QEMUFile *f, DBM= LoadState *s) nothing =3D s->flags =3D=3D (s->flags & DIRTY_BITMAP_MIG_FLAG_EOS); if (s->flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) { - if (!qemu_get_counted_string(f, s->node_name)) { - error_report("Unable to read node name string"); + if (!qemu_get_counted_string(f, s->node_alias)) { + error_report("Unable to read node alias string"); return -EINVAL; } + if (!s->cancelled) { - s->bs =3D bdrv_lookup_bs(s->node_name, s->node_name, &local_er= r); + if (alias_map) { + const AliasMapInnerNode *amin; + + amin =3D g_hash_table_lookup(alias_map, s->node_alias); + if (!amin) { + error_setg(&local_err, "Error: Unknown node alias '%s'= ", + s->node_alias); + s->bs =3D NULL; + } else { + bitmap_alias_map =3D amin->subtree; + s->bs =3D bdrv_lookup_bs(NULL, amin->string, &local_er= r); + } + } else { + s->bs =3D bdrv_lookup_bs(s->node_alias, s->node_alias, + &local_err); + } if (!s->bs) { error_report_err(local_err); cancel_incoming_locked(s); } } - } else if (!s->bs && !nothing && !s->cancelled) { + } else if (s->bs) { + if (alias_map) { + const AliasMapInnerNode *amin; + + /* Must be present in the map, or s->bs would not be set */ + amin =3D g_hash_table_lookup(alias_map, s->node_alias); + assert(amin !=3D NULL); + + bitmap_alias_map =3D amin->subtree; + } + } else if (!nothing && !s->cancelled) { error_report("Error: block device name is not set"); cancel_incoming_locked(s); } + assert(nothing || s->cancelled || !!alias_map =3D=3D !!bitmap_alias_ma= p); + if (s->flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) { - if (!qemu_get_counted_string(f, s->bitmap_name)) { - error_report("Unable to read bitmap name string"); + const char *bitmap_name; + + if (!qemu_get_counted_string(f, s->bitmap_alias)) { + error_report("Unable to read bitmap alias string"); return -EINVAL; } + if (!s->cancelled) { + if (bitmap_alias_map) { + bitmap_name =3D g_hash_table_lookup(bitmap_alias_map, + s->bitmap_alias); + if (!bitmap_name) { + error_report("Error: Unknown bitmap alias '%s' on node= " + "'%s' (alias '%s')", s->bitmap_alias, + s->bs->node_name, s->node_alias); + cancel_incoming_locked(s); + } + } else { + bitmap_name =3D s->bitmap_alias; + } + } + + if (!s->cancelled) { + g_strlcpy(s->bitmap_name, bitmap_name, sizeof(s->bitmap_name)); s->bitmap =3D bdrv_find_dirty_bitmap(s->bs, s->bitmap_name); /* @@ -811,7 +1103,7 @@ static int dirty_bitmap_load_header(QEMUFile *f, DBMLo= adState *s) if (!s->bitmap && !(s->flags & DIRTY_BITMAP_MIG_FLAG_START)) { error_report("Error: unknown dirty bitmap " "'%s' for block device '%s'", - s->bitmap_name, s->node_name); + s->bitmap_name, s->bs->node_name); cancel_incoming_locked(s); } } @@ -835,6 +1127,8 @@ static int dirty_bitmap_load_header(QEMUFile *f, DBMLo= adState *s) */ static int dirty_bitmap_load(QEMUFile *f, void *opaque, int version_id) { + GHashTable *alias_map =3D NULL; + const MigrationParameters *mig_params =3D &migrate_get_current()->para= meters; DBMLoadState *s =3D &((DBMState *)opaque)->load; int ret =3D 0; @@ -846,13 +1140,18 @@ static int dirty_bitmap_load(QEMUFile *f, void *opaq= ue, int version_id) return -EINVAL; } + if (mig_params->has_block_bitmap_mapping) { + alias_map =3D construct_alias_map(mig_params->block_bitmap_mapping, + false, &error_abort); + } + do { QEMU_LOCK_GUARD(&s->lock); - ret =3D dirty_bitmap_load_header(f, s); + ret =3D dirty_bitmap_load_header(f, s, alias_map); if (ret < 0) { cancel_incoming_locked(s); - return ret; + goto fail; } if (s->flags & DIRTY_BITMAP_MIG_FLAG_START) { @@ -869,12 +1168,17 @@ static int dirty_bitmap_load(QEMUFile *f, void *opaq= ue, int version_id) if (ret) { cancel_incoming_locked(s); - return ret; + goto fail; } } while (!(s->flags & DIRTY_BITMAP_MIG_FLAG_EOS)); trace_dirty_bitmap_load_success(); - return 0; + ret =3D 0; +fail: + if (alias_map) { + g_hash_table_destroy(alias_map); + } + return ret; } static int dirty_bitmap_save_setup(QEMUFile *f, void *opaque) diff --git a/migration/migration.c b/migration/migration.c index 8fe36339dbe8..dbd4afa1e895 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -36,6 +36,7 @@ #include "block/block.h" #include "qapi/error.h" #include "qapi/clone-visitor.h" +#include "qapi/qapi-visit-migration.h" #include "qapi/qapi-visit-sockets.h" #include "qapi/qapi-commands-migration.h" #include "qapi/qapi-events-migration.h" @@ -843,6 +844,13 @@ MigrationParameters *qmp_query_migrate_parameters(Erro= r **errp) params->has_announce_step =3D true; params->announce_step =3D s->parameters.announce_step; + if (s->parameters.has_block_bitmap_mapping) { + params->has_block_bitmap_mapping =3D true; + params->block_bitmap_mapping =3D + QAPI_CLONE(BitmapMigrationNodeAliasList, + s->parameters.block_bitmap_mapping); + } + return params; } @@ -1308,6 +1316,13 @@ static bool migrate_params_check(MigrationParameters= *params, Error **errp) "is invalid, it must be in the range of 1 to 10000 ms"); return false; } + + if (params->has_block_bitmap_mapping && + !check_dirty_bitmap_mig_alias_map(params->block_bitmap_mapping, er= rp)) { + error_prepend(errp, "Invalid mapping given for block-bitmap-mappin= g: "); + return false; + } + return true; } @@ -1402,6 +1417,11 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, if (params->has_announce_step) { dest->announce_step =3D params->announce_step; } + + if (params->has_block_bitmap_mapping) { + dest->has_block_bitmap_mapping =3D true; + dest->block_bitmap_mapping =3D params->block_bitmap_mapping; + } } static void migrate_params_apply(MigrateSetParameters *params, Error **err= p) @@ -1514,6 +1534,16 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) if (params->has_announce_step) { s->parameters.announce_step =3D params->announce_step; } + + if (params->has_block_bitmap_mapping) { + qapi_free_BitmapMigrationNodeAliasList( + s->parameters.block_bitmap_mapping); + + s->parameters.has_block_bitmap_mapping =3D true; + s->parameters.block_bitmap_mapping =3D + QAPI_CLONE(BitmapMigrationNodeAliasList, + params->block_bitmap_mapping); + } } void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index ae4b6a4246b5..7711726fd222 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -469,6 +469,32 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) monitor_printf(mon, "%s: '%s'\n", MigrationParameter_str(MIGRATION_PARAMETER_TLS_AUTHZ), params->tls_authz); + + if (params->has_block_bitmap_mapping) { + const BitmapMigrationNodeAliasList *bmnal; + + monitor_printf(mon, "%s:\n", + MigrationParameter_str( + MIGRATION_PARAMETER_BLOCK_BITMAP_MAPPING)); + + for (bmnal =3D params->block_bitmap_mapping; + bmnal; + bmnal =3D bmnal->next) + { + const BitmapMigrationNodeAlias *bmna =3D bmnal->value; + const BitmapMigrationBitmapAliasList *bmbal; + + monitor_printf(mon, " '%s' -> '%s'\n", + bmna->node_name, bmna->alias); + + for (bmbal =3D bmna->bitmaps; bmbal; bmbal =3D bmbal->next= ) { + const BitmapMigrationBitmapAlias *bmba =3D bmbal->valu= e; + + monitor_printf(mon, " '%s' -> '%s'\n", + bmba->name, bmba->alias); + } + } + } } qapi_free_MigrationParameters(params); @@ -1384,6 +1410,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) p->has_announce_step =3D true; visit_type_size(v, param, &p->announce_step, &err); break; + case MIGRATION_PARAMETER_BLOCK_BITMAP_MAPPING: + error_setg(&err, "The block-bitmap-mapping parameter can only be s= et " + "through QMP"); + break; default: assert(0); } --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019569; cv=none; d=zohomail.com; s=zohoarc; b=TDBcfyUoGwlpz9QnEo5TOAgaY2HxgZZ/IwtiVZBriV0siV9NXsmztm4MrQpRLae34PYlu9sHxUGobZY5NZMNMygYbZVXKcB1YwU3AZ5M13e3cHbOlsK4G49refVZ54Si9n6N7M7lkhkav4kIsQ3QdDlekv5j1oFB5EGyowhDuQs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019569; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=92W2/VNpKH3NOOOD4Q+AtHcwoQhnZFYJS8bzwG1EkWE=; b=LhUpzDjhmtEiSN6CVqU6ilLJc/FJPbnBJj3rDqQH0gP/EFQKvqrGW5Kc+BBtBnlERg1XQPCwSo8l626uTHLBgAxvPBgHRpCA/lG5mQpYiwA8mQvFfY3xg/LAMCXEe0GBcV0ZfPzhqBe2BJ4d18nFHBMCwXCoAjvP/hQymrucJu0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019569523515.6081340553667; Fri, 21 Aug 2020 07:19:29 -0700 (PDT) Received: from localhost ([::1]:58440 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97tA-0003sB-5c for importer@patchew.org; Fri, 21 Aug 2020 10:19:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54634) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97j5-0007c8-O0 for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:09:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:55980) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97j3-0004a6-QT for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:09:03 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-356-wQt3JxkqOf2kEqwoqCnhuQ-1; Fri, 21 Aug 2020 10:08:44 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5A5A581CAFA; Fri, 21 Aug 2020 14:08:43 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id EEC14614F1; Fri, 21 Aug 2020 14:08:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018941; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=92W2/VNpKH3NOOOD4Q+AtHcwoQhnZFYJS8bzwG1EkWE=; b=O5t8xel0J7dxmjnrCK7BpxY5QChOXYzNIIZIOYKw0CEYD0z2vOlKaDKSv8BWfpg1VnwiOV 435rFdNDydGAVvIQ/9dqxvqzglYIBo/gEiCifa+Bo8uhAa1CtMPTK/Js1rwgIXbgUVV0qt j2HEV+3f9ybJh66tFbxPr6SgPioOxYY= X-MC-Unique: wQt3JxkqOf2kEqwoqCnhuQ-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 13/14] iotests.py: Let wait_migration() return on failure Date: Fri, 21 Aug 2020 09:08:25 -0500 Message-Id: <20200821140826.194322-14-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 02:43:55 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Max Reitz Let wait_migration() return on failure (with the return value indicating whether the migration was completed or has failed), so we can use it for migrations that are expected to fail, too. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <20200820150725.68687-3-mreitz@redhat.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/iotests.py | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 717b5b652c45..e197c73ca501 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -729,16 +729,22 @@ class VM(qtest.QEMUQtestMachine): } ])) - def wait_migration(self, expect_runstate): + def wait_migration(self, expect_runstate: Optional[str]) -> bool: while True: event =3D self.event_wait('MIGRATION') log(event, filters=3D[filter_qmp_event]) - if event['data']['status'] =3D=3D 'completed': + if event['data']['status'] in ('completed', 'failed'): break - # The event may occur in finish-migrate, so wait for the expected - # post-migration runstate - while self.qmp('query-status')['return']['status'] !=3D expect_run= state: - pass + + if event['data']['status'] =3D=3D 'completed': + # The event may occur in finish-migrate, so wait for the expec= ted + # post-migration runstate + runstate =3D None + while runstate !=3D expect_runstate: + runstate =3D self.qmp('query-status')['return']['status'] + return True + else: + return False def node_info(self, node_name): nodes =3D self.qmp('query-named-block-nodes') --=20 2.28.0 From nobody Fri May 17 03:54:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1598019288; cv=none; d=zohomail.com; s=zohoarc; b=ZtrxqgilfKJKbc8x++V1rUwfSdvSODPJokmjZGbbtK3KwIFlEtn7KeqIG5XB/sBBpSFke/3YSR6ScoLWJvPNPlfrdvwXntCES9q/QJf/16iFjXyW7PiJ/Aezh+Kwy/wVpiSn6DKdysvnh7dzLAJnbuDIDlINgpnS9nw4fLO8yGU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598019288; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=utOUJzpcufX05SSRrX/MVtEK87W4K1Te9K0n/0E9ctc=; b=OD3ovtkzw06xy+cGwXFnYYW/VOLfEI/+EcPngs2XzJvFZsOYPOX5yUKoqVDVTEkqB3Ys9xp0TbslHApuJSxEuQITOgoKCzOuNmJaspFxi6WTTeoyKM8WERvTatO//1et1j3P82X9ts0NHnvvo/04BU6PYxJeTv4vVJW+gvgsDfo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598019288098792.2303866077824; Fri, 21 Aug 2020 07:14:48 -0700 (PDT) Received: from localhost ([::1]:33302 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k97oc-00023F-Q4 for importer@patchew.org; Fri, 21 Aug 2020 10:14:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54682) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k97j8-0007k9-Lu for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:09:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50399) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k97j4-0004aM-Ek for qemu-devel@nongnu.org; Fri, 21 Aug 2020 10:09:06 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-365-PG26rx8JO56qPydtWUH5YA-1; Fri, 21 Aug 2020 10:08:44 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F3786101962F; Fri, 21 Aug 2020 14:08:43 +0000 (UTC) Received: from blue.redhat.com (ovpn-113-220.phx2.redhat.com [10.3.113.220]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C4087AEC5; Fri, 21 Aug 2020 14:08:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1598018941; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=utOUJzpcufX05SSRrX/MVtEK87W4K1Te9K0n/0E9ctc=; b=bb9OzKl+vKTz/e0C+jvbHae8zruFz5MeP/rW1neKNRISav+5HnzMD18d5FaZcBnlG1ljMD VCLTckdug2Lp8XsRB5j86ao0mpN2XU2IA0gkDM+/YsOk3FbrMsrdSUbZOgMWtTPZW2BTfS xzsT7UmPaidWlwiYD0U3bWkqQ0EqliQ= X-MC-Unique: PG26rx8JO56qPydtWUH5YA-1 From: Eric Blake To: qemu-devel@nongnu.org Subject: [PULL 14/14] iotests: Test node/bitmap aliases during migration Date: Fri, 21 Aug 2020 09:08:26 -0500 Message-Id: <20200821140826.194322-15-eblake@redhat.com> In-Reply-To: <20200821140826.194322-1-eblake@redhat.com> References: <20200821140826.194322-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0.002 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/21 04:26:32 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Max Reitz Signed-off-by: Max Reitz Message-Id: <20200820150725.68687-4-mreitz@redhat.com> Reviewed-by: Eric Blake Tested-by: Eric Blake [eblake: fold in python cleanups recommended by Vladimir] Signed-off-by: Eric Blake --- tests/qemu-iotests/300 | 593 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/300.out | 5 + tests/qemu-iotests/group | 1 + 3 files changed, 599 insertions(+) create mode 100755 tests/qemu-iotests/300 create mode 100644 tests/qemu-iotests/300.out diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300 new file mode 100755 index 000000000000..5b75121b8496 --- /dev/null +++ b/tests/qemu-iotests/300 @@ -0,0 +1,593 @@ +#!/usr/bin/env python3 +# +# Copyright (C) 2020 Red Hat, Inc. +# +# Tests for dirty bitmaps migration with node aliases +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import os +import random +import re +from typing import Dict, List, Optional, Union +import iotests +import qemu + +BlockBitmapMapping =3D List[Dict[str, Union[str, List[Dict[str, str]]]]] + +assert iotests.sock_dir is not None +mig_sock =3D os.path.join(iotests.sock_dir, 'mig_sock') + + +class TestDirtyBitmapMigration(iotests.QMPTestCase): + src_node_name: str =3D '' + dst_node_name: str =3D '' + src_bmap_name: str =3D '' + dst_bmap_name: str =3D '' + + def setUp(self) -> None: + self.vm_a =3D iotests.VM(path_suffix=3D'-a') + self.vm_a.add_blockdev(f'node-name=3D{self.src_node_name},' + 'driver=3Dnull-co') + self.vm_a.launch() + + self.vm_b =3D iotests.VM(path_suffix=3D'-b') + self.vm_b.add_blockdev(f'node-name=3D{self.dst_node_name},' + 'driver=3Dnull-co') + self.vm_b.add_incoming(f'unix:{mig_sock}') + self.vm_b.launch() + + result =3D self.vm_a.qmp('block-dirty-bitmap-add', + node=3Dself.src_node_name, + name=3Dself.src_bmap_name) + self.assert_qmp(result, 'return', {}) + + # Dirty some random megabytes + for _ in range(9): + mb_ofs =3D random.randrange(1024) + self.vm_a.hmp_qemu_io(self.src_node_name, f'discard {mb_ofs}M = 1M') + + result =3D self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256', + node=3Dself.src_node_name, + name=3Dself.src_bmap_name) + self.bitmap_hash_reference =3D result['return']['sha256'] + + caps =3D [{'capability': name, 'state': True} + for name in ('dirty-bitmaps', 'events')] + + for vm in (self.vm_a, self.vm_b): + result =3D vm.qmp('migrate-set-capabilities', capabilities=3Dc= aps) + self.assert_qmp(result, 'return', {}) + + def tearDown(self) -> None: + self.vm_a.shutdown() + self.vm_b.shutdown() + try: + os.remove(mig_sock) + except OSError: + pass + + def check_bitmap(self, bitmap_name_valid: bool) -> None: + result =3D self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', + node=3Dself.dst_node_name, + name=3Dself.dst_bmap_name) + if bitmap_name_valid: + self.assert_qmp(result, 'return/sha256', + self.bitmap_hash_reference) + else: + self.assert_qmp(result, 'error/desc', + f"Dirty bitmap '{self.dst_bmap_name}' not foun= d") + + def migrate(self, bitmap_name_valid: bool =3D True, + migration_success: bool =3D True) -> None: + result =3D self.vm_a.qmp('migrate', uri=3Df'unix:{mig_sock}') + self.assert_qmp(result, 'return', {}) + + with iotests.Timeout(5, 'Timeout waiting for migration to complete= '): + self.assertEqual(self.vm_a.wait_migration('postmigrate'), + migration_success) + self.assertEqual(self.vm_b.wait_migration('running'), + migration_success) + + if migration_success: + self.check_bitmap(bitmap_name_valid) + + def verify_dest_error(self, msg: Optional[str]) -> None: + """ + Check whether the given error message is present in vm_b's log. + (vm_b is shut down to do so.) + If @msg is None, check that there has not been any error. + """ + self.vm_b.shutdown() + if msg is None: + self.assertNotIn('qemu-system-', self.vm_b.get_log()) + else: + self.assertIn(msg, self.vm_b.get_log()) + + @staticmethod + def mapping(node_name: str, node_alias: str, + bitmap_name: str, bitmap_alias: str) -> BlockBitmapMapping: + return [{ + 'node-name': node_name, + 'alias': node_alias, + 'bitmaps': [{ + 'name': bitmap_name, + 'alias': bitmap_alias + }] + }] + + def set_mapping(self, vm: iotests.VM, mapping: BlockBitmapMapping, + error: Optional[str] =3D None) -> None: + """ + Invoke migrate-set-parameters on @vm to set the given @mapping. + Check for success if @error is None, or verify the error message + if it is not. + On success, verify that "info migrate_parameters" on HMP returns + our mapping. (Just to check its formatting code.) + """ + result =3D vm.qmp('migrate-set-parameters', + block_bitmap_mapping=3Dmapping) + + if error is None: + self.assert_qmp(result, 'return', {}) + + result =3D vm.qmp('human-monitor-command', + command_line=3D'info migrate_parameters') + + m =3D re.search(r'^block-bitmap-mapping:\r?(\n .*)*\n', + result['return'], flags=3Dre.MULTILINE) + hmp_mapping =3D m.group(0).replace('\r', '') if m else None + + self.assertEqual(hmp_mapping, self.to_hmp_mapping(mapping)) + else: + self.assert_qmp(result, 'error/desc', error) + + @staticmethod + def to_hmp_mapping(mapping: BlockBitmapMapping) -> str: + result =3D 'block-bitmap-mapping:\n' + + for node in mapping: + result +=3D f" '{node['node-name']}' -> '{node['alias']}'\n" + + assert isinstance(node['bitmaps'], list) + for bitmap in node['bitmaps']: + result +=3D f" '{bitmap['name']}' -> '{bitmap['alias']}= '\n" + + return result + + +class TestAliasMigration(TestDirtyBitmapMigration): + src_node_name =3D 'node0' + dst_node_name =3D 'node0' + src_bmap_name =3D 'bmap0' + dst_bmap_name =3D 'bmap0' + + def test_migration_without_alias(self) -> None: + self.migrate(self.src_node_name =3D=3D self.dst_node_name and + self.src_bmap_name =3D=3D self.dst_bmap_name) + + # Check for error message on the destination + if self.src_node_name !=3D self.dst_node_name: + self.verify_dest_error(f"Cannot find " + f"device=3D{self.src_node_name} nor " + f"node_name=3D{self.src_node_name}") + else: + self.verify_dest_error(None) + + def test_alias_on_src_migration(self) -> None: + mapping =3D self.mapping(self.src_node_name, self.dst_node_name, + self.src_bmap_name, self.dst_bmap_name) + + self.set_mapping(self.vm_a, mapping) + self.migrate() + self.verify_dest_error(None) + + def test_alias_on_dst_migration(self) -> None: + mapping =3D self.mapping(self.dst_node_name, self.src_node_name, + self.dst_bmap_name, self.src_bmap_name) + + self.set_mapping(self.vm_b, mapping) + self.migrate() + self.verify_dest_error(None) + + def test_alias_on_both_migration(self) -> None: + src_map =3D self.mapping(self.src_node_name, 'node-alias', + self.src_bmap_name, 'bmap-alias') + + dst_map =3D self.mapping(self.dst_node_name, 'node-alias', + self.dst_bmap_name, 'bmap-alias') + + self.set_mapping(self.vm_a, src_map) + self.set_mapping(self.vm_b, dst_map) + self.migrate() + self.verify_dest_error(None) + + +class TestNodeAliasMigration(TestAliasMigration): + src_node_name =3D 'node-src' + dst_node_name =3D 'node-dst' + + +class TestBitmapAliasMigration(TestAliasMigration): + src_bmap_name =3D 'bmap-src' + dst_bmap_name =3D 'bmap-dst' + + +class TestFullAliasMigration(TestAliasMigration): + src_node_name =3D 'node-src' + dst_node_name =3D 'node-dst' + src_bmap_name =3D 'bmap-src' + dst_bmap_name =3D 'bmap-dst' + + +class TestLongBitmapNames(TestAliasMigration): + # Giving long bitmap names is OK, as long as there is a short alias for + # migration + src_bmap_name =3D 'a' * 512 + dst_bmap_name =3D 'b' * 512 + + # Skip all tests that do not use the intermediate alias + def test_migration_without_alias(self) -> None: + pass + + def test_alias_on_src_migration(self) -> None: + pass + + def test_alias_on_dst_migration(self) -> None: + pass + + +class TestBlockBitmapMappingErrors(TestDirtyBitmapMigration): + src_node_name =3D 'node0' + dst_node_name =3D 'node0' + src_bmap_name =3D 'bmap0' + dst_bmap_name =3D 'bmap0' + + """ + Note that mapping nodes or bitmaps that do not exist is not an error. + """ + + def test_non_injective_node_mapping(self) -> None: + mapping: BlockBitmapMapping =3D [ + { + 'node-name': 'node0', + 'alias': 'common-alias', + 'bitmaps': [{ + 'name': 'bmap0', + 'alias': 'bmap-alias0' + }] + }, + { + 'node-name': 'node1', + 'alias': 'common-alias', + 'bitmaps': [{ + 'name': 'bmap1', + 'alias': 'bmap-alias1' + }] + } + ] + + self.set_mapping(self.vm_a, mapping, + "Invalid mapping given for block-bitmap-mapping: " + "The node alias 'common-alias' is used twice") + + def test_non_injective_bitmap_mapping(self) -> None: + mapping: BlockBitmapMapping =3D [{ + 'node-name': 'node0', + 'alias': 'node-alias0', + 'bitmaps': [ + { + 'name': 'bmap0', + 'alias': 'common-alias' + }, + { + 'name': 'bmap1', + 'alias': 'common-alias' + } + ] + }] + + self.set_mapping(self.vm_a, mapping, + "Invalid mapping given for block-bitmap-mapping: " + "The bitmap alias 'node-alias0'/'common-alias' is= " + "used twice") + + def test_ambiguous_node_mapping(self) -> None: + mapping: BlockBitmapMapping =3D [ + { + 'node-name': 'node0', + 'alias': 'node-alias0', + 'bitmaps': [{ + 'name': 'bmap0', + 'alias': 'bmap-alias0' + }] + }, + { + 'node-name': 'node0', + 'alias': 'node-alias1', + 'bitmaps': [{ + 'name': 'bmap0', + 'alias': 'bmap-alias0' + }] + } + ] + + self.set_mapping(self.vm_a, mapping, + "Invalid mapping given for block-bitmap-mapping: " + "The node name 'node0' is mapped twice") + + def test_ambiguous_bitmap_mapping(self) -> None: + mapping: BlockBitmapMapping =3D [{ + 'node-name': 'node0', + 'alias': 'node-alias0', + 'bitmaps': [ + { + 'name': 'bmap0', + 'alias': 'bmap-alias0' + }, + { + 'name': 'bmap0', + 'alias': 'bmap-alias1' + } + ] + }] + + self.set_mapping(self.vm_a, mapping, + "Invalid mapping given for block-bitmap-mapping: " + "The bitmap 'node0'/'bmap0' is mapped twice") + + def test_migratee_node_is_not_mapped_on_src(self) -> None: + self.set_mapping(self.vm_a, []) + # Should just ignore all bitmaps on unmapped nodes + self.migrate(False) + self.verify_dest_error(None) + + def test_migratee_node_is_not_mapped_on_dst(self) -> None: + self.set_mapping(self.vm_b, []) + self.migrate(False) + self.verify_dest_error(f"Unknown node alias '{self.src_node_name}'= ") + + def test_migratee_bitmap_is_not_mapped_on_src(self) -> None: + mapping: BlockBitmapMapping =3D [{ + 'node-name': self.src_node_name, + 'alias': self.dst_node_name, + 'bitmaps': [] + }] + + self.set_mapping(self.vm_a, mapping) + # Should just ignore all unmapped bitmaps + self.migrate(False) + self.verify_dest_error(None) + + def test_migratee_bitmap_is_not_mapped_on_dst(self) -> None: + mapping: BlockBitmapMapping =3D [{ + 'node-name': self.dst_node_name, + 'alias': self.src_node_name, + 'bitmaps': [] + }] + + self.set_mapping(self.vm_b, mapping) + self.migrate(False) + self.verify_dest_error(f"Unknown bitmap alias " + f"'{self.src_bmap_name}' " + f"on node '{self.dst_node_name}' " + f"(alias '{self.src_node_name}')") + + def test_unused_mapping_on_dst(self) -> None: + # Let the source not send any bitmaps + self.set_mapping(self.vm_a, []) + + # Establish some mapping on the destination + self.set_mapping(self.vm_b, []) + + # The fact that there is a mapping on B without any bitmaps + # being received should be fine, not fatal + self.migrate(False) + self.verify_dest_error(None) + + def test_non_wellformed_node_alias(self) -> None: + alias =3D '123-foo' + + mapping: BlockBitmapMapping =3D [{ + 'node-name': self.src_node_name, + 'alias': alias, + 'bitmaps': [] + }] + + self.set_mapping(self.vm_a, mapping, + f"Invalid mapping given for block-bitmap-mapping:= " + f"The node alias '{alias}' is not well-formed") + + def test_node_alias_too_long(self) -> None: + alias =3D 'a' * 256 + + mapping: BlockBitmapMapping =3D [{ + 'node-name': self.src_node_name, + 'alias': alias, + 'bitmaps': [] + }] + + self.set_mapping(self.vm_a, mapping, + f"Invalid mapping given for block-bitmap-mapping:= " + f"The node alias '{alias}' is longer than 255 byt= es") + + def test_bitmap_alias_too_long(self) -> None: + alias =3D 'a' * 256 + + mapping =3D self.mapping(self.src_node_name, self.dst_node_name, + self.src_bmap_name, alias) + + self.set_mapping(self.vm_a, mapping, + f"Invalid mapping given for block-bitmap-mapping:= " + f"The bitmap alias '{alias}' is longer than 255 " + f"bytes") + + def test_bitmap_name_too_long(self) -> None: + name =3D 'a' * 256 + + result =3D self.vm_a.qmp('block-dirty-bitmap-add', + node=3Dself.src_node_name, + name=3Dname) + self.assert_qmp(result, 'return', {}) + + self.migrate(False, False) + + # Check for the error in the source's log + self.vm_a.shutdown() + self.assertIn(f"Cannot migrate bitmap '{name}' on node " + f"'{self.src_node_name}': Name is longer than 255 by= tes", + self.vm_a.get_log()) + + # Expect abnormal shutdown of the destination VM because of + # the failed migration + try: + self.vm_b.shutdown() + except qemu.machine.AbnormalShutdown: + pass + + def test_aliased_bitmap_name_too_long(self) -> None: + # Longer than the maximum for bitmap names + self.dst_bmap_name =3D 'a' * 1024 + + mapping =3D self.mapping(self.dst_node_name, self.src_node_name, + self.dst_bmap_name, self.src_bmap_name) + + # We would have to create this bitmap during migration, and + # that would fail, because the name is too long. Better to + # catch it early. + self.set_mapping(self.vm_b, mapping, + f"Invalid mapping given for block-bitmap-mapping:= " + f"The bitmap name '{self.dst_bmap_name}' is longe= r " + f"than 1023 bytes") + + def test_node_name_too_long(self) -> None: + # Longer than the maximum for node names + self.dst_node_name =3D 'a' * 32 + + mapping =3D self.mapping(self.dst_node_name, self.src_node_name, + self.dst_bmap_name, self.src_bmap_name) + + # During migration, this would appear simply as a node that + # cannot be found. Still better to catch impossible node + # names early (similar to test_non_wellformed_node_alias). + self.set_mapping(self.vm_b, mapping, + f"Invalid mapping given for block-bitmap-mapping:= " + f"The node name '{self.dst_node_name}' is longer " + f"than 31 bytes") + + +class TestCrossAliasMigration(TestDirtyBitmapMigration): + """ + Swap aliases, both to see that qemu does not get confused, and + that we can migrate multiple things at once. + + So we migrate this: + node-a.bmap-a -> node-b.bmap-b + node-a.bmap-b -> node-b.bmap-a + node-b.bmap-a -> node-a.bmap-b + node-b.bmap-b -> node-a.bmap-a + """ + + src_node_name =3D 'node-a' + dst_node_name =3D 'node-b' + src_bmap_name =3D 'bmap-a' + dst_bmap_name =3D 'bmap-b' + + def setUp(self) -> None: + TestDirtyBitmapMigration.setUp(self) + + # Now create another block device and let both have two bitmaps ea= ch + result =3D self.vm_a.qmp('blockdev-add', + node_name=3D'node-b', driver=3D'null-co') + self.assert_qmp(result, 'return', {}) + + result =3D self.vm_b.qmp('blockdev-add', + node_name=3D'node-a', driver=3D'null-co') + self.assert_qmp(result, 'return', {}) + + bmaps_to_add =3D (('node-a', 'bmap-b'), + ('node-b', 'bmap-a'), + ('node-b', 'bmap-b')) + + for (node, bmap) in bmaps_to_add: + result =3D self.vm_a.qmp('block-dirty-bitmap-add', + node=3Dnode, name=3Dbmap) + self.assert_qmp(result, 'return', {}) + + @staticmethod + def cross_mapping() -> BlockBitmapMapping: + return [ + { + 'node-name': 'node-a', + 'alias': 'node-b', + 'bitmaps': [ + { + 'name': 'bmap-a', + 'alias': 'bmap-b' + }, + { + 'name': 'bmap-b', + 'alias': 'bmap-a' + } + ] + }, + { + 'node-name': 'node-b', + 'alias': 'node-a', + 'bitmaps': [ + { + 'name': 'bmap-b', + 'alias': 'bmap-a' + }, + { + 'name': 'bmap-a', + 'alias': 'bmap-b' + } + ] + } + ] + + def verify_dest_has_all_bitmaps(self) -> None: + bitmaps =3D self.vm_b.query_bitmaps() + + # Extract and sort bitmap names + for node in bitmaps: + bitmaps[node] =3D sorted((bmap['name'] for bmap in bitmaps[nod= e])) + + self.assertEqual(bitmaps, + {'node-a': ['bmap-a', 'bmap-b'], + 'node-b': ['bmap-a', 'bmap-b']}) + + def test_alias_on_src(self) -> None: + self.set_mapping(self.vm_a, self.cross_mapping()) + + # Checks that node-a.bmap-a was migrated to node-b.bmap-b, and + # that is enough + self.migrate() + self.verify_dest_has_all_bitmaps() + self.verify_dest_error(None) + + def test_alias_on_dst(self) -> None: + self.set_mapping(self.vm_b, self.cross_mapping()) + + # Checks that node-a.bmap-a was migrated to node-b.bmap-b, and + # that is enough + self.migrate() + self.verify_dest_has_all_bitmaps() + self.verify_dest_error(None) + + +if __name__ =3D=3D '__main__': + iotests.main(supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/300.out b/tests/qemu-iotests/300.out new file mode 100644 index 000000000000..cafb8161f7b1 --- /dev/null +++ b/tests/qemu-iotests/300.out @@ -0,0 +1,5 @@ +..................................... +---------------------------------------------------------------------- +Ran 37 tests + +OK diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index ecff2621cddc..a53ea7f78b81 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -307,6 +307,7 @@ 296 rw 297 meta 299 auto quick +300 migration 301 backing quick 302 quick 303 rw quick --=20 2.28.0