From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676596; cv=none; d=zohomail.com; s=zohoarc; b=J/gyWpp5M+p90EvdtNGAEEN9CME0eO8xumOYUbVM3FGZ0levHACqMyxdglgjmv7YZbgsXSVeJfeDBP7scy5UV09/nV80LtHH7J/VbN83UeE6zDFAsCKcynTdGukau1BtsqvoiyP/lYi/dLhOQxvt00jA1q2iSQ0EQQckoh+wP/s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676596; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=7s0EkTEnB2xxcrMrSY6owviWDfEtDPHiESXi+CviVuo=; b=Ryk/vyyxbDm45FGLFSLMYMwnn6/4n3fACMN8SHbdC7HTVcrQ4XDcrmQVT95BvxnQQfNFNYah7sSDw7KMk8a8TBCB3tVmDGwtxEPfRDwvrFXo+Fj+sT9kedBlSQboyh5yxS8R1L070AomVVUaVT1s2w5OOCFMo9UksK1+o02bJ7E= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159467659669836.701469337733556; Mon, 13 Jul 2020 14:43:16 -0700 (PDT) Received: from localhost ([::1]:52422 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv6EF-0004lr-IV for importer@patchew.org; Mon, 13 Jul 2020 17:43:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40632) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv68L-0003Jn-BV; Mon, 13 Jul 2020 17:37:09 -0400 Received: from relay.sw.ru ([185.231.240.75]:52832 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv68J-00052C-Rs; Mon, 13 Jul 2020 17:37:09 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-6Y; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 01/10] qcow2: Fix capitalization of header extension constant. Date: Tue, 14 Jul 2020 00:36:34 +0300 Message-Id: <1594676203-436999-2-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make the capitalization of the hexadecimal numbers consistent for the QCOW2 header extension constants in docs/interop/qcow2.txt. Suggested-by: Eric Blake Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 2 +- docs/interop/qcow2.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index ea33673..6c6bee3 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -66,7 +66,7 @@ typedef struct { } QEMU_PACKED QCowExtension; =20 #define QCOW2_EXT_MAGIC_END 0 -#define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xE2792ACA +#define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xe2792aca #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 #define QCOW2_EXT_MAGIC_CRYPTO_HEADER 0x0537be77 #define QCOW2_EXT_MAGIC_BITMAPS 0x23852875 diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index cb72346..f072e27 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -231,7 +231,7 @@ be stored. Each extension has a structure like the foll= owing: =20 Byte 0 - 3: Header extension type: 0x00000000 - End of the header extension area - 0xE2792ACA - Backing file format name string + 0xe2792aca - Backing file format name string 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer --=20 1.8.3.1 From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676476; cv=none; d=zohomail.com; s=zohoarc; b=DgFV6Pw94KqPiSqSzQhU/18JHfKNbB8Rj4X4TYVwf4fQry+mu//6oEJK8BbtqFIw+7dhSkPcMQ6nZs6cqgXCd2AT06rQOERLglBmsJDmSgdO4Jz4qMhmqA0sM/+b2HmlkWwcxemxjUqlBVUawmOQRN8kP/ioTgU3E7sMKlfh53w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676476; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=jTaQe+3C4V+UcRj4btutD2ZIpAv4td/tmxwTbSW57kA=; b=WDWMZ+6IhtEJzsHhZovNjD5Y52QDK1TvRuX/dY/AQuUHdfIhfhRsU7kMjEsu8OrEQzRTP+s6MWSgaL6RjNpNvn+mgysXCBm8aqKJJT2tyJjgWtmnc/jVVJfZN8UAE2K6hhB8TauSzMPBRQ/35/JAZqaDXCA17whij3jjEGdUE9A= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594676476594173.13181359875603; Mon, 13 Jul 2020 14:41:16 -0700 (PDT) Received: from localhost ([::1]:44202 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv6CJ-0001St-Cq for importer@patchew.org; Mon, 13 Jul 2020 17:41:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40500) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv685-0002zz-GC; Mon, 13 Jul 2020 17:36:53 -0400 Received: from relay.sw.ru ([185.231.240.75]:52830 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv680-00052A-4l; Mon, 13 Jul 2020 17:36:53 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-7U; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 02/10] qcow2_format.py: make printable data an extension class member Date: Tue, 14 Jul 2020 00:36:35 +0300 Message-Id: <1594676203-436999-3-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Let us differ binary data type from string one for the extension data variable and keep the string as the QcowHeaderExtension class member. Signed-off-by: Andrey Shinkevich Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/qcow2_format.py | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index cc432e7..2f3681b 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -165,6 +165,13 @@ class QcowHeaderExtension(Qcow2Struct): self.data =3D fd.read(padded) assert self.data is not None =20 + data_str =3D self.data[:self.length] + if all(c in string.printable.encode('ascii') for c in data_str): + data_str =3D f"'{ data_str.decode('ascii') }'" + else: + data_str =3D '' + self.data_str =3D data_str + if self.magic =3D=3D QCOW2_EXT_MAGIC_BITMAPS: self.obj =3D Qcow2BitmapExt(data=3Dself.data) else: @@ -174,12 +181,7 @@ class QcowHeaderExtension(Qcow2Struct): super().dump() =20 if self.obj is None: - data =3D self.data[:self.length] - if all(c in string.printable.encode('ascii') for c in data): - data =3D f"'{ data.decode('ascii') }'" - else: - data =3D '' - print(f'{"data":<25} {data}') + print(f'{"data":<25} {self.data_str}') else: self.obj.dump() =20 --=20 1.8.3.1 From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676469; cv=none; d=zohomail.com; s=zohoarc; b=Bo65vA4/wOYW8tzqJOQRxhQk2GBVsrkYKKno51mwqPR77K9Bfuq5CGT6HWain4shJq40ZGOGxoT55X3XeD3/vymtNJLUNceIVJsgnucKjDAu2bTlbAx2BPhJK9bw8eehdLucp9RzgS6rYkBCjZ+k2qny1dDZr9jaA9it+ZG9AOI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676469; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=8/9SERp38qKYxdpOgqqXuHSK/voJLwiZ1TiQSse6FDQ=; b=aeFUhznM0LLZEs9j3TnEbY7V0o+u4eoRl8qKchK0+mZTrpco65B4AASP3QuIVAz4D5liZ+prh4y13/pOBNB6conCkZfejntn10NXyxOd0pNCsHUGYdXhcyZpZZijvFAm02+nu8IQKgqzAzNx7BuxYmUgUt3+th0iA1WkJhxFVYE= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594676469747314.3259514410222; Mon, 13 Jul 2020 14:41:09 -0700 (PDT) Received: from localhost ([::1]:43592 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv6CC-0001E7-Gg for importer@patchew.org; Mon, 13 Jul 2020 17:41:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40488) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv684-0002xB-HI; Mon, 13 Jul 2020 17:36:52 -0400 Received: from relay.sw.ru ([185.231.240.75]:52838 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv67z-00052D-UR; Mon, 13 Jul 2020 17:36:52 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-8U; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 03/10] qcow2_format.py: change Qcow2BitmapExt initialization method Date: Tue, 14 Jul 2020 00:36:36 +0300 Message-Id: <1594676203-436999-4-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" There are two ways to initialize a class derived from Qcow2Struct: 1. Pass a block of binary data to the constructor. 2. Pass the file descriptor to allow reading the file from constructor. Let's change the Qcow2BitmapExt initialization method from 1 to 2 to support a scattered reading in the initialization chain. The implementation comes with the patch that follows. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/qcow2_format.py | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index 2f3681b..cbaffc4 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -113,6 +113,11 @@ class Qcow2BitmapExt(Qcow2Struct): ('u64', '{:#x}', 'bitmap_directory_offset') ) =20 + def __init__(self, fd): + super().__init__(fd=3Dfd) + pad =3D (struct.calcsize(self.fmt) + 7) & ~7 + if pad: + fd.seek(pad, 1) =20 QCOW2_EXT_MAGIC_BITMAPS =3D 0x23852875 =20 @@ -161,21 +166,21 @@ class QcowHeaderExtension(Qcow2Struct): else: assert all(v is None for v in (magic, length, data)) super().__init__(fd=3Dfd) - padded =3D (self.length + 7) & ~7 - self.data =3D fd.read(padded) - assert self.data is not None - - data_str =3D self.data[:self.length] - if all(c in string.printable.encode('ascii') for c in data_str): - data_str =3D f"'{ data_str.decode('ascii') }'" - else: - data_str =3D '' - self.data_str =3D data_str + if self.magic =3D=3D QCOW2_EXT_MAGIC_BITMAPS: + self.obj =3D Qcow2BitmapExt(fd=3Dfd) + else: + padded =3D (self.length + 7) & ~7 + self.data =3D fd.read(padded) + assert self.data is not None + self.obj =3D None + data_str =3D self.data[:self.length] + if all(c in string.printable.encode( + 'ascii') for c in data_str): + data_str =3D f"'{ data_str.decode('ascii') }'" + else: + data_str =3D '' + self.data_str =3D data_str =20 - if self.magic =3D=3D QCOW2_EXT_MAGIC_BITMAPS: - self.obj =3D Qcow2BitmapExt(data=3Dself.data) - else: - self.obj =3D None =20 def dump(self): super().dump() --=20 1.8.3.1 From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676465; cv=none; d=zohomail.com; s=zohoarc; b=C6kPUk3hORSvvtShD/Gx7v9ob4uxuv+5z/2vvG3knYPImyUh1wZ/PCHjrFEuL6zlzND2uhhCg0y/zTtuuy1at3HKHf034rcQRVncvIz3qjLDCronw63IVziJNNuWfHPzBFaUgXewh5PTpgwZa4Z+64y67jKpSsZo4bPgdcFHZzY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676465; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=1pElELHDBlZZrnA7IaCgaPhawBR2l6Debnq8ziuRlpo=; b=JYw9gPd0eGNUfmaB5/mSCNLszPZW/Y1VDWyJQadcBYndPDj7r2JSG/cOrT8fRZ1KQ0GCeqPLmDAJqCelsxUWc1eVR4ddUOpDWuRkqd+9vpmdh9Tf8x/xDU5+1XsXPhKWTr0pOVq5gBMUx9WIPX8NMx1XKfFgK/YoeCQGTWc/teM= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594676465454938.2984432380542; Mon, 13 Jul 2020 14:41:05 -0700 (PDT) Received: from localhost ([::1]:43210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv6C8-00014o-1x for importer@patchew.org; Mon, 13 Jul 2020 17:41:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40462) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv683-0002vS-DG; Mon, 13 Jul 2020 17:36:51 -0400 Received: from relay.sw.ru ([185.231.240.75]:52834 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv680-00052E-0C; Mon, 13 Jul 2020 17:36:51 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-AU; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 04/10] qcow2_format.py: dump bitmap flags in human readable way. Date: Tue, 14 Jul 2020 00:36:37 +0300 Message-Id: <1594676203-436999-5-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Introduce the class BitmapFlags that parses a bitmap flags mask. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/qcow2_format.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index cbaffc4..e77c831 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -40,6 +40,22 @@ class Flags64(Qcow2Field): return str(bits) =20 =20 +class BitmapFlags(Qcow2Field): + + flags =3D { + 0x1: 'in-use', + 0x2: 'auto' + } + + def __str__(self): + bits =3D [] + for bit in range(64): + flag =3D self.value & (1 << bit) + if flag: + bits.append(self.flags.get(flag, f'bit-{bit}')) + return f'{self.value:#x} ({bits})' + + class Enum(Qcow2Field): =20 def __str__(self): --=20 1.8.3.1 From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676559; cv=none; d=zohomail.com; s=zohoarc; b=K9EPcv/EgC9Z8001Z67w8aVJDcfp69tsFz6/EG9SuxR84/mdisClS4N3NrkU9LPpmNVj3e2+hwfPh2aBYawT/Qs++oRMTFefILMDHGpdxTaWyLM9Iqd2Xq5ffcPolDFQfhUeQ4fqrzbCT7N5P0+BrJf4mAXy7/gYdqQLLuyQKEo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676559; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=0BXOehT7M/0q/XghHwi5C5nCx3tfIXKZajYg1AmhE2o=; b=cqpIehsJ5Wcqw2tRQPJhhhn+k0meODb7n+MB8dEI+MSQveRFiO4NogTV9qMJk+W6nGZruuoURcAeUgNRsekr/Qv6TTe2q56mf3wWuhBY4vVCy6pz8uENOzfkMr5EJ0q+pzL9HuMZ8Ev0ccSQQDipwkas6/yZoNA4a2WI+XpVKSg= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594676559169135.87904566632142; Mon, 13 Jul 2020 14:42:39 -0700 (PDT) Received: from localhost ([::1]:50012 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv6De-0003ng-2J for importer@patchew.org; Mon, 13 Jul 2020 17:42:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40496) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv685-0002ym-3V; Mon, 13 Jul 2020 17:36:53 -0400 Received: from relay.sw.ru ([185.231.240.75]:52848 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv680-00052G-4j; Mon, 13 Jul 2020 17:36:52 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-BT; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 05/10] qcow2_format.py: Dump bitmap directory information Date: Tue, 14 Jul 2020 00:36:38 +0300 Message-Id: <1594676203-436999-6-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Read and dump entries from the bitmap directory of QCOW2 image. Header extension: magic 0x23852875 (Bitmaps) ... Bitmap name bitmap-1 bitmap_table_offset 0xf0000 bitmap_table_size 1 flags 0x2 (['auto']) type 1 granularity_bits 16 name_size 8 extra_data_size 0 Suggested-by: Kevin Wolf Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/qcow2_format.py | 47 ++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index e77c831..f0db9f4 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -134,6 +134,53 @@ class Qcow2BitmapExt(Qcow2Struct): pad =3D (struct.calcsize(self.fmt) + 7) & ~7 if pad: fd.seek(pad, 1) + position =3D fd.tell() + self.read_bitmap_directory(fd) + fd.seek(position) + + def read_bitmap_directory(self, fd): + fd.seek(self.bitmap_directory_offset) + self.bitmap_directory =3D \ + [Qcow2BitmapDirEntry(fd) for _ in range(self.nb_bitmaps)] + + def dump(self): + super().dump() + for entry in self.bitmap_directory: + print() + entry.dump() + + +class Qcow2BitmapDirEntry(Qcow2Struct): + + fields =3D ( + ('u64', '{:#x}', 'bitmap_table_offset'), + ('u32', '{}', 'bitmap_table_size'), + ('u32', BitmapFlags, 'flags'), + ('u8', '{}', 'type'), + ('u8', '{}', 'granularity_bits'), + ('u16', '{}', 'name_size'), + ('u32', '{}', 'extra_data_size') + ) + + def __init__(self, fd): + super().__init__(fd=3Dfd) + # Seek relative to the current position in the file + fd.seek(self.extra_data_size, 1) + bitmap_name =3D fd.read(self.name_size) + self.name =3D bitmap_name.decode('ascii') + # Move position to the end of the entry in the directory + entry_raw_size =3D self.bitmap_dir_entry_raw_size() + padding =3D ((entry_raw_size + 7) & ~7) - entry_raw_size + fd.seek(padding, 1) + + def bitmap_dir_entry_raw_size(self): + return struct.calcsize(self.fmt) + self.name_size + \ + self.extra_data_size + + def dump(self): + print(f'{"Bitmap name":<25} {self.name}') + super(Qcow2BitmapDirEntry, self).dump() + =20 QCOW2_EXT_MAGIC_BITMAPS =3D 0x23852875 =20 --=20 1.8.3.1 From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676378; cv=none; d=zohomail.com; s=zohoarc; b=CEhV8ULEETrKWKTboVUTWzM6DpXRCqyl6c2aAwJi5MvPLPq1Tq9AsVZrzHkyeTIR/CFFvarIY5L0W52DkVei4PDiZVuwMXOhZ146b9GQS5mda0e0vbyXu/baMS6ClLeSyMiUMFrqMq/7OmhD2DPnXO1zK8ZguW/vFnOkV7d/5KQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676378; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=S+7klTEwa59P2Hr27bFd/1d7Q0iPwyfGxdEN8UjMhfQ=; b=X07+cPFtsD7M8XOJ6SsxTxHjFnHzMD20vbL6cKeJL+8MeSFW8yQeXH6iEmo0BDfHX/9W098GYnHyyS9z0Y8s/YsCn8iiQjqqPJyfs6WuZEtOQYp7o0Df1fxgAwpGHI+irp+1jQ/ZvJKeJ4GugvvOZpeOSThOwWdAxx6mBwR+Uc8= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594676378381402.44411756508475; Mon, 13 Jul 2020 14:39:38 -0700 (PDT) Received: from localhost ([::1]:36740 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv6Ai-0006oc-RA for importer@patchew.org; Mon, 13 Jul 2020 17:39:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40480) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv684-0002wQ-5W; Mon, 13 Jul 2020 17:36:52 -0400 Received: from relay.sw.ru ([185.231.240.75]:52836 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv680-00052B-0l; Mon, 13 Jul 2020 17:36:51 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-EM; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 06/10] qcow2_format.py: pass cluster size to substructures Date: Tue, 14 Jul 2020 00:36:39 +0300 Message-Id: <1594676203-436999-7-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The cluster size of an image is the QcowHeader class member and may be obtained by dependent extension structures such as Qcow2BitmapExt for further bitmap table details print. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/qcow2_format.py | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index f0db9f4..d9c8513 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -129,19 +129,21 @@ class Qcow2BitmapExt(Qcow2Struct): ('u64', '{:#x}', 'bitmap_directory_offset') ) =20 - def __init__(self, fd): + def __init__(self, fd, cluster_size): super().__init__(fd=3Dfd) pad =3D (struct.calcsize(self.fmt) + 7) & ~7 if pad: fd.seek(pad, 1) position =3D fd.tell() + self.cluster_size =3D cluster_size self.read_bitmap_directory(fd) fd.seek(position) =20 def read_bitmap_directory(self, fd): fd.seek(self.bitmap_directory_offset) self.bitmap_directory =3D \ - [Qcow2BitmapDirEntry(fd) for _ in range(self.nb_bitmaps)] + [Qcow2BitmapDirEntry(fd, cluster_size=3Dself.cluster_size) + for _ in range(self.nb_bitmaps)] =20 def dump(self): super().dump() @@ -162,8 +164,9 @@ class Qcow2BitmapDirEntry(Qcow2Struct): ('u32', '{}', 'extra_data_size') ) =20 - def __init__(self, fd): + def __init__(self, fd, cluster_size): super().__init__(fd=3Dfd) + self.cluster_size =3D cluster_size # Seek relative to the current position in the file fd.seek(self.extra_data_size, 1) bitmap_name =3D fd.read(self.name_size) @@ -203,11 +206,13 @@ class QcowHeaderExtension(Qcow2Struct): # then padding to next multiply of 8 ) =20 - def __init__(self, magic=3DNone, length=3DNone, data=3DNone, fd=3DNone= ): + def __init__(self, magic=3DNone, length=3DNone, data=3DNone, fd=3DNone, + cluster_size=3DNone): """ Support both loading from fd and creation from user data. For fd-based creation current position in a file will be used to r= ead the data. + The cluster_size value may be obtained by dependent structures. =20 This should be somehow refactored and functionality should be move= d to superclass (to allow creation of any qcow2 struct), but then, fiel= ds @@ -230,7 +235,7 @@ class QcowHeaderExtension(Qcow2Struct): assert all(v is None for v in (magic, length, data)) super().__init__(fd=3Dfd) if self.magic =3D=3D QCOW2_EXT_MAGIC_BITMAPS: - self.obj =3D Qcow2BitmapExt(fd=3Dfd) + self.obj =3D Qcow2BitmapExt(fd=3Dfd, cluster_size=3Dcluste= r_size) else: padded =3D (self.length + 7) & ~7 self.data =3D fd.read(padded) @@ -244,7 +249,6 @@ class QcowHeaderExtension(Qcow2Struct): data_str =3D '' self.data_str =3D data_str =20 - def dump(self): super().dump() =20 @@ -316,7 +320,7 @@ class QcowHeader(Qcow2Struct): end =3D self.cluster_size =20 while fd.tell() < end: - ext =3D QcowHeaderExtension(fd=3Dfd) + ext =3D QcowHeaderExtension(fd=3Dfd, cluster_size=3Dself.clust= er_size) if ext.magic =3D=3D 0: break else: --=20 1.8.3.1 From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676556; cv=none; d=zohomail.com; s=zohoarc; b=JwUADyDSehGZQ3Whudop7OmzMT7Tgl3NFEGhi/EDVNyRYuTRSQq+wTMOlbfJmM3r9TmKewSLjSThJc+ndcwSuFhVxqNIxpme/swGiSV9071nzZeqpnLy0rbXNfxzzXt72K0JzeURPVOdFx0AeWCbUzwGGmTOSY5/tjQpW6XHcbA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676556; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=KNsn+O7z7/jniNQlSLxj8tqeROkgysiumwbkETiwG64=; b=OChPmOEUhMjSJND2nm3+tCOxAuiDvYjOQjAyLAD5Mq7f0BHUUU7kzpf14rSlPkuAD5cgUbKGAHWf3B7k4opnKW4g9JFg+6DT35kxJqamlO3pvCRzlvkj+KVX5zRcO3g+nDW39unnSALu2DSg0CZPyntRLz/TWeuMXuV7TdETCFM= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594676556949953.7305886873037; Mon, 13 Jul 2020 14:42:36 -0700 (PDT) Received: from localhost ([::1]:49856 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv6Db-0003jr-NF for importer@patchew.org; Mon, 13 Jul 2020 17:42:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40494) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv684-0002y3-Ta; Mon, 13 Jul 2020 17:36:52 -0400 Received: from relay.sw.ru ([185.231.240.75]:52824 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv680-000527-1d; Mon, 13 Jul 2020 17:36:52 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-Nj; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 07/10] qcow2_format.py: Dump bitmap table serialized entries Date: Tue, 14 Jul 2020 00:36:40 +0300 Message-Id: <1594676203-436999-8-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add bitmap table information to the QCOW2 metadata dump. Bitmap name bitmap-1 ... Bitmap table type offset size 0 serialized 4718592 65536 1 serialized 4294967296 65536 2 serialized 5348033147437056 65536 3 serialized 13792273858822144 65536 4 serialized 4718592 65536 5 serialized 4294967296 65536 6 serialized 4503608217305088 65536 7 serialized 14073748835532800 65536 Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/qcow2_format.py | 42 ++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 42 insertions(+) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index d9c8513..2c78d46 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -175,14 +175,56 @@ class Qcow2BitmapDirEntry(Qcow2Struct): entry_raw_size =3D self.bitmap_dir_entry_raw_size() padding =3D ((entry_raw_size + 7) & ~7) - entry_raw_size fd.seek(padding, 1) + position =3D fd.tell() + self.read_bitmap_table(fd) + fd.seek(position) =20 def bitmap_dir_entry_raw_size(self): return struct.calcsize(self.fmt) + self.name_size + \ self.extra_data_size =20 + def read_bitmap_table(self, fd): + fd.seek(self.bitmap_table_offset) + table_size =3D self.bitmap_table_size * 8 * 8 + table =3D [e[0] for e in struct.iter_unpack('>Q', fd.read(table_si= ze))] + self.bitmap_table =3D Qcow2BitmapTable(raw_table=3Dtable, + cluster_size=3Dself.cluster_s= ize) + def dump(self): print(f'{"Bitmap name":<25} {self.name}') super(Qcow2BitmapDirEntry, self).dump() + self.bitmap_table.dump() + + +class Qcow2BitmapTableEntry: + + BME_TABLE_ENTRY_OFFSET_MASK =3D 0x00fffffffffffe00 + BME_TABLE_ENTRY_FLAG_ALL_ONES =3D 1 + + def __init__(self, entry): + self.offset =3D entry & self.BME_TABLE_ENTRY_OFFSET_MASK + if self.offset: + self.type =3D 'serialized' + elif entry & self.BME_TABLE_ENTRY_FLAG_ALL_ONES: + self.type =3D 'all-ones' + else: + self.type =3D 'all-zeroes' + + +class Qcow2BitmapTable: + + def __init__(self, raw_table, cluster_size): + self.entries =3D [] + self.cluster_size =3D cluster_size + for entry in raw_table: + self.entries.append(Qcow2BitmapTableEntry(entry)) + + def dump(self): + size =3D self.cluster_size + bitmap_table =3D enumerate(self.entries) + print(f'{"Bitmap table":<14} {"type":<15} {"offset":<24} {"size"}') + for i, entry in bitmap_table: + print(f'{i:<14} {entry.type:<15} {entry.offset:<24} {size}') =20 =20 QCOW2_EXT_MAGIC_BITMAPS =3D 0x23852875 --=20 1.8.3.1 From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676378; cv=none; d=zohomail.com; s=zohoarc; b=UcoBMK5m7zQMo103VwiBLfi/wdrAk8wTFvjqLWIW13tF8zX5TAeDv5YxkZnw8VUsoHKKgoiu30nYtLI5RKiRi5AsP+wLdrelgAS77gxzY0H2UAAiDqMCYeZB8SX4BfBtGYAUfEh+Xh7UDuM5WDDP5O1k1zxIUJ+Iw6HdwH/9ScY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676378; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=DImLmvQ+wfgnR4w2dXFTl1PU/CkoRmrbKOcw7Qs8rBU=; b=bcQ8EJaA/9UQkXaDFFPqldPCWjima8PJPyt6Wl4Wo6iphj6PDFBaSfc7jlSQAlf9BW4nFhB5HQ2MmYBJuXHD8IADUUNU6/xCiC9cNbHCLkvelCb72oOyxBHY/wBfwKWeCznFwmGzsLLCzFZ9qslpKbsPor3yoWD7t7zHJkCxztM= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594676378921488.7670317920258; Mon, 13 Jul 2020 14:39:38 -0700 (PDT) Received: from localhost ([::1]:36800 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv6Aj-0006q7-MB for importer@patchew.org; Mon, 13 Jul 2020 17:39:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40442) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv682-0002uP-NL; Mon, 13 Jul 2020 17:36:50 -0400 Received: from relay.sw.ru ([185.231.240.75]:52842 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv67z-00052F-US; Mon, 13 Jul 2020 17:36:50 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-Pc; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 08/10] qcow2.py: Introduce '-j' key to dump in JSON format Date: Tue, 14 Jul 2020 00:36:41 +0300 Message-Id: <1594676203-436999-9-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add the command key to the qcow2.py arguments list to dump QCOW2 metadata in JSON format. Here is the suggested way to do that. The implementation of the dump in JSON format is in the patch that follows. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/qcow2.py | 19 +++++++++++++++---- tests/qemu-iotests/qcow2_format.py | 16 ++++++++-------- 2 files changed, 23 insertions(+), 12 deletions(-) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index 0910e6a..7402279 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -26,16 +26,19 @@ from qcow2_format import ( ) =20 =20 +dump_json =3D False + + def cmd_dump_header(fd): h =3D QcowHeader(fd) - h.dump() + h.dump(dump_json) print() - h.dump_extensions() + h.dump_extensions(dump_json) =20 =20 def cmd_dump_header_exts(fd): h =3D QcowHeader(fd) - h.dump_extensions() + h.dump_extensions(dump_json) =20 =20 def cmd_set_header(fd, name, value): @@ -134,6 +137,11 @@ cmds =3D [ =20 =20 def main(filename, cmd, args): + global dump_json + dump_json =3D '-j' in sys.argv + if dump_json: + sys.argv.remove('-j') + args.remove('-j') fd =3D open(filename, "r+b") try: for name, handler, num_args, desc in cmds: @@ -151,11 +159,14 @@ def main(filename, cmd, args): =20 =20 def usage(): - print("Usage: %s [, ...]" % sys.argv[0]) + print("Usage: %s [, ...] [, ...]" % sys.argv[0]) print("") print("Supported commands:") for name, handler, num_args, desc in cmds: print(" %-20s - %s" % (name, desc)) + print("") + print("Supported keys:") + print(" %-20s - %s" % ('-j', 'Dump in JSON format')) =20 =20 if __name__ =3D=3D '__main__': diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index 2c78d46..e0e14b5 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -109,7 +109,7 @@ class Qcow2Struct(metaclass=3DQcow2StructMeta): self.__dict__ =3D dict((field[2], values[i]) for i, field in enumerate(self.fields)) =20 - def dump(self): + def dump(self, dump_json=3DNone): for f in self.fields: value =3D self.__dict__[f[2]] if isinstance(f[1], str): @@ -145,8 +145,8 @@ class Qcow2BitmapExt(Qcow2Struct): [Qcow2BitmapDirEntry(fd, cluster_size=3Dself.cluster_size) for _ in range(self.nb_bitmaps)] =20 - def dump(self): - super().dump() + def dump(self, dump_json=3DNone): + super().dump(dump_json) for entry in self.bitmap_directory: print() entry.dump() @@ -190,7 +190,7 @@ class Qcow2BitmapDirEntry(Qcow2Struct): self.bitmap_table =3D Qcow2BitmapTable(raw_table=3Dtable, cluster_size=3Dself.cluster_s= ize) =20 - def dump(self): + def dump(self, dump_json=3DNone): print(f'{"Bitmap name":<25} {self.name}') super(Qcow2BitmapDirEntry, self).dump() self.bitmap_table.dump() @@ -291,13 +291,13 @@ class QcowHeaderExtension(Qcow2Struct): data_str =3D '' self.data_str =3D data_str =20 - def dump(self): + def dump(self, dump_json=3DNone): super().dump() =20 if self.obj is None: print(f'{"data":<25} {self.data_str}') else: - self.obj.dump() + self.obj.dump(dump_json) =20 @classmethod def create(cls, magic, data): @@ -396,8 +396,8 @@ class QcowHeader(Qcow2Struct): buf =3D buf[0:header_bytes-1] fd.write(buf) =20 - def dump_extensions(self): + def dump_extensions(self, dump_json=3DNone): for ex in self.extensions: print('Header extension:') - ex.dump() + ex.dump(dump_json) print() --=20 1.8.3.1 From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676279; cv=none; d=zohomail.com; s=zohoarc; b=dVZEGWKomfkSctgztAAzlCYr5fHxfT2g17oCaHb6PVGedL1n8VMEkL/rXXzT36fMTopuhWVhJkqiC6IxCimZYkCBhJRRCAuxPH0FFz0QILvasmCHlIDz8fYm6Pe8pUU2EiBjX/HnWMUyVRs6xUP8vrCDGZyzQP+DPWGdB+IYgNs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676279; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=7NDHGFUrDb3xLK/DYPldfzDLmcUzq6kwJMZZ5g57Ilg=; b=FDTJtWXYRHv1CDA5ZbcB2aZL6NCYHgS58uJnRayay9gzk7RoIME1w4OmEutp3FCKxveGZUvb9r4U+6iJsrUs2MQPPfJvBfLCU6wvMeO2tVEnDjenUWnTgJ2mlF1tWwFrVtg2l+OpjJ6XqafBJvP1oGuLsA/kA/D94T53mYCFHkE= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159467627919521.750688146262632; Mon, 13 Jul 2020 14:37:59 -0700 (PDT) Received: from localhost ([::1]:58716 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv697-0004LP-VY for importer@patchew.org; Mon, 13 Jul 2020 17:37:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40434) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv682-0002uF-Gd; Mon, 13 Jul 2020 17:36:50 -0400 Received: from relay.sw.ru ([185.231.240.75]:52828 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv67z-000528-Vs; Mon, 13 Jul 2020 17:36:50 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-RA; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 09/10] qcow2_format.py: collect fields to dump in JSON format Date: Tue, 14 Jul 2020 00:36:42 +0300 Message-Id: <1594676203-436999-10-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" As __dict__ is being extended with class members we do not want to print, make a light copy of the initial __dict__ and extend the copy by adding lists we have to print in the JSON output. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/qcow2_format.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index e0e14b5..83c3482 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -109,6 +109,8 @@ class Qcow2Struct(metaclass=3DQcow2StructMeta): self.__dict__ =3D dict((field[2], values[i]) for i, field in enumerate(self.fields)) =20 + self.fields_dict =3D self.__dict__.copy() + def dump(self, dump_json=3DNone): for f in self.fields: value =3D self.__dict__[f[2]] @@ -144,6 +146,7 @@ class Qcow2BitmapExt(Qcow2Struct): self.bitmap_directory =3D \ [Qcow2BitmapDirEntry(fd, cluster_size=3Dself.cluster_size) for _ in range(self.nb_bitmaps)] + self.fields_dict.update(bitmap_directory=3Dself.bitmap_directory) =20 def dump(self, dump_json=3DNone): super().dump(dump_json) @@ -189,6 +192,7 @@ class Qcow2BitmapDirEntry(Qcow2Struct): table =3D [e[0] for e in struct.iter_unpack('>Q', fd.read(table_si= ze))] self.bitmap_table =3D Qcow2BitmapTable(raw_table=3Dtable, cluster_size=3Dself.cluster_s= ize) + self.fields_dict.update(bitmap_table=3Dself.bitmap_table) =20 def dump(self, dump_json=3DNone): print(f'{"Bitmap name":<25} {self.name}') --=20 1.8.3.1 From nobody Sat Nov 15 10:12:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1594676280; cv=none; d=zohomail.com; s=zohoarc; b=EhiRyyVqICIhHcl1TAlThCH5F25xr3mPcj6NlomX+5M7sAnvehlNOBqa5ILukdFFQMGuEzoN8FekBjql0kwY9tqbctT0JCRO01Ao8cO6Y/wL7cavnWJO+IXzWUnBFbZhbFJOhZtS65LvsJeHzS+Mu30a5llVlnDe34KfDY/QE7A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594676280; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=mDchtUN9Kz5064LmwQB8PWq+L9Jf/4LQLwRVszMIfow=; b=AZGl6LTrNCTmMQgkCmBzwG2mF4tKzgj0lKqQNlbR3nzT+YlegnIJqfcuBd6ygBl/ixMl+3/Q5lMhrYfQ0EBpePqmdzMlPFOY+KBNWj1e2zwTeOeTtL9XNvbKN3XeEsbI/D1Ooah2pXtnMevkrtZtISsQ95m/f6310WusDz/YyhI= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594676280223155.65324356802125; Mon, 13 Jul 2020 14:38:00 -0700 (PDT) Received: from localhost ([::1]:58754 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jv699-0004MO-25 for importer@patchew.org; Mon, 13 Jul 2020 17:37:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40470) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv683-0002vt-Qs; Mon, 13 Jul 2020 17:36:51 -0400 Received: from relay.sw.ru ([185.231.240.75]:52856 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jv680-00052H-00; Mon, 13 Jul 2020 17:36:51 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jv67m-0000E2-Sd; Tue, 14 Jul 2020 00:36:34 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v10 10/10] qcow2_format.py: support dumping metadata in JSON format Date: Tue, 14 Jul 2020 00:36:43 +0300 Message-Id: <1594676203-436999-11-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594676203-436999-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 17:36:45 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Implementation of dumping QCOW2 image metadata. The sample output: { "Header_extensions": [ { "name": "Feature table", "magic": 1745090647, "length": 192, "data_str": "" }, { "name": "Bitmaps", "magic": 595929205, "length": 24, "data": { "nb_bitmaps": 2, "reserved32": 0, "bitmap_directory_size": 64, "bitmap_directory_offset": 1048576, "bitmap_directory": [ { "name": "bitmap-1", "bitmap_table_offset": 589824, "bitmap_table_size": 1, "flags": 2, "type": 1, "granularity_bits": 15, "name_size": 8, "extra_data_size": 0, "bitmap_table": { "entries": [ { "type": "serialized", "offset": 655360 }, ... Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/qcow2_format.py | 59 ++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 59 insertions(+) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_= format.py index 83c3482..a263858 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -19,6 +19,15 @@ =20 import struct import string +import json + + +class ComplexEncoder(json.JSONEncoder): + def default(self, obj): + if hasattr(obj, 'get_fields_dict'): + return obj.get_fields_dict() + else: + return json.JSONEncoder.default(self, obj) =20 =20 class Qcow2Field: @@ -112,6 +121,11 @@ class Qcow2Struct(metaclass=3DQcow2StructMeta): self.fields_dict =3D self.__dict__.copy() =20 def dump(self, dump_json=3DNone): + if dump_json: + print(json.dumps(self.get_fields_dict(), indent=3D4, + cls=3DComplexEncoder)) + return + for f in self.fields: value =3D self.__dict__[f[2]] if isinstance(f[1], str): @@ -154,6 +168,9 @@ class Qcow2BitmapExt(Qcow2Struct): print() entry.dump() =20 + def get_fields_dict(self): + return self.fields_dict + =20 class Qcow2BitmapDirEntry(Qcow2Struct): =20 @@ -199,6 +216,11 @@ class Qcow2BitmapDirEntry(Qcow2Struct): super(Qcow2BitmapDirEntry, self).dump() self.bitmap_table.dump() =20 + def get_fields_dict(self): + bmp_name =3D dict(name=3Dself.name) + bme_dict =3D {**bmp_name, **self.fields_dict} + return bme_dict + =20 class Qcow2BitmapTableEntry: =20 @@ -214,6 +236,9 @@ class Qcow2BitmapTableEntry: else: self.type =3D 'all-zeroes' =20 + def get_fields_dict(self): + return dict(type=3Dself.type, offset=3Dself.offset) + =20 class Qcow2BitmapTable: =20 @@ -230,6 +255,18 @@ class Qcow2BitmapTable: for i, entry in bitmap_table: print(f'{i:<14} {entry.type:<15} {entry.offset:<24} {size}') =20 + def get_fields_dict(self): + return dict(entries=3Dself.entries) + + +class Qcow2HeaderExtensionsDoc: + + def __init__(self, extensions): + self.extensions =3D extensions + + def get_fields_dict(self): + return dict(Header_extensions=3Dself.extensions) + =20 QCOW2_EXT_MAGIC_BITMAPS =3D 0x23852875 =20 @@ -245,6 +282,9 @@ class QcowHeaderExtension(Qcow2Struct): 0x44415441: 'Data file' } =20 + def get_fields_dict(self): + return self.mapping.get(self.value, "") + fields =3D ( ('u32', Magic, 'magic'), ('u32', '{}', 'length') @@ -303,6 +343,16 @@ class QcowHeaderExtension(Qcow2Struct): else: self.obj.dump(dump_json) =20 + def get_fields_dict(self): + ext_name =3D dict(name=3Dself.Magic(self.magic)) + he_dict =3D {**ext_name, **self.fields_dict} + if self.obj is not None: + he_dict.update(data=3Dself.obj) + else: + he_dict.update(data_str=3Dself.data_str) + + return he_dict + @classmethod def create(cls, magic, data): return QcowHeaderExtension(magic, len(data), data) @@ -401,7 +451,16 @@ class QcowHeader(Qcow2Struct): fd.write(buf) =20 def dump_extensions(self, dump_json=3DNone): + if dump_json: + ext_doc =3D Qcow2HeaderExtensionsDoc(self.extensions) + print(json.dumps(ext_doc.get_fields_dict(), indent=3D4, + cls=3DComplexEncoder)) + return + for ex in self.extensions: print('Header extension:') ex.dump(dump_json) print() + + def get_fields_dict(self): + return self.fields_dict --=20 1.8.3.1