From nobody Fri May 17 05:00:01 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1591019660; cv=none; d=zohomail.com; s=zohoarc; b=NfnrGLUKTdZ/razRjX0TFBXqhTx4tgiqURu2LZIZRWTsW28aD0HOU7KJnC8CfrcDoNlMqODy7GfcD10FSylgDUCBMLZNAcsyzw4sUS8fVGk27q2vXWhCVglaQYlnDPaTzEGH+bi03Tf6EIVqsOkLEPVWo8MH61uzUzVNaWMrteU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591019660; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=12W4lJ3LUJMvYATH0U8FIIhVlEdKwXqMy0XkAP/1dA4=; b=Rn5Z4yEsJrNdlNOrMGv6sOpMK6MjFeMXydcWoZpb0bvrCik0GKJ1pCjWz5/aexA32MIMI5wM67Nj6YUzhD7oTDc3v1OEkrON70PBMCUdQ1kHcTWTZRg+PeYoJ7UW+3G7UjENCi6X+qxmtRv+KvjfmRGW9XdqeCFfXP591Lk5f/c= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591019660187929.2201737679019; Mon, 1 Jun 2020 06:54:20 -0700 (PDT) Received: from localhost ([::1]:37260 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfktO-00033D-O3 for importer@patchew.org; Mon, 01 Jun 2020 09:54:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33144) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfkne-0003od-8L; Mon, 01 Jun 2020 09:48:22 -0400 Received: from relay.sw.ru ([185.231.240.75]:38772 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknc-0001k9-1H; Mon, 01 Jun 2020 09:48:21 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jfknN-0005dY-KW; Mon, 01 Jun 2020 16:48:05 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v3 1/6] iotests: Add extension names to qcow2.py dump Date: Mon, 1 Jun 2020 16:48:08 +0300 Message-Id: <1591019293-211155-2-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 09:48:16 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Header extension: Feature table magic 0x6803f857 length 192 data The change incurs modification of the output in 031, 036 and 061 test cases. Signed-off-by: Andrey Shinkevich Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Tested-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/031.out | 22 +++++++++++----------- tests/qemu-iotests/036.out | 4 ++-- tests/qemu-iotests/061.out | 18 +++++++++--------- tests/qemu-iotests/qcow2.py | 23 ++++++++++++++++++++--- 4 files changed, 42 insertions(+), 25 deletions(-) diff --git a/tests/qemu-iotests/031.out b/tests/qemu-iotests/031.out index 5a4beda..966c8d9 100644 --- a/tests/qemu-iotests/031.out +++ b/tests/qemu-iotests/031.out @@ -24,7 +24,7 @@ autoclear_features [] refcount_order 4 header_length 72 =20 -Header extension: +Header extension: Unknown magic 0x12345678 length 31 data 'This is a test header extension' @@ -52,7 +52,7 @@ autoclear_features [] refcount_order 4 header_length 72 =20 -Header extension: +Header extension: Unknown magic 0x12345678 length 31 data 'This is a test header extension' @@ -80,12 +80,12 @@ autoclear_features [] refcount_order 4 header_length 72 =20 -Header extension: +Header extension: Backing format magic 0xe2792aca length 11 data 'host_device' =20 -Header extension: +Header extension: Unknown magic 0x12345678 length 31 data 'This is a test header extension' @@ -115,12 +115,12 @@ autoclear_features [] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data =20 -Header extension: +Header extension: Unknown magic 0x12345678 length 31 data 'This is a test header extension' @@ -148,12 +148,12 @@ autoclear_features [] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data =20 -Header extension: +Header extension: Unknown magic 0x12345678 length 31 data 'This is a test header extension' @@ -181,17 +181,17 @@ autoclear_features [] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Backing format magic 0xe2792aca length 11 data 'host_device' =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data =20 -Header extension: +Header extension: Unknown magic 0x12345678 length 31 data 'This is a test header extension' diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out index e409acf..81a7366 100644 --- a/tests/qemu-iotests/036.out +++ b/tests/qemu-iotests/036.out @@ -24,7 +24,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D67108= 864 incompatible_features [] compatible_features [] autoclear_features [63] -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data @@ -36,7 +36,7 @@ No errors were found on the image. incompatible_features [] compatible_features [] autoclear_features [] -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index a51ad1b..7821b7f 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -24,7 +24,7 @@ autoclear_features [] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data @@ -82,7 +82,7 @@ autoclear_features [] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data @@ -138,7 +138,7 @@ autoclear_features [] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data @@ -193,7 +193,7 @@ autoclear_features [42] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data @@ -262,7 +262,7 @@ autoclear_features [] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data @@ -324,7 +324,7 @@ autoclear_features [] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data @@ -353,7 +353,7 @@ autoclear_features [] refcount_order 4 header_length 112 =20 -Header extension: +Header extension: Feature table magic 0x6803f857 length 336 data @@ -491,7 +491,7 @@ wrote 65536/65536 bytes at offset 2147483648 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 65536/65536 bytes at offset 3221225472 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) - (0.00/100%) (12.50/100%) (25.00/100%) (37.50/100%) (50.00/100%) (62.50/100%) (75.00/100%) (87.50/100%) (100.00/100%) (100.00/100%) + (0.00/100%) (12.50/100%) (25.00/100%) (37.50/100%) (50.00/= 100%) (62.50/100%) (75.00/100%) (87.50/100%) (100.00/100%) (= 100.00/100%) No errors were found on the image. =20 =3D=3D=3D Testing progress report with snapshot =3D=3D=3D @@ -506,7 +506,7 @@ wrote 65536/65536 bytes at offset 2147483648 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 65536/65536 bytes at offset 3221225472 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) - (0.00/100%) (6.25/100%) (12.50/100%) (18.75/100%) (25.00/100%) (31.25/100%) (37.50/100%) (43.75/100%) (50.00/100%) (56.25/100%) (62.50/100%) (68.75/100%) (75.00/100%) (81.25/100%) (87.50/100%) (93.75/100%) (100.00/100%) (100.00/100%) + (0.00/100%) (6.25/100%) (12.50/100%) (18.75/100%) (25.00/1= 00%) (31.25/100%) (37.50/100%) (43.75/100%) (50.00/100%) (56= .25/100%) (62.50/100%) (68.75/100%) (75.00/100%) (81.25/100%) = (87.50/100%) (93.75/100%) (100.00/100%) (100.00/100%) No errors were found on the image. =20 =3D=3D=3D Testing version downgrade with external data file =3D=3D=3D diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index 94a07b2..e824b09 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -6,19 +6,36 @@ import string =20 class QcowHeaderExtension: =20 + QCOW2_EXT_MAGIC_BACKING_FORMAT =3D 0xE2792ACA + QCOW2_EXT_MAGIC_FEATURE_TABLE =3D 0x6803f857 + QCOW2_EXT_MAGIC_CRYPTO_HEADER =3D 0x0537be77 + QCOW2_EXT_MAGIC_BITMAPS =3D 0x23852875 + QCOW2_EXT_MAGIC_DATA_FILE =3D 0x44415441 + def __init__(self, magic, length, data): if length % 8 !=3D 0: padding =3D 8 - (length % 8) data +=3D b"\0" * padding =20 - self.magic =3D magic + self.magic =3D magic self.length =3D length - self.data =3D data + self.data =3D data + self.name =3D self.extension_name(magic) =20 @classmethod def create(cls, magic, data): return QcowHeaderExtension(magic, len(data), data) =20 + def extension_name(self, magic): + return { + self.QCOW2_EXT_MAGIC_BACKING_FORMAT: 'Backing format', + self.QCOW2_EXT_MAGIC_FEATURE_TABLE: 'Feature table', + self.QCOW2_EXT_MAGIC_CRYPTO_HEADER: 'Crypto header', + self.QCOW2_EXT_MAGIC_BITMAPS: 'Bitmaps', + self.QCOW2_EXT_MAGIC_DATA_FILE: 'Data file', + }.get(magic, 'Unknown') + + class QcowHeader: =20 uint32_t =3D 'I' @@ -151,7 +168,7 @@ class QcowHeader: else: data =3D "" =20 - print("Header extension:") + print("%-25s %s" % ("Header extension:", ex.name)) print("%-25s %#x" % ("magic", ex.magic)) print("%-25s %d" % ("length", ex.length)) print("%-25s %s" % ("data", data)) --=20 1.8.3.1 From nobody Fri May 17 05:00:01 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1591019357; cv=none; d=zohomail.com; s=zohoarc; b=nXMd8tpulH4odmpkZE8oZJd5gLndZD8KwSiQmdoDGTZtL2+iQlU+dUzokjHmjV4xEgzyuUJbFNCTBZYgOiJMfKQ/geiWdx6qbeceRyK/6OZxfwl+EcRrKQDAeFcl5nla8hfpOwoLpOO2Y+osi3S/UUPkShuSqw5DAh1pr6wYr6o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591019357; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=5hr1CgicZnWvF+/++dv2vuMD9TOcMTzJVuSONBhPVAI=; b=VcbTp+f8E5AjwqbRZv/K7q9/EofCKTCyO69QIAuaGA192C2uwikW8XBzEQ/1JZD8M7ehxwB7V/BJxYpmfYjv+n9wyw2FcR3yeN02DFQ8geM0yLWyEe98vWVQrEAyJsMB5XE9VFqteL2MYDvcIdL3b+qqvZRmR49rZg/1m3gusQ4= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591019357658579.7657420362466; Mon, 1 Jun 2020 06:49:17 -0700 (PDT) Received: from localhost ([::1]:49180 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfkoV-0004oO-RB for importer@patchew.org; Mon, 01 Jun 2020 09:49:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33140) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknd-0003oP-St; Mon, 01 Jun 2020 09:48:21 -0400 Received: from relay.sw.ru ([185.231.240.75]:38780 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknc-0001k8-4a; Mon, 01 Jun 2020 09:48:21 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jfknN-0005dY-Lf; Mon, 01 Jun 2020 16:48:05 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v3 2/6] iotests: move check for printable data to QcowHeaderExtension class Date: Mon, 1 Jun 2020 16:48:09 +0300 Message-Id: <1591019293-211155-3-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 09:48:16 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Let us differ binary data type from string one for the extension data variable and keep the string as the QcowHeaderExtension class member in the script qcow2.py. Signed-off-by: Andrey Shinkevich Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/qcow2.py | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index e824b09..18e4923 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -13,6 +13,12 @@ class QcowHeaderExtension: QCOW2_EXT_MAGIC_DATA_FILE =3D 0x44415441 =20 def __init__(self, magic, length, data): + data_str =3D data[:length] + if all(c in string.printable.encode('ascii') for c in data_str): + data_str =3D "'%s'" % data_str.decode('ascii') + else: + data_str =3D "" + if length % 8 !=3D 0: padding =3D 8 - (length % 8) data +=3D b"\0" * padding @@ -21,6 +27,7 @@ class QcowHeaderExtension: self.length =3D length self.data =3D data self.name =3D self.extension_name(magic) + self.data_str =3D data_str =20 @classmethod def create(cls, magic, data): @@ -162,16 +169,10 @@ class QcowHeader: def dump_extensions(self): for ex in self.extensions: =20 - data =3D ex.data[:ex.length] - if all(c in string.printable.encode('ascii') for c in data): - data =3D "'%s'" % data.decode('ascii') - else: - data =3D "" - print("%-25s %s" % ("Header extension:", ex.name)) print("%-25s %#x" % ("magic", ex.magic)) print("%-25s %d" % ("length", ex.length)) - print("%-25s %s" % ("data", data)) + print("%-25s %s" % ("data", ex.data_str)) print("") =20 =20 --=20 1.8.3.1 From nobody Fri May 17 05:00:01 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1591019445; cv=none; d=zohomail.com; s=zohoarc; b=D1cLn+uyQEumuXWL/RHi7+9qZcoR8bSHnCxToP2DsCpru2gutW4SqsB2nKuEXopONqSCQBonukOwlVUyPFrYPdblkYjT/8PBGxgbp0O7ixgWj1LQQUlJ9RjL49qzTXKmUSUMaNdrGgrWhzvrd01NQSTU76IOEUttFkCzSWzDGiE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591019445; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=V1JgwRMUuIE5Otk6IEm04DoJVatgb074DNRYUIS5gms=; b=b9CWbsyiUvTMbGLoZFM38YXLq+qUb7SsYpyGT3chKAmPMYqlAY3hu1IzgRyqpXwRkCBV6/5oon/k7nYD3rWWifSTFEZiQpKBfmYAKAZCsFCFO1h3VQ3MpDGh7x6s16rioHhHi43y03KFtxb0HJbDM55Y9SSt35nTlST+SXoKPjQ= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591019445358646.5664117152016; Mon, 1 Jun 2020 06:50:45 -0700 (PDT) Received: from localhost ([::1]:53446 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfkpv-0006Y6-PM for importer@patchew.org; Mon, 01 Jun 2020 09:50:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33142) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknd-0003oQ-UC; Mon, 01 Jun 2020 09:48:21 -0400 Received: from relay.sw.ru ([185.231.240.75]:38778 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknc-0001k5-5H; Mon, 01 Jun 2020 09:48:21 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jfknN-0005dY-Mq; Mon, 01 Jun 2020 16:48:05 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v3 3/6] iotests: dump bitmap extension data with qcow2.py Date: Mon, 1 Jun 2020 16:48:10 +0300 Message-Id: <1591019293-211155-4-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 09:48:16 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add bitmap header extension data, if any, to the dump in qcow2.py. Header extension: Bitmaps magic 0x23852875 length 24 nb_bitmaps 2 reserved32 0 bitmap_directory_size 0x40 bitmap_directory_offset 0x100000 Signed-off-by: Andrey Shinkevich Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/qcow2.py | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index 18e4923..8286115 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -4,6 +4,39 @@ import sys import struct import string =20 + +class Qcow2BitmapExt: + + uint32_t =3D 'I' + uint64_t =3D 'Q' + + fields =3D [ + [uint32_t, '%d', 'nb_bitmaps'], + [uint32_t, '%d', 'reserved32'], + [uint64_t, '%#x', 'bitmap_directory_size'], + [uint64_t, '%#x', 'bitmap_directory_offset'] + ] + + fmt =3D '>' + ''.join(field[0] for field in fields) + + def __init__(self, data): + + extension =3D struct.unpack(Qcow2BitmapExt.fmt, data) + self.__dict__ =3D dict((field[2], extension[i]) + for i, field in enumerate(Qcow2BitmapExt.fiel= ds)) + + def dump_bitmap_ext(self): + for f in Qcow2BitmapExt.fields: + value =3D self.__dict__[f[2]] + value_str =3D f[1] % value + + print("%-25s" % f[2], value_str) + print("") + + def dump_ext(self): + self.dump_bitmap_ext() + + class QcowHeaderExtension: =20 QCOW2_EXT_MAGIC_BACKING_FORMAT =3D 0xE2792ACA @@ -13,12 +46,16 @@ class QcowHeaderExtension: QCOW2_EXT_MAGIC_DATA_FILE =3D 0x44415441 =20 def __init__(self, magic, length, data): + self.obj =3D None data_str =3D data[:length] if all(c in string.printable.encode('ascii') for c in data_str): data_str =3D "'%s'" % data_str.decode('ascii') else: data_str =3D "" =20 + if magic =3D=3D self.QCOW2_EXT_MAGIC_BITMAPS: + self.obj =3D Qcow2BitmapExt(data) + if length % 8 !=3D 0: padding =3D 8 - (length % 8) data +=3D b"\0" * padding @@ -172,7 +209,10 @@ class QcowHeader: print("%-25s %s" % ("Header extension:", ex.name)) print("%-25s %#x" % ("magic", ex.magic)) print("%-25s %d" % ("length", ex.length)) - print("%-25s %s" % ("data", ex.data_str)) + if ex.obj is not None: + ex.obj.dump_ext() + else: + print("%-25s %s" % ("data", ex.data_str)) print("") =20 =20 --=20 1.8.3.1 From nobody Fri May 17 05:00:01 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1591019369; cv=none; d=zohomail.com; s=zohoarc; b=ejs3AnodUJbs7gIN0pyyjHlwp5iwHp+c5uHKQNdyjRz7p8FIREdqW7DYa/aOY51UWgCQSXSf4BW+kXWAGS9ExePY4/NRD9VEeLPUkjWqWihKJb148sFPhVRtmHINxFMOr8pVWY+X0SNV8esX2G/OqIFUNTphvGDM28aAIEhyfhI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591019369; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=pQzaPJP1EdnCS7ouZKIm5DfS+8zMGLkm8FHj5iSdsrM=; b=eqtUk7GqMPqr0aUDrxEn7kWHUE9SHRxV5Pq38+bGTlPN4S2woNUUQDHADhdqscIOZ/Gid0CjIIcxPO/0Xzw2cgrC2SLQZ7Lvj5UIyINV9S/AQr219FAIWCLrpBbOd+x7z2pRBt+EqkVGd4g5HLM56v0azlOpf9d6xd7M0cYsC+g= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159101936939220.603626715150313; Mon, 1 Jun 2020 06:49:29 -0700 (PDT) Received: from localhost ([::1]:49938 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfkoi-000574-19 for importer@patchew.org; Mon, 01 Jun 2020 09:49:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33136) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfkne-0003oO-14; Mon, 01 Jun 2020 09:48:22 -0400 Received: from relay.sw.ru ([185.231.240.75]:38774 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknc-0001k3-4Y; Mon, 01 Jun 2020 09:48:21 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jfknN-0005dY-O0; Mon, 01 Jun 2020 16:48:05 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v3 4/6] iotests: Dump bitmap directory info with qcow2.py Date: Mon, 1 Jun 2020 16:48:11 +0300 Message-Id: <1591019293-211155-5-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 09:48:16 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Read and dump entries from the bitmap directory of QCOW2 image with the script qcow2.py. Header extension: Bitmaps ... Bitmap name bitmap-1 flag auto bitmap_table_offset 0xf0000 bitmap_table_size 8 flag_bits 2 type 1 granularity_bits 16 name_size 8 extra_data_size 0 Suggested-by: Kevin Wolf Signed-off-by: Andrey Shinkevich Reviewed-by: Eric Blake --- tests/qemu-iotests/qcow2.py | 104 ++++++++++++++++++++++++++++++++++++++++= +++- 1 file changed, 103 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index 8286115..e4453f6 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -5,6 +5,88 @@ import struct import string =20 =20 +class Qcow2BitmapDirEntry: + + name =3D '' + + uint8_t =3D 'B' + uint16_t =3D 'H' + uint32_t =3D 'I' + uint64_t =3D 'Q' + + fields =3D [ + [uint64_t, '%#x', 'bitmap_table_offset'], + [uint32_t, '%d', 'bitmap_table_size'], + [uint32_t, '%d', 'flag_bits'], + [uint8_t, '%d', 'type'], + [uint8_t, '%d', 'granularity_bits'], + [uint16_t, '%d', 'name_size'], + [uint32_t, '%d', 'extra_data_size'] + ] + + fmt =3D '>' + ''.join(field[0] for field in fields) + + def __init__(self, data): + + entry =3D struct.unpack(Qcow2BitmapDirEntry.fmt, data) + self.__dict__ =3D dict((field[2], entry[i]) + for i, field in enumerate( + Qcow2BitmapDirEntry.fields)) + + self.bitmap_table_size =3D self.bitmap_table_size \ + * struct.calcsize(self.uint64_t) + + self.bitmap_flags =3D [] + BME_FLAG_IN_USE =3D 1 + BME_FLAG_AUTO =3D 1 << 1 + if (self.flag_bits & BME_FLAG_IN_USE) !=3D 0: + self.bitmap_flags.append("in-use") + if (self.flag_bits & BME_FLAG_AUTO) !=3D 0: + self.bitmap_flags.append("auto") + + def bitmap_dir_entry_raw_size(self): + return struct.calcsize(self.fmt) + self.name_size + \ + self.extra_data_size + + def dump_bitmap_dir_entry(self): + print("%-25s" % 'Bitmap name', self.name) + + for fl in self.bitmap_flags: + print("%-25s" % 'flag', fl) + + for f in Qcow2BitmapDirEntry.fields: + value =3D self.__dict__[f[2]] + value_str =3D f[1] % value + print("%-25s" % f[2], value_str) + + +class Qcow2BitmapDirectory: + + def __init__(self, bm_header_ext): + self.nb_bitmaps =3D bm_header_ext.nb_bitmaps + self.bitmap_directory_offset =3D bm_header_ext.bitmap_directory_of= fset + self.bitmap_directory_size =3D bm_header_ext.bitmap_directory_size + + def read_bitmap_directory(self, fd): + self.bitmaps =3D [] + fd.seek(self.bitmap_directory_offset) + buf_size =3D struct.calcsize(Qcow2BitmapDirEntry.fmt) + + for n in range(self.nb_bitmaps): + buf =3D fd.read(buf_size) + dir_entry =3D Qcow2BitmapDirEntry(buf) + fd.seek(dir_entry.extra_data_size, 1) + bitmap_name =3D fd.read(dir_entry.name_size) + dir_entry.name =3D bitmap_name.decode('ascii') + self.bitmaps.append(dir_entry) + entry_raw_size =3D dir_entry.bitmap_dir_entry_raw_size() + shift =3D ((entry_raw_size + 7) & ~7) - entry_raw_size + fd.seek(shift, 1) + + def get_bitmaps(self): + return self.bitmaps + + class Qcow2BitmapExt: =20 uint32_t =3D 'I' @@ -33,8 +115,21 @@ class Qcow2BitmapExt: print("%-25s" % f[2], value_str) print("") =20 + def read_bitmap_directory(self, fd): + bm_directory =3D Qcow2BitmapDirectory(self) + bm_directory.read_bitmap_directory(fd) + self.bitmaps =3D bm_directory.get_bitmaps() + + def load(self, fd): + self.read_bitmap_directory(fd) + + def dump_bitmap_directory(self): + for bm in self.bitmaps: + bm.dump_bitmap_dir_entry() + def dump_ext(self): self.dump_bitmap_ext() + self.dump_bitmap_directory() =20 =20 class QcowHeaderExtension: @@ -79,6 +174,10 @@ class QcowHeaderExtension: self.QCOW2_EXT_MAGIC_DATA_FILE: 'Data file', }.get(magic, 'Unknown') =20 + def load(self, fd): + if self.obj is not None: + self.obj.load(fd) + =20 class QcowHeader: =20 @@ -157,7 +256,10 @@ class QcowHeader: else: padded =3D (length + 7) & ~7 data =3D fd.read(padded) - self.extensions.append(QcowHeaderExtension(magic, length, = data)) + self.extensions.append(QcowHeaderExtension(magic, length, + data)) + for ex in self.extensions: + ex.load(fd) =20 def update_extensions(self, fd): =20 --=20 1.8.3.1 From nobody Fri May 17 05:00:01 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1591019467; cv=none; d=zohomail.com; s=zohoarc; b=alv1BpWbzXJLEEJWcDsjeEuxMlIa4/QJtCgkTErCJ8ZRRCD3SDILZMN44zTgIF9V4JWIjb98hNp/+QDXRhz5NOgSZcnot2qaxQOXZz0tByZkxWfWlTy0ZcVE2IrdD6p2zNX1HliGV3EKVRIxxep8qcuqNi5SDK/tEmi/W+8eV+c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591019467; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=GZNfc9NuikkC+4HQGohqR6+vyRCCk1TyngLwPiupT7g=; b=WWnNJ7LQsKihv2psNq5OjoRAtMU+ghIB7hVludBgDhQ1oP8rB5KOuXZH8Ee5quLB3bGG6L2QyMKkxUnqieu+0ybArAJN9XicqUQGG7O2/EXZIQ1yi33V4ks8PWzUbKje4AC4a6SaBLVrVONDspSdoyQEzhmYVNbT3PW6vkcfRZI= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591019467497916.2471842219248; Mon, 1 Jun 2020 06:51:07 -0700 (PDT) Received: from localhost ([::1]:55122 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfkqI-0007Iz-8N for importer@patchew.org; Mon, 01 Jun 2020 09:51:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33148) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknf-0003pA-4a; Mon, 01 Jun 2020 09:48:23 -0400 Received: from relay.sw.ru ([185.231.240.75]:38776 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknc-0001k7-8G; Mon, 01 Jun 2020 09:48:22 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jfknN-0005dY-PD; Mon, 01 Jun 2020 16:48:05 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v3 5/6] iotests: Dump bitmap table entries serialized in QCOW2 image Date: Mon, 1 Jun 2020 16:48:12 +0300 Message-Id: <1591019293-211155-6-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 09:48:16 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add bitmap table info to the QCOW2 metadata dump with qcow2.py. Bitmap name bitmap-1 ... itmap table type offset size 0 serialized 0xa0000 65536 1 all-zeroes 0x0 65536 2 all-zeroes 0x0 65536 3 all-zeroes 0x0 65536 4 all-zeroes 0x0 65536 5 all-zeroes 0x0 65536 6 all-zeroes 0x0 65536 7 all-zeroes 0x0 65536 Signed-off-by: Andrey Shinkevich Reviewed-by: Eric Blake --- tests/qemu-iotests/qcow2.py | 48 +++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 48 insertions(+) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index e4453f6..76e0c69 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -5,6 +5,41 @@ import struct import string =20 =20 +cluster_size =3D 0 + + +class Qcow2BitmapTableEntry: + + BME_TABLE_ENTRY_OFFSET_MASK =3D 0x00fffffffffffe00 + BME_TABLE_ENTRY_FLAG_ALL_ONES =3D 1 + bmte_type =3D ['all-zeroes', 'all-ones', 'serialized'] + + def __init__(self, entry): + self.cluster_size =3D cluster_size + self.offset =3D entry & self.BME_TABLE_ENTRY_OFFSET_MASK + if self.offset !=3D 0: + index =3D 2 + else: + index =3D entry & self.BME_TABLE_ENTRY_FLAG_ALL_ONES + self.type =3D self.bmte_type[index] + + +class Qcow2BitmapTable: + + def __init__(self, raw_table): + self.entries =3D [] + for entry in raw_table: + self.entries.append(Qcow2BitmapTableEntry(entry)) + + def print_bitmap_table(self): + bitmap_table =3D enumerate(self.entries) + print("Bitmap table\ttype\t\toffset\t\tsize") + for i, entry in bitmap_table: + print("\t%-4d\t%s\t%#x\t\t%d" % (i, entry.type, entry.offset, + entry.cluster_size)) + print("") + + class Qcow2BitmapDirEntry: =20 name =3D '' @@ -48,6 +83,12 @@ class Qcow2BitmapDirEntry: return struct.calcsize(self.fmt) + self.name_size + \ self.extra_data_size =20 + def read_bitmap_table(self, fd): + fd.seek(self.bitmap_table_offset) + table_size =3D self.bitmap_table_size * struct.calcsize(self.uint6= 4_t) + table =3D [e[0] for e in struct.iter_unpack('>Q', fd.read(table_si= ze))] + self.bitmap_table =3D Qcow2BitmapTable(table) + def dump_bitmap_dir_entry(self): print("%-25s" % 'Bitmap name', self.name) =20 @@ -59,6 +100,8 @@ class Qcow2BitmapDirEntry: value_str =3D f[1] % value print("%-25s" % f[2], value_str) =20 + self.bitmap_table.print_bitmap_table() + =20 class Qcow2BitmapDirectory: =20 @@ -83,6 +126,9 @@ class Qcow2BitmapDirectory: shift =3D ((entry_raw_size + 7) & ~7) - entry_raw_size fd.seek(shift, 1) =20 + for bm in self.bitmaps: + bm.read_bitmap_table(fd) + def get_bitmaps(self): return self.bitmaps =20 @@ -223,6 +269,8 @@ class QcowHeader: =20 self.set_defaults() self.cluster_size =3D 1 << self.cluster_bits + global cluster_size + cluster_size =3D self.cluster_size =20 fd.seek(self.header_length) self.load_extensions(fd) --=20 1.8.3.1 From nobody Fri May 17 05:00:01 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1591019585; cv=none; d=zohomail.com; s=zohoarc; b=UYadkSZH3Q6U/TrfDW9xI/GpZhbmeGz8LPnZBn4jlsKWeZUwq9wd5r3k5oslRG9dbPFTEvU4MxrcKx7gtbFHjYEX3o+GInv5F0BqB3sBosi6Qvj7BHZejurXpDvSfLTuO7Londvsxyw5lBAixXliLGPDfYZUDgm93D1eIivzxw0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591019585; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=4E8IAG/uHBA7dCbwkNC6NXtdET6S7Gfrt7TjP6+S6CU=; b=GA56UdwYy5EcW1Apc5ya+cH8pvqRBMysFkG709v9WU7hbrHZ/20nqZmqP3QnimTtcWujPUuW9eSCtQwju3XbAdXq7Z9odM4WIoVke5G0h719zY0ZlxDqZCS3kgqguWm8rh6xtc/W9z4oHElBCCRsgXYj40MMnbh7OdfUCjsdhw4= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591019585072867.0530944264149; Mon, 1 Jun 2020 06:53:05 -0700 (PDT) Received: from localhost ([::1]:33298 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfksB-0001SY-Sk for importer@patchew.org; Mon, 01 Jun 2020 09:53:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33152) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknf-0003pp-EV; Mon, 01 Jun 2020 09:48:23 -0400 Received: from relay.sw.ru ([185.231.240.75]:38782 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfknc-0001k4-50; Mon, 01 Jun 2020 09:48:23 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jfknN-0005dY-Qd; Mon, 01 Jun 2020 16:48:05 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v3 6/6] iotests: Dump QCOW2 image metadata in JSON format with qcow2.py Date: Mon, 1 Jun 2020 16:48:13 +0300 Message-Id: <1591019293-211155-7-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1591019293-211155-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 09:48:16 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Represent QCOW2 metadata dumping with qcow2.py script in JSON format { "QCOW2_header_extensions": [ { "Header_extension": "Feature table", "magic": "0x6803f857", "length": 192, "data_str": "" }, { "Header_extension": "Bitmaps", "magic": "0x23852875", "length": 24, "data": { "nb_bitmaps": 2, "reserved32": 0, "bitmap_directory_size": 64, "bitmap_directory_offset": 1048576, "entries": [ { "name": "bitmap-1", "flags": [], "flag_bits": 0, "bitmap_table_offset": 589824, "bitmap_table_size": 8, "type": 1, "granularity": 16, "name_size": 8, "extra_data_size": 0, "bitmap_table": { "table_entries": [ { "type": "serialized", "offset": 655360, "size": 65536 }, Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Eric Blake --- tests/qemu-iotests/qcow2.py | 108 ++++++++++++++++++++++++++++++++++++++++= ++-- 1 file changed, 105 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index 76e0c69..fd1ef4f 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -3,11 +3,21 @@ import sys import struct import string +import json =20 =20 +dump_json =3D False cluster_size =3D 0 =20 =20 +class ComplexEncoder(json.JSONEncoder): + def default(self, obj): + if hasattr(obj, 'get_info_dict'): + return obj.get_info_dict() + else: + return json.JSONEncoder.default(self, obj) + + class Qcow2BitmapTableEntry: =20 BME_TABLE_ENTRY_OFFSET_MASK =3D 0x00fffffffffffe00 @@ -23,6 +33,9 @@ class Qcow2BitmapTableEntry: index =3D entry & self.BME_TABLE_ENTRY_FLAG_ALL_ONES self.type =3D self.bmte_type[index] =20 + def get_info_dict(self): + return dict(type=3Dself.type, offset=3Dself.offset, size=3Dself.cl= uster_size) + =20 class Qcow2BitmapTable: =20 @@ -39,6 +52,9 @@ class Qcow2BitmapTable: entry.cluster_size)) print("") =20 + def get_info_dict(self): + return dict(table_entries=3Dself.entries) + =20 class Qcow2BitmapDirEntry: =20 @@ -102,6 +118,18 @@ class Qcow2BitmapDirEntry: =20 self.bitmap_table.print_bitmap_table() =20 + def get_info_dict(self): + return dict(name=3Dself.name, + flags=3Dself.bitmap_flags, + flag_bits=3Dself.flag_bits, + bitmap_table_offset=3Dself.bitmap_table_offset, + bitmap_table_size=3Dself.bitmap_table_size, + type=3Dself.type, + granularity=3Dself.granularity_bits, + name_size=3Dself.name_size, + extra_data_size=3Dself.extra_data_size, + bitmap_table=3Dself.bitmap_table) + =20 class Qcow2BitmapDirectory: =20 @@ -177,6 +205,31 @@ class Qcow2BitmapExt: self.dump_bitmap_ext() self.dump_bitmap_directory() =20 + def get_info_dict(self): + return dict(nb_bitmaps=3Dself.nb_bitmaps, + reserved32=3Dself.reserved32, + bitmap_directory_size=3Dself.bitmap_directory_size, + bitmap_directory_offset=3Dself.bitmap_directory_offset, + entries=3Dself.bitmaps) + + +class Qcow2HeaderDoc: + + def __init__(self, h): + self.header =3D h + + def get_info_dict(self): + return dict(QCOW2_header=3Dself.header) + + +class Qcow2HeaderExtensionsDoc: + + def __init__(self, extensions): + self.extensions =3D extensions + + def get_info_dict(self): + return dict(QCOW2_header_extensions=3Dself.extensions) + =20 class QcowHeaderExtension: =20 @@ -224,6 +277,17 @@ class QcowHeaderExtension: if self.obj is not None: self.obj.load(fd) =20 + def get_info_dict(self): + he_dict =3D dict(Header_extension=3Dself.name, + magic=3Dhex(self.magic), + length=3Dself.length) + if self.obj is not None: + he_dict.update(data=3Dself.obj) + else: + he_dict.update(data_str=3Dself.data_str) + + return he_dict + =20 class QcowHeader: =20 @@ -353,9 +417,34 @@ class QcowHeader: print("%-25s" % f[2], value_str) print("") =20 + def get_info_dict(self): + return dict(magic=3Dhex(self.magic), + version=3Dself.version, + backing_file_offset=3Dhex(self.backing_file_offset), + backing_file_size=3Dself.backing_file_size, + cluster_bits=3Dself.cluster_bits, + size=3Dself.size, + crypt_method=3Dself.crypt_method, + l1_size=3Dself.l1_size, + l1_table_offset=3Dhex(self.l1_table_offset), + refcount_table_offset=3Dhex(self.refcount_table_offset= ), + refcount_table_clusters=3Dself.refcount_table_clusters, + nb_snapshots=3Dself.nb_snapshots, + snapshot_offset=3Dhex(self.snapshot_offset), + incompatible_features=3Dself.incompatible_features, + compatible_features=3Dself.compatible_features, + autoclear_features=3Dself.autoclear_features, + refcount_order=3Dself.refcount_order, + header_length=3Dself.header_length) + def dump_extensions(self): - for ex in self.extensions: + if dump_json: + ext_doc =3D Qcow2HeaderExtensionsDoc(self.extensions) + print(json.dumps(ext_doc.get_info_dict(), indent=3D4, + cls=3DComplexEncoder)) + return =20 + for ex in self.extensions: print("%-25s %s" % ("Header extension:", ex.name)) print("%-25s %#x" % ("magic", ex.magic)) print("%-25s %d" % ("length", ex.length)) @@ -368,7 +457,11 @@ class QcowHeader: =20 def cmd_dump_header(fd): h =3D QcowHeader(fd) - h.dump() + if dump_json: + h_doc =3D Qcow2HeaderDoc(h) + print(json.dumps(h_doc.get_info_dict(), indent=3D4, cls=3DComplexE= ncoder)) + else: + h.dump() h.dump_extensions() =20 def cmd_dump_header_exts(fd): @@ -460,6 +553,12 @@ cmds =3D [ ] =20 def main(filename, cmd, args): + global dump_json + dump_json =3D '-j' in sys.argv + if dump_json: + sys.argv.remove('-j') + args.remove('-j') + fd =3D open(filename, "r+b") try: for name, handler, num_args, desc in cmds: @@ -476,11 +575,14 @@ def main(filename, cmd, args): fd.close() =20 def usage(): - print("Usage: %s [, ...]" % sys.argv[0]) + print("Usage: %s [, ...] [, ...]" % sys.argv[0]) print("") print("Supported commands:") for name, handler, num_args, desc in cmds: print(" %-20s - %s" % (name, desc)) + print("") + print("Supported keys:") + print(" %-20s - %s" % ('-j', 'Dump in JSON format')) =20 if __name__ =3D=3D '__main__': if len(sys.argv) < 3: --=20 1.8.3.1