From nobody Sun Apr 28 23:44:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1502899853399441.9237649163289; Wed, 16 Aug 2017 09:10:53 -0700 (PDT) Received: from localhost ([::1]:57158 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1di0uB-000415-PS for importer@patchew.org; Wed, 16 Aug 2017 12:10:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33159) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1di0sb-0002v3-NT for qemu-devel@nongnu.org; Wed, 16 Aug 2017 12:09:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1di0sX-0007Gh-3c for qemu-devel@nongnu.org; Wed, 16 Aug 2017 12:09:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39768) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1di0sW-0007GJ-QS for qemu-devel@nongnu.org; Wed, 16 Aug 2017 12:09:09 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EFC58C058EC2 for ; Wed, 16 Aug 2017 15:59:34 +0000 (UTC) Received: from dverma.bos.com (dhcp-17-188.bos.redhat.com [10.18.17.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 52A2F60BE6; Wed, 16 Aug 2017 15:59:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com EFC58C058EC2 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=dverma@redhat.com From: dverma To: qemu-devel@nongnu.org Date: Wed, 16 Aug 2017 11:59:26 -0400 Message-Id: <1502899168-19852-2-git-send-email-dverma@redhat.com> In-Reply-To: <1502899168-19852-1-git-send-email-dverma@redhat.com> References: <1502899168-19852-1-git-send-email-dverma@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 16 Aug 2017 15:59:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 1/3] Fix format and styles; make code more pythonic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: dverma , dgilbert@redhat.com, jen@redhat.com, quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" - Format fixes, cleaned up the print statement - Style fixes, e.g. changed "if not x in y" to "if x not in y" - Improved variable names Changes v1->v2: 1. Fix patchew warnings about exceeding 80 characters Signed-off-by: Deepak Verma --- scripts/vmstate-static-checker.py | 111 +++++++++++++++++++++-------------= ---- 1 file changed, 62 insertions(+), 49 deletions(-) diff --git a/scripts/vmstate-static-checker.py b/scripts/vmstate-static-che= cker.py index bcef7ee..b416b66 100755 --- a/scripts/vmstate-static-checker.py +++ b/scripts/vmstate-static-checker.py @@ -19,6 +19,11 @@ # You should have received a copy of the GNU General Public License along # with this program; if not, see . =20 +# +# 2017 Deepak Verma +# Added few functions and fields for whitelisting +# + import argparse import json import sys @@ -26,6 +31,7 @@ import sys # Count the number of errors found taint =3D 0 =20 + def bump_taint(): global taint =20 @@ -92,7 +98,7 @@ def check_fields_match(name, s_field, d_field): 'io_win_size', 'mig_io_win_size'], } =20 - if not name in changed_names: + if name not in changed_names: return False =20 if s_field in changed_names[name] and d_field in changed_names[name]: @@ -100,6 +106,7 @@ def check_fields_match(name, s_field, d_field): =20 return False =20 + def get_changed_sec_name(sec): # Section names can change -- see commit 292b1634 for an example. changes =3D { @@ -114,16 +121,17 @@ def get_changed_sec_name(sec): return item return "" =20 + def exists_in_substruct(fields, item): # Some QEMU versions moved a few fields inside a substruct. This # kept the on-wire format the same. This function checks if # something got shifted inside a substruct. For example, the # change in commit 1f42d22233b4f3d1a2933ff30e8d6a6d9ee2d08f =20 - if not "Description" in fields: + if "Description" not in fields: return False =20 - if not "Fields" in fields["Description"]: + if "Fields" not in fields["Description"]: return False =20 substruct_fields =3D fields["Description"]["Fields"] @@ -176,10 +184,10 @@ def check_fields(src_fields, dest_fields, desc, sec): except StopIteration: if d_iter_list =3D=3D []: # We were not in a substruct - print "Section \"" + sec + "\",", - print "Description " + "\"" + desc + "\":", - print "expected field \"" + s_item["field"] + "\",", - print "while dest has no further fields" + print('Section "' + sec + '", ' + 'Description "' + desc + '": ' + 'expected field "' + s_item["field"] + '", ' + 'while dest has no further fields') bump_taint() break =20 @@ -191,30 +199,28 @@ def check_fields(src_fields, dest_fields, desc, sec): advance_dest =3D True =20 if unused_count !=3D 0: - if advance_dest =3D=3D False: + if not advance_dest: unused_count =3D unused_count - s_item["size"] if unused_count =3D=3D 0: advance_dest =3D True continue if unused_count < 0: - print "Section \"" + sec + "\",", - print "Description \"" + desc + "\":", - print "unused size mismatch near \"", - print s_item["field"] + "\"" + print('Section "' + sec + '", ' + 'Description "' + desc + '": ' + 'unused size mismatch near "' + s_item["field"] = + '"') bump_taint() break continue =20 - if advance_src =3D=3D False: + if not advance_src: unused_count =3D unused_count - d_item["size"] if unused_count =3D=3D 0: advance_src =3D True continue if unused_count < 0: - print "Section \"" + sec + "\",", - print "Description \"" + desc + "\":", - print "unused size mismatch near \"", - print d_item["field"] + "\"" + print('Section "' + sec + '", ' + 'Description "' + desc + '": ' + 'unused size mismatch near "' + d_item["field"] = + '"') bump_taint() break continue @@ -262,16 +268,16 @@ def check_fields(src_fields, dest_fields, desc, sec): unused_count =3D s_item["size"] - d_item["size"] continue =20 - print "Section \"" + sec + "\",", - print "Description \"" + desc + "\":", - print "expected field \"" + s_item["field"] + "\",", - print "got \"" + d_item["field"] + "\"; skipping rest" + print('Section "' + sec + '", ' + 'Description "' + desc + '": ' + 'expected field "' + s_item["field"] + '", ' + 'got "' + d_item["field"] + '"; skipping rest') bump_taint() break =20 check_version(s_item, d_item, sec, desc) =20 - if not "Description" in s_item: + if "Description" not in s_item: # Check size of this field only if it's not a VMSTRUCT entry check_size(s_item, d_item, sec, desc, s_item["field"]) =20 @@ -289,18 +295,20 @@ def check_subsections(src_sub, dest_sub, desc, sec): check_descriptions(s_item, d_item, sec) =20 if not found: - print "Section \"" + sec + "\", Description \"" + desc + "\":", - print "Subsection \"" + s_item["name"] + "\" not found" + print('Section "' + sec + '", ' + 'Description "' + desc + '": ' + 'Subsection "' + s_item["name"] + '" not found') bump_taint() =20 =20 def check_description_in_list(s_item, d_item, sec, desc): - if not "Description" in s_item: + if "Description" not in s_item: return =20 - if not "Description" in d_item: - print "Section \"" + sec + "\", Description \"" + desc + "\",", - print "Field \"" + s_item["field"] + "\": missing description" + if "Description" not in d_item: + print('Section "' + sec + '", ' + 'Description "' + desc + '", ' + 'Field "' + s_item["field"] + '": missing description') bump_taint() return =20 @@ -311,17 +319,17 @@ def check_descriptions(src_desc, dest_desc, sec): check_version(src_desc, dest_desc, sec, src_desc["name"]) =20 if not check_fields_match(sec, src_desc["name"], dest_desc["name"]): - print "Section \"" + sec + "\":", - print "Description \"" + src_desc["name"] + "\"", - print "missing, got \"" + dest_desc["name"] + "\" instead; skippin= g" + print('Section "' + sec + '": ' + 'Description "' + src_desc["name"] + '" ' + 'missing, got "' + dest_desc["name"] + '" instead; skipping') bump_taint() return =20 for f in src_desc: if not f in dest_desc: - print "Section \"" + sec + "\"", - print "Description \"" + src_desc["name"] + "\":", - print "Entry \"" + f + "\" missing" + print('Section "' + sec + '" ' + 'Description "' + src_desc["name"] + '": ' + 'Entry "' + field + '" missing') bump_taint() continue =20 @@ -340,15 +348,15 @@ def check_version(s, d, sec, desc=3DNone): print "version error:", s["version_id"], ">", d["version_id"] bump_taint() =20 - if not "minimum_version_id" in d: + if "minimum_version_id" not in dest_ver: return =20 if s["version_id"] < d["minimum_version_id"]: print "Section \"" + sec + "\"", if desc: - print "Description \"" + desc + "\":", - print "minimum version error:", s["version_id"], "<", - print d["minimum_version_id"] + print('Description "' + desc + '": ' + + 'minimum version error: ' + str(src_ver["version_id"]) += ' < ' + + str(dest_ver["minimum_version_id"])) bump_taint() =20 =20 @@ -363,15 +371,21 @@ def check_size(s, d, sec, desc=3DNone, field=3DNone): bump_taint() =20 =20 -def check_machine_type(s, d): - if s["Name"] !=3D d["Name"]: - print "Warning: checking incompatible machine types:", - print "\"" + s["Name"] + "\", \"" + d["Name"] + "\"" + +def check_machine_type(src, dest): + if src["Name"] !=3D dest["Name"]: + print('Warning: checking incompatible machine types: ' + '"' + src["Name"] + '", "' + dest["Name"] + '"') return =20 =20 def main(): - help_text =3D "Parse JSON-formatted vmstate dumps from QEMU in files S= RC and DEST. Checks whether migration from SRC to DEST QEMU versions would= break based on the VMSTATE information contained within the JSON outputs. = The JSON output is created from a QEMU invocation with the -dump-vmstate p= arameter and a filename argument to it. Other parameters to QEMU do not ma= tter, except the -M (machine type) parameter." + help_text =3D ("Parse JSON-formatted vmstate dumps from QEMU in files " + "SRC and DEST. Checks whether migration from SRC to DEST QEMU version= s " + "would break based on the VMSTATE information contained within the JSO= N " + "outputs. The JSON output is created from a QEMU invocation with the " + "-dump-vmstate parameter and a filename argument to it. Other paramete= rs to " + "QEMU do not matter, except the -M (machine type) parameter.") =20 parser =3D argparse.ArgumentParser(description=3Dhelp_text) parser.add_argument('-s', '--src', type=3Dfile, required=3DTrue, @@ -395,12 +409,12 @@ def main(): =20 for sec in src_data: dest_sec =3D sec - if not dest_sec in dest_data: + if dest_sec not in dest_data: # Either the section name got changed, or the section # doesn't exist in dest. dest_sec =3D get_changed_sec_name(sec) - if not dest_sec in dest_data: - print "Section \"" + sec + "\" does not exist in dest" + if dest_sec not in dest_data: + print('Section "' + sec + '" does not exist in dest') bump_taint() continue =20 @@ -414,9 +428,8 @@ def main(): check_version(s, d, sec) =20 for entry in s: - if not entry in d: - print "Section \"" + sec + "\": Entry \"" + entry + "\"", - print "missing" + if entry not in d: + print('Section "' + sec + '": Entry "' + entry + '" missin= g') bump_taint() continue =20 --=20 1.8.3.1 From nobody Sun Apr 28 23:44:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1502899914079581.6643053945054; Wed, 16 Aug 2017 09:11:54 -0700 (PDT) Received: from localhost ([::1]:57230 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1di0vA-0004vL-Qu for importer@patchew.org; Wed, 16 Aug 2017 12:11:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60767) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1di0q8-0000gv-32 for qemu-devel@nongnu.org; Wed, 16 Aug 2017 12:06:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1di0q2-0006T5-1V for qemu-devel@nongnu.org; Wed, 16 Aug 2017 12:06:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35120) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1di0q1-0006St-RN for qemu-devel@nongnu.org; Wed, 16 Aug 2017 12:06:33 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 42A57404325 for ; Wed, 16 Aug 2017 15:59:37 +0000 (UTC) Received: from dverma.bos.com (dhcp-17-188.bos.redhat.com [10.18.17.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 19807619CF; Wed, 16 Aug 2017 15:59:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 42A57404325 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=dverma@redhat.com From: dverma To: qemu-devel@nongnu.org Date: Wed, 16 Aug 2017 11:59:27 -0400 Message-Id: <1502899168-19852-3-git-send-email-dverma@redhat.com> In-Reply-To: <1502899168-19852-1-git-send-email-dverma@redhat.com> References: <1502899168-19852-1-git-send-email-dverma@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 16 Aug 2017 15:59:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 2/3] Update the existing whitelist X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: dverma , dgilbert@redhat.com, jen@redhat.com, quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Appended newer fields and introduced new names in the whitelist Changes v1->v2: 1. Fix patchew warnings about exceeding 80 characters Signed-off-by: Deepak Verma --- scripts/vmstate-static-checker.py | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/scripts/vmstate-static-checker.py b/scripts/vmstate-static-che= cker.py index b416b66..ae41e44 100755 --- a/scripts/vmstate-static-checker.py +++ b/scripts/vmstate-static-checker.py @@ -49,7 +49,6 @@ def check_fields_match(name, s_field, d_field): # is used to whitelist such changes in each section / description. changed_names =3D { 'apic': ['timer', 'timer_expiry'], - 'e1000': ['dev', 'parent_obj'], 'ehci': ['dev', 'pcidev'], 'I440FX': ['dev', 'parent_obj'], 'ich9_ahci': ['card', 'parent_obj'], @@ -73,7 +72,6 @@ def check_fields_match(name, s_field, d_field): 'tmr.timer', 'ar.tmr.timer', 'tmr.overflow_time', 'ar.tmr.overflow_time', 'gpe', 'ar.gpe'], - 'rtl8139': ['dev', 'parent_obj'], 'qxl': ['num_surfaces', 'ssd.num_surfaces'], 'usb-ccid': ['abProtocolDataStructure', 'abProtocolDataStructure.d= ata'], 'usb-host': ['dev', 'parent_obj'], @@ -96,6 +94,26 @@ def check_fields_match(name, s_field, d_field): 'mem_win_size', 'mig_mem_win_size', 'io_win_addr', 'mig_io_win_addr', 'io_win_size', 'mig_io_win_size'], + 'rtl8139': ['dev', 'parent_obj'], + 'e1000e': ['PCIDevice', 'PCIEDevice', 'intr_state', 'redhat_7_3_in= tr_state'], + 'nec-usb-xhci': ['PCIDevice', 'PCIEDevice'], + 'xhci-intr': ['er_full_unused', 'er_full'], + 'e1000': ['dev', 'parent_obj', + 'tx.ipcss', 'tx.props.ipcss', + 'tx.ipcso', 'tx.props.ipcso', + 'tx.ipcse', 'tx.props.ipcse', + 'tx.tucss', 'tx.props.tucss', + 'tx.tucso', 'tx.props.tucso', + 'tx.tucse', 'tx.props.tucse', + 'tx.paylen', 'tx.props.paylen', + 'tx.hdr_len', 'tx.props.hdr_len', + 'tx.mss', 'tx.props.mss', + 'tx.sum_needed', 'tx.props.sum_needed', + 'tx.ip', 'tx.props.ip', + 'tx.tcp', 'tx.props.tcp', + 'tx.ipcss', 'tx.props.ipcss', + 'tx.ipcss', 'tx.props.ipcss', + ] } =20 if name not in changed_names: --=20 1.8.3.1 From nobody Sun Apr 28 23:44:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 150290002635676.7407602817658; Wed, 16 Aug 2017 09:13:46 -0700 (PDT) Received: from localhost ([::1]:57377 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1di0wy-0006Ov-UN for importer@patchew.org; Wed, 16 Aug 2017 12:13:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32912) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1di0s3-0002Nj-9f for qemu-devel@nongnu.org; Wed, 16 Aug 2017 12:08:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1di0s0-000751-4C for qemu-devel@nongnu.org; Wed, 16 Aug 2017 12:08:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59182) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1di0rz-00074o-Qs for qemu-devel@nongnu.org; Wed, 16 Aug 2017 12:08:36 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 46B35C05A1DB for ; Wed, 16 Aug 2017 15:59:38 +0000 (UTC) Received: from dverma.bos.com (dhcp-17-188.bos.redhat.com [10.18.17.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 648D5619CF; Wed, 16 Aug 2017 15:59:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 46B35C05A1DB Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=dverma@redhat.com From: dverma To: qemu-devel@nongnu.org Date: Wed, 16 Aug 2017 11:59:28 -0400 Message-Id: <1502899168-19852-4-git-send-email-dverma@redhat.com> In-Reply-To: <1502899168-19852-1-git-send-email-dverma@redhat.com> References: <1502899168-19852-1-git-send-email-dverma@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 16 Aug 2017 15:59:38 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 3/3] Add new functions for whitelisting and their calls X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: dverma , dgilbert@redhat.com, jen@redhat.com, quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The 'check_updated_properties' function keeps track of properties that were added/removed from fields across qemu versions. The 'check_updated_sizes' function reduces false positives generated especially while testing backward migration by keeping a list of common size/version changes. The 'check_new_sections' function is used to check for sections that got deprecated or were introduced in different versions of qemu and will show as false positives while testing forward migration. Improved the variable names and added multiple blank newlines to keep Python PEP8 warning away. Changes v1->v2: 1. Fix patchew warnings about exceeding 80 characters Signed-off-by: Deepak Verma --- scripts/vmstate-static-checker.py | 254 ++++++++++++++++++++++++++++++----= ---- 1 file changed, 200 insertions(+), 54 deletions(-) diff --git a/scripts/vmstate-static-checker.py b/scripts/vmstate-static-che= cker.py index ae41e44..ebcc133 100755 --- a/scripts/vmstate-static-checker.py +++ b/scripts/vmstate-static-checker.py @@ -40,6 +40,108 @@ def bump_taint(): if taint < 255: taint =3D taint + 1 =20 +# Sections gain/lose new fields with time. +# These are not name changes thats handled by another list. +# These will be 'missing' or 'not found' in different versions of qemu + + +def check_updated_properties(src_desc, field): + src_desc =3D str(src_desc) + field =3D str(field) + updated_property =3D { + 'ICH9LPC': ['ICH9LPC/smi_feat'], + 'ide_bus/error': ['retry_sector_num', 'retry_nsector', 'retry_unit'], + 'e1000': ['e1000/full_mac_state'], + 'ich9_pm': ['ich9_pm/tco', 'ich9_pm/cpuhp'] + } + + if src_desc in updated_property and field in updated_property[src_desc= ]: + return True + + return False + + +# A lot of errors are generated due to differences in sizes some of which = are +# false positives. This list is used to save those common changes +def check_updated_sizes(field, old_size, new_size): + new_sizes_list =3D { + 'tally_counters.TxOk': [8, 64], + 'intel-iommu': [0, 1], + 'iommu-intel': [0, 1] + } + + if field not in new_sizes_list: + return False + + if(old_size in new_sizes_list[field] and new_size in + new_sizes_list[field]): + return True + + return False + + +# With time new sections/hardwares supported and old ones are depreciated = on +# chipsets. +# There is no separate list for new or dead sections as it's relative to w= hich +# qemu version you compare too. +# Update this list with such sections. +# some items in this list might overlap with changed sections names. +def check_new_sections(sec): + new_sections_list =3D [ + 'virtio-balloon-device', + 'virtio-rng-device', + 'virtio-scsi-device', + 'virtio-blk-device', + 'virtio-serial-device', + 'virtio-net-device', + 'vhost-vsock-device', + 'virtio-input-host-device', + 'virtio-input-hid-device', + 'virtio-mouse-device', + 'virtio-keyboard-device', + 'virtio-vga', + 'virtio-input-device', + 'virtio-gpu-device', + 'virtio-tablet-device', + 'isa-pcspk', + 'qemu-xhci', + 'base-xhci', + 'vmgenid', + 'intel-iommu', + 'i8257', + 'i82801b11-bridge', + 'ivshmem', + 'ivshmem-doorbell', + 'ivshmem-plain', + 'usb-storage-device', + 'usb-storage-dev', + 'pci-qxl', + 'pci-uhci-usb', + 'pci-piix3', + 'pci-vga', + 'pci-bridge-seat', + 'pcie-root-port', + 'fw_cfg_io', + 'fw_cfg_mem', + 'exynos4210-ehci-usb', + 'sysbus-ehci-usb', + 'tegra2-ehci-usb', + 'kvm-apic', + 'fusbh200-ehci-usb', + 'apic', + 'apic-common', + 'xlnx,ps7-usb', + 'e1000e', + 'e1000-82544gc', + 'e1000-82545em'] + + if sec in new_sections_list: + return True + + return False + +# Fields might change name with time across qemu versions. + =20 def check_fields_match(name, s_field, d_field): if s_field =3D=3D d_field: @@ -57,7 +159,7 @@ def check_fields_match(name, s_field, d_field): 'ioh-3240-express-root-port': ['port.br.dev', 'parent_obj.parent_obj.parent_obj', 'port.br.dev.exp.aer_log', - 'parent_obj.parent_obj.parent_obj.exp.aer_= log'], + 'parent_obj.parent_obj.parent_obj.exp.aer_log'= ], 'cirrus_vga': ['hw_cursor_x', 'vga.hw_cursor_x', 'hw_cursor_y', 'vga.hw_cursor_y'], 'lsiscsi': ['dev', 'parent_obj'], @@ -73,7 +175,8 @@ def check_fields_match(name, s_field, d_field): 'tmr.overflow_time', 'ar.tmr.overflow_time', 'gpe', 'ar.gpe'], 'qxl': ['num_surfaces', 'ssd.num_surfaces'], - 'usb-ccid': ['abProtocolDataStructure', 'abProtocolDataStructure.d= ata'], + 'usb-ccid': ['abProtocolDataStructure', + 'abProtocolDataStructure.data'], 'usb-host': ['dev', 'parent_obj'], 'usb-mouse': ['usb-ptr-queue', 'HIDPointerEventQueue'], 'usb-tablet': ['usb-ptr-queue', 'HIDPointerEventQueue'], @@ -84,7 +187,7 @@ def check_fields_match(name, s_field, d_field): 'xio3130-express-downstream-port': ['port.br.dev', 'parent_obj.parent_obj.parent_= obj', 'port.br.dev.exp.aer_log', - 'parent_obj.parent_obj.parent_obj.exp.aer_= log'], + 'parent_obj.parent_obj.parent_obj.exp.aer_log'= ], 'xio3130-downstream': ['PCIDevice', 'PCIEDevice'], 'xio3130-express-upstream-port': ['br.dev', 'parent_obj.parent_obj= ', 'br.dev.exp.aer_log', @@ -95,7 +198,8 @@ def check_fields_match(name, s_field, d_field): 'io_win_addr', 'mig_io_win_addr', 'io_win_size', 'mig_io_win_size'], 'rtl8139': ['dev', 'parent_obj'], - 'e1000e': ['PCIDevice', 'PCIEDevice', 'intr_state', 'redhat_7_3_in= tr_state'], + 'e1000e': ['PCIDevice', 'PCIEDevice', 'intr_state', + 'redhat_7_3_intr_state'], 'nec-usb-xhci': ['PCIDevice', 'PCIEDevice'], 'xhci-intr': ['er_full_unused', 'er_full'], 'e1000': ['dev', 'parent_obj', @@ -225,7 +329,8 @@ def check_fields(src_fields, dest_fields, desc, sec): if unused_count < 0: print('Section "' + sec + '", ' 'Description "' + desc + '": ' - 'unused size mismatch near "' + s_item["field"] = + '"') + 'unused size mismatch near "' + + s_item["field"] + '"') bump_taint() break continue @@ -238,7 +343,8 @@ def check_fields(src_fields, dest_fields, desc, sec): if unused_count < 0: print('Section "' + sec + '", ' 'Description "' + desc + '": ' - 'unused size mismatch near "' + d_item["field"] = + '"') + 'unused size mismatch near "' + d_item["field"] + + '"') bump_taint() break continue @@ -286,12 +392,22 @@ def check_fields(src_fields, dest_fields, desc, sec): unused_count =3D s_item["size"] - d_item["size"] continue =20 - print('Section "' + sec + '", ' - 'Description "' + desc + '": ' - 'expected field "' + s_item["field"] + '", ' - 'got "' + d_item["field"] + '"; skipping rest') - bump_taint() - break + # commit 20daa90a20d, extra field 'config' was added in newer + # releases there will be a mismatch in the number of fields of + # irq_state and config it's a known false positive so skip it + if (desc in ["PCIDevice", "PCIEDevice"]): + if((s_item["field"] in ["irq_state", "config"]) and + (d_item["field"] in ["irq_state", "config"])): + break + + # some fields are new some dead, but are not errors. + if not check_fields_match(desc, s_item["field"], d_item["field= "]): + print('Section "' + sec + '", ' + 'Description "' + desc + '": ' + 'expected field "' + s_item["field"] + '", ' + 'got "' + d_item["field"] + '"; skipping rest') + bump_taint() + break =20 check_version(s_item, d_item, sec, desc) =20 @@ -312,7 +428,8 @@ def check_subsections(src_sub, dest_sub, desc, sec): found =3D True check_descriptions(s_item, d_item, sec) =20 - if not found: + # check the updated properties list before throwing error + if not found and (not check_updated_properties(desc, s_item["name"= ])): print('Section "' + sec + '", ' 'Description "' + desc + '": ' 'Subsection "' + s_item["name"] + '" not found') @@ -343,51 +460,66 @@ def check_descriptions(src_desc, dest_desc, sec): bump_taint() return =20 - for f in src_desc: - if not f in dest_desc: - print('Section "' + sec + '" ' - 'Description "' + src_desc["name"] + '": ' - 'Entry "' + field + '" missing') - bump_taint() - continue + for field in src_desc: + if field not in dest_desc: + # check the updated list of changed properties + # before throwing error + if check_updated_properties(src_desc["name"], field): + continue + else: + print('Section "' + sec + '" ' + 'Description "' + src_desc["name"] + '": ' + 'Entry "' + field + '" missing') + bump_taint() + continue =20 - if f =3D=3D 'Fields': - check_fields(src_desc[f], dest_desc[f], src_desc["name"], sec) + if field =3D=3D 'Fields': + check_fields(src_desc[field], dest_desc[field], + src_desc["name"], sec) =20 - if f =3D=3D 'Subsections': - check_subsections(src_desc[f], dest_desc[f], src_desc["name"],= sec) + if field =3D=3D 'Subsections': + check_subsections(src_desc[field], dest_desc[field], + src_desc["name"], sec) =20 =20 -def check_version(s, d, sec, desc=3DNone): - if s["version_id"] > d["version_id"]: - print "Section \"" + sec + "\"", - if desc: - print "Description \"" + desc + "\":", - print "version error:", s["version_id"], ">", d["version_id"] - bump_taint() +def check_version(src_ver, dest_ver, sec, desc=3DNone): + if src_ver["version_id"] > dest_ver["version_id"]: + if not check_updated_sizes(sec, src_ver["version_id"], + dest_ver["version_id"]): + print ('Section "' + sec + '"'), + if desc: + print ('Description "' + desc + '":'), + print ('version error: ' + str(src_ver["version_id"]) + ' > ' + + str(dest_ver["version_id"])) + bump_taint() =20 if "minimum_version_id" not in dest_ver: return =20 - if s["version_id"] < d["minimum_version_id"]: - print "Section \"" + sec + "\"", - if desc: - print('Description "' + desc + '": ' + - 'minimum version error: ' + str(src_ver["version_id"]) += ' < ' + - str(dest_ver["minimum_version_id"])) - bump_taint() + if src_ver["version_id"] < dest_ver["minimum_version_id"]: + if not check_updated_sizes(sec, src_ver["version_id"], + dest_ver["minimum_version_id"]): + print ('Section "' + sec + '"'), + if desc: + print('Description "' + desc + '": ' + + 'minimum version error: ' + str(src_ver["version_id"= ]) + + ' < ' + str(dest_ver["minimum_version_id"])) + bump_taint() =20 =20 -def check_size(s, d, sec, desc=3DNone, field=3DNone): - if s["size"] !=3D d["size"]: - print "Section \"" + sec + "\"", - if desc: - print "Description \"" + desc + "\"", - if field: - print "Field \"" + field + "\"", - print "size mismatch:", s["size"], ",", d["size"] - bump_taint() +def check_size(src, dest, sec, desc=3DNone, field=3DNone): + if src["size"] !=3D dest["size"]: + # check updated sizes list before throwing error =20 + if not check_updated_sizes(field, src["size"], dest["size"]): + print ('Section "' + sec + '"'), + if desc: + print ('Description "' + desc + '"'), + if field: + print ('Field "' + field + '"'), + print ('size mismatch: ' + str(src["size"]) + ' , ' + + str(dest["size"])) + bump_taint() =20 =20 def check_machine_type(src, dest): @@ -402,8 +534,8 @@ def main(): "SRC and DEST. Checks whether migration from SRC to DEST QEMU version= s " "would break based on the VMSTATE information contained within the JSO= N " "outputs. The JSON output is created from a QEMU invocation with the " - "-dump-vmstate parameter and a filename argument to it. Other paramete= rs to " - "QEMU do not matter, except the -M (machine type) parameter.") + "-dump-vmstate parameter and a filename argument to it. Other paramete= rs " + " to QEMU do not matter, except the -M (machine type) parameter.") =20 parser =3D argparse.ArgumentParser(description=3Dhelp_text) parser.add_argument('-s', '--src', type=3Dfile, required=3DTrue, @@ -428,13 +560,27 @@ def main(): for sec in src_data: dest_sec =3D sec if dest_sec not in dest_data: - # Either the section name got changed, or the section - # doesn't exist in dest. + # Either the section name got changed, or + # the section doesn't exist in dest or + # section was newly supported. dest_sec =3D get_changed_sec_name(sec) - if dest_sec not in dest_data: - print('Section "' + sec + '" does not exist in dest') - bump_taint() + if dest_sec =3D=3D "": + if not check_new_sections(sec): + # section not in newly supported list and not in dest + print('Section "' + sec + '" does not exist in dest') + bump_taint() continue + else: + # section name changed + if dest_sec not in dest_data: + # new name not found in dest. + if check_new_sections(dest_sec): + continue + else: + # new name not in dest and not newly supported + print('Section "' + sec + '" does not exist in des= t') + bump_taint() + continue =20 s =3D src_data[sec] d =3D dest_data[dest_sec] --=20 1.8.3.1