From nobody Wed Jan 15 12:38:22 2025 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; spf=none (zohomail.com: 8.43.85.245 is neither permitted nor denied by domain of lists.libvirt.org) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1705422462139554.8264757428329; Tue, 16 Jan 2024 08:27:42 -0800 (PST) Received: by lists.libvirt.org (Postfix, from userid 996) id 14C001C7A; Tue, 16 Jan 2024 11:27:41 -0500 (EST) Received: from lists.libvirt.org.85.43.8.in-addr.arpa (localhost [IPv6:::1]) by lists.libvirt.org (Postfix) with ESMTP id 68B3F1D63; Tue, 16 Jan 2024 11:13:42 -0500 (EST) Received: by lists.libvirt.org (Postfix, from userid 996) id CF9D51C7A; Tue, 16 Jan 2024 11:13:00 -0500 (EST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 540341C74 for ; Tue, 16 Jan 2024 11:12:57 -0500 (EST) Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-421-Ql2YyXGJNwKA675ZioFp9g-1; Tue, 16 Jan 2024 11:12:55 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 423303C1E9C3 for ; Tue, 16 Jan 2024 16:12:55 +0000 (UTC) Received: from speedmetal.lan (unknown [10.45.242.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id B06041C060AF for ; Tue, 16 Jan 2024 16:12:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.4 X-MC-Unique: Ql2YyXGJNwKA675ZioFp9g-1 From: Peter Krempa To: devel@lists.libvirt.org Subject: [PATCH 08/10] qemu-replies-tool: Add mode to dump all QMP schema query strings Date: Tue, 16 Jan 2024 17:12:42 +0100 Message-ID: <128dbd2bd7b721f93e8aa37d18cb3b6b587c8828.1705421341.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Message-ID-Hash: 2UFXG2TVS7B7JIPH6FJXY2AILZ5CLXXV X-Message-ID-Hash: 2UFXG2TVS7B7JIPH6FJXY2AILZ5CLXXV X-MailFrom: pkrempa@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-config-2; header-match-config-3; header-match-devel.lists.libvirt.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.2 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1705422462428100001 Make the tool useful also for non-testing purposes by adding 'dump' mode, which will process the data and output information about the qemu version. The first 'dump' mode produces all possible valid query strings per virQEMUQAPISchemaPathGet/virQEMUCapsQMPSchemaQueries. This is useful for users to look up a query string via 'grep' rather than trying to come up with it manually. Additionally the data as represented by qemu changes naming very often and that makes it un-reviewable to find changes between two qemu builds. By using the dump mode, which produces results in stable order we can use it to 'diff' two .replies file without churn. Sample output '[...]' denotes an arbitrary trim: $ ./scripts/qemu-replies-tool.py tests/qemucapabilitiesdata/caps_9.0.0_x86_= 64.replies --dump-qmp-query-strings [...] (qmp) blockdev-add (qmp) blockdev-add/arg-type/auto-read-only (qmp) blockdev-add/arg-type/auto-read-only/!bool (qmp) blockdev-add/arg-type/cache (qmp) blockdev-add/arg-type/cache/direct (qmp) blockdev-add/arg-type/cache/direct/!bool (qmp) blockdev-add/arg-type/cache/no-flush (qmp) blockdev-add/arg-type/cache/no-flush/!bool (qmp) blockdev-add/arg-type/detect-zeroes (qmp) blockdev-add/arg-type/detect-zeroes/^off (qmp) blockdev-add/arg-type/detect-zeroes/^on (qmp) blockdev-add/arg-type/detect-zeroes/^unmap [...] (qmp) blockdev-add/arg-type/driver (qmp) blockdev-add/arg-type/driver/^blkdebug (qmp) blockdev-add/arg-type/driver/^blklogwrites (qmp) blockdev-add/arg-type/driver/^blkreplay (qmp) blockdev-add/arg-type/driver/^blkverify (qmp) blockdev-add/arg-type/driver/^bochs (qmp) blockdev-add/arg-type/driver/^cloop [...] (qmp) blockdev-add/arg-type/+blkdebug (qmp) blockdev-add/arg-type/+blkdebug/align (qmp) blockdev-add/arg-type/+blkdebug/align/!int (qmp) blockdev-add/arg-type/+blkdebug/config (qmp) blockdev-add/arg-type/+blkdebug/config/!str (qmp) blockdev-add/arg-type/+blkdebug/image (qmp) blockdev-add/arg-type/+blkdebug/image (recursion) (qmp) blockdev-add/arg-type/+blkdebug/image/!str (qmp) blockdev-add/arg-type/+blkdebug/inject-error Signed-off-by: Peter Krempa Reviewed-by: Andrea Bolognani --- scripts/qemu-replies-tool.py | 112 +++++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) diff --git a/scripts/qemu-replies-tool.py b/scripts/qemu-replies-tool.py index c5dee9a66a..5dcc975756 100755 --- a/scripts/qemu-replies-tool.py +++ b/scripts/qemu-replies-tool.py @@ -290,9 +290,95 @@ def validate_qmp_schema(schemalist): raise qmpSchemaException("unknown or missing 'meta-type' i= n schema entry '%s'" % entry) +# Recursively traverse the schema and print out the schema query strings f= or +# the corresponding entries. In certain cases the schema references itself, +# which is handled by passing a 'trace' list which contains the current pa= th +def dump_qmp_probe_strings_iter(name, cur, trace, schema): + obj =3D schema[name] + + if name in trace: + print('%s (recursion)' % cur) + return + + trace =3D trace + [name] + + match obj['meta-type']: + case 'command' | 'event': + arguments =3D obj.get('arg-type', None) + returns =3D obj.get('ret-type', None) + + print(cur) + + for f in obj.get('features', []): + print('%s/$%s' % (cur, f)) + + if arguments: + dump_qmp_probe_strings_iter(arguments, cur + '/arg-type', = trace, schema) + + if returns: + dump_qmp_probe_strings_iter(returns, cur + '/ret-type', tr= ace, schema) + + case 'object': + members =3D sorted(obj.get('members', []), key=3Dlambda d: d['= name']) + variants =3D sorted(obj.get('variants', []), key=3Dlambda d: d= ['case']) + + for f in obj.get('features', []): + print('%s/$%s' % (cur, f)) + + for memb in members: + membpath =3D "%s/%s" % (cur, memb['name']) + print(membpath) + + for f in memb.get('features', []): + print('%s/$%s' % (membpath, f)) + + dump_qmp_probe_strings_iter(memb['type'], membpath, trace,= schema) + + for var in variants: + varpath =3D "%s/+%s" % (cur, var['case']) + print(varpath) + dump_qmp_probe_strings_iter(var['type'], varpath, trace, s= chema) + + case 'enum': + members =3D sorted(obj.get('members', []), key=3Dlambda d: d['= name']) + + for m in members: + print('%s/^%s' % (cur, m['name'])) + + for f in m.get('features', []): + print('%s/^%s/$%s' % (cur, m['name'], f)) + + case 'array': + dump_qmp_probe_strings_iter(obj['element-type'], cur, trace, s= chema) + + case 'builtin': + print('%s/!%s' % (cur, name)) + + case 'alternate': + for var in obj['members']: + dump_qmp_probe_strings_iter(var['type'], cur, trace, schem= a) + + +def dump_qmp_probe_strings(schemalist): + schemadict =3D {} + toplevel =3D [] + + for memb in schemalist: + schemadict[memb['name']] =3D memb + + if memb['meta-type'] =3D=3D 'command' or memb['meta-type'] =3D=3D = 'event': + toplevel.append(memb['name']) + + toplevel.sort() + + for c in toplevel: + dump_qmp_probe_strings_iter(c, '(qmp) ' + c, [], schemadict) + + def process_one(filename, args): try: conv =3D qemu_replies_load(filename) + dumped =3D False modify_replies(conv) @@ -300,6 +386,13 @@ def process_one(filename, args): if cmd['execute'] =3D=3D 'query-qmp-schema': validate_qmp_schema(rep['return']) + if args.dump_all or args.dump_qmp_query_strings: + dump_qmp_probe_strings(rep['return']) + dumped =3D True + + if dumped: + return True + qemu_replies_compare_or_replace(filename, conv, args.regenerate) except qrtException as e: @@ -327,6 +420,19 @@ The default mode is validation which checks the follow= ing: - the input file has the expected JSON formatting - the QMP schema from qemu is fully covered by libvirt's code +In 'dump' mode if '-dump-all' or one of the specific '-dump-*' flags (belo= w) +is selected the script outputs information gathered from the given '.repli= es' +file. The data is also usable for comparing two '.replies' files in a "dif= fable" +fashion as many of the query commands may change ordering or naming without +functional impact on libvirt. + + --dump-qmp-query-strings + + Dumps all possible valid QMP capability query strings based on the cur= rent + qemu version in format used by virQEMUQAPISchemaPathGet or + virQEMUCapsQMPSchemaQueries. It's useful to find specific query string + without having to piece the information together from 'query-qmp-schem= a' + The tool can be also used to programmaticaly modify the '.replies' file by editting the 'modify_replies' method directly in the source, or for re-formatting and re-numbering the '.replies' file to conform with the req= uired @@ -351,6 +457,12 @@ parser.add_argument('--repliesdir', default=3D'', parser.add_argument('replyfile', nargs=3D'?', help=3D'path to .replies file to use') +parser.add_argument('--dump-all', action=3D'store_true', + help=3D'invoke all --dump-* sub-commands') + +parser.add_argument('--dump-qmp-query-strings', action=3D'store_true', + help=3D'dump QMP schema in form of query strings used = to probe capabilities') + args =3D parser.parse_args() if not args.replyfile and not args.repliesdir: --=20 2.43.0 _______________________________________________ Devel mailing list -- devel@lists.libvirt.org To unsubscribe send an email to devel-leave@lists.libvirt.org