From nobody Mon Feb 9 17:37:25 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 151834271902091.68813181228586; Sun, 11 Feb 2018 01:51:59 -0800 (PST) Received: from localhost ([::1]:40827 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ekoIb-0002e4-32 for importer@patchew.org; Sun, 11 Feb 2018 04:51:53 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54049) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eko3c-00071G-Og for qemu-devel@nongnu.org; Sun, 11 Feb 2018 04:36:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eko3W-0006JC-9K for qemu-devel@nongnu.org; Sun, 11 Feb 2018 04:36:24 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:39190 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eko3W-0006Hb-0x for qemu-devel@nongnu.org; Sun, 11 Feb 2018 04:36:18 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8901940201A3; Sun, 11 Feb 2018 09:36:16 +0000 (UTC) Received: from blackfin.pond.sub.org (ovpn-116-43.ams2.redhat.com [10.36.116.43]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 25D4B2166BAE; Sun, 11 Feb 2018 09:36:16 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id F119B113866D; Sun, 11 Feb 2018 10:36:07 +0100 (CET) From: Markus Armbruster To: qemu-devel@nongnu.org Date: Sun, 11 Feb 2018 10:35:58 +0100 Message-Id: <20180211093607.27351-21-armbru@redhat.com> In-Reply-To: <20180211093607.27351-1-armbru@redhat.com> References: <20180211093607.27351-1-armbru@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Sun, 11 Feb 2018 09:36:16 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Sun, 11 Feb 2018 09:36:16 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'armbru@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v2 20/29] qapi/types qapi/visit: Generate built-in stuff into separate files X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marcandre.lureau@redhat.com, mdroth@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Linking code from multiple separate QAPI schemata into the same program is possible, but involves some weirdness around built-in types: * We generate code for built-in types into .c only with option --builtins. The user is responsible for generating code for exactly one QAPI schema per program with --builtins. * We generate code for built-in types into .h regardless of --builtins, but guarded by #ifndef QAPI_VISIT_BUILTIN. Because all copies of this code are exactly the same, including any combination of these headers works. Replace this contraption by something more conventional: generate code for built-in types into their very own files: qapi-builtin-types.c, qapi-builtin-visit.c, qapi-builtin-types.h, qapi-builtin-visit.h, but only with --builtins. Obey --output-dir, but ignore --prefix for them. Make qapi-types.h include qapi-builtin-types.h. With multiple schemata you now have multiple qapi-types.[ch], but only one qapi-builtin-types.[ch]. Same for qapi-visit.[ch] and qapi-builtin-visit.[ch]. Bonus: if all you need is built-in stuff, you can include a much smaller header. To be exploited shortly. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Reviewed-by: Marc-Andr=C3=A9 Lureau Reviewed-by: Michael Roth --- .gitignore | 2 ++ Makefile | 13 ++++++---- Makefile.objs | 2 ++ scripts/qapi/common.py | 61 ++++++++++++++++++++++++++++++++++++++++------- scripts/qapi/types.py | 61 +++++++++++++++++++---------------------------- scripts/qapi/visit.py | 64 +++++++++++++++++++++-------------------------= ---- 6 files changed, 116 insertions(+), 87 deletions(-) diff --git a/.gitignore b/.gitignore index 7d783e6e66..9477a08b6b 100644 --- a/.gitignore +++ b/.gitignore @@ -29,6 +29,8 @@ /qga/qapi-generated /qapi-generated /qapi-gen-timestamp +/qapi-builtin-types.[ch] +/qapi-builtin-visit.[ch] /qapi-types.[ch] /qapi-visit.[ch] /qapi-event.[ch] diff --git a/Makefile b/Makefile index 164a38578e..60ddc9c945 100644 --- a/Makefile +++ b/Makefile @@ -90,10 +90,13 @@ endif include $(SRC_PATH)/rules.mak =20 GENERATED_FILES =3D qemu-version.h config-host.h qemu-options.def -GENERATED_FILES +=3D qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h -GENERATED_FILES +=3D qmp-commands.c qapi-types.c qapi-visit.c qapi-event.c -GENERATED_FILES +=3D qmp-introspect.h -GENERATED_FILES +=3D qmp-introspect.c +GENERATED_FILES +=3D qapi-builtin-types.h qapi-builtin-types.c +GENERATED_FILES +=3D qapi-types.h qapi-types.c +GENERATED_FILES +=3D qapi-builtin-visit.h qapi-builtin-visit.c +GENERATED_FILES +=3D qapi-visit.h qapi-visit.c +GENERATED_FILES +=3D qmp-commands.h qmp-commands.c +GENERATED_FILES +=3D qapi-event.h qapi-event.c +GENERATED_FILES +=3D qmp-introspect.c qmp-introspect.h GENERATED_FILES +=3D qapi-doc.texi =20 GENERATED_FILES +=3D trace/generated-tcg-tracers.h @@ -520,7 +523,9 @@ qapi-modules =3D $(SRC_PATH)/qapi-schema.json $(SRC_PAT= H)/qapi/common.json \ $(SRC_PATH)/qapi/transaction.json \ $(SRC_PATH)/qapi/ui.json =20 +qapi-builtin-types.c qapi-builtin-types.h \ qapi-types.c qapi-types.h \ +qapi-builtin-visit.c qapi-builtin-visit.h \ qapi-visit.c qapi-visit.h \ qmp-commands.h qmp-commands.c \ qapi-event.c qapi-event.h \ diff --git a/Makefile.objs b/Makefile.objs index d255aaf194..2813e984fd 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -2,6 +2,8 @@ # Common libraries for tools and emulators stub-obj-y =3D stubs/ crypto/ util-obj-y =3D util/ qobject/ qapi/ +util-obj-y +=3D qapi-builtin-types.o +util-obj-y +=3D qapi-builtin-visit.o util-obj-y +=3D qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o =20 chardev-obj-y =3D chardev/ diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py index 31d2f73e7e..de12f8469a 100644 --- a/scripts/qapi/common.py +++ b/scripts/qapi/common.py @@ -1531,11 +1531,10 @@ class QAPISchema(object): =20 def _def_builtin_type(self, name, json_type, c_type): self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type)) - # TODO As long as we have QAPI_TYPES_BUILTIN to share multiple - # qapi-types.h from a single .c, all arrays of builtins must be - # declared in the first file whether or not they are used. Nicer - # would be to use lazy instantiation, while figuring out how to - # avoid compilation issues with multiple qapi-types.h. + # Instantiating only the arrays that are actually used would + # be nice, but we can't as long as their generated code + # (qapi-builtin-types.[ch]) may be shared by some other + # schema. self._make_array_type(name, None) =20 def _def_predefineds(self): @@ -1992,14 +1991,15 @@ class QAPIGen(object): return '' =20 def write(self, output_dir, fname): - if output_dir: + pathname =3D os.path.join(output_dir, fname) + dir =3D os.path.dirname(pathname) + if dir: try: - os.makedirs(output_dir) + os.makedirs(dir) except os.error as e: if e.errno !=3D errno.EEXIST: raise - fd =3D os.open(os.path.join(output_dir, fname), - os.O_RDWR | os.O_CREAT, 0666) + fd =3D os.open(pathname, os.O_RDWR | os.O_CREAT, 0666) f =3D os.fdopen(fd, 'r+') text =3D (self._top(fname) + self._preamble + self._body + self._bottom(fname)) @@ -2046,6 +2046,7 @@ class QAPIGenH(QAPIGenC): =20 =20 class QAPIGenDoc(QAPIGen): + def _top(self, fname): return (QAPIGen._top(self, fname) + '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n\n') @@ -2062,3 +2063,45 @@ class QAPISchemaMonolithicCVisitor(QAPISchemaVisitor= ): def write(self, output_dir): self._genc.write(output_dir, self._prefix + self._what + '.c') self._genh.write(output_dir, self._prefix + self._what + '.h') + + +class QAPISchemaModularCVisitor(QAPISchemaVisitor): + + def __init__(self, prefix, what, blurb, pydoc): + self._prefix =3D prefix + self._what =3D what + self._blurb =3D blurb + self._pydoc =3D pydoc + self._module =3D {} + + def _module_basename(self, what, name): + if name is None: + return re.sub(r'-', '-builtin-', what) + return self._prefix + what + + def _add_module(self, name, blurb): + genc =3D QAPIGenC(blurb, self._pydoc) + genh =3D QAPIGenH(blurb, self._pydoc) + self._module[name] =3D (genc, genh) + self._set_module(name) + + def _set_module(self, name): + self._genc, self._genh =3D self._module[name] + + def write(self, output_dir, opt_builtins): + for name in self._module: + if name is None and not opt_builtins: + continue + basename =3D self._module_basename(self._what, name) + (genc, genh) =3D self._module[name] + genc.write(output_dir, basename + '.c') + genh.write(output_dir, basename + '.h') + + def _begin_module(self, name): + pass + + def visit_module(self, name): + if len(self._module) !=3D 1: + return + self._add_module(name, self._blurb) + self._begin_module(name) diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py index d2b8423479..59826b1162 100644 --- a/scripts/qapi/types.py +++ b/scripts/qapi/types.py @@ -167,64 +167,51 @@ void qapi_free_%(c_name)s(%(c_name)s *obj) return ret =20 =20 -class QAPISchemaGenTypeVisitor(QAPISchemaMonolithicCVisitor): +class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor): =20 - def __init__(self, prefix, opt_builtins): - QAPISchemaMonolithicCVisitor.__init__( + def __init__(self, prefix): + QAPISchemaModularCVisitor.__init__( self, prefix, 'qapi-types', ' * Schema-defined QAPI types', __doc__) - self._opt_builtins =3D opt_builtins + self._add_module(None, ' * Built-in QAPI types') + self._genc.preamble_add(mcgen(''' +#include "qemu/osdep.h" +#include "qapi/dealloc-visitor.h" +#include "qapi-builtin-types.h" +#include "qapi-builtin-visit.h" +''')) + self._genh.preamble_add(mcgen(''' +#include "qapi/util.h" +''')) + + def _begin_module(self, name): self._genc.preamble_add(mcgen(''' #include "qemu/osdep.h" #include "qapi/dealloc-visitor.h" #include "%(prefix)sqapi-types.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=3Dprefix)) + prefix=3Dself._prefix)) self._genh.preamble_add(mcgen(''' -#include "qapi/util.h" +#include "qapi-builtin-types.h" ''')) - self._btin =3D '\n' + guardstart('QAPI_TYPES_BUILTIN') =20 def visit_begin(self, schema): # gen_object() is recursive, ensure it doesn't visit the empty type objects_seen.add(schema.the_empty_object_type.name) =20 - def visit_end(self): - # To avoid header dependency hell, we always generate - # declarations for built-in types in our header files and - # simply guard them. See also opt_builtins (command line - # option -b). - self._btin +=3D guardend('QAPI_TYPES_BUILTIN') - self._genh.preamble_add(self._btin) - self._btin =3D None - def _gen_type_cleanup(self, name): self._genh.add(gen_type_cleanup_decl(name)) self._genc.add(gen_type_cleanup(name)) =20 def visit_enum_type(self, name, info, values, prefix): - # Special case for our lone builtin enum type - # TODO use something cleaner than existence of info - if not info: - self._btin +=3D gen_enum(name, values, prefix) - if self._opt_builtins: - self._genc.add(gen_enum_lookup(name, values, prefix)) - else: - self._genh.preamble_add(gen_enum(name, values, prefix)) - self._genc.add(gen_enum_lookup(name, values, prefix)) + self._genh.preamble_add(gen_enum(name, values, prefix)) + self._genc.add(gen_enum_lookup(name, values, prefix)) =20 def visit_array_type(self, name, info, element_type): - if isinstance(element_type, QAPISchemaBuiltinType): - self._btin +=3D gen_fwd_object_or_array(name) - self._btin +=3D gen_array(name, element_type) - self._btin +=3D gen_type_cleanup_decl(name) - if self._opt_builtins: - self._genc.add(gen_type_cleanup(name)) - else: - self._genh.preamble_add(gen_fwd_object_or_array(name)) - self._genh.add(gen_array(name, element_type)) - self._gen_type_cleanup(name) + self._genh.preamble_add(gen_fwd_object_or_array(name)) + self._genh.add(gen_array(name, element_type)) + self._gen_type_cleanup(name) =20 def visit_object_type(self, name, info, base, members, variants): # Nothing to do for the special empty builtin @@ -248,6 +235,6 @@ class QAPISchemaGenTypeVisitor(QAPISchemaMonolithicCVis= itor): =20 =20 def gen_types(schema, output_dir, prefix, opt_builtins): - vis =3D QAPISchemaGenTypeVisitor(prefix, opt_builtins) + vis =3D QAPISchemaGenTypeVisitor(prefix) schema.visit(vis) - vis.write(output_dir) + vis.write(output_dir, opt_builtins) diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py index 3d09d44265..9b678e7263 100644 --- a/scripts/qapi/visit.py +++ b/scripts/qapi/visit.py @@ -263,13 +263,27 @@ out: c_name=3Dc_name(name)) =20 =20 -class QAPISchemaGenVisitVisitor(QAPISchemaMonolithicCVisitor): +class QAPISchemaGenVisitVisitor(QAPISchemaModularCVisitor): =20 - def __init__(self, prefix, opt_builtins): - QAPISchemaMonolithicCVisitor.__init__( + def __init__(self, prefix): + QAPISchemaModularCVisitor.__init__( self, prefix, 'qapi-visit', ' * Schema-defined QAPI visitors', __doc__) - self._opt_builtins =3D opt_builtins + self._add_module(None, ' * Built-in QAPI visitors') + self._genc.preamble_add(mcgen(''' +#include "qemu/osdep.h" +#include "qemu-common.h" +#include "qapi/error.h" +#include "qapi-builtin-visit.h" +''')) + self._genh.preamble_add(mcgen(''' +#include "qapi/visitor.h" +#include "qapi-builtin-types.h" + +''', + prefix=3Dprefix)) + + def _begin_module(self, name): self._genc.preamble_add(mcgen(''' #include "qemu/osdep.h" #include "qemu-common.h" @@ -277,45 +291,21 @@ class QAPISchemaGenVisitVisitor(QAPISchemaMonolithicC= Visitor): #include "qapi/qmp/qerror.h" #include "%(prefix)sqapi-visit.h" ''', - prefix=3Dprefix)) + prefix=3Dself._prefix)) self._genh.preamble_add(mcgen(''' -#include "qapi/visitor.h" +#include "qapi-builtin-visit.h" #include "%(prefix)sqapi-types.h" =20 ''', - prefix=3Dprefix)) - self._btin =3D guardstart('QAPI_VISIT_BUILTIN') - - def visit_end(self): - # To avoid header dependency hell, we always generate - # declarations for built-in types in our header files and - # simply guard them. See also opt_builtins (command line - # option -b). - self._btin +=3D guardend('QAPI_VISIT_BUILTIN') - self._genh.preamble_add(self._btin) - self._btin =3D None + prefix=3Dself._prefix)) =20 def visit_enum_type(self, name, info, values, prefix): - # Special case for our lone builtin enum type - # TODO use something cleaner than existence of info - if not info: - self._btin +=3D gen_visit_decl(name, scalar=3DTrue) - if self._opt_builtins: - self._genc.add(gen_visit_enum(name)) - else: - self._genh.add(gen_visit_decl(name, scalar=3DTrue)) - self._genc.add(gen_visit_enum(name)) + self._genh.add(gen_visit_decl(name, scalar=3DTrue)) + self._genc.add(gen_visit_enum(name)) =20 def visit_array_type(self, name, info, element_type): - decl =3D gen_visit_decl(name) - defn =3D gen_visit_list(name, element_type) - if isinstance(element_type, QAPISchemaBuiltinType): - self._btin +=3D decl - if self._opt_builtins: - self._genc.add(defn) - else: - self._genh.add(decl) - self._genc.add(defn) + self._genh.add(gen_visit_decl(name)) + self._genc.add(gen_visit_list(name, element_type)) =20 def visit_object_type(self, name, info, base, members, variants): # Nothing to do for the special empty builtin @@ -336,6 +326,6 @@ class QAPISchemaGenVisitVisitor(QAPISchemaMonolithicCVi= sitor): =20 =20 def gen_visit(schema, output_dir, prefix, opt_builtins): - vis =3D QAPISchemaGenVisitVisitor(prefix, opt_builtins) + vis =3D QAPISchemaGenVisitVisitor(prefix) schema.visit(vis) - vis.write(output_dir) + vis.write(output_dir, opt_builtins) --=20 2.13.6