From nobody Mon Feb 9 10:12:37 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1599190918; cv=none; d=zohomail.com; s=zohoarc; b=UGGH7trhuKwuXMT49NvbrTID6OHLWGzz0vi/plITJi+KZ6m41V0lnf/X/MzN41LHsiiTa6oApDddaLP78iPnmkiQacjAK5ptm8BJ8wTyf9Msmb4r1XrpD1krUKny73fxkLK2zfzpRrTMXVRnD0kuTVWp1kmd/p9T+Tq2YsJU1Vw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1599190918; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RGO3VYU3oBRwD0yr5kemO3b2N3Gg1nOcfrFKVlUN4ms=; b=hYxcwRptEWq3BmF5uhc6YIRX9SrMBrkphBBqxgOurlAoEPE7VbJm7XzCSKIXkYaTMePmftgXn1CCgp53DenUbFra1fpANayZSjg5qeonjlJJbn3gFT17s+LY0mU8fous2JMP7+yr87js/gvks8ohXofnXAOjO8g+GAdApAOzm98= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1599190918485209.03283297939004; Thu, 3 Sep 2020 20:41:58 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-327-1nlMIYSoNqWOqsE0IL_GiQ-1; Thu, 03 Sep 2020 23:41:55 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0DA5C85EE96; Fri, 4 Sep 2020 03:41:45 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DDEFC5C1C2; Fri, 4 Sep 2020 03:41:44 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id AA3A01800FF5; Fri, 4 Sep 2020 03:41:44 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 0843fAMU030559 for ; Thu, 3 Sep 2020 23:41:11 -0400 Received: by smtp.corp.redhat.com (Postfix) id 1F24D10379B; Fri, 4 Sep 2020 03:41:10 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1007EEE873 for ; Fri, 4 Sep 2020 03:41:08 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3C39A800962 for ; Fri, 4 Sep 2020 03:41:08 +0000 (UTC) Received: from mail-m24147.qiye.163.com (mail-m24147.qiye.163.com [220.194.24.147]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-166-nschr7QuOruTJCSzWaERjA-1; Thu, 03 Sep 2020 23:41:03 -0400 Received: from localhost.localdomain (unknown [58.56.27.130]) by smtp4 (Coremail) with SMTP id JedpCgAHcNgStlFfYHYgCg--.1426S3; Fri, 04 Sep 2020 11:35:46 +0800 (CST) X-MC-Unique: 1nlMIYSoNqWOqsE0IL_GiQ-1 X-MC-Unique: nschr7QuOruTJCSzWaERjA-1 From: Shi Lei To: libvir-list@redhat.com Subject: [RFCv2 01/46] scripts: Add a tool to generate xml parse/format functions Date: Fri, 4 Sep 2020 11:34:53 +0800 Message-Id: <20200904033538.418579-2-shi_lei@massclouds.com> In-Reply-To: <20200904033538.418579-1-shi_lei@massclouds.com> References: <20200904033538.418579-1-shi_lei@massclouds.com> MIME-Version: 1.0 X-CM-TRANSID: JedpCgAHcNgStlFfYHYgCg--.1426S3 X-Coremail-Antispam: 1Uf129KBjvAXoWfKw13tw1DArWrKr1fXF4Dtwb_yoW7JF4fCo WfK3W8tF1rur43KrykKF97Wr1kur90gr47J3s3Gry5Wa1qqrW5u3WrZa13Wa90vr4UCr90 vr1Iva4fWF4DAFyfn29KB7ZKAUJUUUU8529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UbIYCTnIWIevJa73UjIFyTuYvj4iK9auUUUUU X-Originating-IP: [58.56.27.130] X-CM-SenderInfo: pvklsz1hl6ztxvvfz0xxgvhudrp/1tbiWhSVT1f4pvWJpQAAsE X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false; X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0.002 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This tool is used to generate parsexml/formatbuf functions. It is based on libclang and its python-binding. Some directives (such as genparse, xmlattr, etc.) need to be added on the declarations of structs to direct the tool. Signed-off-by: Shi Lei --- po/POTFILES.in | 1 + scripts/xmlgen/directive.py | 1115 +++++++++++++++++++++++++++++++++++ scripts/xmlgen/go | 7 + scripts/xmlgen/main.py | 439 ++++++++++++++ scripts/xmlgen/utils.py | 121 ++++ 5 files changed, 1683 insertions(+) create mode 100644 scripts/xmlgen/directive.py create mode 100755 scripts/xmlgen/go create mode 100755 scripts/xmlgen/main.py create mode 100644 scripts/xmlgen/utils.py diff --git a/po/POTFILES.in b/po/POTFILES.in index 3d6c20c..c9c0262 100644 --- a/po/POTFILES.in +++ b/po/POTFILES.in @@ -5,6 +5,7 @@ @BUILDDIR@src/admin/admin_server_dispatch_stubs.h @BUILDDIR@src/remote/remote_client_bodies.h @BUILDDIR@src/remote/remote_daemon_dispatch_stubs.h +@SRCDIR@scripts/xmlgen/directive.py @SRCDIR@src/access/viraccessdriverpolkit.c @SRCDIR@src/access/viraccessmanager.c @SRCDIR@src/admin/admin_server.c diff --git a/scripts/xmlgen/directive.py b/scripts/xmlgen/directive.py new file mode 100644 index 0000000..9b73931 --- /dev/null +++ b/scripts/xmlgen/directive.py @@ -0,0 +1,1115 @@ +# +# Copyright (C) 2020 Shandong Massclouds Co.,Ltd. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# + +import json +from collections import OrderedDict +from utils import singleton, dedup, Block, Terms, render + +BUILTIN_TYPES =3D { + 'String': {}, + 'Bool': {}, + 'Chars': { + 'conv': 'virStrcpyStatic(def->${name}, ${name}Str)' + }, + 'UChars': { + 'conv': 'virStrcpyStatic((char *)def->${name}, ${mdvar})' + }, + 'Int': { + 'fmt': '%d', + 'conv': 'virStrToLong_i(${mdvar}, NULL, 0, &def->${name})' + }, + 'UInt': { + 'fmt': '%u', + 'conv': 'virStrToLong_uip(${mdvar}, NULL, 0, &def->${name})' + }, + 'ULong': { + 'fmt': '%lu', + 'conv': 'virStrToLong_ulp(${mdvar}, NULL, 0, &def->${name})' + }, + 'ULongLong': { + 'fmt': '%llu', + 'conv': 'virStrToLong_ullp(${mdvar}, NULL, 0, &def->${name})' + }, + 'U8': { + 'fmt': '%u', + 'conv': 'virStrToLong_u8p(${mdvar}, NULL, 0, &def->${name})' + }, + 'U32': { + 'fmt': '%u', + 'conv': 'virStrToLong_uip(${mdvar}, NULL, 0, &def->${name})' + }, + 'Time': { + 'conv': 'virStrToTime(${mdvar}, &def->${name})' + }, +} + + +@singleton +class TypeTable(OrderedDict): + def __init__(self): + OrderedDict.__init__(self) + for name, kvs in BUILTIN_TYPES.items(): + kvs['name'] =3D name + kvs['meta'] =3D 'Builtin' + self[name] =3D kvs + + def register(self, kvs): + name =3D kvs['name'] + if name not in self: + self[name] =3D kvs + return name + + def get(self, name): + if name in self: + return self[name] + return {'meta': 'Struct', 'name': name, 'external': True} + + def check(self, name): + return name in self + + +T_NAMESPACE_PARSE =3D [ + 'if (xmlopt)', + ' def->ns =3D xmlopt->ns;', + 'if (def->ns.parse) {', + ' if (virXMLNamespaceRegister(ctxt, &def->ns) < 0)', + ' goto error;', + ' if ((def->ns.parse)(ctxt, &def->namespaceData) < 0)', + ' goto error;', + '}' +] + +T_NAMESPACE_FORMAT_BEGIN =3D [ + 'if (def->namespaceData && def->ns.format)', + ' virXMLNamespaceFormatNS(buf, &def->ns);' +] + +T_NAMESPACE_FORMAT_END =3D [ + 'if (def->namespaceData && def->ns.format) {', + ' if ((def->ns.format)(buf, def->namespaceData) < 0)', + ' return -1;', + '}' +] + + +def funcSignature(rtype, name, args): + alignment =3D ' ' * (len(name) + 1) + connector =3D ',\n' + alignment + + ret =3D [] + ret.append(rtype) + ret.append('%s(%s)' % (name, connector.join(args))) + return Block(ret) + + +def counterName(member): + if isinstance(member['array'], bool): + return 'n' + member['name'] + return member['array'] + + +def loop(count, block): + assert isinstance(block, list) and len(block) > 0 + lp =3D ' {' if len(block) > 1 else '' + + ret =3D Block() + ret.format('if (${count} > 0) {', count=3Dcount) + ret.format(' size_t i;') + ret.format(' for (i =3D 0; i < ${count}; i++)${lp}', count=3Dcount,= lp=3Dlp) + ret.mapfmt(' ${_each_line_}', block) + ret.format(' }' if lp else None) + ret.format('}') + return ret + + +def ispointer(member): + mtype =3D TypeTable().get(member['type']) + return member['pointer'] and not mtype.get('external') + + +def clearMember(member, pre_name=3DNone): + + # Callback for make_switch + def _switch_clear_cb(child, switch, _): + return clearMember(child, switch['name']) + + if member.get('xmlswitch'): + return makeSwitch(member, _switch_clear_cb, None) + + mtype =3D TypeTable().get(member['type']) + + if pre_name: + ref =3D 'def->%s.%s' % (pre_name, member['name']) + else: + ref =3D 'def->%s' % member['name'] + + if member.get('array'): + ref +=3D '[i]' + + ret =3D Block() + if mtype['meta'] =3D=3D 'Struct': + if ispointer(member): + ret.format('${tname}Clear(${ref});', tname=3Dmtype['name'], re= f=3Dref) + ret.format('g_free(${ref});', ref=3Dref) + ret.format('${ref} =3D NULL;', ref=3Dref) + else: + ret.format('${tname}Clear(&${ref});', tname=3Dmtype['name'], r= ef=3Dref) + elif mtype['name'] =3D=3D 'String': + ret.format('g_free(${ref});', ref=3Dref) + ret.format('${ref} =3D NULL;', ref=3Dref) + elif mtype['name'] in ['Chars', 'UChars']: + ret.format('memset(${ref}, 0, sizeof(${ref}));', ref=3Dref) + elif not member.get('array'): + ret.format('${ref} =3D 0;', ref=3Dref) + + if member.get('specified'): + assert not member.get('array'), "'specified' can't come with 'arra= y'." + if isinstance(member['specified'], str): + ret.format('def->${sname} =3D false;', sname=3Dmember['specifi= ed']) + else: + ret.format('${ref}_specified =3D false;', ref=3Dref) + + if member.get('array') and len(ret) > 0: + count =3D 'def->' + counterName(member) + ret =3D loop(count, ret) + ret.format('g_free(def->${name});', name=3Dmember['name']) + ret.format('def->${name} =3D 0;', name=3Dmember['name']) + ret.format('${count} =3D 0;', count=3Dcount) + + return ret + + +def makeClearFunc(writer, atype): + if 'genparse' not in atype: + return + + args =3D ['%sPtr def' % atype['name']] + signature =3D funcSignature('void', atype['name'] + 'Clear', args) + writer.write(atype, 'clearfunc', '.h', signature.output() + ';') + + ret =3D Block() + ret.extend(signature) + ret.format('{') + ret.format(' if (!def)') + ret.format(' return;') + ret.newline() + + delay_members =3D [] + for member in atype['members']: + if member.get('delay_clear'): + delay_members.append(member) + continue + + ret.mapfmt(' ${_each_line_}', clearMember(member)) + ret.newline() + + for member in delay_members: + ret.mapfmt(' ${_each_line_}', clearMember(member)) + ret.newline() + + if 'namespace' in atype: + ret.format(' if (def->namespaceData && def->ns.free)') + ret.format(' (def->ns.free)(def->namespaceData);') + else: + ret.pop() # Remove last newline + + ret.format('}') + writer.write(atype, 'clearfunc', '.c', ret.output()) + + +def reportInvalid(tname, mdvar): + ret =3D Block() + ret.append('virReportError(VIR_ERR_XML_ERROR,') + ret.append(" _(\"Invalid '%s' setting '%s' in '%s'\"),") + ret.format(' "${tname}", ${mdvar}, instname);', + tname=3Dtname, mdvar=3Dmdvar) + return ret + + +def reportMissing(tname): + ret =3D Block() + ret.append('virReportError(VIR_ERR_XML_ERROR,') + ret.append(" _(\"Missing '%s' setting in '%s'\"),") + ret.format(' "${tname}", instname);', tname=3Dtname) + return ret + + +def checkError(kind, cond, tname, mdvar): + if kind =3D=3D 'Invalid': + report =3D reportInvalid(tname, mdvar) + elif kind =3D=3D 'Missing': + report =3D reportMissing(tname) + else: + assert False, '%s is unsupported.' % kind + + ret =3D Block() + ret.format('if (${cond}) {', cond=3Dcond) + ret.mapfmt(' ${_each_line_}', report) + ret.append(' goto error;') + ret.append('}') + return ret + + +def parseMember(member, parent, tmpvars, pre_name=3DNone): + mtype =3D TypeTable().get(member['type']) + + # + # Helper functions + # + def _middle_var(member): + ret =3D member['name'].replace('.', '_') + if member.get('xmlgroup'): + ret =3D 'node' + elif mtype['name'] =3D=3D 'String': + ret =3D 'def->' + ret + elif member.get('xmlattr') or mtype['meta'] !=3D 'Struct': + ret +=3D 'Str' + else: + ret +=3D 'Node' + return ret + + def _read_xml_func(mdvar, tname): + if member.get('xmlattr'): + if '/' in tname: + funcname =3D 'virXMLChildPropString' + else: + funcname =3D 'virXMLPropString' + elif member.get('xmlelem'): + if mtype['meta'] =3D=3D 'Struct': + funcname =3D 'virXMLChildNode' + else: + funcname =3D 'virXMLChildNodeContent' + else: + return None + + return render('${mdvar} =3D ${funcname}(node, "${tname}");', + mdvar=3Dmdvar, funcname=3Dfuncname, tname=3Dtname) + + def _assign_struct(name, member, mdvar): + ret =3D Block() + if ispointer(member): + ret.format('def->${name} =3D g_new0(typeof(*def->${name}), 1);= ', + name=3Dname) + + func =3D mtype['name'] + 'ParseXML' + amp =3D '' if ispointer(member) else '&' + tmpl =3D '${func}(${mdvar}, ${amp}def->${name}, instname, def, opa= que)' + fn =3D render(tmpl, func=3Dfunc, mdvar=3Dmdvar, amp=3Damp, name=3D= name) + return if_cond(fn + ' < 0', ['goto error;']) + + def _assign_non_struct(name, member, mdvar): + if mtype['meta'] =3D=3D 'Enum': + typename =3D mtype['name'] + if not typename.endswith('Type'): + typename +=3D 'Type' + expr =3D render('(def->${name} =3D ${typename}FromString(${mdv= ar}))', + name=3Dname, typename=3Dtypename, mdvar=3Dmdvar) + expr +=3D ' <=3D 0' + elif mtype['name'] =3D=3D 'Bool': + truevalue =3D member.get('truevalue', 'yes') + expr =3D render('virStrToBool(${mdvar}, "${truth}", &def->${na= me})', + mdvar=3Dmdvar, truth=3Dtruevalue, name=3Dname) + expr +=3D ' < 0' + else: + builtin =3D BUILTIN_TYPES.get(mtype['name']) + assert builtin, mtype['name'] + tmpl =3D builtin.get('conv', None) + if tmpl: + expr =3D render(tmpl, name=3Dname, mdvar=3Dmdvar, tname=3D= tname) + expr +=3D ' < 0' + else: + return None + + return checkError('Invalid', expr, tname, mdvar) + + def _parse_array(name, member, tname): + num =3D 'n%sNodes' % Terms.upperInitial(tname) + tmpvars.append(num) + tmpvars.append('nodes') + count =3D counterName(member) + + if mtype['meta'] =3D=3D 'Struct': + item =3D _assign_struct(name + '[i]', member, 'tnode') + else: + item =3D ['def->%s[i] =3D virXMLNodeContentString(tnode);' % n= ame] + + ret =3D Block() + ret.format('${num} =3D virXMLChildNodeSet(node, "${tname}", &nodes= );', + num=3Dnum, tname=3Dtname) + ret.format('if (${num} > 0) {', num=3Dnum) + ret.format(' size_t i;') + ret.newline() + ret.format(' def->${name} =3D g_new0(typeof(*def->${name}), ${n= um});', + name=3Dname, num=3Dnum) + ret.format(' for (i =3D 0; i < ${num}; i++) {', num=3Dnum) + ret.format(' xmlNodePtr tnode =3D nodes[i];') + ret.mapfmt(' ${_each_line_}', item) + ret.format(' }') + ret.format(' def->${count} =3D ${num};', count=3Dcount, num=3Dn= um) + ret.format(' g_free(nodes);') + ret.format(' nodes =3D NULL;') + ret.format('} else if (${num} < 0) {', num=3Dnum) + ret.format(' virReportError(VIR_ERR_XML_ERROR,') + ret.format(' _("Invalid %s element found."),') + ret.format(' "${tname}");', tname=3Dtname) + ret.format(' goto error;') + + if member.get('required'): + ret.append('} else {') + ret.mapfmt(' ${_each_line_}', reportMissing(tname)) + ret.append(' goto error;') + + ret.append('}') + return ret + + # + # Main routine + # + if not member.get('xmlattr') and not member.get('xmlelem') \ + and not member.get('xmlgroup'): + return None + + if pre_name: + name =3D pre_name + '.' + member['name'] + else: + name =3D member['name'] + + tname =3D None + if member.get('xmlattr'): + tname =3D member['xmlattr'] + elif member.get('xmlelem'): + tname =3D member['xmlelem'] + + # For array member + if member.get('array'): + return _parse_array(name, member, tname) + + # For common member + mdvar =3D _middle_var(member) + if mdvar.endswith('Str') or mdvar.endswith('Node'): + tmpvars.append(mdvar) + + block =3D Block() + if tname: + block.append(_read_xml_func(mdvar, tname)) + if member.get('required'): + cond =3D render('${mdvar} =3D=3D NULL', mdvar=3Dmdvar) + block.extend(checkError('Missing', cond, tname, mdvar)) + + if mtype['meta'] =3D=3D 'Struct': + assignment =3D _assign_struct(name, member, mdvar) + else: + assignment =3D _assign_non_struct(name, member, mdvar) + if not assignment: + return block + + if member.get('specified'): + if isinstance(member['specified'], str): + assignment.format('def->${name} =3D true;', name=3Dmember['spe= cified']) + else: + assignment.format('def->${name}_specified =3D true;', name=3Dn= ame) + + if tname: + block.extend(if_cond(mdvar, assignment)) + else: + block.extend(assignment) + + return block + + +def makeParseFunc(writer, atype): + + # + # Helper functions + # + def _switch_parse_cb(child, switch, tmpvars): + return parseMember(child, atype, tmpvars, switch['name']) + + def _members_block(tmpvars): + block =3D Block() + for member in atype['members']: + if member.get('xmlswitch'): + block.extend(makeSwitch(member, _switch_parse_cb, tmpvars)) + else: + block.extend(parseMember(member, atype, tmpvars)) + block.newline() + return block + + def _post_hook(tmpvars): + if atype['genparse'] not in ['withhook', 'concisehook']: + return None + + args =3D ['node', 'def', 'instname', 'parent', 'opaque'] + if 'namespace' in atype: + args.append('ctxt') + args.append('xmlopt') + + if atype['genparse'] =3D=3D 'withhook': + args.extend(tmpvars) + if 'nodes' in args: + args.remove('nodes') + + funcname =3D atype['name'] + 'ParseXMLHook' + cond =3D '%s(%s) < 0' % (funcname, ', '.join(args)) + return if_cond(cond, ['goto error;']) + + def _handle_tmpvars(tmpvars): + args, heads, tails =3D [], [], [] + for var in tmpvars: + if var =3D=3D 'nodes': + heads.append('xmlNodePtr *nodes =3D NULL;') + tails.append('g_free(nodes);') + tails.append('nodes =3D NULL;') + elif var.endswith('Str'): + heads.append('g_autofree char *%s =3D NULL;' % var) + args.append('const char *%s' % var) + elif var.endswith('Node'): + heads.append('xmlNodePtr %s =3D NULL;' % var) + args.append('xmlNodePtr %s' % var) + else: + assert var.endswith('Nodes') and var.startswith('n') + heads.append('int %s =3D 0;' % var) + args.append('int %s' % var) + + if atype['genparse'] !=3D 'withhook': + args =3D [] + + if heads: + heads.append('') + + heads.append('if (!def)') + heads.append(' goto error;') + tails.append('%sClear(def);' % atype['name']) + return args, heads, tails + + # + # Composite + # + if 'genparse' not in atype: + return + + typename =3D atype['name'] + funcname =3D typename + 'ParseXML' + + # Declare virXXXParseXML + args =3D Block([ + 'xmlNodePtr node', + '%sPtr def' % typename, + 'const char *instname', + 'void *parent', + 'void *opaque' + ]) + + if 'namespace' in atype: + args.append('xmlXPathContextPtr ctxt') + args.append('virNetworkXMLOptionPtr xmlopt') + + signature =3D funcSignature('int', funcname, args) + writer.write(atype, 'parsefunc', '.h', signature.output() + ';') + + # Prepare for implementation + tmpvars =3D [] + parseblock =3D _members_block(tmpvars) + tmpvars =3D dedup(tmpvars) + tmpargs, headlines, cleanup =3D _handle_tmpvars(tmpvars) + posthook =3D _post_hook(tmpvars) + + if atype['genparse'] in ['withhook', 'concisehook']: + # Declare virXXXParseXMLHook + signature =3D funcSignature('int', funcname + 'Hook', args + tmpar= gs) + writer.write(atype, 'parsefunc', '.h', signature.output() + ';') + else: + # Without hook, instname, parent and opaque are unused. + args[2] +=3D ' G_GNUC_UNUSED' + args[3] +=3D ' G_GNUC_UNUSED' + args[4] +=3D ' G_GNUC_UNUSED' + + # Implement virXXXParseXML + impl =3D funcSignature('int', funcname, args) + impl.format('{') + impl.mapfmt(' ${_each_line_}', headlines) + impl.newline() + impl.mapfmt(' ${_each_line_}', parseblock) + + if posthook: + impl.mapfmt(' ${_each_line_}', posthook) + impl.newline() + + if 'namespace' in atype: + impl.extend(' ${_each_line_}', T_NAMESPACE_PARSE) + + impl.format(' return 0;') + impl.newline() + impl.format(' error:') + impl.mapfmt(' ${_each_line_}', cleanup) + impl.format(' return -1;') + impl.format('}') + + writer.write(atype, 'parsefunc', '.c', impl.output()) + + +def if_cond(condition, block): + assert isinstance(block, list) and len(block) > 0 + lp =3D ' {' if len(block) > 1 else '' + + ret =3D Block() + ret.format('if (${condition})${lp}', condition=3Dcondition, lp=3Dlp) + ret.mapfmt(' ${_each_line_}', block) + ret.format('}' if lp else None) + return ret + + +def formatMember(member, parent): + mtype =3D TypeTable().get(member['type']) + + # + # Helper functions. + # + def _checkOnCondition(var): + ret =3D None + if ispointer(member): + ret =3D var + elif member.get('specified'): + if isinstance(member['specified'], str): + ret =3D 'def->' + member['specified'] + else: + ret =3D var + '_specified' + if ret.startswith('&'): + ret =3D ret[1:] + elif mtype['meta'] =3D=3D 'Struct': + ret =3D '%sCheck(&%s, def, opaque)' % (mtype['name'], var) + elif member.get('required'): + pass + elif mtype['meta'] =3D=3D 'Enum': + ret =3D var + elif mtype['meta'] =3D=3D 'Builtin': + if mtype['name'] in ['Chars', 'UChars']: + ret =3D var + '[0]' + else: + ret =3D var + + if 'formatflag' in member: + flag =3D member['formatflag'] + if flag =3D=3D '_ALWAYS_': + return None + + exclusive =3D False + reverse =3D False + if flag[0] =3D=3D '%': + flag =3D flag[1:] + exclusive =3D True + if flag[0] =3D=3D '!': + flag =3D flag[1:] + reverse =3D True + cond =3D '(virXMLFlag(opaque) & %s)' % flag + if reverse: + cond =3D '!' + cond + + if ret and not exclusive: + ret =3D ret + ' && ' + cond + else: + ret =3D cond + + return ret + + def _handleMore(code): + counter =3D 'def->' + counterName(member) + return loop(counter, code), counter + + def _format(layout, var): + if mtype['meta'] =3D=3D 'Struct': + if not ispointer(member): + var =3D '&' + var + + fname =3D mtype['name'] + 'FormatBuf' + cond =3D '%s(buf, "%s", %s, def, opaque) < 0' % (fname, layout= , var) + return if_cond(cond, ['return -1;']) + elif mtype['meta'] =3D=3D 'Enum': + name =3D mtype['name'] + if not name.endswith('Type'): + name +=3D 'Type' + + ret =3D Block() + ret.format('const char *str =3D ${name}ToString(${var});', + name=3Dname, var=3Dvar) + ret.format('if (!str) {') + ret.format(' virReportError(VIR_ERR_INTERNAL_ERROR,') + ret.format(' _("Unknown %s type %d"),') + ret.format(' "${tname}", ${var});', + tname=3Dmember['xmlattr'], var=3Dvar) + ret.format(' return -1;') + ret.format('}') + ret.format('virBufferAsprintf(buf, "${layout}", str);', + layout=3Dlayout) + return ret + elif mtype['name'] =3D=3D 'Bool': + truevalue =3D member.get('truevalue', 'yes') + if truevalue =3D=3D 'yes': + var =3D '%s ? "yes" : "no"' % var + elif truevalue =3D=3D 'on': + var =3D '%s ? "on" : "off"' % var + else: + var =3D '%s ? "%s" : ""' % (var, truevalue) + return ['virBufferEscapeString(buf, "%s", %s);' % (layout, var= )] + elif mtype['name'] =3D=3D 'String': + return ['virBufferEscapeString(buf, "%s", %s);' % (layout, var= )] + elif mtype['name'] =3D=3D 'Time': + return ['virTimeFormatBuf(buf, "%s", %s);' % (layout, var)] + else: + return ['virBufferAsprintf(buf, "%s", %s);' % (layout, var)] + + def _handleAttr(tagname, var): + if 'xmlattr' not in member: + return None + + fmt =3D '%s' + if mtype['meta'] =3D=3D 'Builtin': + fmt =3D BUILTIN_TYPES[mtype['name']].get('fmt', '%s') + + layout =3D " %s=3D'%s'" % (tagname, fmt) + return _format(layout, var) + + def _handleElem(tagname, var): + if 'xmlattr' in member: + return None + + if mtype['meta'] !=3D 'Struct': + layout =3D '<%s>%%s\\n' % (tagname, tagname) + else: + layout =3D tagname + + code =3D _format(layout, var) + return code + + # + # Main routine + # + name =3D member['name'] + if member.get('array'): + name =3D name + '[i]' + var =3D 'def->' + name + + ret =3D None + if 'xmlattr' in member: + tagname =3D member['xmlattr'] + ret =3D _handleAttr(tagname, var) + else: + tagname =3D member['xmlelem'] + ret =3D _handleElem(tagname, var) + + if not ret: + return None, None + + if member.get('array'): + return _handleMore(ret) + + checks =3D _checkOnCondition(var) + if checks: + ret =3D if_cond(checks, ret) + + if checks: + if '&&' in checks or '||' in checks: + checks =3D '(%s)' % checks + + return ret, checks + + +def makeSwitch(switch, callback, opaque=3DNone): + assert switch.get('xmlswitch', None) and switch.get('switch_type', Non= e) + + captype =3D Terms.allcaps(switch['switch_type']) + block =3D Block() + block.format('switch (def->${etype}) {', etype=3Dswitch['xmlswitch']) + block.newline() + for child in switch['members']: + value =3D captype + '_' + Terms.allcaps(child['name']) + block.format('case ${value}:', value=3Dvalue) + block.mapfmt(' ${_each_line_}', callback(child, switch, opaque)) + block.format(' break;') + block.newline() + block.format('case ${captype}_NONE:', captype=3Dcaptype) + block.format('case ${captype}_LAST:', captype=3Dcaptype) + block.format(' break;') + block.format('}') + return block + + +def makeFormatFunc(writer, atype): + if 'genformat' not in atype: + return + + # + # Helper functions and classes. + # + def _handleHeads(tmpvars, typename, kind=3D''): + tmpvars =3D dedup(tmpvars) + heads =3D Block() + if tmpvars: + heads.mapfmt( + 'g_auto(virBuffer) ${_each_line_} =3D VIR_BUFFER_INITIALIZ= ER;', + tmpvars + ) + heads.newline() + + heads.format('if (!def || !buf)') + heads.format(' return 0;') + + if tmpvars: + funcname =3D typename + 'Format' + kind + 'Hook' + + heads.newline() + args =3D Block(['def', 'parent', 'opaque']) + args.mapfmt('&${_each_line_}', tmpvars) + heads.format('if (%s(%s) < 0)' % (funcname, ', '.join(args))) + heads.format(' return -1;') + + args =3D Block([ + 'const %s *def' % typename, + 'const void *parent', + 'const void *opaque' + ]) + args.mapfmt('virBufferPtr ${_each_line_}', tmpvars) + signature =3D funcSignature('int', funcname, args) + writer.write(atype, 'formatfunc', '.h', signature.output() + '= ;') + + args =3D [ + 'const %s *def' % typename, + 'const void *parent', + 'void *opaque', + 'bool value' + ] + funcname =3D typename + 'Check' + kind + 'Hook' + signature =3D funcSignature('bool', funcname, args) + writer.write(atype, 'formatfunc', '.h', signature.output() + '= ;') + + return '\n'.join(heads) + + def _format_group(child, switch, kind): + kind =3D kind if kind else '' + prefix =3D (switch['name'] + '.') if switch else '' + mtype =3D TypeTable().get(child['type']) + funcname =3D '%sFormat%s' % (mtype['name'], kind) + var =3D 'def->%s%s' % (prefix, child['name']) + if not ispointer(child): + var =3D '&' + var + r_check =3D '%sCheck%s(%s, def, opaque)' % (mtype['name'], kind, v= ar) + cond =3D '%s(buf, %s, def, opaque) < 0' % (funcname, var) + return if_cond(cond, ['return -1;']), r_check + + def _switch_format_cb(child, switch, kind): + return _format_group(child, switch, kind)[0] + + def _handle_wrap_attr(member): + wrap, member['xmlattr'] =3D member['xmlattr'].split('/') + ret, check =3D formatMember(member, atype) + return ret, check, wrap + + def _switch_check_cb(child, switch, kind): + return ['ret =3D %s;' % _format_group(child, switch, kind)[1]] + + def _prepare_member(member, atype): + wrap, attr, attr_chk, elem, elem_chk =3D None, None, None, None, N= one + if member.get('xmlswitch'): + attr =3D makeSwitch(member, _switch_format_cb, 'Attr') + elem =3D makeSwitch(member, _switch_format_cb, 'Elem') + basename =3D atype['name'] + Terms.upperInitial(member['name']) + attr_chk =3D '%sCheckAttr(def, opaque)' % basename + elem_chk =3D '%sCheckElem(def, opaque)' % basename + + # Declare virXXXCheck[Attr|Elem] for switch. + for kind in ['Attr', 'Elem']: + args =3D ['const %s *def' % atype['name'], 'void *opaque'] + decl =3D funcSignature('bool', basename + 'Check' + kind, = args) + writer.write(atype, 'formatfunc', '.h', decl.output() + ';= ') + + # Implement virXXXCheck[Attr|Elem] for switch. + checks =3D makeSwitch(member, _switch_check_cb, kind) + + args[1] +=3D ' G_GNUC_UNUSED' + impl =3D funcSignature('bool', basename + 'Check' + kind, = args) + impl.format('{') + impl.format(' bool ret =3D false;') + impl.format(' if (!def)') + impl.format(' return false;') + impl.newline() + impl.mapfmt(' ${_each_line_}', checks) + impl.newline() + impl.format(' return ret;') + impl.format('}') + writer.write(atype, 'formatfunc', '.c', impl.output()) + + elif member.get('xmlattr'): + if '/' in member['xmlattr']: + attr, attr_chk, wrap =3D _handle_wrap_attr(member) + else: + attr, attr_chk =3D formatMember(member, atype) + elif member.get('xmlelem'): + elem, elem_chk =3D formatMember(member, atype) + elif member.get('xmlgroup'): + attr, attr_chk =3D _format_group(member, None, 'Attr') + elem, elem_chk =3D _format_group(member, None, 'Elem') + return wrap, attr, attr_chk, elem, elem_chk + + def _prepare_hook(member): + assert member.get('xmlattr') or member.get('xmlelem') + buf =3D member['name'] + 'Buf' + ret =3D if_cond('virBufferUse(&%s)' % buf, + ['virBufferAddBuffer(buf, &%s);' % buf]) + return ret, buf + + class _WrapItem: + def __init__(self): + self.attrs =3D Block() + self.checks =3D [] + self.optional =3D True + self.pos =3D 0 + + def _prepare(): + attrs =3D Block() + elems =3D Block() + attr_checks =3D [] + elem_checks =3D [] + attr_hook_vars =3D [] + elem_hook_vars =3D [] + attrs_optional =3D True + elems_optional =3D True + wraps =3D OrderedDict() + for member in atype['members']: + if member.get('formathook'): + block, hookvar =3D _prepare_hook(member) + if member.get('xmlattr'): + attrs.extend(block) + attrs.newline(block) + attr_hook_vars.append(hookvar) + elif member.get('xmlelem'): + elems.extend(block) + elems.newline(block) + elem_hook_vars.append(hookvar) + else: + assert False, 'formathook is only with [xmlattr|xmlele= m].' + else: + wrap, attr, attr_chk, elem, elem_chk =3D \ + _prepare_member(member, atype) + if wrap: + item =3D wraps.setdefault(wrap, _WrapItem()) + item.pos =3D len(elems) + item.attrs.extend(attr) + item.attrs.newline() + item.checks.append(attr_chk) + if member.get('required'): + item.optional =3D False + continue + + attrs.extend(attr) + attrs.newline(attr) + elems.extend(elem) + elems.newline(elem) + if attr_chk: + attr_checks.append(attr_chk) + if elem_chk: + elem_checks.append(elem_chk) + if member.get('required'): + attrs_optional =3D False + + while wraps: + wrap, item =3D wraps.popitem() + lines =3D Block() + lines.format('virBufferAddLit(buf, "<${name}");', name=3Dwrap) + lines.newline() + lines.extend(item.attrs) + lines.format('virBufferAddLit(buf, "/>\\n");') + if item.optional: + elem_checks.extend(item.checks) + lines =3D if_cond(' || '.join(item.checks), lines) + lines.newline() + else: + elems_optional =3D False + + for line in reversed(lines): + elems.insert(item.pos, line) + + attr_checks =3D dedup(attr_checks) + elem_checks =3D dedup(elem_checks) + return (attrs, attr_checks, attrs_optional, attr_hook_vars, + elems, elem_checks, elems_optional, elem_hook_vars) + + def _check_null(optional, checks, has_hook, kind=3D''): + if not optional: + return None + + # Declare virXXXCheck[Attr|Elem] + typename =3D atype['name'] + funcname =3D typename + 'Check' + kind + args =3D [ + 'const %s *def' % typename, + 'const void *parent', + 'void *opaque' + ] + signature =3D funcSignature('bool', funcname, args) + writer.write(atype, 'formatfunc', '.h', signature.output() + ';') + + # Implement virXXXFormat[Attr|Elem] + check =3D ' || '.join(checks) if optional else 'true' + if not check: + check =3D 'true' + + if has_hook: + check =3D '%sCheck%sHook(def, parent, opaque, %s)' \ + % (typename, kind, check) + + args[1] +=3D ' G_GNUC_UNUSED' + args[2] +=3D ' G_GNUC_UNUSED' + + impl =3D funcSignature('bool', funcname, args) + impl.format('{') + impl.format(' if (!def)') + impl.format(' return false;') + impl.newline() + impl.format(' return ${check};', check=3Dcheck) + impl.format('}') + writer.write(atype, 'formatfunc', '.c', impl.output()) + + return if_cond('!%s(def, parent, opaque)' % funcname, ['return 0;'= ]) + + def _compose_full(attrs, attr_checks, attrs_optional, attr_hook_vars, + elems, elem_checks, elems_optional, elem_hook_vars): + has_hook =3D (attr_hook_vars or elem_hook_vars) + typename =3D atype['name'] + + # Declare virXXXFormatBuf + args =3D [ + 'virBufferPtr buf', + 'const char *name', + 'const %s *def' % typename, + 'const void *parent', + 'void *opaque' + ] + signature =3D funcSignature('int', typename + 'FormatBuf', args) + writer.write(atype, 'formatfunc', '.h', signature.output() + ';') + + # Implement virXXXFormatBuf + headlines =3D _handleHeads(attr_hook_vars + elem_hook_vars, typena= me) + checknull =3D _check_null(attrs_optional and elems_optional, + attr_checks + elem_checks, has_hook) + + args[3] +=3D ' G_GNUC_UNUSED' + args[4] +=3D ' G_GNUC_UNUSED' + + impl =3D funcSignature('int', typename + 'FormatBuf', args) + impl.format('{') + impl.mapfmt(' ${_each_line_}', headlines.split('\n')) + impl.newline() + + if checknull: + impl.mapfmt(' ${_each_line_}', checknull) + impl.newline() + + impl.format(' virBufferAsprintf(buf, "<%s", name);') + impl.newline() + + if 'namespace' in atype: + impl.mapfmt(' ${_each_line}', T_NAMESPACE_FORMAT_BEGIN) + + impl.mapfmt(' ${_each_line_}', attrs) + + if elems: + if attrs and elems_optional: + impl.format(' if (!(${checks})) {', + checks=3D' || '.join(elem_checks)) + impl.format(' virBufferAddLit(buf, "/>\\n");') + impl.format(' return 0;') + impl.format(' }') + impl.newline() + + if 'namespace' in atype: + impl.mapfmt(' ${_each_line}', T_NAMESPACE_FORMAT_END) + + impl.format(' virBufferAddLit(buf, ">\\n");') + impl.newline() + impl.format(' virBufferAdjustIndent(buf, 2);') + impl.newline() + impl.mapfmt(' ${_each_line_}', elems) + impl.format(' virBufferAdjustIndent(buf, -2);') + impl.format(' virBufferAsprintf(buf, "\\n", name);') + else: + impl.format(' virBufferAddLit(buf, "/>\\n");') + + impl.newline() + impl.format(' return 0;') + impl.format('}') + writer.write(atype, 'formatfunc', '.c', impl.output()) + + def _compose_part(kind, block, checks, optional, hook_vars): + typename =3D atype['name'] + funcname =3D typename + 'Format' + kind + headlines =3D _handleHeads(hook_vars, atype['name'], kind) + if not block: + block =3D ['/* empty */', ''] + + checknull =3D _check_null(optional, checks, len(hook_vars), kind) + + # Declare virXXXFormat[Attr|Elem] + args =3D [ + 'virBufferPtr buf', + 'const %s *def' % typename, + 'const void *parent', + 'void *opaque' + ] + signature =3D funcSignature('int', funcname, args) + writer.write(atype, 'formatfunc', '.h', signature.output() + ';') + + # Implement virXXXFormat[Attr|Elem] + args[2] +=3D ' G_GNUC_UNUSED' + args[3] +=3D ' G_GNUC_UNUSED' + + impl =3D funcSignature('int', funcname, args) + impl.format('{') + impl.mapfmt(' ${_each_line_}', headlines.split('\n')) + impl.newline() + + if checknull: + impl.mapfmt(' ${_each_line_}', checknull) + impl.newline() + + impl.mapfmt(' ${_each_line_}', block) + impl.format(' return 0;') + impl.format('}') + writer.write(atype, 'formatfunc', '.c', impl.output()) + + # + # Main routine of formating. + # + (attrs, attr_checks, attrs_optional, attr_hook_vars, + elems, elem_checks, elems_optional, elem_hook_vars) =3D _prepare() + + if atype['genformat'] in ['separate', 'onlyattrs', 'onlyelems']: + if atype['genformat'] in ['separate', 'onlyattrs']: + _compose_part('Attr', attrs, attr_checks, + attrs_optional, attr_hook_vars) + + if atype['genformat'] in ['separate', 'onlyelems']: + _compose_part('Elem', elems, elem_checks, + elems_optional, elem_hook_vars) + else: + _compose_full(attrs, attr_checks, attrs_optional, attr_hook_vars, + elems, elem_checks, elems_optional, elem_hook_vars) + + +def showDirective(atype): + print('\n[Directive]\n') + print(json.dumps(atype, indent=3D4)) diff --git a/scripts/xmlgen/go b/scripts/xmlgen/go new file mode 100755 index 0000000..3821ce0 --- /dev/null +++ b/scripts/xmlgen/go @@ -0,0 +1,7 @@ +# This is a command-line tool + +export PYTHONDONTWRITEBYTECODE=3D1 +WORK_DIR=3D$(cd $(dirname $0); pwd) +SRCDIR=3D"${WORK_DIR}/../../src" +BUILDDIR=3D"${WORK_DIR}/../../build/src" +${WORK_DIR}/main.py -s ${SRCDIR} -b ${BUILDDIR} $@ diff --git a/scripts/xmlgen/main.py b/scripts/xmlgen/main.py new file mode 100755 index 0000000..5054640 --- /dev/null +++ b/scripts/xmlgen/main.py @@ -0,0 +1,439 @@ +#!/usr/bin/env python3 +# +# Copyright (C) 2020 Shandong Massclouds Co.,Ltd. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# + +import os +import re +import sys +import argparse +from clang.cindex import Index, CursorKind +from clang.cindex import SourceLocation, SourceRange, TokenKind +from datetime import datetime +from directive import TypeTable, showDirective +from directive import makeClearFunc, makeParseFunc, makeFormatFunc +from utils import Terms + +TOOL_DESC =3D ''' +Generate xml parse/format functions based on directives. + +Subcommand:\n + list: List types. By default, only list structs tagged by + 'genparse'/'genformat'. When the option '-a' is specified, + list all types discovered by this tool.\n + show: Show the target type's directives and its code for preview. + Specify target type by its name. The option '-k' indicates + the kinds of code for preview.\n + generate: Generate code. To be called by Makefile. + It needs option -k to filter output. + +Option:\n + -k: Specify kinds to filter code output. More than one kind can be + specified, 'c' for clearfunc; 'p' for parsefunc; + 'f' for formatfunc.\n + The option '-k' is only valid for show and generate. +''' + +# Three builtin types need to be handled specially: +# 'char *' =3D> String +# 'char XXXX[...]' =3D> Chars +# 'unsigned char XXXX[...]' =3D> UChars +BUILTIN_MAP =3D { + 'bool': 'Bool', + 'char': 'Char', + 'unsigned char': 'UChar', + 'int': 'Int', + 'unsigned int': 'UInt', + 'long': 'Long', + 'unsigned long': 'ULong', + 'long long': 'LongLong', + 'unsigned long long': 'ULongLong', + 'uint8_t': 'U8', + 'uint32_t': 'U32', + 'time_t': 'Time', +} + + +def getBuiltinType(ctype, ptr=3DFalse, size=3DNone): + if ctype =3D=3D 'char': + if ptr: + return 'String' + elif size: + return 'Chars' + + if ctype =3D=3D 'unsigned char' and size: + return 'UChars' + + return BUILTIN_MAP.get(ctype, None) + + +def cursorLineExtent(cursor, tu): + loc =3D cursor.location + start =3D SourceLocation.from_position(tu, loc.file, loc.line, 1) + end =3D SourceLocation.from_position(tu, loc.file, loc.line, -1) + return SourceRange.from_locations(start, end) + + +def getTokens(cursor, tu): + return tu.get_tokens(extent=3DcursorLineExtent(cursor, tu)) + + +DIRECTIVES =3D [ + 'genparse', 'genformat', 'namespace', 'xmlattr', 'xmlelem', 'xmlgroup', + 'required', 'array', 'specified', 'truevalue', 'formathook', 'xmlswitc= h', + 'formatflag' +] + + +def createDirectives(text): + tlist =3D re.findall(r'/\*(.*)\*/', text) + if len(tlist) !=3D 1: + return None + + tlist =3D tlist[0].split(',') + if len(tlist) =3D=3D 0: + return None + + directives =3D {} + for item in tlist: + item =3D item.strip() + if ':' in item: + key, value =3D item.split(':') + else: + key, value =3D item, None + + if key in DIRECTIVES: + directives[key] =3D value + return directives + + +def getDirectives(tokens, cursor): + for token in tokens: + if token.location.column <=3D cursor.location.column: + continue + if token.kind =3D=3D TokenKind.COMMENT: + directive =3D createDirectives(token.spelling) + if directive: + return directive + return None + + +def determinType(kvs, tokens, cursor): + prefix =3D [] + kind =3D None + for token in tokens: + if token.location.column >=3D cursor.location.column: + break + if not kind: + kind =3D token.kind + prefix.append(token.spelling) + + suffix =3D [] + for token in tokens: + if token.spelling =3D=3D ';': + break + suffix.append(token.spelling) + + size =3D None + if len(suffix) =3D=3D 3 and suffix[0] =3D=3D '[' and suffix[2] =3D=3D = ']': + size =3D suffix[1] + + assert kind in [TokenKind.IDENTIFIER, TokenKind.KEYWORD], \ + 'Bad field "%s" (%s).' % (cursor.spelling, kind) + + assert prefix + typename =3D ' '.join(prefix) + + # For array, remove the most-outer pointer + if kvs.get('array'): + if typename.endswith('Ptr'): + typename =3D typename[:-3] + elif typename.endswith('*'): + typename =3D typename[:-1].strip() + + ptr =3D False + if typename.endswith('Ptr'): + typename =3D typename[:-3] + ptr =3D True + elif prefix[-1] =3D=3D '*': + typename =3D typename[:-1].strip() + ptr =3D True + + ret =3D getBuiltinType(typename, ptr, size) + if ret: + typename =3D ret + + kvs.update({'type': typename, 'pointer': ptr}) + if size: + kvs['size'] =3D size + return kvs + + +def analyseMember(cursor, tu): + dvs =3D getDirectives(getTokens(cursor, tu), cursor) + if not dvs: + return None + + kvs =3D {'name': cursor.spelling} + kvs.update(dvs) + + # Formalize member + for key in ['array', 'required', 'specified', 'formathook']: + if key in kvs and not kvs[key]: + kvs[key] =3D True + + if 'formatflag' in kvs: + assert kvs.get('formatflag'), 'Directive "formatflag" is None' + + for tag in ['xmlattr', 'xmlelem', 'xmlgroup']: + if tag in kvs: + if not kvs[tag]: + if kvs.get('array'): + kvs[tag] =3D Terms.singularize(kvs['name']) + else: + kvs[tag] =3D kvs['name'] + + return determinType(kvs, getTokens(cursor, tu), cursor) + + +def findMember(struct, name): + for member in struct['members']: + if member['name'] =3D=3D name: + return member + + return None + + +def analyseStruct(struct, cursor, tu): + tokens =3D getTokens(cursor, tu) + kvs =3D getDirectives(tokens, cursor) + if kvs: + path, _ =3D os.path.splitext(cursor.location.file.name) + path, filename =3D os.path.split(path) + _, dirname =3D os.path.split(path) + kvs['output'] =3D dirname + '/' + filename + struct.update(kvs) + + last_kind =3D None + inner_members =3D [] + for child in cursor.get_children(): + if inner_members: + if last_kind =3D=3D CursorKind.STRUCT_DECL: + # Flatten the members of embedded struct + for member in inner_members: + member['name'] =3D child.spelling + '.' + member['= name'] + struct['members'].append(member) + else: + # Embedded union + union =3D getDirectives(getTokens(child, tu), child) + if union and 'xmlswitch' in union: + switch =3D findMember(struct, union['xmlswitch']) + switch['delay_clear'] =3D True + union['switch_type'] =3D switch['type'] + union['name'] =3D child.spelling + union['members'] =3D inner_members + struct['members'].append(union) + struct['xmlswitch'] =3D union['xmlswitch'] + + inner_members =3D [] + last_kind =3D None + continue + + if child.kind in [CursorKind.STRUCT_DECL, CursorKind.UNION_DEC= L]: + last_kind =3D child.kind + for ichild in child.get_children(): + member =3D analyseMember(ichild, tu) + if member: + inner_members.append(member) + continue + + member =3D analyseMember(child, tu) + if member: + struct['members'].append(member) + + return struct + + +def discoverStructures(tu): + for cursor in tu.cursor.get_children(): + if cursor.kind =3D=3D CursorKind.STRUCT_DECL and cursor.is_definit= ion(): + # Detect structs + name =3D cursor.spelling + if not name: + continue + if name.startswith('_'): + name =3D name[1:] + if TypeTable().check(name): + continue + struct =3D {'name': name, 'meta': 'Struct', 'members': []} + analyseStruct(struct, cursor, tu) + TypeTable().register(struct) + elif cursor.kind =3D=3D CursorKind.TYPEDEF_DECL: + # Detect enums + # We can't seek out enums by CursorKind.ENUM_DECL, + # since almost all enums are anonymous. + token =3D cursor.get_tokens() + try: + next(token) # skip 'typedef' + if next(token).spelling =3D=3D 'enum': + if not TypeTable().check(cursor.spelling): + enum =3D {'name': cursor.spelling, 'meta': 'Enum'} + TypeTable().register(enum) + except StopIteration: + pass + + +class CodeWriter(object): + def __init__(self, args): + self._builddir =3D args.builddir + self._cmd =3D args.cmd + self._files =3D {} + self._filters =3D {} + self._filters['clearfunc'] =3D args.kinds and 'c' in args.kinds + self._filters['parsefunc'] =3D args.kinds and 'p' in args.kinds + self._filters['formatfunc'] =3D args.kinds and 'f' in args.kinds + if args.cmd =3D=3D 'show': + self._filters['target'] =3D args.target + + def _getFile(self, path, ext): + assert ext in ['.h', '.c'] + _, basename =3D os.path.split(path) + path =3D '%s.generated%s' % (path, ext) + f =3D self._files.get(path) + if f is None: + f =3D open(path, 'w') + f.write('/* Generated by scripts/xmlgen */\n\n') + if ext in ['.c']: + f.write('#include \n') + f.write('#include "%s.h"\n' % basename) + f.write('#include "viralloc.h"\n') + f.write('#include "virerror.h"\n') + f.write('#include "virstring.h"\n\n') + f.write('#define VIR_FROM_THIS VIR_FROM_NONE\n') + else: + f.write('#pragma once\n\n') + f.write('#include "internal.h"\n') + f.write('#include "virxml.h"\n') + self._files[path] =3D f + return f + + def write(self, atype, kind, extname, content): + if not self._filters[kind]: + return + + if self._cmd =3D=3D 'show': + target =3D self._filters['target'] + if not target or target =3D=3D atype['name']: + if extname =3D=3D '.h': + print('\n[.h]') + else: + print('\n[.c]') + print('\n' + content) + return + + assert self._cmd =3D=3D 'generate' + + if atype.get('output'): + lfs =3D '\n' if extname =3D=3D '.h' else '\n\n' + path =3D self._builddir + '/' + atype['output'] + f =3D self._getFile(path, extname) + f.write(lfs + content + '\n') + + def complete(self): + for name in self._files: + self._files[name].close() + self._files.clear() + + +def getHFiles(path): + retlist =3D [] + for fname in os.listdir(path): + if fname.endswith('.h'): + retlist.append(os.path.join(path, fname)) + return retlist + + +HELP_LIST =3D 'list structs tagged by "genparse"/"genformat"' +HELP_LIST_ALL =3D 'list all discovered types' + +if __name__ =3D=3D "__main__": + parser =3D argparse.ArgumentParser( + formatter_class=3Dargparse.RawDescriptionHelpFormatter, + description=3DTOOL_DESC) + parser.add_argument('-s', dest=3D'srcdir') + parser.add_argument('-b', dest=3D'builddir') + subparsers =3D parser.add_subparsers(dest=3D'cmd') + parser_list =3D subparsers.add_parser('list', help=3DHELP_LIST) + parser_list.add_argument('-a', dest=3D'list_all', action=3D'store_true= ', + default=3DFalse, help=3DHELP_LIST_ALL) + parser_show =3D subparsers.add_parser('show', help=3D'show target code= ') + parser_show.add_argument('target', help=3D'target for being previewed') + parser_show.add_argument('-k', dest=3D'kinds', + help=3D'kinds of code to be previewed') + parser_generate =3D subparsers.add_parser('generate', help=3D'generate= code') + parser_generate.add_argument('-k', dest=3D'kinds', + help=3D'kinds of code to be generated') + args =3D parser.parse_args() + + if not args.cmd or not args.srcdir or not args.builddir: + parser.print_help() + sys.exit(1) + + if args.cmd =3D=3D 'generate': + print('###### xmlgen: start ... ######') + if not args.kinds: + print("[dry run]: no kinds specified for 'generate'") + + timestamp =3D datetime.now() + + # Examine all *.h in "$(srcdir)/[util|conf]" + index =3D Index.create() + srcdir =3D args.srcdir + hfiles =3D getHFiles(srcdir + '/util') + getHFiles(srcdir + '/conf') + for hfile in hfiles: + tu =3D index.parse(hfile) + discoverStructures(tu) # find all structs and enums + + if args.cmd =3D=3D 'list': + print('%-64s %s' % ('TYPENAME', 'META')) + for name, kvs in TypeTable().items(): + if not args.list_all: + if not ('genparse' in kvs or 'genparse' in kvs): + continue + print('%-64s %s' % (name, kvs['meta'])) + sys.exit(0) + elif args.cmd =3D=3D 'show': + assert args.target, args + atype =3D TypeTable().get(args.target) + if not atype: + sys.exit(0) + showDirective(atype) + + writer =3D CodeWriter(args) + + for atype in TypeTable().values(): + makeClearFunc(writer, atype) + makeParseFunc(writer, atype) + makeFormatFunc(writer, atype) + + writer.complete() + + if args.cmd =3D=3D 'generate': + elapse =3D (datetime.now() - timestamp).microseconds + print('\n###### xmlgen: elapse %d(us) ######\n' % elapse) + + sys.exit(0) diff --git a/scripts/xmlgen/utils.py b/scripts/xmlgen/utils.py new file mode 100644 index 0000000..4ac1f39 --- /dev/null +++ b/scripts/xmlgen/utils.py @@ -0,0 +1,121 @@ +# +# Copyright (C) 2020 Shandong Massclouds Co.,Ltd. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# + +from string import Template + + +def singleton(cls): + _instances =3D {} + + def inner(): + if cls not in _instances: + _instances[cls] =3D cls() + return _instances[cls] + return inner + + +class Terms(object): + abbrs =3D ['uuid', 'pci', 'zpci', 'ptr', 'mac', 'mtu', 'dns', 'ip', 'd= hcp'] + plurals =3D {'addresses': 'address'} + caps =3D {'NET_DEV': 'NETDEV', 'MACTABLE': 'MAC_TABLE'} + + @classmethod + def _split(cls, word): + ret =3D [] + if not word: + return ret + head =3D 0 + for pos in range(1, len(word)): + if word[pos].isupper() and not word[pos - 1].isupper(): + ret.append(word[head:pos]) + head =3D pos + ret.append(word[head:]) + return ret + + @classmethod + def singularize(cls, name): + ret =3D cls.plurals.get(name, None) + if ret: + return ret + assert name.endswith('s') + return name[:-1] + + # Don't use str.capitalize() which force other letters to be lowercase. + @classmethod + def upperInitial(cls, word): + if not word: + return '' + if word in cls.abbrs: + return word.upper() + if len(word) > 0 and word[0].isupper(): + return word + return word[0].upper() + word[1:] + + @classmethod + def allcaps(cls, word): + if len(word) =3D=3D 0: + return word + parts =3D cls._split(word) + ret =3D '_'.join([part.upper() for part in parts]) + for key, value in cls.caps.items(): + ret =3D ret.replace('_%s_' % key, '_%s_' % value) + return ret + + +def render(template, **kwargs): + return Template(template).substitute(kwargs) + + +class Block(list): + def format(self, template, **args): + if template: + self.append(Template(template).substitute(**args)) + + def extend(self, block): + if isinstance(block, list): + super(Block, self).extend(block) + + # ${_each_line_} is the only legal key for template + # and represents each line of the block. + def mapfmt(self, template, block): + if not block or not template: + return + + assert isinstance(block, list), block + for line in block: + if line: + self.append(Template(template).substitute(_each_line_=3Dli= ne)) + else: + self.append('') + + def newline(self, condition=3DTrue): + if condition: + super(Block, self).append('') + + def output(self, connector=3D'\n'): + return connector.join(self) + + +def dedup(alist): + assert isinstance(alist, list) + ret =3D [] + for e in alist: + if e not in ret: + ret.append(e) + + return ret --=20 2.25.1