From nobody Sun Oct 5 01:45:11 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 15517269382360.6954925197507009; Mon, 4 Mar 2019 11:15:38 -0800 (PST) Received: from localhost ([127.0.0.1]:59420 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0t3h-0006A8-Fn for importer@patchew.org; Mon, 04 Mar 2019 14:15:29 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45655) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0sCO-0001lF-AL for qemu-devel@nongnu.org; Mon, 04 Mar 2019 13:20:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h0sCM-0003eq-3h for qemu-devel@nongnu.org; Mon, 04 Mar 2019 13:20:24 -0500 Received: from mail-wr1-x42e.google.com ([2a00:1450:4864:20::42e]:39620) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1h0sCL-0003dR-Nr for qemu-devel@nongnu.org; Mon, 04 Mar 2019 13:20:22 -0500 Received: by mail-wr1-x42e.google.com with SMTP id l5so6630487wrw.6 for ; Mon, 04 Mar 2019 10:20:21 -0800 (PST) Received: from 640k.lan ([93.56.166.5]) by smtp.gmail.com with ESMTPSA id q5sm8371364wrn.43.2019.03.04.10.20.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Mar 2019 10:20:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OhD6HQS2fsUbDNTMT18XrFba46DAk0OcTFU5gDGhqPY=; b=djl58/fxhuZSHXKM8Bov8JG2Q2mYfl3yIn6M9jjqV0rikw9S3mhRsDQZeFoYonv0Zo lRNznaVHvSYz/EzmOpcUn0xZyeEzRnc9OVaWWeEpZN3WGvfbQfoSUcqIvE2vLEVok5s1 P4HgF0pLZTfqXKv0RXkF8FkJp1/Qs0xHwKfuopNt3k9dpGRQLdekVVIwJjk8Dm3Ev/j9 0QWpz4ezIHGIztB600AKF1SzHMJq3NSnpsmz760enbRi1euRC/tmJTkc/H/S6EeBCPQM I88ixOiMXm3p/YNGlYy+1EcHvs1tRxbWiCjG3fr87J26W0qEoVn8eedoGc9eZbk77qMR EbrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=OhD6HQS2fsUbDNTMT18XrFba46DAk0OcTFU5gDGhqPY=; b=h2dlkvA8iwuKuil/dRBd7eMDxEz2MtsHPmj4oqH2G/I+963ck8ZfZRSWgSWHDVkKu1 ENGK99VF7gGc4/mgtv/Wn2sUdt5cnuVcJQbB+AnqXkPkvgFpAng0K/eGr50FLQOWUmL8 rRPUD2dCLFM98cJek5MrkVt7FHGOqptG6uLaR1oc9V1kuUG22yZgT7XFix3QZZMaThP7 +R4G5edbl6koNlWKrXm1IwIsAxseBcZNKSxq3XP+e7bu42RfieF4JyV8vxL+ddusj1bZ 6bQf3kzhHFzpXf2DDNciZgimniyHJkhpoM/+5vZH/KRCaaX2hcY1AudAdpW0osBweBUr pNWQ== X-Gm-Message-State: APjAAAUmuqov6bq0mSBHFzebJL/mJ3I7nBsN+bDwJr7upsUKGrMStkGe VF87UVPEvTDQ4BXAC0q9c4jExPXb X-Google-Smtp-Source: APXvYqys+FL1zIqoLolhgZhVfOPGhDWg4FuudiPmSo9EQCAkGT4iqt7GsCLRCTiDTC/Tg1CXMP7Kxg== X-Received: by 2002:a5d:4147:: with SMTP id c7mr14380829wrq.235.1551723620199; Mon, 04 Mar 2019 10:20:20 -0800 (PST) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Mon, 4 Mar 2019 19:19:24 +0100 Message-Id: <1551723614-1823-5-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1551723614-1823-1-git-send-email-pbonzini@redhat.com> References: <1551723614-1823-1-git-send-email-pbonzini@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::42e Subject: [Qemu-devel] [PULL 04/54] minikconfig: add parser skeleton X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yang Zhong , thuth@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This implements a scanner and recursive descent parser for Kconfig-like configuration files. The only "action" of the parser is for now to detect undefined variables and process include files. The main differences between Kconfig and this are: * only the "bool" type is supported * variables can only be defined once * choices are not supported (but they could be added as syntactic sugar for multiple Boolean values) * menus and other graphical concepts (prompts, help text) are not supported * assignments ("CONFIG_FOO=3Dy", "CONFIG_FOO=3Dn") are parsed as part of the Kconfig language, not as a separate file. The idea was originally by =C3=81kos Kov=C3=A1cs, but I could not find his implementation so I had to redo it. Signed-off-by: Paolo Bonzini Message-Id: <20190123065618.3520-23-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini --- scripts/minikconf.py | 441 +++++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 441 insertions(+) create mode 100644 scripts/minikconf.py diff --git a/scripts/minikconf.py b/scripts/minikconf.py new file mode 100644 index 0000000..fd75d96 --- /dev/null +++ b/scripts/minikconf.py @@ -0,0 +1,441 @@ +# +# Mini-Kconfig parser +# +# Copyright (c) 2015 Red Hat Inc. +# +# Authors: +# Paolo Bonzini +# +# This work is licensed under the terms of the GNU GPL, version 2 +# or, at your option, any later version. See the COPYING file in +# the top-level directory. + +from __future__ import print_function +import os +import sys + +__all__ =3D [ 'KconfigParserError', 'KconfigData', 'KconfigParser' ] + +def debug_print(*args): + #print('# ' + (' '.join(str(x) for x in args))) + pass + +# ------------------------------------------- +# KconfigData implements the Kconfig semantics. For now it can only +# detect undefined symbols, i.e. symbols that were referenced in +# assignments or dependencies but were not declared with "config FOO". +# +# Semantic actions are represented by methods called do_*. The do_var +# method return the semantic value of a variable (which right now is +# just its name). +# ------------------------------------------- + +class KconfigData: + def __init__(self): + self.previously_included =3D [] + self.incl_info =3D None + self.defined_vars =3D set() + self.referenced_vars =3D set() + + # semantic analysis ------------- + + def check_undefined(self): + undef =3D False + for i in self.referenced_vars: + if not (i in self.defined_vars): + print("undefined symbol %s" % (i), file=3Dsys.stderr) + undef =3D True + return undef + + # semantic actions ------------- + + def do_declaration(self, var): + if (var in self.defined_vars): + raise Exception('variable "' + var + '" defined twice') + + self.defined_vars.add(var) + + # var is a string with the variable's name. + # + # For now this just returns the variable's name itself. + def do_var(self, var): + self.referenced_vars.add(var) + return var + + def do_assignment(self, var, val): + pass + + def do_default(self, var, val, cond=3DNone): + pass + + def do_depends_on(self, var, expr): + pass + + def do_select(self, var, symbol, cond=3DNone): + pass + + def do_imply(self, var, symbol, cond=3DNone): + pass + +# ------------------------------------------- +# KconfigParser implements a recursive descent parser for (simplified) +# Kconfig syntax. +# ------------------------------------------- + +# tokens table +TOKENS =3D {} +TOK_NONE =3D -1 +TOK_LPAREN =3D 0; TOKENS[TOK_LPAREN] =3D '"("'; +TOK_RPAREN =3D 1; TOKENS[TOK_RPAREN] =3D '")"'; +TOK_EQUAL =3D 2; TOKENS[TOK_EQUAL] =3D '"=3D"'; +TOK_AND =3D 3; TOKENS[TOK_AND] =3D '"&&"'; +TOK_OR =3D 4; TOKENS[TOK_OR] =3D '"||"'; +TOK_NOT =3D 5; TOKENS[TOK_NOT] =3D '"!"'; +TOK_DEPENDS =3D 6; TOKENS[TOK_DEPENDS] =3D '"depends"'; +TOK_ON =3D 7; TOKENS[TOK_ON] =3D '"on"'; +TOK_SELECT =3D 8; TOKENS[TOK_SELECT] =3D '"select"'; +TOK_IMPLY =3D 9; TOKENS[TOK_IMPLY] =3D '"imply"'; +TOK_CONFIG =3D 10; TOKENS[TOK_CONFIG] =3D '"config"'; +TOK_DEFAULT =3D 11; TOKENS[TOK_DEFAULT] =3D '"default"'; +TOK_Y =3D 12; TOKENS[TOK_Y] =3D '"y"'; +TOK_N =3D 13; TOKENS[TOK_N] =3D '"n"'; +TOK_SOURCE =3D 14; TOKENS[TOK_SOURCE] =3D '"source"'; +TOK_BOOL =3D 15; TOKENS[TOK_BOOL] =3D '"bool"'; +TOK_IF =3D 16; TOKENS[TOK_IF] =3D '"if"'; +TOK_ID =3D 17; TOKENS[TOK_ID] =3D 'identifier'; +TOK_EOF =3D 18; TOKENS[TOK_EOF] =3D 'end of file'; + +class KconfigParserError(Exception): + def __init__(self, parser, msg, tok=3DNone): + self.loc =3D parser.location() + tok =3D tok or parser.tok + if tok !=3D TOK_NONE: + location =3D TOKENS.get(tok, None) or ('"%s"' % tok) + msg =3D '%s before %s' % (msg, location) + self.msg =3D msg + + def __str__(self): + return "%s: %s" % (self.loc, self.msg) + +class KconfigParser: + @classmethod + def parse(self, fp): + data =3D KconfigData() + parser =3D KconfigParser(data) + parser.parse_file(fp) + if data.check_undefined(): + raise KconfigParserError(parser, "there were undefined symbols= ") + + return data + + def __init__(self, data): + self.data =3D data + + def parse_file(self, fp): + self.abs_fname =3D os.path.abspath(fp.name) + self.fname =3D fp.name + self.data.previously_included.append(self.abs_fname) + self.src =3D fp.read() + if self.src =3D=3D '' or self.src[-1] !=3D '\n': + self.src +=3D '\n' + self.cursor =3D 0 + self.line =3D 1 + self.line_pos =3D 0 + self.get_token() + self.parse_config() + + # file management ----- + + def error_path(self): + inf =3D self.data.incl_info + res =3D "" + while inf: + res =3D ("In file included from %s:%d:\n" % (inf['file'], + inf['line'])) + res + inf =3D inf['parent'] + return res + + def location(self): + col =3D 1 + for ch in self.src[self.line_pos:self.pos]: + if ch =3D=3D '\t': + col +=3D 8 - ((col - 1) % 8) + else: + col +=3D 1 + return '%s%s:%d:%d' %(self.error_path(), self.fname, self.line, co= l) + + def do_include(self, include): + incl_abs_fname =3D os.path.join(os.path.dirname(self.abs_fname), + include) + # catch inclusion cycle + inf =3D self.data.incl_info + while inf: + if incl_abs_fname =3D=3D os.path.abspath(inf['file']): + raise KconfigParserError(self, "Inclusion loop for %s" + % include) + inf =3D inf['parent'] + + # skip multiple include of the same file + if incl_abs_fname in self.data.previously_included: + return + try: + fp =3D open(incl_abs_fname, 'r') + except IOError as e: + raise KconfigParserError(self, + '%s: %s' % (e.strerror, include)) + + inf =3D self.data.incl_info + self.data.incl_info =3D { 'file': self.fname, 'line': self.line, + 'parent': inf } + KconfigParser(self.data).parse_file(fp) + self.data.incl_info =3D inf + + # recursive descent parser ----- + + # y_or_n: Y | N + def parse_y_or_n(self): + if self.tok =3D=3D TOK_Y: + self.get_token() + return True + if self.tok =3D=3D TOK_N: + self.get_token() + return False + raise KconfigParserError(self, 'Expected "y" or "n"') + + # var: ID + def parse_var(self): + if self.tok =3D=3D TOK_ID: + val =3D self.val + self.get_token() + return self.data.do_var(val) + else: + raise KconfigParserError(self, 'Expected identifier') + + # assignment_var: ID (starting with "CONFIG_") + def parse_assignment_var(self): + if self.tok =3D=3D TOK_ID: + val =3D self.val + if not val.startswith("CONFIG_"): + raise KconfigParserError(self, + 'Expected identifier starting with "CONFIG_"', = TOK_NONE) + self.get_token() + return self.data.do_var(val[7:]) + else: + raise KconfigParserError(self, 'Expected identifier') + + # assignment: var EQUAL y_or_n + def parse_assignment(self): + var =3D self.parse_assignment_var() + if self.tok !=3D TOK_EQUAL: + raise KconfigParserError(self, 'Expected "=3D"') + self.get_token() + self.data.do_assignment(var, self.parse_y_or_n()) + + # primary: NOT primary + # | LPAREN expr RPAREN + # | var + def parse_primary(self): + if self.tok =3D=3D TOK_NOT: + self.get_token() + self.parse_primary() + elif self.tok =3D=3D TOK_LPAREN: + self.get_token() + self.parse_expr() + if self.tok !=3D TOK_RPAREN: + raise KconfigParserError(self, 'Expected ")"') + self.get_token() + elif self.tok =3D=3D TOK_ID: + self.parse_var() + else: + raise KconfigParserError(self, 'Expected "!" or "(" or identif= ier') + + # disj: primary (OR primary)* + def parse_disj(self): + self.parse_primary() + while self.tok =3D=3D TOK_OR: + self.get_token() + self.parse_primary() + + # expr: disj (AND disj)* + def parse_expr(self): + self.parse_disj() + while self.tok =3D=3D TOK_AND: + self.get_token() + self.parse_disj() + + # condition: IF expr + # | empty + def parse_condition(self): + if self.tok =3D=3D TOK_IF: + self.get_token() + return self.parse_expr() + else: + return None + + # property: DEFAULT y_or_n condition + # | DEPENDS ON expr + # | SELECT var condition + # | BOOL + def parse_property(self, var): + if self.tok =3D=3D TOK_DEFAULT: + self.get_token() + val =3D self.parse_y_or_n() + cond =3D self.parse_condition() + self.data.do_default(var, val, cond) + elif self.tok =3D=3D TOK_DEPENDS: + self.get_token() + if self.tok !=3D TOK_ON: + raise KconfigParserError(self, 'Expected "on"') + self.get_token() + self.data.do_depends_on(var, self.parse_expr()) + elif self.tok =3D=3D TOK_SELECT: + self.get_token() + symbol =3D self.parse_var() + cond =3D self.parse_condition() + self.data.do_select(var, symbol, cond) + elif self.tok =3D=3D TOK_IMPLY: + self.get_token() + symbol =3D self.parse_var() + cond =3D self.parse_condition() + self.data.do_imply(var, symbol, cond) + elif self.tok =3D=3D TOK_BOOL: + self.get_token() + else: + raise KconfigParserError(self, 'Error in recursive descent?') + + # properties: properties property + # | /* empty */ + def parse_properties(self, var): + had_default =3D False + while self.tok =3D=3D TOK_DEFAULT or self.tok =3D=3D TOK_DEPENDS o= r \ + self.tok =3D=3D TOK_SELECT or self.tok =3D=3D TOK_BOOL or \ + self.tok =3D=3D TOK_IMPLY: + self.parse_property(var) + self.data.do_default(var, False) + + # for nicer error message + if self.tok !=3D TOK_SOURCE and self.tok !=3D TOK_CONFIG and \ + self.tok !=3D TOK_ID and self.tok !=3D TOK_EOF: + raise KconfigParserError(self, 'expected "source", "config", i= dentifier, ' + + '"default", "depends on", "imply" or "select"') + + # declaration: config var properties + def parse_declaration(self): + if self.tok =3D=3D TOK_CONFIG: + self.get_token() + var =3D self.parse_var() + self.data.do_declaration(var) + self.parse_properties(var) + else: + raise KconfigParserError(self, 'Error in recursive descent?') + + # clause: SOURCE + # | declaration + # | assignment + def parse_clause(self): + if self.tok =3D=3D TOK_SOURCE: + val =3D self.val + self.get_token() + self.do_include(val) + elif self.tok =3D=3D TOK_CONFIG: + self.parse_declaration() + elif self.tok =3D=3D TOK_ID: + self.parse_assignment() + else: + raise KconfigParserError(self, 'expected "source", "config" or= identifier') + + # config: clause+ EOF + def parse_config(self): + while self.tok !=3D TOK_EOF: + self.parse_clause() + return self.data + + # scanner ----- + + def get_token(self): + while True: + self.tok =3D self.src[self.cursor] + self.pos =3D self.cursor + self.cursor +=3D 1 + + self.val =3D None + self.tok =3D self.scan_token() + if self.tok is not None: + return + + def check_keyword(self, rest): + if not self.src.startswith(rest, self.cursor): + return False + length =3D len(rest) + if self.src[self.cursor + length].isalnum() or self.src[self.curso= r + length] =3D=3D '|': + return False + self.cursor +=3D length + return True + + def scan_token(self): + if self.tok =3D=3D '#': + self.cursor =3D self.src.find('\n', self.cursor) + return None + elif self.tok =3D=3D '=3D': + return TOK_EQUAL + elif self.tok =3D=3D '(': + return TOK_LPAREN + elif self.tok =3D=3D ')': + return TOK_RPAREN + elif self.tok =3D=3D '&' and self.src[self.pos+1] =3D=3D '&': + self.cursor +=3D 1 + return TOK_AND + elif self.tok =3D=3D '|' and self.src[self.pos+1] =3D=3D '|': + self.cursor +=3D 1 + return TOK_OR + elif self.tok =3D=3D '!': + return TOK_NOT + elif self.tok =3D=3D 'd' and self.check_keyword("epends"): + return TOK_DEPENDS + elif self.tok =3D=3D 'o' and self.check_keyword("n"): + return TOK_ON + elif self.tok =3D=3D 's' and self.check_keyword("elect"): + return TOK_SELECT + elif self.tok =3D=3D 'i' and self.check_keyword("mply"): + return TOK_IMPLY + elif self.tok =3D=3D 'c' and self.check_keyword("onfig"): + return TOK_CONFIG + elif self.tok =3D=3D 'd' and self.check_keyword("efault"): + return TOK_DEFAULT + elif self.tok =3D=3D 'b' and self.check_keyword("ool"): + return TOK_BOOL + elif self.tok =3D=3D 'i' and self.check_keyword("f"): + return TOK_IF + elif self.tok =3D=3D 'y' and self.check_keyword(""): + return TOK_Y + elif self.tok =3D=3D 'n' and self.check_keyword(""): + return TOK_N + elif (self.tok =3D=3D 's' and self.check_keyword("ource")) or \ + self.tok =3D=3D 'i' and self.check_keyword("nclude"): + # source FILENAME + # include FILENAME + while self.src[self.cursor].isspace(): + self.cursor +=3D 1 + start =3D self.cursor + self.cursor =3D self.src.find('\n', self.cursor) + self.val =3D self.src[start:self.cursor] + return TOK_SOURCE + elif self.tok.isalpha(): + # identifier + while self.src[self.cursor].isalnum() or self.src[self.cursor]= =3D=3D '_': + self.cursor +=3D 1 + self.val =3D self.src[self.pos:self.cursor] + return TOK_ID + elif self.tok =3D=3D '\n': + if self.cursor =3D=3D len(self.src): + return TOK_EOF + self.line +=3D 1 + self.line_pos =3D self.cursor + elif not self.tok.isspace(): + raise KconfigParserError(self, 'invalid input') + + return None + +if __name__ =3D=3D '__main__': + fname =3D len(sys.argv) > 1 and sys.argv[1] or 'Kconfig.test' + KconfigParser.parse(open(fname, 'r')) --=20 1.8.3.1