From nobody Mon Apr 29 07:27:06 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1518389319988677.8224475485708; Sun, 11 Feb 2018 14:48:39 -0800 (PST) Received: from localhost ([::1]:36879 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1el0Q9-0004Ts-J7 for importer@patchew.org; Sun, 11 Feb 2018 17:48:29 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46336) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1el0Oo-0003yI-0D for qemu-devel@nongnu.org; Sun, 11 Feb 2018 17:47:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1el0Oj-0000iZ-Eq for qemu-devel@nongnu.org; Sun, 11 Feb 2018 17:47:06 -0500 Received: from mail-pl0-x22b.google.com ([2607:f8b0:400e:c01::22b]:47027) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1el0Oj-0000i5-2f for qemu-devel@nongnu.org; Sun, 11 Feb 2018 17:47:01 -0500 Received: by mail-pl0-x22b.google.com with SMTP id x2so165719plr.13 for ; Sun, 11 Feb 2018 14:47:00 -0800 (PST) Received: from cloudburst.twiddle.net (174-21-6-47.tukw.qwest.net. [174.21.6.47]) by smtp.gmail.com with ESMTPSA id y5sm19796979pgq.80.2018.02.11.14.46.56 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 11 Feb 2018 14:46:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=hiFXgP478WkpZQetxd2zbNYxyk5OqwLc88rEPPdMSJY=; b=STEBuSI8CbrXgKRZe3ELmn33tsveBnH0i271P1DdIZlrkoc2ocoI+67AVTfGqU6pXa tc3KuVaNudoFaQ3G5D2FDCU1h3L+hyPmaZkiwgC4jG9kmmFPyRHUFx4U1L5Cl+FwRzC2 bIdOtJkQSnb9cYzWg1YT5CjBPOrwc3VXstIaU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=hiFXgP478WkpZQetxd2zbNYxyk5OqwLc88rEPPdMSJY=; b=lRh0VuQ95Cei6JQk0bigdvWxOqRcLydzvGo2bRxsX3qrW+NzlKafBNzP5xZVTKpL4M uIoxvccHPhIqd8aDV8Z+uxnN58aFTfsp1J86s7CZM084Ij8XxzV6D9JjHzXGFk4hYEyC 1MW7q0ehCxgrPB7hm5aghI65pSj4wiVnzUiyq++9KL5QpN765sZw+q+oiGoAZz3GEj6j J07TdVrNJrKTeDqR4zt77wXP8bRqFEGzSBR04A9e1CqunW1jbSfwWUE9ws7ISd9MX+pe dD9ul5DgMmtvpk/JUZIUVv1zqbGkN0AMQQR8bEXpaJ5SwDscDnCxubkftr6YPahkHSrJ /1/Q== X-Gm-Message-State: APf1xPD2Krhw+QxZ/BtI+os/IY1mpSg/pFdOHDVq6POHl3pzWDUSnDss 4BtiIOEMMO/fRE5XR/QHy7Fs9vkGJzQ= X-Google-Smtp-Source: AH8x226GLmAHvaIy5iTsUlXoATEleUZLWylsOWYbkM11r6qhPToCq1ySaUMh1rA0eSV/dsf67lBMHg== X-Received: by 2002:a17:902:1a2:: with SMTP id b31-v6mr8606512plb.28.1518389218945; Sun, 11 Feb 2018 14:46:58 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Sun, 11 Feb 2018 14:46:56 -0800 Message-Id: <20180211224656.12767-1-richard.henderson@linaro.org> X-Mailer: git-send-email 2.14.3 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::22b Subject: [Qemu-devel] [PATCH v2] scripts: Add decodetree.py X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To be used to decode ARM SVE, but could be used for any fixed-width ISA. Signed-off-by: Richard Henderson --- Changes since v1: * Pass pycodestyle-{2,3}. * Support 16-bit and 32-bit insns (I have a def file for thumb1). * Testsuite (only negative tests so far). * Called translate functions default to static. * Notice duplicate assignments and missing assignments to fields. * Use '-' to indicate a non-decoded bit, as opposed to '.' which must be filled in elsewhere by a format or a field. --- scripts/decodetree.py | 1026 +++++++++++++++++++++++++++++++++++++= ++++ tests/Makefile.include | 9 +- tests/decode/check.sh | 16 + tests/decode/err_argset1.def | 1 + tests/decode/err_argset2.def | 1 + tests/decode/err_field1.def | 1 + tests/decode/err_field2.def | 1 + tests/decode/err_field3.def | 1 + tests/decode/err_field4.def | 2 + tests/decode/err_field5.def | 1 + tests/decode/err_init1.def | 2 + tests/decode/err_init2.def | 2 + tests/decode/err_init3.def | 3 + tests/decode/err_init4.def | 3 + tests/decode/err_overlap1.def | 2 + tests/decode/err_overlap2.def | 2 + tests/decode/err_overlap3.def | 2 + tests/decode/err_overlap4.def | 2 + tests/decode/err_overlap5.def | 1 + tests/decode/err_overlap6.def | 2 + tests/decode/err_overlap7.def | 2 + 21 files changed, 1081 insertions(+), 1 deletion(-) create mode 100755 scripts/decodetree.py create mode 100755 tests/decode/check.sh create mode 100644 tests/decode/err_argset1.def create mode 100644 tests/decode/err_argset2.def create mode 100644 tests/decode/err_field1.def create mode 100644 tests/decode/err_field2.def create mode 100644 tests/decode/err_field3.def create mode 100644 tests/decode/err_field4.def create mode 100644 tests/decode/err_field5.def create mode 100644 tests/decode/err_init1.def create mode 100644 tests/decode/err_init2.def create mode 100644 tests/decode/err_init3.def create mode 100644 tests/decode/err_init4.def create mode 100644 tests/decode/err_overlap1.def create mode 100644 tests/decode/err_overlap2.def create mode 100644 tests/decode/err_overlap3.def create mode 100644 tests/decode/err_overlap4.def create mode 100644 tests/decode/err_overlap5.def create mode 100644 tests/decode/err_overlap6.def create mode 100644 tests/decode/err_overlap7.def diff --git a/scripts/decodetree.py b/scripts/decodetree.py new file mode 100755 index 0000000000..3c20acf700 --- /dev/null +++ b/scripts/decodetree.py @@ -0,0 +1,1026 @@ +#!/usr/bin/env python +# +# Generate a decoding tree from a specification file. +# +# The tree is built from instruction "patterns". A pattern may represent +# a single architectural instruction or a group of same, depending on what +# is convenient for further processing. +# +# Each pattern has "fixedbits" & "fixedmask", the combination of which +# describes the condition under which the pattern is matched: +# +# (insn & fixedmask) =3D=3D fixedbits +# +# Each pattern may have "fields", which are extracted from the insn and +# passed along to the translator. Examples of such are registers, +# immediates, and sub-opcodes. +# +# In support of patterns, one may declare fields, argument sets, and +# formats, each of which may be re-used to simplify further definitions. +# +# *** Field syntax: +# +# field_def :=3D '%' identifier ( unnamed_field )+ ( !function=3Dident= ifier )? +# unnamed_field :=3D number ':' ( 's' ) number +# +# For unnamed_field, the first number is the least-significant bit positio= n of +# the field and the second number is the length of the field. If the 's' = is +# present, the field is considered signed. If multiple unnamed_fields are +# present, they are concatenated. In this way one can define disjoint fie= lds. +# +# If !function is specified, the concatenated result is passed through the +# named function, taking and returning an integral value. +# +# FIXME: the fields of the structure into which this result will be stored +# is restricted to "int". Which means that we cannot expand 64-bit items. +# +# Field examples: +# +# %disp 0:s16 -- sextract(i, 0, 16) +# %imm9 16:6 10:3 -- extract(i, 16, 6) << 3 | extract(i, 10, 3) +# %disp12 0:s1 1:1 2:10 -- sextract(i, 0, 1) << 11 +# | extract(i, 1, 1) << 10 +# | extract(i, 2, 10) +# %shimm8 5:s8 13:1 !function=3Dexpand_shimm8 +# -- expand_shimm8(sextract(i, 5, 8) << 1 +# | extract(i, 13, 1)) +# +# *** Argument set syntax: +# +# args_def :=3D '&' identifier ( args_elt )+ +# args_elt :=3D identifier +# +# Each args_elt defines an argument within the argument set. +# Each argument set will be rendered as a C structure "arg_$name" +# with each of the fields being one of the member arguments. +# +# Argument set examples: +# +# ®3 ra rb rc +# &loadstore reg base offset +# +# *** Format syntax: +# +# fmt_def :=3D '@' identifier ( fmt_elt )+ +# fmt_elt :=3D fixedbit_elt | field_elt | field_ref | args_ref +# fixedbit_elt :=3D [01.-]+ +# field_elt :=3D identifier ':' 's'? number +# field_ref :=3D '%' identifier | identifier '=3D' '%' identifier +# args_ref :=3D '&' identifier +# +# Defining a format is a handy way to avoid replicating groups of fields +# across many instruction patterns. +# +# A fixedbit_elt describes a contiguous sequence of bits that must +# be 1, 0, [.-] for don't care. The difference between '.' and '-' +# is that '.' means that the bit will be covered with a field and +# '-' means that the bit is really ignored by the cpu and will not +# be covered by a field. +# +# A field_elt describes a simple field only given a width; the position of +# the field is implied by its position with respect to other fixedbit_elt +# and field_elt. +# +# If any fixedbit_elt or field_elt appear then all bits must be defined. +# Padding with a fixedbit_elt of all '.' is an easy way to accomplish that. +# +# A field_ref incorporates a field by reference. This is the only way to +# add a complex field to a format. A field may be renamed in the process +# via assignment to another identifier. This is intended to allow the +# same argument set be used with disjoint named fields. +# +# A single args_ref may specify an argument set to use for the format. +# The set of fields in the format must be a subset of the arguments in +# the argument set. If an argument set is not specified, one will be +# inferred from the set of fields. +# +# It is recommended, but not required, that all field_ref and args_ref +# appear at the end of the line, not interleaving with fixedbit_elf or +# field_elt. +# +# Format examples: +# +# @opr ...... ra:5 rb:5 ... 0 ....... rc:5 +# @opi ...... ra:5 lit:8 1 ....... rc:5 +# +# *** Pattern syntax: +# +# pat_def :=3D identifier ( pat_elt )+ +# pat_elt :=3D fixedbit_elt | field_elt | field_ref +# | args_ref | fmt_ref | const_elt +# fmt_ref :=3D '@' identifier +# const_elt :=3D identifier '=3D' number +# +# The fixedbit_elt and field_elt specifiers are unchanged from formats. +# A pattern that does not specify a named format will have one inferred +# from a referenced argument set (if present) and the set of fields. +# +# A const_elt allows a argument to be set to a constant value. This may +# come in handy when fields overlap between patterns and one has to +# include the values in the fixedbit_elt instead. +# +# The decoder will call a translator function for each pattern matched. +# +# Pattern examples: +# +# addl_r 010000 ..... ..... .... 0000000 ..... @opr +# addl_i 010000 ..... ..... .... 0000000 ..... @opi +# +# which will, in part, invoke +# +# trans_addl_r(ctx, &arg_opr, insn) +# and +# trans_addl_i(ctx, &arg_opi, insn) +# + +import io +import re +import sys +import getopt +import pdb + +insnwidth =3D 32 +fields =3D {} +arguments =3D {} +formats =3D {} +patterns =3D [] + +translate_prefix =3D 'trans' +translate_scope =3D 'static ' +output_file =3D sys.stdout +input_file =3D '' +insntype =3D 'uint32_t' + +re_ident =3D '[a-zA-Z][a-zA-Z0-9_]*' + + +def error(lineno, *args): + """Print an error message from file:line and args and exit.""" + if lineno: + r =3D '{0}:{1}: error:'.format(input_file, lineno) + elif input_file: + r =3D '{0}: error:'.format(input_file) + else: + r =3D 'error:' + for a in args: + r +=3D ' ' + str(a) + r +=3D '\n' + sys.stderr.write(r) + exit(1) + + +def output(*args): + global output_file + for a in args: + output_file.write(a) + + +if sys.version_info >=3D (3, 0): + re_fullmatch =3D re.fullmatch +else: + def re_fullmatch(pat, str): + return re.match('^' + pat + '$', str) + + +def output_autogen(): + output('/* This file is autogenerated by scripts/decodetree.py. */\n\= n') + + +def str_indent(c): + """Return a string with C spaces""" + return ' ' * c + + +def str_fields(fields): + """Return a string uniquely identifing FIELDS""" + r =3D '' + for n in sorted(fields.keys()): + r +=3D '_' + n + return r[1:] + + +def str_match_bits(bits, mask): + """Return a string pretty-printing BITS/MASK""" + global insnwidth + + i =3D 1 << (insnwidth - 1) + space =3D 0x01010100 + r =3D '' + while i !=3D 0: + if i & mask: + if i & bits: + r +=3D '1' + else: + r +=3D '0' + else: + r +=3D '.' + if i & space: + r +=3D ' ' + i >>=3D 1 + return r + + +def is_pow2(x): + """Return true iff X is equal to a power of 2.""" + return (x & (x - 1)) =3D=3D 0 + + +def ctz(x): + """Return the number of times 2 factors into X.""" + r =3D 0 + while ((x >> r) & 1) =3D=3D 0: + r +=3D 1 + return r + + +def is_contiguous(bits): + shift =3D ctz(bits) + if is_pow2((bits >> shift) + 1): + return shift + else: + return -1 + + +def eq_fields_for_args(flds_a, flds_b): + if len(flds_a) !=3D len(flds_b): + return False + for k, a in flds_a.items(): + if k not in flds_b: + return False + return True + + +def eq_fields_for_fmts(flds_a, flds_b): + if len(flds_a) !=3D len(flds_b): + return False + for k, a in flds_a.items(): + if k not in flds_b: + return False + b =3D flds_b[k] + if a.__class__ !=3D b.__class__ or a !=3D b: + return False + return True + + +class Field: + """Class representing a simple instruction field""" + def __init__(self, sign, pos, len): + self.sign =3D sign + self.pos =3D pos + self.len =3D len + self.mask =3D ((1 << len) - 1) << pos + + def __str__(self): + if self.sign: + s =3D 's' + else: + s =3D '' + return str(pos) + ':' + s + str(len) + + def str_extract(self): + if self.sign: + extr =3D 'sextract32' + else: + extr =3D 'extract32' + return '{0}(insn, {1}, {2})'.format(extr, self.pos, self.len) + + def __eq__(self, other): + return self.sign =3D=3D other.sign and self.sign =3D=3D other.sign + + def __ne__(self, other): + return not self.__eq__(other) +# end Field + + +class MultiField: + """Class representing a compound instruction field""" + def __init__(self, subs, mask): + self.subs =3D subs + self.sign =3D subs[0].sign + self.mask =3D mask + + def __str__(self): + return str(self.subs) + + def str_extract(self): + ret =3D '0' + pos =3D 0 + for f in reversed(self.subs): + if pos =3D=3D 0: + ret =3D f.str_extract() + else: + ret =3D 'deposit32({0}, {1}, {2}, {3})' \ + .format(ret, pos, 32 - pos, f.str_extract()) + pos +=3D f.len + return ret + + def __ne__(self, other): + if len(self.subs) !=3D len(other.subs): + return True + for a, b in zip(self.subs, other.subs): + if a.__class__ !=3D b.__class__ or a !=3D b: + return True + return False + + def __eq__(self, other): + return not self.__ne__(other) +# end MultiField + + +class ConstField: + """Class representing an argument field with constant value""" + def __init__(self, value): + self.value =3D value + self.mask =3D 0 + self.sign =3D value < 0 + + def __str__(self): + return str(self.value) + + def str_extract(self): + return str(self.value) + + def __cmp__(self, other): + return self.value - other.value +# end ConstField + + +class FunctionField: + """Class representing a field passed through an expander""" + def __init__(self, func, base): + self.mask =3D base.mask + self.sign =3D base.sign + self.base =3D base + self.func =3D func + + def __str__(self): + return self.func + '(' + str(self.base) + ')' + + def str_extract(self): + return self.func + '(' + self.base.str_extract() + ')' + + def __eq__(self, other): + return self.func =3D=3D other.func and self.base =3D=3D other.base + + def __ne__(self, other): + return not self.__eq__(other) +# end FunctionField + + +class Arguments: + """Class representing the extracted fields of a format""" + def __init__(self, nm, flds): + self.name =3D nm + self.fields =3D sorted(flds) + + def __str__(self): + return self.name + ' ' + str(self.fields) + + def struct_name(self): + return 'arg_' + self.name + + def output_def(self): + output('typedef struct {\n') + for n in self.fields: + output(' int ', n, ';\n') + output('} ', self.struct_name(), ';\n\n') +# end Arguments + + +class General: + """Common code between instruction formats and instruction patterns""" + def __init__(self, name, lineno, base, fixb, fixm, udfm, fldm, flds): + self.name =3D name + self.lineno =3D lineno + self.base =3D base + self.fixedbits =3D fixb + self.fixedmask =3D fixm + self.undefmask =3D udfm + self.fieldmask =3D fldm + self.fields =3D flds + + def __str__(self): + r =3D self.name + if self.base: + r =3D r + ' ' + self.base.name + else: + r =3D r + ' ' + str(self.fields) + r =3D r + ' ' + str_match_bits(self.fixedbits, self.fixedmask) + return r + + def str1(self, i): + return str_indent(i) + self.__str__() +# end General + + +class Format(General): + """Class representing an instruction format""" + + def extract_name(self): + return 'extract_' + self.name + + def output_extract(self): + output('static void ', self.extract_name(), '(', + self.base.struct_name(), ' *a, ', insntype, ' insn)\n{\n') + for n, f in self.fields.items(): + output(' a->', n, ' =3D ', f.str_extract(), ';\n') + output('}\n\n') +# end Format + + +class Pattern(General): + """Class representing an instruction pattern""" + + def output_decl(self): + global translate_scope + global translate_prefix + output('typedef ', self.base.base.struct_name(), + ' arg_', self.name, ';\n') + output(translate_scope, 'void ', translate_prefix, '_', self.name, + '(DisasContext *ctx, arg_', self.name, + ' *a, ', insntype, ' insn);\n') + + def output_code(self, i, extracted, outerbits, outermask): + global translate_prefix + ind =3D str_indent(i) + arg =3D self.base.base.name + output(ind, '/* line ', str(self.lineno), ' */\n') + if not extracted: + output(ind, self.base.extract_name(), '(&u.f_', arg, ', insn);= \n') + for n, f in self.fields.items(): + output(ind, 'u.f_', arg, '.', n, ' =3D ', f.str_extract(), ';\= n') + output(ind, translate_prefix, '_', self.name, + '(ctx, &u.f_', arg, ', insn);\n') + output(ind, 'return true;\n') +# end Pattern + + +def parse_field(lineno, name, toks): + """Parse one instruction field from TOKS at LINENO""" + global fields + global re_ident + global insnwidth + + # A "simple" field will have only one entry; + # a "multifield" will have several. + subs =3D [] + width =3D 0 + func =3D None + for t in toks: + if re_fullmatch('!function=3D' + re_ident, t): + if func: + error(lineno, 'duplicate function') + func =3D t.split('=3D') + func =3D func[1] + continue + + if re_fullmatch('[0-9]+:s[0-9]+', t): + # Signed field extract + subtoks =3D t.split(':s') + sign =3D True + elif re_fullmatch('[0-9]+:[0-9]+', t): + # Unsigned field extract + subtoks =3D t.split(':') + sign =3D False + else: + error(lineno, 'invalid field token "{0}"'.format(t)) + po =3D int(subtoks[0]) + le =3D int(subtoks[1]) + if po + le > insnwidth: + error(lineno, 'field {0} too large'.format(t)) + f =3D Field(sign, po, le) + subs.append(f) + width +=3D le + + if width > insnwidth: + error(lineno, 'field too large') + if len(subs) =3D=3D 1: + f =3D subs[0] + else: + mask =3D 0 + for s in subs: + if mask & s.mask: + error(lineno, 'field components overlap') + mask |=3D s.mask + f =3D MultiField(subs, mask) + if func: + f =3D FunctionField(func, f) + + if name in fields: + error(lineno, 'duplicate field', name) + fields[name] =3D f +# end parse_field + + +def parse_arguments(lineno, name, toks): + """Parse one argument set from TOKS at LINENO""" + global arguments + global re_ident + + flds =3D [] + for t in toks: + if not re_fullmatch(re_ident, t): + error(lineno, 'invalid argument set token "{0}"'.format(t)) + if t in flds: + error(lineno, 'duplicate argument "{0}"'.format(t)) + flds.append(t) + + if name in arguments: + error(lineno, 'duplicate argument set', name) + arguments[name] =3D Arguments(name, flds) +# end parse_arguments + + +def lookup_field(lineno, name): + global fields + if name in fields: + return fields[name] + error(lineno, 'undefined field', name) + + +def add_field(lineno, flds, new_name, f): + if new_name in flds: + error(lineno, 'duplicate field', new_name) + flds[new_name] =3D f + return flds + + +def add_field_byname(lineno, flds, new_name, old_name): + return add_field(lineno, flds, new_name, lookup_field(lineno, old_name= )) + + +def infer_argument_set(flds): + global arguments + + for arg in arguments.values(): + if eq_fields_for_args(flds, arg.fields): + return arg + + name =3D str(len(arguments)) + arg =3D Arguments(name, flds.keys()) + arguments[name] =3D arg + return arg + + +def infer_format(arg, fieldmask, flds): + global arguments + global formats + + const_flds =3D {} + var_flds =3D {} + for n, c in flds.items(): + if c is ConstField: + const_flds[n] =3D c + else: + var_flds[n] =3D c + + # Look for an existing format with the same argument set and fields + for fmt in formats.values(): + if arg and fmt.base !=3D arg: + continue + if fieldmask !=3D fmt.fieldmask: + continue + if not eq_fields_for_fmts(flds, fmt.fields): + continue + return (fmt, const_flds) + + name =3D 'Fmt_' + str(len(formats)) + if not arg: + arg =3D infer_argument_set(flds) + + fmt =3D Format(name, 0, arg, 0, 0, 0, fieldmask, var_flds) + formats[name] =3D fmt + + return (fmt, const_flds) +# end infer_format + + +def parse_generic(lineno, is_format, name, toks): + """Parse one instruction format from TOKS at LINENO""" + global fields + global arguments + global formats + global patterns + global re_ident + global insnwidth + + fixedmask =3D 0 + fixedbits =3D 0 + undefmask =3D 0 + width =3D 0 + flds =3D {} + arg =3D None + fmt =3D None + for t in toks: + # '&Foo' gives a format an explcit argument set. + if t[0] =3D=3D '&': + tt =3D t[1:] + if arg: + error(lineno, 'multiple argument sets') + if tt in arguments: + arg =3D arguments[tt] + else: + error(lineno, 'undefined argument set', t) + continue + + # '@Foo' gives a pattern an explicit format. + if t[0] =3D=3D '@': + tt =3D t[1:] + if fmt: + error(lineno, 'multiple formats') + if tt in formats: + fmt =3D formats[tt] + else: + error(lineno, 'undefined format', t) + continue + + # '%Foo' imports a field. + if t[0] =3D=3D '%': + tt =3D t[1:] + flds =3D add_field_byname(lineno, flds, tt, tt) + continue + + # 'Foo=3D%Bar' imports a field with a different name. + if re_fullmatch(re_ident + '=3D%' + re_ident, t): + (fname, iname) =3D t.split('=3D%') + flds =3D add_field_byname(lineno, flds, fname, iname) + continue + + # 'Foo=3Dnumber' sets an argument field to a constant value + if re_fullmatch(re_ident + '=3D[0-9]+', t): + (fname, value) =3D t.split('=3D') + value =3D int(value) + flds =3D add_field(lineno, flds, fname, ConstField(value)) + continue + + # Pattern of 0s, 1s, dots and dashes indicate required zeros, + # required ones, or dont-cares. + if re_fullmatch('[01.-]+', t): + shift =3D len(t) + fms =3D t.replace('0', '1') + fms =3D fms.replace('.', '0') + fms =3D fms.replace('-', '0') + fbs =3D t.replace('.', '0') + fbs =3D fbs.replace('-', '0') + ubm =3D t.replace('1', '0') + ubm =3D ubm.replace('.', '0') + ubm =3D ubm.replace('-', '1') + fms =3D int(fms, 2) + fbs =3D int(fbs, 2) + ubm =3D int(ubm, 2) + fixedbits =3D (fixedbits << shift) | fbs + fixedmask =3D (fixedmask << shift) | fms + undefmask =3D (undefmask << shift) | ubm + # Otherwise, fieldname:fieldwidth + elif re_fullmatch(re_ident + ':s?[0-9]+', t): + (fname, flen) =3D t.split(':') + sign =3D False + if flen[0] =3D=3D 's': + sign =3D True + flen =3D flen[1:] + shift =3D int(flen, 10) + f =3D Field(sign, insnwidth - width - shift, shift) + flds =3D add_field(lineno, flds, fname, f) + fixedbits <<=3D shift + fixedmask <<=3D shift + undefmask <<=3D shift + else: + error(lineno, 'invalid token "{0}"'.format(t)) + width +=3D shift + + # We should have filled in all of the bits of the instruction. + if not (is_format and width =3D=3D 0) and width !=3D insnwidth: + error(lineno, 'definition has {0} bits'.format(width)) + + # Do not check for fields overlaping fields; one valid usage + # is to be able to duplicate fields via import. + fieldmask =3D 0 + for f in flds.values(): + fieldmask |=3D f.mask + + # Fix up what we've parsed to match either a format or a pattern. + if is_format: + # Formats cannot reference formats. + if fmt: + error(lineno, 'format referencing format') + # If an argument set is given, then there should be no fields + # without a place to store it. + if arg: + for f in flds.keys(): + if f not in arg.fields: + error(lineno, 'field {0} not in argument set {1}' + .format(f, arg.name)) + else: + arg =3D infer_argument_set(flds) + if name in formats: + error(lineno, 'duplicate format name', name) + fmt =3D Format(name, lineno, arg, fixedbits, fixedmask, + undefmask, fieldmask, flds) + formats[name] =3D fmt + else: + # Patterns can reference a format ... + if fmt: + # ... but not an argument simultaneously + if arg: + error(lineno, 'pattern specifies both format and argument = set') + if fixedmask & fmt.fixedmask: + error(lineno, 'pattern fixed bits overlap format fixed bit= s') + fieldmask |=3D fmt.fieldmask + fixedbits |=3D fmt.fixedbits + fixedmask |=3D fmt.fixedmask + undefmask |=3D fmt.undefmask + else: + (fmt, flds) =3D infer_format(arg, fieldmask, flds) + arg =3D fmt.base + for f in flds.keys(): + if f not in arg.fields: + error(lineno, 'field {0} not in argument set {1}' + .format(f, arg.name)) + for f in arg.fields: + if f not in flds.keys() and f not in fmt.fields.keys(): + error(lineno, 'field {0} not initialized'.format(f)) + pat =3D Pattern(name, lineno, fmt, fixedbits, fixedmask, + undefmask, fieldmask, flds) + patterns.append(pat) + + # Validate the masks that we have assembled. + if fieldmask & fixedmask: + error(lineno, 'fieldmask overlaps fixedmask (0x{0:08x} & 0x{1:08x}= )' + .format(fieldmask, fixedmask)) + if fieldmask & undefmask: + error(lineno, 'fieldmask overlaps undefmask (0x{0:08x} & 0x{1:08x}= )' + .format(fieldmask, undefmask)) + if fixedmask & undefmask: + error(lineno, 'fixedmask overlaps undefmask (0x{0:08x} & 0x{1:08x}= )' + .format(fixedmask, undefmask)) +# end parse_general + + +def parse_file(f): + """Parse all of the patterns within a file""" + + # Read all of the lines of the file. Concatenate lines + # ending in backslash; discard empty lines and comments. + toks =3D [] + lineno =3D 0 + for line in f: + lineno +=3D 1 + + # Discard comments + end =3D line.find('#') + if end >=3D 0: + line =3D line[:end] + + t =3D line.split() + if len(toks) !=3D 0: + # Next line after continuation + toks.extend(t) + elif len(t) =3D=3D 0: + # Empty line + continue + else: + toks =3D t + + # Continuation? + if toks[-1] =3D=3D '\\': + toks.pop() + continue + + if len(toks) < 2: + error(lineno, 'short line') + + name =3D toks[0] + del toks[0] + + # Determine the type of object needing to be parsed. + if name[0] =3D=3D '%': + parse_field(lineno, name[1:], toks) + elif name[0] =3D=3D '&': + parse_arguments(lineno, name[1:], toks) + elif name[0] =3D=3D '@': + parse_generic(lineno, True, name[1:], toks) + else: + parse_generic(lineno, False, name, toks) + toks =3D [] +# end parse_file + + +class Tree: + """Class representing a node in a decode tree""" + + def __init__(self, fm, tm): + self.fixedmask =3D fm + self.thismask =3D tm + self.subs =3D [] + self.base =3D None + + def str1(self, i): + ind =3D str_indent(i) + r =3D '{0}{1:08x}'.format(ind, self.fixedmask) + if self.format: + r +=3D ' ' + self.format.name + r +=3D ' [\n' + for (b, s) in self.subs: + r +=3D '{0} {1:08x}:\n'.format(ind, b) + r +=3D s.str1(i + 4) + '\n' + r +=3D ind + ']' + return r + + def __str__(self): + return self.str1(0) + + def output_code(self, i, extracted, outerbits, outermask): + ind =3D str_indent(i) + + # If we identified all nodes below have the same format, + # extract the fields now. + if not extracted and self.base: + output(ind, self.base.extract_name(), + '(&u.f_', self.base.base.name, ', insn);\n') + extracted =3D True + + # Attempt to aid the compiler in producing compact switch statemen= ts. + # If the bits in the mask are contiguous, extract them. + sh =3D is_contiguous(self.thismask) + if sh > 0: + # Propagate SH down into the local functions. + def str_switch(b, sh=3Dsh): + return '(insn >> {0}) & 0x{1:x}'.format(sh, b >> sh) + + def str_case(b, sh=3Dsh): + return '0x{0:x}'.format(b >> sh) + else: + def str_switch(b): + return 'insn & 0x{0:08x}'.format(b) + + def str_case(b): + return '0x{0:08x}'.format(b) + + output(ind, 'switch (', str_switch(self.thismask), ') {\n') + for b, s in sorted(self.subs): + assert (self.thismask & ~s.fixedmask) =3D=3D 0 + innermask =3D outermask | self.thismask + innerbits =3D outerbits | b + output(ind, 'case ', str_case(b), ':\n') + output(ind, ' /* ', + str_match_bits(innerbits, innermask), ' */\n') + s.output_code(i + 4, extracted, innerbits, innermask) + output(ind, '}\n') + output(ind, 'return false;\n') +# end Tree + + +def build_tree(pats, outerbits, outermask): + # Find the intersection of all remaining fixedmask. + innermask =3D ~outermask + for i in pats: + innermask &=3D i.fixedmask + + if innermask =3D=3D 0: + pnames =3D [] + for p in pats: + pnames.append(p.name + ':' + str(p.lineno)) + error(pats[0].lineno, 'overlapping patterns:', pnames) + + fullmask =3D outermask | innermask + + # Sort each element of pats into the bin selected by the mask. + bins =3D {} + for i in pats: + fb =3D i.fixedbits & innermask + if fb in bins: + bins[fb].append(i) + else: + bins[fb] =3D [i] + + # We must recurse if any bin has more than one element or if + # the single element in the bin has not been fully matched. + t =3D Tree(fullmask, innermask) + + for b, l in bins.items(): + s =3D l[0] + if len(l) > 1 or s.fixedmask & ~fullmask !=3D 0: + s =3D build_tree(l, b | outerbits, fullmask) + t.subs.append((b, s)) + + return t +# end build_tree + + +def prop_format(tree): + """Propagate Format objects into the decode tree""" + + # Depth first search. + for (b, s) in tree.subs: + if isinstance(s, Tree): + prop_format(s) + + # If all entries in SUBS have the same format, then + # propagate that into the tree. + f =3D None + for (b, s) in tree.subs: + if f is None: + f =3D s.base + if f is None: + return + if f is not s.base: + return + tree.base =3D f +# end prop_format + + +def main(): + global arguments + global formats + global patterns + global translate_scope + global translate_prefix + global output_file + global input_file + global insnwidth + global insntype + + c_file =3D None + decode_function =3D 'decode' + + long_opts =3D ['decode=3D', 'translate=3D', 'output=3D', 'insnwidth=3D= '] + try: + (opts, args) =3D getopt.getopt(sys.argv[1:], 'o:w:', long_opts) + except getopt.GetoptError as err: + error(0, err) + for o, a in opts: + if o in ('-o', '--output'): + c_file =3D a + elif o =3D=3D '--decode': + decode_function =3D a + elif o =3D=3D '--translate': + translate_prefix =3D a + translate_scope =3D '' + elif o in ('-w', '--insnwidth'): + insnwidth =3D int(a) + if insnwidth =3D=3D 16: + insntype =3D 'uint16_t' + elif insnwidth !=3D 32: + error(0, 'cannot handle insns of width', insnwidth) + else: + assert False, 'unhandled option' + + if len(args) < 1: + error(0, 'missing input file') + input_file =3D args[0] + f =3D open(input_file, 'r') + parse_file(f) + f.close() + + t =3D build_tree(patterns, 0, 0) + prop_format(t) + + if c_file: + output_file =3D open(c_file, 'w') + else: + output_file =3D sys.stdout + + output_autogen() + for n in sorted(arguments.keys()): + f =3D arguments[n] + f.output_def() + + # A single translate function can be invoked for different patterns. + # Make sure that the argument sets are the same, and declare the + # function only once. + out_pats =3D {} + for i in patterns: + if i.name in out_pats: + p =3D out_pats[i.name] + if i.base.base !=3D p.base.base: + error(0, i.name, ' has conflicting argument sets') + else: + i.output_decl() + out_pats[i.name] =3D i + output('\n') + + for n in sorted(formats.keys()): + f =3D formats[n] + f.output_extract() + + output('bool ', decode_function, + '(DisasContext *ctx, ', insntype, ' insn)\n{\n') + + i4 =3D str_indent(4) + output(i4, 'union {\n') + for n in sorted(arguments.keys()): + f =3D arguments[n] + output(i4, i4, f.struct_name(), ' f_', f.name, ';\n') + output(i4, '} u;\n\n') + + t.output_code(4, False, 0, 0) + + output('}\n') + + if c_file: + output_file.close() +# end main + + +if __name__ =3D=3D '__main__': + main() diff --git a/tests/Makefile.include b/tests/Makefile.include index f41da235ae..8cc7d56e97 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -928,6 +928,13 @@ $(patsubst %, check-%, $(check-qapi-schema-y)): check-= %.json: $(SRC_PATH)/%.json check-tests/qapi-schema/doc-good.texi: tests/qapi-schema/doc-good.test.texi @diff -q $(SRC_PATH)/tests/qapi-schema/doc-good.texi $< =20 +.PHONY: check-decodetree +check-decodetree: + $(call quiet-command, \ + cd $(SRC_PATH)/tests/decode && \ + ./check.sh "$(PYTHON)" "$(SRC_PATH)/scripts/decodetree.py", \ + TEST, decodetree.py) + # Consolidated targets =20 .PHONY: check-qapi-schema check-qtest check-unit check check-clean @@ -936,7 +943,7 @@ check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGET= S)) check-unit: $(patsubst %,check-%, $(check-unit-y)) check-speed: $(patsubst %,check-%, $(check-speed-y)) check-block: $(patsubst %,check-%, $(check-block-y)) -check: check-qapi-schema check-unit check-qtest +check: check-qapi-schema check-unit check-qtest check-decodetree check-clean: $(MAKE) -C tests/tcg clean rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) diff --git a/tests/decode/check.sh b/tests/decode/check.sh new file mode 100755 index 0000000000..6eb1392593 --- /dev/null +++ b/tests/decode/check.sh @@ -0,0 +1,16 @@ +#!/bin/sh + +PYTHON=3D$1 +DECODETREE=3D$2 +E=3D0 + +# All of these tests should produce errors +for i in err_*.def; do + if $PYTHON $DECODETREE $i > /dev/null 2> /dev/null; then + # Pass, aka failed to fail. + echo FAIL: $i 1>&2 + E=3D1 + fi +done + +exit $E diff --git a/tests/decode/err_argset1.def b/tests/decode/err_argset1.def new file mode 100644 index 0000000000..65d089d582 --- /dev/null +++ b/tests/decode/err_argset1.def @@ -0,0 +1 @@ +&args a a diff --git a/tests/decode/err_argset2.def b/tests/decode/err_argset2.def new file mode 100644 index 0000000000..16a812cf0d --- /dev/null +++ b/tests/decode/err_argset2.def @@ -0,0 +1 @@ +&args a b c d0 0e diff --git a/tests/decode/err_field1.def b/tests/decode/err_field1.def new file mode 100644 index 0000000000..075404ced1 --- /dev/null +++ b/tests/decode/err_field1.def @@ -0,0 +1 @@ +%field asdf diff --git a/tests/decode/err_field2.def b/tests/decode/err_field2.def new file mode 100644 index 0000000000..08933bf8c9 --- /dev/null +++ b/tests/decode/err_field2.def @@ -0,0 +1 @@ +%field 0:33 diff --git a/tests/decode/err_field3.def b/tests/decode/err_field3.def new file mode 100644 index 0000000000..ecb6427a40 --- /dev/null +++ b/tests/decode/err_field3.def @@ -0,0 +1 @@ +%field 31:2 diff --git a/tests/decode/err_field4.def b/tests/decode/err_field4.def new file mode 100644 index 0000000000..2844afc24a --- /dev/null +++ b/tests/decode/err_field4.def @@ -0,0 +1,2 @@ +%field 0:1 +%field 0:1 diff --git a/tests/decode/err_field5.def b/tests/decode/err_field5.def new file mode 100644 index 0000000000..cc3ea844ae --- /dev/null +++ b/tests/decode/err_field5.def @@ -0,0 +1 @@ +%field 0:1 !function=3Da !function=3Da diff --git a/tests/decode/err_init1.def b/tests/decode/err_init1.def new file mode 100644 index 0000000000..2c986cf627 --- /dev/null +++ b/tests/decode/err_init1.def @@ -0,0 +1,2 @@ +&args a b +insn 00000000 00000000 00000000 b:8 &args diff --git a/tests/decode/err_init2.def b/tests/decode/err_init2.def new file mode 100644 index 0000000000..7c80854ea5 --- /dev/null +++ b/tests/decode/err_init2.def @@ -0,0 +1,2 @@ +&args a b +insn 00000000 00000000 a:8 b:8 &args a=3D1 diff --git a/tests/decode/err_init3.def b/tests/decode/err_init3.def new file mode 100644 index 0000000000..15a3060c61 --- /dev/null +++ b/tests/decode/err_init3.def @@ -0,0 +1,3 @@ +&args a +@format ........ ........ a:16 &args +insn 00000000 00000000 a:16 @format diff --git a/tests/decode/err_init4.def b/tests/decode/err_init4.def new file mode 100644 index 0000000000..b84d968acd --- /dev/null +++ b/tests/decode/err_init4.def @@ -0,0 +1,3 @@ +&args a b +@format ........ ........ a:16 &args +insn 00000000 00000000 ........ ........ @format diff --git a/tests/decode/err_overlap1.def b/tests/decode/err_overlap1.def new file mode 100644 index 0000000000..5d39c2ddad --- /dev/null +++ b/tests/decode/err_overlap1.def @@ -0,0 +1,2 @@ +%field 0:1 +insn 00000000 00000000 00000000 00000000 %field diff --git a/tests/decode/err_overlap2.def b/tests/decode/err_overlap2.def new file mode 100644 index 0000000000..38e4f8ae31 --- /dev/null +++ b/tests/decode/err_overlap2.def @@ -0,0 +1,2 @@ +@format ........ ........ ........ ....... fld:1 +insn 00000000 00000000 00000000 00000000 @format diff --git a/tests/decode/err_overlap3.def b/tests/decode/err_overlap3.def new file mode 100644 index 0000000000..90c73f4f30 --- /dev/null +++ b/tests/decode/err_overlap3.def @@ -0,0 +1,2 @@ +%field 0:1 +insn 00000000 00000000 00000000 -------- %field diff --git a/tests/decode/err_overlap4.def b/tests/decode/err_overlap4.def new file mode 100644 index 0000000000..d83f8e3153 --- /dev/null +++ b/tests/decode/err_overlap4.def @@ -0,0 +1,2 @@ +@format ........ ........ ........ .......- +insn 00000000 00000000 00000000 00000000 @format diff --git a/tests/decode/err_overlap5.def b/tests/decode/err_overlap5.def new file mode 100644 index 0000000000..2f4359bb9b --- /dev/null +++ b/tests/decode/err_overlap5.def @@ -0,0 +1 @@ +%field 3:5 0:5 diff --git a/tests/decode/err_overlap6.def b/tests/decode/err_overlap6.def new file mode 100644 index 0000000000..bf446a3092 --- /dev/null +++ b/tests/decode/err_overlap6.def @@ -0,0 +1,2 @@ +@format ........ ........ ........ .......1 +insn 00000000 00000000 00000000 00000000 @format diff --git a/tests/decode/err_overlap7.def b/tests/decode/err_overlap7.def new file mode 100644 index 0000000000..44d914163b --- /dev/null +++ b/tests/decode/err_overlap7.def @@ -0,0 +1,2 @@ +insn1 00000000 00000000 00000000 00000000 +insn2 00000000 00000000 00000000 00000000 --=20 2.14.3