From: Alessandro Di Federico <ale@rev.ng>

This patchset introduces the idef-parser for target/hexagon.
It is based on "[PATCH v8 00/35] Hexagon patch series".

`idef-parser` is a build-time tool built using flex and bison. Its aim
is to generate a large part of the tiny code generator frontend for
Hexagon. The prototype of idef-parser has been presented at KVM Forum
2019 ("QEMU-Hexagon: Automatic Translation of the ISA Manual Pseudcode
to Tiny Code Instructions"):

    https://www.youtube.com/watch?v=3EpnTYBOXCI

`target/hexagon/idef-parser/README.rst` provides an overview of the
parser and its inner working.

Please consider this an RFC, this is our first large patchset. Feedback
is more than welcome.

A couple of notes:

* `idef-parser` also supports certain things that are not used in the
  most recently submitted version of the "Hexagon patch
  series". However, they will be needed and stripping them out of the
  parser is quite a bit of work.
* checkpatch.pl complains on a single macro which has a trailing
  semi-colon, which is required.

Alessandro Di Federico (4):
  target/hexagon: update MAINTAINERS for idef-parser
  target/hexagon: import README for idef-parser
  target/hexagon: prepare input for the idef-parser
  target/hexagon: call idef-parser functions

Niccolò Izzo (2):
  target/hexagon: introduce new helper functions
  target/hexagon: import additional tests

Paolo Montesel (4):
  target/hexagon: make helper functions non-static
  target/hexagon: expose next PC in DisasContext
  target/hexagon: import lexer for idef-parser
  target/hexagon: import parser for idef-parser

 MAINTAINERS                                 |    8 +
 target/hexagon/README                       |    5 +
 target/hexagon/gen_idef_parser_funcs.py     |  114 ++
 target/hexagon/gen_tcg_funcs.py             |   28 +-
 target/hexagon/genptr.c                     |  237 ++-
 target/hexagon/genptr.h                     |   26 +
 target/hexagon/hex_common.py                |   10 +
 target/hexagon/idef-parser/README.rst       |  446 +++++
 target/hexagon/idef-parser/idef-lexer.lex   |  648 +++++++
 target/hexagon/idef-parser/idef-parser.h    |  245 +++
 target/hexagon/idef-parser/idef-parser.y    | 1250 ++++++++++++
 target/hexagon/idef-parser/macros.inc       |  166 ++
 target/hexagon/idef-parser/parser-helpers.c | 1925 +++++++++++++++++++
 target/hexagon/idef-parser/parser-helpers.h |  293 +++
 target/hexagon/idef-parser/prepare          |   33 +
 target/hexagon/meson.build                  |   65 +-
 target/hexagon/translate.c                  |    4 +-
 target/hexagon/translate.h                  |    1 +
 tests/tcg/hexagon/Makefile.target           |   35 +-
 tests/tcg/hexagon/crt.S                     |   28 +
 tests/tcg/hexagon/first.S                   |   24 +-
 tests/tcg/hexagon/test_abs.S                |   20 +
 tests/tcg/hexagon/test_add.S                |   20 +
 tests/tcg/hexagon/test_andp.S               |   23 +
 tests/tcg/hexagon/test_bitcnt.S             |   42 +
 tests/tcg/hexagon/test_bitsplit.S           |   25 +
 tests/tcg/hexagon/test_call.S               |   63 +
 tests/tcg/hexagon/test_clobber.S            |   35 +
 tests/tcg/hexagon/test_cmp.S                |   34 +
 tests/tcg/hexagon/test_cmpy.S               |   31 +
 tests/tcg/hexagon/test_djump.S              |   24 +
 tests/tcg/hexagon/test_dotnew.S             |   39 +
 tests/tcg/hexagon/test_dstore.S             |   29 +
 tests/tcg/hexagon/test_ext.S                |   18 +
 tests/tcg/hexagon/test_fibonacci.S          |   33 +
 tests/tcg/hexagon/test_hello.S              |   21 +
 tests/tcg/hexagon/test_hl.S                 |   19 +
 tests/tcg/hexagon/test_hwloops.S            |   25 +
 tests/tcg/hexagon/test_jmp.S                |   25 +
 tests/tcg/hexagon/test_lsr.S                |   39 +
 tests/tcg/hexagon/test_mpyi.S               |   20 +
 tests/tcg/hexagon/test_packet.S             |   26 +
 tests/tcg/hexagon/test_reorder.S            |   31 +
 tests/tcg/hexagon/test_round.S              |   31 +
 tests/tcg/hexagon/test_vavgw.S              |   33 +
 tests/tcg/hexagon/test_vcmpb.S              |   32 +
 tests/tcg/hexagon/test_vcmpw.S              |   29 +
 tests/tcg/hexagon/test_vcmpy.S              |   50 +
 tests/tcg/hexagon/test_vlsrw.S              |   23 +
 tests/tcg/hexagon/test_vmaxh.S              |   37 +
 tests/tcg/hexagon/test_vminh.S              |   37 +
 tests/tcg/hexagon/test_vpmpyh.S             |   30 +
 tests/tcg/hexagon/test_vspliceb.S           |   33 +
 53 files changed, 6540 insertions(+), 28 deletions(-)
 create mode 100644 target/hexagon/gen_idef_parser_funcs.py
 create mode 100644 target/hexagon/idef-parser/README.rst
 create mode 100644 target/hexagon/idef-parser/idef-lexer.lex
 create mode 100644 target/hexagon/idef-parser/idef-parser.h
 create mode 100644 target/hexagon/idef-parser/idef-parser.y
 create mode 100644 target/hexagon/idef-parser/macros.inc
 create mode 100644 target/hexagon/idef-parser/parser-helpers.c
 create mode 100644 target/hexagon/idef-parser/parser-helpers.h
 create mode 100755 target/hexagon/idef-parser/prepare
 create mode 100644 tests/tcg/hexagon/crt.S
 create mode 100644 tests/tcg/hexagon/test_abs.S
 create mode 100644 tests/tcg/hexagon/test_add.S
 create mode 100644 tests/tcg/hexagon/test_andp.S
 create mode 100644 tests/tcg/hexagon/test_bitcnt.S
 create mode 100644 tests/tcg/hexagon/test_bitsplit.S
 create mode 100644 tests/tcg/hexagon/test_call.S
 create mode 100644 tests/tcg/hexagon/test_clobber.S
 create mode 100644 tests/tcg/hexagon/test_cmp.S
 create mode 100644 tests/tcg/hexagon/test_cmpy.S
 create mode 100644 tests/tcg/hexagon/test_djump.S
 create mode 100644 tests/tcg/hexagon/test_dotnew.S
 create mode 100644 tests/tcg/hexagon/test_dstore.S
 create mode 100644 tests/tcg/hexagon/test_ext.S
 create mode 100644 tests/tcg/hexagon/test_fibonacci.S
 create mode 100644 tests/tcg/hexagon/test_hello.S
 create mode 100644 tests/tcg/hexagon/test_hl.S
 create mode 100644 tests/tcg/hexagon/test_hwloops.S
 create mode 100644 tests/tcg/hexagon/test_jmp.S
 create mode 100644 tests/tcg/hexagon/test_lsr.S
 create mode 100644 tests/tcg/hexagon/test_mpyi.S
 create mode 100644 tests/tcg/hexagon/test_packet.S
 create mode 100644 tests/tcg/hexagon/test_reorder.S
 create mode 100644 tests/tcg/hexagon/test_round.S
 create mode 100644 tests/tcg/hexagon/test_vavgw.S
 create mode 100644 tests/tcg/hexagon/test_vcmpb.S
 create mode 100644 tests/tcg/hexagon/test_vcmpw.S
 create mode 100644 tests/tcg/hexagon/test_vcmpy.S
 create mode 100644 tests/tcg/hexagon/test_vlsrw.S
 create mode 100644 tests/tcg/hexagon/test_vmaxh.S
 create mode 100644 tests/tcg/hexagon/test_vminh.S
 create mode 100644 tests/tcg/hexagon/test_vpmpyh.S
 create mode 100644 tests/tcg/hexagon/test_vspliceb.S

-- 
2.30.0

On 2/11/21 1:50 PM, Alessandro Di Federico via wrote:
> From: Alessandro Di Federico <ale@rev.ng>
> 
> This patchset introduces the idef-parser for target/hexagon.
> It is based on "[PATCH v8 00/35] Hexagon patch series".
> 
> `idef-parser` is a build-time tool built using flex and bison. Its aim
> is to generate a large part of the tiny code generator frontend for
> Hexagon. The prototype of idef-parser has been presented at KVM Forum
> 2019 ("QEMU-Hexagon: Automatic Translation of the ISA Manual Pseudcode
> to Tiny Code Instructions"):
> 
>     https://www.youtube.com/watch?v=3EpnTYBOXCI
> 
> `target/hexagon/idef-parser/README.rst` provides an overview of the
> parser and its inner working.
> 
> Please consider this an RFC, this is our first large patchset. Feedback
> is more than welcome.

Well, first off, this fails to build with our gitlab CI.

There are several problems, apart from the new bison parse.error option
previously reported:

(1) bison and flex not added to dockerfiles, so the packages are not available
to CI.

(2) idef-parser program not marked with "native: true", so we get an early
meson error about that.

(3) flex = generator() does not specify both output files, so the proper
dependency on idef-parser.yy.h is not present.

(4) idef-parser.yy.h is placed in the wrong directory, because you used
'%option header-file="xyz"' instead of the command-line
'--headerfile=@OUTPUT1@'.  This results in an immediate compilation failure,
because the file isn't found.

(5) There's an odd mismatch in file naming, in which you have
idef-lexer.lex.yy.c but idef-parser.yy.h.  Not a bug, really, but it looks odd
when fixing 3 & 4.

I can provide a patch for these if you like, but they're fairly obvious when
you try it.


r~

On Sun, 14 Feb 2021 13:52:25 -0800
Richard Henderson <richard.henderson@linaro.org> wrote:

> Well, first off, this fails to build with our gitlab CI.
> 
> There are several problems, apart from the new bison parse.error
> option previously reported:
> 
> (1) bison and flex not added to dockerfiles, so the packages are not
> available to CI.
> 
> (2) idef-parser program not marked with "native: true", so we get an
> early meson error about that.
> 
> (3) flex = generator() does not specify both output files, so the
> proper dependency on idef-parser.yy.h is not present.
> 
> (4) idef-parser.yy.h is placed in the wrong directory, because you
> used '%option header-file="xyz"' instead of the command-line
> '--headerfile=@OUTPUT1@'.  This results in an immediate compilation
> failure, because the file isn't found.
> 
> (5) There's an odd mismatch in file naming, in which you have
> idef-lexer.lex.yy.c but idef-parser.yy.h.  Not a bug, really, but it
> looks odd when fixing 3 & 4.

We've integrated the suggested changes and tested the v2 on the Gitlab
CI. It should now work properly.

-- 
Alessandro Di Federico
rev.ng