This rewrites the json-parser to use a push parser aka state machine.
While push parsers are inherently more complex than recursive descent,
the grammar for JSON is simple enough that the parser remains readable.
There is therefore no need to use e.g. QEMU coroutines.
Unlike the suggestion in commit 62815d85aed ("json: Redesign the callback
to consume JSON values", 2018-08-24), I kept the json-streamer concept.
It helps in handling input limits, it performs error recovery, and it
converts the token-at-a-time push interface to callbacks---all things
that are more easily done in a separate layer to keep the parser clean.
However, there is no need anymore for it to store partial JSON objects
in tokenized form, because the current state is stored in the push
parser's stack.
Another benefit is that QEMU can report the first parsing error
immediately, without waiting for parentheses to be balanced or for a
lexing error. Error recovery then proceeds as before (i.e., the next
parse still starts after balanced parentheses or a lexing error).
On top of the benefits intrinsic in the push architecture, it so happens
that it's really easy to add a location to JSON parsing errors now, so
do that as well.
The diffstat is unfavorable, but most of the new lines delta is really
new comments explaining the grammar and state machines.
Almost the same as v3, the only substantial change being to
restore the "expecting value" (actually now "expecting key") error
for not having a string or interpolation where a key is expected.
Other changes:
- Add some extra comments to json-parser.c
- Consistently use "top-level"
- Move json_parser_reset() right after the out_emit label
- Avoid <= comparisons for unsigned variables
- Add extra comments about error recovery situations in json-streamer.c
- Avoid double "JSON parser error, JSON parser error, stray '%s'"
- Spell location as %d:%d rather than "at line %d, column %d"
Thanks,
Paolo
Paolo Bonzini (6):
json-parser: replace with a push parser
json-streamer: reuse parser
json-streamer: make brace/bracket count unsigned
json-streamer: remove token queue
json-streamer: do not heap-allocate JSONToken
json-parser: add location to JSON parsing errors
include/qobject/json-parser.h | 16 +-
qobject/json-parser-int.h | 13 +-
qobject/json-lexer.c | 11 +-
qobject/json-parser.c | 589 +++++++++++++++++++---------------
qobject/json-streamer.c | 121 +++----
5 files changed, 427 insertions(+), 323 deletions(-)
--
2.54.0