From nobody Wed Nov 5 16:36:54 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1535140634048155.0442019281021; Fri, 24 Aug 2018 12:57:14 -0700 (PDT) Received: from localhost ([::1]:43310 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ftICm-0007qU-S7 for importer@patchew.org; Fri, 24 Aug 2018 15:57:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33236) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ftHoi-0002Ic-UK for qemu-devel@nongnu.org; Fri, 24 Aug 2018 15:32:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ftHoe-0001gO-LT for qemu-devel@nongnu.org; Fri, 24 Aug 2018 15:32:20 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:41644 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ftHod-0001Zz-Fu for qemu-devel@nongnu.org; Fri, 24 Aug 2018 15:32:16 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AFE84804BAAD for ; Fri, 24 Aug 2018 19:32:10 +0000 (UTC) Received: from blackfin.pond.sub.org (ovpn-116-97.ams2.redhat.com [10.36.116.97]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6F33A215670B for ; Fri, 24 Aug 2018 19:32:10 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id D2DE81132930; Fri, 24 Aug 2018 21:32:06 +0200 (CEST) From: Markus Armbruster To: qemu-devel@nongnu.org Date: Fri, 24 Aug 2018 21:31:27 +0200 Message-Id: <20180824193206.25475-20-armbru@redhat.com> In-Reply-To: <20180824193206.25475-1-armbru@redhat.com> References: <20180824193206.25475-1-armbru@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 24 Aug 2018 19:32:10 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 24 Aug 2018 19:32:10 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'armbru@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 19/58] json: Revamp lexer documentation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180823164025.12553-20-armbru@redhat.com> --- qobject/json-lexer.c | 80 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 71 insertions(+), 9 deletions(-) diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c index e85e9a78ff..902fe60846 100644 --- a/qobject/json-lexer.c +++ b/qobject/json-lexer.c @@ -18,21 +18,83 @@ #define MAX_TOKEN_SIZE (64ULL << 20) =20 /* - * Required by JSON (RFC 7159): + * From RFC 8259 "The JavaScript Object Notation (JSON) Data + * Interchange Format", with [comments in brackets]: * - * \"([^\\\"]|\\[\"'\\/bfnrt]|\\u[0-9a-fA-F]{4})*\" - * -?(0|[1-9][0-9]*)(.[0-9]+)?([eE][-+]?[0-9]+)? - * [{}\[\],:] - * [a-z]+ # covers null, true, false + * The set of tokens includes six structural characters, strings, + * numbers, and three literal names. * - * Extension of '' strings: + * These are the six structural characters: * - * '([^\\']|\\[\"'\\/bfnrt]|\\u[0-9a-fA-F]{4})*' + * begin-array =3D ws %x5B ws ; [ left square bracket + * begin-object =3D ws %x7B ws ; { left curly bracket + * end-array =3D ws %x5D ws ; ] right square bracket + * end-object =3D ws %x7D ws ; } right curly bracket + * name-separator =3D ws %x3A ws ; : colon + * value-separator =3D ws %x2C ws ; , comma * - * Extension for vararg handling in JSON construction: + * Insignificant whitespace is allowed before or after any of the six + * structural characters. + * [This lexer accepts it before or after any token, which is actually + * the same, as the grammar always has structural characters between + * other tokens.] * - * %((l|ll|I64)?d|[ipsf]) + * ws =3D *( + * %x20 / ; Space + * %x09 / ; Horizontal tab + * %x0A / ; Line feed or New line + * %x0D ) ; Carriage return * + * [...] three literal names: + * false null true + * [This lexer accepts [a-z]+, and leaves rejecting unknown literal + * names to the parser.] + * + * [Numbers:] + * + * number =3D [ minus ] int [ frac ] [ exp ] + * decimal-point =3D %x2E ; . + * digit1-9 =3D %x31-39 ; 1-9 + * e =3D %x65 / %x45 ; e E + * exp =3D e [ minus / plus ] 1*DIGIT + * frac =3D decimal-point 1*DIGIT + * int =3D zero / ( digit1-9 *DIGIT ) + * minus =3D %x2D ; - + * plus =3D %x2B ; + + * zero =3D %x30 ; 0 + * + * [Strings:] + * string =3D quotation-mark *char quotation-mark + * + * char =3D unescaped / + * escape ( + * %x22 / ; " quotation mark U+0022 + * %x5C / ; \ reverse solidus U+005C + * %x2F / ; / solidus U+002F + * %x62 / ; b backspace U+0008 + * %x66 / ; f form feed U+000C + * %x6E / ; n line feed U+000A + * %x72 / ; r carriage return U+000D + * %x74 / ; t tab U+0009 + * %x75 4HEXDIG ) ; uXXXX U+XXXX + * escape =3D %x5C ; \ + * quotation-mark =3D %x22 ; " + * unescaped =3D %x20-21 / %x23-5B / %x5D-10FFFF + * + * + * Extensions over RFC 8259: + * - Extra escape sequence in strings: + * 0x27 (apostrophe) is recognized after escape, too + * - Single-quoted strings: + * Like double-quoted strings, except they're delimited by %x27 + * (apostrophe) instead of %x22 (quotation mark), and can't contain + * unescaped apostrophe, but can contain unescaped quotation mark. + * - Interpolation: + * interpolation =3D %((l|ll|I64)[du]|[ipsf]) + * + * Note: + * - Input must be encoded in UTF-8. + * - Decoding and validating is left to the parser. */ =20 enum json_lexer_state { --=20 2.17.1