From nobody Wed Nov 5 13:04:21 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15345190421481020.8773529569281; Fri, 17 Aug 2018 08:17:22 -0700 (PDT) Received: from localhost ([::1]:34687 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqgUz-0000Vu-Pn for importer@patchew.org; Fri, 17 Aug 2018 11:17:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46306) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fqgKI-00036T-TJ for qemu-devel@nongnu.org; Fri, 17 Aug 2018 11:06:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fqgKF-0001lG-32 for qemu-devel@nongnu.org; Fri, 17 Aug 2018 11:06:10 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46438 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fqgKE-0001i8-B7 for qemu-devel@nongnu.org; Fri, 17 Aug 2018 11:06:06 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 49EE58011454; Fri, 17 Aug 2018 15:06:05 +0000 (UTC) Received: from blackfin.pond.sub.org (ovpn-116-56.ams2.redhat.com [10.36.116.56]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0651510D18E0; Fri, 17 Aug 2018 15:06:05 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 270D3110E655; Fri, 17 Aug 2018 17:06:00 +0200 (CEST) From: Markus Armbruster To: qemu-devel@nongnu.org Date: Fri, 17 Aug 2018 17:05:17 +0200 Message-Id: <20180817150559.16243-19-armbru@redhat.com> In-Reply-To: <20180817150559.16243-1-armbru@redhat.com> References: <20180817150559.16243-1-armbru@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 17 Aug 2018 15:06:05 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 17 Aug 2018 15:06:05 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'armbru@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v2 18/60] json: Revamp lexer documentation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marcandre.lureau@redhat.com, mdroth@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-lexer.c | 80 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 71 insertions(+), 9 deletions(-) diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c index e85e9a78ff..109a7d8bb8 100644 --- a/qobject/json-lexer.c +++ b/qobject/json-lexer.c @@ -18,21 +18,83 @@ #define MAX_TOKEN_SIZE (64ULL << 20) =20 /* - * Required by JSON (RFC 7159): + * From RFC 7159 "The JavaScript Object Notation (JSON) Data + * Interchange Format", with [comments in brackets]: * - * \"([^\\\"]|\\[\"'\\/bfnrt]|\\u[0-9a-fA-F]{4})*\" - * -?(0|[1-9][0-9]*)(.[0-9]+)?([eE][-+]?[0-9]+)? - * [{}\[\],:] - * [a-z]+ # covers null, true, false + * The set of tokens includes six structural characters, strings, + * numbers, and three literal names. * - * Extension of '' strings: + * These are the six structural characters: * - * '([^\\']|\\[\"'\\/bfnrt]|\\u[0-9a-fA-F]{4})*' + * begin-array =3D ws %x5B ws ; [ left square bracket + * begin-object =3D ws %x7B ws ; { left curly bracket + * end-array =3D ws %x5D ws ; ] right square bracket + * end-object =3D ws %x7D ws ; } right curly bracket + * name-separator =3D ws %x3A ws ; : colon + * value-separator =3D ws %x2C ws ; , comma * - * Extension for vararg handling in JSON construction: + * Insignificant whitespace is allowed before or after any of the six + * structural characters. + * [This lexer accepts it before or after any token, which is actually + * the same, as the grammar always has structural characters between + * other tokens.] * - * %((l|ll|I64)?d|[ipsf]) + * ws =3D *( + * %x20 / ; Space + * %x09 / ; Horizontal tab + * %x0A / ; Line feed or New line + * %x0D ) ; Carriage return * + * [...] three literal names: + * false null true + * [This lexer accepts [a-z]+, and leaves rejecting unknown literal + * names to the parser.] + * + * [Numbers:] + * + * number =3D [ minus ] int [ frac ] [ exp ] + * decimal-point =3D %x2E ; . + * digit1-9 =3D %x31-39 ; 1-9 + * e =3D %x65 / %x45 ; e E + * exp =3D e [ minus / plus ] 1*DIGIT + * frac =3D decimal-point 1*DIGIT + * int =3D zero / ( digit1-9 *DIGIT ) + * minus =3D %x2D ; - + * plus =3D %x2B ; + + * zero =3D %x30 ; 0 + * + * [Strings:] + * string =3D quotation-mark *char quotation-mark + * + * char =3D unescaped / + * escape ( + * %x22 / ; " quotation mark U+0022 + * %x5C / ; \ reverse solidus U+005C + * %x2F / ; / solidus U+002F + * %x62 / ; b backspace U+0008 + * %x66 / ; f form feed U+000C + * %x6E / ; n line feed U+000A + * %x72 / ; r carriage return U+000D + * %x74 / ; t tab U+0009 + * %x75 4HEXDIG ) ; uXXXX U+XXXX + * escape =3D %x5C ; \ + * quotation-mark =3D %x22 ; " + * unescaped =3D %x20-21 / %x23-5B / %x5D-10FFFF + * + * + * Extensions over RFC 7159: + * - Extra escape sequence in strings: + * 0x27 (apostrophe) is recognized after escape, too + * - Single-quoted strings: + * Like double-quoted strings, except they're delimited by %x27 + * (apostrophe) instead of %x22 (quotation mark), and can't contain + * unescaped apostrophe, but can contain unescaped quotation mark. + * - Interpolation: + * interpolation =3D %((l|ll|I64)[du]|[ipsf]) + * + * Note: + * - Input must be encoded in UTF-8. + * - Decoding and validating is left to the parser. */ =20 enum json_lexer_state { --=20 2.17.1