From nobody Wed Nov 5 10:30:43 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 153504264423086.58357065380983; Thu, 23 Aug 2018 09:44:04 -0700 (PDT) Received: from localhost ([::1]:37709 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fssiI-00058x-D1 for importer@patchew.org; Thu, 23 Aug 2018 12:44:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60967) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fssgE-0003Sq-BC for qemu-devel@nongnu.org; Thu, 23 Aug 2018 12:41:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fssev-0003bc-3Z for qemu-devel@nongnu.org; Thu, 23 Aug 2018 12:40:37 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:44754 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fsseu-0003aI-IT for qemu-devel@nongnu.org; Thu, 23 Aug 2018 12:40:32 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 16FFB40216F6; Thu, 23 Aug 2018 16:40:32 +0000 (UTC) Received: from blackfin.pond.sub.org (ovpn-116-97.ams2.redhat.com [10.36.116.97]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 449B110CD897; Thu, 23 Aug 2018 16:40:31 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 121461156572; Thu, 23 Aug 2018 18:40:26 +0200 (CEST) From: Markus Armbruster To: qemu-devel@nongnu.org Date: Thu, 23 Aug 2018 18:39:53 +0200 Message-Id: <20180823164025.12553-27-armbru@redhat.com> In-Reply-To: <20180823164025.12553-1-armbru@redhat.com> References: <20180823164025.12553-1-armbru@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 23 Aug 2018 16:40:32 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 23 Aug 2018 16:40:32 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'armbru@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 26/58] json: Leave rejecting invalid escape sequences to parser X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marcandre.lureau@redhat.com, mdroth@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Both lexer and parser reject invalid escape sequences in strings. The parser's check is useless. The lexer ends the token right after the first non-well-formed byte. This tends to lead to suboptimal error reporting. For instance, input {"abc\@ijk": 1} produces the tokens JSON_LCURLY { JSON_ERROR "abc\@ JSON_KEYWORD ijk JSON_ERROR ": 1}\n The parser then reports three errors Invalid JSON syntax JSON parse error, invalid keyword 'ijk' Invalid JSON syntax before it recovers at the newline. Drop the lexer's escape sequence checking, and make it accept the same characters after backslash it accepts elsewhere in strings. It now produces JSON_LCURLY { JSON_STRING "abc\@ijk" JSON_COLON : JSON_INTEGER 1 JSON_RCURLY and the parser reports just JSON parse error, invalid escape sequence in string While there, fix parse_string()'s inaccurate function comment. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-lexer.c | 72 +++---------------------------------------- qobject/json-parser.c | 56 +++++++++++++++++++-------------- 2 files changed, 37 insertions(+), 91 deletions(-) diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c index 4c402f62d3..0731779470 100644 --- a/qobject/json-lexer.c +++ b/qobject/json-lexer.c @@ -80,6 +80,8 @@ * escape =3D %x5C ; \ * quotation-mark =3D %x22 ; " * unescaped =3D %x20-21 / %x23-5B / %x5D-10FFFF + * [This lexer accepts any non-control character after escape, and + * leaves rejecting invalid ones to the parser.] * * * Extensions over RFC 8259: @@ -99,16 +101,8 @@ =20 enum json_lexer_state { IN_ERROR =3D 0, /* must really be 0, see json_lexer[] */ - IN_DQ_UCODE3, - IN_DQ_UCODE2, - IN_DQ_UCODE1, - IN_DQ_UCODE0, IN_DQ_STRING_ESCAPE, IN_DQ_STRING, - IN_SQ_UCODE3, - IN_SQ_UCODE2, - IN_SQ_UCODE1, - IN_SQ_UCODE0, IN_SQ_STRING_ESCAPE, IN_SQ_STRING, IN_ZERO, @@ -144,37 +138,8 @@ static const uint8_t json_lexer[][256] =3D { /* Relies on default initialization to IN_ERROR! */ =20 /* double quote string */ - [IN_DQ_UCODE3] =3D { - ['0' ... '9'] =3D IN_DQ_STRING, - ['a' ... 'f'] =3D IN_DQ_STRING, - ['A' ... 'F'] =3D IN_DQ_STRING, - }, - [IN_DQ_UCODE2] =3D { - ['0' ... '9'] =3D IN_DQ_UCODE3, - ['a' ... 'f'] =3D IN_DQ_UCODE3, - ['A' ... 'F'] =3D IN_DQ_UCODE3, - }, - [IN_DQ_UCODE1] =3D { - ['0' ... '9'] =3D IN_DQ_UCODE2, - ['a' ... 'f'] =3D IN_DQ_UCODE2, - ['A' ... 'F'] =3D IN_DQ_UCODE2, - }, - [IN_DQ_UCODE0] =3D { - ['0' ... '9'] =3D IN_DQ_UCODE1, - ['a' ... 'f'] =3D IN_DQ_UCODE1, - ['A' ... 'F'] =3D IN_DQ_UCODE1, - }, [IN_DQ_STRING_ESCAPE] =3D { - ['b'] =3D IN_DQ_STRING, - ['f'] =3D IN_DQ_STRING, - ['n'] =3D IN_DQ_STRING, - ['r'] =3D IN_DQ_STRING, - ['t'] =3D IN_DQ_STRING, - ['/'] =3D IN_DQ_STRING, - ['\\'] =3D IN_DQ_STRING, - ['\''] =3D IN_DQ_STRING, - ['\"'] =3D IN_DQ_STRING, - ['u'] =3D IN_DQ_UCODE0, + [0x20 ... 0xFD] =3D IN_DQ_STRING, }, [IN_DQ_STRING] =3D { [0x20 ... 0xFD] =3D IN_DQ_STRING, @@ -183,37 +148,8 @@ static const uint8_t json_lexer[][256] =3D { }, =20 /* single quote string */ - [IN_SQ_UCODE3] =3D { - ['0' ... '9'] =3D IN_SQ_STRING, - ['a' ... 'f'] =3D IN_SQ_STRING, - ['A' ... 'F'] =3D IN_SQ_STRING, - }, - [IN_SQ_UCODE2] =3D { - ['0' ... '9'] =3D IN_SQ_UCODE3, - ['a' ... 'f'] =3D IN_SQ_UCODE3, - ['A' ... 'F'] =3D IN_SQ_UCODE3, - }, - [IN_SQ_UCODE1] =3D { - ['0' ... '9'] =3D IN_SQ_UCODE2, - ['a' ... 'f'] =3D IN_SQ_UCODE2, - ['A' ... 'F'] =3D IN_SQ_UCODE2, - }, - [IN_SQ_UCODE0] =3D { - ['0' ... '9'] =3D IN_SQ_UCODE1, - ['a' ... 'f'] =3D IN_SQ_UCODE1, - ['A' ... 'F'] =3D IN_SQ_UCODE1, - }, [IN_SQ_STRING_ESCAPE] =3D { - ['b'] =3D IN_SQ_STRING, - ['f'] =3D IN_SQ_STRING, - ['n'] =3D IN_SQ_STRING, - ['r'] =3D IN_SQ_STRING, - ['t'] =3D IN_SQ_STRING, - ['/'] =3D IN_SQ_STRING, - ['\\'] =3D IN_SQ_STRING, - ['\''] =3D IN_SQ_STRING, - ['\"'] =3D IN_SQ_STRING, - ['u'] =3D IN_SQ_UCODE0, + [0x20 ... 0xFD] =3D IN_SQ_STRING, }, [IN_SQ_STRING] =3D { [0x20 ... 0xFD] =3D IN_SQ_STRING, diff --git a/qobject/json-parser.c b/qobject/json-parser.c index a9b227f56c..7437827c24 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -106,30 +106,40 @@ static int hex2decimal(char ch) } =20 /** - * parse_string(): Parse a json string and return a QObject + * parse_string(): Parse a JSON string * - * string - * "" - * " chars " - * chars - * char - * char chars - * char - * any-Unicode-character- - * except-"-or-\-or- - * control-character - * \" - * \\ - * \/ - * \b - * \f - * \n - * \r - * \t - * \u four-hex-digits=20 + * From RFC 8259 "The JavaScript Object Notation (JSON) Data + * Interchange Format": + * + * char =3D unescaped / + * escape ( + * %x22 / ; " quotation mark U+0022 + * %x5C / ; \ reverse solidus U+005C + * %x2F / ; / solidus U+002F + * %x62 / ; b backspace U+0008 + * %x66 / ; f form feed U+000C + * %x6E / ; n line feed U+000A + * %x72 / ; r carriage return U+000D + * %x74 / ; t tab U+0009 + * %x75 4HEXDIG ) ; uXXXX U+XXXX + * escape =3D %x5C ; \ + * quotation-mark =3D %x22 ; " + * unescaped =3D %x20-21 / %x23-5B / %x5D-10FFFF + * + * Extensions over RFC 8259: + * - Extra escape sequence in strings: + * 0x27 (apostrophe) is recognized after escape, too + * - Single-quoted strings: + * Like double-quoted strings, except they're delimited by %x27 + * (apostrophe) instead of %x22 (quotation mark), and can't contain + * unescaped apostrophe, but can contain unescaped quotation mark. + * + * Note: + * - Encoding is modified UTF-8. + * - Invalid Unicode characters are rejected. + * - Control characters \x00..\x1F are rejected by the lexer. */ -static QString *qstring_from_escaped_str(JSONParserContext *ctxt, - JSONToken *token) +static QString *parse_string(JSONParserContext *ctxt, JSONToken *token) { const char *ptr =3D token->str; QString *str; @@ -495,7 +505,7 @@ static QObject *parse_literal(JSONParserContext *ctxt) =20 switch (token->type) { case JSON_STRING: - return QOBJECT(qstring_from_escaped_str(ctxt, token)); + return QOBJECT(parse_string(ctxt, token)); case JSON_INTEGER: { /* * Represent JSON_INTEGER as QNUM_I64 if possible, else as --=20 2.17.1