From nobody Sun Jul 26 11:51:48 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782469076; cv=none; d=zohomail.com; s=zohoarc; b=j2+jRca7Y4zN/FO8SF+SLIDRwigsvzf+eoNZI2fvGiR6IYCIkiYnPpW/UcisnfJMYakO/r1vVxi4YMUwFe5qV673pEgHL2ZOH44iBH6r8xOoqBHgJ2FPjlZHSWgfv/98A+axg5qcwZQ8qzFxykW7vMBNlDx54Lm9myrnY0Hvo58= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782469076; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fYWIqbUMZiNIGFSZanWK622KV92npvNyKOKjAPQEbgk=; b=Btf7PqeEZsgf56djRkF/PYjlYRABX19xfItowCGPtNumkuU7DhuuMjimsI6of5l4lwVm0g4qWoo941X/B8uLg/l7BrfjepkcGBOzTBd1r7nxQMwlfepOMyU5ADDJSs5SRsqKmFQbCHOqA2HHw5ydfvui+izoYLDDfahNW7AK25Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782469076347702.9231082828238; Fri, 26 Jun 2026 03:17:56 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wd3cx-0006Ds-Vz; Fri, 26 Jun 2026 06:17:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3cw-0006Cv-6Q for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3ct-00053i-4Z for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:37 -0400 Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-468-9g4otM6POm6wJRHVyNwoCg-1; Fri, 26 Jun 2026 06:17:33 -0400 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-463dd955dafso424756f8f.1 for ; Fri, 26 Jun 2026 03:17:32 -0700 (PDT) Received: from [192.168.10.48] ([151.95.124.208]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-46f9db29a94sm719858f8f.22.2026.06.26.03.17.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2026 03:17:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782469054; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fYWIqbUMZiNIGFSZanWK622KV92npvNyKOKjAPQEbgk=; b=jK6c3/QXepTz/3jnNW2K/ePtk65W1a9FeHn8GAEL8doJgeZwjSgkDbaYRGo2k2Zio5GUiw KaDKEhPMB/1aT3WOpEorONzvbYUH2Oq8vZdbZSvlxxPCUEBQWVBjl81k5ky8zs797QN0U5 X4732SF0Kba1obMftkFUY/Mry00FRsA= X-MC-Unique: 9g4otM6POm6wJRHVyNwoCg-1 X-Mimecast-MFC-AGG-ID: 9g4otM6POm6wJRHVyNwoCg_1782469052 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1782469052; x=1783073852; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fYWIqbUMZiNIGFSZanWK622KV92npvNyKOKjAPQEbgk=; b=sNtmLi1c374xydDaWW2us3kEDxGkpK7AqIFu9Yu73VkZ3ZqwRgNCahyvOUWaCSdDst hXsYNCDrbbyIR85iDIKWGCFZjO1I3dS1VRIaqyAZF5hpt0ObDJsVH3dhwUKujAve+AM8 2tgGCgYtkngVmLVXYIzjNPSDZ2NzvpkLNYD1hHAyzDGTUwHY2iAmDErF+SHZrdPHoKiQ XQBSAJ+iauPmuZK8962IgOYlzPQ0pViGCRut21BjTv9utkpOpFppT7+q0Uo2yuC2H4Kq PuZPNcVsL4OjPe20NVGkwaJFcfkGzCAo4foEsBu8z2D1hwEOsK2MqO7s12EYePZp1Cjo Q/6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782469052; x=1783073852; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=fYWIqbUMZiNIGFSZanWK622KV92npvNyKOKjAPQEbgk=; b=g/lBlddj0w7drfHvtPvlEm/ura1FyG1Bz4j0HQ9sVSwQ1iFb+ygY9l59S1o9LCaSa0 7et3GurSyE9RJfCJM3UmtF+A1aekdEKPiE/OWb7veOOT8sKTTvW5D1zouDoQ4RNmAW0/ NWvgUAgJhjiSmjyRIxApBj843DVqjIshRo/DFfTLPOdXafh/JOWf45SSMIVmvgbpQSbe DUTQDGSTLcB0TuiduyP8YlKtr5muuMTuFgiG+sOBlolOIYWoVRldvSk5wvWvaNk3I1QW 6VS8AmnuD7uYy+GchTfpu3GMo12m9KUEizM9CH+4L6etjgRHv4jiwi4PT0Siv+2Quqr7 88fw== X-Gm-Message-State: AOJu0YzTNmYplNSyQc9s/xboCK0reTgQm+iUl1X44F+XuHXZCOGn7Od3 FY899WSXHpYGxuCHGHnQ6DKShAXBxIyViRqQWBD3cejcYKy131+8uwc4rGRJEzErrRmHN/DBbsU YJsQOzot4HFXvPQXA/OdrQhudsPQecs6/JF9JJbu5h2HjozET4SerMQy8pcINN8ZohcQik3BC3C GeZmfK2p9Y3usqMb4NX+Kz01JSqdPlkdG8cbGmMdFk X-Gm-Gg: AfdE7cnvsVruMXHi+JLUBrFnfDWeXKCVCSqinxfKsP+cAl1Z+3S/EIPZA3f3Pyk0oJ9 1u+Dlzbb0BS+sKyODJeDOBkjSIPgB4OBeGHsIA1IoUUAgVUEYw9MkdrCqdBz/lyB8VCExIHCX8d EvYm2VpA3xkPgCr6zs9Rxks1jtAa69fZ5ls0edyeUJLZPD8vzwuaGQG/h2M/02L2JsQejxXNoSI 0tFUMJ2OHsFf+cIvXm6Thb8qlTCqedloOwP6oKEG5+KOClSQxQABq249pYwNvGg/7XBtqZezdhn aiJtwu1IPqap8vPk28H5WGAT+83g5EC2War299fs+BulT6r1NPhtUBMZdRAGfHm8QsB8YjKwu2L NQf40+qJp6nehLIAmekCai02S3pg/d7j7VeLZoP9jem5vSTQHLx52vzvIImiOU9KTTPFoJCDnQk DuxuXV/AYCz5ZojT9e X-Received: by 2002:a05:600c:8b4b:b0:490:ba0a:1178 with SMTP id 5b1f17b1804b1-4926688974amr86016645e9.28.1782469051489; Fri, 26 Jun 2026 03:17:31 -0700 (PDT) X-Received: by 2002:a05:600c:8b4b:b0:490:ba0a:1178 with SMTP id 5b1f17b1804b1-4926688974amr86015755e9.28.1782469050789; Fri, 26 Jun 2026 03:17:30 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: armbru@redhat.com Subject: [PATCH 1/6] json-parser: replace with a push parser Date: Fri, 26 Jun 2026 12:17:21 +0200 Message-ID: <20260626101727.1727389-2-pbonzini@redhat.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626101727.1727389-1-pbonzini@redhat.com> References: <20260626101727.1727389-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782469077116158500 In order to avoid stashing all the tokens corresponding to a JSON value, embed the parsing stack and state machine in JSONParser. This is more efficient and allows for more prompt error recovery; it also does not make the code substantially larger than the current recursive descent parser, though the state machine is probably a bit harder to follow. The stack consists of QLists and QDicts corresponding to open brackets and braces, plus optionally a QString with the current key on top of each QDict. After each value is parsed, it is added to the top array or dictionary or, if the stack is empty, json_parser_feed returns the complete QObject. For now, json-streamer.c keeps tracking the tokens up until braces and brackets are balanced, and then shoves the whole queue of tokens into the push parser. The only logic change is that JSON_END_OF_INPUT always triggers the emptying of the queue; the parser takes notice and checks that there is nothing on the stack. Not using brace_count and bracket_count for this is the first step towards improved separation of concerns between json-parser.c and json-streamer.c. Signed-off-by: Paolo Bonzini Reviewed-by: Markus Armbruster --- include/qobject/json-parser.h | 6 + qobject/json-parser-int.h | 5 +- qobject/json-parser.c | 565 ++++++++++++++++++++-------------- qobject/json-streamer.c | 21 +- 4 files changed, 359 insertions(+), 238 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 7345a9bd5cb..05346fa816b 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -20,6 +20,12 @@ typedef struct JSONLexer { int x, y; } JSONLexer; =20 +typedef struct JSONParserContext { + Error *err; + GQueue *stack; + va_list *ap; +} JSONParserContext; + typedef struct JSONMessageParser { void (*emit)(void *opaque, QObject *json, Error *err); void *opaque; diff --git a/qobject/json-parser-int.h b/qobject/json-parser-int.h index 8c01f236276..1f435cb8eb2 100644 --- a/qobject/json-parser-int.h +++ b/qobject/json-parser-int.h @@ -49,6 +49,9 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, =20 /* json-parser.c */ JSONToken *json_token(JSONTokenType type, int x, int y, GString *tokstr); -QObject *json_parser_parse(GQueue *tokens, va_list *ap, Error **errp); +void json_parser_init(JSONParserContext *ctxt, va_list *ap); +void json_parser_reset(JSONParserContext *ctxt); +QObject *json_parser_feed(JSONParserContext *ctxt, const JSONToken *token,= Error **errp); +void json_parser_destroy(JSONParserContext *ctxt); =20 #endif diff --git a/qobject/json-parser.c b/qobject/json-parser.c index f6622b82b0a..845da3699aa 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -31,12 +31,111 @@ struct JSONToken { char str[]; }; =20 -typedef struct JSONParserContext { - Error *err; - JSONToken *current; - GQueue *buf; - va_list *ap; -} JSONParserContext; +/* + * The JSON parser is a push parser, returning to the caller after every + * token. Therefore it has an explicit representation of its parser + * stack; each stack entry consists of a parser state and a QObject: + * - a QList, for an array that is being added to + * - a QDict, for a dictionary that is being added to + * - a QString, for the key of the next pair that will be added to a QDict + * + * The stack represents an arbitrary nesting of arrays and dictionaries + * (whose next key has been parsed); it can also have a dictionary whose + * next key has not been parsed, but that can only happen at the top level. + * Because of this, the stack contents are always of the form + * "(QList | QDict QString)* QDict?". + * + * An empty stack represents the beginning of the parsing process, with + * start state BEFORE_VALUE. + */ + +typedef enum JSONParserState { + AFTER_LCURLY, + AFTER_LSQUARE, + BEFORE_KEY, + BEFORE_VALUE, + END_OF_KEY, + END_OF_VALUE, +} JSONParserState; + +typedef struct JSONParserStackEntry { + /* + * State when the container is completed or, for the top of the stack, + * entry state for the next token. + */ + JSONParserState state; + + /* + * A QString with the last parsed key, or a QList/QDict for the current + * container. + */ + QObject *partial; +} JSONParserStackEntry; + +/* + * This is the JSON grammar that's parsed, with the state transition and + * action at each point of the grammar. While this is not a formal + * description, "-> action" represents the pseudocode of the action + * and "-> STATE" sets the top stack entry's state to STATE. + * + * The state alone is enough to tell you what to parse; the state plus + * the type of the top of stack tells you which action to take. + * + * // The initial state is BEFORE_VALUE. + * input :=3D value -> END_OF_VALUE -> return parsed value + * (input | END_OF_INPUT) + * + * // entered on BEFORE_VALUE; after any of these rules are processed, the + * // parser has completed a QObject and is in the END_OF_VALUE state. + * // + * // When the parser reaches the END_OF_VALUE state, it examines the + * // top of the stack to see if it's coming from "input" (stack empty), + * // "array_items" (TOS is a QList) or "dict_pairs" (TOS is a QString; the + * // item below will be a QDict). It then proceeds with the corresponding + * // actions, which will be one of: + * // - return parsed value + * // - add value to QList + * // - pop QString with the key, add key/value to the QDict + * value :=3D literal -> END_OF_VALUE + * | '[' -> push empty QList -> AFTER_LSQUARE + * after_lsquare -> END_OF_VALUE + * | '{' -> push empty QDict -> AFTER_LCURLY + * after_lcurly -> END_OF_VALUE + * + * // non-recursive values, entered on BEFORE_VALUE + * literal :=3D INTEGER -> END_OF_VALUE + * | FLOAT -> END_OF_VALUE + * | KEYWORD -> END_OF_VALUE + * | STRING -> END_OF_VALUE + * | INTERP -> END_OF_VALUE + * + * // entered on AFTER_LSQUARE + * after_lsquare :=3D ']' -> pop completed QList -> END_OF_VALUE + * | =CF=B5 -> BEFORE_VALUE + * array_items -> END_OF_VALUE + * + * // entered on BEFORE_VALUE, with TOS being a QList + * array_items :=3D value -> add value to QList -> END_OF_VALUE + * (']' -> pop completed QList -> END_OF_VALUE + * | ',' -> BEFORE_VALUE + * array_items) -> END_OF_VALUE + * + * // entered on AFTER_LCURLY + * after_lcurly :=3D '}' -> pop completed QDict -> END_OF_VALUE + * | =CF=B5 -> BEFORE_KEY + * dict_pairs -> END_OF_VALUE + * + * // entered on BEFORE_KEY, with TOS being a QDict + * dict_pairs :=3D (STRING | INTERP) -> push QString -> END_OF_KEY + * ':' -> BEFORE_VALUE + * value -> pop QString + add pair to QDict -> END_OF_VA= LUE + * ('}' -> pop completed QDict -> END_OF_VALUE + * | ',' -> BEFORE_KEY + * dict_pairs) -> END_OF_VALUE + * + * Parse errors ignore the token. json_parser_reset() can be + * called to restart parsing from scratch, with an empty stack. + */ =20 #define BUG_ON(cond) assert(!(cond)) =20 @@ -49,7 +148,27 @@ typedef struct JSONParserContext { * 4) deal with premature EOI */ =20 -static QObject *parse_value(JSONParserContext *ctxt); +static inline JSONParserStackEntry *current_entry(JSONParserContext *ctxt) +{ + return g_queue_peek_tail(ctxt->stack); +} + +static void push_entry(JSONParserContext *ctxt, QObject *partial, + JSONParserState state) +{ + JSONParserStackEntry *entry =3D g_new(JSONParserStackEntry, 1); + entry->partial =3D partial; + entry->state =3D state; + g_queue_push_tail(ctxt->stack, entry); +} + +/* Drop the top entry and return the new top entry. */ +static JSONParserStackEntry *pop_entry(JSONParserContext *ctxt) +{ + JSONParserStackEntry *entry =3D g_queue_pop_tail(ctxt->stack); + g_free(entry); + return current_entry(ctxt); +} =20 /** * Error handler @@ -236,200 +355,10 @@ out: return NULL; } =20 -/* Note: the token object returned by parser_context_peek_token or - * parser_context_pop_token is deleted as soon as parser_context_pop_token - * is called again. - */ -static const JSONToken *parser_context_pop_token(JSONParserContext *ctxt) +/* Terminals */ + +static QObject *parse_keyword(JSONParserContext *ctxt, const JSONToken *to= ken) { - g_free(ctxt->current); - ctxt->current =3D g_queue_pop_head(ctxt->buf); - return ctxt->current; -} - -static const JSONToken *parser_context_peek_token(JSONParserContext *ctxt) -{ - return g_queue_peek_head(ctxt->buf); -} - -/** - * Parsing rules - */ -static int parse_pair(JSONParserContext *ctxt, QDict *dict) -{ - QObject *key_obj =3D NULL; - QString *key; - QObject *value; - const JSONToken *peek, *token; - - peek =3D parser_context_peek_token(ctxt); - if (peek =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - key_obj =3D parse_value(ctxt); - key =3D qobject_to(QString, key_obj); - if (!key) { - parse_error(ctxt, peek, "key is not a string in object"); - goto out; - } - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - if (token->type !=3D JSON_COLON) { - parse_error(ctxt, token, "missing : in object pair"); - goto out; - } - - value =3D parse_value(ctxt); - if (value =3D=3D NULL) { - parse_error(ctxt, token, "Missing value in dict"); - goto out; - } - - if (qdict_haskey(dict, qstring_get_str(key))) { - parse_error(ctxt, token, "duplicate key"); - goto out; - } - - qdict_put_obj(dict, qstring_get_str(key), value); - - qobject_unref(key_obj); - return 0; - -out: - qobject_unref(key_obj); - return -1; -} - -static QObject *parse_object(JSONParserContext *ctxt) -{ - QDict *dict =3D NULL; - const JSONToken *token, *peek; - - token =3D parser_context_pop_token(ctxt); - assert(token && token->type =3D=3D JSON_LCURLY); - - dict =3D qdict_new(); - - peek =3D parser_context_peek_token(ctxt); - if (peek =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - if (peek->type !=3D JSON_RCURLY) { - if (parse_pair(ctxt, dict) =3D=3D -1) { - goto out; - } - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - while (token->type !=3D JSON_RCURLY) { - if (token->type !=3D JSON_COMMA) { - parse_error(ctxt, token, "expected separator in dict"); - goto out; - } - - if (parse_pair(ctxt, dict) =3D=3D -1) { - goto out; - } - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - } - } else { - (void)parser_context_pop_token(ctxt); - } - - return QOBJECT(dict); - -out: - qobject_unref(dict); - return NULL; -} - -static QObject *parse_array(JSONParserContext *ctxt) -{ - QList *list =3D NULL; - const JSONToken *token, *peek; - - token =3D parser_context_pop_token(ctxt); - assert(token && token->type =3D=3D JSON_LSQUARE); - - list =3D qlist_new(); - - peek =3D parser_context_peek_token(ctxt); - if (peek =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - if (peek->type !=3D JSON_RSQUARE) { - QObject *obj; - - obj =3D parse_value(ctxt); - if (obj =3D=3D NULL) { - parse_error(ctxt, token, "expecting value"); - goto out; - } - - qlist_append_obj(list, obj); - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - while (token->type !=3D JSON_RSQUARE) { - if (token->type !=3D JSON_COMMA) { - parse_error(ctxt, token, "expected separator in list"); - goto out; - } - - obj =3D parse_value(ctxt); - if (obj =3D=3D NULL) { - parse_error(ctxt, token, "expecting value"); - goto out; - } - - qlist_append_obj(list, obj); - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - } - } else { - (void)parser_context_pop_token(ctxt); - } - - return QOBJECT(list); - -out: - qobject_unref(list); - return NULL; -} - -static QObject *parse_keyword(JSONParserContext *ctxt) -{ - const JSONToken *token; - - token =3D parser_context_pop_token(ctxt); assert(token && token->type =3D=3D JSON_KEYWORD); =20 if (!strcmp(token->str, "true")) { @@ -443,11 +372,9 @@ static QObject *parse_keyword(JSONParserContext *ctxt) return NULL; } =20 -static QObject *parse_interpolation(JSONParserContext *ctxt) +static QObject *parse_interpolation(JSONParserContext *ctxt, + const JSONToken *token) { - const JSONToken *token; - - token =3D parser_context_pop_token(ctxt); assert(token && token->type =3D=3D JSON_INTERP); =20 if (!strcmp(token->str, "%p")) { @@ -479,11 +406,8 @@ static QObject *parse_interpolation(JSONParserContext = *ctxt) return NULL; } =20 -static QObject *parse_literal(JSONParserContext *ctxt) +static QObject *parse_literal(JSONParserContext *ctxt, const JSONToken *to= ken) { - const JSONToken *token; - - token =3D parser_context_pop_token(ctxt); assert(token); =20 switch (token->type) { @@ -531,35 +455,174 @@ static QObject *parse_literal(JSONParserContext *ctx= t) } } =20 -static QObject *parse_value(JSONParserContext *ctxt) +/* Parsing state machine */ + +static QObject *parse_begin_value(JSONParserContext *ctxt, + const JSONToken *token) { - const JSONToken *token; - - token =3D parser_context_peek_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - return NULL; - } - switch (token->type) { case JSON_LCURLY: - return parse_object(ctxt); + push_entry(ctxt, QOBJECT(qdict_new()), AFTER_LCURLY); + return NULL; case JSON_LSQUARE: - return parse_array(ctxt); + push_entry(ctxt, QOBJECT(qlist_new()), AFTER_LSQUARE); + return NULL; case JSON_INTERP: - return parse_interpolation(ctxt); + return parse_interpolation(ctxt, token); case JSON_INTEGER: case JSON_FLOAT: case JSON_STRING: - return parse_literal(ctxt); + return parse_literal(ctxt, token); case JSON_KEYWORD: - return parse_keyword(ctxt); + return parse_keyword(ctxt, token); default: parse_error(ctxt, token, "expecting value"); return NULL; } } =20 +static QObject *parse_token(JSONParserContext *ctxt, const JSONToken *toke= n) +{ + JSONParserStackEntry *entry; + JSONParserState state; + QString *key; + QObject *key_obj =3D NULL, *value =3D NULL; + + entry =3D current_entry(ctxt); + state =3D entry ? entry->state : BEFORE_VALUE; + switch (state) { + case AFTER_LCURLY: + /* Grab '}' for empty object or fall through to BEFORE_KEY */ + assert(qobject_type(entry->partial) =3D=3D QTYPE_QDICT); + if (token->type =3D=3D JSON_RCURLY) { + value =3D entry->partial; + entry =3D pop_entry(ctxt); + break; + } + entry->state =3D BEFORE_KEY; + /* fall through */ + + case BEFORE_KEY: + /* Expecting object key */ + assert(qobject_type(entry->partial) =3D=3D QTYPE_QDICT); + if (token->type !=3D JSON_STRING && token->type !=3D JSON_INTERP) { + parse_error(ctxt, token, "expecting key"); + return NULL; + } + + key_obj =3D parse_begin_value(ctxt, token); + if (!key_obj) { + /* Parse error already reported */ + } else if (qobject_type(key_obj) !=3D QTYPE_QSTRING) { + /* An interpolation was valid syntactically but not %s */ + parse_error(ctxt, token, "key is not a string in object"); + } else { + /* Store key in a special entry on the stack */ + push_entry(ctxt, key_obj, END_OF_KEY); + } + return NULL; + + case END_OF_KEY: + /* Expecting ':' after key */ + assert(qobject_type(entry->partial) =3D=3D QTYPE_QSTRING); + if (token->type =3D=3D JSON_COLON) { + entry->state =3D BEFORE_VALUE; + } else { + parse_error(ctxt, token, "expecting ':'"); + } + return NULL; + + case AFTER_LSQUARE: + /* Grab ']' for empty array or fall through to BEFORE_VALUE */ + assert(qobject_type(entry->partial) =3D=3D QTYPE_QLIST); + if (token->type =3D=3D JSON_RSQUARE) { + value =3D entry->partial; + entry =3D pop_entry(ctxt); + break; + } + entry->state =3D BEFORE_VALUE; + /* fall through */ + + case BEFORE_VALUE: + /* Expecting value */ + assert(!entry || qobject_type(entry->partial) !=3D QTYPE_QDICT); + value =3D parse_begin_value(ctxt, token); + if (!value) { + /* Error or '['/'{' */ + return NULL; + } + /* Return value or insert it into a container */ + break; + + case END_OF_VALUE: + /* Grab ',' or ']' for array; ',' or '}' for object */ + if (qobject_to(QList, entry->partial)) { + /* Array */ + if (token->type !=3D JSON_RSQUARE) { + if (token->type =3D=3D JSON_COMMA) { + entry->state =3D BEFORE_VALUE; + } else { + parse_error(ctxt, token, "expected ',' or ']'"); + } + return NULL; + } + } else if (qobject_to(QDict, entry->partial)) { + /* Object */ + if (token->type !=3D JSON_RCURLY) { + if (token->type =3D=3D JSON_COMMA) { + entry->state =3D BEFORE_KEY; + } else { + parse_error(ctxt, token, "expected ',' or '}'"); + } + return NULL; + } + } else { + g_assert_not_reached(); + } + + /* Got ']' or '}'; return full value or insert into parent contain= er */ + value =3D entry->partial; + entry =3D pop_entry(ctxt); + break; + } + + assert(value); + if (entry =3D=3D NULL) { + /* Parse stack now empty, the top-level value is complete. */ + return value; + } + + /* + * Parse stack is not empty and entry->partial is the top of stack. + * It's a QString with the key (and a QDict is below it) if we're + * parsing an object, or a QList if we're parsing an array. + */ + key =3D qobject_to(QString, entry->partial); + if (key) { + const char *key_str; + QDict *dict; + + /* Pop off key, and store (key, value) in QDict. */ + entry =3D pop_entry(ctxt); + dict =3D qobject_to(QDict, entry->partial); + assert(dict); + key_str =3D qstring_get_str(key); + if (qdict_haskey(dict, key_str)) { + parse_error(ctxt, token, "duplicate key"); + qobject_unref(value); + return NULL; + } + qdict_put_obj(dict, key_str, value); + qobject_unref(key); + } else { + /* Array, just store value in the QList. */ + qlist_append_obj(qobject_to(QList, entry->partial), value); + } + + entry->state =3D END_OF_VALUE; + return NULL; +} + JSONToken *json_token(JSONTokenType type, int x, int y, GString *tokstr) { JSONToken *token =3D g_malloc(sizeof(JSONToken) + tokstr->len + 1); @@ -572,20 +635,56 @@ JSONToken *json_token(JSONTokenType type, int x, int = y, GString *tokstr) return token; } =20 -QObject *json_parser_parse(GQueue *tokens, va_list *ap, Error **errp) +void json_parser_reset(JSONParserContext *ctxt) { - JSONParserContext ctxt =3D { .buf =3D tokens, .ap =3D ap }; - QObject *result; + JSONParserStackEntry *entry; =20 - result =3D parse_value(&ctxt); - assert(ctxt.err || g_queue_is_empty(ctxt.buf)); - - error_propagate(errp, ctxt.err); - - while (!g_queue_is_empty(ctxt.buf)) { - parser_context_pop_token(&ctxt); + ctxt->err =3D NULL; + while ((entry =3D g_queue_pop_tail(ctxt->stack)) !=3D NULL) { + qobject_unref(entry->partial); + g_free(entry); } - g_free(ctxt.current); +} =20 +void json_parser_init(JSONParserContext *ctxt, va_list *ap) +{ + ctxt->stack =3D g_queue_new(); + ctxt->ap =3D ap; + json_parser_reset(ctxt); +} + +void json_parser_destroy(JSONParserContext *ctxt) +{ + json_parser_reset(ctxt); + g_queue_free(ctxt->stack); + ctxt->stack =3D NULL; +} + +/* + * Advance the parser based on the token that is passed. + * Return the finished top-level value if the token completes it. + * If an error is returned, the function must not be called without + * first resetting the parser. + */ +QObject *json_parser_feed(JSONParserContext *ctxt, const JSONToken *token, + Error **errp) +{ + QObject *result =3D NULL; + + assert(!ctxt->err); + switch (token->type) { + case JSON_END_OF_INPUT: + /* Check for premature end of input */ + if (!g_queue_is_empty(ctxt->stack)) { + parse_error(ctxt, token, "premature end of input"); + } + break; + + default: + result =3D parse_token(ctxt, token); + break; + } + + error_propagate(errp, ctxt->err); return result; } diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index b93d97b995f..6c93e6fd78d 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -32,6 +32,7 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, JSONTokenType type, int x, int y) { JSONMessageParser *parser =3D container_of(lexer, JSONMessageParser, l= exer); + JSONParserContext ctxt; QObject *json =3D NULL; Error *err =3D NULL; JSONToken *token; @@ -56,8 +57,7 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, if (g_queue_is_empty(&parser->tokens)) { return; } - json =3D json_parser_parse(&parser->tokens, parser->ap, &err); - goto out_emit; + break; default: break; } @@ -85,11 +85,24 @@ void json_message_process_token(JSONLexer *lexer, GStri= ng *input, g_queue_push_tail(&parser->tokens, token); =20 if ((parser->brace_count > 0 || parser->bracket_count > 0) - && parser->brace_count >=3D 0 && parser->bracket_count >=3D 0) { + && parser->brace_count >=3D 0 && parser->bracket_count >=3D 0 + && type !=3D JSON_END_OF_INPUT) { return; } =20 - json =3D json_parser_parse(&parser->tokens, parser->ap, &err); + json_parser_init(&ctxt, parser->ap); + + /* Process all tokens in the queue */ + while (!g_queue_is_empty(&parser->tokens)) { + token =3D g_queue_pop_head(&parser->tokens); + json =3D json_parser_feed(&ctxt, token, &err); + g_free(token); + if (json || err) { + break; + } + } + + json_parser_destroy(&ctxt); =20 out_emit: parser->brace_count =3D 0; --=20 2.54.0 From nobody Sun Jul 26 11:51:48 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782469078; cv=none; d=zohomail.com; s=zohoarc; b=I4tBMEp6UeOK4ywI355En/hrv17qAGTElp5S/RXtu5YjxBMpHvP3CPmfsFxPp/i9OdeMf/9XvG/E1PEkwBUoMH3XN4WVC4xf1Bd8UKAVpbxWUTWRWtdBVJLwrHNVZ6g7ma1RxWgVP+1X7S+dJgsdWPYOuLj/9ojjv5O3e39G1Zk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782469078; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=aGGvWJRtvW7IOFkOBW6H2Jyklms6smfTnzqEtf9KrDE=; b=hg9PMnnHifnzOmolldsySdjBhbvM/7KXbawtmdXbg1DA1YukQrWkkkUL05zVK+IrlVFW/hsZGlySIGb72rtlak//Ksv2cWtKzPc1cWSQcWfuirx1sR/5uoX+fjHq5tt3v4QuMfilw1LHalW6obiImlHUD8mKmARvVsDGVbQVZFQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782469078385298.3711198151825; Fri, 26 Jun 2026 03:17:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wd3cz-0006ER-GN; Fri, 26 Jun 2026 06:17:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3cx-0006DV-JL for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3cv-00053u-WB for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:39 -0400 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-636-WN7Sc6kHNx63CjYt_og4zg-1; Fri, 26 Jun 2026 06:17:36 -0400 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-49244130073so5942155e9.1 for ; Fri, 26 Jun 2026 03:17:35 -0700 (PDT) Received: from [192.168.10.48] ([151.95.124.208]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4926f94f213sm5785055e9.12.2026.06.26.03.17.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2026 03:17:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782469057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aGGvWJRtvW7IOFkOBW6H2Jyklms6smfTnzqEtf9KrDE=; b=ErPmjDygtDTA43YwLdcKrRRQiMh5yywyXri12Q2KDDoTkXYxDQoNNGFNpS7LeQCSCU1TSI Tn4FQQwQEa71fyXReEjKG03aoWOMnBEdLU1xPHPZZloWl21UO6BkXmj50XMDeu8HD97Zvm 79iTIqZo9i2Osdy89EGM9+jkqQxaE8o= X-MC-Unique: WN7Sc6kHNx63CjYt_og4zg-1 X-Mimecast-MFC-AGG-ID: WN7Sc6kHNx63CjYt_og4zg_1782469055 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1782469055; x=1783073855; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aGGvWJRtvW7IOFkOBW6H2Jyklms6smfTnzqEtf9KrDE=; b=NNp1WLmz9dCCIYlYn+gavtDJJM/uLGKce3bE/NDomSBcjPzGhZWMYT3A00nlQP9a/k P2KoU0HFcnyDGqAcmtU1G835rti+TxsPwvaLT+1HN1wLR3M2+lrIOS/WjwuuyNjy/1Yk z8gKZHSnpg28xvJoQ7HoGS/aDMRcytAbIqO93oEL0gDKyi5s+24PyeBjnj6nzocgCa+y KBgUSOWhKwYOfdS5De3uN5zKr+bqJfBcMMK9t8ADmNJZvjQbe1tFsC+tEGfP0RbhjFLe QyvjYV1fBOimSdwjlx5e0iLd2Gwsl0gkZ1zG90n6lbKoMq96ozxSwAMOd0Iss4OfB2pj QxAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782469055; x=1783073855; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=aGGvWJRtvW7IOFkOBW6H2Jyklms6smfTnzqEtf9KrDE=; b=ka5aRx/ctXth3H881XpoQ0dcqzTAf4pVvuewXv8U8NuRq+L1rFFD00a0V2nNGrm569 +unnJ24r/MVHaHz48VkG9oPmD+fy4daJh7sU8jnFb9EOPAde9RJtttQZ4efG51YqgN51 rmor3hwp4QHF/ecv05t417wBcYg9NGgYyyYfrkpA+nN8CHpdBGuuEXERXjuskPK8p+k/ A7yLhMe8h7acVwmSLkTH3+Cby4UfRtDKHQz8oK3UDbwTnVG/y6k05By1U/lSL+Gep/I6 sC99sMpgUo5DsId9FcsvSvHF1pcTwURVO+VjW4PhODVSooEM/NgtlDLrGOxwQapOOHIr G7ew== X-Gm-Message-State: AOJu0YygtP9U2DjNReLsKDxBHOOM7zZOXWpZ8NmRaByqNjxV2hqre764 uOsVekbKgYH4U0jSjCYm+gIJQdwahNM1I5qwwVhJcEDDJju/OiC/l9cVLbie8ME7kIa/3R4pwR8 TSTq+kj84DfQqABK2JCnUbzKomxZBqWNCsSpG5pTUENH+AoMqGwKuse4N0ET65Am0RZl4kdtDWh nvMC3RJUKWBlyEzB+I29my6lIwWv6H/wMavnn79/if X-Gm-Gg: AfdE7ckYPOmb2cqYAx3+G6QSmfNhrZJUT7B5NzCZJE3eI6b3b82pAIv1ZdIZ50WxWHy JECl6GZBwbXmssNdxys+FYUPofNCPhstuU+Vnsiym9yOWbYqSl8wCNYcUcu8YcR1B8bQ4pKcEfn t6/b2KsyixU0eD9EFSRzj6Pa3qi8tXoK2eLdIgoTX0ln5PNNuwn0IsPF1ooCBzAThU/O8HteXVg K8HoEAbhXgOhC3XlaQmDtBYAuAYoYPKcVNuK7uxxvyRaygDJVUxe4mkSTwpBR/euzbnigBTCXws QUGYooiRHOJY9vRlHbvzM0TpYMQ8J0Q4LRqc2ZV74jO5iyt0hoIFaWW9trs1+zpUTchfm1RzOXm h6q7B615bKoO+4IYPTQFoX7o7bstHHq/ZdEV5oRbdc/WAyTdc8Z7jfWu1XjsjNjF6Lqc3AweL4M YqDeSq3v+71OAh6tMK X-Received: by 2002:a05:600c:4e4b:b0:490:e196:eaee with SMTP id 5b1f17b1804b1-49266872cd7mr83692345e9.10.1782469054778; Fri, 26 Jun 2026 03:17:34 -0700 (PDT) X-Received: by 2002:a05:600c:4e4b:b0:490:e196:eaee with SMTP id 5b1f17b1804b1-49266872cd7mr83691625e9.10.1782469053943; Fri, 26 Jun 2026 03:17:33 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: armbru@redhat.com Subject: [PATCH 2/6] json-streamer: reuse parser Date: Fri, 26 Jun 2026 12:17:22 +0200 Message-ID: <20260626101727.1727389-3-pbonzini@redhat.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626101727.1727389-1-pbonzini@redhat.com> References: <20260626101727.1727389-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782469080776158500 Content-Type: text/plain; charset="utf-8" The push parser can be reset, so reuse it when the json-streamer detects a completed toplevel object. Reviewed-by: Markus Armbruster Signed-off-by: Paolo Bonzini Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- include/qobject/json-parser.h | 2 +- qobject/json-streamer.c | 11 ++++------- 2 files changed, 5 insertions(+), 8 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 05346fa816b..4c3d89f751f 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -29,8 +29,8 @@ typedef struct JSONParserContext { typedef struct JSONMessageParser { void (*emit)(void *opaque, QObject *json, Error *err); void *opaque; - va_list *ap; JSONLexer lexer; + JSONParserContext parser; int brace_count; int bracket_count; GQueue tokens; diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index 6c93e6fd78d..6c4f99b3e7f 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -32,7 +32,6 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, JSONTokenType type, int x, int y) { JSONMessageParser *parser =3D container_of(lexer, JSONMessageParser, l= exer); - JSONParserContext ctxt; QObject *json =3D NULL; Error *err =3D NULL; JSONToken *token; @@ -90,21 +89,18 @@ void json_message_process_token(JSONLexer *lexer, GStri= ng *input, return; } =20 - json_parser_init(&ctxt, parser->ap); - /* Process all tokens in the queue */ while (!g_queue_is_empty(&parser->tokens)) { token =3D g_queue_pop_head(&parser->tokens); - json =3D json_parser_feed(&ctxt, token, &err); + json =3D json_parser_feed(&parser->parser, token, &err); g_free(token); if (json || err) { break; } } =20 - json_parser_destroy(&ctxt); - out_emit: + json_parser_reset(&parser->parser); parser->brace_count =3D 0; parser->bracket_count =3D 0; json_message_free_tokens(parser); @@ -119,12 +115,12 @@ void json_message_parser_init(JSONMessageParser *pars= er, { parser->emit =3D emit; parser->opaque =3D opaque; - parser->ap =3D ap; parser->brace_count =3D 0; parser->bracket_count =3D 0; g_queue_init(&parser->tokens); parser->token_size =3D 0; =20 + json_parser_init(&parser->parser, ap); json_lexer_init(&parser->lexer, !!ap); } =20 @@ -144,4 +140,5 @@ void json_message_parser_destroy(JSONMessageParser *par= ser) { json_lexer_destroy(&parser->lexer); json_message_free_tokens(parser); + json_parser_destroy(&parser->parser); } --=20 2.54.0 From nobody Sun Jul 26 11:51:48 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782469081; cv=none; d=zohomail.com; s=zohoarc; b=WB5l55JgZs+TmdAYko8dOo+tjteoLMR+sPQVReqGh/GP37/GaeTp5N/e4Rcp/OSkpAiex+gW491Y2rAdX0Ts8V6g3sB0R+1XJIao8K1ViE+iyPj9HaOeSpGJqbN0PzpP7tfITkgkLE5HPwzhSLOH7KS+zRw4o3BJlYKAgtW8MqI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782469081; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=byDGriG36cugk0iUUjibQB2wOLS/oNL2G+fOn7UmSQs=; b=jgEnD0S/8CkCfyXWFU0c+NJCnXGpWKC89TNwdHKtToxnsLEOpFJIxusmC0dlk8qD6MtdGyZeKLmFxUs8+/v4dEavluaHKywSl/zBiVWJaNnOsaGopRgvGTlCRcsoIhyZLqsdlsv35PkHoP7kRpWWnDuRBiX1SOWEwS76Yt9fF5s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782469081329472.88241078229385; Fri, 26 Jun 2026 03:18:01 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wd3d3-0006F6-Uh; Fri, 26 Jun 2026 06:17:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3d0-0006EV-06 for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3cy-00054G-CS for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:41 -0400 Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-384-8_eRVnhbM8urCiXdnTllYw-1; Fri, 26 Jun 2026 06:17:38 -0400 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-490b7959b9fso6333565e9.2 for ; Fri, 26 Jun 2026 03:17:37 -0700 (PDT) Received: from [192.168.10.48] ([151.95.124.208]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49269020266sm65293625e9.15.2026.06.26.03.17.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2026 03:17:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782469059; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=byDGriG36cugk0iUUjibQB2wOLS/oNL2G+fOn7UmSQs=; b=VU4WL31Wkc7E5RGsvyeNc0Rqaexq6NG7yhhoDHNb4HcZy6cNLMTAC+GVdcyjxsSOjuLj5y 8hNJffl+6a67ygRk+d5kKuQ/XxtOdHGj8HsTkMogWuQDU+RgyRxzTECPWKIGPQkZXjtvuK sezaK00qQNcSzHtNt9WcZ6slAg9BPJk= X-MC-Unique: 8_eRVnhbM8urCiXdnTllYw-1 X-Mimecast-MFC-AGG-ID: 8_eRVnhbM8urCiXdnTllYw_1782469057 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1782469057; x=1783073857; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=byDGriG36cugk0iUUjibQB2wOLS/oNL2G+fOn7UmSQs=; b=RfQcMKVkUlZZygDVQlDCXRtoqz8XtJesjR1/L9EFuauJZlO838d2Bu5fw76fNTzUjG FjdV5pCAO3euC/N+offqZrh6hpiIiARNDGcWpqaSQd++ppJGj2yX0z2dc72QRyAQ0KrD grDbeNT8yrHCYJe/od3lIPaNBjp2Oy9i5pXHIgHGrTD64vMgKkqKelWJ2Ln1IYEBktrP EUIarE/8bnqgB14tXf02CTVi50kfTLzTMnle2nUJm7WX8zQwckUQl7X8gBM2TbWeXgMz U3G+p8JgpJB/Y/6dFo8323Gwksj2Hmlx4c5hrph8F0Ib7lZ2Ln3B9V6OvffL1Cklj5IB oxuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782469057; x=1783073857; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=byDGriG36cugk0iUUjibQB2wOLS/oNL2G+fOn7UmSQs=; b=np0qz9evFof/k/WylFRmDDsKdBJN9rSPiqYMzBAn/frezMKBRpJYWnEadzTDplMSb9 UcAYFNLwAqckXEerFwGlOAm8nyboyMeArbvWs8BspMYVCpEty77fQZ26vwJtWnKzSTn8 zWYT/QCn4D0yQa2bqb+58T8G7Hkk8+8QzriHQDD5tihaD9w4ZZZlUEOv2KkYqflwEAp8 p5VuRpsmSSQLdzv4p5FKw57mkQvZeSqtwiuObagOzmHnLwLQmnWgx1SbLx6KlBUaUC8f 0CHmaS5cBII36REJhxIQDDM7SUNg7gr+WqdTISxIIaffzXmdXbQfgkwVhBUtSqNId5K7 NMwQ== X-Gm-Message-State: AOJu0Yz+KZBlrdeb3ALt9OppsEkMvB1qoI1TlkrW/TIDRc4LB8j734d+ DRc4TTwaAotKZ99g5YkOQH7NTzAaKXCzcpZKYDAQHIsEP3HZSAUBdJASi+WyCLv1Kx68T2dL0/V BSHujYoby9WRPyX+5mnlvZNCXE9CBMCrtIxtx7ZhcRAIEp6p8jWNoPiEgnB8pHyY7DUOrvVm4jQ 9/M/g9cac1/OO1S5mCQaHFHxk8KchWx+vyj3xUwcoW X-Gm-Gg: AfdE7cnw+41i9T/P0PekH9KzyF6OC6EW3j8Ov3kJ2/bUZcppuOn32OikJyPRVn42duq Ndw+BgMoJF07SQiJ9rVGGCDxo1LEcZ4fxul9RH5LYu8PFmcwJSnthY826ICAJYfFp901M28y/9Y V+U5BxC+Wcl0XMtraAa0e+PbyiRQhB2Q52vPT5taiCehe5wONcnFRU38cnWeWoN0DeKApz27ETN YAsV+WYcTg6YhkecuwrF+CJvQSKY4qGkFVEwKfA+J/fHFiWolGqbcCpvO4OmGdzJPNtCMd9vj8e TLRe2Unafzmd3V0r6+svNz8RrJt2jZAoBj4mhgJdEbBnwiqB7dpvlVJaBS/lxSe+MSPeJPpeupH vVhtDqFacdJHcPmGaTbrMR9xen6DIC2fx5c8U16iXdCdZj7YPPYjMkEjxc9rGOu7Xq+5MQ4jaBL tsoWyB5NEmtiv4KgTe X-Received: by 2002:a05:600c:c059:20b0:490:adb6:793d with SMTP id 5b1f17b1804b1-4926689fb12mr73048235e9.26.1782469056856; Fri, 26 Jun 2026 03:17:36 -0700 (PDT) X-Received: by 2002:a05:600c:c059:20b0:490:adb6:793d with SMTP id 5b1f17b1804b1-4926689fb12mr73047775e9.26.1782469056328; Fri, 26 Jun 2026 03:17:36 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: armbru@redhat.com Subject: [PATCH 3/6] json-streamer: make brace/bracket count unsigned Date: Fri, 26 Jun 2026 12:17:23 +0200 Message-ID: <20260626101727.1727389-4-pbonzini@redhat.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626101727.1727389-1-pbonzini@redhat.com> References: <20260626101727.1727389-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782469082744158501 Content-Type: text/plain; charset="utf-8" It makes no sense to let brace_count and bracket_count go negative, also because it immediately ends error recovery and sets them both back to zero. Instead set them to zero *before* choosing whether to process the token queue; this makes it possible to have the fields as unsigned. Note that JSON_END_OF_INPUT now forces the parentheses to appear balanced, so that the queue is emptied and an error is reported; hence, the "type !=3D JSON_END_OF_INPUT" condition can be removed. Signed-off-by: Paolo Bonzini Reviewed-by: Markus Armbruster --- include/qobject/json-parser.h | 4 ++-- qobject/json-streamer.c | 24 +++++++++++++++++++++--- 2 files changed, 23 insertions(+), 5 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 4c3d89f751f..0cf6932ecdc 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -31,8 +31,8 @@ typedef struct JSONMessageParser { void *opaque; JSONLexer lexer; JSONParserContext parser; - int brace_count; - int bracket_count; + unsigned int brace_count; + unsigned int bracket_count; GQueue tokens; uint64_t token_size; } JSONMessageParser; diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index 6c4f99b3e7f..9e1f650bad8 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -41,21 +41,41 @@ void json_message_process_token(JSONLexer *lexer, GStri= ng *input, parser->brace_count++; break; case JSON_RCURLY: + if (!parser->brace_count) { + goto end_error_recovery; + } parser->brace_count--; break; case JSON_LSQUARE: parser->bracket_count++; break; case JSON_RSQUARE: + if (!parser->bracket_count) { + goto end_error_recovery; + } parser->bracket_count--; break; case JSON_ERROR: error_setg(&err, "JSON parse error, stray '%s'", input->str); goto out_emit; case JSON_END_OF_INPUT: + /* + * Force the parentheses to appear balanced and the queue + * to be emptied, causing a parse error if it wasn't. + */ if (g_queue_is_empty(&parser->tokens)) { return; } + end_error_recovery: + /* + * We goto here due to receiving either JSON_ERROR or a + * JSON_R{CURLY,SQUARE}) that is known to be unbalanced. + * If in error recovery, end it immediately. If not in + * error recovery, json_parser_feed() will raise an error + * but error recovery won't be entered at all. + */ + parser->brace_count =3D 0; + parser->bracket_count =3D 0; break; default: break; @@ -83,9 +103,7 @@ void json_message_process_token(JSONLexer *lexer, GStrin= g *input, =20 g_queue_push_tail(&parser->tokens, token); =20 - if ((parser->brace_count > 0 || parser->bracket_count > 0) - && parser->brace_count >=3D 0 && parser->bracket_count >=3D 0 - && type !=3D JSON_END_OF_INPUT) { + if (parser->brace_count > 0 || parser->bracket_count > 0) { return; } =20 --=20 2.54.0 From nobody Sun Jul 26 11:51:48 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782469187; cv=none; d=zohomail.com; s=zohoarc; b=X/ttKR74UEz6lv+ptTx7rdkO07Rb61KFG/YoiY67m43GzrbKTU6IBde86DjfdvPynW7269CbfkFTlL6vZHitlmMOm2wfZ9nsNClNyvsn/XCp5sb0TJ24XpBnfuA2MVZjCWKuGttw8JRP5APic7KWqM7deo0RKyQBdYgqIU6K5wU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782469187; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=OXLdP7Z/ImFU396DY/eG/3Y3KVTSa70n4PlfMSVXj5A=; b=Pac+KN3Gih9PKxpd/OcvWUD6rgry+GtKkpiVzJxI9H6aOIAZ1VK7hdh0mSV8YGbKxG8FR3T4vgA5hb5jqp9MGwFpu6lksmP+CFpy72yJmk0sonl2KlUvZGOnnBc4elNHFXwVEI62xgK0BM6qW1+9bo1E40PyDGaTPkLQe8viFWE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782469187642687.2819454289504; Fri, 26 Jun 2026 03:19:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wd3d6-0006Fb-1N; Fri, 26 Jun 2026 06:17:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3d4-0006FK-H9 for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3d1-00054R-On for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:46 -0400 Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-556-_8ylBa-YMlS7vEz_0eF0Jg-1; Fri, 26 Jun 2026 06:17:40 -0400 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-4923411f041so8754865e9.2 for ; Fri, 26 Jun 2026 03:17:40 -0700 (PDT) Received: from [192.168.10.48] ([151.95.124.208]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49268fc0d36sm112576045e9.2.2026.06.26.03.17.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2026 03:17:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782469062; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OXLdP7Z/ImFU396DY/eG/3Y3KVTSa70n4PlfMSVXj5A=; b=gTQVmP+0tT26FqFDTLWvGuXPCqw3XfpybuRp7fZCKkfyIiMb/EoYOrmynCerc19PnsCUCJ bts5XL0Eb5hOFZAMFkZZzSgGZzP50jiRgEHfIXogc+0uRnyFyeO1dmAOPiQOcFOndrJSoS iwiNbVHl19D4SA1wSdnLDLzm2jDbTic= X-MC-Unique: _8ylBa-YMlS7vEz_0eF0Jg-1 X-Mimecast-MFC-AGG-ID: _8ylBa-YMlS7vEz_0eF0Jg_1782469060 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1782469060; x=1783073860; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OXLdP7Z/ImFU396DY/eG/3Y3KVTSa70n4PlfMSVXj5A=; b=NPZs/KFUOejvXsJHXNQ22+3Fyq51PjBGdgtnYgJNJhJisXU/+d4EHAwbhsZYBlu747 VM94wTFbT0g6ahhmmkru17jQwKfi7jHYHQNxTTCwEEZ/PQDlyVexUmfOrzHxv721WlrR T4wIjFUvKs4h+Zfd78un0wBrVijwVwouah30IkM0lKhCdXZt5mOi4zrDV86+YSyEEcqz TWFIaIKbMDpXFYP141F1N8YwgBK/dCEht+yTkoyLTPVQJj+uVA3z9CUdU1U+tsbD3/8r gYnnR06ztzDJxwAWvTRkaJL/D/CEtkKExMqVlWDi4vCZja3jJ36PCwm49V1Jg4mN2aPx QduA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782469060; x=1783073860; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=OXLdP7Z/ImFU396DY/eG/3Y3KVTSa70n4PlfMSVXj5A=; b=Ex7+J5CcyHz7mqfwvSFy3jH+OhXzKFbN3YhjrIaf0bdxP0HhdCirEngYF57g9QNySx 5F5AlKUwJ/tYsit0s4iHov3sZC+fIPzx612Px8l2HUT1/5Jhj6izmP2MxlU2qsEUDBAg xuJYbSxuBedzyWUn/3hht92PYrAlVi4tK56OgjHOABQwP08hDIZuj3/hYwdCnU4ANxU2 D66mxidpPox1HxkkwuR0nDCldPaC2qUijOLArdA1ax7ZHzl0fAveKQrtyyWLBLra1afG 9zAtMcBy/Ysj17gh4mtd9DQ1ySVVsBUwi5XzTuJAhQTlb3gEMxoOneyw9bXvorAWC1b6 5eeQ== X-Gm-Message-State: AOJu0Yy1tDtMMtVpU0GyFOIqT1TfvODtrL6ypzpLkYbVF3xohKFAP0a/ u9K5oWRPY3wgJYvB5fI5Ub9jYv/n32tHxu2M7w7nHig8wP2B2dR6W9d0Pyj+2e9vjfL3L26kR0Y Cb0QJBgMmGwO9hbFHAR+OFfycMfcCoCT4upMj8R+uLZ7O4lM6e/Ns7/Xyntn77spnYNRRwjpi6y xIXOwwO7f5HIgK+ALyTAZpkKAji6QOhQLoRnd+usTs X-Gm-Gg: AfdE7cmGiPbc3yae+HY56MSB6hTqr4HVatM8ixack9Vz136/mTI8H1VJwF5ETsqldOb LQaoKb5ZmOYcYCMRcgxYyxynO/96VrNwOy7LLabi8rkviI4OUMG2bF+fo9R+h51/wLuGdgOd7aU FxflvblIcisJw02GMuOv91JYCay5VE09tfgwapfP0FL6Vo3qemkbWHUN0e2TzDuzeJQps6dq6ab IaLxRD+/jdfpul48ep4POfxMHTKpcRiJWeMRgc2nz9XJigLuMZIc2xR3BBt1weqSFTvrN2PRCJN iNEMyGrV3FPiCdhJgeuLfBXg4wn0W+gCIu6mJwCJXw5ySN7mUoTkKn0pVI9fV8cqw24sWsO5VSk kVaaVftkdmb4AOX8fgqkbRaOmWgvry8h6PcfxTNCXVDPT/LmwdKWqqsNtbbozaeHIcUjFeWYRZ1 1KivTwdKXRYfmU9Tjf X-Received: by 2002:a05:600c:6309:b0:490:bd1d:4732 with SMTP id 5b1f17b1804b1-4926687e7a7mr82981005e9.23.1782469059534; Fri, 26 Jun 2026 03:17:39 -0700 (PDT) X-Received: by 2002:a05:600c:6309:b0:490:bd1d:4732 with SMTP id 5b1f17b1804b1-4926687e7a7mr82980425e9.23.1782469058929; Fri, 26 Jun 2026 03:17:38 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: armbru@redhat.com Subject: [PATCH 4/6] json-streamer: remove token queue Date: Fri, 26 Jun 2026 12:17:24 +0200 Message-ID: <20260626101727.1727389-5-pbonzini@redhat.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626101727.1727389-1-pbonzini@redhat.com> References: <20260626101727.1727389-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782469189416158501 Content-Type: text/plain; charset="utf-8" Now fully exploit the push parser, feeding it one token at a time without having to wait until braces and brackets are balanced. While the nesting counts are retained for error recovery purposes, the system can now report the first parsing error without waiting for parentheses to be balanced. This also means that JSON_ERROR can be handled in json-parser.c, not json-streamer.c. After reporting the error, json-streamer.c then enters an error recovery mode where subsequent errors are suppressed. This mimics the previous error reporting behavior, but it provides prompt feedback on parsing errors. As an example, here is an example interaction with qemu-ga. BEFORE (error reported only once braces are balanced): >> {"execute":foo >> } << {"error": {"class": "GenericError", "desc": "JSON parse error, invali= d keyword 'foo'"}} >> {"execute":"somecommand"} << {"error": {"class": "CommandNotFound", "desc": "The command somecomma= nd has not been found"}} AFTER (error reported immediately, but similar error recovery as before): >> {"execute":foo << {"error": {"class": "GenericError", "desc": "JSON parse error, invali= d keyword 'foo'"}} >> } >> {"execute":"somecommand"} << {"error": {"class": "CommandNotFound", "desc": "The command somecomma= nd has not been found"}} Signed-off-by: Paolo Bonzini Reviewed-by: Markus Armbruster --- include/qobject/json-parser.h | 3 +- qobject/json-parser.c | 4 ++ qobject/json-streamer.c | 106 +++++++++++++--------------------- 3 files changed, 47 insertions(+), 66 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 0cf6932ecdc..3479e637588 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -33,7 +33,8 @@ typedef struct JSONMessageParser { JSONParserContext parser; unsigned int brace_count; unsigned int bracket_count; - GQueue tokens; + unsigned int token_count; + bool error; uint64_t token_size; } JSONMessageParser; =20 diff --git a/qobject/json-parser.c b/qobject/json-parser.c index 845da3699aa..484956deae4 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -673,6 +673,10 @@ QObject *json_parser_feed(JSONParserContext *ctxt, con= st JSONToken *token, =20 assert(!ctxt->err); switch (token->type) { + case JSON_ERROR: + parse_error(ctxt, token, "stray '%s'", token->str); + break; + case JSON_END_OF_INPUT: /* Check for premature end of input */ if (!g_queue_is_empty(ctxt->stack)) { diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index 9e1f650bad8..9526f815f00 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -1,5 +1,5 @@ /* - * JSON streaming support + * JSON parser - callback interface and error recovery * * Copyright IBM, Corp. 2009 * @@ -19,23 +19,16 @@ #define MAX_TOKEN_COUNT (2ULL << 20) #define MAX_NESTING (1 << 10) =20 -static void json_message_free_tokens(JSONMessageParser *parser) -{ - JSONToken *token; - - while ((token =3D g_queue_pop_head(&parser->tokens))) { - g_free(token); - } -} - void json_message_process_token(JSONLexer *lexer, GString *input, JSONTokenType type, int x, int y) { JSONMessageParser *parser =3D container_of(lexer, JSONMessageParser, l= exer); - QObject *json =3D NULL; Error *err =3D NULL; - JSONToken *token; =20 + parser->token_size +=3D input->len; + parser->token_count++; + + /* Detect message boundaries for error recovery purposes. */ switch (type) { case JSON_LCURLY: parser->brace_count++; @@ -56,19 +49,9 @@ void json_message_process_token(JSONLexer *lexer, GStrin= g *input, parser->bracket_count--; break; case JSON_ERROR: - error_setg(&err, "JSON parse error, stray '%s'", input->str); - goto out_emit; - case JSON_END_OF_INPUT: - /* - * Force the parentheses to appear balanced and the queue - * to be emptied, causing a parse error if it wasn't. - */ - if (g_queue_is_empty(&parser->tokens)) { - return; - } end_error_recovery: /* - * We goto here due to receiving either JSON_ERROR or a + * We come here due to receiving either JSON_ERROR or a * JSON_R{CURLY,SQUARE}) that is known to be unbalanced. * If in error recovery, end it immediately. If not in * error recovery, json_parser_feed() will raise an error @@ -81,49 +64,43 @@ void json_message_process_token(JSONLexer *lexer, GStri= ng *input, break; } =20 - /* - * Security consideration, we limit total memory allocated per object - * and the maximum recursion depth that a message can force. - */ - if (parser->token_size + input->len + 1 > MAX_TOKEN_SIZE) { - error_setg(&err, "JSON token size limit exceeded"); - goto out_emit; - } - if (g_queue_get_length(&parser->tokens) + 1 > MAX_TOKEN_COUNT) { - error_setg(&err, "JSON token count limit exceeded"); - goto out_emit; - } - if (parser->bracket_count + parser->brace_count > MAX_NESTING) { - error_setg(&err, "JSON nesting depth limit exceeded"); - goto out_emit; - } + if (parser->error) { + /* error recovery, eat tokens until parentheses balance */ + } else { + /* + * Safety consideration, we limit total memory allocated per object + * and the maximum nesting depth that a message can force. + */ + if (parser->token_size > MAX_TOKEN_SIZE) { + error_setg(&err, "JSON token size limit exceeded"); + } else if (parser->token_count > MAX_TOKEN_COUNT) { + error_setg(&err, "JSON token count limit exceeded"); + } else if (parser->bracket_count + parser->brace_count > MAX_NESTI= NG) { + error_setg(&err, "JSON nesting depth limit exceeded"); + } else { + g_autofree JSONToken *token =3D json_token(type, x, y, input); + QObject *json =3D json_parser_feed(&parser->parser, token, &er= r); + if (json) { + parser->emit(parser->opaque, json, NULL); + } + } =20 - token =3D json_token(type, x, y, input); - parser->token_size +=3D input->len; - - g_queue_push_tail(&parser->tokens, token); - - if (parser->brace_count > 0 || parser->bracket_count > 0) { - return; - } - - /* Process all tokens in the queue */ - while (!g_queue_is_empty(&parser->tokens)) { - token =3D g_queue_pop_head(&parser->tokens); - json =3D json_parser_feed(&parser->parser, token, &err); - g_free(token); - if (json || err) { - break; + if (err) { + parser->emit(parser->opaque, NULL, err); + /* start recovery */ + parser->error =3D true; } } =20 -out_emit: - json_parser_reset(&parser->parser); - parser->brace_count =3D 0; - parser->bracket_count =3D 0; - json_message_free_tokens(parser); - parser->token_size =3D 0; - parser->emit(parser->opaque, json, err); + if ((parser->brace_count =3D=3D 0 && parser->bracket_count =3D=3D 0) + || type =3D=3D JSON_END_OF_INPUT) { + json_parser_reset(&parser->parser); + parser->error =3D false; + parser->brace_count =3D 0; + parser->bracket_count =3D 0; + parser->token_count =3D 0; + parser->token_size =3D 0; + } } =20 void json_message_parser_init(JSONMessageParser *parser, @@ -133,9 +110,10 @@ void json_message_parser_init(JSONMessageParser *parse= r, { parser->emit =3D emit; parser->opaque =3D opaque; + parser->error =3D false; parser->brace_count =3D 0; parser->bracket_count =3D 0; - g_queue_init(&parser->tokens); + parser->token_count =3D 0; parser->token_size =3D 0; =20 json_parser_init(&parser->parser, ap); @@ -151,12 +129,10 @@ void json_message_parser_feed(JSONMessageParser *pars= er, void json_message_parser_flush(JSONMessageParser *parser) { json_lexer_flush(&parser->lexer); - assert(g_queue_is_empty(&parser->tokens)); } =20 void json_message_parser_destroy(JSONMessageParser *parser) { json_lexer_destroy(&parser->lexer); - json_message_free_tokens(parser); json_parser_destroy(&parser->parser); } --=20 2.54.0 From nobody Sun Jul 26 11:51:48 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782469176; cv=none; d=zohomail.com; s=zohoarc; b=kAthQJEjk4enczaV/ZDt/nv/sFGDYtxApWXMi7pjmAJ3Ajv3qrpmAiYt6aEovqXS7UFFrUozsnDDNkEp7Qan1MguqaebBjObXa62n4AQmbxX5pFQy9IT+CDr8AYh5jbgyEQ+yUQu3H7QgCdmftgCYmGfV4R+vDdXNGw78X4fKMA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782469176; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=hXJQgLsRLvqft6rTlkd6uJkNVGDg/FaswUbPamdfO7g=; b=Ip6NPIuBClms/w6CYeikS553fO7qbwSJpl9qNyrLGN+l5FrK3oZVkQTHiHP6boh+wPwEOUEolzCgFQio4kyz3W8RmsdvA4meVIYSu/sOgdAIyRRK5ctjagkqdykaLZg01XPYgmQ/r8ViyYC+ZdmfjD2LfkS5aB4IMfj3a4xRL7M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782469176894863.8662654486395; Fri, 26 Jun 2026 03:19:36 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wd3d6-0006Fq-Ek; Fri, 26 Jun 2026 06:17:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3d5-0006FT-DF for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3d3-00055V-HO for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:47 -0400 Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-516-CZZesMOCNJWgeFAdBbAwAg-1; Fri, 26 Jun 2026 06:17:43 -0400 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-46dc84ca722so464997f8f.1 for ; Fri, 26 Jun 2026 03:17:42 -0700 (PDT) Received: from [192.168.10.48] ([151.95.124.208]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4926c286442sm23524675e9.1.2026.06.26.03.17.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2026 03:17:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782469065; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hXJQgLsRLvqft6rTlkd6uJkNVGDg/FaswUbPamdfO7g=; b=TwzaGWVmiTfrWEYY15bxPa5fDJFkQjfu5yG5FynzZufVQFYTDD6kXjvp+DCz6FNPACiyuL uXvbZ3skFrEYjWrxJWDy82p7AzXKtcYu6Dub0OrM/TW5Ah5+iS60HID9PaS57tqq0OMaAC a4K1tm2uEFpf+m5wLUr4lcaGZtr+IiQ= X-MC-Unique: CZZesMOCNJWgeFAdBbAwAg-1 X-Mimecast-MFC-AGG-ID: CZZesMOCNJWgeFAdBbAwAg_1782469062 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1782469062; x=1783073862; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hXJQgLsRLvqft6rTlkd6uJkNVGDg/FaswUbPamdfO7g=; b=CCDdQWMV6aDwtLcZpK3Z8LoJ/BRI/k4cYueKRktqsHdYZZItIq/kT3duOFe2UBONwK NPSYzNYWgvtFC+MjPHuoBdTi5JbjG1rphiNARiB1Pzui58b4gAoEQCoGGAmFRN+qJCVm VhSoZYiNDKbj36ZNX00Tgl4V+hzKpgwLTqU4a1N0cUAm35FxXF/z7nxkYHTZFzFZ4232 gQErNUX7LnGEQ5VlvrhzJmeE/vEbcqpaQ1YcMoG2fbmeIMUQ1QQT7Lha7mc3dzMt73Sb /xDn5wbFURKmM/PegSCULB8rjVdDXD4okzoAjiAyNPW9Ivs/WjyY45eOgUwDv/ZbXq0I 0pLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782469062; x=1783073862; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=hXJQgLsRLvqft6rTlkd6uJkNVGDg/FaswUbPamdfO7g=; b=opqLYglYtyLwDsBhPAGZL3Aw252/Tkm+MK+fbjIdeD5UrhUv+rJiQWu6a9XMziRqge ixDsN3ROdlGkKA5l4Ka8GAe/qT+GrjW2IA7L56b6CYE5iV+VzTCXFfn7WCkNbrUYi7Rf qx8KqjBLUVDIF8KMnMQpfZqagwiVFBEWp9aT+X6N2TWKYLxmTecipvbrsfFKTULS8Tof kxwkvgQM/+EyXK+ygo1LhSJ0n9ppe+B34Ur8nflg2224ZuFPM2+BGUU1X5SFowtQSB/j MUbVP2CvXocnD3O5ciiK0f/KOci24GApAym3BSQCHgduzd4sZoNgP9Vmz/pJJW7fLnPZ m54A== X-Gm-Message-State: AOJu0YwkbIReNtaFLB93xMzK56naUcZcUZYJPnsJOtoKXQiUX3/4V9I+ Pnu26+L9ehpNGtUXNjA9AXf3a/15btVgv3pSs6LCOPSWNwI5/TL6R7aXYY2oVhXqb6A7YeKzaAU Q3CwEO/ktDrpf4vHfUyN4DCPjJTziTAiPu4u4iSkyCyu53YN7dyTkwZNBnZlZgJsIArdJfisPP6 4j1QDaryeGwLB/5IgqLvBXFv0x/IdhhNn1dg85SKlQ X-Gm-Gg: AfdE7cmwBdjJ2uRyrQB54g0i4nMjb2kcr7yurFfR4wauMezHFTNQmsCjtz1NxEN4qXt KgG15uYc8wdkv04PexeiyIVYuFL1UqmXT4TQYQt1uvXZeoqwDh/bYY/0e1ZxEVyxnhaJw805jCX Eu8ReGFbCZi+8OGiomq/W80H/4sUja1CEoQMcBo9pPbPdwmwJUSGD3ckz0RLbPqQhA/9n5WrqDT Rb2WsNDsv1CEN/I3EYOMxmAF88Mc96Gd7pjXFj9mnYBR5TNklxNYP/14e0u57crOmFh9rjKWo6N jyPIMVK8fwu+m8DgHDuyqh9NYfnt51xGcwvPap0ztoGSxc5jkYdo8a0HRySm1kbbDGQEdfuQbrx luK0M/F66jmHRK1IJ6O18jakND/FDhoMLKfFjpXOOrk6u2uLSWU8kkVdO4DNQDlC6I5a3JC6b0S 72PwnaTs9z7o1HHyGW X-Received: by 2002:a05:600d:644e:20b0:492:3fc4:bd6b with SMTP id 5b1f17b1804b1-49266862939mr69554505e9.7.1782469061844; Fri, 26 Jun 2026 03:17:41 -0700 (PDT) X-Received: by 2002:a05:600d:644e:20b0:492:3fc4:bd6b with SMTP id 5b1f17b1804b1-49266862939mr69554125e9.7.1782469061317; Fri, 26 Jun 2026 03:17:41 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: armbru@redhat.com Subject: [PATCH 5/6] json-streamer: do not heap-allocate JSONToken Date: Fri, 26 Jun 2026 12:17:25 +0200 Message-ID: <20260626101727.1727389-6-pbonzini@redhat.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626101727.1727389-1-pbonzini@redhat.com> References: <20260626101727.1727389-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782469177273158500 Content-Type: text/plain; charset="utf-8" This is not needed with a push parser. Since it processes tokens immediately, the JSONToken can be created directly on the stack and does not need to copy the lexer's string data. Reviewed-by: Markus Armbruster Signed-off-by: Paolo Bonzini --- qobject/json-parser-int.h | 8 ++++++-- qobject/json-parser.c | 18 ------------------ qobject/json-streamer.c | 9 +++++++-- 3 files changed, 13 insertions(+), 22 deletions(-) diff --git a/qobject/json-parser-int.h b/qobject/json-parser-int.h index 1f435cb8eb2..5a6b5c9af90 100644 --- a/qobject/json-parser-int.h +++ b/qobject/json-parser-int.h @@ -35,7 +35,12 @@ typedef enum json_token_type { JSON_MAX =3D JSON_END_OF_INPUT } JSONTokenType; =20 -typedef struct JSONToken JSONToken; +typedef struct JSONToken { + JSONTokenType type; + int x; + int y; + char *str; +} JSONToken; =20 /* json-lexer.c */ void json_lexer_init(JSONLexer *lexer, bool enable_interpolation); @@ -48,7 +53,6 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, JSONTokenType type, int x, int y); =20 /* json-parser.c */ -JSONToken *json_token(JSONTokenType type, int x, int y, GString *tokstr); void json_parser_init(JSONParserContext *ctxt, va_list *ap); void json_parser_reset(JSONParserContext *ctxt); QObject *json_parser_feed(JSONParserContext *ctxt, const JSONToken *token,= Error **errp); diff --git a/qobject/json-parser.c b/qobject/json-parser.c index 484956deae4..4a3f5866129 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -24,13 +24,6 @@ #include "qobject/qstring.h" #include "json-parser-int.h" =20 -struct JSONToken { - JSONTokenType type; - int x; - int y; - char str[]; -}; - /* * The JSON parser is a push parser, returning to the caller after every * token. Therefore it has an explicit representation of its parser @@ -623,17 +616,6 @@ static QObject *parse_token(JSONParserContext *ctxt, c= onst JSONToken *token) return NULL; } =20 -JSONToken *json_token(JSONTokenType type, int x, int y, GString *tokstr) -{ - JSONToken *token =3D g_malloc(sizeof(JSONToken) + tokstr->len + 1); - - token->type =3D type; - memcpy(token->str, tokstr->str, tokstr->len); - token->str[tokstr->len] =3D 0; - token->x =3D x; - token->y =3D y; - return token; -} =20 void json_parser_reset(JSONParserContext *ctxt) { diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index 9526f815f00..6d7c947f94a 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -78,8 +78,13 @@ void json_message_process_token(JSONLexer *lexer, GStrin= g *input, } else if (parser->bracket_count + parser->brace_count > MAX_NESTI= NG) { error_setg(&err, "JSON nesting depth limit exceeded"); } else { - g_autofree JSONToken *token =3D json_token(type, x, y, input); - QObject *json =3D json_parser_feed(&parser->parser, token, &er= r); + JSONToken token =3D (JSONToken) { + .type =3D type, + .x =3D x, + .y =3D y, + .str =3D input->str + }; + QObject *json =3D json_parser_feed(&parser->parser, &token, &e= rr); if (json) { parser->emit(parser->opaque, json, NULL); } --=20 2.54.0 From nobody Sun Jul 26 11:51:48 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782469126; cv=none; d=zohomail.com; s=zohoarc; b=A+5RMt3k0JuCYLOzbsPkgon7yyhmGxUFgEVCR4We3nkgWJybpBzXNpzD73SSTy5LbIBMsEaCpDxrxZ+pEqRY0VT8QDhH4trQ5GZmZOfqckcPRhl5dERzoJimeD0/AHlEnzzivTSt+GOqDkyIClCUoDbFo1s/vJopHbD8wuvtgVY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782469126; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=b6g1WIFxg79eY4POLrBwb5BZYFE2C39RrZvOqTPAeZM=; b=GfeNuYTPvokEW6H2UNQTTTAa7MDL+wRiSmxU1yjiEjoQlbJr3UlcFf7x+edTgjEjNcaxVM79syQbFOdTNKLu/EouqlACNJ9nFId8RNAq/l0Fv+JDfJgKgMy5TzerAFBcnkpPTSgK+mjkTauMvkWW8InLaUAZG3LaeSLC6JorH5A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782469126242499.08817196282655; Fri, 26 Jun 2026 03:18:46 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wd3dE-0006RE-BS; Fri, 26 Jun 2026 06:17:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3d9-0006Pf-Bs for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wd3d7-00055v-Lu for qemu-devel@nongnu.org; Fri, 26 Jun 2026 06:17:51 -0400 Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-662-dPNPG5LrNuiOGC18sGfuCg-1; Fri, 26 Jun 2026 06:17:47 -0400 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-4625e71d3ccso1428191f8f.0 for ; Fri, 26 Jun 2026 03:17:47 -0700 (PDT) Received: from [192.168.10.48] ([151.95.124.208]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49269016362sm63345335e9.13.2026.06.26.03.17.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2026 03:17:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782469069; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b6g1WIFxg79eY4POLrBwb5BZYFE2C39RrZvOqTPAeZM=; b=VG2kg/l+NaaTRz2CuJW1UirgJl0gjQpE0bZxU4BQChJCfp3hGlOpvA0PEmjbO2R/eYIsbv kvd3NpOorhwLoAJ8eo/tgs4zdMDtdJYC1W3I+VTuYfjoeWh9fFXJ9NuvDfVxZVvtyH9k1S jZz5c2Vt+K/sdKToc0TnMhGUoqvOTIw= X-MC-Unique: dPNPG5LrNuiOGC18sGfuCg-1 X-Mimecast-MFC-AGG-ID: dPNPG5LrNuiOGC18sGfuCg_1782469067 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1782469066; x=1783073866; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b6g1WIFxg79eY4POLrBwb5BZYFE2C39RrZvOqTPAeZM=; b=kIplujsWyWAQNxLfi8U71LFD0fwG/xSQGd/3irbe1Ie6HfYQ+gbhWrQjQYSwra7lJp iWej993cxezQxIQ6KdvRq4MJKI6Na+abIzxk+VNC0ttWnxPyhtILrTPMwtx38y9nqT9g e7ciLWTVpEWkKi3YwZqRPs4VvUQf5+Wj2WtobbnzD7ira4YJ+tk/tXA0LExqXmC3Vhhx bZs/ycqBDke+ebOk3G55QOtApz5gLhSioCZU84XdEmSzrJLWknJ0KPB78L9wQvnZcQ9b Rzmg9UAeUJ+zTWs5cvnfK/SlmIRqkw+6ZxtlSjafGVIRd6Yb6VScQLvBNlCabRZogOmp xcGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782469066; x=1783073866; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=b6g1WIFxg79eY4POLrBwb5BZYFE2C39RrZvOqTPAeZM=; b=o3Ho+iftNR0SXN4xGbFGKbRLLKibtdbEiX3B0upcNibXWdTV6fZe7ivGEVlNuciKv9 DoQEfTfiKDfxYMrOEoXbmXJtaRnmVfJ1YYOsJBT/qPbMSdlqqmAaZJ3HfBeTsi0vOpyX aq8xwn4hvfR0fee/RT1t2mzdBVjV0x3Sosg7jnlM4Z19c1QHrB3xE+duqATKOdwcWWSs KbLl3mV5m0NkSqW7kJjVvrDoI55HZKNoWixQhQkNzhg8fvQsT1ynkMq39tYRk5Q6jZNv 5n0kuOMJJKHqVxM99BWUExLGABCRpkcIK0aneL4dEVKDZa02dDuQmHxalK1sT6rioJdw zhQA== X-Gm-Message-State: AOJu0YyUK8rfhDRplAvabu5rvK76DosRcYV2D5YAfaXo76A/TlR4LkHE 8EwI75lLATqK/ZTL7msaOe46U7Z6QG3Z/7FFT4+gqW7bhegJ7Pr6NUDzHhtMjdHGxOkK+SwYpCo l3LjZOL13jPGQndN38owcv+tyWWNae5Rt31hSO1chNdH4lfArDlPTa5wDUVgCm+6Z0CXCGV50jb 4j0vLACU9O9SdJsoCRMP2gQyDJyCQEzg9tKkO2SbA/ X-Gm-Gg: AfdE7cl/ycW0GekhRA9tbXZtRiNFUWQaUR5YNkz9QoM1RpPADBXedVAUern8rT2Kdv/ cX8sRfnRsvieefzB5CS63XgwcjJDz1B29XPLq7QtXs0bcd1matfRzH40XUVFWikL6Ttik8TAW6b kAZBhyC3Zp6GIdfZNYhf4uw4HRN8s3B3lIcvBeYeUmx3JFSprTxiJVZdcG0686b7phDnl+J1lgR 6raT+XyX+jr9fJWu4RGIFaAdtiMed+jnxj6MCFD0J9pYwaKI/8/MyOnhwnmDIqCFDonykVysNG2 flyh5a2cd4GHDMF0u/vqQNKSAnJORTnijO8w+gqAaoINCqjRyXjUWeIP0u+D3ZFN5Exc9rO/1lO XT2DeZY90+mVkQlKssYOtPO83SAZfysES0TSGoFnyLjP0i44KVQdrmAGoDy+fV+AhwgZF31kGM/ aMMyeP1QCPqsQBphUU X-Received: by 2002:a05:600c:524a:b0:490:bbc1:c9be with SMTP id 5b1f17b1804b1-49266192a51mr76953315e9.0.1782469066465; Fri, 26 Jun 2026 03:17:46 -0700 (PDT) X-Received: by 2002:a05:600c:524a:b0:490:bbc1:c9be with SMTP id 5b1f17b1804b1-49266192a51mr76952935e9.0.1782469065771; Fri, 26 Jun 2026 03:17:45 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: armbru@redhat.com Subject: [PATCH 6/6] json-parser: add location to JSON parsing errors Date: Fri, 26 Jun 2026 12:17:26 +0200 Message-ID: <20260626101727.1727389-7-pbonzini@redhat.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626101727.1727389-1-pbonzini@redhat.com> References: <20260626101727.1727389-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782469127431158500 Content-Type: text/plain; charset="utf-8" Now that all calls to parse_error have a token, add the line and column to the message. As far as I can see the two important TODOs (better errors and better EOI handling) are done, and the others (token range information and "parsed size"?) do not really matter or are handled better by json-streamer.c. So remove the list, which had sat unchanged since 2009. This needs some adjustments to provide a good x and y for error messages. First of all, they switch from zero-based to one-based, which is safe because they were both sitting unused. Second, right now the x and y are those of the *last* character in the token. Modify json-lexer.c to freeze tok->x and tok->y at the first character added to the GString. Signed-off-by: Paolo Bonzini Reviewed-by: Markus Armbruster --- include/qobject/json-parser.h | 1 + qobject/json-lexer.c | 11 +++++++---- qobject/json-parser.c | 12 ++---------- 3 files changed, 10 insertions(+), 14 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 3479e637588..e078b36b2d5 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -17,6 +17,7 @@ typedef struct JSONLexer { int start_state, state; GString *token; + int cur_x, cur_y; int x, y; } JSONLexer; =20 diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c index 51341d96e49..7753ba6c092 100644 --- a/qobject/json-lexer.c +++ b/qobject/json-lexer.c @@ -277,7 +277,8 @@ void json_lexer_init(JSONLexer *lexer, bool enable_inte= rpolation) lexer->start_state =3D lexer->state =3D enable_interpolation ? IN_START_INTERP : IN_START; lexer->token =3D g_string_sized_new(3); - lexer->x =3D lexer->y =3D 0; + lexer->cur_x =3D lexer->cur_y =3D 1; + lexer->x =3D lexer->y =3D 1; } =20 static void json_lexer_feed_char(JSONLexer *lexer, char ch, bool flush) @@ -285,10 +286,10 @@ static void json_lexer_feed_char(JSONLexer *lexer, ch= ar ch, bool flush) int new_state; bool char_consumed =3D false; =20 - lexer->x++; + lexer->cur_x++; if (ch =3D=3D '\n') { - lexer->x =3D 0; - lexer->y++; + lexer->cur_x =3D 1; + lexer->cur_y++; } =20 while (flush ? lexer->state !=3D lexer->start_state : !char_consumed) { @@ -316,6 +317,8 @@ static void json_lexer_feed_char(JSONLexer *lexer, char= ch, bool flush) case IN_START: g_string_truncate(lexer->token, 0); new_state =3D lexer->start_state; + lexer->x =3D lexer->cur_x; + lexer->y =3D lexer->cur_y; break; case JSON_ERROR: json_message_process_token(lexer, lexer->token, JSON_ERROR, diff --git a/qobject/json-parser.c b/qobject/json-parser.c index 4a3f5866129..a188d58d006 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -132,15 +132,6 @@ typedef struct JSONParserStackEntry { =20 #define BUG_ON(cond) assert(!(cond)) =20 -/** - * TODO - * - * 0) make errors meaningful again - * 1) add geometry information to tokens - * 3) should we return a parsed size? - * 4) deal with premature EOI - */ - static inline JSONParserStackEntry *current_entry(JSONParserContext *ctxt) { return g_queue_peek_tail(ctxt->stack); @@ -179,7 +170,8 @@ static void G_GNUC_PRINTF(3, 4) parse_error(JSONParserC= ontext *ctxt, va_start(ap, msg); vsnprintf(message, sizeof(message), msg, ap); va_end(ap); - error_setg(&ctxt->err, "JSON parse error, %s", message); + error_setg(&ctxt->err, "%d:%d: JSON parse error, %s", + token->y, token->x, message); } =20 static int cvt4hex(const char *s) --=20 2.54.0