From nobody Sun Jul 26 11:03:38 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782996024; cv=none; d=zohomail.com; s=zohoarc; b=A5yN6DBGvr2BavsANfldEihMHiKfPtHwJBmx50woFNgN0OoWy2XFDWjHllRFLT6lLH0AZYNu8Bct8vpHsaOddaScq9Lldl/L48kpXMi8NxBy+QY4a6MYg83+MfFiGkrveykGLRlY1bYW03qg65Sz2yyVbv246SPN2/3Nfsrrgnc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782996024; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=eXirk1eW7vDiQ6qUhFzNl6PwXc18LGY6D23talZ7Ckw=; b=GohJSROmrAdPNs/CNo0UkrB0YTvqg1eycbA/uug8KhB/+Wk0N8aWFtMYAhgng+k2DA2IqsdGbmEdZHZmi9+G92qHKQMENOaWPZfzGx4eNKVQ6acH0ueGo9VyvMvCtfYSa+L25a/qV2ccr2jBcaey7A9VwaIB7MQIi8Wjgk1W0ts= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17829960248158.890086038202412; Thu, 2 Jul 2026 05:40:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wfGiE-0004fa-Pv; Thu, 02 Jul 2026 08:40:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi6-0004cY-W6 for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi3-0000bH-I5 for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:05 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-97-ikZia_W8OuSxuzeRT_Q8YQ-1; Thu, 02 Jul 2026 08:39:59 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3B3981944CED; Thu, 2 Jul 2026 12:39:58 +0000 (UTC) Received: from blackfin.pond.sub.org (unknown [10.44.22.4]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 998661971E71; Thu, 2 Jul 2026 12:39:57 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 02F1A21E692F; Thu, 02 Jul 2026 14:39:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782996002; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eXirk1eW7vDiQ6qUhFzNl6PwXc18LGY6D23talZ7Ckw=; b=c4da/zQfu2ZDAoueOnD3VVQhGk+Z8Aphss0IJb2ThShKIl4rHa9ulOBbz/JFsrF0jzVjiC vYp9p7vWpN7wu5tQ1C/uRk7nSmkkO0+QECCY+PaDtldTVT+nAYB98/gFC1AYgoTI0Xs/Yg 5PMqK6zJG8vVt/KHS36c9KT2R2cmlrk= X-MC-Unique: ikZia_W8OuSxuzeRT_Q8YQ-1 X-Mimecast-MFC-AGG-ID: ikZia_W8OuSxuzeRT_Q8YQ_1782995998 From: Markus Armbruster To: qemu-devel@nongnu.org Cc: stefanha@redhat.com, Bin Guo , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL 1/7] qobject/json-writer: preallocate output buffer Date: Thu, 2 Jul 2026 14:39:48 +0200 Message-ID: <20260702123954.1284221-2-armbru@redhat.com> In-Reply-To: <20260702123954.1284221-1-armbru@redhat.com> References: <20260702123954.1284221-1-armbru@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=armbru@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782996026863158500 From: Bin Guo json_writer_new() creates the output GString with g_string_new(NULL), which starts at the GLib default of 64 bytes. Serializing typical QMP responses then requires multiple reallocations as the buffer grows -- for query-qmp-schema the GString is reallocated 12+ times. Preallocate JSON_WRITER_INITIAL_SIZE (4096) bytes. This covers most QMP responses without any reallocation. The JSONWriter is a short-lived object so the preallocation does not accumulate. Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Markus Armbruster Signed-off-by: Bin Guo Message-ID: <20260603022538.92780-1-guobin@linux.alibaba.com> Signed-off-by: Markus Armbruster --- qobject/json-writer.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/qobject/json-writer.c b/qobject/json-writer.c index aac2c6ab71..c23c81709b 100644 --- a/qobject/json-writer.c +++ b/qobject/json-writer.c @@ -24,13 +24,16 @@ struct JSONWriter { GByteArray *container_is_array; }; =20 +/* Should cover most QMP responses without reallocation */ +#define JSON_WRITER_INITIAL_SIZE 4096 + JSONWriter *json_writer_new(bool pretty) { JSONWriter *writer =3D g_new(JSONWriter, 1); =20 writer->pretty =3D pretty; writer->need_comma =3D false; - writer->contents =3D g_string_new(NULL); + writer->contents =3D g_string_sized_new(JSON_WRITER_INITIAL_SIZE); writer->container_is_array =3D g_byte_array_new(); return writer; } --=20 2.54.0 From nobody Sun Jul 26 11:03:38 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782996027; cv=none; d=zohomail.com; s=zohoarc; b=JOlDYwVpuBtTM0gq54D/bmyF8QloP9l+rpxMLT43OfstcDgSWX+cuEkmOsk0oHD7G9xC0tvgmt1IB8XaMS55vvHwXngfVYKwwIXZDSn+JYcT/wL/TnxnpfIxy8rAfBr0h1qGzpPDSPs6ILuHsHPW/O7iuL49N9quQqylDpfCr8E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782996027; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9V4bJjBDJ8487kyP/8ZTK8RQxOnnadaVdyX7xF2mxos=; b=nqcWrud129ddud7lEWOj23nYzcp0posfrXAc80MDk4jc1pxDUx1sMMtT2oRPFGLgwSTMet80JwHmR2jSbti0DY25YW4zSgEuYGgYtSAvvKY0SZnNslQF0IRxBnx8jpkwhKnxDqVzWJLb3Gpup2vFlb5J6nmo+KqbJkjBbkSFJK8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782996027253739.4513121450722; Thu, 2 Jul 2026 05:40:27 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wfGiD-0004fY-Na; Thu, 02 Jul 2026 08:40:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi9-0004d3-01 for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi1-0000ZE-C0 for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:07 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-492-L_tT6c0DOIqmD27DJYVjKQ-1; Thu, 02 Jul 2026 08:39:58 -0400 Received: from mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.95]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F335F1944B36 for ; Thu, 2 Jul 2026 12:39:57 +0000 (UTC) Received: from blackfin.pond.sub.org (unknown [10.44.22.4]) by mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7E0FF36F0B for ; Thu, 2 Jul 2026 12:39:57 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 0798021E6931; Thu, 02 Jul 2026 14:39:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782996000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9V4bJjBDJ8487kyP/8ZTK8RQxOnnadaVdyX7xF2mxos=; b=EOB4rSGv2lj9KsUxhbx2sB34CG6snWamsm16TCwcvOP5eLnrM55McG1t6v+e19FuQhie0M ZeFpOtoT075KIFEtCC9aFvr2RRYIVOGCoDow58J1iNTNGTD0WhNMQ0pQs07KxvvZNz+9e3 IjhkYaq9PPTcuja69xzsDw/4LHrJ6z0= X-MC-Unique: L_tT6c0DOIqmD27DJYVjKQ-1 X-Mimecast-MFC-AGG-ID: L_tT6c0DOIqmD27DJYVjKQ_1782995998 From: Markus Armbruster To: qemu-devel@nongnu.org Cc: stefanha@redhat.com, Paolo Bonzini Subject: [PULL 2/7] json-parser: replace with a push parser Date: Thu, 2 Jul 2026 14:39:49 +0200 Message-ID: <20260702123954.1284221-3-armbru@redhat.com> In-Reply-To: <20260702123954.1284221-1-armbru@redhat.com> References: <20260702123954.1284221-1-armbru@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.6 on 10.30.177.95 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=armbru@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782996028919158500 From: Paolo Bonzini In order to avoid stashing all the tokens corresponding to a JSON value, embed the parsing stack and state machine in JSONParser. This is more efficient and allows for more prompt error recovery; it also does not make the code substantially larger than the current recursive descent parser, though the state machine is probably a bit harder to follow. The stack consists of QLists and QDicts corresponding to open brackets and braces, plus optionally a QString with the current key on top of each QDict. After each value is parsed, it is added to the top array or dictionary or, if the stack is empty, json_parser_feed returns the complete QObject. For now, json-streamer.c keeps tracking the tokens up until braces and brackets are balanced, and then shoves the whole queue of tokens into the push parser. The only logic change is that JSON_END_OF_INPUT always triggers the emptying of the queue; the parser takes notice and checks that there is nothing on the stack. Not using brace_count and bracket_count for this is the first step towards improved separation of concerns between json-parser.c and json-streamer.c. Signed-off-by: Paolo Bonzini Message-ID: <20260626101727.1727389-2-pbonzini@redhat.com> Reviewed-by: Markus Armbruster [Minor comment improvements] Signed-off-by: Markus Armbruster --- include/qobject/json-parser.h | 6 + qobject/json-parser-int.h | 5 +- qobject/json-parser.c | 567 ++++++++++++++++++++-------------- qobject/json-streamer.c | 21 +- 4 files changed, 361 insertions(+), 238 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 7345a9bd5c..05346fa816 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -20,6 +20,12 @@ typedef struct JSONLexer { int x, y; } JSONLexer; =20 +typedef struct JSONParserContext { + Error *err; + GQueue *stack; + va_list *ap; +} JSONParserContext; + typedef struct JSONMessageParser { void (*emit)(void *opaque, QObject *json, Error *err); void *opaque; diff --git a/qobject/json-parser-int.h b/qobject/json-parser-int.h index 8c01f23627..1f435cb8eb 100644 --- a/qobject/json-parser-int.h +++ b/qobject/json-parser-int.h @@ -49,6 +49,9 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, =20 /* json-parser.c */ JSONToken *json_token(JSONTokenType type, int x, int y, GString *tokstr); -QObject *json_parser_parse(GQueue *tokens, va_list *ap, Error **errp); +void json_parser_init(JSONParserContext *ctxt, va_list *ap); +void json_parser_reset(JSONParserContext *ctxt); +QObject *json_parser_feed(JSONParserContext *ctxt, const JSONToken *token,= Error **errp); +void json_parser_destroy(JSONParserContext *ctxt); =20 #endif diff --git a/qobject/json-parser.c b/qobject/json-parser.c index f6622b82b0..5ed1aa1c8d 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -31,12 +31,112 @@ struct JSONToken { char str[]; }; =20 -typedef struct JSONParserContext { - Error *err; - JSONToken *current; - GQueue *buf; - va_list *ap; -} JSONParserContext; +/* + * The JSON parser is a push parser, returning a completed top-level + * object, an error, or NULL (if the object is incomplete and no error + * happened) after every token. Therefore it has an explicit + * representation of its parser stack; each stack entry consists of a + * parser state and a QObject: - a QList, for an array that is being + * added to - a QDict, for a dictionary that is being added to - a + * QString, for the key of the next pair that will be added to a QDict + * + * The stack represents an arbitrary nesting of arrays and dictionaries + * (whose next key has been parsed); it can also have a dictionary whose + * next key has not been parsed, but that can only happen at the top level. + * Because of this, the stack contents are always of the form + * "(QList | QDict QString)* QDict?". + * + * An empty stack represents the beginning of the parsing process, with + * start state BEFORE_VALUE. + */ + +typedef enum JSONParserState { + AFTER_LCURLY, + AFTER_LSQUARE, + BEFORE_KEY, + BEFORE_VALUE, + END_OF_KEY, + END_OF_VALUE, +} JSONParserState; + +typedef struct JSONParserStackEntry { + /* + * State when the container is completed or, for the top of the stack, + * entry state for the next token. + */ + JSONParserState state; + + /* + * A QString with the last parsed key, or a QList/QDict for the current + * container. + */ + QObject *partial; +} JSONParserStackEntry; + +/* + * This is the JSON grammar that's parsed, with the state transition and + * action at each point of the grammar. While this is not a formal + * description, "-> action" represents the pseudocode of the action + * and "-> STATE" sets the top stack entry's state to STATE. + * + * The state alone is enough to tell you what to parse; the state plus + * the type of the top of stack tells you which action to take. + * + * // The initial state is BEFORE_VALUE. + * input :=3D value -> END_OF_VALUE -> return parsed value + * (input | END_OF_INPUT) + * + * // entered on BEFORE_VALUE; after any of these rules are processed, the + * // parser has completed a QObject and is in the END_OF_VALUE state. + * // + * // When the parser reaches the END_OF_VALUE state, it examines the + * // top of the stack to see if it's coming from "input" (stack empty), + * // "array_items" (TOS is a QList) or "dict_pairs" (TOS is a QString; the + * // item below will be a QDict). It then proceeds with the corresponding + * // actions, which will be one of: + * // - return parsed value + * // - add value to QList + * // - pop QString with the key, add key/value to the QDict + * value :=3D literal -> END_OF_VALUE + * | '[' -> push empty QList -> AFTER_LSQUARE + * after_lsquare -> END_OF_VALUE + * | '{' -> push empty QDict -> AFTER_LCURLY + * after_lcurly -> END_OF_VALUE + * + * // non-recursive values, entered on BEFORE_VALUE + * literal :=3D INTEGER -> END_OF_VALUE + * | FLOAT -> END_OF_VALUE + * | KEYWORD -> END_OF_VALUE + * | STRING -> END_OF_VALUE + * | INTERP -> END_OF_VALUE + * + * // entered on AFTER_LSQUARE + * after_lsquare :=3D ']' -> pop completed QList -> END_OF_VALUE + * | =CF=B5 -> BEFORE_VALUE + * array_items -> END_OF_VALUE + * + * // entered on BEFORE_VALUE, with TOS being a QList + * array_items :=3D value -> add value to QList -> END_OF_VALUE + * (']' -> pop completed QList -> END_OF_VALUE + * | ',' -> BEFORE_VALUE + * array_items) -> END_OF_VALUE + * + * // entered on AFTER_LCURLY + * after_lcurly :=3D '}' -> pop completed QDict -> END_OF_VALUE + * | =CF=B5 -> BEFORE_KEY + * dict_pairs -> END_OF_VALUE + * + * // entered on BEFORE_KEY, with TOS being a QDict + * dict_pairs :=3D (STRING | INTERP) -> push QString -> END_OF_KEY + * ':' -> BEFORE_VALUE + * value -> pop QString + add pair to QDict -> END_OF_VA= LUE + * ('}' -> pop completed QDict -> END_OF_VALUE + * | ',' -> BEFORE_KEY + * dict_pairs) -> END_OF_VALUE + * + * Parse errors ignore the token. json_parser_reset() can be + * called to restart parsing from scratch, with an empty stack. + */ =20 #define BUG_ON(cond) assert(!(cond)) =20 @@ -49,7 +149,27 @@ typedef struct JSONParserContext { * 4) deal with premature EOI */ =20 -static QObject *parse_value(JSONParserContext *ctxt); +static inline JSONParserStackEntry *current_entry(JSONParserContext *ctxt) +{ + return g_queue_peek_tail(ctxt->stack); +} + +static void push_entry(JSONParserContext *ctxt, QObject *partial, + JSONParserState state) +{ + JSONParserStackEntry *entry =3D g_new(JSONParserStackEntry, 1); + entry->partial =3D partial; + entry->state =3D state; + g_queue_push_tail(ctxt->stack, entry); +} + +/* Drop the top entry and return the new top entry. */ +static JSONParserStackEntry *pop_entry(JSONParserContext *ctxt) +{ + JSONParserStackEntry *entry =3D g_queue_pop_tail(ctxt->stack); + g_free(entry); + return current_entry(ctxt); +} =20 /** * Error handler @@ -236,200 +356,10 @@ out: return NULL; } =20 -/* Note: the token object returned by parser_context_peek_token or - * parser_context_pop_token is deleted as soon as parser_context_pop_token - * is called again. - */ -static const JSONToken *parser_context_pop_token(JSONParserContext *ctxt) +/* Terminals */ + +static QObject *parse_keyword(JSONParserContext *ctxt, const JSONToken *to= ken) { - g_free(ctxt->current); - ctxt->current =3D g_queue_pop_head(ctxt->buf); - return ctxt->current; -} - -static const JSONToken *parser_context_peek_token(JSONParserContext *ctxt) -{ - return g_queue_peek_head(ctxt->buf); -} - -/** - * Parsing rules - */ -static int parse_pair(JSONParserContext *ctxt, QDict *dict) -{ - QObject *key_obj =3D NULL; - QString *key; - QObject *value; - const JSONToken *peek, *token; - - peek =3D parser_context_peek_token(ctxt); - if (peek =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - key_obj =3D parse_value(ctxt); - key =3D qobject_to(QString, key_obj); - if (!key) { - parse_error(ctxt, peek, "key is not a string in object"); - goto out; - } - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - if (token->type !=3D JSON_COLON) { - parse_error(ctxt, token, "missing : in object pair"); - goto out; - } - - value =3D parse_value(ctxt); - if (value =3D=3D NULL) { - parse_error(ctxt, token, "Missing value in dict"); - goto out; - } - - if (qdict_haskey(dict, qstring_get_str(key))) { - parse_error(ctxt, token, "duplicate key"); - goto out; - } - - qdict_put_obj(dict, qstring_get_str(key), value); - - qobject_unref(key_obj); - return 0; - -out: - qobject_unref(key_obj); - return -1; -} - -static QObject *parse_object(JSONParserContext *ctxt) -{ - QDict *dict =3D NULL; - const JSONToken *token, *peek; - - token =3D parser_context_pop_token(ctxt); - assert(token && token->type =3D=3D JSON_LCURLY); - - dict =3D qdict_new(); - - peek =3D parser_context_peek_token(ctxt); - if (peek =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - if (peek->type !=3D JSON_RCURLY) { - if (parse_pair(ctxt, dict) =3D=3D -1) { - goto out; - } - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - while (token->type !=3D JSON_RCURLY) { - if (token->type !=3D JSON_COMMA) { - parse_error(ctxt, token, "expected separator in dict"); - goto out; - } - - if (parse_pair(ctxt, dict) =3D=3D -1) { - goto out; - } - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - } - } else { - (void)parser_context_pop_token(ctxt); - } - - return QOBJECT(dict); - -out: - qobject_unref(dict); - return NULL; -} - -static QObject *parse_array(JSONParserContext *ctxt) -{ - QList *list =3D NULL; - const JSONToken *token, *peek; - - token =3D parser_context_pop_token(ctxt); - assert(token && token->type =3D=3D JSON_LSQUARE); - - list =3D qlist_new(); - - peek =3D parser_context_peek_token(ctxt); - if (peek =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - if (peek->type !=3D JSON_RSQUARE) { - QObject *obj; - - obj =3D parse_value(ctxt); - if (obj =3D=3D NULL) { - parse_error(ctxt, token, "expecting value"); - goto out; - } - - qlist_append_obj(list, obj); - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - - while (token->type !=3D JSON_RSQUARE) { - if (token->type !=3D JSON_COMMA) { - parse_error(ctxt, token, "expected separator in list"); - goto out; - } - - obj =3D parse_value(ctxt); - if (obj =3D=3D NULL) { - parse_error(ctxt, token, "expecting value"); - goto out; - } - - qlist_append_obj(list, obj); - - token =3D parser_context_pop_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - goto out; - } - } - } else { - (void)parser_context_pop_token(ctxt); - } - - return QOBJECT(list); - -out: - qobject_unref(list); - return NULL; -} - -static QObject *parse_keyword(JSONParserContext *ctxt) -{ - const JSONToken *token; - - token =3D parser_context_pop_token(ctxt); assert(token && token->type =3D=3D JSON_KEYWORD); =20 if (!strcmp(token->str, "true")) { @@ -443,11 +373,9 @@ static QObject *parse_keyword(JSONParserContext *ctxt) return NULL; } =20 -static QObject *parse_interpolation(JSONParserContext *ctxt) +static QObject *parse_interpolation(JSONParserContext *ctxt, + const JSONToken *token) { - const JSONToken *token; - - token =3D parser_context_pop_token(ctxt); assert(token && token->type =3D=3D JSON_INTERP); =20 if (!strcmp(token->str, "%p")) { @@ -479,11 +407,8 @@ static QObject *parse_interpolation(JSONParserContext = *ctxt) return NULL; } =20 -static QObject *parse_literal(JSONParserContext *ctxt) +static QObject *parse_literal(JSONParserContext *ctxt, const JSONToken *to= ken) { - const JSONToken *token; - - token =3D parser_context_pop_token(ctxt); assert(token); =20 switch (token->type) { @@ -531,35 +456,174 @@ static QObject *parse_literal(JSONParserContext *ctx= t) } } =20 -static QObject *parse_value(JSONParserContext *ctxt) +/* Parsing state machine */ + +static QObject *parse_begin_value(JSONParserContext *ctxt, + const JSONToken *token) { - const JSONToken *token; - - token =3D parser_context_peek_token(ctxt); - if (token =3D=3D NULL) { - parse_error(ctxt, NULL, "premature EOI"); - return NULL; - } - switch (token->type) { case JSON_LCURLY: - return parse_object(ctxt); + push_entry(ctxt, QOBJECT(qdict_new()), AFTER_LCURLY); + return NULL; case JSON_LSQUARE: - return parse_array(ctxt); + push_entry(ctxt, QOBJECT(qlist_new()), AFTER_LSQUARE); + return NULL; case JSON_INTERP: - return parse_interpolation(ctxt); + return parse_interpolation(ctxt, token); case JSON_INTEGER: case JSON_FLOAT: case JSON_STRING: - return parse_literal(ctxt); + return parse_literal(ctxt, token); case JSON_KEYWORD: - return parse_keyword(ctxt); + return parse_keyword(ctxt, token); default: parse_error(ctxt, token, "expecting value"); return NULL; } } =20 +static QObject *parse_token(JSONParserContext *ctxt, const JSONToken *toke= n) +{ + JSONParserStackEntry *entry; + JSONParserState state; + QString *key; + QObject *key_obj =3D NULL, *value =3D NULL; + + entry =3D current_entry(ctxt); + state =3D entry ? entry->state : BEFORE_VALUE; + switch (state) { + case AFTER_LCURLY: + /* Grab '}' for empty object or fall through to BEFORE_KEY */ + assert(qobject_type(entry->partial) =3D=3D QTYPE_QDICT); + if (token->type =3D=3D JSON_RCURLY) { + value =3D entry->partial; + entry =3D pop_entry(ctxt); + break; + } + entry->state =3D BEFORE_KEY; + /* fall through */ + + case BEFORE_KEY: + /* Expecting object key */ + assert(qobject_type(entry->partial) =3D=3D QTYPE_QDICT); + if (token->type !=3D JSON_STRING && token->type !=3D JSON_INTERP) { + parse_error(ctxt, token, "expecting key"); + return NULL; + } + + key_obj =3D parse_begin_value(ctxt, token); + if (!key_obj) { + /* Parse error already reported */ + } else if (qobject_type(key_obj) !=3D QTYPE_QSTRING) { + /* An interpolation was valid syntactically but not %s */ + parse_error(ctxt, token, "key is not a string in object"); + } else { + /* Store key in a special entry on the stack */ + push_entry(ctxt, key_obj, END_OF_KEY); + } + return NULL; + + case END_OF_KEY: + /* Expecting ':' after key */ + assert(qobject_type(entry->partial) =3D=3D QTYPE_QSTRING); + if (token->type =3D=3D JSON_COLON) { + entry->state =3D BEFORE_VALUE; + } else { + parse_error(ctxt, token, "expecting ':'"); + } + return NULL; + + case AFTER_LSQUARE: + /* Grab ']' for empty array or fall through to BEFORE_VALUE */ + assert(qobject_type(entry->partial) =3D=3D QTYPE_QLIST); + if (token->type =3D=3D JSON_RSQUARE) { + value =3D entry->partial; + entry =3D pop_entry(ctxt); + break; + } + entry->state =3D BEFORE_VALUE; + /* fall through */ + + case BEFORE_VALUE: + /* Expecting value */ + assert(!entry || qobject_type(entry->partial) !=3D QTYPE_QDICT); + value =3D parse_begin_value(ctxt, token); + if (!value) { + /* Error or '['/'{' */ + return NULL; + } + /* Return value or insert it into a container */ + break; + + case END_OF_VALUE: + /* Grab ',' or ']' for array; ',' or '}' for object */ + if (qobject_to(QList, entry->partial)) { + /* Array */ + if (token->type !=3D JSON_RSQUARE) { + if (token->type =3D=3D JSON_COMMA) { + entry->state =3D BEFORE_VALUE; + } else { + parse_error(ctxt, token, "expected ',' or ']'"); + } + return NULL; + } + } else if (qobject_to(QDict, entry->partial)) { + /* Object */ + if (token->type !=3D JSON_RCURLY) { + if (token->type =3D=3D JSON_COMMA) { + entry->state =3D BEFORE_KEY; + } else { + parse_error(ctxt, token, "expected ',' or '}'"); + } + return NULL; + } + } else { + g_assert_not_reached(); + } + + /* Got ']' or '}'; return full value or insert into parent contain= er */ + value =3D entry->partial; + entry =3D pop_entry(ctxt); + break; + } + + assert(value); + if (entry =3D=3D NULL) { + /* Parse stack now empty, the top-level value is complete. */ + return value; + } + + /* + * Parse stack is not empty and entry->partial is the top of stack. + * It's a QString with the key (and a QDict is below it) if we're + * parsing an object, or a QList if we're parsing an array. + */ + key =3D qobject_to(QString, entry->partial); + if (key) { + const char *key_str; + QDict *dict; + + /* Pop off key, and store (key, value) in QDict. */ + entry =3D pop_entry(ctxt); + dict =3D qobject_to(QDict, entry->partial); + assert(dict); + key_str =3D qstring_get_str(key); + if (qdict_haskey(dict, key_str)) { + parse_error(ctxt, token, "duplicate key"); + qobject_unref(value); + return NULL; + } + qdict_put_obj(dict, key_str, value); + qobject_unref(key); + } else { + /* Array, just store value in the QList. */ + qlist_append_obj(qobject_to(QList, entry->partial), value); + } + + entry->state =3D END_OF_VALUE; + return NULL; +} + JSONToken *json_token(JSONTokenType type, int x, int y, GString *tokstr) { JSONToken *token =3D g_malloc(sizeof(JSONToken) + tokstr->len + 1); @@ -572,20 +636,57 @@ JSONToken *json_token(JSONTokenType type, int x, int = y, GString *tokstr) return token; } =20 -QObject *json_parser_parse(GQueue *tokens, va_list *ap, Error **errp) +void json_parser_reset(JSONParserContext *ctxt) { - JSONParserContext ctxt =3D { .buf =3D tokens, .ap =3D ap }; - QObject *result; + JSONParserStackEntry *entry; =20 - result =3D parse_value(&ctxt); - assert(ctxt.err || g_queue_is_empty(ctxt.buf)); - - error_propagate(errp, ctxt.err); - - while (!g_queue_is_empty(ctxt.buf)) { - parser_context_pop_token(&ctxt); + ctxt->err =3D NULL; + while ((entry =3D g_queue_pop_tail(ctxt->stack)) !=3D NULL) { + qobject_unref(entry->partial); + g_free(entry); } - g_free(ctxt.current); +} =20 +void json_parser_init(JSONParserContext *ctxt, va_list *ap) +{ + ctxt->stack =3D g_queue_new(); + ctxt->ap =3D ap; + json_parser_reset(ctxt); +} + +void json_parser_destroy(JSONParserContext *ctxt) +{ + json_parser_reset(ctxt); + g_queue_free(ctxt->stack); + ctxt->stack =3D NULL; +} + +/* + * Advance the parser based on the token that is passed. + * Return the finished top-level value if the token completes it, else + * NULL. + * Once an error is returned, the function must not be called again + * without first resetting the parser. + */ +QObject *json_parser_feed(JSONParserContext *ctxt, const JSONToken *token, + Error **errp) +{ + QObject *result =3D NULL; + + assert(!ctxt->err); + switch (token->type) { + case JSON_END_OF_INPUT: + /* Check for premature end of input */ + if (!g_queue_is_empty(ctxt->stack)) { + parse_error(ctxt, token, "premature end of input"); + } + break; + + default: + result =3D parse_token(ctxt, token); + break; + } + + error_propagate(errp, ctxt->err); return result; } diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index b93d97b995..6c93e6fd78 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -32,6 +32,7 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, JSONTokenType type, int x, int y) { JSONMessageParser *parser =3D container_of(lexer, JSONMessageParser, l= exer); + JSONParserContext ctxt; QObject *json =3D NULL; Error *err =3D NULL; JSONToken *token; @@ -56,8 +57,7 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, if (g_queue_is_empty(&parser->tokens)) { return; } - json =3D json_parser_parse(&parser->tokens, parser->ap, &err); - goto out_emit; + break; default: break; } @@ -85,11 +85,24 @@ void json_message_process_token(JSONLexer *lexer, GStri= ng *input, g_queue_push_tail(&parser->tokens, token); =20 if ((parser->brace_count > 0 || parser->bracket_count > 0) - && parser->brace_count >=3D 0 && parser->bracket_count >=3D 0) { + && parser->brace_count >=3D 0 && parser->bracket_count >=3D 0 + && type !=3D JSON_END_OF_INPUT) { return; } =20 - json =3D json_parser_parse(&parser->tokens, parser->ap, &err); + json_parser_init(&ctxt, parser->ap); + + /* Process all tokens in the queue */ + while (!g_queue_is_empty(&parser->tokens)) { + token =3D g_queue_pop_head(&parser->tokens); + json =3D json_parser_feed(&ctxt, token, &err); + g_free(token); + if (json || err) { + break; + } + } + + json_parser_destroy(&ctxt); =20 out_emit: parser->brace_count =3D 0; --=20 2.54.0 From nobody Sun Jul 26 11:03:38 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782996043; cv=none; d=zohomail.com; s=zohoarc; b=mEYF2Y/+8sqN7FcEAUIaqJkgqYIeUvCcWKHmSQ1Z11Gvf6pjDndjLBackFU2VNd+R4g0RtUnI2ecjXmPRstmIm/4B/vURH3qB6Aftqwy5Fsrb/9rLLOGF1ulvWmHJJQOMmpKU50fErdwWbWssoS7wK43kmI9O7dDRdTD/URjf68= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782996043; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=8L15Bcx32fMBn9ToX54ay9RXWuDg7j6veFMiwl9j+H0=; b=Dwe2quRlnCHt4dm0t2iukgEOjkABChbHy07B1gfv/+uKKSdoQMgmpaThgVQpV848A8Z484k+QPB6z1x9yAK+FjfKqm11pOi4q/3ReEuMXGrx66BnJ2X/3dy4B6rsphXpcvLlz98qYwol1E3QwGxGuiohaquxbG0wNhZjV8JdXoU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782996043434796.7671002490179; Thu, 2 Jul 2026 05:40:43 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wfGiD-0004e4-1a; Thu, 02 Jul 2026 08:40:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi7-0004ca-0n for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi1-0000ZC-Bg for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:05 -0400 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-584-If-u7y84Ok22PhFxp3ADpQ-1; Thu, 02 Jul 2026 08:39:59 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F2BB9183338B; Thu, 2 Jul 2026 12:39:57 +0000 (UTC) Received: from blackfin.pond.sub.org (unknown [10.44.22.4]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A99A71971E6F; Thu, 2 Jul 2026 12:39:57 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 0BE1621E68B4; Thu, 02 Jul 2026 14:39:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782996000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8L15Bcx32fMBn9ToX54ay9RXWuDg7j6veFMiwl9j+H0=; b=Stl83micMVTg5NbOCb9LTvUrUTFZtiMVSYiIFXhuyP6UhzJ8wSK0VsdKRqn285/TD29CuN n3rUNDuyYjZci1b8sBaacSOpDM60p2BO9eKZwKBxyUYEl4I0Y5XLaibE6AJq3TWpcYJ54D T/CK7kQc3wHhB9k1FpjB/bIp+rcpRhk= X-MC-Unique: If-u7y84Ok22PhFxp3ADpQ-1 X-Mimecast-MFC-AGG-ID: If-u7y84Ok22PhFxp3ADpQ_1782995998 From: Markus Armbruster To: qemu-devel@nongnu.org Cc: stefanha@redhat.com, Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL 3/7] json-streamer: reuse parser Date: Thu, 2 Jul 2026 14:39:50 +0200 Message-ID: <20260702123954.1284221-4-armbru@redhat.com> In-Reply-To: <20260702123954.1284221-1-armbru@redhat.com> References: <20260702123954.1284221-1-armbru@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=armbru@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782996044925158500 From: Paolo Bonzini The push parser can be reset, so reuse it when the json-streamer detects a completed toplevel object. Reviewed-by: Markus Armbruster Signed-off-by: Paolo Bonzini Message-ID: <20260626101727.1727389-3-pbonzini@redhat.com> Reviewed-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Markus Armbruster --- include/qobject/json-parser.h | 2 +- qobject/json-streamer.c | 11 ++++------- 2 files changed, 5 insertions(+), 8 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 05346fa816..4c3d89f751 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -29,8 +29,8 @@ typedef struct JSONParserContext { typedef struct JSONMessageParser { void (*emit)(void *opaque, QObject *json, Error *err); void *opaque; - va_list *ap; JSONLexer lexer; + JSONParserContext parser; int brace_count; int bracket_count; GQueue tokens; diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index 6c93e6fd78..6c4f99b3e7 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -32,7 +32,6 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, JSONTokenType type, int x, int y) { JSONMessageParser *parser =3D container_of(lexer, JSONMessageParser, l= exer); - JSONParserContext ctxt; QObject *json =3D NULL; Error *err =3D NULL; JSONToken *token; @@ -90,21 +89,18 @@ void json_message_process_token(JSONLexer *lexer, GStri= ng *input, return; } =20 - json_parser_init(&ctxt, parser->ap); - /* Process all tokens in the queue */ while (!g_queue_is_empty(&parser->tokens)) { token =3D g_queue_pop_head(&parser->tokens); - json =3D json_parser_feed(&ctxt, token, &err); + json =3D json_parser_feed(&parser->parser, token, &err); g_free(token); if (json || err) { break; } } =20 - json_parser_destroy(&ctxt); - out_emit: + json_parser_reset(&parser->parser); parser->brace_count =3D 0; parser->bracket_count =3D 0; json_message_free_tokens(parser); @@ -119,12 +115,12 @@ void json_message_parser_init(JSONMessageParser *pars= er, { parser->emit =3D emit; parser->opaque =3D opaque; - parser->ap =3D ap; parser->brace_count =3D 0; parser->bracket_count =3D 0; g_queue_init(&parser->tokens); parser->token_size =3D 0; =20 + json_parser_init(&parser->parser, ap); json_lexer_init(&parser->lexer, !!ap); } =20 @@ -144,4 +140,5 @@ void json_message_parser_destroy(JSONMessageParser *par= ser) { json_lexer_destroy(&parser->lexer); json_message_free_tokens(parser); + json_parser_destroy(&parser->parser); } --=20 2.54.0 From nobody Sun Jul 26 11:03:38 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782996014; cv=none; d=zohomail.com; s=zohoarc; b=kSDw853MqoEntr59Dx8venOsfihFkLecCj1GUwB4eeeeKxYq6jJ3s51IsDEtAo+PGKPMW/HD4YDKYd4Rt4IAihj3ZDSnVp7LHmCoAwkSo9dZCXAqIiasdp7UenlNZEJG5IF3CxRdobu+yfSTekJSxIx32BGnvLbch//vWxF+oqM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782996014; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AzkgJH733zQ1/24sgPR1KcHp8RMMso47pRoPPIy81gs=; b=HhY79EJmo+ua7/LuJI2KBX6rjoYcVaVISe3GK33bKBJBEviTPLSD4xGaXYyk/lrSb5Xz1Ed+Z9MmD0leW/M4Xftq22P/5TMXUKbHVvGq3o5zrWec6RLD2Z8G8ll+4+QJLD45fAEBpInFOQrRw3JJ0lJU4qqLp+VJSnvIaGS0kDk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782996014304674.8752799836998; Thu, 2 Jul 2026 05:40:14 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wfGi6-0004cX-Vh; Thu, 02 Jul 2026 08:40:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi4-0004c3-T9 for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi1-0000Yz-8C for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:04 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-584-w4Zcfsv3MHG2-Kb0W2BJXQ-1; Thu, 02 Jul 2026 08:39:58 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1B7991944F01 for ; Thu, 2 Jul 2026 12:39:58 +0000 (UTC) Received: from blackfin.pond.sub.org (unknown [10.44.22.4]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B4FCC194510C for ; Thu, 2 Jul 2026 12:39:57 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 1193B21E6837; Thu, 02 Jul 2026 14:39:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782996000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AzkgJH733zQ1/24sgPR1KcHp8RMMso47pRoPPIy81gs=; b=NE/BD8mjHXayYfbyFuvcffN5enh6V1l9Prjyu+WlyeCrJlcEL+hZKoA6kRLXslZ7UDY/fT fSfxVzxCGJ2RX32mhNPsF9+6rZ9AhT+3j5X7CxT6awZyiCaryQPjCJXHV/9a905F0mtKPd 6hM9WL4HLwwfClpK82jiXexorO/g0VY= X-MC-Unique: w4Zcfsv3MHG2-Kb0W2BJXQ-1 X-Mimecast-MFC-AGG-ID: w4Zcfsv3MHG2-Kb0W2BJXQ_1782995998 From: Markus Armbruster To: qemu-devel@nongnu.org Cc: stefanha@redhat.com, Paolo Bonzini Subject: [PULL 4/7] json-streamer: make brace/bracket count unsigned Date: Thu, 2 Jul 2026 14:39:51 +0200 Message-ID: <20260702123954.1284221-5-armbru@redhat.com> In-Reply-To: <20260702123954.1284221-1-armbru@redhat.com> References: <20260702123954.1284221-1-armbru@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=armbru@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782996016914158500 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini It makes no sense to let brace_count and bracket_count go negative, also because it immediately ends error recovery and sets them both back to zero. Instead set them to zero *before* choosing whether to process the token queue; this makes it possible to have the fields as unsigned. Note that JSON_END_OF_INPUT now forces the parentheses to appear balanced, so that the queue is emptied and an error is reported; hence, the "type !=3D JSON_END_OF_INPUT" condition can be removed. Signed-off-by: Paolo Bonzini Message-ID: <20260626101727.1727389-4-pbonzini@redhat.com> Reviewed-by: Markus Armbruster [Comment tweaked] Signed-off-by: Markus Armbruster --- include/qobject/json-parser.h | 4 ++-- qobject/json-streamer.c | 24 +++++++++++++++++++++--- 2 files changed, 23 insertions(+), 5 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 4c3d89f751..0cf6932ecd 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -31,8 +31,8 @@ typedef struct JSONMessageParser { void *opaque; JSONLexer lexer; JSONParserContext parser; - int brace_count; - int bracket_count; + unsigned int brace_count; + unsigned int bracket_count; GQueue tokens; uint64_t token_size; } JSONMessageParser; diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index 6c4f99b3e7..e18fd9bb0e 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -41,21 +41,41 @@ void json_message_process_token(JSONLexer *lexer, GStri= ng *input, parser->brace_count++; break; case JSON_RCURLY: + if (!parser->brace_count) { + goto end_error_recovery; + } parser->brace_count--; break; case JSON_LSQUARE: parser->bracket_count++; break; case JSON_RSQUARE: + if (!parser->bracket_count) { + goto end_error_recovery; + } parser->bracket_count--; break; case JSON_ERROR: error_setg(&err, "JSON parse error, stray '%s'", input->str); goto out_emit; case JSON_END_OF_INPUT: + /* + * Force the parentheses to appear balanced and the queue + * to be emptied, causing a parse error if it wasn't. + */ if (g_queue_is_empty(&parser->tokens)) { return; } + end_error_recovery: + /* + * We come here due to receiving either JSON_ERROR or a + * JSON_R{CURLY,SQUARE}) that is known to be unbalanced. + * If in error recovery, end it immediately. If not in + * error recovery, json_parser_feed() will raise an error + * but error recovery won't be entered at all. + */ + parser->brace_count =3D 0; + parser->bracket_count =3D 0; break; default: break; @@ -83,9 +103,7 @@ void json_message_process_token(JSONLexer *lexer, GStrin= g *input, =20 g_queue_push_tail(&parser->tokens, token); =20 - if ((parser->brace_count > 0 || parser->bracket_count > 0) - && parser->brace_count >=3D 0 && parser->bracket_count >=3D 0 - && type !=3D JSON_END_OF_INPUT) { + if (parser->brace_count > 0 || parser->bracket_count > 0) { return; } =20 --=20 2.54.0 From nobody Sun Jul 26 11:03:38 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782996056; cv=none; d=zohomail.com; s=zohoarc; b=ns9Aozt8VX8zEwLjAM7lUrwncAfoHIe2MlsUOj4/N84tNP6sNyQZcvguWJn9iU6JRwbqSUFgbGPD1QL8U6LK2pl9yZGgu65OqHRW5YsU+7L5QTUQMABmA0RSJJwFZsV1IahpkFDGPX/ihE3yvNnJ7tO1VNBslus9XbxW5GVY/ZA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782996056; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2EQI2xs4DbG+uKQbd6CA4mkkG9SkGAaPYno2FXeAKP8=; b=Zj2kuRONVmhdr5TH9AC1WFoQKixEadazIJ5H/I9pY3hdogMtq5x3/0dpuz84zGP3QCNVR6cFKnnP0oyilf+4cAwj3INhgKtBik+nzXtLsQeOO1mHiigMPIgiFmhIVFzGqjJs4VBbrS+kYmhy+tlYgJtg9sXRp7SzQbVO2EBY+R8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782996056439223.32114839112955; Thu, 2 Jul 2026 05:40:56 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wfGiI-0004hU-47; Thu, 02 Jul 2026 08:40:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi9-0004d4-02 for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi4-0000bn-K1 for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:07 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-275-eZ13an4tMiOgTUmswzHb5A-1; Thu, 02 Jul 2026 08:40:01 -0400 Received: from mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.95]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E9EB11944B04 for ; Thu, 2 Jul 2026 12:40:00 +0000 (UTC) Received: from blackfin.pond.sub.org (unknown [10.44.22.4]) by mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 71E7136F11 for ; Thu, 2 Jul 2026 12:40:00 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 1454B21E6742; Thu, 02 Jul 2026 14:39:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782996003; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2EQI2xs4DbG+uKQbd6CA4mkkG9SkGAaPYno2FXeAKP8=; b=AaqSDbU6e7v2QMz94dCPhzvbelSuGtfk8uQXHlTETrhIZieILwHT29Pv63S3qwsleIdedJ gaej7IIYMJmtOYDKuvOxpYhJph9P+yXY6rD4TO08GmtM0a3HB6BS14ObdCzKVXqWrzHK/U +ZAoZwLQAcIPmerF3CFux82HcPmHNts= X-MC-Unique: eZ13an4tMiOgTUmswzHb5A-1 X-Mimecast-MFC-AGG-ID: eZ13an4tMiOgTUmswzHb5A_1782996001 From: Markus Armbruster To: qemu-devel@nongnu.org Cc: stefanha@redhat.com, Paolo Bonzini Subject: [PULL 5/7] json-streamer: remove token queue Date: Thu, 2 Jul 2026 14:39:52 +0200 Message-ID: <20260702123954.1284221-6-armbru@redhat.com> In-Reply-To: <20260702123954.1284221-1-armbru@redhat.com> References: <20260702123954.1284221-1-armbru@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.6 on 10.30.177.95 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=armbru@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782996057119158500 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini Now fully exploit the push parser, feeding it one token at a time without having to wait until braces and brackets are balanced. While the nesting counts are retained for error recovery purposes, the system can now report the first parsing error without waiting for parentheses to be balanced. This also means that JSON_ERROR can be handled in json-parser.c, not json-streamer.c. After reporting the error, json-streamer.c then enters an error recovery mode where subsequent errors are suppressed. This mimics the previous error reporting behavior, but it provides prompt feedback on parsing errors. As an example, here is an example interaction with qemu-ga. BEFORE (error reported only once braces are balanced): >> {"execute":foo >> } << {"error": {"class": "GenericError", "desc": "JSON parse error, invali= d keyword 'foo'"}} >> {"execute":"somecommand"} << {"error": {"class": "CommandNotFound", "desc": "The command somecomma= nd has not been found"}} AFTER (error reported immediately, but similar error recovery as before): >> {"execute":foo << {"error": {"class": "GenericError", "desc": "JSON parse error, invali= d keyword 'foo'"}} >> } >> {"execute":"somecommand"} << {"error": {"class": "CommandNotFound", "desc": "The command somecomma= nd has not been found"}} Signed-off-by: Paolo Bonzini Message-ID: <20260626101727.1727389-5-pbonzini@redhat.com> Reviewed-by: Markus Armbruster [Token size limit check off-by-one fixed] Signed-off-by: Markus Armbruster --- include/qobject/json-parser.h | 3 +- qobject/json-parser.c | 4 ++ qobject/json-streamer.c | 104 +++++++++++++--------------------- 3 files changed, 46 insertions(+), 65 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 0cf6932ecd..3479e63758 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -33,7 +33,8 @@ typedef struct JSONMessageParser { JSONParserContext parser; unsigned int brace_count; unsigned int bracket_count; - GQueue tokens; + unsigned int token_count; + bool error; uint64_t token_size; } JSONMessageParser; =20 diff --git a/qobject/json-parser.c b/qobject/json-parser.c index 5ed1aa1c8d..79a2989e7a 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -675,6 +675,10 @@ QObject *json_parser_feed(JSONParserContext *ctxt, con= st JSONToken *token, =20 assert(!ctxt->err); switch (token->type) { + case JSON_ERROR: + parse_error(ctxt, token, "stray '%s'", token->str); + break; + case JSON_END_OF_INPUT: /* Check for premature end of input */ if (!g_queue_is_empty(ctxt->stack)) { diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index e18fd9bb0e..48c5506771 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -1,5 +1,5 @@ /* - * JSON streaming support + * JSON parser - callback interface and error recovery * * Copyright IBM, Corp. 2009 * @@ -19,23 +19,16 @@ #define MAX_TOKEN_COUNT (2ULL << 20) #define MAX_NESTING (1 << 10) =20 -static void json_message_free_tokens(JSONMessageParser *parser) -{ - JSONToken *token; - - while ((token =3D g_queue_pop_head(&parser->tokens))) { - g_free(token); - } -} - void json_message_process_token(JSONLexer *lexer, GString *input, JSONTokenType type, int x, int y) { JSONMessageParser *parser =3D container_of(lexer, JSONMessageParser, l= exer); - QObject *json =3D NULL; Error *err =3D NULL; - JSONToken *token; =20 + parser->token_size +=3D input->len; + parser->token_count++; + + /* Detect message boundaries for error recovery purposes. */ switch (type) { case JSON_LCURLY: parser->brace_count++; @@ -56,16 +49,6 @@ void json_message_process_token(JSONLexer *lexer, GStrin= g *input, parser->bracket_count--; break; case JSON_ERROR: - error_setg(&err, "JSON parse error, stray '%s'", input->str); - goto out_emit; - case JSON_END_OF_INPUT: - /* - * Force the parentheses to appear balanced and the queue - * to be emptied, causing a parse error if it wasn't. - */ - if (g_queue_is_empty(&parser->tokens)) { - return; - } end_error_recovery: /* * We come here due to receiving either JSON_ERROR or a @@ -81,49 +64,43 @@ void json_message_process_token(JSONLexer *lexer, GStri= ng *input, break; } =20 - /* - * Security consideration, we limit total memory allocated per object - * and the maximum recursion depth that a message can force. - */ - if (parser->token_size + input->len + 1 > MAX_TOKEN_SIZE) { - error_setg(&err, "JSON token size limit exceeded"); - goto out_emit; - } - if (g_queue_get_length(&parser->tokens) + 1 > MAX_TOKEN_COUNT) { - error_setg(&err, "JSON token count limit exceeded"); - goto out_emit; - } - if (parser->bracket_count + parser->brace_count > MAX_NESTING) { - error_setg(&err, "JSON nesting depth limit exceeded"); - goto out_emit; - } + if (parser->error) { + /* error recovery, eat tokens until parentheses balance */ + } else { + /* + * Safety consideration, we limit total memory allocated per object + * and the maximum nesting depth that a message can force. + */ + if (parser->token_size >=3D MAX_TOKEN_SIZE) { + error_setg(&err, "JSON token size limit exceeded"); + } else if (parser->token_count > MAX_TOKEN_COUNT) { + error_setg(&err, "JSON token count limit exceeded"); + } else if (parser->bracket_count + parser->brace_count > MAX_NESTI= NG) { + error_setg(&err, "JSON nesting depth limit exceeded"); + } else { + g_autofree JSONToken *token =3D json_token(type, x, y, input); + QObject *json =3D json_parser_feed(&parser->parser, token, &er= r); + if (json) { + parser->emit(parser->opaque, json, NULL); + } + } =20 - token =3D json_token(type, x, y, input); - parser->token_size +=3D input->len; - - g_queue_push_tail(&parser->tokens, token); - - if (parser->brace_count > 0 || parser->bracket_count > 0) { - return; - } - - /* Process all tokens in the queue */ - while (!g_queue_is_empty(&parser->tokens)) { - token =3D g_queue_pop_head(&parser->tokens); - json =3D json_parser_feed(&parser->parser, token, &err); - g_free(token); - if (json || err) { - break; + if (err) { + parser->emit(parser->opaque, NULL, err); + /* start recovery */ + parser->error =3D true; } } =20 -out_emit: - json_parser_reset(&parser->parser); - parser->brace_count =3D 0; - parser->bracket_count =3D 0; - json_message_free_tokens(parser); - parser->token_size =3D 0; - parser->emit(parser->opaque, json, err); + if ((parser->brace_count =3D=3D 0 && parser->bracket_count =3D=3D 0) + || type =3D=3D JSON_END_OF_INPUT) { + json_parser_reset(&parser->parser); + parser->error =3D false; + parser->brace_count =3D 0; + parser->bracket_count =3D 0; + parser->token_count =3D 0; + parser->token_size =3D 0; + } } =20 void json_message_parser_init(JSONMessageParser *parser, @@ -133,9 +110,10 @@ void json_message_parser_init(JSONMessageParser *parse= r, { parser->emit =3D emit; parser->opaque =3D opaque; + parser->error =3D false; parser->brace_count =3D 0; parser->bracket_count =3D 0; - g_queue_init(&parser->tokens); + parser->token_count =3D 0; parser->token_size =3D 0; =20 json_parser_init(&parser->parser, ap); @@ -151,12 +129,10 @@ void json_message_parser_feed(JSONMessageParser *pars= er, void json_message_parser_flush(JSONMessageParser *parser) { json_lexer_flush(&parser->lexer); - assert(g_queue_is_empty(&parser->tokens)); } =20 void json_message_parser_destroy(JSONMessageParser *parser) { json_lexer_destroy(&parser->lexer); - json_message_free_tokens(parser); json_parser_destroy(&parser->parser); } --=20 2.54.0 From nobody Sun Jul 26 11:03:38 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782996068; cv=none; d=zohomail.com; s=zohoarc; b=PUQ/7P0qvOYzpDyPpKt8fmGMWOjqmTNLbhXHkhRH4VQJrVSsR513TtMxzcpH+/P8CwuN19eHvOIEPs5kIMLaxbN6Bqp5j48CnOeXCU1Jca9vRE7aE6zl/3N+Oq4ba5ZZxKpTi5WIDh+9wCyUcXTca4Z7NsMUlbfiLM/nsmaJBk8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782996068; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=/cKsmNByVUGfSieyuA+9LR/rYjRAcBMJs3v8ne67gDM=; b=lbnqL8akSqFNjIcJCFi2QL38dmtO8E5jlZ5gbFCS4Gz1twuzwISc8JHxItDhZiEL2ff7rpcMfKQnMVxoOGIUXHMwE/u9QKvHKvcDvZD0gtJFPJ4Ks+2ju2vmFZvp5fhreKBjZobNZulhyvZjX+25t7w1mA/NDibRrDlfnW8noI4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782996068767382.8987040406465; Thu, 2 Jul 2026 05:41:08 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wfGiH-0004g9-6V; Thu, 02 Jul 2026 08:40:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi7-0004cZ-0a for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi3-0000bO-Fg for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:05 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-449-SceaZBtAOn6rP6m5dK1pCw-1; Thu, 02 Jul 2026 08:40:01 -0400 Received: from mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.95]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id ADFBF1944B01 for ; Thu, 2 Jul 2026 12:40:00 +0000 (UTC) Received: from blackfin.pond.sub.org (unknown [10.44.22.4]) by mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6C57A36F0B for ; Thu, 2 Jul 2026 12:40:00 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 182CA21E66CA; Thu, 02 Jul 2026 14:39:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782996002; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/cKsmNByVUGfSieyuA+9LR/rYjRAcBMJs3v8ne67gDM=; b=MWmuTo91BxRVYukDkxiUpGpQyIzVNgt96T4oVifYN/boA8sfpwDu7jUIKhzKVvimQI5YhY /VMQA04sODX9PqXCDN/orONUyv5yBmlV/A6PeFZPIv6Zvuip6mF8fBBqnXG2f9HgQNTgFA LYg/nMHjchPsII8C6J7BUHz5VZ/s4+I= X-MC-Unique: SceaZBtAOn6rP6m5dK1pCw-1 X-Mimecast-MFC-AGG-ID: SceaZBtAOn6rP6m5dK1pCw_1782996000 From: Markus Armbruster To: qemu-devel@nongnu.org Cc: stefanha@redhat.com, Paolo Bonzini Subject: [PULL 6/7] json-streamer: do not heap-allocate JSONToken Date: Thu, 2 Jul 2026 14:39:53 +0200 Message-ID: <20260702123954.1284221-7-armbru@redhat.com> In-Reply-To: <20260702123954.1284221-1-armbru@redhat.com> References: <20260702123954.1284221-1-armbru@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.6 on 10.30.177.95 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=armbru@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782996069178158500 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini This is not needed with a push parser. Since it processes tokens immediately, the JSONToken can be created directly on the stack and does not need to copy the lexer's string data. Reviewed-by: Markus Armbruster Signed-off-by: Paolo Bonzini Message-ID: <20260626101727.1727389-6-pbonzini@redhat.com> Signed-off-by: Markus Armbruster --- qobject/json-parser-int.h | 8 ++++++-- qobject/json-parser.c | 18 ------------------ qobject/json-streamer.c | 9 +++++++-- 3 files changed, 13 insertions(+), 22 deletions(-) diff --git a/qobject/json-parser-int.h b/qobject/json-parser-int.h index 1f435cb8eb..5a6b5c9af9 100644 --- a/qobject/json-parser-int.h +++ b/qobject/json-parser-int.h @@ -35,7 +35,12 @@ typedef enum json_token_type { JSON_MAX =3D JSON_END_OF_INPUT } JSONTokenType; =20 -typedef struct JSONToken JSONToken; +typedef struct JSONToken { + JSONTokenType type; + int x; + int y; + char *str; +} JSONToken; =20 /* json-lexer.c */ void json_lexer_init(JSONLexer *lexer, bool enable_interpolation); @@ -48,7 +53,6 @@ void json_message_process_token(JSONLexer *lexer, GString= *input, JSONTokenType type, int x, int y); =20 /* json-parser.c */ -JSONToken *json_token(JSONTokenType type, int x, int y, GString *tokstr); void json_parser_init(JSONParserContext *ctxt, va_list *ap); void json_parser_reset(JSONParserContext *ctxt); QObject *json_parser_feed(JSONParserContext *ctxt, const JSONToken *token,= Error **errp); diff --git a/qobject/json-parser.c b/qobject/json-parser.c index 79a2989e7a..0dcc94fefd 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -24,13 +24,6 @@ #include "qobject/qstring.h" #include "json-parser-int.h" =20 -struct JSONToken { - JSONTokenType type; - int x; - int y; - char str[]; -}; - /* * The JSON parser is a push parser, returning a completed top-level * object, an error, or NULL (if the object is incomplete and no error @@ -624,17 +617,6 @@ static QObject *parse_token(JSONParserContext *ctxt, c= onst JSONToken *token) return NULL; } =20 -JSONToken *json_token(JSONTokenType type, int x, int y, GString *tokstr) -{ - JSONToken *token =3D g_malloc(sizeof(JSONToken) + tokstr->len + 1); - - token->type =3D type; - memcpy(token->str, tokstr->str, tokstr->len); - token->str[tokstr->len] =3D 0; - token->x =3D x; - token->y =3D y; - return token; -} =20 void json_parser_reset(JSONParserContext *ctxt) { diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index 48c5506771..b634bbe99c 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -78,8 +78,13 @@ void json_message_process_token(JSONLexer *lexer, GStrin= g *input, } else if (parser->bracket_count + parser->brace_count > MAX_NESTI= NG) { error_setg(&err, "JSON nesting depth limit exceeded"); } else { - g_autofree JSONToken *token =3D json_token(type, x, y, input); - QObject *json =3D json_parser_feed(&parser->parser, token, &er= r); + JSONToken token =3D (JSONToken) { + .type =3D type, + .x =3D x, + .y =3D y, + .str =3D input->str + }; + QObject *json =3D json_parser_feed(&parser->parser, &token, &e= rr); if (json) { parser->emit(parser->opaque, json, NULL); } --=20 2.54.0 From nobody Sun Jul 26 11:03:38 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1782996054; cv=none; d=zohomail.com; s=zohoarc; b=lmpISJc4yjNF9cII04vAiIMhruAlawAkkGHroIDgD23FmJCRHByiT++sGzynNfmNdrNmhqNWRIclfNFpoRKxlBgd7jZacPzPiNcyh1hAJDN3WnZX7lfefeHAIIJ0LsQIqskprqbUgdRZRyAp0WsaUmd1nUs/GCnnYU9FP9t93Ds= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1782996054; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=U9Jvfizcauxl+X/zlHc1DSC3xHHM3vTwIZ1eT7VSYK8=; b=KQpsK/fOLQJ5wNYrHAy3lANdC5L4bn+ol84mSf3Awe4FayfIsY8Cx+0aS/nNGpQP794aZp2BzFflYMKZ9nDVCQCscLKfY6QIpukCIBCo+u1a9h7nu9mT0jqjUX1jpMoyYD4SR+7iHrkH47fh/kHWBCJbtVWhwP4TNz7GXPDwKyw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1782996054592809.7633625963304; Thu, 2 Jul 2026 05:40:54 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wfGiC-0004dp-7E; Thu, 02 Jul 2026 08:40:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi7-0004cb-5T for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wfGi4-0000bo-Ja for qemu-devel@nongnu.org; Thu, 02 Jul 2026 08:40:06 -0400 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-84-zTj_k50HNmWaZ-znQXL67A-1; Thu, 02 Jul 2026 08:40:01 -0400 Received: from mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.95]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EFA861944F2E for ; Thu, 2 Jul 2026 12:40:00 +0000 (UTC) Received: from blackfin.pond.sub.org (unknown [10.44.22.4]) by mx-prod-int-10.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A8FBA404 for ; Thu, 2 Jul 2026 12:40:00 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 1C20221E668C; Thu, 02 Jul 2026 14:39:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782996003; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U9Jvfizcauxl+X/zlHc1DSC3xHHM3vTwIZ1eT7VSYK8=; b=KbZ3Z9Ec9nRH1jlenfHnmeDL8ths2qIjDGlTxZqkt+E1/vfoDJE3CZ11a3RQu4RltCP8R4 oJyz5WgKd1mM0zrBD6Yok/IdMPVo0OOFBJP9lPEaT5cUUjnH1KwJeTioeD/5TjlSIaPrWu js434pkndflM7iWP7DQF7UXXIhBowU0= X-MC-Unique: zTj_k50HNmWaZ-znQXL67A-1 X-Mimecast-MFC-AGG-ID: zTj_k50HNmWaZ-znQXL67A_1782996001 From: Markus Armbruster To: qemu-devel@nongnu.org Cc: stefanha@redhat.com, Paolo Bonzini Subject: [PULL 7/7] json-parser: add location to JSON parsing errors Date: Thu, 2 Jul 2026 14:39:54 +0200 Message-ID: <20260702123954.1284221-8-armbru@redhat.com> In-Reply-To: <20260702123954.1284221-1-armbru@redhat.com> References: <20260702123954.1284221-1-armbru@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.6 on 10.30.177.95 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=armbru@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1782996055052158501 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini Now that all calls to parse_error have a token, add the line and column to the message. As far as I can see the two important TODOs (better errors and better EOI handling) are done, and the others (token range information and "parsed size"?) do not really matter or are handled better by json-streamer.c. So remove the list, which had sat unchanged since 2009. This needs some adjustments to provide a good x and y for error messages. First of all, they switch from zero-based to one-based, which is safe because they were both sitting unused. Second, right now the x and y are those of the *last* character in the token. Modify json-lexer.c to freeze tok->x and tok->y at the first character added to the GString. Signed-off-by: Paolo Bonzini Message-ID: <20260626101727.1727389-7-pbonzini@redhat.com> Reviewed-by: Markus Armbruster Signed-off-by: Markus Armbruster --- include/qobject/json-parser.h | 1 + qobject/json-lexer.c | 11 +++++++---- qobject/json-parser.c | 12 ++---------- 3 files changed, 10 insertions(+), 14 deletions(-) diff --git a/include/qobject/json-parser.h b/include/qobject/json-parser.h index 3479e63758..e078b36b2d 100644 --- a/include/qobject/json-parser.h +++ b/include/qobject/json-parser.h @@ -17,6 +17,7 @@ typedef struct JSONLexer { int start_state, state; GString *token; + int cur_x, cur_y; int x, y; } JSONLexer; =20 diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c index 51341d96e4..7753ba6c09 100644 --- a/qobject/json-lexer.c +++ b/qobject/json-lexer.c @@ -277,7 +277,8 @@ void json_lexer_init(JSONLexer *lexer, bool enable_inte= rpolation) lexer->start_state =3D lexer->state =3D enable_interpolation ? IN_START_INTERP : IN_START; lexer->token =3D g_string_sized_new(3); - lexer->x =3D lexer->y =3D 0; + lexer->cur_x =3D lexer->cur_y =3D 1; + lexer->x =3D lexer->y =3D 1; } =20 static void json_lexer_feed_char(JSONLexer *lexer, char ch, bool flush) @@ -285,10 +286,10 @@ static void json_lexer_feed_char(JSONLexer *lexer, ch= ar ch, bool flush) int new_state; bool char_consumed =3D false; =20 - lexer->x++; + lexer->cur_x++; if (ch =3D=3D '\n') { - lexer->x =3D 0; - lexer->y++; + lexer->cur_x =3D 1; + lexer->cur_y++; } =20 while (flush ? lexer->state !=3D lexer->start_state : !char_consumed) { @@ -316,6 +317,8 @@ static void json_lexer_feed_char(JSONLexer *lexer, char= ch, bool flush) case IN_START: g_string_truncate(lexer->token, 0); new_state =3D lexer->start_state; + lexer->x =3D lexer->cur_x; + lexer->y =3D lexer->cur_y; break; case JSON_ERROR: json_message_process_token(lexer, lexer->token, JSON_ERROR, diff --git a/qobject/json-parser.c b/qobject/json-parser.c index 0dcc94fefd..5935730c0c 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -133,15 +133,6 @@ typedef struct JSONParserStackEntry { =20 #define BUG_ON(cond) assert(!(cond)) =20 -/** - * TODO - * - * 0) make errors meaningful again - * 1) add geometry information to tokens - * 3) should we return a parsed size? - * 4) deal with premature EOI - */ - static inline JSONParserStackEntry *current_entry(JSONParserContext *ctxt) { return g_queue_peek_tail(ctxt->stack); @@ -180,7 +171,8 @@ static void G_GNUC_PRINTF(3, 4) parse_error(JSONParserC= ontext *ctxt, va_start(ap, msg); vsnprintf(message, sizeof(message), msg, ap); va_end(ap); - error_setg(&ctxt->err, "JSON parse error, %s", message); + error_setg(&ctxt->err, "%d:%d: JSON parse error, %s", + token->y, token->x, message); } =20 static int cvt4hex(const char *s) --=20 2.54.0