From nobody Mon Feb 9 02:27:54 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488196540767661.6098839290986; Mon, 27 Feb 2017 03:55:40 -0800 (PST) Received: from localhost ([::1]:51970 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ciJtz-0003Yz-5e for importer@patchew.org; Mon, 27 Feb 2017 06:55:39 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44834) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ciJMd-0006Gn-QQ for qemu-devel@nongnu.org; Mon, 27 Feb 2017 06:21:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ciJMb-00018r-Eu for qemu-devel@nongnu.org; Mon, 27 Feb 2017 06:21:11 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51330) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ciJMO-0000vr-Q5; Mon, 27 Feb 2017 06:20:57 -0500 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EEC43C054C3C; Mon, 27 Feb 2017 11:20:56 +0000 (UTC) Received: from blackfin.pond.sub.org (ovpn-116-55.ams2.redhat.com [10.36.116.55]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1RBKtWX021136 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 27 Feb 2017 06:20:56 -0500 Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 5DEAA11384AE; Mon, 27 Feb 2017 12:20:50 +0100 (CET) From: Markus Armbruster To: qemu-devel@nongnu.org Date: Mon, 27 Feb 2017 12:20:50 +0100 Message-Id: <1488194450-28056-25-git-send-email-armbru@redhat.com> In-Reply-To: <1488194450-28056-1-git-send-email-armbru@redhat.com> References: <1488194450-28056-1-git-send-email-armbru@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 27 Feb 2017 11:20:57 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 24/24] keyval: Support lists X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, "mdroth@linux.vnet.ibm.commdroth"@linux.vnet.ibm.com, pkrempa@redhat.com, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Additionally permit non-negative integers as key components. A dictionary's keys must either be all integers or none. If all keys are integers, convert the dictionary to a list. The set of keys must be [0,N]. Examples: * list.1=3Dgoner,list.0=3Dnull,list.1=3Deins,list.2=3Dzwei is equivalent to JSON [ "null", "eins", "zwei" ] * a.b.c=3D1,a.b.0=3D2 is inconsistent: a.b.c clashes with a.b.0 * list.0=3Dnull,list.2=3Deins,list.2=3Dzwei has a hole: list.1 is missing Similar design flaw as for objects: there is now way to denote an empty list. While interpreting "key absent" as empty list seems natural (removing a list member from the input string works when there are multiple ones, so why not when there's just one), it doesn't work: "key absent" already means "optional list absent", which isn't the same as "empty list present". Update the keyval object visitor to use this a.0 syntax in error messages rather than the usual a[0]. Signed-off-by: Markus Armbruster --- qapi/qobject-input-visitor.c | 5 +- tests/test-keyval.c | 117 ++++++++++++++++++++++++++++ util/keyval.c | 177 +++++++++++++++++++++++++++++++++++++++= +--- 3 files changed, 286 insertions(+), 13 deletions(-) diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c index 1d7b420..4c159e0 100644 --- a/qapi/qobject-input-visitor.c +++ b/qapi/qobject-input-visitor.c @@ -41,6 +41,7 @@ struct QObjectInputVisitor { =20 /* Root of visit at visitor creation. */ QObject *root; + bool keyval; /* Assume @root made with keyval_parse() */ =20 /* Stack of objects being visited (all entries will be either * QDict or QList). */ @@ -73,7 +74,9 @@ static const char *full_name_nth(QObjectInputVisitor *qiv= , const char *name, g_string_prepend(qiv->errname, name ?: ""); g_string_prepend_c(qiv->errname, '.'); } else { - snprintf(buf, sizeof(buf), "[%u]", so->index); + snprintf(buf, sizeof(buf), + qiv->keyval ? ".%u" : "[%u]", + so->index); g_string_prepend(qiv->errname, buf); } name =3D so->name; diff --git a/tests/test-keyval.c b/tests/test-keyval.c index 6eceafb..1ff6035 100644 --- a/tests/test-keyval.c +++ b/tests/test-keyval.c @@ -12,6 +12,7 @@ =20 #include "qemu/osdep.h" #include "qapi/error.h" +#include "qapi/qmp/qstring.h" #include "qapi/qobject-input-visitor.h" #include "qemu/cutils.h" #include "qemu/option.h" @@ -183,6 +184,71 @@ static void test_keyval_parse(void) g_assert(!qdict); } =20 +static void check_list012(QList *qlist) +{ + static const char *expected[] =3D { "null", "eins", "zwei" }; + int i; + QString *qstr; + + g_assert(qlist); + for (i =3D 0; i < ARRAY_SIZE(expected); i++) { + qstr =3D qobject_to_qstring(qlist_pop(qlist)); + g_assert(qstr); + g_assert_cmpstr(qstring_get_str(qstr), =3D=3D, expected[i]); + } + g_assert(qlist_empty(qlist)); +} + +static void test_keyval_parse_list(void) +{ + Error *err =3D NULL; + QDict *qdict, *sub_qdict; + + /* Root can't be a list */ + qdict =3D keyval_parse("0=3D1", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* List elements need not be in order */ + qdict =3D keyval_parse("list.0=3Dnull,list.2=3Dzwei,list.1=3Deins", + NULL, &error_abort); + g_assert_cmpint(qdict_size(qdict), =3D=3D, 1); + check_list012(qdict_get_qlist(qdict, "list")); + QDECREF(qdict); + + /* Multiple indexes, last one wins */ + qdict =3D keyval_parse("list.1=3Dgoner,list.0=3Dnull,list.1=3Deins,lis= t.2=3Dzwei", + NULL, &error_abort); + g_assert_cmpint(qdict_size(qdict), =3D=3D, 1); + check_list012(qdict_get_qlist(qdict, "list")); + QDECREF(qdict); + + /* List at deeper nesting */ + qdict =3D keyval_parse("a.list.1=3Deins,a.list.0=3Dnull,a.list.2=3Dzwe= i", + NULL, &error_abort); + g_assert_cmpint(qdict_size(qdict), =3D=3D, 1); + sub_qdict =3D qdict_get_qdict(qdict, "a"); + g_assert_cmpint(qdict_size(sub_qdict), =3D=3D, 1); + check_list012(qdict_get_qlist(sub_qdict, "list")); + QDECREF(qdict); + + /* Inconsistent dotted keys: both list and dictionary */ + qdict =3D keyval_parse("a.b.c=3D1,a.b.0=3D2", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + qdict =3D keyval_parse("a.0.c=3D1,a.b.c=3D2", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Missing list indexes */ + qdict =3D keyval_parse("list.2=3Dlonely", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + qdict =3D keyval_parse("list.0=3Dnull,list.2=3Deins,list.2=3Dzwei", NU= LL, &err); + error_free_or_abort(&err); + g_assert(!qdict); +} + static void test_keyval_visit_bool(void) { Error *err =3D NULL; @@ -459,6 +525,55 @@ static void test_keyval_visit_dict(void) visit_free(v); } =20 +static void test_keyval_visit_list(void) +{ + Error *err =3D NULL; + Visitor *v; + QDict *qdict; + char *s; + + qdict =3D keyval_parse("a.0=3D,a.1=3DI,a.2.0=3DII", NULL, &error_abort= ); + /* TODO empty list */ + v =3D qobject_input_visitor_new_keyval(QOBJECT(qdict)); + QDECREF(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_start_list(v, "a", NULL, 0, &error_abort); + visit_type_str(v, NULL, &s, &error_abort); + g_assert_cmpstr(s, =3D=3D, ""); + visit_type_str(v, NULL, &s, &error_abort); + g_assert_cmpstr(s, =3D=3D, "I"); + visit_start_list(v, NULL, NULL, 0, &error_abort); + visit_type_str(v, NULL, &s, &error_abort); + g_assert_cmpstr(s, =3D=3D, "II"); + visit_check_list(v, &error_abort); + visit_end_list(v, NULL); + visit_check_list(v, &error_abort); + visit_end_list(v, NULL); + visit_check_struct(v, &error_abort); + visit_end_struct(v, NULL); + visit_free(v); + + qdict =3D keyval_parse("a.0=3D,b.0.0=3Dhead", NULL, &error_abort); + v =3D qobject_input_visitor_new_keyval(QOBJECT(qdict)); + QDECREF(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_start_list(v, "a", NULL, 0, &error_abort); + visit_check_list(v, &err); /* a[0] unexpected */ + error_free_or_abort(&err); + visit_end_list(v, NULL); + visit_start_list(v, "b", NULL, 0, &error_abort); + visit_start_list(v, NULL, NULL, 0, &error_abort); + visit_type_str(v, NULL, &s, &error_abort); + g_assert_cmpstr(s, =3D=3D, "head"); + visit_type_str(v, NULL, &s, &err); /* b[0][1] missing */ + error_free_or_abort(&err); + visit_end_list(v, NULL); + visit_end_list(v, NULL); + visit_check_struct(v, &error_abort); + visit_end_struct(v, NULL); + visit_free(v); +} + static void test_keyval_visit_optional(void) { Visitor *v; @@ -492,10 +607,12 @@ int main(int argc, char *argv[]) { g_test_init(&argc, &argv, NULL); g_test_add_func("/keyval/keyval_parse", test_keyval_parse); + g_test_add_func("/keyval/keyval_parse/list", test_keyval_parse_list); g_test_add_func("/keyval/visit/bool", test_keyval_visit_bool); g_test_add_func("/keyval/visit/number", test_keyval_visit_number); g_test_add_func("/keyval/visit/size", test_keyval_visit_size); g_test_add_func("/keyval/visit/dict", test_keyval_visit_dict); + g_test_add_func("/keyval/visit/list", test_keyval_visit_list); g_test_add_func("/keyval/visit/optional", test_keyval_visit_optional); g_test_run(); return 0; diff --git a/util/keyval.c b/util/keyval.c index 1170dad..3621f28 100644 --- a/util/keyval.c +++ b/util/keyval.c @@ -21,10 +21,12 @@ * * Semantics defined by reduction to JSON: * - * key-vals defines a tree of objects rooted at R + * key-vals is a tree of objects and arrays rooted at object R * where for each key-val =3D key-fragment . ... =3D val in key-vals * R op key-fragment op ... =3D val' - * where (left-associative) op is member reference L.key-fragment + * where (left-associative) op is + * array subscript L[key-fragment] for numeric key-fragment + * member reference L.key-fragment otherwise * val' is val with ',,' replaced by ',' * and only R may be empty. * @@ -34,16 +36,16 @@ * doesn't have one, because R.a must be an object to satisfy a.b=3D1 * and a string to satisfy a=3D2. * - * Key-fragments must be valid QAPI names. + * Key-fragments must be valid QAPI names or consist only of digits. * * The length of any key-fragment must be between 1 and 127. * - * Design flaw: there is no way to denote an empty non-root object. - * While interpreting "key absent" as empty object seems natural + * Design flaw: there is no way to denote an empty array or non-root + * object. While interpreting "key absent" as empty seems natural * (removing a key-val from the input string removes the member when * there are more, so why not when it's the last), it doesn't work: - * "key absent" already means "optional object absent", which isn't - * the same as "empty object present". + * "key absent" already means "optional object/array absent", which + * isn't the same as "empty object/array present". * * Additional syntax for use with an implied key: * @@ -51,17 +53,43 @@ * val-no-key =3D / [^,]* / * * where no-key is syntactic sugar for implied-key=3Dval-no-key. - * - * TODO support lists */ =20 #include "qemu/osdep.h" #include "qapi/error.h" #include "qapi/qmp/qstring.h" #include "qapi/util.h" +#include "qemu/cutils.h" #include "qemu/option.h" =20 /* + * Convert @key to a list index. + * Convert all leading digits to a (non-negative) number, capped at + * INT_MAX. + * If @end is non-null, assign a pointer to the first character after + * the number to *@end. + * Else, fail if any characters follow. + * On success, return the converted number. + * On failure, return a negative value. + * Note: since only digits are converted, no two keys can map to the + * same number, except by overflow to INT_MAX. + */ +static int key_to_index(const char *key, const char **end) +{ + int ret; + unsigned long index; + + if (*key < '0' || *key > '9') { + return -EINVAL; + } + ret =3D qemu_strtoul(key, end, 10, &index); + if (ret) { + return ret =3D=3D -ERANGE ? INT_MAX : ret; + } + return index <=3D INT_MAX ? index : INT_MAX; +} + +/* * Ensure @cur maps @key_in_cur the right way. * If @value is null, it needs to map to a QDict, else to this * QString. @@ -113,7 +141,7 @@ static const char *keyval_parse_one(QDict *qdict, const= char *params, const char *implied_key, Error **errp) { - const char *key, *key_end, *s; + const char *key, *key_end, *s, *end; size_t len; char key_in_cur[128]; QDict *cur; @@ -137,8 +165,13 @@ static const char *keyval_parse_one(QDict *qdict, cons= t char *params, cur =3D qdict; s =3D key; for (;;) { - ret =3D parse_qapi_name(s, false); - len =3D ret < 0 ? 0 : ret; + /* Want a key index (unless it's first) or a QAPI name */ + if (s !=3D key && key_to_index(s, &end) >=3D 0) { + len =3D end - s; + } else { + ret =3D parse_qapi_name(s, false); + len =3D ret < 0 ? 0 : ret; + } assert(s + len <=3D key_end); if (!len || (s + len < key_end && s[len] !=3D '.')) { assert(key !=3D implied_key); @@ -205,6 +238,119 @@ static const char *keyval_parse_one(QDict *qdict, con= st char *params, return s; } =20 +static char *reassemble_key(GSList *key) +{ + GString *s =3D g_string_new(""); + GSList *p; + + for (p =3D key; p; p =3D p->next) { + g_string_prepend_c(s, '.'); + g_string_prepend(s, (char *)p->data); + } + + return g_string_free(s, FALSE); +} + +/* + * Listify @cur recursively. + * Replace QDicts whose keys are all valid list indexes by QLists. + * @key_of_cur is the list of key fragments leading up to @cur. + * On success, return either @cur or its replacement. + * On failure, store an error through @errp and return NULL. + */ +static QObject *keyval_listify(QDict *cur, GSList *key_of_cur, Error **err= p) +{ + GSList key_node; + bool has_index, has_member; + const QDictEntry *ent; + QDict *qdict; + QObject *val; + char *key; + size_t nelt; + QObject **elt; + int index, i; + QList *list; + + key_node.next =3D key_of_cur; + + /* + * Recursively listify @cur's members, and figure out whether @cur + * itself is to be listified. + */ + has_index =3D false; + has_member =3D false; + for (ent =3D qdict_first(cur); ent; ent =3D qdict_next(cur, ent)) { + if (key_to_index(ent->key, NULL) >=3D 0) { + has_index =3D true; + } else { + has_member =3D true; + } + + qdict =3D qobject_to_qdict(ent->value); + if (!qdict) { + continue; + } + + key_node.data =3D ent->key; + val =3D keyval_listify(qdict, &key_node, errp); + if (!val) { + return NULL; + } + if (val !=3D ent->value) { + qdict_put_obj(cur, ent->key, val); + } + } + + if (has_index && has_member) { + key =3D reassemble_key(key_of_cur); + error_setg(errp, "Parameters '%s*' used inconsistently", key); + g_free(key); + return NULL; + } + if (!has_index) { + return QOBJECT(cur); + } + + /* Copy @cur's values to @elt[] */ + nelt =3D qdict_size(cur); + elt =3D g_new0(QObject *, nelt); + for (ent =3D qdict_first(cur); ent; ent =3D qdict_next(cur, ent)) { + index =3D key_to_index(ent->key, NULL); + assert(index >=3D 0); + /* + * We iterate @nelt times. Because the dictionary keys are + * distinct, the indexes are also distinct (key_to_index() + * ensures it). If we get one exceeding @nelt here, we will + * leave a hole in @elt[], triggering the error in the next + * loop. + * + * Well, I lied. key_to_index() can return INT_MAX multiple + * times, but INT_MAX surely exceeds @nelt. + */ + if ((size_t)index >=3D nelt) { + continue; + } + assert(!elt[index]); + elt[index] =3D ent->value; + qobject_incref(elt[index]); + } + + /* Make a list from @elt[] */ + list =3D qlist_new(); + for (i =3D 0; i < nelt; i++) { + if (!elt[i]) { + key =3D reassemble_key(key_of_cur); + error_setg(errp, "Parameter '%s%d' missing", key, i); + g_free(key); + QDECREF(list); + return NULL; + } + qlist_append_obj(list, elt[i]); + } + + return QOBJECT(list); +} + /* * Parse @params in QEMU's traditional KEY=3DVALUE,... syntax. * If @implied_key, the first KEY=3D can be omitted. @implied_key is @@ -216,6 +362,7 @@ QDict *keyval_parse(const char *params, const char *imp= lied_key, Error **errp) { QDict *qdict =3D qdict_new(); + QObject *listified; const char *s; =20 s =3D params; @@ -228,5 +375,11 @@ QDict *keyval_parse(const char *params, const char *im= plied_key, implied_key =3D NULL; } =20 + listified =3D keyval_listify(qdict, NULL, errp); + if (!listified) { + QDECREF(qdict); + return NULL; + } + assert(listified =3D=3D QOBJECT(qdict)); return qdict; } --=20 2.7.4