From nobody Sun Feb 8 13:53:08 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488321120377747.727307324238; Tue, 28 Feb 2017 14:32:00 -0800 (PST) Received: from localhost ([::1]:37406 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ciqJK-0000rR-TC for importer@patchew.org; Tue, 28 Feb 2017 17:31:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33601) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ciqDt-00056b-UO for qemu-devel@nongnu.org; Tue, 28 Feb 2017 17:26:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ciqDq-0000Ww-Uj for qemu-devel@nongnu.org; Tue, 28 Feb 2017 17:26:21 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50980) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ciqDq-0000Vn-Mq for qemu-devel@nongnu.org; Tue, 28 Feb 2017 17:26:18 -0500 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E07983B71F for ; Tue, 28 Feb 2017 22:26:18 +0000 (UTC) Received: from blackfin.pond.sub.org (ovpn-116-55.ams2.redhat.com [10.36.116.55]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SMQHrH017505 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 28 Feb 2017 17:26:18 -0500 Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id B39491138650; Tue, 28 Feb 2017 23:26:15 +0100 (CET) From: Markus Armbruster To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 23:25:54 +0100 Message-Id: <1488320775-9849-4-git-send-email-armbru@redhat.com> In-Reply-To: <1488320775-9849-1-git-send-email-armbru@redhat.com> References: <1488320775-9849-1-git-send-email-armbru@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 28 Feb 2017 22:26:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 03/24] keyval: New keyval_parse() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" keyval_parse() parses KEY=3DVALUE,... into a QDict. Works like qemu_opts_parse(), except: * Returns a QDict instead of a QemuOpts (d'oh). * Supports nesting, unlike QemuOpts: a KEY is split into key fragments at '.' (dotted key convention; the block layer does something similar on top of QemuOpts). The key fragments are QDict keys, and the last one's value is updated to VALUE. * Each key fragment may be up to 127 bytes long. qemu_opts_parse() limits the entire key to 127 bytes. * Overlong key fragments are rejected. qemu_opts_parse() silently truncates them. * Empty key fragments are rejected. qemu_opts_parse() happily accepts empty keys. * It does not store the returned value. qemu_opts_parse() stores it in the QemuOptsList. * It does not treat parameter "id" specially. qemu_opts_parse() ignores all but the first "id", and fails when its value isn't id_wellformed(), or duplicate (a QemuOpts with the same ID is already stored). It also screws up when a value contains ",id=3D". * Implied value is not supported. qemu_opts_parse() desugars "foo" to "foo=3Don", and "nofoo" to "foo=3Doff". * An implied key's value can't be empty, and can't contain ','. I intend to grow this into a saner replacement for QemuOpts. It'll take time, though. Note: keyval_parse() provides no way to do lists, and its key syntax is incompatible with the __RFQDN_ prefix convention for downstream extensions, because it blindly splits at '.', even in __RFQDN_. Both issues will be addressed later in the series. Signed-off-by: Markus Armbruster Message-Id: <1488317230-26248-4-git-send-email-armbru@redhat.com> --- include/qemu/option.h | 3 + tests/.gitignore | 1 + tests/Makefile.include | 3 + tests/test-keyval.c | 180 ++++++++++++++++++++++++++++++++++++++ util/Makefile.objs | 1 + util/keyval.c | 231 +++++++++++++++++++++++++++++++++++++++++++++= ++++ 6 files changed, 419 insertions(+) create mode 100644 tests/test-keyval.c create mode 100644 util/keyval.c diff --git a/include/qemu/option.h b/include/qemu/option.h index e786df0..f7338db 100644 --- a/include/qemu/option.h +++ b/include/qemu/option.h @@ -141,4 +141,7 @@ void qemu_opts_print_help(QemuOptsList *list); void qemu_opts_free(QemuOptsList *list); QemuOptsList *qemu_opts_append(QemuOptsList *dst, QemuOptsList *list); =20 +QDict *keyval_parse(const char *params, const char *implied_key, + Error **errp); + #endif diff --git a/tests/.gitignore b/tests/.gitignore index dc37519..30b7740 100644 --- a/tests/.gitignore +++ b/tests/.gitignore @@ -47,6 +47,7 @@ test-io-channel-file.txt test-io-channel-socket test-io-channel-tls test-io-task +test-keyval test-logging test-mul64 test-opts-visitor diff --git a/tests/Makefile.include b/tests/Makefile.include index 35d07e4..e3b4f27 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -94,6 +94,8 @@ check-unit-y +=3D tests/check-qom-proplist$(EXESUF) gcov-files-check-qom-proplist-y =3D qom/object.c check-unit-y +=3D tests/test-qemu-opts$(EXESUF) gcov-files-test-qemu-opts-y =3D util/qemu-option.c +check-unit-y +=3D tests/test-keyval$(EXESUF) +gcov-files-test-keyval-y =3D util/keyval.c check-unit-y +=3D tests/test-write-threshold$(EXESUF) gcov-files-test-write-threshold-y =3D block/write-threshold.c check-unit-y +=3D tests/test-crypto-hash$(EXESUF) @@ -720,6 +722,7 @@ tests/vhost-user-test$(EXESUF): tests/vhost-user-test.o= $(test-util-obj-y) \ $(chardev-obj-y) tests/qemu-iotests/socket_scm_helper$(EXESUF): tests/qemu-iotests/socket_s= cm_helper.o tests/test-qemu-opts$(EXESUF): tests/test-qemu-opts.o $(test-util-obj-y) +tests/test-keyval$(EXESUF): tests/test-keyval.o $(test-util-obj-y) tests/test-write-threshold$(EXESUF): tests/test-write-threshold.o $(test-b= lock-obj-y) tests/test-netfilter$(EXESUF): tests/test-netfilter.o $(qtest-obj-y) tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-= y) diff --git a/tests/test-keyval.c b/tests/test-keyval.c new file mode 100644 index 0000000..27f6625 --- /dev/null +++ b/tests/test-keyval.c @@ -0,0 +1,180 @@ +/* + * Unit tests for parsing of KEY=3DVALUE,... strings + * + * Copyright (C) 2017 Red Hat Inc. + * + * Authors: + * Markus Armbruster , + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/option.h" + +static void test_keyval_parse(void) +{ + Error *err =3D NULL; + QDict *qdict, *sub_qdict; + char long_key[129]; + char *params; + + /* Nothing */ + qdict =3D keyval_parse("", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 0); + QDECREF(qdict); + + /* Empty key (qemu_opts_parse() accepts this) */ + qdict =3D keyval_parse("=3Dval", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Empty key fragment */ + qdict =3D keyval_parse(".", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + qdict =3D keyval_parse("key.", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Overlong key */ + memset(long_key, 'a', 127); + long_key[127] =3D 'z'; + long_key[128] =3D 0; + params =3D g_strdup_printf("k.%s=3Dv", long_key); + qdict =3D keyval_parse(params + 2, NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Overlong key fragment */ + qdict =3D keyval_parse(params, NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + g_free(params); + + /* Long key (qemu_opts_parse() accepts and truncates silently) */ + params =3D g_strdup_printf("k.%s=3Dv", long_key + 1); + qdict =3D keyval_parse(params + 2, NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, long_key + 1), =3D=3D, "v"); + QDECREF(qdict); + + /* Long key fragment */ + qdict =3D keyval_parse(params, NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 1); + sub_qdict =3D qdict_get_qdict(qdict, "k"); + g_assert(sub_qdict); + g_assert_cmpuint(qdict_size(sub_qdict), =3D=3D, 1); + g_assert_cmpstr(qdict_get_try_str(sub_qdict, long_key + 1), =3D=3D, "v= "); + QDECREF(qdict); + g_free(params); + + /* Multiple keys, last one wins */ + qdict =3D keyval_parse("a=3D1,b=3D2,,x,a=3D3", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 2); + g_assert_cmpstr(qdict_get_try_str(qdict, "a"), =3D=3D, "3"); + g_assert_cmpstr(qdict_get_try_str(qdict, "b"), =3D=3D, "2,x"); + QDECREF(qdict); + + /* Even when it doesn't in qemu_opts_parse() */ + qdict =3D keyval_parse("id=3Dfoo,id=3Dbar", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, "id"), =3D=3D, "bar"); + QDECREF(qdict); + + /* Dotted keys */ + qdict =3D keyval_parse("a.b.c=3D1,a.b.c=3D2,d=3D3", NULL, &error_abort= ); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 2); + sub_qdict =3D qdict_get_qdict(qdict, "a"); + g_assert(sub_qdict); + g_assert_cmpuint(qdict_size(sub_qdict), =3D=3D, 1); + sub_qdict =3D qdict_get_qdict(sub_qdict, "b"); + g_assert(sub_qdict); + g_assert_cmpuint(qdict_size(sub_qdict), =3D=3D, 1); + g_assert_cmpstr(qdict_get_try_str(sub_qdict, "c"), =3D=3D, "2"); + g_assert_cmpstr(qdict_get_try_str(qdict, "d"), =3D=3D, "3"); + QDECREF(qdict); + + /* Inconsistent dotted keys */ + qdict =3D keyval_parse("a.b=3D1,a=3D2", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + qdict =3D keyval_parse("a.b=3D1,a.b.c=3D2", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Trailing comma is ignored */ + qdict =3D keyval_parse("x=3Dy,", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, "x"), =3D=3D, "y"); + QDECREF(qdict); + + /* Except when it isn't */ + qdict =3D keyval_parse(",", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Value containing ,id=3D not misinterpreted as qemu_opts_parse() doe= s */ + qdict =3D keyval_parse("x=3D,,id=3Dbar", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, "x"), =3D=3D, ",id=3Dbar"); + QDECREF(qdict); + + /* Anti-social ID is left to caller (qemu_opts_parse() rejects it) */ + qdict =3D keyval_parse("id=3D666", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, "id"), =3D=3D, "666"); + QDECREF(qdict); + + /* Implied value not supported (unlike qemu_opts_parse()) */ + qdict =3D keyval_parse("an,noaus,noaus=3D", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Implied value, key "no" (qemu_opts_parse(): negated empty key) */ + qdict =3D keyval_parse("no", NULL, &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Implied key */ + qdict =3D keyval_parse("an,aus=3Doff,noaus=3D", "implied", &error_abor= t); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 3); + g_assert_cmpstr(qdict_get_try_str(qdict, "implied"), =3D=3D, "an"); + g_assert_cmpstr(qdict_get_try_str(qdict, "aus"), =3D=3D, "off"); + g_assert_cmpstr(qdict_get_try_str(qdict, "noaus"), =3D=3D, ""); + QDECREF(qdict); + + /* Implied dotted key */ + qdict =3D keyval_parse("val", "eins.zwei", &error_abort); + g_assert_cmpuint(qdict_size(qdict), =3D=3D, 1); + sub_qdict =3D qdict_get_qdict(qdict, "eins"); + g_assert(sub_qdict); + g_assert_cmpuint(qdict_size(sub_qdict), =3D=3D, 1); + g_assert_cmpstr(qdict_get_try_str(sub_qdict, "zwei"), =3D=3D, "val"); + QDECREF(qdict); + + /* Implied key with empty value (qemu_opts_parse() accepts this) */ + qdict =3D keyval_parse(",", "implied", &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Likewise (qemu_opts_parse(): implied key with comma value) */ + qdict =3D keyval_parse(",,,a=3D1", "implied", &err); + error_free_or_abort(&err); + g_assert(!qdict); + + /* Empty key is not an implied key */ + qdict =3D keyval_parse("=3Dval", "implied", &err); + error_free_or_abort(&err); + g_assert(!qdict); +} + +int main(int argc, char *argv[]) +{ + g_test_init(&argc, &argv, NULL); + g_test_add_func("/keyval/keyval_parse", test_keyval_parse); + g_test_run(); + return 0; +} diff --git a/util/Makefile.objs b/util/Makefile.objs index bc629e2..06366b5 100644 --- a/util/Makefile.objs +++ b/util/Makefile.objs @@ -24,6 +24,7 @@ util-obj-y +=3D error.o qemu-error.o util-obj-y +=3D id.o util-obj-y +=3D iov.o qemu-config.o qemu-sockets.o uri.o notify.o util-obj-y +=3D qemu-option.o qemu-progress.o +util-obj-y +=3D keyval.o util-obj-y +=3D hexdump.o util-obj-y +=3D crc32c.o util-obj-y +=3D uuid.o diff --git a/util/keyval.c b/util/keyval.c new file mode 100644 index 0000000..990126f --- /dev/null +++ b/util/keyval.c @@ -0,0 +1,231 @@ +/* + * Parsing KEY=3DVALUE,... strings + * + * Copyright (C) 2017 Red Hat Inc. + * + * Authors: + * Markus Armbruster , + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +/* + * KEY=3DVALUE,... syntax: + * + * key-vals =3D [ key-val { ',' key-val } [ ',' ] ] + * key-val =3D key '=3D' val + * key =3D key-fragment { '.' key-fragment } + * key-fragment =3D / [^=3D,.]* / + * val =3D { / [^,]* / | ',,' } + * + * Semantics defined by reduction to JSON: + * + * key-vals defines a tree of objects rooted at R + * where for each key-val =3D key-fragment . ... =3D val in key-vals + * R op key-fragment op ... =3D val' + * where (left-associative) op is member reference L.key-fragment + * val' is val with ',,' replaced by ',' + * and only R may be empty. + * + * Duplicate keys are permitted; all but the last one are ignored. + * + * The equations must have a solution. Counter-example: a.b=3D1,a=3D2 + * doesn't have one, because R.a must be an object to satisfy a.b=3D1 + * and a string to satisfy a=3D2. + * + * The length of any key-fragment must be between 1 and 127. + * + * Design flaw: there is no way to denote an empty non-root object. + * While interpreting "key absent" as empty object seems natural + * (removing a key-val from the input string removes the member when + * there are more, so why not when it's the last), it doesn't work: + * "key absent" already means "optional object absent", which isn't + * the same as "empty object present". + * + * Additional syntax for use with an implied key: + * + * key-vals-ik =3D val-no-key [ ',' key-vals ] + * val-no-key =3D / [^=3D,]* / + * + * where no-key is syntactic sugar for implied-key=3Dval-no-key. + * + * TODO support lists + * TODO support key-fragment with __RFQDN_ prefix (downstream extensions) + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qapi/qmp/qstring.h" +#include "qemu/option.h" + +/* + * Ensure @cur maps @key_in_cur the right way. + * If @value is null, it needs to map to a QDict, else to this + * QString. + * If @cur doesn't have @key_in_cur, put an empty QDict or @value, + * respectively. + * Else, if it needs to map to a QDict, and already does, do nothing. + * Else, if it needs to map to this QString, and already maps to a + * QString, replace it by @value. + * Else, fail because we have conflicting needs on how to map + * @key_in_cur. + * In any case, take over the reference to @value, i.e. if the caller + * wants to hold on to a reference, it needs to QINCREF(). + * Use @key up to @key_cursor to identify the key in error messages. + * On success, return the mapped value. + * On failure, store an error through @errp and return NULL. + */ +static QObject *keyval_parse_put(QDict *cur, + const char *key_in_cur, QString *value, + const char *key, const char *key_cursor, + Error **errp) +{ + QObject *old, *new; + + old =3D qdict_get(cur, key_in_cur); + if (old) { + if (qobject_type(old) !=3D (value ? QTYPE_QSTRING : QTYPE_QDICT)) { + error_setg(errp, "Parameters '%.*s.*' used inconsistently", + (int)(key_cursor - key), key); + QDECREF(value); + return NULL; + } + if (!value) { + return old; /* already QDict, do nothing */ + } + new =3D QOBJECT(value); /* replacement */ + } else { + new =3D QOBJECT(value) ?: QOBJECT(qdict_new()); + } + qdict_put_obj(cur, key_in_cur, new); + return new; +} + +/* + * Parse one KEY=3DVALUE from @params, store result in @qdict. + * The first fragment of KEY applies to @qdict. Subsequent fragments + * apply to nested QDicts, which are created on demand. @implied_key + * is as in keyval_parse(). + * On success, return a pointer to the next KEY=3DVALUE, or else to '\0'. + * On failure, return NULL. + */ +static const char *keyval_parse_one(QDict *qdict, const char *params, + const char *implied_key, + Error **errp) +{ + const char *key, *key_end, *s; + size_t len; + char key_in_cur[128]; + QDict *cur; + QObject *next; + QString *val; + + key =3D params; + len =3D strcspn(params, "=3D,"); + if (implied_key && len && key[len] !=3D '=3D') { + /* Desugar implied key */ + key =3D implied_key; + len =3D strlen(implied_key); + } + key_end =3D key + len; + + /* + * Loop over key fragments: @s points to current fragment, it + * applies to @cur. @key_in_cur[] holds the previous fragment. + */ + cur =3D qdict; + s =3D key; + for (;;) { + for (len =3D 0; s + len < key_end && s[len] !=3D '.'; len++) { + } + if (!len) { + assert(key !=3D implied_key); + error_setg(errp, "Invalid parameter '%.*s'", + (int)(key_end - key), key); + return NULL; + } + if (len >=3D sizeof(key_in_cur)) { + assert(key !=3D implied_key); + error_setg(errp, "Parameter%s '%.*s' is too long", + s !=3D key || s + len !=3D key_end ? " fragment" : = "", + (int)len, s); + return NULL; + } + + if (s !=3D key) { + next =3D keyval_parse_put(cur, key_in_cur, NULL, + key, s - 1, errp); + if (!next) { + return NULL; + } + cur =3D qobject_to_qdict(next); + assert(cur); + } + + memcpy(key_in_cur, s, len); + key_in_cur[len] =3D 0; + s +=3D len; + + if (*s !=3D '.') { + break; + } + s++; + } + + if (key =3D=3D implied_key) { + assert(!*s); + s =3D params; + } else { + if (*s !=3D '=3D') { + error_setg(errp, "Expected '=3D' after parameter '%.*s'", + (int)(s - key), key); + return NULL; + } + s++; + } + + val =3D qstring_new(); + for (;;) { + if (!*s) { + break; + } else if (*s =3D=3D ',') { + s++; + if (*s !=3D ',') { + break; + } + } + qstring_append_chr(val, *s++); + } + + if (!keyval_parse_put(cur, key_in_cur, val, key, key_end, errp)) { + return NULL; + } + return s; +} + +/* + * Parse @params in QEMU's traditional KEY=3DVALUE,... syntax. + * If @implied_key, the first KEY=3D can be omitted. @implied_key is + * implied then, and VALUE can't be empty or contain ',' or '=3D'. + * On success, return a dictionary of the parsed keys and values. + * On failure, store an error through @errp and return NULL. + */ +QDict *keyval_parse(const char *params, const char *implied_key, + Error **errp) +{ + QDict *qdict =3D qdict_new(); + const char *s; + + s =3D params; + while (*s) { + s =3D keyval_parse_one(qdict, s, implied_key, errp); + if (!s) { + QDECREF(qdict); + return NULL; + } + implied_key =3D NULL; + } + + return qdict; +} --=20 2.7.4