[PATCH 14/20] qapi: fix non-compliant JSON examples

John Snow posted 20 patches 6 months, 2 weeks ago
Maintainers: Markus Armbruster <armbru@redhat.com>, Michael Roth <michael.roth@amd.com>, Peter Maydell <peter.maydell@linaro.org>, Eric Blake <eblake@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>, Igor Mammedov <imammedo@redhat.com>, Ani Sinha <anisinha@redhat.com>, Gerd Hoffmann <kraxel@redhat.com>, "Marc-André Lureau" <marcandre.lureau@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, Eduardo Habkost <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Yanan Wang <wangyanan55@huawei.com>, Peter Xu <peterx@redhat.com>, Fabiano Rosas <farosas@suse.de>, Jason Wang <jasowang@redhat.com>, Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>, Jiri Pirko <jiri@resnulli.us>, Stefan Berger <stefanb@linux.vnet.ibm.com>, Stefan Hajnoczi <stefanha@redhat.com>, Mads Ynddal <mads@ynddal.dk>, Lukas Straub <lukasstraub2@web.de>, Konstantin Kostiuk <kkostiuk@redhat.com>
[PATCH 14/20] qapi: fix non-compliant JSON examples
Posted by John Snow 6 months, 2 weeks ago
If we parse all examples as QMP, we need them to conform to a standard
so that they render correctly. Once the QMP lexer is active for
examples, these will produce warning messages and fail the build.

The QMP lexer still supports elisions, but they must be represented as
the value "...", so two examples have been adjusted to support that
format here.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 qapi/control.json   | 3 ++-
 qapi/machine.json   | 2 +-
 qapi/migration.json | 2 +-
 qapi/misc.json      | 3 ++-
 qapi/net.json       | 6 +++---
 qapi/rocker.json    | 2 +-
 qapi/ui.json        | 2 +-
 7 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/qapi/control.json b/qapi/control.json
index 6bdbf077c2e..10c906fa0e7 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -145,7 +145,8 @@
 #             },
 #             {
 #                "name":"system_powerdown"
-#             }
+#             },
+#             ...
 #          ]
 #        }
 #
diff --git a/qapi/machine.json b/qapi/machine.json
index bce6e1bbc41..64a77557571 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1057,7 +1057,7 @@
 #            "vcpus-count": 1 },
 #          { "props": { "core-id": 0 }, "type": "POWER8-spapr-cpu-core",
 #            "vcpus-count": 1, "qom-path": "/machine/unattached/device[0]"}
-#        ]}'
+#        ]}
 #
 #     For pc machine type started with -smp 1,maxcpus=2:
 #
diff --git a/qapi/migration.json b/qapi/migration.json
index a351fd37143..89047d46c7c 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -2078,7 +2078,7 @@
 # Example:
 #
 #     -> {"execute": "calc-dirty-rate", "arguments": {"calc-time": 1,
-#                                                     'sample-pages': 512} }
+#                                                     "sample-pages": 512} }
 #     <- { "return": {} }
 #
 #     Measure dirty rate using dirty bitmap for 500 milliseconds:
diff --git a/qapi/misc.json b/qapi/misc.json
index ec30e5c570a..4b41e15dcd4 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -287,7 +287,8 @@
 #
 # Example:
 #
-#     -> { "execute": "get-win32-socket", "arguments": { "info": "abcd123..", fdname": "skclient" } }
+#     -> { "execute": "get-win32-socket",
+#          "arguments": { "info": "abcd123..", "fdname": "skclient" } }
 #     <- { "return": {} }
 ##
 { 'command': 'get-win32-socket', 'data': {'info': 'str', 'fdname': 'str'}, 'if': 'CONFIG_WIN32' }
diff --git a/qapi/net.json b/qapi/net.json
index 0f5a259475e..c19df435a53 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -1003,9 +1003,9 @@
 #
 # Example:
 #
-#     <- { 'event': 'NETDEV_STREAM_DISCONNECTED',
-#          'data': {'netdev-id': 'netdev0'},
-#          'timestamp': {'seconds': 1663330937, 'microseconds': 526695} }
+#     <- { "event": "NETDEV_STREAM_DISCONNECTED",
+#          "data": {"netdev-id": "netdev0"},
+#          "timestamp": {"seconds": 1663330937, "microseconds": 526695} }
 ##
 { 'event': 'NETDEV_STREAM_DISCONNECTED',
   'data': { 'netdev-id': 'str' } }
diff --git a/qapi/rocker.json b/qapi/rocker.json
index 5635cf174fd..f5225eb62cc 100644
--- a/qapi/rocker.json
+++ b/qapi/rocker.json
@@ -250,7 +250,7 @@
 #                       "action": {"goto-tbl": 10},
 #                       "mask": {"in-pport": 4294901760}
 #                      },
-#                      {...more...},
+#                      {...},
 #        ]}
 ##
 { 'command': 'query-rocker-of-dpa-flows',
diff --git a/qapi/ui.json b/qapi/ui.json
index f610bce118a..c12f5292571 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -361,7 +361,7 @@
 #                    "channel-id": 0,
 #                    "tls": false
 #                 },
-#                 [ ... more channels follow ... ]
+#                 ...
 #              ]
 #           }
 #        }
-- 
2.44.0
Re: [PATCH 14/20] qapi: fix non-compliant JSON examples
Posted by Markus Armbruster 5 months, 1 week ago
John Snow <jsnow@redhat.com> writes:

> If we parse all examples as QMP, we need them to conform to a standard
> so that they render correctly. Once the QMP lexer is active for
> examples, these will produce warning messages and fail the build.
>
> The QMP lexer still supports elisions, but they must be represented as
> the value "...", so two examples have been adjusted to support that
> format here.

I think this could use a bit more context.  I believe you're referring
to docs/sphinx/qmp_lexer.py.  It describes itself as "a Sphinx extension
that provides a QMP lexer for code blocks."

"If we parse all examples as QMP" and "Once the QMP lexer is active for
examples" suggests we're *not* using it for (some?) examples.  So what
are we using it for?

> Signed-off-by: John Snow <jsnow@redhat.com>

Patch looks lovely.

Hat tip to Victor Toso, who fixed up most examples two years ago.  Back
then we couldn't decide how to do elisions, so we left some unfixed.
Re: [PATCH 14/20] qapi: fix non-compliant JSON examples
Posted by John Snow 5 months, 1 week ago
On Fri, Jun 14, 2024, 6:55 AM Markus Armbruster <armbru@redhat.com> wrote:

> John Snow <jsnow@redhat.com> writes:
>
> > If we parse all examples as QMP, we need them to conform to a standard
> > so that they render correctly. Once the QMP lexer is active for
> > examples, these will produce warning messages and fail the build.
> >
> > The QMP lexer still supports elisions, but they must be represented as
> > the value "...", so two examples have been adjusted to support that
> > format here.
>
> I think this could use a bit more context.  I believe you're referring
> to docs/sphinx/qmp_lexer.py.  It describes itself as "a Sphinx extension
> that provides a QMP lexer for code blocks."
>

That's our guy! I explain its use a bit more in ... some other patch,
somewhere...


> "If we parse all examples as QMP" and "Once the QMP lexer is active for
> examples" suggests we're *not* using it for (some?) examples.  So what
> are we using it for?
>

My incremental backup doc makes use of it; you have to "opt in" to using
the QMP lexer instead of the generic lexer.

The example conversion patch later in this series opts all of the qapi docs
into using it.

((Later, it's possible to make "Example::" choose the QMP lexer by default
on any of our generated QMP pages. (and opting out would require explicit
code-block syntax with the lexer of choice named.)))


> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> Patch looks lovely.
>
> Hat tip to Victor Toso, who fixed up most examples two years ago.  Back
> then we couldn't decide how to do elisions, so we left some unfixed.
>

Sorry I didn't chime in back then! "..." is arbitrary, but it's what we
already use for the qmp lexer and in the incremental backup/bitmap docs, so
I figured consistency was good.

The QMP lexer has syntax support for ->, <- and ... and otherwise requires
the examples to be valid JSON. It doesn't understand grammar, though, so
it's kind of "dumb", but this is one small protection.

>
Re: [PATCH 14/20] qapi: fix non-compliant JSON examples
Posted by Markus Armbruster 5 months, 1 week ago
John Snow <jsnow@redhat.com> writes:

> On Fri, Jun 14, 2024, 6:55 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > If we parse all examples as QMP, we need them to conform to a standard
>> > so that they render correctly. Once the QMP lexer is active for
>> > examples, these will produce warning messages and fail the build.
>> >
>> > The QMP lexer still supports elisions, but they must be represented as
>> > the value "...", so two examples have been adjusted to support that
>> > format here.
>>
>> I think this could use a bit more context.  I believe you're referring
>> to docs/sphinx/qmp_lexer.py.  It describes itself as "a Sphinx extension
>> that provides a QMP lexer for code blocks."
>>
>
> That's our guy! I explain its use a bit more in ... some other patch,
> somewhere...
>
>
>> "If we parse all examples as QMP" and "Once the QMP lexer is active for
>> examples" suggests we're *not* using it for (some?) examples.  So what
>> are we using it for?
>>
>
> My incremental backup doc makes use of it; you have to "opt in" to using
> the QMP lexer instead of the generic lexer.

The ".. code-block:: QMP" lines I can see in a few files?  Namely:

    docs/devel/s390-cpu-topology.rst
    docs/interop/bitmaps.rst
    docs/interop/qmp-spec.rst

> The example conversion patch later in this series opts all of the qapi docs
> into using it.
>
> ((Later, it's possible to make "Example::" choose the QMP lexer by default
> on any of our generated QMP pages. (and opting out would require explicit
> code-block syntax with the lexer of choice named.)))

The patch does two related things:

1. Fix invalid JSON.  Doesn't need justification.

2. Normalize elisions to ...  You pick ... because that's what
   qmp_lexer.py wants.

Doing both in one patch is fine.

Perhaps

    qapi: Fix invalid JSON in examples, and normalize elisions

    A few examples elide part of the output.  Normalize elision to
    exactly ...  Together with the JSON fixing, this enables use of
    docs/sphinx/qmp_lexer.py to highlight the examples in a later patch.

>> > Signed-off-by: John Snow <jsnow@redhat.com>
>>
>> Patch looks lovely.
>>
>> Hat tip to Victor Toso, who fixed up most examples two years ago.  Back
>> then we couldn't decide how to do elisions, so we left some unfixed.
>>
>
> Sorry I didn't chime in back then! "..." is arbitrary, but it's what we
> already use for the qmp lexer and in the incremental backup/bitmap docs, so
> I figured consistency was good.

It is.

> The QMP lexer has syntax support for ->, <- and ... and otherwise requires
> the examples to be valid JSON. It doesn't understand grammar, though, so
> it's kind of "dumb", but this is one small protection.