[PATCH 00/12] qapi: add formal "intro" section

John Snow posted 12 patches 1 month ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20260423220022.2180059-1-jsnow@redhat.com
Maintainers: Markus Armbruster <armbru@redhat.com>, Michael Roth <michael.roth@amd.com>, Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>, John Snow <jsnow@redhat.com>, Peter Maydell <peter.maydell@linaro.org>, Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, Eric Blake <eblake@redhat.com>, Richard Henderson <richard.henderson@linaro.org>, Paolo Bonzini <pbonzini@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, "Michael S. Tsirkin" <mst@redhat.com>, Igor Mammedov <imammedo@redhat.com>, Ani Sinha <anisinha@redhat.com>, Gerd Hoffmann <kraxel@redhat.com>, "Marc-André Lureau" <marcandre.lureau@redhat.com>
There is a newer version of this series
docs/devel/qapi-code-gen.rst            |  17 +-
docs/sphinx/qapidoc.py                  |  52 +++---
qapi/accelerator.json                   |  24 +--
qapi/acpi-hest.json                     |   5 +-
qapi/acpi.json                          |  23 +--
qapi/audio.json                         | 112 ++++---------
.editorconfig                           |   1 +
scripts/qapi/parser.py                  | 204 ++++++++++++++----------
tests/qapi-schema/doc-good.out          |  30 ++--
tests/qapi-schema/doc-missing-colon.err |   2 +-
tests/qapi-schema/test-qapi.py          |  15 +-
11 files changed, 239 insertions(+), 246 deletions(-)
[PATCH 00/12] qapi: add formal "intro" section
Posted by John Snow 1 month ago
Hiya, this is a series that explores a potential syntax for a
designated "Intro" section. Markus knows why I want this, but for
everyone else: a designated "Introduction" section is useful for the
desired "inliner" feature for the new QAPI doc system. Commits explain
a bit more. This is prep work and doesn't really change anything
tangibly except source code syntax for the QAPI docs.

It is designed so that this conversion can happen incrementally with
no actual difference to the rendered manuals, so each QAPI module can
be converted one at a time for easier review and merging in an
arbitrary order.

This series demonstrates conversion of just four modules; if I'm given
a thumbs up, I will convert the rest of QAPI, one module (or
maintainer stanza) per patch like how I handled adding
cross-references.

John Snow (12):
  tests/qapi: generate output in source order
  qapi/docs: remove unused QAPIDoc subsection members
  qapi/docs: make remaining subsection members "private"
  qapi/docs: add "Intro" section
  qapi/docs: adjust stub member insertion algorithm
  qapi/docs: remove implicit Plain section
  qapi/docs: add "Intro" section parsing
  qapi/docs: Add rendering for INTRO sections
  qapi: convert intro sections for accelerator.json
  qapi: convert intro sections for acpi-hest.json
  qapi: convert intro sections for acpi.json
  qapi: convert intro sections for audio.json

 docs/devel/qapi-code-gen.rst            |  17 +-
 docs/sphinx/qapidoc.py                  |  52 +++---
 qapi/accelerator.json                   |  24 +--
 qapi/acpi-hest.json                     |   5 +-
 qapi/acpi.json                          |  23 +--
 qapi/audio.json                         | 112 ++++---------
 .editorconfig                           |   1 +
 scripts/qapi/parser.py                  | 204 ++++++++++++++----------
 tests/qapi-schema/doc-good.out          |  30 ++--
 tests/qapi-schema/doc-missing-colon.err |   2 +-
 tests/qapi-schema/test-qapi.py          |  15 +-
 11 files changed, 239 insertions(+), 246 deletions(-)

-- 
2.53.0

Re: [PATCH 00/12] qapi: add formal "intro" section
Posted by Markus Armbruster 1 month ago
John Snow <jsnow@redhat.com> writes:

> Hiya, this is a series that explores a potential syntax for a
> designated "Intro" section. Markus knows why I want this, but for
> everyone else: a designated "Introduction" section is useful for the
> desired "inliner" feature for the new QAPI doc system. Commits explain
> a bit more. This is prep work and doesn't really change anything
> tangibly except source code syntax for the QAPI docs.

Let me elaborate a bit.

1. Why "intro"?

Generated documentation often refers to types like this:

    Command announce-self (Since: 4.0)

       Trigger generation of broadcast RARP frames to update network
       switches.  This can be useful when network bonds fail-over the
       active slave.

       Arguments:
          * The members of "AnnounceParameters".

       Example::

          -> { "execute": "announce-self",
               "arguments": {
                   "initial": 50, "max": 550, "rounds": 10, "step": 50,
                   "interfaces": ["vn2", "vn3"], "id": "bob" } }
          <- { "return": {} }

The arguments are hidden behind link "AnnounceParameters".  Following
leads to:

    Object AnnounceParameters (Since: 4.0)

       Parameters for self-announce timers

       Members:
          * initial ("int") -- Initial delay (in ms) before sending
            the first GARP/RARP announcement

          * max ("int") -- Maximum delay (in ms) between GARP/RARP
            announcement packets

          * rounds ("int") -- Number of self-announcement attempts

          * step ("int") -- Delay increase (in ms) after each self-
            announcement attempt

          * interfaces ("[string]", *optional*) -- An optional list of
            interface names, which restricts the announcement to the
            listed interfaces.  (Since 4.1)

          * id ("string", *optional*) -- A name to be used to identify
            an instance of announce-timers and to allow it to modified
            later.  Not for use as part of the migration parameters.
            (Since 4.1)

This is bad UX.  Especially when you have to chase multiple links.

John's doc inliner inlines AnnounceParameters documentation into
announce-self documentation.

However, we don't want to inline "Parameters for self-announce timers".

Definitions typically start with a short description of the thing being
defined.  John calls this "intro".  We don't want to inline these.

Syntactically, a doc comment begins with optional plain paragraphs.
These end when something else begins: argument or member description,
tagged section such as "Returns:", ...

Most of the time, these initial paragraphs are exactly the "intro".  But
not always.

Even without the inliner, we often need to know where "intro" ends.  In
the example above, "Arguments:" is auto-generated right after the
"intro", and for that, the generator needs to know where "intro" ends.

In his "qapi: enforce section ordering" series, John proposed syntax to
explicitly end "intro": a line "Details:".  He had to add it to roughly
one in 25 definition docs.

I fear adding "Details:" when needed is easily forgotten when it's so
rarely needed.  And I don't even trust myself to catch it patch review.
So we looked for clearer syntax.

This series implements one: instead of letting intro end when something
else starts, use indentation like we do for descriptions and tagged
sections.

Let's have a look at the current doc source for announce-self:

    ##
    # @announce-self:
    #
    # Trigger generation of broadcast RARP frames to update network
    # switches.  This can be useful when network bonds fail-over the
    # active slave.
    #
    # TODO: This line is a hack to separate the example from the body
    #
    # .. qmp-example::
    #
    #     -> { "execute": "announce-self",
    #          "arguments": {
    #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
    #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
    #     <- { "return": {} }
    #
    # Since: 4.0
    ##

"Intro" ends when something else starts, namely the TODO: section.  We
need this hack so the doc generator inserts the "Arguments:" part where
we want it.

John's "qapi: enforce section ordering" series replaces the TODO hack
with a "Details:" line.

In new syntax, this instead becomes:

    ##
    # @announce-self: Trigger generation of broadcast RARP frames to
    #     update network switches.  This can be useful when network bonds
    #     fail-over the active slave.
    #
    # .. qmp-example::
    #
    #     -> { "execute": "announce-self",
    #          "arguments": {
    #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
    #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
    #     <- { "return": {} }
    #
    # Since: 4.0
    ##

Now "intro" is indented, and its end is obvious.  Mistakes seem
unlikely.

We could instead do:

    ##
    # @announce-self:
    #     Trigger generation of broadcast RARP frames to update network
    #     switches.  This can be useful when network bonds fail-over the
    #     active slave.
    #
    # .. qmp-example::
    #
    #     -> { "execute": "announce-self",
    #          "arguments": {
    #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
    #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
    #     <- { "return": {} }
    #
    # Since: 4.0
    ##

I think I like this a bit better.

> It is designed so that this conversion can happen incrementally with
> no actual difference to the rendered manuals, so each QAPI module can
> be converted one at a time for easier review and merging in an
> arbitrary order.

Why not wholesale conversion?  As long as generated output stays the
same.  It does in this series, except for harmless line breaks.

> This series demonstrates conversion of just four modules; if I'm given
> a thumbs up, I will convert the rest of QAPI, one module (or
> maintainer stanza) per patch like how I handled adding
> cross-references.
Re: [PATCH 00/12] qapi: add formal "intro" section
Posted by John Snow 1 month ago
On Fri, Apr 24, 2026, 8:50 AM Markus Armbruster <armbru@redhat.com> wrote:

> John Snow <jsnow@redhat.com> writes:
>
> > Hiya, this is a series that explores a potential syntax for a
> > designated "Intro" section. Markus knows why I want this, but for
> > everyone else: a designated "Introduction" section is useful for the
> > desired "inliner" feature for the new QAPI doc system. Commits explain
> > a bit more. This is prep work and doesn't really change anything
> > tangibly except source code syntax for the QAPI docs.
>
> Let me elaborate a bit.
>
> 1. Why "intro"?
>
> Generated documentation often refers to types like this:
>
>     Command announce-self (Since: 4.0)
>
>        Trigger generation of broadcast RARP frames to update network
>        switches.  This can be useful when network bonds fail-over the
>        active slave.
>
>        Arguments:
>           * The members of "AnnounceParameters".
>
>        Example::
>
>           -> { "execute": "announce-self",
>                "arguments": {
>                    "initial": 50, "max": 550, "rounds": 10, "step": 50,
>                    "interfaces": ["vn2", "vn3"], "id": "bob" } }
>           <- { "return": {} }
>
> The arguments are hidden behind link "AnnounceParameters".  Following
> leads to:
>
>     Object AnnounceParameters (Since: 4.0)
>
>        Parameters for self-announce timers
>
>        Members:
>           * initial ("int") -- Initial delay (in ms) before sending
>             the first GARP/RARP announcement
>
>           * max ("int") -- Maximum delay (in ms) between GARP/RARP
>             announcement packets
>
>           * rounds ("int") -- Number of self-announcement attempts
>
>           * step ("int") -- Delay increase (in ms) after each self-
>             announcement attempt
>
>           * interfaces ("[string]", *optional*) -- An optional list of
>             interface names, which restricts the announcement to the
>             listed interfaces.  (Since 4.1)
>
>           * id ("string", *optional*) -- A name to be used to identify
>             an instance of announce-timers and to allow it to modified
>             later.  Not for use as part of the migration parameters.
>             (Since 4.1)
>
> This is bad UX.  Especially when you have to chase multiple links.
>
> John's doc inliner inlines AnnounceParameters documentation into
> announce-self documentation.
>
> However, we don't want to inline "Parameters for self-announce timers".
>
> Definitions typically start with a short description of the thing being
> defined.  John calls this "intro".  We don't want to inline these.
>
> Syntactically, a doc comment begins with optional plain paragraphs.
> These end when something else begins: argument or member description,
> tagged section such as "Returns:", ...
>
> Most of the time, these initial paragraphs are exactly the "intro".  But
> not always.
>
> Even without the inliner, we often need to know where "intro" ends.  In
> the example above, "Arguments:" is auto-generated right after the
> "intro", and for that, the generator needs to know where "intro" ends.
>
> In his "qapi: enforce section ordering" series, John proposed syntax to
> explicitly end "intro": a line "Details:".  He had to add it to roughly
> one in 25 definition docs.
>
> I fear adding "Details:" when needed is easily forgotten when it's so
> rarely needed.  And I don't even trust myself to catch it patch review.
> So we looked for clearer syntax.
>
> This series implements one: instead of letting intro end when something
> else starts, use indentation like we do for descriptions and tagged
> sections.
>
> Let's have a look at the current doc source for announce-self:
>
>     ##
>     # @announce-self:
>     #
>     # Trigger generation of broadcast RARP frames to update network
>     # switches.  This can be useful when network bonds fail-over the
>     # active slave.
>     #
>     # TODO: This line is a hack to separate the example from the body
>     #
>     # .. qmp-example::
>     #
>     #     -> { "execute": "announce-self",
>     #          "arguments": {
>     #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
>     #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
>     #     <- { "return": {} }
>     #
>     # Since: 4.0
>     ##
>
> "Intro" ends when something else starts, namely the TODO: section.  We
> need this hack so the doc generator inserts the "Arguments:" part where
> we want it.
>
> John's "qapi: enforce section ordering" series replaces the TODO hack
> with a "Details:" line.
>
> In new syntax, this instead becomes:
>
>     ##
>     # @announce-self: Trigger generation of broadcast RARP frames to
>     #     update network switches.  This can be useful when network bonds
>     #     fail-over the active slave.
>     #
>     # .. qmp-example::
>     #
>     #     -> { "execute": "announce-self",
>     #          "arguments": {
>     #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
>     #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
>     #     <- { "return": {} }
>     #
>     # Since: 4.0
>     ##
>
> Now "intro" is indented, and its end is obvious.  Mistakes seem
> unlikely.
>
> We could instead do:
>
>     ##
>     # @announce-self:
>     #     Trigger generation of broadcast RARP frames to update network
>     #     switches.  This can be useful when network bonds fail-over the
>     #     active slave.
>     #
>     # .. qmp-example::
>     #
>     #     -> { "execute": "announce-self",
>     #          "arguments": {
>     #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
>     #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
>     #     <- { "return": {} }
>     #
>     # Since: 4.0
>     ##
>
> I think I like this a bit better.
>
> > It is designed so that this conversion can happen incrementally with
> > no actual difference to the rendered manuals, so each QAPI module can
> > be converted one at a time for easier review and merging in an
> > arbitrary order.
>
> Why not wholesale conversion?  As long as generated output stays the
> same.  It does in this series, except for harmless line breaks.
>

When we last spoke, we weren't convinced this was the right approach and we
both wanted to see it. So I did enough to get a taste.


> > This series demonstrates conversion of just four modules; if I'm given
> > a thumbs up, I will convert the rest of QAPI, one module (or
> > maintainer stanza) per patch like how I handled adding
> > cross-references.
>
>
Re: [PATCH 00/12] qapi: add formal "intro" section
Posted by Markus Armbruster 1 month ago
John Snow <jsnow@redhat.com> writes:

> On Fri, Apr 24, 2026, 8:50 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Hiya, this is a series that explores a potential syntax for a
>> > designated "Intro" section. Markus knows why I want this, but for
>> > everyone else: a designated "Introduction" section is useful for the
>> > desired "inliner" feature for the new QAPI doc system. Commits explain
>> > a bit more. This is prep work and doesn't really change anything
>> > tangibly except source code syntax for the QAPI docs.

[...]

>> Why not wholesale conversion?  As long as generated output stays the
>> same.  It does in this series, except for harmless line breaks.
>>
>
> When we last spoke, we weren't convinced this was the right approach and we
> both wanted to see it. So I did enough to get a taste.

Yes, and it makes sense.

>> > This series demonstrates conversion of just four modules; if I'm given
>> > a thumbs up, I will convert the rest of QAPI, one module (or
>> > maintainer stanza) per patch like how I handled adding
>> > cross-references.
Re: [PATCH 00/12] qapi: add formal "intro" section
Posted by John Snow 1 month ago
On Thu, Apr 23, 2026, 6:00 PM John Snow <jsnow@redhat.com> wrote:

> Hiya, this is a series that explores a potential syntax for a
> designated "Intro" section. Markus knows why I want this, but for
> everyone else: a designated "Introduction" section is useful for the
> desired "inliner" feature for the new QAPI doc system. Commits explain
> a bit more. This is prep work and doesn't really change anything
> tangibly except source code syntax for the QAPI docs.
>
> It is designed so that this conversion can happen incrementally with
> no actual difference to the rendered manuals, so each QAPI module can
> be converted one at a time for easier review and merging in an
> arbitrary order.
>
> This series demonstrates conversion of just four modules; if I'm given
> a thumbs up, I will convert the rest of QAPI, one module (or
> maintainer stanza) per patch like how I handled adding
> cross-references.
>
> John Snow (12):
>   tests/qapi: generate output in source order
>   qapi/docs: remove unused QAPIDoc subsection members
>   qapi/docs: make remaining subsection members "private"
>   qapi/docs: add "Intro" section
>   qapi/docs: adjust stub member insertion algorithm
>   qapi/docs: remove implicit Plain section
>   qapi/docs: add "Intro" section parsing
>   qapi/docs: Add rendering for INTRO sections
>   qapi: convert intro sections for accelerator.json
>   qapi: convert intro sections for acpi-hest.json
>   qapi: convert intro sections for acpi.json
>   qapi: convert intro sections for audio.json
>
>  docs/devel/qapi-code-gen.rst            |  17 +-
>  docs/sphinx/qapidoc.py                  |  52 +++---
>  qapi/accelerator.json                   |  24 +--
>  qapi/acpi-hest.json                     |   5 +-
>  qapi/acpi.json                          |  23 +--
>  qapi/audio.json                         | 112 ++++---------
>  .editorconfig                           |   1 +
>

Whoops, that snuck in by accident. Ignore! I needed some help doing the
line wraps in qapi :)

 scripts/qapi/parser.py                  | 204 ++++++++++++++----------
>  tests/qapi-schema/doc-good.out          |  30 ++--
>  tests/qapi-schema/doc-missing-colon.err |   2 +-
>  tests/qapi-schema/test-qapi.py          |  15 +-
>  11 files changed, 239 insertions(+), 246 deletions(-)
>
> --
> 2.53.0
>
>
>
Re: [PATCH 00/12] qapi: add formal "intro" section
Posted by Markus Armbruster 1 month ago
John Snow <jsnow@redhat.com> writes:

> On Thu, Apr 23, 2026, 6:00 PM John Snow <jsnow@redhat.com> wrote:
>
>> Hiya, this is a series that explores a potential syntax for a
>> designated "Intro" section. Markus knows why I want this, but for
>> everyone else: a designated "Introduction" section is useful for the
>> desired "inliner" feature for the new QAPI doc system. Commits explain
>> a bit more. This is prep work and doesn't really change anything
>> tangibly except source code syntax for the QAPI docs.
>>
>> It is designed so that this conversion can happen incrementally with
>> no actual difference to the rendered manuals, so each QAPI module can
>> be converted one at a time for easier review and merging in an
>> arbitrary order.
>>
>> This series demonstrates conversion of just four modules; if I'm given
>> a thumbs up, I will convert the rest of QAPI, one module (or
>> maintainer stanza) per patch like how I handled adding
>> cross-references.
>>
>> John Snow (12):
>>   tests/qapi: generate output in source order
>>   qapi/docs: remove unused QAPIDoc subsection members
>>   qapi/docs: make remaining subsection members "private"
>>   qapi/docs: add "Intro" section
>>   qapi/docs: adjust stub member insertion algorithm
>>   qapi/docs: remove implicit Plain section
>>   qapi/docs: add "Intro" section parsing
>>   qapi/docs: Add rendering for INTRO sections
>>   qapi: convert intro sections for accelerator.json
>>   qapi: convert intro sections for acpi-hest.json
>>   qapi: convert intro sections for acpi.json
>>   qapi: convert intro sections for audio.json
>>
>>  docs/devel/qapi-code-gen.rst            |  17 +-
>>  docs/sphinx/qapidoc.py                  |  52 +++---
>>  qapi/accelerator.json                   |  24 +--
>>  qapi/acpi-hest.json                     |   5 +-
>>  qapi/acpi.json                          |  23 +--
>>  qapi/audio.json                         | 112 ++++---------
>>  .editorconfig                           |   1 +
>>
>
> Whoops, that snuck in by accident. Ignore! I needed some help doing the
> line wraps in qapi :)

Would it make sense as a separate patch?

>>  scripts/qapi/parser.py                  | 204 ++++++++++++++----------
>>  tests/qapi-schema/doc-good.out          |  30 ++--
>>  tests/qapi-schema/doc-missing-colon.err |   2 +-
>>  tests/qapi-schema/test-qapi.py          |  15 +-
>>  11 files changed, 239 insertions(+), 246 deletions(-)
>>
>> --
>> 2.53.0
>>
>>
>>