docs/index.rst | 1 + docs/qapi/index.rst | 53 ++++++ docs/sphinx/qapidoc.py | 419 ++++++++++++++++++++++++++++++++++++++--- scripts/qapi/parser.py | 97 +++++++--- scripts/qapi/source.py | 4 +- 5 files changed, 524 insertions(+), 50 deletions(-) create mode 100644 docs/qapi/index.rst
based-on: https://patchew.org/QEMU/20241213011307.2942030-1-jsnow@redhat.com/ Hi! This series is a very, very barebones implementation for the new QAPI doc generator. It does not have many features that I presented on at KVM Forum; the point of this patch set is instead to present a stripped down basis for ongoing work so we can discuss on-list with full context of the code available to do so. The documentation this series generates is *not suitable* for replacing the current document generator, it has a few glaring omissions - on purpose - those features have been factored out intentionally so they can be reviewed with fuller context and more careful review. What this series does: - Adds the new "Transmogrifier" rST generator to qapidoc.py, which generates an in-memory rST document using qapi-domain directives. - Adds a test document that showcases this new transmogrifier. What this series very notably does not do (yet): - "ifcond" data for anything other than top-level entities is not considered or rendered. This means "if" statements for features and members are entirely absent. - The inliner is not present at all. This series renders only documentation exactly as it is exists in the source files. - *branches* are themselves not considered at all; they're skipped entirely for now. They will be included alongside the inliner in either a subsequent series or a followup to this series. - Undocumented members and return statements are not autogenerated. - Pseudofeatures (Things like allow-oob) are not generated as documented features. - Documentation culling: all entities are documented whether or not they're relevant to the wire format. My goal in doing it this way is to save the "fancy" features for later so we can focus on reviewing and tightening up the core functionality of the transmogrifier. Once we're on steadier ground, I will re-add the fanciful features while adjusting the qapi-domain mechanisms. Once everything looks "roughly right, give or take some minor nits", I will switch back to the qapi-domain series itself for review before we merge everything together. John Snow (23): docs/qapidoc: support header-less freeform sections qapi/parser: adjust info location for doc body section docs/qapidoc: remove example section support qapi: expand tags to all doc sections qapi/schema: add __repr__ to QAPIDoc.Section docs/qapidoc: add transmogrifier stub docs/qapidoc: add transmogrifier class stub docs/qapidoc: add visit_module() method qapi/source: allow multi-line QAPISourceInfo advancing docs/qapidoc: add visit_freeform() method docs/qapidoc: add preamble() method docs/qapidoc: add visit_paragraph() method docs/qapidoc: add visit_errors() method docs/qapidoc: add format_type() method docs/qapidoc: add add_field() and generate_field() helper methods docs/qapidoc: add visit_feature() method docs/qapidoc: record current documented entity in transmogrifier docs/qapidoc: add visit_returns() method docs/qapidoc: add visit_member() method docs/qapidoc: add visit_sections() method docs/qapidoc: add visit_entity() docs/qapidoc: implement transmogrify() method docs/qapidoc: add transmogrifier test document docs/index.rst | 1 + docs/qapi/index.rst | 53 ++++++ docs/sphinx/qapidoc.py | 419 ++++++++++++++++++++++++++++++++++++++--- scripts/qapi/parser.py | 97 +++++++--- scripts/qapi/source.py | 4 +- 5 files changed, 524 insertions(+), 50 deletions(-) create mode 100644 docs/qapi/index.rst -- 2.47.0
John Snow <jsnow@redhat.com> writes:
> based-on: https://patchew.org/QEMU/20241213011307.2942030-1-jsnow@redhat.com/
>
> Hi!
>
> This series is a very, very barebones implementation for the new QAPI
> doc generator. It does not have many features that I presented on at KVM
> Forum; the point of this patch set is instead to present a stripped down
> basis for ongoing work so we can discuss on-list with full context of
> the code available to do so.
>
> The documentation this series generates is *not suitable* for replacing
> the current document generator, it has a few glaring omissions - on
> purpose - those features have been factored out intentionally so they
> can be reviewed with fuller context and more careful review.
>
> What this series does:
>
> - Adds the new "Transmogrifier" rST generator to qapidoc.py, which
> generates an in-memory rST document using qapi-domain directives.
> - Adds a test document that showcases this new transmogrifier.
Note to other reviewers: transmogrifier output is
docs/manual/qapi/index.html.
> What this series very notably does not do (yet):
>
> - "ifcond" data for anything other than top-level entities is not
> considered or rendered. This means "if" statements for features and
> members are entirely absent.
>
> - The inliner is not present at all. This series renders only
> documentation exactly as it is exists in the source files.
This item is not even a regression.
> - *branches* are themselves not considered at all; they're skipped
> entirely for now. They will be included alongside the inliner in
> either a subsequent series or a followup to this series.
>
> - Undocumented members and return statements are not autogenerated.
The current doc generator auto-generates missing member documentation
("Not documented"). It doesn't auto-generate missing returns
documentation. I explored auto-generating them, but shelved my work to
not interfere with yours.
> - Pseudofeatures (Things like allow-oob) are not generated as documented
> features.
What exactly are "pseudofeatures"?
> - Documentation culling: all entities are documented whether or not
> they're relevant to the wire format.
Also not a regression.
> My goal in doing it this way is to save the "fancy" features for later
> so we can focus on reviewing and tightening up the core functionality of
> the transmogrifier. Once we're on steadier ground, I will re-add the
> fanciful features while adjusting the qapi-domain mechanisms. Once
> everything looks "roughly right, give or take some minor nits", I will
> switch back to the qapi-domain series itself for review before we merge
> everything together.
Makes sense to me.
[...]
> docs/index.rst | 1 +
> docs/qapi/index.rst | 53 ++++++
> docs/sphinx/qapidoc.py | 419 ++++++++++++++++++++++++++++++++++++++---
> scripts/qapi/parser.py | 97 +++++++---
> scripts/qapi/source.py | 4 +-
> 5 files changed, 524 insertions(+), 50 deletions(-)
> create mode 100644 docs/qapi/index.rst
The changes to the QAPI generator core (scripts/qapi/) are small, and
spread over just four patches:
qapi/source: allow multi-line QAPISourceInfo advancing
qapi/schema: add __repr__ to QAPIDoc.Section
qapi: expand tags to all doc sections
qapi/parser: adjust info location for doc body section
On Thu, Dec 19, 2024 at 7:31 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > based-on:
> https://patchew.org/QEMU/20241213011307.2942030-1-jsnow@redhat.com/
> >
> > Hi!
> >
> > This series is a very, very barebones implementation for the new QAPI
> > doc generator. It does not have many features that I presented on at KVM
> > Forum; the point of this patch set is instead to present a stripped down
> > basis for ongoing work so we can discuss on-list with full context of
> > the code available to do so.
> >
> > The documentation this series generates is *not suitable* for replacing
> > the current document generator, it has a few glaring omissions - on
> > purpose - those features have been factored out intentionally so they
> > can be reviewed with fuller context and more careful review.
> >
> > What this series does:
> >
> > - Adds the new "Transmogrifier" rST generator to qapidoc.py, which
> > generates an in-memory rST document using qapi-domain directives.
> > - Adds a test document that showcases this new transmogrifier.
>
> Note to other reviewers: transmogrifier output is
> docs/manual/qapi/index.html.
>
> > What this series very notably does not do (yet):
> >
> > - "ifcond" data for anything other than top-level entities is not
> > considered or rendered. This means "if" statements for features and
> > members are entirely absent.
> >
> > - The inliner is not present at all. This series renders only
> > documentation exactly as it is exists in the source files.
>
> This item is not even a regression.
>
No; but the version of this series as sent also does not add "The members
of ..." stubs, which would be a regression. I didn't necessarily intend for
this to be merged as-is; more of a "part one, with additional tricky
elements that require more careful thought isolated into separate patches
for later".
where "later" means "in v2" or "as a follow-up series as we stage things in
a development branch before final submission for inclusion to
origin/master" or whatever the actual mechanism is. I don't have a strong
vision there, really; I just wanted to nail down the basics out in the open
even if that was just between you (Markus) and I and we have a gentleman's
agreement that it looks tentatively OK.
>
> > - *branches* are themselves not considered at all; they're skipped
> > entirely for now. They will be included alongside the inliner in
> > either a subsequent series or a followup to this series.
> >
> > - Undocumented members and return statements are not autogenerated.
>
> The current doc generator auto-generates missing member documentation
> ("Not documented"). It doesn't auto-generate missing returns
> documentation. I explored auto-generating them, but shelved my work to
> not interfere with yours.
>
> > - Pseudofeatures (Things like allow-oob) are not generated as documented
> > features.
>
> What exactly are "pseudofeatures"?
>
What I've named things like allow-oob that aren't features, but ought to be
documented. We may well decide to promote them to real-deal special
features, or maybe not. My work-in-progress branch currently just adds
"dummy" features to document them. We can discuss this later alongside the
patch that implements this.
>
> > - Documentation culling: all entities are documented whether or not
> > they're relevant to the wire format.
>
> Also not a regression.
>
> > My goal in doing it this way is to save the "fancy" features for later
> > so we can focus on reviewing and tightening up the core functionality of
> > the transmogrifier. Once we're on steadier ground, I will re-add the
> > fanciful features while adjusting the qapi-domain mechanisms. Once
> > everything looks "roughly right, give or take some minor nits", I will
> > switch back to the qapi-domain series itself for review before we merge
> > everything together.
>
> Makes sense to me.
>
> [...]
>
> > docs/index.rst | 1 +
> > docs/qapi/index.rst | 53 ++++++
> > docs/sphinx/qapidoc.py | 419 ++++++++++++++++++++++++++++++++++++++---
> > scripts/qapi/parser.py | 97 +++++++---
> > scripts/qapi/source.py | 4 +-
> > 5 files changed, 524 insertions(+), 50 deletions(-)
> > create mode 100644 docs/qapi/index.rst
>
> The changes to the QAPI generator core (scripts/qapi/) are small, and
> spread over just four patches:
>
> qapi/source: allow multi-line QAPISourceInfo advancing
> qapi/schema: add __repr__ to QAPIDoc.Section
> qapi: expand tags to all doc sections
> qapi/parser: adjust info location for doc body section
>
>
John Snow <jsnow@redhat.com> writes:
> On Thu, Dec 19, 2024 at 7:31 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > based-on:
>> https://patchew.org/QEMU/20241213011307.2942030-1-jsnow@redhat.com/
>> >
>> > Hi!
>> >
>> > This series is a very, very barebones implementation for the new QAPI
>> > doc generator. It does not have many features that I presented on at KVM
>> > Forum; the point of this patch set is instead to present a stripped down
>> > basis for ongoing work so we can discuss on-list with full context of
>> > the code available to do so.
>> >
>> > The documentation this series generates is *not suitable* for replacing
>> > the current document generator, it has a few glaring omissions - on
>> > purpose - those features have been factored out intentionally so they
>> > can be reviewed with fuller context and more careful review.
>> >
>> > What this series does:
>> >
>> > - Adds the new "Transmogrifier" rST generator to qapidoc.py, which
>> > generates an in-memory rST document using qapi-domain directives.
>> > - Adds a test document that showcases this new transmogrifier.
>>
>> Note to other reviewers: transmogrifier output is
>> docs/manual/qapi/index.html.
>>
>> > What this series very notably does not do (yet):
>> >
>> > - "ifcond" data for anything other than top-level entities is not
>> > considered or rendered. This means "if" statements for features and
>> > members are entirely absent.
>> >
>> > - The inliner is not present at all. This series renders only
>> > documentation exactly as it is exists in the source files.
>>
>> This item is not even a regression.
>>
>
> No; but the version of this series as sent also does not add "The members
> of ..." stubs, which would be a regression.
Right.
> I didn't necessarily intend for
> this to be merged as-is; more of a "part one, with additional tricky
> elements that require more careful thought isolated into separate patches
> for later".
>
> where "later" means "in v2" or "as a follow-up series as we stage things in
> a development branch before final submission for inclusion to
> origin/master" or whatever the actual mechanism is. I don't have a strong
> vision there, really; I just wanted to nail down the basics out in the open
> even if that was just between you (Markus) and I and we have a gentleman's
> agreement that it looks tentatively OK.
Got it.
>> > - *branches* are themselves not considered at all; they're skipped
>> > entirely for now. They will be included alongside the inliner in
>> > either a subsequent series or a followup to this series.
>> >
>> > - Undocumented members and return statements are not autogenerated.
>>
>> The current doc generator auto-generates missing member documentation
>> ("Not documented"). It doesn't auto-generate missing returns
>> documentation. I explored auto-generating them, but shelved my work to
>> not interfere with yours.
>>
>> > - Pseudofeatures (Things like allow-oob) are not generated as documented
>> > features.
>>
>> What exactly are "pseudofeatures"?
>>
>
> What I've named things like allow-oob that aren't features, but ought to be
> documented. We may well decide to promote them to real-deal special
> features, or maybe not. My work-in-progress branch currently just adds
> "dummy" features to document them. We can discuss this later alongside the
> patch that implements this.
I agree this is a digression, so feel free to ignore the remainder of my
reply for now.
We have two kinds of flags in the QAPI schema language: features and ad
hoc flags. The ad hoc flags are 'boxed' (commands and events), 'gen',
'success-response', 'allow-oob', 'allow-preconfig', 'coroutine'
(commands only).
The flags sort into three buckets:
1. Code generation directives that do not affect the external interface,
and thus should not be visible in introspection: 'boxed', 'gen',
'coroutine'.
2. Flags that are visible at the external interface, but don't affect
code generation beyond making them visible in introspection: the
non-special features.
3. Code generation directives that affect the external interface, and
thus are (or should be) visible in introspection.
3a. The special features: are visible.
3b. 'allow-oob': is visible, but differently, because it predates
special features.
3c. 'allow-preconfig': not visible.
3d. 'success-response': not visible, because we use it for QGA only,
which doesn't provide introspection.
Bucket 3 could use cleanup.
[...]
On Thu, Jan 9, 2025, 6:48 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > On Thu, Dec 19, 2024 at 7:31 AM Markus Armbruster <armbru@redhat.com>
> wrote:
> >
> >> John Snow <jsnow@redhat.com> writes:
> >>
> >> > based-on:
> >> https://patchew.org/QEMU/20241213011307.2942030-1-jsnow@redhat.com/
> >> >
> >> > Hi!
> >> >
> >> > This series is a very, very barebones implementation for the new QAPI
> >> > doc generator. It does not have many features that I presented on at
> KVM
> >> > Forum; the point of this patch set is instead to present a stripped
> down
> >> > basis for ongoing work so we can discuss on-list with full context of
> >> > the code available to do so.
> >> >
> >> > The documentation this series generates is *not suitable* for
> replacing
> >> > the current document generator, it has a few glaring omissions - on
> >> > purpose - those features have been factored out intentionally so they
> >> > can be reviewed with fuller context and more careful review.
> >> >
> >> > What this series does:
> >> >
> >> > - Adds the new "Transmogrifier" rST generator to qapidoc.py, which
> >> > generates an in-memory rST document using qapi-domain directives.
> >> > - Adds a test document that showcases this new transmogrifier.
> >>
> >> Note to other reviewers: transmogrifier output is
> >> docs/manual/qapi/index.html.
> >>
> >> > What this series very notably does not do (yet):
> >> >
> >> > - "ifcond" data for anything other than top-level entities is not
> >> > considered or rendered. This means "if" statements for features and
> >> > members are entirely absent.
> >> >
> >> > - The inliner is not present at all. This series renders only
> >> > documentation exactly as it is exists in the source files.
> >>
> >> This item is not even a regression.
> >>
> >
> > No; but the version of this series as sent also does not add "The members
> > of ..." stubs, which would be a regression.
>
> Right.
>
> > I didn't necessarily intend
> for
> > this to be merged as-is; more of a "part one, with additional tricky
> > elements that require more careful thought isolated into separate patches
> > for later".
> >
> > where "later" means "in v2" or "as a follow-up series as we stage things
> in
> > a development branch before final submission for inclusion to
> > origin/master" or whatever the actual mechanism is. I don't have a strong
> > vision there, really; I just wanted to nail down the basics out in the
> open
> > even if that was just between you (Markus) and I and we have a
> gentleman's
> > agreement that it looks tentatively OK.
>
> Got it.
>
> >> > - *branches* are themselves not considered at all; they're skipped
> >> > entirely for now. They will be included alongside the inliner in
> >> > either a subsequent series or a followup to this series.
> >> >
> >> > - Undocumented members and return statements are not autogenerated.
> >>
> >> The current doc generator auto-generates missing member documentation
> >> ("Not documented"). It doesn't auto-generate missing returns
> >> documentation. I explored auto-generating them, but shelved my work to
> >> not interfere with yours.
> >>
> >> > - Pseudofeatures (Things like allow-oob) are not generated as
> documented
> >> > features.
> >>
> >> What exactly are "pseudofeatures"?
> >>
> >
> > What I've named things like allow-oob that aren't features, but ought to
> be
> > documented. We may well decide to promote them to real-deal special
> > features, or maybe not. My work-in-progress branch currently just adds
> > "dummy" features to document them. We can discuss this later alongside
> the
> > patch that implements this.
>
> I agree this is a digression, so feel free to ignore the remainder of my
> reply for now.
>
> We have two kinds of flags in the QAPI schema language: features and ad
> hoc flags. The ad hoc flags are 'boxed' (commands and events), 'gen',
> 'success-response', 'allow-oob', 'allow-preconfig', 'coroutine'
> (commands only).
>
> The flags sort into three buckets:
>
> 1. Code generation directives that do not affect the external interface,
> and thus should not be visible in introspection: 'boxed', 'gen',
> 'coroutine'.
>
> 2. Flags that are visible at the external interface, but don't affect
> code generation beyond making them visible in introspection: the
> non-special features.
>
> 3. Code generation directives that affect the external interface, and
> thus are (or should be) visible in introspection.
>
> 3a. The special features: are visible.
>
> 3b. 'allow-oob': is visible, but differently, because it predates
> special features.
>
> 3c. 'allow-preconfig': not visible.
>
> 3d. 'success-response': not visible, because we use it for QGA only,
> which doesn't provide introspection.
>
> Bucket 3 could use cleanup.
>
Yep. We can get into it on the pseudofeatures patch. Your analysis here
matches my understanding.
> [...]
>
>
© 2016 - 2025 Red Hat, Inc.