[PATCH 57/57] docs/qapidoc: Add "the members of" pointers

John Snow posted 57 patches 4 weeks ago
There is a newer version of this series
[PATCH 57/57] docs/qapidoc: Add "the members of" pointers
Posted by John Snow 4 weeks ago
Add "the members of ..." pointers to Members and Arguments lists where
appropriate, with clickable cross-references - so it's a slight
improvement over the old system :)

This patch is meant to be a temporary solution until we can review and
merge the inliner.

The implementation of this patch is a little bit of a hack: Sphinx is
not designed to allow you to mix fields of different "type"; i.e. mixing
member descriptions and free-form text under the same heading. To
accomplish this with a minimum of hackery, we technically document a
"dummy field" and then just strip off the documentation for that dummy
field in a post-processing step. We use the "q_dummy" variable for this
purpose, then strip it back out before final processing. If this
processing step should fail, you'll see warnings for a bad
cross-reference. (So if you don't see any, it must be working!)

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/sphinx/qapi_domain.py | 22 +++++++++++++--
 docs/sphinx/qapidoc.py     | 58 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 77 insertions(+), 3 deletions(-)

diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 077ab7ae47a..a3c89a85f44 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -165,6 +165,24 @@ def since_validator(param: str) -> str:
     return param
 
 
+class SpecialTypedField(CompatTypedField):
+    def make_field(self, *args: Any, **kwargs: Any) -> nodes.field:
+        ret = super().make_field(*args, **kwargs)
+
+        # Look for the characteristic " -- " text node that Sphinx
+        # inserts for each TypedField entry ...
+        for node in ret.traverse(lambda n: str(n) == " -- "):
+            par = node.parent
+            if par.children[0].astext() != "q_dummy":
+                continue
+
+            # If the first node's text is q_dummy, this is a dummy
+            # field we want to strip down to just its contents.
+            del par.children[:-1]
+
+        return ret
+
+
 class QAPIObject(ParserFix):
     """
     Description of a generic QAPI object.
@@ -451,7 +469,7 @@ class QAPICommand(QAPIObject):
     doc_field_types.extend(
         [
             # :arg TypeName ArgName: descr
-            CompatTypedField(
+            SpecialTypedField(
                 "argument",
                 label=_("Arguments"),
                 names=("arg",),
@@ -519,7 +537,7 @@ class QAPIObjectWithMembers(QAPIObject):
     doc_field_types.extend(
         [
             # :member type name: descr
-            CompatTypedField(
+            SpecialTypedField(
                 "member",
                 label=_("Members"),
                 names=("memb",),
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 511bab1592c..daddb709628 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -47,8 +47,10 @@
     QAPISchemaCommand,
     QAPISchemaDefinition,
     QAPISchemaEnumMember,
+    QAPISchemaEvent,
     QAPISchemaFeature,
     QAPISchemaMember,
+    QAPISchemaObjectType,
     QAPISchemaObjectTypeMember,
     QAPISchemaType,
     QAPISchemaVisitor,
@@ -300,11 +302,61 @@ def preamble(self, ent: QAPISchemaDefinition) -> None:
 
         self.ensure_blank_line()
 
+    def _insert_member_pointer(self, ent: QAPISchemaDefinition) -> None:
+
+        def _get_target(
+            ent: QAPISchemaDefinition,
+        ) -> Optional[QAPISchemaDefinition]:
+            if isinstance(ent, (QAPISchemaCommand, QAPISchemaEvent)):
+                return ent.arg_type
+            if isinstance(ent, QAPISchemaObjectType):
+                return ent.base
+            return None
+
+        target = _get_target(ent)
+        if target is not None and not target.is_implicit():
+            assert ent.info
+            self.add_field(
+                self.member_field_type,
+                "q_dummy",
+                f"The members of :qapi:type:`{target.name}`.",
+                ent.info,
+                "q_dummy",
+            )
+
+        if isinstance(ent, QAPISchemaObjectType) and ent.branches is not None:
+            for variant in ent.branches.variants:
+                if variant.type.name == "q_empty":
+                    continue
+                assert ent.info
+                self.add_field(
+                    self.member_field_type,
+                    "q_dummy",
+                    f" When ``{ent.branches.tag_member.name}`` is "
+                    f"``{variant.name}``; "
+                    f"The members of :qapi:type:`{variant.type.name}`.",
+                    ent.info,
+                    "q_dummy",
+                )
+
     def visit_sections(self, ent: QAPISchemaDefinition) -> None:
         sections = ent.doc.all_sections if ent.doc else []
 
+        # Determine the index location at which we should generate
+        # documentation for "The members of ..." pointers. This should
+        # go at the end of the members section(s) if any. Note that
+        # index 0 is assumed to be a plain intro section, even if it is
+        # empty; and that a members section if present will always
+        # immediately follow the opening PLAIN section.
+        gen_index = 1
+        if len(sections) > 1:
+            while sections[gen_index].kind == QAPIDoc.Kind.MEMBER:
+                gen_index += 1
+                if gen_index >= len(sections):
+                    break
+
         # Add sections *in the order they are documented*:
-        for section in sections:
+        for i, section in enumerate(sections):
             # @var is translated to ``var``:
             section.text = re.sub(r"@([\w-]+)", r"``\1``", section.text)
 
@@ -326,6 +378,10 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None:
             else:
                 assert False
 
+            # Generate "The members of ..." entries if necessary:
+            if i == gen_index - 1:
+                self._insert_member_pointer(ent)
+
         self.ensure_blank_line()
 
     # Transmogrification core methods
-- 
2.48.1
Re: [PATCH 57/57] docs/qapidoc: Add "the members of" pointers
Posted by Markus Armbruster 3 weeks, 5 days ago
John Snow <jsnow@redhat.com> writes:

> Add "the members of ..." pointers to Members and Arguments lists where
> appropriate, with clickable cross-references - so it's a slight
> improvement over the old system :)
>
> This patch is meant to be a temporary solution until we can review and
> merge the inliner.
>
> The implementation of this patch is a little bit of a hack: Sphinx is
> not designed to allow you to mix fields of different "type"; i.e. mixing
> member descriptions and free-form text under the same heading. To
> accomplish this with a minimum of hackery, we technically document a
> "dummy field" and then just strip off the documentation for that dummy
> field in a post-processing step. We use the "q_dummy" variable for this
> purpose, then strip it back out before final processing. If this
> processing step should fail, you'll see warnings for a bad
> cross-reference. (So if you don't see any, it must be working!)
>
> Signed-off-by: John Snow <jsnow@redhat.com>

Yes, it's a hack, and possibly fragile, but it'll all go away when the
inliner lands.  I understand the inliner already exists, but you're
holding it back to not balloon the series even more.

I'm on board.
Re: [PATCH 57/57] docs/qapidoc: Add "the members of" pointers
Posted by John Snow 3 weeks, 4 days ago
On Fri, Mar 7, 2025 at 7:37 AM Markus Armbruster <armbru@redhat.com> wrote:

> John Snow <jsnow@redhat.com> writes:
>
> > Add "the members of ..." pointers to Members and Arguments lists where
> > appropriate, with clickable cross-references - so it's a slight
> > improvement over the old system :)
> >
> > This patch is meant to be a temporary solution until we can review and
> > merge the inliner.
> >
> > The implementation of this patch is a little bit of a hack: Sphinx is
> > not designed to allow you to mix fields of different "type"; i.e. mixing
> > member descriptions and free-form text under the same heading. To
> > accomplish this with a minimum of hackery, we technically document a
> > "dummy field" and then just strip off the documentation for that dummy
> > field in a post-processing step. We use the "q_dummy" variable for this
> > purpose, then strip it back out before final processing. If this
> > processing step should fail, you'll see warnings for a bad
> > cross-reference. (So if you don't see any, it must be working!)
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> Yes, it's a hack, and possibly fragile, but it'll all go away when the
> inliner lands.  I understand the inliner already exists, but you're
> holding it back to not balloon the series even more.
>

Just couldn't find any other way to do it with less SLOC. It's a weird
limitation in the bowels of Sphinx, and the Field classes aren't factored
aggressively enough to just write my own new Field subclass without also
having to do a lot of re-implementation that touches a lot of APIs I'd
rather not muck around with: I literally think that approach, while
"cleaner" in the traditional sense, is likely *more* porcelain than what
I've done here.

Thought it was better to just let Sphinx do whatever it does in make_xref,
and then make some tactical edits on the tail.

Could write a John Steinbeck novella's analysis on all the other approaches
I tried and why I ruled them out, but. Eh. Just read a John Steinbeck
novella instead if you want to.


> I'm on board.
>
>
Thank goodness.