[PATCH 56/57] docs/qapidoc: add intermediate output debugger

John Snow posted 57 patches 4 weeks ago
There is a newer version of this series
[PATCH 56/57] docs/qapidoc: add intermediate output debugger
Posted by John Snow 4 weeks ago
Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
will produce .ir files (one for each qapidoc directive) that write the
generated rst file to disk to allow for easy debugging and verification
of the generated document.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 docs/sphinx/qapidoc.py | 41 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 4 deletions(-)

diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index c84fff95697..511bab1592c 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -37,7 +37,7 @@
 from typing import TYPE_CHECKING
 
 from docutils import nodes
-from docutils.parsers.rst import Directive, directives
+from docutils.parsers.rst import directives
 from docutils.statemachine import StringList
 from qapi.error import QAPIError
 from qapi.parser import QAPIDoc
@@ -60,7 +60,7 @@
 from sphinx.directives.code import CodeBlock
 from sphinx.errors import ExtensionError
 from sphinx.util import logging
-from sphinx.util.docutils import switch_source_input
+from sphinx.util.docutils import SphinxDirective, switch_source_input
 from sphinx.util.nodes import nested_parse_with_titles
 
 
@@ -422,7 +422,7 @@ def visit_module(self, name: str) -> None:
         super().visit_module(name)
 
 
-class NestedDirective(Directive):
+class NestedDirective(SphinxDirective):
     def run(self) -> Sequence[nodes.Node]:
         raise NotImplementedError
 
@@ -491,10 +491,43 @@ def transmogrify(self, schema: QAPISchema) -> nodes.Element:
                 node.document = self.state.document
                 self.state.nested_parse(content, 0, contentnode)
         logger.info("Transmogrifier's nested parse completed.")
+
+        if self.env.app.verbosity >= 2 or os.environ.get("DEBUG"):
+            argname = "_".join(Path(self.arguments[0]).parts)
+            name = Path(argname).stem + ".ir"
+            self.write_intermediate(content, name)
+
         sys.stdout.flush()
-
         return contentnode
 
+    def write_intermediate(self, content: StringList, filename: str) -> None:
+        logger.info(
+            "writing intermediate rST for '%s' to '%s'",
+            self.arguments[0],
+            filename,
+        )
+
+        srctree = Path(self.env.app.config.qapidoc_srctree).resolve()
+        outlines = []
+        lcol_width = 0
+
+        for i, line in enumerate(content):
+            src, lineno = content.info(i)
+            srcpath = Path(src).resolve()
+            srcpath = srcpath.relative_to(srctree)
+
+            lcol = f"{srcpath}:{lineno:04d}"
+            lcol_width = max(lcol_width, len(lcol))
+            outlines.append((lcol, line))
+
+        with open(filename, "w", encoding="UTF-8") as outfile:
+            for lcol, rcol in outlines:
+                outfile.write(lcol.rjust(lcol_width))
+                outfile.write(" |")
+                if rcol:
+                    outfile.write(f" {rcol}")
+                outfile.write("\n")
+
     def legacy(self, schema: QAPISchema) -> nodes.Element:
         vis = QAPISchemaGenRSTVisitor(self)
         vis.visit_begin(schema)
-- 
2.48.1
Re: [PATCH 56/57] docs/qapidoc: add intermediate output debugger
Posted by Markus Armbruster 3 weeks, 5 days ago
John Snow <jsnow@redhat.com> writes:

> Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
> will produce .ir files (one for each qapidoc directive) that write the
> generated rst file to disk to allow for easy debugging and verification
> of the generated document.
>
> Signed-off-by: John Snow <jsnow@redhat.com>

I understand we generally need to examine these .ir files only when
things go wrong, or maybe to help understanding the transmogrifier.  I
guess few people will care, and only rarely.  But when we care, we
likely care a *lot*.  Sure we want to dig the information on how to get
.ir files out of a commit message then?
Re: [PATCH 56/57] docs/qapidoc: add intermediate output debugger
Posted by John Snow 3 weeks, 4 days ago
On Fri, Mar 7, 2025 at 7:34 AM Markus Armbruster <armbru@redhat.com> wrote:

> John Snow <jsnow@redhat.com> writes:
>
> > Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
> > will produce .ir files (one for each qapidoc directive) that write the
> > generated rst file to disk to allow for easy debugging and verification
> > of the generated document.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> I understand we generally need to examine these .ir files only when
> things go wrong, or maybe to help understanding the transmogrifier.  I
> guess few people will care, and only rarely.  But when we care, we
> likely care a *lot*.  Sure we want to dig the information on how to get
> .ir files out of a commit message then?
>

Intend to advertise it in the transmogrifier doc and/or extend the
doc-writing section of the qapi code gen doc.
Re: [PATCH 56/57] docs/qapidoc: add intermediate output debugger
Posted by Markus Armbruster 3 weeks, 4 days ago
John Snow <jsnow@redhat.com> writes:

> On Fri, Mar 7, 2025 at 7:34 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
>> > will produce .ir files (one for each qapidoc directive) that write the
>> > generated rst file to disk to allow for easy debugging and verification
>> > of the generated document.
>> >
>> > Signed-off-by: John Snow <jsnow@redhat.com>
>>
>> I understand we generally need to examine these .ir files only when
>> things go wrong, or maybe to help understanding the transmogrifier.  I
>> guess few people will care, and only rarely.  But when we care, we
>> likely care a *lot*.  Sure we want to dig the information on how to get
>> .ir files out of a commit message then?
>>
>
> Intend to advertise it in the transmogrifier doc and/or extend the
> doc-writing section of the qapi code gen doc.

Okay!