From nobody Mon Nov 25 11:39:21 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1715724081; cv=none; d=zohomail.com; s=zohoarc; b=lvgTkSBSWFVw0/TygKKtq3CjTZ5H+PnauYeLZxMOofKHI2s/KoXnJ/455X+8+U8ylX+pkwpCu4g5P5f37z6EcyjSs0RjrRp+sXeF6whu6MX/Ric7xNK9lmRK3rZgjktK4juOFtbbwPzYAIG8GlXfoyN0VKTYBeJOQnvV8NNUJJM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1715724081; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=7/DfOPlH5gs0diXQtgz3pXaLIlH4rV8K/i8zUNxA/3Q=; b=gTHBO0Z0m8+SDHl3wCcXvxp6IF4kCkyQ9rz/PpvROJuic1AAd6xg4ly3wyeSpWbZEfReH5Ay+d0bIGuu926XEqgsXJ5zbNDu5Uboi0ljbntbt65h1qfJLwH0w6Rs5wYTYtJV361gNjqd9jQm0Syc7/zfS0hhCEVuJNcKY1hkWBI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1715724081826802.2744733176834; Tue, 14 May 2024 15:01:21 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s709w-0005Xg-8v; Tue, 14 May 2024 17:58:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s709s-0005WK-Tg for qemu-devel@nongnu.org; Tue, 14 May 2024 17:58:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s709p-0008Pj-Tu for qemu-devel@nongnu.org; Tue, 14 May 2024 17:58:04 -0400 Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-20-Ot5wjhPmNbaPm-mGJu2axA-1; Tue, 14 May 2024 17:57:58 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 90DC63801EDA; Tue, 14 May 2024 21:57:57 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.17.156]) by smtp.corp.redhat.com (Postfix) with ESMTP id AD6FB3C27; Tue, 14 May 2024 21:57:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715723881; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7/DfOPlH5gs0diXQtgz3pXaLIlH4rV8K/i8zUNxA/3Q=; b=JxVlKQFZjRCpIKNOBMX2DlrzSNq89Qlb/JYz61raYkmmd9qRbh5gfpI3UtLuFvGOL42s7g MxWIV7oRbAW/Oebr13Cow86/+eXmfBPzCp9Xbs4e86OXITofiLka4W5H4qi7z9dmQIxNnD 9XjbKSzYnbM/c/9h4/oRskL+AVZAU3Y= X-MC-Unique: Ot5wjhPmNbaPm-mGJu2axA-1 From: John Snow To: qemu-devel@nongnu.org Cc: Peter Xu , Marcel Apfelbaum , Gerd Hoffmann , Fabiano Rosas , Pavel Dovgalyuk , Markus Armbruster , Ani Sinha , Michael Roth , Kevin Wolf , Jiri Pirko , Mads Ynddal , Jason Wang , Igor Mammedov , Peter Maydell , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Stefan Hajnoczi , Paolo Bonzini , Eduardo Habkost , "Michael S. Tsirkin" , qemu-block@nongnu.org, Stefan Berger , Victor Toso de Carvalho , Eric Blake , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Konstantin Kostiuk , Lukas Straub , Yanan Wang , Hanna Reitz , John Snow Subject: [PATCH 04/20] qapi/parser: preserve indentation in QAPIDoc sections Date: Tue, 14 May 2024 17:57:23 -0400 Message-ID: <20240514215740.940155-5-jsnow@redhat.com> In-Reply-To: <20240514215740.940155-1-jsnow@redhat.com> References: <20240514215740.940155-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.974, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1715724082980100001 Content-Type: text/plain; charset="utf-8" Prior to this patch, a section like this: @name: lorem ipsum dolor sit amet consectetur adipiscing elit would be parsed as: "lorem ipsum\ndolor sit amet\n consectetur adipiscing elit" We want to preserve the indentation for even the first body line so that the entire block can be parsed directly as rST. This patch would now parse that segment as: "lorem ipsum\n dolor sit amet\n consectetur adipiscing elit" This understandably breaks qapidoc.py; so a new function is added there to re-dedent the text. Once the new generator is merged, this function will not be needed any longer and can be dropped. (I verified this patch changes absolutely nothing by comparing the md5sums of the QMP ref html pages both before and after the change, so it's certified inert. QAPI test output has been updated to reflect the new strategy of preserving indents for rST.) Signed-off-by: John Snow --- docs/sphinx/qapidoc.py | 36 +++++++++++++++++++++++++++++----- scripts/qapi/parser.py | 8 ++++++-- tests/qapi-schema/doc-good.out | 32 +++++++++++++++--------------- 3 files changed, 53 insertions(+), 23 deletions(-) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index 1655682d4c7..2e3ffcbafb7 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -26,6 +26,7 @@ =20 import os import re +import textwrap =20 from docutils import nodes from docutils.parsers.rst import Directive, directives @@ -51,6 +52,28 @@ __version__ =3D "1.0" =20 =20 +def dedent(text: str) -> str: + # Temporary: In service of the new QAPI domain, the QAPI doc parser + # now preserves indents in args/members/features text. QAPIDoc does + # not handle this well, so undo that change here. + + # QAPIDoc is being rewritten and will be replaced soon, + # but this function is here in the interim as transition glue. + + lines =3D text.splitlines(True) + if len(lines) > 1: + if re.match(r"\s+", lines[0]): + # First line is indented; description started on + # the line after the name. dedent the whole block. + return textwrap.dedent(text) + else: + # Descr started on same line. Dedent line 2+. + return lines[0] + textwrap.dedent("".join(lines[1:])) + else: + # Descr was a single line; dedent entire line. + return textwrap.dedent(text) + + # fmt: off # Function borrowed from pydash, which is under the MIT license def intersperse(iterable, separator): @@ -169,7 +192,7 @@ def _nodes_for_members(self, doc, what, base=3DNone, br= anches=3DNone): term =3D self._nodes_for_one_member(section.member) # TODO drop fallbacks when undocumented members are outlawed if section.text: - defn =3D section.text + defn =3D dedent(section.text) else: defn =3D [nodes.Text('Not documented')] =20 @@ -207,7 +230,7 @@ def _nodes_for_enum_values(self, doc): termtext.extend(self._nodes_for_ifcond(section.member.ifco= nd)) # TODO drop fallbacks when undocumented members are outlawed if section.text: - defn =3D section.text + defn =3D dedent(section.text) else: defn =3D [nodes.Text('Not documented')] =20 @@ -242,7 +265,7 @@ def _nodes_for_features(self, doc): dlnode =3D nodes.definition_list() for section in doc.features.values(): dlnode +=3D self._make_dlitem( - [nodes.literal('', section.member.name)], section.text) + [nodes.literal('', section.member.name)], dedent(section.t= ext)) seen_item =3D True =20 if not seen_item: @@ -265,9 +288,12 @@ def _nodes_for_sections(self, doc): continue snode =3D self._make_section(section.tag) if section.tag and section.tag.startswith('Example'): - snode +=3D self._nodes_for_example(section.text) + snode +=3D self._nodes_for_example(dedent(section.text)) else: - self._parse_text_into_node(section.text, snode) + self._parse_text_into_node( + dedent(section.text) if section.tag else section.text, + snode, + ) nodelist.append(snode) return nodelist =20 diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 7b13a583ac1..8cdd5334ec6 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -448,7 +448,10 @@ def get_doc_indented(self, doc: 'QAPIDoc') -> Optional= [str]: indent =3D must_match(r'\s*', line).end() if not indent: return line - doc.append_line(line[indent:]) + + # Preserve the indent, it's needed for rST formatting. + doc.append_line(line) + prev_line_blank =3D False while True: self.accept(False) @@ -465,7 +468,8 @@ def get_doc_indented(self, doc: 'QAPIDoc') -> Optional[= str]: self, "unexpected de-indent (expected at least %d spaces)" % indent) - doc.append_line(line[indent:]) + # Again, preserve the indent for ReST. + doc.append_line(line) prev_line_blank =3D True =20 def get_doc_paragraph(self, doc: 'QAPIDoc') -> Optional[str]: diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out index 716a9a41026..435f6e6d768 100644 --- a/tests/qapi-schema/doc-good.out +++ b/tests/qapi-schema/doc-good.out @@ -117,8 +117,8 @@ doc symbol=3DBase body=3D =20 arg=3Dbase1 -description starts on a new line, -minimally indented + description starts on a new line, + minimally indented doc symbol=3DVariant1 body=3D A paragraph @@ -145,8 +145,8 @@ doc symbol=3DAlternate =20 arg=3Di description starts on the same line -remainder indented the same -@b is undocumented + remainder indented the same + @b is undocumented arg=3Db =20 feature=3Dalt-feat @@ -158,11 +158,11 @@ doc symbol=3Dcmd body=3D =20 arg=3Darg1 -description starts on a new line, -indented + description starts on a new line, + indented arg=3Darg2 description starts on the same line -remainder indented differently + remainder indented differently arg=3Darg3 =20 feature=3Dcmd-feat1 @@ -178,16 +178,16 @@ some section=3DTODO frobnicate section=3DNotes -- Lorem ipsum dolor sit amet -- Ut enim ad minim veniam + - Lorem ipsum dolor sit amet + - Ut enim ad minim veniam =20 -Duis aute irure dolor + Duis aute irure dolor section=3DExample --> in -<- out + -> in + <- out section=3DExamples -- *verbatim* -- {braces} + - *verbatim* + - {braces} section=3DSince 2.10 doc symbol=3Dcmd-boxed @@ -198,9 +198,9 @@ a feature feature=3Dcmd-feat2 another feature section=3DExample --> in + -> in =20 -<- out + <- out doc symbol=3DEVT_BOXED body=3D =20 --=20 2.44.0