From nobody Tue Feb 10 01:14:45 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1581617512683552.1481812638037; Thu, 13 Feb 2020 10:11:52 -0800 (PST) Received: from localhost ([::1]:57422 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2Ixr-00047r-AC for importer@patchew.org; Thu, 13 Feb 2020 13:11:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:34574) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2Ijx-0000wT-8J for qemu-devel@nongnu.org; Thu, 13 Feb 2020 12:57:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j2Ijr-0003Bu-L4 for qemu-devel@nongnu.org; Thu, 13 Feb 2020 12:57:27 -0500 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]:37500) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j2Ijr-00039L-Cw for qemu-devel@nongnu.org; Thu, 13 Feb 2020 12:57:23 -0500 Received: by mail-wm1-x32b.google.com with SMTP id a6so7802107wme.2 for ; Thu, 13 Feb 2020 09:57:23 -0800 (PST) Received: from orth.archaic.org.uk (orth.archaic.org.uk. [81.2.115.148]) by smtp.gmail.com with ESMTPSA id p26sm3692913wmc.24.2020.02.13.09.57.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2020 09:57:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yx82A80BgFchuY4arBJB9y5p01nca9y/eUUsgMiGXgw=; b=AGXIeWWwOSyBDQEJVbx1ioDFD2c2SCfA5DIqRnh9pAt60DvOalv0BT/R2sOcc0hjJc BJwB/8nSYIqM4sn8wkj7g/IEZjZ60QuMMNUjF7QatyDegfKPGPWxMpeq5410aMal7AFq oMUpO4lC1wZR/CwgQNI3GsTEgZ7CgaOX5ArdRykHItCemPJrqj4I1S8hajXOrRPbm5Il s3gFsvtFky4yiU7na+LQtYl8zRz+lBNP7slkU9D1YYqK9Zy+acPd1XsJJLI3+3qgZAEc tYB4RqxaAaIu+EljWyxOS+PqU2IDVMzN9urBA4YuRTz/zeVLquZ/2K4FUnZk2nWiY1yC lhIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yx82A80BgFchuY4arBJB9y5p01nca9y/eUUsgMiGXgw=; b=nYqL2fqzYrbJxOfLBm9mtZhAPleJOxKZIfgcU9GCa8TFMfDipZO5LKwC1knOUZ8vQD HIhuxzZJy8FQO0A7q2bIAo1ClCPwPvSj52iIIKFSed29BACCI2yzBSQdIcDpIPec2Aey tCO0JhYjpHR/ucZKY7XUWi7VGNGcJB6GhBUIi06oZVcm84MTMPgUIxmPloiruoqirX0K 58ZiiplC+ErC6j7ekwFnS5I9o6EM5BcYhPcRMVxd3pR895vRP6MMzY9REVmm7vH+nwf7 TFt2l/hhdAseQv4BFzSD3vM30dT+J9EVulqPVT3aEqtl4/kninB5bTOa2gKYKIvmUXb6 h70w== X-Gm-Message-State: APjAAAUewIFP+MlRb+pyq2DjcJYdlobasgt6oIJPsvfqdoSeyYoUuA55 JHPs/MiH/mk4ly3pnjDauGgfRPJIAV4= X-Google-Smtp-Source: APXvYqzF+MrCwUbTjQA8GEfPXADSGZWVDB/5FVZx1t9CLvKLYIghdClHUWlpOyRE2quNQOtUSDgFhA== X-Received: by 2002:a7b:c8d7:: with SMTP id f23mr6818891wml.173.1581616641818; Thu, 13 Feb 2020 09:57:21 -0800 (PST) From: Peter Maydell To: qemu-devel@nongnu.org Subject: [PATCH v2 23/30] scripts/qapi/parser.py: improve doc comment indent handling Date: Thu, 13 Feb 2020 17:56:40 +0000 Message-Id: <20200213175647.17628-24-peter.maydell@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200213175647.17628-1-peter.maydell@linaro.org> References: <20200213175647.17628-1-peter.maydell@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::32b X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Markus Armbruster , Michael Roth , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Make the handling of indentation in doc comments more sophisticated, so that when we see a section like: Notes: some text some more text indented line 3 we save it for the doc-comment processing code as: some text some more text indented line 3 and when we see a section with the heading on its own line: Notes: some text some more text indented text we also accept that and save it in the same form. The exception is that we always retain indentation as-is for Examples sections, because these are literal text. If we detect that the comment document text is not indented as much as we expect it to be, we throw a parse error. (We don't complain about over-indented sections, because for rST this can be legitimate markup.) The golden reference for the doc comment text is updated to remove the two 'wrong' indents; these now form a test case that we correctly stripped leading whitespace from an indented multi-line argument definition. Signed-off-by: Peter Maydell --- v1->v2: Update doc-good.out as per final para. --- scripts/qapi/parser.py | 82 +++++++++++++++++++++++++++------- tests/qapi-schema/doc-good.out | 4 +- 2 files changed, 67 insertions(+), 19 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 2196ec5de1e..66f802641c9 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -313,18 +313,32 @@ class QAPIDoc(object): """ =20 class Section(object): - def __init__(self, name=3DNone): + def __init__(self, parser, name=3DNone, indent=3D0): + # parser, for error messages about indentation + self._parser =3D parser # optional section name (argument/member or section name) self.name =3D name # the list of lines for this section self.text =3D '' + # the expected indent level of the text of this section + self._indent =3D indent =20 def append(self, line): + # Strip leading spaces corresponding to the expected indent le= vel + # Blank lines are always OK. + if line: + spacecount =3D len(line) - len(line.lstrip(" ")) + if spacecount > self._indent: + spacecount =3D self._indent + if spacecount < self._indent: + raise QAPIParseError(self._parser, "unexpected de-inde= nt") + line =3D line[spacecount:] + self.text +=3D line.rstrip() + '\n' =20 class ArgSection(Section): - def __init__(self, name): - QAPIDoc.Section.__init__(self, name) + def __init__(self, parser, name, indent=3D0): + QAPIDoc.Section.__init__(self, parser, name, indent) self.member =3D None =20 def connect(self, member): @@ -338,7 +352,7 @@ class QAPIDoc(object): self._parser =3D parser self.info =3D info self.symbol =3D None - self.body =3D QAPIDoc.Section() + self.body =3D QAPIDoc.Section(parser) # dict mapping parameter name to ArgSection self.args =3D OrderedDict() self.features =3D OrderedDict() @@ -443,7 +457,18 @@ class QAPIDoc(object): =20 if name.startswith('@') and name.endswith(':'): line =3D line[len(name)+1:] - self._start_args_section(name[1:-1]) + if not line or line.isspace(): + # Line was just the "@arg:" header; following lines + # are not indented + indent =3D 0 + line =3D '' + else: + # Line is "@arg: first line of description"; following + # lines should be indented by len(name) + 1, and we + # pad out this first line so it is handled the same way + indent =3D len(name) + 1 + line =3D ' ' * indent + line + self._start_args_section(name[1:-1], indent) elif self._is_section_tag(name): self._append_line =3D self._append_various_line self._append_various_line(line) @@ -465,7 +490,17 @@ class QAPIDoc(object): =20 if name.startswith('@') and name.endswith(':'): line =3D line[len(name)+1:] - self._start_features_section(name[1:-1]) + if not line or line.isspace(): + # Line is just the "@name:" header, no ident for following= lines + indent =3D 0 + line =3D '' + else: + # Line is "@arg: first line of description"; following + # lines should be indented by len(name) + 3, and we + # pad out this first line so it is handled the same way + indent =3D len(name) + 1 + line =3D ' ' * indent + line + self._start_features_section(name[1:-1], indent) elif self._is_section_tag(name): self._append_line =3D self._append_various_line self._append_various_line(line) @@ -498,11 +533,23 @@ class QAPIDoc(object): % (name, self.sections[0].name)) elif self._is_section_tag(name): line =3D line[len(name)+1:] - self._start_section(name[:-1]) + if not line or line.isspace(): + # Line is just "SectionName:", no indent for following lin= es + indent =3D 0 + line =3D '' + elif name.startswith("Example"): + # The "Examples" section is literal-text, so preserve + # all the indentation as-is + indent =3D 0 + else: + # Line is "SectionName: some text", indent required + indent =3D len(name) + 1 + line =3D ' ' * indent + line + self._start_section(name[:-1], indent) =20 self._append_freeform(line) =20 - def _start_symbol_section(self, symbols_dict, name): + def _start_symbol_section(self, symbols_dict, name, indent): # FIXME invalid names other than the empty string aren't flagged if not name: raise QAPIParseError(self._parser, "invalid parameter name") @@ -511,21 +558,21 @@ class QAPIDoc(object): "'%s' parameter name duplicated" % name) assert not self.sections self._end_section() - self._section =3D QAPIDoc.ArgSection(name) + self._section =3D QAPIDoc.ArgSection(self._parser, name, indent) symbols_dict[name] =3D self._section =20 - def _start_args_section(self, name): - self._start_symbol_section(self.args, name) + def _start_args_section(self, name, indent): + self._start_symbol_section(self.args, name, indent) =20 - def _start_features_section(self, name): - self._start_symbol_section(self.features, name) + def _start_features_section(self, name, indent): + self._start_symbol_section(self.features, name, indent) =20 - def _start_section(self, name=3DNone): + def _start_section(self, name=3DNone, indent=3D0): if name in ('Returns', 'Since') and self.has_section(name): raise QAPIParseError(self._parser, "duplicated '%s' section" % name) self._end_section() - self._section =3D QAPIDoc.Section(name) + self._section =3D QAPIDoc.Section(self._parser, name, indent) self.sections.append(self._section) =20 def _end_section(self): @@ -548,7 +595,7 @@ class QAPIDoc(object): def connect_member(self, member): if member.name not in self.args: # Undocumented TODO outlaw - self.args[member.name] =3D QAPIDoc.ArgSection(member.name) + self.args[member.name] =3D QAPIDoc.ArgSection(self._parser, me= mber.name) self.args[member.name].connect(member) =20 def connect_feature(self, feature): @@ -556,7 +603,8 @@ class QAPIDoc(object): raise QAPISemError(feature.info, "feature '%s' lacks documentation" % feature.name) - self.features[feature.name] =3D QAPIDoc.ArgSection(feature.nam= e) + self.features[feature.name] =3D QAPIDoc.ArgSection(self._parse= r, + feature.name) self.features[feature.name].connect(feature) =20 def check_expr(self, expr): diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out index a65bce639ff..0302ce0bde1 100644 --- a/tests/qapi-schema/doc-good.out +++ b/tests/qapi-schema/doc-good.out @@ -144,7 +144,7 @@ doc symbol=3DAlternate =20 arg=3Di an integer - @b is undocumented +@b is undocumented arg=3Db =20 doc freeform @@ -157,7 +157,7 @@ doc symbol=3Dcmd the first argument arg=3Darg2 the second - argument +argument arg=3Darg3 =20 feature=3Dcmd-feat1 --=20 2.20.1