From nobody Thu Oct 9 01:13:37 2025 Received: from ms.lwn.net (ms.lwn.net [45.79.88.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4E7B17C220; Sat, 21 Jun 2025 20:35:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.79.88.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538130; cv=none; b=A7WFSJzIWx72iWrfvXIhvMbQesNJy6QZOqNZyQv95yNzFd0061MZH+0+N8834vacIkw6IEYS49/6muZunbF8+gChcgeMgFhBwq+nCsFJhu+53oHZ3+1AZWZCiIYsgvJ45bwwZt2BhcDrgvnhDOgt3v+bno6ZeWqrmR2qlKoyOdI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538130; c=relaxed/simple; bh=PIgnXZoNcczk9sjzileC3R46aHm/pWzyDBtsHom8YH4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZR6b2CjJLasL9yn/yInfMBF9q6Ch1J8uvxhyeyFdiC4ZZyNcxeWGsoiOiSJ3gJ0fkHxNtthW6TDRsorAohgqSgj+XkEZxcxFmUE9yU26SElJMET/SB+r0C+0K5HVPxodwf9A6wKcH4Ql+71b3KY6V322521Q6qScV0Aqi7F8JRg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net; spf=pass smtp.mailfrom=lwn.net; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b=Mz2ph1H/; arc=none smtp.client-ip=45.79.88.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lwn.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b="Mz2ph1H/" DKIM-Filter: OpenDKIM Filter v2.11.0 ms.lwn.net B9DD441F2B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lwn.net; s=20201203; t=1750538128; bh=TJA19eEJ7nfHXs0zlOqFBQnq1kbpG1/6/C2QTSEnHhA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Mz2ph1H/iEpDNhi142UwH+rdYCRoRi5TzF9G3Onhg3sULbbPBIa5aS25Fj7mLNQ+/ 4zu/Vg1ovIz7NeRCfRSkxTe0r/k9deHbyeq58XVlPNDPU98Jyc7CzUoz/5CtFGYmmE jUxJ4HKzkvtfRdGcpo+9ir1UznEBlGis5PLvpF2CtCna5j3UB7ppRLz5hfv7gcpV00 j6ly0lGSwxMKJ+79M2ar34m1fGF7eYxgh5Bl8uUpLslYO1kLtmd8TUGZBAYmDF8J69 wWCK3E3HCdj2oBzdZRF28hSUJukipOASPe+aXMeeuemPSmQ6o+TMctZqqlO0WgOBlZ gSiomlkhxs9fQ== Received: from trenco.lwn.net (unknown [IPv6:2601:280:4600:2da9:67c:16ff:fe81:5f9b]) by ms.lwn.net (Postfix) with ESMTPA id B9DD441F2B; Sat, 21 Jun 2025 20:35:27 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Akira Yokosawa , Jonathan Corbet Subject: [PATCH 1/9] docs: kdoc: Make body_with_blank_line parsing more flexible Date: Sat, 21 Jun 2025 14:35:04 -0600 Message-ID: <20250621203512.223189-2-corbet@lwn.net> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250621203512.223189-1-corbet@lwn.net> References: <20250621203512.223189-1-corbet@lwn.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The regex in the BODY_WITH_BLANK_LINE case was looking for lines starting with " * ", where exactly one space was allowed before the following text. There are many kerneldoc comments where the authors have put multiple spaces instead, leading to mis-formatting of the documentation. Specifically, in this case, the description portion is associated with the last of the parameters. Allow multiple spaces in this context. See, for example, synchronize_hardirq() and how its documentation is formatted before and after the change. Signed-off-by: Jonathan Corbet Acked-by: Mauro Carvalho Chehab Tested-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_parser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index 42b2e0936b72..c46e1b6a7d4b 100644 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -1309,7 +1309,7 @@ class KernelDoc: """ =20 if self.state =3D=3D state.BODY_WITH_BLANK_LINE: - r =3D KernRe(r"\s*\*\s?\S") + r =3D KernRe(r"\s*\*\s*\S") if r.match(line): self.dump_section() self.entry.section =3D SECTION_DEFAULT --=20 2.49.0 From nobody Thu Oct 9 01:13:37 2025 Received: from ms.lwn.net (ms.lwn.net [45.79.88.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4741C1865EB; Sat, 21 Jun 2025 20:35:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.79.88.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538130; cv=none; b=aJvf1WxdpG4xZcjLFHHMmuS1VrcYU4rDRkCBdc2aWTXsUZg7ig50c54WuLtgT49EmA+ruvLLQdwYz9mP7bPdDsqfY9rcRLKaoDBIWXWKb610eqkxYYlxlYVxE3w9EFBlgQ+B2HPfdNTHBeLZ+buAKHCgZv0kmUdjuPvbDkz/QVY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538130; c=relaxed/simple; bh=5b38RPJ//eThz4XG3UGkeE0z0kHG07y+zKvKopTT4YQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c8GqmBmLw6Kj4m2kFgV4rUHjunx6aoo1mC0E0jOiNhGhjXQnne/TlicvVsSiS27ETNjNStT17nRagffdFa6uY9dJamKAEMIfODyFlqyGsKDNGbA/e10jhlLbbxChOFN/s9n9n1TdqV4w2m9pYth9GC3SJ2Wfoae5aHvpEONXmIA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net; spf=pass smtp.mailfrom=lwn.net; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b=IxlMR5dd; arc=none smtp.client-ip=45.79.88.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lwn.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b="IxlMR5dd" DKIM-Filter: OpenDKIM Filter v2.11.0 ms.lwn.net 5157441F2F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lwn.net; s=20201203; t=1750538128; bh=ySfGZziezFZvQWxqd0tnXEAmNLxr6M/Rt7g7yarGNiI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IxlMR5dd6FlQZno6reMhNvPm0ABAmn6ZwrhrmMe6wYi3ge4VEgLPQq7XBB9YRYzO8 AUfzTzrQb8ZKwnrJEABzy8LfNQy/wXeNirKrbbxohpG4+Vzixeu818Iy+iixCWB8yU PFxugBmCKpcp1FTu8csNl5vGIg0+VEmwsU7CQOLJffC+0gz/OuIFl544ccsrAtd9Lr HW5GcE1aoYeROcOhXGkPFaVsp6ixnrGFtLJRTR3F090ZeSMYDBTMghu6v1oTjvTnbi HeUqznvX44/9y4tEAvHHQ1ZqimdetfG9adphMMh6oRItVlEte2XmbgUGQGSH2I8MFp IYNiD+Vl1NOQA== Received: from trenco.lwn.net (unknown [IPv6:2601:280:4600:2da9:67c:16ff:fe81:5f9b]) by ms.lwn.net (Postfix) with ESMTPA id 5157441F2F; Sat, 21 Jun 2025 20:35:28 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Akira Yokosawa , Jonathan Corbet Subject: [PATCH 2/9] docs: kdoc: consolidate the "begin section" logic Date: Sat, 21 Jun 2025 14:35:05 -0600 Message-ID: <20250621203512.223189-3-corbet@lwn.net> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250621203512.223189-1-corbet@lwn.net> References: <20250621203512.223189-1-corbet@lwn.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Pull the repeated "begin a section" logic into a single place and hide it within the KernelEntry class. Signed-off-by: Jonathan Corbet Reviewed-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_parser.py | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index c46e1b6a7d4b..d29a61a06f6d 100644 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -169,6 +169,15 @@ class KernelEntry: self.warnings.append(log_msg) return =20 + # + # Begin a new section. + # + def begin_section(self, line_no, title =3D SECTION_DEFAULT, dump =3D F= alse): + if dump: + self.dump_section(start_new =3D True) + self.section =3D title + self.new_start_line =3D line_no + def dump_section(self, start_new=3DTrue): """ Dumps section contents to arrays/hashes intended for that purpose. @@ -1231,12 +1240,11 @@ class KernelDoc: # Check for a DOC: block and handle them specially. # if doc_block.search(line): - self.entry.new_start_line =3D ln =20 if not doc_block.group(1): - self.entry.section =3D "Introduction" + self.entry.begin_section(ln, "Introduction") else: - self.entry.section =3D doc_block.group(1) + self.entry.begin_section(ln, doc_block.group(1)) =20 self.entry.identifier =3D self.entry.section self.state =3D state.DOCBLOCK @@ -1270,8 +1278,7 @@ class KernelDoc: self.state =3D state.BODY self.entry.identifier =3D self.entry.identifier.strip(" ") # if there's no @param blocks need to set up default section h= ere - self.entry.section =3D SECTION_DEFAULT - self.entry.new_start_line =3D ln + 1 + self.entry.begin_section(ln + 1) # # Find the description portion, which *should* be there but # isn't always. @@ -1312,8 +1319,7 @@ class KernelDoc: r =3D KernRe(r"\s*\*\s*\S") if r.match(line): self.dump_section() - self.entry.section =3D SECTION_DEFAULT - self.entry.new_start_line =3D ln + self.entry.begin_section(ln) self.entry.contents =3D "" =20 if doc_sect.search(line): @@ -1340,8 +1346,7 @@ class KernelDoc: if self.entry.contents.strip("\n"): self.dump_section() =20 - self.entry.new_start_line =3D ln - self.entry.section =3D newsection + self.entry.begin_section(ln, newsection) self.entry.leading_space =3D None =20 self.entry.contents =3D newcontents.lstrip() @@ -1370,9 +1375,7 @@ class KernelDoc: =20 if cont =3D=3D "": if self.entry.section =3D=3D self.section_context: - self.dump_section() - - self.entry.new_start_line =3D ln + self.entry.begin_section(ln, dump =3D True) self.state =3D state.BODY else: if self.entry.section !=3D SECTION_DEFAULT: @@ -1427,8 +1430,7 @@ class KernelDoc: =20 if self.inline_doc_state =3D=3D state.INLINE_NAME and \ doc_inline_sect.search(line): - self.entry.section =3D doc_inline_sect.group(1) - self.entry.new_start_line =3D ln + self.entry.begin_section(ln, doc_inline_sect.group(1)) =20 self.entry.contents =3D doc_inline_sect.group(2).lstrip() if self.entry.contents !=3D "": @@ -1627,7 +1629,7 @@ class KernelDoc: """STATE_PROTO: reading a function/whatever prototype.""" =20 if doc_inline_oneline.search(line): - self.entry.section =3D doc_inline_oneline.group(1) + self.entry.begin_section(ln, doc_inline_oneline.group(1)) self.entry.contents =3D doc_inline_oneline.group(2) =20 if self.entry.contents !=3D "": --=20 2.49.0 From nobody Thu Oct 9 01:13:37 2025 Received: from ms.lwn.net (ms.lwn.net [45.79.88.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC25926FD8E; Sat, 21 Jun 2025 20:35:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.79.88.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538131; cv=none; b=jLc8mXA9lO+Ab9C9RGdUSgrgAGceJpdptTjVZxzR0xhHzOrlsifKS8EVs5xzhf9OTLm+2+A2Lp3Xzg3HAwv7LtjeoNCS369lMnIazsBwU8P83FmDpoa+6jF3CQvpANeiafl0IbmXczk1jsdvZXGXRYgiJ4hBAfF8IBHLLWG2p3g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538131; c=relaxed/simple; bh=9IHMMT5ypODexH2PM/uQIE+G0IbtBkxfFwsgKLeyQww=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HqMLn6dfOPbgeBbxxdLq7vBAMFbsvHhgX6NxbOcpAvpFqHWwtNhGcK8rt/fJo7+hFkAIFIJpABO+clFqN6l8CRUcg3n85v0sTLrLCnFfQkF5Vo7vRV/e/c8W5L9qYdAfTOX5VY9wUw+B+kjSnJ43TQmnrnXkzTfL1/Or1gsVJYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net; spf=pass smtp.mailfrom=lwn.net; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b=ODw9ppL3; arc=none smtp.client-ip=45.79.88.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lwn.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b="ODw9ppL3" DKIM-Filter: OpenDKIM Filter v2.11.0 ms.lwn.net D538441F30 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lwn.net; s=20201203; t=1750538129; bh=mlDZRfSBfR9P22HI7T1naTG3fh1siu5biEocXS2vcq8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ODw9ppL3B1FBkqBE8eWKNBdoJ9ZuvuWm0QAbqCmBBxABfPq7DvpKg6sYtL7sTT/i4 6+zfSuXXFPA6+dxUsYRGE8gJFuLyNNOxZ+JL5NuLVUJxCyD8Dt1/fMLhzQ5vrwI7+h NqheAWrsDA5vocf8S5t5md4i9sqVnL4QlJY4p5MKES86sebV/Ewg8UVen9kuloA5mm EA/O64H6ovPwx460vPmbo98QNVgWcLijZSU4VL8XEZj9PnAe5fwpL6VF6rV1QwmAAW KlDsVxOADC7H8b2R+BsaodpfuIXo0KdhF+RFTT4zaChio9MOS6O3DPycg+pf8H2t4j H6eX6fttuLvyg== Received: from trenco.lwn.net (unknown [IPv6:2601:280:4600:2da9:67c:16ff:fe81:5f9b]) by ms.lwn.net (Postfix) with ESMTPA id D538441F30; Sat, 21 Jun 2025 20:35:28 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Akira Yokosawa , Jonathan Corbet Subject: [PATCH 3/9] docs: kdoc: separate out the handling of the declaration phase Date: Sat, 21 Jun 2025 14:35:06 -0600 Message-ID: <20250621203512.223189-4-corbet@lwn.net> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250621203512.223189-1-corbet@lwn.net> References: <20250621203512.223189-1-corbet@lwn.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The BODY_MAYBE state really describes the "we are in a declaration" state. Rename it accordingly, and split the handling of this state out from that of the other BODY* states. This change introduces a fair amount of duplicated code that will be coalesced in a later patch. Signed-off-by: Jonathan Corbet Reviewed-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_parser.py | 93 +++++++++++++++++++++++++++------ 1 file changed, 78 insertions(+), 15 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index d29a61a06f6d..f1491f8c88e7 100644 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -86,7 +86,7 @@ class state: # Parser states NORMAL =3D 0 # normal code NAME =3D 1 # looking for function name - BODY_MAYBE =3D 2 # body - or maybe more description + DECLARATION =3D 2 # We have seen a declaration which might no= t be done BODY =3D 3 # the body of the comment BODY_WITH_BLANK_LINE =3D 4 # the body which has a blank line PROTO =3D 5 # scanning prototype @@ -96,7 +96,7 @@ class state: name =3D [ "NORMAL", "NAME", - "BODY_MAYBE", + "DECLARATION", "BODY", "BODY_WITH_BLANK_LINE", "PROTO", @@ -1287,7 +1287,7 @@ class KernelDoc: r =3D KernRe("[-:](.*)") if r.search(line): self.entry.declaration_purpose =3D trim_whitespace(r.group= (1)) - self.state =3D state.BODY_MAYBE + self.state =3D state.DECLARATION else: self.entry.declaration_purpose =3D "" =20 @@ -1310,9 +1310,82 @@ class KernelDoc: else: self.emit_msg(ln, f"Cannot find identifier on line:\n{line}") =20 + def process_decl(self, ln, line): + """ + STATE_DECLARATION: We've seen the beginning of a declaration + """ + if doc_sect.search(line): + self.entry.in_doc_sect =3D True + newsection =3D doc_sect.group(1) + + if newsection.lower() in ["description", "context"]: + newsection =3D newsection.title() + + # Special case: @return is a section, not a param description + if newsection.lower() in ["@return", "@returns", + "return", "returns"]: + newsection =3D "Return" + + # Perl kernel-doc has a check here for contents before section= s. + # the logic there is always false, as in_doc_sect variable is + # always true. So, just don't implement Wcontents_before_secti= ons + + # .title() + newcontents =3D doc_sect.group(2) + if not newcontents: + newcontents =3D "" + + if self.entry.contents.strip("\n"): + self.dump_section() + + self.entry.begin_section(ln, newsection) + self.entry.leading_space =3D None + + self.entry.contents =3D newcontents.lstrip() + if self.entry.contents: + self.entry.contents +=3D "\n" + + self.state =3D state.BODY + return + + if doc_end.search(line): + self.dump_section() + + # Look for doc_com + + doc_end: + r =3D KernRe(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') + if r.match(line): + self.emit_msg(ln, f"suspicious ending line: {line}") + + self.entry.prototype =3D "" + self.entry.new_start_line =3D ln + 1 + + self.state =3D state.PROTO + return + + if doc_content.search(line): + cont =3D doc_content.group(1) + + if cont =3D=3D "": + self.state =3D state.BODY + self.entry.contents +=3D "\n" # needed? + + else: + # Continued declaration purpose + self.entry.declaration_purpose =3D self.entry.declaration_= purpose.rstrip() + self.entry.declaration_purpose +=3D " " + cont + + r =3D KernRe(r"\s+") + self.entry.declaration_purpose =3D r.sub(' ', + self.entry.declarat= ion_purpose) + return + + # Unknown line, ignore + self.emit_msg(ln, f"bad line: {line}") + + def process_body(self, ln, line): """ - STATE_BODY and STATE_BODY_MAYBE: the bulk of a kerneldoc comment. + STATE_BODY: the bulk of a kerneldoc comment. """ =20 if self.state =3D=3D state.BODY_WITH_BLANK_LINE: @@ -1385,16 +1458,6 @@ class KernelDoc: =20 self.entry.contents +=3D "\n" =20 - elif self.state =3D=3D state.BODY_MAYBE: - - # Continued declaration purpose - self.entry.declaration_purpose =3D self.entry.declaration_= purpose.rstrip() - self.entry.declaration_purpose +=3D " " + cont - - r =3D KernRe(r"\s+") - self.entry.declaration_purpose =3D r.sub(' ', - self.entry.declarat= ion_purpose) - else: if self.entry.section.startswith('@') or \ self.entry.section =3D=3D self.section_context: @@ -1687,7 +1750,7 @@ class KernelDoc: state.NORMAL: process_normal, state.NAME: process_name, state.BODY: process_body, - state.BODY_MAYBE: process_body, + state.DECLARATION: process_decl, state.BODY_WITH_BLANK_LINE: process_body, state.INLINE: process_inline, state.PROTO: process_proto, --=20 2.49.0 From nobody Thu Oct 9 01:13:37 2025 Received: from ms.lwn.net (ms.lwn.net [45.79.88.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E4AA27054B; Sat, 21 Jun 2025 20:35:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.79.88.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538131; cv=none; b=a4zZZBEK9DpUOypBGRkIywfYBJR20VNS9qNF1Nck5Vv6GD4E6CdoHCTtE77n0Au30p8RpXxnEw1L9XVLrppxPJaUQbRQ/nRTmVEc0h5ze7JQ/KZTCPvI/zm6Ur14RiTtuZwmsCxfoo5flDlCe6TLWiwNxGwhy8ZnWDcqPLCj1uc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538131; c=relaxed/simple; bh=O0CUyK50FgOTsjRuQq2n40FMPsir8dSq5R3XD3XXlXE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TiWxuqOaQ8gYkPfP53P6Hn/Ss5TAa+KrGTseETppa+Hxjvq2ZutLq+nCzJqzHmTT+XL1T0jyuNRrTM3R+x5bVsFOazXfRHXMC5jV9+gNc1aV7VoTvPF8yKoHd+6R7MjCnrcP3+4f3JqPCq35XEI3uUA1DGIz4BFaW9zFwAEi+mE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net; spf=pass smtp.mailfrom=lwn.net; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b=G21Hr1gx; arc=none smtp.client-ip=45.79.88.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lwn.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b="G21Hr1gx" DKIM-Filter: OpenDKIM Filter v2.11.0 ms.lwn.net 63F2B41F33 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lwn.net; s=20201203; t=1750538129; bh=oEr6nuzMBz/c+VtKctlDDd8uhHmdT8aD/TJ5fWa+6dY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G21Hr1gxsX1ou4Y3AUcWf1kAdoTzeKMdGp+8y2v9qg/ElcUgPIaGzPRQ4g4DCqZAM JM6xCB6j9dCozlh0KjOKg5NflfhoLVdi6KtRzsFo9tOgAq708SEYZ73T2Fb0Bx9mQg 3xrgWsLv6CRmPUO19/3zEl/mAmwBGcRfM8Amo4QYLSl8nZnKnJtU8jCidTWjXfGyta OR0urqcq8mSZ52XndkxNs3cgO1VWT7vTT6WXUXLvqJAJ3jv9KG6dAajGFhYWEou9lJ WC7LN/e9+gztc3W46e/fFfu7tbfF/Vrkx8bkSc1uWoMKDVr+hnMsKLrHKt012cqvnH QCXQeIetz8Trg== Received: from trenco.lwn.net (unknown [IPv6:2601:280:4600:2da9:67c:16ff:fe81:5f9b]) by ms.lwn.net (Postfix) with ESMTPA id 63F2B41F33; Sat, 21 Jun 2025 20:35:29 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Akira Yokosawa , Jonathan Corbet Subject: [PATCH 4/9] docs: kdoc: split out the special-section state Date: Sat, 21 Jun 2025 14:35:07 -0600 Message-ID: <20250621203512.223189-5-corbet@lwn.net> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250621203512.223189-1-corbet@lwn.net> References: <20250621203512.223189-1-corbet@lwn.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The state known as BODY_WITH_BLANK_LINE really, in a convoluted way, indicates a "special section" that is terminated by a blank line or the beginning of a new section. That is either "@param: desc" sections, or the weird "context" section that plays by the same rules. Rename the state to SPECIAL_SECTION and split its processing into a separate function; no real changes to the logic yet. Signed-off-by: Jonathan Corbet Reviewed-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_parser.py | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index f1491f8c88e7..185ffe4e1469 100644 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -88,7 +88,7 @@ class state: NAME =3D 1 # looking for function name DECLARATION =3D 2 # We have seen a declaration which might no= t be done BODY =3D 3 # the body of the comment - BODY_WITH_BLANK_LINE =3D 4 # the body which has a blank line + SPECIAL_SECTION =3D 4 # doc section ending with a blank line PROTO =3D 5 # scanning prototype DOCBLOCK =3D 6 # documentation block INLINE =3D 7 # gathering doc outside main block @@ -98,7 +98,7 @@ class state: "NAME", "DECLARATION", "BODY", - "BODY_WITH_BLANK_LINE", + "SPECIAL_SECTION", "PROTO", "DOCBLOCK", "INLINE", @@ -1383,18 +1383,18 @@ class KernelDoc: self.emit_msg(ln, f"bad line: {line}") =20 =20 + def process_special(self, ln, line): + """ + STATE_SPECIAL_SECTION: a section ending with a blank line + """ + if KernRe(r"\s*\*\s*\S").match(line): + self.entry.begin_section(ln, dump =3D True) + self.process_body(ln, line) + def process_body(self, ln, line): """ STATE_BODY: the bulk of a kerneldoc comment. """ - - if self.state =3D=3D state.BODY_WITH_BLANK_LINE: - r =3D KernRe(r"\s*\*\s*\S") - if r.match(line): - self.dump_section() - self.entry.begin_section(ln) - self.entry.contents =3D "" - if doc_sect.search(line): self.entry.in_doc_sect =3D True newsection =3D doc_sect.group(1) @@ -1452,7 +1452,7 @@ class KernelDoc: self.state =3D state.BODY else: if self.entry.section !=3D SECTION_DEFAULT: - self.state =3D state.BODY_WITH_BLANK_LINE + self.state =3D state.SPECIAL_SECTION else: self.state =3D state.BODY =20 @@ -1751,7 +1751,7 @@ class KernelDoc: state.NAME: process_name, state.BODY: process_body, state.DECLARATION: process_decl, - state.BODY_WITH_BLANK_LINE: process_body, + state.SPECIAL_SECTION: process_special, state.INLINE: process_inline, state.PROTO: process_proto, state.DOCBLOCK: process_docblock, --=20 2.49.0 From nobody Thu Oct 9 01:13:37 2025 Received: from ms.lwn.net (ms.lwn.net [45.79.88.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B67B7270EC3; Sat, 21 Jun 2025 20:35:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.79.88.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538132; cv=none; b=LzJH6Sef7Xi2spfZLEBLDk6BMP7qz46y8MAjyFEownVQdEZgp1GFAEdmz48zH43pUc0Igw2VO+NVNyccuQKexUiex2ASWQI7t7nIE1sYTmRDmqQHAQ0ytJnQk8kwjoF40uhgCOv/01aKQMyTg7GdeaWfoSOfndFEIWwPxBosN2c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538132; c=relaxed/simple; bh=+tg+/selIsk1K4SgMwFprCsk6T3OcPFBih5+vmWXiG4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YYr/7EQ0d0/4ghGTAK0M818g+TUiY82g+skfL9ajO6HPhBNjA8ms5hPLrBS6KZRYSD+5oDmqcDMURPVsigKuQpU1ellINRbMMbOUUCvTAVZo7MVsR99uaOhUl+SbV5ry57w9xU48sA+rL/KK2u0R3NrCZ629XR3J/6is8ZkmiSM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net; spf=pass smtp.mailfrom=lwn.net; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b=Uktu6SBA; arc=none smtp.client-ip=45.79.88.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lwn.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b="Uktu6SBA" DKIM-Filter: OpenDKIM Filter v2.11.0 ms.lwn.net E6C8F41AD8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lwn.net; s=20201203; t=1750538130; bh=JVMKs6c/9IMxrm7+AbKDFaW9ihwnMSixkjpZKCTY79I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Uktu6SBAGTvlkdfQEahylKALHjr49IeEiEB6ooqSFtcmR8DomUdc0AK+jiE1u/EtY 0TCLIcRFlgJf6V4RMss2HvNBWgjXbi8ucUZSgTQQwepUDwGlx3jixjzjlI3EpRNrPe 5pal/FpGpkLxhDB27qH8Z5xlFb+N58tj7Uzdmz1ulZOPvdFmnG3zMrGel11pew1RXc FFUBBKSLrf+MM2kkHCy27QrJ20I/GA076SOF6eHWLxT3Dh0Vge13zzVf/EIdFsvRL/ U6RPnSmWpsoqU2S8jxvAL4PoGP2ogk8uZq6UdrM0JmrGjn3RH9jbxyyUhatfEfqhTW N9VnwfBcRuZsA== Received: from trenco.lwn.net (unknown [IPv6:2601:280:4600:2da9:67c:16ff:fe81:5f9b]) by ms.lwn.net (Postfix) with ESMTPA id E6C8F41AD8; Sat, 21 Jun 2025 20:35:29 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Akira Yokosawa , Jonathan Corbet Subject: [PATCH 5/9] docs: kdoc: coalesce the new-section handling Date: Sat, 21 Jun 2025 14:35:08 -0600 Message-ID: <20250621203512.223189-6-corbet@lwn.net> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250621203512.223189-1-corbet@lwn.net> References: <20250621203512.223189-1-corbet@lwn.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Merge the duplicated code back into a single implementation. Code movement only, no logic changes. Signed-off-by: Jonathan Corbet Reviewed-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_parser.py | 49 +++++++++------------------------ 1 file changed, 13 insertions(+), 36 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index 185ffe4e1469..a336d543e72b 100644 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -1310,10 +1310,10 @@ class KernelDoc: else: self.emit_msg(ln, f"Cannot find identifier on line:\n{line}") =20 - def process_decl(self, ln, line): - """ - STATE_DECLARATION: We've seen the beginning of a declaration - """ + # + # Helper function to determine if a new section is being started. + # + def is_new_section(self, ln, line): if doc_sect.search(line): self.entry.in_doc_sect =3D True newsection =3D doc_sect.group(1) @@ -1346,6 +1346,14 @@ class KernelDoc: self.entry.contents +=3D "\n" =20 self.state =3D state.BODY + return True + return False + + def process_decl(self, ln, line): + """ + STATE_DECLARATION: We've seen the beginning of a declaration + """ + if self.is_new_section(ln, line): return =20 if doc_end.search(line): @@ -1395,38 +1403,7 @@ class KernelDoc: """ STATE_BODY: the bulk of a kerneldoc comment. """ - if doc_sect.search(line): - self.entry.in_doc_sect =3D True - newsection =3D doc_sect.group(1) - - if newsection.lower() in ["description", "context"]: - newsection =3D newsection.title() - - # Special case: @return is a section, not a param description - if newsection.lower() in ["@return", "@returns", - "return", "returns"]: - newsection =3D "Return" - - # Perl kernel-doc has a check here for contents before section= s. - # the logic there is always false, as in_doc_sect variable is - # always true. So, just don't implement Wcontents_before_secti= ons - - # .title() - newcontents =3D doc_sect.group(2) - if not newcontents: - newcontents =3D "" - - if self.entry.contents.strip("\n"): - self.dump_section() - - self.entry.begin_section(ln, newsection) - self.entry.leading_space =3D None - - self.entry.contents =3D newcontents.lstrip() - if self.entry.contents: - self.entry.contents +=3D "\n" - - self.state =3D state.BODY + if self.is_new_section(ln, line): return =20 if doc_end.search(line): --=20 2.49.0 From nobody Thu Oct 9 01:13:37 2025 Received: from ms.lwn.net (ms.lwn.net [45.79.88.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B7A0271454; Sat, 21 Jun 2025 20:35:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.79.88.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538132; cv=none; b=HCJ4Q1pJxPBAdHsvGfJ/z872phBfty/ipKFnzgEsSshfjozzjKUZgk9AymuNovLozCTMRBLfzERXI2VIwYHf+9TTiNavVVaWjIzGhH2ZHZsF3neu8CYUe0eQkJfyD2Qwwo4vwSJ1vgfztgClrcILGeuOG4z1d54aLhldG0BViAk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538132; c=relaxed/simple; bh=hgyF5qXQdsldMVeKTROkq4cIqxZZPrXcb24sgEudAuY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kZy6yJQxpMo9pvXOYvoEOEsbgrm47FH8ZPNq8GNeoepDLwIaHBKQtZ8PSxmnbzB2memWbZF9iNDGTE0VfbN9k0uEMH7kFKLa6cRNx041Paf7rbA+OyTmi9GMbO417BUJKBCepMwkQmYNQWFZYFgE61hBOsUqJYswri+x67Uvq4w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net; spf=pass smtp.mailfrom=lwn.net; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b=L8VdigIi; arc=none smtp.client-ip=45.79.88.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lwn.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b="L8VdigIi" DKIM-Filter: OpenDKIM Filter v2.11.0 ms.lwn.net 757B441F2B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lwn.net; s=20201203; t=1750538130; bh=t5O6xZ0asJB+PqlMyDbZnxO5S+2Br3NznwxuBU17ipc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L8VdigIi2YeigH9E6OnEpO551bOSOSz+jABgghdJX9mJGQWzmB0rxwrKeMjGk8Fd6 /CxU5z2SXq7tpYBxe2IQ5xmgj47UD/TimGVAGtD/bFwxkSBsp1dDGOw9rKXryNh4KH q6yjBAWkHOYYOO/Q70hZ2Htw0G0Y07tuUIBOvpu4NGLCm61f7nzcd1x0b1y7X/YH8K a2nTuEPBPQyt9yGUQwDCMQXeH3+8JIK/ptLyXD8i9k/f936JW84NxRYyffBauUUKHG qIRyO7F9cCwtYmMoBjamqCHcYQMPuC21XAGtgAQyFkIb1I+kX4mQHi8RIsJgK6B4/r jQISqHclwTTpQ== Received: from trenco.lwn.net (unknown [IPv6:2601:280:4600:2da9:67c:16ff:fe81:5f9b]) by ms.lwn.net (Postfix) with ESMTPA id 757B441F2B; Sat, 21 Jun 2025 20:35:30 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Akira Yokosawa , Jonathan Corbet Subject: [PATCH 6/9] docs: kdoc: rework the handling of SPECIAL_SECTION Date: Sat, 21 Jun 2025 14:35:09 -0600 Message-ID: <20250621203512.223189-7-corbet@lwn.net> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250621203512.223189-1-corbet@lwn.net> References: <20250621203512.223189-1-corbet@lwn.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the recognition of this state to when we enter it, rather than when we exit, eliminating some twisty logic along the way. Some changes in output do result from this shift, generally for kerneldoc comments that do not quite fit the format. See, for example, struct irqdomain. As far as I can tell, the new behavior is more correct in each case. Signed-off-by: Jonathan Corbet Reviewed-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_parser.py | 48 ++++++++++++++------------------- 1 file changed, 20 insertions(+), 28 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index a336d543e72b..5998b02ca3a0 100644 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -1316,21 +1316,25 @@ class KernelDoc: def is_new_section(self, ln, line): if doc_sect.search(line): self.entry.in_doc_sect =3D True + self.state =3D state.BODY + # + # Pick out the name of our new section, tweaking it if need be. + # newsection =3D doc_sect.group(1) - - if newsection.lower() in ["description", "context"]: - newsection =3D newsection.title() - - # Special case: @return is a section, not a param description - if newsection.lower() in ["@return", "@returns", - "return", "returns"]: + if newsection.lower() =3D=3D 'description': + newsection =3D 'Description' + elif newsection.lower() =3D=3D 'context': + newsection =3D 'Context' + self.state =3D state.SPECIAL_SECTION + elif newsection.lower() in ["@return", "@returns", + "return", "returns"]: newsection =3D "Return" - - # Perl kernel-doc has a check here for contents before section= s. - # the logic there is always false, as in_doc_sect variable is - # always true. So, just don't implement Wcontents_before_secti= ons - - # .title() + self.state =3D state.SPECIAL_SECTION + elif newsection[0] =3D=3D '@': + self.state =3D state.SPECIAL_SECTION + # + # Initialize the contents, and get the new section going. + # newcontents =3D doc_sect.group(2) if not newcontents: newcontents =3D "" @@ -1344,8 +1348,6 @@ class KernelDoc: self.entry.contents =3D newcontents.lstrip() if self.entry.contents: self.entry.contents +=3D "\n" - - self.state =3D state.BODY return True return False =20 @@ -1395,8 +1397,9 @@ class KernelDoc: """ STATE_SPECIAL_SECTION: a section ending with a blank line """ - if KernRe(r"\s*\*\s*\S").match(line): + if KernRe(r"\s*\*\s*$").match(line): self.entry.begin_section(ln, dump =3D True) + self.state =3D state.BODY self.process_body(ln, line) =20 def process_body(self, ln, line): @@ -1424,20 +1427,9 @@ class KernelDoc: cont =3D doc_content.group(1) =20 if cont =3D=3D "": - if self.entry.section =3D=3D self.section_context: - self.entry.begin_section(ln, dump =3D True) - self.state =3D state.BODY - else: - if self.entry.section !=3D SECTION_DEFAULT: - self.state =3D state.SPECIAL_SECTION - else: - self.state =3D state.BODY - self.entry.contents +=3D "\n" - else: - if self.entry.section.startswith('@') or \ - self.entry.section =3D=3D self.section_context: + if self.state =3D=3D state.SPECIAL_SECTION: if self.entry.leading_space is None: r =3D KernRe(r'^(\s+)') if r.match(cont): --=20 2.49.0 From nobody Thu Oct 9 01:13:37 2025 Received: from ms.lwn.net (ms.lwn.net [45.79.88.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEB5827146D; Sat, 21 Jun 2025 20:35:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.79.88.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538133; cv=none; b=JaV1p1HbZNoXGS68Im07tIqhkOW1/fD0fYIKg0S2LjBnm9TVU5PMyQVG1T9GqWQU+BfjM8D/JC+Ktjx1B7qTB9ChOu+NIwhLB7XF9oskDt7Hrbo23cGKgT8fzp/cgfXbJhIDF+DUuF6Ij4gQUcsBBW0oAHAl34EM9aAKhIRBid4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538133; c=relaxed/simple; bh=DIzPZ/e60/l8noMEDLFx3LiQtyQoc1JBMRgfRJt8SzI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FE7oYA5y/SxQ47dXsDkkYSFbGL1J7QxoZg6y64J41fzRfpzTVQTG4WCE+SoeKuIhJ3UMIYvn760KJxRFo3j9NwuDGJ+fAj9DTznRZoOFSareviC84jLNmRCHod85MYYYvKK1foyhreAFeYa+G1yFmItOvXewwGaBi0YxfhPceFE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net; spf=pass smtp.mailfrom=lwn.net; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b=Nj2BFMAj; arc=none smtp.client-ip=45.79.88.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lwn.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b="Nj2BFMAj" DKIM-Filter: OpenDKIM Filter v2.11.0 ms.lwn.net 052DC41F2F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lwn.net; s=20201203; t=1750538131; bh=igzwgHfO6uct+1CCVP7j6YFnebjHFin2PWnOhANtugQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Nj2BFMAj3DjijgU7IIaAPaQgNX4rJKImRCfWslRpYc+Cq8TfeOfTD51iGNLr/cYlB Iu+5MtlH/F1od6OF7AQMasQTkValYA2XmVFtAjmJAqy7Yqf4cP6C0RzJyBPWwkrmG+ gdsfyLBEX7SuRMLrQ6Uhwoc3m7KtG/+M/+8YHl9cbUtKPhGdIRrw5u+4ToAAaEm6UI 7jbabHg+9VFORNJURl7R1JBzBdnm9eWJrW7ZhV9Ne+4vW1EEX2gsdHqnj8TFsKj+wB klxa3HLAv5SSy2WLDyhDlIBLqOSVr5n40Zx0XHZaG13nAu1gkovKGUsgzkpk6n5n1s AkjZB6ah2WEOw== Received: from trenco.lwn.net (unknown [IPv6:2601:280:4600:2da9:67c:16ff:fe81:5f9b]) by ms.lwn.net (Postfix) with ESMTPA id 052DC41F2F; Sat, 21 Jun 2025 20:35:30 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Akira Yokosawa , Jonathan Corbet Subject: [PATCH 7/9] docs: kdoc: coalesce the end-of-comment processing Date: Sat, 21 Jun 2025 14:35:10 -0600 Message-ID: <20250621203512.223189-8-corbet@lwn.net> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250621203512.223189-1-corbet@lwn.net> References: <20250621203512.223189-1-corbet@lwn.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Separate out the end-of-comment logic into its own helper and remove the duplicated code introduced earlier. Signed-off-by: Jonathan Corbet Reviewed-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_parser.py | 36 +++++++++++++-------------------- 1 file changed, 14 insertions(+), 22 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index 5998b02ca3a0..f7a5b85a8ed7 100644 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -1351,13 +1351,10 @@ class KernelDoc: return True return False =20 - def process_decl(self, ln, line): - """ - STATE_DECLARATION: We've seen the beginning of a declaration - """ - if self.is_new_section(ln, line): - return - + # + # Helper function to detect (and effect) the end of a kerneldoc commen= t. + # + def is_comment_end(self, ln, line): if doc_end.search(line): self.dump_section() =20 @@ -1370,6 +1367,15 @@ class KernelDoc: self.entry.new_start_line =3D ln + 1 =20 self.state =3D state.PROTO + return True + return False + + + def process_decl(self, ln, line): + """ + STATE_DECLARATION: We've seen the beginning of a declaration + """ + if self.is_new_section(ln, line) or self.is_comment_end(ln, line): return =20 if doc_content.search(line): @@ -1406,21 +1412,7 @@ class KernelDoc: """ STATE_BODY: the bulk of a kerneldoc comment. """ - if self.is_new_section(ln, line): - return - - if doc_end.search(line): - self.dump_section() - - # Look for doc_com + + doc_end: - r =3D KernRe(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') - if r.match(line): - self.emit_msg(ln, f"suspicious ending line: {line}") - - self.entry.prototype =3D "" - self.entry.new_start_line =3D ln + 1 - - self.state =3D state.PROTO + if self.is_new_section(ln, line) or self.is_comment_end(ln, line): return =20 if doc_content.search(line): --=20 2.49.0 From nobody Thu Oct 9 01:13:37 2025 Received: from ms.lwn.net (ms.lwn.net [45.79.88.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 713252727F2; Sat, 21 Jun 2025 20:35:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.79.88.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538133; cv=none; b=rAZneXyLmcGEsMGiUUgvWjOKzJ9CH0llKarrCrpsMQ/WW4GZyP0h3ACWBaMuQvvSih4hkD2gteEdKDEttkhp6cHne8MIj9RS1bGUOZnEtDtxM1lkmGiKed1upAtDpm0aWKS2xWVmcPIav68ElVHDlzfc06IGmH6iF2PxnYqG3ig= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538133; c=relaxed/simple; bh=Wfjmd+ywBZv+xvzl7vSnUpl5jf1Ie+l67lrJTU+43cA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H3LCdYdU4Zgji0b42BVfPft2qYjus+eWwdMhH2YNoy+G+chHunqHsfrCi/l4Nj2SZ7Vb1HZFY0CYj0GBR2di6wWmo0zZNlBcWjiBUMoOzYD94uZ06XpFkGLLA2Z0OyAXDGI65+d85wlF9NziDhCaii2IiYgWpW0mP5DV6WQnoZU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net; spf=pass smtp.mailfrom=lwn.net; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b=bi8xpYjU; arc=none smtp.client-ip=45.79.88.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lwn.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b="bi8xpYjU" DKIM-Filter: OpenDKIM Filter v2.11.0 ms.lwn.net 871D341F30 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lwn.net; s=20201203; t=1750538131; bh=Rl9BVfHMxyNwzTDlVL5SRCWJsUt7PJhS8xa7wZIJ02w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bi8xpYjUU8puIsuAAbWRj+eEVGeLHTbH0oyg162nLGUEQF6/C4irMhjIHo7XSOCcK IOvysHv3zz/CcPfu9wzy5SQ8RIW2Ox2gsUH5ho9OHgMxPFba6jPGZEJzLWHw6WUDvo PumV3QmbwnqHF58/Co+jxbHzIltseFeKWfMJCF/MmBtEdomeoppAUfkPhuQBXRYY9W PWkA1zm+eX476LXz4g8syzxxA++DgtXXSHRjnQgU4BcCdwYCC1rY8RlpWkXBAyOyI8 vVGPEu1uBb2pVfPPKD0H1yj6vZLAO2H0g/cFdVDclRpgY3AutSc4w5/oJJReQIuoPF nmynTzutTL6EQ== Received: from trenco.lwn.net (unknown [IPv6:2601:280:4600:2da9:67c:16ff:fe81:5f9b]) by ms.lwn.net (Postfix) with ESMTPA id 871D341F30; Sat, 21 Jun 2025 20:35:31 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Akira Yokosawa , Jonathan Corbet Subject: [PATCH 8/9] docs: kdoc: Add some comments to process_decl() Date: Sat, 21 Jun 2025 14:35:11 -0600 Message-ID: <20250621203512.223189-9-corbet@lwn.net> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250621203512.223189-1-corbet@lwn.net> References: <20250621203512.223189-1-corbet@lwn.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the function can actually fit into a human brain, add a few comments. While I was at it, I switched to the trim_whitespace() helper rather than open-coding it. Signed-off-by: Jonathan Corbet Reviewed-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_parser.py | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index f7a5b85a8ed7..a6ee8bac378d 100644 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -1377,26 +1377,28 @@ class KernelDoc: """ if self.is_new_section(ln, line) or self.is_comment_end(ln, line): return - + # + # Look for anything with the " * " line beginning. + # if doc_content.search(line): cont =3D doc_content.group(1) - + # + # A blank line means that we have moved out of the declaration + # part of the comment (without any "special section" parameter + # descriptions). + # if cont =3D=3D "": self.state =3D state.BODY self.entry.contents +=3D "\n" # needed? - + # + # Otherwise we have more of the declaration section to soak up. + # else: - # Continued declaration purpose - self.entry.declaration_purpose =3D self.entry.declaration_= purpose.rstrip() - self.entry.declaration_purpose +=3D " " + cont - - r =3D KernRe(r"\s+") - self.entry.declaration_purpose =3D r.sub(' ', - self.entry.declarat= ion_purpose) - return - - # Unknown line, ignore - self.emit_msg(ln, f"bad line: {line}") + self.entry.declaration_purpose =3D \ + trim_whitespace(self.entry.declaration_purpose + ' ' += cont) + else: + # Unknown line, ignore + self.emit_msg(ln, f"bad line: {line}") =20 =20 def process_special(self, ln, line): --=20 2.49.0 From nobody Thu Oct 9 01:13:37 2025 Received: from ms.lwn.net (ms.lwn.net [45.79.88.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFC9F272E42; Sat, 21 Jun 2025 20:35:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.79.88.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538134; cv=none; b=T6f4mDgv/shnNjmdPQe9WKh/kl6m7pgUwse3vjzGV/oNC7x8hieYKPwJgtcix8DR7hahga4m40wbyqwXoxBAM3YZyhNYiuspF8LiNCW+e0KhXngG5YdhskmL8XXDU4LYAfyGK3SZ4gbld0I0qDu7A2xpODXCDCgaSGL9XBCejk4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750538134; c=relaxed/simple; bh=ozdZY6KDzV6ie0g+Vi70jxsyCl/MWPzeGOSBtx/vfqs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dd0rM5XjSCiMdaChBWJrZSACB0choOriEBX+HBLD12fQBMeD+W1lVp1AVIzKn4Wcu8KT8mocDhfQv8fzXcrdAafCybJRi4nfeQXn6dnH4ZKXwXHUPqd6Fg76SM47mb9akEaWsIo5N0w4YCVlB4FWRA879wKICw7nqWKllCNVhss= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net; spf=pass smtp.mailfrom=lwn.net; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b=bVRh2zv5; arc=none smtp.client-ip=45.79.88.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lwn.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lwn.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lwn.net header.i=@lwn.net header.b="bVRh2zv5" DKIM-Filter: OpenDKIM Filter v2.11.0 ms.lwn.net 18C8F41F33 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lwn.net; s=20201203; t=1750538132; bh=lxJdgmDPCnFc/wdxyRX/baI+nLdfvPrKONhfAX9HTxw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bVRh2zv50ruq9t+3JB7R/Txc4s7Aapvz2onZp3z8EmhIg2LRdBpxAghjmwf2P6PNJ PxWpM2lYfxSfcFG3l7/UW028EqzvbBw/6OUttnlRGQpJ/IXZ45JKYScBQPmok9uLTW 8SPv2tnWp7RYITqzYh0Y5OSkxoVgHu2XVDkW530xiL4SiMdSDY2MuEyBNFmBf8E8gM goYbbhbvRFKRfldJLAX15mGMibMmZHS2ZkbMM7IfUODogMLTyr5bxwDG7MkfuMiNGz /S839Dt2naqAQ+oAuXR5OSyPwwPnHenZT74W3sV2x5PeDoSb8MfrNo+0798rUOpM3Y uNH4JlsmJfLgQ== Received: from trenco.lwn.net (unknown [IPv6:2601:280:4600:2da9:67c:16ff:fe81:5f9b]) by ms.lwn.net (Postfix) with ESMTPA id 18C8F41F33; Sat, 21 Jun 2025 20:35:32 +0000 (UTC) From: Jonathan Corbet To: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Akira Yokosawa , Jonathan Corbet Subject: [PATCH 9/9] docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION states Date: Sat, 21 Jun 2025 14:35:12 -0600 Message-ID: <20250621203512.223189-10-corbet@lwn.net> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250621203512.223189-1-corbet@lwn.net> References: <20250621203512.223189-1-corbet@lwn.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the last SPECIAL_SECTION special case into the proper handler function, getting rid of more if/then/else logic. The leading-space tracking was tightened up a bit in the move. Add some comments describing what is going on. No changes to the generated output. Signed-off-by: Jonathan Corbet Reviewed-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_parser.py | 80 ++++++++++++++++++++------------- 1 file changed, 48 insertions(+), 32 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index a6ee8bac378d..3557c512c85a 100644 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -1405,10 +1405,53 @@ class KernelDoc: """ STATE_SPECIAL_SECTION: a section ending with a blank line """ + # + # If we have hit a blank line (only the " * " marker), then this + # section is done. + # if KernRe(r"\s*\*\s*$").match(line): self.entry.begin_section(ln, dump =3D True) + self.entry.contents +=3D '\n' self.state =3D state.BODY - self.process_body(ln, line) + return + # + # Not a blank line, look for the other ways to end the section. + # + if self.is_new_section(ln, line) or self.is_comment_end(ln, line): + return + # + # OK, we should have a continuation of the text for this section. + # + if doc_content.search(line): + cont =3D doc_content.group(1) + # + # If the lines of text after the first in a special section ha= ve + # leading white space, we need to trim it out or Sphinx will g= et + # confused. For the second line (the None case), see what we + # find there and remember it. + # + if self.entry.leading_space is None: + r =3D KernRe(r'^(\s+)') + if r.match(cont): + self.entry.leading_space =3D len(r.group(1)) + else: + self.entry.leading_space =3D 0 + # + # Otherwise, before trimming any leading chars, be *sure* + # that they are white space. We should maybe warn if this + # isn't the case. + # + for i in range(0, self.entry.leading_space): + if cont[i] !=3D " ": + self.entry.leading_space =3D i + break + # + # Add the trimmed result to the section and we're done. + # + self.entry.contents +=3D cont[self.entry.leading_space:] + '\n' + else: + # Unknown line, ignore + self.emit_msg(ln, f"bad line: {line}") =20 def process_body(self, ln, line): """ @@ -1419,37 +1462,10 @@ class KernelDoc: =20 if doc_content.search(line): cont =3D doc_content.group(1) - - if cont =3D=3D "": - self.entry.contents +=3D "\n" - else: - if self.state =3D=3D state.SPECIAL_SECTION: - if self.entry.leading_space is None: - r =3D KernRe(r'^(\s+)') - if r.match(cont): - self.entry.leading_space =3D len(r.group(1)) - else: - self.entry.leading_space =3D 0 - - # Double-check if leading space are realy spaces - pos =3D 0 - for i in range(0, self.entry.leading_space): - if cont[i] !=3D " ": - break - pos +=3D 1 - - cont =3D cont[pos:] - - # NEW LOGIC: - # In case it is different, update it - if self.entry.leading_space !=3D pos: - self.entry.leading_space =3D pos - - self.entry.contents +=3D cont + "\n" - return - - # Unknown line, ignore - self.emit_msg(ln, f"bad line: {line}") + self.entry.contents +=3D cont + "\n" + else: + # Unknown line, ignore + self.emit_msg(ln, f"bad line: {line}") =20 def process_inline(self, ln, line): """STATE_INLINE: docbook comments within a prototype.""" --=20 2.49.0