From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57E72279791; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=sQAob0fqpio84UkoNHPIVJhWa/NVAc+rfSgQybzE6qsUoqIqjlnFgZEj/Yodvs/v0NKnwBmZ3hsQyfy4CXaFOv+BQdlsuoVCRYwjrwsJFrAhfewLA32tSUTCHHUo0bv+B+AJJtzySrPffSa+/cxB/1ORXlq6bLDh9GlZ78Qtw9k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=XDxTPT2c6W8eHR7BT6Ec6vb15gsGsFJ8CPPzepOj7LY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aEQV5HtrA/WcFw4+R9AXahMpX3/C38kyJNTh3U0kleUiHzKWrQufro+ZKt9dsSCkHJRRaIrCHHJXa+5CAWzHqKw0hT9jideMhGlcce9V63K9A1xWsbcN/RCdSyZIEt5Q/5o4BdKFMo2sjs7CsekyaOAehzWT+YIBHmcHR+ufF3E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AjCPEr+S; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AjCPEr+S" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0C6E1C4CEEB; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=XDxTPT2c6W8eHR7BT6Ec6vb15gsGsFJ8CPPzepOj7LY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AjCPEr+S/XdiTJptR5n3uyAVivqrZGWjiLttZqAJf4tUJZ89h0Wj0euE5S4i3bZbX 9zrmAe2gI0pyeg899Mq3yYYSzGn+eWzuOocK3XNMYy1p+yNg748v4QT1Apzku+u2BB 8335srwSXNgxc1UFYX4wRb94bMX9VpwTgv1iFxAfcuC+I/1sL/ydi4qCldmLznAsGS rSElgV2+oPFpvwbb8Hdo7WIJv2uBC5Ot8+rnxdWhf1JuqRzJCxkXf5B0Syk1pQL/Cs Gvs04mE/sYrxJS8RU1OA2TYC6rdUWoNl7Ccquk5I4mSJNaKsg68SH84lLY5/DWWOEm Iy7xsVh0lVFPw== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT84-028y; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Mauro Carvalho Chehab , linux-kernel@vger.kernel.org Subject: [PATCH 01/24] docs: parse-headers.pl: improve its debug output format Date: Thu, 21 Aug 2025 16:21:07 +0200 Message-ID: <3d23cd095d482715284fc2e0a46199b00e10e851.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Change the --debug logic to help comparing its results with a new python script. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/parse-headers.pl | 31 ++++++++++++++++++++------- 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/Documentation/sphinx/parse-headers.pl b/Documentation/sphinx/p= arse-headers.pl index 7b1458544e2e..560685926cdb 100755 --- a/Documentation/sphinx/parse-headers.pl +++ b/Documentation/sphinx/parse-headers.pl @@ -31,8 +31,6 @@ my %enums; my %enum_symbols; my %structs; =20 -require Data::Dumper if ($debug); - # # read the file and get identifiers # @@ -197,6 +195,9 @@ if ($file_exceptions) { } else { $reftype =3D $def_reftype{$type}; } + if (!$reftype) { + print STDERR "Warning: can't find ref type for $type"; + } $new =3D "$reftype:`$old <$new>`"; =20 if ($type eq "ioctl") { @@ -229,12 +230,26 @@ if ($file_exceptions) { } =20 if ($debug) { - print Data::Dumper->Dump([\%ioctls], [qw(*ioctls)]) if (%ioctls); - print Data::Dumper->Dump([\%typedefs], [qw(*typedefs)]) if (%typedefs); - print Data::Dumper->Dump([\%enums], [qw(*enums)]) if (%enums); - print Data::Dumper->Dump([\%structs], [qw(*structs)]) if (%structs); - print Data::Dumper->Dump([\%defines], [qw(*defines)]) if (%defines); - print Data::Dumper->Dump([\%enum_symbols], [qw(*enum_symbols)]) if (%enum= _symbols); + my @all_hashes =3D ( + {ioctl =3D> \%ioctls}, + {typedef =3D> \%typedefs}, + {enum =3D> \%enums}, + {struct =3D> \%structs}, + {define =3D> \%defines}, + {symbol =3D> \%enum_symbols} + ); + + foreach my $hash (@all_hashes) { + while (my ($name, $hash_ref) =3D each %$hash) { + next unless %$hash_ref; # Skip empty hashes + + print "$name:\n"; + for my $key (sort keys %$hash_ref) { + print " $key -> $hash_ref->{$key}\n"; + } + print "\n"; + } + } } =20 # --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8DBB2857D7; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=ZuXhAFaJxi43z4hvoE9UuN1wuJtoxKUEGl4s9bsC7uw3YlZBXmK54tIH9qpZT0CfGWDL33Mqnvme8chPWcQhM/lSIDwP0qPBkbQOvHFbq7/nzsmejUD4HQcFzuqs9psz6o0uXzQZmL0zqZ+qzwYX4Xp7ZaO359SZrjiibUUTjN8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=IkR2tDIMqOGFWIt+/VpJUQvrkHkeqcw1eRxVumPctes=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G3rLXJQYVC6SGicmw/QnELeYoaeR1qih4a3wCPbGu0LW+Vtg/CGZooj1k+os+Y3tmDXqQfOTpR+JP4BR+J9gyy+pNvtgKLuf6opd3EZN/YBiB27lXrLHRckspPW3IiKKsnZnUSs2qCNtKO37MpBA2sikU/KQixxnpO6Bp0rkP0k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=icc9ggfl; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="icc9ggfl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 201FCC19421; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=IkR2tDIMqOGFWIt+/VpJUQvrkHkeqcw1eRxVumPctes=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=icc9ggfljy7yd1pHywAHH+Y0Yah4ts1u52SEckexaK3AZM6YySkU29VTVtZC2OPNx W6DkXP8Hsg3creXeGKIrjpIchidUf1hh5rmKbDccHY1Qw+Jy5SkWBODAuX9TRvTqge UoXrP8CcCLjAnGxEIb4vo70Nz6RTk4WB6TSEfOu+iwdA7gIboL13xFWoXE3uA64ney lpnDhfZKKTzjpX5LcDg7t8SdigjMNUz0dHZeL0GEi7MUUxquOgGyMQyYO6syH1JZ50 BCbVmxarGv9ZQlPdOgl7g9rMLfNQBNNq4VU0wYw6YIbRcvEY+uuv1z496TqgMdq1+a xtF72kRSm0hzQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT88-090I; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , linux-kernel@vger.kernel.org Subject: [PATCH 02/24] docs: parse-headers.py: convert parse-headers.pl Date: Thu, 21 Aug 2025 16:21:08 +0200 Message-ID: <507f8916a05a1c3bebde2d154a06f41d17671452.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" When the Kernel started to use Sphinx, we had to come up with a solution to parse media headers. On that time, we didn't have much experience with Sphinx extensions. So, we came up with our own script-based solution that were basically implementing a set of rules we used to have at the Makefile. Convert it to Python, keeping it bug-compatible with the original script. While here, try to better document it. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/parse-headers.py | 429 ++++++++++++++++++++++++++ 1 file changed, 429 insertions(+) create mode 100755 Documentation/sphinx/parse-headers.py diff --git a/Documentation/sphinx/parse-headers.py b/Documentation/sphinx/p= arse-headers.py new file mode 100755 index 000000000000..b39284d21090 --- /dev/null +++ b/Documentation/sphinx/parse-headers.py @@ -0,0 +1,429 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2016 by Mauro Carvalho Chehab . +# pylint: disable=3DC0103,R0902,R0912,R0914,R0915 + +""" +Convert a C header or source file (C_FILE), into a ReStructured Text +included via ..parsed-literal block with cross-references for the +documentation files that describe the API. It accepts an optional +EXCEPTIONS_FILE with describes what elements will be either ignored or +be pointed to a non-default reference. + +The output is written at the (OUT_FILE). + +It is capable of identifying defines, functions, structs, typedefs, +enums and enum symbols and create cross-references for all of them. +It is also capable of distinguish #define used for specifying a Linux +ioctl. + +The EXCEPTIONS_FILE contains a set of rules like: + + ignore ioctl VIDIOC_ENUM_FMT + replace ioctl VIDIOC_DQBUF vidioc_qbuf + replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ :c:type:`v4l2_event_mot= ion_det` +""" + +import argparse +import os +import re +import sys + + +class ParseHeader: + """ + Creates an enriched version of a Kernel header file with cross-links + to each C data structure type. + + It is meant to allow having a more comprehensive documentation, where + uAPI headers will create cross-reference links to the code. + + It is capable of identifying defines, functions, structs, typedefs, + enums and enum symbols and create cross-references for all of them. + It is also capable of distinguish #define used for specifying a Linux + ioctl. + + By default, it create rules for all symbols and defines, but it also + allows parsing an exception file. Such file contains a set of rules + using the syntax below: + + 1. Ignore rules: + + ignore ` + + Removes the symbol from reference generation. + + 2. Replace rules: + + replace + + Replaces how old_symbol with a new reference. The new_reference can be: + - A simple symbol name; + - A full Sphinx reference. + + On both cases, can be: + - ioctl: for defines that end with _IO*, e.g. ioctl definitions + - define: for other defines + - symbol: for symbols defined within enums; + - typedef: for typedefs; + - enum: for the name of a non-anonymous enum; + - struct: for structs. + + Examples: + + ignore define __LINUX_MEDIA_H + ignore ioctl VIDIOC_ENUM_FMT + replace ioctl VIDIOC_DQBUF vidioc_qbuf + replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ :c:type:`v4l2_event= _motion_det` + """ + + # Parser regexes with multiple ways to capture enums and structs + RE_ENUMS =3D [ + re.compile(r"^\s*enum\s+([\w_]+)\s*\{"), + re.compile(r"^\s*enum\s+([\w_]+)\s*$"), + re.compile(r"^\s*typedef\s*enum\s+([\w_]+)\s*\{"), + re.compile(r"^\s*typedef\s*enum\s+([\w_]+)\s*$"), + ] + RE_STRUCTS =3D [ + re.compile(r"^\s*struct\s+([_\w][\w\d_]+)\s*\{"), + re.compile(r"^\s*struct\s+([_\w][\w\d_]+)$"), + re.compile(r"^\s*typedef\s*struct\s+([_\w][\w\d_]+)\s*\{"), + re.compile(r"^\s*typedef\s*struct\s+([_\w][\w\d_]+)$"), + ] + + # FIXME: the original code was written a long time before Sphinx C + # domain to have multiple namespaces. To avoid to much turn at the + # existing hyperlinks, the code kept using "c:type" instead of the + # right types. To change that, we need to change the types not only + # here, but also at the uAPI media documentation. + DEF_SYMBOL_TYPES =3D { + "ioctl": { + "prefix": "\\ ", + "suffix": "\\ ", + "ref_type": ":ref", + }, + "define": { + "prefix": "\\ ", + "suffix": "\\ ", + "ref_type": ":ref", + }, + # We're calling each definition inside an enum as "symbol" + "symbol": { + "prefix": "\\ ", + "suffix": "\\ ", + "ref_type": ":ref", + }, + "typedef": { + "prefix": "\\ ", + "suffix": "\\ ", + "ref_type": ":c:type", + }, + # This is the name of the enum itself + "enum": { + "prefix": "", + "suffix": "\\ ", + "ref_type": ":c:type", + }, + "struct": { + "prefix": "", + "suffix": "\\ ", + "ref_type": ":c:type", + }, + } + + def __init__(self, debug: bool =3D False): + """Initialize internal vars""" + self.debug =3D debug + self.data =3D "" + + self.symbols =3D {} + + for symbol_type in self.DEF_SYMBOL_TYPES: + self.symbols[symbol_type] =3D {} + + def store_type(self, symbol_type: str, symbol: str, + ref_name: str =3D None, replace_underscores: bool =3D T= rue): + """ + Stores a new symbol at self.symbols under symbol_type. + + By default, underscores are replaced by "-" + """ + defs =3D self.DEF_SYMBOL_TYPES[symbol_type] + + prefix =3D defs.get("prefix", "") + suffix =3D defs.get("suffix", "") + ref_type =3D defs.get("ref_type") + + # Determine ref_link based on symbol type + if ref_type: + if symbol_type =3D=3D "enum": + ref_link =3D f"{ref_type}:`{symbol}`" + else: + if not ref_name: + ref_name =3D symbol.lower() + + if replace_underscores: + ref_name =3D ref_name.replace("_", "-") + + ref_link =3D f"{ref_type}:`{symbol} <{ref_name}>`" + else: + ref_link =3D symbol + + self.symbols[symbol_type][symbol] =3D f"{prefix}{ref_link}{suffix}" + + def store_line(self, line): + """Stores a line at self.data, properly indented""" + line =3D " " + line.expandtabs() + self.data +=3D line.rstrip(" ") + + def parse_file(self, file_in: str): + """Reads a C source file and get identifiers""" + self.data =3D "" + is_enum =3D False + is_comment =3D False + multiline =3D "" + + with open(file_in, "r", + encoding=3D"utf-8", errors=3D"backslashreplace") as f: + for line_no, line in enumerate(f): + self.store_line(line) + line =3D line.strip("\n") + + # Handle continuation lines + if line.endswith(r"\\"): + multiline +=3D line[-1] + continue + + if multiline: + line =3D multiline + line + multiline =3D "" + + # Handle comments. They can be multilined + if not is_comment: + if re.search(r"/\*.*", line): + is_comment =3D True + else: + # Strip C99-style comments + line =3D re.sub(r"(//.*)", "", line) + + if is_comment: + if re.search(r".*\*/", line): + is_comment =3D False + else: + multiline =3D line + continue + + # At this point, line variable may be a multilined stateme= nt, + # if lines end with \ or if they have multi-line comments + # With that, it can safely remove the entire comments, + # and there's no need to use re.DOTALL for the logic below + + line =3D re.sub(r"(/\*.*\*/)", "", line) + if not line.strip(): + continue + + # It can be useful for debug purposes to print the file af= ter + # having comments stripped and multi-lines grouped. + if self.debug > 1: + print(f"line {line_no + 1}: {line}") + + # Now the fun begins: parse each type and store it. + + # We opted for a two parsing logic here due to: + # 1. it makes easier to debug issues not-parsed symbols; + # 2. we want symbol replacement at the entire content, not + # just when the symbol is detected. + + if is_enum: + match =3D re.match(r"^\s*([_\w][\w\d_]+)\s*[\,=3D]?", = line) + if match: + self.store_type("symbol", match.group(1)) + if "}" in line: + is_enum =3D False + continue + + match =3D re.match(r"^\s*#\s*define\s+([\w_]+)\s+_IO", lin= e) + if match: + self.store_type("ioctl", match.group(1), + replace_underscores=3DFalse) + continue + + match =3D re.match(r"^\s*#\s*define\s+([\w_]+)(\s+|$)", li= ne) + if match: + self.store_type("define", match.group(1)) + continue + + match =3D re.match(r"^\s*typedef\s+([_\w][\w\d_]+)\s+(.*)\= s+([_\w][\w\d_]+);", + line) + if match: + name =3D match.group(2).strip() + symbol =3D match.group(3) + self.store_type("typedef", symbol, ref_name=3Dname, + replace_underscores=3DFalse) + continue + + for re_enum in self.RE_ENUMS: + match =3D re_enum.match(line) + if match: + self.store_type("enum", match.group(1)) + is_enum =3D True + break + + for re_struct in self.RE_STRUCTS: + match =3D re_struct.match(line) + if match: + self.store_type("struct", match.group(1), + replace_underscores=3DFalse) + break + + def process_exceptions(self, fname: str): + """ + Process exceptions file with rules to ignore or replace references. + """ + if not fname: + return + + name =3D os.path.basename(fname) + + with open(fname, "r", encoding=3D"utf-8", errors=3D"backslashrepla= ce") as f: + for ln, line in enumerate(f): + ln +=3D 1 + line =3D line.strip() + if not line or line.startswith("#"): + continue + + # Handle ignore rules + match =3D re.match(r"^ignore\s+(\w+)\s+(\S+)", line) + if match: + c_type =3D match.group(1) + symbol =3D match.group(2) + + if c_type not in self.DEF_SYMBOL_TYPES: + sys.exit(f"{name}:{ln}: {c_type} is invalid") + + d =3D self.symbols[c_type] + if symbol in d: + del d[symbol] + + continue + + # Handle replace rules + match =3D re.match(r"^replace\s+(\S+)\s+(\S+)\s+(\S+)", li= ne) + if not match: + sys.exit(f"{name}:{ln}: invalid line: {line}") + + c_type, old, new =3D match.groups() + + if c_type not in self.DEF_SYMBOL_TYPES: + sys.exit(f"{name}:{ln}: {c_type} is invalid") + + reftype =3D None + + # Parse reference type when the type is specified + + match =3D re.match(r"^\:c\:(data|func|macro|type)\:\`(.+)\= `", new) + if match: + reftype =3D f":c:{match.group(1)}" + new =3D match.group(2) + else: + match =3D re.search(r"(\:ref)\:\`(.+)\`", new) + if match: + reftype =3D match.group(1) + new =3D match.group(2) + + # If the replacement rule doesn't have a type, get default + if not reftype: + reftype =3D self.DEF_SYMBOL_TYPES[c_type].get("ref_typ= e") + if not reftype: + reftype =3D self.DEF_SYMBOL_TYPES[c_type].get("rea= l_type") + + new_ref =3D f"{reftype}:`{old} <{new}>`" + + # Change self.symbols to use the replacement rule + if old in self.symbols[c_type]: + self.symbols[c_type][old] =3D new_ref + else: + print(f"{name}:{ln}: Warning: can't find {old} {c_type= }") + + def debug_print(self): + """ + Print debug information containing the replacement rules per symbo= l. + To make easier to check, group them per type. + """ + if not self.debug: + return + + for c_type, refs in self.symbols.items(): + if not refs: # Skip empty dictionaries + continue + + print(f"{c_type}:") + + for symbol, ref in sorted(refs.items()): + print(f" {symbol} -> {ref}") + + print() + + def write_output(self, file_in: str, file_out: str): + """Write the formatted output to a file.""" + + # Avoid extra blank lines + text =3D re.sub(r"\s+$", "", self.data) + "\n" + text =3D re.sub(r"\n\s+\n", "\n\n", text) + + # Escape Sphinx special characters + text =3D re.sub(r"([\_\`\*\<\>\&\\\\:\/\|\%\$\#\{\}\~\^])", r"\\\1= ", text) + + # Source uAPI files may have special notes. Use bold font for them + text =3D re.sub(r"DEPRECATED", "**DEPRECATED**", text) + + # Delimiters to catch the entire symbol after escaped + start_delim =3D r"([ \n\t\(=3D\*\@])" + end_delim =3D r"(\s|,|\\=3D|\\:|\;|\)|\}|\{)" + + # Process all reference types + for ref_dict in self.symbols.values(): + for symbol, replacement in ref_dict.items(): + symbol =3D re.escape(re.sub(r"([\_\`\*\<\>\&\\\\:\/])", r"= \\\1", symbol)) + text =3D re.sub(fr'{start_delim}{symbol}{end_delim}', + fr'\1{replacement}\2', text) + + # Remove "\ " where not needed: before spaces and at the end of li= nes + text =3D re.sub(r"\\ ([\n ])", r"\1", text) + + title =3D os.path.basename(file_in) + + with open(file_out, "w", encoding=3D"utf-8", errors=3D"backslashre= place") as f: + f.write(".. -*- coding: utf-8; mode: rst -*-\n\n") + f.write(f"{title}\n") + f.write("=3D" * len(title)) + f.write("\n\n.. parsed-literal::\n\n") + f.write(text) + + +def main(): + """Main function""" + parser =3D argparse.ArgumentParser(description=3D__doc__, + formatter_class=3Dargparse.RawDescrip= tionHelpFormatter) + + parser.add_argument("-d", "--debug", action=3D"count", default=3D0, + help=3D"Increase debug level. Can be used multiple= times") + parser.add_argument("file_in", help=3D"Input C file") + parser.add_argument("file_out", help=3D"Output RST file") + parser.add_argument("file_exceptions", nargs=3D"?", + help=3D"Exceptions file (optional)") + + args =3D parser.parse_args() + + parser =3D ParseHeader(debug=3Dargs.debug) + parser.parse_file(args.file_in) + + if args.file_exceptions: + parser.process_exceptions(args.file_exceptions) + + parser.debug_print() + parser.write_output(args.file_in, args.file_out) + + +if __name__ =3D=3D "__main__": + main() --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 81D9D27AC2A; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=GPQzK7hhGflfHyz6y+xDOCVEiJwTUEmbPk/1CARhhpJdvUtDCJPUtGuAGgIgvc282m5xA5l95k9pK1bX94QYYBwhQd9DNfxIZJ0l+u+ukO7VJQNuNd7VSM0XJV35+VKXDBNwbO89+1aEpYvAhQeBn5gSpBGoIUzjg0tqmiuEJ+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=/wRz/bm5opGP5gUAlYf6YzB0TWc+kftZ82g5lfHklv8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=J1kc9C0HkBngFhG5YEGQq5SRoi0qFEcm2Tq7NCItAAB48rgcmOEUww+ozZI/yobPruWzG3nXXXh2w0L+x/zfwmm8MLiq3W9tE+g6klL4ccCkqITOWCAt37rDAlv6DoR6vIb6Xp0PKcwZdVjTNoOBIXOneQa/PvobaT5oFZzXNck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LkM3VPIC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LkM3VPIC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 167BBC116C6; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=/wRz/bm5opGP5gUAlYf6YzB0TWc+kftZ82g5lfHklv8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LkM3VPICjUXquRQ26JDE0AY8UPKLVzZkS6t0WwXQCyviz7+fz266jVJPYOpAddtGg HRJzpMhM0vAGRjivgBf/kEE9eNwtvoWcWDoFPKeguA+gvU5aZrSDiNaHcfCPd2F5TN Tk0LY8v3FHyEb4el4oU3PPQZsWXliSIIRg4NoXm6VdAFw2qJkiwvjCXJqoSWTEDdM6 bll6wcehDP9NiYXxmMEkrtBR6+jHOgqeKvYI2kXe99H1pfzAaPJwdBIHRsic1J6CbB QhsEVBI7X0qq2NAbIAg/zUjAH6nNXhOuTCKnQpKY5ps6K4R7328q1eSxaxH05b6dzQ LCi39YZMarjmw== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8C-0Fgr; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , linux-kernel@vger.kernel.org Subject: [PATCH 03/24] docs: parse-headers.py: improve --help logic Date: Thu, 21 Aug 2025 16:21:09 +0200 Message-ID: <8ffefe7813007a4a55c1dbb43da09616250e7037.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" When printing --help, we'd like the name of the files from __doc__ to match the displayed positional arguments at both usage and argument description lines. Use a custom formatter class to convert ``foo`` into ANSI SGR code to bold the argument, if is TTY, and adjust the help text to match the argument names. Here on Plasma, that makes it display it colored, wich is really cool. Yet, I opted for SGR, as the best is to follow the terminal color schema for bold. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/parse-headers.py | 67 +++++++++++++++++++++++---- 1 file changed, 58 insertions(+), 9 deletions(-) diff --git a/Documentation/sphinx/parse-headers.py b/Documentation/sphinx/p= arse-headers.py index b39284d21090..650f9c9a68d1 100755 --- a/Documentation/sphinx/parse-headers.py +++ b/Documentation/sphinx/parse-headers.py @@ -4,20 +4,20 @@ # pylint: disable=3DC0103,R0902,R0912,R0914,R0915 =20 """ -Convert a C header or source file (C_FILE), into a ReStructured Text +Convert a C header or source file ``FILE_IN``, into a ReStructured Text included via ..parsed-literal block with cross-references for the documentation files that describe the API. It accepts an optional -EXCEPTIONS_FILE with describes what elements will be either ignored or -be pointed to a non-default reference. +``FILE_RULES`` file to describes what elements will be either ignored or +be pointed to a non-default reference type/name. =20 -The output is written at the (OUT_FILE). +The output is written at ``FILE_OUT``. =20 It is capable of identifying defines, functions, structs, typedefs, enums and enum symbols and create cross-references for all of them. It is also capable of distinguish #define used for specifying a Linux ioctl. =20 -The EXCEPTIONS_FILE contains a set of rules like: +The optional ``FILE_RULES`` contains a set of rules like: =20 ignore ioctl VIDIOC_ENUM_FMT replace ioctl VIDIOC_DQBUF vidioc_qbuf @@ -400,17 +400,66 @@ class ParseHeader: f.write("\n\n.. parsed-literal::\n\n") f.write(text) =20 +class EnrichFormatter(argparse.HelpFormatter): + """ + Better format the output, making easier to identify the positional args + and how they're used at the __doc__ description. + """ + def __init__(self, *args, **kwargs): + """Initialize class and check if is TTY""" + super().__init__(*args, **kwargs) + self._tty =3D sys.stdout.isatty() + + def enrich_text(self, text): + """Handle ReST markups (currently, only ``foo``)""" + if self._tty and text: + # Replace ``text`` with ANSI bold + return re.sub(r'\`\`(.+?)\`\`', + lambda m: f'\033[1m{m.group(1)}\033[0m', text) + return text + + def _fill_text(self, text, width, indent): + """Enrich descriptions with markups on it""" + enriched =3D self.enrich_text(text) + return "\n".join(indent + line for line in enriched.splitlines()) + + def _format_usage(self, usage, actions, groups, prefix): + """Enrich positional arguments at usage: line""" + + prog =3D self._prog + parts =3D [] + + for action in actions: + if action.option_strings: + opt =3D action.option_strings[0] + if action.nargs !=3D 0: + opt +=3D f" {action.dest.upper()}" + parts.append(f"[{opt}]") + else: + # Positional argument + parts.append(self.enrich_text(f"``{action.dest.upper()}``"= )) + + usage_text =3D f"{prefix or 'usage: '} {prog} {' '.join(parts)}\n" + return usage_text + + def _format_action_invocation(self, action): + """Enrich argument names""" + if not action.option_strings: + return self.enrich_text(f"``{action.dest.upper()}``") + else: + return ", ".join(action.option_strings) + =20 def main(): """Main function""" parser =3D argparse.ArgumentParser(description=3D__doc__, - formatter_class=3Dargparse.RawDescrip= tionHelpFormatter) + formatter_class=3DEnrichFormatter) =20 parser.add_argument("-d", "--debug", action=3D"count", default=3D0, help=3D"Increase debug level. Can be used multiple= times") parser.add_argument("file_in", help=3D"Input C file") parser.add_argument("file_out", help=3D"Output RST file") - parser.add_argument("file_exceptions", nargs=3D"?", + parser.add_argument("file_rules", nargs=3D"?", help=3D"Exceptions file (optional)") =20 args =3D parser.parse_args() @@ -418,8 +467,8 @@ def main(): parser =3D ParseHeader(debug=3Dargs.debug) parser.parse_file(args.file_in) =20 - if args.file_exceptions: - parser.process_exceptions(args.file_exceptions) + if args.file_rules: + parser.process_exceptions(args.file_rules) =20 parser.debug_print() parser.write_output(args.file_in, args.file_out) --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B29E28134C; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=RVd8OD0tyC94gxDSJTfXw5mPW0pjWZUgCui6l0jlwGf0qhEPJKlZhHXUgGcBlNiZNIFy0ps2Y/dIvnYMQ4UtpvSZeV3iLF2m+AQvyOFqJLwVXuDOoH2a31GfEZKiOlYTp1rqnFm80/6iHZY8IQttCbGaFWqkF4HiPTJxqnZdYDc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=HiN7NqutO6hirOm4EnCnsuZhXt+MMY3adGpEmCAC7NY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=blJVjb3H0M/ojp3ExZm+Xj7A/G/ygnkKfWTEGonK1nQBfE7FqFOsXZ6h27G2P0F1CzAbB0aj5PSrTBs3uI/YWI1TIOGcbZbFYkcb9O9JVlDbDJJ55B5np46iD2zJxRSCj2zeiQ6+GV86uOYj/P9jDlRs3E840iftAGpkbwoL1NE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TYV5s84B; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TYV5s84B" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1B7E3C4CEED; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=HiN7NqutO6hirOm4EnCnsuZhXt+MMY3adGpEmCAC7NY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TYV5s84Bgs1X5htuk44nmyIRe+YxE4Ga/73kTS/2zGuuZQwn9W0eiJAzNuPo1RcPD JQh1asCNqz5GDYR8NqdO0xr53xgOU2sfKEHu56W+8GpyVMHYloL2/nwUrW4EJKHcK9 G3aAqn1IKswVYU+UjQWZcodtDKPoewqCw59wROqPSrDWstHo7tDYSTHncZ7utpRzlZ 1EN8ReMFSZgyz2nyLuSh+/ZobxAiXXVwD+hRsxcqEuybel3t//mx/ZFHLMpslF9qN7 cT4u1929Q2s4e8SqsO9qAZdjxC6ghDuP4oyVAjaQwJLJgEkpjlrB0h9w4AwDIj4Ibf 3ClSBa06FP/Zg== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8G-0MP6; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , linux-kernel@vger.kernel.org Subject: [PATCH 04/24] docs: parse-headers.py: better handle @var arguments Date: Thu, 21 Aug 2025 16:21:10 +0200 Message-ID: <7a8bb6f607dd3d614cd27ae1c3176aab404605ee.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The kernel-doc markups inside headers may contain @var markups. With the current rule, this would be converted into: \* @:c:type:`DMX_BUFFER_FLAG_DISCONTINUITY_INDICATOR `\: Fix it adding a non-printed space if needed. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/parse-headers.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/Documentation/sphinx/parse-headers.py b/Documentation/sphinx/p= arse-headers.py index 650f9c9a68d1..f4ab9c49d2f5 100755 --- a/Documentation/sphinx/parse-headers.py +++ b/Documentation/sphinx/parse-headers.py @@ -120,12 +120,12 @@ class ParseHeader: }, # This is the name of the enum itself "enum": { - "prefix": "", + "prefix": "\\ ", "suffix": "\\ ", "ref_type": ":c:type", }, "struct": { - "prefix": "", + "prefix": "\\ ", "suffix": "\\ ", "ref_type": ":c:type", }, @@ -390,6 +390,8 @@ class ParseHeader: =20 # Remove "\ " where not needed: before spaces and at the end of li= nes text =3D re.sub(r"\\ ([\n ])", r"\1", text) + text =3D re.sub(r" \\ ", " ", text) + =20 title =3D os.path.basename(file_in) =20 --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6ABA4279DBB; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=e5RV5fbJUEteFUJQavPIvXBPXjo5KA0rwVK/u3Rubdnz6Snuao76pFxPdgGmtOVnWH9A7OcXbw7Etb7e4kHvWQQvwz32vYeuo7nYb/vKn/ELrD4oSZ9v384b2DDvQTDjXJ9xW4qDXB9NUPxrR7evNd8tVh6sJYWmjH4dTmgb29M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=xaVYPkTp3UCaFGLlpkZHV5Y3yGQI0qRfBiGX9ydW30c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VOzYofgx58BZSXJ/D1IsuJELJDYJV/FnC/m4t817ry20eyoDChlJIRj6K/Xd+yG0VVsSob5RwKdbn4JO6OHBSXVTfJZvrKrXQL8ajSBAD5jQvpsBo2FbmbKZgISw/0EL9q7NuBNCA3xlFFjttN01QTK0maHaho8aqTJiiYzeAxM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rGK50HqV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rGK50HqV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 12CEEC113CF; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=xaVYPkTp3UCaFGLlpkZHV5Y3yGQI0qRfBiGX9ydW30c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rGK50HqV4D2ftJ2wJq/Of1l9eBbyC4BggYHUAfuBniHo9urdjCJ7IDlYwSoMBj7gF gm8NDxRgbuQW4RFF5/xx++rbT64HbhwNvgRzBosFVnQwgmQ1fsTyYpL/2il5sMBtN0 b2mnyjM0zsZcyf20ZmPXNa+Ug/hj4vYeJll7axZebnqjQQy5RpexOuVrBqhRNa+Xc1 mwOCWaGkwrsV4p39o+Zorhx1wORB0g2132toww4ME8F6rq2fBWZWwSvYpClJaHMUgj CPHyCEntf1vSxs0PZBFWkx1soa+/Ot0V1V7of1I8qZlc+hmybUXqwSqJPyzxFLyXgG edf6tdxcgRJpA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8K-0TDw; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , linux-kernel@vger.kernel.org Subject: [PATCH 05/24] docs: parse-headers.py: simplify the rules for hashes Date: Thu, 21 Aug 2025 16:21:11 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Normal :ref domain accept either hashes or underscores, but c-domain ones don't. Fix it and remove unneeded places where we opt to disable underscore transformation. Ideally, we should have a rule about the default, or change the way media docs have their references. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/parse-headers.py | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/Documentation/sphinx/parse-headers.py b/Documentation/sphinx/p= arse-headers.py index f4ab9c49d2f5..344090ef259c 100755 --- a/Documentation/sphinx/parse-headers.py +++ b/Documentation/sphinx/parse-headers.py @@ -162,7 +162,8 @@ class ParseHeader: if not ref_name: ref_name =3D symbol.lower() =20 - if replace_underscores: + # c-type references don't support hash + if ref_type =3D=3D ":ref" and replace_underscores: ref_name =3D ref_name.replace("_", "-") =20 ref_link =3D f"{ref_type}:`{symbol} <{ref_name}>`" @@ -258,8 +259,7 @@ class ParseHeader: if match: name =3D match.group(2).strip() symbol =3D match.group(3) - self.store_type("typedef", symbol, ref_name=3Dname, - replace_underscores=3DFalse) + self.store_type("typedef", symbol, ref_name=3Dname) continue =20 for re_enum in self.RE_ENUMS: @@ -272,8 +272,7 @@ class ParseHeader: for re_struct in self.RE_STRUCTS: match =3D re_struct.match(line) if match: - self.store_type("struct", match.group(1), - replace_underscores=3DFalse) + self.store_type("struct", match.group(1)) break =20 def process_exceptions(self, fname: str): --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D90732857DA; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=cMwJLAjEv1vq3k7mFMiwL3Bcza6kWRKrrLFE0HcD382KDaaNL8dzS3jDFZ9EA3QYTim0itdZcffarkg2N0DdRtnzZUPJej1tZ3SL0Z4tUB+JZletwuSg3JlydlsFblv6rOiH/4qUjB3FAYTiEtaO9a39pt151KqpWy6exArHIcQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=RcaqdZwLfefMvyrZbcNpDpPDg5fNh39ClTRYBACxEw0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EgK6PM/DnMR59h/oCD+amoqeHD9er5lGVwbuE9My6z74rL346KTVpFJLgFIgBhgoO7algsl4Kn1jEiLKhEbpniFGu8/zrq//+bQkg6qGxOarH/AxuYl21DlbTSnyaoPXD/G5hmOSiL9d1GD0WIMthwLqzSmguVr+H3DbEJVHqKw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZPvniHZH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZPvniHZH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2761FC19424; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=RcaqdZwLfefMvyrZbcNpDpPDg5fNh39ClTRYBACxEw0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZPvniHZHCg+vn/mYW+ke9xN7gioni/R3dB7XDYnyFN1EZAjMKckMLYhN3qeSrhicc N85vgcLuNSQsfHI0YQG9fiFeUKFtB7suMPDsNYi7okinQ7QNPLtKXMmTnzGS1/q3/R sgyVe2geQlowjhFHnTLm5gPVLdwkxNxSPAu3McYLWPO+yMShuHy7RC3GMWSTDPgSEB SYmfNrTK55S4/42ZlrP4FD6LD8NFM2XgP/kqIJyP18VcFIoQSxJIcl/jXz41sI0gNH ee/0TYo3zS4wItAFYgDSFfRdNlRlaYRvIuy98/h+lDw3gY21i9mq8KcfutK76Vqu6/ zw6w0OBRKnYfg== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8O-0a5b; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Donald Hunter , Jakub Kicinski , Jan Stancek , linux-kernel@vger.kernel.org Subject: [PATCH 06/24] tools: docs: parse-headers.py: move it from sphinx dir Date: Thu, 21 Aug 2025 16:21:12 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" As suggested by Jon, we should start having a tools/docs directory, instead of placing everything under scripts. In the specific case of parse-headers.py, the previous location is where we're placing Sphinx extensions, which is not the right place for execs. Move it to tools/docs/parse-headers.py. Signed-off-by: Mauro Carvalho Chehab --- .pylintrc | 2 +- tools/docs/lib/__init__.py | 0 tools/docs/lib/enrich_formatter.py | 70 ++++++++++++++ .../docs/lib/parse_data_structs.py | 95 ++----------------- tools/docs/parse-headers.py | 57 +++++++++++ 5 files changed, 135 insertions(+), 89 deletions(-) create mode 100644 tools/docs/lib/__init__.py create mode 100644 tools/docs/lib/enrich_formatter.py rename Documentation/sphinx/parse-headers.py =3D> tools/docs/lib/parse_dat= a_structs.py (80%) create mode 100755 tools/docs/parse-headers.py diff --git a/.pylintrc b/.pylintrc index f1d21379254b..ad2476751f80 100644 --- a/.pylintrc +++ b/.pylintrc @@ -1,2 +1,2 @@ [MASTER] -init-hook=3D'import sys; sys.path +=3D ["scripts/lib", "scripts/lib/kdoc",= "scripts/lib/abi"]' +init-hook=3D'import sys; sys.path +=3D ["scripts/lib", "scripts/lib/kdoc",= "scripts/lib/abi", "tools/docs/lib"]' diff --git a/tools/docs/lib/__init__.py b/tools/docs/lib/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/docs/lib/enrich_formatter.py b/tools/docs/lib/enrich_for= matter.py new file mode 100644 index 000000000000..bb171567a4ca --- /dev/null +++ b/tools/docs/lib/enrich_formatter.py @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 by Mauro Carvalho Chehab . + +""" +Ancillary argparse HelpFormatter class that works on a similar way as +argparse.RawDescriptionHelpFormatter, e.g. description maintains line +breaks, but it also implement transformations to the help text. The +actual transformations ar given by enrich_text(), if the output is tty. + +Currently, the follow transformations are done: + + - Positional arguments are shown in upper cases; + - if output is TTY, ``var`` and positional arguments are shown prepend= ed + by an ANSI SGR code. This is usually translated to bold. On some + terminals, like, konsole, this is translated into a colored bold tex= t. +""" + +import argparse +import re +import sys + +class EnrichFormatter(argparse.HelpFormatter): + """ + Better format the output, making easier to identify the positional args + and how they're used at the __doc__ description. + """ + def __init__(self, *args, **kwargs): + """Initialize class and check if is TTY""" + super().__init__(*args, **kwargs) + self._tty =3D sys.stdout.isatty() + + def enrich_text(self, text): + """Handle ReST markups (currently, only ``foo``)""" + if self._tty and text: + # Replace ``text`` with ANSI SGR (bold) + return re.sub(r'\`\`(.+?)\`\`', + lambda m: f'\033[1m{m.group(1)}\033[0m', text) + return text + + def _fill_text(self, text, width, indent): + """Enrich descriptions with markups on it""" + enriched =3D self.enrich_text(text) + return "\n".join(indent + line for line in enriched.splitlines()) + + def _format_usage(self, usage, actions, groups, prefix): + """Enrich positional arguments at usage: line""" + + prog =3D self._prog + parts =3D [] + + for action in actions: + if action.option_strings: + opt =3D action.option_strings[0] + if action.nargs !=3D 0: + opt +=3D f" {action.dest.upper()}" + parts.append(f"[{opt}]") + else: + # Positional argument + parts.append(self.enrich_text(f"``{action.dest.upper()}``"= )) + + usage_text =3D f"{prefix or 'usage: '} {prog} {' '.join(parts)}\n" + return usage_text + + def _format_action_invocation(self, action): + """Enrich argument names""" + if not action.option_strings: + return self.enrich_text(f"``{action.dest.upper()}``") + + return ", ".join(action.option_strings) diff --git a/Documentation/sphinx/parse-headers.py b/tools/docs/lib/parse_d= ata_structs.py similarity index 80% rename from Documentation/sphinx/parse-headers.py rename to tools/docs/lib/parse_data_structs.py index 344090ef259c..2b7fa6bd8321 100755 --- a/Documentation/sphinx/parse-headers.py +++ b/tools/docs/lib/parse_data_structs.py @@ -1,36 +1,32 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: GPL-2.0 -# Copyright (c) 2016 by Mauro Carvalho Chehab . -# pylint: disable=3DC0103,R0902,R0912,R0914,R0915 +# Copyright (c) 2016-2025 by Mauro Carvalho Chehab . +# pylint: disable=3DR0912,R0915 =20 """ -Convert a C header or source file ``FILE_IN``, into a ReStructured Text -included via ..parsed-literal block with cross-references for the -documentation files that describe the API. It accepts an optional -``FILE_RULES`` file to describes what elements will be either ignored or -be pointed to a non-default reference type/name. +Parse a source file or header, creating ReStructured Text cross references. =20 -The output is written at ``FILE_OUT``. +It accepts an optional file to change the default symbol reference or to +suppress symbols from the output. =20 It is capable of identifying defines, functions, structs, typedefs, enums and enum symbols and create cross-references for all of them. It is also capable of distinguish #define used for specifying a Linux ioctl. =20 -The optional ``FILE_RULES`` contains a set of rules like: +The optional rules file contains a set of rules like: =20 ignore ioctl VIDIOC_ENUM_FMT replace ioctl VIDIOC_DQBUF vidioc_qbuf replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ :c:type:`v4l2_event_mot= ion_det` """ =20 -import argparse import os import re import sys =20 =20 -class ParseHeader: +class ParseDataStructs: """ Creates an enriched version of a Kernel header file with cross-links to each C data structure type. @@ -400,80 +396,3 @@ class ParseHeader: f.write("=3D" * len(title)) f.write("\n\n.. parsed-literal::\n\n") f.write(text) - -class EnrichFormatter(argparse.HelpFormatter): - """ - Better format the output, making easier to identify the positional args - and how they're used at the __doc__ description. - """ - def __init__(self, *args, **kwargs): - """Initialize class and check if is TTY""" - super().__init__(*args, **kwargs) - self._tty =3D sys.stdout.isatty() - - def enrich_text(self, text): - """Handle ReST markups (currently, only ``foo``)""" - if self._tty and text: - # Replace ``text`` with ANSI bold - return re.sub(r'\`\`(.+?)\`\`', - lambda m: f'\033[1m{m.group(1)}\033[0m', text) - return text - - def _fill_text(self, text, width, indent): - """Enrich descriptions with markups on it""" - enriched =3D self.enrich_text(text) - return "\n".join(indent + line for line in enriched.splitlines()) - - def _format_usage(self, usage, actions, groups, prefix): - """Enrich positional arguments at usage: line""" - - prog =3D self._prog - parts =3D [] - - for action in actions: - if action.option_strings: - opt =3D action.option_strings[0] - if action.nargs !=3D 0: - opt +=3D f" {action.dest.upper()}" - parts.append(f"[{opt}]") - else: - # Positional argument - parts.append(self.enrich_text(f"``{action.dest.upper()}``"= )) - - usage_text =3D f"{prefix or 'usage: '} {prog} {' '.join(parts)}\n" - return usage_text - - def _format_action_invocation(self, action): - """Enrich argument names""" - if not action.option_strings: - return self.enrich_text(f"``{action.dest.upper()}``") - else: - return ", ".join(action.option_strings) - - -def main(): - """Main function""" - parser =3D argparse.ArgumentParser(description=3D__doc__, - formatter_class=3DEnrichFormatter) - - parser.add_argument("-d", "--debug", action=3D"count", default=3D0, - help=3D"Increase debug level. Can be used multiple= times") - parser.add_argument("file_in", help=3D"Input C file") - parser.add_argument("file_out", help=3D"Output RST file") - parser.add_argument("file_rules", nargs=3D"?", - help=3D"Exceptions file (optional)") - - args =3D parser.parse_args() - - parser =3D ParseHeader(debug=3Dargs.debug) - parser.parse_file(args.file_in) - - if args.file_rules: - parser.process_exceptions(args.file_rules) - - parser.debug_print() - parser.write_output(args.file_in, args.file_out) - - -if __name__ =3D=3D "__main__": - main() diff --git a/tools/docs/parse-headers.py b/tools/docs/parse-headers.py new file mode 100755 index 000000000000..07d3b47c4834 --- /dev/null +++ b/tools/docs/parse-headers.py @@ -0,0 +1,57 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2016, 2025 by Mauro Carvalho Chehab . +# pylint: disable=3DC0103 + +""" +Convert a C header or source file ``FILE_IN``, into a ReStructured Text +included via ..parsed-literal block with cross-references for the +documentation files that describe the API. It accepts an optional +``FILE_RULES`` file to describes what elements will be either ignored or +be pointed to a non-default reference type/name. + +The output is written at ``FILE_OUT``. + +It is capable of identifying defines, functions, structs, typedefs, +enums and enum symbols and create cross-references for all of them. +It is also capable of distinguish #define used for specifying a Linux +ioctl. + +The optional ``FILE_RULES`` contains a set of rules like: + + ignore ioctl VIDIOC_ENUM_FMT + replace ioctl VIDIOC_DQBUF vidioc_qbuf + replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ :c:type:`v4l2_event_mot= ion_det` +""" + +import argparse + +from lib.parse_data_structs import ParseDataStructs +from lib.enrich_formatter import EnrichFormatter + +def main(): + """Main function""" + parser =3D argparse.ArgumentParser(description=3D__doc__, + formatter_class=3DEnrichFormatter) + + parser.add_argument("-d", "--debug", action=3D"count", default=3D0, + help=3D"Increase debug level. Can be used multiple= times") + parser.add_argument("file_in", help=3D"Input C file") + parser.add_argument("file_out", help=3D"Output RST file") + parser.add_argument("file_rules", nargs=3D"?", + help=3D"Exceptions file (optional)") + + args =3D parser.parse_args() + + parser =3D ParseDataStructs(debug=3Dargs.debug) + parser.parse_file(args.file_in) + + if args.file_rules: + parser.process_exceptions(args.file_rules) + + parser.debug_print() + parser.write_output(args.file_in, args.file_out) + + +if __name__ =3D=3D "__main__": + main() --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 982D5280328; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=ABGjkJfWcFCtJRMJC61nv+RnJoYCQewFgV63JAPNDYKI9PlBDpcLGXAw+xFoEMSqz1ulZpz4gb++gRk/lDDKru7ApNON6FmAVySIfnlgMAEM/fj9FjPcGVbwQElXgcz43b+rTDRdrlIgKLBy+Bqlqf0o78CIm8WxrDDF08hjub8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=uE8fFpGnhMJcK5GXfCzqeiiea156Y1/8xBUIdOSHKgI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VV/tR9p0d1Bxz+WPEsp+Mj6qNKd1SyHi/I06GUSSf8xQulczhIsevLi+FapxvTlHOQS23u57gw+fyeIOsfe9hZvy+AdJfCQGIm9clCRJswZKqRPxm9JmOwJCVW2fy6Bnz2t4kAUNteeJyo3fniz4WTyNtGERWQIkHaeI08YEnN4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BtlkHiPL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BtlkHiPL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 24C2FC19422; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=uE8fFpGnhMJcK5GXfCzqeiiea156Y1/8xBUIdOSHKgI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BtlkHiPLBRNfXYO9ys83tffbtgIgbT6sHMOnthh2f9Mpm3nG/fSOkKLxnm+5bs3fR l63JoB1qK0FXSZQKP9aW/H/YcdjiXdX+ILuST7FEZx+bGmblHEX5HpoSB3YNH+zaT9 AQtNNGsxssIMfql2DSJypEYL+UyYqPJ89fGfpZY3rzM1ZtfUgsjUUrnYnwlAFF+xU9 UPsSy+xSmA1OJJf3TfemoBl86gFY21BqEfZaCbawAznJG2l99M5aGglgrSMFXyqZ2u CN2yefe9E4B2zIG6VoPz3rg2rdSB51PTzXrp8okl42jHIlADDPlMMLzCu3qd9YyOwH mxuBU5dEEc7kA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8S-0gsH; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , linux-kernel@vger.kernel.org Subject: [PATCH 07/24] tools: docs: parse_data_structs.py: add methods to return output Date: Thu, 21 Aug 2025 16:21:13 +0200 Message-ID: <1588eefd164040e5b2d946201705459d36f273de.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" When running it from command line, we want to write an output file, but when used as a class, one may just want the output content returned as a string. Split write_output() on two methods to allow both usecases. Also add an extra method to produce a TOC. Signed-off-by: Mauro Carvalho Chehab --- tools/docs/lib/parse_data_structs.py | 62 ++++++++++++++++++++++++++-- tools/docs/parse-headers.py | 5 ++- 2 files changed, 62 insertions(+), 5 deletions(-) diff --git a/tools/docs/lib/parse_data_structs.py b/tools/docs/lib/parse_da= ta_structs.py index 2b7fa6bd8321..a5aa2e182052 100755 --- a/tools/docs/lib/parse_data_structs.py +++ b/tools/docs/lib/parse_data_structs.py @@ -97,33 +97,39 @@ class ParseDataStructs: "prefix": "\\ ", "suffix": "\\ ", "ref_type": ":ref", + "description": "IOCTL Commands", }, "define": { "prefix": "\\ ", "suffix": "\\ ", "ref_type": ":ref", + "description": "Macros and Definitions", }, # We're calling each definition inside an enum as "symbol" "symbol": { "prefix": "\\ ", "suffix": "\\ ", "ref_type": ":ref", + "description": "Enumeration values", }, "typedef": { "prefix": "\\ ", "suffix": "\\ ", "ref_type": ":c:type", + "description": "Type Definitions", }, - # This is the name of the enum itself + # This is the description of the enum itself "enum": { "prefix": "\\ ", "suffix": "\\ ", "ref_type": ":c:type", + "description": "Enumerations", }, "struct": { "prefix": "\\ ", "suffix": "\\ ", "ref_type": ":c:type", + "description": "Structures", }, } =20 @@ -359,7 +365,7 @@ class ParseDataStructs: =20 print() =20 - def write_output(self, file_in: str, file_out: str): + def gen_output(self): """Write the formatted output to a file.""" =20 # Avoid extra blank lines @@ -387,12 +393,60 @@ class ParseDataStructs: text =3D re.sub(r"\\ ([\n ])", r"\1", text) text =3D re.sub(r" \\ ", " ", text) =20 + return text =20 + def gen_toc(self): + """ + Create a TOC table pointing to each symbol from the header + """ + text =3D [] + + # Add header + text.append(".. contents:: Table of Contents") + text.append(" :depth: 2") + text.append(" :local:") + text.append("") + + # Sort symbol types per description + symbol_descriptions =3D [] + for k, v in self.DEF_SYMBOL_TYPES.items(): + symbol_descriptions.append((v['description'], k)) + + symbol_descriptions.sort() + + # Process each category + for description, c_type in symbol_descriptions: + + refs =3D self.symbols[c_type] + if not refs: # Skip empty categories + continue + + text.append(f"{description}") + text.append("-" * len(description)) + text.append("") + + # Sort symbols alphabetically + for symbol, ref in sorted(refs.items()): + text.append(f"* :{ref}:") + + text.append("") # Add empty line between categories + + return "\n".join(text) + + def write_output(self, file_in: str, file_out: str, toc: bool): title =3D os.path.basename(file_in) =20 + if toc: + text =3D self.gen_toc() + else: + text =3D self.gen_output() + with open(file_out, "w", encoding=3D"utf-8", errors=3D"backslashre= place") as f: f.write(".. -*- coding: utf-8; mode: rst -*-\n\n") f.write(f"{title}\n") - f.write("=3D" * len(title)) - f.write("\n\n.. parsed-literal::\n\n") + f.write("=3D" * len(title) + "\n\n") + + if not toc: + f.write(".. parsed-literal::\n\n") + f.write(text) diff --git a/tools/docs/parse-headers.py b/tools/docs/parse-headers.py index 07d3b47c4834..bfa4e46a53e3 100755 --- a/tools/docs/parse-headers.py +++ b/tools/docs/parse-headers.py @@ -36,6 +36,9 @@ def main(): =20 parser.add_argument("-d", "--debug", action=3D"count", default=3D0, help=3D"Increase debug level. Can be used multiple= times") + parser.add_argument("-t", "--toc", action=3D"store_true", + help=3D"instead of a literal block, outputs a TOC = table at the RST file") + parser.add_argument("file_in", help=3D"Input C file") parser.add_argument("file_out", help=3D"Output RST file") parser.add_argument("file_rules", nargs=3D"?", @@ -50,7 +53,7 @@ def main(): parser.process_exceptions(args.file_rules) =20 parser.debug_print() - parser.write_output(args.file_in, args.file_out) + parser.write_output(args.file_in, args.file_out, args.toc) =20 =20 if __name__ =3D=3D "__main__": --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 899E527B35E; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=qlrojwn1sOB75QGDOA1zlG/ne6LEXxKDmxaK2s5UuVjqrpYBRxKL369zCh9vdIL2c4l+cooUL9QDwHce47MlXh3GQ20Tnnbje2/HDKi7F835UVOYZYC5GnjOw+aRlHg7FY71IkycPOiLHpz/0m6lEbficIR6+TQAdSbdseGZI0w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=RsM+00sJ/1A8GHlMF3o/XgMw54KHkYT1Q+rNZSrXntI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=izW+MPX5QWIZedR/vIZid9GOV2mEOkx5zuQmRC57zUjevOoLG6Gm98VyYoVr9b+kjDUk9h3/8F2evTRxT5YpvGL2Rg00QmmkF2XN0jlz2A2gQbIPhGNWgK+lshsCBXopyX5ZiiRrW/GLedv/iJBCDS6hcxL7NEKygVpbe5vY1Fc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CTKeLVys; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CTKeLVys" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 19066C116D0; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=RsM+00sJ/1A8GHlMF3o/XgMw54KHkYT1Q+rNZSrXntI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CTKeLVys+hrJbhlfnDpW0iUYE59ReSib+zuyv2bmcUgIpK9WCpiyzLRRWTwUgntev jdeD6gj7Lcp6DNZsft3iwl0pEJ4C0SxSyxa/6jFAM7fzKy4t1nDvWUXR2+o/CFFZpt lCpDJ38kXUi1cgZVpBwS+KadUG0hGwirF8S/kXqYb4iePNKtjlmHwZkuPjn2N8Vr8y gcR8tXoQz0G6xfFv20oho+lf0imqytWMN30AgZxNCsT4Mrf5qx3GCG64fFsXgWdOnU 7EFqYBYvyA6llStOvxrVhKV0qZOb+UoyS1J26GZgRT/kS8A0Qaf9jGFM2FmzOLZJUe tlaEeVOh/TddQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8W-0nXw; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , linux-kernel@vger.kernel.org Subject: [PATCH 08/24] MAINTAINERS: add files from tools/docs to documentation entry Date: Thu, 21 Aug 2025 16:21:14 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" As we now have a tools directory for docs, add it to its corresponding entry. Signed-off-by: Mauro Carvalho Chehab --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index dafc11712544..ef87548b8f88 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -7308,6 +7308,7 @@ F: scripts/get_abi.py F: scripts/kernel-doc* F: scripts/lib/abi/* F: scripts/lib/kdoc/* +F: tools/docs/* F: tools/net/ynl/pyynl/lib/doc_generator.py F: scripts/sphinx-pre-install X: Documentation/ABI/ --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6AB38279DB6; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=YpWPVzZllUlvxqabbZrPp6W4GjB6LuZupkXVhb/w3JlwcTbsGdKSUGimevHLokUOw+xc5zJTV1MXlssBK/lAq2zIBCtIRNE7ofCfy2vC7p6FZul81L3wVAPuOHbS4Zn6jr0YIIe1H84yRmlrc9ax0wxKowkbDi7K1GczLP2L/Fo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=6KNyilChtfEfKAJL2KTz7S3+EmJ+UL4s3OTBvprovpQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=k+PunpL4vdGsXE/aGQdU2km4QF4xJ7EyaGijHfV74GJF16pUKn945Hxt/sroxbtP4BVt5Xo72ZxIZyy5x+/d357XetwaAniwjETa5nd/OG2xpfp8vMuKcJm3o89w9U12z57fJ+Y+X2UG9E75pEKcJXnR3TOOLpALIw5R6rVUkKk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MGd/ulva; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MGd/ulva" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0FE08C4CEF4; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=6KNyilChtfEfKAJL2KTz7S3+EmJ+UL4s3OTBvprovpQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MGd/ulvamzFNYwH8HntAI+wJCTDGmKqFD+/O6YmHkSRqjxWmSOl1kFLxtf1xUhCiB j6egBehk81HRFuGoVZ79wloy7jC7cLXZZN6jZ2wmICNfpy6wCJpnyzsqv4AM2f2Q3k 7ztjg83kBZG8lZ8L07tuB1ewqXKPrwBbHgK0qSSkqtvSt+yEeFGm4lBOwhtDZm7CHH Cq9Dh0nohIYuD72buLaK1YwjK/tdjPF1M1vHQwKSbIr5mlrdn8YfVxqto4slbXfBJN 1D98HLM2zqrwUjZczkMH4HLrr0iEO1FE0nF/ZRrJvoprbx4NG0jntmR/LoER/veBFu v/ZrC2xoz1wlA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8a-0uCv; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Mauro Carvalho Chehab , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org Subject: [PATCH 09/24] docs: uapi: media: Makefile: use parse-headers.py Date: Thu, 21 Aug 2025 16:21:15 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Now that we have a new parser, use it. Signed-off-by: Mauro Carvalho Chehab --- Documentation/userspace-api/media/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/Makefile b/Documentation/use= rspace-api/media/Makefile index 3d8aaf5c253b..accc734d045a 100644 --- a/Documentation/userspace-api/media/Makefile +++ b/Documentation/userspace-api/media/Makefile @@ -3,7 +3,7 @@ # Rules to convert a .h file to inline RST documentation =20 SRC_DIR=3D$(srctree)/Documentation/userspace-api/media -PARSER =3D $(srctree)/Documentation/sphinx/parse-headers.pl +PARSER =3D $(srctree)/tools/docs/parse-headers.py UAPI =3D $(srctree)/include/uapi/linux KAPI =3D $(srctree)/include/linux =20 --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7A88285C9D; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; cv=none; b=hAkZc5/vVtyCw6TZH6bLTEe5bRszbE2MRJ9cnwTTBjJKVnHSIpvFTIPIUJsbV8sZARvE+ojgtTMMnU1fgN6U1G1k0AN1MqHAzXPW+EbK90rTYTe7mY3vWBK6DWUxoeIwIqZoRQW6LdhgQVrC9bDSoSKaDvGY/G7EoYqmZN9SVQw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; c=relaxed/simple; bh=QHi7tmIl6nunuTAnM3ij6+YulhA5kpLqxpviwyDbWAA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PRLqwhDZyPapofQQQ+cqwcq+50rL1M2MMOLSJp+ZbB5UfKZe8HozIijeoUIMX3K+xTP2I/33WTe1LMpaUWkwmY+/8u4aSomt/cFmt8TirRvkNZ6iGmEst0Y+yeFluXN7zK+ABF+nFRW/R4R+ogntSYD3x3CW1gI3blO+j5NuA2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ln2v4AlA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ln2v4AlA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2644BC2BC86; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=QHi7tmIl6nunuTAnM3ij6+YulhA5kpLqxpviwyDbWAA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ln2v4AlA8O21gO/gkX0Ap2mbzGys9N9GousLxteJMiuK44zLRQmeIlvWvKDvky1dU quCDx3bGJKNp4cF/yLdrDGvonoPnrRyK5mWQxT2etx+7ItprxaPzmPCyxsRJts1hve vnbQN8oJG3xC+2JA0Nie4SoGo9Oymz1uCPsCGpiX09qugkMOxtGdQBR0Y7YgLtTnw3 IQ1F9GtJu+YX8Q4vGpCnBvEtBm1pFQU9RkBzV9zDrTs+EMM46/pK4oOuzJe+FYZWxl 3vOT0ELBVVN4R5o5mfL76UnOSef6xEJXEchfRqiJ7gZzjPcZJxvzym3qa+1KegQ0vS k9pyiDo4K8USQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8e-115S; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 10/24] docs: kernel_include.py: Update its coding style Date: Thu, 21 Aug 2025 16:21:16 +0200 Message-ID: <6ff25e39e2f0548ef8906c7c4ecc5edbcd9f1b70.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" With the help of tools like black, pylint, autopep8 and flake, improve the code style in preparation for further changes. No functional changes. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 100 ++++++++++++------------- 1 file changed, 47 insertions(+), 53 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 1e566e87ebcd..1212786ac516 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -1,7 +1,6 @@ #!/usr/bin/env python3 -# -*- coding: utf-8; mode: python -*- # SPDX-License-Identifier: GPL-2.0 -# pylint: disable=3DR0903, C0330, R0914, R0912, E0401 +# pylint: disable=3DR0903, R0912, R0914, R0915, C0209,W0707 =20 """ kernel-include @@ -40,41 +39,38 @@ from docutils.parsers.rst import directives from docutils.parsers.rst.directives.body import CodeBlock, NumberLines from docutils.parsers.rst.directives.misc import Include =20 -__version__ =3D '1.0' +__version__ =3D "1.0" + =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D def setup(app): -# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D - + """Setup Sphinx exension""" app.add_directive("kernel-include", KernelInclude) - return dict( - version =3D __version__, - parallel_read_safe =3D True, - parallel_write_safe =3D True - ) + return { + "version": __version__, + "parallel_read_safe": True, + "parallel_write_safe": True, + } + =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D class KernelInclude(Include): -# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D - """KernelInclude (``kernel-include``) directive""" =20 def run(self): env =3D self.state.document.settings.env - path =3D os.path.realpath( - os.path.expandvars(self.arguments[0])) + path =3D os.path.realpath(os.path.expandvars(self.arguments[0])) =20 # to get a bit security back, prohibit /etc: if path.startswith(os.sep + "etc"): - raise self.severe( - 'Problems with "%s" directive, prohibited path: %s' - % (self.name, path)) + raise self.severe('Problems with "%s" directive, prohibited pa= th: %s' % + (self.name, path)) =20 self.arguments[0] =3D path =20 env.note_dependency(os.path.abspath(path)) =20 - #return super(KernelInclude, self).run() # won't work, see HINTs i= n _run() + # return super(KernelInclude, self).run() # won't work, see HINTs = in _run() return self._run() =20 def _run(self): @@ -87,41 +83,39 @@ class KernelInclude(Include): =20 if not self.state.document.settings.file_insertion_enabled: raise self.warning('"%s" directive disabled.' % self.name) - source =3D self.state_machine.input_lines.source( - self.lineno - self.state_machine.input_offset - 1) + source =3D self.state_machine.input_lines.source(self.lineno - + self.state_machine.= input_offset - 1) source_dir =3D os.path.dirname(os.path.abspath(source)) path =3D directives.path(self.arguments[0]) - if path.startswith('<') and path.endswith('>'): + if path.startswith("<") and path.endswith(">"): path =3D os.path.join(self.standard_include_path, path[1:-1]) path =3D os.path.normpath(os.path.join(source_dir, path)) =20 # HINT: this is the only line I had to change / commented out: - #path =3D utils.relative_path(None, path) + # path =3D utils.relative_path(None, path) =20 - encoding =3D self.options.get( - 'encoding', self.state.document.settings.input_encoding) - e_handler=3Dself.state.document.settings.input_encoding_error_hand= ler - tab_width =3D self.options.get( - 'tab-width', self.state.document.settings.tab_width) + encoding =3D self.options.get("encoding", + self.state.document.settings.input_enc= oding) + e_handler =3D self.state.document.settings.input_encoding_error_ha= ndler + tab_width =3D self.options.get("tab-width", + self.state.document.settings.tab_widt= h) try: self.state.document.settings.record_dependencies.add(path) - include_file =3D io.FileInput(source_path=3Dpath, - encoding=3Dencoding, + include_file =3D io.FileInput(source_path=3Dpath, encoding=3De= ncoding, error_handler=3De_handler) - except UnicodeEncodeError as error: + except UnicodeEncodeError: raise self.severe('Problems with "%s" directive path:\n' 'Cannot encode input file path "%s" ' - '(wrong locale?).' % - (self.name, SafeString(path))) + "(wrong locale?)." % (self.name, SafeString(= path))) except IOError as error: - raise self.severe('Problems with "%s" directive path:\n%s.' % - (self.name, ErrorString(error))) - startline =3D self.options.get('start-line', None) - endline =3D self.options.get('end-line', None) + raise self.severe('Problems with "%s" directive path:\n%s.' + % (self.name, ErrorString(error))) + startline =3D self.options.get("start-line", None) + endline =3D self.options.get("end-line", None) try: if startline or (endline is not None): lines =3D include_file.readlines() - rawtext =3D ''.join(lines[startline:endline]) + rawtext =3D "".join(lines[startline:endline]) else: rawtext =3D include_file.read() except UnicodeError as error: @@ -129,43 +123,43 @@ class KernelInclude(Include): (self.name, ErrorString(error))) # start-after/end-before: no restrictions on newlines in match-tex= t, # and no restrictions on matching inside lines vs. line boundaries - after_text =3D self.options.get('start-after', None) + after_text =3D self.options.get("start-after", None) if after_text: # skip content in rawtext before *and incl.* a matching text after_index =3D rawtext.find(after_text) if after_index < 0: raise self.severe('Problem with "start-after" option of "%= s" ' - 'directive:\nText not found.' % self.nam= e) - rawtext =3D rawtext[after_index + len(after_text):] - before_text =3D self.options.get('end-before', None) + "directive:\nText not found." % self.nam= e) + rawtext =3D rawtext[after_index + len(after_text) :] + before_text =3D self.options.get("end-before", None) if before_text: # skip content in rawtext after *and incl.* a matching text before_index =3D rawtext.find(before_text) if before_index < 0: raise self.severe('Problem with "end-before" option of "%s= " ' - 'directive:\nText not found.' % self.nam= e) + "directive:\nText not found." % self.nam= e) rawtext =3D rawtext[:before_index] =20 include_lines =3D statemachine.string2lines(rawtext, tab_width, convert_whitespace=3DTru= e) - if 'literal' in self.options: + if "literal" in self.options: # Convert tabs to spaces, if `tab_width` is positive. if tab_width >=3D 0: text =3D rawtext.expandtabs(tab_width) else: text =3D rawtext literal_block =3D nodes.literal_block(rawtext, source=3Dpath, - classes=3Dself.options.get('class', []= )) + classes=3Dself.options.get= ("class", []) + ) literal_block.line =3D 1 self.add_name(literal_block) - if 'number-lines' in self.options: + if "number-lines" in self.options: try: - startline =3D int(self.options['number-lines'] or 1) + startline =3D int(self.options["number-lines"] or 1) except ValueError: - raise self.error(':number-lines: with non-integer ' - 'start value') + raise self.error(":number-lines: with non-integer star= t value") endline =3D startline + len(include_lines) - if text.endswith('\n'): + if text.endswith("\n"): text =3D text[:-1] tokens =3D NumberLines([([], text)], startline, endline) for classes, value in tokens: @@ -177,12 +171,12 @@ class KernelInclude(Include): else: literal_block +=3D nodes.Text(text, text) return [literal_block] - if 'code' in self.options: - self.options['source'] =3D path + if "code" in self.options: + self.options["source"] =3D path codeblock =3D CodeBlock(self.name, - [self.options.pop('code')], # arguments + [self.options.pop("code")], # arguments self.options, - include_lines, # content + include_lines, # content self.lineno, self.content_offset, self.block_text, --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C46571F91C8; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=ewwS6oS1NR/GqRCH1jOJsslQhoGyd/8t9YbmEK/0D0QnL4sSOc9B7WiSSxYCY2TO3Sp59se7PymB5xhiqhgEtcN9Btqdl38/bRbWl7nV5pPPu95J+BeE5ypRrBjGGpyAqvAkksis3GoMz+efapHos0cf4LWZEt/ddzKBX4AEyeE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=lRH9SEB4iDb1LfHInxrSMa8aPsx5ggsWJbqLQRJAoqQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Dnqodbs2GFq6A53YbrKT6yirHt5sruFikzY+gojlT0setTR/PkVmoDYChOXbMtpyuIghAsRtfuxFBwVccoksPiYyvQKUfqq3V3sUkByH7Nr+gzX34jKbdSnUlgsPYn2al3wApBVQTAK3Q+Mg3SxfOaVxeLWU9+heYDG+leNkRLk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LdUJS5bj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LdUJS5bj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1DF21C16AAE; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=lRH9SEB4iDb1LfHInxrSMa8aPsx5ggsWJbqLQRJAoqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LdUJS5bjPd3m1j90lbHlIC5VKp8nJ32DTzMXAaTtF6EKyzXABJdliCu/gSadIPbKS cOW7oLiG5nziwKncu79UYUY1Ov/r8qc/fMEcFSpOW11xPJJ/LgicTMf4I+LmYlXdh/ 3KekxDWiDk8Iowx6fHA/zO9JHnetiZC2dwD241mDXM4NHPIl8a8UXqIAiCNjDSsgTi KaD0+wO3ygjfOhClkzwMhvMKkW8Yqt2WDQqclJA4qIIib/vpa+sZUq7Ud6NP+FZZ7D VXysuvN6OnQHw0Hb49Ny0wQNYe0u5ZcTIy+8lUXhsQHA6oaUbG9K4YX7zNW3NFfwY0 6FBUfnd77Q6bA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8i-17v9; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 11/24] docs: kernel_include.py: allow cross-reference generation Date: Thu, 21 Aug 2025 16:21:17 +0200 Message-ID: <1e30edc994a3edba976aafbf182eef2ed5439ede.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" kernel_include extension was originally designed to be used by the media comprehensive uAPI documentation, where, instead of simpler kernel-doc markups, the uAPI documentation is enriched with a larger text, with images, complex tables, graphs, etc. There, we wanted to include the much simpler yet documented .h file. This extension is needed to include files from other parts of the Kernel tree outside Documentation, because the original Sphinx include tag doesn't allow going outside of the directory passed via sphinx-build command line. Yet, the cross-references themselves to the full documentation were using a perl script to create cross-references against the comprehensive documentation. As the perl script is now converted to Phython and there is a Python class producing an include-compatible output with cross references, add two optional arguments to kernel_include.py: 1. :generate-cross-refs: If present, instead of reading the file, it calls ParseDataStructs() class, which converts C data structures into cross-references to be linked to ReST files containing a more comprehensive documentati= on; Don't use it together with :start-line: and/or :end-line:, as filtering input file line range is currently not supported. 2. :exception-file: Used together with :generate-cross-refs:. Points to a file containi= ng rules to ignore C data structs or to use a different reference name, optionally using a different reference type. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 94 ++++++++++++++++++++------ 1 file changed, 74 insertions(+), 20 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 1212786ac516..fc37e6fa9d96 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -25,6 +25,24 @@ Substrings of the form $name or ${name} are replaced by the value of environment variable name. Malformed variable names and references to non-existing variables are left unchanged. + + This extension overrides Sphinx include directory, adding two extra + arguments: + + 1. :generate-cross-refs: + + If present, instead of reading the file, it calls ParseDataStructs= () + class, which converts C data structures into cross-references to + be linked to ReST files containing a more comprehensive documentat= ion; + + Don't use it together with :start-line: and/or :end-line:, as + filtering input file line range is currently not supported. + + 2. :exception-file: + + Used together with :generate-cross-refs:. Points to a file contain= ing + rules to ignore C data structs or to use a different reference nam= e, + optionally using a different reference type. """ =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D @@ -32,6 +50,7 @@ # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D =20 import os.path +import sys =20 from docutils import io, nodes, statemachine from docutils.utils.error_reporting import SafeString, ErrorString @@ -39,6 +58,11 @@ from docutils.parsers.rst import directives from docutils.parsers.rst.directives.body import CodeBlock, NumberLines from docutils.parsers.rst.directives.misc import Include =20 +srctree =3D os.path.abspath(os.environ["srctree"]) +sys.path.insert(0, os.path.join(srctree, "tools/docs/lib")) + +from parse_data_structs import ParseDataStructs + __version__ =3D "1.0" =20 =20 @@ -57,6 +81,14 @@ def setup(app): class KernelInclude(Include): """KernelInclude (``kernel-include``) directive""" =20 + # Add extra options + option_spec =3D Include.option_spec.copy() + + option_spec.update({ + 'generate-cross-refs': directives.flag, + 'exception-file': directives.unchanged, + }) + def run(self): env =3D self.state.document.settings.env path =3D os.path.realpath(os.path.expandvars(self.arguments[0])) @@ -99,28 +131,49 @@ class KernelInclude(Include): e_handler =3D self.state.document.settings.input_encoding_error_ha= ndler tab_width =3D self.options.get("tab-width", self.state.document.settings.tab_widt= h) - try: - self.state.document.settings.record_dependencies.add(path) - include_file =3D io.FileInput(source_path=3Dpath, encoding=3De= ncoding, - error_handler=3De_handler) - except UnicodeEncodeError: - raise self.severe('Problems with "%s" directive path:\n' - 'Cannot encode input file path "%s" ' - "(wrong locale?)." % (self.name, SafeString(= path))) - except IOError as error: - raise self.severe('Problems with "%s" directive path:\n%s.' - % (self.name, ErrorString(error))) startline =3D self.options.get("start-line", None) endline =3D self.options.get("end-line", None) - try: - if startline or (endline is not None): - lines =3D include_file.readlines() - rawtext =3D "".join(lines[startline:endline]) - else: - rawtext =3D include_file.read() - except UnicodeError as error: - raise self.severe('Problem with "%s" directive:\n%s' % - (self.name, ErrorString(error))) + + # Get optional arguments to related to cross-references generation + if 'generate-cross-refs' in self.options: + parser =3D ParseDataStructs() + parser.parse_file(path) + + exceptions_file =3D self.options.get('exception-file') + if exceptions_file: + exceptions_file =3D os.path.join(source_dir, exceptions_fi= le) + parser.process_exceptions(exceptions_file) + + title =3D os.path.basename(path) + rawtext =3D parser.gen_output() + if startline or endline: + raise self.severe('generate-cross-refs can\'t be used toge= ther with "start-line" or "end-line"') + + if "code" not in self.options: + rawtext =3D ".. parsed-literal::\n\n" + rawtext + else: + try: + self.state.document.settings.record_dependencies.add(path) + include_file =3D io.FileInput(source_path=3Dpath, encoding= =3Dencoding, + error_handler=3De_handler) + except UnicodeEncodeError: + raise self.severe('Problems with "%s" directive path:\n' + 'Cannot encode input file path "%s" ' + "(wrong locale?)." % (self.name, SafeStrin= g(path))) + except IOError as error: + raise self.severe('Problems with "%s" directive path:\n%s.' + % (self.name, ErrorString(error))) + + try: + if startline or (endline is not None): + lines =3D include_file.readlines() + rawtext =3D "".join(lines[startline:endline]) + else: + rawtext =3D include_file.read() + except UnicodeError as error: + raise self.severe('Problem with "%s" directive:\n%s' % + (self.name, ErrorString(error))) + # start-after/end-before: no restrictions on newlines in match-tex= t, # and no restrictions on matching inside lines vs. line boundaries after_text =3D self.options.get("start-after", None) @@ -171,6 +224,7 @@ class KernelInclude(Include): else: literal_block +=3D nodes.Text(text, text) return [literal_block] + if "code" in self.options: self.options["source"] =3D path codeblock =3D CodeBlock(self.name, --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1E5E284685; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=PZXBChTjm5Yvpqs3mZIgUGub+qBeTcpvViqrin+OZ7/2F4BMUwTNUA9iNXVOTwYj+q4JCdO+k/JLRHB7C+t7Pm5YEbg85UIJ+8DcDw4sEH94jjN4Cw48LJz74AAoZL5fQtfke5S2J+7fgU8MmkGXcnB7YcjlrgStWgZZytQvt5c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=UW2xelo4TnzX16OxyxtwPo8LfwSKWCjViRI3KVdm6R4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Dbc9uaCeI5uuhQ5wrI2s1OwloYTgrm/YX4zkx4+3GjGEvgNCpyOOJEMCw112s7a0uUC60CDAPhh8Mw5HQU1l18agC4hgHX/N2+85D/TCvUdT203LeJruLXe897KQ6zSU6TDKLLe1RM2UKV8mbWgKDSyEXV64S0hRG3FO6L//8MY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bvQPhZY4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bvQPhZY4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2DE97C2BC87; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=UW2xelo4TnzX16OxyxtwPo8LfwSKWCjViRI3KVdm6R4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bvQPhZY4nHQ1YyOL93Na7XeAh5dPqn4BvSGFmtRoWUs5DOdm3rw4znxFweyIMQ/QC PD/87OSFXRK5PU9rxq8qqAzzIB62xpQCKMM8agLYjH4lcIP8zOQXBE9qPGjfLP4AbJ tlyY1wrzJxpN8nVC3SdiFt74bpps52wj1XSK5u3KzTAx8eDNt0CrYrqseUZkTN/Ue5 tGfuHe0/o7WLPwP5qoE241NKU2NvVDoprVega4K4De37xJBY1u5fjMnwUEeCrtInpZ sQ+Jn7SqsvaKtVc/0glPACJfMmvWRoh15IHPoAkajRfftNi5g+8kejMMS63+xjyYYo +IkDXS+ltVIJg== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8m-1EiT; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 12/24] docs: kernel_include.py: generate warnings for broken refs Date: Thu, 21 Aug 2025 16:21:18 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab In the past, Sphinx used to warn about broken references. That's basically the rationale for adding media uAPI files: to get warnings about missed symbols. This is not true anymore. So, we need to explicitly check them after doctree-resolved event. While here, move setup() to the end, to make it closer to what we do on other extensions. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 108 ++++++++++++++++++++----- 1 file changed, 89 insertions(+), 19 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index fc37e6fa9d96..0a3e5377dd1e 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -26,7 +26,7 @@ environment variable name. Malformed variable names and references to non-existing variables are left unchanged. =20 - This extension overrides Sphinx include directory, adding two extra + This extension overrides Sphinx include directory, adding some extra arguments: =20 1. :generate-cross-refs: @@ -35,14 +35,20 @@ class, which converts C data structures into cross-references to be linked to ReST files containing a more comprehensive documentat= ion; =20 - Don't use it together with :start-line: and/or :end-line:, as - filtering input file line range is currently not supported. - 2. :exception-file: =20 - Used together with :generate-cross-refs:. Points to a file contain= ing - rules to ignore C data structs or to use a different reference nam= e, - optionally using a different reference type. + Used together with :generate-cross-refs + + Points to a file containing rules to ignore C data structs or to + use a different reference name, optionally using a different + reference type. + + 3. :warn-broken: + + Used together with :generate-cross-refs: + + Detect if the auto-generated cross references doesn't exist. + """ =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D @@ -50,6 +56,7 @@ # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D =20 import os.path +import re import sys =20 from docutils import io, nodes, statemachine @@ -58,23 +65,18 @@ from docutils.parsers.rst import directives from docutils.parsers.rst.directives.body import CodeBlock, NumberLines from docutils.parsers.rst.directives.misc import Include =20 +from sphinx.util import logging + srctree =3D os.path.abspath(os.environ["srctree"]) sys.path.insert(0, os.path.join(srctree, "tools/docs/lib")) =20 from parse_data_structs import ParseDataStructs =20 __version__ =3D "1.0" +logger =3D logging.getLogger(__name__) =20 - -# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D -def setup(app): - """Setup Sphinx exension""" - app.add_directive("kernel-include", KernelInclude) - return { - "version": __version__, - "parallel_read_safe": True, - "parallel_write_safe": True, - } +RE_DOMAIN_REF =3D re.compile(r'\\ :(ref|c:type|c:func):`([^<`]+)(?:<([^>]+= )>)?`\\') +RE_SIMPLE_REF =3D re.compile(r'`([^`]+)`') =20 =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D @@ -86,6 +88,7 @@ class KernelInclude(Include): =20 option_spec.update({ 'generate-cross-refs': directives.flag, + 'warn-broken': directives.flag, 'exception-file': directives.unchanged, }) =20 @@ -103,9 +106,9 @@ class KernelInclude(Include): env.note_dependency(os.path.abspath(path)) =20 # return super(KernelInclude, self).run() # won't work, see HINTs = in _run() - return self._run() + return self._run(env) =20 - def _run(self): + def _run(self, env): """Include a file as part of the content of this reST file.""" =20 # HINT: I had to copy&paste the whole Include.run method. I'am not= happy @@ -151,6 +154,10 @@ class KernelInclude(Include): =20 if "code" not in self.options: rawtext =3D ".. parsed-literal::\n\n" + rawtext + + # Store references on a symbol dict to be used at check time + if 'warn-broken' in self.options: + env._xref_files.add(path) else: try: self.state.document.settings.record_dependencies.add(path) @@ -239,3 +246,66 @@ class KernelInclude(Include): return codeblock.run() self.state_machine.insert_input(include_lines, path) return [] + +# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D + +reported =3D set() + +def check_missing_refs(app, env, node, contnode): + """Check broken refs for the files it creates xrefs""" + if not node.source: + return None + + try: + xref_files =3D env._xref_files + except AttributeError: + logger.critical("FATAL: _xref_files not initialized!") + raise + + # Only show missing references for kernel-include reference-parsed fil= es + if node.source not in xref_files: + return None + + target =3D node.get('reftarget', '') + domain =3D node.get('refdomain', 'std') + reftype =3D node.get('reftype', '') + + msg =3D f"can't link to: {domain}:{reftype}:: {target}" + + # Don't duplicate warnings + data =3D (node.source, msg) + if data in reported: + return None + reported.add(data) + + logger.warning(msg, location=3Dnode, type=3D'ref', subtype=3D'missing') + + return None + +def merge_xref_info(app, env, docnames, other): + """ + As each process modify env._xref_files, we need to merge them back. + """ + if not hasattr(other, "_xref_files"): + return + env._xref_files.update(getattr(other, "_xref_files", set())) + +def init_xref_docs(app, env, docnames): + """Initialize a list of files that we're generating cross references= =C2=A8""" + app.env._xref_files =3D set() + +# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D + +def setup(app): + """Setup Sphinx exension""" + + app.connect("env-before-read-docs", init_xref_docs) + app.connect("env-merge-info", merge_xref_info) + app.add_directive("kernel-include", KernelInclude) + app.connect("missing-reference", check_missing_refs) + + return { + "version": __version__, + "parallel_read_safe": True, + "parallel_write_safe": True, + } --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEEF728467D; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=SpXqYTKC+zZ2f0yRtftW7rqqXkcZ7HjUcj/tVpjm91HTfwQxJ9T5DChx2fUdX4yYPC1k1YnWfvFL9pKuXazq4TOAnMAzeKcCvd5QFxza5c3fpAK32t2At8byea8D+8auW+7JvVW3Drg4cxrwvO91gG/7y/MsYWn1wF7pRfCfE0U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=DL+ejH7HzWPaC/HHDV1RfIxKyreXW8+q4TsBZyqpgb0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DrNENVUIgUUSSWpF5xk14CTrxyjxBgmtM58PA+qxfdmb/qcPdzX7/WhNDLlZ8TeZb9Ip86/XjNVWZ+P7elbm7q6Lze9lpBWxJnOlod85JG7MCkVM5ggzeVa4hMy4ghTZREWfGU38+L5Zt/tNCnlI5JGZ6+DsUI1XK3lV4scRzbQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Xpdtc4Bf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Xpdtc4Bf" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A7E3C2BC9E; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=DL+ejH7HzWPaC/HHDV1RfIxKyreXW8+q4TsBZyqpgb0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Xpdtc4BfErkzjj+NKJxwbMyjP6bV2A1wbvwX1jSuCEit5IQ07vXMw5qQKRzQ5r8VN nYaSB/7L2jfjjGeoj8cWGM9e+xA971ho2oFg3zo0BYT2FLABE4wCVwxjsjx6in7gfb KwZLb/6YqG1eG3ibCkh2RTjP1YabhFKWRcalCkf2Yc8ihmCAlKsXH+XfQnLYAZngDp ohW/EjThMSmlOgj50dPmu0ue4ZIGD6FZbGHchFF4OdYPF40JScA9v2H43XFIC640lb wJu2CBfp+F7SpJOR7GpYQ4m8w6kvzV4/GbNh8lsh6zfBmXZRxLYza+2g5Sk595iXMQ 2sivLNi52bPeQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8q-1LWT; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 13/24] docs: kernel_include.py: move rawtext logic to separate functions Date: Thu, 21 Aug 2025 16:21:19 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The run function is too complex. merge run() and _run() into a single function and move the read logic to separate functions. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 82 ++++++++++++++------------ 1 file changed, 43 insertions(+), 39 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 0a3e5377dd1e..ef86ee9e79d6 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -92,7 +92,47 @@ class KernelInclude(Include): 'exception-file': directives.unchanged, }) =20 + def read_rawtext(self, path, encoding): + """Read and process file content with error handling""" + try: + self.state.document.settings.record_dependencies.add(path) + include_file =3D io.FileInput(source_path=3Dpath, + encoding=3Dencoding, + error_handler=3Dself.state.doc= ument.settings.input_encoding_error_handler) + except UnicodeEncodeError: + raise self.severe('Problems with directive path:\n' + 'Cannot encode input file path "%s" ' + '(wrong locale?).' % SafeString(path)) + except IOError as error: + raise self.severe('Problems with directive path:\n%s.' % E= rrorString(error)) + + try: + return include_file.read() + except UnicodeError as error: + raise self.severe('Problem with directive:\n%s' % ErrorStr= ing(error)) + + def read_rawtext_with_xrefs(self, env, path): + parser =3D ParseDataStructs() + parser.parse_file(path) + + if 'exception-file' in self.options: + source_dir =3D os.path.dirname(os.path.abspath( + self.state_machine.input_lines.source( + self.lineno - self.state_machine.input_offset - 1))) + exceptions_file =3D os.path.join(source_dir, self.options['exc= eption-file']) + parser.process_exceptions(exceptions_file) + + if self.options.get("start-line") or self.options.get("end-line"): + raise self.severe('generate-cross-refs can\'t be used with "st= art-line" or "end-line"') + + # Store references on a symbol dict to be used at check time + if 'warn-broken' in self.options: + env._xref_files.add(path) + + return parser.gen_output() + def run(self): + """Include a file as part of the content of this reST file.""" env =3D self.state.document.settings.env path =3D os.path.realpath(os.path.expandvars(self.arguments[0])) =20 @@ -105,12 +145,6 @@ class KernelInclude(Include): =20 env.note_dependency(os.path.abspath(path)) =20 - # return super(KernelInclude, self).run() # won't work, see HINTs = in _run() - return self._run(env) - - def _run(self, env): - """Include a file as part of the content of this reST file.""" - # HINT: I had to copy&paste the whole Include.run method. I'am not= happy # with this, but due to security reasons, the Include.run method d= oes # not allow absolute or relative pathnames pointing to locations *= above* @@ -139,47 +173,17 @@ class KernelInclude(Include): =20 # Get optional arguments to related to cross-references generation if 'generate-cross-refs' in self.options: - parser =3D ParseDataStructs() - parser.parse_file(path) - - exceptions_file =3D self.options.get('exception-file') - if exceptions_file: - exceptions_file =3D os.path.join(source_dir, exceptions_fi= le) - parser.process_exceptions(exceptions_file) + rawtext =3D self.read_rawtext_with_xrefs(env, path) =20 title =3D os.path.basename(path) - rawtext =3D parser.gen_output() + if startline or endline: raise self.severe('generate-cross-refs can\'t be used toge= ther with "start-line" or "end-line"') =20 if "code" not in self.options: rawtext =3D ".. parsed-literal::\n\n" + rawtext - - # Store references on a symbol dict to be used at check time - if 'warn-broken' in self.options: - env._xref_files.add(path) else: - try: - self.state.document.settings.record_dependencies.add(path) - include_file =3D io.FileInput(source_path=3Dpath, encoding= =3Dencoding, - error_handler=3De_handler) - except UnicodeEncodeError: - raise self.severe('Problems with "%s" directive path:\n' - 'Cannot encode input file path "%s" ' - "(wrong locale?)." % (self.name, SafeStrin= g(path))) - except IOError as error: - raise self.severe('Problems with "%s" directive path:\n%s.' - % (self.name, ErrorString(error))) - - try: - if startline or (endline is not None): - lines =3D include_file.readlines() - rawtext =3D "".join(lines[startline:endline]) - else: - rawtext =3D include_file.read() - except UnicodeError as error: - raise self.severe('Problem with "%s" directive:\n%s' % - (self.name, ErrorString(error))) + rawtext =3D self.read_rawtext(path, encoding) =20 # start-after/end-before: no restrictions on newlines in match-tex= t, # and no restrictions on matching inside lines vs. line boundaries --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD795283FFF; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=Y1Q9u0G2AD0eYzUsSLLlYxsazmTXoY5k8Bam40EQtvQDFlU4ZB5OHxeOjQE6VbU459zqpIkH4BGoE7kbpfQgS7uOEzLzVlEJDbXtiAtaEWQuh7mpyCm3GsvOwAypd92McE3QlCJuQX7qa4CTk6yKPgmStopT8LwB6ajsti6E96Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=qu5eqKXlag7ALsP16SkP+d9fl76226iDf9IK5Sgc2pE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=e2aEEb9WZWc+jk4DktyLu5jirh5n9r8aWiwRDO/6PG9f15yb5idiTUU2BWcP+b0/6O4wPhWOZuRgaQKWZqLbaaEKH/GqG0aglnhDu674CbdmSjraXKA0eDlfQs1wuAwOQejI64b8klTYeWouOi93JL8ZvypvZP2SnX1wfOwguvA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=iRinbi0d; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="iRinbi0d" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A2FDC19425; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=qu5eqKXlag7ALsP16SkP+d9fl76226iDf9IK5Sgc2pE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iRinbi0d6r5qMfaikiw5DtkpvrdOOyetl0qHefidh+cg+lNQdpwcx+xA6+sfu1QeN KQ0JmDXxCTRcMhXD2jUgJ0K+zO7K891bjKeGpVHmpkdjQfbusB7/W6ovjI10+ASGgD RQejKDkArVxdx8ZIwnwTiBRNDdQ26eOPiiEe4uzalKCianmRw3TJ1sR39EpuzaVb7t n3DVWanaRyhQDddqN2AoEC8TrSJgSGyApATG2krXPQQFAPSnoK7pHlW04sQK5nk3ey 4t0IN/TFjo0lGd8fGifxuBJqSPwYxEOJHruLwK3aKCzxetWHqDWnEJzZtXcAbjquNP 1anIs8lJN6AXQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8u-1SLl; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 14/24] docs: kernel_include.py: move range logic to a separate function Date: Thu, 21 Aug 2025 16:21:20 +0200 Message-ID: <811121f0b40c788936c6aec92a27500767b9b0dd.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Cleanup run() function by moving the range logic to a separate function. Here, I ended checking the current Sphinx implementation, as it has some extra logic for the range check. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 51 +++++++++++++++++--------- 1 file changed, 33 insertions(+), 18 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index ef86ee9e79d6..c5f4f34e22cb 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -131,6 +131,38 @@ class KernelInclude(Include): =20 return parser.gen_output() =20 + def apply_range(self, rawtext): + # Get to-be-included content + startline =3D self.options.get('start-line', None) + endline =3D self.options.get('end-line', None) + try: + if startline or (endline is not None): + lines =3D rawtext.splitlines() + rawtext =3D '\n'.join(lines[startline:endline]) + except UnicodeError as error: + raise self.severe(f'Problem with "{self.name}" directive:\n' + + io.error_string(error)) + # start-after/end-before: no restrictions on newlines in match-tex= t, + # and no restrictions on matching inside lines vs. line boundaries + after_text =3D self.options.get("start-after", None) + if after_text: + # skip content in rawtext before *and incl.* a matching text + after_index =3D rawtext.find(after_text) + if after_index < 0: + raise self.severe('Problem with "start-after" option of "%= s" ' + "directive:\nText not found." % self.nam= e) + rawtext =3D rawtext[after_index + len(after_text) :] + before_text =3D self.options.get("end-before", None) + if before_text: + # skip content in rawtext after *and incl.* a matching text + before_index =3D rawtext.find(before_text) + if before_index < 0: + raise self.severe('Problem with "end-before" option of "%s= " ' + "directive:\nText not found." % self.nam= e) + rawtext =3D rawtext[:before_index] + + return rawtext + def run(self): """Include a file as part of the content of this reST file.""" env =3D self.state.document.settings.env @@ -185,24 +217,7 @@ class KernelInclude(Include): else: rawtext =3D self.read_rawtext(path, encoding) =20 - # start-after/end-before: no restrictions on newlines in match-tex= t, - # and no restrictions on matching inside lines vs. line boundaries - after_text =3D self.options.get("start-after", None) - if after_text: - # skip content in rawtext before *and incl.* a matching text - after_index =3D rawtext.find(after_text) - if after_index < 0: - raise self.severe('Problem with "start-after" option of "%= s" ' - "directive:\nText not found." % self.nam= e) - rawtext =3D rawtext[after_index + len(after_text) :] - before_text =3D self.options.get("end-before", None) - if before_text: - # skip content in rawtext after *and incl.* a matching text - before_index =3D rawtext.find(before_text) - if before_index < 0: - raise self.severe('Problem with "end-before" option of "%s= " ' - "directive:\nText not found." % self.nam= e) - rawtext =3D rawtext[:before_index] + rawtext =3D self.apply_range(rawtext) =20 include_lines =3D statemachine.string2lines(rawtext, tab_width, convert_whitespace=3DTru= e) --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2E832741D1; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=l3AiaCeoSu/73/fG/0rgfgsudu/Ef9Hkj6WS4Sf0bCBLegNFUZJ04A2VOarPt4wo4TYMc/3E5PrfPJPmhMXF3eGWZnxSozhyZqvxQn1CjFEukkBYcOtSHyL4clGZkmR5C4YqnZaBIrbzhQfQ93z5HRy4u32lujLVmYXtbf6YC7I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=wQmzELiyLlP6dYnGbR5xm0Wl2cXgsAK4Vx4o5Slpgnw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gLjMRP0UMEUqH7Qas3vXizUuFhXgfOzWjcNWmZVYlfe+a9aFgPx/zYnM5MTFzhoyoiTwpnn07BsmMV8qMqEiy5I4uChA+Z+1sWRMgZ1r4dk2nLilrKqtNrPmOE0TrBTAlHVLXUECWohJD6uWLfM+JfxnAIyjQmLpofMrbVF5RfM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MouVHQ/S; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MouVHQ/S" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D5DBC4AF0C; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=wQmzELiyLlP6dYnGbR5xm0Wl2cXgsAK4Vx4o5Slpgnw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MouVHQ/SUz3h0bXRfAPJnNzPcYS1KVcbXEpqYXmzjR9pv6IyOCGPq9HcaPZSWCVjK NezXokJ5LOPYcYmU4rR8kq78U/FJ+o50TA8FBODGNmk04ZWLGrMeKOBE9SDhNg+5qB IJ+cV2BfTZSXWILLqPIC0xs+Z0Dpp/kkbDEcID7+wHH7zvE4gzQkS2+SmObfANlx9h 66CL2bZ5FPEcdEYcBo8Dzl7ftnvR+ko7Ypm+z2Q86can4pFMwMJN1L9lvNbv2UiPV8 bminWawp9iEX++gneBk7Ht6cs8als6I1NPK8cocagTSZYXt3ToNMWpz2CulPuqxT4l 0vxxfZM/z+kEg== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8y-1Z67; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 15/24] docs: kernel_include.py: remove range restriction for gen docs Date: Thu, 21 Aug 2025 16:21:21 +0200 Message-ID: <92f8d6245a5891918281bca9cf7575da206aa27a.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Originally, parse-readers were generating an output where the first two lines were setting a literal block. The script now gets only the actual parsed data without that, so it is now safe to allow start-line and end-line parameters to be handled. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 6 ------ 1 file changed, 6 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index c5f4f34e22cb..4cdd1c77982e 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -122,9 +122,6 @@ class KernelInclude(Include): exceptions_file =3D os.path.join(source_dir, self.options['exc= eption-file']) parser.process_exceptions(exceptions_file) =20 - if self.options.get("start-line") or self.options.get("end-line"): - raise self.severe('generate-cross-refs can\'t be used with "st= art-line" or "end-line"') - # Store references on a symbol dict to be used at check time if 'warn-broken' in self.options: env._xref_files.add(path) @@ -209,9 +206,6 @@ class KernelInclude(Include): =20 title =3D os.path.basename(path) =20 - if startline or endline: - raise self.severe('generate-cross-refs can\'t be used toge= ther with "start-line" or "end-line"') - if "code" not in self.options: rawtext =3D ".. parsed-literal::\n\n" + rawtext else: --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D84C02857CF; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=XixoRb/5uheiGz9kbwCW8ycGpcpydByJLDlWDi/2+2X0JXzRA00u8BGvGBEwb29YIzDKK1qaM5JtccZJJN186L97ePSV77haiROzFAS2cwGn750U34SCj2xVTrGX3UtM1ZOiiDBQ+cXh55KF0Y4+YuQzc0d6ryQe1iIr4TG/rjw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=XLpGjVmiz7KWSgkDl0zwCX9wJOnCvElrKCUuUORJrj4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HhhXKY2f4sNaCkeA2usVInqZD94rYkuq4v7KirfdUuF/vsuZD/BX+1jnDp4lbBkQlImekujx6T5vw7xcWGPp2BbW9DfXYUGXlDUsaslRvw3yneAL6ro4aICZiN+CXjVRl1WJ01nM1fvE9SlHBPFR9/+6FaDFSqN15T+foJRopHk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=S0vyeHCE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="S0vyeHCE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39A51C2BCB5; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=XLpGjVmiz7KWSgkDl0zwCX9wJOnCvElrKCUuUORJrj4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S0vyeHCE3qfDsq0D4MUgUVrAQDhsIuCEzI/P548tnM9nZ5Muk91VVYMiRaeedD7Wt vTONdyjILJMfX/ePvRar4MwZo7QGt34OAeIoPCd2OiuiJX8XR38CqmKFkqo4p0AJFu xe9TEBOUOyTC4wzlOy7TWgGRz4VWvem0b0vZDy3NQVRv0ouIJqdPhpfxJZMsfeQwG0 phEHbF5YW44ATkBx4awRTIX9X5NjyexQ2E9szQYUqsurXoPTOYpndaqupjL2EfLWmh BKUKPFEkGBom6ji4hWIoDLbspnhuSiYjN7Q3MNksPH8NBWlAzjNhxqXFaoHax/GD+V /60orX2LtmQCA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT92-1ftd; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 16/24] docs: kernel_include.py: move code and literal functions Date: Thu, 21 Aug 2025 16:21:22 +0200 Message-ID: <32c0cee9ef85f8d4789f50514d77f719118b3217.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Simplify run() even more by moving the code which handles with code and literal blocks to their own functions. No functional changes. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 100 +++++++++++++++---------- 1 file changed, 59 insertions(+), 41 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 4cdd1c77982e..0909eb3a07ea 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -160,6 +160,52 @@ class KernelInclude(Include): =20 return rawtext =20 + def literal(self, path, tab_width, rawtext): + """Output a literal block""" + + # Convert tabs to spaces, if `tab_width` is positive. + if tab_width >=3D 0: + text =3D rawtext.expandtabs(tab_width) + else: + text =3D rawtext + literal_block =3D nodes.literal_block(rawtext, source=3Dpath, + classes=3Dself.options.get("cl= ass", [])) + literal_block.line =3D 1 + self.add_name(literal_block) + if "number-lines" in self.options: + try: + startline =3D int(self.options["number-lines"] or 1) + except ValueError: + raise self.error(":number-lines: with non-integer start va= lue") + endline =3D startline + len(include_lines) + if text.endswith("\n"): + text =3D text[:-1] + tokens =3D NumberLines([([], text)], startline, endline) + for classes, value in tokens: + if classes: + literal_block +=3D nodes.inline(value, value, + classes=3Dclasses) + else: + literal_block +=3D nodes.Text(value, value) + else: + literal_block +=3D nodes.Text(text, text) + return [literal_block] + + def code(self, path, include_lines): + """Output a code block""" + + self.options["source"] =3D path + codeblock =3D CodeBlock(self.name, + [self.options.pop("code")], # arguments + self.options, + include_lines, + self.lineno, + self.content_offset, + self.block_text, + self.state, + self.state_machine) + return codeblock.run() + def run(self): """Include a file as part of the content of this reST file.""" env =3D self.state.document.settings.env @@ -200,6 +246,13 @@ class KernelInclude(Include): startline =3D self.options.get("start-line", None) endline =3D self.options.get("end-line", None) =20 + if "literal" in self.options: + ouptut_type =3D "literal" + elif "code" in self.options: + ouptut_type =3D "code" + else: + ouptut_type =3D "normal" + # Get optional arguments to related to cross-references generation if 'generate-cross-refs' in self.options: rawtext =3D self.read_rawtext_with_xrefs(env, path) @@ -213,50 +266,15 @@ class KernelInclude(Include): =20 rawtext =3D self.apply_range(rawtext) =20 + if ouptut_type =3D=3D "literal": + return self.literal(path, tab_width, rawtext) + include_lines =3D statemachine.string2lines(rawtext, tab_width, convert_whitespace=3DTru= e) - if "literal" in self.options: - # Convert tabs to spaces, if `tab_width` is positive. - if tab_width >=3D 0: - text =3D rawtext.expandtabs(tab_width) - else: - text =3D rawtext - literal_block =3D nodes.literal_block(rawtext, source=3Dpath, - classes=3Dself.options.get= ("class", []) - ) - literal_block.line =3D 1 - self.add_name(literal_block) - if "number-lines" in self.options: - try: - startline =3D int(self.options["number-lines"] or 1) - except ValueError: - raise self.error(":number-lines: with non-integer star= t value") - endline =3D startline + len(include_lines) - if text.endswith("\n"): - text =3D text[:-1] - tokens =3D NumberLines([([], text)], startline, endline) - for classes, value in tokens: - if classes: - literal_block +=3D nodes.inline(value, value, - classes=3Dclasses) - else: - literal_block +=3D nodes.Text(value, value) - else: - literal_block +=3D nodes.Text(text, text) - return [literal_block] =20 - if "code" in self.options: - self.options["source"] =3D path - codeblock =3D CodeBlock(self.name, - [self.options.pop("code")], # arguments - self.options, - include_lines, # content - self.lineno, - self.content_offset, - self.block_text, - self.state, - self.state_machine) - return codeblock.run() + if ouptut_type =3D=3D "code": + return self.code(path, include_lines) + self.state_machine.insert_input(include_lines, path) return [] =20 --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6385285042; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=d5HjfOdT0clD3yoiM6lYhZiILrRjQR7I9Kl7Qz85tOx7I4uMk6S6P6cC9Ei44CV5QFW/SaMGjeD6n5/ICrmmQrfhiZI98OzMVr2RO+SA4t9gmJNbsH2P9iL7OWIAkRMuCosZj8iMNfWbesJ9vihHl6d/q2QY1mrahBog4xHNrdk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=R6Yr4duEC8arn5hYajMGz65rXfC8mfkSt1jOlvNU8yA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=W3cGBM9HPZzwH7ZzG05iHCDTyu8qwZO1kHx1KvtGHfaQIJ+IJOyF4Jhw/aS/CFYzDksRblOYQ3UC1XID2bk1Vug7UES7e24fwdECwspLgXx7pHFVUTVHfa94p3lT8oqbd4iSzXGpyy213SCigTNtfGi25ysP6s1QQAdMQhab/4s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HqH23nLl; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HqH23nLl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3A596C2BCB6; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=R6Yr4duEC8arn5hYajMGz65rXfC8mfkSt1jOlvNU8yA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HqH23nLlB0O2qhbrws+Op98u1vfopuebC0S2m0L4PBLY4VIgLiBJTHsiQm0+bw92I LzYrzHa/8mx+UIQdWdBgobQFIlQPm58BtinK8j4wzICBF86AcMkPri3BPA/1U9AvN+ vdQmOJqgrNmpeoYnOsScGMTGP7/k1zsEQH5Q6DnHYHD64XKO8nysuoCzfdnORQmKX9 ycW9haBIgMrQArSNfcvXfWSxYMmVXwcstKvXNo6zRIvz10adlDFJV4J3EaL5HUI7i+ F0O/Mew5N5fGsjQSGBw+qQRNFus8PZpQbtQeyawiegAVKCcwkcwNicckrNWapXU/aU /tpvOmsvce/6Q== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT96-1mXD; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 17/24] docs: kernel_include.py: add support to generate a TOC table Date: Thu, 21 Aug 2025 16:21:23 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" When generate-cross-refs is used, instead of just implementing the default of generating a literal block, we can also generate a ReST file as a TOC. The advantage is that, by being a ReST file, missing references will point to the place inside the header file that has the broken link. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 36 ++++++++++++++++---------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 0909eb3a07ea..79682408105e 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -89,6 +89,7 @@ class KernelInclude(Include): option_spec.update({ 'generate-cross-refs': directives.flag, 'warn-broken': directives.flag, + 'toc': directives.flag, 'exception-file': directives.unchanged, }) =20 @@ -111,7 +112,7 @@ class KernelInclude(Include): except UnicodeError as error: raise self.severe('Problem with directive:\n%s' % ErrorStr= ing(error)) =20 - def read_rawtext_with_xrefs(self, env, path): + def read_rawtext_with_xrefs(self, env, path, output_type): parser =3D ParseDataStructs() parser.parse_file(path) =20 @@ -126,7 +127,10 @@ class KernelInclude(Include): if 'warn-broken' in self.options: env._xref_files.add(path) =20 - return parser.gen_output() + if output_type =3D=3D "toc": + return parser.gen_toc() + + return ".. parsed-literal::\n\n" + parser.gen_output() =20 def apply_range(self, rawtext): # Get to-be-included content @@ -243,39 +247,43 @@ class KernelInclude(Include): e_handler =3D self.state.document.settings.input_encoding_error_ha= ndler tab_width =3D self.options.get("tab-width", self.state.document.settings.tab_widt= h) - startline =3D self.options.get("start-line", None) - endline =3D self.options.get("end-line", None) =20 if "literal" in self.options: - ouptut_type =3D "literal" + output_type =3D "literal" elif "code" in self.options: - ouptut_type =3D "code" + output_type =3D "code" else: - ouptut_type =3D "normal" + output_type =3D "rst" =20 # Get optional arguments to related to cross-references generation - if 'generate-cross-refs' in self.options: - rawtext =3D self.read_rawtext_with_xrefs(env, path) + if "generate-cross-refs" in self.options: + if "toc" in self.options: + output_type =3D "toc" + + rawtext =3D self.read_rawtext_with_xrefs(env, path, output_typ= e) + + # When :generate-cross-refs: is used, the input is always a C + # file, so it has to be handled as a parsed-literal + if output_type =3D=3D "rst": + output_type =3D "literal" =20 title =3D os.path.basename(path) - - if "code" not in self.options: - rawtext =3D ".. parsed-literal::\n\n" + rawtext else: rawtext =3D self.read_rawtext(path, encoding) =20 rawtext =3D self.apply_range(rawtext) =20 - if ouptut_type =3D=3D "literal": + if output_type =3D=3D "literal": return self.literal(path, tab_width, rawtext) =20 include_lines =3D statemachine.string2lines(rawtext, tab_width, convert_whitespace=3DTru= e) =20 - if ouptut_type =3D=3D "code": + if output_type =3D=3D "code": return self.code(path, include_lines) =20 self.state_machine.insert_input(include_lines, path) + return [] =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC03A283FDE; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=iA+jKjbPvnz7K9cGyJBfhSi19bsrGSgEYGkS9JIAfNUvhHkwJHiMJlrpXGoRQIFzyr1H87DJWCqGqIVSdVZuZA7NFjzDBYGhHb/lnCQ6hgc0vmk6MYvwH2Ifec8ePk5qIV+XtfqsEM5F3xdPeZyDbJGoL8Rwu4fRuNxycwkc08k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=Ryz5fK1HBEzZjQk/ZMX2f1BMYW5Jfa7MSrJhKnqCZ5k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pJIhD38jehjwoe0f2zNOYXNTaCkn04yvO+rmH6s5MYZRlIlouCP/xrUwYVFaqcyormQ1rpTrW/vzI567yH55uvGSufO9lruXRfdIK71jqm8OA/1VEvslsdnAbWpJL66dLu33zHKQ34G70h6kyamtdOmGOYWlgjunsrObujeLyII= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kVdoq3vS; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kVdoq3vS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3822BC2BCB3; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=Ryz5fK1HBEzZjQk/ZMX2f1BMYW5Jfa7MSrJhKnqCZ5k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kVdoq3vS2SugNOsq7BvkiRhD2lu+lhaIXr0UJ0b1qzyBBkhZmF5RUpJ8dHB1kKvYi 8/9QuyCURlQALDJuCWRYTapDSBNqDX/P3o13b4zKg7vuDMLI10o/bt/ayNDTaBZCoX Vn9iD9Zino0PkhKbXaC1y0tfRp4xSQ5lPE59akzhyECskxQ1QA8R6IDhxRem22A1sp vZkXrTn7xQfRwONFuHZ8J6siXyZ9PFumkfbcWcpFZjn8NgKiFLhvhMGXeIDuyzW1ZM LQi8qUpg6vuY2XHpt+whPR2EoLn5uU918NFAicDZtiK5vQM2l4Yd2ExlM/h+zPqkhY lIJfwxcCJV93A== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT9A-1tBm; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 18/24] docs: kernel_include.py: append line numbers to better report errors Date: Thu, 21 Aug 2025 16:21:24 +0200 Message-ID: <4e6309dbd113648a481c1bbe364bfe477cc3f598.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" It is best to point to the original line of code that generated an error than to point to the beginning of a directive. Add support for it. It should be noticed that this won't work for literal or code blocks, as Sphinx will ignore it, pointing to the beginning of the directive. Yet, when the output is known to be in ReST format, like on TOC, this makes the error a lot more easier to be handled. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 81 ++++++++++++++------------ 1 file changed, 44 insertions(+), 37 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 79682408105e..90ed8428f776 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -60,6 +60,7 @@ import re import sys =20 from docutils import io, nodes, statemachine +from docutils.statemachine import ViewList from docutils.utils.error_reporting import SafeString, ErrorString from docutils.parsers.rst import directives from docutils.parsers.rst.directives.body import CodeBlock, NumberLines @@ -112,7 +113,14 @@ class KernelInclude(Include): except UnicodeError as error: raise self.severe('Problem with directive:\n%s' % ErrorStr= ing(error)) =20 - def read_rawtext_with_xrefs(self, env, path, output_type): + def xref_text(self, env, path, tab_width): + """ + Read and add contents from a C file parsed to have cross reference= s. + + There are two types of supported output here: + - A C source code with cross-references; + - a TOC table containing cross references. + """ parser =3D ParseDataStructs() parser.parse_file(path) =20 @@ -127,10 +135,33 @@ class KernelInclude(Include): if 'warn-broken' in self.options: env._xref_files.add(path) =20 - if output_type =3D=3D "toc": - return parser.gen_toc() + if "toc" in self.options: + rawtext =3D parser.gen_toc() + else: + rawtext =3D ".. parsed-literal::\n\n" + parser.gen_output() + self.apply_range(rawtext) =20 - return ".. parsed-literal::\n\n" + parser.gen_output() + title =3D os.path.basename(path) + + include_lines =3D statemachine.string2lines(rawtext, tab_width, + convert_whitespace=3DTru= e) + + # Append line numbers data + + startline =3D self.options.get('start-line', None) + + result =3D ViewList() + if startline and startline > 0: + offset =3D startline - 1 + else: + offset =3D 0 + + for ln, line in enumerate(include_lines, start=3Doffset): + result.append(line, path, ln) + + self.state_machine.insert_input(result, path) + + return [] =20 def apply_range(self, rawtext): # Get to-be-included content @@ -195,9 +226,12 @@ class KernelInclude(Include): literal_block +=3D nodes.Text(text, text) return [literal_block] =20 - def code(self, path, include_lines): + def code(self, path, tab_width): """Output a code block""" =20 + include_lines =3D statemachine.string2lines(rawtext, tab_width, + convert_whitespace=3DTru= e) + self.options["source"] =3D path codeblock =3D CodeBlock(self.name, [self.options.pop("code")], # arguments @@ -244,47 +278,20 @@ class KernelInclude(Include): =20 encoding =3D self.options.get("encoding", self.state.document.settings.input_enc= oding) - e_handler =3D self.state.document.settings.input_encoding_error_ha= ndler tab_width =3D self.options.get("tab-width", self.state.document.settings.tab_widt= h) =20 - if "literal" in self.options: - output_type =3D "literal" - elif "code" in self.options: - output_type =3D "code" - else: - output_type =3D "rst" - # Get optional arguments to related to cross-references generation if "generate-cross-refs" in self.options: - if "toc" in self.options: - output_type =3D "toc" - - rawtext =3D self.read_rawtext_with_xrefs(env, path, output_typ= e) - - # When :generate-cross-refs: is used, the input is always a C - # file, so it has to be handled as a parsed-literal - if output_type =3D=3D "rst": - output_type =3D "literal" - - title =3D os.path.basename(path) - else: - rawtext =3D self.read_rawtext(path, encoding) + return self.xref_text(env, path, tab_width) =20 + rawtext =3D self.read_rawtext(path, encoding) rawtext =3D self.apply_range(rawtext) =20 - if output_type =3D=3D "literal": - return self.literal(path, tab_width, rawtext) + if "code" in self.options: + return self.code(path, tab_width, rawtext) =20 - include_lines =3D statemachine.string2lines(rawtext, tab_width, - convert_whitespace=3DTru= e) - - if output_type =3D=3D "code": - return self.code(path, include_lines) - - self.state_machine.insert_input(include_lines, path) - - return [] + return self.literal(path, tab_width, rawtext) =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D =20 --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC820284B4E; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=U/Ti2l6qQyv+kF8bqyIblIJoyvAZnTQc/VZVgl6vIgp+Z7cLLUcJnH1bvlTt1ccNejCmtxLNG0EPbpJAglo732XDiEvJhc3hwcl0H3mXUmB2jGAvABJXn8K2q+8J1lvKPXHLOnPgTwvgQ//x2jwfwPyt++c4PGTyqZV0gXVoaMo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=Ws1QXmr61TBgn6INp6mbk145NP3+6J63X8KSGxb87Js=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WoEISo7Nd+uHPPulCtc/kvB2EVRZYVLz8TbwDRb6KwDweu4anvqpU1erDivEx0RDvvtVJ/yj1akL+IuJfbYL0jIcHjl5LDlPSvNxxumxG8lZbTDi0NuX909woGxDAnWGN3AhdcCwlK2lnwsXZNbpeHO0TfQro+i9C4dPKDJeyxo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Po/npVGg; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Po/npVGg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3860BC4AF0E; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=Ws1QXmr61TBgn6INp6mbk145NP3+6J63X8KSGxb87Js=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Po/npVGgnsdCdD5oVXuzmKjwuCyVuhasVX1oMR6sWXAwhUpxLoJFf+q4e8RoDOHB5 kvYtePU5w6gcHXlsqtg/Gi1fMQ8mFkxOx5ZMr3y/nDpdQ0HtVNyUNMFhegA8up6chy aS1l9rKxltnST7NQt65ito0gKqBsZpGW5MBDqwfTmjJNVRh7c3C7bg0aC0OML8K8Fd xRtltGsKMikh5K63IsvcC5LXZg4Q9q0Vklo4KQxhBIDpAeiiqgqXzomlTjjuYqing1 PZdGRhscAfmkHQA3nBeWSsgIgAhogRLHN/Vb97z703MOZGUrFMwcqHGYxGnjWJS0lZ LueazZwTfK05w== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT9E-1zuD; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 19/24] docs: kernel_include.py: move apply_range() and add a docstring Date: Thu, 21 Aug 2025 16:21:25 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" While not required, better to have caller functions at the end. As apply_range() is now called by xref_text(), move it to be before the latter. No functional changes. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 68 ++++++++++++++------------ 1 file changed, 36 insertions(+), 32 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 90ed8428f776..fd4887f80577 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -113,6 +113,42 @@ class KernelInclude(Include): except UnicodeError as error: raise self.severe('Problem with directive:\n%s' % ErrorStr= ing(error)) =20 + def apply_range(self, rawtext): + """ + Handles start-line, end-line, start-after and end-before parameters + """ + + # Get to-be-included content + startline =3D self.options.get('start-line', None) + endline =3D self.options.get('end-line', None) + try: + if startline or (endline is not None): + lines =3D rawtext.splitlines() + rawtext =3D '\n'.join(lines[startline:endline]) + except UnicodeError as error: + raise self.severe(f'Problem with "{self.name}" directive:\n' + + io.error_string(error)) + # start-after/end-before: no restrictions on newlines in match-tex= t, + # and no restrictions on matching inside lines vs. line boundaries + after_text =3D self.options.get("start-after", None) + if after_text: + # skip content in rawtext before *and incl.* a matching text + after_index =3D rawtext.find(after_text) + if after_index < 0: + raise self.severe('Problem with "start-after" option of "%= s" ' + "directive:\nText not found." % self.nam= e) + rawtext =3D rawtext[after_index + len(after_text) :] + before_text =3D self.options.get("end-before", None) + if before_text: + # skip content in rawtext after *and incl.* a matching text + before_index =3D rawtext.find(before_text) + if before_index < 0: + raise self.severe('Problem with "end-before" option of "%s= " ' + "directive:\nText not found." % self.nam= e) + rawtext =3D rawtext[:before_index] + + return rawtext + def xref_text(self, env, path, tab_width): """ Read and add contents from a C file parsed to have cross reference= s. @@ -163,38 +199,6 @@ class KernelInclude(Include): =20 return [] =20 - def apply_range(self, rawtext): - # Get to-be-included content - startline =3D self.options.get('start-line', None) - endline =3D self.options.get('end-line', None) - try: - if startline or (endline is not None): - lines =3D rawtext.splitlines() - rawtext =3D '\n'.join(lines[startline:endline]) - except UnicodeError as error: - raise self.severe(f'Problem with "{self.name}" directive:\n' - + io.error_string(error)) - # start-after/end-before: no restrictions on newlines in match-tex= t, - # and no restrictions on matching inside lines vs. line boundaries - after_text =3D self.options.get("start-after", None) - if after_text: - # skip content in rawtext before *and incl.* a matching text - after_index =3D rawtext.find(after_text) - if after_index < 0: - raise self.severe('Problem with "start-after" option of "%= s" ' - "directive:\nText not found." % self.nam= e) - rawtext =3D rawtext[after_index + len(after_text) :] - before_text =3D self.options.get("end-before", None) - if before_text: - # skip content in rawtext after *and incl.* a matching text - before_index =3D rawtext.find(before_text) - if before_index < 0: - raise self.severe('Problem with "end-before" option of "%s= " ' - "directive:\nText not found." % self.nam= e) - rawtext =3D rawtext[:before_index] - - return rawtext - def literal(self, path, tab_width, rawtext): """Output a literal block""" =20 --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E30FF285C89; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; cv=none; b=Xp9ObjSle798aUpTvRdijWGfzLqyW9v6rTPjHnhNyyvBjiE1WmeIs/Fm/iWOM01vJ30dsSQLd8xjFpYzxre9oUgFXmaPyw/5Tck2/CaCw+x3nsdbIS9E7yyrO3MMryDzFuMHtIzAMfIcbRiBjMJoKfzITjM5KzIuuovZXThidKc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; c=relaxed/simple; bh=TiBwJFe/nRO7mW633U4ruwZsiS7f5kmzwOmkA8jDJ8g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WwjJfkTSJR0fDhFYAxd5AT/v9ZsYgYwx2T0vn0gGCVSpQD26jAtZj2WoGn7+WhWC783uIgFhEG8ITJNILDcG5HQ3f2zf+Rpim8S+oD7ywscG043NBeYWOENUqx9XWDK60xLCk/PdmCQYKVYv8Uds8/ZLV174XI7iaRsvH2d7dPo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ULo+QUmB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ULo+QUmB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4C141C2BCC9; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=TiBwJFe/nRO7mW633U4ruwZsiS7f5kmzwOmkA8jDJ8g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ULo+QUmB54xEzR0C4mSfwuNIvkI0BqwOrRW9zwTCpXX2KoiCZyBzA8LWulT1N/TRE injM4rnqbmpUMHAnQhm43jBXklaQO0JIqq/X/ZfCUGnkVnCdn9gs2jCgWRlzuB4es7 AaEIO+oZ49T2gZxFWBllVu9cg0ABIyEaK1aJr5s1L4NVgL9IXxMNH5mqFHIBnn7JJy eV20V956KN9i0ODMDmOR7zx9i4O1GQhSLVMBNRuayqyg0QY2ltyVJra9IvjFvYGXx9 ripHaUqePJ84z0uyUlbvn/RhgcEFkdfLWysnG20D2DM7+KJP6sdOpHbTVmRzbQeGMt n7Zcc96zrVysA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT9I-26Yc; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 20/24] docs: kernel_include.py: remove line numbers from parsed-literal Date: Thu, 21 Aug 2025 16:21:26 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" When parsed-literal directive is added to rawtext, while cross references will be properly displayed, Sphinx will ignore line numbers. So, it is not worth adding them. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index fd4887f80577..3a1753486319 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -171,13 +171,24 @@ class KernelInclude(Include): if 'warn-broken' in self.options: env._xref_files.add(path) =20 - if "toc" in self.options: - rawtext =3D parser.gen_toc() - else: + if "toc" not in self.options: + rawtext =3D ".. parsed-literal::\n\n" + parser.gen_output() self.apply_range(rawtext) =20 - title =3D os.path.basename(path) + include_lines =3D statemachine.string2lines(rawtext, tab_width, + convert_whitespace= =3DTrue) + + # Sphinx always blame the ".. ", so placing + # line numbers here won't make any difference + + self.state_machine.insert_input(include_lines, path) + return [] + + # TOC output is a ReST file, not a literal. So, we can add line + # numbers + + rawtext =3D parser.gen_toc() =20 include_lines =3D statemachine.string2lines(rawtext, tab_width, convert_whitespace=3DTru= e) --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E280A2857F7; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; cv=none; b=pxY7T9luwTGbR55Dna40FH2ZBuoLJ0xGsuNxU93/YK+AM01EA8fHP1XW7PIxLxv6wC/Ij7uXVzF9/npoGvdgf7+xnZvEdqCNviuh5OYvSSu9DcuNrTaEduiP1lIvaiPJr8w1gKKE1h0snzl2kJgdwnZRHAVooKV1uUEV4c3qijI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; c=relaxed/simple; bh=LQfriqJ7w45m1TsmRVR1XR73bImTQyUEqk93YtSCuzA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=N8XPqzh21EX9QPpYlZyiZlxpYyuzZHupActfJPI91rWqVmRvNvmx9KZw963CgTIntYYM39wQgXSBJir931XKFhvsZ/dXNTuDG0fTTANoP7D64IdrWkqGO0DYKyvUdkSvoCa0OinYASPHlvG91XB3x0Pv+cwJsBdyuOjIn8BIykE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HrSDWfIf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HrSDWfIf" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4D934C2BCF5; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=LQfriqJ7w45m1TsmRVR1XR73bImTQyUEqk93YtSCuzA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HrSDWfIf7ufCni7PxZ62/b7X3KzBuHr3HM03lebFs6XM9xW98cOU+TplnNkMgCi1d kyx2B39dd6dIqq/LaBPX0pf9c+whwhkjz/BugfmXqYA7v9gk+3Q1FWeTSswzBWgLO7 gkcZgyanlORJ1W5VX7ZVZyCU6dLvYThv9wGu3wfh8J3yFxe3/MfwuBHry8dZBRFJ9k eptdpIda1A7cMJY76XXOSfGp3Y93DzPdVvrPORrhrozdSmz3LELXVhrYDceuUUOyDz JGzLZWMNW/Qvabcl32hLlsBLaAvHR2kReC5OjOMOTK5UnGmOvGU7Bg4+/UBPsEz8Cf Bq3UlqEqMT2TQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT9M-2DIg; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 21/24] docs: kernel_include.py: remove Include class inheritance Date: Thu, 21 Aug 2025 16:21:27 +0200 Message-ID: <69206f9a5709c39591be515397fde851658eedc0.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" While the original code came from the Sphinx Include class, such class is monolithic: it has only one function that does everything, and 3 variables that are used: - required_arguments - optional_arguments - option_spec So, basically those are the only members that remain from the original class, but hey! Those are the same vars that every other Sphinx directive extension has to define! In summary, keeping inheritance here doesn't make much sense. Worse than that, kernel-include doesn't support the current set of options that the original Include class has, but it also has its own set of options. So, let's fill in the argument vars with what it does support, dropping the rest. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 40 ++++++++++++++++++++------ 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 3a1753486319..e6f734476ab3 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -62,9 +62,8 @@ import sys from docutils import io, nodes, statemachine from docutils.statemachine import ViewList from docutils.utils.error_reporting import SafeString, ErrorString -from docutils.parsers.rst import directives +from docutils.parsers.rst import Directive, directives from docutils.parsers.rst.directives.body import CodeBlock, NumberLines -from docutils.parsers.rst.directives.misc import Include =20 from sphinx.util import logging =20 @@ -81,18 +80,43 @@ RE_SIMPLE_REF =3D re.compile(r'`([^`]+)`') =20 =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D -class KernelInclude(Include): - """KernelInclude (``kernel-include``) directive""" +class KernelInclude(Directive): + """ + KernelInclude (``kernel-include``) directive =20 - # Add extra options - option_spec =3D Include.option_spec.copy() + Most of the stuff here came from Include directive defined at: + docutils/parsers/rst/directives/misc.py =20 - option_spec.update({ + Yet, overriding the class don't has any benefits: the original class + only have run() and argument list. Not all of them are implemented, + when checked against latest Sphinx version, as with time more arguments + were added. + + So, keep its own list of supported arguments + """ + + required_arguments =3D 1 + optional_arguments =3D 0 + final_argument_whitespace =3D True + option_spec =3D { + 'literal': directives.flag, + 'code': directives.unchanged, + 'encoding': directives.encoding, + 'tab-width': int, + 'start-line': int, + 'end-line': int, + 'start-after': directives.unchanged_required, + 'end-before': directives.unchanged_required, + # ignored except for 'literal' or 'code': + 'number-lines': directives.unchanged, # integer or None + 'class': directives.class_option, + + # Arguments that aren't from Sphinx Include directive 'generate-cross-refs': directives.flag, 'warn-broken': directives.flag, 'toc': directives.flag, 'exception-file': directives.unchanged, - }) + } =20 def read_rawtext(self, path, encoding): """Read and process file content with error handling""" --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7CCB28505A; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=PLma+dfdsW0I+FS31gyBAoXfCazmLJ/0zhQmXbDalqoQ/UAeBYKbakL3QncMPDyGhAZfmpfDzVCTeHNk9btM/LvKxLg0U/M3vXcgx10BNzUMIHeMJAzdR39bvfO0hwqpehei7tNHNlS7sHTt3SN0t3waeDxEjHvNdasERRvwzuQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=LbPxF3kGKvZLO+UE6KiArHqAGzi0BSWTiNNAdqSOJS4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DFqiXuD6jYfKxth6luaYGSQHx7kJz7X1gaW5WKAaJj02nOswNO/9ENPGSzbL8vc+lcywj4lybmr0Cv3S7ReoaxnWhF9gxQIaYNbIrGP6QixkfP5n2Dk1jgvk1/X3V2UEC2XnlsIiribBiCoOfop8mLnos2zI9w//wm6xIHjoytU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Fb53XVar; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Fb53XVar" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4D4FDC2BCC7; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=LbPxF3kGKvZLO+UE6KiArHqAGzi0BSWTiNNAdqSOJS4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Fb53XVar4QcXj4ZqCEwqyRSFWaW6oNAes6/hAsfIcE/O0nvOYDjoBkRkmrF3bFrAk ZLnSl0IxtpE+IBFG7usy81JmKgKh/WxzsNprAaBzAa0kxzmw9Z5AX6F4otQvch/1vM cHlhLEX3zetONcuvTdbNVo7ID0eflU9vGvZNEBNHdsm9KbDbD4g92Hk0SvHtk3nLWw eOsfzTFmvZ8Vu+3j/f6BUTQjuWXOZ2poEjFc6R6odv98EGZ+1Sy9xwwgg4bR4uHj8z PeG8gaLUrPLQhCWXmltDoauEfZR/EX9M/a4EdpJDk30CjjvVC5YQ6OAlywOmry/l9B QjcvpzWHCC4dw== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT9Q-2JzK; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 22/24] docs: kernel_include.py: document all supported parameters Date: Thu, 21 Aug 2025 16:21:28 +0200 Message-ID: <2bbd518941dd4cae29b3a6f86c6afae711347266.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" As we're actually a fork of Sphinx Include, update its docstring to contain the documentation for the actual implemented parameters. Let's use :param: for parameters, as defined at: https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 88 +++++++++++++++++--------- 1 file changed, 58 insertions(+), 30 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index e6f734476ab3..23566ab74866 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -2,53 +2,81 @@ # SPDX-License-Identifier: GPL-2.0 # pylint: disable=3DR0903, R0912, R0914, R0915, C0209,W0707 =20 + """ - kernel-include - ~~~~~~~~~~~~~~ +Implementation of the ``kernel-include`` reST-directive. =20 - Implementation of the ``kernel-include`` reST-directive. +:copyright: Copyright (C) 2016 Markus Heiser +:license: GPL Version 2, June 1991 see linux/COPYING for details. =20 - :copyright: Copyright (C) 2016 Markus Heiser - :license: GPL Version 2, June 1991 see linux/COPYING for details. +The ``kernel-include`` reST-directive is a replacement for the ``include`` +directive. The ``kernel-include`` directive expand environment variables in +the path name and allows to include files from arbitrary locations. =20 - The ``kernel-include`` reST-directive is a replacement for the ``inclu= de`` - directive. The ``kernel-include`` directive expand environment variabl= es in - the path name and allows to include files from arbitrary locations. +.. hint:: =20 - .. hint:: + Including files from arbitrary locations (e.g. from ``/etc``) is a + security risk for builders. This is why the ``include`` directive from + docutils *prohibit* pathnames pointing to locations *above* the filesy= stem + tree where the reST document with the include directive is placed. =20 - Including files from arbitrary locations (e.g. from ``/etc``) is a - security risk for builders. This is why the ``include`` directive fr= om - docutils *prohibit* pathnames pointing to locations *above* the file= system - tree where the reST document with the include directive is placed. +Substrings of the form $name or ${name} are replaced by the value of +environment variable name. Malformed variable names and references to +non-existing variables are left unchanged. =20 - Substrings of the form $name or ${name} are replaced by the value of - environment variable name. Malformed variable names and references to - non-existing variables are left unchanged. +**Supported Sphinx Include Options**: =20 - This extension overrides Sphinx include directory, adding some extra - arguments: +:param literal: + If present, the included file is inserted as a literal block. =20 - 1. :generate-cross-refs: +:param code: + Specify the language for syntax highlighting (e.g., 'c', 'python'). =20 - If present, instead of reading the file, it calls ParseDataStructs= () - class, which converts C data structures into cross-references to - be linked to ReST files containing a more comprehensive documentat= ion; +:param encoding: + Specify the encoding of the included file (default: 'utf-8'). =20 - 2. :exception-file: +:param tab-width: + Specify the number of spaces that a tab represents. =20 - Used together with :generate-cross-refs +:param start-line: + Line number at which to start including the file (1-based). =20 - Points to a file containing rules to ignore C data structs or to - use a different reference name, optionally using a different - reference type. +:param end-line: + Line number at which to stop including the file (inclusive). =20 - 3. :warn-broken: +:param start-after: + Include lines after the first line matching this text. =20 - Used together with :generate-cross-refs: +:param end-before: + Include lines before the first line matching this text. =20 - Detect if the auto-generated cross references doesn't exist. +:param number-lines: + Number the included lines (integer specifies start number). + Only effective with 'literal' or 'code' options. =20 +:param class: + Specify HTML class attribute for the included content. + +**Kernel-specific Extensions**: + +:param generate-cross-refs: + If present, instead of directly including the file, it calls + ParseDataStructs() to convert C data structures into cross-references + that link to comprehensive documentation in other ReST files. + +:param exception-file: + (Used with generate-cross-refs) + + Path to a file containing rules for handling special cases: + - Ignore specific C data structures + - Use alternative reference names + - Specify different reference types + +:param warn-broken: + (Used with generate-cross-refs) + + Enables warnings when auto-generated cross-references don't point to + existing documentation targets. """ =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7277228CF49; Thu, 21 Aug 2025 14:21:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; cv=none; b=kx7UYxjKnVS++k9OrgTG/JIpZLdmXueBGxX4F7/Ysgky8TGqxvKITAlKDOjI6cNrdyMFUKTCHSYb7jJ7gjfc+UX3psqQgTUrevheu/LSRh4qhQuqLqKNryf8x0oJpHkJhUvHG0dA+m5l9MmAJVH+56oBDHTyxi0FELc/vF9kAOc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; c=relaxed/simple; bh=KsD1vNVCtNtIDrdngUGcQADwaGx5iOiHWVt8kHIS/5Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KM3fBZGJVn873cMptQJXkrXnZjsjCYZ6v/bMUohEz1hYdRTr7y7Rd9uesCvhs32CTOeIHTuIx8GM6pG/GLmoGlM9ACPOYIIEy2HYoRoAR5J3oDFr5jY4Y+4hz4FXvn5sLHA1U8tLgJAtv5crK2X+o4jfkTjPnlDh1cUqgxrQgjU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ExsOIfGY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ExsOIfGY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D1A7C4CEF4; Thu, 21 Aug 2025 14:21:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786100; bh=KsD1vNVCtNtIDrdngUGcQADwaGx5iOiHWVt8kHIS/5Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ExsOIfGYWwOaZ5MoutZDuMpVbAfW2+2P6Snm8/PlalBhiMJuxudAABK0DOTLiHK2L ehTj9R2/5GjSPN7ipKhryQfAfOH8w3/Mt1Y2Kj6C4GtMs/qK7DhQ+mhhE9K2G5rW/C fIcRPdV6ZYw9mEy2fUjhcKv8Pp1LdGD5y09MMPrMxBpQraMsXBTxEy8iLpvy5jvt5F T4V9X7n1xWvgJjuaB1exw+ZmnHONTkLfaquuoiJghBclOlToR9bOefjBxxm01Tzhs8 5huvuWg1s2IbOIHfdK32ckJ6QocptTJlxpxLlJ8BOd4VMGMdzHJiyYOFhXr+TSalPY S3BwYpixHvN3w== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT9U-2R6C; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Benjamin Gaignard , Erling Ljunggren , Hans Verkuil , Hans de Goede , Mauro Carvalho Chehab , Ricardo Ribalda , Sean Young , Yunke Cao , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org Subject: [PATCH 23/24] scripts: sphinx-build-wrapper: get rid of uapi/media Makefile Date: Thu, 21 Aug 2025 16:21:29 +0200 Message-ID: <72c95996d5202b467ad55b5948cc8e70d99ac5cd.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Now that kernel-include directive supports parsing data structs directly, we can finally get rid of the horrible hack we added to support parsing media uAPI symbols. As a side effect, Documentation/output doesn't have anymore media auto-generated .rst files on it. Signed-off-by: Mauro Carvalho Chehab --- Documentation/userspace-api/media/Makefile | 64 ------------------- .../userspace-api/media/cec/cec-header.rst | 5 +- .../media/{ =3D> cec}/cec.h.rst.exceptions | 0 .../media/{ =3D> dvb}/ca.h.rst.exceptions | 0 .../media/{ =3D> dvb}/dmx.h.rst.exceptions | 0 .../media/{ =3D> dvb}/frontend.h.rst.exceptions | 0 .../userspace-api/media/dvb/headers.rst | 17 +++-- .../media/{ =3D> dvb}/net.h.rst.exceptions | 0 .../media/mediactl/media-header.rst | 5 +- .../{ =3D> mediactl}/media.h.rst.exceptions | 0 .../userspace-api/media/rc/lirc-header.rst | 4 +- .../media/{ =3D> rc}/lirc.h.rst.exceptions | 0 .../userspace-api/media/v4l/videodev.rst | 4 +- .../{ =3D> v4l}/videodev2.h.rst.exceptions | 0 scripts/sphinx-build-wrapper | 48 -------------- 15 files changed, 25 insertions(+), 122 deletions(-) delete mode 100644 Documentation/userspace-api/media/Makefile rename Documentation/userspace-api/media/{ =3D> cec}/cec.h.rst.exceptions = (100%) rename Documentation/userspace-api/media/{ =3D> dvb}/ca.h.rst.exceptions (= 100%) rename Documentation/userspace-api/media/{ =3D> dvb}/dmx.h.rst.exceptions = (100%) rename Documentation/userspace-api/media/{ =3D> dvb}/frontend.h.rst.except= ions (100%) rename Documentation/userspace-api/media/{ =3D> dvb}/net.h.rst.exceptions = (100%) rename Documentation/userspace-api/media/{ =3D> mediactl}/media.h.rst.exce= ptions (100%) rename Documentation/userspace-api/media/{ =3D> rc}/lirc.h.rst.exceptions = (100%) rename Documentation/userspace-api/media/{ =3D> v4l}/videodev2.h.rst.excep= tions (100%) diff --git a/Documentation/userspace-api/media/Makefile b/Documentation/use= rspace-api/media/Makefile deleted file mode 100644 index accc734d045a..000000000000 --- a/Documentation/userspace-api/media/Makefile +++ /dev/null @@ -1,64 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0 - -# Rules to convert a .h file to inline RST documentation - -SRC_DIR=3D$(srctree)/Documentation/userspace-api/media -PARSER =3D $(srctree)/tools/docs/parse-headers.py -UAPI =3D $(srctree)/include/uapi/linux -KAPI =3D $(srctree)/include/linux - -FILES =3D ca.h.rst dmx.h.rst frontend.h.rst net.h.rst \ - videodev2.h.rst media.h.rst cec.h.rst lirc.h.rst - -TARGETS :=3D $(addprefix $(BUILDDIR)/, $(FILES)) - -gen_rst =3D \ - echo ${PARSER} $< $@ $(SRC_DIR)/$(notdir $@).exceptions; \ - ${PARSER} $< $@ $(SRC_DIR)/$(notdir $@).exceptions - -quiet_gen_rst =3D echo ' PARSE $(patsubst $(srctree)/%,%,$<)'; \ - ${PARSER} $< $@ $(SRC_DIR)/$(notdir $@).exceptions - -silent_gen_rst =3D ${gen_rst} - -$(BUILDDIR)/ca.h.rst: ${UAPI}/dvb/ca.h ${PARSER} $(SRC_DIR)/ca.h.rst.excep= tions - @$($(quiet)gen_rst) - -$(BUILDDIR)/dmx.h.rst: ${UAPI}/dvb/dmx.h ${PARSER} $(SRC_DIR)/dmx.h.rst.ex= ceptions - @$($(quiet)gen_rst) - -$(BUILDDIR)/frontend.h.rst: ${UAPI}/dvb/frontend.h ${PARSER} $(SRC_DIR)/fr= ontend.h.rst.exceptions - @$($(quiet)gen_rst) - -$(BUILDDIR)/net.h.rst: ${UAPI}/dvb/net.h ${PARSER} $(SRC_DIR)/net.h.rst.ex= ceptions - @$($(quiet)gen_rst) - -$(BUILDDIR)/videodev2.h.rst: ${UAPI}/videodev2.h ${PARSER} $(SRC_DIR)/vide= odev2.h.rst.exceptions - @$($(quiet)gen_rst) - -$(BUILDDIR)/media.h.rst: ${UAPI}/media.h ${PARSER} $(SRC_DIR)/media.h.rst.= exceptions - @$($(quiet)gen_rst) - -$(BUILDDIR)/cec.h.rst: ${UAPI}/cec.h ${PARSER} $(SRC_DIR)/cec.h.rst.except= ions - @$($(quiet)gen_rst) - -$(BUILDDIR)/lirc.h.rst: ${UAPI}/lirc.h ${PARSER} $(SRC_DIR)/lirc.h.rst.exc= eptions - @$($(quiet)gen_rst) - -# Media build rules - -.PHONY: all html texinfo epub xml latex - -all: $(IMGDOT) $(BUILDDIR) ${TARGETS} -html: all -texinfo: all -epub: all -xml: all -latex: $(IMGPDF) all -linkcheck: - -clean: - -rm -f $(DOTTGT) $(IMGTGT) ${TARGETS} 2>/dev/null - -$(BUILDDIR): - $(Q)mkdir -p $@ diff --git a/Documentation/userspace-api/media/cec/cec-header.rst b/Documen= tation/userspace-api/media/cec/cec-header.rst index d70736ac2b1d..f67003bb8740 100644 --- a/Documentation/userspace-api/media/cec/cec-header.rst +++ b/Documentation/userspace-api/media/cec/cec-header.rst @@ -6,5 +6,6 @@ CEC Header File *************** =20 -.. kernel-include:: $BUILDDIR/cec.h.rst - +.. kernel-include:: include/uapi/linux/cec.h + :generate-cross-refs: + :exception-file: cec.h.rst.exceptions diff --git a/Documentation/userspace-api/media/cec.h.rst.exceptions b/Docum= entation/userspace-api/media/cec/cec.h.rst.exceptions similarity index 100% rename from Documentation/userspace-api/media/cec.h.rst.exceptions rename to Documentation/userspace-api/media/cec/cec.h.rst.exceptions diff --git a/Documentation/userspace-api/media/ca.h.rst.exceptions b/Docume= ntation/userspace-api/media/dvb/ca.h.rst.exceptions similarity index 100% rename from Documentation/userspace-api/media/ca.h.rst.exceptions rename to Documentation/userspace-api/media/dvb/ca.h.rst.exceptions diff --git a/Documentation/userspace-api/media/dmx.h.rst.exceptions b/Docum= entation/userspace-api/media/dvb/dmx.h.rst.exceptions similarity index 100% rename from Documentation/userspace-api/media/dmx.h.rst.exceptions rename to Documentation/userspace-api/media/dvb/dmx.h.rst.exceptions diff --git a/Documentation/userspace-api/media/frontend.h.rst.exceptions b/= Documentation/userspace-api/media/dvb/frontend.h.rst.exceptions similarity index 100% rename from Documentation/userspace-api/media/frontend.h.rst.exceptions rename to Documentation/userspace-api/media/dvb/frontend.h.rst.exceptions diff --git a/Documentation/userspace-api/media/dvb/headers.rst b/Documentat= ion/userspace-api/media/dvb/headers.rst index 88c3eb33a89e..c75f64cf21d5 100644 --- a/Documentation/userspace-api/media/dvb/headers.rst +++ b/Documentation/userspace-api/media/dvb/headers.rst @@ -7,10 +7,19 @@ Digital TV uAPI header files Digital TV uAPI headers *********************** =20 -.. kernel-include:: $BUILDDIR/frontend.h.rst +.. kernel-include:: include/uapi/linux/dvb/frontend.h + :generate-cross-refs: + :exception-file: frontend.h.rst.exceptions =20 -.. kernel-include:: $BUILDDIR/dmx.h.rst +.. kernel-include:: include/uapi/linux/dvb/dmx.h + :generate-cross-refs: + :exception-file: dmx.h.rst.exceptions =20 -.. kernel-include:: $BUILDDIR/ca.h.rst +.. kernel-include:: include/uapi/linux/dvb/ca.h + :generate-cross-refs: + :exception-file: ca.h.rst.exceptions + +.. kernel-include:: include/uapi/linux/dvb/net.h + :generate-cross-refs: + :exception-file: net.h.rst.exceptions =20 -.. kernel-include:: $BUILDDIR/net.h.rst diff --git a/Documentation/userspace-api/media/net.h.rst.exceptions b/Docum= entation/userspace-api/media/dvb/net.h.rst.exceptions similarity index 100% rename from Documentation/userspace-api/media/net.h.rst.exceptions rename to Documentation/userspace-api/media/dvb/net.h.rst.exceptions diff --git a/Documentation/userspace-api/media/mediactl/media-header.rst b/= Documentation/userspace-api/media/mediactl/media-header.rst index c674271c93f5..d561d2845f3d 100644 --- a/Documentation/userspace-api/media/mediactl/media-header.rst +++ b/Documentation/userspace-api/media/mediactl/media-header.rst @@ -6,5 +6,6 @@ Media Controller Header File **************************** =20 -.. kernel-include:: $BUILDDIR/media.h.rst - +.. kernel-include:: include/uapi/linux/media.h + :generate-cross-refs: + :exception-file: media.h.rst.exceptions diff --git a/Documentation/userspace-api/media/media.h.rst.exceptions b/Doc= umentation/userspace-api/media/mediactl/media.h.rst.exceptions similarity index 100% rename from Documentation/userspace-api/media/media.h.rst.exceptions rename to Documentation/userspace-api/media/mediactl/media.h.rst.exceptions diff --git a/Documentation/userspace-api/media/rc/lirc-header.rst b/Documen= tation/userspace-api/media/rc/lirc-header.rst index 54cb40b8a065..a53328327847 100644 --- a/Documentation/userspace-api/media/rc/lirc-header.rst +++ b/Documentation/userspace-api/media/rc/lirc-header.rst @@ -6,5 +6,7 @@ LIRC Header File **************** =20 -.. kernel-include:: $BUILDDIR/lirc.h.rst +.. kernel-include:: include/uapi/linux/lirc.h + :generate-cross-refs: + :exception-file: lirc.h.rst.exceptions =20 diff --git a/Documentation/userspace-api/media/lirc.h.rst.exceptions b/Docu= mentation/userspace-api/media/rc/lirc.h.rst.exceptions similarity index 100% rename from Documentation/userspace-api/media/lirc.h.rst.exceptions rename to Documentation/userspace-api/media/rc/lirc.h.rst.exceptions diff --git a/Documentation/userspace-api/media/v4l/videodev.rst b/Documenta= tion/userspace-api/media/v4l/videodev.rst index c866fec417eb..cde485bc9a5f 100644 --- a/Documentation/userspace-api/media/v4l/videodev.rst +++ b/Documentation/userspace-api/media/v4l/videodev.rst @@ -6,4 +6,6 @@ Video For Linux Two Header File ******************************* =20 -.. kernel-include:: $BUILDDIR/videodev2.h.rst +.. kernel-include:: include/uapi/linux/videodev2.h + :generate-cross-refs: + :exception-file: videodev2.h.rst.exceptions diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b= /Documentation/userspace-api/media/v4l/videodev2.h.rst.exceptions similarity index 100% rename from Documentation/userspace-api/media/videodev2.h.rst.exceptions rename to Documentation/userspace-api/media/v4l/videodev2.h.rst.exceptions diff --git a/scripts/sphinx-build-wrapper b/scripts/sphinx-build-wrapper index 0d13c19f6df3..abe8c26ae137 100755 --- a/scripts/sphinx-build-wrapper +++ b/scripts/sphinx-build-wrapper @@ -463,56 +463,10 @@ class SphinxBuilder: except subprocess.CalledProcessError as e: sys.exit(f"Error generating info docs: {e}") =20 - def get_make_media(self): - """ - The media uAPI requires an additional Makefile target. - """ - - mediadir =3D f"{self.obj}/userspace-api/media" - - make =3D os.environ.get("MAKE", "make") - build =3D os.environ.get("build", "-f $(srctree)/scripts/Makefile.= build obj") - - # Check if the script was started outside docs Makefile - if not os.environ.get("obj"): - mediadir =3D os.path.abspath(mediadir) - - # the build makefile var contains macros that require expand - make_media =3D f"{make} {build}=3D{mediadir}" - make_media =3D make_media.replace("$(", "${").replace(")", "}") - make_media =3D os.path.expandvars(make_media) - - # As it also contains multiple arguments, use shlex to split it - return shlex.split(make_media) - - def prepare_media(self, builder): - """ - Run userspace-api/media Makefile. - - The logic behind it are from the initial ports to Sphinx. - They're old and need to be replaced by a proper Sphinx extension. - While we don't do that, we need to explicitly call media Makefile - to build some files. - """ - - cmd =3D self.get_make_media() + [builder] - - if self.verbose: - print(" ".join(cmd)) - - with JobserverExec() as jobserver: - rc =3D jobserver.run(cmd, env=3Dself.env) - - if rc: - cmd_str =3D " ".join(cmd) - sys.exit(f"Failed to run {cmd_str}") - def cleandocs(self, builder): =20 shutil.rmtree(self.builddir, ignore_errors=3DTrue) =20 - self.prepare_media(builder) - def build(self, target, sphinxdirs=3DNone, conf=3D"conf.py", theme=3DNone, css=3DNone, paper=3DNone): """ @@ -533,8 +487,6 @@ class SphinxBuilder: if not sphinxbuild: sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n") =20 - self.prepare_media(builder) - if builder =3D=3D "latex": if not self.pdflatex_cmd and not self.latexmk_cmd: sys.exit("Error: pdflatex or latexmk required for PDF gene= ration") --=20 2.50.1 From nobody Sat Oct 4 00:27:00 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D0AE28AB0B; Thu, 21 Aug 2025 14:21:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; cv=none; b=RR/CP2Eup3K6XuXBQa0T4hI3JHLaWsK9A9U+teEGhYGW60J5ZKfsZFppLFTCQQMnWpA6PDYxrOFTILlLWmS9eGsqFf49K50d8MDiI7n6nkmEOmZ6rORfJa3vhm1NRHoxnF51wuSVtkzYzEps0Qn1PSOW0BkycRKFei2tQpxZDN4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786100; c=relaxed/simple; bh=uYEK1a2ZG5UzKHVraa8W5O9cCQbPaE3FbRWgQy8mxrI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VM6Bgr1A/iQu7NqVe/bNMmW06GQmJ+GGLa+j6DIhtTzh7OD7E53r9XLf6hqVP+9O3wUIbJYnlIGaDpNc/SG2a86mSU0AacSoJKrOzaUvfGwmFZPZaZCOoE/75lN/2FkOeZVaeKshrjYeesngUR36b2+wqzJtlsiHIFrXzXK5/i8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BYKypHZ/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BYKypHZ/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0C5FFC16AAE; Thu, 21 Aug 2025 14:21:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786100; bh=uYEK1a2ZG5UzKHVraa8W5O9cCQbPaE3FbRWgQy8mxrI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BYKypHZ/p75IMUVHV2u1aeOlh/lFPeeUMsCfZP8rey7YwoACZ0n8h3ZQh1cZxNNIt u9+y8UykM5RNnBnG9A6UqEVNuf6umgGmEeblRhJQ0ciUFv8IVJxEHVGdqHSJ3HkD6/ RlfgexA0Fq47+1g/jP2bBx4/pZ3GN2tokd9NdiY/gc91beWj5dO0OYR28yEYbX389n F0EfglEi0Pb/7mn0Ffyq1HAkXkWuPs8eM4YG9fbXgd09yHwSUXseVGYta1CZUM81li nXetHbQCfzUvgEw+tt6J6+ZKmBw15zHKVbLUyRFh7iP8CBBhXBaMiPytWizJqKRxmo FdCLGFrrfFbtQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT9Y-2XwE; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Mauro Carvalho Chehab , linux-kernel@vger.kernel.org Subject: [PATCH 24/24] docs: sphinx: drop parse-headers.pl Date: Thu, 21 Aug 2025 16:21:30 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Now that we have a replacement in place, drop the old version. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/parse-headers.pl | 419 -------------------------- 1 file changed, 419 deletions(-) delete mode 100755 Documentation/sphinx/parse-headers.pl diff --git a/Documentation/sphinx/parse-headers.pl b/Documentation/sphinx/p= arse-headers.pl deleted file mode 100755 index 560685926cdb..000000000000 --- a/Documentation/sphinx/parse-headers.pl +++ /dev/null @@ -1,419 +0,0 @@ -#!/usr/bin/env perl -# SPDX-License-Identifier: GPL-2.0 -# Copyright (c) 2016 by Mauro Carvalho Chehab . - -use strict; -use Text::Tabs; -use Getopt::Long; -use Pod::Usage; - -my $debug; -my $help; -my $man; - -GetOptions( - "debug" =3D> \$debug, - 'usage|?' =3D> \$help, - 'help' =3D> \$man -) or pod2usage(2); - -pod2usage(1) if $help; -pod2usage(-exitstatus =3D> 0, -verbose =3D> 2) if $man; -pod2usage(2) if (scalar @ARGV < 2 || scalar @ARGV > 3); - -my ($file_in, $file_out, $file_exceptions) =3D @ARGV; - -my $data; -my %ioctls; -my %defines; -my %typedefs; -my %enums; -my %enum_symbols; -my %structs; - -# -# read the file and get identifiers -# - -my $is_enum =3D 0; -my $is_comment =3D 0; -open IN, $file_in or die "Can't open $file_in"; -while () { - $data .=3D $_; - - my $ln =3D $_; - if (!$is_comment) { - $ln =3D~ s,/\*.*(\*/),,g; - - $is_comment =3D 1 if ($ln =3D~ s,/\*.*,,); - } else { - if ($ln =3D~ s,^(.*\*/),,) { - $is_comment =3D 0; - } else { - next; - } - } - - if ($is_enum && $ln =3D~ m/^\s*([_\w][\w\d_]+)\s*[\,=3D]?/) { - my $s =3D $1; - my $n =3D $1; - $n =3D~ tr/A-Z/a-z/; - $n =3D~ tr/_/-/; - - $enum_symbols{$s} =3D "\\ :ref:`$s <$n>`\\ "; - - $is_enum =3D 0 if ($is_enum && m/\}/); - next; - } - $is_enum =3D 0 if ($is_enum && m/\}/); - - if ($ln =3D~ m/^\s*#\s*define\s+([_\w][\w\d_]+)\s+_IO/) { - my $s =3D $1; - my $n =3D $1; - $n =3D~ tr/A-Z/a-z/; - - $ioctls{$s} =3D "\\ :ref:`$s <$n>`\\ "; - next; - } - - if ($ln =3D~ m/^\s*#\s*define\s+([_\w][\w\d_]+)\s+/) { - my $s =3D $1; - my $n =3D $1; - $n =3D~ tr/A-Z/a-z/; - $n =3D~ tr/_/-/; - - $defines{$s} =3D "\\ :ref:`$s <$n>`\\ "; - next; - } - - if ($ln =3D~ m/^\s*typedef\s+([_\w][\w\d_]+)\s+(.*)\s+([_\w][\w\d_]+);/) { - my $s =3D $2; - my $n =3D $3; - - $typedefs{$n} =3D "\\ :c:type:`$n <$s>`\\ "; - next; - } - if ($ln =3D~ m/^\s*enum\s+([_\w][\w\d_]+)\s+\{/ - || $ln =3D~ m/^\s*enum\s+([_\w][\w\d_]+)$/ - || $ln =3D~ m/^\s*typedef\s*enum\s+([_\w][\w\d_]+)\s+\{/ - || $ln =3D~ m/^\s*typedef\s*enum\s+([_\w][\w\d_]+)$/) { - my $s =3D $1; - - $enums{$s} =3D "enum :c:type:`$s`\\ "; - - $is_enum =3D $1; - next; - } - if ($ln =3D~ m/^\s*struct\s+([_\w][\w\d_]+)\s+\{/ - || $ln =3D~ m/^\s*struct\s+([[_\w][\w\d_]+)$/ - || $ln =3D~ m/^\s*typedef\s*struct\s+([_\w][\w\d_]+)\s+\{/ - || $ln =3D~ m/^\s*typedef\s*struct\s+([[_\w][\w\d_]+)$/ - ) { - my $s =3D $1; - - $structs{$s} =3D "struct $s\\ "; - next; - } -} -close IN; - -# -# Handle multi-line typedefs -# - -my @matches =3D ($data =3D~ m/typedef\s+struct\s+\S+?\s*\{[^\}]+\}\s*(\S+)= \s*\;/g, - $data =3D~ m/typedef\s+enum\s+\S+?\s*\{[^\}]+\}\s*(\S+)\s*\;/g,); -foreach my $m (@matches) { - my $s =3D $m; - - $typedefs{$s} =3D "\\ :c:type:`$s`\\ "; - next; -} - -# -# Handle exceptions, if any -# - -my %def_reftype =3D ( - "ioctl" =3D> ":ref", - "define" =3D> ":ref", - "symbol" =3D> ":ref", - "typedef" =3D> ":c:type", - "enum" =3D> ":c:type", - "struct" =3D> ":c:type", -); - -if ($file_exceptions) { - open IN, $file_exceptions or die "Can't read $file_exceptions"; - while () { - next if (m/^\s*$/ || m/^\s*#/); - - # Parsers to ignore a symbol - - if (m/^ignore\s+ioctl\s+(\S+)/) { - delete $ioctls{$1} if (exists($ioctls{$1})); - next; - } - if (m/^ignore\s+define\s+(\S+)/) { - delete $defines{$1} if (exists($defines{$1})); - next; - } - if (m/^ignore\s+typedef\s+(\S+)/) { - delete $typedefs{$1} if (exists($typedefs{$1})); - next; - } - if (m/^ignore\s+enum\s+(\S+)/) { - delete $enums{$1} if (exists($enums{$1})); - next; - } - if (m/^ignore\s+struct\s+(\S+)/) { - delete $structs{$1} if (exists($structs{$1})); - next; - } - if (m/^ignore\s+symbol\s+(\S+)/) { - delete $enum_symbols{$1} if (exists($enum_symbols{$1})); - next; - } - - # Parsers to replace a symbol - my ($type, $old, $new, $reftype); - - if (m/^replace\s+(\S+)\s+(\S+)\s+(\S+)/) { - $type =3D $1; - $old =3D $2; - $new =3D $3; - } else { - die "Can't parse $file_exceptions: $_"; - } - - if ($new =3D~ m/^\:c\:(data|func|macro|type)\:\`(.+)\`/) { - $reftype =3D ":c:$1"; - $new =3D $2; - } elsif ($new =3D~ m/\:ref\:\`(.+)\`/) { - $reftype =3D ":ref"; - $new =3D $1; - } else { - $reftype =3D $def_reftype{$type}; - } - if (!$reftype) { - print STDERR "Warning: can't find ref type for $type"; - } - $new =3D "$reftype:`$old <$new>`"; - - if ($type eq "ioctl") { - $ioctls{$old} =3D $new if (exists($ioctls{$old})); - next; - } - if ($type eq "define") { - $defines{$old} =3D $new if (exists($defines{$old})); - next; - } - if ($type eq "symbol") { - $enum_symbols{$old} =3D $new if (exists($enum_symbols{$old})); - next; - } - if ($type eq "typedef") { - $typedefs{$old} =3D $new if (exists($typedefs{$old})); - next; - } - if ($type eq "enum") { - $enums{$old} =3D $new if (exists($enums{$old})); - next; - } - if ($type eq "struct") { - $structs{$old} =3D $new if (exists($structs{$old})); - next; - } - - die "Can't parse $file_exceptions: $_"; - } -} - -if ($debug) { - my @all_hashes =3D ( - {ioctl =3D> \%ioctls}, - {typedef =3D> \%typedefs}, - {enum =3D> \%enums}, - {struct =3D> \%structs}, - {define =3D> \%defines}, - {symbol =3D> \%enum_symbols} - ); - - foreach my $hash (@all_hashes) { - while (my ($name, $hash_ref) =3D each %$hash) { - next unless %$hash_ref; # Skip empty hashes - - print "$name:\n"; - for my $key (sort keys %$hash_ref) { - print " $key -> $hash_ref->{$key}\n"; - } - print "\n"; - } - } -} - -# -# Align block -# -$data =3D expand($data); -$data =3D " " . $data; -$data =3D~ s/\n/\n /g; -$data =3D~ s/\n\s+$/\n/g; -$data =3D~ s/\n\s+\n/\n\n/g; - -# -# Add escape codes for special characters -# -$data =3D~ s,([\_\`\*\<\>\&\\\\:\/\|\%\$\#\{\}\~\^]),\\$1,g; - -$data =3D~ s,DEPRECATED,**DEPRECATED**,g; - -# -# Add references -# - -my $start_delim =3D "[ \n\t\(\=3D\*\@]"; -my $end_delim =3D "(\\s|,|\\\\=3D|\\\\:|\\;|\\\)|\\}|\\{)"; - -foreach my $r (keys %ioctls) { - my $s =3D $ioctls{$r}; - - $r =3D~ s,([\_\`\*\<\>\&\\\\:\/]),\\\\$1,g; - - print "$r -> $s\n" if ($debug); - - $data =3D~ s/($start_delim)($r)$end_delim/$1$s$3/g; -} - -foreach my $r (keys %defines) { - my $s =3D $defines{$r}; - - $r =3D~ s,([\_\`\*\<\>\&\\\\:\/]),\\\\$1,g; - - print "$r -> $s\n" if ($debug); - - $data =3D~ s/($start_delim)($r)$end_delim/$1$s$3/g; -} - -foreach my $r (keys %enum_symbols) { - my $s =3D $enum_symbols{$r}; - - $r =3D~ s,([\_\`\*\<\>\&\\\\:\/]),\\\\$1,g; - - print "$r -> $s\n" if ($debug); - - $data =3D~ s/($start_delim)($r)$end_delim/$1$s$3/g; -} - -foreach my $r (keys %enums) { - my $s =3D $enums{$r}; - - $r =3D~ s,([\_\`\*\<\>\&\\\\:\/]),\\\\$1,g; - - print "$r -> $s\n" if ($debug); - - $data =3D~ s/enum\s+($r)$end_delim/$s$2/g; -} - -foreach my $r (keys %structs) { - my $s =3D $structs{$r}; - - $r =3D~ s,([\_\`\*\<\>\&\\\\:\/]),\\\\$1,g; - - print "$r -> $s\n" if ($debug); - - $data =3D~ s/struct\s+($r)$end_delim/$s$2/g; -} - -foreach my $r (keys %typedefs) { - my $s =3D $typedefs{$r}; - - $r =3D~ s,([\_\`\*\<\>\&\\\\:\/]),\\\\$1,g; - - print "$r -> $s\n" if ($debug); - $data =3D~ s/($start_delim)($r)$end_delim/$1$s$3/g; -} - -$data =3D~ s/\\ ([\n\s])/\1/g; - -# -# Generate output file -# - -my $title =3D $file_in; -$title =3D~ s,.*/,,; - -open OUT, "> $file_out" or die "Can't open $file_out"; -print OUT ".. -*- coding: utf-8; mode: rst -*-\n\n"; -print OUT "$title\n"; -print OUT "=3D" x length($title); -print OUT "\n\n.. parsed-literal::\n\n"; -print OUT $data; -close OUT; - -__END__ - -=3Dhead1 NAME - -parse_headers.pl - parse a C file, in order to identify functions, structs, -enums and defines and create cross-references to a Sphinx book. - -=3Dhead1 SYNOPSIS - -B [] [] - -Where can be: --debug, --help or --usage. - -=3Dhead1 OPTIONS - -=3Dover 8 - -=3Ditem B<--debug> - -Put the script in verbose mode, useful for debugging. - -=3Ditem B<--usage> - -Prints a brief help message and exits. - -=3Ditem B<--help> - -Prints a more detailed help message and exits. - -=3Dback - -=3Dhead1 DESCRIPTION - -Convert a C header or source file (C_FILE), into a ReStructured Text -included via ..parsed-literal block with cross-references for the -documentation files that describe the API. It accepts an optional -EXCEPTIONS_FILE with describes what elements will be either ignored or -be pointed to a non-default reference. - -The output is written at the (OUT_FILE). - -It is capable of identifying defines, functions, structs, typedefs, -enums and enum symbols and create cross-references for all of them. -It is also capable of distinguish #define used for specifying a Linux -ioctl. - -The EXCEPTIONS_FILE contain two rules to allow ignoring a symbol or -to replace the default references by a custom one. - -Please read Documentation/doc-guide/parse-headers.rst at the Kernel's -tree for more details. - -=3Dhead1 BUGS - -Report bugs to Mauro Carvalho Chehab - -=3Dhead1 COPYRIGHT - -Copyright (c) 2016 by Mauro Carvalho Chehab . - -License GPLv2: GNU GPL version 2 . - -This is free software: you are free to change and redistribute it. -There is NO WARRANTY, to the extent permitted by law. - -=3Dcut --=20 2.50.1