From nobody Sat Oct 4 00:28:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF8563126B1; Fri, 22 Aug 2025 14:19:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755872388; cv=none; b=Q1WbwRKw9q4dt0JRqD9H2vsqRBB/XtOHG01q9GwsPDFT5AvtnOcbJSKzapKTbCo+gH4lkARKhiAMjL0O1puBu1FfxYTIb1SwY6S85dxZpo/cjF22PEyzYoQheL8e9z+VckeD6JsA/TQPxsbwCmPh5bIqBz3oIZ8XwtI8KrSXM48= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755872388; c=relaxed/simple; bh=lRH9SEB4iDb1LfHInxrSMa8aPsx5ggsWJbqLQRJAoqQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CQ1c0Kig6TsI7xkObiYUUYrRviFuuEGjeTbCNp+Mgs3dpEVhgvA571QQqEd8tuudwgAnc6dpHyBBRX+mlqbSe35qUdL4NsIHjqR1Za23dbYGlSSx84TOXa6yk3OtOlPE1tU+Sh+lu5qetVgFgx2Kv8RoHRM0QeABH1ol1p58lXs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QYYkXOU3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QYYkXOU3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2BF0DC2BC9E; Fri, 22 Aug 2025 14:19:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755872388; bh=lRH9SEB4iDb1LfHInxrSMa8aPsx5ggsWJbqLQRJAoqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QYYkXOU3/ipHgWgtXTfBSbsdEnwC0+UoegWmTxdTXZQOpSYSzrLpdKU4f3lhpusYn lSnbym7pyKO3LtuYqcYYe4XzIud6e8FLAxGIhgq+LRAVhbyzSTBjN4QIYkfs5XVnjC cz5B3MyuMFnZfMDYoOn4MC4GPFmp0RhcgaNmxUKRuYw8BvAh+crlrMLLwQsXx6jhlu f9JaYTDhyJ2ZEItjf8VRSF4uhFfOwIxH7c195ejBIPJQY3zGLXkktFTNnjGgowKP1I yA2Ut026HP8Fxcb8I6irFIj1XSM310Qg20EkG5Rt/Frm/yYs6Ukg0bzP/m7vK0w2K2 0iUtRLNrtt3iQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1upScM-0000000CCrZ-1fRm; Fri, 22 Aug 2025 16:19:46 +0200 From: Mauro Carvalho Chehab To: Jonathan Corbet , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH v2 11/24] docs: kernel_include.py: allow cross-reference generation Date: Fri, 22 Aug 2025 16:19:23 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" kernel_include extension was originally designed to be used by the media comprehensive uAPI documentation, where, instead of simpler kernel-doc markups, the uAPI documentation is enriched with a larger text, with images, complex tables, graphs, etc. There, we wanted to include the much simpler yet documented .h file. This extension is needed to include files from other parts of the Kernel tree outside Documentation, because the original Sphinx include tag doesn't allow going outside of the directory passed via sphinx-build command line. Yet, the cross-references themselves to the full documentation were using a perl script to create cross-references against the comprehensive documentation. As the perl script is now converted to Phython and there is a Python class producing an include-compatible output with cross references, add two optional arguments to kernel_include.py: 1. :generate-cross-refs: If present, instead of reading the file, it calls ParseDataStructs() class, which converts C data structures into cross-references to be linked to ReST files containing a more comprehensive documentati= on; Don't use it together with :start-line: and/or :end-line:, as filtering input file line range is currently not supported. 2. :exception-file: Used together with :generate-cross-refs:. Points to a file containi= ng rules to ignore C data structs or to use a different reference name, optionally using a different reference type. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 94 ++++++++++++++++++++------ 1 file changed, 74 insertions(+), 20 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 1212786ac516..fc37e6fa9d96 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -25,6 +25,24 @@ Substrings of the form $name or ${name} are replaced by the value of environment variable name. Malformed variable names and references to non-existing variables are left unchanged. + + This extension overrides Sphinx include directory, adding two extra + arguments: + + 1. :generate-cross-refs: + + If present, instead of reading the file, it calls ParseDataStructs= () + class, which converts C data structures into cross-references to + be linked to ReST files containing a more comprehensive documentat= ion; + + Don't use it together with :start-line: and/or :end-line:, as + filtering input file line range is currently not supported. + + 2. :exception-file: + + Used together with :generate-cross-refs:. Points to a file contain= ing + rules to ignore C data structs or to use a different reference nam= e, + optionally using a different reference type. """ =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D @@ -32,6 +50,7 @@ # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D =20 import os.path +import sys =20 from docutils import io, nodes, statemachine from docutils.utils.error_reporting import SafeString, ErrorString @@ -39,6 +58,11 @@ from docutils.parsers.rst import directives from docutils.parsers.rst.directives.body import CodeBlock, NumberLines from docutils.parsers.rst.directives.misc import Include =20 +srctree =3D os.path.abspath(os.environ["srctree"]) +sys.path.insert(0, os.path.join(srctree, "tools/docs/lib")) + +from parse_data_structs import ParseDataStructs + __version__ =3D "1.0" =20 =20 @@ -57,6 +81,14 @@ def setup(app): class KernelInclude(Include): """KernelInclude (``kernel-include``) directive""" =20 + # Add extra options + option_spec =3D Include.option_spec.copy() + + option_spec.update({ + 'generate-cross-refs': directives.flag, + 'exception-file': directives.unchanged, + }) + def run(self): env =3D self.state.document.settings.env path =3D os.path.realpath(os.path.expandvars(self.arguments[0])) @@ -99,28 +131,49 @@ class KernelInclude(Include): e_handler =3D self.state.document.settings.input_encoding_error_ha= ndler tab_width =3D self.options.get("tab-width", self.state.document.settings.tab_widt= h) - try: - self.state.document.settings.record_dependencies.add(path) - include_file =3D io.FileInput(source_path=3Dpath, encoding=3De= ncoding, - error_handler=3De_handler) - except UnicodeEncodeError: - raise self.severe('Problems with "%s" directive path:\n' - 'Cannot encode input file path "%s" ' - "(wrong locale?)." % (self.name, SafeString(= path))) - except IOError as error: - raise self.severe('Problems with "%s" directive path:\n%s.' - % (self.name, ErrorString(error))) startline =3D self.options.get("start-line", None) endline =3D self.options.get("end-line", None) - try: - if startline or (endline is not None): - lines =3D include_file.readlines() - rawtext =3D "".join(lines[startline:endline]) - else: - rawtext =3D include_file.read() - except UnicodeError as error: - raise self.severe('Problem with "%s" directive:\n%s' % - (self.name, ErrorString(error))) + + # Get optional arguments to related to cross-references generation + if 'generate-cross-refs' in self.options: + parser =3D ParseDataStructs() + parser.parse_file(path) + + exceptions_file =3D self.options.get('exception-file') + if exceptions_file: + exceptions_file =3D os.path.join(source_dir, exceptions_fi= le) + parser.process_exceptions(exceptions_file) + + title =3D os.path.basename(path) + rawtext =3D parser.gen_output() + if startline or endline: + raise self.severe('generate-cross-refs can\'t be used toge= ther with "start-line" or "end-line"') + + if "code" not in self.options: + rawtext =3D ".. parsed-literal::\n\n" + rawtext + else: + try: + self.state.document.settings.record_dependencies.add(path) + include_file =3D io.FileInput(source_path=3Dpath, encoding= =3Dencoding, + error_handler=3De_handler) + except UnicodeEncodeError: + raise self.severe('Problems with "%s" directive path:\n' + 'Cannot encode input file path "%s" ' + "(wrong locale?)." % (self.name, SafeStrin= g(path))) + except IOError as error: + raise self.severe('Problems with "%s" directive path:\n%s.' + % (self.name, ErrorString(error))) + + try: + if startline or (endline is not None): + lines =3D include_file.readlines() + rawtext =3D "".join(lines[startline:endline]) + else: + rawtext =3D include_file.read() + except UnicodeError as error: + raise self.severe('Problem with "%s" directive:\n%s' % + (self.name, ErrorString(error))) + # start-after/end-before: no restrictions on newlines in match-tex= t, # and no restrictions on matching inside lines vs. line boundaries after_text =3D self.options.get("start-after", None) @@ -171,6 +224,7 @@ class KernelInclude(Include): else: literal_block +=3D nodes.Text(text, text) return [literal_block] + if "code" in self.options: self.options["source"] =3D path codeblock =3D CodeBlock(self.name, --=20 2.50.1