From nobody Sat Oct 4 01:41:05 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C46571F91C8; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; cv=none; b=ewwS6oS1NR/GqRCH1jOJsslQhoGyd/8t9YbmEK/0D0QnL4sSOc9B7WiSSxYCY2TO3Sp59se7PymB5xhiqhgEtcN9Btqdl38/bRbWl7nV5pPPu95J+BeE5ypRrBjGGpyAqvAkksis3GoMz+efapHos0cf4LWZEt/ddzKBX4AEyeE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755786099; c=relaxed/simple; bh=lRH9SEB4iDb1LfHInxrSMa8aPsx5ggsWJbqLQRJAoqQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Dnqodbs2GFq6A53YbrKT6yirHt5sruFikzY+gojlT0setTR/PkVmoDYChOXbMtpyuIghAsRtfuxFBwVccoksPiYyvQKUfqq3V3sUkByH7Nr+gzX34jKbdSnUlgsPYn2al3wApBVQTAK3Q+Mg3SxfOaVxeLWU9+heYDG+leNkRLk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LdUJS5bj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LdUJS5bj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1DF21C16AAE; Thu, 21 Aug 2025 14:21:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1755786099; bh=lRH9SEB4iDb1LfHInxrSMa8aPsx5ggsWJbqLQRJAoqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LdUJS5bjPd3m1j90lbHlIC5VKp8nJ32DTzMXAaTtF6EKyzXABJdliCu/gSadIPbKS cOW7oLiG5nziwKncu79UYUY1Ov/r8qc/fMEcFSpOW11xPJJ/LgicTMf4I+LmYlXdh/ 3KekxDWiDk8Iowx6fHA/zO9JHnetiZC2dwD241mDXM4NHPIl8a8UXqIAiCNjDSsgTi KaD0+wO3ygjfOhClkzwMhvMKkW8Yqt2WDQqclJA4qIIib/vpa+sZUq7Ud6NP+FZZ7D VXysuvN6OnQHw0Hb49Ny0wQNYe0u5ZcTIy+8lUXhsQHA6oaUbG9K4YX7zNW3NFfwY0 6FBUfnd77Q6bA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1up6Ab-0000000BT8i-17v9; Thu, 21 Aug 2025 16:21:37 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , Kees Cook , linux-kernel@vger.kernel.org Subject: [PATCH 11/24] docs: kernel_include.py: allow cross-reference generation Date: Thu, 21 Aug 2025 16:21:17 +0200 Message-ID: <1e30edc994a3edba976aafbf182eef2ed5439ede.1755784930.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" kernel_include extension was originally designed to be used by the media comprehensive uAPI documentation, where, instead of simpler kernel-doc markups, the uAPI documentation is enriched with a larger text, with images, complex tables, graphs, etc. There, we wanted to include the much simpler yet documented .h file. This extension is needed to include files from other parts of the Kernel tree outside Documentation, because the original Sphinx include tag doesn't allow going outside of the directory passed via sphinx-build command line. Yet, the cross-references themselves to the full documentation were using a perl script to create cross-references against the comprehensive documentation. As the perl script is now converted to Phython and there is a Python class producing an include-compatible output with cross references, add two optional arguments to kernel_include.py: 1. :generate-cross-refs: If present, instead of reading the file, it calls ParseDataStructs() class, which converts C data structures into cross-references to be linked to ReST files containing a more comprehensive documentati= on; Don't use it together with :start-line: and/or :end-line:, as filtering input file line range is currently not supported. 2. :exception-file: Used together with :generate-cross-refs:. Points to a file containi= ng rules to ignore C data structs or to use a different reference name, optionally using a different reference type. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kernel_include.py | 94 ++++++++++++++++++++------ 1 file changed, 74 insertions(+), 20 deletions(-) diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/= kernel_include.py index 1212786ac516..fc37e6fa9d96 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -25,6 +25,24 @@ Substrings of the form $name or ${name} are replaced by the value of environment variable name. Malformed variable names and references to non-existing variables are left unchanged. + + This extension overrides Sphinx include directory, adding two extra + arguments: + + 1. :generate-cross-refs: + + If present, instead of reading the file, it calls ParseDataStructs= () + class, which converts C data structures into cross-references to + be linked to ReST files containing a more comprehensive documentat= ion; + + Don't use it together with :start-line: and/or :end-line:, as + filtering input file line range is currently not supported. + + 2. :exception-file: + + Used together with :generate-cross-refs:. Points to a file contain= ing + rules to ignore C data structs or to use a different reference nam= e, + optionally using a different reference type. """ =20 # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D @@ -32,6 +50,7 @@ # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D =20 import os.path +import sys =20 from docutils import io, nodes, statemachine from docutils.utils.error_reporting import SafeString, ErrorString @@ -39,6 +58,11 @@ from docutils.parsers.rst import directives from docutils.parsers.rst.directives.body import CodeBlock, NumberLines from docutils.parsers.rst.directives.misc import Include =20 +srctree =3D os.path.abspath(os.environ["srctree"]) +sys.path.insert(0, os.path.join(srctree, "tools/docs/lib")) + +from parse_data_structs import ParseDataStructs + __version__ =3D "1.0" =20 =20 @@ -57,6 +81,14 @@ def setup(app): class KernelInclude(Include): """KernelInclude (``kernel-include``) directive""" =20 + # Add extra options + option_spec =3D Include.option_spec.copy() + + option_spec.update({ + 'generate-cross-refs': directives.flag, + 'exception-file': directives.unchanged, + }) + def run(self): env =3D self.state.document.settings.env path =3D os.path.realpath(os.path.expandvars(self.arguments[0])) @@ -99,28 +131,49 @@ class KernelInclude(Include): e_handler =3D self.state.document.settings.input_encoding_error_ha= ndler tab_width =3D self.options.get("tab-width", self.state.document.settings.tab_widt= h) - try: - self.state.document.settings.record_dependencies.add(path) - include_file =3D io.FileInput(source_path=3Dpath, encoding=3De= ncoding, - error_handler=3De_handler) - except UnicodeEncodeError: - raise self.severe('Problems with "%s" directive path:\n' - 'Cannot encode input file path "%s" ' - "(wrong locale?)." % (self.name, SafeString(= path))) - except IOError as error: - raise self.severe('Problems with "%s" directive path:\n%s.' - % (self.name, ErrorString(error))) startline =3D self.options.get("start-line", None) endline =3D self.options.get("end-line", None) - try: - if startline or (endline is not None): - lines =3D include_file.readlines() - rawtext =3D "".join(lines[startline:endline]) - else: - rawtext =3D include_file.read() - except UnicodeError as error: - raise self.severe('Problem with "%s" directive:\n%s' % - (self.name, ErrorString(error))) + + # Get optional arguments to related to cross-references generation + if 'generate-cross-refs' in self.options: + parser =3D ParseDataStructs() + parser.parse_file(path) + + exceptions_file =3D self.options.get('exception-file') + if exceptions_file: + exceptions_file =3D os.path.join(source_dir, exceptions_fi= le) + parser.process_exceptions(exceptions_file) + + title =3D os.path.basename(path) + rawtext =3D parser.gen_output() + if startline or endline: + raise self.severe('generate-cross-refs can\'t be used toge= ther with "start-line" or "end-line"') + + if "code" not in self.options: + rawtext =3D ".. parsed-literal::\n\n" + rawtext + else: + try: + self.state.document.settings.record_dependencies.add(path) + include_file =3D io.FileInput(source_path=3Dpath, encoding= =3Dencoding, + error_handler=3De_handler) + except UnicodeEncodeError: + raise self.severe('Problems with "%s" directive path:\n' + 'Cannot encode input file path "%s" ' + "(wrong locale?)." % (self.name, SafeStrin= g(path))) + except IOError as error: + raise self.severe('Problems with "%s" directive path:\n%s.' + % (self.name, ErrorString(error))) + + try: + if startline or (endline is not None): + lines =3D include_file.readlines() + rawtext =3D "".join(lines[startline:endline]) + else: + rawtext =3D include_file.read() + except UnicodeError as error: + raise self.severe('Problem with "%s" directive:\n%s' % + (self.name, ErrorString(error))) + # start-after/end-before: no restrictions on newlines in match-tex= t, # and no restrictions on matching inside lines vs. line boundaries after_text =3D self.options.get("start-after", None) @@ -171,6 +224,7 @@ class KernelInclude(Include): else: literal_block +=3D nodes.Text(text, text) return [literal_block] + if "code" in self.options: self.options["source"] =3D path codeblock =3D CodeBlock(self.name, --=20 2.50.1