From nobody Thu Oct 9 04:12:36 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BFBF23236F; Fri, 20 Jun 2025 08:11:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; cv=none; b=VhXBCxlh8uEozNTks2Q8Nn2Bl2ojI6/dVeYJoer8fpTVI6Ubt0Bw+wCiBUOA+i8+1353eCEVWOqZTBnsQGx1MyIayEBjbECHpvqt+jmZZ+wJoQdSoLXY4Oa/JMpDXlGAGGdYHPsFWzrSx/t7/Jw3K+JKlBDrEs8JRf2Lu7CYWIY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; c=relaxed/simple; bh=JvTI+DF/1Dszyn4T0n7ZKhi6vNtj6FoC1Ae/TMd5FHM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ieOjYaC7VCvdhtGrVGnS6qppY/+KKfWogpx/4oEYWieM+Gx1hbZeP3Rr2iNSImMnX3+jwg2VMe0ubnf1GX2B983+J4/xDnxmd+fIr5EFCk+a47bKduKzN1LY37XqQe5oP9r6eiEU6zAk1xARz6Ku/NRfflygEHrw342NeT8UPEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=R9RBbEPy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="R9RBbEPy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BD1A0C4CEF0; Fri, 20 Jun 2025 08:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1750407113; bh=JvTI+DF/1Dszyn4T0n7ZKhi6vNtj6FoC1Ae/TMd5FHM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R9RBbEPybIDPIl5sxqWXk0nqMmbTRyx+Fh7+IEbpD2P0UFiQvaByzvjRhvhjo5aiX AvQgoX+g64q+H2ETUi05ul1jL6nxGUlNNg4s0Meqxx+CWwoQ/i0KKJE8aHLUwirvjy ifi4gpzk+lACr7ttf5qMWV1KjmfVYsA9MVPdlbDhqCaNKD0pQO2ZRM2TSbWOWn4FG2 S8ZqZ4H8Zi+yDyAZGoxILDsX1NLOTQVeV4Kx9TN2YkOSKqSv6A41XjBQqXCGqZl3QM eULTtvbfoqtG9xReD0CfhtiFj3kceu/Zz+VHhx3WFqY+N9M9n0tPrCvKrM5xiHM82a WwPbTYf2sCEqg== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1uSWql-00000004TWn-3Lm5; Fri, 20 Jun 2025 10:11:51 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org, Donald Hunter Subject: [PATCH 1/6] docs: conf.py: properly handle include and exclude patterns Date: Fri, 20 Jun 2025 10:11:41 +0200 Message-ID: <737b08e891765dc10bd944d4d42f8b1e37b80275.1750406900.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" When one does: make SPHINXDIRS=3D"foo" htmldocs All patterns would be relative to Documentation/foo, which causes the include/exclude patterns like: include_patterns =3D [ ... f'foo/*.{ext}', ] to break. This is not what it is expected. Address it by adding a logic to dynamically adjust the pattern when SPHINXDIRS is used. That allows adding parsers for other file types. It should be noticed that include_patterns was added on Sphinx 5.1: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-incl= ude_patterns So, a backward-compatible code is needed when we start using it for real. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Donald Hunter --- Documentation/conf.py | 67 ++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 63 insertions(+), 4 deletions(-) diff --git a/Documentation/conf.py b/Documentation/conf.py index 12de52a2b17e..4ba4ee45e599 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -17,6 +17,66 @@ import os import sphinx import shutil =20 +# Get Sphinx version +major, minor, patch =3D sphinx.version_info[:3] + +# Include_patterns were added on Sphinx 5.1 +if (major < 5) or (major =3D=3D 5 and minor < 1): + has_include_patterns =3D False +else: + has_include_patterns =3D True + # Include patterns that don't contain directory names, in glob format + include_patterns =3D ['**.rst'] + +# Location of Documentation/ directory +doctree =3D os.path.abspath('.') + +# Exclude of patterns that don't contain directory names, in glob format. +exclude_patterns =3D [] + +# List of patterns that contain directory names in glob format. +dyn_include_patterns =3D [] +dyn_exclude_patterns =3D ['output'] + +# Properly handle include/exclude patterns +# ---------------------------------------- + +def update_patterns(app): + + """ + On Sphinx, all directories are relative to what it is passed as + SOURCEDIR parameter for sphinx-build. Due to that, all patterns + that have directory names on it need to be dynamically set, after + converting them to a relative patch. + + As Sphinx doesn't include any patterns outside SOURCEDIR, we should + exclude relative patterns that start with "../". + """ + + sourcedir =3D app.srcdir # full path to the source directory + builddir =3D os.environ.get("BUILDDIR") + + # setup include_patterns dynamically + if has_include_patterns: + for p in dyn_include_patterns: + full =3D os.path.join(doctree, p) + + rel_path =3D os.path.relpath(full, start =3D app.srcdir) + if rel_path.startswith("../"): + continue + + app.config.include_patterns.append(rel_path) + + # setup exclude_patterns dynamically + for p in dyn_exclude_patterns: + full =3D os.path.join(doctree, p) + + rel_path =3D os.path.relpath(full, start =3D app.srcdir) + if rel_path.startswith("../"): + continue + + app.config.exclude_patterns.append(rel_path) + # helper # ------ =20 @@ -219,10 +279,6 @@ language =3D 'en' # Else, today_fmt is used as the format for a strftime call. #today_fmt =3D '%B %d, %Y' =20 -# List of patterns, relative to source directory, that match files and -# directories to ignore when looking for source files. -exclude_patterns =3D ['output'] - # The reST default role (used for this markup: `text`) to use for all # documents. #default_role =3D None @@ -516,3 +572,6 @@ kerneldoc_srctree =3D '..' # the last statement in the conf.py file # ------------------------------------------------------------------------= ------ loadConfig(globals()) + +def setup(app): + app.connect('builder-inited', update_patterns) --=20 2.49.0 From nobody Thu Oct 9 04:12:36 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C0D3233155; Fri, 20 Jun 2025 08:11:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; cv=none; b=k1vMfgXubEGVaN+x+PC3oYJWAfgqGVQ4b76W0aFJEtY7QkK/sMgnepxAQYUgGU+RxoQzyGJ/bTUKhG8sjeXxfvmxys9uWRZ0tXvAkm6n0ODX17qgk73Z7I7+tA0ncoToGuPP0nfso2U71ugyxWna/cOzGwhe2iL2cipDGaGlsgM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; c=relaxed/simple; bh=9sv3o8O8M0IyOR5wJkoSEusaPecRlQcEL5SoMo0iJ7A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Sp3xFBFPIxN3ysuUVzSkqo5j/3qNmtozaYKf1JEU7Dn/3UZzjlzjP/0LiBot+RWbqZ9mDZ57/ytvybqmhT8C2QoebnUF2uuIdWXL46QayljJyA0Rk6L6DTsqlFOfA1tW4suZlPE28pygK3szrmErxPEP7lnvPVD1VSgvcdStRIM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aCHDGw9c; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aCHDGw9c" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C98F5C4CEF4; Fri, 20 Jun 2025 08:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1750407113; bh=9sv3o8O8M0IyOR5wJkoSEusaPecRlQcEL5SoMo0iJ7A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aCHDGw9cy6UhOjC3zruGzQFc4XAmAHl+VJTOwHDTUCvwDR6mD4oySGu70XhlZuQpF Q61cJvFHmCD+yf4yG6YpioNJu4LlwD+oj8msZXmvFN1c6l+ZxrF8IRZ8DaIepmx0uI bNVcoNEdh0Rn4KFpIUjVmH3CxSrJs+ehA/oOIo/LT+oUgvaHErOv09FzYp+vAMyQTf ucIaZPjuxKXcgGTF2ODls1d8eCla5n7+ivA0l55qwNyWhw/cvXvmutWDHK5ACgXNdj b+G9WRJX6r8M3ZxwWGuiLMPjmZuO4dLBNt/Bo9jo/Bdu+VyJyDliNamVLCiLSAFkFn m+vPnXDOvM81Q== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1uSWql-00000004TWr-3SYV; Fri, 20 Jun 2025 10:11:51 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org, Breno Leitao , Donald Hunter Subject: [PATCH 2/6] docs: Makefile: disable check rules on make cleandocs Date: Fri, 20 Jun 2025 10:11:42 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" It doesn't make sense to check for missing ABI and documents when cleaning the tree. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Breno Leitao Reviewed-by: Donald Hunter --- Documentation/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/Makefile b/Documentation/Makefile index d30d66ddf1ad..b98477df5ddf 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -5,6 +5,7 @@ # for cleaning subdir- :=3D devicetree/bindings =20 +ifneq ($(MAKECMDGOALS),cleandocs) # Check for broken documentation file references ifeq ($(CONFIG_WARN_MISSING_DOCUMENTS),y) $(shell $(srctree)/scripts/documentation-file-ref-check --warn) @@ -14,6 +15,7 @@ endif ifeq ($(CONFIG_WARN_ABI_ERRORS),y) $(shell $(srctree)/scripts/get_abi.py --dir $(srctree)/Documentation/ABI v= alidate) endif +endif =20 # You can set these variables from the command line. SPHINXBUILD =3D sphinx-build --=20 2.49.0 From nobody Thu Oct 9 04:12:36 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C38623371E; Fri, 20 Jun 2025 08:11:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; cv=none; b=l50skHv5fw5bw+V+I2t91IJkFwQ3Bkn7vjJk78BrVM/GbR32uyX2/gX/x7pFvuJB8gfBvJiDmwzfHf1jOt7dGSoI4/GfQanFxnJxg9FbVb/hAr5Pqs1nEPlakLiu9s7xv9RH6w0hz9pp8II0/uFQrQzg79YwmqAhi8hnk2bcDCc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; c=relaxed/simple; bh=wx0BDvaHpPo9dP1wTRMAhz/5i+/h4gNQceHfFlSoK7g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hDJOght8J0JD3RSFe0mbFJbG+y+mX46DB5WMf50bUafO5PWYrrtayQBPm2nq+3iUT1Ip9JMM+Smk11Z/sd1g9FkSgNS+F3ii+Hz01BNb4eQwHvkFLHJwqCwoI1urodxLewwibbibLDP9pEqVB7m0U1H9hkvMV011Ff/dTsswZnY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=o7y94HaH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="o7y94HaH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C792DC4AF09; Fri, 20 Jun 2025 08:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1750407114; bh=wx0BDvaHpPo9dP1wTRMAhz/5i+/h4gNQceHfFlSoK7g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=o7y94HaHSE67lSEPEmuqgyCglT7T6bKhAla/OLEqFQkcisrE6dNuZxo4z1UDOCJcR /ipKl4JCApwsr6TWAyyRZBZsqY9W6+n+sDKu/g3SdzaX7VTiAjQO2hVEtzubjO4gPZ RluTjkE1W88llGp3iWDpQPKz7g8Qe8e2AL+0KsDFSZtKORzjK4Y2WFRYUdaf6WP7Vc 1Z0dKMgPCzxyuFz/tb5PHT7qExcHnUVNcc+4PohotqMZopVckKLNwLx1rHI7AASMc8 ba3VnPheZ+/ShiCyKbvcksBsMvBjG8ShYYW/8utfyChyZzpV4XXs7rLl46w76pMZFd Ws92II0KtnzaQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1uSWql-00000004TWv-3ZJx; Fri, 20 Jun 2025 10:11:51 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 3/6] scripts: scripts/test_doc_build.py: add script to test doc build Date: Fri, 20 Jun 2025 10:11:43 +0200 Message-ID: <1858fa581a86a1c684cf1c6044be98d482022251.1750406900.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Testing Sphinx backward-compatibility is hard, as per version minimal Python dependency requirements can be a nightmare. Add a script to help automate such checks. Signed-off-by: Mauro Carvalho Chehab --- scripts/test_doc_build.py | 241 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 241 insertions(+) create mode 100755 scripts/test_doc_build.py diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py new file mode 100755 index 000000000000..482716fbe91d --- /dev/null +++ b/scripts/test_doc_build.py @@ -0,0 +1,241 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# Copyright(c) 2025: Mauro Carvalho Chehab +# +# pylint: disable=3DC0103,R1715 + +""" +Install minimal supported requirements for different Sphinx versions +and optionally test the build. +""" + +import argparse +import os.path +import sys +import time + +from subprocess import run + +# Minimal python version supported by the building system +python_bin =3D "python3.9" + +# Starting from 8.0.2, Python 3.9 becomes too old +python_changes =3D {(8, 0, 2): "python3"} + +# Sphinx versions to be installed and their incremental requirements +sphinx_requirements =3D { + (3, 4, 3): { + "alabaster": "0.7.13", + "babel": "2.17.0", + "certifi": "2025.6.15", + "charset-normalizer": "3.4.2", + "docutils": "0.15", + "idna": "3.10", + "imagesize": "1.4.1", + "Jinja2": "3.0.3", + "MarkupSafe": "2.0", + "packaging": "25.0", + "Pygments": "2.19.1", + "PyYAML": "5.1", + "requests": "2.32.4", + "snowballstemmer": "3.0.1", + "sphinxcontrib-applehelp": "1.0.4", + "sphinxcontrib-devhelp": "1.0.2", + "sphinxcontrib-htmlhelp": "2.0.1", + "sphinxcontrib-jsmath": "1.0.1", + "sphinxcontrib-qthelp": "1.0.3", + "sphinxcontrib-serializinghtml": "1.1.5", + "urllib3": "2.4.0", + }, + (3, 5, 4): {}, + (4, 0, 3): { + "docutils": "0.17.1", + "PyYAML": "5.1", + }, + (4, 1, 2): {}, + (4, 3, 2): {}, + (4, 4, 0): {}, + (4, 5, 0): {}, + (5, 0, 2): {}, + (5, 1, 1): {}, + (5, 2, 3): { + "Jinja2": "3.1.2", + "MarkupSafe": "2.0", + "PyYAML": "5.3.1", + }, + (5, 3, 0): { + "docutils": "0.18.1", + "PyYAML": "5.3.1", + }, + (6, 0, 1): {}, + (6, 1, 3): {}, + (6, 2, 1): { + "PyYAML": "5.4.1", + }, + (7, 0, 1): {}, + (7, 1, 2): {}, + (7, 2, 3): { + "PyYAML": "6.0.1", + "sphinxcontrib-serializinghtml": "1.1.9", + }, + (7, 3, 7): { + "alabaster": "0.7.14", + "PyYAML": "6.0.1", + }, + (7, 4, 7): { + "docutils": "0.20", + "PyYAML": "6.0.1", + }, + (8, 0, 2): {}, + (8, 1, 3): { + "PyYAML": "6.0.1", + "sphinxcontrib-applehelp": "1.0.7", + "sphinxcontrib-devhelp": "1.0.6", + "sphinxcontrib-htmlhelp": "2.0.6", + "sphinxcontrib-qthelp": "1.0.6", + }, + (8, 2, 3): { + "PyYAML": "6.0.1", + "sphinxcontrib-serializinghtml": "1.1.9", + }, +} + + +def parse_version(ver_str): + """Convert a version string into a tuple.""" + + return tuple(map(int, ver_str.split("."))) + + +parser =3D argparse.ArgumentParser(description=3D"Build docs for different= sphinx_versions.") + +parser.add_argument('-v', '--version', help=3D'Sphinx single version', + type=3Dparse_version) +parser.add_argument('--min-version', "--min", help=3D'Sphinx minimal versi= on', + type=3Dparse_version) +parser.add_argument('--max-version', "--max", help=3D'Sphinx maximum versi= on', + type=3Dparse_version) +parser.add_argument('-a', '--make_args', + help=3D'extra arguments for make htmldocs, like SPHINX= DIRS=3Dnetlink/specs', + nargs=3D"*") +parser.add_argument('-w', '--write', help=3D'write a requirements.txt file= ', + action=3D'store_true') +parser.add_argument('-m', '--make', + help=3D'Make documentation', + action=3D'store_true') +parser.add_argument('-i', '--wait-input', + help=3D'Wait for an enter before going to the next ver= sion', + action=3D'store_true') + +args =3D parser.parse_args() + +if not args.make_args: + args.make_args =3D [] + +if args.version: + if args.min_version or args.max_version: + sys.exit("Use either --version or --min-version/--max-version") + else: + args.min_version =3D args.version + args.max_version =3D args.version + +sphinx_versions =3D sorted(list(sphinx_requirements.keys())) + +if not args.min_version: + args.min_version =3D sphinx_versions[0] + +if not args.max_version: + args.max_version =3D sphinx_versions[-1] + +first_run =3D True +cur_requirements =3D {} +built_time =3D {} + +for cur_ver, new_reqs in sphinx_requirements.items(): + cur_requirements.update(new_reqs) + + if cur_ver in python_changes: + python_bin =3D python_changes[cur_ver] + + ver =3D ".".join(map(str, cur_ver)) + + if args.min_version: + if cur_ver < args.min_version: + continue + + if args.max_version: + if cur_ver > args.max_version: + break + + if not first_run and args.wait_input and args.make: + ret =3D input("Press Enter to continue or 'a' to abort: ").strip()= .lower() + if ret =3D=3D "a": + print("Aborted.") + sys.exit() + else: + first_run =3D False + + venv_dir =3D f"Sphinx_{ver}" + req_file =3D f"requirements_{ver}.txt" + + print(f"\nSphinx {ver} with {python_bin}") + + # Create venv + run([python_bin, "-m", "venv", venv_dir], check=3DTrue) + pip =3D os.path.join(venv_dir, "bin/pip") + + # Create install list + reqs =3D [] + for pkg, verstr in cur_requirements.items(): + reqs.append(f"{pkg}=3D=3D{verstr}") + + reqs.append(f"Sphinx=3D=3D{ver}") + + run([pip, "install"] + reqs, check=3DTrue) + + # Freeze environment + result =3D run([pip, "freeze"], capture_output=3DTrue, text=3DTrue, ch= eck=3DTrue) + + # Pip install succeeded. Write requirements file + if args.write: + with open(req_file, "w", encoding=3D"utf-8") as fp: + fp.write(result.stdout) + + if args.make: + start_time =3D time.time() + + # Prepare a venv environment + env =3D os.environ.copy() + bin_dir =3D os.path.join(venv_dir, "bin") + env["PATH"] =3D bin_dir + ":" + env["PATH"] + env["VIRTUAL_ENV"] =3D venv_dir + if "PYTHONHOME" in env: + del env["PYTHONHOME"] + + # Test doc build + run(["make", "cleandocs"], env=3Denv, check=3DTrue) + make =3D ["make"] + args.make_args + ["htmldocs"] + + print(f". {bin_dir}/activate") + print(" ".join(make)) + print("deactivate") + run(make, env=3Denv, check=3DTrue) + + end_time =3D time.time() + elapsed_time =3D end_time - start_time + hours, minutes =3D divmod(elapsed_time, 3600) + minutes, seconds =3D divmod(minutes, 60) + + hours =3D int(hours) + minutes =3D int(minutes) + seconds =3D int(seconds) + + built_time[ver] =3D f"{hours:02d}:{minutes:02d}:{seconds:02d}" + + print(f"Finished doc build for Sphinx {ver}. Elapsed time: {built_= time[ver]}") + +if args.make: + print() + print("Summary:") + for ver, elapsed_time in sorted(built_time.items()): + print(f"\tSphinx {ver} elapsed time: {elapsed_time}") --=20 2.49.0 From nobody Thu Oct 9 04:12:36 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5716B23372E; Fri, 20 Jun 2025 08:11:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; cv=none; b=YumXZn6rtG/5Ky9VO9a3WlckWs/HhOre3poaf4Hrh9nEEjlTvFvqi4ukPmG+7km6DvPxuVVXK/FWURtXMMwgEHpoTqpbrlmIDPOKhFp3dGqceyxDz9n0f1hOHYIV+u7yKJC11JIrEuNPqv133J7YxzhEWIxmvbYnAXRswUbtZ+I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; c=relaxed/simple; bh=XT+7KR3Tg4iAhlUU3c6mKrOGy1MIPvbKyEPBCyZpAmw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kdodHir61/LYfcls6eRavHt7TmTa5v18xiWTq5bjYF1uWtK7AW4ZDQ99JUL0S3ySxPybYUhjSgAT3RfJNC9wmQZybaWmTgRbXQnZf6hnKE3pXRJSW70g2SzVf0/2ni6JTHDwAsWA0Y0glzVXpDQ1oku4MjGC/gkuyfJvAeKviZY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RT2Bq3n5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RT2Bq3n5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C2F80C4CEF5; Fri, 20 Jun 2025 08:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1750407114; bh=XT+7KR3Tg4iAhlUU3c6mKrOGy1MIPvbKyEPBCyZpAmw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RT2Bq3n5H2XOU69Jn3UhOPJSdWL88/4TC+yf8M9PnNCRpcTxIh4QQ4lYT6N/jk0rV epkKT0ISnQ0DDs7TLuVezdvvwNrAfWEdAH4Gpn+pzsbP0QQksy6YIqRJq8TRouZg7M s8+dXzwAKuA7x94h3DlcflUt6zFLrMlqp220t6FGnfiZIsyO9sPfEuSkPYCx9T6ru5 R1O61P22PlPsXNBk+d5Vwk2KRwiS5jsuPCHF7K6aj1eZC56zSVDx+QMUOl3POBkfiV mVPd8aLkbwJzHweWKLURwhb4KKm0vIKjKm6TJoNaDQMP3Jxs7w8DgBSXPuEG71J9C/ xVuKFH5INHXVg== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1uSWql-00000004TWz-3gE9; Fri, 20 Jun 2025 10:11:51 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 4/6] scripts/test_doc_build.py: make capture assynchronous Date: Fri, 20 Jun 2025 10:11:44 +0200 Message-ID: <55f4ce45bc0b48c5d1a2fcac2e92f6cdc3d727d6.1750406900.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Prepare the tool to allow writing the output into log files. For such purpose, receive stdin/stdout messages asynchronously. Signed-off-by: Mauro Carvalho Chehab --- scripts/test_doc_build.py | 393 +++++++++++++++++++++++++------------- 1 file changed, 255 insertions(+), 138 deletions(-) diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py index 482716fbe91d..94f2f2d8c3b7 100755 --- a/scripts/test_doc_build.py +++ b/scripts/test_doc_build.py @@ -2,7 +2,7 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright(c) 2025: Mauro Carvalho Chehab # -# pylint: disable=3DC0103,R1715 +# pylint: disable=3DR0903,R0913,R0914,R0917 =20 """ Install minimal supported requirements for different Sphinx versions @@ -10,20 +10,20 @@ and optionally test the build. """ =20 import argparse +import asyncio import os.path import sys import time - -from subprocess import run +import subprocess =20 # Minimal python version supported by the building system -python_bin =3D "python3.9" +MINIMAL_PYTHON_VERSION =3D "python3.9" =20 # Starting from 8.0.2, Python 3.9 becomes too old -python_changes =3D {(8, 0, 2): "python3"} +PYTHON_VER_CHANGES =3D {(8, 0, 2): "python3"} =20 # Sphinx versions to be installed and their incremental requirements -sphinx_requirements =3D { +SPHINX_REQUIREMENTS =3D { (3, 4, 3): { "alabaster": "0.7.13", "babel": "2.17.0", @@ -101,141 +101,258 @@ sphinx_requirements =3D { } =20 =20 +class AsyncCommands: + """Excecute command synchronously""" + + stdout =3D None + stderr =3D None + output =3D None + + async def _read(self, stream, verbose, is_info): + """Ancillary routine to capture while displaying""" + + while stream is not None: + line =3D await stream.readline() + if line: + out =3D line.decode("utf-8", errors=3D"backslashreplace") + self.output +=3D out + if is_info: + if verbose: + print(out.rstrip("\n")) + + self.stdout +=3D out + else: + if verbose: + print(out.rstrip("\n"), file=3Dsys.stderr) + + self.stderr +=3D out + else: + break + + async def run(self, cmd, capture_output=3DFalse, check=3DFalse, + env=3DNone, verbose=3DTrue): + + """ + Execute an arbitrary command, handling errors. + + Please notice that this class is not thread safe + """ + + self.stdout =3D "" + self.stderr =3D "" + self.output =3D "" + + if verbose: + print("$ ", " ".join(cmd)) + + proc =3D await asyncio.create_subprocess_exec(cmd[0], + *cmd[1:], + env=3Denv, + stdout=3Dasyncio.subpr= ocess.PIPE, + stderr=3Dasyncio.subpr= ocess.PIPE) + + # Handle input and output in realtime + await asyncio.gather( + self._read(proc.stdout, verbose, True), + self._read(proc.stderr, verbose, False), + ) + + await proc.wait() + + if check and proc.returncode > 0: + raise subprocess.CalledProcessError(returncode=3Dproc.returnco= de, + cmd=3D" ".join(cmd), + output=3Dself.stdout, + stderr=3Dself.stderr) + + if capture_output: + if proc.returncode > 0: + print("Error {proc.returncode}", file=3Dsys.stderr) + return "" + + return self.output + + ret =3D subprocess.CompletedProcess(args=3Dcmd, + returncode=3Dproc.returncode, + stdout=3Dself.stdout, + stderr=3Dself.stderr) + + return ret + + +class SphinxVenv: + """ + Installs Sphinx on one virtual env per Sphinx version with a minimal + set of dependencies, adjusting them to each specific version. + """ + + def __init__(self): + """Initialize instance variables""" + + self.built_time =3D {} + self.first_run =3D True + + async def _handle_version(self, args, cur_ver, cur_requirements, pytho= n_bin): + """Handle a single Sphinx version""" + + cmd =3D AsyncCommands() + + ver =3D ".".join(map(str, cur_ver)) + + if not self.first_run and args.wait_input and args.make: + ret =3D input("Press Enter to continue or 'a' to abort: ").str= ip().lower() + if ret =3D=3D "a": + print("Aborted.") + sys.exit() + else: + self.first_run =3D False + + venv_dir =3D f"Sphinx_{ver}" + req_file =3D f"requirements_{ver}.txt" + + print(f"\nSphinx {ver} with {python_bin}") + + # Create venv + await cmd.run([python_bin, "-m", "venv", venv_dir], check=3DTrue) + pip =3D os.path.join(venv_dir, "bin/pip") + + # Create install list + reqs =3D [] + for pkg, verstr in cur_requirements.items(): + reqs.append(f"{pkg}=3D=3D{verstr}") + + reqs.append(f"Sphinx=3D=3D{ver}") + + await cmd.run([pip, "install"] + reqs, check=3DTrue, verbose=3DTru= e) + + # Freeze environment + result =3D await cmd.run([pip, "freeze"], verbose=3DFalse, check= =3DTrue) + + # Pip install succeeded. Write requirements file + if args.write: + with open(req_file, "w", encoding=3D"utf-8") as fp: + fp.write(result.stdout) + + if args.make: + start_time =3D time.time() + + # Prepare a venv environment + env =3D os.environ.copy() + bin_dir =3D os.path.join(venv_dir, "bin") + env["PATH"] =3D bin_dir + ":" + env["PATH"] + env["VIRTUAL_ENV"] =3D venv_dir + if "PYTHONHOME" in env: + del env["PYTHONHOME"] + + # Test doc build + await cmd.run(["make", "cleandocs"], env=3Denv, check=3DTrue) + make =3D ["make"] + args.make_args + ["htmldocs"] + + print(f". {bin_dir}/activate") + print(" ".join(make)) + print("deactivate") + await cmd.run(make, env=3Denv, check=3DTrue) + + end_time =3D time.time() + elapsed_time =3D end_time - start_time + hours, minutes =3D divmod(elapsed_time, 3600) + minutes, seconds =3D divmod(minutes, 60) + + hours =3D int(hours) + minutes =3D int(minutes) + seconds =3D int(seconds) + + self.built_time[ver] =3D f"{hours:02d}:{minutes:02d}:{seconds:= 02d}" + + print(f"Finished doc build for Sphinx {ver}. Elapsed time: {se= lf.built_time[ver]}") + + async def run(self, args): + """ + Navigate though multiple Sphinx versions, handling each of them + on a loop. + """ + + cur_requirements =3D {} + python_bin =3D MINIMAL_PYTHON_VERSION + + for cur_ver, new_reqs in SPHINX_REQUIREMENTS.items(): + cur_requirements.update(new_reqs) + + if cur_ver in PYTHON_VER_CHANGES: # pylint: disable= =3DR1715 + + python_bin =3D PYTHON_VER_CHANGES[cur_ver] + + if args.min_version: + if cur_ver < args.min_version: + continue + + if args.max_version: + if cur_ver > args.max_version: + break + + await self._handle_version(args, cur_ver, cur_requirements, + python_bin) + + if args.make: + print() + print("Summary:") + for ver, elapsed_time in sorted(self.built_time.items()): + print(f"\tSphinx {ver} elapsed time: {elapsed_time}") + + def parse_version(ver_str): """Convert a version string into a tuple.""" =20 return tuple(map(int, ver_str.split("."))) =20 =20 -parser =3D argparse.ArgumentParser(description=3D"Build docs for different= sphinx_versions.") - -parser.add_argument('-v', '--version', help=3D'Sphinx single version', - type=3Dparse_version) -parser.add_argument('--min-version', "--min", help=3D'Sphinx minimal versi= on', - type=3Dparse_version) -parser.add_argument('--max-version', "--max", help=3D'Sphinx maximum versi= on', - type=3Dparse_version) -parser.add_argument('-a', '--make_args', - help=3D'extra arguments for make htmldocs, like SPHINX= DIRS=3Dnetlink/specs', - nargs=3D"*") -parser.add_argument('-w', '--write', help=3D'write a requirements.txt file= ', - action=3D'store_true') -parser.add_argument('-m', '--make', - help=3D'Make documentation', - action=3D'store_true') -parser.add_argument('-i', '--wait-input', - help=3D'Wait for an enter before going to the next ver= sion', - action=3D'store_true') - -args =3D parser.parse_args() - -if not args.make_args: - args.make_args =3D [] - -if args.version: - if args.min_version or args.max_version: - sys.exit("Use either --version or --min-version/--max-version") - else: - args.min_version =3D args.version - args.max_version =3D args.version - -sphinx_versions =3D sorted(list(sphinx_requirements.keys())) - -if not args.min_version: - args.min_version =3D sphinx_versions[0] - -if not args.max_version: - args.max_version =3D sphinx_versions[-1] - -first_run =3D True -cur_requirements =3D {} -built_time =3D {} - -for cur_ver, new_reqs in sphinx_requirements.items(): - cur_requirements.update(new_reqs) - - if cur_ver in python_changes: - python_bin =3D python_changes[cur_ver] - - ver =3D ".".join(map(str, cur_ver)) - - if args.min_version: - if cur_ver < args.min_version: - continue - - if args.max_version: - if cur_ver > args.max_version: - break - - if not first_run and args.wait_input and args.make: - ret =3D input("Press Enter to continue or 'a' to abort: ").strip()= .lower() - if ret =3D=3D "a": - print("Aborted.") - sys.exit() - else: - first_run =3D False - - venv_dir =3D f"Sphinx_{ver}" - req_file =3D f"requirements_{ver}.txt" - - print(f"\nSphinx {ver} with {python_bin}") - - # Create venv - run([python_bin, "-m", "venv", venv_dir], check=3DTrue) - pip =3D os.path.join(venv_dir, "bin/pip") - - # Create install list - reqs =3D [] - for pkg, verstr in cur_requirements.items(): - reqs.append(f"{pkg}=3D=3D{verstr}") - - reqs.append(f"Sphinx=3D=3D{ver}") - - run([pip, "install"] + reqs, check=3DTrue) - - # Freeze environment - result =3D run([pip, "freeze"], capture_output=3DTrue, text=3DTrue, ch= eck=3DTrue) - - # Pip install succeeded. Write requirements file - if args.write: - with open(req_file, "w", encoding=3D"utf-8") as fp: - fp.write(result.stdout) - - if args.make: - start_time =3D time.time() - - # Prepare a venv environment - env =3D os.environ.copy() - bin_dir =3D os.path.join(venv_dir, "bin") - env["PATH"] =3D bin_dir + ":" + env["PATH"] - env["VIRTUAL_ENV"] =3D venv_dir - if "PYTHONHOME" in env: - del env["PYTHONHOME"] - - # Test doc build - run(["make", "cleandocs"], env=3Denv, check=3DTrue) - make =3D ["make"] + args.make_args + ["htmldocs"] - - print(f". {bin_dir}/activate") - print(" ".join(make)) - print("deactivate") - run(make, env=3Denv, check=3DTrue) - - end_time =3D time.time() - elapsed_time =3D end_time - start_time - hours, minutes =3D divmod(elapsed_time, 3600) - minutes, seconds =3D divmod(minutes, 60) - - hours =3D int(hours) - minutes =3D int(minutes) - seconds =3D int(seconds) - - built_time[ver] =3D f"{hours:02d}:{minutes:02d}:{seconds:02d}" - - print(f"Finished doc build for Sphinx {ver}. Elapsed time: {built_= time[ver]}") - -if args.make: - print() - print("Summary:") - for ver, elapsed_time in sorted(built_time.items()): - print(f"\tSphinx {ver} elapsed time: {elapsed_time}") +async def main(): + """Main program""" + + parser =3D argparse.ArgumentParser(description=3D"Build docs for diffe= rent sphinx_versions.") + + parser.add_argument('-v', '--version', help=3D'Sphinx single version', + type=3Dparse_version) + parser.add_argument('--min-version', "--min", help=3D'Sphinx minimal v= ersion', + type=3Dparse_version) + parser.add_argument('--max-version', "--max", help=3D'Sphinx maximum v= ersion', + type=3Dparse_version) + parser.add_argument('-a', '--make_args', + help=3D'extra arguments for make htmldocs, like SP= HINXDIRS=3Dnetlink/specs', + nargs=3D"*") + parser.add_argument('-w', '--write', help=3D'write a requirements.txt = file', + action=3D'store_true') + parser.add_argument('-m', '--make', + help=3D'Make documentation', + action=3D'store_true') + parser.add_argument('-i', '--wait-input', + help=3D'Wait for an enter before going to the next= version', + action=3D'store_true') + + args =3D parser.parse_args() + + if not args.make_args: + args.make_args =3D [] + + if args.version: + if args.min_version or args.max_version: + sys.exit("Use either --version or --min-version/--max-version") + else: + args.min_version =3D args.version + args.max_version =3D args.version + + sphinx_versions =3D sorted(list(SPHINX_REQUIREMENTS.keys())) + + if not args.min_version: + args.min_version =3D sphinx_versions[0] + + if not args.max_version: + args.max_version =3D sphinx_versions[-1] + + venv =3D SphinxVenv() + await venv.run(args) + + +# Call main method +if __name__ =3D=3D "__main__": + asyncio.run(main()) --=20 2.49.0 From nobody Thu Oct 9 04:12:36 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9362423504D; Fri, 20 Jun 2025 08:11:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; cv=none; b=g+pglMCsEbLtyn9r09mAAEUJB8qVQf+6vOEUQ8BmRZGUiURo2zKMbBgosuW0tAuh2IQcKQlndlK1S7m9XzD7YRYZIZO62Zapz+E+hhwn/lT3h88hDcDALXezp3Ilhl4oCIrNpsrMsD35LcYj+GdZfnzUbsFzSiJ1beXs1vSIQ5s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; c=relaxed/simple; bh=t6WKKzUZ3jaz/IWeMR8qJbIf2p6B9ADIoUTLRFgr0u0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pN3Ftav/CWwsEHSTtori3z2JWodmt5UOVXQD7iWZh/a7sxeMRXh4ZCwAiQS3FhWEzf5uo+sysdqPKVnoGLnJfZQReH2SetxNr93Z5ZsXLQ0vVSpijKoSg/KPre3pOuAJhCKMkg/onjXJDpYhDaE9HYqrv2QJ+3LpXkO/4kLV11Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bv8JAdSh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bv8JAdSh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CC25EC4CEF8; Fri, 20 Jun 2025 08:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1750407114; bh=t6WKKzUZ3jaz/IWeMR8qJbIf2p6B9ADIoUTLRFgr0u0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bv8JAdShpRMIbnveqo5nklz8ukjqn7HR+jMc8Fdu2t7EtPnd4f1BmWGViH3LXIrn/ 3NUjpP8NckKDlrPxt1cZ5BP/6KYneUBzkPnId4oOPtb4Id5dTzDdic655Cu4F2GlOa Dj75A01yeRzdFJg4+L9hCK5UWg87WPu0GN1v+1zhhAoXgkVZbFSvKubTuiE2MU00Fj 7KCIP9HYyRtAreUqfL7KBwGL3P4UkwW2GNRNFBzWQHvQQjhYK/ZxIcIJBwxsiK7zX9 gyIPikBgBMwfyfXHGggnrqrdNXZMcNZxp3uZmJTnIDsQodh9RF6FSmY3jh0pcVoK2u 9HwGL7brI6csg== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1uSWql-00000004TX3-3mwU; Fri, 20 Jun 2025 10:11:51 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 5/6] scripts: test_doc_build.py: better control its output Date: Fri, 20 Jun 2025 10:11:45 +0200 Message-ID: <5bdec8c0811ffb66470146b45e70a4edafbc52ad.1750406900.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Now that asyncio is supported, allow userspace to adjust verbosity level and direct the script output to a file. Signed-off-by: Mauro Carvalho Chehab --- scripts/test_doc_build.py | 78 +++++++++++++++++++++++++-------------- 1 file changed, 51 insertions(+), 27 deletions(-) diff --git a/scripts/test_doc_build.py b/scripts/test_doc_build.py index 94f2f2d8c3b7..5b9eb2c0bf01 100755 --- a/scripts/test_doc_build.py +++ b/scripts/test_doc_build.py @@ -2,7 +2,7 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright(c) 2025: Mauro Carvalho Chehab # -# pylint: disable=3DR0903,R0913,R0914,R0917 +# pylint: disable=3DR0903,R0912,R0913,R0914,R0917,C0301 =20 """ Install minimal supported requirements for different Sphinx versions @@ -104,9 +104,22 @@ SPHINX_REQUIREMENTS =3D { class AsyncCommands: """Excecute command synchronously""" =20 - stdout =3D None - stderr =3D None - output =3D None + def __init__(self, fp=3DNone): + + self.stdout =3D None + self.stderr =3D None + self.output =3D None + self.fp =3D fp + + def log(self, out, verbose, is_info=3DTrue): + if verbose: + if is_info: + print(out.rstrip("\n")) + else: + print(out.rstrip("\n"), file=3Dsys.stderr) + + if self.fp: + self.fp.write(out.rstrip("\n") + "\n") =20 async def _read(self, stream, verbose, is_info): """Ancillary routine to capture while displaying""" @@ -115,16 +128,10 @@ class AsyncCommands: line =3D await stream.readline() if line: out =3D line.decode("utf-8", errors=3D"backslashreplace") - self.output +=3D out + self.log(out, verbose, is_info) if is_info: - if verbose: - print(out.rstrip("\n")) - self.stdout +=3D out else: - if verbose: - print(out.rstrip("\n"), file=3Dsys.stderr) - self.stderr +=3D out else: break @@ -140,10 +147,8 @@ class AsyncCommands: =20 self.stdout =3D "" self.stderr =3D "" - self.output =3D "" =20 - if verbose: - print("$ ", " ".join(cmd)) + self.log("$ " + " ".join(cmd), verbose) =20 proc =3D await asyncio.create_subprocess_exec(cmd[0], *cmd[1:], @@ -167,7 +172,7 @@ class AsyncCommands: =20 if capture_output: if proc.returncode > 0: - print("Error {proc.returncode}", file=3Dsys.stderr) + self.log(f"Error {proc.returncode}", verbose=3DTrue, is_in= fo=3DFalse) return "" =20 return self.output @@ -192,10 +197,11 @@ class SphinxVenv: self.built_time =3D {} self.first_run =3D True =20 - async def _handle_version(self, args, cur_ver, cur_requirements, pytho= n_bin): + async def _handle_version(self, args, fp, + cur_ver, cur_requirements, python_bin): """Handle a single Sphinx version""" =20 - cmd =3D AsyncCommands() + cmd =3D AsyncCommands(fp) =20 ver =3D ".".join(map(str, cur_ver)) =20 @@ -210,10 +216,11 @@ class SphinxVenv: venv_dir =3D f"Sphinx_{ver}" req_file =3D f"requirements_{ver}.txt" =20 - print(f"\nSphinx {ver} with {python_bin}") + cmd.log(f"\nSphinx {ver} with {python_bin}", verbose=3DTrue) =20 # Create venv - await cmd.run([python_bin, "-m", "venv", venv_dir], check=3DTrue) + await cmd.run([python_bin, "-m", "venv", venv_dir], + verbose=3Dargs.verbose, check=3DTrue) pip =3D os.path.join(venv_dir, "bin/pip") =20 # Create install list @@ -223,7 +230,7 @@ class SphinxVenv: =20 reqs.append(f"Sphinx=3D=3D{ver}") =20 - await cmd.run([pip, "install"] + reqs, check=3DTrue, verbose=3DTru= e) + await cmd.run([pip, "install"] + reqs, check=3DTrue, verbose=3Darg= s.verbose) =20 # Freeze environment result =3D await cmd.run([pip, "freeze"], verbose=3DFalse, check= =3DTrue) @@ -248,10 +255,11 @@ class SphinxVenv: await cmd.run(["make", "cleandocs"], env=3Denv, check=3DTrue) make =3D ["make"] + args.make_args + ["htmldocs"] =20 - print(f". {bin_dir}/activate") - print(" ".join(make)) - print("deactivate") - await cmd.run(make, env=3Denv, check=3DTrue) + if args.verbose: + print(f". {bin_dir}/activate") + await cmd.run(make, env=3Denv, check=3DTrue, verbose=3DTrue) + if args.verbose: + print("deactivate") =20 end_time =3D time.time() elapsed_time =3D end_time - start_time @@ -264,7 +272,7 @@ class SphinxVenv: =20 self.built_time[ver] =3D f"{hours:02d}:{minutes:02d}:{seconds:= 02d}" =20 - print(f"Finished doc build for Sphinx {ver}. Elapsed time: {se= lf.built_time[ver]}") + cmd.log(f"Finished doc build for Sphinx {ver}. Elapsed time: {= self.built_time[ver]}", verbose=3DTrue) =20 async def run(self, args): """ @@ -272,6 +280,15 @@ class SphinxVenv: on a loop. """ =20 + if args.log: + fp =3D open(args.log, "w", encoding=3D"utf-8") + if not args.verbose: + args.verbose =3D False + else: + fp =3D None + if not args.verbose: + args.verbose =3D True + cur_requirements =3D {} python_bin =3D MINIMAL_PYTHON_VERSION =20 @@ -290,7 +307,7 @@ class SphinxVenv: if cur_ver > args.max_version: break =20 - await self._handle_version(args, cur_ver, cur_requirements, + await self._handle_version(args, fp, cur_ver, cur_requirements, python_bin) =20 if args.make: @@ -299,6 +316,8 @@ class SphinxVenv: for ver, elapsed_time in sorted(self.built_time.items()): print(f"\tSphinx {ver} elapsed time: {elapsed_time}") =20 + if fp: + fp.close() =20 def parse_version(ver_str): """Convert a version string into a tuple.""" @@ -311,7 +330,7 @@ async def main(): =20 parser =3D argparse.ArgumentParser(description=3D"Build docs for diffe= rent sphinx_versions.") =20 - parser.add_argument('-v', '--version', help=3D'Sphinx single version', + parser.add_argument('-V', '--version', help=3D'Sphinx single version', type=3Dparse_version) parser.add_argument('--min-version', "--min", help=3D'Sphinx minimal v= ersion', type=3Dparse_version) @@ -328,6 +347,11 @@ async def main(): parser.add_argument('-i', '--wait-input', help=3D'Wait for an enter before going to the next= version', action=3D'store_true') + parser.add_argument('-v', '--verbose', + help=3D'Verbose all commands', + action=3D'store_true') + parser.add_argument('-l', '--log', + help=3D'Log command output on a file') =20 args =3D parser.parse_args() =20 --=20 2.49.0 From nobody Thu Oct 9 04:12:36 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C05E233128; Fri, 20 Jun 2025 08:11:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; cv=none; b=amrOhu5Q3hgp+L++bEqWWnu0+jp6PrlzYyjHo9JxWOj3+NCAUvQHVVro5ozGdrqJiZH7SYOm49as/m7Rtl/LDdP5CkC7ztiTs8H1b5Fqfd6L8JD/wH2wxnEu4idANtK+GQCVWeL7DJOfX3/j7KkNfLl6Q3NmS3dw11OoeWNKstE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750407114; c=relaxed/simple; bh=3yI0hd++pTAVk6UlNj1uaAE4va43OAIfWkR3JnQPolk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bs31cMlVAdCwVnQzI9tSRMJumH+pdG6VicXYBB19j11laoKGWXG1E1NBVtMHTYbhrlqoUGwIJYb1qlG6Ul3U1i/ZlCQT0kY++QIeERL2mAmRQfE8zZv4Ff8hQFRBSKm5lMrCLbAG2XYDEIZgi7uk0+Ia3ic5QbH6SH0YJHlUBc4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GhtEtjUs; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GhtEtjUs" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C51A1C4CEF6; Fri, 20 Jun 2025 08:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1750407113; bh=3yI0hd++pTAVk6UlNj1uaAE4va43OAIfWkR3JnQPolk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GhtEtjUssPMeJCzgvWaJTcFljk/sjLv3sGESjb9Aol0m3SWcPlxcthRoivaXHSHp8 oWrFWbVyl/Ji57YmHHEv0aAUsz91X559+PH+7OUS5mfXnmqNM8N5A4DQNrZ7jTYAaU im3oRk2caUP7Ryru+PXUVGQUKMuwF5hUVQ1+XePmMexg68nLIk61jbg3MfJSK4ruuE s/pGR7tSyssqLxdTRP4EamOSwRVylOWpJUP82PY+MY87TWfgM9f1ICM2w3DNE6CJij 5iAWxucqcUSdBdz10I5wbs1v1Qd7JrBrmuJ+PZ2YBE+44//0EERjiDveP5vuNtA0QM Q4wst4q63xbUQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1uSWql-00000004TX7-3tjQ; Fri, 20 Jun 2025 10:11:51 +0200 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Mauro Carvalho Chehab" , Randy Dunlap , linux-kernel@vger.kernel.org Subject: [PATCH 6/6] docs: sphinx: add a file with the requirements for lowest version Date: Fri, 20 Jun 2025 10:11:46 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Those days, it is hard to install a virtual env that would build docs with Sphinx 3.4.3, as even python 3.13 is not compatible anymore with it. /usr/bin/python3.9 -m venv sphinx_3.4.3 . sphinx_3.4.3/bin/activate pip install -r Documentation/sphinx/min_requirements.txt Signed-off-by: Mauro Carvalho Chehab --- Documentation/doc-guide/sphinx.rst | 23 +++++++++++++++++++++++ Documentation/sphinx/min_requirements.txt | 10 ++++++++++ 2 files changed, 33 insertions(+) create mode 100644 Documentation/sphinx/min_requirements.txt diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/s= phinx.rst index 5a91df105141..607589592bfb 100644 --- a/Documentation/doc-guide/sphinx.rst +++ b/Documentation/doc-guide/sphinx.rst @@ -131,6 +131,29 @@ It supports two optional parameters: ``--no-virtualenv`` Use OS packaging for Sphinx instead of Python virtual environment. =20 +Installing Sphinx Minimal Version +--------------------------------- + +When changing Sphinx build system, it is important to ensure that +the minimal version will still be supported. Nowadays, it is +becoming harder to do that on modern distributions, as it is not +possible to install with Python 3.13 and above. + +Testing with the lowest supported Python version as defined at +Documentation/process/changes.rst can be done by creating +a venv with it with, and install minimal requirements with:: + + /usr/bin/python3.9 -m venv sphinx_min + . sphinx_min/bin/activate + pip install -r Documentation/sphinx/min_requirements.txt + +A more comprehensive test can be done by using: + + scripts/test_doc_build.py + +Such script create one Python venv per supported version, +optionally building documentation for a range of Sphinx versions. + =20 Sphinx Build =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/Documentation/sphinx/min_requirements.txt b/Documentation/sphi= nx/min_requirements.txt new file mode 100644 index 000000000000..52d9f27010e8 --- /dev/null +++ b/Documentation/sphinx/min_requirements.txt @@ -0,0 +1,10 @@ +alabaster >=3D0.7,<0.8 +docutils>=3D0.15,<0.18 +jinja2>=3D2.3,<3.1 +PyYAML>=3D5.1,<6.1 +Sphinx=3D=3D3.4.3 +sphinxcontrib-applehelp=3D=3D1.0.2 +sphinxcontrib-devhelp=3D=3D1.0.1 +sphinxcontrib-htmlhelp=3D=3D1.0.3 +sphinxcontrib-qthelp=3D=3D1.0.2 +sphinxcontrib-serializinghtml=3D=3D1.1.4 --=20 2.49.0