As reported by Akira, there were incompatibility issues with
Sphinx and docutils with docutils 0.19. There's already
a fix for it, but, as there are incompatibility issues with
different versions, better to add a check to verify if the
combination is supported/tested.
After check Sphinx release notes, it seems that the
version compatibility is given by:
======= ============ ============
Sphinx Min Docutils Max Docutils
Version Version Version
------- ------------ ------------
3.4.3 >= 0.12.0 < 0.18.0
4.0.0 >= 0.12.0 < 0.19.0
6.0.0 >= 0.18.0 < 0.20.0
7.0.0 >= 0.18.1 < 0.21.0
7.2.0 >= 0.18.1 < 0.20.0
7.4.0 >= 0.18.1 < 0.21.0
8.0.0 >= 0.20.0 < 0.22.0
8.2.3 >= 0.20.0 < 0.22.0
======= ============ ============
For now, add a logic inside conf.py to check the above
compatibility list, emitting warnings if the docutils
version doesn't match it.
This way, when we discover incompatibility issues, we
can easily adjust the table.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/conf.py | 70 +++++++++++++++++++++++++++++++++++++++++--
1 file changed, 67 insertions(+), 3 deletions(-)
diff --git a/Documentation/conf.py b/Documentation/conf.py
index 700516238d3f..df99a4d96b58 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -9,7 +9,11 @@ import os
import shutil
import sys
+import docutils
import sphinx
+from sphinx.util import logging
+
+logger = logging.getLogger(__name__)
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
@@ -21,11 +25,71 @@ from load_config import loadConfig # pylint: disable=C0413,E0401
# Minimal supported version
needs_sphinx = "3.4.3"
-# Get Sphinx version
-major, minor, patch = sphinx.version_info[:3] # pylint: disable=I1101
+# Get Sphinx and docutils versions
+sphinx_ver = sphinx.version_info[:3] # pylint: disable=I1101
+docutils_ver = docutils.__version_info__[:3]
+
+sphinx_ver_str = ".".join([str(x) for x in sphinx_ver])
+docutils_ver_str = ".".join([str(x) for x in docutils_ver])
+
+# Docutils min/max versions.
+# The dockutils version check logic is done with:
+# ver >= min and ver < max
+SPHINX_DOCUTILS_VERS = {
+ (3, 4, 3): {
+ "min": (0, 12, 0),
+ "max": (0, 18, 0)
+ },
+ (4, 0, 0): {
+ "min": (0, 12, 0),
+ "max": (0, 19, 0)
+ },
+ (6, 0, 0): {
+ "min": (0, 18, 0),
+ "max": (0, 20, 0)
+ },
+ (7, 0, 0): {
+ "min": (0, 18, 1),
+ "max": (0, 21, 0)
+ },
+ (7, 2, 0): {
+ "min": (0, 18, 1),
+ "max": (0, 20, 0)
+ },
+ (7, 4, 0): {
+ "min": (0, 18, 1),
+ "max": (0, 21, 0)
+ },
+ (8, 0, 0): {
+ "min": (0, 20, 0),
+ "max": (0, 22, 0)
+ },
+ (8, 2, 3): {
+ "min": (0, 20, 0),
+ "max": (0, 22, 0)
+ },
+}
+
+du_min = None
+du_max = None
+for sp_ver, doc_vers in SPHINX_DOCUTILS_VERS.items():
+ if sp_ver > sphinx_ver:
+ break
+
+ du_min = doc_vers.get("min")
+ du_max = doc_vers.get("max")
+
+if sphinx_ver > sorted(SPHINX_DOCUTILS_VERS.keys())[-1]:
+ logger.warning(f"Sphinx version {sphinx_ver_str} is too new and not tested. Doc generation may fail")
+elif not du_min or not du_max:
+ logger.warning(f"Sphinx version {sphinx_ver_str} is not tested. Doc generation may fail")
+elif docutils_ver < du_min:
+ logger.warning(f"Docutils {docutils_ver_str} is too old for Sphinx {sphinx_ver_str}. Doc generation may fail")
+elif docutils_ver >= du_max:
+ logger.warning(f"Docutils {docutils_ver_str} could be too new for Sphinx {sphinx_ver_str}. Doc generation may fail")
# Include_patterns were added on Sphinx 5.1
-if (major < 5) or (major == 5 and minor < 1):
+if sphinx_ver < (5, 1, 0):
has_include_patterns = False
else:
has_include_patterns = True
--
2.49.0On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote:
> As reported by Akira, there were incompatibility issues with
> Sphinx and docutils with docutils 0.19. There's already
> a fix for it, but, as there are incompatibility issues with
> different versions, better to add a check to verify if the
> combination is supported/tested.
>
I've been skeptical of adding such checks in conf.py.
What happened the other day was that Jon used a deprecated (and rarely
used) method of docutils which failed to work properly only in
docutils 0.19 (there is no mention of related issues in its and
nearby release notes).
Your integration of parser_yaml extension will raise the minimum
version of docutils to 0.17.1. I think all you will need is just
to check:
docutils < 0.17.1
, and to make a warning regardless of Sphinx versions.
> After check Sphinx release notes, it seems that the
> version compatibility is given by:
>
> ======= ============ ============
> Sphinx Min Docutils Max Docutils
> Version Version Version
> ------- ------------ ------------
> 3.4.3 >= 0.12.0 < 0.18.0
> 4.0.0 >= 0.12.0 < 0.19.0
> 6.0.0 >= 0.18.0 < 0.20.0
> 7.0.0 >= 0.18.1 < 0.21.0
> 7.2.0 >= 0.18.1 < 0.20.0
> 7.4.0 >= 0.18.1 < 0.21.0
> 8.0.0 >= 0.20.0 < 0.22.0
> 8.2.3 >= 0.20.0 < 0.22.0
> ======= ============ ============
>
<nitpick>
So this is what I see in the changelog (not looked into < 6.0.0):
================= ====================
Sphinx version docutils version
(since) min max
----------------- ------------ -------
6.0.0 >= 0.18 < 0.20
6.2.0 >= 0.18.1 < 0.20
7.0.1 >= 0.18.1 < 0.21
7.3.0 >= 0.18.1 < 0.22
7.4.0 >= 0.20 < 0.22
(up to 8.2.3)
================= ====================
This Looks quite different from yours ...
</nitpick>
That said, these dependencies should be recognized and taken care of by
pip/PyPI (or whatever distro package management system), and already met;
unless you have done something strange/dangerous and screwed up your Sphinx
installation.
My limited imagination prevents me from coming up with any plausible
scenario where these checks at every sphinx-build invocation can be helpful.
> For now, add a logic inside conf.py to check the above
> compatibility list, emitting warnings if the docutils
> version doesn't match it.
>
> This way, when we discover incompatibility issues, we
> can easily adjust the table.
Does not sound convincing enough to me. Sorry.
Thanks, Akira
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> Documentation/conf.py | 70 +++++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 67 insertions(+), 3 deletions(-)
[...]
Akira Yokosawa <akiyks@gmail.com> writes: > On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote: >> As reported by Akira, there were incompatibility issues with >> Sphinx and docutils with docutils 0.19. There's already >> a fix for it, but, as there are incompatibility issues with >> different versions, better to add a check to verify if the >> combination is supported/tested. >> > > I've been skeptical of adding such checks in conf.py. I have to kind of agree with this concern. We have managed without this complexity so far. It looks like we could always be behind on maintaining it going forward. Do we *really* need this one? Thanks, jon
Em Sun, 22 Jun 2025 14:58:04 -0600 Jonathan Corbet <corbet@lwn.net> escreveu: > Akira Yokosawa <akiyks@gmail.com> writes: > > > On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote: > >> As reported by Akira, there were incompatibility issues with > >> Sphinx and docutils with docutils 0.19. There's already > >> a fix for it, but, as there are incompatibility issues with > >> different versions, better to add a check to verify if the > >> combination is supported/tested. > >> > > > > I've been skeptical of adding such checks in conf.py. > > I have to kind of agree with this concern. We have managed without this > complexity so far. It looks like we could always be behind on > maintaining it going forward. Do we *really* need this one? IMO having a check is interesting, as the dependency between Sphinx and docutils is high. Yet, with the testing script, this may not be needed, provided that we run it to check if changes at Sphinx extensions won't cause regressions. Still, the dependency check at test_doc_build.py is not complete. Anyway, if you prefer, don't pick this one. We can revisit it later when needed. Thanks, Mauro
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes: > Em Sun, 22 Jun 2025 14:58:04 -0600 > Jonathan Corbet <corbet@lwn.net> escreveu: > >> Akira Yokosawa <akiyks@gmail.com> writes: >> >> > On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote: >> >> As reported by Akira, there were incompatibility issues with >> >> Sphinx and docutils with docutils 0.19. There's already >> >> a fix for it, but, as there are incompatibility issues with >> >> different versions, better to add a check to verify if the >> >> combination is supported/tested. >> >> >> > >> > I've been skeptical of adding such checks in conf.py. >> >> I have to kind of agree with this concern. We have managed without this >> complexity so far. It looks like we could always be behind on >> maintaining it going forward. Do we *really* need this one? > > IMO having a check is interesting, as the dependency between > Sphinx and docutils is high. Yet, with the testing script, this may > not be needed, provided that we run it to check if changes at Sphinx > extensions won't cause regressions. Still, the dependency check > at test_doc_build.py is not complete. > > Anyway, if you prefer, don't pick this one. We can revisit it later > when needed. I've left it out for now, but applied the rest of the series. Keep it around, we may yet decide we need it... Thanks, jon
Em Wed, 25 Jun 2025 12:37:37 -0600 Jonathan Corbet <corbet@lwn.net> escreveu: > Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes: > > > Em Sun, 22 Jun 2025 14:58:04 -0600 > > Jonathan Corbet <corbet@lwn.net> escreveu: > > > >> Akira Yokosawa <akiyks@gmail.com> writes: > >> > >> > On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote: > >> >> As reported by Akira, there were incompatibility issues with > >> >> Sphinx and docutils with docutils 0.19. There's already > >> >> a fix for it, but, as there are incompatibility issues with > >> >> different versions, better to add a check to verify if the > >> >> combination is supported/tested. > >> >> > >> > > >> > I've been skeptical of adding such checks in conf.py. > >> > >> I have to kind of agree with this concern. We have managed without this > >> complexity so far. It looks like we could always be behind on > >> maintaining it going forward. Do we *really* need this one? > > > > IMO having a check is interesting, as the dependency between > > Sphinx and docutils is high. Yet, with the testing script, this may > > not be needed, provided that we run it to check if changes at Sphinx > > extensions won't cause regressions. Still, the dependency check > > at test_doc_build.py is not complete. > > > > Anyway, if you prefer, don't pick this one. We can revisit it later > > when needed. > > I've left it out for now, but applied the rest of the series. Keep it > around, we may yet decide we need it... Ok, I placed on my scratch tree on github, on a separate branch: https://github.com/mchehab/linux/commits/check_sphinx_at_conf_py/ The patch is here: https://github.com/mchehab/linux/commit/178f37fce4aa16592b0c0b567ea0ffca744c3af5 (I'm documenting here just in case we forget and need it again ;-) ) Regards, Mauro > > Thanks, > > jon Thanks, Mauro
Em Sun, 22 Jun 2025 20:19:52 +0900 Akira Yokosawa <akiyks@gmail.com> escreveu: > > After check Sphinx release notes, it seems that the > > version compatibility is given by: > > > > ======= ============ ============ > > Sphinx Min Docutils Max Docutils > > Version Version Version > > ------- ------------ ------------ > > 3.4.3 >= 0.12.0 < 0.18.0 > > 4.0.0 >= 0.12.0 < 0.19.0 > > 6.0.0 >= 0.18.0 < 0.20.0 > > 7.0.0 >= 0.18.1 < 0.21.0 > > 7.2.0 >= 0.18.1 < 0.20.0 > > 7.4.0 >= 0.18.1 < 0.21.0 > > 8.0.0 >= 0.20.0 < 0.22.0 > > 8.2.3 >= 0.20.0 < 0.22.0 > > ======= ============ ============ > > > > <nitpick> > So this is what I see in the changelog (not looked into < 6.0.0): > > ================= ==================== > Sphinx version docutils version > (since) min max > ----------------- ------------ ------- > 6.0.0 >= 0.18 < 0.20 > 6.2.0 >= 0.18.1 < 0.20 > 7.0.1 >= 0.18.1 < 0.21 > 7.3.0 >= 0.18.1 < 0.22 > 7.4.0 >= 0.20 < 0.22 > (up to 8.2.3) > ================= ==================== > > This Looks quite different from yours ... > </nitpick> Thanks for your feedback! Getting this right is not too easy, specially since not all changelog mentions docutil version. Btw, one of the factors I used when building my table were to check the release dates of Sphinx and docutils. So, I manually adjusted some entries, as changelog usually say something like: Release 5.0.0 (released May 30, 2022)¶ Dependencies 5.0.0 b1 #10164: Support Docutils 0.18. Patch by Adam Turner. But doesn't mention minimal supported version. I'll re-check and adjust it as needed, if Jon decides to pick it. Thanks, Mauro
Hi Akira,
Em Sun, 22 Jun 2025 20:19:52 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:
> On Sun, 22 Jun 2025 08:02:44 +0200, Mauro Carvalho Chehab wrote:
> > As reported by Akira, there were incompatibility issues with
> > Sphinx and docutils with docutils 0.19. There's already
> > a fix for it, but, as there are incompatibility issues with
> > different versions, better to add a check to verify if the
> > combination is supported/tested.
> >
>
> I've been skeptical of adding such checks in conf.py.
>
> What happened the other day was that Jon used a deprecated (and rarely
> used) method of docutils which failed to work properly only in
> docutils 0.19 (there is no mention of related issues in its and
> nearby release notes).
True, but the same also happened with me at the parser_yaml code.
>
> Your integration of parser_yaml extension will raise the minimum
> version of docutils to 0.17.1.
>
> I think all you will need is just
> to check:
>
> docutils < 0.17.1
> , and to make a warning regardless of Sphinx versions.
That's not the best solution.
See, Docutils 0.17 was released in Jun, 2021. and 0.17.1 on
October, 2021. If we look at the Sphinx releases, we have:
====== ===================
Sphinx Release Date
------ -------------------
4.1.0 June 10, 2021
4.1.1 July 25, 2021
4.1.2 August 1, 2021
4.1.3 August 15, 2021
4.2.0 September 28, 2021
====== ===================
As Sphinx policy is to not even fix bugs when a new release
happens, it means that, at the best, the minimal version made to be
compatible with 0.17 is 4.1.x, and the minimal compatible with
0.17.1 is 4.2.x.
So, if we want not to raise the minimal version to 4.2.x (*), the right
fix is this (I'll be sending at the upcoming YAML patch series I'll be
sending next week):
diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/parser_yaml.py
index 8288e2ff7c7c..1602b31f448e 100755
--- a/Documentation/sphinx/parser_yaml.py
+++ b/Documentation/sphinx/parser_yaml.py
@@ -77,6 +77,10 @@ class YamlParser(Parser):
result.append(line, document.current_source, lineoffset)
+ # Fix backward compatibility with docutils < 0.17.1
+ if "tab_width" not in vars(document.settings):
+ document.settings.tab_width = 8
+
rst_parser = RSTParser()
rst_parser.parse('\n'.join(result), document)
What happens is that Sphinx is a shell case built on the top of
docutils: the actual ReST parser is inside docutils.
(*) the exact patch where 0.17.1 support was added/tested on
Sphinx needs to be checked
Now, there is another alternative: raise the bar, increasing minimal
versions for Python and Sphinx. Yet, I would do it only once per year.
> That said, these dependencies should be recognized and taken care of by
> pip/PyPI (or whatever distro package management system), and already met;
> unless you have done something strange/dangerous and screwed up your Sphinx
> installation.
>
> My limited imagination prevents me from coming up with any plausible
> scenario where these checks at every sphinx-build invocation can be helpful.
>
Heh, playing with scripts/test_doc_build.py, pip allowed to create
several environments that are valid for the package management software
but either:
- cause sphinx-build to crash even before returning Sphinx version;
- crash in the middle of the compilation.
As I said, Sphinx is a shell on the top of docutils. Some of the code
there seem to depend on subtile behaviors, like the one with
the tab_width affecting parser_yaml.py on certain Sphinx/docutils
combinations or the one fixed by Jon with regards to setup a class
for the broken references.
The idea of this patch is to warn the one building the docs
that he might be navigating turbulent waters when it is unknown
if the combination is supported. It also better defines the
scope of backward compatibility we want to provide.
Thanks,
Mauro
© 2016 - 2025 Red Hat, Inc.