Once upon a time, "sphinx-build" on certain RPM platforms invoked
specifically a Python 2.x version, while "sphinx-build-3" was a distro
shim for the Python 3.x version.
These days, none of our supported platforms utilize a 2.x version, so it
should be safe to search for 'sphinx-build' prior to 'sphinx-build-3',
which will prefer pip/venv installed versions of sphinx if they're
available.
This adds an extremely convenient ability to test document building
ability in QEMU across multiple versions of Sphinx for the purposes of
compatibility testing.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/meson.build | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/meson.build b/docs/meson.build
index 9136fed3b73..906034f9a87 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -1,5 +1,5 @@
if get_option('sphinx_build') == ''
- sphinx_build = find_program(['sphinx-build-3', 'sphinx-build'],
+ sphinx_build = find_program(['sphinx-build', 'sphinx-build-3'],
required: get_option('docs'))
else
sphinx_build = find_program(get_option('sphinx_build'),
--
2.39.0On 2/21/23 02:24, John Snow wrote:
> Once upon a time, "sphinx-build" on certain RPM platforms invoked
> specifically a Python 2.x version, while "sphinx-build-3" was a distro
> shim for the Python 3.x version.
>
> These days, none of our supported platforms utilize a 2.x version, so it
> should be safe to search for 'sphinx-build' prior to 'sphinx-build-3',
> which will prefer pip/venv installed versions of sphinx if they're
> available.
>
> This adds an extremely convenient ability to test document building
> ability in QEMU across multiple versions of Sphinx for the purposes of
> compatibility testing.
Can we just use "$PYTHON -m sphinx.cmd.build" instead, to ensure that we don't
escape the virtual environment? Or even better, we could have a simple script
like this:
#! /usr/bin/env python3
from pkg_resources import load_entry_point
if __name__ == '__main__':
if sys.argv[1] == '--check':
try:
load_entry_point(sys.argv[2], 'console_scripts', sys.argv[3])
sys.exit(0)
except ImportError:
sys.exit(1)
else:
entry_point = load_entry_point(sys.argv[1], 'console_scripts', sys.argv[2])
# The second argument to python-run.py becomes sys.argv[0]
del sys.argv[0:1]
sys.exit(entry_point())
then docs/meson.build can do this:
python_run = find_program('scripts/python-run.py')
build_docs = false
if get_feature('docs') \
.require(run_command(python_run, '--check', 'sphinx', 'sphinx-build',
check: false).returncode() == 0,
error: 'Could not find sphinx installation') \
.allowed()
# The sphinx module is installed
SPHINX_ARGS = ['env', 'CONFDIR=' + qemu_confdir,
python_run, 'sphinx', 'sphinx-build', '-q']
...
build_docs = (sphinx_build_test_out.returncode() == 0)
...
endif
This again ensures that sphinx-build will not escape the virtual environment
if there is one. configure can also use the script to run meson, though that
can come later.
Paolo
On Tue, Feb 21, 2023, 6:31 AM Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 2/21/23 02:24, John Snow wrote:
> > Once upon a time, "sphinx-build" on certain RPM platforms invoked
> > specifically a Python 2.x version, while "sphinx-build-3" was a distro
> > shim for the Python 3.x version.
> >
> > These days, none of our supported platforms utilize a 2.x version, so it
> > should be safe to search for 'sphinx-build' prior to 'sphinx-build-3',
> > which will prefer pip/venv installed versions of sphinx if they're
> > available.
> >
> > This adds an extremely convenient ability to test document building
> > ability in QEMU across multiple versions of Sphinx for the purposes of
> > compatibility testing.
>
> Can we just use "$PYTHON -m sphinx.cmd.build" instead, to ensure that we
> don't
> escape the virtual environment? Or even better, we could have a simple
> script
> like this:
>
> #! /usr/bin/env python3
>
> from pkg_resources import load_entry_point
>
> if __name__ == '__main__':
> if sys.argv[1] == '--check':
> try:
> load_entry_point(sys.argv[2], 'console_scripts', sys.argv[3])
> sys.exit(0)
> except ImportError:
> sys.exit(1)
> else:
> entry_point = load_entry_point(sys.argv[1], 'console_scripts',
> sys.argv[2])
> # The second argument to python-run.py becomes sys.argv[0]
> del sys.argv[0:1]
> sys.exit(entry_point())
>
> then docs/meson.build can do this:
>
> python_run = find_program('scripts/python-run.py')
> build_docs = false
> if get_feature('docs') \
> .require(run_command(python_run, '--check', 'sphinx', 'sphinx-build',
> check: false).returncode() == 0,
> error: 'Could not find sphinx installation') \
> .allowed()
> # The sphinx module is installed
> SPHINX_ARGS = ['env', 'CONFDIR=' + qemu_confdir,
> python_run, 'sphinx', 'sphinx-build', '-q']
> ...
> build_docs = (sphinx_build_test_out.returncode() == 0)
> ...
> endif
>
> This again ensures that sphinx-build will not escape the virtual
> environment
> if there is one. configure can also use the script to run meson, though
> that
> can come later.
>
> Paolo
>
Yeah, I proposed we use "python3 -m sphinx.cmd.build" once, but Peter did
not like the idea of Sphinx becoming a python dependency instead of being
treated as a black box.
Obviously circumstances are shifting somewhat and we may be more open to
treating Sphinx as a python dependency given that we need to enforce
compatibility with custom plugins written in qemu.git.
If I was trying to please absolutely nobody but me, I'd certainly use the
`$python -m sphinx` approach; especially because it means that for
qapi-gen, the code is run under the same environment in both cases (native
qapi-gen exec and sphinx doc building).
I'm for it, but lost appetite for making the argument some time back.
>
On 2/21/23 12:31, Paolo Bonzini wrote:
> On 2/21/23 02:24, John Snow wrote:
>> Once upon a time, "sphinx-build" on certain RPM platforms invoked
>> specifically a Python 2.x version, while "sphinx-build-3" was a distro
>> shim for the Python 3.x version.
>>
>> These days, none of our supported platforms utilize a 2.x version, so it
>> should be safe to search for 'sphinx-build' prior to 'sphinx-build-3',
>> which will prefer pip/venv installed versions of sphinx if they're
>> available.
>>
>> This adds an extremely convenient ability to test document building
>> ability in QEMU across multiple versions of Sphinx for the purposes of
>> compatibility testing.
>
> Can we just use "$PYTHON -m sphinx.cmd.build" instead, to ensure that we
> don't
> escape the virtual environment? Or even better, we could have a simple
> script
> like this:
>
> #! /usr/bin/env python3
>
> from pkg_resources import load_entry_point
>
> if __name__ == '__main__':
> if sys.argv[1] == '--check':
> try:
> load_entry_point(sys.argv[2], 'console_scripts', sys.argv[3])
> sys.exit(0)
> except ImportError:
> sys.exit(1)
> else:
> entry_point = load_entry_point(sys.argv[1], 'console_scripts',
> sys.argv[2])
> # The second argument to python-run.py becomes sys.argv[0]
> del sys.argv[0:1]
> sys.exit(entry_point())
>
> then docs/meson.build can do this:
>
> python_run = find_program('scripts/python-run.py')
> build_docs = false
> if get_feature('docs') \
> .require(run_command(python_run, '--check', 'sphinx', 'sphinx-build',
> check: false).returncode() == 0,
> error: 'Could not find sphinx installation') \
> .allowed()
> # The sphinx module is installed
> SPHINX_ARGS = ['env', 'CONFDIR=' + qemu_confdir,
> python_run, 'sphinx', 'sphinx-build', '-q']
> ...
> build_docs = (sphinx_build_test_out.returncode() == 0)
> ...
> endif
>
> This again ensures that sphinx-build will not escape the virtual
> environment
> if there is one. configure can also use the script to run meson, though
> that
> can come later.
Ok, it's a bit harder but it works. Patch coming later today.
Paolo
John Snow <jsnow@redhat.com> writes:
> Once upon a time, "sphinx-build" on certain RPM platforms invoked
> specifically a Python 2.x version, while "sphinx-build-3" was a distro
> shim for the Python 3.x version.
>
> These days, none of our supported platforms utilize a 2.x version, so it
> should be safe to search for 'sphinx-build' prior to 'sphinx-build-3',
> which will prefer pip/venv installed versions of sphinx if they're
> available.
>
> This adds an extremely convenient ability to test document building
> ability in QEMU across multiple versions of Sphinx for the purposes of
> compatibility testing.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/meson.build | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/docs/meson.build b/docs/meson.build
> index 9136fed3b73..906034f9a87 100644
> --- a/docs/meson.build
> +++ b/docs/meson.build
> @@ -1,5 +1,5 @@
> if get_option('sphinx_build') == ''
> - sphinx_build = find_program(['sphinx-build-3', 'sphinx-build'],
> + sphinx_build = find_program(['sphinx-build', 'sphinx-build-3'],
> required: get_option('docs'))
> else
> sphinx_build = find_program(get_option('sphinx_build'),
Do we still need to check for sphinx-build-3? Or asked differently, is
there any supported build host that provides only sphinx-build-3?
On Tue, Feb 21, 2023, 1:50 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Once upon a time, "sphinx-build" on certain RPM platforms invoked
> > specifically a Python 2.x version, while "sphinx-build-3" was a distro
> > shim for the Python 3.x version.
> >
> > These days, none of our supported platforms utilize a 2.x version, so it
> > should be safe to search for 'sphinx-build' prior to 'sphinx-build-3',
> > which will prefer pip/venv installed versions of sphinx if they're
> > available.
> >
> > This adds an extremely convenient ability to test document building
> > ability in QEMU across multiple versions of Sphinx for the purposes of
> > compatibility testing.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/meson.build | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/docs/meson.build b/docs/meson.build
> > index 9136fed3b73..906034f9a87 100644
> > --- a/docs/meson.build
> > +++ b/docs/meson.build
> > @@ -1,5 +1,5 @@
> > if get_option('sphinx_build') == ''
> > - sphinx_build = find_program(['sphinx-build-3', 'sphinx-build'],
> > + sphinx_build = find_program(['sphinx-build', 'sphinx-build-3'],
> > required: get_option('docs'))
> > else
> > sphinx_build = find_program(get_option('sphinx_build'),
>
> Do we still need to check for sphinx-build-3? Or asked differently, is
> there any supported build host that provides only sphinx-build-3?
>
Yes, modern Fedora still uses "sphinx-build-3" as the name in /usr/bin for
the rpm-packaged version of sphinx.
It's just that the only platforms where "sphinx-build" is the 2.x version
are platforms on which we want to drop 3.6 support anyway, so it's OK to
invert the search priority in the context of this series.
(All pip/pypi versions use "sphinx-build" as the binary name. In effect,
this patch means we prefer pip/pypi versions if they're in your $PATH.)
>
John Snow <jsnow@redhat.com> writes:
> On Tue, Feb 21, 2023, 1:50 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Once upon a time, "sphinx-build" on certain RPM platforms invoked
>> > specifically a Python 2.x version, while "sphinx-build-3" was a distro
>> > shim for the Python 3.x version.
>> >
>> > These days, none of our supported platforms utilize a 2.x version, so it
>> > should be safe to search for 'sphinx-build' prior to 'sphinx-build-3',
>> > which will prefer pip/venv installed versions of sphinx if they're
>> > available.
>> >
>> > This adds an extremely convenient ability to test document building
>> > ability in QEMU across multiple versions of Sphinx for the purposes of
>> > compatibility testing.
>> >
>> > Signed-off-by: John Snow <jsnow@redhat.com>
>> > ---
>> > docs/meson.build | 2 +-
>> > 1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/docs/meson.build b/docs/meson.build
>> > index 9136fed3b73..906034f9a87 100644
>> > --- a/docs/meson.build
>> > +++ b/docs/meson.build
>> > @@ -1,5 +1,5 @@
>> > if get_option('sphinx_build') == ''
>> > - sphinx_build = find_program(['sphinx-build-3', 'sphinx-build'],
>> > + sphinx_build = find_program(['sphinx-build', 'sphinx-build-3'],
>> > required: get_option('docs'))
>> > else
>> > sphinx_build = find_program(get_option('sphinx_build'),
>>
>> Do we still need to check for sphinx-build-3? Or asked differently, is
>> there any supported build host that provides only sphinx-build-3?
>>
>
> Yes, modern Fedora still uses "sphinx-build-3" as the name in /usr/bin for
> the rpm-packaged version of sphinx.
For what it's worth, python3-sphinx-5.0.2-2.fc37.noarch provides
/usr/bin/sphinx-build
/usr/bin/sphinx-build-3
/usr/bin/sphinx-build-3.11
where the latter two are symbolic links to the first. No need to check
for sphinx-build-3 here.
> It's just that the only platforms where "sphinx-build" is the 2.x version
> are platforms on which we want to drop 3.6 support anyway, so it's OK to
> invert the search priority in the context of this series.
>
> (All pip/pypi versions use "sphinx-build" as the binary name. In effect,
> this patch means we prefer pip/pypi versions if they're in your $PATH.)
On Wed, Feb 22, 2023 at 2:15 AM Markus Armbruster <armbru@redhat.com> wrote:
>
> John Snow <jsnow@redhat.com> writes:
>
> > On Tue, Feb 21, 2023, 1:50 AM Markus Armbruster <armbru@redhat.com> wrote:
> >
> >> John Snow <jsnow@redhat.com> writes:
> >>
> >> > Once upon a time, "sphinx-build" on certain RPM platforms invoked
> >> > specifically a Python 2.x version, while "sphinx-build-3" was a distro
> >> > shim for the Python 3.x version.
> >> >
> >> > These days, none of our supported platforms utilize a 2.x version, so it
> >> > should be safe to search for 'sphinx-build' prior to 'sphinx-build-3',
> >> > which will prefer pip/venv installed versions of sphinx if they're
> >> > available.
> >> >
> >> > This adds an extremely convenient ability to test document building
> >> > ability in QEMU across multiple versions of Sphinx for the purposes of
> >> > compatibility testing.
> >> >
> >> > Signed-off-by: John Snow <jsnow@redhat.com>
> >> > ---
> >> > docs/meson.build | 2 +-
> >> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >> >
> >> > diff --git a/docs/meson.build b/docs/meson.build
> >> > index 9136fed3b73..906034f9a87 100644
> >> > --- a/docs/meson.build
> >> > +++ b/docs/meson.build
> >> > @@ -1,5 +1,5 @@
> >> > if get_option('sphinx_build') == ''
> >> > - sphinx_build = find_program(['sphinx-build-3', 'sphinx-build'],
> >> > + sphinx_build = find_program(['sphinx-build', 'sphinx-build-3'],
> >> > required: get_option('docs'))
> >> > else
> >> > sphinx_build = find_program(get_option('sphinx_build'),
> >>
> >> Do we still need to check for sphinx-build-3? Or asked differently, is
> >> there any supported build host that provides only sphinx-build-3?
> >>
> >
> > Yes, modern Fedora still uses "sphinx-build-3" as the name in /usr/bin for
> > the rpm-packaged version of sphinx.
>
> For what it's worth, python3-sphinx-5.0.2-2.fc37.noarch provides
>
> /usr/bin/sphinx-build
> /usr/bin/sphinx-build-3
> /usr/bin/sphinx-build-3.11
>
> where the latter two are symbolic links to the first. No need to check
> for sphinx-build-3 here.
Oh, I see. I guess it should be fine, but only if we explicitly drop
support for the 3.6 version that comes with CentOS. I'm not entirely
sure if "sphinx-build-3" is used anywhere else, I *think* it's just an
rpm-ism.
On Thu, Feb 23, 2023 at 5:40 AM John Snow <jsnow@redhat.com> wrote:> >
For what it's worth, python3-sphinx-5.0.2-2.fc37.noarch provides
> >
> > /usr/bin/sphinx-build
> > /usr/bin/sphinx-build-3
> > /usr/bin/sphinx-build-3.11
> >
> > where the latter two are symbolic links to the first. No need to check
> > for sphinx-build-3 here.
>
> Oh, I see. I guess it should be fine, but only if we explicitly drop
> support for the 3.6 version that comes with CentOS. I'm not entirely
> sure if "sphinx-build-3" is used anywhere else, I *think* it's just an
> rpm-ism.
Are you sure CentOS has a problem there? I checked
python3-sphinx-1.7.6-3.el8 and it has sphinx-build as a symlink too:
$ rpm2cpio ../python3-sphinx-1.7.6-3.el8.noarch.rpm |cpio -idv
$ ls -l usr/bin/
total 16
-rwxr-xr-x. 1 pbonzini users 403 Feb 23 09:50 sphinx-apidoc
lrwxrwxrwx. 1 pbonzini users 13 Feb 23 09:50 sphinx-apidoc-3 -> sphinx-apidoc
lrwxrwxrwx. 1 pbonzini users 13 Feb 23 09:50 sphinx-apidoc-3.6 -> sphinx-apidoc
-rwxr-xr-x. 1 pbonzini users 405 Feb 23 09:50 sphinx-autogen
lrwxrwxrwx. 1 pbonzini users 14 Feb 23 09:50 sphinx-autogen-3 -> sphinx-autogen
lrwxrwxrwx. 1 pbonzini users 14 Feb 23 09:50 sphinx-autogen-3.6 ->
sphinx-autogen
-rwxr-xr-x. 1 pbonzini users 401 Feb 23 09:50 sphinx-build
lrwxrwxrwx. 1 pbonzini users 12 Feb 23 09:50 sphinx-build-3 -> sphinx-build
lrwxrwxrwx. 1 pbonzini users 12 Feb 23 09:50 sphinx-build-3.6 -> sphinx-build
-rwxr-xr-x. 1 pbonzini users 411 Feb 23 09:50 sphinx-quickstart
lrwxrwxrwx. 1 pbonzini users 17 Feb 23 09:50 sphinx-quickstart-3 ->
sphinx-quickstart
lrwxrwxrwx. 1 pbonzini users 17 Feb 23 09:50 sphinx-quickstart-3.6 ->
sphinx-quickstart
And it's 3.6-based:
$ ls -l usr/lib
total 0
drwxr-xr-x. 3 pbonzini users 27 Feb 23 09:50 python3.6
$ head -4 usr/bin/sphinx-build
#!/usr/libexec/platform-python
# EASY-INSTALL-ENTRY-SCRIPT: 'Sphinx==1.7.6','console_scripts','sphinx-build'
__requires__ = 'Sphinx==1.7.6'
import re
This alternative, simpler patch should do it:
diff --git a/docs/meson.build b/docs/meson.build
index 9136fed3b730..1ab5a9882018 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -1,10 +1,5 @@
-if get_option('sphinx_build') == ''
- sphinx_build = find_program(['sphinx-build-3', 'sphinx-build'],
- required: get_option('docs'))
-else
- sphinx_build = find_program(get_option('sphinx_build'),
- required: get_option('docs'))
-endif
+sphinx_build = find_program(get_option('sphinx_build'),
+ required: get_option('docs'))
# Check if tools are available to build documentation.
build_docs = false
diff --git a/meson_options.txt b/meson_options.txt
index 7e5801db90f9..92440429b1d0 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -12,7 +12,7 @@ option('pkgversion', type : 'string', value : '',
description: 'use specified string as sub-version of the package')
option('smbd', type : 'string', value : '',
description: 'Path to smbd for slirp networking')
-option('sphinx_build', type : 'string', value : '',
+option('sphinx_build', type : 'string', value : 'sphinx-build',
description: 'Use specified sphinx-build for building document')
option('iasl', type : 'string', value : '',
description: 'Path to ACPI disassembler')
Paolo
John Snow <jsnow@redhat.com> writes:
> On Wed, Feb 22, 2023 at 2:15 AM Markus Armbruster <armbru@redhat.com> wrote:
>>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > On Tue, Feb 21, 2023, 1:50 AM Markus Armbruster <armbru@redhat.com> wrote:
>> >
>> >> John Snow <jsnow@redhat.com> writes:
>> >>
>> >> > Once upon a time, "sphinx-build" on certain RPM platforms invoked
>> >> > specifically a Python 2.x version, while "sphinx-build-3" was a distro
>> >> > shim for the Python 3.x version.
>> >> >
>> >> > These days, none of our supported platforms utilize a 2.x version, so it
>> >> > should be safe to search for 'sphinx-build' prior to 'sphinx-build-3',
>> >> > which will prefer pip/venv installed versions of sphinx if they're
>> >> > available.
>> >> >
>> >> > This adds an extremely convenient ability to test document building
>> >> > ability in QEMU across multiple versions of Sphinx for the purposes of
>> >> > compatibility testing.
>> >> >
>> >> > Signed-off-by: John Snow <jsnow@redhat.com>
>> >> > ---
>> >> > docs/meson.build | 2 +-
>> >> > 1 file changed, 1 insertion(+), 1 deletion(-)
>> >> >
>> >> > diff --git a/docs/meson.build b/docs/meson.build
>> >> > index 9136fed3b73..906034f9a87 100644
>> >> > --- a/docs/meson.build
>> >> > +++ b/docs/meson.build
>> >> > @@ -1,5 +1,5 @@
>> >> > if get_option('sphinx_build') == ''
>> >> > - sphinx_build = find_program(['sphinx-build-3', 'sphinx-build'],
>> >> > + sphinx_build = find_program(['sphinx-build', 'sphinx-build-3'],
>> >> > required: get_option('docs'))
>> >> > else
>> >> > sphinx_build = find_program(get_option('sphinx_build'),
>> >>
>> >> Do we still need to check for sphinx-build-3? Or asked differently, is
>> >> there any supported build host that provides only sphinx-build-3?
>> >>
>> >
>> > Yes, modern Fedora still uses "sphinx-build-3" as the name in /usr/bin for
>> > the rpm-packaged version of sphinx.
>>
>> For what it's worth, python3-sphinx-5.0.2-2.fc37.noarch provides
>>
>> /usr/bin/sphinx-build
>> /usr/bin/sphinx-build-3
>> /usr/bin/sphinx-build-3.11
>>
>> where the latter two are symbolic links to the first. No need to check
>> for sphinx-build-3 here.
>
> Oh, I see. I guess it should be fine, but only if we explicitly drop
> support for the 3.6 version that comes with CentOS. I'm not entirely
> sure if "sphinx-build-3" is used anywhere else, I *think* it's just an
> rpm-ism.
I can see just two reasons for trying sphinx-build-3:
1. sphinx-build does not exist.
2. sphinx-build exists, but uses Python 2, which doesn't work with our
Sphinx extension.
The commit message seems to claim it's not 2.
So, what is it?
© 2016 - 2026 Red Hat, Inc.