Hi Jon,

This patch adds support for not running sphinx-build at the wrapper
tool. It was requested by Akira, who wanted to be able to ignore
Sphinx errors during latex build and still try to build PDF.

This patch is against docs/build-script and applies after the 3 patch
series I sent yesterday:

    https://lore.kernel.org/linux-doc/cover.1758361087.git.mchehab+huawei@kernel.org/


While Akira's original intention is to have pdfdocs target depend on
latexdocs, IMO, this is overkill, as probably only Akira and a couple
of other developers might want to have such behavior.

See, after all changes, the makefile rule for *all* doc build targets
is simple:

        $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check
        +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \
                --sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \
                --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
                --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)

After applying patch 1 from this series, it is really easy to replicate 
"make -i" by writing a small script that does:

	tools/docs/sphinx-pre-install --version-check	
	tools/docs/sphinx-build-wrapper latexdocs || echo "LaTeX build failed, but we'll try build PDF anyway"
	tools/docs/sphinx-build-wrapper -s pdfdocs

So, I'm not proposing doing any changes to Makefile, as IMHO, for
most people, the intermediate steps don't matter.

Mauro Carvalho Chehab (1):
  tools/docs/sphinx-build-wrapper: allow skipping sphinx-build step

 tools/docs/sphinx-build-wrapper | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

-- 
2.51.0

On Sun, 21 Sep 2025 11:13:24 +0200, Mauro Carvalho Chehab wrote:
> Hi Jon,
> 
> This patch adds support for not running sphinx-build at the wrapper
> tool. It was requested by Akira, who wanted to be able to ignore
> Sphinx errors during latex build and still try to build PDF.

Thank you for trying to figure out my intention.
However, you failed to see the point.

> 
> This patch is against docs/build-script and applies after the 3 patch
> series I sent yesterday:
> 
>     https://lore.kernel.org/linux-doc/cover.1758361087.git.mchehab+huawei@kernel.org/
> 
> 
> While Akira's original intention is to have pdfdocs target depend on
> latexdocs, IMO, this is overkill, as probably only Akira and a couple
> of other developers might want to have such behavior.

I think it is only you who don't want such behavior.

> 
> See, after all changes, the makefile rule for *all* doc build targets
> is simple:
> 
>         $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check
>         +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \
>                 --sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \
>                 --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
>                 --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)
> 
> After applying patch 1 from this series, it is really easy to replicate 
> "make -i" by writing a small script that does:
> 
> 	tools/docs/sphinx-pre-install --version-check	
> 	tools/docs/sphinx-build-wrapper latexdocs || echo "LaTeX build failed, but we'll try build PDF anyway"
> 	tools/docs/sphinx-build-wrapper -s pdfdocs
> 

Hello?

You are the one who is changing the way "make pdfdocs" behaves.
All I want is to restore the current behavior, without any need to
use such an ad-hoc script.

Sorry, but I think I have to NAK this.

Furthermore, your "cleanup" is obfuscating the very fact that "pdfdocs"
needs a successful "latexdocs" stage.

I believe Documentation/Makefile is the right place to describe it.

Good luck,
Akira

Em Mon, 22 Sep 2025 20:30:40 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> On Sun, 21 Sep 2025 11:13:24 +0200, Mauro Carvalho Chehab wrote:
> > Hi Jon,
> > 
> > This patch adds support for not running sphinx-build at the wrapper
> > tool. It was requested by Akira, who wanted to be able to ignore
> > Sphinx errors during latex build and still try to build PDF.  
> 
> Thank you for trying to figure out my intention.
> However, you failed to see the point.
> 
> > 
> > This patch is against docs/build-script and applies after the 3 patch
> > series I sent yesterday:
> > 
> >     https://lore.kernel.org/linux-doc/cover.1758361087.git.mchehab+huawei@kernel.org/
> > 
> > 
> > While Akira's original intention is to have pdfdocs target depend on
> > latexdocs, IMO, this is overkill, as probably only Akira and a couple
> > of other developers might want to have such behavior.  
> 
> I think it is only you who don't want such behavior.

Whom else wants to use "make -i" to skip failed latexdocs build and
still do pdf builds? For what reason?

> > See, after all changes, the makefile rule for *all* doc build targets
> > is simple:
> > 
> >         $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check
> >         +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \
> >                 --sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \
> >                 --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
> >                 --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)
> > 
> > After applying patch 1 from this series, it is really easy to replicate 
> > "make -i" by writing a small script that does:
> > 
> > 	tools/docs/sphinx-pre-install --version-check	
> > 	tools/docs/sphinx-build-wrapper latexdocs || echo "LaTeX build failed, but we'll try build PDF anyway"
> > 	tools/docs/sphinx-build-wrapper -s pdfdocs
> >   
> 
> Hello?
> 
> You are the one who is changing the way "make pdfdocs" behaves.

It doesn't change. if latex is not built, it won't try to build
PDF files, just like before.

See, for almost all targets, docs build is a two-step procedure. That
includes htmldocs as well, as it requires a post-build step to copy
css and static files. Right now, the logic is hidden inside complex
make macros.

The new logic place the target-dependent extra steps on separate functions
which are now properly documented, executing them only if the first
step works.

With the "-s" parameter, one can skip the first step, running only
the second one.

> All I want is to restore the current behavior, without any need to
> use such an ad-hoc script.

Huh?

You said you want to be able to do just the reverse: to ignore
failures at latex build ("make -i").  If you want to have exactly
the same behavior, you don't need anything.

> Sorry, but I think I have to NAK this.
> 
> Furthermore, your "cleanup" is obfuscating the very fact that "pdfdocs"
> needs a successful "latexdocs" stage.
> 
> I believe Documentation/Makefile is the right place to describe it.

If you want to propose such change, be my guest. As I said, *I* won't propose
it, as IMO it is a bad idea, but if you want, feel free to submit a patch 
after this one similar to:

	-htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
	+htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs linkcheckdocs:
	+
	+pdfdocs: latexdocs
	+        +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ -s \
	+                --sphinxdirs="$(SPHINXDIRS)" \
	+                --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
	+                --paper=$(PAPER)

and let's discuss its pros/cons in separate.

Thanks,
Mauro

On Mon, 22 Sep 2025 14:23:30 +0200, Mauro Carvalho Chehab wrote:
> Em Mon, 22 Sep 2025 20:30:40 +0900
> Akira Yokosawa <akiyks@gmail.com> escreveu:
> 
>> On Sun, 21 Sep 2025 11:13:24 +0200, Mauro Carvalho Chehab wrote:
>>> Hi Jon,
>>>
>>> This patch adds support for not running sphinx-build at the wrapper
>>> tool. It was requested by Akira, who wanted to be able to ignore
>>> Sphinx errors during latex build and still try to build PDF.  
>>
>> Thank you for trying to figure out my intention.
>> However, you failed to see the point.
>>
>>>
>>> This patch is against docs/build-script and applies after the 3 patch
>>> series I sent yesterday:
>>>
>>>     https://lore.kernel.org/linux-doc/cover.1758361087.git.mchehab+huawei@kernel.org/
>>>
>>>
>>> While Akira's original intention is to have pdfdocs target depend on
>>> latexdocs, IMO, this is overkill, as probably only Akira and a couple
>>> of other developers might want to have such behavior.  
>>
>> I think it is only you who don't want such behavior.
> 
> Whom else wants to use "make -i" to skip failed latexdocs build and
> still do pdf builds? For what reason?

Hmm, looks like you were suffered from context mismatch.

My message you tried to understand (archived at:

    https://lore.kernel.org/5031e0c4-f17e-41b8-8955-959989e797f2@gmail.com/

) started this way:

    Hi, (just got v8, but sending anyway ...),

, and the message assumed the "make pdfdocs" behavior of v7.

At the time, I thought it was what you had intended and it looked as though
you wanted latexmk/xelatex to continue running as much as possible ignoring
any intermediate errors.

You have since fixed the missed exception handling at the latexdocs phase,
but your rapid respins of this series simply overwhelmed me and I have
failed to see "make pdfdocs" now gives up at latexdocs error.

My intention of mentioning GNU Make's options of -i, -k, etc. in the message
was to show you that the make--sub-make scheme is so flexible and it can
even emulate the buggy behavior of then sphinx-build-wrapper.

In normal cases, I won't use any of those options.
But as a human developer, I sometimes use them when I observe build errors
somewhere in "make pdfdocs".

> 
>>> See, after all changes, the makefile rule for *all* doc build targets
>>> is simple:
>>>
>>>         $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check
>>>         +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \
>>>                 --sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \
>>>                 --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
>>>                 --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)
>>>
>>> After applying patch 1 from this series, it is really easy to replicate 
>>> "make -i" by writing a small script that does:
>>>
>>> 	tools/docs/sphinx-pre-install --version-check	
>>> 	tools/docs/sphinx-build-wrapper latexdocs || echo "LaTeX build failed, but we'll try build PDF anyway"
>>> 	tools/docs/sphinx-build-wrapper -s pdfdocs
>>>   
>>
>> Hello?
>>
>> You are the one who is changing the way "make pdfdocs" behaves.
> 
> It doesn't change. if latex is not built, it won't try to build
> PDF files, just like before.

Didn't you mean "if latexdocs is not build"?
Yes, I now see this is the current behavior.

> 
> See, for almost all targets, docs build is a two-step procedure. That
> includes htmldocs as well, as it requires a post-build step to copy
> css and static files. Right now, the logic is hidden inside complex
> make macros.
> 
> The new logic place the target-dependent extra steps on separate functions
> which are now properly documented, executing them only if the first
> step works.
> 
> With the "-s" parameter, one can skip the first step, running only
> the second one.
> 
>> All I want is to restore the current behavior, without any need to
>> use such an ad-hoc script.
> 
> Huh?
> 
> You said you want to be able to do just the reverse: to ignore
> failures at latex build ("make -i").

No, as I said above, I don't want it in most cases.
But in rare occasions, I might want it.  That depends on the situation
at hand.

>                                       If you want to have exactly
> the same behavior, you don't need anything.
> 
>> Sorry, but I think I have to NAK this.
>>
>> Furthermore, your "cleanup" is obfuscating the very fact that "pdfdocs"
>> needs a successful "latexdocs" stage.
>>
>> I believe Documentation/Makefile is the right place to describe it.
> 
> If you want to propose such change, be my guest. As I said, *I* won't propose
> it, as IMO it is a bad idea, but if you want, feel free to submit a patch 
> after this one similar to:
> 
> 	-htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
> 	+htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs linkcheckdocs:
> 	+
> 	+pdfdocs: latexdocs
> 	+        +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ -s \
> 	+                --sphinxdirs="$(SPHINXDIRS)" \
> 	+                --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
> 	+                --paper=$(PAPER)
> 
> and let's discuss its pros/cons in separate.

OK. This looks like the way forward to me.

I'll prepare a patch (set) and submit it after v6.18-rc1.

In the meantime, I think we might be better off if we could root-cause
the "latexmk bug" you repeatedly claimed you had observed, without providing
any steps to reproduce.

Let me continue in another thread.
It would be far more interesting for me to (try to) figure out what you
had observed.

Thanks,
Akira

Em Wed, 24 Sep 2025 16:03:59 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> On Mon, 22 Sep 2025 14:23:30 +0200, Mauro Carvalho Chehab wrote:
> > Em Mon, 22 Sep 2025 20:30:40 +0900
> > Akira Yokosawa <akiyks@gmail.com> escreveu:
> >   
> >> On Sun, 21 Sep 2025 11:13:24 +0200, Mauro Carvalho Chehab wrote:  
> >>> Hi Jon,
> >>>
> >>> This patch adds support for not running sphinx-build at the wrapper
> >>> tool. It was requested by Akira, who wanted to be able to ignore
> >>> Sphinx errors during latex build and still try to build PDF.    
> >>
> >> Thank you for trying to figure out my intention.
> >> However, you failed to see the point.
> >>  
> >>>
> >>> This patch is against docs/build-script and applies after the 3 patch
> >>> series I sent yesterday:
> >>>
> >>>     https://lore.kernel.org/linux-doc/cover.1758361087.git.mchehab+huawei@kernel.org/
> >>>
> >>>
> >>> While Akira's original intention is to have pdfdocs target depend on
> >>> latexdocs, IMO, this is overkill, as probably only Akira and a couple
> >>> of other developers might want to have such behavior.    
> >>
> >> I think it is only you who don't want such behavior.  
> > 
> > Whom else wants to use "make -i" to skip failed latexdocs build and
> > still do pdf builds? For what reason?  
> 
> Hmm, looks like you were suffered from context mismatch.
> 
> My message you tried to understand (archived at:
> 
>     https://lore.kernel.org/5031e0c4-f17e-41b8-8955-959989e797f2@gmail.com/
> 
> ) started this way:
> 
>     Hi, (just got v8, but sending anyway ...),
> 
> , and the message assumed the "make pdfdocs" behavior of v7.
> 
> At the time, I thought it was what you had intended and it looked as though
> you wanted latexmk/xelatex to continue running as much as possible ignoring
> any intermediate errors.

No, this was a bug, as I answered you already and sent the fix:
if sphinx-build fails, the script shall bail out.

> You have since fixed the missed exception handling at the latexdocs phase,
> but your rapid respins of this series simply overwhelmed me and I have
> failed to see "make pdfdocs" now gives up at latexdocs error.
> 
> My intention of mentioning GNU Make's options of -i, -k, etc. in the message
> was to show you that the make--sub-make scheme is so flexible and it can
> even emulate the buggy behavior of then sphinx-build-wrapper.
> 
> In normal cases, I won't use any of those options.
> But as a human developer, I sometimes use them when I observe build errors
> somewhere in "make pdfdocs".
...
> >> All I want is to restore the current behavior, without any need to
> >> use such an ad-hoc script.  
> > 
> > Huh?
> > 
> > You said you want to be able to do just the reverse: to ignore
> > failures at latex build ("make -i").  
> 
> No, as I said above, I don't want it in most cases.
> But in rare occasions, I might want it.  That depends on the situation
> at hand.

There is exactly the usecase where running directly via makefile
may not be enough, as you may want to play with different scenarios.

On such cases, you'll be better served by calling the script directly, 
as it provides more flexibility without requiring to pass parameters via
env vars nor nor enable/disable different venvs to test with different
versions of docutils/spinx packages.

On an hypothetical scenario where pdfdocs is not working fine, calling
the script directly allows to easily test different scenarios, and it is
faster, as you won't do anything else but running directly the doc build
logic:

	# equivalent to "make pdfdocs":
	# build tex. If sphinx-build succeeds, build pdf for each tex
	./tools/docs/sphinx-build-wrapper pdfdocs

	# equivalent to "make latexdocs"
	./tools/docs/sphinx-build-wrapper latexdocs

	# generate latex with Sphinx 3.4.3 in verbose mode
	# equivalent to:
	#	. Sphinx_3.4.3/bin/activate
	#	make V=1 latexdocs
	#	deactivate
	./tools/docs/sphinx-build-wrapper latexdocs -V Sphinx_3.4.3 -v

	# build only translations.tex with Sphinx 5.2.0 in verbose mode
	./tools/docs/sphinx-build-wrapper latexdocs -V Sphinx_5.2.0 -v --sphinxdirs translations

Once you're happy with the above, you could proceed with the pdf build step,
forcing the script to skip sphinx-build step with:

	# Run just PDF build step, producing a summary at the end
	$ ./tools/docs/sphinx-build-wrapper pdfdocs  -s -v
	...
	Summary
	=======
	...
	translations/pdf/translations.pdf: FAILED
	...

	# After building latexdocs for translations.tex and core-api.tex
	# run the run test PDF build, producing a single line
	# with all failures for each generated .tex file:
	$ ./tools/docs/sphinx-build-wrapper pdfdocs  -s  --sphinxdirs translations core-api
	...
	Error: Can't build 1 PDF file(s): translations/pdf/translations.pdf

	# Use a different variable font deny configuration directory
	$ ./tools/docs/sphinx-build-wrapper pdfdocs  -s  --sphinxdirs translations --deny-vf ~/new-deny-vf/
	
> >> Furthermore, your "cleanup" is obfuscating the very fact that "pdfdocs"
> >> needs a successful "latexdocs" stage.
> >>
> >> I believe Documentation/Makefile is the right place to describe it.  
> > 
> > If you want to propose such change, be my guest. As I said, *I* won't propose
> > it, as IMO it is a bad idea, but if you want, feel free to submit a patch 
> > after this one similar to:
> > 
> > 	-htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
> > 	+htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs linkcheckdocs:
> > 	+
> > 	+pdfdocs: latexdocs
> > 	+        +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ -s \
> > 	+                --sphinxdirs="$(SPHINXDIRS)" \
> > 	+                --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
> > 	+                --paper=$(PAPER)
> > 
> > and let's discuss its pros/cons in separate.  
> 
> OK. This looks like the way forward to me.
> 
> I'll prepare a patch (set) and submit it after v6.18-rc1.

Ok, but as I said, IMO we're better served running the script
directly while debugging issues.

> In the meantime, I think we might be better off if we could root-cause
> the "latexmk bug" you repeatedly claimed you had observed, without providing
> any steps to reproduce.

This is on my todo list. 

With regards to the steps to reproduce you need to remove this from make:

	|| sh $(srctree)/scripts/check-variable-fonts.sh || exit

to ensure that errors will be handled by make.

As we depend on error handling logic, it means that you may need to build docs
with different configurations, different types of latex errors produced by
sphinx-build, and test with different versions of docutils, sphinx, xelatex and
latexmk, and hope to have a .tex file that has errors but still generate .pdf. 

Getting such issues is time consuming and it needs a certain amount of luck
to catch it, as you depend on changes at the *.rst files that will produce
a .tex file with the exact amount of spicy to produce such behavior.

With me, it only happens when I don't want it to happen ;-)

-

Another option would be to try to manually generate a defective *.tex file
that doesn't produce a critical error but still generates a.pdf output when
xelatex is called by hand.

Anyway, this requires a calm week where you don't have anything else
to do, and you're prepared to waste several hours trying to pick the
hot spot on away that it would be reproducible by others.

So, this has low priority.

> Let me continue in another thread.
> It would be far more interesting for me to (try to) figure out what you
> had observed.
> 
> Thanks,
> Akira
> 



Thanks,
Mauro