[PATCH v1 05/30] docs: reporting-issues: outline why reporting is complicated

Thorsten Leemhuis posted 30 patches 3 months, 2 weeks ago
[PATCH v1 05/30] docs: reporting-issues: outline why reporting is complicated
Posted by Thorsten Leemhuis 3 months, 2 weeks ago
Replace the closing words with a section that describes why reporting
Linux kernel bugs is more complicated than in other FLOSS projects.

Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
---
 .../admin-guide/reporting-issues.rst          | 67 ++++++++++++++++---
 1 file changed, 59 insertions(+), 8 deletions(-)

diff --git a/Documentation/admin-guide/reporting-issues.rst b/Documentation/admin-guide/reporting-issues.rst
index 745e698cb6be8b..2629fde3aa4b8f 100644
--- a/Documentation/admin-guide/reporting-issues.rst
+++ b/Documentation/admin-guide/reporting-issues.rst
@@ -1756,14 +1756,65 @@ But do not worry too much about all of this, a lot of drivers have active
 maintainers who are quite interested in fixing as many issues as possible.
 
 
-Closing words
-=============
-
-Compared with other Free/Libre & Open Source Software it's hard to report
-issues to the Linux kernel developers: the length and complexity of this
-document and the implications between the lines illustrate that. But that's how
-it is for now. The main author of this text hopes documenting the state of the
-art will lay some groundwork to improve the situation over time.
+Why reporting Linux kernel bugs is somewhat complicated
+-------------------------------------------------------
+
+The Linux kernel's development model differs from typical Open Source projects
+in a few important aspects. Four of those complicate bug reporting:
+
+1. Developers are free to solely focus on the latest mainline codebase.
+
+2. The 'stable team' maintains the stable and longterm kernel series, but is not
+   allowed to fix many bugs just there if they happen in mainline, too.
+
+3. There is no central bug tracker.
+
+4. Most kernels used in Linux distributions are totally unsuitable for reporting
+   bugs upstream.
+
+Due to the first aspect, some of the developers ignore or react coldly to
+reports about bugs in, say, Linux 6.1 when 6.2-rc1 is already out.
+
+The combination of the first and the second aspect is why some of the
+developers are unwilling to look into reports with stable or longterm kernels:
+the problem might never have happened in the code they work on, for example
+because the stable team did something wrong between 6.1.1 and 6.1.2.
+
+The stable team due to those two aspects is often in a similar but opposite
+situation when it comes to bugs reported using a kernel like 6.1.2: If that
+issue already happened in 6.1 and still happens in the latest mainline kernel,
+then it must be fixed there first. That is among the reasons why reporters in
+the end always have to check if mainline is affected, as the stable team often
+is the wrong point of contact, unless it is a series specific regression.
+
+There are various reasons why no central bug tracker exists. They, among others,
+were not a thing yet when Linux started, which is why reporting my email was
+the norm initially -- and still is, as quite a few developers prefer to handle
+all aspects of kernel development via email. Some, on the other hand, saw
+benefits in trackers and set up bugzilla.kernel.org, which due to the
+aforementioned aspect never became mandatory and has some problematic aspects;
+this is why it frequently does not even forward newly filed reports to the
+appropriate developers. Yet other developers prefer the comfort of Git forges
+for development and issue tracking; but subsystems use various forges, so those
+trackers are spread over the web.
+
+The fourth aspect is explained in the second point of the reference section
+already: Old codebases, modified sources, and add-ons lead to bugs that were
+fixed a long time ago already or never happened upstream in the first place.
+These are problems for many other software packaged by Linux distributions as
+well. But there it is usually a smaller problem, as the modifications and
+extensions distributors apply to the kernel are often much bigger and more
+dangerous; the kernel also changes way more quickly, is a lot more
+complex, and naturally more fragile. Due to these aspects, many developers
+expect reports to be based on fresh and vanilla kernels. Furthermore, most of
+them receive way more bug reports than they are able to handle, which is
+why they prioritize the ones that look more promising.
+
+Reporting a bug due to these and other unmentioned aspects is harder than in
+other Free/Libre & Open Source Software projects -- the complexity of this
+document proves that. But that is the state of affairs for now. The primary
+author of this text hopes documenting it will lay some groundwork to improve
+the situation over time.
 
 
 ..
-- 
2.51.0
Re: [PATCH v1 05/30] docs: reporting-issues: outline why reporting is complicated
Posted by Jonathan Corbet 3 months, 2 weeks ago
Thorsten Leemhuis <linux@leemhuis.info> writes:

> Replace the closing words with a section that describes why reporting
> Linux kernel bugs is more complicated than in other FLOSS projects.
>
> Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
> ---
>  .../admin-guide/reporting-issues.rst          | 67 ++++++++++++++++---
>  1 file changed, 59 insertions(+), 8 deletions(-)

So the text is OK but ... this is now the second section that is
essentially a long apology for the kernel process being so difficult.
It seems redundant with the other text, and I'm not convinced we need
it.

Again, length is an impediment to getting people to actually read this
stuff; we should be trying to be as concise as we can.  Do we really
need this?

Thanks,

jon
Re: [PATCH v1 05/30] docs: reporting-issues: outline why reporting is complicated
Posted by Thorsten Leemhuis 3 weeks, 5 days ago
On 10/27/25 18:44, Jonathan Corbet wrote:
> Thorsten Leemhuis <linux@leemhuis.info> writes:
> 
>> Replace the closing words with a section that describes why reporting
>> Linux kernel bugs is more complicated than in other FLOSS projects.
>>
>> Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
>> ---
>>  .../admin-guide/reporting-issues.rst          | 67 ++++++++++++++++---
>>  1 file changed, 59 insertions(+), 8 deletions(-)
> 
> So the text is OK but ... this is now the second section that is
> essentially a long apology for the kernel process being so difficult.
> It seems redundant with the other text, and I'm not convinced we need
> it.
> 
> Again, length is an impediment to getting people to actually read this
> stuff; we should be trying to be as concise as we can.  Do we really
> need this?

I thought about this for a while, but in the end I this section and the
one from 4/30 are worth it. Let me explain my line of thought:

* Yes, they make the document a longer, but it's in an appendix, which
makes it relatively obvious that these are optional background details
you don't have to read.

* You in your reply to 3/30 asked to "Consider also soliciting patches
to improve it" -- something I in the context had found obvious and not
worth mentioning, as I thought, "everybody that knows how to send
patches will do so without us mentioning it here". But for these two
sections it's different: is stuff that is obvious for us, but not
obvious for many non-developers (at least from what I see when
processing bugs/regressions reports= and relevant for them. That's
because the kernel IMHO is not "an open-source program like any other",
as you mentioned in a reply to 4/30:

- Other FLOSS software is relatively independent, but the kernel is
mostly drivers -- so without the right hardware, there sometimes is
nothing that developers can do in case of bugs. We also normally don't
remove orphaned code due to the "no regressions" rule, unless it's
getting really old and/or in the way, so there is nothing anybody will
do about a bug report -- which is unusual. The stuff in 4/30 explains
such aspects.

- The kernel development model is different in some important aspects,
which can be even confusing for people familiar with FLOSS practices.
Like the "mainline developers might not care about bugs in
stable/longterm kernels, while the stable team will leave things to the
mainline developers if the problem happens there, too". This is rare and
IMHO needs to be explained somewhere, as then we can point people to
this explanation that otherwise will just feel pushed around and become
annoyed by the kernel (some obviously will nevertheless feel that, but
that way we can get at least some people to understand things and/or on
our side).

Ciao, Thorsten
Re: [PATCH v1 05/30] docs: reporting-issues: outline why reporting is complicated
Posted by Thorsten Leemhuis 3 weeks, 4 days ago
On 1/13/26 17:07, Thorsten Leemhuis wrote:

> I thought about this for a while, but in the end I this section and the
> one from 4/30 are worth it.

And having slept about this once more, I after my reply yesterday now
think it might be wise to merge the important parts from 4/30 into this
patch -- that should get things shorter and still get across chat I care
about. Sorry, took me a moment to realize that.

Ciao, Thorsten
Re: [PATCH v1 05/30] docs: reporting-issues: outline why reporting is complicated
Posted by Thorsten Leemhuis 3 weeks, 2 days ago

On 1/14/26 06:02, Thorsten Leemhuis wrote:
> On 1/13/26 17:07, Thorsten Leemhuis wrote:
> 
>> I thought about this for a while, but in the end I this section and the
>> one from 4/30 are worth it.
> 
> And having slept about this once more, I after my reply yesterday now
> think it might be wise to merge the important parts from 4/30 into this
> patch -- that should get things shorter and still get across chat I care
> about. Sorry, took me a moment to realize that.

FYI, wrote something new (see below) which is even a few lines shorter
than what is in the document right now, yeah. :-D

Ciao, Thorsten

---

Appendix: Why it's harder to report kernel bugs
===============================================

The developers are well aware that reporting Linux kernel bugs is harder
than in other Free/Libre Open Source Projects. Many reasons for that lie
in the nature of the kernel, its development model, and how the world
uses it:

* *Most kernels of Linux distributions are totally unsuitable for
reporting bugs upstream.* The reference section above already explained
this in detail: outdated codebases as well as modifications and add-ons
lead to kernel bugs that were fixed upstream a long time ago or never
happened there in the first place. Developers of other Open Source
software face these problems as well, but the situation is a lot worse
when it comes to the Linux kernel, as the changes and their impact are
much more severe -- which is why many developers expect reports with
kernels built from fresh and nearly unmodified sources.

* *Bugs often only occur in a special environment.* That is because
Linux is mostly drivers and can be used in a multitude of ways.
Developers often do not have a matching setup at hand -- and therefore
frequently have to rely on bug reporters for isolating a bug's cause and
testing proposed fixes.

* *The kernel has hundreds of maintainers, and all-rounders are
extremely rare.* This is also due to the multitude of drivers and
features, which is why many know little about areas related to their
code and even less about unrelated.

* *It is hard finding where to report issues to, among others, due to
the lack of a central bug tracker.* This is something even some kernel
developers dislike, but that's the situation everyone has to deal with
currently.

* *Stable and longterm kernels are primarily maintained by a dedicated
'stable team', which usually is forbidden to fix bugs just there that
happen in mainline, too.* The team thus will almost always want to know
if mainline is also affected when someone reports a bug, say, using
Linux 6.1.2: if it already happened in 6.1 and still occurs with
6.2-rc1, it is best left to the regular developers, as they know the
area best.

* *Kernel developers are free to solely focus on the latest mainline
kernel.* Some developers, therefore, ignore or react coldly to reports
about bugs in, say, Linux 6.1 when 6.2-rc1 is already out; a similar
reaction might occur when it comes to reports with 6.1.1 or 6.1.2, as
the stable team might have caused the breakage.

* *There might be nobody to help.* That can happen due to the lack of
hardware documentation -- for example, when a driver was built using
reverse engineering or taken over by spare-time developers when the
hardware manufacturer left it behind. Other times there is nobody to
even report bugs to: when maintainers move on without a replacement,
their code often remains as long as it's useful.

Some of these aspects could be improved to facilitate bug reporting --
many Linux kernel developers are well aware of this and would be glad if
a few individuals or an entity would make this their mission.