Since Linux 7.0, kfuncs annotated with KF_IMPLICIT_ARGS require pahole
v1.26 or later. Without it, such kfuncs have incorrect BTF prototypes in
vmlinux, causing BPF programs to fail with 'func_proto incompatible with
vmlinux' error.

This affects all sched_ext kfuncs (e.g. scx_bpf_create_dsq,
scx_bpf_dispatch) and other KF_IMPLICIT_ARGS kfuncs across the kernel.
Ubuntu 24.04 LTS ships pahole v1.25 by default, causing 23/30 sched_ext
selftests to fail on affected systems.

Document this requirement in Documentation/process/changes.rst so users
understand the failure mode and can upgrade pahole appropriately.

Signed-off-by: zhidao su <suzhidao@xiaomi.com>
---
 Documentation/process/changes.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 6b373e193548..141a4576c24d 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -145,6 +145,11 @@ Since Linux 5.2, if CONFIG_DEBUG_INFO_BTF is selected, the build system
 generates BTF (BPF Type Format) from DWARF in vmlinux, a bit later from kernel
 modules as well.  This requires pahole v1.22 or later.
 
+Since Linux 7.0, kfuncs annotated with KF_IMPLICIT_ARGS require pahole v1.26
+or later.  Without it, such kfuncs will have incorrect BTF prototypes in
+vmlinux, causing BPF programs to fail to load with a "func_proto incompatible
+with vmlinux" error.  Many sched_ext kfuncs are affected.
+
 It is found in the 'dwarves' or 'pahole' distro packages or from
 https://fedorapeople.org/~acme/dwarves/.
 
-- 
2.43.0

On Mon, Mar 23, 2026 at 11:22 PM zhidao su <soolaugust@gmail.com> wrote:
>
> Since Linux 7.0, kfuncs annotated with KF_IMPLICIT_ARGS require pahole
> v1.26 or later. Without it, such kfuncs have incorrect BTF prototypes in
> vmlinux, causing BPF programs to fail with 'func_proto incompatible with
> vmlinux' error.
>
> This affects all sched_ext kfuncs (e.g. scx_bpf_create_dsq,
> scx_bpf_dispatch) and other KF_IMPLICIT_ARGS kfuncs across the kernel.
> Ubuntu 24.04 LTS ships pahole v1.25 by default, causing 23/30 sched_ext
> selftests to fail on affected systems.

I don't think that's true.
At least when implicit args were designed the goal was to avoid
pahole dependencies.

Please share exact steps to reproduce.

Updating doc is definitely not an answer.

pw-bot: cr

Since Linux 7.0, kfuncs annotated with KF_IMPLICIT_ARGS require pahole
v1.26 or later.  Without it, such kfuncs will have incorrect BTF
prototypes in vmlinux, causing BPF programs to fail to load with a
"func_proto incompatible with vmlinux" error.  Many sched_ext kfuncs
are affected (e.g. scx_bpf_create_dsq, scx_bpf_kick_cpu).

The root cause: scripts/Makefile.btf passes --btf_features=decl_tag_kfuncs
to pahole only when pahole >= 1.26.  Without that flag, pahole emits no
DECL_TAG BTF entries for __bpf_kfunc-annotated functions.  As a result,
resolve_btfids/main.c::collect_kfuncs() finds no bpf_kfunc DECL_TAGs,
short-circuits at line 1002, and btf2btf() never creates the _impl
variants or strips the implicit 'aux' argument from the visible proto.
The vmlinux BTF retains the 3-param prototype while BPF programs declare
the 2-param version, triggering the mismatch.

Raise the minimum version in the requirements table from 1.22 to 1.26
and add a note explaining the failure mode, so users understand why
their BPF programs fail on distributions shipping pahole v1.25 (e.g.
Ubuntu 24.04 LTS).

Suggested-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: zhidao su <suzhidao@xiaomi.com>
---
 Documentation/process/changes.rst | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 6b373e193548..02068d72a101 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -38,7 +38,7 @@ bash                   4.2              bash --version
 binutils               2.30             ld -v
 flex                   2.5.35           flex --version
 bison                  2.0              bison --version
-pahole                 1.22             pahole --version
+pahole                 1.26             pahole --version
 util-linux             2.10o            mount --version
 kmod                   13               depmod -V
 e2fsprogs              1.41.4           e2fsck -V
@@ -145,6 +145,11 @@ Since Linux 5.2, if CONFIG_DEBUG_INFO_BTF is selected, the build system
 generates BTF (BPF Type Format) from DWARF in vmlinux, a bit later from kernel
 modules as well.  This requires pahole v1.22 or later.
 
+Since Linux 7.0, kfuncs annotated with KF_IMPLICIT_ARGS require pahole v1.26
+or later.  Without it, such kfuncs will have incorrect BTF prototypes in
+vmlinux, causing BPF programs to fail to load with a "func_proto incompatible
+with vmlinux" error.  Many sched_ext kfuncs are affected.
+
 It is found in the 'dwarves' or 'pahole' distro packages or from
 https://fedorapeople.org/~acme/dwarves/.
 
-- 
2.43.0

Applied to sched_ext/for-7.1.

Thanks.

-- 
tejun

zhidao su <soolaugust@gmail.com> writes:

> Since Linux 7.0, kfuncs annotated with KF_IMPLICIT_ARGS require pahole
> v1.26 or later. Without it, such kfuncs have incorrect BTF prototypes in
> vmlinux, causing BPF programs to fail with 'func_proto incompatible with
> vmlinux' error.
>
> This affects all sched_ext kfuncs (e.g. scx_bpf_create_dsq,
> scx_bpf_dispatch) and other KF_IMPLICIT_ARGS kfuncs across the kernel.
> Ubuntu 24.04 LTS ships pahole v1.25 by default, causing 23/30 sched_ext
> selftests to fail on affected systems.
>
> Document this requirement in Documentation/process/changes.rst so users
> understand the failure mode and can upgrade pahole appropriately.
>
> Signed-off-by: zhidao su <suzhidao@xiaomi.com>
> ---
>  Documentation/process/changes.rst | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> index 6b373e193548..141a4576c24d 100644
> --- a/Documentation/process/changes.rst
> +++ b/Documentation/process/changes.rst
> @@ -145,6 +145,11 @@ Since Linux 5.2, if CONFIG_DEBUG_INFO_BTF is selected, the build system
>  generates BTF (BPF Type Format) from DWARF in vmlinux, a bit later from kernel
>  modules as well.  This requires pahole v1.22 or later.
>  
> +Since Linux 7.0, kfuncs annotated with KF_IMPLICIT_ARGS require pahole v1.26
> +or later.  Without it, such kfuncs will have incorrect BTF prototypes in
> +vmlinux, causing BPF programs to fail to load with a "func_proto incompatible
> +with vmlinux" error.  Many sched_ext kfuncs are affected.

This seems like reasonable information, but is there a reason to not
just raise the minimum pahole version to 1.26 and be done with it?

Thanks,

jon