[v2] Upgrading uprobe and kprobe to their `multi` counterparts.

[RFC PATCH bpf-next v2 2/3] libbpf: Add FEAT_KPROBE_MULTI_LINK feature probe.

Posted by Varun R Mallya 2 days, 15 hours ago

Add FEAT_KPROBE_MULTI_LINK, similar to UPROBE_MULTI_LINK
by loading and creating a link for a small BPF program with
BPF_TRACE_KPROBE_MULTI as the expected attach type, and
then checking the success of the operation.

Signed-off-by: Varun R Mallya <varunrmallya@gmail.com>
---
 tools/lib/bpf/features.c        | 37 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf_internal.h |  2 ++
 2 files changed, 39 insertions(+)

diff --git a/tools/lib/bpf/features.c b/tools/lib/bpf/features.c
index adcad221c601..13227c9ea69d 100644
--- a/tools/lib/bpf/features.c
+++ b/tools/lib/bpf/features.c
@@ -424,6 +424,40 @@ static int probe_uprobe_multi_link(int token_fd)
 	return link_fd < 0 && err == -EINVAL;
 }
 
+static int probe_kprobe_multi_link(int token_fd)
+{
+	LIBBPF_OPTS(bpf_prog_load_opts, load_opts,
+		    .expected_attach_type = BPF_TRACE_KPROBE_MULTI,
+		    .token_fd = token_fd,
+		    .prog_flags = token_fd ? BPF_F_TOKEN_FD : 0,
+	);
+	LIBBPF_OPTS(bpf_link_create_opts, link_opts);
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	int prog_fd, link_fd, err;
+	const char *sym = "bpf_map_lookup_elem"; /* stable, always present */
+
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_KPROBE, NULL, "GPL",
+				insns, ARRAY_SIZE(insns), &load_opts);
+	if (prog_fd < 0)
+		return -errno;
+
+	/* attaching to a valid symbol should succeed */
+	link_opts.kprobe_multi.syms = &sym;
+	link_opts.kprobe_multi.cnt = 1;
+	link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_KPROBE_MULTI, &link_opts);
+	err = -errno;
+	if (link_fd >= 0)
+		close(link_fd);
+	close(prog_fd);
+	/* if kprobe_multi is supported, link creation either succeeds or
+	 * fails with something other than -EINVAL due to permissions,
+	 */
+	return link_fd >= 0 || (err != -EINVAL);
+}
+
 static int probe_kern_bpf_cookie(int token_fd)
 {
 	struct bpf_insn insns[] = {
@@ -658,6 +692,9 @@ static struct kern_feature_desc {
 	[FEAT_UPROBE_MULTI_LINK] = {
 		"BPF multi-uprobe link support", probe_uprobe_multi_link,
 	},
+	[FEAT_KPROBE_MULTI_LINK] = {
+		"BPF multi-kprobe link support", probe_kprobe_multi_link,
+	},
 	[FEAT_ARG_CTX_TAG] = {
 		"kernel-side __arg_ctx tag", probe_kern_arg_ctx_tag,
 	},
diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index 4bcb6ca69bb1..633106a687c7 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -388,6 +388,8 @@ enum kern_feature_id {
 	FEAT_SYSCALL_WRAPPER,
 	/* BPF multi-uprobe link support */
 	FEAT_UPROBE_MULTI_LINK,
+	/* BPF multi-kprobe link support */
+	FEAT_KPROBE_MULTI_LINK,
 	/* Kernel supports arg:ctx tag (__arg_ctx) for global subprogs natively */
 	FEAT_ARG_CTX_TAG,
 	/* Kernel supports '?' at the front of datasec names */
-- 
2.52.0

Re: [RFC PATCH bpf-next v2 2/3] libbpf: Add FEAT_KPROBE_MULTI_LINK feature probe.

Posted by Jiri Olsa 2 days, 11 hours ago

On Mon, Mar 30, 2026 at 04:30:18PM +0530, Varun R Mallya wrote:
> Add FEAT_KPROBE_MULTI_LINK, similar to UPROBE_MULTI_LINK
> by loading and creating a link for a small BPF program with
> BPF_TRACE_KPROBE_MULTI as the expected attach type, and
> then checking the success of the operation.
> 
> Signed-off-by: Varun R Mallya <varunrmallya@gmail.com>
> ---
>  tools/lib/bpf/features.c        | 37 +++++++++++++++++++++++++++++++++
>  tools/lib/bpf/libbpf_internal.h |  2 ++
>  2 files changed, 39 insertions(+)
> 
> diff --git a/tools/lib/bpf/features.c b/tools/lib/bpf/features.c
> index adcad221c601..13227c9ea69d 100644
> --- a/tools/lib/bpf/features.c
> +++ b/tools/lib/bpf/features.c
> @@ -424,6 +424,40 @@ static int probe_uprobe_multi_link(int token_fd)
>  	return link_fd < 0 && err == -EINVAL;
>  }
>  
> +static int probe_kprobe_multi_link(int token_fd)
> +{
> +	LIBBPF_OPTS(bpf_prog_load_opts, load_opts,
> +		    .expected_attach_type = BPF_TRACE_KPROBE_MULTI,
> +		    .token_fd = token_fd,
> +		    .prog_flags = token_fd ? BPF_F_TOKEN_FD : 0,
> +	);
> +	LIBBPF_OPTS(bpf_link_create_opts, link_opts);
> +	struct bpf_insn insns[] = {
> +		BPF_MOV64_IMM(BPF_REG_0, 0),
> +		BPF_EXIT_INSN(),
> +	};
> +	int prog_fd, link_fd, err;
> +	const char *sym = "bpf_map_lookup_elem"; /* stable, always present */
> +
> +	prog_fd = bpf_prog_load(BPF_PROG_TYPE_KPROBE, NULL, "GPL",
> +				insns, ARRAY_SIZE(insns), &load_opts);
> +	if (prog_fd < 0)
> +		return -errno;
> +
> +	/* attaching to a valid symbol should succeed */
> +	link_opts.kprobe_multi.syms = &sym;
> +	link_opts.kprobe_multi.cnt = 1;

maybe we could set cnt > MAX_KPROBE_MULTI_CNT and check if the fail
is E2BIG ? this could save some cycles, but haven't tried that

jirka

> +	link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_KPROBE_MULTI, &link_opts);
> +	err = -errno;
> +	if (link_fd >= 0)
> +		close(link_fd);
> +	close(prog_fd);
> +	/* if kprobe_multi is supported, link creation either succeeds or
> +	 * fails with something other than -EINVAL due to permissions,
> +	 */
> +	return link_fd >= 0 || (err != -EINVAL);
> +}
> +

SNIP

Re: [RFC PATCH bpf-next v2 2/3] libbpf: Add FEAT_KPROBE_MULTI_LINK feature probe.

Posted by Varun R Mallya 16 hours ago

On Mon, Mar 30, 2026 at 04:52:46PM +0200, Jiri Olsa wrote:
> > +
> > +	/* attaching to a valid symbol should succeed */
> > +	link_opts.kprobe_multi.syms = &sym;
> > +	link_opts.kprobe_multi.cnt = 1;
> 
> maybe we could set cnt > MAX_KPROBE_MULTI_CNT and check if the fail
> is E2BIG ? this could save some cycles, but haven't tried that
> 
> jirka
>
Thanks for the suggestion!
I tested this out on an older kernel (<5.8) to see if it fails with a different error
code and it did. On newer kernels it always fails with -E2BIG.
Implemented this in my latest version (which I will send out in a while).

> > +	link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_KPROBE_MULTI, &link_opts);
> > +	err = -errno;
> > +	if (link_fd >= 0)
> > +		close(link_fd);
> > +	close(prog_fd);
> > +	/* if kprobe_multi is supported, link creation either succeeds or
> > +	 * fails with something other than -EINVAL due to permissions,
> > +	 */
> > +	return link_fd >= 0 || (err != -EINVAL);
> > +}
> > +
> 
> SNIP

Re: [RFC PATCH bpf-next v2 2/3] libbpf: Add FEAT_KPROBE_MULTI_LINK feature probe.

Posted by Leon Hwang 2 days, 12 hours ago

On 2026/3/30 19:00, Varun R Mallya wrote:
> diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> index 4bcb6ca69bb1..633106a687c7 100644
> --- a/tools/lib/bpf/libbpf_internal.h
> +++ b/tools/lib/bpf/libbpf_internal.h
> @@ -388,6 +388,8 @@ enum kern_feature_id {
>  	FEAT_SYSCALL_WRAPPER,
>  	/* BPF multi-uprobe link support */
>  	FEAT_UPROBE_MULTI_LINK,
> +	/* BPF multi-kprobe link support */
> +	FEAT_KPROBE_MULTI_LINK,

Even though it seems good to put FEAT_KPROBE_MULTI_LINK here, better to
put it at last to keep backwards compatibility?

When users compile application using old libbpf and run it against such
new libbpf.so, will they get incorrect feature support?

Thanks,
Leon

>  	/* Kernel supports arg:ctx tag (__arg_ctx) for global subprogs natively */
>  	FEAT_ARG_CTX_TAG,
>  	/* Kernel supports '?' at the front of datasec names */

Re: [RFC PATCH bpf-next v2 2/3] libbpf: Add FEAT_KPROBE_MULTI_LINK feature probe.

Posted by Varun R Mallya 16 hours ago

On Mon, Mar 30, 2026 at 10:42:57PM +0800, Leon Hwang wrote:
> On 2026/3/30 19:00, Varun R Mallya wrote:
> > diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> > index 4bcb6ca69bb1..633106a687c7 100644
> > --- a/tools/lib/bpf/libbpf_internal.h
> > +++ b/tools/lib/bpf/libbpf_internal.h
> > @@ -388,6 +388,8 @@ enum kern_feature_id {
> >  	FEAT_SYSCALL_WRAPPER,
> >  	/* BPF multi-uprobe link support */
> >  	FEAT_UPROBE_MULTI_LINK,
> > +	/* BPF multi-kprobe link support */
> > +	FEAT_KPROBE_MULTI_LINK,
> 
> Even though it seems good to put FEAT_KPROBE_MULTI_LINK here, better to
> put it at last to keep backwards compatibility?
> 
> When users compile application using old libbpf and run it against such
> new libbpf.so, will they get incorrect feature support?
> 
> Thanks,
> Leon
Changing in v3. Thanks!
> >  	/* Kernel supports arg:ctx tag (__arg_ctx) for global subprogs natively */
> >  	FEAT_ARG_CTX_TAG,
> >  	/* Kernel supports '?' at the front of datasec names */
>