From nobody Sun Feb 8 10:16:55 2026 Received: from out-188.mta1.migadu.com (out-188.mta1.migadu.com [95.215.58.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C873477E26 for ; Tue, 20 Jan 2026 22:27:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768948044; cv=none; b=C7ih1veyl2AgDAn9PUeKisYIQUuuHQt5JPEBMb+1U1d1V98CGF3fwId8QAYIIQc7A4w9aEycaZefrrf40BwozDn4cyZ/Re2yUImzJ25CdPzqhVyr66dipp2LImOLtU4BSU0p1tISLyZWIe69k6hvuzg4Q30Qq8kWEz+Ub9VjPAk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768948044; c=relaxed/simple; bh=fZd7cddqiPk6zPaSlVrEHfeGpkBMCpfB6PeLJZ2//Fg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HClIQGKs5RxyPWDco6mlfgv1nRQfHMIT/4Q8GShYJgA8CbRpI0GJn/Ie8kRhQza9cFI0wB8GXVCyjfFyTAEe7Va/qKofnGTGbffoWRkWGRR7HgdkaGLBkeepoSXEd9yzoE+/Hj9i54TZUjrb2fiq8yRlTgpoQ6etpRpe5EPt3Bo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=b9+Xt2Si; arc=none smtp.client-ip=95.215.58.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="b9+Xt2Si" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1768948034; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TWe3QQvOop6uqL2wzPpgOCmxLOQh8a9rRjwy3D8L9Fo=; b=b9+Xt2SibU/mO19AYnQwE79UYSHSrL/tYBg5vzT5ar37dk0ecf1PN8MS6vJA10fdJ0jWb9 BqUQpjFcBZV/27m6IaSMzpHTK7F+Oi5ZlUg9smYkGxbYtCDvHnUp4vMxLTQ8ynPA/bGKe2 r5RXSflylTQ0OcdcG9l5hJ43fV8wi0I= From: Ihor Solodrai To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman Cc: Mykyta Yatsenko , Tejun Heo , Alan Maguire , Benjamin Tissoires , Jiri Kosina , Amery Hung , bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-input@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH bpf-next v3 03/13] bpf: Verifier support for KF_IMPLICIT_ARGS Date: Tue, 20 Jan 2026 14:26:28 -0800 Message-ID: <20260120222638.3976562-4-ihor.solodrai@linux.dev> In-Reply-To: <20260120222638.3976562-1-ihor.solodrai@linux.dev> References: <20260120222638.3976562-1-ihor.solodrai@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" A kernel function bpf_foo marked with KF_IMPLICIT_ARGS flag is expected to have two associated types in BTF: * `bpf_foo` with a function prototype that omits implicit arguments * `bpf_foo_impl` with a function prototype that matches the kernel declaration of `bpf_foo`, but doesn't have a ksym associated with its name In order to support kfuncs with implicit arguments, the verifier has to know how to resolve a call of `bpf_foo` to the correct BTF function prototype and address. To implement this, in add_kfunc_call() kfunc flags are checked for KF_IMPLICIT_ARGS. For such kfuncs a BTF func prototype is adjusted to the one found for `bpf_foo_impl` (func_name + "_impl" suffix, by convention) function in BTF. This effectively changes the signature of the `bpf_foo` kfunc in the context of verification: from one without implicit args to the one with full argument list. The values of implicit arguments by design are provided by the verifier, and so they can only be of particular types. In this patch the only allowed implicit arg type is a pointer to struct bpf_prog_aux. In order for the verifier to correctly set an implicit bpf_prog_aux arg value at runtime, is_kfunc_arg_prog() is extended to check for the arg type. At a point when prog arg is determined in check_kfunc_args() the kfunc with implicit args already has a prototype with full argument list, so the existing value patch mechanism just works. If a new kfunc with KF_IMPLICIT_ARG is declared for an existing kfunc that uses a __prog argument (a legacy case), the prototype substitution works in exactly the same way, assuming the kfunc follows the _impl naming convention. The difference is only in how _impl prototype is added to the BTF, which is not the verifier's concern. See a subsequent resolve_btfids patch for details. __prog suffix is still supported at this point, but will be removed in a subsequent patch, after current users are moved to KF_IMPLICIT_ARGS. Introduction of KF_IMPLICIT_ARGS revealed an issue with zero-extension tracking, because an explicit rX =3D 0 in place of the verifier-supplied argument is now absent if the arg is implicit (the BPF prog doesn't pass a dummy NULL anymore). To mitigate this, reset the subreg_def of all caller saved registers in check_kfunc_call() [1]. [1] https://lore.kernel.org/bpf/b4a760ef828d40dac7ea6074d39452bb0dc82caa.ca= mel@gmail.com/ Acked-by: Eduard Zingerman Signed-off-by: Ihor Solodrai --- include/linux/btf.h | 1 + kernel/bpf/verifier.c | 58 ++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 55 insertions(+), 4 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index a2f4f383f5b6..48108471c5b1 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -78,6 +78,7 @@ #define KF_ARENA_RET (1 << 13) /* kfunc returns an arena pointer */ #define KF_ARENA_ARG1 (1 << 14) /* kfunc takes an arena pointer as its f= irst argument */ #define KF_ARENA_ARG2 (1 << 15) /* kfunc takes an arena pointer as its s= econd argument */ +#define KF_IMPLICIT_ARGS (1 << 16) /* kfunc has implicit arguments supplie= d by the verifier */ =20 /* * Tag marking a kernel function as a kfunc. This is meant to minimize the diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 87f0e113b356..adc24a2ce5b6 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3271,6 +3271,34 @@ static struct btf *find_kfunc_desc_btf(struct bpf_ve= rifier_env *env, s16 offset) return btf_vmlinux ?: ERR_PTR(-ENOENT); } =20 +#define KF_IMPL_SUFFIX "_impl" + +static const struct btf_type *find_kfunc_impl_proto(struct bpf_verifier_en= v *env, + struct btf *btf, + const char *func_name) +{ + char *buf =3D env->tmp_str_buf; + const struct btf_type *func; + s32 impl_id; + int len; + + len =3D snprintf(buf, TMP_STR_BUF_LEN, "%s%s", func_name, KF_IMPL_SUFFIX); + if (len < 0 || len >=3D TMP_STR_BUF_LEN) { + verbose(env, "function name %s%s is too long\n", func_name, KF_IMPL_SUFF= IX); + return NULL; + } + + impl_id =3D btf_find_by_name_kind(btf, buf, BTF_KIND_FUNC); + if (impl_id <=3D 0) { + verbose(env, "cannot find function %s in BTF\n", buf); + return NULL; + } + + func =3D btf_type_by_id(btf, impl_id); + + return btf_type_by_id(btf, func->type); +} + static int fetch_kfunc_meta(struct bpf_verifier_env *env, s32 func_id, s16 offset, @@ -3308,7 +3336,16 @@ static int fetch_kfunc_meta(struct bpf_verifier_env = *env, } =20 func_name =3D btf_name_by_offset(btf, func->name_off); - func_proto =3D btf_type_by_id(btf, func->type); + + /* + * An actual prototype of a kfunc with KF_IMPLICIT_ARGS flag + * can be found through the counterpart _impl kfunc. + */ + if (kfunc_flags && (*kfunc_flags & KF_IMPLICIT_ARGS)) + func_proto =3D find_kfunc_impl_proto(env, btf, func_name); + else + func_proto =3D btf_type_by_id(btf, func->type); + if (!func_proto || !btf_type_is_func_proto(func_proto)) { verbose(env, "kernel function btf_id %d does not have a valid func_proto= \n", func_id); @@ -12174,9 +12211,11 @@ static bool is_kfunc_arg_irq_flag(const struct btf= *btf, const struct btf_param return btf_param_match_suffix(btf, arg, "__irq_flag"); } =20 +static bool is_kfunc_arg_prog_aux(const struct btf *btf, const struct btf_= param *arg); + static bool is_kfunc_arg_prog(const struct btf *btf, const struct btf_para= m *arg) { - return btf_param_match_suffix(btf, arg, "__prog"); + return btf_param_match_suffix(btf, arg, "__prog") || is_kfunc_arg_prog_au= x(btf, arg); } =20 static bool is_kfunc_arg_scalar_with_name(const struct btf *btf, @@ -12207,6 +12246,7 @@ enum { KF_ARG_WORKQUEUE_ID, KF_ARG_RES_SPIN_LOCK_ID, KF_ARG_TASK_WORK_ID, + KF_ARG_PROG_AUX_ID }; =20 BTF_ID_LIST(kf_arg_btf_ids) @@ -12218,6 +12258,7 @@ BTF_ID(struct, bpf_rb_node) BTF_ID(struct, bpf_wq) BTF_ID(struct, bpf_res_spin_lock) BTF_ID(struct, bpf_task_work) +BTF_ID(struct, bpf_prog_aux) =20 static bool __is_kfunc_ptr_arg_type(const struct btf *btf, const struct btf_param *arg, int type) @@ -12298,6 +12339,11 @@ static bool is_kfunc_arg_callback(struct bpf_verif= ier_env *env, const struct btf return true; } =20 +static bool is_kfunc_arg_prog_aux(const struct btf *btf, const struct btf_= param *arg) +{ + return __is_kfunc_ptr_arg_type(btf, arg, KF_ARG_PROG_AUX_ID); +} + /* Returns true if struct is composed of scalars, 4 levels of nesting allo= wed */ static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env, const struct btf *btf, @@ -14177,8 +14223,12 @@ static int check_kfunc_call(struct bpf_verifier_en= v *env, struct bpf_insn *insn, } } =20 - for (i =3D 0; i < CALLER_SAVED_REGS; i++) - mark_reg_not_init(env, regs, caller_saved[i]); + for (i =3D 0; i < CALLER_SAVED_REGS; i++) { + u32 regno =3D caller_saved[i]; + + mark_reg_not_init(env, regs, regno); + regs[regno].subreg_def =3D DEF_NOT_SUBREG; + } =20 /* Check return type */ t =3D btf_type_skip_modifiers(desc_btf, meta.func_proto->type, NULL); --=20 2.52.0