From nobody Thu Apr 2 12:07:16 2026 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0ACDF36EAA4 for ; Sun, 29 Mar 2026 14:05:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793127; cv=none; b=HHCTBVIzRhT/yYMyVr4T0Dr6XcwN5J4oqCtama1xW7bJx1yeyMMCmo6eWkPydxAj+T+7Il8wvQAJVX1au0zQLvpXtsIWyywXScwmh7CjpKTnT2YWDIDSPKDoGv/Ak7vkBAvtamVb5Md8qAlGY5D6y15m8gqNtj4jRUHCAu4mfeE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793127; c=relaxed/simple; bh=4gRW1+mtRMcdZfAHAzXm2KNnewoZoCSS46EQVHhhBP4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i9UksDMSvKDvuNA7itR0g/4yaoIuzJFR/kRUWGzn5X3rQ7EA4gp1A5xlzZNe22pz5xpFXnuEDtEU9vowCe9nMnX2NZs1ygzlknEwM9dIQ3LlXvEmOsrFCZshn42y28y2Ol0ZKvUAuLE9dRRlRhsAg3d0QdB0extVI6LOEncI8ds= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CsqBWlf/; arc=none smtp.client-ip=209.85.216.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CsqBWlf/" Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-35c1d101355so1525806a91.1 for ; Sun, 29 Mar 2026 07:05:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774793125; x=1775397925; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mi9szT9Syd9CFHAlHUGFYC/MCx4DDJNTxIJ9cbtZIrc=; b=CsqBWlf/TJjr0wVtBvbWZ5At3LF6/QmJjR6a0TOMhtQJ8zyKgqZkYjYyo3KXw/MQEH Nyn0nyOn22yAfoXAcFAkJ2RkmNn0kwQQX7JMUn9KSlJJ7qx9lp/lIyneRS6pj5kotAlH fKu6gqiqPCTLFy+UL9/8lZbSs3D0wGu4toU0d6NCBt2TtXkIHYhfur1/td5GguN+vz3l 1twyGiIzGzO6qEbPsS3M3XtIK1XQHGNpDrutVJGTNC4ugHHJLtrY5E4KId2HV8cb7uNh +etHvajuyQBZELZFTq4l+cz3PgHQqJKDEKP69aVRbc/bvXxVqp/15bkBmV0G4yKQxbQ6 kZUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774793125; x=1775397925; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=mi9szT9Syd9CFHAlHUGFYC/MCx4DDJNTxIJ9cbtZIrc=; b=sguSidZ+LZ8lWKZbUIRzy42KVIapzVn87HZ5SbGDJnmrRCJsxFbUwfgE/v15qXz9tw OaHMqyuy3DU5KtX8jnuv8LirxwtQNoUfZXkSxU3TfgWUcICrQOGQgS8GE6KqxiGPj0U/ cxhp/KSq6keCnwrZMngW1xA3R0tulufj5PeGhHIoAylihSIdQ4+lkotya/4exBsI6fkn B/V1Mt4P7LBaGCRu7D/ltDj7L+CCXovGF59lBWcaicdMhCW3xoDn/O7zZf6g1BgutVvp PAS//u6CClhV8L3ThNpfxSfTQbj/QiBvsyScFCHrrIKWvWN2Gg0f2hSRnz8eySn4cw07 UK1Q== X-Forwarded-Encrypted: i=1; AJvYcCUNwZut+iJKmEQSgeGZxZdkwIU4QeU0NYHtBuKhZ3rhZ0BfYtM8nC9qdDuTmzAZb1YcdBa8K+q3Wx+Cj54=@vger.kernel.org X-Gm-Message-State: AOJu0Yx9AXVeRiCea0bG4nVZ7dpo/V2iR+hDz/h6945Tu9lUkpEMUjZ9 7MvLfaDYnZ+kiZGzRh+WBDTTuqwVFMYt5DnbfO/+J+rDi6zOFWgm+qFp X-Gm-Gg: ATEYQzw3gvSCg6GzAtjPSoSz2T16Ri8KKBDdKZwm0dHHKirrYAnOkNY4dkNHlnUwdnL XceZgZ+jMNecyfvud0UXsQdpXC97CRALaS+l2E2dlkEwhKAHgMQ5PhIc8+/PwADTvSZzAmDSCcO zvSFGTa+jvHjn+MXJOnJlmZMyUy9LOfDM3Tn8vbzjgtyEXp51JqvRgqwNcUwNJyDESrY0WW8YwL L/R9iq+bTMyzD6DEfJrEoHrduhGNOn56TlUt+yIR/MF8c/kY9i1hoxTLv0u+dtMGPHTyMI+4F9E 6YmtU4nt9b5NPseI1JzqY0IYkqJKXYoeTt0YALQb8mcnXvhohNjDV4Dbd9N1L0qEeB5fQ/TEmJn uX/iDHZG+cWn4kCzuncTGJiYts1UIa/kiRB7zRs9X6CxVRZpNkpBgE48ZbQxM4y1taER8/hvokA lxIW8RBNo9exvdA6CE9YwCj/y7tXJcB8KvttWb5J1OnfFTHLt18wLzngHP+KsGXobp X-Received: by 2002:a17:90a:1048:b0:35d:93ff:2854 with SMTP id 98e67ed59e1d1-35d93ff2b0amr3089975a91.8.1774793125137; Sun, 29 Mar 2026 07:05:25 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.111]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35d950bd630sm4417454a91.16.2026.03.29.07.05.18 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Mar 2026 07:05:24 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v9 1/9] bpf: refactor kfunc checks using table-driven approach in verifier Date: Sun, 29 Mar 2026 22:04:58 +0800 Message-ID: <20260329140506.9595-2-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260329140506.9595-1-pilgrimtao@gmail.com> References: <20260329140506.9595-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Replace per-kfunc btf_id chains check with btf_id_in_kfunc_table() and static kfunc tables for easier maintenance. Prepare for future extensions to the bpf_list API family. Signed-off-by: Kaitao Cheng --- kernel/bpf/verifier.c | 261 +++++++++++++++++++++++------------------- 1 file changed, 144 insertions(+), 117 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 4fbacd2149cd..f2d9863bb290 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -544,9 +544,6 @@ static bool is_async_callback_calling_kfunc(u32 btf_id); static bool is_callback_calling_kfunc(u32 btf_id); static bool is_bpf_throw_kfunc(struct bpf_insn *insn); =20 -static bool is_bpf_wq_set_callback_kfunc(u32 btf_id); -static bool is_task_work_add_kfunc(u32 func_id); - static bool is_sync_callback_calling_function(enum bpf_func_id func_id) { return func_id =3D=3D BPF_FUNC_for_each_map_elem || @@ -586,7 +583,7 @@ static bool is_async_cb_sleepable(struct bpf_verifier_e= nv *env, struct bpf_insn =20 /* bpf_wq and bpf_task_work callbacks are always sleepable. */ if (bpf_pseudo_kfunc_call(insn) && insn->off =3D=3D 0 && - (is_bpf_wq_set_callback_kfunc(insn->imm) || is_task_work_add_kfunc(in= sn->imm))) + is_async_callback_calling_kfunc(insn->imm)) return true; =20 verifier_bug(env, "unhandled async callback in is_async_cb_sleepable"); @@ -11203,31 +11200,6 @@ static int set_task_work_schedule_callback_state(s= truct bpf_verifier_env *env, return 0; } =20 -static bool is_rbtree_lock_required_kfunc(u32 btf_id); - -/* Are we currently verifying the callback for a rbtree helper that must - * be called with lock held? If so, no need to complain about unreleased - * lock - */ -static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env) -{ - struct bpf_verifier_state *state =3D env->cur_state; - struct bpf_insn *insn =3D env->prog->insnsi; - struct bpf_func_state *callee; - int kfunc_btf_id; - - if (!state->curframe) - return false; - - callee =3D state->frame[state->curframe]; - - if (!callee->in_callback_fn) - return false; - - kfunc_btf_id =3D insn[callee->callsite].imm; - return is_rbtree_lock_required_kfunc(kfunc_btf_id); -} - static bool retval_range_within(struct bpf_retval_range range, const struc= t bpf_reg_state *reg) { if (range.return_32bit) @@ -12639,11 +12611,103 @@ BTF_ID(func, bpf_session_is_return) BTF_ID(func, bpf_stream_vprintk) BTF_ID(func, bpf_stream_print_stack) =20 -static bool is_task_work_add_kfunc(u32 func_id) -{ - return func_id =3D=3D special_kfunc_list[KF_bpf_task_work_schedule_signal= ] || - func_id =3D=3D special_kfunc_list[KF_bpf_task_work_schedule_resume= ]; -} +/* Kfunc family related to list. */ +static const enum special_kfunc_type bpf_list_api_kfuncs[] =3D { + KF_bpf_list_push_front_impl, + KF_bpf_list_push_back_impl, + KF_bpf_list_pop_front, + KF_bpf_list_pop_back, + KF_bpf_list_front, + KF_bpf_list_back, +}; + +/* Kfuncs that take a list node argument (bpf_list_node *). */ +static const enum special_kfunc_type bpf_list_node_api_kfuncs[] =3D { + KF_bpf_list_push_front_impl, + KF_bpf_list_push_back_impl, +}; + +/* Kfuncs that take an rbtree node argument (bpf_rb_node *). */ +static const enum special_kfunc_type bpf_rbtree_node_api_kfuncs[] =3D { + KF_bpf_rbtree_remove, + KF_bpf_rbtree_add_impl, + KF_bpf_rbtree_left, + KF_bpf_rbtree_right, +}; + +/* Kfunc family related to rbtree. */ +static const enum special_kfunc_type bpf_rbtree_api_kfuncs[] =3D { + KF_bpf_rbtree_add_impl, + KF_bpf_rbtree_remove, + KF_bpf_rbtree_first, + KF_bpf_rbtree_root, + KF_bpf_rbtree_left, + KF_bpf_rbtree_right, +}; + +/* Kfunc family related to spin_lock. */ +static const enum special_kfunc_type bpf_res_spin_lock_api_kfuncs[] =3D { + KF_bpf_res_spin_lock, + KF_bpf_res_spin_unlock, + KF_bpf_res_spin_lock_irqsave, + KF_bpf_res_spin_unlock_irqrestore, +}; + +/* Kfunc family related to iter_num. */ +static const enum special_kfunc_type bpf_iter_num_api_kfuncs[] =3D { + KF_bpf_iter_num_new, + KF_bpf_iter_num_next, + KF_bpf_iter_num_destroy, +}; + +/* Kfunc family related to arena. */ +static const enum special_kfunc_type bpf_arena_api_kfuncs[] =3D { + KF_bpf_arena_alloc_pages, + KF_bpf_arena_free_pages, + KF_bpf_arena_reserve_pages, +}; + +/* Kfunc family related to stream. */ +static const enum special_kfunc_type bpf_stream_api_kfuncs[] =3D { + KF_bpf_stream_vprintk, + KF_bpf_stream_print_stack, +}; + +/* Kfuncs that must be called when inserting a node in list/rbtree. */ +static const enum special_kfunc_type bpf_collection_insert_kfuncs[] =3D { + KF_bpf_list_push_front_impl, + KF_bpf_list_push_back_impl, + KF_bpf_rbtree_add_impl, +}; + +/* KF_ACQUIRE kfuncs whose vmlinux BTF return type is void* */ +static const enum special_kfunc_type bpf_obj_acquire_ptr_kfuncs[] =3D { + KF_bpf_obj_new_impl, + KF_bpf_percpu_obj_new_impl, + KF_bpf_refcount_acquire_impl, +}; + +/* Kfunc family related to task_work. */ +static const enum special_kfunc_type bpf_task_work_api_kfuncs[] =3D { + KF_bpf_task_work_schedule_signal, + KF_bpf_task_work_schedule_resume, +}; + +/* __kfuncs must be an array identifier (not a pointer), for ARRAY_SIZE. */ +#define btf_id_in_kfunc_table(__btf_id, __kfuncs) \ + ({ \ + u32 ___id =3D (__btf_id); \ + unsigned int ___i; \ + bool ___found =3D false; \ + \ + for (___i =3D 0; ___i < ARRAY_SIZE(__kfuncs); ___i++) { \ + if (___id =3D=3D special_kfunc_list[(__kfuncs)[___i]]) { \ + ___found =3D true; \ + break; \ + } \ + } \ + ___found; \ + }) =20 static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { @@ -12680,6 +12744,29 @@ static bool is_kfunc_pkt_changing(struct bpf_kfunc= _call_arg_meta *meta) return meta->func_id =3D=3D special_kfunc_list[KF_bpf_xdp_pull_data]; } =20 +/* Are we currently verifying the callback for a rbtree helper that must + * be called with lock held? If so, no need to complain about unreleased + * lock + */ +static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env) +{ + struct bpf_verifier_state *state =3D env->cur_state; + struct bpf_insn *insn =3D env->prog->insnsi; + struct bpf_func_state *callee; + int kfunc_btf_id; + + if (!state->curframe) + return false; + + callee =3D state->frame[state->curframe]; + + if (!callee->in_callback_fn) + return false; + + kfunc_btf_id =3D insn[callee->callsite].imm; + return btf_id_in_kfunc_table(kfunc_btf_id, bpf_rbtree_api_kfuncs); +} + static enum kfunc_ptr_arg_type get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, @@ -13036,65 +13123,20 @@ static int check_reg_allocation_locked(struct bpf= _verifier_env *env, struct bpf_ return 0; } =20 -static bool is_bpf_list_api_kfunc(u32 btf_id) -{ - return btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_front] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_back] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_front] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_back]; -} - -static bool is_bpf_rbtree_api_kfunc(u32 btf_id) -{ - return btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_remove] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_first] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_root] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_left] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_right]; -} - -static bool is_bpf_iter_num_api_kfunc(u32 btf_id) -{ - return btf_id =3D=3D special_kfunc_list[KF_bpf_iter_num_new] || - btf_id =3D=3D special_kfunc_list[KF_bpf_iter_num_next] || - btf_id =3D=3D special_kfunc_list[KF_bpf_iter_num_destroy]; -} - static bool is_bpf_graph_api_kfunc(u32 btf_id) { - return is_bpf_list_api_kfunc(btf_id) || is_bpf_rbtree_api_kfunc(btf_id) || + return btf_id_in_kfunc_table(btf_id, bpf_list_api_kfuncs) || + btf_id_in_kfunc_table(btf_id, bpf_rbtree_api_kfuncs) || btf_id =3D=3D special_kfunc_list[KF_bpf_refcount_acquire_impl]; } =20 -static bool is_bpf_res_spin_lock_kfunc(u32 btf_id) -{ - return btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_lock] || - btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_unlock] || - btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_lock_irqsave] || - btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_unlock_irqrestore= ]; -} - -static bool is_bpf_arena_kfunc(u32 btf_id) -{ - return btf_id =3D=3D special_kfunc_list[KF_bpf_arena_alloc_pages] || - btf_id =3D=3D special_kfunc_list[KF_bpf_arena_free_pages] || - btf_id =3D=3D special_kfunc_list[KF_bpf_arena_reserve_pages]; -} - -static bool is_bpf_stream_kfunc(u32 btf_id) -{ - return btf_id =3D=3D special_kfunc_list[KF_bpf_stream_vprintk] || - btf_id =3D=3D special_kfunc_list[KF_bpf_stream_print_stack]; -} - static bool kfunc_spin_allowed(u32 btf_id) { - return is_bpf_graph_api_kfunc(btf_id) || is_bpf_iter_num_api_kfunc(btf_id= ) || - is_bpf_res_spin_lock_kfunc(btf_id) || is_bpf_arena_kfunc(btf_id) || - is_bpf_stream_kfunc(btf_id); + return is_bpf_graph_api_kfunc(btf_id) || + btf_id_in_kfunc_table(btf_id, bpf_iter_num_api_kfuncs) || + btf_id_in_kfunc_table(btf_id, bpf_res_spin_lock_api_kfuncs) || + btf_id_in_kfunc_table(btf_id, bpf_arena_api_kfuncs) || + btf_id_in_kfunc_table(btf_id, bpf_stream_api_kfuncs); } =20 static bool is_sync_callback_calling_kfunc(u32 btf_id) @@ -13102,12 +13144,6 @@ static bool is_sync_callback_calling_kfunc(u32 btf= _id) return btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]; } =20 -static bool is_async_callback_calling_kfunc(u32 btf_id) -{ - return is_bpf_wq_set_callback_kfunc(btf_id) || - is_task_work_add_kfunc(btf_id); -} - static bool is_bpf_throw_kfunc(struct bpf_insn *insn) { return bpf_pseudo_kfunc_call(insn) && insn->off =3D=3D 0 && @@ -13119,15 +13155,16 @@ static bool is_bpf_wq_set_callback_kfunc(u32 btf_= id) return btf_id =3D=3D special_kfunc_list[KF_bpf_wq_set_callback]; } =20 -static bool is_callback_calling_kfunc(u32 btf_id) +static bool is_async_callback_calling_kfunc(u32 btf_id) { - return is_sync_callback_calling_kfunc(btf_id) || - is_async_callback_calling_kfunc(btf_id); + return is_bpf_wq_set_callback_kfunc(btf_id) || + btf_id_in_kfunc_table(btf_id, bpf_task_work_api_kfuncs); } =20 -static bool is_rbtree_lock_required_kfunc(u32 btf_id) +static bool is_callback_calling_kfunc(u32 btf_id) { - return is_bpf_rbtree_api_kfunc(btf_id); + return is_sync_callback_calling_kfunc(btf_id) || + is_async_callback_calling_kfunc(btf_id); } =20 static bool check_kfunc_is_graph_root_api(struct bpf_verifier_env *env, @@ -13138,10 +13175,10 @@ static bool check_kfunc_is_graph_root_api(struct = bpf_verifier_env *env, =20 switch (head_field_type) { case BPF_LIST_HEAD: - ret =3D is_bpf_list_api_kfunc(kfunc_btf_id); + ret =3D btf_id_in_kfunc_table(kfunc_btf_id, bpf_list_api_kfuncs); break; case BPF_RB_ROOT: - ret =3D is_bpf_rbtree_api_kfunc(kfunc_btf_id); + ret =3D btf_id_in_kfunc_table(kfunc_btf_id, bpf_rbtree_api_kfuncs); break; default: verbose(env, "verifier internal error: unexpected graph root argument ty= pe %s\n", @@ -13163,14 +13200,10 @@ static bool check_kfunc_is_graph_node_api(struct = bpf_verifier_env *env, =20 switch (node_field_type) { case BPF_LIST_NODE: - ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_i= mpl] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl= ]); + ret =3D btf_id_in_kfunc_table(kfunc_btf_id, bpf_list_node_api_kfuncs); break; case BPF_RB_NODE: - ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_remove] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_left] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_right]); + ret =3D btf_id_in_kfunc_table(kfunc_btf_id, bpf_rbtree_node_api_kfuncs); break; default: verbose(env, "verifier internal error: unexpected graph node argument ty= pe %s\n", @@ -13878,7 +13911,7 @@ static int check_kfunc_args(struct bpf_verifier_env= *env, struct bpf_kfunc_call_ return -EINVAL; } =20 - if (!is_bpf_res_spin_lock_kfunc(meta->func_id)) + if (!btf_id_in_kfunc_table(meta->func_id, bpf_res_spin_lock_api_kfuncs)) return -EFAULT; if (meta->func_id =3D=3D special_kfunc_list[KF_bpf_res_spin_lock] || meta->func_id =3D=3D special_kfunc_list[KF_bpf_res_spin_lock_irqsav= e]) @@ -14215,7 +14248,7 @@ static int check_kfunc_call(struct bpf_verifier_env= *env, struct bpf_insn *insn, } } =20 - if (is_task_work_add_kfunc(meta.func_id)) { + if (btf_id_in_kfunc_table(meta.func_id, bpf_task_work_api_kfuncs)) { err =3D push_callback_call(env, insn, insn_idx, meta.subprogno, set_task_work_schedule_callback_state); if (err) { @@ -14304,9 +14337,7 @@ static int check_kfunc_call(struct bpf_verifier_env= *env, struct bpf_insn *insn, return err; } =20 - if (meta.func_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] || - meta.func_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl] || - meta.func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { + if (btf_id_in_kfunc_table(meta.func_id, bpf_collection_insert_kfuncs)) { release_ref_obj_id =3D regs[BPF_REG_2].ref_obj_id; insn_aux->insert_off =3D regs[BPF_REG_2].var_off.value; insn_aux->kptr_struct_meta =3D btf_find_struct_meta(meta.arg_btf, meta.a= rg_btf_id); @@ -14354,11 +14385,9 @@ static int check_kfunc_call(struct bpf_verifier_en= v *env, struct bpf_insn *insn, t =3D btf_type_skip_modifiers(desc_btf, meta.func_proto->type, NULL); =20 if (is_kfunc_acquire(&meta) && !btf_type_is_struct_ptr(meta.btf, t)) { - /* Only exception is bpf_obj_new_impl */ + /* Only exception is bpf_obj_acquire_ptr_kfuncs */ if (meta.btf !=3D btf_vmlinux || - (meta.func_id !=3D special_kfunc_list[KF_bpf_obj_new_impl] && - meta.func_id !=3D special_kfunc_list[KF_bpf_percpu_obj_new_impl] && - meta.func_id !=3D special_kfunc_list[KF_bpf_refcount_acquire_impl])= ) { + !btf_id_in_kfunc_table(meta.func_id, bpf_obj_acquire_ptr_kfuncs)) { verbose(env, "acquire kernel function does not return PTR_TO_BTF_ID\n"); return -EINVAL; } @@ -23316,9 +23345,7 @@ static int fixup_kfunc_call(struct bpf_verifier_env= *env, struct bpf_insn *insn, insn_buf[1] =3D addr[1]; insn_buf[2] =3D *insn; *cnt =3D 3; - } else if (desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_= impl] || - desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] = || - desc->func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { + } else if (btf_id_in_kfunc_table(desc->func_id, bpf_collection_insert_kfu= ncs)) { struct btf_struct_meta *kptr_struct_meta =3D env->insn_aux_data[insn_idx= ].kptr_struct_meta; int struct_meta_reg =3D BPF_REG_3; int node_offset_reg =3D BPF_REG_4; --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 2 12:07:16 2026 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A1F0378D76 for ; Sun, 29 Mar 2026 14:05:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793134; cv=none; b=sgz2DxcPJSHkjb9CWauUGZ350nInrQAL8xk8DBAiqN4rTGDMg+OcRltqpmRqxXHyfAGNkOPVL6lY9kmlPBsvEmYtpUeuaacinOM+o33ZLQ+P8vxHr/75SKKb/sOENNqtpUrOtBhcpfLXKF2F/OrbtNC/LBgbn9UA+iPUQBKxRcc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793134; c=relaxed/simple; bh=E61zFnKODeiQLWRE/+6fpJeHwp6AmheZ+meXAQg2l9E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=f2Lv/stc6iTLbk5uT7W3XWRJydJ180qOopg1/RGrLtQLgmQgM4YsCox/FMxQCt9LZNN/r6aV9phdBsWwpxjFXqSCR3G21SJ/iNyMPM4dLxzNmL7mR3PuI8Y5GrVPgff/cfhWK4FTaJ+R5Ug6bMmPdHuwGT9ELSs6oXU5CyozoVE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=A/3RI0rO; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="A/3RI0rO" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-35da9692ec3so28754a91.1 for ; Sun, 29 Mar 2026 07:05:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774793132; x=1775397932; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uQ8JJnoPn08kd5GRnT5LaskZM0x4pGuXiAVfsNtLskI=; b=A/3RI0rO31z5IHqB2icQoJ4QrQAthNJjKeDcGD7YYsMq3bnE3oYzNQEcuyyvwJB6Bd 47jOn1NO70ZXo1Y0Y2WfrC0wz6ovDTv+h2PgbF2EY3d3+R4kim+UG565RDu2cBN7Whwc IcCz2ioMluMPi0s0BkrswV9JpyVaPNYvtynQ5fQwzmTd8xhdrFlTgEJ30GyuER41AS68 /qZh42uZ7aOuhCjYqaW5AyKJXANqoYChcP8axw7I/npmeClHl89sgldHIgXZ/9lA9wQa /1f1ZsL8RR3IN6o/DzMDDE3rVe4c72xQ/HT3tDwesMRxnrkx1LZJeuWgpTPvzr/3amPW W04g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774793132; x=1775397932; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=uQ8JJnoPn08kd5GRnT5LaskZM0x4pGuXiAVfsNtLskI=; b=UnlVEAjjMx3pTsZlqKEZ5nCQxoBA2hZhSt9wDcjW+i2cpLVWWNTn0EbYXLXbLMYM3o 8DXPiGGa2gYaQQfmh5x3DSlVaMcpwTjMufKZIzsG6hHZGVnHbEIivhwN6s8QUbtdyTSi dN8MckEeUBSL1Ttr9ScNEG21++99q+JnWoFITwXfepV1QX3/PmeEtSYgIbXWKOFQyxQd r+YXerZflykfagDhjQQaI2mNl1PN/vFG0RV/TwvTH0KblwHNO7/zdto//1QG0wdKZvjI z9vnMDeKvieLyOGNXjWCmPdBQZ9KdISdUwjyQOP7JufT4jBtn/YYtdYb5SJVpUCxTtdA Es/g== X-Forwarded-Encrypted: i=1; AJvYcCUDgTGy79qhfIMCNv1GGZET8xRWSCOd9TRzCNHXrQ8s4i4VAwqw6IBmnBtgsMAuEkMaTVBMMVskknAHwgE=@vger.kernel.org X-Gm-Message-State: AOJu0Yx4V35VO+l6HxEpoocvVI2vathWbykMI3X7dNh+hdWQNv9vkLHB rZyKsceAXXU/+3S2VFoA0eGyJ3h/XyiJSRAySCkgVTE/7MU/UuFPhXhu X-Gm-Gg: ATEYQzw41BAf6kPirXYLcsNd8dwdAOGJYCuk72bgo+qwZWKbsIIvuFov2Mr9LrJX1Rq nDkKNxDtLYFi69gg0RDW53t0zYzJJIGZrD5RHlUJfKL3f7cQ/dTNT+cfjxJzaCjIlfSAPCg0MDf 9WmC+v3tNpSX6WB1HS3kG2VCMmu5BA6KVDDA3N1MyxiMmhgJNGqBOCRsLBsKtqV0xUbi8tR/OfV UkGdDOnyNwoHH5sBfsF6k83My6RDLiIPfO/CkzCtjlDdV+J2Xm+HawUJH94wsXt0DbpfvScnq5Z 9uqWek5SFszYnFI5clIyjh3Ppe5fSfP6AoKynhsVQefBTc0PaMe5NLKphP1du2z11f+YXCrhuoK sDa/1jBK2392/jMdHYsmjhu1rBm0TqxgFy0Gxy1hFAKkJ6CIS2WL/u3SNIN+aZDtp9dQnRdpATY YspFtPWNJmvKXM22U9mr7wPWntsCvY/BMGKLT6cbaCop6naVUt07XWMw== X-Received: by 2002:a17:90a:f16:b0:35d:943e:3c2c with SMTP id 98e67ed59e1d1-35d943e3f12mr3381608a91.5.1774793131953; Sun, 29 Mar 2026 07:05:31 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.111]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35d950bd630sm4417454a91.16.2026.03.29.07.05.25 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Mar 2026 07:05:31 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v9 2/9] bpf: refactor __bpf_list_del to take list node pointer Date: Sun, 29 Mar 2026 22:04:59 +0800 Message-ID: <20260329140506.9595-3-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260329140506.9595-1-pilgrimtao@gmail.com> References: <20260329140506.9595-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Refactor __bpf_list_del to accept (head, struct list_head *n) instead of (head, bool tail). The caller now passes the specific node to remove: bpf_list_pop_front passes h->next, bpf_list_pop_back passes h->prev. Prepares for introducing bpf_list_del(head, node) kfunc to remove an arbitrary node when the user holds ownership. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index cb6d242bd093..94fcd4ab39e9 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2426,22 +2426,25 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_= list_head *head, return __bpf_list_add(n, head, true, meta ? meta->record : NULL, off); } =20 -static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bo= ol tail) +static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, + struct list_head *n) { - struct list_head *n, *h =3D (void *)head; + struct list_head *h =3D (void *)head; struct bpf_list_node_kern *node; =20 /* If list_head was 0-initialized by map, bpf_obj_init_field wasn't * called on its fields, so init here */ - if (unlikely(!h->next)) + if (unlikely(!h->next)) { INIT_LIST_HEAD(h); + return NULL; + } + if (list_empty(h)) return NULL; =20 - n =3D tail ? h->prev : h->next; node =3D container_of(n, struct bpf_list_node_kern, list_head); - if (WARN_ON_ONCE(READ_ONCE(node->owner) !=3D head)) + if (unlikely(READ_ONCE(node->owner) !=3D head)) return NULL; =20 list_del_init(n); @@ -2451,12 +2454,16 @@ static struct bpf_list_node *__bpf_list_del(struct = bpf_list_head *head, bool tai =20 __bpf_kfunc struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head = *head) { - return __bpf_list_del(head, false); + struct list_head *h =3D (void *)head; + + return __bpf_list_del(head, h->next); } =20 __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *= head) { - return __bpf_list_del(head, true); + struct list_head *h =3D (void *)head; + + return __bpf_list_del(head, h->prev); } =20 __bpf_kfunc struct bpf_list_node *bpf_list_front(struct bpf_list_head *hea= d) --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 2 12:07:16 2026 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A563037996C for ; Sun, 29 Mar 2026 14:05:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793140; cv=none; b=XBdZxJSTFL0l6hm7SmTSXf0Ssa8xhdUx2l8JTyPhMbfzJfxUSdRDSbNmkJGh+EPf9AGLrz6Xnd64htnZ0cQfiEGYyMovWGRdPh+g0gBSxL+5duRg/SX9pi3AyzNRX71Q+Rnhl5XU6pttLs/DRWyO3xOQACcxTIRHz6943TrDPyM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793140; c=relaxed/simple; bh=RnEU1GUyt7gozCKuyT2/MNHb4AIB3MO0vNXjvqRs7mk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uPHe58SI0HSnpAsjDCIu2ALItmX6yz3vhbdrBBBcdk2tDKen+pM6SSOxdYfD2t8cKSFiH1g3pkCvsg4UGO0eQuadKMrh5s+Ad02H7L9fq3CNY+wMRCDHCGiJmcKbujV74mTxnnyl7KrVoK7kmXGFmBXWbam9OyFm5gcckp8CGw4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=intMsBj+; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="intMsBj+" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-35d971fbcddso482153a91.1 for ; Sun, 29 Mar 2026 07:05:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774793139; x=1775397939; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZdhoS7VmQGxLal6rB9oI7wDO5oAIKWN4gPVOSCiN2cw=; b=intMsBj+V01XL0Fa0ij1l+7KwOoF883DylHpXpmYNllwiBSv7WITH3F5uVAB1r0Wpw S81E4HN4ESyxoXsgW8nnW+5w29DpgpKljjgsutqXMzZac8kYth1W83HFfkWfSsgua7I2 O8EmoA+YELqb+9etsW2LcOBkC6pcQ6PVgdPcf7ihTPJq3GFPinhqipYGYuSaRV/hZ1VY 1FK+sIpx4vYZFgLMVMyzZzOinhCR4feFwXMEK1p3I5+nwJ4OT0Bfrn+oTPQKh9tCRqO3 6fXJrGg39unLUVv4I+KtKIPsA0T96SnkFt+q9U3FZAkg3dpDJDbHHG7oe7w0oltuYbVC ovQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774793139; x=1775397939; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ZdhoS7VmQGxLal6rB9oI7wDO5oAIKWN4gPVOSCiN2cw=; b=Jom0EkP09elp31df+YQ/9EI+8KvdllrnhS/NTjp6FXtbmyODeWU42qLZ82YRGgXQ2b TUiDCyekGp1XPxecpnpzHsFX6kBYepEHK/ieBn5k/bJLWpnazwO/NNpp/jFWV3Twfmf2 UJIRTNml1KxI8hieKJy5LY/cfPw4TzMdHw9m62+MnPrs11/n/N2FIbEhhCjvjv8qVpyD e3cAEDsQcMdHQpjce9UyvGop+fFwOS4sJx6ntJ1EBbb6tB6phvLpsKpPlTXCS7jIt14r kpBF5oUxagrkB09GXQuXF8hgZl2soptJxkPuEBxEj/DoVuhFzTI2kKns9iqD7NUiSKMd k1Gw== X-Forwarded-Encrypted: i=1; AJvYcCW8NrrtzEurqpnYJZck7h0KDqBQZpc+xcYVgE2TFpT6Zm0hKmFCXlcQfKhyaswDPiJcLgoH3Hqbz8WqUtY=@vger.kernel.org X-Gm-Message-State: AOJu0Yx894sup3oCDjffomT7R3tpTyUXw1a0JmTE4kR9I0sqhLdWQ2Xr 9e5t78aerkMhKARxcRNadrZR1FP0Xih0Qjh3gfW73+gy2Gaxqr+yFSIJ X-Gm-Gg: ATEYQzzcC8q3p1cqk0SFup2X5MrWiA5uYzRT3M4yfR2sSY1YyNpTNgS8W+3f9wlFIOl WDyo3hboxsCjWs+khujalSG7jjX06/LrcBDOzkrcaDYotZ2NdcVF6Xks2PxM0mSK9CGvXbQX+I2 fSEVHWYl9Bkor6NBl3hGbHe1kfE47NQB7FjwaBmv0vTe48LXbAwWzy/qK1L166oh72qeSDql9ab A1cqPyd4TwDL6nwIBLLbsKUZiu8xSotgKxdUPltigGvc4I5F0HBdBij6p3rHNst/F55v67oOW15 UIZ3ZTfLhB9xR/cpgSygJrbcp+hmwHs84wahfpnKXzgp0uCqbL8szczEteHSGUTNmaO9dbodPjz BxoYJzesJSFHSIpL7UJpc/BaRtB+ucUBoAXJ79w0qaY+6Ff2RgqubY6sQSMVrQoZeYDHT5LPdKK YNellSWCUMW7+u50kIRatWFaxWuSze5+Bxizj1vweSv3daO6a2IgjBTnLT4etNE5BP X-Received: by 2002:a17:90b:3f46:b0:35b:e4f0:f9b4 with SMTP id 98e67ed59e1d1-35c30d790admr7033452a91.10.1774793139041; Sun, 29 Mar 2026 07:05:39 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.111]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35d950bd630sm4417454a91.16.2026.03.29.07.05.32 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Mar 2026 07:05:38 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v9 3/9] bpf: clear list node owner and unlink before drop Date: Sun, 29 Mar 2026 22:05:00 +0800 Message-ID: <20260329140506.9595-4-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260329140506.9595-1-pilgrimtao@gmail.com> References: <20260329140506.9595-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng When draining a BPF list_head, clear each node's owner pointer while still holding the spinlock, so concurrent readers always see a consistent owner. Delink each node with list_del_init() before calling __bpf_obj_drop_impl(), preventing subsequent users who hold a reference count to the node from acquiring an invalid next node. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 94fcd4ab39e9..8abb99712043 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2232,7 +2232,7 @@ EXPORT_SYMBOL_GPL(bpf_base_func_proto); void bpf_list_head_free(const struct btf_field *field, void *list_head, struct bpf_spin_lock *spin_lock) { - struct list_head *head =3D list_head, *orig_head =3D list_head; + struct list_head *head =3D list_head, *orig_head =3D list_head, *pos; =20 BUILD_BUG_ON(sizeof(struct list_head) > sizeof(struct bpf_list_head)); BUILD_BUG_ON(__alignof__(struct list_head) > __alignof__(struct bpf_list_= head)); @@ -2247,6 +2247,9 @@ void bpf_list_head_free(const struct btf_field *field= , void *list_head, if (!head->next || list_empty(head)) goto unlock; head =3D head->next; + /* Clear owner under spinlock to ensure the owner is always valid */ + for (pos =3D head; pos !=3D orig_head; pos =3D pos->next) + WRITE_ONCE(container_of(pos, struct bpf_list_node_kern, list_head)->owne= r, NULL); unlock: INIT_LIST_HEAD(orig_head); __bpf_spin_unlock_irqrestore(spin_lock); @@ -2255,7 +2258,9 @@ void bpf_list_head_free(const struct btf_field *field= , void *list_head, void *obj =3D head; =20 obj -=3D field->graph_root.node_offset; + pos =3D head; head =3D head->next; + list_del_init(pos); /* The contained type can also have resources, including a * bpf_list_head which needs to be freed. */ --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 2 12:07:16 2026 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EED9379974 for ; Sun, 29 Mar 2026 14:05:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793147; cv=none; b=b/XnZcBlPxJmRETDa8HvK7DaG4ye7azoBEFLToUTl9XO/5i4LTWWzWs73NELV9KJjUdqg/5C0QvPNLoIhDdoowKwgIzHEAZUJDCqpRBK0pbtW8gH5FKF/1hyoaJAMmPBbjCMpqSHM2f+D4nAS2bIY7M0xrzz2xGiYX4+0XYTsds= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793147; c=relaxed/simple; bh=QHI8JtS+6j5swan0XlMnxTRWUG/+dPcF49oye/Dz5Bg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O3P18i6VYQNOHqTeWBXMRbAnuHH/oC7JidIEdb/bg2uinQMio+xNKAHeddMMEkuPGs6tcYcd6RLzkcAuw34i7Iclj72eZ8KHWdxchzpbMLfCt9v1xab7K0PPFlL6DCSDkGIsAkUm5mwfjSJ16T73gdlEpmJ9TK3MO7XejHErgYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ii/+f5ay; arc=none smtp.client-ip=209.85.216.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ii/+f5ay" Received: by mail-pj1-f49.google.com with SMTP id 98e67ed59e1d1-35d971fbcddso482194a91.1 for ; Sun, 29 Mar 2026 07:05:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774793146; x=1775397946; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iXAn5RbGKGbLHjyb3tFwIH2zU+FTjnwaItuZJq6CaG4=; b=ii/+f5aybcPjv6Aw1lnR2CbgsvvUs50Y0qKbM/D/VZv/fq8qsIgeMrVq5Rpdvu0idB ub0plH8h5Sx5G8bADZR96KSCj5gHBE9VMSpyXEDOVyvWgAKSrty4mutXSb+XRP+iqV2n V2VqEPZNQp0zSGYE4KZA4dtNs6NzAiNINUrGPlynRQZdMrHB9cF02AIs6QfvZQOFBs5B /o8kal7KPtweDxoLWBt78kNLqieXlqc/3iK/SQ6hZGOWANpJwIEgbdwjJjOAWjtOgy4z fLRIITmrjiocwSiHJZ+s9tY+Q8ODd8vYoAs3EsXDMMA79ZU68vgeXDICWE41zB0//3vB IKIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774793146; x=1775397946; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=iXAn5RbGKGbLHjyb3tFwIH2zU+FTjnwaItuZJq6CaG4=; b=OmVJDUk2DFhiOdyxA8dyOZGOI+mks5FUOmP6oBgVgQIdeEoerTS1Bp1+nk6v1FJaT8 OK1BpnJxCKq/fPDXnFW+H2QvRAWzYiIrFej2JxeoZR2t3O6fmJ8FftJJtFA+Wtic9H82 UWyre7iDE90yJgP9HsKP3lz+lFR1wJD2/+LkScdbli3dyapfKwICySx7q3zEnq+5qhsK 8Otxp71FzfHZkMneG1EvWn5Y8ii9SGZeR8aWCd9l1GaS18tSNiPxp4WCCFKqPO++eB0G RHVMFb4IKJkgMYllfdGaJJvb6R6k65cn0VPt5SpofctDdB02hk/hhy+GuN68swlF89Ic zIbg== X-Forwarded-Encrypted: i=1; AJvYcCVDM4tJ2M9WkPMzOEK2PTjD744JxGlmT3pQg8d/8hMr9fUTn/pg3ZvjA9hNk3gsNeu3Hw6uKyaDimwNsIY=@vger.kernel.org X-Gm-Message-State: AOJu0YzA9CUMTxpSkw8o1kSj91CsiE7VczqJth19S1lGCR16JMcoQmto jZfv0rBUf63GpJnTXlFu/qHMHIVQ+4JfbP/JJxgv3FprMryzyELAkeIY X-Gm-Gg: ATEYQzwAYaphDHBaovaUjw9NAoWFOTHotBaXu15kgS9AftxSfZlmLZfzVQ7gBkKRLSy iA4Dbt0xkbDN22/45Sgflz7nRkVkcXcz7X3LTopztxcu2IRyy1ohv1DOTtdAnvWVB6AnlWFYvm6 VFfj3eljhtZVBvp64Au6wpjiOcr6iLI4mZ5QbFlF62NMf7JpdeRRH4kewCiE7sVF/ZGRjQF8S3S z2hfszk9H7Qabm6zl8GZqCnax4w8HJH29CEkLwqKpWWOtkBuIpP7TsKyA9UIs8yuAE8+AB/nriA daSzZo2UKBMEgZibMqW8cW9Q1arPS/sxWUGomFKzy4oCMmThb6blrWIptJepbpOj5xvfX0iQ9di URNkdiH6SjeXN2539WFmlJCmpPc/QrMkBTInutrPwW+S1qnobu10saVgBbczQGviV8YebvFV+vc xW09+8IonP2SXkszOc9m2sGR4PY/xkgx95hPlJhMOyBgi+oRRQ/4uHwA== X-Received: by 2002:a17:90b:2887:b0:359:8ca0:308d with SMTP id 98e67ed59e1d1-35c30dc50b6mr6972922a91.14.1774793145740; Sun, 29 Mar 2026 07:05:45 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.111]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35d950bd630sm4417454a91.16.2026.03.29.07.05.39 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Mar 2026 07:05:45 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v9 4/9] bpf: Introduce the bpf_list_del kfunc. Date: Sun, 29 Mar 2026 22:05:01 +0800 Message-ID: <20260329140506.9595-5-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260329140506.9595-1-pilgrimtao@gmail.com> References: <20260329140506.9595-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Allow users to remove any node from a linked list. We have added an additional parameter bpf_list_head *head to bpf_list_del, as the verifier requires the head parameter to check whether the lock is being held. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 10 ++++++++++ kernel/bpf/verifier.c | 4 ++++ 2 files changed, 14 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 8abb99712043..6dddb2377047 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2471,6 +2471,15 @@ __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(= struct bpf_list_head *head) return __bpf_list_del(head, h->prev); } =20 +__bpf_kfunc struct bpf_list_node *bpf_list_del(struct bpf_list_head *head, + struct bpf_list_node *node) +{ + struct bpf_list_node_kern *kn =3D (void *)node; + + /* verifier to guarantee n is a list node rather than the head */ + return __bpf_list_del(head, &kn->list_head); +} + __bpf_kfunc struct bpf_list_node *bpf_list_front(struct bpf_list_head *hea= d) { struct list_head *h =3D (struct list_head *)head; @@ -4557,6 +4566,7 @@ BTF_ID_FLAGS(func, bpf_list_push_front_impl) BTF_ID_FLAGS(func, bpf_list_push_back_impl) BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index f2d9863bb290..ae8d1f2e32de 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12480,6 +12480,7 @@ enum special_kfunc_type { KF_bpf_list_push_back_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, + KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, KF_bpf_cast_to_kern_ctx, @@ -12540,6 +12541,7 @@ BTF_ID(func, bpf_list_push_front_impl) BTF_ID(func, bpf_list_push_back_impl) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) +BTF_ID(func, bpf_list_del) BTF_ID(func, bpf_list_front) BTF_ID(func, bpf_list_back) BTF_ID(func, bpf_cast_to_kern_ctx) @@ -12617,6 +12619,7 @@ static const enum special_kfunc_type bpf_list_api_k= funcs[] =3D { KF_bpf_list_push_back_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, + KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, }; @@ -12625,6 +12628,7 @@ static const enum special_kfunc_type bpf_list_api_k= funcs[] =3D { static const enum special_kfunc_type bpf_list_node_api_kfuncs[] =3D { KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_del, }; =20 /* Kfuncs that take an rbtree node argument (bpf_rb_node *). */ --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 2 12:07:16 2026 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD7AC36EA9B for ; Sun, 29 Mar 2026 14:05:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793153; cv=none; b=XYmNsELYsxGpRNobGceQnnPRGe7vdjL+v3fjXSva6xmGOZxPL18vpVbzRSB6O3unI4ks5//4Zr/eNGywprroe1pmReZfQhhtGh1pA8QXhQntJQ+n9oX8aa6QGCoOuvqzjtk0uHmNosjS6WGGTqR1a5HC2SCx63onXkrROC4OFk8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793153; c=relaxed/simple; bh=m4Dp/QO1CQwIaQ7zOXjACVsoK8KivPGSGTVahKCzCOY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Xr+2YyvRIlo+JId29MusDkLrDrPeVOgB7/F69DqejmQYwj04LZJ/eDIiA/uKzq+yTDpgvx78RpV+8dYZcWK3ALll4F/12P4gLTjhnvnpkR+5b0gNIS9qT9SoN1FXSJATphjSvT/cWAah9y8ijVl0QcFmSrzyhXaCG77IPbbvGJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=pfMv6AxA; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pfMv6AxA" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-35d94f4ee36so562000a91.3 for ; Sun, 29 Mar 2026 07:05:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774793152; x=1775397952; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=38DTT79mtCXKXJ8Z/5bB5pjQPLerUM4+fkGr6yskgL4=; b=pfMv6AxAxDbMNVnjgOEadQHLR8OA1E5im8YgJTmUjR4QcXZm0C4froJjGFHhsQiFhr 1r/sFhu784VbykkK8IpvC4umPnpUL8EfHIzrRuWg5rXZc2EM820GnNcTZHuKTjG7QdJF dg5/UI5Oh3BTMFoixYZXYs49+D/ADEv2uG6yqgIF2y/gJuP2gbDO/UHbG38DLsCOTcES zp88WxyglGBSjcbkjBOrtU2oaHyRjkXgl6tO+yHzkCHS2ppRpz6G339OkmjG+NnZaPbj EmIjQ/DlJQIvLN75YOM1TbohXXDyd+l5RzHgEyL1RHvOjdXtn1JYKAdYscq6Nx4aIhxa /mkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774793152; x=1775397952; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=38DTT79mtCXKXJ8Z/5bB5pjQPLerUM4+fkGr6yskgL4=; b=I8fX+vj1APRaZpBukdGewI7Ad2YhyI0x/V+Hg0RAzT9dYbDRXSeX4kjUga1QK40C60 tCNNU/p5Wh++JFx4fMpkFGXnzuBhzpaEl3EXXeWBSvRSe8ShNbniZSrGLD7/8uDMeFm2 GYByvhvyGMbXRDQKuvUjPpgNi6cxkmwtMJP3g1cZSoBjV4OePCY9dobdSRLmRMgKre8b f/wDZAtFqpAOfgcf3piFiMxtqr5ujVQoPds1kN4BrI9x+5c3jh/Gpuc3l7doRuRwUPqk +TRCzGiRWeEnvhfA0dCck1wUv30+3ZDuKpLBuARalqJ/tWe24lJY/cKqiL2J52mGs28K YOUw== X-Forwarded-Encrypted: i=1; AJvYcCV1df+cutis8ZPLgqNt7TKw2Gmjw9e0WRAQWqvpYmn9JGHVlVJnR66THe2J8bT38zaAmdqo+DwLksyvkbI=@vger.kernel.org X-Gm-Message-State: AOJu0Yw7w/iFq60COOGz6MTiprYjXzRSw9D9SKLKJATeyVL10zoP5pdC 4E4E51T3IRGt2wAOjY7UNqMRe1AEv5XDfoTNEHNOTZ+XgEqpUCpSLfkI X-Gm-Gg: ATEYQzw3TS20CA4xqdFRuJ85SERwG52pGlgDjCmouwPZrt3MnhccS3xviJedRnJCcb8 6VCGv49KBEVcxPX/DsPNLUCrDUAEQYzKi0YQ7GOsIdqW+7xTFhx/wt8F/qtRJqAS90qnbulQV8E EIZ4idJhkN85op5fXLsIw52Ru8MT2ytaZVZLb0lt/O485lbFp2reDNXF61jI7WhzvhrgwaINL/K iukMkmHCXJN9MNsdSrb2bFqqDU1zLLXRS3Mq30WUoplmc5vpr4KHPp83zIR50Fs4dH6y+Val/Kr dKlwK7bLuckXOxD5HFlwGW7uhDnDvCHoa87JoplklYcRho7ZXkgTFgLd/xpwWD4VrUcP+tH64w+ oSIE1+LxKaGEZX/Y42FRxHxOhmnoKkCpGnEEcuRAK9bJVzFRKQmGmkNeDn0U5CygguozoLY80Fa V6WtixkoHlMeYeTHZi1E+EonGLMXclOu3RQzcwXDN5/VAGOIIPJUCSEw== X-Received: by 2002:a17:90a:1009:b0:35d:9560:3efc with SMTP id 98e67ed59e1d1-35d9560431dmr3049031a91.14.1774793151838; Sun, 29 Mar 2026 07:05:51 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.111]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35d950bd630sm4417454a91.16.2026.03.29.07.05.46 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Mar 2026 07:05:51 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v9 5/9] bpf: refactor __bpf_list_add to take insertion point via **prev_ptr Date: Sun, 29 Mar 2026 22:05:02 +0800 Message-ID: <20260329140506.9595-6-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260329140506.9595-1-pilgrimtao@gmail.com> References: <20260329140506.9595-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Refactor __bpf_list_add to accept (node, head, struct list_head **prev_ptr, ..) instead of (node, head, bool tail, ..). Load prev from *prev_ptr after INIT_LIST_HEAD(h), so we never dereference an uninitialized h->prev when head was 0-initialized (e.g. push_back passes &h->prev). When prev is not the list head, validate that prev is in the list via its owner. Prepares for bpf_list_add_impl(head, new, prev, ..) to insert after a given list node. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 6dddb2377047..669e380746a6 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2386,9 +2386,11 @@ __bpf_kfunc void *bpf_refcount_acquire_impl(void *p_= _refcounted_kptr, void *meta =20 static int __bpf_list_add(struct bpf_list_node_kern *node, struct bpf_list_head *head, - bool tail, struct btf_record *rec, u64 off) + struct list_head **prev_ptr, + struct btf_record *rec, u64 off) { struct list_head *n =3D &node->list_head, *h =3D (void *)head; + struct list_head *prev; =20 /* If list_head was 0-initialized by map, bpf_obj_init_field wasn't * called on its fields, so init here @@ -2396,19 +2398,31 @@ static int __bpf_list_add(struct bpf_list_node_kern= *node, if (unlikely(!h->next)) INIT_LIST_HEAD(h); =20 + prev =3D *prev_ptr; + + /* When prev is not the list head, it must be a node in this list. */ + if (prev !=3D h) { + struct bpf_list_node_kern *prev_kn =3D + container_of(prev, struct bpf_list_node_kern, list_head); + + if (unlikely(READ_ONCE(prev_kn->owner) !=3D head)) + goto fail; + } + /* node->owner !=3D NULL implies !list_empty(n), no need to separately * check the latter */ - if (cmpxchg(&node->owner, NULL, BPF_PTR_POISON)) { - /* Only called from BPF prog, no need to migrate_disable */ - __bpf_obj_drop_impl((void *)n - off, rec, false); - return -EINVAL; - } + if (cmpxchg(&node->owner, NULL, BPF_PTR_POISON)) + goto fail; =20 - tail ? list_add_tail(n, h) : list_add(n, h); + list_add(n, prev); WRITE_ONCE(node->owner, head); - return 0; + +fail: + /* Only called from BPF prog, no need to migrate_disable */ + __bpf_obj_drop_impl((void *)n - off, rec, false); + return -EINVAL; } =20 __bpf_kfunc int bpf_list_push_front_impl(struct bpf_list_head *head, @@ -2417,8 +2431,9 @@ __bpf_kfunc int bpf_list_push_front_impl(struct bpf_l= ist_head *head, { struct bpf_list_node_kern *n =3D (void *)node; struct btf_struct_meta *meta =3D meta__ign; + struct list_head *h =3D (void *)head; =20 - return __bpf_list_add(n, head, false, meta ? meta->record : NULL, off); + return __bpf_list_add(n, head, &h, meta ? meta->record : NULL, off); } =20 __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head, @@ -2427,8 +2442,9 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_li= st_head *head, { struct bpf_list_node_kern *n =3D (void *)node; struct btf_struct_meta *meta =3D meta__ign; + struct list_head *h =3D (void *)head; =20 - return __bpf_list_add(n, head, true, meta ? meta->record : NULL, off); + return __bpf_list_add(n, head, &h->prev, meta ? meta->record : NULL, off); } =20 static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 2 12:07:16 2026 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 918243793B0 for ; Sun, 29 Mar 2026 14:05:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793158; cv=none; b=SCGCy+nvf1fN16NLVEN481AqDlxk0i53NUjU72zA2rpK7OsMTGcD2oMsCinNT6faHWI2WDM6KFooAntaePixybNrdEv69rhC9zP5IvtEg5hVEnIPeUjCWZc46SW32QbzR4H/7iupjlXSvizI1iuHCccNUwtq3qV3l1HOP/s9CDk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793158; c=relaxed/simple; bh=YKd6ls40YDD26tsO07ifajw47M3k7vqF3qVCfUoibiM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XAU/UGlE97yYgSbX1Gb/1XoM6sgcRAzmNjmqaoFvuSEGBPMC1koHW8lbzbj3BBZtb6TzA114fuLcJdXFYtRQJeQzqCQWKQ/9yMm3AkIRgQMo+6XwDCUT7G9d3lAAvW+B7dZT9aIi3a0cSoAgAamN+DrY/sGaGs1ylrsNQyB2v1U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=sBq6pgx3; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sBq6pgx3" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-35c1a131946so2170094a91.0 for ; Sun, 29 Mar 2026 07:05:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774793157; x=1775397957; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eOBfBfgSL8HGIwmMW7fxQx1ON8sALgbRfX0R/Gl6Mv8=; b=sBq6pgx3PG0f8brRpkuOjZdmjwU1Ur5Zd/zylQQWUrp4GLwpGDZSdZ9v/atBi6CaSj /NJjGvN07f7h5qsp/G6EvqFFlkS8Y90Td1TdLs6xgOIjdMhTX4q1V5Ah4YVWhOLWqBf/ 4Ah+W27HjSRKENOYjHcw2f+h7tsWkTkTNNxVZj5yzh7iRcTsZEdCzZomY+GFVhyhNGel mb1kcO33rMxS3I6QMQQtT32CdmOzrFEcxZ3Rb75k+xgUbxj4gzfSerV9IqGP6gp7HK8F WQ2GnUyVdVmM2EeuABRC43QmzsUvJ2olxkP/QMKzMFFYhSvp2INrAFOoYt8Npxc7fxoa bXmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774793157; x=1775397957; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=eOBfBfgSL8HGIwmMW7fxQx1ON8sALgbRfX0R/Gl6Mv8=; b=M2uVthF1P3NfufmPACSTbqcRoMLEg4R4MmCjMAzs5vLli40x+X1b4DwtkhIsWyC9oM tLw7/Jm/9fxtG9kGDogvLj64nM99m7yqC1Y7WFPVd5JS1HJnGMleGpV7K5JO6R3rvrEj 4ZzOSLfnqdd/8TZMIHHRN1D0xmY0mRHJ4sv30nu4QTLRJvDnL6LkyQ1VyZMsSqWlNnOS Z/fi4xsX/p6ZWk+apML7fDqX1IlrW80Dqw0cJemJ/V+Iwj8PaO764a6haZ9LfaNYV3tU 2qON0vKZZIr0HNnLAPPBzqSEyY/5QsgIWcyoDSSb+jkeYYvUaChHhnbnlZWPyJr1gGEk Jmbg== X-Forwarded-Encrypted: i=1; AJvYcCXCbPdnNLH0UGhV2oxvJA+zNbM5z20qiJylqJQ0VCs9mbDpfRCO2tgKYS/3sPAeQD3Vc+PBOft0uPsGLMQ=@vger.kernel.org X-Gm-Message-State: AOJu0YxNnd7vGgrO/BxdAJL6HApO3US9Hvnfj7qXIH/HEysDL3SRbyB9 8OtzBr/AA/EUa8i27Itw/NLEjpP3g4Hz8EwTG1HACFml7tchZBymwRVk X-Gm-Gg: ATEYQzzK/DgunmCG3C+cjG49kP5PcyhA/zwV29lA3A+dfFJC/RjNU2CCmbZWm+6kVmf E19h1uoDnvdfFmFToxohorOBzfKmxN/tzvkOxk6ZhNLaTGd2/NMBdL9y6YWXKEqwWxTlE7TFjIp r6UpHC/p70BH55JMSa679cpwTqKfOLO2PxaT9bU5647cYXKwdCEfgqrqaa5hm6wDHMpkrOYhkuF UYJixSL/gqY3pfb3nymdyxzKVlgAFA623IrLiKjR3vI8sOYk3Jh+VGlhvQKMKZDYfiq6aIP0T+7 KLONpev26NNX7oBYz7Esq7OK31tu2OPBlA3l2TscgRzAHWVkuADcIDB9Ifo9l0aU52+2tdksGef nc0EbibQRxaE84H2Am9jJiLLDC0svNqboDkrDERZWT+ty6MlRE3XMN0bNSq5qhc/kpvkKin1tPf eqchCNauamxrQCOcbtZpHbluQSZ7H20kLZ6AKMz77U+25hnymCeV3ePw== X-Received: by 2002:a17:90b:4c4f:b0:35d:a559:5c0a with SMTP id 98e67ed59e1d1-35da5595e3bmr893075a91.6.1774793156865; Sun, 29 Mar 2026 07:05:56 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.111]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35d950bd630sm4417454a91.16.2026.03.29.07.05.52 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Mar 2026 07:05:56 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v9 6/9] bpf: Add bpf_list_add_impl to insert node after a given list node Date: Sun, 29 Mar 2026 22:05:03 +0800 Message-ID: <20260329140506.9595-7-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260329140506.9595-1-pilgrimtao@gmail.com> References: <20260329140506.9595-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Add a new kfunc bpf_list_add_impl(head, new, prev, meta, off) that inserts 'new' after 'prev' in the BPF linked list. Both must be in the same list; 'prev' must already be in the list. The new node must be an owning reference (e.g. from bpf_obj_new); the kfunc consumes that reference and the node becomes non-owning once inserted. We have added an additional parameter bpf_list_head *head to bpf_list_add_impl, as the verifier requires the head parameter to check whether the lock is being held. Returns 0 on success, -EINVAL if 'prev' is not in a list or 'new' is already in a list (or duplicate insertion). On failure, the kernel drops the passed-in node. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 14 ++++++++++++++ kernel/bpf/verifier.c | 12 ++++++++++-- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 669e380746a6..68c83a009275 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2447,6 +2447,19 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_l= ist_head *head, return __bpf_list_add(n, head, &h->prev, meta ? meta->record : NULL, off); } =20 +__bpf_kfunc int bpf_list_add_impl(struct bpf_list_head *head, + struct bpf_list_node *new, + struct bpf_list_node *prev, + void *meta__ign, u64 off) +{ + struct bpf_list_node_kern *n =3D (void *)new, *p =3D (void *)prev; + struct btf_struct_meta *meta =3D meta__ign; + struct list_head *prev_ptr =3D &p->list_head; + + return __bpf_list_add(n, head, &prev_ptr, + meta ? meta->record : NULL, off); +} + static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, struct list_head *n) { @@ -4585,6 +4598,7 @@ BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF= _RET_NULL) BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_list_add_impl) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE) BTF_ID_FLAGS(func, bpf_rbtree_remove, KF_ACQUIRE | KF_RET_NULL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index ae8d1f2e32de..69dcf0105973 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12478,6 +12478,7 @@ enum special_kfunc_type { KF_bpf_refcount_acquire_impl, KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, KF_bpf_list_del, @@ -12539,6 +12540,7 @@ BTF_ID(func, bpf_obj_drop_impl) BTF_ID(func, bpf_refcount_acquire_impl) BTF_ID(func, bpf_list_push_front_impl) BTF_ID(func, bpf_list_push_back_impl) +BTF_ID(func, bpf_list_add_impl) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) BTF_ID(func, bpf_list_del) @@ -12617,6 +12619,7 @@ BTF_ID(func, bpf_stream_print_stack) static const enum special_kfunc_type bpf_list_api_kfuncs[] =3D { KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, KF_bpf_list_del, @@ -12628,6 +12631,7 @@ static const enum special_kfunc_type bpf_list_api_k= funcs[] =3D { static const enum special_kfunc_type bpf_list_node_api_kfuncs[] =3D { KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, KF_bpf_list_del, }; =20 @@ -12681,6 +12685,7 @@ static const enum special_kfunc_type bpf_stream_api= _kfuncs[] =3D { static const enum special_kfunc_type bpf_collection_insert_kfuncs[] =3D { KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, KF_bpf_rbtree_add_impl, }; =20 @@ -23354,8 +23359,11 @@ static int fixup_kfunc_call(struct bpf_verifier_en= v *env, struct bpf_insn *insn, int struct_meta_reg =3D BPF_REG_3; int node_offset_reg =3D BPF_REG_4; =20 - /* rbtree_add has extra 'less' arg, so args-to-fixup are in diff regs */ - if (desc->func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { + /* list/rbtree_add_impl have an extra arg (prev/less), + * so args-to-fixup are in different regs. + */ + if (desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || + desc->func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { struct_meta_reg =3D BPF_REG_4; node_offset_reg =3D BPF_REG_5; } --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 2 12:07:16 2026 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEE8D36EA9B for ; Sun, 29 Mar 2026 14:06:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793166; cv=none; b=Ba9EOIX1XNzx+31GX/mRUSmE1liFhmtlml2bsyctT1TqDdnl6OFYpu7WZvDkjZw/MeriN0KyLCKI0HAe9nNzkxvSL+8+FFH+m+pc+WipoZRb1GKcWSL2AQ+rEl/cDBm0idh1PVwHS81My0OoEQE+QlxmC9ZKQ0OgMB+QPV3wZ2E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793166; c=relaxed/simple; bh=tPdGTAhKk9uUuJbJVbIiih1qpIQRVY60ISz1bsCYIG0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q8DGUCsfdT6Yze34Ms4AUmp5avjzhQ5ymsT2mTmT+ZA4hutm1GLWo/RMz64dEStGJoqCv2X5VtcQIq4YRVged5rAVUXah12KsnPUDGQyfHYkZGHBKyJ95kVt9VoHVJu20yPypARobEAVkPQI64rHPUkGySGweNNFSZdIihmFY0s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kv8As0F1; arc=none smtp.client-ip=209.85.216.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kv8As0F1" Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-35da01fc0baso332860a91.2 for ; Sun, 29 Mar 2026 07:06:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774793164; x=1775397964; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IsS0LRWkeTNfaLxRCLgnZWxO5ozeSApl7pBPb8LQn1A=; b=kv8As0F1GUJD4wS+pDMuBkgYUIsxCymnFkkshp2uUFxyVoaD4n5MjSIY2lI4PpUvBq hHiUdQnlo2do0TWNsXCCe3a/7ySP/w+vPUYBmtJydUKeuPmsKHNoD5xA/8cKd0whkTxN nMEnEcaEARUAoOlzibec9Cwn39VsD/iKqTvVt0xn6/MMZJivjs5JjkpFkhon0yooxHxn MozslC7Gg6q2lKt79KPM1RhgHRrOdnpXuL3+9spACD0YRqULBPggshPS0MUynZqosyY5 FSXIgp2iWDPFoRgwm+gG7L3uk3zodtlyvOB5IPDmpaPxTxf+av1m5grXDfMysJ3qxmsm 36+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774793164; x=1775397964; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=IsS0LRWkeTNfaLxRCLgnZWxO5ozeSApl7pBPb8LQn1A=; b=CvmtrUBtg5RI+hu1Eyd6Nli08DYpygpBmNf/Gw7fhoIXr+EJUBffBoKNPvWQJ5NuXk x6+dREIW5htWLvuuhUpJ0F6p6tHwfAxa8WxLvN1tISXZhlPl5F5W7GUMMXyroGF0/jln o0+FPufmtVKkuoq9XuyMTmpq80v3vwln58d9Cid1js+lRQ0WuQvOktnPzlIMLXfL0Tog wnpKFXNlJEqXjkvRDKeOrAsSsS9qlHHDFS/7ukyH1wb4J+FEi1z5FfCJ1qVfNMRqtzwW aT1oYp5XRl7jT8qw/O5O/8tcIW+3hJAa3PvmlQ5NbXkjqIoM5ugKd1IDum+GbTolPWmC 7VQA== X-Forwarded-Encrypted: i=1; AJvYcCVaxTyuUjq2mxlxbVbQhDONhoaV3BD8vGXr3cBDNbD/gkN2P2laaNFr+Pz/5j+13YUZgy6yP/kmweAuhKg=@vger.kernel.org X-Gm-Message-State: AOJu0YyGWwQfjst4FCLcGWz7vSgPM4NVHYHZjtPUCgxBUmOuMHiDlLYI Y1kpYl3udqRRd5YENFm3HFLYGYi8R//w32xsw9ZRWwEvU23y9K7roo78 X-Gm-Gg: ATEYQzypjGYlyYScWKDiAl1AY1scJVPbW8DSF3idHFisWMq+AgdTvInuU5It6pWsfUf QULZtW/0Ty47lFTdVrWYLZQHCoZ2+BrEnTHkTkBATwRw3g9OhWYNMmHdO1PU7WErO2rymPla07a Sfij7gtFAF9Zwjget4SwW+fDQfSEL2E/48yXZmBzNJRD6FemoTw5Uvc5lLR7fP0jWFeAMLnGRv9 uNMby2fcie5vaWO4mOGlgzMV9U9x2w2BOUvvRXAx8CRZPf/MkkL0coQVqRcf0NJ/5LJbGxmMWK7 0R6fqqPBepDu+QIWylM3E49kpY/6OsJ2RF22TF9eQkPm5/RTk/3KUaUt2rfJX1jU1zyH0U3vM9b JiXHQX63CSfJijd1gMWsXewDH09fzZteIfprCcw8W8yclkWmGSaXsVFkMyno1cWDSazyj1aTbu8 RhYlM4l3bhBCtjOioMY9rGLYg6RzcNgoO0Oq+ZW1BMNNEFLCAeqOVwnA== X-Received: by 2002:a17:90b:558d:b0:35b:9720:98d0 with SMTP id 98e67ed59e1d1-35c2ffa8265mr8695085a91.5.1774793164035; Sun, 29 Mar 2026 07:06:04 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.111]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35d950bd630sm4417454a91.16.2026.03.29.07.05.57 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Mar 2026 07:06:03 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v9 7/9] bpf: allow bpf_list_front/back result as the prev argument of bpf_list_add_impl Date: Sun, 29 Mar 2026 22:05:04 +0800 Message-ID: <20260329140506.9595-8-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260329140506.9595-1-pilgrimtao@gmail.com> References: <20260329140506.9595-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng KF_ARG_PTR_TO_LIST_NODE normally requires an owning reference (PTR_TO_BTF_ID | MEM_ALLOC and ref_obj_id). For bpf_list_add_impl's third argument (prev), allow a non-owning reference with ref_obj_id=3D=3D0 so that the result of bpf_list_front() or bpf_list_back() can be passed as the insertion point. When prev is such a non-owning ref, skip the MEM_ALLOC/ref_obj_id checks and jump to the shared list-node processing. Owning refs (e.g. from pop + refcount_acquire) still pass the existing checks and reach the same label. Add BTF suffix __nonown_allowed (is_kfunc_arg_nonown_allowed) and document it under kfuncs.rst. Signed-off-by: Kaitao Cheng --- Documentation/bpf/kfuncs.rst | 20 +++++++++++++++++++- kernel/bpf/helpers.c | 4 ++-- kernel/bpf/verifier.c | 13 +++++++++++++ 3 files changed, 34 insertions(+), 3 deletions(-) diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst index 75e6c078e0e7..6760c547dd32 100644 --- a/Documentation/bpf/kfuncs.rst +++ b/Documentation/bpf/kfuncs.rst @@ -207,7 +207,25 @@ Here, the buffer may be NULL. If the buffer is not NUL= L, it must be at least buffer__szk bytes in size. The kfunc is responsible for checking if the bu= ffer is NULL before using it. =20 -2.3.5 __str Annotation +2.3.5 __nonown_allowed Annotation +---------------------------------- + +This annotation is used to indicate that the parameter may be a non-owning= reference. + +An example is given below:: + + __bpf_kfunc int bpf_list_add_impl(..., struct bpf_list_node + *prev__nonown_allowed, ...) + { + ... + } + +For the ``prev__nonown_allowed`` parameter (resolved as ``KF_ARG_PTR_TO_LI= ST_NODE``), +suffix ``__nonown_allowed`` retains the usual owning-pointer rules and also +permits a non-owning reference with no ref_obj_id (e.g. the return value of +bpf_list_front() / bpf_list_back()). + +2.3.6 __str Annotation ---------------------------- This annotation is used to indicate that the argument is a constant string. =20 diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 68c83a009275..743341aae5c0 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2449,10 +2449,10 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_= list_head *head, =20 __bpf_kfunc int bpf_list_add_impl(struct bpf_list_head *head, struct bpf_list_node *new, - struct bpf_list_node *prev, + struct bpf_list_node *prev__nonown_allowed, void *meta__ign, u64 off) { - struct bpf_list_node_kern *n =3D (void *)new, *p =3D (void *)prev; + struct bpf_list_node_kern *n =3D (void *)new, *p =3D (void *)prev__nonown= _allowed; struct btf_struct_meta *meta =3D meta__ign; struct list_head *prev_ptr =3D &p->list_head; =20 diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 69dcf0105973..514a0aab93b8 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12265,6 +12265,11 @@ static bool is_kfunc_arg_nullable(const struct btf= *btf, const struct btf_param return btf_param_match_suffix(btf, arg, "__nullable"); } =20 +static bool is_kfunc_arg_nonown_allowed(const struct btf *btf, const struc= t btf_param *arg) +{ + return btf_param_match_suffix(btf, arg, "__nonown_allowed"); +} + static bool is_kfunc_arg_const_str(const struct btf *btf, const struct btf= _param *arg) { return btf_param_match_suffix(btf, arg, "__str"); @@ -13736,6 +13741,13 @@ static int check_kfunc_args(struct bpf_verifier_en= v *env, struct bpf_kfunc_call_ return ret; break; case KF_ARG_PTR_TO_LIST_NODE: + if (is_kfunc_arg_nonown_allowed(btf, &args[i]) && + type_is_non_owning_ref(reg->type) && !reg->ref_obj_id) { + /* Allow bpf_list_front/back return value as + * list_add_impl's third arg (R3). + */ + goto check_ok; + } if (reg->type !=3D (PTR_TO_BTF_ID | MEM_ALLOC)) { verbose(env, "arg#%d expected pointer to allocated object\n", i); return -EINVAL; @@ -13744,6 +13756,7 @@ static int check_kfunc_args(struct bpf_verifier_env= *env, struct bpf_kfunc_call_ verbose(env, "allocated object must be referenced\n"); return -EINVAL; } +check_ok: ret =3D process_kf_arg_ptr_to_list_node(env, reg, regno, meta); if (ret < 0) return ret; --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 2 12:07:16 2026 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9DD0737A481 for ; Sun, 29 Mar 2026 14:06:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793172; cv=none; b=gKzHJFA6CnrrKdv/8PhCHmj2lMFqe5BL53LbhJ/KA9SvSRcQnEzhq0SVSYzdrHZNfcPOOlWDaUsJY4KbFTap9jfMDCMCuLmbSZNcfbyNzNoy7DkSsR1pw2UNlLN1L5Nm47oY6OBMGPKNTa8hGM6UM6jJrWkgkDGQ0JU9erPwED8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793172; c=relaxed/simple; bh=PJGK20G/EyaqmP1Vc9CVFdFypEO545O2570T0iwtLEY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aRDYKVlSwa77+BiwzVOi8OCaBfAcRIPCHGGFPYmXc4vdzC67gg3DfsTUMpsnuKXGTB18C+Bj6TBMMfjPwnwR0van/HRGmKJSKACIXQaiZi/hMtlZwY03+rFGGcKhDQAWs0yp45/rODwXqCfqLdYEoHpRlbKoLV5xX+W0r90Ecxk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=rHkXOTrE; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rHkXOTrE" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-35d8e548a05so1756314a91.1 for ; Sun, 29 Mar 2026 07:06:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774793170; x=1775397970; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZEXeG5kJjdDQbEYeMVL6/LNMSdciV0H1QuI3w9LhyDw=; b=rHkXOTrENKwiy9WklIkCf2zrc7DOhw3wkHUaNSNPlUDXiT0cpJgHUfefMDBtQhqzMA lGf/6ibMq8nZBl4jSTQDo/H6HaO7TUMDnHk1telNES9f5uYhEnP9eZv8pVKc00NjPHTu dXgb8OpIDf5KtevFNr7nNqdgQnNithTQIKZMYUL5Wyj4g6l7GKc00wV+2dRgayj69b1m 1TO8wlecinEdHHIfFtu3YGlrVv/sX6uzp6kmh+2O4EgSxQvnIM38GFDWHlpwscXjaOni Ghl/6NRC/IVGioTWPkbUFR6zKJK79nJ887hJcdGOJSV0E+tG27h21tTETxyBx+NWAEd0 M04w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774793170; x=1775397970; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ZEXeG5kJjdDQbEYeMVL6/LNMSdciV0H1QuI3w9LhyDw=; b=lIBIzlYCg4qxRTObbMF+9TUJzkv2UKRNxe5nsYh6eeZaZdSCQomHXxAmY1jN1nAO8m xdesfi9nfQpDGDD0CLQ437U/cZpAuE79ctEdLw5Kk/4aXF58EOJOq3i542vAU6yC7rdB iv4ue33S6jIvHuE6OfEKgS72zbGmtPNao8TdAp1xHKmA+KcYCQMm9PIw6syRKAjdjKcC ppzZTLkXLyL+x6EVqiye5s5a8RuQdxJy98UtJwCFOfqOlilBdPOQG0vGAxD568dkwQNo 1ask3IsHUkMNWw8GB+PmFWaqpel1Bu9waeXsbfHklgX4yGyZfm8f0C2HFpuUMxzWdJtF Wcqw== X-Forwarded-Encrypted: i=1; AJvYcCWi86UA4Y8M3W7pBnEKritdjW6JZrAkdnVkVX/lrSn34xepdhKr86d9P0sXTpaw9cxMV01EIXWZV7QM9Os=@vger.kernel.org X-Gm-Message-State: AOJu0YzwZBF6SjpTofty9R6af5uM9NpPGAWm083T9ErDiFDrb/SJS95g Bq+5mgu0kndz6SIP/1N5j19p1lgLa0bRCuISM3S/0lx1b9DDAoJhGfVQ X-Gm-Gg: ATEYQzws3e6QDo0tSqtcDcCy5QoYfHxapzCQ5XlJqQuXki+zFXmQ3UgELURjXNCEGXh Eelb13y8nPCPXPq28AcWGHi43HcxNlc4Ib/nvb8rcOyAhXS5R7JWir5Rsprm0NlV1JNnvamhIlL dXSV4kHweb6D0Nf5FululyBYiqTv3GiUW7w5GpWB5zJgTs/buIAeyL+TMEPWEV6Bq25gdNPbhSO HRMEZIsRTEzql2DZ4hhltlIwdys7P/cEK/xEc2yg/B71nYU3d9TrgAdQvwLz+LqqWsjomUGvsGO Mpc/tanADLtFtQOFlDp2zpUxrUk8FYSA31D8hRkUa6xHr9Ka0uaDIB7MpBBMyMXuT7aVZ+n3qa8 m0lfpBFFSRC5IR+W5KULYco32/lNkYYv8fu2HvC3zxJ3T5w1rVEb4saiwGbKC8NAdgs48plJUdo 4fZ3I7eB1FR0K/p9GYeBMmsHdATcOGN40NJ3Q0ckfhLEIRK3nq25z525yqZSLA9dGg X-Received: by 2002:a17:90b:3141:b0:35c:936:d4cc with SMTP id 98e67ed59e1d1-35c3002fd1fmr8949840a91.14.1774793169987; Sun, 29 Mar 2026 07:06:09 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.111]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35d950bd630sm4417454a91.16.2026.03.29.07.06.04 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Mar 2026 07:06:09 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Emil Tsalapatis Subject: [PATCH bpf-next v9 8/9] bpf: add bpf_list_is_first/last/empty kfuncs Date: Sun, 29 Mar 2026 22:05:05 +0800 Message-ID: <20260329140506.9595-9-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260329140506.9595-1-pilgrimtao@gmail.com> References: <20260329140506.9595-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Add three kfuncs for BPF linked list queries: - bpf_list_is_first(head, node): true if node is the first in the list. - bpf_list_is_last(head, node): true if node is the last in the list. - bpf_list_empty(head): true if the list has no entries. Currently, without these kfuncs, to implement the above functionality it is necessary to first call bpf_list_pop_front/back to retrieve the first or last node before checking whether the passed-in node was the first or last one. After the check, the node had to be pushed back into the list using bpf_list_push_front/back, which was very inefficient. Now, with the bpf_list_is_first/last/empty kfuncs, we can directly check whether a node is the first, last, or whether the list is empty, without having to first retrieve the node. Signed-off-by: Kaitao Cheng Reviewed-by: Emil Tsalapatis --- kernel/bpf/helpers.c | 38 ++++++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 11 +++++++++++ 2 files changed, 49 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 743341aae5c0..5f778da2e560 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2529,6 +2529,41 @@ __bpf_kfunc struct bpf_list_node *bpf_list_back(stru= ct bpf_list_head *head) return (struct bpf_list_node *)h->prev; } =20 +__bpf_kfunc bool bpf_list_is_first(struct bpf_list_head *head, struct bpf_= list_node *node) +{ + struct list_head *h =3D (struct list_head *)head; + struct bpf_list_node_kern *kn =3D (struct bpf_list_node_kern *)node; + + if (READ_ONCE(kn->owner) !=3D head) + return false; + + return list_is_first(&kn->list_head, h); +} + +__bpf_kfunc bool bpf_list_is_last(struct bpf_list_head *head, struct bpf_l= ist_node *node) +{ + struct list_head *h =3D (struct list_head *)head; + struct bpf_list_node_kern *kn =3D (struct bpf_list_node_kern *)node; + + if (READ_ONCE(kn->owner) !=3D head) + return false; + + return list_is_last(&kn->list_head, h); +} + +__bpf_kfunc bool bpf_list_empty(struct bpf_list_head *head) +{ + struct list_head *h =3D (struct list_head *)head; + + /* If list_head was 0-initialized by map, bpf_obj_init_field wasn't + * called on its fields, so init here + */ + if (unlikely(!h->next)) + INIT_LIST_HEAD(h); + + return list_empty(h); +} + __bpf_kfunc struct bpf_rb_node *bpf_rbtree_remove(struct bpf_rb_root *root, struct bpf_rb_node *node) { @@ -4599,6 +4634,9 @@ BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_= NULL) BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_add_impl) +BTF_ID_FLAGS(func, bpf_list_is_first) +BTF_ID_FLAGS(func, bpf_list_is_last) +BTF_ID_FLAGS(func, bpf_list_empty) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE) BTF_ID_FLAGS(func, bpf_rbtree_remove, KF_ACQUIRE | KF_RET_NULL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 514a0aab93b8..33fae482cd56 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12489,6 +12489,9 @@ enum special_kfunc_type { KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, + KF_bpf_list_is_first, + KF_bpf_list_is_last, + KF_bpf_list_empty, KF_bpf_cast_to_kern_ctx, KF_bpf_rdonly_cast, KF_bpf_rcu_read_lock, @@ -12551,6 +12554,9 @@ BTF_ID(func, bpf_list_pop_back) BTF_ID(func, bpf_list_del) BTF_ID(func, bpf_list_front) BTF_ID(func, bpf_list_back) +BTF_ID(func, bpf_list_is_first) +BTF_ID(func, bpf_list_is_last) +BTF_ID(func, bpf_list_empty) BTF_ID(func, bpf_cast_to_kern_ctx) BTF_ID(func, bpf_rdonly_cast) BTF_ID(func, bpf_rcu_read_lock) @@ -12630,6 +12636,9 @@ static const enum special_kfunc_type bpf_list_api_k= funcs[] =3D { KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, + KF_bpf_list_is_first, + KF_bpf_list_is_last, + KF_bpf_list_empty, }; =20 /* Kfuncs that take a list node argument (bpf_list_node *). */ @@ -12638,6 +12647,8 @@ static const enum special_kfunc_type bpf_list_node_= api_kfuncs[] =3D { KF_bpf_list_push_back_impl, KF_bpf_list_add_impl, KF_bpf_list_del, + KF_bpf_list_is_first, + KF_bpf_list_is_last, }; =20 /* Kfuncs that take an rbtree node argument (bpf_rb_node *). */ --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 2 12:07:16 2026 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A55A437A487 for ; Sun, 29 Mar 2026 14:06:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793182; cv=none; b=pjr2TBOYVnpVBpkb0BFLndZbK8KMr22VsKCauT1hMWrAsyZyWt9Z/Zx/fh/dCyKxUNOZudVP5plUMi4+W9a8vadMb4/CpZ70cGRPz18cODQMfIFsEfqNH7Szc55C9lgRBZZ2PTn9p28OJ70OzvCBsfQyUTmCXvUrPH6MdPEqsEM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774793182; c=relaxed/simple; bh=iFNWymfmfxX/QJqdbdhLQocWSeqa+lLLvreHPgxAa/o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T4fCUCZlmlhEuAZyElzdvDRs57tiTiyN5h3/dlOX3AElM5j3Mmkmst7Hw5EXQeSfrQEkOeBWfAjpgyu5PFPAqfK34j7DFKXU6AcckewQROHBzuF48CyJhKBe6OeGM2vYb4jM41peOiKJIJVES5wDTzz3RTBEEjCDwlea6sj/WHk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ClAJhtE9; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ClAJhtE9" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-35d9f68d011so312072a91.2 for ; Sun, 29 Mar 2026 07:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774793180; x=1775397980; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sZLdK6dkEj8TjVSrlUlFfcQoiH20fwv6lL/iTYgpM30=; b=ClAJhtE9Wo1SkS82Zy2w3iJlP3CYqk6H/cJ+vzeECj/kFaPFi2LOv0wiAD/gCktkIv hNdo+xkZrGObOurHi3tqYE3djX/rQcvjMq/cjskor7Nu/eEHI7Be0jQe0HSmjetDNsqT ZGWEogWoZ5HRQLNUexH0b727E9AEMhFR3Dwqt47LiAqe6S3c+p8371K6lmwRgoAi6bGQ aSFj4RtQbL6GoBZjv7PzmwVtFHXm7EroCuw1rJA4gSwh2O2m4a+HQgtacj0W59DlWGrH zvzjbhHg5tc5od4mUhq5nQTjfloaqR6X1+wzvDVhaSiXBX7OgCQzT7H14R0K9FbJIGvV xlZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774793180; x=1775397980; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=sZLdK6dkEj8TjVSrlUlFfcQoiH20fwv6lL/iTYgpM30=; b=azLFtU0ntpHIYEsUIe+RuoHZCk8+xPc5Ri8XuSkpjaA9UXI3R4euabCesfYaiRGYcp v7L8kjdJHjF/8uVd0f0OpGd/ZH+PSpe06KH/cdD5B69xL/SWoh2v6z1S6AOKN67a5YOx 6bX4Ql0vb5/i7fpuXSHvfqITdyfj/Q2YrosN7K5RlTTWrwF2aK15EwmjvZlZnfvDHwCM eYPx2/Pz/C+3IiOLSL30rRR4h1DgzHce8K4+lIV0WyeDMpklkzeBudESgRAYnF5lt4g6 CijFV3wymNxRhdleEiXWWbZQhn/ENc7qnOSdungUcx/9I9Bea4zhhRUTBF5UzSj692Wm vEbQ== X-Forwarded-Encrypted: i=1; AJvYcCU2sEssnXINarCbFcuZRjs1bhpZX1USwwFT0E2hqaYPDt+oG9wrOH+Wkjb2tOrNiFJwVAYXROMzYSKh8nw=@vger.kernel.org X-Gm-Message-State: AOJu0YyjypbGZ6EofEPJ2xZcQQxVcXZHxeRQLoyhg5oLfMVoVs3g0BYw wcfkRqCp6kKQAYDszoFhCghMOYPQkyiwvvqQTrjYtDQQF+7XzfYlH0rM X-Gm-Gg: ATEYQzyvG5sSTUKud8VFJGDJTgT48GEmHhus/M8rgecN217/aOjdBfpJkHNByefih0w dQKjUDGHvruL7l8WJf653NAEhHofaVIDMj/lGaICm0W4MwIomsp0PK4WuqNZUENQokXLYu8Qk9T QYbG21W51tdTCPziuOP68AS+UvpniEjmlQk2zUE4DlKsf90OkAVCp8ztS4EeCZnFqFqxrW+sPow PKk8N3RqLp1JEsBS1d3kyQexfribYMbnOwuSHKv91bFqU40mUaII9ZfA1lsYFKgddlPkhqAOUaf UPU9k69jGvShlO5CHbkPJKtvbz0ApcjsOCJrPQMUw4Py6WzlCpMI7GhgfurLE3hQSEm00jaTJ9d 7eRoLB+N9MHWFclCYTgYP19lTravOZXuzck4lRU2j7PkNZbG7TAq2h51r0Lu1CMxCcpout1B9HN cjnaKUQmvlW25HLBBhH4JxcXu1ErNJLZu246vqECtkhvhb914MMZ3C+Nw+azMUQ9JL X-Received: by 2002:a17:90b:3b44:b0:35c:1695:24a3 with SMTP id 98e67ed59e1d1-35c300f9c52mr8062463a91.23.1774793179945; Sun, 29 Mar 2026 07:06:19 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.111]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35d950bd630sm4417454a91.16.2026.03.29.07.06.10 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Mar 2026 07:06:19 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v9 9/9] selftests/bpf: Add test cases for bpf_list_del/add/is_first/is_last/empty Date: Sun, 29 Mar 2026 22:05:06 +0800 Message-ID: <20260329140506.9595-10-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260329140506.9595-1-pilgrimtao@gmail.com> References: <20260329140506.9595-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Extend refcounted_kptr with tests for bpf_list_add (including prev from bpf_list_front and bpf_refcount_acquire), bpf_list_del (including node from bpf_rbtree_remove and bpf_refcount_acquire), bpf_list_empty, bpf_list_is_first/last, and push_back on uninit head. To verify the validity of bpf_list_del/add, the test also expects the verifier to reject calls to bpf_list_del/add made without holding the spin_lock. Signed-off-by: Kaitao Cheng --- .../testing/selftests/bpf/bpf_experimental.h | 16 + .../selftests/bpf/progs/refcounted_kptr.c | 311 ++++++++++++++++++ 2 files changed, 327 insertions(+) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing= /selftests/bpf/bpf_experimental.h index 44466acf8083..5821f0000e1f 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -85,6 +85,22 @@ extern int bpf_list_push_back_impl(struct bpf_list_head = *head, /* Convenience macro to wrap over bpf_list_push_back_impl */ #define bpf_list_push_back(head, node) bpf_list_push_back_impl(head, node,= NULL, 0) =20 +/* Description + * Insert 'new' after 'prev' in the BPF linked list with head 'head'. + * The bpf_spin_lock protecting the list must be held. 'prev' must already + * be in that list; 'new' must not be in any list. The 'meta' and 'off' + * parameters are rewritten by the verifier, no need for BPF programs to + * set them. + * Returns + * 0 on success, -EINVAL if head is NULL, prev is not in the list with hea= d, + * or new is already in a list. + */ +extern int bpf_list_add_impl(struct bpf_list_head *head, struct bpf_list_n= ode *new, + struct bpf_list_node *prev, void *meta, __u64 off) __ksym; + +/* Convenience macro to wrap over bpf_list_add_impl */ +#define bpf_list_add(head, new, prev) bpf_list_add_impl(head, new, prev, N= ULL, 0) + /* Description * Remove the entry at the beginning of the BPF linked list. * Returns diff --git a/tools/testing/selftests/bpf/progs/refcounted_kptr.c b/tools/te= sting/selftests/bpf/progs/refcounted_kptr.c index c847398837cc..e5558994a76d 100644 --- a/tools/testing/selftests/bpf/progs/refcounted_kptr.c +++ b/tools/testing/selftests/bpf/progs/refcounted_kptr.c @@ -367,6 +367,317 @@ long insert_rbtree_and_stash__del_tree_##rem_tree(voi= d *ctx) \ INSERT_STASH_READ(true, "insert_stash_read: remove from tree"); INSERT_STASH_READ(false, "insert_stash_read: don't remove from tree"); =20 +SEC("tc") +__description("list_empty_test: list empty before add, non-empty after add= ") +__success __retval(0) +int list_empty_test(void *ctx) +{ + struct node_data *node_new; + + bpf_spin_lock(&lock); + if (!bpf_list_empty(&head)) { + bpf_spin_unlock(&lock); + return -1; + } + bpf_spin_unlock(&lock); + + node_new =3D bpf_obj_new(typeof(*node_new)); + if (!node_new) + return -2; + + bpf_spin_lock(&lock); + bpf_list_push_front(&head, &node_new->l); + + if (bpf_list_empty(&head)) { + bpf_spin_unlock(&lock); + return -3; + } + bpf_spin_unlock(&lock); + return 0; +} + +static struct node_data *__add_in_list(struct bpf_list_head *head, + struct bpf_spin_lock *lock) +{ + struct node_data *node_new, *node_ref; + + node_new =3D bpf_obj_new(typeof(*node_new)); + if (!node_new) + return NULL; + + node_ref =3D bpf_refcount_acquire(node_new); + + bpf_spin_lock(lock); + bpf_list_push_front(head, &node_new->l); + bpf_spin_unlock(lock); + return node_ref; +} + +SEC("tc") +__description("list_is_edge_test: is_first on first node, is_last on last = node") +__success __retval(0) +int list_is_edge_test(void *ctx) +{ + struct node_data *node_first, *node_last; + int err =3D 0; + + node_last =3D __add_in_list(&head, &lock); + if (!node_last) + return -1; + + node_first =3D __add_in_list(&head, &lock); + if (!node_first) { + bpf_obj_drop(node_last); + return -2; + } + + bpf_spin_lock(&lock); + if (!bpf_list_is_first(&head, &node_first->l)) { + err =3D -3; + goto fail; + } + if (!bpf_list_is_last(&head, &node_last->l)) + err =3D -4; + +fail: + bpf_spin_unlock(&lock); + bpf_obj_drop(node_first); + bpf_obj_drop(node_last); + return err; +} + +SEC("tc") +__description("list_del_test1: del returns removed nodes") +__success __retval(0) +int list_del_test1(void *ctx) +{ + struct node_data *node_first, *node_last; + struct bpf_list_node *bpf_node_first, *bpf_node_last; + int err =3D 0; + + node_last =3D __add_in_list(&head, &lock); + if (!node_last) + return -1; + + node_first =3D __add_in_list(&head, &lock); + if (!node_first) { + bpf_obj_drop(node_last); + return -2; + } + + bpf_spin_lock(&lock); + bpf_node_last =3D bpf_list_del(&head, &node_last->l); + bpf_node_first =3D bpf_list_del(&head, &node_first->l); + bpf_spin_unlock(&lock); + + if (bpf_node_first) + bpf_obj_drop(container_of(bpf_node_first, struct node_data, l)); + else + err =3D -3; + + if (bpf_node_last) + bpf_obj_drop(container_of(bpf_node_last, struct node_data, l)); + else + err =3D -4; + + bpf_obj_drop(node_first); + bpf_obj_drop(node_last); + return err; +} + +SEC("tc") +__description("list_del_test2: remove an arbitrary node from the list") +__success __retval(0) +int list_del_test2(void *ctx) +{ + struct bpf_rb_node *rb; + struct bpf_list_node *l; + struct node_data *n; + long err; + + err =3D __insert_in_tree_and_list(&head, &root, &lock); + if (err) + return err; + + bpf_spin_lock(&lock); + rb =3D bpf_rbtree_first(&root); + if (!rb) { + bpf_spin_unlock(&lock); + return -4; + } + + rb =3D bpf_rbtree_remove(&root, rb); + if (!rb) { + bpf_spin_unlock(&lock); + return -5; + } + + n =3D container_of(rb, struct node_data, r); + l =3D bpf_list_del(&head, &n->l); + bpf_spin_unlock(&lock); + bpf_obj_drop(n); + if (!l) + return -6; + + bpf_obj_drop(container_of(l, struct node_data, l)); + return 0; +} + +SEC("tc") +__description("list_add_test1: insert new node after prev") +__success __retval(0) +int list_add_test1(void *ctx) +{ + struct node_data *node_first; + struct node_data *new_node; + long err =3D 0; + + node_first =3D __add_in_list(&head, &lock); + if (!node_first) + return -1; + + new_node =3D bpf_obj_new(typeof(*new_node)); + if (!new_node) { + err =3D -2; + goto fail; + } + + bpf_spin_lock(&lock); + err =3D bpf_list_add(&head, &new_node->l, &node_first->l); + bpf_spin_unlock(&lock); + if (err) { + err =3D -3; + goto fail; + } + +fail: + bpf_obj_drop(node_first); + return 0; +} + +SEC("tc") +__description("list_add_test2: list_add accepts list_front return value as= prev") +__success __retval(0) +int list_add_test2(void *ctx) +{ + struct node_data *new_node, *tmp; + struct bpf_list_node *bpf_node; + long err =3D 0; + + tmp =3D __add_in_list(&head, &lock); + if (!tmp) + return -1; + + new_node =3D bpf_obj_new(typeof(*new_node)); + if (!new_node) { + err =3D -2; + goto fail; + } + + bpf_spin_lock(&lock); + bpf_node =3D bpf_list_front(&head); + if (!bpf_node) { + bpf_spin_unlock(&lock); + bpf_obj_drop(new_node); + err =3D -3; + goto fail; + } + + err =3D bpf_list_add(&head, &new_node->l, bpf_node); + bpf_spin_unlock(&lock); + if (err) { + err =3D -4; + goto fail; + } + +fail: + bpf_obj_drop(tmp); + return err; +} + +struct uninit_head_val { + struct bpf_spin_lock lock; + struct bpf_list_head head __contains(node_data, l); +}; + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct uninit_head_val); + __uint(max_entries, 1); +} uninit_head_map SEC(".maps"); + +SEC("tc") +__description("list_push_back_uninit_head: push_back on 0-initialized list= head") +__success __retval(0) +int list_push_back_uninit_head(void *ctx) +{ + struct uninit_head_val *st; + struct node_data *node; + int ret =3D -1, key =3D 0; + + st =3D bpf_map_lookup_elem(&uninit_head_map, &key); + if (!st) + return -1; + + node =3D bpf_obj_new(typeof(*node)); + if (!node) + return -1; + + bpf_spin_lock(&st->lock); + ret =3D bpf_list_push_back(&st->head, &node->l); + bpf_spin_unlock(&st->lock); + + return ret; +} + +SEC("?tc") +__failure __msg("bpf_spin_lock at off=3D32 must be held for bpf_list_head") +long list_del_without_lock_fail(void *ctx) +{ + struct bpf_rb_node *rb; + struct bpf_list_node *l; + struct node_data *n; + + bpf_spin_lock(&lock); + rb =3D bpf_rbtree_first(&root); + bpf_spin_unlock(&lock); + if (!rb) + return -1; + + n =3D container_of(rb, struct node_data, r); + /* Error case: delete list node without holding lock */ + l =3D bpf_list_del(&head, &n->l); + if (!l) + return -2; + bpf_obj_drop(container_of(l, struct node_data, l)); + + return 0; +} + +SEC("?tc") +__failure __msg("bpf_spin_lock at off=3D32 must be held for bpf_list_head") +long list_add_without_lock_fail(void *ctx) +{ + struct bpf_rb_node *rb; + struct bpf_list_node *l; + struct node_data *n; + + bpf_spin_lock(&lock); + rb =3D bpf_rbtree_first(&root); + l =3D bpf_list_front(&head); + bpf_spin_unlock(&lock); + if (!rb || !l) + return -1; + + n =3D container_of(l, struct node_data, l); + /* Error case: add list node without holding lock */ + if (bpf_list_add(&head, &n->l, l)) + return -2; + + return 0; +} + SEC("tc") __success long rbtree_refcounted_node_ref_escapes(void *ctx) --=20 2.50.1 (Apple Git-155)