From nobody Tue Apr 7 04:14:22 2026 Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E927126E142 for ; Mon, 16 Mar 2026 11:29:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660545; cv=none; b=aEFej21cfTZNFvJdrRB05RvuTU1eTkMsMiCEdWftbbKYQzGeVAAMrcr5/JbVNlhIztvzk48/CZpHUFny5h06H4TLeIwlEjP0TZMIXxjKUUr5GBWwfKIffitFRcWj4z3ZJWBfET14jHNmYKnWnl/Vg9QYVjS9eSZqTyIZv513AzA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660545; c=relaxed/simple; bh=qRnnF/d9ao+ix7V4nQlSqXFAnI53UccsqHirYypmr0M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AFNxo5+hiZaufsyEQrXFPeP02vg7JAqUn9r8SLr3xokDUYKsYuYY/TFeB7amKhMzi5CiIVZnCRzcxjluO6LvqQA8eWDeWDALBM5F+3jlc2JMzexir6fCZyiUJGdAPfIUwie/eedvs/ONDZ82CQQf5zuBobv3+yCOCWL3dHQ5Or4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=a2CapIAX; arc=none smtp.client-ip=209.85.215.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="a2CapIAX" Received: by mail-pg1-f177.google.com with SMTP id 41be03b00d2f7-c7358a7a8d1so3151719a12.3 for ; Mon, 16 Mar 2026 04:29:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773660543; x=1774265343; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wxXRm33amG5I2paW33zzs2AvrNuv3vxVlJtBFRjkyIk=; b=a2CapIAX4AC4yyZJZ1kNSUXcjU3XlYiK9J/6m1BwZD18azY8N3PySNJxqwclrsA6hT rgCKwZPIWNlRT5/AfHw9FrtYgJNmgrWiLda6bDeGp+LNS6ij3kUVFmWLYvA9Ed/El9sL fzjXOLexbLkoX2yJ+cE3weQcpe69kwuk+oYvLzp7ZTqCETkpxecSYrNDRkOJoA2r9xbs N0YNkt2fe0i8VX3SXvfXjvVJ3UCBdyn+2A41XpO+5LA1DjxRGQsxJkgEIZYz9K/Wn7zi EJcXSChuTJ/bfhpA8O3jjGW6Zo4EsELFn/SnPLrr3lodYuUHcZDQXhBrvK4bN+3UqS5m NENA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773660543; x=1774265343; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=wxXRm33amG5I2paW33zzs2AvrNuv3vxVlJtBFRjkyIk=; b=sJemBNbZg29mzu00sdUkId/8fP/mrRaCDK6kwQQ4tMoB8ayU+UyL46Z7K3N5XenyGJ yYFALHSVXbqACrEN9d6wI6iEqHNIgdKcIMbPG05GU5hGdN4efenQWdoZtLlhPHmDLP0I QzrH77hyn+EnTXz2+Q2PrZKY0Ms8ad/orKCXNIIMHUxH/ZJGshtwzUUJdVZZfCAf4G/6 DniQszFZuiLWwvRebM17cAlS3Krq/hg1N5MfxFnPf9GgcPieMXx8wDS9lg+2sG0hQG8m rd9JvOFyaGfwX3WZPUiMSQBVQM7cBxKXR5rrobkcEacPNJxrot6MbOf6cWqP2k/p3Z+N zQqg== X-Forwarded-Encrypted: i=1; AJvYcCV3+1sDy1RKPlHlj3/X0dpfZK+8GwVgtgQvIsrYOq1l04PCRS9z7XOW/YQp9TY2AvvyYzaVrohDcUC7Gv0=@vger.kernel.org X-Gm-Message-State: AOJu0YwMIVV07EnMFRy3tIuWoKVWZtiAH2VQLQeMbOYYbke0WW0JvOJE sWBxqjoveUdjmc83MwV9XiVtHDDEDHSKEFTs1UgZQ8Cz5+L6sLKvA+DM X-Gm-Gg: ATEYQzyJq0xABXPTx+e76itRVLzETcywl20H1X4OnDoeZPJJsq2W8rAq5EQv8GHieNs UUnVQGygVnPDelOUYOeonLtC8+UHnKvjsmVt/h0XRBTkXNbx5+Bb5rxMqB9YPo6PjnymcOcwDCs 0b/+Atz57WfsHQiOiMkPWW8Q/TQ66/tBlXErBkKyMN+ZVdGX4lLUIuaNsrYuBrn5rZjXTLhw1Lw CzeGbK1kSs5NulvLa16q6pZg4R+XNnim6+OnynBHtUIMF7WmSIKkuc4w6wIvigH/LEWjJvtJ5lc ihPEwCUz2x0andL4sZ699t5F0/KOUJbeFnKgwwvJw7BLsggikMpjy1GP6sXb63oisnDjfxhpJz0 Fx3R+c2q9y4o9ikvhSVR3ZGy7R8uPrQ7fUngOcNJPtXIBLzr3CdrdXWnpZziCv1nbv/tTnwCgkV uTjzzrLoJzg3lVo8OMAIh1P6BiWQo/VrWxJL/PEJBPNQ+GmnrkLpg4lf4= X-Received: by 2002:a05:6a21:3388:b0:398:a306:d64f with SMTP id adf61e73a8af0-398eca9d7ebmr11676718637.30.1773660543229; Mon, 16 Mar 2026 04:29:03 -0700 (PDT) Received: from localhost.localdomain ([116.128.244.171]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c7401588ecesm4684940a12.32.2026.03.16.04.28.56 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 16 Mar 2026 04:29:02 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v8 1/8] bpf: refactor kfunc checks using table-driven approach in verifier Date: Mon, 16 Mar 2026 19:28:36 +0800 Message-ID: <20260316112843.78657-2-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260316112843.78657-1-pilgrimtao@gmail.com> References: <20260316112843.78657-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Replace per-kfunc btf_id chains in list/rbtree/res_lock and graph node checks with btf_id_in_kfunc_table() and static kfunc tables for easier maintenance. Prepare for future extensions to the bpf_list API family. Signed-off-by: Kaitao Cheng Reviewed-by: Emil Tsalapatis --- kernel/bpf/verifier.c | 79 +++++++++++++++++++++++++++++++------------ 1 file changed, 57 insertions(+), 22 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 4fbacd2149cd..64c1f8343dfa 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12639,6 +12639,53 @@ BTF_ID(func, bpf_session_is_return) BTF_ID(func, bpf_stream_vprintk) BTF_ID(func, bpf_stream_print_stack) =20 +static const enum special_kfunc_type bpf_list_api_kfuncs[] =3D { + KF_bpf_list_push_front_impl, + KF_bpf_list_push_back_impl, + KF_bpf_list_pop_front, + KF_bpf_list_pop_back, + KF_bpf_list_front, + KF_bpf_list_back, +}; + +/* Kfuncs that take a list node argument (bpf_list_node *). */ +static const enum special_kfunc_type bpf_list_node_api_kfuncs[] =3D { + KF_bpf_list_push_front_impl, + KF_bpf_list_push_back_impl, +}; + +/* Kfuncs that take an rbtree node argument (bpf_rb_node *). */ +static const enum special_kfunc_type bpf_rbtree_node_api_kfuncs[] =3D { + KF_bpf_rbtree_remove, + KF_bpf_rbtree_add_impl, + KF_bpf_rbtree_left, + KF_bpf_rbtree_right, +}; + +static const enum special_kfunc_type bpf_rbtree_api_kfuncs[] =3D { + KF_bpf_rbtree_add_impl, + KF_bpf_rbtree_remove, + KF_bpf_rbtree_first, + KF_bpf_rbtree_root, + KF_bpf_rbtree_left, + KF_bpf_rbtree_right, +}; + +static const enum special_kfunc_type bpf_res_spin_lock_kfuncs[] =3D { + KF_bpf_res_spin_lock, + KF_bpf_res_spin_unlock, + KF_bpf_res_spin_lock_irqsave, + KF_bpf_res_spin_unlock_irqrestore, +}; + +static bool btf_id_in_kfunc_table(u32 btf_id, const enum special_kfunc_typ= e *kfuncs, int n) +{ + for (int i =3D 0; i < n; i++) + if (btf_id =3D=3D special_kfunc_list[kfuncs[i]]) + return true; + return false; +} + static bool is_task_work_add_kfunc(u32 func_id) { return func_id =3D=3D special_kfunc_list[KF_bpf_task_work_schedule_signal= ] || @@ -13038,22 +13085,14 @@ static int check_reg_allocation_locked(struct bpf= _verifier_env *env, struct bpf_ =20 static bool is_bpf_list_api_kfunc(u32 btf_id) { - return btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_front] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_back] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_front] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_back]; + return btf_id_in_kfunc_table(btf_id, bpf_list_api_kfuncs, + ARRAY_SIZE(bpf_list_api_kfuncs)); } =20 static bool is_bpf_rbtree_api_kfunc(u32 btf_id) { - return btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_remove] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_first] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_root] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_left] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_right]; + return btf_id_in_kfunc_table(btf_id, bpf_rbtree_api_kfuncs, + ARRAY_SIZE(bpf_rbtree_api_kfuncs)); } =20 static bool is_bpf_iter_num_api_kfunc(u32 btf_id) @@ -13071,10 +13110,8 @@ static bool is_bpf_graph_api_kfunc(u32 btf_id) =20 static bool is_bpf_res_spin_lock_kfunc(u32 btf_id) { - return btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_lock] || - btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_unlock] || - btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_lock_irqsave] || - btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_unlock_irqrestore= ]; + return btf_id_in_kfunc_table(btf_id, bpf_res_spin_lock_kfuncs, + ARRAY_SIZE(bpf_res_spin_lock_kfuncs)); } =20 static bool is_bpf_arena_kfunc(u32 btf_id) @@ -13163,14 +13200,12 @@ static bool check_kfunc_is_graph_node_api(struct = bpf_verifier_env *env, =20 switch (node_field_type) { case BPF_LIST_NODE: - ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_i= mpl] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl= ]); + ret =3D btf_id_in_kfunc_table(kfunc_btf_id, bpf_list_node_api_kfuncs, + ARRAY_SIZE(bpf_list_node_api_kfuncs)); break; case BPF_RB_NODE: - ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_remove] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_left] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_right]); + ret =3D btf_id_in_kfunc_table(kfunc_btf_id, bpf_rbtree_node_api_kfuncs, + ARRAY_SIZE(bpf_rbtree_node_api_kfuncs)); break; default: verbose(env, "verifier internal error: unexpected graph node argument ty= pe %s\n", --=20 2.50.1 (Apple Git-155) From nobody Tue Apr 7 04:14:22 2026 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E521374738 for ; Mon, 16 Mar 2026 11:29:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660551; cv=none; b=CafOt6iF9rIe0dpxL6gRaAs7Lj2VP9euEa8yIRD2GekxEjSVRTpfCz+ouUNPF/Br5XM9RYLJQNqzCi435GW4uP+UA+RzCPIx86mYzt7sv7AWFraCQA9FVvK8CsE9Sh3qhSzH74V8qowQVigznoJBiVjAANkl4kiOOlzeMr3eoYY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660551; c=relaxed/simple; bh=UjK10OBfDG4cifl9Mz4i7HLK/ZoPL07F/KNEv3xPjSA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OUoBwE4S+jCoLRFzydNZMQUNcjdDq3cPUvc4oFEeHi62adbjHafNjBTrmyD9H3HYh1NNDATowidQ9un2b0FBjdp0IoTlVQ6CW3lqHBjRFP0sFOgQxD6DnClA9xwMqbREsKmfchmIDMp7NwupFk7VNOkenhLcdojPV+iGHpRRv/c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YTIRw/ct; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YTIRw/ct" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-82984c077b2so2210246b3a.1 for ; Mon, 16 Mar 2026 04:29:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773660550; x=1774265350; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HpPRbYYnTe0Bs7rJxn1jlZ275hZTs9fjUlBButpAMuA=; b=YTIRw/ctKVt+YLqTE4cSt64k3AZn3q9DpzeRgQ4uzBbtOm05XInj7EAuJorrxbWpo1 /iyU0ob19rJfbHve6oJcNrN5GpQj2GyrEhCXS9nV9OoIaDeTjYIzT4M7Bl6Hx9NMMKWb +vTiUEH2kGaQUfk1sWHR/FTy/VaxTqR9uuHCO8NqVF35gjuFWVhwn7B149eq6+9n/ash 883Fhwzb5EA3lYb0MMdw0lra1Bxk8MSADXHvsepaYHSWtRm0jEvgoldKB+Ba5PP+DsaN ASl56O377Wa1ZgGRjxWXSUxzFjdTPHHWz8QJwYniZCvuJWEYOtq0gQurzFqOse+SiroQ cMkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773660550; x=1774265350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=HpPRbYYnTe0Bs7rJxn1jlZ275hZTs9fjUlBButpAMuA=; b=FH+JJh4wSbzAKt+VMJTuKtcBqs0KzqpPwphOARrfQlwKe+JZy94wji1hzMuTXlO4hq W4kzSgxQkiG/J+pEVx8OkBek+dX9WFVFH7qU1QLnoaIZrnat3/m6mjzhsGn74TGpl/A+ bPIYOsiWUBGQJvhJREif++DhCmks8qnpNQc1sSqaqrTNt9K6EBkcNK4R2nnPwuC4R3v8 t4iSf4TPsF0uba9CyKf75riK/NZ/7KRM87JDpa9OuYVlkpS7WqQwiQEBv19PrJV6UDun QaV/c23/QAQrWelNt/WStBXS/LD26l9sl2+wf/yj+KWI2j6lmU9oQJJDKXWmVECNKCv4 PiNA== X-Forwarded-Encrypted: i=1; AJvYcCUfC6B2w91j9gJ4NzJ34S0+l6k44KNVufy5B3YrSNhn59akM74ZNxphH3/IOzmmlYaCIi12bbRNawAQ3VA=@vger.kernel.org X-Gm-Message-State: AOJu0YyjAf4aKAA+u/kFlrBsEv7TUzXcklwzPFHghN05QManVYhH34XG JBfLg2RzfbBStq5Kpflui8oPM1qN2tX0X4YTkgjPx8ReSsvl5un91XD+ X-Gm-Gg: ATEYQzwY6RYDVyCJJwOj2x8wt4n2hQApwHmMYCOw11CifTG4gsJHy8uo8+E+eA/gBI9 JGUuI7gq4mvfMT+4IrmNItXLadLnyXVCv6rFN6f2tLl9aomlou+F5moop7Xoj7+5HouGTgLL/n3 8ylAo3Dr1h4gnhxfTxuVEKHkTUB9eujyhMK9XJsCjJ/g40goa5a9acYuG6QYPO9CopKfaHloLRa 3viUBa75MMr0lCo5Yvqav78thPhVCoRrD+yOR5JyCxCA5I0trOxDupwZEd5izaruLi91h8WKniO DZ6V9b6I2blohUwm0RCLwkGdrbDKgcUP1nn68K/wJr+Ci1+i2eXcOxi1Ifls8I75RpxN+bbQKPX Lls9NWJYnZ69avYtxwNEenGHpGYilBScsUrfAJRLI//ZUtmFLx8LOrCWuR9wL3PRkx7KG5pgMmk 6xOoAGuOXEAHfTMYlJmQLNhrdvtzvFwHKY0HdXy6gC808fyLhAQp03FLtQWaT6iAumCw== X-Received: by 2002:a05:6a21:4d8c:b0:38e:54b8:60a1 with SMTP id adf61e73a8af0-398eca2fc4amr11781405637.4.1773660549984; Mon, 16 Mar 2026 04:29:09 -0700 (PDT) Received: from localhost.localdomain ([116.128.244.171]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c7401588ecesm4684940a12.32.2026.03.16.04.29.03 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 16 Mar 2026 04:29:09 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v8 2/8] bpf: refactor __bpf_list_del to take list node pointer Date: Mon, 16 Mar 2026 19:28:37 +0800 Message-ID: <20260316112843.78657-3-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260316112843.78657-1-pilgrimtao@gmail.com> References: <20260316112843.78657-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Refactor __bpf_list_del to accept (head, struct list_head *n) instead of (head, bool tail). The caller now passes the specific node to remove: bpf_list_pop_front passes h->next, bpf_list_pop_back passes h->prev. Prepares for introducing bpf_list_del(head, node) kfunc to remove an arbitrary node when the user holds ownership. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index cb6d242bd093..e87b263c5fe6 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2426,9 +2426,10 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_l= ist_head *head, return __bpf_list_add(n, head, true, meta ? meta->record : NULL, off); } =20 -static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bo= ol tail) +static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, + struct list_head *n) { - struct list_head *n, *h =3D (void *)head; + struct list_head *h =3D (void *)head; struct bpf_list_node_kern *node; =20 /* If list_head was 0-initialized by map, bpf_obj_init_field wasn't @@ -2439,7 +2440,6 @@ static struct bpf_list_node *__bpf_list_del(struct bp= f_list_head *head, bool tai if (list_empty(h)) return NULL; =20 - n =3D tail ? h->prev : h->next; node =3D container_of(n, struct bpf_list_node_kern, list_head); if (WARN_ON_ONCE(READ_ONCE(node->owner) !=3D head)) return NULL; @@ -2451,12 +2451,16 @@ static struct bpf_list_node *__bpf_list_del(struct = bpf_list_head *head, bool tai =20 __bpf_kfunc struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head = *head) { - return __bpf_list_del(head, false); + struct list_head *h =3D (void *)head; + + return __bpf_list_del(head, h->next); } =20 __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *= head) { - return __bpf_list_del(head, true); + struct list_head *h =3D (void *)head; + + return __bpf_list_del(head, h->prev); } =20 __bpf_kfunc struct bpf_list_node *bpf_list_front(struct bpf_list_head *hea= d) --=20 2.50.1 (Apple Git-155) From nobody Tue Apr 7 04:14:22 2026 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8733C18AFE for ; Mon, 16 Mar 2026 11:29:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660558; cv=none; b=eZzQUpXVXdypZ1o0e4Xid0TktGZzNp8k151m20d4F4sg4Nxpc8SCaW6b8E1kzdC2f+qJTmN+C05lLtwGLFhsRtiVHikx7bSXLdcUF+TB/SxCtS94/V8GyW6m32AS8cP0RZbnA4px8LXE77DmDip6ZFjdVp/00XTGX5GH801NaRQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660558; c=relaxed/simple; bh=X6NpjHI33SxFJZtV58lEW1oBuVxFCOVbuRWaUibfXDA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=k2BK6+3IVOCefRVDrGZQ1EXI+ZpNjgPPJYLDaPkshfFp8uvkvQFuJW5BwSNrr+2ndcKL5v4em7zNmmehiytvK3F4EGZzn3tBjUdUwPat0GdKTI3bTp2uZ/HJ8z9LvJmkEelPdLUUb+IZvvev68ep2VXn9w7DySU0++1P70Y+S54= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UDIu095M; arc=none smtp.client-ip=209.85.215.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UDIu095M" Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-c739d32b72cso3073256a12.2 for ; Mon, 16 Mar 2026 04:29:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773660557; x=1774265357; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Xcdf8NjYRYFE5tIQAMhp6zvGC7XeDqC/XeKloLYP+1w=; b=UDIu095M5+CBbIhot34O5IZt9TCm4RWOVJCY3mcOFpqfQr2ymU2SO5wrOagnpN2Jr5 Dpo09el23CakLIboz9Ow6mxjyZ8pryC8v2BJu7VHlKLCrTrxUwlqqyxJpvdvIIROo+7e ubq4iXRbNjuVygDfDCHz8xbBAynkTOJgbXn8xu1hArezT3yLzoTAmGNG/iAMv6kieDcW v3FzqsvJagorsYPfufdfVbya/eUOOgapaT3eC8ok0u06pnGZozdXxjDXhZb8R4+MDm7C twfxUa6tIAB0C0bg2lEExfHcqmkXPGUH34o/3hqmCyEVQ5cPztDSEZGcaMLEBcO+7W3e sXsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773660557; x=1774265357; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Xcdf8NjYRYFE5tIQAMhp6zvGC7XeDqC/XeKloLYP+1w=; b=U9R82Pp+4SKcx2Z5gfqHmGZVtXiiRR0No9ES7ldhgUbpWFlwTfCbPdNsAsQ5Tf08gN d3UlN5P0acVJyCP5e8x4Q8613FlLOHZwCJIQhZ/88vZs2MnzemLE+KNABLmBSGUulJwe A3RLfkFgSBZ1mKKO6VKXogO8vHv4Yn7DFLMeW7xBF1pKylidykb0hgZ+RTXkjmPlChWv +i1lACmgydTjMSG7oJmkZXawlWSCrc/lvTjld0Ut0kqQSSizfGyATiKdnIh0qC6H4Ffd 6h5BtOc8BECMjwEnqEeHqPdl5JFXOKzL89rTg7SKgMs/ahmFrdgnRy4NMEYFBZQoxrJz TM4g== X-Forwarded-Encrypted: i=1; AJvYcCVhvjW2MHBXCRzYUWqyhP2hlf9fus5ZFLvs8RTFDxO4UAsxPApbaSeXIIjCJqDWxDI38dzKLiB0HXFAx4Y=@vger.kernel.org X-Gm-Message-State: AOJu0Yy+2wApv544hN2epTP2xD1le/Z3ZrszhvowuxrC39alsIw/TECb 8DVaVOUNSue99Je7fDGyoZX0Y60eQFbPnvhL4hF+MjKF57G/XpjLxc69 X-Gm-Gg: ATEYQzwEg4bEvYxbLXKH8/UghvgF9jaDJW6hLREET8WzRi9RQTBlUwgLElY5kncbA4j TFVR2WXB2rHQc9v0cw9wtP33xI9RgxZx6eWtj9fzOs0jxw5zpgmfGhRJyCp9gpJQTP2i2Swt5DX xFYYjuYO6Yy76S3Pg8x8uhnKinfx6fP118EUzGLN9gtrVHFPFS0AN9sVLnwXHfX1dAPZPLK6vbC f9MimwqgAjfj989zVdw6x4V2t7YPM8JYdJROuYzd8xJRDAjA5aEMApynstLSmbcQklw6r/Y6vZs 4qEeOOBhZGpaS2JVJJC4AjsN/+9eJJOApvZ47SSr5n8P3CEzBKWVTKCKz3agg/dC4hhv2WcM9rc bPRLgkyjzP3N3j/EFnyzZZ5KJSC95XT3LjemnbAboEF85PSU5vHIA5kRX++HSdltp35kncDlr3u 6DtSqj0gvYQ7opjBsOkV7jPe03i9od7r2Etmndv55VZKJ+OgONNhIQe9o= X-Received: by 2002:a05:6a21:6317:b0:398:a060:a967 with SMTP id adf61e73a8af0-398eca11578mr11470604637.11.1773660556858; Mon, 16 Mar 2026 04:29:16 -0700 (PDT) Received: from localhost.localdomain ([116.128.244.171]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c7401588ecesm4684940a12.32.2026.03.16.04.29.10 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 16 Mar 2026 04:29:16 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v8 3/8] bpf: Introduce the bpf_list_del kfunc. Date: Mon, 16 Mar 2026 19:28:38 +0800 Message-ID: <20260316112843.78657-4-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260316112843.78657-1-pilgrimtao@gmail.com> References: <20260316112843.78657-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng If a user holds ownership of a node in the middle of a list, they can directly remove it from the list without strictly adhering to deletion rules from the head or tail. We have added an additional parameter bpf_list_head *head to bpf_list_del, as the verifier requires the head parameter to check whether the lock is being held. This is typically paired with bpf_refcount. After calling bpf_list_del, it is generally necessary to drop the reference to the list node twice to prevent reference count leaks. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 11 +++++++++++ kernel/bpf/verifier.c | 4 ++++ 2 files changed, 15 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index e87b263c5fe6..dac346eb1e2f 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2437,6 +2437,8 @@ static struct bpf_list_node *__bpf_list_del(struct bp= f_list_head *head, */ if (unlikely(!h->next)) INIT_LIST_HEAD(h); + + /* verifier to guarantee n is a list node rather than the head */ if (list_empty(h)) return NULL; =20 @@ -2463,6 +2465,14 @@ __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(= struct bpf_list_head *head) return __bpf_list_del(head, h->prev); } =20 +__bpf_kfunc struct bpf_list_node *bpf_list_del(struct bpf_list_head *head, + struct bpf_list_node *node) +{ + struct bpf_list_node_kern *kn =3D (void *)node; + + return __bpf_list_del(head, &kn->list_head); +} + __bpf_kfunc struct bpf_list_node *bpf_list_front(struct bpf_list_head *hea= d) { struct list_head *h =3D (struct list_head *)head; @@ -4549,6 +4559,7 @@ BTF_ID_FLAGS(func, bpf_list_push_front_impl) BTF_ID_FLAGS(func, bpf_list_push_back_impl) BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 64c1f8343dfa..e928ad4290c7 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12508,6 +12508,7 @@ enum special_kfunc_type { KF_bpf_list_push_back_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, + KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, KF_bpf_cast_to_kern_ctx, @@ -12568,6 +12569,7 @@ BTF_ID(func, bpf_list_push_front_impl) BTF_ID(func, bpf_list_push_back_impl) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) +BTF_ID(func, bpf_list_del) BTF_ID(func, bpf_list_front) BTF_ID(func, bpf_list_back) BTF_ID(func, bpf_cast_to_kern_ctx) @@ -12644,6 +12646,7 @@ static const enum special_kfunc_type bpf_list_api_k= funcs[] =3D { KF_bpf_list_push_back_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, + KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, }; @@ -12652,6 +12655,7 @@ static const enum special_kfunc_type bpf_list_api_k= funcs[] =3D { static const enum special_kfunc_type bpf_list_node_api_kfuncs[] =3D { KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_del, }; =20 /* Kfuncs that take an rbtree node argument (bpf_rb_node *). */ --=20 2.50.1 (Apple Git-155) From nobody Tue Apr 7 04:14:22 2026 Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B797E378D8D for ; Mon, 16 Mar 2026 11:29:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660565; cv=none; b=i+BrfaRw8fLGYz9PT9X9FI+KI0tN/rv5BxAnjDzV3uSOkDt/5inBbqLLWQ+loHQ4rXDUgbmGcH7PLFPArV2CjG/S3Y7senOff18Au8rAOLeN+N0jEEtxlvNfM8lPcxYarc2yZACq9neuhwOPZyeUdkcURTZjHxYcnLGxr0eB/l8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660565; c=relaxed/simple; bh=NqfU9bo52FgrDZTokYHoNf2PxzwuddES9W5iZTUFhMU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pWOGJjb+CtxZKmSeH05OYGFkDOoWcgdlIxxR4AV65K8NSi0UyL64AAlc13paW+lgHW3VmjUO43it3hFwjJybaHudeDfNjZ6V4f4YQQCqetKhGDFVBs0OhpkzOKUhZag01PTs2htGsCsgESbvCihbxEqPkdKfI1GAruNs3Leqa4g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MBtz2wHK; arc=none smtp.client-ip=209.85.215.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MBtz2wHK" Received: by mail-pg1-f173.google.com with SMTP id 41be03b00d2f7-c74029e07d3so373261a12.2 for ; Mon, 16 Mar 2026 04:29:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773660564; x=1774265364; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uOOxYXSdyRk2XDewcy/Jq7LBl3oSgedlPl85DwcVZpI=; b=MBtz2wHK6Qjfc4JaYc0KYGQKJrAOORzHKxDwHHQqsseo+RvD2s2HgDcdUKQsng2egO ZaiQFDKUGcgGu99CNd1eNoo3cTzC6phsbHuQ1f+PvQSLPqo9NteDGmENAuaO2YIo/xIS qPx91AQ9OkgOi1tsvqsoPYor4aG2Mh42pNdzvEObNpTrqMSXGJrRcFWS+DFCCPP+BkO9 dBHwODAZl57pTe6WEoFSFVndq0LxpJtnG0XTiXRN3Fkl+cP4TGGhqIvmdnM+hjDCYd32 H+yHcXk5JVRd8TXUWRVXWuJTq0w2mbOm9kUx2WQ/kbWL12KpIf/N1sFnjA75zsZ8jV44 Mc2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773660564; x=1774265364; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=uOOxYXSdyRk2XDewcy/Jq7LBl3oSgedlPl85DwcVZpI=; b=Cl2psbTd7g6y0UjxfU4TbdW/07CmwiuSAXzVvRdFbqM9edRUt3LelwoHZIg1abuk1Q 5/UEjLPYcgj7R1fOTwxK9N2BWP/oCZEuG0Lbv0kFEooEuDZUVw/ktR4rLZQLWDi7tmM4 K7yn/FbJT4OajJWUf+AEQ37Vp7Txa9toXgj2QAbKZBLosexqX+il6ZiNA14+jPSu2PZD mAzkLY0Dt9tfAp4HFCAWj1ljafVD+IoKPCUbkf+b59LVV6vgs7zw2OjKsKsQiYqPABC4 DddxFnbkRQnzKiGtV/MSiM4M1uwQB1yQpELkfZBwpZzCo5viIUGChIB7lPMXhDmPcANP koag== X-Forwarded-Encrypted: i=1; AJvYcCUWsL9Pz08mpkyqrermhICM48FNqHN4q0R6laStS/yRr9uSZeqMOdGUUnXiNtBq3qejc0P8sLo3KWI9vfg=@vger.kernel.org X-Gm-Message-State: AOJu0YwMPF7NuNn34vcnMpomb4XEcUeq0FIgwjDRUGvv1Sh3DQuHwrqg iXBLeDb9mTiJ+y21VfRq6ARUJ9rSxRDo5RoJWhtKVC4Ufc4I4lIjRQlB X-Gm-Gg: ATEYQzwbsBWIXAO0H2yVDAox7oy7msmpuett8aM+DcazvEe1PqOQKX01O+NKH4R3Zlr 7WLYrtyYwK/fSEekn/wGFRGOY/HgQNMVzWdLbLBTov9GPYPi8k1p0d4PYKtwoujnfY2LxBgrpcA ynTYpN0iVsD2eU+Ly9tETfk98VyUTOuRlAVjoxhZ8fFWFvvSCYbLObvXBwP4XPpAwVH6xdqGQIX bll24hPaMxoUWWAvGo7kzHuQ3vbcz98IS+bCPG+Yw6iguukh9wbJoFrqQcYDC0t9wuv2PSHZfG4 uUQJ+DJrKHPH9dAuqylu0zd/AGrT4SkR/VwlApb2CDSin7NC6Pjq7UjCGE4wDDYh+fGkDIdllPE HFEcmevo5BoWBVnXNVfG5QG5f9svrpveW/iOwBFJzaOuEP5QUIua3+DLDGAE9dxASYHYgeVm4FC 2YGLta6YNmR5JkgMTZ2iEpWNiPnr8VnVd2F/Uo8KWImsShus6PyO9VC9VhqIU2xG59Tg== X-Received: by 2002:a05:6a21:a8d:b0:35e:8b76:c960 with SMTP id adf61e73a8af0-398ecd38ab4mr11422656637.48.1773660564045; Mon, 16 Mar 2026 04:29:24 -0700 (PDT) Received: from localhost.localdomain ([116.128.244.171]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c7401588ecesm4684940a12.32.2026.03.16.04.29.17 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 16 Mar 2026 04:29:23 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v8 4/8] bpf: refactor __bpf_list_add to take insertion point via **prev_ptr Date: Mon, 16 Mar 2026 19:28:39 +0800 Message-ID: <20260316112843.78657-5-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260316112843.78657-1-pilgrimtao@gmail.com> References: <20260316112843.78657-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Refactor __bpf_list_add to accept (new, head, struct list_head **prev_ptr, ..) instead of (node, head, bool tail, ..). Load prev from *prev_ptr after INIT_LIST_HEAD(h), so we never dereference an uninitialized h->prev when head was 0-initialized (e.g. push_back passes &h->prev). When prev is not the list head, validate that prev is in the list via its owner. Prepares for bpf_list_add_impl(head, new, prev, ..) to insert after a given list node. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 44 ++++++++++++++++++++++++++++---------------- 1 file changed, 28 insertions(+), 16 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index dac346eb1e2f..a9665f97b3bc 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2379,11 +2379,13 @@ __bpf_kfunc void *bpf_refcount_acquire_impl(void *p= __refcounted_kptr, void *meta return (void *)p__refcounted_kptr; } =20 -static int __bpf_list_add(struct bpf_list_node_kern *node, +static int __bpf_list_add(struct bpf_list_node_kern *new, struct bpf_list_head *head, - bool tail, struct btf_record *rec, u64 off) + struct list_head **prev_ptr, + struct btf_record *rec, u64 off) { - struct list_head *n =3D &node->list_head, *h =3D (void *)head; + struct list_head *n =3D &new->list_head, *h =3D (void *)head; + struct list_head *prev; =20 /* If list_head was 0-initialized by map, bpf_obj_init_field wasn't * called on its fields, so init here @@ -2391,39 +2393,49 @@ static int __bpf_list_add(struct bpf_list_node_kern= *node, if (unlikely(!h->next)) INIT_LIST_HEAD(h); =20 - /* node->owner !=3D NULL implies !list_empty(n), no need to separately + prev =3D *prev_ptr; + + /* When prev is not the list head, it must be a node in this list. */ + if (prev !=3D h && WARN_ON_ONCE(READ_ONCE(container_of( + prev, struct bpf_list_node_kern, list_head)->owner) !=3D head)) + goto fail; + + /* new->owner !=3D NULL implies !list_empty(n), no need to separately * check the latter */ - if (cmpxchg(&node->owner, NULL, BPF_PTR_POISON)) { - /* Only called from BPF prog, no need to migrate_disable */ - __bpf_obj_drop_impl((void *)n - off, rec, false); - return -EINVAL; - } - - tail ? list_add_tail(n, h) : list_add(n, h); - WRITE_ONCE(node->owner, head); + if (cmpxchg(&new->owner, NULL, BPF_PTR_POISON)) + goto fail; =20 + list_add(n, prev); + WRITE_ONCE(new->owner, head); return 0; + +fail: + /* Only called from BPF prog, no need to migrate_disable */ + __bpf_obj_drop_impl((void *)n - off, rec, false); + return -EINVAL; } =20 __bpf_kfunc int bpf_list_push_front_impl(struct bpf_list_head *head, struct bpf_list_node *node, void *meta__ign, u64 off) { - struct bpf_list_node_kern *n =3D (void *)node; + struct bpf_list_node_kern *new =3D (void *)node; struct btf_struct_meta *meta =3D meta__ign; + struct list_head *h =3D (void *)head; =20 - return __bpf_list_add(n, head, false, meta ? meta->record : NULL, off); + return __bpf_list_add(new, head, &h, meta ? meta->record : NULL, off); } =20 __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head, struct bpf_list_node *node, void *meta__ign, u64 off) { - struct bpf_list_node_kern *n =3D (void *)node; + struct bpf_list_node_kern *new =3D (void *)node; struct btf_struct_meta *meta =3D meta__ign; + struct list_head *h =3D (void *)head; =20 - return __bpf_list_add(n, head, true, meta ? meta->record : NULL, off); + return __bpf_list_add(new, head, &h->prev, meta ? meta->record : NULL, of= f); } =20 static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, --=20 2.50.1 (Apple Git-155) From nobody Tue Apr 7 04:14:22 2026 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D83CB374738 for ; Mon, 16 Mar 2026 11:29:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660574; cv=none; b=tDUpQT5XVBBBKfGzDte4yoDAFSq6cpxK8bXMqRLeh7pd83x2iB+J3TEpOA+Xt/OWCC3ndj+0GFRxCLCmKkf2/cAhi5pfjlthv+3yChrAkO2YhcCMLDQtMgV3UCGx3BgVqSmBUpShF1YGiEDgJ71zjT86XwX1ekwbPvAWcSpGCHs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660574; c=relaxed/simple; bh=f254u9m1QCRh+1RTnHbpIKWIkCtIaFhXm7XPT4JMgW0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Wr/xVETBKLkeMQxBo68jUf2L/WkfGwvpcWLMAfVApeDAvQaWCnZxft5z1qqzLVqyJcKSNdh9snMa9jlkmUgBATQ3m82BxCv5ucJFG6kc7CLlpqsikm3uQVWT429+oFIXQpYr9HKWFsgZtlqIj1pkdWedslwd/rsLxiqz4fTTAgg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IQPRwHW0; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IQPRwHW0" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-829a568f3ccso2413630b3a.0 for ; Mon, 16 Mar 2026 04:29:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773660571; x=1774265371; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W2EXT8xQIbLD8Uq3gaj+lgUvs/CfjmF64+/VehA8xmM=; b=IQPRwHW0hdmJVPw/QB5HYZNDRiS8IfS4WLz41X0HY2Wi8q71hSicH9jLwWCntcPxe4 RiF3YoIXAi++m72YWJzYt/jzVQsQ/zWJdVcrwi6JD1IY6oy7/Ek5YOQXiusUo7tGDxJO bCvJo1v5OjpW+UaZN7dhS6XfJx3qWvq20sbck3xG9Ud6NDuKtLylJxLanOr1//J1Vwf1 wPt/ky/KNhLkml5FdBKck+cQJW9ne8NiIKn4dXyCpv9sTMDqYf4/a/VNfGNV1pJAzyNV trgC+OXHldlww/0TJHygLmkC+RQ9ti/iIV0FUlqrwnYyU1pXHFYFZsblfJrmaQipglgL lJSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773660571; x=1774265371; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=W2EXT8xQIbLD8Uq3gaj+lgUvs/CfjmF64+/VehA8xmM=; b=hadXLu/EE0O038tZ3ZdZzS/ZPH2vUNTQ6xY6iw+NAcxtk7wmCQjkkRniHeb3Kzx0e1 EIrlXwKvK3FaADQwDo1esYMkZfujgPJ8IbCZtMZLWLLfn7iFfZkDA1AlGv8NmllIk0nQ 4qz6AyOaVD/7u5IYxmVGf3pBlmdwgYiv/1CPAsEMKurr9j3rlpPlGLWXg39rihWL222r +z7bLfQtXfAfx/UAsfxErU0g4ziQtAV4JaRmQVQnIPzFWPKYVC3Wnb+zNx1JcI4OuGbl hJXMPw88yyL+M6SeN1y8O2/cF0DYAqkcB7D8dvAFMiSyvJddaS0Z70iGzFNOLj4L4lFf Arew== X-Forwarded-Encrypted: i=1; AJvYcCUNSIQykQPyxNvky8tOAnh4YTnmT68oWWFym1VAxW6p8V/JwOFP3LQRYBIzWOuzIWhytxK6HRFl2aPohgQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzEsQ1EpZ0HHQv+0XxZzZ16zvCQLxr2/JxMU+ChKrj70wdNKdk6 6WTM4oBbrWiS+4vQs1e72qBUfYCQhCKzdxm6CK/Khjai3bn2H7GFHCkm X-Gm-Gg: ATEYQzxEz8eEcQXkbXM62yfcTtvYrG5/MbW0QW97bI+urxhh9JnLNZ1AQAP5Hswgvr4 xNiw/WqkKVaFQ0CrKjzdZIJRqddYrzSOSVorC5o+WIt4ND0L9p9BArG2yjgWHHkgocQd3tb8Vf7 URpeBylw9pM0W4Lfibj1/HZP4fZeup/OnECRCc9TJHhZnu17SanRV1HSmScYNkNSoeSLfODsutd yWlifnpBukCXzMlv2V6SX051zx2d2Ek5nFfGdjcUbS8sWG8C1VYYQrckKCJr8iFJUN9H5EAnE+v Y2DydVgb7B88/kle48RdbIhEZcgtlWtkBvsDRg6m54Ttay8jeDR26AEjwzGYE7t7AiW7JCoKcHA vsp+uVC+Q1aOEIzE1Tu5RsNM3n5en6rjeWP1uZ9mpKnRuXT7QU/VbVLaxynM7Im+blP27prWRdo z+hapWCzty0PgFbj0T0u2sIrgjeGRS8IxHggrA2+vyebBOnuZSZmkX5D8= X-Received: by 2002:a05:6a21:7106:b0:398:c3c3:8686 with SMTP id adf61e73a8af0-398eca2d969mr10979592637.4.1773660571067; Mon, 16 Mar 2026 04:29:31 -0700 (PDT) Received: from localhost.localdomain ([116.128.244.171]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c7401588ecesm4684940a12.32.2026.03.16.04.29.24 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 16 Mar 2026 04:29:30 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v8 5/8] bpf: Add bpf_list_add_impl to insert node after a given list node Date: Mon, 16 Mar 2026 19:28:40 +0800 Message-ID: <20260316112843.78657-6-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260316112843.78657-1-pilgrimtao@gmail.com> References: <20260316112843.78657-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Add a new kfunc bpf_list_add_impl(head, new, prev, meta, off) that inserts 'new' after 'prev' in the BPF linked list. Both must be in the same list; 'prev' must already be in the list. The new node must be an owning reference (e.g. from bpf_obj_new); the kfunc consumes that reference and the node becomes non-owning once inserted. We have added an additional parameter bpf_list_head *head to bpf_list_add_impl, as the verifier requires the head parameter to check whether the lock is being held. Returns 0 on success, -EINVAL if 'prev' is not in a list or 'new' is already in a list (or duplicate insertion). On failure, the kernel drops the passed-in node. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 14 ++++++++++++++ kernel/bpf/verifier.c | 13 +++++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index a9665f97b3bc..dc4f8b4eec01 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2438,6 +2438,19 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_l= ist_head *head, return __bpf_list_add(new, head, &h->prev, meta ? meta->record : NULL, of= f); } =20 +__bpf_kfunc int bpf_list_add_impl(struct bpf_list_head *head, + struct bpf_list_node *new, + struct bpf_list_node *prev, + void *meta__ign, u64 off) +{ + struct bpf_list_node_kern *kn =3D (void *)new, *kp =3D (void *)prev; + struct btf_struct_meta *meta =3D meta__ign; + struct list_head *prev_ptr =3D &kp->list_head; + + return __bpf_list_add(kn, head, &prev_ptr, + meta ? meta->record : NULL, off); +} + static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, struct list_head *n) { @@ -4574,6 +4587,7 @@ BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF= _RET_NULL) BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_list_add_impl) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE) BTF_ID_FLAGS(func, bpf_rbtree_remove, KF_ACQUIRE | KF_RET_NULL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index e928ad4290c7..98ddb370feb5 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12506,6 +12506,7 @@ enum special_kfunc_type { KF_bpf_refcount_acquire_impl, KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, KF_bpf_list_del, @@ -12567,6 +12568,7 @@ BTF_ID(func, bpf_obj_drop_impl) BTF_ID(func, bpf_refcount_acquire_impl) BTF_ID(func, bpf_list_push_front_impl) BTF_ID(func, bpf_list_push_back_impl) +BTF_ID(func, bpf_list_add_impl) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) BTF_ID(func, bpf_list_del) @@ -12644,6 +12646,7 @@ BTF_ID(func, bpf_stream_print_stack) static const enum special_kfunc_type bpf_list_api_kfuncs[] =3D { KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, KF_bpf_list_del, @@ -12655,6 +12658,7 @@ static const enum special_kfunc_type bpf_list_api_k= funcs[] =3D { static const enum special_kfunc_type bpf_list_node_api_kfuncs[] =3D { KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, KF_bpf_list_del, }; =20 @@ -14345,6 +14349,7 @@ static int check_kfunc_call(struct bpf_verifier_env= *env, struct bpf_insn *insn, =20 if (meta.func_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] || meta.func_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl] || + meta.func_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || meta.func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { release_ref_obj_id =3D regs[BPF_REG_2].ref_obj_id; insn_aux->insert_off =3D regs[BPF_REG_2].var_off.value; @@ -23357,13 +23362,17 @@ static int fixup_kfunc_call(struct bpf_verifier_e= nv *env, struct bpf_insn *insn, *cnt =3D 3; } else if (desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_= impl] || desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] = || + desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || desc->func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { struct btf_struct_meta *kptr_struct_meta =3D env->insn_aux_data[insn_idx= ].kptr_struct_meta; int struct_meta_reg =3D BPF_REG_3; int node_offset_reg =3D BPF_REG_4; =20 - /* rbtree_add has extra 'less' arg, so args-to-fixup are in diff regs */ - if (desc->func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { + /* list/rbtree_add_impl have an extra arg (prev/less), + * so args-to-fixup are in different regs. + */ + if (desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || + desc->func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { struct_meta_reg =3D BPF_REG_4; node_offset_reg =3D BPF_REG_5; } --=20 2.50.1 (Apple Git-155) From nobody Tue Apr 7 04:14:22 2026 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFADA34DCCC for ; Mon, 16 Mar 2026 11:29:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660579; cv=none; b=Y5Y4j4UDCR63petji1p1A28W2/eVZZuXJxgb8AuQkgzj4pX3V6E331Lp9Xk/gIkC2QQMBKW9eawhAQFzb4F3tTHHA669q3T0Onul4fI7YwE1z8FESxBXNHPfc/JG6aCWIEMv1C7b08hC4x98tf7+raxritR7lQoWuTeCJstLpjQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660579; c=relaxed/simple; bh=La0oxEXQHp19d2nfILBob+w4HOT1QdSwN6buq4oJMss=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WiDAZslVF5Z8QgLcDnPNIcnUCqBHkYg/uA/e9NI4MPH/ePx+iI7DA8BK/6ZbREzI9hxg3Ql28R0EC0ytLMyPszT51u1CbVVzvGh4pwMrLZ32x50C5W0H0topPdQCXO/5bvvrrqlGlyGQojjmQNT64HK6N2DK36ujq9kMqUjfdSQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GLtOWS+g; arc=none smtp.client-ip=209.85.215.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GLtOWS+g" Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-c70e27e2b74so1527309a12.0 for ; Mon, 16 Mar 2026 04:29:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773660578; x=1774265378; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CbAiNsDyIMAUQsD2xc40hS2bONCwm72xxTw7O95hbw8=; b=GLtOWS+gxB0+/9R5HXYsnUJsT5nYFt+dfG5i1Ixsq/cSjtFZRhu0lZ7mDosAcpcwEc 5231JfNKHzMA9VHusbBs8EwmPkvZjAQfXOk2UEwZZ99qCxdpOasNw+bGKk+wg72W2msq Yjh09Z5M+rik487o1bbqbymlhAa/74WoLPFcE2Hx0NmBkQiaLLUacdG67U1uX58L3vIh HYDD0q+TSqOL+N28koFeB4ynQDVKZfl4gLQKYvFGJWSK1xn/KaxEESSVt7MhRt4FHWTI x5O+JV03Iv4AzDiiXTzk/B4LTNuh5tVlbr+PS946hkzWyINX6Btu9RGMX6AWTi2e+6uU +CAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773660578; x=1774265378; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CbAiNsDyIMAUQsD2xc40hS2bONCwm72xxTw7O95hbw8=; b=T50dJfJYtrEtaPTkqDYU3/a86C3TZ3Lc2H8yO/ZtVwC9ZC57JSdnu8QEodTkrR+uRQ DxT8Dty0XaJhGSkAojLQM1LFWVOZKkWwTU1sE1qPbRxhCdE3O3bpyQr0KhRu8tBFKiMm wB1vfLoowhzdbBP0xZ2Pxp5ZNOZCUNzx/CLs9VHGWGxefAHlq/HBKnWpH6vlx6G3L5rW UnVjDeBnpTzORjjV8OCCPXFvuKBtWLjQ1uMeVbDEVVPDfJDgRKmXWPz1FNhcZ9mhmlQL BPuIPjyFOXxbm4saSlbaDXfE1ipxjqg2IjFUx26rS7891QzdHeTQGz0mIodNqBNpVt45 d8Kw== X-Forwarded-Encrypted: i=1; AJvYcCWjsVmomMiK8TpDXlL1UTGaeDyoaZ9AaQFHWxNqbOyOBDT1c+9Yc3Mj9oxixPFS3vJqeu5X791stcRlx0c=@vger.kernel.org X-Gm-Message-State: AOJu0Ywedd0fetyz/YoBUIRBWLBWOuY3UgV24svl4yoROnZs3i2F4dcQ w4DDPj78ggnCLaFEbChMQxIa5ff+TpM6hJVqOhloyJ+3vBH0YzNTD6A3 X-Gm-Gg: ATEYQzwycI22GRbU/jttANd7DyR5R3p7c9t7b+PshaRmPJ1oVRnr6Nh64s7dLh9uSN+ e6QHHqFKFsfhOg5u1bmvz7GQGNODis5hYV1YMUFGhGB/kUt6gHZyWY5XQzl2M6KiiAVvFIr6a7r Js2OgF6XjtglthhiKr45XW8r6otnObNB1HcMpqGdPL89MkcaOmvIviNhLSPuSbldiZtrexZk/Uv jQ/x5vII9phohO8n7NuQ4rR3cMJnZsDARGZicT3FLrvT/bGxLwWxZxoVJXcKx38/KnIZoq9hvNN ypgIc+GzTwpG6l4PnD6IdY5c4JGxDj4zj7hs076pPa87fGneGCRJy2L4Tn7OqLyosatCEQlFk96 LtOpN+98MNTn0v/4U9FNkwWcKpWQrZ1AUiYOvoztJj5jEvtMqof7lcjn+f33EP/fUa0dQ1uRU9f ywDodozA96Ta80Q0fzG+oAqTau0nRMNnlF7JBc++NLpeWacM/J9+oOtl/45RhUDMeSZw== X-Received: by 2002:a05:6a21:7a97:b0:398:78ca:c9d2 with SMTP id adf61e73a8af0-398eca24a5bmr11968023637.15.1773660578050; Mon, 16 Mar 2026 04:29:38 -0700 (PDT) Received: from localhost.localdomain ([116.128.244.171]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c7401588ecesm4684940a12.32.2026.03.16.04.29.31 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 16 Mar 2026 04:29:37 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v8 6/8] bpf: allow bpf_list_front/back result as the prev argument of bpf_list_add_impl Date: Mon, 16 Mar 2026 19:28:41 +0800 Message-ID: <20260316112843.78657-7-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260316112843.78657-1-pilgrimtao@gmail.com> References: <20260316112843.78657-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng KF_ARG_PTR_TO_LIST_NODE normally requires an owning reference (PTR_TO_BTF_ID | MEM_ALLOC and ref_obj_id). For bpf_list_add_impl's third argument (prev), allow a non-owning reference with ref_obj_id=3D=3D0 so that the result of bpf_list_front() or bpf_list_back() can be passed as the insertion point. When prev is such a non-owning ref, skip the MEM_ALLOC/ref_obj_id checks and jump to the shared list-node processing. Owning refs (e.g. from pop + refcount_acquire) still pass the existing checks and reach the same label. Signed-off-by: Kaitao Cheng --- kernel/bpf/verifier.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 98ddb370feb5..0a6f2e6a5d28 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -13737,6 +13737,14 @@ static int check_kfunc_args(struct bpf_verifier_en= v *env, struct bpf_kfunc_call_ return ret; break; case KF_ARG_PTR_TO_LIST_NODE: + if (meta->func_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] + && i =3D=3D 2 && type_is_non_owning_ref(reg->type) + && !reg->ref_obj_id) { + /* Allow bpf_list_front/back return value as + * list_add_impl's third arg (R3). + */ + goto check_ok; + } if (reg->type !=3D (PTR_TO_BTF_ID | MEM_ALLOC)) { verbose(env, "arg#%d expected pointer to allocated object\n", i); return -EINVAL; @@ -13745,6 +13753,7 @@ static int check_kfunc_args(struct bpf_verifier_env= *env, struct bpf_kfunc_call_ verbose(env, "allocated object must be referenced\n"); return -EINVAL; } +check_ok: ret =3D process_kf_arg_ptr_to_list_node(env, reg, regno, meta); if (ret < 0) return ret; --=20 2.50.1 (Apple Git-155) From nobody Tue Apr 7 04:14:22 2026 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01E2938236E for ; Mon, 16 Mar 2026 11:29:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660587; cv=none; b=mGLjt9rAUmk0UbBTBR4lgnGIs7kBDSmWol2vWIWWQbLzeTDEXdBlm05vA5Wrxl+VDTbb6uArKeVMa6yy2JHRL5zxLxp3tIiwUYtn2gFGX6+hhUK1LSUc8/jr6RaRIxkjrb3jiDUfhheuCoocMn5CBBjCau2ynzD+kjwNpDASGnU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660587; c=relaxed/simple; bh=fkvvyCOJcW+OjU44LfyjgAgZvKAy8Us0SsZ8+/q2mLo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p3uaFyA2KuH9b5Y+Q0W64NwuVRp0iugViorn6VyRhgBw/9s//nN0fbS7AszKUfsjadXjz24b6ot79LjPTKWYw/nRtX5hpNx7Qas+KH4VR5BccGn2Snj/0ohz99kb6by05BSfvQWnEBeXrwM7sPC1SA267UXbiJyxLQUTJljDgFA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CbBTNKaQ; arc=none smtp.client-ip=209.85.215.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CbBTNKaQ" Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-c06cb8004e8so1779603a12.0 for ; Mon, 16 Mar 2026 04:29:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773660585; x=1774265385; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+XQhRYOi6IQvYzgEEWmUGpE0n+4jdX31PpEv2AHuppI=; b=CbBTNKaQjJnFYn+G6p1nEp1W2MVa6/LVBtq4TuSCCpIoshl9gCghe61/Dm6Uex6jKC tUTsRKOtd6/RRpP1PCKVDEOvaHtlvn+wwnXDbRB2MdMZbDs+fkdBkMa9IlwYlwSc3zxu L8B7AcuG3Hh86kGsGdYuALh+S6vLNJFh11GN1zmHJUeD4FmHX7N5fAUqq+sXES3iM3NU 2DAVPXI4OodeKnl49g8wN3DVea/Ak1PWN16mQyOfdTyytOWjFtgkaYtzxp7DKBDNccpj YIjAVr8636+slQooj89QESuYXCRGpjZ00oZnQmzEyCxBGLDRMX89CObVC94xfnfW8/08 peNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773660585; x=1774265385; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=+XQhRYOi6IQvYzgEEWmUGpE0n+4jdX31PpEv2AHuppI=; b=mYN8XSDEm44rWTug5OiNkMA5jVxaD3M3Y//T/6ivfB5QE1kqvnYRorPRjVzTSlnbMb +zxhngwf0getXNlOdrkhJ2T+V8Ymaz6kRiCK4/0jAM9vkKkqvSGh7dMOnuiDvjpU+Xrg gnCRdxF1WtvAMcPbpm7S9DP0m57WG7Vd6CUaxlbxNB5JHf1AMQ9NYhRV5VjMOKo0il/4 fwvB0lxTzhe2ebkC9XoobIQ0NjXeMjegDwHyjYq8PsMdVqYDTXL8S8A/11fr70GSPcW5 gHzpJGjkq1f+QUGfe0mqNsJSVV43D92quaEF06H//fqkS8zN04elJIQ/1iFPG1zWPnV5 c1XA== X-Forwarded-Encrypted: i=1; AJvYcCVco6mYFJs/JtAvgZyQz1aR7tcsm1sgca4MF0v+nyou+mWiKYnxH7/+pkVe73olC2o5QEyT/xKVQjhmfJo=@vger.kernel.org X-Gm-Message-State: AOJu0Yxg571VaoLNQWbcx20ZoAVDQy2v1YuTiCqGPoN78Ob4SX/O1JbT s8SsDXQXfQxf613Wm6OvTlFKH6KnVYdq4PRUZw1TiG3lysigmylZNMfU X-Gm-Gg: ATEYQzzsD4P5z9l2oMA3Y9U3tsqiwWXrmRlxI7s5+FNbEbknXmkwnFs+pIlCC7Q8MPE WBSAkuTIX4eSjM2c8BhWwyDhjbyihuhRan4ECsGvKMbAIB7M6BJfM9fuiHLFnFjRUB/qHGBce2n dr++seHtk97yQa9UItr2KaPqXFE32t2fOp0vwL1vUAnSuOo1dmLV8P4HTaAeo4aWmFQKmMtySul oarme6PrulH2XbdVnm2Ye/x+iAEA1xzia7tNLKz8CjERC5g5/M2fCVSi5xa+COQ3OkG6K9uiTL1 tf0jxclA9Fi7WcnXvPp5xhw5GCz8tHF5L9iPPlnT+EiQm0vK9EUBWSLqAyp8iUIJt5wgfqPzk/E jGdid3zq7gZ75KCnEmVYFrg2LpL2lxVfkYqkYaTbOzVKTFR+Q3lhwKwK+3JouAuhdFfVdj+JYLZ Ow10FRG79874B8b1TRzo/uKOtFd5tmhff8CDbB20rCIKNGquMTl1mCcXw= X-Received: by 2002:a05:6a21:494:b0:398:7d3e:95d7 with SMTP id adf61e73a8af0-398ecabc6e4mr11097936637.22.1773660585359; Mon, 16 Mar 2026 04:29:45 -0700 (PDT) Received: from localhost.localdomain ([116.128.244.171]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c7401588ecesm4684940a12.32.2026.03.16.04.29.38 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 16 Mar 2026 04:29:44 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v8 7/8] bpf: add bpf_list_is_first/last/empty kfuncs Date: Mon, 16 Mar 2026 19:28:42 +0800 Message-ID: <20260316112843.78657-8-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260316112843.78657-1-pilgrimtao@gmail.com> References: <20260316112843.78657-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Add three kfuncs for BPF linked list queries: - bpf_list_is_first(head, node): true if node is the first in the list. - bpf_list_is_last(head, node): true if node is the last in the list. - bpf_list_empty(head): true if the list has no entries. Currently, without these kfuncs, to implement the above functionality it is necessary to first call bpf_list_pop_front/back to retrieve the first or last node before checking whether the passed-in node was the first or last one. After the check, the node had to be pushed back into the list using bpf_list_push_front/back, which was very inefficient. Now, with the bpf_list_is_first/last/empty kfuncs, we can directly check whether a node is the first, last, or whether the list is empty, without having to first retrieve the node. Signed-off-by: Kaitao Cheng Reviewed-by: Emil Tsalapatis --- kernel/bpf/helpers.c | 38 ++++++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 11 +++++++++++ 2 files changed, 49 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index dc4f8b4eec01..da70dcff4ba8 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2518,6 +2518,41 @@ __bpf_kfunc struct bpf_list_node *bpf_list_back(stru= ct bpf_list_head *head) return (struct bpf_list_node *)h->prev; } =20 +__bpf_kfunc bool bpf_list_is_first(struct bpf_list_head *head, struct bpf_= list_node *node) +{ + struct list_head *h =3D (struct list_head *)head; + struct bpf_list_node_kern *kn =3D (struct bpf_list_node_kern *)node; + + if (READ_ONCE(kn->owner) !=3D head) + return false; + + return list_is_first(&kn->list_head, h); +} + +__bpf_kfunc bool bpf_list_is_last(struct bpf_list_head *head, struct bpf_l= ist_node *node) +{ + struct list_head *h =3D (struct list_head *)head; + struct bpf_list_node_kern *kn =3D (struct bpf_list_node_kern *)node; + + if (READ_ONCE(kn->owner) !=3D head) + return false; + + return list_is_last(&kn->list_head, h); +} + +__bpf_kfunc bool bpf_list_empty(struct bpf_list_head *head) +{ + struct list_head *h =3D (struct list_head *)head; + + /* If list_head was 0-initialized by map, bpf_obj_init_field wasn't + * called on its fields, so init here + */ + if (unlikely(!h->next)) + INIT_LIST_HEAD(h); + + return list_empty(h); +} + __bpf_kfunc struct bpf_rb_node *bpf_rbtree_remove(struct bpf_rb_root *root, struct bpf_rb_node *node) { @@ -4588,6 +4623,9 @@ BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_= NULL) BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_add_impl) +BTF_ID_FLAGS(func, bpf_list_is_first) +BTF_ID_FLAGS(func, bpf_list_is_last) +BTF_ID_FLAGS(func, bpf_list_empty) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE) BTF_ID_FLAGS(func, bpf_rbtree_remove, KF_ACQUIRE | KF_RET_NULL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 0a6f2e6a5d28..c2cc622bb3a1 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12512,6 +12512,9 @@ enum special_kfunc_type { KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, + KF_bpf_list_is_first, + KF_bpf_list_is_last, + KF_bpf_list_empty, KF_bpf_cast_to_kern_ctx, KF_bpf_rdonly_cast, KF_bpf_rcu_read_lock, @@ -12574,6 +12577,9 @@ BTF_ID(func, bpf_list_pop_back) BTF_ID(func, bpf_list_del) BTF_ID(func, bpf_list_front) BTF_ID(func, bpf_list_back) +BTF_ID(func, bpf_list_is_first) +BTF_ID(func, bpf_list_is_last) +BTF_ID(func, bpf_list_empty) BTF_ID(func, bpf_cast_to_kern_ctx) BTF_ID(func, bpf_rdonly_cast) BTF_ID(func, bpf_rcu_read_lock) @@ -12652,6 +12658,9 @@ static const enum special_kfunc_type bpf_list_api_k= funcs[] =3D { KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, + KF_bpf_list_is_first, + KF_bpf_list_is_last, + KF_bpf_list_empty, }; =20 /* Kfuncs that take a list node argument (bpf_list_node *). */ @@ -12660,6 +12669,8 @@ static const enum special_kfunc_type bpf_list_node_= api_kfuncs[] =3D { KF_bpf_list_push_back_impl, KF_bpf_list_add_impl, KF_bpf_list_del, + KF_bpf_list_is_first, + KF_bpf_list_is_last, }; =20 /* Kfuncs that take an rbtree node argument (bpf_rb_node *). */ --=20 2.50.1 (Apple Git-155) From nobody Tue Apr 7 04:14:22 2026 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1CDE374738 for ; Mon, 16 Mar 2026 11:29:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660594; cv=none; b=Rl7RHEFnYa/2ChUmV8VRrYyKFkJMhdhRd6ROzq/47/+6aDFSJFFytbQhTMWKNaDTeXzfB2X/ZZJaRCsjw46dBqLlUvmJ0VDZGhQVxQfOcKmslKPwAp6ZhCx0aTDK/9KfsrDO5vHAmUOK2cqcRfKg1OLxPll5Cf9jtzKaRQ7iBLA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773660594; c=relaxed/simple; bh=iFNWymfmfxX/QJqdbdhLQocWSeqa+lLLvreHPgxAa/o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rUVfwmkVGeaFs3xw2qKLprE/XRxvR5bSAkyxTYIttKRojRxkplLEFiM+Wq2nk3DaSVZ/Zwlrqa2dfFLB+KDEEkVTg8HxvIHNzRhvv4d+YqoliRVLXGqu5m3d6n8wh6dkkSUzZ9ZB/7dCpmwNtPv8zhnKJYISthMZrNhH9JpJXEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OSrMChax; arc=none smtp.client-ip=209.85.215.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OSrMChax" Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-c73aabd620bso2959808a12.1 for ; Mon, 16 Mar 2026 04:29:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773660593; x=1774265393; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sZLdK6dkEj8TjVSrlUlFfcQoiH20fwv6lL/iTYgpM30=; b=OSrMChaxTPi4rDkJrzJHNphoewSkJtP5z7riBpE2UEna6QWdR8LbiXTQG3+mFnfa5s btHmPhExpGSCLUAMdyKWw+1Huy2EbqZDP6kVlEYskK7LaNubH0TLP+0dK6iuk7SFMz0j 20MyrehWCqBTVe5yYbrMqv5ofIW/Xmdp5nWr2MhjqMmsrYnBoLWe5EHZSJTL54RuQmI7 x2ceCsGGuayqUps9/n18uygPhisCZCIHHcSm8SmY3vjfuHlZhH0l89FQF5+kH2kXMo4Z n3xI6ZcKMo9s9jRh1aWrhg59+W04FEGsHJiNht/YJBykyvTEA1HePlaXRjWgih2TT2gT wqMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773660593; x=1774265393; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=sZLdK6dkEj8TjVSrlUlFfcQoiH20fwv6lL/iTYgpM30=; b=eokcwWuCccCRzkKF8hgAA7eL1l6xyi2JvGoMMS4JH6jaEmKc8B13yU+JuC517NrkAn 6x2JisW4eKkVMB2ykBXvgyiH2WnWomVXhEEnv/HP22ZNBuTuIIVCechKfqzGWaMtBpLA HDT6OegNTupM8Hj+ciyLvlwyYrHEMG2zJxycoVb8CfLQ+Nj/PpaGNPQ9ZXN4sS0bVFX4 xN0iSoZWfK/QFF9Ghp2RQ4lOWUu2tf5EdRj0c5kUejtywXWdO4NYmDRDiM+3aYuiIvcz KBthPaB47I6Z4EDA/mZcDKWQiRtQu0r6bn6aoymb4oUpd49749q0of4tw6Rvf5DngNAL AASg== X-Forwarded-Encrypted: i=1; AJvYcCWRsjZXrvDfLf2jljuGXbwQoSKpZXmRrbcNsiC8MaYyt0skcvIF1gkkVtSfXHOSqk7pWxgwDHgtpZxhLvk=@vger.kernel.org X-Gm-Message-State: AOJu0Ywgp6/Wu2FbOoU5swbgQEoywrIoNPj2kvGdZJZ6OtKXDFh6Xllm qzh+0sQ9PV2eMFMSio9KU74M5XNjKp3BjtIyEoc62r1ZUVazwgRS4w5B X-Gm-Gg: ATEYQzwdsUSnFtg3V2xy2ZUJi5qsxKYj/S0CKhiMtIS7YWPbuTdcLrkQ2yAEorcr6wP tnVOtOJ8BVZ3bq27DV9Aoe3bSFb3QN/geuTXdKToy0FFOuB/PHbcbuWDwmiscYGqOV1X9+Epexh OWdDBzQKwEfUj0tU5wG7XyISN6PjolGMr5tcWJOPwzN9BflKS/ihDVypWNghQhdzLzsxdcuQPWa kHjmgxIChYu9X3jPhbMnEmL8pnN3Pmg/9yX0XZoPUCaciJEIsHRa89Oqjtw+spbUWvPlS1dqLtR PGafsdylmCNJaL27EmCHnwYjzYRhd9o6ZmMu75wa1GRs93U3dNdDSO5JfIe3+QswSEVi/ItS3DT QqZV+8Y9Q3FnyoKRzVE7YYEsIzKA2jWD3ZQOcLqSUGbieEB1QJsnkJkeR73m9lf/Ji/cctKH/3W IZb2FPgydYRFTW7KyH1ne/Rta7KBG1e6rWeBW/xgs2GQinOuaKL1o4vNc2i47tos3ZlQ== X-Received: by 2002:a05:6a21:685:b0:398:6ea8:21d8 with SMTP id adf61e73a8af0-398ec9b4a31mr11583444637.15.1773660592920; Mon, 16 Mar 2026 04:29:52 -0700 (PDT) Received: from localhost.localdomain ([116.128.244.171]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c7401588ecesm4684940a12.32.2026.03.16.04.29.45 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 16 Mar 2026 04:29:51 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v8 8/8] selftests/bpf: Add test cases for bpf_list_del/add/is_first/is_last/empty Date: Mon, 16 Mar 2026 19:28:43 +0800 Message-ID: <20260316112843.78657-9-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260316112843.78657-1-pilgrimtao@gmail.com> References: <20260316112843.78657-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Extend refcounted_kptr with tests for bpf_list_add (including prev from bpf_list_front and bpf_refcount_acquire), bpf_list_del (including node from bpf_rbtree_remove and bpf_refcount_acquire), bpf_list_empty, bpf_list_is_first/last, and push_back on uninit head. To verify the validity of bpf_list_del/add, the test also expects the verifier to reject calls to bpf_list_del/add made without holding the spin_lock. Signed-off-by: Kaitao Cheng --- .../testing/selftests/bpf/bpf_experimental.h | 16 + .../selftests/bpf/progs/refcounted_kptr.c | 311 ++++++++++++++++++ 2 files changed, 327 insertions(+) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing= /selftests/bpf/bpf_experimental.h index 44466acf8083..5821f0000e1f 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -85,6 +85,22 @@ extern int bpf_list_push_back_impl(struct bpf_list_head = *head, /* Convenience macro to wrap over bpf_list_push_back_impl */ #define bpf_list_push_back(head, node) bpf_list_push_back_impl(head, node,= NULL, 0) =20 +/* Description + * Insert 'new' after 'prev' in the BPF linked list with head 'head'. + * The bpf_spin_lock protecting the list must be held. 'prev' must already + * be in that list; 'new' must not be in any list. The 'meta' and 'off' + * parameters are rewritten by the verifier, no need for BPF programs to + * set them. + * Returns + * 0 on success, -EINVAL if head is NULL, prev is not in the list with hea= d, + * or new is already in a list. + */ +extern int bpf_list_add_impl(struct bpf_list_head *head, struct bpf_list_n= ode *new, + struct bpf_list_node *prev, void *meta, __u64 off) __ksym; + +/* Convenience macro to wrap over bpf_list_add_impl */ +#define bpf_list_add(head, new, prev) bpf_list_add_impl(head, new, prev, N= ULL, 0) + /* Description * Remove the entry at the beginning of the BPF linked list. * Returns diff --git a/tools/testing/selftests/bpf/progs/refcounted_kptr.c b/tools/te= sting/selftests/bpf/progs/refcounted_kptr.c index c847398837cc..e5558994a76d 100644 --- a/tools/testing/selftests/bpf/progs/refcounted_kptr.c +++ b/tools/testing/selftests/bpf/progs/refcounted_kptr.c @@ -367,6 +367,317 @@ long insert_rbtree_and_stash__del_tree_##rem_tree(voi= d *ctx) \ INSERT_STASH_READ(true, "insert_stash_read: remove from tree"); INSERT_STASH_READ(false, "insert_stash_read: don't remove from tree"); =20 +SEC("tc") +__description("list_empty_test: list empty before add, non-empty after add= ") +__success __retval(0) +int list_empty_test(void *ctx) +{ + struct node_data *node_new; + + bpf_spin_lock(&lock); + if (!bpf_list_empty(&head)) { + bpf_spin_unlock(&lock); + return -1; + } + bpf_spin_unlock(&lock); + + node_new =3D bpf_obj_new(typeof(*node_new)); + if (!node_new) + return -2; + + bpf_spin_lock(&lock); + bpf_list_push_front(&head, &node_new->l); + + if (bpf_list_empty(&head)) { + bpf_spin_unlock(&lock); + return -3; + } + bpf_spin_unlock(&lock); + return 0; +} + +static struct node_data *__add_in_list(struct bpf_list_head *head, + struct bpf_spin_lock *lock) +{ + struct node_data *node_new, *node_ref; + + node_new =3D bpf_obj_new(typeof(*node_new)); + if (!node_new) + return NULL; + + node_ref =3D bpf_refcount_acquire(node_new); + + bpf_spin_lock(lock); + bpf_list_push_front(head, &node_new->l); + bpf_spin_unlock(lock); + return node_ref; +} + +SEC("tc") +__description("list_is_edge_test: is_first on first node, is_last on last = node") +__success __retval(0) +int list_is_edge_test(void *ctx) +{ + struct node_data *node_first, *node_last; + int err =3D 0; + + node_last =3D __add_in_list(&head, &lock); + if (!node_last) + return -1; + + node_first =3D __add_in_list(&head, &lock); + if (!node_first) { + bpf_obj_drop(node_last); + return -2; + } + + bpf_spin_lock(&lock); + if (!bpf_list_is_first(&head, &node_first->l)) { + err =3D -3; + goto fail; + } + if (!bpf_list_is_last(&head, &node_last->l)) + err =3D -4; + +fail: + bpf_spin_unlock(&lock); + bpf_obj_drop(node_first); + bpf_obj_drop(node_last); + return err; +} + +SEC("tc") +__description("list_del_test1: del returns removed nodes") +__success __retval(0) +int list_del_test1(void *ctx) +{ + struct node_data *node_first, *node_last; + struct bpf_list_node *bpf_node_first, *bpf_node_last; + int err =3D 0; + + node_last =3D __add_in_list(&head, &lock); + if (!node_last) + return -1; + + node_first =3D __add_in_list(&head, &lock); + if (!node_first) { + bpf_obj_drop(node_last); + return -2; + } + + bpf_spin_lock(&lock); + bpf_node_last =3D bpf_list_del(&head, &node_last->l); + bpf_node_first =3D bpf_list_del(&head, &node_first->l); + bpf_spin_unlock(&lock); + + if (bpf_node_first) + bpf_obj_drop(container_of(bpf_node_first, struct node_data, l)); + else + err =3D -3; + + if (bpf_node_last) + bpf_obj_drop(container_of(bpf_node_last, struct node_data, l)); + else + err =3D -4; + + bpf_obj_drop(node_first); + bpf_obj_drop(node_last); + return err; +} + +SEC("tc") +__description("list_del_test2: remove an arbitrary node from the list") +__success __retval(0) +int list_del_test2(void *ctx) +{ + struct bpf_rb_node *rb; + struct bpf_list_node *l; + struct node_data *n; + long err; + + err =3D __insert_in_tree_and_list(&head, &root, &lock); + if (err) + return err; + + bpf_spin_lock(&lock); + rb =3D bpf_rbtree_first(&root); + if (!rb) { + bpf_spin_unlock(&lock); + return -4; + } + + rb =3D bpf_rbtree_remove(&root, rb); + if (!rb) { + bpf_spin_unlock(&lock); + return -5; + } + + n =3D container_of(rb, struct node_data, r); + l =3D bpf_list_del(&head, &n->l); + bpf_spin_unlock(&lock); + bpf_obj_drop(n); + if (!l) + return -6; + + bpf_obj_drop(container_of(l, struct node_data, l)); + return 0; +} + +SEC("tc") +__description("list_add_test1: insert new node after prev") +__success __retval(0) +int list_add_test1(void *ctx) +{ + struct node_data *node_first; + struct node_data *new_node; + long err =3D 0; + + node_first =3D __add_in_list(&head, &lock); + if (!node_first) + return -1; + + new_node =3D bpf_obj_new(typeof(*new_node)); + if (!new_node) { + err =3D -2; + goto fail; + } + + bpf_spin_lock(&lock); + err =3D bpf_list_add(&head, &new_node->l, &node_first->l); + bpf_spin_unlock(&lock); + if (err) { + err =3D -3; + goto fail; + } + +fail: + bpf_obj_drop(node_first); + return 0; +} + +SEC("tc") +__description("list_add_test2: list_add accepts list_front return value as= prev") +__success __retval(0) +int list_add_test2(void *ctx) +{ + struct node_data *new_node, *tmp; + struct bpf_list_node *bpf_node; + long err =3D 0; + + tmp =3D __add_in_list(&head, &lock); + if (!tmp) + return -1; + + new_node =3D bpf_obj_new(typeof(*new_node)); + if (!new_node) { + err =3D -2; + goto fail; + } + + bpf_spin_lock(&lock); + bpf_node =3D bpf_list_front(&head); + if (!bpf_node) { + bpf_spin_unlock(&lock); + bpf_obj_drop(new_node); + err =3D -3; + goto fail; + } + + err =3D bpf_list_add(&head, &new_node->l, bpf_node); + bpf_spin_unlock(&lock); + if (err) { + err =3D -4; + goto fail; + } + +fail: + bpf_obj_drop(tmp); + return err; +} + +struct uninit_head_val { + struct bpf_spin_lock lock; + struct bpf_list_head head __contains(node_data, l); +}; + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct uninit_head_val); + __uint(max_entries, 1); +} uninit_head_map SEC(".maps"); + +SEC("tc") +__description("list_push_back_uninit_head: push_back on 0-initialized list= head") +__success __retval(0) +int list_push_back_uninit_head(void *ctx) +{ + struct uninit_head_val *st; + struct node_data *node; + int ret =3D -1, key =3D 0; + + st =3D bpf_map_lookup_elem(&uninit_head_map, &key); + if (!st) + return -1; + + node =3D bpf_obj_new(typeof(*node)); + if (!node) + return -1; + + bpf_spin_lock(&st->lock); + ret =3D bpf_list_push_back(&st->head, &node->l); + bpf_spin_unlock(&st->lock); + + return ret; +} + +SEC("?tc") +__failure __msg("bpf_spin_lock at off=3D32 must be held for bpf_list_head") +long list_del_without_lock_fail(void *ctx) +{ + struct bpf_rb_node *rb; + struct bpf_list_node *l; + struct node_data *n; + + bpf_spin_lock(&lock); + rb =3D bpf_rbtree_first(&root); + bpf_spin_unlock(&lock); + if (!rb) + return -1; + + n =3D container_of(rb, struct node_data, r); + /* Error case: delete list node without holding lock */ + l =3D bpf_list_del(&head, &n->l); + if (!l) + return -2; + bpf_obj_drop(container_of(l, struct node_data, l)); + + return 0; +} + +SEC("?tc") +__failure __msg("bpf_spin_lock at off=3D32 must be held for bpf_list_head") +long list_add_without_lock_fail(void *ctx) +{ + struct bpf_rb_node *rb; + struct bpf_list_node *l; + struct node_data *n; + + bpf_spin_lock(&lock); + rb =3D bpf_rbtree_first(&root); + l =3D bpf_list_front(&head); + bpf_spin_unlock(&lock); + if (!rb || !l) + return -1; + + n =3D container_of(l, struct node_data, l); + /* Error case: add list node without holding lock */ + if (bpf_list_add(&head, &n->l, l)) + return -2; + + return 0; +} + SEC("tc") __success long rbtree_refcounted_node_ref_escapes(void *ctx) --=20 2.50.1 (Apple Git-155)