From nobody Thu Apr 9 13:31:47 2026 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B23F8229B38 for ; Sun, 8 Mar 2026 13:46:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977611; cv=none; b=fJHFNK4sVh0IRTvNkb487q+c67Uky/aKpP0Tw05OF8WG2pYRQA1nQRD5i29cOXXQKez1KpdfXsaS3I6HovoSYnWIAgKzcen1D+vdW7ILRCNMiSGg8u1P66VG3wH1tYmn9xJxyPE5FeuSKOvd/ZdrEN05qLwmMx7oWpbIp8g8deo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977611; c=relaxed/simple; bh=TryhRL/uyiBcF0xS7N76PWhRBXatsJzx9ww2XVRgX3o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pCYVzFFa3LmtymlSlq2wbPt2+mmj9luz0SUI7A2Sr/0zcdPkmUyW28s0u9G/V2MfbuivyQ4Wnc+WGemYAIi78OnOZZMOCXgUVclcPiBwwPJ9WYRWo8E2pkFoHVD5qrKrT2cWJTTP33ORI7SWqJwtEg72RetWLSWKP0EuyPtLYWg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=h+BZebWv; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="h+BZebWv" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-829b2019b39so779474b3a.3 for ; Sun, 08 Mar 2026 06:46:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772977610; x=1773582410; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aF8gwyBWdT2w+mKfu7Jf1DaykLIqQz7Y61RbBhsAuck=; b=h+BZebWvFI8GCT+0hRezjtfn2P5k/o1vqWASZhAa4W9BNygoKN76w9k9+Y2qbHSrDz oCibCvMJtF+CLHGN8714XZAauGVs6AMOWmfxiwA2wfCc7LO/D7Hmv8PBIJDTIorL8IQk nF5Pk8Gc+SGUifNB2BDL+5xX+ZvyfRblxvGFcPOyUR8al5tY5O1DX0U+AuOtGQK0Uw8I jXT5ZX9K/qMlpaBtx1ye75UA9mMiv0H5jygRfRIv2kDsitVlRwz64ZHhYuXo/Xp6evz7 pqNUyh5p3WKFKNN9Q6J/q7nnmDzJAJOoGmdWbrJRjqS3xsrJ5TUV7W3mS/oNd2PpI1XS eYdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772977610; x=1773582410; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=aF8gwyBWdT2w+mKfu7Jf1DaykLIqQz7Y61RbBhsAuck=; b=rJ39cy+iq27Ic1DblyLf/MPjRqoFLs3NlA+ZCO49VyQNdCWpHPIFiC4Mjnk4HKWZAx fW+qDTywsW4VwXtDRupj00tJeEl61ZLlpEh3zN7zXzIVz4bZJHKlZ5890nqwXhE0+Rcb GWYi2gDRDzR/FUIIfHdYnOy4m5DRCwiH/q8397PzTWL+8aXQ42TxG1eU2hTwqP0eBH7O jcuUb8dBqDywEKJY5OOSn5H5IMPC5KuL7VXQOEfnvw4blxRci0F2a3T3k1QjmFm87fDc 3fLgM1Q6otsxEDDfnMoZxf91EwlxxnSJ1hvNsWp4Sc0HBW6zFTuCol2RzZB4xZATJj+R EXNA== X-Forwarded-Encrypted: i=1; AJvYcCUihyNEIhI8gPZxDf6AHyEQ7ymn4dscEnz+PQoORtfuq6g0T8ZMKzlR6e6xypxKppEONIKAWonn3mFe6wc=@vger.kernel.org X-Gm-Message-State: AOJu0YzV0GQo9r7sfHNuHMCW5lJHZ+Uh+zBiTcqNcVWQEROweRZw6jw+ buqiJZBOm841alMsQ9Q/Ye4YMFtLc3NGvKBRQZJ3Mah0E4vR+XnF3PWQ X-Gm-Gg: ATEYQzwVYsKKmZGq90nu3CQmJLYKwht+KvCZhDGEiXaHd8/6Pi6ALTxuP8oBiwXIcxG p1O3C0IrHgcH0vUFHCmYkHH8GRlwCIGLQsf4Ts3P4edsw845oZhi+M+1SDmiuZv5vG4Fu/KRh+j djNhLH0PcGdM+6HI76PnIB8w8VV316N4eNYAESrRmowoesSpBJaJKhCJPEsLa0KL6UKkV0BRDcQ oK+s7OxyjIoiLpcjw4rGFcBAFATCPD0ysAfpsCbYIVfSqoPXaJETewyQ54/DxMLzRZr1Gpjp3Xe dimIVGjzQyQsUDMDOxjHP68P/J5r9ABW203exmq2wUy4p9uT69r60+V3cygcK8TVS7SWdQ04B3V LVQzJO9ULyi+73rTAfdwY4/yuoF97dIAuTBg0YHyECqUip74cJTPupuG16e7b0leUlZQdnafz1w Btcy4tuu/OQpTxv6TihwOBLxWzWEEYq+N545U6zPkDKDu0kYYR X-Received: by 2002:a05:6a00:b83:b0:829:809e:8972 with SMTP id d2e1a72fcca58-829a2db6331mr7125950b3a.3.1772977610015; Sun, 08 Mar 2026 06:46:50 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.35]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-829a4636968sm7095056b3a.9.2026.03.08.06.46.34 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 08 Mar 2026 06:46:49 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v7 1/5] bpf: Introduce the bpf_list_del kfunc. Date: Sun, 8 Mar 2026 21:46:10 +0800 Message-ID: <20260308134614.29711-2-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260308134614.29711-1-pilgrimtao@gmail.com> References: <20260308134614.29711-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng If a user holds ownership of a node in the middle of a list, they can directly remove it from the list without strictly adhering to deletion rules from the head or tail. We have added an additional parameter bpf_list_head *head to bpf_list_del, as the verifier requires the head parameter to check whether the lock is being held. This is typically paired with bpf_refcount. After calling bpf_list_del, it is generally necessary to drop the reference to the list node twice to prevent reference count leaks. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 30 +++++++++++++++++++++++------- kernel/bpf/verifier.c | 6 +++++- 2 files changed, 28 insertions(+), 8 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 6eb6c82ed2ee..01b74c4ac00d 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2426,20 +2426,23 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_= list_head *head, return __bpf_list_add(n, head, true, meta ? meta->record : NULL, off); } =20 -static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bo= ol tail) +static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, + struct list_head *n) { - struct list_head *n, *h =3D (void *)head; + struct list_head *h =3D (void *)head; struct bpf_list_node_kern *node; =20 /* If list_head was 0-initialized by map, bpf_obj_init_field wasn't * called on its fields, so init here */ - if (unlikely(!h->next)) + if (unlikely(!h->next)) { INIT_LIST_HEAD(h); - if (list_empty(h)) + return NULL; + } + + if (n =3D=3D h) return NULL; =20 - n =3D tail ? h->prev : h->next; node =3D container_of(n, struct bpf_list_node_kern, list_head); if (WARN_ON_ONCE(READ_ONCE(node->owner) !=3D head)) return NULL; @@ -2451,12 +2454,24 @@ static struct bpf_list_node *__bpf_list_del(struct = bpf_list_head *head, bool tai =20 __bpf_kfunc struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head = *head) { - return __bpf_list_del(head, false); + struct list_head *h =3D (void *)head; + + return __bpf_list_del(head, h->next); } =20 __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *= head) { - return __bpf_list_del(head, true); + struct list_head *h =3D (void *)head; + + return __bpf_list_del(head, h->prev); +} + +__bpf_kfunc struct bpf_list_node *bpf_list_del(struct bpf_list_head *head, + struct bpf_list_node *node) +{ + struct bpf_list_node_kern *kn =3D (void *)node; + + return __bpf_list_del(head, &kn->list_head); } =20 __bpf_kfunc struct bpf_list_node *bpf_list_front(struct bpf_list_head *hea= d) @@ -4545,6 +4560,7 @@ BTF_ID_FLAGS(func, bpf_list_push_front_impl) BTF_ID_FLAGS(func, bpf_list_push_back_impl) BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 67c09b43a497..c9557d3fb8dd 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12461,6 +12461,7 @@ enum special_kfunc_type { KF_bpf_list_push_back_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, + KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, KF_bpf_cast_to_kern_ctx, @@ -12521,6 +12522,7 @@ BTF_ID(func, bpf_list_push_front_impl) BTF_ID(func, bpf_list_push_back_impl) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) +BTF_ID(func, bpf_list_del) BTF_ID(func, bpf_list_front) BTF_ID(func, bpf_list_back) BTF_ID(func, bpf_cast_to_kern_ctx) @@ -12996,6 +12998,7 @@ static bool is_bpf_list_api_kfunc(u32 btf_id) btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_front] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_back] || + btf_id =3D=3D special_kfunc_list[KF_bpf_list_del] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_front] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_back]; } @@ -13118,7 +13121,8 @@ static bool check_kfunc_is_graph_node_api(struct bp= f_verifier_env *env, switch (node_field_type) { case BPF_LIST_NODE: ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_i= mpl] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl= ]); + kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl= ] || + kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_del]); break; case BPF_RB_NODE: ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_remove] || --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 9 13:31:47 2026 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE06C22FE0E for ; Sun, 8 Mar 2026 13:47:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977624; cv=none; b=DPrp1+KweJt09ebqDZnwUSyDegXL1KY39x6OS3V6R1GXWbNQf4monBMscypl4OQniVW6Rt6+9m6TnxgCUzMxzereVkO24YySgMJvVZKTp8Z63USEwpos4YJb3brZl96FxPZp4ADv57FIohZo/K8mC+ljG84fx/B2YgK3Qk6KVJU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977624; c=relaxed/simple; bh=vlnT0qx9z2tgLchicqex4emzHsHHdPmTZee/+gMzAko=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Cx1PoWnvwo14b3f6dfscwlgyROLSib4thR2nOjUzv/i1lY3sog7lc0gB5q6B4GilLK/Iavg3c+XKAalQgx5UK8q8a6FQybMfskzNg5STZYs6thk1d0y2yqhPbwk9geJ+2eYcemN7+93GXewm4q9/HJedd6UIAkyJK6uZXGhYtrs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Y9A+AyJV; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Y9A+AyJV" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-829928e512aso1640610b3a.2 for ; Sun, 08 Mar 2026 06:47:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772977622; x=1773582422; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LE0gvW5I8wxwMECBfOzg+fw5DMaARSTRZ4S0cwcKh68=; b=Y9A+AyJVUz9IV8H+qjB5vnmrWiC60SMaBMr1FVkQw5MVJ1gUkD6rbi2Ce0vEnHHyl1 VbO2B7OAozOaJVaQ9AREpmUCQjZpyWO5e6AuqXHWMZ13LWN7oQDUFGbe08WjZlxooAam FOoPtzw+c6TLAkxiZYMcd/zJuejwFT4BUShQvkwSulUn71cTqnj8hNnX9dHb+gvCoop8 CjL5LDeFSh2BY8OGrP+szO2M7jlgfpxof3LPWa0NS3JVqQA5+BwUDKmtkcmeWR1khKt9 zIxTemwQeM8y+g2RevzyShFBv+RWyHQeQjlV+ZTel9QhJyLGeCNY7bI9bF+oEt1Oefud yt8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772977622; x=1773582422; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=LE0gvW5I8wxwMECBfOzg+fw5DMaARSTRZ4S0cwcKh68=; b=h0EJg3gzzy0m/Ld/BOnvW8/WWcWJATn0X1WBcQHhwH9EFYp3eor4AjAouHoS5Lp2lA VZZGcS9Ql3Qxm82aLgU/dTE0DYizv/xEXIL009QR5+wBdraHMEBqH+bROq6XtX8bm8Dg 64GgDpYMvsRrcyfLbt2WICGS2fwCvd7LrTTaEGJiuy7oSBMvuACFVUEgXxeRsjTGNW7i WWIg76N9idir5f0/vFL50Ud9wBHv9L1TTwqN++RmREDA9gQwceYr+DnOORV5M4IzyO2q wKvCRUq38YtWOhg6JCAghUr4h1qvUg1fQsptkveCGZCB9+o1brCG8SwG7nJ/ZXEE/d1S MBMg== X-Forwarded-Encrypted: i=1; AJvYcCX3YSCceGcAB1BsMQyiozbE+0f85iYzhV7LxGd2U2fhH6VQZceEgH5epl+jNefHD4hdSJSCiM0KslHMZRw=@vger.kernel.org X-Gm-Message-State: AOJu0YyHcJ6PgsWeo2xMlHZ+BGZDczZi7HhNCPVyTpxJSiRzjvH0U4zu cp98QEvl9w7EZ9PeW6vSg7yGNGHTBMK6GLTNEysJGjkB5laMAYkrOvhp X-Gm-Gg: ATEYQzx0B0bo6QrqoOS8Y6blcRwite4+txc2d1RakcfDYdHjAaTJgyi2mrdlfcDf7iI hrx34av5SFC2xWMI7B4xkj1hEyr7EsGc3gThccoUjMo9yvuZiKITkbW8SEtHISkG18/evOJRg00 7cjYYvfVxrwqVW2cHqrQCdHI6Tvu+9/YP6vIgbLsP6AFb+H/J5ekv09dsxmq43Twwlio+0dyPRS oclauHRdKbXpHR5VaE95i67Vj3y3xPDG/LGVfIOlIRg18IF9gD7XpeMPDMptjdEww0S7yQwhP/P U47IimVfE5LmborqY0K5OWUBFoSimFRKJZkdigwjhQcpsNV8uCfdBvhgjPIg+TpHbWBGJqBejRy 11PIVrnPHMPkdvkMdKJWktw4CFJleNkTPlQSE3mwYGAxOr8b7w7gNNDAvEL99mSAeEfcIXja5QE 9OS3M01WkdYGNxEa+nRMIQFrPyC/pWjHm3q5R5eFhsgohxdxuo X-Received: by 2002:a05:6a00:21d4:b0:824:36f8:3461 with SMTP id d2e1a72fcca58-829a2e14048mr6726119b3a.23.1772977622138; Sun, 08 Mar 2026 06:47:02 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.35]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-829a4636968sm7095056b3a.9.2026.03.08.06.46.50 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 08 Mar 2026 06:47:01 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v7 2/5] bpf: Add bpf_list_add_impl to insert node after a given list node Date: Sun, 8 Mar 2026 21:46:11 +0800 Message-ID: <20260308134614.29711-3-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260308134614.29711-1-pilgrimtao@gmail.com> References: <20260308134614.29711-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Add a new kfunc bpf_list_add_impl(head, new, prev, meta, off) that inserts 'new' after 'prev' in the BPF linked list. Both must be in the same list; 'prev' must already be in the list. The new node must be an owning reference (e.g. from bpf_obj_new); the kfunc consumes that reference and the node becomes non-owning once inserted. We have added an additional parameter bpf_list_head *head to bpf_list_add_impl, as the verifier requires the head parameter to check whether the lock is being held. Returns 0 on success, -EINVAL if 'prev' is not in a list or 'new' is already in a list (or duplicate insertion). On failure, the kernel drops the passed-in node. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 56 ++++++++++++++++++++++++++++++------------- kernel/bpf/verifier.c | 13 ++++++++-- 2 files changed, 50 insertions(+), 19 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 01b74c4ac00d..407520fde668 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2379,11 +2379,12 @@ __bpf_kfunc void *bpf_refcount_acquire_impl(void *p= __refcounted_kptr, void *meta return (void *)p__refcounted_kptr; } =20 -static int __bpf_list_add(struct bpf_list_node_kern *node, - struct bpf_list_head *head, - bool tail, struct btf_record *rec, u64 off) +static int __bpf_list_add(struct bpf_list_head *head, + struct bpf_list_node_kern *new, + struct list_head *prev, + struct btf_record *rec, u64 off) { - struct list_head *n =3D &node->list_head, *h =3D (void *)head; + struct list_head *n =3D &new->list_head, *h =3D (void *)head; =20 /* If list_head was 0-initialized by map, bpf_obj_init_field wasn't * called on its fields, so init here @@ -2391,39 +2392,59 @@ static int __bpf_list_add(struct bpf_list_node_kern= *node, if (unlikely(!h->next)) INIT_LIST_HEAD(h); =20 - /* node->owner !=3D NULL implies !list_empty(n), no need to separately + /* When prev is not the list head, it must be a node in this list. */ + if (prev !=3D h && WARN_ON_ONCE(READ_ONCE(container_of( + prev, struct bpf_list_node_kern, list_head)->owner) !=3D head)) + goto fail; + + /* new->owner !=3D NULL implies !list_empty(n), no need to separately * check the latter */ - if (cmpxchg(&node->owner, NULL, BPF_PTR_POISON)) { - /* Only called from BPF prog, no need to migrate_disable */ - __bpf_obj_drop_impl((void *)n - off, rec, false); - return -EINVAL; - } - - tail ? list_add_tail(n, h) : list_add(n, h); - WRITE_ONCE(node->owner, head); + if (cmpxchg(&new->owner, NULL, BPF_PTR_POISON)) + goto fail; =20 + list_add(n, prev); + WRITE_ONCE(new->owner, head); return 0; + +fail: + /* Only called from BPF prog, no need to migrate_disable */ + __bpf_obj_drop_impl((void *)n - off, rec, false); + return -EINVAL; } =20 __bpf_kfunc int bpf_list_push_front_impl(struct bpf_list_head *head, struct bpf_list_node *node, void *meta__ign, u64 off) { - struct bpf_list_node_kern *n =3D (void *)node; + struct bpf_list_node_kern *new =3D (void *)node; struct btf_struct_meta *meta =3D meta__ign; + struct list_head *h =3D (void *)head; =20 - return __bpf_list_add(n, head, false, meta ? meta->record : NULL, off); + return __bpf_list_add(head, new, h, meta ? meta->record : NULL, off); } =20 __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head, struct bpf_list_node *node, void *meta__ign, u64 off) { - struct bpf_list_node_kern *n =3D (void *)node; + struct bpf_list_node_kern *new =3D (void *)node; + struct btf_struct_meta *meta =3D meta__ign; + struct list_head *h =3D (void *)head; + + return __bpf_list_add(head, new, h->prev, meta ? meta->record : NULL, off= ); +} + +__bpf_kfunc int bpf_list_add_impl(struct bpf_list_head *head, + struct bpf_list_node *new, + struct bpf_list_node *prev, + void *meta__ign, u64 off) +{ + struct bpf_list_node_kern *kn =3D (void *)new, *kp =3D (void *)prev; struct btf_struct_meta *meta =3D meta__ign; =20 - return __bpf_list_add(n, head, true, meta ? meta->record : NULL, off); + return __bpf_list_add(head, kn, &kp->list_head, + meta ? meta->record : NULL, off); } =20 static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, @@ -4563,6 +4584,7 @@ BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF= _RET_NULL) BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_list_add_impl) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE) BTF_ID_FLAGS(func, bpf_rbtree_remove, KF_ACQUIRE | KF_RET_NULL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index c9557d3fb8dd..5f55b68ed935 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12459,6 +12459,7 @@ enum special_kfunc_type { KF_bpf_refcount_acquire_impl, KF_bpf_list_push_front_impl, KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, KF_bpf_list_pop_front, KF_bpf_list_pop_back, KF_bpf_list_del, @@ -12520,6 +12521,7 @@ BTF_ID(func, bpf_obj_drop_impl) BTF_ID(func, bpf_refcount_acquire_impl) BTF_ID(func, bpf_list_push_front_impl) BTF_ID(func, bpf_list_push_back_impl) +BTF_ID(func, bpf_list_add_impl) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) BTF_ID(func, bpf_list_del) @@ -12996,6 +12998,7 @@ static bool is_bpf_list_api_kfunc(u32 btf_id) { return btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl] || + btf_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_front] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_back] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_del] || @@ -13122,6 +13125,7 @@ static bool check_kfunc_is_graph_node_api(struct bp= f_verifier_env *env, case BPF_LIST_NODE: ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_i= mpl] || kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl= ] || + kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_del]); break; case BPF_RB_NODE: @@ -14264,6 +14268,7 @@ static int check_kfunc_call(struct bpf_verifier_env= *env, struct bpf_insn *insn, =20 if (meta.func_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] || meta.func_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl] || + meta.func_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || meta.func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { release_ref_obj_id =3D regs[BPF_REG_2].ref_obj_id; insn_aux->insert_off =3D regs[BPF_REG_2].off; @@ -23230,13 +23235,17 @@ static int fixup_kfunc_call(struct bpf_verifier_e= nv *env, struct bpf_insn *insn, *cnt =3D 3; } else if (desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_= impl] || desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] = || + desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || desc->func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { struct btf_struct_meta *kptr_struct_meta =3D env->insn_aux_data[insn_idx= ].kptr_struct_meta; int struct_meta_reg =3D BPF_REG_3; int node_offset_reg =3D BPF_REG_4; =20 - /* rbtree_add has extra 'less' arg, so args-to-fixup are in diff regs */ - if (desc->func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { + /* list/rbtree_add_impl have an extra arg (prev/less), + * so args-to-fixup are in different regs. + */ + if (desc->func_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || + desc->func_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl]) { struct_meta_reg =3D BPF_REG_4; node_offset_reg =3D BPF_REG_5; } --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 9 13:31:47 2026 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E74CB245005 for ; Sun, 8 Mar 2026 13:47:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977639; cv=none; b=J9bMCY0QWZIwMoMn30fLX0T6o0WzMXnWvMvlELSAemYESh9DWEQckBAtEuPKFYEc4nQRl+COFRMIaQ2C8d87407ZRCd7HmGFQcwE08weGiE5T4wjW9ynfRVWyy6+4C08bc9IuMfGiVW/o4viyUsbEKPl4xFQD7GJMWXzftCeSWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977639; c=relaxed/simple; bh=1eWxWto6KUk7PXfmWo0yP2+4m0Bt3mH32o0CRbZIsgk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iryHo3hzz8FXCDLnbMu07ONypP7ugi2PiNbtzNN778kIjl0yVRo3qECPl8tjtFHnjzpfE6/qq4hnhlyviAgGMJO3Rd0RJCgPxw6ib8e4sKbZg1Wfb5wFhPWjofvSQUESVjA/cO4pKFJcXRacML8prDU7ePlLzpcCZyHyOUpMrpQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SizQlioL; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SizQlioL" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-8297310ce0aso2669739b3a.2 for ; Sun, 08 Mar 2026 06:47:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772977637; x=1773582437; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C5ogfvh3LzPgqc+SKxRTFIvNBgOyOocO0BrAiL+IkvQ=; b=SizQlioLetZYYJsXqd9lDAv4ffIprn6V7/fq3EN1oqWu4ZFdWFWW7RxPoeaU1aA0yf 3XmN7iVs7r6P1qG2XnbAVTJZNQ4WxBpuIDMXiIMxNKSQ9oilrpAeu1Jay9PC34pXStLg wDg4IW/lVtnSPbHc56B7Er4aPL9q93PfYepUEBylAvLTbh3xmkvoeAHsxsT6IIVJLeXZ Xgt8IUQm/TS2XPioA2OTTBSCwhL27utzgXeq4pjTF6b0mRrqyZWLybsFWM/xbzTa+cc9 Lwc8L9dYFuucngbAvYsCeBiuAQ6H67cZEGzpT4/33wfrxZ0rZ01eO9FEDF7b55R3se/Y 1n/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772977637; x=1773582437; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=C5ogfvh3LzPgqc+SKxRTFIvNBgOyOocO0BrAiL+IkvQ=; b=tvcydsW2UowTiuspQo3tYM09hcEHyzC3zXeyYJEPGxpLy/9tMmCoxVztaOLIUWAgzl BiWeMjgzoKFycDuryuh5RrqaoNIfkCaYKJPvdXIAEN8GJcwe+vLW1iNGLExe5hG/yR8S DTpPuXCjGOxPNM7Ol6Cp28PbXZqSdSeJXgZ3WhbRiGeMg9csenbfIK8ZXPoGmZ/RORUM G2XvXoSBNi+x4U+YICljnURPOvUWUDwZ9dOghVwO/BbyTIEaZwHOzLx+BAm+3+W9A1w5 4/0mMUEl2iftL6gnSSw3P8cbsUvmhWfj+Fx93O1PkM1iPXX0BeDqdSev1WAFJYj+rdOV BQpg== X-Forwarded-Encrypted: i=1; AJvYcCXVMfTWvuOlHikN73C7Cl+wCJdMyAX99ACc1hzBhCHuYkmhftxCTrHooSruSGhBlSW8AbJFR4Vc7QsNMn8=@vger.kernel.org X-Gm-Message-State: AOJu0YwmVe62wCo0AwN0yaeUVpmFRuoZWbWeFuA0RIGuJY6Uv3b4k+1Z 2rcv/BqVPVe/cd08VA20ZxVl4sWFhPG9URfzJHKOFyI1+2gUT/uQz9Dv X-Gm-Gg: ATEYQzx2mmm5HjUsJZM06FXZcTn61ZeYMppwtztSEayivXa09kNPhcBss58kX2hekjW OWk/GxZWCh0Jn6uRawduBCIT3uJEXRFu1GjcU6tU33At+wszVzdX+pj/JD3lbP9PJZcO5FjqiPR l0YYj8/v3SU/15xhmZakcWsLilEUhQfMPbmNoS0bUAOiJC40S/otzY1uMvzB2tfEa2v2gspPwyT f5w5Y20KCdJtNyN0aMW3q0YxsG0A+CO1lmaetHbSf8IqVOx6l0Yf+RapetTxWeqr5IIesuM5F/W ilk8gz5xGWgQC4/wB2+Fboo3NnRiGSkBHmn3dpuzHpflKa2yIY4EZMWK6TcO4DgfnqvlUtkmcaQ vbm1UNqrYSAeC2OxsWfnZzcvYzsDIaqNlBBtpsfU6kNU1uPk3/+lTkVkoBcJrwfDt8CaWYGN/Nt 2EeSLCfPJ1WiZ45EK+87jqa00H9bzriG1T5cUl4rwt7SU/OMzh X-Received: by 2002:a05:6a00:3c8d:b0:821:81ef:5de8 with SMTP id d2e1a72fcca58-829a2dc08bbmr7422879b3a.12.1772977637188; Sun, 08 Mar 2026 06:47:17 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.35]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-829a4636968sm7095056b3a.9.2026.03.08.06.47.02 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 08 Mar 2026 06:47:16 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v7 3/5] bpf: add bpf_list_is_first/last/empty kfuncs Date: Sun, 8 Mar 2026 21:46:12 +0800 Message-ID: <20260308134614.29711-4-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260308134614.29711-1-pilgrimtao@gmail.com> References: <20260308134614.29711-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Add three kfuncs for BPF linked list queries: - bpf_list_is_first(head, node): true if node is the first in the list. - bpf_list_is_last(head, node): true if node is the last in the list. - bpf_list_empty(head): true if the list has no entries. In previous versions, to implement the above functionality, it was necessary to first call bpf_list_pop_front/back to retrieve the first or last node before checking whether the passed-in node was the first or last one. After the check, the node had to be pushed back into the list using bpf_list_push_front/back, which was very inefficient. Now, with the bpf_list_is_first/last/empty kfuncs, we can directly check whether a node is the first, last, or whether the list is empty, without having to first retrieve the node. Signed-off-by: Kaitao Cheng --- kernel/bpf/helpers.c | 38 ++++++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 15 +++++++++++++-- 2 files changed, 51 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 407520fde668..476f5ad319e2 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2515,6 +2515,41 @@ __bpf_kfunc struct bpf_list_node *bpf_list_back(stru= ct bpf_list_head *head) return (struct bpf_list_node *)h->prev; } =20 +__bpf_kfunc bool bpf_list_is_first(struct bpf_list_head *head, struct bpf_= list_node *node) +{ + struct list_head *h =3D (struct list_head *)head; + struct bpf_list_node_kern *kn =3D (struct bpf_list_node_kern *)node; + + if (READ_ONCE(kn->owner) !=3D head) + return false; + + return list_is_first(&kn->list_head, h); +} + +__bpf_kfunc bool bpf_list_is_last(struct bpf_list_head *head, struct bpf_l= ist_node *node) +{ + struct list_head *h =3D (struct list_head *)head; + struct bpf_list_node_kern *kn =3D (struct bpf_list_node_kern *)node; + + if (READ_ONCE(kn->owner) !=3D head) + return false; + + return list_is_last(&kn->list_head, h); +} + +__bpf_kfunc bool bpf_list_empty(struct bpf_list_head *head) +{ + struct list_head *h =3D (struct list_head *)head; + + /* If list_head was 0-initialized by map, bpf_obj_init_field wasn't + * called on its fields, so init here + */ + if (unlikely(!h->next)) + INIT_LIST_HEAD(h); + + return list_empty(h); +} + __bpf_kfunc struct bpf_rb_node *bpf_rbtree_remove(struct bpf_rb_root *root, struct bpf_rb_node *node) { @@ -4585,6 +4620,9 @@ BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_= NULL) BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_add_impl) +BTF_ID_FLAGS(func, bpf_list_is_first) +BTF_ID_FLAGS(func, bpf_list_is_last) +BTF_ID_FLAGS(func, bpf_list_empty) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE) BTF_ID_FLAGS(func, bpf_rbtree_remove, KF_ACQUIRE | KF_RET_NULL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5f55b68ed935..5e32e02429c4 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12465,6 +12465,9 @@ enum special_kfunc_type { KF_bpf_list_del, KF_bpf_list_front, KF_bpf_list_back, + KF_bpf_list_is_first, + KF_bpf_list_is_last, + KF_bpf_list_empty, KF_bpf_cast_to_kern_ctx, KF_bpf_rdonly_cast, KF_bpf_rcu_read_lock, @@ -12527,6 +12530,9 @@ BTF_ID(func, bpf_list_pop_back) BTF_ID(func, bpf_list_del) BTF_ID(func, bpf_list_front) BTF_ID(func, bpf_list_back) +BTF_ID(func, bpf_list_is_first) +BTF_ID(func, bpf_list_is_last) +BTF_ID(func, bpf_list_empty) BTF_ID(func, bpf_cast_to_kern_ctx) BTF_ID(func, bpf_rdonly_cast) BTF_ID(func, bpf_rcu_read_lock) @@ -13003,7 +13009,10 @@ static bool is_bpf_list_api_kfunc(u32 btf_id) btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_back] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_del] || btf_id =3D=3D special_kfunc_list[KF_bpf_list_front] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_back]; + btf_id =3D=3D special_kfunc_list[KF_bpf_list_back] || + btf_id =3D=3D special_kfunc_list[KF_bpf_list_is_first] || + btf_id =3D=3D special_kfunc_list[KF_bpf_list_is_last] || + btf_id =3D=3D special_kfunc_list[KF_bpf_list_empty]; } =20 static bool is_bpf_rbtree_api_kfunc(u32 btf_id) @@ -13126,7 +13135,9 @@ static bool check_kfunc_is_graph_node_api(struct bp= f_verifier_env *env, ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_i= mpl] || kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl= ] || kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_del]); + kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_del] || + kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_is_first] || + kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_is_last]); break; case BPF_RB_NODE: ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_remove] || --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 9 13:31:47 2026 Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 386022441A6 for ; Sun, 8 Mar 2026 13:47:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977662; cv=none; b=ojhItAX4uotKS/8OAn6Gam4MXBUKCQ8B/tZfSXBJsxnNie/udhYIG07a5TbMQKHyo45ra2c3uGnAobILeGjeQKZLHF5E4mq7T2xUqQJqSfi0ipeKtQl6Qw96T8QEDs5AO2j4B7qWTk1O+KUpwD7uUt/ZWV0OtC6voVjJHOu6N8s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977662; c=relaxed/simple; bh=1fVjL/BWbWrnK0xa23u3xDjOZ/zHvb6gwBS6fUH3NPc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JyK1/58m66jYOEirs0s+BU2YgLT7PDk/j7WiSRpq301h+r5J7ewgFa1B7YXd4UBJoc4O0vzb9Wb04z3MC2baeE+gq1e0XU56SpCK64aHvAwYZbYmu5+05U1T+eFFSMB4UhnY9HgVvEnfV/9rXek9Y4Hywx8Z6WnCKGkmlMSGhB0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cRNkOgmx; arc=none smtp.client-ip=209.85.215.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cRNkOgmx" Received: by mail-pg1-f173.google.com with SMTP id 41be03b00d2f7-c7358a7a8d1so4715720a12.3 for ; Sun, 08 Mar 2026 06:47:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772977660; x=1773582460; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6XrESepKNhObGU0gsyPBOvwXryFumcINMmhOSkp1V30=; b=cRNkOgmxbvhTcfHxC7op0J5RqHPKFcgpXHyFlDY/MdfNl9YATozj81SGpV0dvG4t2Q Z3v0+nsJxjmNcgBJcW58xRS8oomnBTVuUXmy0Zcco1Zf8MyhqNLk+SwO81iwpFOgtfu0 t3fh8v20WcHKEL30avHjeHS9w8u/rzFPlQ9I6jf48HAYciFSBcT7C/99+7+SKLMqbzQm CsCmszDooyECRukZ/noPw0adp+hXrDv6wCqgRVorXmmJmUXyGDnv/rgLvIvB1eecTEa/ NNBdU1+yI9u40mJU9pdwJrktIr2sO39g2HMGc9HXw2bQ5z65a3my9ErX3M2i5q1hXtJw mOcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772977660; x=1773582460; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=6XrESepKNhObGU0gsyPBOvwXryFumcINMmhOSkp1V30=; b=irXvasU7thPo3HzIEsR+ctmncNUjZJveb8s9VX2WP9txJbx6H44wJmm6WlSWNHFAJS EYQjmI0ln17quNGRniBr1hE5plhp7XJuGA47AnkZanz6c9bunrLX19Eimq5O52dHS+9X q6RKjxauMCcfOQGwQe59CIM8vUqfapgBcZojLPBU0XA1zOw7Nhhf581q8dMQS2Ox2Gx+ o9korIE5gYUa6lh6BPWUqeZtoNCDctjg1LHK0ix5nmg+ieEoyy0zxHVOUnmUdRpKpiOj woUYL8USTGvQ/Wo0tlWl7KeUK66Y4ONDt/CucfWZDwODJeNXzNfB88XGq8+smFLQA/Nq h/Wg== X-Forwarded-Encrypted: i=1; AJvYcCWR/Gi4KR1tfdI+WYnMG+OuaL+YpmBFD7IaQIDLRL5bYaaPDD2xa//q+kgkhZBzcRfso3PzbgEDAtOg3TM=@vger.kernel.org X-Gm-Message-State: AOJu0YyCMdr1H23dj5+K/s4Va9rGw1XDLzY+WaS3eCGQBpDIiAES0JNv InTCf1AxOlXG+n+YKofLGUTLLb7y8xpc4yGIt96q+q0bLRRVMlhKpGGs X-Gm-Gg: ATEYQzzhuolYNFucj2dvLuhJ3ToPjazZZTute94ehKvajYo9E406h/pZwgX27GsthLn d2Xj/6Xxvp9h7c8fEZvhEY2yR3VFHhQ1cr7tI3tKU6ajCzJeI7Oq91u2dswUXIoo521koIcEC0x BIu7pdJJK1v4bhS2rZbIEqoYmV6FcPVwIIqQVPpm8bjUbm5jctbNQZ9Kg3qK807rE2+XOzhQsgt 1hItvyWfi50YducCv/yGSyy8j/XSqD8fEZ4hvhdQU0zsBivTh8cX/jI2oTQuWCUL1BgdAMWyCFF NgC42n4J3880JwkYLEsZZ+C7kcKaWPIsNzwesUqCJg0SYvxyUlvR2iYIHv8PWeEP+HLYcLIiyxE wALr7rNeue9OYAMK/VCgXGgkE7htHay1G5ebCJGYtCvldWNFNqCUa5odyazcVKQ6P7flHfq0eCW GV3gmn30I46ETws/V44P/VtZeQTJDJPVtUFIJzis2ktGgXjrsu X-Received: by 2002:a05:6a20:7350:b0:394:58eb:48fa with SMTP id adf61e73a8af0-39858fb419bmr8732290637.6.1772977660602; Sun, 08 Mar 2026 06:47:40 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.35]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-829a4636968sm7095056b3a.9.2026.03.08.06.47.19 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 08 Mar 2026 06:47:40 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v7 4/5] selftests/bpf: Add test cases for bpf_list_del/add/is_first/is_last/empty Date: Sun, 8 Mar 2026 21:46:13 +0800 Message-ID: <20260308134614.29711-5-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260308134614.29711-1-pilgrimtao@gmail.com> References: <20260308134614.29711-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Extend the refcounted_kptr test: add a node to both an rbtree and a list, retrieve the node from the rbtree to obtain the node pointer, then add a new node after the first in the list, and finally use bpf_list_del to remove both nodes. The test asserts that the list is non-empty after insert, asserts the first and last nodes after bpf_list_add, and asserts that the list is empty after removing both nodes. To verify the validity of bpf_list_del/add, the test also expects the verifier to reject calls to bpf_list_del/add made without holding the spin_lock. Signed-off-by: Kaitao Cheng --- .../testing/selftests/bpf/bpf_experimental.h | 16 ++ .../selftests/bpf/progs/refcounted_kptr.c | 140 ++++++++++++++++++ 2 files changed, 156 insertions(+) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing= /selftests/bpf/bpf_experimental.h index 4b7210c318dd..005ca9d84677 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -85,6 +85,22 @@ extern int bpf_list_push_back_impl(struct bpf_list_head = *head, /* Convenience macro to wrap over bpf_list_push_back_impl */ #define bpf_list_push_back(head, node) bpf_list_push_back_impl(head, node,= NULL, 0) =20 +/* Description + * Insert 'new' after 'prev' in the BPF linked list with head 'head'. + * The bpf_spin_lock protecting the list must be held. 'prev' must already + * be in that list; 'new' must not be in any list. The 'meta' and 'off' + * parameters are rewritten by the verifier, no need for BPF programs to + * set them. + * Returns + * 0 on success, -EINVAL if head is NULL, prev is not in the list with hea= d, + * or new is already in a list. + */ +extern int bpf_list_add_impl(struct bpf_list_head *head, struct bpf_list_n= ode *new, + struct bpf_list_node *prev, void *meta, __u64 off) __ksym; + +/* Convenience macro to wrap over bpf_list_add_impl */ +#define bpf_list_add(head, new, prev) bpf_list_add_impl(head, new, prev, N= ULL, 0) + /* Description * Remove the entry at the beginning of the BPF linked list. * Returns diff --git a/tools/testing/selftests/bpf/progs/refcounted_kptr.c b/tools/te= sting/selftests/bpf/progs/refcounted_kptr.c index 1aca85d86aeb..c2defa991acd 100644 --- a/tools/testing/selftests/bpf/progs/refcounted_kptr.c +++ b/tools/testing/selftests/bpf/progs/refcounted_kptr.c @@ -367,6 +367,146 @@ long insert_rbtree_and_stash__del_tree_##rem_tree(voi= d *ctx) \ INSERT_STASH_READ(true, "insert_stash_read: remove from tree"); INSERT_STASH_READ(false, "insert_stash_read: don't remove from tree"); =20 +/* + * Insert one node in tree and list, remove it from tree, add a second node + * after it with bpf_list_add, check bpf_list_is_first/is_last/empty, then + * remove both nodes from list via bpf_list_del. + */ +SEC("tc") +__description("list_add_del_and_check: test bpf_list_add/del/is_first/is_l= ast/empty") +__success __retval(0) +long list_add_del_and_check(void *ctx) +{ + long err =3D 0; + struct bpf_rb_node *rb; + struct bpf_list_node *l_node, *l_node_ref; + struct node_data *n_rb, *n_new, *n_new_ref; + + err =3D __insert_in_tree_and_list(&head, &root, &lock); + if (err) + return err; + + bpf_spin_lock(&lock); + /* Test1: bpf_list_empty */ + if (bpf_list_empty(&head)) { + bpf_spin_unlock(&lock); + return -4; + } + + rb =3D bpf_rbtree_first(&root); + if (!rb) { + bpf_spin_unlock(&lock); + return -5; + } + + rb =3D bpf_rbtree_remove(&root, rb); + bpf_spin_unlock(&lock); + if (!rb) + return -6; + + n_rb =3D container_of(rb, struct node_data, r); + n_new =3D bpf_obj_new(typeof(*n_new)); + if (!n_new) { + bpf_obj_drop(n_rb); + return -7; + } + n_new_ref =3D bpf_refcount_acquire(n_new); + if (!n_new_ref) { + bpf_obj_drop(n_rb); + bpf_obj_drop(n_new); + return -8; + } + + bpf_spin_lock(&lock); + /* Test2: bpf_list_add */ + if (bpf_list_add(&head, &n_new->l, &n_rb->l)) { + bpf_spin_unlock(&lock); + bpf_obj_drop(n_rb); + bpf_obj_drop(n_new_ref); + return -9; + } + + /* Test3: bpf_list_is_first/is_last */ + if (!bpf_list_is_first(&head, &n_rb->l) || + !bpf_list_is_last(&head, &n_new_ref->l)) { + bpf_spin_unlock(&lock); + bpf_obj_drop(n_rb); + bpf_obj_drop(n_new_ref); + return -10; + } + + /* Test4: bpf_list_del */ + l_node =3D bpf_list_del(&head, &n_rb->l); + l_node_ref =3D bpf_list_del(&head, &n_new_ref->l); + bpf_spin_unlock(&lock); + bpf_obj_drop(n_rb); + bpf_obj_drop(n_new_ref); + + if (l_node) + bpf_obj_drop(container_of(l_node, struct node_data, l)); + else + err =3D -11; + + if (l_node_ref) + bpf_obj_drop(container_of(l_node_ref, struct node_data, l)); + else + err =3D -12; + + bpf_spin_lock(&lock); + /* Test5: bpf_list_empty */ + if (!bpf_list_empty(&head)) + err =3D -13; + bpf_spin_unlock(&lock); + return err; +} + +SEC("?tc") +__failure __msg("bpf_spin_lock at off=3D32 must be held for bpf_list_head") +long list_del_without_lock_fail(void *ctx) +{ + struct bpf_rb_node *rb; + struct bpf_list_node *l; + struct node_data *n; + + bpf_spin_lock(&lock); + rb =3D bpf_rbtree_first(&root); + bpf_spin_unlock(&lock); + if (!rb) + return -1; + + n =3D container_of(rb, struct node_data, r); + /* Error case: delete list node without holding lock */ + l =3D bpf_list_del(&head, &n->l); + if (!l) + return -2; + bpf_obj_drop(container_of(l, struct node_data, l)); + + return 0; +} + +SEC("?tc") +__failure __msg("bpf_spin_lock at off=3D32 must be held for bpf_list_head") +long list_add_without_lock_fail(void *ctx) +{ + struct bpf_rb_node *rb; + struct bpf_list_node *l; + struct node_data *n; + + bpf_spin_lock(&lock); + rb =3D bpf_rbtree_first(&root); + l =3D bpf_list_front(&head); + bpf_spin_unlock(&lock); + if (!rb || !l) + return -1; + + n =3D container_of(l, struct node_data, l); + /* Error case: add list node without holding lock */ + if (bpf_list_add(&head, &n->l, l)) + return -2; + + return 0; +} + SEC("tc") __success long rbtree_refcounted_node_ref_escapes(void *ctx) --=20 2.50.1 (Apple Git-155) From nobody Thu Apr 9 13:31:47 2026 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 171BE259CA9 for ; Sun, 8 Mar 2026 13:47:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977674; cv=none; b=B7+vB7BGKfsT8Fn7E6dUeYqOkNWQeaGo478ypyj18CRLEFRMFgTYU3vf+ThWjHb6O2SfFJ/hYPeOZ96PqrZeUjC6hicnBIUcY/ImRkQ7I37zFirA57NzWyDVmtwKa3wyGATAA6mNbUJX+4Zd8d6mnBQXOrpeIZDLc1IU/POFZJQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772977674; c=relaxed/simple; bh=Zw6ImnSf6lR/+yeVgzkNmTem6QdcwnLp6DHs2X5E5qk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JYfn/FyhJs06hBNJAqumY5oX2bLaOwuO8xNLcgaEfAYTKCU0+GCwQbTcg6h87TCwgPo+ILsLUrMxouE+usZk8xF4x0IRQ6g9f0IzsLDA6ujmnJO8Otw4cEJdZFKJzrCK7qYMS88Q7pUr3q/RyDOvfSjHxAuTPeDFDPnBfRyGIkM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Tgij0Y4L; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Tgij0Y4L" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-82989744ee0so2739188b3a.2 for ; Sun, 08 Mar 2026 06:47:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772977672; x=1773582472; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eNkJj3izrk8VdKqI7b13r9RBEyjaruaTSc4dC/sIOZs=; b=Tgij0Y4LmHt0UGkjj+6F8Xs8IU15gzIZSkyIpnMhEcvq1QAPo/iu1Da1Ot3MEMkdUP Ko9MUOhROb2hqmrjOtJ9Xdqg6ppAoeE0HOOyJxnsPAbsdoymGi9ctrOaqulxaMvjLxhp dbMr0oOCy83r6KHYWJKK1UuIFvhemI0sl1ukTG2v60NUFhKSkHUDz12iHRbZ75XyJFdB HnkYoLVTj3jyFtBWpzHvkeoAYWnHR0VixurK1x00Os2Axh+kwbLzTJupwCoeYS5hIrgl OEYgzbbFL9FdrioeWDLbkDpvLV46Jm/8rhBbeXPhK3orVVh3YZdCyveMdD+gTLmTovmn Im0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772977672; x=1773582472; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=eNkJj3izrk8VdKqI7b13r9RBEyjaruaTSc4dC/sIOZs=; b=BaOTGnNLtu5FTwvuI6gE/aWwmwsj/DJjpg4QkbHxKzYM7HgqUQOUDYwis643i/RreE g4PKqFdC5ffd+FOY5/Xp1rJG4oe76VxAAhuugc4JNGUt9fbCJ2PwrrGiceFiGl+LINbw yfL4Nla97z6q1NnQ43PbnCtRHAbNHTGnn2UYohHYRbz/4G3e7gCJy/Ftc7wiIDUIfOjX qwX0+y79b81QWMch4Pwg6QxGOeJLyqaj1vX4vrROpmmW4FwNI0/Na/25IpJFeHnIWKsb soqv3kPNZTtclcIpP+b5ABAIQ3lU2x6TKXmYKKw2B3l0vOjm2sWOC0gUmnurhZyD0vpt iw+Q== X-Forwarded-Encrypted: i=1; AJvYcCU8sE06ZOkjI/91Ec6HbctJzdQRJw4N876ddLD5uUGnQWCh7APE6j1J13TTEKa7NIC2CdtV78HH1D4WlX4=@vger.kernel.org X-Gm-Message-State: AOJu0YwkuwUESDc4/UAdDDsCFqTQKLVqZmRQu89gycNZXKQFX7VI/L45 3Ijf0Rw1scSMrMyPLxdtbeH6rEQnlFlqArde1sOfgX/85XsIfjgmb/LD X-Gm-Gg: ATEYQzyTquTTdXorjejwgH30/hUHKwWBeBfdRygNwTBlwPwR21J2PqHq0IODsoUaNsV 0wzPyPsQOsLLoX8HeUtrB+da4tpmNDbaZ6bBrXHyCZ34dXpL53ddtFxQCwzaZQ8audPYa+UpJqf I+X6xjl5/4yKUT/1PnZCGKCbYkyzkCTH1V7QUTlYmC05+y4ug+KuZFMO3pa/itSkCkMSDndgPg9 xfJ2vcJiL7K5assFe3Caac4p1H8OO/djLA/Irfr1cKVh3bSV0AxJ5rXMwk36l6DXGOhGMXQpFqU AwIOadZ3flmEwI2ik/ks/EreTuNUqVsRXHKM75/0+VZBf7ft3b8e0SrMHEVO4Sp/q0U3k13dSN1 hOzdYWSDJWY+qfQ+5Qq2yZPMJs8P4IBQtV1Y04w7o9xv7vM+CCa56DNKNh8jf4QFObT5CCNlAxT m3RFJexrxJoZ7OyIxeEwMormXUOn4uXaXdN9UzrR6Zgr0JJdEJ X-Received: by 2002:a05:6a00:f0e:b0:829:8ac8:ea35 with SMTP id d2e1a72fcca58-829a2f6f491mr7428397b3a.51.1772977672416; Sun, 08 Mar 2026 06:47:52 -0700 (PDT) Received: from localhost.localdomain ([113.218.252.35]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-829a4636968sm7095056b3a.9.2026.03.08.06.47.41 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 08 Mar 2026 06:47:52 -0700 (PDT) From: Chengkaitao To: martin.lau@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, chengkaitao@kylinos.cn, linux-kselftest@vger.kernel.org Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v7 5/5] bpf: refactor kfunc checks using table-driven approach in verifier Date: Sun, 8 Mar 2026 21:46:14 +0800 Message-ID: <20260308134614.29711-6-pilgrimtao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260308134614.29711-1-pilgrimtao@gmail.com> References: <20260308134614.29711-1-pilgrimtao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaitao Cheng Replace per-kfunc btf_id chains in list/rbtree/res_lock and graph node checks with btf_id_in_kfunc_table() and static kfunc tables for easier maintenance. Signed-off-by: Kaitao Cheng --- kernel/bpf/verifier.c | 97 +++++++++++++++++++++++++++++-------------- 1 file changed, 66 insertions(+), 31 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5e32e02429c4..853716f599ce 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12602,6 +12602,62 @@ BTF_ID(func, bpf_session_is_return) BTF_ID(func, bpf_stream_vprintk) BTF_ID(func, bpf_stream_print_stack) =20 +static const enum special_kfunc_type bpf_list_api_kfuncs[] =3D { + KF_bpf_list_push_front_impl, + KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, + KF_bpf_list_pop_front, + KF_bpf_list_pop_back, + KF_bpf_list_del, + KF_bpf_list_front, + KF_bpf_list_back, + KF_bpf_list_is_first, + KF_bpf_list_is_last, + KF_bpf_list_empty, +}; + +/* Kfuncs that take a list node argument (bpf_list_node *). */ +static const enum special_kfunc_type bpf_list_node_api_kfuncs[] =3D { + KF_bpf_list_push_front_impl, + KF_bpf_list_push_back_impl, + KF_bpf_list_add_impl, + KF_bpf_list_del, + KF_bpf_list_is_first, + KF_bpf_list_is_last, +}; + +/* Kfuncs that take an rbtree node argument (bpf_rb_node *). */ +static const enum special_kfunc_type bpf_rbtree_node_api_kfuncs[] =3D { + KF_bpf_rbtree_remove, + KF_bpf_rbtree_add_impl, + KF_bpf_rbtree_left, + KF_bpf_rbtree_right, +}; + +static const enum special_kfunc_type bpf_rbtree_api_kfuncs[] =3D { + KF_bpf_rbtree_add_impl, + KF_bpf_rbtree_remove, + KF_bpf_rbtree_first, + KF_bpf_rbtree_root, + KF_bpf_rbtree_left, + KF_bpf_rbtree_right, +}; + +static const enum special_kfunc_type bpf_res_spin_lock_kfuncs[] =3D { + KF_bpf_res_spin_lock, + KF_bpf_res_spin_unlock, + KF_bpf_res_spin_lock_irqsave, + KF_bpf_res_spin_unlock_irqrestore, +}; + +static bool btf_id_in_kfunc_table(u32 btf_id, const enum special_kfunc_typ= e *kfuncs, int n) +{ + for (int i =3D 0; i < n; i++) + if (btf_id =3D=3D special_kfunc_list[kfuncs[i]]) + return true; + return false; +} + static bool is_task_work_add_kfunc(u32 func_id) { return func_id =3D=3D special_kfunc_list[KF_bpf_task_work_schedule_signal= ] || @@ -13002,27 +13058,14 @@ static int check_reg_allocation_locked(struct bpf= _verifier_env *env, struct bpf_ =20 static bool is_bpf_list_api_kfunc(u32 btf_id) { - return btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_front] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_pop_back] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_del] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_front] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_back] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_is_first] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_is_last] || - btf_id =3D=3D special_kfunc_list[KF_bpf_list_empty]; + return btf_id_in_kfunc_table(btf_id, bpf_list_api_kfuncs, + ARRAY_SIZE(bpf_list_api_kfuncs)); } =20 static bool is_bpf_rbtree_api_kfunc(u32 btf_id) { - return btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_remove] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_first] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_root] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_left] || - btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_right]; + return btf_id_in_kfunc_table(btf_id, bpf_rbtree_api_kfuncs, + ARRAY_SIZE(bpf_rbtree_api_kfuncs)); } =20 static bool is_bpf_iter_num_api_kfunc(u32 btf_id) @@ -13040,10 +13083,8 @@ static bool is_bpf_graph_api_kfunc(u32 btf_id) =20 static bool is_bpf_res_spin_lock_kfunc(u32 btf_id) { - return btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_lock] || - btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_unlock] || - btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_lock_irqsave] || - btf_id =3D=3D special_kfunc_list[KF_bpf_res_spin_unlock_irqrestore= ]; + return btf_id_in_kfunc_table(btf_id, bpf_res_spin_lock_kfuncs, + ARRAY_SIZE(bpf_res_spin_lock_kfuncs)); } =20 static bool is_bpf_arena_kfunc(u32 btf_id) @@ -13132,18 +13173,12 @@ static bool check_kfunc_is_graph_node_api(struct = bpf_verifier_env *env, =20 switch (node_field_type) { case BPF_LIST_NODE: - ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_front_i= mpl] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_push_back_impl= ] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_add_impl] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_del] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_is_first] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_list_is_last]); + ret =3D btf_id_in_kfunc_table(kfunc_btf_id, bpf_list_node_api_kfuncs, + ARRAY_SIZE(bpf_list_node_api_kfuncs)); break; case BPF_RB_NODE: - ret =3D (kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_remove] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_add_impl] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_left] || - kfunc_btf_id =3D=3D special_kfunc_list[KF_bpf_rbtree_right]); + ret =3D btf_id_in_kfunc_table(kfunc_btf_id, bpf_rbtree_node_api_kfuncs, + ARRAY_SIZE(bpf_rbtree_node_api_kfuncs)); break; default: verbose(env, "verifier internal error: unexpected graph node argument ty= pe %s\n", --=20 2.50.1 (Apple Git-155)