From nobody Sun Oct 5 17:56:35 2025 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5995E2BDC1D; Thu, 31 Jul 2025 09:24:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.195 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753953886; cv=none; b=Q/guKvcBkc/UTqtX3WrkMXlC38DWoGI2mq4r1AlExNtfKDpwzJYbMxai1lTbzCA4nCaAwkDFM52JKsMsWAfWvYyiDGW6Nv76xDK2518T6AxfdOZ5F/Eh8xtQ9B9Nl+8/ouoqvQOo02zzhXPGxVxsqqSDWJuX6C3Nedo6ka2nLck= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753953886; c=relaxed/simple; bh=/tgD51DEMY/99i9Q6tGBTqJ1Jrsh0iL1bQAqAHmMO0E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QkJJGPYrdhq5YZfUeyURa05gfrzvlfJrQM94r9NrEsXZyKLIu6lNgTJB0Tw68nJCQe70/vRq22I232xDCS9IzwnjihUqWMT5hs5FvPrIdei31S9+ncUOiNn/XgKH2B/FCh3ge8vJZk9YEmYAXcSMTh4kP+BN3fXvnXxyV+O/5iA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AVNV+t4E; arc=none smtp.client-ip=209.85.210.195 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AVNV+t4E" Received: by mail-pf1-f195.google.com with SMTP id d2e1a72fcca58-76b0724d64bso645578b3a.1; Thu, 31 Jul 2025 02:24:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753953883; x=1754558683; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6U5gNA8e+vROYthJvOv16nHX/WnYabMsA44r8KL57Dk=; b=AVNV+t4Ewt84wNvC/yC37v8khrlY2LXU2nKlIm/OFnwByAe1hLADc0JrPoNTiApsD6 gVYWqDWnTjWkepI5G4q5d0Kzkv1ODCBbBNeBrCt0YEJr/czemIAk45A5DAaxtpDfz3Cz pvaAU4h07vZeFCHOMhKGpN1IJX2K6nKtpF7nTg8lsWgx38sE0iNmuiD4tez4/ecoY0MR CyzyyLfaX2TsUAkbXRPwqEL5SE/zh3c0JNLfPsYcnAjOI3WMi0X5Tgonks8KicoyovWW ALGvxVmAVojYKIiRMfYaEWkbZ1cCgHrD8L194A7aeTtFxIcuhE1h8ukUSzL/SBp35JVH gciw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753953883; x=1754558683; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6U5gNA8e+vROYthJvOv16nHX/WnYabMsA44r8KL57Dk=; b=ablBTLGHl0ZgFErl6AS7xiLRoZNxubucnyvFTqHgKWg4yrcxaRkwYxp2uGi7aZ4Vhi OEmvhE0s8i/Eb8kCd61nH9fbglFPuAJyv1KRGAyKEpcYmGb+PxiHUCoOs+aMJTCNCypP 1WYfwIItHSgxFaZYjszLG3RaUhUcpmddMlQN42GMQG3MJHtR6BdGP31+wunhUuld7b3x aRQ/0hNqXhWLD3AzvUhTpBaWTlaB0M3bE/41WcZXztY88oMYfkPtoWgKrE0dy7lWoCl+ DC1bBUF2lGx0ZP4jxbb7IfkOcm5QKGB4Onaz0HYUcTO5eyau4hwVKruAT0+Aq+5Vm8AG 6geA== X-Forwarded-Encrypted: i=1; AJvYcCU/wM9DpisK/ZwOAZyy4N/yOfcar/mlxwNu76DyjKiXzkaDKBZKrsnGVhfYqAygg5uLj6xRfPHD6JT8kzYTHgxd3R2U@vger.kernel.org, AJvYcCURinOdyoEzl4JrNWFBJp/rdoNxH/wgrRy1C4LOkml72aKiCR4M4PfA5psPhP6RwBDh2Ig=@vger.kernel.org, AJvYcCWKQol28I9iZGDJuXq87AmCY0Ys7vDgAJSv5KhXPG51cBJFJUPT0ieehr5wJrpVW5QuOf6tkE8xrde56Qn/@vger.kernel.org X-Gm-Message-State: AOJu0YxEe33vxWqv7ml9C+diguCtAabTLEmV1W+mNVTxvlcMIQOKRvAv om9H2K77YImonWT9lz5QNl0G3WIKeE2ZqYgPuh6ZMsXNUsIno3/zwBpx X-Gm-Gg: ASbGncv38Z2DFPZGDn/xcfA/7LLp0w1RdZzKEbYEdkT+2B4tSoi/6DkKZ3MIJ4SGcOz EKzGgCWUeHNW1tFy5Nu/wsSqE81ec/sSh93/p/BXXGF7qVGK9tfaAT7rbyPmPaGxK2l6Da2ZtQI dYBO2xAdqUxL33VnjcnBJRM8/JhZ2p2/RDsqJclpBLxb7y11dTVNNodBtMHVATd7x/98LY+88i/ vkfvYOvID3Hen9lJQ0L6yJGW0tQBvShmKy0OeZRTnFk4VNPMPrUvCUlCIGKZPd2yRalJ/oU9wWd hoDOWb+DGHUzJmmWVixjkdVrYlxbNcIhb65v0DXLFHrH96LI0HzKM1DRjwvNyy+kGAepwpavFSX gFbsDH2ItY+QE/Qtn7W0= X-Google-Smtp-Source: AGHT+IH52k6ooqrIzYDpa3HfSnK3ao3abrWD+aDjQHMdpsdsc3JRM2ZaGRlvfdFYm8f0AE6KNOBC2A== X-Received: by 2002:a05:6a00:1897:b0:742:a77b:8c3 with SMTP id d2e1a72fcca58-76ab0827e7dmr10346584b3a.4.1753953883311; Thu, 31 Jul 2025 02:24:43 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76bccfbd1a7sm1108143b3a.73.2025.07.31.02.24.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 02:24:42 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: mhiramat@kernel.org, olsajiri@gmail.com Cc: rostedt@goodmis.org, mathieu.desnoyers@efficios.com, hca@linux.ibm.com, revest@chromium.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 1/4] fprobe: use rhltable for fprobe_ip_table Date: Thu, 31 Jul 2025 17:24:24 +0800 Message-ID: <20250731092433.49367-2-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731092433.49367-1-dongml2@chinatelecom.cn> References: <20250731092433.49367-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For now, all the kernel functions who are hooked by the fprobe will be added to the hash table "fprobe_ip_table". The key of it is the function address, and the value of it is "struct fprobe_hlist_node". The budget of the hash table is FPROBE_IP_TABLE_SIZE, which is 256. And this means the overhead of the hash table lookup will grow linearly if the count of the functions in the fprobe more than 256. When we try to hook all the kernel functions, the overhead will be huge. Therefore, replace the hash table with rhltable to reduce the overhead. Signed-off-by: Menglong Dong --- v3: - some format optimization - handle the error that returned from rhltable_insert in insert_fprobe_node --- include/linux/fprobe.h | 3 +- kernel/trace/fprobe.c | 154 +++++++++++++++++++++++------------------ 2 files changed, 90 insertions(+), 67 deletions(-) diff --git a/include/linux/fprobe.h b/include/linux/fprobe.h index 702099f08929..f5d8982392b9 100644 --- a/include/linux/fprobe.h +++ b/include/linux/fprobe.h @@ -7,6 +7,7 @@ #include #include #include +#include #include =20 struct fprobe; @@ -26,7 +27,7 @@ typedef void (*fprobe_exit_cb)(struct fprobe *fp, unsigne= d long entry_ip, * @fp: The fprobe which owns this. */ struct fprobe_hlist_node { - struct hlist_node hlist; + struct rhlist_head hlist; unsigned long addr; struct fprobe *fp; }; diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c index ba7ff14f5339..2f1683a26c10 100644 --- a/kernel/trace/fprobe.c +++ b/kernel/trace/fprobe.c @@ -41,47 +41,46 @@ * - RCU hlist traversal under disabling preempt */ static struct hlist_head fprobe_table[FPROBE_TABLE_SIZE]; -static struct hlist_head fprobe_ip_table[FPROBE_IP_TABLE_SIZE]; +static struct rhltable fprobe_ip_table; static DEFINE_MUTEX(fprobe_mutex); =20 -/* - * Find first fprobe in the hlist. It will be iterated twice in the entry - * probe, once for correcting the total required size, the second time is - * calling back the user handlers. - * Thus the hlist in the fprobe_table must be sorted and new probe needs to - * be added *before* the first fprobe. - */ -static struct fprobe_hlist_node *find_first_fprobe_node(unsigned long ip) +static u32 fprobe_node_hashfn(const void *data, u32 len, u32 seed) { - struct fprobe_hlist_node *node; - struct hlist_head *head; + return hash_ptr(*(unsigned long **)data, 32); +} =20 - head =3D &fprobe_ip_table[hash_ptr((void *)ip, FPROBE_IP_HASH_BITS)]; - hlist_for_each_entry_rcu(node, head, hlist, - lockdep_is_held(&fprobe_mutex)) { - if (node->addr =3D=3D ip) - return node; - } - return NULL; +static int fprobe_node_cmp(struct rhashtable_compare_arg *arg, + const void *ptr) +{ + unsigned long key =3D *(unsigned long *)arg->key; + const struct fprobe_hlist_node *n =3D ptr; + + return n->addr !=3D key; } -NOKPROBE_SYMBOL(find_first_fprobe_node); =20 -/* Node insertion and deletion requires the fprobe_mutex */ -static void insert_fprobe_node(struct fprobe_hlist_node *node) +static u32 fprobe_node_obj_hashfn(const void *data, u32 len, u32 seed) { - unsigned long ip =3D node->addr; - struct fprobe_hlist_node *next; - struct hlist_head *head; + const struct fprobe_hlist_node *n =3D data; + + return hash_ptr((void *)n->addr, 32); +} + +static const struct rhashtable_params fprobe_rht_params =3D { + .head_offset =3D offsetof(struct fprobe_hlist_node, hlist), + .key_offset =3D offsetof(struct fprobe_hlist_node, addr), + .key_len =3D sizeof_field(struct fprobe_hlist_node, addr), + .hashfn =3D fprobe_node_hashfn, + .obj_hashfn =3D fprobe_node_obj_hashfn, + .obj_cmpfn =3D fprobe_node_cmp, + .automatic_shrinking =3D true, +}; =20 +/* Node insertion and deletion requires the fprobe_mutex */ +static int insert_fprobe_node(struct fprobe_hlist_node *node) +{ lockdep_assert_held(&fprobe_mutex); =20 - next =3D find_first_fprobe_node(ip); - if (next) { - hlist_add_before_rcu(&node->hlist, &next->hlist); - return; - } - head =3D &fprobe_ip_table[hash_ptr((void *)ip, FPROBE_IP_HASH_BITS)]; - hlist_add_head_rcu(&node->hlist, head); + return rhltable_insert(&fprobe_ip_table, &node->hlist, fprobe_rht_params); } =20 /* Return true if there are synonims */ @@ -92,9 +91,11 @@ static bool delete_fprobe_node(struct fprobe_hlist_node = *node) /* Avoid double deleting */ if (READ_ONCE(node->fp) !=3D NULL) { WRITE_ONCE(node->fp, NULL); - hlist_del_rcu(&node->hlist); + rhltable_remove(&fprobe_ip_table, &node->hlist, + fprobe_rht_params); } - return !!find_first_fprobe_node(node->addr); + return !!rhltable_lookup(&fprobe_ip_table, &node->addr, + fprobe_rht_params); } =20 /* Check existence of the fprobe */ @@ -249,9 +250,10 @@ static inline int __fprobe_kprobe_handler(unsigned lon= g ip, unsigned long parent static int fprobe_entry(struct ftrace_graph_ent *trace, struct fgraph_ops = *gops, struct ftrace_regs *fregs) { - struct fprobe_hlist_node *node, *first; unsigned long *fgraph_data =3D NULL; unsigned long func =3D trace->func; + struct fprobe_hlist_node *node; + struct rhlist_head *head, *pos; unsigned long ret_ip; int reserved_words; struct fprobe *fp; @@ -260,14 +262,12 @@ static int fprobe_entry(struct ftrace_graph_ent *trac= e, struct fgraph_ops *gops, if (WARN_ON_ONCE(!fregs)) return 0; =20 - first =3D node =3D find_first_fprobe_node(func); - if (unlikely(!first)) - return 0; - + rcu_read_lock(); + head =3D rhltable_lookup(&fprobe_ip_table, &func, fprobe_rht_params); reserved_words =3D 0; - hlist_for_each_entry_from_rcu(node, hlist) { + rhl_for_each_entry_rcu(node, pos, head, hlist) { if (node->addr !=3D func) - break; + continue; fp =3D READ_ONCE(node->fp); if (!fp || !fp->exit_handler) continue; @@ -278,17 +278,19 @@ static int fprobe_entry(struct ftrace_graph_ent *trac= e, struct fgraph_ops *gops, reserved_words +=3D FPROBE_HEADER_SIZE_IN_LONG + SIZE_IN_LONG(fp->entry_data_size); } - node =3D first; + rcu_read_unlock(); if (reserved_words) { fgraph_data =3D fgraph_reserve_data(gops->idx, reserved_words * sizeof(l= ong)); if (unlikely(!fgraph_data)) { - hlist_for_each_entry_from_rcu(node, hlist) { + rcu_read_lock(); + rhl_for_each_entry_rcu(node, pos, head, hlist) { if (node->addr !=3D func) - break; + continue; fp =3D READ_ONCE(node->fp); if (fp && !fprobe_disabled(fp)) fp->nmissed++; } + rcu_read_unlock(); return 0; } } @@ -299,12 +301,12 @@ static int fprobe_entry(struct ftrace_graph_ent *trac= e, struct fgraph_ops *gops, */ ret_ip =3D ftrace_regs_get_return_address(fregs); used =3D 0; - hlist_for_each_entry_from_rcu(node, hlist) { + rhl_for_each_entry_rcu(node, pos, head, hlist) { int data_size; void *data; =20 if (node->addr !=3D func) - break; + continue; fp =3D READ_ONCE(node->fp); if (!fp || fprobe_disabled(fp)) continue; @@ -448,25 +450,21 @@ static int fprobe_addr_list_add(struct fprobe_addr_li= st *alist, unsigned long ad return 0; } =20 -static void fprobe_remove_node_in_module(struct module *mod, struct hlist_= head *head, - struct fprobe_addr_list *alist) +static void fprobe_remove_node_in_module(struct module *mod, struct fprobe= _hlist_node *node, + struct fprobe_addr_list *alist) { - struct fprobe_hlist_node *node; int ret =3D 0; =20 - hlist_for_each_entry_rcu(node, head, hlist, - lockdep_is_held(&fprobe_mutex)) { - if (!within_module(node->addr, mod)) - continue; - if (delete_fprobe_node(node)) - continue; - /* - * If failed to update alist, just continue to update hlist. - * Therefore, at list user handler will not hit anymore. - */ - if (!ret) - ret =3D fprobe_addr_list_add(alist, node->addr); - } + if (!within_module(node->addr, mod)) + return; + if (delete_fprobe_node(node)) + return; + /* + * If failed to update alist, just continue to update hlist. + * Therefore, at list user handler will not hit anymore. + */ + if (!ret) + ret =3D fprobe_addr_list_add(alist, node->addr); } =20 /* Handle module unloading to manage fprobe_ip_table. */ @@ -474,8 +472,9 @@ static int fprobe_module_callback(struct notifier_block= *nb, unsigned long val, void *data) { struct fprobe_addr_list alist =3D {.size =3D FPROBE_IPS_BATCH_INIT}; + struct fprobe_hlist_node *node; + struct rhashtable_iter iter; struct module *mod =3D data; - int i; =20 if (val !=3D MODULE_STATE_GOING) return NOTIFY_DONE; @@ -486,8 +485,16 @@ static int fprobe_module_callback(struct notifier_bloc= k *nb, return NOTIFY_DONE; =20 mutex_lock(&fprobe_mutex); - for (i =3D 0; i < FPROBE_IP_TABLE_SIZE; i++) - fprobe_remove_node_in_module(mod, &fprobe_ip_table[i], &alist); + rhashtable_walk_enter(&fprobe_ip_table.ht, &iter); + do { + rhashtable_walk_start(&iter); + + while ((node =3D rhashtable_walk_next(&iter)) && !IS_ERR(node)) + fprobe_remove_node_in_module(mod, node, &alist); + + rhashtable_walk_stop(&iter); + } while (node =3D=3D ERR_PTR(-EAGAIN)); + rhashtable_walk_exit(&iter); =20 if (alist.index < alist.size && alist.index > 0) ftrace_set_filter_ips(&fprobe_graph_ops.ops, @@ -722,8 +729,16 @@ int register_fprobe_ips(struct fprobe *fp, unsigned lo= ng *addrs, int num) ret =3D fprobe_graph_add_ips(addrs, num); if (!ret) { add_fprobe_hash(fp); - for (i =3D 0; i < hlist_array->size; i++) - insert_fprobe_node(&hlist_array->array[i]); + for (i =3D 0; i < hlist_array->size; i++) { + ret =3D insert_fprobe_node(&hlist_array->array[i]); + if (ret) + break; + } + /* fallback on insert error */ + if (ret) { + for (i--; i >=3D 0; i--) + delete_fprobe_node(&hlist_array->array[i]); + } } mutex_unlock(&fprobe_mutex); =20 @@ -819,3 +834,10 @@ int unregister_fprobe(struct fprobe *fp) return ret; } EXPORT_SYMBOL_GPL(unregister_fprobe); + +static int __init fprobe_initcall(void) +{ + rhltable_init(&fprobe_ip_table, &fprobe_rht_params); + return 0; +} +late_initcall(fprobe_initcall); --=20 2.50.1 From nobody Sun Oct 5 17:56:35 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EDB52BE021; Thu, 31 Jul 2025 09:24:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753953889; cv=none; b=N61nqGV+mw210Nax6aOLcmgRRD3r8YpOSgUXjcHZOabWE0D0M+WC1JQT6Cr0HpsjjVI/KMCDeSjwY5buqIcO1SEAzIrtMqIlFdfJGQHytVMkc8O+6Y58ZIxnLavp1szxVkOFBzVNPot6b8aISYt54sCjHYs59aIhVld0vZX4a3A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753953889; c=relaxed/simple; bh=zudiWkkAlg6yySZkWMGW/IXw6hzgirR0YopZ0oI3JAs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I1ECuT0tF4lo3okt6Xc0N2BJbvMST/RDg2JuTEnL8ZvoDOlRXFx+ZYTHG7D048ldxqwyeZdUFgxan97pC38a4nTjrDa3KudaP+qm5JyrchoFnhlGQspfRLeO5DtWzKG9GeexcqbrC4WWTXV6Lp0UjXLSf8fiLGnTCGbmKrugz20= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XxgUpiUD; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XxgUpiUD" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-76a3818eb9bso631334b3a.3; Thu, 31 Jul 2025 02:24:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753953887; x=1754558687; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=y8iwRQl4S0Ri7auKWOImDztrR8W3w7sQvsgC0R12G/0=; b=XxgUpiUDcCeM13T6ki143KfG9fhA5SXRW3WRMYAwXLTzqVASWoH6SZMQTpFcGsBUKg yTGgJkUMe8rQMyk9ZbYIWRKY8UHiM7l89uL6T0bvaRVRSrGH7hGWBHaCNqBiZ3AoApUe jl06ymP54+QsizKsikas0jrS6PsMjW0REbocuSRVAoBABfO/hzSHQKy2AK0BF6e1WMiX Au7HEpuv3xJCRAbJmEdzpJZkzNHOIYJjldj1MNfmzqLVaD2oXpnzZvbQR25GKvDC5ZFN aulw+PA4hwpu4iWvoi3zSgQRazBfZTskzqU69bvdEKxPlC5dy0XzZVaGlBNNuOgdx2i/ gslQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753953887; x=1754558687; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y8iwRQl4S0Ri7auKWOImDztrR8W3w7sQvsgC0R12G/0=; b=mFGqzCQCIvIfmC9wwzvPfIWnbve46jTHpxN0DGEuSVn0qFhgKkIgf6TzmysVkjLXK3 WzhYrPHrovUTlC9lS3/oqj0+UC+hfLal5RLZRbJ1LG1DBEynAS1XnAV9aWnaAta2+izf nQuD1uYUMoM6yMEnPvi+bpTVBcLxC5MkIFbCzT7BJ4y4Ga+Oq35as/0t95VSma9M2mSO WKtqIvZlYrrmO209dNE3Kst5ftVNMbmwsxRfjL2MfxUKhu8gOck9RYpkm1h8HSYUhNNQ 0B/yymlZmn7Z2iYp81RLbYzNC/h7n6t1eVhAcLc0YChV97DkXaobbBGLALxHpmeGGyqs Po4Q== X-Forwarded-Encrypted: i=1; AJvYcCV68VPNRZ/4wKjNRQ0uEYkIKE3zzNk0PCLiNhmKVAX239W4jmOecjrvjG6KrvRyQR8Zv/7eDdARtxKNBMSw/lKrq46S@vger.kernel.org, AJvYcCVIRdS2gCtxApY9TwVAsd4TK4m1S8cAuOiAWtOZ2C4+dtB98GuGC9WPGeR8h2G4D8+6SX8=@vger.kernel.org, AJvYcCWcCkNLo/VI7vZUJaJcyMNfb02TiTY4hZH+vRw1k5lUAvD3ir9VXCsm/hZkZY9DmyF9PL0kvrfVGa2zNxlY@vger.kernel.org X-Gm-Message-State: AOJu0YzRd7HRHlx02AMR1wASYIng+4xy5wYvFSwzOnbZ0DObLFqQsC9B /8QzIEsHlLytXmP/SWrSza2PXeQb1D8+oLgDTFIfy40p825IQdNMG/k4 X-Gm-Gg: ASbGncsB+tWCJf8jWX6qoKN8u34jP6wpraF9iDURquY4+3QcG9ijnkZoAU+31q5gvAg wd8bM9TdORFtORqtNQjl4moJcEM2osLkM1oqp4YR0utK/v5UhWpO0EgVcVHLazK+SW2dHkJbbsO MlHt1IShjwZFTtFY3RLqg7Ng6IxwwpPtmWfv+3B728Mo9t6tUHqsVypiJLHY/X07GkWsS1BybLX jc7OS/zHL3o87aNUFcRVIepyI71tD1R1w7eA4wlkdvs+0HKPbkRvbB6c++y80gm2cDGtaJYXdCp ckxRHMY1zjuHLL5ZalZreyEFs9C16/vAmBSTH3tc5fPNdG6YPTXJd5JQupbIsmGXe5b1k+dVMtw A2PvrrVxPrfeR5vUazlvInoZ2Dn80Jw== X-Google-Smtp-Source: AGHT+IE69IrDX4EBsii1X3G+FeMs7JJnWEdul/j2MjZ8EJTWz2KT10E5CGTtSAT1HFtTXb9arqBJiQ== X-Received: by 2002:a05:6a00:368f:b0:76b:8b13:e076 with SMTP id d2e1a72fcca58-76b8b2328cdmr6144314b3a.5.1753953886752; Thu, 31 Jul 2025 02:24:46 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76bccfbd1a7sm1108143b3a.73.2025.07.31.02.24.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 02:24:46 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: mhiramat@kernel.org, olsajiri@gmail.com Cc: rostedt@goodmis.org, mathieu.desnoyers@efficios.com, hca@linux.ibm.com, revest@chromium.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 2/4] selftests/bpf: move get_ksyms and get_addrs to trace_helpers.c Date: Thu, 31 Jul 2025 17:24:25 +0800 Message-ID: <20250731092433.49367-3-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731092433.49367-1-dongml2@chinatelecom.cn> References: <20250731092433.49367-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We need to get all the kernel function that can be traced sometimes, so we move the get_syms() and get_addrs() in kprobe_multi_test.c to trace_helpers.c and rename it to bpf_get_ksyms() and bpf_get_addrs(). Signed-off-by: Menglong Dong --- .../bpf/prog_tests/kprobe_multi_test.c | 220 +----------------- tools/testing/selftests/bpf/trace_helpers.c | 214 +++++++++++++++++ tools/testing/selftests/bpf/trace_helpers.h | 3 + 3 files changed, 220 insertions(+), 217 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c b/t= ools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c index e19ef509ebf8..171706e78da8 100644 --- a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c @@ -422,220 +422,6 @@ static void test_unique_match(void) kprobe_multi__destroy(skel); } =20 -static size_t symbol_hash(long key, void *ctx __maybe_unused) -{ - return str_hash((const char *) key); -} - -static bool symbol_equal(long key1, long key2, void *ctx __maybe_unused) -{ - return strcmp((const char *) key1, (const char *) key2) =3D=3D 0; -} - -static bool is_invalid_entry(char *buf, bool kernel) -{ - if (kernel && strchr(buf, '[')) - return true; - if (!kernel && !strchr(buf, '[')) - return true; - return false; -} - -static bool skip_entry(char *name) -{ - /* - * We attach to almost all kernel functions and some of them - * will cause 'suspicious RCU usage' when fprobe is attached - * to them. Filter out the current culprits - arch_cpu_idle - * default_idle and rcu_* functions. - */ - if (!strcmp(name, "arch_cpu_idle")) - return true; - if (!strcmp(name, "default_idle")) - return true; - if (!strncmp(name, "rcu_", 4)) - return true; - if (!strcmp(name, "bpf_dispatcher_xdp_func")) - return true; - if (!strncmp(name, "__ftrace_invalid_address__", - sizeof("__ftrace_invalid_address__") - 1)) - return true; - return false; -} - -/* Do comparision by ignoring '.llvm.' suffixes. */ -static int compare_name(const char *name1, const char *name2) -{ - const char *res1, *res2; - int len1, len2; - - res1 =3D strstr(name1, ".llvm."); - res2 =3D strstr(name2, ".llvm."); - len1 =3D res1 ? res1 - name1 : strlen(name1); - len2 =3D res2 ? res2 - name2 : strlen(name2); - - if (len1 =3D=3D len2) - return strncmp(name1, name2, len1); - if (len1 < len2) - return strncmp(name1, name2, len1) <=3D 0 ? -1 : 1; - return strncmp(name1, name2, len2) >=3D 0 ? 1 : -1; -} - -static int load_kallsyms_compare(const void *p1, const void *p2) -{ - return compare_name(((const struct ksym *)p1)->name, ((const struct ksym = *)p2)->name); -} - -static int search_kallsyms_compare(const void *p1, const struct ksym *p2) -{ - return compare_name(p1, p2->name); -} - -static int get_syms(char ***symsp, size_t *cntp, bool kernel) -{ - size_t cap =3D 0, cnt =3D 0; - char *name =3D NULL, *ksym_name, **syms =3D NULL; - struct hashmap *map; - struct ksyms *ksyms; - struct ksym *ks; - char buf[256]; - FILE *f; - int err =3D 0; - - ksyms =3D load_kallsyms_custom_local(load_kallsyms_compare); - if (!ASSERT_OK_PTR(ksyms, "load_kallsyms_custom_local")) - return -EINVAL; - - /* - * The available_filter_functions contains many duplicates, - * but other than that all symbols are usable in kprobe multi - * interface. - * Filtering out duplicates by using hashmap__add, which won't - * add existing entry. - */ - - if (access("/sys/kernel/tracing/trace", F_OK) =3D=3D 0) - f =3D fopen("/sys/kernel/tracing/available_filter_functions", "r"); - else - f =3D fopen("/sys/kernel/debug/tracing/available_filter_functions", "r"); - - if (!f) - return -EINVAL; - - map =3D hashmap__new(symbol_hash, symbol_equal, NULL); - if (IS_ERR(map)) { - err =3D libbpf_get_error(map); - goto error; - } - - while (fgets(buf, sizeof(buf), f)) { - if (is_invalid_entry(buf, kernel)) - continue; - - free(name); - if (sscanf(buf, "%ms$*[^\n]\n", &name) !=3D 1) - continue; - if (skip_entry(name)) - continue; - - ks =3D search_kallsyms_custom_local(ksyms, name, search_kallsyms_compare= ); - if (!ks) { - err =3D -EINVAL; - goto error; - } - - ksym_name =3D ks->name; - err =3D hashmap__add(map, ksym_name, 0); - if (err =3D=3D -EEXIST) { - err =3D 0; - continue; - } - if (err) - goto error; - - err =3D libbpf_ensure_mem((void **) &syms, &cap, - sizeof(*syms), cnt + 1); - if (err) - goto error; - - syms[cnt++] =3D ksym_name; - } - - *symsp =3D syms; - *cntp =3D cnt; - -error: - free(name); - fclose(f); - hashmap__free(map); - if (err) - free(syms); - return err; -} - -static int get_addrs(unsigned long **addrsp, size_t *cntp, bool kernel) -{ - unsigned long *addr, *addrs, *tmp_addrs; - int err =3D 0, max_cnt, inc_cnt; - char *name =3D NULL; - size_t cnt =3D 0; - char buf[256]; - FILE *f; - - if (access("/sys/kernel/tracing/trace", F_OK) =3D=3D 0) - f =3D fopen("/sys/kernel/tracing/available_filter_functions_addrs", "r"); - else - f =3D fopen("/sys/kernel/debug/tracing/available_filter_functions_addrs"= , "r"); - - if (!f) - return -ENOENT; - - /* In my local setup, the number of entries is 50k+ so Let us initially - * allocate space to hold 64k entries. If 64k is not enough, incrementally - * increase 1k each time. - */ - max_cnt =3D 65536; - inc_cnt =3D 1024; - addrs =3D malloc(max_cnt * sizeof(long)); - if (addrs =3D=3D NULL) { - err =3D -ENOMEM; - goto error; - } - - while (fgets(buf, sizeof(buf), f)) { - if (is_invalid_entry(buf, kernel)) - continue; - - free(name); - if (sscanf(buf, "%p %ms$*[^\n]\n", &addr, &name) !=3D 2) - continue; - if (skip_entry(name)) - continue; - - if (cnt =3D=3D max_cnt) { - max_cnt +=3D inc_cnt; - tmp_addrs =3D realloc(addrs, max_cnt); - if (!tmp_addrs) { - err =3D -ENOMEM; - goto error; - } - addrs =3D tmp_addrs; - } - - addrs[cnt++] =3D (unsigned long)addr; - } - - *addrsp =3D addrs; - *cntp =3D cnt; - -error: - free(name); - fclose(f); - if (err) - free(addrs); - return err; -} - static void do_bench_test(struct kprobe_multi_empty *skel, struct bpf_kpro= be_multi_opts *opts) { long attach_start_ns, attach_end_ns; @@ -670,7 +456,7 @@ static void test_kprobe_multi_bench_attach(bool kernel) char **syms =3D NULL; size_t cnt =3D 0; =20 - if (!ASSERT_OK(get_syms(&syms, &cnt, kernel), "get_syms")) + if (!ASSERT_OK(bpf_get_ksyms(&syms, &cnt, kernel), "bpf_get_ksyms")) return; =20 skel =3D kprobe_multi_empty__open_and_load(); @@ -696,13 +482,13 @@ static void test_kprobe_multi_bench_attach_addr(bool = kernel) size_t cnt =3D 0; int err; =20 - err =3D get_addrs(&addrs, &cnt, kernel); + err =3D bpf_get_addrs(&addrs, &cnt, kernel); if (err =3D=3D -ENOENT) { test__skip(); return; } =20 - if (!ASSERT_OK(err, "get_addrs")) + if (!ASSERT_OK(err, "bpf_get_addrs")) return; =20 skel =3D kprobe_multi_empty__open_and_load(); diff --git a/tools/testing/selftests/bpf/trace_helpers.c b/tools/testing/se= lftests/bpf/trace_helpers.c index 81943c6254e6..d24baf244d1f 100644 --- a/tools/testing/selftests/bpf/trace_helpers.c +++ b/tools/testing/selftests/bpf/trace_helpers.c @@ -17,6 +17,7 @@ #include #include #include +#include "bpf/hashmap.h" #include "bpf/libbpf_internal.h" =20 #define TRACEFS_PIPE "/sys/kernel/tracing/trace_pipe" @@ -519,3 +520,216 @@ void read_trace_pipe(void) { read_trace_pipe_iter(trace_pipe_cb, NULL, 0); } + +static size_t symbol_hash(long key, void *ctx __maybe_unused) +{ + return str_hash((const char *) key); +} + +static bool symbol_equal(long key1, long key2, void *ctx __maybe_unused) +{ + return strcmp((const char *) key1, (const char *) key2) =3D=3D 0; +} + +static bool is_invalid_entry(char *buf, bool kernel) +{ + if (kernel && strchr(buf, '[')) + return true; + if (!kernel && !strchr(buf, '[')) + return true; + return false; +} + +static bool skip_entry(char *name) +{ + /* + * We attach to almost all kernel functions and some of them + * will cause 'suspicious RCU usage' when fprobe is attached + * to them. Filter out the current culprits - arch_cpu_idle + * default_idle and rcu_* functions. + */ + if (!strcmp(name, "arch_cpu_idle")) + return true; + if (!strcmp(name, "default_idle")) + return true; + if (!strncmp(name, "rcu_", 4)) + return true; + if (!strcmp(name, "bpf_dispatcher_xdp_func")) + return true; + if (!strncmp(name, "__ftrace_invalid_address__", + sizeof("__ftrace_invalid_address__") - 1)) + return true; + return false; +} + +/* Do comparison by ignoring '.llvm.' suffixes. */ +static int compare_name(const char *name1, const char *name2) +{ + const char *res1, *res2; + int len1, len2; + + res1 =3D strstr(name1, ".llvm."); + res2 =3D strstr(name2, ".llvm."); + len1 =3D res1 ? res1 - name1 : strlen(name1); + len2 =3D res2 ? res2 - name2 : strlen(name2); + + if (len1 =3D=3D len2) + return strncmp(name1, name2, len1); + if (len1 < len2) + return strncmp(name1, name2, len1) <=3D 0 ? -1 : 1; + return strncmp(name1, name2, len2) >=3D 0 ? 1 : -1; +} + +static int load_kallsyms_compare(const void *p1, const void *p2) +{ + return compare_name(((const struct ksym *)p1)->name, ((const struct ksym = *)p2)->name); +} + +static int search_kallsyms_compare(const void *p1, const struct ksym *p2) +{ + return compare_name(p1, p2->name); +} + +int bpf_get_ksyms(char ***symsp, size_t *cntp, bool kernel) +{ + size_t cap =3D 0, cnt =3D 0; + char *name =3D NULL, *ksym_name, **syms =3D NULL; + struct hashmap *map; + struct ksyms *ksyms; + struct ksym *ks; + char buf[256]; + FILE *f; + int err =3D 0; + + ksyms =3D load_kallsyms_custom_local(load_kallsyms_compare); + if (!ksyms) + return -EINVAL; + + /* + * The available_filter_functions contains many duplicates, + * but other than that all symbols are usable to trace. + * Filtering out duplicates by using hashmap__add, which won't + * add existing entry. + */ + + if (access("/sys/kernel/tracing/trace", F_OK) =3D=3D 0) + f =3D fopen("/sys/kernel/tracing/available_filter_functions", "r"); + else + f =3D fopen("/sys/kernel/debug/tracing/available_filter_functions", "r"); + + if (!f) + return -EINVAL; + + map =3D hashmap__new(symbol_hash, symbol_equal, NULL); + if (IS_ERR(map)) { + err =3D libbpf_get_error(map); + goto error; + } + + while (fgets(buf, sizeof(buf), f)) { + if (is_invalid_entry(buf, kernel)) + continue; + + free(name); + if (sscanf(buf, "%ms$*[^\n]\n", &name) !=3D 1) + continue; + if (skip_entry(name)) + continue; + + ks =3D search_kallsyms_custom_local(ksyms, name, search_kallsyms_compare= ); + if (!ks) { + err =3D -EINVAL; + goto error; + } + + ksym_name =3D ks->name; + err =3D hashmap__add(map, ksym_name, 0); + if (err =3D=3D -EEXIST) { + err =3D 0; + continue; + } + if (err) + goto error; + + err =3D libbpf_ensure_mem((void **) &syms, &cap, + sizeof(*syms), cnt + 1); + if (err) + goto error; + + syms[cnt++] =3D ksym_name; + } + + *symsp =3D syms; + *cntp =3D cnt; + +error: + free(name); + fclose(f); + hashmap__free(map); + if (err) + free(syms); + return err; +} + +int bpf_get_addrs(unsigned long **addrsp, size_t *cntp, bool kernel) +{ + unsigned long *addr, *addrs, *tmp_addrs; + int err =3D 0, max_cnt, inc_cnt; + char *name =3D NULL; + size_t cnt =3D 0; + char buf[256]; + FILE *f; + + if (access("/sys/kernel/tracing/trace", F_OK) =3D=3D 0) + f =3D fopen("/sys/kernel/tracing/available_filter_functions_addrs", "r"); + else + f =3D fopen("/sys/kernel/debug/tracing/available_filter_functions_addrs"= , "r"); + + if (!f) + return -ENOENT; + + /* In my local setup, the number of entries is 50k+ so Let us initially + * allocate space to hold 64k entries. If 64k is not enough, incrementally + * increase 1k each time. + */ + max_cnt =3D 65536; + inc_cnt =3D 1024; + addrs =3D malloc(max_cnt * sizeof(long)); + if (addrs =3D=3D NULL) { + err =3D -ENOMEM; + goto error; + } + + while (fgets(buf, sizeof(buf), f)) { + if (is_invalid_entry(buf, kernel)) + continue; + + free(name); + if (sscanf(buf, "%p %ms$*[^\n]\n", &addr, &name) !=3D 2) + continue; + if (skip_entry(name)) + continue; + + if (cnt =3D=3D max_cnt) { + max_cnt +=3D inc_cnt; + tmp_addrs =3D realloc(addrs, max_cnt); + if (!tmp_addrs) { + err =3D -ENOMEM; + goto error; + } + addrs =3D tmp_addrs; + } + + addrs[cnt++] =3D (unsigned long)addr; + } + + *addrsp =3D addrs; + *cntp =3D cnt; + +error: + free(name); + fclose(f); + if (err) + free(addrs); + return err; +} diff --git a/tools/testing/selftests/bpf/trace_helpers.h b/tools/testing/se= lftests/bpf/trace_helpers.h index 2ce873c9f9aa..9437bdd4afa5 100644 --- a/tools/testing/selftests/bpf/trace_helpers.h +++ b/tools/testing/selftests/bpf/trace_helpers.h @@ -41,4 +41,7 @@ ssize_t get_rel_offset(uintptr_t addr); =20 int read_build_id(const char *path, char *build_id, size_t size); =20 +int bpf_get_ksyms(char ***symsp, size_t *cntp, bool kernel); +int bpf_get_addrs(unsigned long **addrsp, size_t *cntp, bool kernel); + #endif --=20 2.50.1 From nobody Sun Oct 5 17:56:35 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A74FC2BE043; Thu, 31 Jul 2025 09:24:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753953892; cv=none; b=ZeBj6ezPbmzSoh40r6mpJdLcjtPl+rG2Z4L1xH5sZ8xl0Rk4s6VUuGG52wpo0GG4h5uzD5IqRfTMNKw8pKWf6XVYYpKP+Cpie5Y1cuZQhrufcSRcroW7QukasEjR/+IqFCTV1Wa8pOfSnhoQXL/ZkH7el8YQ09wrkIR7iJdDhGI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753953892; c=relaxed/simple; bh=TGw2ul2ZAldEcVHaU0fnT+T9Zm5AaMY7QcEq1fLdVSA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EJb+KUTqWQnE7RXHaLOKusrCcYMVthr2uADY0JqfuEz1ha7UJ8IcU9gPyDJp0n1UFSzCZqw9IMaOly1e+wUjFoGywDX0rvF1srUZaL1tR3Wul3V6x8V58wFA1T1zqk7JyxhTFHH7KxiSc7HQepyrR2MOaR2vEiRg5uhynO3Nyqg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BHooIH2I; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BHooIH2I" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-7425bd5a83aso627117b3a.0; Thu, 31 Jul 2025 02:24:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753953890; x=1754558690; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DGBbLIiLGqrmFvSwaYDbAojdCre78exL+6ZjHnBuXvE=; b=BHooIH2IAYMcpBhNatGwdhjNO8bOKMHIyiTQaJ6M8kDzcsuXyBev00hRITLy0eSzLV tgJTHkGptSaaNACfQF28riBvR2+FoQ0P8NJwBjdqY9PC9hHu3o8UP8KjtnvnGfrBDqxW YeYf3pAIOSgt1LZWerlMCVROyOu9OowTfqaRAlvIKUnn9xUT0AyHt6fNnbAkls5Ptu8N XIsS7kc2h7hG6WQJhw4VSsI6gWNNrkg2IvuNgjgBlUlkO96VgIeBBWCadGJNP5AM0FMs fpaDtICwHQeh6C+sSYhaWTQqMPyJRdwwlKE0QWto0PAm7tljC5PKJsTj+Ts22f/nMJvR A0BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753953890; x=1754558690; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DGBbLIiLGqrmFvSwaYDbAojdCre78exL+6ZjHnBuXvE=; b=SKMwxY8tIAhSIlFh53Hqw7GQpNrQwMtCFK89Q+Jc9idFl2AHpDblJd+VlAHehSrdpj 68ZIJeF0tE5oh1ZDXduJR5V5EmSOBp/R7wlH7P7P/blCQ+T+AQ3KSxx5zYq++dG099Iq RgylsN7dbLGmsYyGIulvInFOVE1ZxuxSMr/mUqhkN2OnOQeEgsllU/+Ylrp4lspWvcYB 2Cxttr6atYmnCGHsY4kvPC4x58uPWQdwufxVegWoyJoFThU+4cCopUUdHTLcsKtnCyPn aF2Vbez/PVL6kClnn2632VhTY/5XVJaPZV62u12OLtmGDgY7Fy/EKM03uQ+2M7qQtDZO kkCQ== X-Forwarded-Encrypted: i=1; AJvYcCUhSNCjNY10GHaYDjuEN6mhBpgWz0N58Y7y2/1Mow8/f2BF3HQJ5hhbGklObUgJ5+fitH4ycz9/3FsF4uhx@vger.kernel.org, AJvYcCV+knHiYisP6dyv1C+xLEnQk9GDlYX8W/w3QNeTNPMemoIvtw5zrRk9S4lPLWA7wk5v3Rs=@vger.kernel.org, AJvYcCXV5fc86LHwWxipvKPv/mLfb2OcGXI2tWduWJlyJL+KOJ0i6rN8T4XEK8snU+tq9kCQVYTKfpKul/GISKvLu2TNhiKp@vger.kernel.org X-Gm-Message-State: AOJu0Yx/bT+8ho4h7V0znoVRhifb+0INtBHp7kbymlg5EB4cJucMHc3U cHm9HrHI2W7grq6wl+F1PJYCENuWXQChUOBTwacs4oYPiJV8B4mByfdc X-Gm-Gg: ASbGncsWO7ktd4piWxjicIEu8AK33S1UPEVKD8BavOHXnarTCVFXwkyzp3DRbTX91CN AKv9uoyR+tG9yy2b/wDUUmUNCkVcGSI8Q/LNUyCNnPTvHmB7kBYQkr5VmZnTiisXrM1W8s4xAmg zMoItgbU19H9gdMbzQYm6CYAC0UED2/cwOXBNIh7Uf+KNe8HKEYMun2AFMeaq2g8l8RTuZomrdu 25dkHEslMg6PjjjddfrqWoumgLqHLMSwFN5qmFpMQ5jFM0WkQVff8zADG/Wdb62H8kb3aKQv8r/ ck1zoDD28nkOmc8qGbWPHY8ZhdElvyeHnmeyZXEW28CIKzV03Sg22VCKnBzrhvffTVTSUKRycHd E9tDkr7tYGZ+edGJ4nfrrgOEQ4Kd/Hg== X-Google-Smtp-Source: AGHT+IHNh/ANB2dkQCa0nO9BTR+DxCH4IthcpzCHKS7zI734G3ATILS/eyr6az5628hpfcauKorqWA== X-Received: by 2002:a05:6a20:258f:b0:237:d013:8a78 with SMTP id adf61e73a8af0-23dc0e99cd2mr11483916637.37.1753953889900; Thu, 31 Jul 2025 02:24:49 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76bccfbd1a7sm1108143b3a.73.2025.07.31.02.24.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 02:24:49 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: mhiramat@kernel.org, olsajiri@gmail.com Cc: rostedt@goodmis.org, mathieu.desnoyers@efficios.com, hca@linux.ibm.com, revest@chromium.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 3/4] selftests/bpf: skip recursive functions for kprobe_multi Date: Thu, 31 Jul 2025 17:24:26 +0800 Message-ID: <20250731092433.49367-4-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731092433.49367-1-dongml2@chinatelecom.cn> References: <20250731092433.49367-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Some functions is recursive for the kprobe_multi and impact the benchmark results. So just skip them. Signed-off-by: Menglong Dong --- tools/testing/selftests/bpf/trace_helpers.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/tools/testing/selftests/bpf/trace_helpers.c b/tools/testing/se= lftests/bpf/trace_helpers.c index d24baf244d1f..9da9da51b132 100644 --- a/tools/testing/selftests/bpf/trace_helpers.c +++ b/tools/testing/selftests/bpf/trace_helpers.c @@ -559,6 +559,22 @@ static bool skip_entry(char *name) if (!strncmp(name, "__ftrace_invalid_address__", sizeof("__ftrace_invalid_address__") - 1)) return true; + + if (!strcmp(name, "migrate_disable")) + return true; + if (!strcmp(name, "migrate_enable")) + return true; + if (!strcmp(name, "rcu_read_unlock_strict")) + return true; + if (!strcmp(name, "preempt_count_add")) + return true; + if (!strcmp(name, "preempt_count_sub")) + return true; + if (!strcmp(name, "__rcu_read_lock")) + return true; + if (!strcmp(name, "__rcu_read_unlock")) + return true; + return false; } =20 --=20 2.50.1 From nobody Sun Oct 5 17:56:35 2025 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2915F2BE63A; Thu, 31 Jul 2025 09:24:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753953895; cv=none; b=j/Y3ItID6aBCaCzjrgYw3Z+GeSgPm0g0EzPDAByybyxcKI9k9bH+xH1WjMr162poZwtp2YtPRgo26AMznQy61wzUDNXinycfMRbJZFOl8rP1FVBQFO/nhsEaIEmI0Puf9mxa/UEKmibT/n91++gyHTtACNLaVWq74DGi7deWvJk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753953895; c=relaxed/simple; bh=Y0sVgiMIvCN1tEPgiAVLe8svWdYQEvpuSTM/2lhhTGQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Fo+EXo0Q1dSUTowz5HYtg+QNwkwxPQxw6fmZ62CjLcuAii4Rpwq6ge2JBW9YrL1m1RHbSNZ541gDvgwdgASSqjGuik8rm2uccgDoGAU1+CkLb7B/o2YQ13RmiyGU3JiPA8J946dGfT8yc7zGge4mZEzWEy7jsaNr5w0rwf5V7hE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ig7PDkl0; arc=none smtp.client-ip=209.85.210.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ig7PDkl0" Received: by mail-pf1-f193.google.com with SMTP id d2e1a72fcca58-74b54cead6cso136443b3a.1; Thu, 31 Jul 2025 02:24:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753953893; x=1754558693; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=I4PDMe7SP8CZO+JhWjjdFtQ1vR89wVuSvH9ObtDEfac=; b=ig7PDkl0M5auHWQoARXS5sH/IQCbiUWi5/T64V94jJnBTJkigBBG0OsRepWZKKvRCi NyK34v0r10REl8ySw28Ho9RRx6Y7hjZ2wEJJxQsSqJqPu+8a766yA2H5niO4lZi+ktoC jDLhyQReHyqIGv/smugXdqDMO+vWNBBVruFGoaOk+WojHhJWxdWCvUKH8S6fmxzeLfjW ySlfFspSbToJKDyABvxL+yE4Xz9yCjriOqgkrIVyCOxwnAKOf2Xzib/mRXD88urP00PA Uoavq4hbAWx0Og8p45hpflqwgE61J5kpBFsR5y2nTlKc8XhzJtXB2SrKHGGHoN79moVL 8Iow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753953893; x=1754558693; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I4PDMe7SP8CZO+JhWjjdFtQ1vR89wVuSvH9ObtDEfac=; b=N3l3dl96YcJ+OD/66H2QalJ6uOJI6D3Hy+K+bfZrmsH3FVgEbjKJgC50ivRBCXEFJ1 LFvpLg3LaOJSGStUXKEoKiKNgLKp4U2BoBJyCHDmyZAkBnvM3Hw5b++dVtTiXkULEued 5vWi8MPFc2z3HZ6II5dQPTY0qu5M/AHXLuKCmaosRuuPrnrff81UryXIRq6f9+S9sBTv EdtTzg6xfS2X3DZYz9ELdV4Ygiz082MBmO9WV1r6j2Zi6kDs/PUwMMTP1PlTEwQzNegv yJ72wWCYBQc/lG6mE0vk5SOckrJWY2O9jTRWCCKxK5ndsIzgOGuyUTlImE0UnJ4JSEqT vYiQ== X-Forwarded-Encrypted: i=1; AJvYcCVzopYKldJTvZMv/K1fw8HnY2QeaLY9M7eh8DXkgjtSLLHuBWkje6ndQdtMVLJznx+TSfE=@vger.kernel.org, AJvYcCWza5tktBslYk/3RZVWU5ynyIcwwlwYXu8aUVqWtA2mvGfJKTOhOT2H0OQut7/s6mLJfpdYAw8z8C/GB6odDKdZqMof@vger.kernel.org, AJvYcCXXesEK/3WJp/tkSioMM2zVJaBu6ODvBxu1qHlqYjyCF2VagTMYjylgaetOgZumO3Uk85I/aBB/uUWlEe3m@vger.kernel.org X-Gm-Message-State: AOJu0Yx9ibUpOkgSlH20p6D282RYO0AgNsOxB21LjvdJqCkMC6NqwhPy oUleHqtk10wDoy0vt/JCvp2WV9e6iJCIWTtlSfNLyjP9uemoHwMIZseC X-Gm-Gg: ASbGnct1IiNZgh5SY6T/0dapFQDEhyMGkOoKZPu9ePh3z1r7b0JaEqKPtyeg1oUWTJX r/DEfgTDpGuZXfMmJJSfDpdH2KFU7kdkMfb0tbdQ7gobyteODsP5+zJheTMZ4vhGeTTOitqd4Go EKnRhF07qXi0N00cyUg52PPbqoNpCvegJP3bz6zRbgBKGXRxpDRMh0O1P9hfN5f3tY21l5ymqjd 7E+m4K2LzTEuAKcfsw4PdCCNevDg6DOZD8lKPHKOAbH9jAAmkXQXFolyvIU+aTG/EDhunwVQuso Kg+F7kYDh27QFr7mBC+yt4LY76MYdvpNE1P7/AncnECh8HNUm5HA52SChLy3sbueKkfvMhZqJRb aqPBNWgvmYWVyhuoTdZg= X-Google-Smtp-Source: AGHT+IE3kSfjGpm4vF9P6oGhvDfaGmV6e2g/GrQ5BCL1131NXnerrwWXlBf/hlhOgLJeyGeAWXcjjg== X-Received: by 2002:a05:6a20:258a:b0:20a:bde0:beb9 with SMTP id adf61e73a8af0-23dc0ea7f3emr9520807637.1.1753953893229; Thu, 31 Jul 2025 02:24:53 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76bccfbd1a7sm1108143b3a.73.2025.07.31.02.24.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Jul 2025 02:24:52 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: mhiramat@kernel.org, olsajiri@gmail.com Cc: rostedt@goodmis.org, mathieu.desnoyers@efficios.com, hca@linux.ibm.com, revest@chromium.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 4/4] selftests/bpf: add benchmark testing for kprobe-multi-all Date: Thu, 31 Jul 2025 17:24:27 +0800 Message-ID: <20250731092433.49367-5-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250731092433.49367-1-dongml2@chinatelecom.cn> References: <20250731092433.49367-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For now, the benchmark for kprobe-multi is single, which means there is only 1 function is hooked during testing. Add the testing "kprobe-multi-all", which will hook all the kernel functions during the benchmark. And the "kretprobe-multi-all" is added too. Signed-off-by: Menglong Dong --- v3: - add testcase kretprobe-multi-all - attach a empty kprobe-multi/kretprobe-multi prog to all the kernel functions except bpf_get_numa_node_id to make the bench result more accurate --- tools/testing/selftests/bpf/bench.c | 4 ++ .../selftests/bpf/benchs/bench_trigger.c | 54 +++++++++++++++++++ .../selftests/bpf/benchs/run_bench_trigger.sh | 4 +- .../selftests/bpf/progs/trigger_bench.c | 12 +++++ tools/testing/selftests/bpf/trace_helpers.c | 3 ++ 5 files changed, 75 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/bench.c b/tools/testing/selftests/= bpf/bench.c index ddd73d06a1eb..29dbf937818a 100644 --- a/tools/testing/selftests/bpf/bench.c +++ b/tools/testing/selftests/bpf/bench.c @@ -510,6 +510,8 @@ extern const struct bench bench_trig_kretprobe; extern const struct bench bench_trig_kprobe_multi; extern const struct bench bench_trig_kretprobe_multi; extern const struct bench bench_trig_fentry; +extern const struct bench bench_trig_kprobe_multi_all; +extern const struct bench bench_trig_kretprobe_multi_all; extern const struct bench bench_trig_fexit; extern const struct bench bench_trig_fmodret; extern const struct bench bench_trig_tp; @@ -578,6 +580,8 @@ static const struct bench *benchs[] =3D { &bench_trig_kprobe_multi, &bench_trig_kretprobe_multi, &bench_trig_fentry, + &bench_trig_kprobe_multi_all, + &bench_trig_kretprobe_multi_all, &bench_trig_fexit, &bench_trig_fmodret, &bench_trig_tp, diff --git a/tools/testing/selftests/bpf/benchs/bench_trigger.c b/tools/tes= ting/selftests/bpf/benchs/bench_trigger.c index 82327657846e..c6634a64a7c0 100644 --- a/tools/testing/selftests/bpf/benchs/bench_trigger.c +++ b/tools/testing/selftests/bpf/benchs/bench_trigger.c @@ -226,6 +226,58 @@ static void trigger_fentry_setup(void) attach_bpf(ctx.skel->progs.bench_trigger_fentry); } =20 +static void attach_ksyms_all(struct bpf_program *empty, bool kretprobe) +{ + LIBBPF_OPTS(bpf_kprobe_multi_opts, opts); + char **syms =3D NULL; + size_t cnt =3D 0; + + if (bpf_get_ksyms(&syms, &cnt, true)) { + printf("failed to get ksyms\n"); + exit(1); + } + + printf("found %zu ksyms\n", cnt); + opts.syms =3D (const char **) syms; + opts.cnt =3D cnt; + opts.retprobe =3D kretprobe; + /* attach empty to all the kernel functions except bpf_get_numa_node_id. = */ + if (!bpf_program__attach_kprobe_multi_opts(empty, NULL, &opts)) { + printf("failed to attach bpf_program__attach_kprobe_multi_opts to all\n"= ); + exit(1); + } +} + +static void trigger_kprobe_multi_all_setup(void) +{ + struct bpf_program *prog, *empty; + + setup_ctx(); + empty =3D ctx.skel->progs.bench_kprobe_multi_empty; + prog =3D ctx.skel->progs.bench_trigger_kprobe_multi; + bpf_program__set_autoload(empty, true); + bpf_program__set_autoload(prog, true); + load_ctx(); + + attach_ksyms_all(empty, false); + attach_bpf(prog); +} + +static void trigger_kretprobe_multi_all_setup(void) +{ + struct bpf_program *prog, *empty; + + setup_ctx(); + empty =3D ctx.skel->progs.bench_kretprobe_multi_empty; + prog =3D ctx.skel->progs.bench_trigger_kretprobe_multi; + bpf_program__set_autoload(empty, true); + bpf_program__set_autoload(prog, true); + load_ctx(); + + attach_ksyms_all(empty, true); + attach_bpf(prog); +} + static void trigger_fexit_setup(void) { setup_ctx(); @@ -512,6 +564,8 @@ BENCH_TRIG_KERNEL(kretprobe, "kretprobe"); BENCH_TRIG_KERNEL(kprobe_multi, "kprobe-multi"); BENCH_TRIG_KERNEL(kretprobe_multi, "kretprobe-multi"); BENCH_TRIG_KERNEL(fentry, "fentry"); +BENCH_TRIG_KERNEL(kprobe_multi_all, "kprobe-multi-all"); +BENCH_TRIG_KERNEL(kretprobe_multi_all, "kretprobe-multi-all"); BENCH_TRIG_KERNEL(fexit, "fexit"); BENCH_TRIG_KERNEL(fmodret, "fmodret"); BENCH_TRIG_KERNEL(tp, "tp"); diff --git a/tools/testing/selftests/bpf/benchs/run_bench_trigger.sh b/tool= s/testing/selftests/bpf/benchs/run_bench_trigger.sh index a690f5a68b6b..f7573708a0c3 100755 --- a/tools/testing/selftests/bpf/benchs/run_bench_trigger.sh +++ b/tools/testing/selftests/bpf/benchs/run_bench_trigger.sh @@ -6,8 +6,8 @@ def_tests=3D( \ usermode-count kernel-count syscall-count \ fentry fexit fmodret \ rawtp tp \ - kprobe kprobe-multi \ - kretprobe kretprobe-multi \ + kprobe kprobe-multi kprobe-multi-all \ + kretprobe kretprobe-multi kretprobe-multi-all \ ) =20 tests=3D("$@") diff --git a/tools/testing/selftests/bpf/progs/trigger_bench.c b/tools/test= ing/selftests/bpf/progs/trigger_bench.c index 044a6d78923e..3d5f30c29ae3 100644 --- a/tools/testing/selftests/bpf/progs/trigger_bench.c +++ b/tools/testing/selftests/bpf/progs/trigger_bench.c @@ -97,6 +97,12 @@ int bench_trigger_kprobe_multi(void *ctx) return 0; } =20 +SEC("?kprobe.multi/bpf_get_numa_node_id") +int bench_kprobe_multi_empty(void *ctx) +{ + return 0; +} + SEC("?kretprobe.multi/bpf_get_numa_node_id") int bench_trigger_kretprobe_multi(void *ctx) { @@ -104,6 +110,12 @@ int bench_trigger_kretprobe_multi(void *ctx) return 0; } =20 +SEC("?kretprobe.multi/bpf_get_numa_node_id") +int bench_kretprobe_multi_empty(void *ctx) +{ + return 0; +} + SEC("?fentry/bpf_get_numa_node_id") int bench_trigger_fentry(void *ctx) { diff --git a/tools/testing/selftests/bpf/trace_helpers.c b/tools/testing/se= lftests/bpf/trace_helpers.c index 9da9da51b132..78cf1aab09d8 100644 --- a/tools/testing/selftests/bpf/trace_helpers.c +++ b/tools/testing/selftests/bpf/trace_helpers.c @@ -575,6 +575,9 @@ static bool skip_entry(char *name) if (!strcmp(name, "__rcu_read_unlock")) return true; =20 + if (!strcmp(name, "bpf_get_numa_node_id")) + return true; + return false; } =20 --=20 2.50.1