From nobody Mon Feb 9 19:05:10 2026 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D0503DBA8 for ; Mon, 11 Mar 2024 09:35:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710149733; cv=none; b=pPDJXsjAMWMxZxSaJGfqqzYTZ+vf52UHOT6dITIO6nkYVNJDi/m5Tdt6F1EWberWpmNME+i74lCN7q6TaxOOGqKIX3jq+zphk4fJu+2w6dQfImuErXw0GgMIDgTnVtq/qIT564VVimute7bVkhU0sVWXqXezZAcYmmIR/m669To= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710149733; c=relaxed/simple; bh=ibzml+PzSFzKH42zzI5+IahEHBtbwOKK2hMI0h78KBM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pB0CtKt0NEnz1KtDxdeZAUHs5fqU5XYInGB4lsUKoli/McN5iw0YtRP7MvVe3Wq73aTWpDLRGHkMiiJzmeipqXc6eOXjIz84UXiIDOu3TsL380ASxj+UK8qdUg6wD5xvJBTkPVqGWiWYAN9KPyBo9HKWV5aRHPl1jUFyOEKTRGk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=IJw1BOVE; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="IJw1BOVE" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-1dd10a37d68so34587595ad.2 for ; Mon, 11 Mar 2024 02:35:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1710149731; x=1710754531; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JFf08lEs3m84/dbvV4ZlLUOMjMFpJvm5xegIfbyJ9HU=; b=IJw1BOVEKlp2EdC5OdG9+ILZTcu8cTvJvjCj56vkSBEDC3OZrC7HF3TOVTr+vFIiq4 FSIvwTwJlo1In9aEsOEp+lPbCVGRKr78CMgSnTgggi46BE3Lhj5i5GtVpgGjfsADGSnO Kf4dR0pU6Bf8V8TXsn+cyi5IE3GMaJ8C2COzMosFZSuXJPDhkiGJXPznCFJRvIeTa1cL CngqTNIQRxBl/7yQZ3xFFqBH95DBKK0oaklwawrBmD5gyjh6i798cJofSmslddv0URsh fw1mGBkYfMIR4nBibl2QlMk0dAY8QvpgvIYpQObYYRM5fnI4GW2WbaG8IZufYBxTjIeT vf0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710149731; x=1710754531; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JFf08lEs3m84/dbvV4ZlLUOMjMFpJvm5xegIfbyJ9HU=; b=fba8oh/9WUkRTfgDStrD+UoTpn0zVFsMICUhEzXXcNoUMcoZjp8uzJDNu4XHVEg12I cptXu7bARUkWnv9rRr754rzs07a5tQvWuFd0bYKI/5PsvLBCG9OFgV0tJTrzxe++Izop yULT6DNrow7yG/TH3z13JD+0jUOMQa1KdJYTAz9yJWFJVNXX6vrX9HRnqaLPpjaHWMYe DZGwgzX+9CUy+DH70nZzVCEeXltsdLt2euvhcCo9BVL7jJon5qLk+Wem34FkB/02EIJJ Kssadt7RIaOCoXYqP68pepBqmk05fpUmjKwfZIXGBy/x6r5qBHiQFLulhtP7R4sI3wTy ioHQ== X-Forwarded-Encrypted: i=1; AJvYcCVSIXEosC+PSQQNBw5zvGcxFjshDhcF7EjxQjJlyYSqKVW0IUwmqCAlx7nO+sQTHkWNZFtbsMMzvGSQd04DsExwzPtQ+UJkUxKUcQMa X-Gm-Message-State: AOJu0YxDAwfWqCy5PuE/RBYcGJyjs0OvWaZXQKHULerh9vn/DDWhjMui Gl0N/6AUFcyMSnWLBDP9WsL+MsKZjazMVffEdlMg3Wjmf1YFNTC50y2uzicT7X8= X-Google-Smtp-Source: AGHT+IFQIVN/SFlbgP5cEEuZ7moW9KP5l9jB6qy51la06bqNfTxjZ7YG5WDDYrE2U6yDVhQ3L+9Yxw== X-Received: by 2002:a17:902:988c:b0:1dd:772e:d504 with SMTP id s12-20020a170902988c00b001dd772ed504mr4091891plp.66.1710149729346; Mon, 11 Mar 2024 02:35:29 -0700 (PDT) Received: from localhost.localdomain ([43.129.25.208]) by smtp.gmail.com with ESMTPSA id h9-20020a170902f7c900b001dcad9cbf8bsm4253365plw.239.2024.03.11.02.35.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Mar 2024 02:35:29 -0700 (PDT) From: Menglong Dong To: andrii@kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, davem@davemloft.net, dsahern@kernel.org, dave.hansen@linux.intel.com, x86@kernel.org, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, quentin@isovalent.com, bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com, Menglong Dong Subject: [PATCH bpf-next v2 8/9] libbpf: add support for the multi-link of tracing Date: Mon, 11 Mar 2024 17:35:25 +0800 Message-Id: <20240311093526.1010158-9-dongmenglong.8@bytedance.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240311093526.1010158-1-dongmenglong.8@bytedance.com> References: <20240311093526.1010158-1-dongmenglong.8@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add support for the attach types of: BPF_TRACE_FENTRY_MULTI BPF_TRACE_FEXIT_MULTI BPF_MODIFY_RETURN_MULTI Signed-off-by: Menglong Dong --- tools/bpf/bpftool/common.c | 3 + tools/lib/bpf/bpf.c | 10 +++ tools/lib/bpf/bpf.h | 6 ++ tools/lib/bpf/libbpf.c | 168 ++++++++++++++++++++++++++++++++++++- tools/lib/bpf/libbpf.h | 14 ++++ tools/lib/bpf/libbpf.map | 1 + 6 files changed, 199 insertions(+), 3 deletions(-) diff --git a/tools/bpf/bpftool/common.c b/tools/bpf/bpftool/common.c index cc6e6aae2447..ffc85256671d 100644 --- a/tools/bpf/bpftool/common.c +++ b/tools/bpf/bpftool/common.c @@ -1089,6 +1089,9 @@ const char *bpf_attach_type_input_str(enum bpf_attach= _type t) case BPF_TRACE_FENTRY: return "fentry"; case BPF_TRACE_FEXIT: return "fexit"; case BPF_MODIFY_RETURN: return "mod_ret"; + case BPF_TRACE_FENTRY_MULTI: return "fentry_multi"; + case BPF_TRACE_FEXIT_MULTI: return "fexit_multi"; + case BPF_MODIFY_RETURN_MULTI: return "mod_ret_multi"; case BPF_SK_REUSEPORT_SELECT: return "sk_skb_reuseport_select"; case BPF_SK_REUSEPORT_SELECT_OR_MIGRATE: return "sk_skb_reuseport_select_= or_migrate"; default: return libbpf_bpf_attach_type_str(t); diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 97ec005c3c47..63d4734dbae4 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -793,6 +793,16 @@ int bpf_link_create(int prog_fd, int target_fd, if (!OPTS_ZEROED(opts, tracing)) return libbpf_err(-EINVAL); break; + case BPF_TRACE_FENTRY_MULTI: + case BPF_TRACE_FEXIT_MULTI: + case BPF_MODIFY_RETURN_MULTI: + attr.link_create.tracing_multi.btf_ids =3D ptr_to_u64(OPTS_GET(opts, tra= cing_multi.btf_ids, 0)); + attr.link_create.tracing_multi.tgt_fds =3D ptr_to_u64(OPTS_GET(opts, tra= cing_multi.tgt_fds, 0)); + attr.link_create.tracing_multi.cookies =3D ptr_to_u64(OPTS_GET(opts, tra= cing_multi.cookies, 0)); + attr.link_create.tracing_multi.cnt =3D OPTS_GET(opts, tracing_multi.cnt,= 0); + if (!OPTS_ZEROED(opts, tracing_multi)) + return libbpf_err(-EINVAL); + break; case BPF_NETFILTER: attr.link_create.netfilter.pf =3D OPTS_GET(opts, netfilter.pf, 0); attr.link_create.netfilter.hooknum =3D OPTS_GET(opts, netfilter.hooknum,= 0); diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index df0db2f0cdb7..e28c88d6cfa4 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -419,6 +419,12 @@ struct bpf_link_create_opts { struct { __u64 cookie; } tracing; + struct { + __u32 cnt; + const __u32 *btf_ids; + const __u32 *tgt_fds; + const __u64 *cookies; + } tracing_multi; struct { __u32 pf; __u32 hooknum; diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index fd5428494a7e..821214774941 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -132,6 +132,9 @@ static const char * const attach_type_name[] =3D { [BPF_TRACE_UPROBE_MULTI] =3D "trace_uprobe_multi", [BPF_NETKIT_PRIMARY] =3D "netkit_primary", [BPF_NETKIT_PEER] =3D "netkit_peer", + [BPF_TRACE_FENTRY_MULTI] =3D "trace_fentry_multi", + [BPF_TRACE_FEXIT_MULTI] =3D "trace_fexit_multi", + [BPF_MODIFY_RETURN_MULTI] =3D "modify_return_multi", }; =20 static const char * const link_type_name[] =3D { @@ -381,6 +384,8 @@ enum sec_def_flags { SEC_XDP_FRAGS =3D 16, /* Setup proper attach type for usdt probes. */ SEC_USDT =3D 32, + /* attachment target is multi-link */ + SEC_ATTACH_BTF_MULTI =3D 64, }; =20 struct bpf_sec_def { @@ -7160,9 +7165,9 @@ static int libbpf_prepare_prog_load(struct bpf_progra= m *prog, if ((def & SEC_USDT) && kernel_supports(prog->obj, FEAT_UPROBE_MULTI_LINK= )) prog->expected_attach_type =3D BPF_TRACE_UPROBE_MULTI; =20 - if ((def & SEC_ATTACH_BTF) && !prog->attach_btf_id) { + if ((def & (SEC_ATTACH_BTF | SEC_ATTACH_BTF_MULTI)) && !prog->attach_btf_= id) { int btf_obj_fd =3D 0, btf_type_id =3D 0, err; - const char *attach_name; + const char *attach_name, *name_end; =20 attach_name =3D strchr(prog->sec_name, '/'); if (!attach_name) { @@ -7181,7 +7186,27 @@ static int libbpf_prepare_prog_load(struct bpf_progr= am *prog, } attach_name++; /* skip over / */ =20 - err =3D libbpf_find_attach_btf_id(prog, attach_name, &btf_obj_fd, &btf_t= ype_id); + name_end =3D strchr(attach_name, ','); + /* for multi-link tracing, use the first target symbol during + * loading. + */ + if ((def & SEC_ATTACH_BTF_MULTI) && name_end) { + int len =3D name_end - attach_name + 1; + char *first_tgt; + + first_tgt =3D malloc(len); + if (!first_tgt) + return -ENOMEM; + strncpy(first_tgt, attach_name, len); + first_tgt[len - 1] =3D '\0'; + err =3D libbpf_find_attach_btf_id(prog, first_tgt, &btf_obj_fd, + &btf_type_id); + free(first_tgt); + } else { + err =3D libbpf_find_attach_btf_id(prog, attach_name, &btf_obj_fd, + &btf_type_id); + } + if (err) return err; =20 @@ -9149,6 +9174,7 @@ static int attach_kprobe_multi(const struct bpf_progr= am *prog, long cookie, stru static int attach_uprobe_multi(const struct bpf_program *prog, long cookie= , struct bpf_link **link); static int attach_lsm(const struct bpf_program *prog, long cookie, struct = bpf_link **link); static int attach_iter(const struct bpf_program *prog, long cookie, struct= bpf_link **link); +static int attach_trace_multi(const struct bpf_program *prog, long cookie,= struct bpf_link **link); =20 static const struct bpf_sec_def section_defs[] =3D { SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE), @@ -9192,6 +9218,13 @@ static const struct bpf_sec_def section_defs[] =3D { SEC_DEF("fentry.s+", TRACING, BPF_TRACE_FENTRY, SEC_ATTACH_BTF | SEC_SLE= EPABLE, attach_trace), SEC_DEF("fmod_ret.s+", TRACING, BPF_MODIFY_RETURN, SEC_ATTACH_BTF | SEC_= SLEEPABLE, attach_trace), SEC_DEF("fexit.s+", TRACING, BPF_TRACE_FEXIT, SEC_ATTACH_BTF | SEC_SLEEP= ABLE, attach_trace), + SEC_DEF("tp_btf+", TRACING, BPF_TRACE_RAW_TP, SEC_ATTACH_BTF, attach_tra= ce), + SEC_DEF("fentry.multi+", TRACING, BPF_TRACE_FENTRY_MULTI, SEC_ATTACH_BTF_= MULTI, attach_trace_multi), + SEC_DEF("fmod_ret.multi+", TRACING, BPF_MODIFY_RETURN_MULTI, SEC_ATTACH_B= TF_MULTI, attach_trace_multi), + SEC_DEF("fexit.multi+", TRACING, BPF_TRACE_FEXIT_MULTI, SEC_ATTACH_BTF_M= ULTI, attach_trace_multi), + SEC_DEF("fentry.multi.s+", TRACING, BPF_TRACE_FENTRY_MULTI, SEC_ATTACH_BT= F_MULTI | SEC_SLEEPABLE, attach_trace_multi), + SEC_DEF("fmod_ret.multi.s+", TRACING, BPF_MODIFY_RETURN_MULTI, SEC_ATTACH= _BTF_MULTI | SEC_SLEEPABLE, attach_trace_multi), + SEC_DEF("fexit.multi.s+", TRACING, BPF_TRACE_FEXIT_MULTI, SEC_ATTACH_BTF_= MULTI | SEC_SLEEPABLE, attach_trace_multi), SEC_DEF("freplace+", EXT, 0, SEC_ATTACH_BTF, attach_trace), SEC_DEF("lsm+", LSM, BPF_LSM_MAC, SEC_ATTACH_BTF, attach_lsm), SEC_DEF("lsm.s+", LSM, BPF_LSM_MAC, SEC_ATTACH_BTF | SEC_SLEEPABLE, atta= ch_lsm), @@ -12300,6 +12333,135 @@ static int attach_trace(const struct bpf_program = *prog, long cookie, struct bpf_ return libbpf_get_error(*link); } =20 +struct bpf_link *bpf_program__attach_trace_multi_opts(const struct bpf_pro= gram *prog, + const struct bpf_trace_multi_opts *opts) +{ + LIBBPF_OPTS(bpf_link_create_opts, link_opts); + __u32 *btf_ids =3D NULL, *tgt_fds =3D NULL; + struct bpf_link *link =3D NULL; + char errmsg[STRERR_BUFSIZE]; + int prog_fd, pfd, cnt, err; + + if (!OPTS_VALID(opts, bpf_trace_multi_opts)) + return libbpf_err_ptr(-EINVAL); + + prog_fd =3D bpf_program__fd(prog); + if (prog_fd < 0) { + pr_warn("prog '%s': can't attach before loaded\n", prog->name); + return libbpf_err_ptr(-EINVAL); + } + + cnt =3D OPTS_GET(opts, cnt, 0); + if (opts->syms) { + int btf_obj_fd, btf_type_id, i; + + if (opts->btf_ids || opts->tgt_fds) { + pr_warn("can set both opts->syms and opts->btf_ids\n"); + return libbpf_err_ptr(-EINVAL); + } + + btf_ids =3D malloc(sizeof(*btf_ids) * cnt); + tgt_fds =3D malloc(sizeof(*tgt_fds) * cnt); + if (!btf_ids || !tgt_fds) { + err =3D -ENOMEM; + goto err_free; + } + for (i =3D 0; i < cnt; i++) { + btf_obj_fd =3D btf_type_id =3D 0; + + err =3D find_kernel_btf_id(prog->obj, opts->syms[i], + prog->expected_attach_type, &btf_obj_fd, + &btf_type_id); + if (err) + goto err_free; + btf_ids[i] =3D btf_type_id; + tgt_fds[i] =3D btf_obj_fd; + } + link_opts.tracing_multi.btf_ids =3D btf_ids; + link_opts.tracing_multi.tgt_fds =3D tgt_fds; + } else { + link_opts.tracing_multi.btf_ids =3D OPTS_GET(opts, btf_ids, 0); + link_opts.tracing_multi.tgt_fds =3D OPTS_GET(opts, tgt_fds, 0); + } + + link =3D calloc(1, sizeof(*link)); + if (!link) { + err =3D -ENOMEM; + goto err_free; + } + link->detach =3D &bpf_link__detach_fd; + + link_opts.tracing_multi.cookies =3D OPTS_GET(opts, cookies, 0); + link_opts.tracing_multi.cnt =3D cnt; + + pfd =3D bpf_link_create(prog_fd, 0, bpf_program__expected_attach_type(pro= g), &link_opts); + if (pfd < 0) { + err =3D -errno; + pr_warn("prog '%s': failed to attach: %s\n", + prog->name, libbpf_strerror_r(pfd, errmsg, sizeof(errmsg))); + goto err_free; + } + link->fd =3D pfd; + + free(btf_ids); + free(tgt_fds); + return link; +err_free: + free(btf_ids); + free(tgt_fds); + free(link); + return libbpf_err_ptr(err); +} + +static int attach_trace_multi(const struct bpf_program *prog, long cookie,= struct bpf_link **link) +{ + LIBBPF_OPTS(bpf_trace_multi_opts, opts); + int i, err, len, cnt =3D 1; + char **syms, *buf, *name; + const char *spec; + + spec =3D strchr(prog->sec_name, '/'); + if (!spec || !*(++spec)) + return -EINVAL; + + len =3D strlen(spec); + buf =3D malloc(len + 1); + if (!buf) + return -ENOMEM; + + strcpy(buf, spec); + for (i =3D 0; i < len; i++) { + if (buf[i] =3D=3D ',') + cnt++; + } + + syms =3D malloc(sizeof(*syms) * cnt); + if (!syms) { + err =3D -ENOMEM; + goto out_free; + } + + opts.syms =3D (const char **)syms; + opts.cnt =3D cnt; + name =3D buf; + err =3D -EINVAL; + while (name) { + if (*name =3D=3D '\0') + goto out_free; + *(syms++) =3D name; + name =3D strchr(name, ','); + if (name) + *(name++) =3D '\0'; + } + + *link =3D bpf_program__attach_trace_multi_opts(prog, &opts); + err =3D libbpf_get_error(*link); +out_free: + free(buf); + free(opts.syms); + return err; +} + static int attach_lsm(const struct bpf_program *prog, long cookie, struct = bpf_link **link) { *link =3D bpf_program__attach_lsm(prog); diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index c41a909ea4c1..9bca44d5adfa 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -790,6 +790,20 @@ bpf_program__attach_xdp(const struct bpf_program *prog= , int ifindex); LIBBPF_API struct bpf_link * bpf_program__attach_freplace(const struct bpf_program *prog, int target_fd, const char *attach_func_name); +struct bpf_trace_multi_opts { + /* size of this struct, for forward/backward compatibility */ + size_t sz; + const char **syms; + __u32 *btf_ids; + __u32 *tgt_fds; + __u64 *cookies; + size_t cnt; +}; +#define bpf_trace_multi_opts__last_field cnt + +LIBBPF_API struct bpf_link * +bpf_program__attach_trace_multi_opts(const struct bpf_program *prog, + const struct bpf_trace_multi_opts *opts); =20 struct bpf_netfilter_opts { /* size of this struct, for forward/backward compatibility */ diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index 57642b78917f..94933898df44 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -414,4 +414,5 @@ LIBBPF_1.4.0 { btf__new_split; btf_ext__raw_data; bpf_object__free_btfs; + bpf_program__attach_trace_multi_opts; } LIBBPF_1.3.0; --=20 2.39.2