From nobody Tue Sep 29 01:59:56 2026 Received: from mta0.migadu.com (out-111.mta0.migadu.com [91.218.175.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 919BE489868 for ; Thu, 13 Aug 2026 15:23:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.111 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634631; cv=none; b=n0fcNMDx8FHuIhBVV3NmvxCBSZqHm8TktC7O5DbZkH1XquKhqQX1CHVAeWdwse6VYpUO4UNwDvKYLXJ2BQZF/WSy1VlHbnSh7aT1UhWBEpdc6WXC/otE8boLxzsR4PnZvqaxN4yX0jSdemWC8ADX6lMsmEwNPYGVTjG8ebybdDU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634631; c=relaxed/simple; bh=KChodiFWtYmnh7jV1ssWT8mlezg6RNZ+DZqpuqnYLno=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y8/LLsIwcIq7j+bXbQisHrHfJt37hG+tvW+3QX8Wrfn3w6UuMp8FL7kxadGgdQn3/nbGKx2TiEZjWkDv8St6cQ3D7355gmLHqHyd2Eh/tGtEdgDroVadG0d2FFIDYQe4XGq7VwCZ7Tmo74CEydAgPWfCJ7ieAexCV+i0CUtdngw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=blAh1gie; arc=none smtp.client-ip=91.218.175.111 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="blAh1gie" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=KChodiFWtYmnh7jV1ssWT8mlezg6RNZ+DZqpuqnYLno=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634626; v=1; x=1787239426; b=blAh1gie5UD7vfpxelVoNrpDD1Fsdm3hyku1niEglbJD2r6Bq/sJs9FeIP/DCdTWeQfh7IXM Jh+DwVTNruRjnCfTJ2PLywTTjwNK4N7XTivv0jRL7GwJc4aOShx9g4sNYW74bzHY/u0z7wHF9sM cAaV3gUja5AXBgzm+aqUsZOg= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id cbfd9e92d5adc6de; Thu, 13 Aug 2026 15:23:46 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com, Emil Tsalapatis Subject: [PATCH bpf-next v12 01/10] bpf: Drop duplicate blank lines in kernel/bpf/ Date: Thu, 13 Aug 2026 23:23:14 +0800 Message-ID: <20260813152324.97937-2-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" There are many adjacent blank lines in kernel/bpf/ that have accumulated over time. Drop them for cleanup. No functional changes intended. Reviewed-by: Emil Tsalapatis Signed-off-by: Leon Hwang --- kernel/bpf/backtrack.c | 2 -- kernel/bpf/btf.c | 1 - kernel/bpf/cfg.c | 1 - kernel/bpf/fixups.c | 1 - kernel/bpf/hashtab.c | 2 -- kernel/bpf/helpers.c | 1 - kernel/bpf/liveness.c | 2 -- kernel/bpf/queue_stack_maps.c | 1 - kernel/bpf/syscall.c | 5 ----- kernel/bpf/verifier.c | 14 -------------- 10 files changed, 30 deletions(-) diff --git a/kernel/bpf/backtrack.c b/kernel/bpf/backtrack.c index 40bd04421a99..a2b18a9f1694 100644 --- a/kernel/bpf/backtrack.c +++ b/kernel/bpf/backtrack.c @@ -214,7 +214,6 @@ static inline bool bt_is_reg_set(struct backtrack_state= *bt, u32 reg) return bt->reg_masks[bt->frame] & (1 << reg); } =20 - /* format registers bitmask, e.g., "r0,r2,r4" for 0x15 mask */ static void fmt_reg_mask(char *buf, ssize_t buf_sz, u32 reg_mask) { @@ -254,7 +253,6 @@ void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 = stack_mask) } } =20 - /* For given verifier state backtrack_insn() is called from the last insn = to * the first insn. Its purpose is to compute a bitmask of registers and * stack slots that needs precision in the parent verifier state. diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 6606187ed4f4..87ffde865a50 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -2534,7 +2534,6 @@ static void btf_bitfield_show(void *data, u8 bits_off= set, btf_int128_print(show, print_num); } =20 - static void btf_int_bits_show(const struct btf *btf, const struct btf_type *t, void *data, u8 bits_offset, diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c index db3416a7c904..818f7afac83a 100644 --- a/kernel/bpf/cfg.c +++ b/kernel/bpf/cfg.c @@ -47,7 +47,6 @@ enum { BRANCH =3D 2, }; =20 - static void mark_subprog_changes_pkt_data(struct bpf_verifier_env *env, in= t off) { struct bpf_subprog_info *subprog; diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c index 2417a3461652..0caf1bbd9494 100644 --- a/kernel/bpf/fixups.c +++ b/kernel/bpf/fixups.c @@ -1466,7 +1466,6 @@ int bpf_fixup_call_args(struct bpf_verifier_env *env) return err; } =20 - /* The function requires that first instruction in 'patch' is insnsi[prog-= >len - 1] */ static int add_hidden_subprog(struct bpf_verifier_env *env, struct bpf_ins= n *patch, int len) { diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 9f394e1aa2e8..d40cb5dd446c 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -998,7 +998,6 @@ static void dec_elem_count(struct bpf_htab *htab) atomic_dec(&htab->count); } =20 - static void free_htab_elem(struct bpf_htab *htab, struct htab_elem *l) { htab_put_fd_value(htab, l); @@ -2970,7 +2969,6 @@ static int rhtab_delete_elem(struct bpf_rhtab *rhtab,= struct rhtab_elem *elem, v return 0; } =20 - static long rhtab_map_delete_elem(struct bpf_map *map, void *key) { struct bpf_rhtab *rhtab =3D container_of(map, struct bpf_rhtab, map); diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 6388b6b23e49..45e2f19387b2 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -4871,7 +4871,6 @@ static const struct btf_kfunc_id_set generic_kfunc_se= t =3D { .set =3D &generic_btf_ids, }; =20 - BTF_ID_LIST(generic_dtor_ids) BTF_ID(struct, task_struct) BTF_ID(func, bpf_task_release_dtor) diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c index 1c997aeba6fa..74fc4b3f80d6 100644 --- a/kernel/bpf/liveness.c +++ b/kernel/bpf/liveness.c @@ -269,7 +269,6 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 i= dx) =20 __diag_pop(); =20 - static inline bool update_insn(struct bpf_verifier_env *env, struct func_instance *instance, u32 frame, u32 insn_idx) { @@ -1862,7 +1861,6 @@ static int analyze_subprog(struct bpf_verifier_env *e= nv, if (need_resched()) cond_resched(); =20 - /* * When an instance is reused (must_write_initialized =3D=3D true), * record into a fresh instance and merge afterward. This avoids diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index c1c9dee4dcdd..6e8b18c32a10 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -123,7 +123,6 @@ static long __queue_map_get(struct bpf_map *map, void *= value, bool delete) return err; } =20 - static long __stack_map_get(struct bpf_map *map, void *value, bool delete) { struct bpf_queue_stack *qs =3D bpf_queue_stack(map); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 8d111da88655..7d8c3e8e6d62 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -636,7 +636,6 @@ int bpf_map_alloc_pages(const struct bpf_map *map, int = nid, return ret; } =20 - static int btf_field_cmp(const void *a, const void *b) { const struct btf_field *f1 =3D a, *f2 =3D b; @@ -1830,7 +1829,6 @@ static int map_lookup_elem(union bpf_attr *attr) return err; } =20 - #define BPF_MAP_UPDATE_ELEM_LAST_FIELD flags =20 static int map_update_elem(union bpf_attr *attr, bpfptr_t uattr) @@ -3497,7 +3495,6 @@ int bpf_link_prime(struct bpf_link *link, struct bpf_= link_primer *primer) if (fd < 0) return fd; =20 - id =3D bpf_link_alloc_id(link); if (id < 0) { put_unused_fd(fd); @@ -5505,7 +5502,6 @@ static int bpf_link_get_info_by_fd(struct file *file, return 0; } =20 - static int token_get_info_by_fd(struct file *file, struct bpf_token *token, const union bpf_attr *attr, @@ -6507,7 +6503,6 @@ BPF_CALL_3(bpf_sys_bpf, int, cmd, union bpf_attr *, a= ttr, u32, attr_size) return __sys_bpf(cmd, KERNEL_BPFPTR(attr), attr_size, KERNEL_BPFPTR(NULL)= , 0); } =20 - /* To shut up -Wmissing-prototypes. * This function is used by the kernel light skeleton * to load bpf programs when modules are loaded or during kernel boot. diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 164d16c243ca..cdb79a66b156 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -635,7 +635,6 @@ static void __mark_dynptr_reg(struct bpf_reg_state *reg, enum bpf_dynptr_type type, bool first_slot, int id, int parent_id); =20 - static void mark_dynptr_stack_regs(struct bpf_verifier_env *env, struct bpf_reg_state *sreg1, struct bpf_reg_state *sreg2, @@ -1674,7 +1673,6 @@ static bool same_callsites(struct bpf_verifier_state = *a, struct bpf_verifier_sta return true; } =20 - void bpf_free_backedges(struct bpf_scc_visit *visit) { struct bpf_scc_backedge *backedge, *next; @@ -2291,7 +2289,6 @@ static struct bpf_verifier_state *push_async_cb(struc= t bpf_verifier_env *env, return &elem->st; } =20 - static int cmp_subprogs(const void *a, const void *b) { return ((struct bpf_subprog_info *)a)->start - @@ -3969,7 +3966,6 @@ static int check_stack_read(struct bpf_verifier_env *= env, return err; } =20 - /* check_stack_write dispatches to check_stack_write_fixed_off or * check_stack_write_var_off. * @@ -4767,7 +4763,6 @@ static int check_sock_access(struct bpf_verifier_env = *env, int insn_idx, valid =3D false; } =20 - if (valid) { env->insn_aux_data[insn_idx].ctx_field_size =3D info.ctx_field_size; @@ -6635,7 +6630,6 @@ static int check_stack_range_initialized( if (err) return err; =20 - if (tnum_is_const(reg->var_off)) { min_off =3D max_off =3D reg->var_off.value + off; } else { @@ -7347,7 +7341,6 @@ static bool is_iter_new_kfunc(struct bpf_call_arg_met= a *meta) return meta->kfunc_flags & KF_ITER_NEW; } =20 - static bool is_iter_destroy_kfunc(struct bpf_call_arg_meta *meta) { return meta->kfunc_flags & KF_ITER_DESTROY; @@ -11607,7 +11600,6 @@ static int process_irq_flag(struct bpf_verifier_env= *env, struct bpf_reg_state * return 0; } =20 - static int ref_set_non_owning(struct bpf_verifier_env *env, struct bpf_reg= _state *reg) { struct btf_record *rec =3D reg_btf_record(reg); @@ -16412,7 +16404,6 @@ static int check_ld_abs(struct bpf_verifier_env *en= v, struct bpf_insn *insn) return 0; } =20 - static bool return_retval_range(struct bpf_verifier_env *env, struct bpf_r= etval_range *range) { enum bpf_prog_type prog_type =3D resolve_prog_type(env->prog); @@ -18361,8 +18352,6 @@ static void release_insn_arrays(struct bpf_verifier= _env *env) bpf_insn_array_release(env->insn_array_maps[i]); } =20 - - /* The verifier does more data flow analysis than llvm and will not * explore branches that are dead at run time. Malicious programs can * have dead code too. Therefore replace all dead at-run-time code @@ -18390,8 +18379,6 @@ static void sanitize_dead_code(struct bpf_verifier_= env *env) } } =20 - - static void free_states(struct bpf_verifier_env *env) { struct bpf_verifier_state_list *sl; @@ -18678,7 +18665,6 @@ static int do_check_main(struct bpf_verifier_env *e= nv) return ret; } =20 - static void print_verification_stats(struct bpf_verifier_env *env) { /* Skip over hidden subprogs which are not verified. */ --=20 2.55.0 From nobody Tue Sep 29 01:59:56 2026 Received: from mta1.migadu.com (out-211.mta1.migadu.com [95.215.58.211]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90E4847A0B5 for ; Thu, 13 Aug 2026 15:23:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.211 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634636; cv=none; b=U/scaJUGlRwXtOApzHRcpqrRbDtZZe8V7vva/Vx9ANJAfCc88zlnkZZb64Fuw3zoEsYy3A7LHzriuT1nwgX/caHR5mQWcrn7zZiKa7NJQSby1BWYx1w/g/c/1172Ct7cFJHb80fQwJ2esTX1A+HF+h3lTv87f2+uX2cZKPyddEo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634636; c=relaxed/simple; bh=rl2mM/hgJaTMp2JXut3FiR4Nw5ZzCc31Waf+AGGLHso=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GMCu7fc7d0CAI9YlhX4OqT2JL6U1pUmL8bgPhW+ivrOc1JiKBI1yW+KBVtDo6XUkUtQIUSIO/fig0rPPBrKvjvZVyS4Hq2aPwg5PvDhwReddp0e6Us/y1p2h0AygEsonL+0RAr0YwwB7+GZ/EVz5t85wVKQekkUxoW4+oBgKSkE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=QT92BiqO; arc=none smtp.client-ip=95.215.58.211 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="QT92BiqO" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=rl2mM/hgJaTMp2JXut3FiR4Nw5ZzCc31Waf+AGGLHso=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634632; v=1; x=1787239432; b=QT92BiqOI82blshjS0lZV5TdMwXI2BNK7tn1U8l3h3QoGLkKvuz94aWM77MUde3oMSbQXASE L0vBSggRrVtU66ieQ3+MXaK6eOebeCZBhYDI48ZiVpcQvhInySLeZQV9QZ3//XI37m3FKgvP1Cj 5npVOGIGUUYuZhEpNEVeOoiw= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id dcce8eb518750f6d; Thu, 13 Aug 2026 15:23:51 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com Subject: [PATCH bpf-next v12 02/10] bpf: Factor out check_map_mem_read helper in verifier Date: Thu, 13 Aug 2026 23:23:15 +0800 Message-ID: <20260813152324.97937-3-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In the next commit, percpu_array map will add map_direct_value_addr support. IOW, it will add a map_type check in the iff condition of the bpf_map_direct_read() code block, which will reduce the code block readability. Hence, factor out check_map_mem_read helper to improve the readability, and the maintainability for the percpu_array map case. Acked-by: Eduard Zingerman Signed-off-by: Leon Hwang --- kernel/bpf/verifier.c | 75 +++++++++++++++++++++++++------------------ 1 file changed, 43 insertions(+), 32 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index cdb79a66b156..4fac230122d9 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -6078,6 +6078,48 @@ static void add_scalar_to_reg(struct bpf_reg_state *= dst_reg, s64 val) reg_bounds_sync(dst_reg); } =20 +static int check_map_mem_read(struct bpf_verifier_env *env, struct bpf_reg= _state *reg, int off, + int bpf_size, int value_regno, bool is_ldsx) +{ + struct bpf_reg_state *regs =3D cur_regs(env); + int size =3D bpf_size_to_bytes(bpf_size); + struct bpf_map *map =3D reg->map_ptr; + + switch (map->map_type) { + case BPF_MAP_TYPE_INSN_ARRAY: + if (bpf_size !=3D BPF_DW) { + verbose(env, "Invalid read of %d bytes from insn_array\n", size); + return -EACCES; + } + regs[value_regno] =3D *reg; + add_scalar_to_reg(®s[value_regno], off); + regs[value_regno].type =3D PTR_TO_INSN; + return 0; + default: + break; + } + + /* If map is read-only, track its contents as scalars. */ + if (tnum_is_const(reg->var_off) && + bpf_map_is_rdonly(map) && + map->ops->map_direct_value_addr) { + int map_off =3D off + reg->var_off.value; + u64 val =3D 0; + int err; + + err =3D bpf_map_direct_read(map, map_off, size, &val, is_ldsx); + if (err) + return err; + + regs[value_regno].type =3D SCALAR_VALUE; + __mark_reg_known(®s[value_regno], val); + return 0; + } + + mark_reg_unknown(env, regs, value_regno); + return 0; +} + /* check whether memory at (regno + off) is accessible for t =3D (read | w= rite) * if t=3D=3Dwrite, value_regno is a register which value is stored into m= emory * if t=3D=3Dread, value_regno is a register which will receive the value = from memory @@ -6132,38 +6174,7 @@ static int check_mem_access(struct bpf_verifier_env = *env, int insn_idx, struct b if (kptr_field) { err =3D check_map_kptr_access(env, value_regno, insn_idx, kptr_field); } else if (t =3D=3D BPF_READ && value_regno >=3D 0) { - struct bpf_map *map =3D reg->map_ptr; - - /* - * If map is read-only, track its contents as scalars, - * unless it is an insn array (see the special case below) - */ - if (tnum_is_const(reg->var_off) && - bpf_map_is_rdonly(map) && - map->ops->map_direct_value_addr && - map->map_type !=3D BPF_MAP_TYPE_INSN_ARRAY) { - int map_off =3D off + reg->var_off.value; - u64 val =3D 0; - - err =3D bpf_map_direct_read(map, map_off, size, - &val, is_ldsx); - if (err) - return err; - - regs[value_regno].type =3D SCALAR_VALUE; - __mark_reg_known(®s[value_regno], val); - } else if (map->map_type =3D=3D BPF_MAP_TYPE_INSN_ARRAY) { - if (bpf_size !=3D BPF_DW) { - verbose(env, "Invalid read of %d bytes from insn_array\n", - size); - return -EACCES; - } - regs[value_regno] =3D *reg; - add_scalar_to_reg(®s[value_regno], off); - regs[value_regno].type =3D PTR_TO_INSN; - } else { - mark_reg_unknown(env, regs, value_regno); - } + err =3D check_map_mem_read(env, reg, off, bpf_size, value_regno, is_lds= x); } } else if (base_type(reg->type) =3D=3D PTR_TO_MEM) { bool rdonly_mem =3D type_is_rdonly_mem(reg->type); --=20 2.55.0 From nobody Tue Sep 29 01:59:56 2026 Received: from mta1.migadu.com (out-218.mta1.migadu.com [95.215.58.218]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08D2D481672 for ; Thu, 13 Aug 2026 15:24:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.218 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634642; cv=none; b=JM0upHwEt4KscUboA95Wm1FjqdGw6JNwxAeE4EeKBrtDjdvvUF9m9HSrJSUtBIYId3ZrjqwtzJL9Yy2KbrRGuPV1bvAACs87zBIa1nioD4sH/0AeiezjkOR9ZyTHUR0u8hgEP29qRYmmpEqg66nh6pDmXxS5L9Tbd1RF7MfdrkY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634642; c=relaxed/simple; bh=yB48kahetbeEkeaNGZTZRWgCKbMzbovCzAn/9dJ5enc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ag4ZqOdE3/mCj1MUDU0/jbaX02wNf3bCvyfesOyEsRK1hh9oluJrsxwrXINRhL1cdcPnYWS971k9LDAdqMgzg166F6uHhw0/lIEsfnd9TlZFw3RTMyzGIzL9DRLIpaLDPTf2iVwP5f2KSzCW3uew+3Ea8JuzQSJkkcxqnz6OfiE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=mbIWddtF; arc=none smtp.client-ip=95.215.58.218 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="mbIWddtF" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=yB48kahetbeEkeaNGZTZRWgCKbMzbovCzAn/9dJ5enc=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634639; v=1; x=1787239439; b=mbIWddtFokRYBVysPCScuMjsF3AWgBjhHUunn6CT2FUC7jQ+GnWDUWY7daTNGpBOjW17AQqp NV0Rdy8Onw9XGht0LFCv65J9/f4r2p/3zuQ/HTpiciGF8RGA0ib4OLjTFYJQU+C3CtxQ4wFIVzX rUKeevXYu1bUCP/zztvCTIo4= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id 19b2e3cb29af70c6; Thu, 13 Aug 2026 15:23:58 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com Subject: [PATCH bpf-next v12 03/10] bpf: Introduce global percpu data Date: Thu, 13 Aug 2026 23:23:16 +0800 Message-ID: <20260813152324.97937-4-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce global percpu data, inspired by the commit 6316f78306c1 ("Merge branch 'support-global-data'"). It enables the definition of global percpu variables in BPF, similar to the include/linux/percpu-defs.h::DEFINE_PER_CPU() macro. For example, in BPF, it is able to define a global percpu variable like: int data SEC(".percpu"); With this patch, tools like retsnoop [1] and bpfsnoop [2] can simplify their BPF code for handling LBRs. The code can be updated from static struct perf_branch_entry lbrs[1][MAX_LBR_ENTRIES] SEC(".data.lbrs"); to static struct perf_branch_entry lbrs[MAX_LBR_ENTRIES] SEC(".percpu.lbrs"); This eliminates the need to retrieve the CPU ID using the bpf_get_smp_processor_id() helper. Additionally, by reusing global percpu data map, sharing information between tail callers and callees or freplace callers and callees becomes simpler compared to reusing percpu_array maps. Links: [1] https://github.com/anakryiko/retsnoop [2] https://github.com/bpfsnoop/bpfsnoop Acked-by: Eduard Zingerman Signed-off-by: Leon Hwang --- kernel/bpf/arraymap.c | 38 ++++++++++++++++++++++++++++++++++++-- kernel/bpf/const_fold.c | 1 - kernel/bpf/fixups.c | 37 +++++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 11 +++++++++++ 4 files changed, 84 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 248b4818178c..34865701f7f7 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -259,6 +259,37 @@ static void *percpu_array_map_lookup_elem(struct bpf_m= ap *map, void *key) return this_cpu_ptr(array->pptrs[index & array->index_mask]); } =20 +static int percpu_array_map_direct_value_addr(const struct bpf_map *map, u= 64 *imm, u32 off) +{ + struct bpf_array *array =3D container_of(map, struct bpf_array, map); + + if (!bpf_jit_supports_percpu_insn()) + return -EOPNOTSUPP; + if (map->max_entries !=3D 1) + return -EOPNOTSUPP; + if (off >=3D map->value_size) + return -EINVAL; + + *imm =3D (u64)(__force unsigned long) array->pptrs[0]; + return 0; +} + +static int percpu_array_map_direct_value_meta(const struct bpf_map *map, u= 64 imm, u32 *off) +{ + struct bpf_array *array =3D container_of(map, struct bpf_array, map); + u64 base =3D (u64)(__force unsigned long) array->pptrs[0]; + + if (!bpf_jit_supports_percpu_insn()) + return -EOPNOTSUPP; + if (map->max_entries !=3D 1) + return -EOPNOTSUPP; + if (imm < base || imm >=3D base + array->elem_size) + return -ENOENT; + + *off =3D imm - base; + return 0; +} + /* emit BPF instructions equivalent to C code of percpu_array_map_lookup_e= lem() */ static int percpu_array_map_gen_lookup(struct bpf_map *map, struct bpf_ins= n *insn_buf) { @@ -551,9 +582,10 @@ static int array_map_check_btf(struct bpf_map *map, const struct btf_type *key_type, const struct btf_type *value_type) { - /* One exception for keyless BTF: .bss/.data/.rodata map */ + /* One exception for keyless BTF: .bss/.data/.rodata/.percpu map */ if (btf_type_is_void(key_type)) { - if (map->map_type !=3D BPF_MAP_TYPE_ARRAY || + if ((map->map_type !=3D BPF_MAP_TYPE_ARRAY && + map->map_type !=3D BPF_MAP_TYPE_PERCPU_ARRAY) || map->max_entries !=3D 1) return -EINVAL; =20 @@ -832,6 +864,8 @@ const struct bpf_map_ops percpu_array_map_ops =3D { .map_get_next_key =3D bpf_array_get_next_key, .map_lookup_elem =3D percpu_array_map_lookup_elem, .map_gen_lookup =3D percpu_array_map_gen_lookup, + .map_direct_value_addr =3D percpu_array_map_direct_value_addr, + .map_direct_value_meta =3D percpu_array_map_direct_value_meta, .map_update_elem =3D array_map_update_elem, .map_delete_elem =3D array_map_delete_elem, .map_lookup_percpu_elem =3D percpu_array_map_lookup_percpu_elem, diff --git a/kernel/bpf/const_fold.c b/kernel/bpf/const_fold.c index 4cf120c7b2cb..7f1b30059cc8 100644 --- a/kernel/bpf/const_fold.c +++ b/kernel/bpf/const_fold.c @@ -182,7 +182,6 @@ static void const_reg_xfer(struct bpf_verifier_env *env= , struct const_arg_info * u64 val =3D 0; =20 if (!bpf_map_is_rdonly(map) || !map->ops->map_direct_value_addr || - map->map_type =3D=3D BPF_MAP_TYPE_INSN_ARRAY || off < 0 || off + size > map->value_size || bpf_map_direct_read(map, off, size, &val, is_ldsx)) { *dst =3D unknown; diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c index 0caf1bbd9494..177a3fcbb63a 100644 --- a/kernel/bpf/fixups.c +++ b/kernel/bpf/fixups.c @@ -1834,6 +1834,43 @@ int bpf_do_misc_fixups(struct bpf_verifier_env *env) goto next_insn; } =20 + if (bpf_jit_supports_percpu_insn() && + insn->code =3D=3D (BPF_LD | BPF_IMM | BPF_DW) && + (insn->src_reg =3D=3D BPF_PSEUDO_MAP_VALUE || + insn->src_reg =3D=3D BPF_PSEUDO_MAP_IDX_VALUE)) { + struct bpf_map *map; + + aux =3D &env->insn_aux_data[i + delta]; + map =3D env->used_maps[aux->map_index]; + if (map->map_type !=3D BPF_MAP_TYPE_PERCPU_ARRAY) + goto next_insn; + + prog->jit_required =3D true; + + /* + * We are *skipping* first half of ld_imm64 insn + * with 'i++;', patching over second half of it + * with that same half + mov64_percpu_reg insn. + * All because bpf_patch_insn_data() can only + * replace one 8-byte insn, which does not work + * well for ld_imm64 insn. + */ + + insn_buf[0] =3D insn[1]; + insn_buf[1] =3D BPF_MOV64_PERCPU_REG(insn->dst_reg, insn->dst_reg); + cnt =3D 2; + + i++; + new_prog =3D bpf_patch_insn_data(env, i + delta, insn_buf, cnt); + if (!new_prog) + return -ENOMEM; + + delta +=3D cnt - 1; + env->prog =3D prog =3D new_prog; + insn =3D new_prog->insnsi + i + delta; + goto next_insn; + } + if (insn->code !=3D (BPF_JMP | BPF_CALL)) goto next_insn; if (insn->src_reg =3D=3D BPF_PSEUDO_CALL) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 4fac230122d9..6ac1afced20b 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -5582,6 +5582,8 @@ int bpf_map_direct_read(struct bpf_map *map, int off,= int size, u64 *val, u64 addr; int err; =20 + if (map->map_type =3D=3D BPF_MAP_TYPE_INSN_ARRAY || map->map_type =3D=3D = BPF_MAP_TYPE_PERCPU_ARRAY) + return -EINVAL; err =3D map->ops->map_direct_value_addr(map, &addr, off); if (err) return err; @@ -6095,6 +6097,8 @@ static int check_map_mem_read(struct bpf_verifier_env= *env, struct bpf_reg_state add_scalar_to_reg(®s[value_regno], off); regs[value_regno].type =3D PTR_TO_INSN; return 0; + case BPF_MAP_TYPE_PERCPU_ARRAY: + goto reg_unknown; default: break; } @@ -6116,6 +6120,7 @@ static int check_map_mem_read(struct bpf_verifier_env= *env, struct bpf_reg_state return 0; } =20 +reg_unknown: mark_reg_unknown(env, regs, value_regno); return 0; } @@ -8129,6 +8134,12 @@ static int check_arg_const_str(struct bpf_verifier_e= nv *env, return -EACCES; } =20 + if (map->map_type =3D=3D BPF_MAP_TYPE_PERCPU_ARRAY) { + verbose(env, "%s points to percpu_array map which cannot be used as cons= t string\n", + reg_arg_name(env, argno)); + return -EACCES; + } + if (!bpf_map_is_rdonly(map)) { verbose(env, "%s does not point to a readonly map'\n", reg_arg_name(env,= argno)); return -EACCES; --=20 2.55.0 From nobody Tue Sep 29 01:59:56 2026 Received: from mta1.migadu.com (out-224.mta1.migadu.com [95.215.58.224]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C938C485CE3 for ; Thu, 13 Aug 2026 15:24:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.224 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634649; cv=none; b=TbjfuMKWQsUkQZAACpsas7Y1ibZCtppFZfNwrsgkPsQjwlU0lPAeMeJtLAIRyhDqNMqKj8PiM7JwQlJQ4cNM828nmz1moKLSYFxLqX6qFRv72j39MEl6RXRSbrezivflApdaMlaLTYvr7bSbSuy4zD26etOkwRPwIsEK7SnHCU4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634649; c=relaxed/simple; bh=5ZCt5mLnoYxldIysQPGgQKI9kJMnrVUrpohc+4vpe68=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ivzHEjKuBbNPnUajyBbEWHs3Fyxb6dGWLkrAOFVusqVOxWLL4T6jxkmkPbAZSFWVhcrIk8iyBY7gCRZ85Ez5PNebPBWxfRZpn1PBtYd1aUAnMWfUgTvJVLpSxorhp0MkjPA/tSpYHZXusqh2C9xmIcom1EexugB70F6HUWHa8ro= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=fduSR/cK; arc=none smtp.client-ip=95.215.58.224 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="fduSR/cK" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=5ZCt5mLnoYxldIysQPGgQKI9kJMnrVUrpohc+4vpe68=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634645; v=1; x=1787239445; b=fduSR/cKP6G5O8DqkFUx0vA2iFdIyR8khH1x88w5S9/1xLXUFryEH+OjLzesnEfbi1XpucjG YTbBaH6AzhrSeRxXuCRCs2yedElLB4SuK1zQ39N8JPM6DgfAjdhNY+7cLUsZlRzQufv+/9J4whl IpNEDjy0qpQ02RR46DyJ+nSM= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id 6c86e6d09adda9ec; Thu, 13 Aug 2026 15:24:05 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com, Emil Tsalapatis Subject: [PATCH bpf-next v12 04/10] libbpf: Probe percpu data feature Date: Thu, 13 Aug 2026 23:23:17 +0800 Message-ID: <20260813152324.97937-5-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" libbpf needs a reliable way to distinguish kernels that can support global percpu data from those that cannot. Add a dedicated feature probe, so libbpf can make capability decisions early and fail predictably when global percpu data is unavailable. Acked-by: Andrii Nakryiko Reviewed-by: Emil Tsalapatis Signed-off-by: Leon Hwang --- tools/lib/bpf/features.c | 35 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_internal.h | 2 ++ 2 files changed, 37 insertions(+) diff --git a/tools/lib/bpf/features.c b/tools/lib/bpf/features.c index b7e388f99d0b..ef9581c11303 100644 --- a/tools/lib/bpf/features.c +++ b/tools/lib/bpf/features.c @@ -620,6 +620,38 @@ static int probe_bpf_syscall_common_attrs(int token_fd) return probe_sys_bpf_ext(); } =20 +static int probe_kern_percpu_data(int token_fd) +{ + struct bpf_insn insns[] =3D { + BPF_LD_MAP_VALUE(BPF_REG_1, 0, 0), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 0), + BPF_EXIT_INSN(), + }; + LIBBPF_OPTS(bpf_map_create_opts, map_opts, + .token_fd =3D token_fd, + .map_flags =3D token_fd ? BPF_F_TOKEN_FD : 0, + ); + LIBBPF_OPTS(bpf_prog_load_opts, prog_opts, + .token_fd =3D token_fd, + .prog_flags =3D token_fd ? BPF_F_TOKEN_FD : 0, + ); + int ret, map, insn_cnt =3D ARRAY_SIZE(insns); + + map =3D bpf_map_create(BPF_MAP_TYPE_PERCPU_ARRAY, "libbpf_percpu", sizeof= (int), 8, 1, + &map_opts); + if (map < 0) { + pr_warn("Error in %s(): %s. Couldn't create simple percpu_array map.\n", + __func__, errstr(map)); + return map; + } + + insns[0].imm =3D map; + + ret =3D bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", insns, in= sn_cnt, &prog_opts); + close(map); + return probe_fd(ret); +} + typedef int (*feature_probe_fn)(int /* token_fd */); =20 static struct kern_feature_cache feature_cache; @@ -707,6 +739,9 @@ static struct kern_feature_desc { [FEAT_BPF_SYSCALL_COMMON_ATTRS] =3D { "BPF syscall common attributes support", probe_bpf_syscall_common_attrs, }, + [FEAT_PERCPU_DATA] =3D { + "kernel supports percpu data", probe_kern_percpu_data, + }, }; =20 bool feat_supported(struct kern_feature_cache *cache, enum kern_feature_id= feat_id) diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_interna= l.h index 7a74abb904f8..4c46d34fc055 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -401,6 +401,8 @@ enum kern_feature_id { FEAT_BTF_LAYOUT, /* Kernel supports BPF syscall common attributes */ FEAT_BPF_SYSCALL_COMMON_ATTRS, + /* Kernel supports percpu data */ + FEAT_PERCPU_DATA, __FEAT_CNT, }; =20 --=20 2.55.0 From nobody Tue Sep 29 01:59:56 2026 Received: from mta1.migadu.com (out-235.mta1.migadu.com [95.215.58.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70852480DD2 for ; Thu, 13 Aug 2026 15:24:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.235 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634671; cv=none; b=bkfcr0lENvHbXzqM8Nr+pPapQrkV85OfMYamm6/z/nxWcN4Y4P/nXOXB7J9MeQ21adPaz2aFTNTY422pnLwD1nvjYnw6o1JKpOG83IpGD0gXHN1Fbb+C6EeWe3kLA1ahujyzqAteUaorjbUQjAnFd7iySCRhjI24oQwY712HweA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634671; c=relaxed/simple; bh=I7mICQjvdzBTJbzjS5cuLrhSBOFhUT9GSMraQWwZXsE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sT6MmvJhzsWaxfH5TxNyCYB9+mZxBG+Oq6AKu4K+OdDpGSHpVkTH22kgmFUOJdPdHZLageTU/3i+eg3i6raFTAYhZlle3Fyl5H6tl6VDqEHWPFXGOAjv3MEXuTt02Oftp/5tOcyDgFuefNpGRtFC0a6N+hLy6nGt/933FE3oJiY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Aec4Xjuk; arc=none smtp.client-ip=95.215.58.235 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Aec4Xjuk" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=I7mICQjvdzBTJbzjS5cuLrhSBOFhUT9GSMraQWwZXsE=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634667; v=1; x=1787239467; b=Aec4XjuknDVKA0CVYumLHkaoApe1aDS7viqpzzCE1EiCbVEJd3M5vO4QjsAWJ3xIdSJsn8Pf 3ZCeev87CDMHxly6qMJ3wiBImx0F6+QvkGnANrN4rbf6OrvT24o4D2xO3+LZ2WQni5BjHnKNOAU 16t1jaHetRx0V8fuERDm/2Rs= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id 3fed13be608966b0; Thu, 13 Aug 2026 15:24:27 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com Subject: [PATCH bpf-next v12 05/10] libbpf: Add support for global percpu data Date: Thu, 13 Aug 2026 23:23:18 +0800 Message-ID: <20260813152324.97937-6-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add support for global percpu data in libbpf by adding a new ".percpu" section, similar to ".data". It enables efficient handling of percpu global variables in bpf programs. When generating loader for lightweight skeleton, update the percpu_array map used for global percpu data using BPF_F_ALL_CPUS, in order to update values across all CPUs using one value slot. Unlike global data, the mmaped data for global percpu data will be marked as read-only after populating the percpu_array map. Thereafter, users can read those initialized percpu data after loading prog. If they want to update the percpu data after loading prog, they have to update the percpu_array map using key=3D0 instead. Signed-off-by: Leon Hwang --- tools/lib/bpf/bpf_gen_internal.h | 3 +- tools/lib/bpf/gen_loader.c | 3 +- tools/lib/bpf/libbpf.c | 81 +++++++++++++++++++++++++++----- 3 files changed, 72 insertions(+), 15 deletions(-) diff --git a/tools/lib/bpf/bpf_gen_internal.h b/tools/lib/bpf/bpf_gen_inter= nal.h index 042569187752..6c5ad6c55e8a 100644 --- a/tools/lib/bpf/bpf_gen_internal.h +++ b/tools/lib/bpf/bpf_gen_internal.h @@ -65,7 +65,8 @@ void bpf_gen__prog_load(struct bpf_gen *gen, enum bpf_prog_type prog_type, const char *prog_name, const char *license, struct bpf_insn *insns, size_t insn_cnt, struct bpf_prog_load_opts *load_attr, int prog_idx); -void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *valu= e, __u32 value_size); +void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *valu= e, __u32 value_size, + __u64 flags); void bpf_gen__map_freeze(struct bpf_gen *gen, int map_idx); void bpf_gen__record_attach_target(struct bpf_gen *gen, const char *name, = enum bpf_attach_type type); void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is= _weak, diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c index 6e3dd5242761..af3a04f161ac 100644 --- a/tools/lib/bpf/gen_loader.c +++ b/tools/lib/bpf/gen_loader.c @@ -1128,7 +1128,7 @@ void bpf_gen__prog_load(struct bpf_gen *gen, } =20 void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *pval= ue, - __u32 value_size) + __u32 value_size, __u64 flags) { int attr_size =3D offsetofend(union bpf_attr, flags); int map_update_attr, value, key; @@ -1136,6 +1136,7 @@ void bpf_gen__map_update_elem(struct bpf_gen *gen, in= t map_idx, void *pvalue, int zero =3D 0; =20 memset(&attr, 0, attr_size); + attr.flags =3D tgt_endian(flags); =20 value =3D add_data(gen, pvalue, value_size); key =3D add_data(gen, &zero, sizeof(zero)); diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 514e4e9daa82..e574870fb716 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -541,6 +541,7 @@ struct bpf_struct_ops { }; =20 #define DATA_SEC ".data" +#define PERCPU_SEC ".percpu" #define BSS_SEC ".bss" #define RODATA_SEC ".rodata" #define KCONFIG_SEC ".kconfig" @@ -555,6 +556,7 @@ enum libbpf_map_type { LIBBPF_MAP_BSS, LIBBPF_MAP_RODATA, LIBBPF_MAP_KCONFIG, + LIBBPF_MAP_PERCPU, }; =20 struct bpf_map_def { @@ -666,6 +668,7 @@ enum sec_type { SEC_DATA, SEC_RODATA, SEC_ST_OPS, + SEC_PERCPU, }; =20 struct elf_sec_desc { @@ -1839,6 +1842,8 @@ static size_t bpf_map_mmap_sz(const struct bpf_map *m= ap) switch (map->def.type) { case BPF_MAP_TYPE_ARRAY: return array_map_mmap_sz(map->def.value_size, map->def.max_entries); + case BPF_MAP_TYPE_PERCPU_ARRAY: + return map->def.value_size; case BPF_MAP_TYPE_ARENA: return page_sz * map->def.max_entries; default: @@ -1866,7 +1871,8 @@ static int bpf_map_mmap_resize(struct bpf_map *map, s= ize_t old_sz, size_t new_sz return 0; } =20 -static char *internal_map_name(struct bpf_object *obj, const char *real_na= me) +static char *internal_map_name(struct bpf_object *obj, const char *real_na= me, + enum libbpf_map_type type) { char map_name[BPF_OBJ_NAME_LEN], *p; int pfx_len, sfx_len =3D max((size_t)7, strlen(real_name)); @@ -1907,8 +1913,11 @@ static char *internal_map_name(struct bpf_object *ob= j, const char *real_name) if (sfx_len >=3D BPF_OBJ_NAME_LEN) sfx_len =3D BPF_OBJ_NAME_LEN - 1; =20 - /* if there are two or more dots in map name, it's a custom dot map */ - if (strchr(real_name + 1, '.') !=3D NULL) + /* + * Don't prefix the bpf_object name if this is a custom dot map + * (containing two or more dots) or a percpu data map. + */ + if (strchr(real_name + 1, '.') !=3D NULL || type =3D=3D LIBBPF_MAP_PERCPU) pfx_len =3D 0; else pfx_len =3D min((size_t)BPF_OBJ_NAME_LEN - sfx_len - 1, strlen(obj->name= )); @@ -1941,6 +1950,13 @@ static bool map_is_mmapable(struct bpf_object *obj, = struct bpf_map *map) if (!map->btf_value_type_id) return false; =20 + /* + * The internal PERCPU maps are not mmapble because the underlying + * percpu_array maps do not have mmap support. + */ + if (map->libbpf_type =3D=3D LIBBPF_MAP_PERCPU) + return false; + t =3D btf__type_by_id(obj->btf, map->btf_value_type_id); if (!btf_is_datasec(t)) return false; @@ -1962,6 +1978,7 @@ static int bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_type= type, const char *real_name, int sec_idx, void *data, size_t data_sz) { + bool is_percpu =3D type =3D=3D LIBBPF_MAP_PERCPU; struct bpf_map_def *def; struct bpf_map *map; size_t mmap_sz; @@ -1975,7 +1992,7 @@ bpf_object__init_internal_map(struct bpf_object *obj,= enum libbpf_map_type type, map->sec_idx =3D sec_idx; map->sec_offset =3D 0; map->real_name =3D strdup(real_name); - map->name =3D internal_map_name(obj, real_name); + map->name =3D internal_map_name(obj, real_name, type); if (!map->real_name || !map->name) { zfree(&map->real_name); zfree(&map->name); @@ -1983,7 +2000,7 @@ bpf_object__init_internal_map(struct bpf_object *obj,= enum libbpf_map_type type, } =20 def =3D &map->def; - def->type =3D BPF_MAP_TYPE_ARRAY; + def->type =3D is_percpu ? BPF_MAP_TYPE_PERCPU_ARRAY : BPF_MAP_TYPE_ARRAY; def->key_size =3D sizeof(int); def->value_size =3D data_sz; def->max_entries =3D 1; @@ -1996,8 +2013,9 @@ bpf_object__init_internal_map(struct bpf_object *obj,= enum libbpf_map_type type, if (map_is_mmapable(obj, map)) def->map_flags |=3D BPF_F_MMAPABLE; =20 - pr_debug("map '%s' (global data): at sec_idx %d, offset %zu, flags %x.\n", - map->name, map->sec_idx, map->sec_offset, def->map_flags); + pr_debug("map '%s' (global %sdata): at sec_idx %d, offset %zu, flags %x.\= n", + map->name, is_percpu ? "percpu " : "", map->sec_idx, + map->sec_offset, def->map_flags); =20 mmap_sz =3D bpf_map_mmap_sz(map); map->mmaped =3D mmap(NULL, mmap_sz, PROT_READ | PROT_WRITE, @@ -2057,6 +2075,13 @@ static int bpf_object__init_global_data_maps(struct = bpf_object *obj) NULL, sec_desc->data->d_size); break; + case SEC_PERCPU: + sec_name =3D elf_sec_name(obj, elf_sec_by_idx(obj, sec_idx)); + err =3D bpf_object__init_internal_map(obj, LIBBPF_MAP_PERCPU, + sec_name, sec_idx, + sec_desc->data->d_buf, + sec_desc->data->d_size); + break; default: /* skip */ break; @@ -4016,6 +4041,11 @@ static int bpf_object__elf_collect(struct bpf_object= *obj) sec_desc->sec_type =3D SEC_RODATA; sec_desc->shdr =3D sh; sec_desc->data =3D data; + } else if (strcmp(name, PERCPU_SEC) =3D=3D 0 || + str_has_pfx(name, PERCPU_SEC ".")) { + sec_desc->sec_type =3D SEC_PERCPU; + sec_desc->shdr =3D sh; + sec_desc->data =3D data; } else if (strcmp(name, STRUCT_OPS_SEC) =3D=3D 0 || strcmp(name, STRUCT_OPS_LINK_SEC) =3D=3D 0 || strcmp(name, "?" STRUCT_OPS_SEC) =3D=3D 0 || @@ -4544,6 +4574,7 @@ static bool bpf_object__shndx_is_data(const struct bp= f_object *obj, case SEC_BSS: case SEC_DATA: case SEC_RODATA: + case SEC_PERCPU: return true; default: return false; @@ -4569,6 +4600,8 @@ bpf_object__section_to_libbpf_map_type(const struct b= pf_object *obj, int shndx) return LIBBPF_MAP_DATA; case SEC_RODATA: return LIBBPF_MAP_RODATA; + case SEC_PERCPU: + return LIBBPF_MAP_PERCPU; default: return LIBBPF_MAP_UNSPEC; } @@ -4944,7 +4977,7 @@ static int map_fill_btf_type_info(struct bpf_object *= obj, struct bpf_map *map) =20 /* * LLVM annotates global data differently in BTF, that is, - * only as '.data', '.bss' or '.rodata'. + * only as '.data', '.bss', '.percpu' or '.rodata'. */ if (!bpf_map__is_internal(map)) return -ENOENT; @@ -5293,18 +5326,20 @@ static int bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *= map) { enum libbpf_map_type map_type =3D map->libbpf_type; + bool is_percpu =3D map_type =3D=3D LIBBPF_MAP_PERCPU; + const __u64 update_flags =3D is_percpu ? BPF_F_ALL_CPUS : 0; int err, zero =3D 0; size_t mmap_sz; =20 if (obj->gen_loader) { bpf_gen__map_update_elem(obj->gen_loader, map - obj->maps, - map->mmaped, map->def.value_size); + map->mmaped, map->def.value_size, update_flags); if (map_type =3D=3D LIBBPF_MAP_RODATA || map_type =3D=3D LIBBPF_MAP_KCON= FIG) bpf_gen__map_freeze(obj->gen_loader, map - obj->maps); return 0; } =20 - err =3D bpf_map_update_elem(map->fd, &zero, map->mmaped, 0); + err =3D bpf_map_update_elem(map->fd, &zero, map->mmaped, update_flags); if (err) { err =3D -errno; pr_warn("map '%s': failed to set initial contents: %s\n", @@ -5349,6 +5384,13 @@ bpf_object__populate_internal_map(struct bpf_object = *obj, struct bpf_map *map) return err; } map->mmaped =3D mmaped; + } else if (is_percpu) { + if (mprotect(map->mmaped, mmap_sz, PROT_READ)) { + err =3D -errno; + pr_warn("map '%s': failed to mprotect() contents: %s\n", + bpf_map__name(map), errstr(err)); + return err; + } } else if (map->mmaped) { munmap(map->mmaped, mmap_sz); map->mmaped =3D NULL; @@ -5624,9 +5666,16 @@ bpf_object__create_maps(struct bpf_object *obj) * runtime due to bpf_program__set_autoload(prog, false), * bpf_object loading will succeed just fine even on old * kernels. + * Same skipping applies to percpu data. */ - if (bpf_map__is_internal(map) && !kernel_supports(obj, FEAT_GLOBAL_DATA)) - map->autocreate =3D false; + if (bpf_map__is_internal(map)) { + bool is_percpu =3D map->libbpf_type =3D=3D LIBBPF_MAP_PERCPU; + enum kern_feature_id feat_id; + + feat_id =3D is_percpu ? FEAT_PERCPU_DATA : FEAT_GLOBAL_DATA; + if (!kernel_supports(obj, feat_id)) + map->autocreate =3D false; + } =20 if (!map->autocreate) { pr_debug("map '%s': skipped auto-creating...\n", map->name); @@ -10807,11 +10856,16 @@ static bool map_uses_real_name(const struct bpf_m= ap *map) * such map's corresponding ELF section name as a map name. * This check distinguishes .data/.rodata from .data.* and .rodata.* * maps to know which name has to be returned to the user. + * Map name of the custom .percpu.* maps might be truncated to + * BPF_OBJ_NAME_LEN-1 chars in internal_map_name(). Hence, percpu data + * maps must use real name for their user-visible name. */ if (map->libbpf_type =3D=3D LIBBPF_MAP_DATA && strcmp(map->real_name, DAT= A_SEC) !=3D 0) return true; if (map->libbpf_type =3D=3D LIBBPF_MAP_RODATA && strcmp(map->real_name, R= ODATA_SEC) !=3D 0) return true; + if (map->libbpf_type =3D=3D LIBBPF_MAP_PERCPU) + return true; return false; } =20 @@ -10976,7 +11030,8 @@ int bpf_map__set_value_size(struct bpf_map *map, __= u32 size) size_t mmap_old_sz, mmap_new_sz; int err; =20 - if (map->def.type !=3D BPF_MAP_TYPE_ARRAY) + if (map->def.type !=3D BPF_MAP_TYPE_ARRAY && + map->def.type !=3D BPF_MAP_TYPE_PERCPU_ARRAY) return libbpf_err(-EOPNOTSUPP); =20 mmap_old_sz =3D bpf_map_mmap_sz(map); --=20 2.55.0 From nobody Tue Sep 29 01:59:56 2026 Received: from mta0.migadu.com (out-147.mta0.migadu.com [91.218.175.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 000C8485CF2 for ; Thu, 13 Aug 2026 15:24:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.147 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634682; cv=none; b=IJr7NZUoYFqtZlJEK8O0TWpTD7cRHJhBLIGeFQscg4BM5AdKkSFbAY+5c+Oj6j5JrIQq4vyCSAsSnmabu+WDTvdbU8KzzKe2Z2PD15eWE5frhzteHU7cJfW6WahEsaKi9vnQ9wD7DXfFcZWTrehwA3xzc+/wNEnaydegQKX2ZnY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634682; c=relaxed/simple; bh=aFosn3y8R2POzRDoyB0GVTPz1E1cfvnc2ChUoOTHu2E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GjANUv6g9sR6uujUpRhiSQIoSh08d3TQPXcaExypeFY3/uK7/Pm1UD5soQ8NhyxtMRNyFBMNiWwvaNGmcrOKeZzDRUGdPf4sWGLebB1MGCW0u2VdnWN3TUBBKQeKJPYUK+otVim53clkYCQb18ww2Hr0HiiG81nc0EE0V6l6tLM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=QVxi/wKd; arc=none smtp.client-ip=91.218.175.147 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="QVxi/wKd" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=aFosn3y8R2POzRDoyB0GVTPz1E1cfvnc2ChUoOTHu2E=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634678; v=1; x=1787239478; b=QVxi/wKd3bwBGQ49/1N838I9NTnVPuqSUDvM8Xa6HMN6b6gdFUx0i5bY6pwQztAHxWANe3cS ddWdXqe3GQtzVn3iEIGY+wC4PkH+SQGH8QibLG4+6sslE+qRXp9ySux5tpICPzO2Y1TtTl4GKlq eiLFl2TwN16oD6sfE52EGngQ= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id 9ff169fd82cf7c6e; Thu, 13 Aug 2026 15:24:38 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com Subject: [PATCH bpf-next v12 06/10] bpftool: Generate skeleton for global percpu data Date: Thu, 13 Aug 2026 23:23:19 +0800 Message-ID: <20260813152324.97937-7-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Enhance bpftool to generate skeletons that properly handle global percpu variables. The generated skeleton now includes a dedicated structure for percpu data, allowing users to initialize and access percpu variables more efficiently. For global percpu variables, the skeleton now includes a nested structure, e.g.: struct test_global_percpu_data { struct bpf_object_skeleton *skeleton; struct bpf_object *obj; struct { struct bpf_map *percpu; } maps; // ... struct test_global_percpu_data__percpu { int data; char run; struct { char set; int i; int nums[7]; } struct_data; int nums[7]; } *percpu; // ... }; * The "struct test_global_percpu_data__percpu *percpu" points to initialized data, which is actually "maps.percpu->mmaped". * Before loading the skeleton, updating the "struct test_global_percpu_data__percpu *percpu" modifies the initial value of the corresponding global percpu variables. * After loading the skeleton, "maps.percpu->mmaped" has been marked as read-only in libbpf. If users want to update the global percpu variables, they have to update the "maps.percpu" map instead. * For lightweight skeleton, "lskel->percpu" will be protected by "mprotect(p, sz, PROT_READ)". * For subskeleton, those variables of global percpu data will be skipped. Acked-by: Quentin Monnet Assisted-by: Codex:gpt-5.5-xhigh Signed-off-by: Leon Hwang --- tools/bpf/bpftool/gen.c | 53 +++++++++++++++++++++++++++-------- tools/lib/bpf/skel_internal.h | 24 ++++++++++++++-- 2 files changed, 63 insertions(+), 14 deletions(-) diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index a01d06d22d1a..a50540ef6521 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -101,6 +101,12 @@ static bool get_map_ident(const struct bpf_map *map, c= har *buf, size_t buf_sz) return true; } =20 + if (bpf_map__type(map) =3D=3D BPF_MAP_TYPE_PERCPU_ARRAY) { + snprintf(buf, buf_sz, "%s", name + 1); + sanitize_identifier(buf); + return true; + } + for (i =3D 0, n =3D ARRAY_SIZE(sfxs); i < n; i++) { const char *sfx =3D sfxs[i], *p; =20 @@ -117,7 +123,7 @@ static bool get_map_ident(const struct bpf_map *map, ch= ar *buf, size_t buf_sz) =20 static bool get_datasec_ident(const char *sec_name, char *buf, size_t buf_= sz) { - static const char *pfxs[] =3D { ".data", ".rodata", ".bss", ".kconfig" }; + static const char *pfxs[] =3D { ".data", ".rodata", ".bss", ".percpu", ".= kconfig" }; int i, n; =20 /* recognize hard coded LLVM section name */ @@ -254,7 +260,7 @@ static const struct btf_type *find_type_for_map(struct = btf *btf, const char *map return NULL; } =20 -static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t s= z) +static bool is_skel_data(const struct bpf_map *map, char *buf, size_t sz) { size_t tmp_sz; =20 @@ -263,13 +269,24 @@ static bool is_mmapable_map(const struct bpf_map *map= , char *buf, size_t sz) return true; } =20 - if (!bpf_map__is_internal(map) || !(bpf_map__map_flags(map) & BPF_F_MMAPA= BLE)) + if (!bpf_map__is_internal(map)) return false; =20 if (!get_map_ident(map, buf, sz)) return false; =20 - return true; + if (bpf_map__map_flags(map) & BPF_F_MMAPABLE) + return true; + + if (bpf_map__type(map) =3D=3D BPF_MAP_TYPE_PERCPU_ARRAY) + return bpf_map__btf_value_type_id(map) !=3D 0; + + return false; +} + +static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t s= z) +{ + return is_skel_data(map, buf, sz) && bpf_map__type(map) !=3D BPF_MAP_TYPE= _PERCPU_ARRAY; } =20 static int codegen_datasecs(struct bpf_object *obj, const char *obj_name) @@ -287,7 +304,7 @@ static int codegen_datasecs(struct bpf_object *obj, con= st char *obj_name) =20 bpf_object__for_each_map(map, obj) { /* only generate definitions for memory-mapped internal maps */ - if (!is_mmapable_map(map, map_ident, sizeof(map_ident))) + if (!is_skel_data(map, map_ident, sizeof(map_ident))) continue; =20 sec =3D find_type_for_map(btf, map_ident); @@ -517,7 +534,7 @@ static void codegen_asserts(struct bpf_object *obj, con= st char *obj_name) ", obj_name); =20 bpf_object__for_each_map(map, obj) { - if (!is_mmapable_map(map, map_ident, sizeof(map_ident))) + if (!is_skel_data(map, map_ident, sizeof(map_ident))) continue; =20 sec =3D find_type_for_map(btf, map_ident); @@ -668,8 +685,7 @@ static void codegen_destroy(struct bpf_object *obj, con= st char *obj_name) bpf_object__for_each_map(map, obj) { if (!get_map_ident(map, ident, sizeof(ident))) continue; - if (bpf_map__is_internal(map) && - (bpf_map__map_flags(map) & BPF_F_MMAPABLE)) + if (is_skel_data(map, ident, sizeof(ident))) printf("\tskel_free_map_data(skel->%1$s, skel->maps.%1$s.initial_value,= %2$zu);\n", ident, bpf_map_mmap_sz(map)); codegen("\ @@ -741,7 +757,7 @@ static int gen_trace(struct bpf_object *obj, const char= *obj_name, const char *h const void *mmap_data =3D NULL; size_t mmap_size =3D 0; =20 - if (!is_mmapable_map(map, ident, sizeof(ident))) + if (!is_skel_data(map, ident, sizeof(ident))) continue; =20 codegen("\ @@ -849,9 +865,23 @@ static int gen_trace(struct bpf_object *obj, const cha= r *obj_name, const char *h bpf_object__for_each_map(map, obj) { const char *mmap_flags; =20 - if (!is_mmapable_map(map, ident, sizeof(ident))) + if (!is_skel_data(map, ident, sizeof(ident))) continue; =20 + if (bpf_map__type(map) =3D=3D BPF_MAP_TYPE_PERCPU_ARRAY) { + codegen("\ + \n\ + err =3D skel_protect_map_data(skel->%1$s, &skel->maps.%1$s.initial_valu= e, %2$zd);\n\ + if (err) \n\ + return err; \n\ + #ifdef __KERNEL__ \n\ + skel->%1$s =3D NULL; \n\ + #endif \n\ + ", + ident, bpf_map_mmap_sz(map)); + continue; + } + if (bpf_map__map_flags(map) & BPF_F_RDONLY_PROG) mmap_flags =3D "PROT_READ"; else @@ -955,8 +985,7 @@ codegen_maps_skeleton(struct bpf_object *obj, size_t ma= p_cnt, bool mmaped, bool map->map =3D &obj->maps.%s; \n\ ", i, bpf_map__name(map), ident); - /* memory-mapped internal maps */ - if (mmaped && is_mmapable_map(map, ident, sizeof(ident))) { + if (mmaped && is_skel_data(map, ident, sizeof(ident))) { printf("\tmap->mmaped =3D (void **)&obj->%s;\n", ident); } =20 diff --git a/tools/lib/bpf/skel_internal.h b/tools/lib/bpf/skel_internal.h index 53fee53d36d5..1f3f332dffbe 100644 --- a/tools/lib/bpf/skel_internal.h +++ b/tools/lib/bpf/skel_internal.h @@ -131,8 +131,10 @@ static inline void skel_free_map_data(void *p, __u64 a= ddr, size_t sz) { if (addr !=3D ~0ULL) kvfree(p); - /* When addr =3D=3D ~0ULL the 'p' points to - * ((struct bpf_array *)map)->value. See skel_finalize_map_data. + /* + * When addr =3D=3D ~0ULL the init buffer has already been released. + * For skel_finalize_map_data(), 'p' points to + * ((struct bpf_array *)map)->value. */ } =20 @@ -170,6 +172,15 @@ static inline void *skel_finalize_map_data(__u64 *init= _val, size_t mmap_sz, int return addr; } =20 +static inline int skel_protect_map_data(void *p, __u64 *init_val, size_t s= z) +{ + (void)sz; + + kvfree(p); + *init_val =3D ~0ULL; + return 0; +} + #else =20 static inline void *skel_alloc(size_t size) @@ -208,6 +219,15 @@ static inline void *skel_finalize_map_data(__u64 *init= _val, size_t mmap_sz, int return NULL; return addr; } + +static inline int skel_protect_map_data(void *p, __u64 *init_val, size_t s= z) +{ + (void)init_val; + + if (mprotect(p, sz, PROT_READ)) + return -errno; + return 0; +} #endif =20 static inline int skel_closenz(int fd) --=20 2.55.0 From nobody Tue Sep 29 01:59:56 2026 Received: from mta0.migadu.com (out-157.mta0.migadu.com [91.218.175.157]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59AC83BE650 for ; Thu, 13 Aug 2026 15:24:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.157 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634693; cv=none; b=LwElXplkDup4qgK3/WU+pP3sUo0IePeGmX6gc+m8gO2CgggMmOoFKfJGWt1MJQ8swo+gMLWQ8Tkut3/6gRjCz5G+CIs8RwN9UlFaFISLl2Kk/lG2d5yq7IvQhwpmigICAa8OU3NmEAYr2wN1koIBzQ0o64KlhOlMlq4Nob9hzd0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634693; c=relaxed/simple; bh=RBxBBkj1k7nfJOBGOm3B3B7ITIfark1MfJS6GJ2p0cw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NNJQRw59HNQ37Xxyc+iVaS1Mq746ng9fElW6lkexb3mezMiem2SON0PlRHmLN+VQ85yGAWNcVNHJd2+l7hYC2lAk8EhAb3GkqpdAk2yLFly0VZSUgJf9hqo+lpvfMjZhqzJJOAT59rfcTq98tVrBZuCXdhjjmcOpwETJ/htGNvU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=eUgCa6OK; arc=none smtp.client-ip=91.218.175.157 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="eUgCa6OK" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=RBxBBkj1k7nfJOBGOm3B3B7ITIfark1MfJS6GJ2p0cw=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634688; v=1; x=1787239488; b=eUgCa6OKch9mChKDnFaJNc4wk5G1Qedkra7/ZNsD1Hg4ixaeuV7QIoqBT+xjn41v9NNvLOli u/DR8mOV1rOpSedHJIkfJkIYeeuNFqpK25j+r2pTeLNJD7kWafrNh0Yxz22TAEe3WjIfDkVfiMR BMC6PknD3dgL8zreX2KjIvz4= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id 774565b0785bbd38; Thu, 13 Aug 2026 15:24:48 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com Subject: [PATCH bpf-next v12 07/10] selftests/bpf: Add tests to verify global percpu data Date: Thu, 13 Aug 2026 23:23:20 +0800 Message-ID: <20260813152324.97937-8-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If the arch, like s390x, does not support percpu insn, these cases won't test global percpu data by checking FEAT_PERCPU_DATA support. The following APIs have been tested for global percpu data: 1. bpf_map__set_initial_value() 2. bpf_map__initial_value() 3. bpf_map__set_value_size() 4. generated percpu struct pointer pointing to internal map's mmaped data 5. bpf_map__lookup_elem() for global percpu data map 6. bpf_map_lookup_elem_flags() for global percpu data map At the same time, the case is also tested with 'bpftool gen skeleton -L'. Assisted-by: Codex:gpt-5.5-xhigh Signed-off-by: Leon Hwang --- tools/testing/selftests/bpf/Makefile | 2 +- .../bpf/prog_tests/global_data_init.c | 188 ++++++++++++++++++ .../bpf/progs/test_global_percpu_data.c | 41 ++++ 3 files changed, 230 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/test_global_percpu_da= ta.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests= /bpf/Makefile index d3655a706482..560ce4016fbf 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -531,7 +531,7 @@ LSKELS_SIGNED :=3D fentry_test.c fexit_test.c atomics.c =20 # Generate both light skeleton and libbpf skeleton for these LSKELS_EXTRA :=3D test_ksyms_module.c test_ksyms_weak.c kfunc_call_test.c \ - kfunc_call_test_subprog.c + kfunc_call_test_subprog.c test_global_percpu_data.c SKEL_BLACKLIST +=3D $$(LSKELS) $$(LSKELS_SIGNED) =20 test_static_linked.skel.h-deps :=3D test_static_linked1.bpf.o test_static_= linked2.bpf.o diff --git a/tools/testing/selftests/bpf/prog_tests/global_data_init.c b/to= ols/testing/selftests/bpf/prog_tests/global_data_init.c index 8466332d7406..06d163a022dc 100644 --- a/tools/testing/selftests/bpf/prog_tests/global_data_init.c +++ b/tools/testing/selftests/bpf/prog_tests/global_data_init.c @@ -1,5 +1,8 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include "bpf/libbpf_internal.h" +#include "test_global_percpu_data.skel.h" +#include "test_global_percpu_data.lskel.h" =20 void test_global_data_init(void) { @@ -60,3 +63,188 @@ void test_global_data_init(void) free(newval); bpf_object__close(obj); } + +static void test_percpu_data_on_cpus(struct bpf_map *map, int map_fd, int = prog_fd, int *runp) +{ + struct test_global_percpu_data__percpu *data =3D NULL; + int i, err, key =3D 0, num_online, run =3D 0; + __u64 args[2] =3D {0x1234ULL, 0x5678ULL}; + size_t data_sz; + bool *online; + LIBBPF_OPTS(bpf_test_run_opts, topts, + .ctx_in =3D args, + .ctx_size_in =3D sizeof(args), + .flags =3D BPF_F_TEST_RUN_ON_CPU, + ); + + err =3D parse_cpu_mask_file("/sys/devices/system/cpu/online", &online, &n= um_online); + if (!ASSERT_OK(err, "parse_cpu_mask_file")) + return; + + data_sz =3D map ? bpf_map__value_size(map) : sizeof(*data); + data =3D calloc(1, data_sz); + if (!ASSERT_OK_PTR(data, "calloc percpu data")) + goto out; + + /* run on every online-CPU */ + for (i =3D 0; i < num_online; i++) { + __u64 flags; + + if (!online[i]) + continue; + + topts.cpu =3D i; + topts.retval =3D -1; + err =3D bpf_prog_test_run_opts(prog_fd, &topts); + ASSERT_OK(err, "bpf_prog_test_run_opts"); + ASSERT_EQ(topts.retval, 0, "bpf_prog_test_run_opts retval"); + + memset(data, 0, data_sz); + flags =3D ((__u64) i << 32) | BPF_F_CPU; + if (map) + err =3D bpf_map__lookup_elem(map, &key, sizeof(key), data, data_sz, fla= gs); + else + err =3D bpf_map_lookup_elem_flags(map_fd, &key, data, flags); + if (!ASSERT_OK(err, "lookup_elem on cpu")) + break; + + ASSERT_EQ(*runp, ++run, "run"); + ASSERT_EQ(data->cpu_id[0], i, "cpu_id"); + ASSERT_EQ(data->data, 1, "data"); + ASSERT_TRUE(data->set, "set"); + ASSERT_EQ(data->nums[6], 0xc0de, "nums[6]"); + ASSERT_EQ(data->struct_data.i, 1, "struct_data.i"); + ASSERT_TRUE(data->struct_data.set, "struct_data.set"); + ASSERT_EQ(data->struct_data.nums[6], 0xc0de, "struct_data.nums[6]"); + } + +out: + free(data); + free(online); +} + +static void test_global_percpu_data_init(void) +{ + struct test_global_percpu_data__percpu init_value =3D {}; + struct test_global_percpu_data__percpu *init_data; + const __u32 desired_sz =3D sysconf(_SC_PAGE_SIZE); + struct test_global_percpu_data *skel =3D NULL; + size_t init_data_sz; + struct bpf_map *map; + int prog_fd, err; + + skel =3D test_global_percpu_data__open(); + if (!ASSERT_OK_PTR(skel, "test_global_percpu_data__open")) + goto out; + if (!ASSERT_OK_PTR(skel->percpu, "skel->percpu")) + goto out; + if (!ASSERT_OK_PTR(skel->data_percpu, "skel->data_percpu")) + goto out; + if (!ASSERT_OK_PTR(skel->percpu_data, "skel->percpu_data")) + goto out; + if (!ASSERT_OK_PTR(skel->percpu_looooooooong, "skel->percpu_looooooooong"= )) + goto out; + + ASSERT_STREQ(bpf_map__name(skel->maps.percpu_data), ".percpu.data", + ".percpu.data map name"); + ASSERT_STREQ(bpf_map__name(skel->maps.data_percpu), ".data.percpu", + ".data.percpu map name"); + ASSERT_STREQ(bpf_map__name(skel->maps.percpu_looooooooong), ".percpu.looo= oooooong", + "long map name"); + ASSERT_STREQ(bpf_map__name(skel->maps.percpu), ".percpu", "map name"); + ASSERT_EQ(skel->percpu->data, -1, "skel->percpu->data"); + ASSERT_FALSE(skel->percpu->set, "skel->percpu->set"); + ASSERT_EQ(skel->percpu->nums[6], 0, "skel->percpu->nums[6]"); + ASSERT_EQ(skel->percpu->struct_data.i, -1, "struct_data.i"); + ASSERT_FALSE(skel->percpu->struct_data.set, "struct_data.set"); + ASSERT_EQ(skel->percpu->struct_data.nums[6], 0, "struct_data.nums[6]"); + + map =3D skel->maps.percpu; + if (!ASSERT_EQ(bpf_map__type(map), BPF_MAP_TYPE_PERCPU_ARRAY, "bpf_map__t= ype")) + goto out; + + init_value.data =3D 2; + init_value.nums[6] =3D -1; + init_value.struct_data.i =3D 2; + init_value.struct_data.nums[6] =3D -1; + err =3D bpf_map__set_initial_value(map, &init_value, sizeof(init_value)); + if (!ASSERT_OK(err, "bpf_map__set_initial_value")) + goto out; + + init_data =3D bpf_map__initial_value(map, &init_data_sz); + if (!ASSERT_OK_PTR(init_data, "bpf_map__initial_value")) + goto out; + + ASSERT_EQ(init_data->data, init_value.data, "init_value data"); + ASSERT_EQ(init_data->set, init_value.set, "init_value set"); + ASSERT_EQ(init_data->struct_data.i, init_value.struct_data.i, "init_value= struct_data.i"); + ASSERT_EQ(init_data->struct_data.nums[6], init_value.struct_data.nums[6], + "init_value struct_data.nums[6]"); + ASSERT_EQ(init_data_sz, sizeof(init_value), "init_value size"); + ASSERT_EQ((void *) init_data, (void *) skel->percpu, "skel->percpu eq ini= t_data"); + ASSERT_EQ(skel->percpu->data, init_value.data, "skel->percpu->data"); + ASSERT_EQ(skel->percpu->set, init_value.set, "skel->percpu->set"); + ASSERT_EQ(skel->percpu->struct_data.i, init_value.struct_data.i, + "skel->percpu->struct_data.i"); + ASSERT_EQ(skel->percpu->struct_data.nums[6], init_value.struct_data.nums[= 6], + "skel->percpu->struct_data.nums[6]"); + + ASSERT_GT(desired_sz, sizeof(init_value), "desired_sz"); + err =3D bpf_map__set_value_size(map, desired_sz); + if (!ASSERT_OK(err, "bpf_map__set_value_size")) + goto out; + if (!ASSERT_EQ(bpf_map__value_size(map), desired_sz, "percpu value size")) + goto out; + if (!ASSERT_NEQ(bpf_map__btf_value_type_id(map), 0, "percpu BTF value typ= e")) + goto out; + + init_data =3D bpf_map__initial_value(map, &init_data_sz); + if (!ASSERT_OK_PTR(init_data, "resized bpf_map__initial_value")) + goto out; + if (!ASSERT_EQ(init_data_sz, desired_sz, "resized initial value size")) + goto out; + if (!ASSERT_EQ(init_data->data, init_value.data, "resized initial value d= ata")) + goto out; + + err =3D test_global_percpu_data__load(skel); + if (!ASSERT_OK(err, "test_global_percpu_data__load")) + goto out; + + ASSERT_OK_PTR(skel->percpu, "skel->percpu"); + + prog_fd =3D bpf_program__fd(skel->progs.update_percpu_data); + test_percpu_data_on_cpus(map, bpf_map__fd(map), prog_fd, &skel->bss->run); + +out: + test_global_percpu_data__destroy(skel); +} + +static void test_global_percpu_data_lskel(void) +{ + struct test_global_percpu_data_lskel *lskel =3D NULL; + int prog_fd, map_fd; + + lskel =3D test_global_percpu_data_lskel__open_and_load(); + if (!ASSERT_OK_PTR(lskel, "test_global_percpu_data_lskel__open_and_load")) + goto out; + + map_fd =3D lskel->maps.percpu.map_fd; + prog_fd =3D lskel->progs.update_percpu_data.prog_fd; + test_percpu_data_on_cpus(NULL, map_fd, prog_fd, &lskel->bss->run); + +out: + test_global_percpu_data_lskel__destroy(lskel); +} + +void test_global_percpu_data(void) +{ + if (!feat_supported(NULL, FEAT_PERCPU_DATA)) { + test__skip(); + return; + } + + if (test__start_subtest("init")) + test_global_percpu_data_init(); + if (test__start_subtest("lskel")) + test_global_percpu_data_lskel(); +} diff --git a/tools/testing/selftests/bpf/progs/test_global_percpu_data.c b/= tools/testing/selftests/bpf/progs/test_global_percpu_data.c new file mode 100644 index 000000000000..416841cd3569 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_global_percpu_data.c @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include "bpf_misc.h" + +/* Used for testing map name. */ +int loong SEC(".percpu.looooooooong"); +int data3 SEC(".data.percpu"); +int data2 SEC(".percpu.data"); + +int run; +/* cpu_id as array to verify map value resizing. */ +int cpu_id[1] SEC(".percpu"); +int data SEC(".percpu") =3D -1; +int nums[7] SEC(".percpu"); +bool set SEC(".percpu") =3D false; +struct { + char set; + int i; + int nums[7]; +} struct_data SEC(".percpu") =3D { + .set =3D 0, + .i =3D -1, +}; + +SEC("raw_tp/task_rename") +__auxiliary +int update_percpu_data(void *ctx) +{ + struct_data.nums[6] =3D 0xc0de; + struct_data.set =3D 1; + struct_data.i =3D 1; + nums[6] =3D 0xc0de; + data =3D 1; + run++; + set =3D true; + cpu_id[0] =3D bpf_get_smp_processor_id(); + return 0; +} + +char _license[] SEC("license") =3D "GPL"; --=20 2.55.0 From nobody Tue Sep 29 01:59:56 2026 Received: from mta1.migadu.com (out-1.mta1.migadu.com [95.215.58.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A8B648989E for ; Thu, 13 Aug 2026 15:25:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634703; cv=none; b=aU+ymJ/0Ev7WuO+oQdy/pZe6D4jzDss2/aAOEmPW1FGxiHHivnDKx+WEOllX3WVCD4Do9pEqeM08KASKKpUwtNoohrbCt/hXQdofIxK0ewu1GM2TW12BA5wYMR3ipKNyFlrTT4ZzuXIY721Rhx/yVZerkt42nWg5+f2iHLDMLPA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634703; c=relaxed/simple; bh=UJsx5keYjMIXh3q0QSQp1PpllH7J02tmA0pv68zLRDA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JO1/oGbnLvcOsTEMM85Fer2ko2Wuo4toi3LjJiZPJcymygYodFYR8TYjIoAY2Goh6zsovsLjTMUVwCVGkU0Q0MzeJkd927V3t34vVT2+aUrUjwqCkMMRd9FTaxTnMUOw97NFyL23QAGHAHE65x7ezrmR/Xdbia+gZFg0ZuOS7L0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Lx5WuJqy; arc=none smtp.client-ip=95.215.58.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Lx5WuJqy" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=UJsx5keYjMIXh3q0QSQp1PpllH7J02tmA0pv68zLRDA=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634699; v=1; x=1787239499; b=Lx5WuJqyuS6HyqcfDuISDPJJil/JZo6FwOzrQKyTRzKjpy3hs8aed9LhOZDi8kPzYPsZW+NN 3E+XlP0N8+GWz0UFXVJUvVMBDiLyI1XIsPfd9iPCE5zXv2qvEBkGQyL++tuNtLiTTilOcCrcSn1 OfEQKMvMNzMCYLCPjhTfLkMw= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id de3dc4ec878090e5; Thu, 13 Aug 2026 15:24:59 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com Subject: [PATCH bpf-next v12 08/10] selftests/bpf: Test direct reading/writing read-only percpu_array map Date: Thu, 13 Aug 2026 23:23:21 +0800 Message-ID: <20260813152324.97937-9-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Verify these two cases: 1. Direct reading the data of read-only percpu data's percpu_array map is allowed. 2. Direct writing the data of read-only percpu data's percpu_array map is disallowed. Assisted-by: Codex:gpt-5.5-xhigh Signed-off-by: Leon Hwang --- .../bpf/prog_tests/global_data_init.c | 90 +++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/global_data_init.c b/to= ols/testing/selftests/bpf/prog_tests/global_data_init.c index 06d163a022dc..9688b417a90b 100644 --- a/tools/testing/selftests/bpf/prog_tests/global_data_init.c +++ b/tools/testing/selftests/bpf/prog_tests/global_data_init.c @@ -236,6 +236,92 @@ static void test_global_percpu_data_lskel(void) test_global_percpu_data_lskel__destroy(lskel); } =20 +static int create_rdonly_percpu_array(void) +{ + LIBBPF_OPTS(bpf_map_create_opts, map_opts, + .map_flags =3D BPF_F_RDONLY_PROG, + ); + int key =3D 0, map_fd, err; + __u64 value =3D 0; + + map_fd =3D bpf_map_create(BPF_MAP_TYPE_PERCPU_ARRAY, "percpu_ro_map", siz= eof(int), + sizeof(__u64), 1, &map_opts); + if (!ASSERT_GE(map_fd, 0, "bpf_map_create")) + return -1; + + err =3D bpf_map_update_elem(map_fd, &key, &value, BPF_F_ALL_CPUS); + if (!ASSERT_OK(err, "bpf_map_update_elem")) + goto out; + + err =3D bpf_map_freeze(map_fd); + if (!ASSERT_OK(err, "bpf_map_freeze")) + goto out; + + return map_fd; + +out: + close(map_fd); + return -1; +} + +static void test_global_percpu_data_rdonly_direct_read(void) +{ + /* + * Raw instructions with manually prepared rdonly percpu_array map + * for testing direct-read global percpu data, because libbpf + * doesn't have rdonly internal percpu_array map support for + * global percpu data. + */ + struct bpf_insn insns[] =3D { + BPF_LD_MAP_VALUE(BPF_REG_1, 0, 0), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 0), + BPF_EXIT_INSN(), + }; + int map_fd, prog_fd; + + map_fd =3D create_rdonly_percpu_array(); + if (map_fd < 0) + return; + + insns[0].imm =3D map_fd; + prog_fd =3D bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, "percpu_ro_prog", = "GPL", insns, + ARRAY_SIZE(insns), NULL); + if (ASSERT_GE(prog_fd, 0, "bpf_prog_load")) + close(prog_fd); + close(map_fd); +} + +static void test_global_percpu_data_rdonly_direct_write(void) +{ + LIBBPF_OPTS(bpf_prog_load_opts, prog_opts); + /* See the comment in test_global_percpu_data_rdonly_direct_read() */ + struct bpf_insn insns[] =3D { + BPF_LD_MAP_VALUE(BPF_REG_1, 0, 0), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 0), + BPF_ST_MEM(BPF_DW, BPF_REG_1, 0, 0), + BPF_EXIT_INSN(), + }; + char log_buf[256] =3D {}; + int map_fd, prog_fd; + + prog_opts.log_buf =3D log_buf; + prog_opts.log_size =3D sizeof(log_buf); + prog_opts.log_level =3D 1; + + map_fd =3D create_rdonly_percpu_array(); + if (map_fd < 0) + return; + + insns[0].imm =3D map_fd; + prog_fd =3D bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, "percpu_ro_prog", = "GPL", insns, + ARRAY_SIZE(insns), &prog_opts); + if (!ASSERT_LT(prog_fd, 0, "bpf_prog_load")) + close(prog_fd); + else + ASSERT_HAS_SUBSTR(log_buf, "write into map forbidden", "verifier log"); + close(map_fd); +} + void test_global_percpu_data(void) { if (!feat_supported(NULL, FEAT_PERCPU_DATA)) { @@ -247,4 +333,8 @@ void test_global_percpu_data(void) test_global_percpu_data_init(); if (test__start_subtest("lskel")) test_global_percpu_data_lskel(); + if (test__start_subtest("rdonly_direct_read")) + test_global_percpu_data_rdonly_direct_read(); + if (test__start_subtest("rdonly_direct_write")) + test_global_percpu_data_rdonly_direct_write(); } --=20 2.55.0 From nobody Tue Sep 29 01:59:56 2026 Received: from mta0.migadu.com (out-169.mta0.migadu.com [91.218.175.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8F2F43634E for ; Thu, 13 Aug 2026 15:25:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634712; cv=none; b=WZLzMzlt7yTwc9VzoGSRRGKFro2oohVUwdj9dpQPmIDC6WHlF9sHWoPT4gyfJlB8uFRpCn80PEuwz8DVM4EJENfFrejeeQDFPC11iTXlEk+eqxOtdqJDW7C7nZB7J8HXFXgW5Pk87SYTvXFRUa39HRW1gJZ79FDJ7DcEpmZRyfo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634712; c=relaxed/simple; bh=ugn5KW5d0MnDrXEms3ZV9ewV8YufdJ3J2dZdBo1FPRI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VDwbLCMpbuxcA17dGC95VOllfMZ5Dh2IWMShYNi/5Y3KZmLa656k/dtqER1iztG7GInC/s7nlXkj++Lgcl3d/kLzbONOCaTLZtxDo5kUtUoRga55mJC3YPtgGdS0dMTd07xT8ZkXobMDSqjqk7pcvLcKYlLS6iRigppnd2/SoFs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=xGlXLnoQ; arc=none smtp.client-ip=91.218.175.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="xGlXLnoQ" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=ugn5KW5d0MnDrXEms3ZV9ewV8YufdJ3J2dZdBo1FPRI=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634709; v=1; x=1787239509; b=xGlXLnoQXXwfkcA3h4/hZVRyiDBuN3Fult+zpzD5Wbc2/0SttuVANKCvPZFTPfU4OIUPwHcd +I9TtGbL3694J4OCk/wWjV0UGSvvh+0vvwEShJUyFVqoZ0GMrdjNYLjWGoZJl01c4UFoct0pCkF AiEENlkiIzOjZR8sk0vIDouY= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id fde2fdddc889e8f1; Thu, 13 Aug 2026 15:25:08 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com, Emil Tsalapatis Subject: [PATCH bpf-next v12 09/10] selftests/bpf: Test verifier log for global percpu data Date: Thu, 13 Aug 2026 23:23:22 +0800 Message-ID: <20260813152324.97937-10-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add two tests to verify the verifier log "R%d points to percpu_array map which cannot be used as const string\n". Reviewed-by: Emil Tsalapatis Signed-off-by: Leon Hwang --- .../bpf/prog_tests/global_data_init.c | 6 +++++ .../bpf/progs/test_global_percpu_data.c | 23 +++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/global_data_init.c b/to= ols/testing/selftests/bpf/prog_tests/global_data_init.c index 9688b417a90b..d308ca3b3045 100644 --- a/tools/testing/selftests/bpf/prog_tests/global_data_init.c +++ b/tools/testing/selftests/bpf/prog_tests/global_data_init.c @@ -322,6 +322,11 @@ static void test_global_percpu_data_rdonly_direct_writ= e(void) close(map_fd); } =20 +static void test_global_percpu_data_verifier_log(void) +{ + RUN_TESTS(test_global_percpu_data); +} + void test_global_percpu_data(void) { if (!feat_supported(NULL, FEAT_PERCPU_DATA)) { @@ -337,4 +342,5 @@ void test_global_percpu_data(void) test_global_percpu_data_rdonly_direct_read(); if (test__start_subtest("rdonly_direct_write")) test_global_percpu_data_rdonly_direct_write(); + test_global_percpu_data_verifier_log(); } diff --git a/tools/testing/selftests/bpf/progs/test_global_percpu_data.c b/= tools/testing/selftests/bpf/progs/test_global_percpu_data.c index 416841cd3569..d086e9417f9f 100644 --- a/tools/testing/selftests/bpf/progs/test_global_percpu_data.c +++ b/tools/testing/selftests/bpf/progs/test_global_percpu_data.c @@ -38,4 +38,27 @@ int update_percpu_data(void *ctx) return 0; } =20 +static const char fmt[] SEC(".percpu.fmt") =3D "data %d\n"; + +SEC("?kprobe") +__failure __msg("R{{[0-9]+}} points to percpu_array map which cannot be us= ed as const string") +int verifier_strncmp(void *ctx) +{ + return bpf_strncmp("test", 5, fmt); +} + +SEC("?kprobe") +__failure __msg("R{{[0-9]+}} points to percpu_array map which cannot be us= ed as const string") +int verifier_snprintf(void *ctx) +{ + u64 args[] =3D { data }; + char buf[128]; + int len; + + len =3D bpf_snprintf(buf, sizeof(buf), fmt, args, sizeof(args)); + if (len > 0) + bpf_printk("snprintf: %s\n", buf); + return 0; +} + char _license[] SEC("license") =3D "GPL"; --=20 2.55.0 From nobody Tue Sep 29 01:59:56 2026 Received: from mta1.migadu.com (out-13.mta1.migadu.com [95.215.58.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0ADF388391 for ; Thu, 13 Aug 2026 15:25:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634729; cv=none; b=imOmIa+JCd8pcsPk8KsmjAgZzg4wIeXlNJpphsnLGmidsWjVR+fQbffvd6NgrGI1vhai68Ko1kM0emJEZFFJ1m/E0aUqWrZ1GxDA4ZwGfqNT4Kz5a0ReYT/v+mpSUO1ImVs/ZLw+xrdiG7tCIKYXbfKy30+aIG5Q9U/B+g9r6D4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786634729; c=relaxed/simple; bh=RsqbY0RCU8Ls/knpKMfiJGL0HaQv4wOyuSTzrYc79yg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TLDtW6nuqCqE/1LvjRn5w597JbyaaXhh2oofisDgKgFmYHDrFP1iNvMU6UVwvbbLryPkUWQCNfMX1A39GSV2+IUonBprihPk5ctaygPV65kt9TMadWlndqWm9VcfCVKlrS39/u4XPq9kDvtxPEeujt9MpGO8IN5ze/QyjqSYFH8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=ookBvmUR; arc=none smtp.client-ip=95.215.58.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="ookBvmUR" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=RsqbY0RCU8Ls/knpKMfiJGL0HaQv4wOyuSTzrYc79yg=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1786634725; v=1; x=1787239525; b=ookBvmURlKNcUxFFgiP/izZmaZnAiKgfVwLlaidDbfgo1h3vhuYMXDJz5XyfdCeoZB2PMNsT cjKRKynyx52wSZTRepe+5od4IjbLbO2lh3VMzfmhQkIW2VZcHvrSZ2MZcKygNovAqHT1R3SFDbq SLaysfAvI7jhjL5yxLeD0Nys= X-Envelope-To: linux-kernel@vger.kernel.org Received: from .linux.dev (117.20.149.103) by smtp.migadu.com with ESMTPS id 3e806838460a431b; Thu, 13 Aug 2026 15:25:25 +0000 X-Migadu-Flow: FLOW_OUT From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , John Fastabend , Quentin Monnet , Shuah Khan , Leon Hwang , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com, Emil Tsalapatis Subject: [PATCH bpf-next v12 10/10] selftests/bpf: Verify bpf_iter for global percpu data Date: Thu, 13 Aug 2026 23:23:23 +0800 Message-ID: <20260813152324.97937-11-leon.hwang@linux.dev> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813152324.97937-1-leon.hwang@linux.dev> References: <20260813152324.97937-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a test to verify that it is OK to iter the percpu_array map used for global percpu data. Reviewed-by: Emil Tsalapatis Signed-off-by: Leon Hwang --- .../bpf/prog_tests/global_data_init.c | 52 +++++++++++++++++++ .../bpf/progs/test_global_percpu_data.c | 25 +++++++++ 2 files changed, 77 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/global_data_init.c b/to= ols/testing/selftests/bpf/prog_tests/global_data_init.c index d308ca3b3045..7d6bda909295 100644 --- a/tools/testing/selftests/bpf/prog_tests/global_data_init.c +++ b/tools/testing/selftests/bpf/prog_tests/global_data_init.c @@ -327,6 +327,56 @@ static void test_global_percpu_data_verifier_log(void) RUN_TESTS(test_global_percpu_data); } =20 +static void test_global_percpu_data_iter(void) +{ + DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); + struct test_global_percpu_data *skel; + union bpf_iter_link_info linfo =3D {}; + struct bpf_link *link =3D NULL; + int fd, num_cpus, len, err; + char buf[16]; + + num_cpus =3D libbpf_num_possible_cpus(); + if (!ASSERT_GT(num_cpus, 0, "libbpf_num_possible_cpus")) + return; + + skel =3D test_global_percpu_data__open(); + if (!ASSERT_OK_PTR(skel, "test_global_percpu_data__open")) + return; + + skel->rodata->num_cpus =3D num_cpus; + skel->rodata->offsetof_num =3D offsetof(struct test_global_percpu_data__p= ercpu, struct_data); + skel->rodata->offsetof_num +=3D sizeof(skel->percpu->struct_data) - sizeo= f(int); + skel->rodata->elem_sz =3D roundup(sizeof(struct test_global_percpu_data__= percpu), 8); + skel->percpu->struct_data.nums[6] =3D 0xc0de; + + err =3D test_global_percpu_data__load(skel); + if (!ASSERT_OK(err, "test_global_percpu_data__load")) + goto out; + + linfo.map.map_fd =3D bpf_map__fd(skel->maps.percpu); + opts.link_info =3D &linfo; + opts.link_info_len =3D sizeof(linfo); + link =3D bpf_program__attach_iter(skel->progs.dump_percpu_data, &opts); + if (!ASSERT_OK_PTR(link, "bpf_program__attach_iter")) + goto out; + + fd =3D bpf_iter_create(bpf_link__fd(link)); + if (!ASSERT_GE(fd, 0, "bpf_iter_create")) + goto out; + + while ((len =3D read(fd, buf, sizeof(buf))) > 0) + do { } while (0); + ASSERT_EQ(len, 0, "read iter"); + ASSERT_TRUE(skel->bss->run_iter, "run_iter"); + ASSERT_EQ(skel->bss->percpu_data_sum, 0xc0de * num_cpus, "percpu_data_sum= "); + + close(fd); +out: + bpf_link__destroy(link); + test_global_percpu_data__destroy(skel); +} + void test_global_percpu_data(void) { if (!feat_supported(NULL, FEAT_PERCPU_DATA)) { @@ -343,4 +393,6 @@ void test_global_percpu_data(void) if (test__start_subtest("rdonly_direct_write")) test_global_percpu_data_rdonly_direct_write(); test_global_percpu_data_verifier_log(); + if (test__start_subtest("iter")) + test_global_percpu_data_iter(); } diff --git a/tools/testing/selftests/bpf/progs/test_global_percpu_data.c b/= tools/testing/selftests/bpf/progs/test_global_percpu_data.c index d086e9417f9f..71ff8d1bf49e 100644 --- a/tools/testing/selftests/bpf/progs/test_global_percpu_data.c +++ b/tools/testing/selftests/bpf/progs/test_global_percpu_data.c @@ -61,4 +61,29 @@ int verifier_snprintf(void *ctx) return 0; } =20 +volatile const __u32 num_cpus =3D 0; +volatile const int offsetof_num; +volatile const int elem_sz; +__u32 percpu_data_sum =3D 0; +bool run_iter =3D false; + +SEC("iter/bpf_map_elem") +__auxiliary +int dump_percpu_data(struct bpf_iter__bpf_map_elem *ctx) +{ + void *pptr =3D ctx->value; + int i; + + if (!pptr) + return 0; + + run_iter =3D true; + + for (i =3D 0; i < num_cpus; i++) { + percpu_data_sum +=3D *(int *) (pptr + offsetof_num); + pptr +=3D elem_sz; + } + return 0; +} + char _license[] SEC("license") =3D "GPL"; --=20 2.55.0