From nobody Sun May 10 09:55:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79667C433F5 for ; Fri, 13 May 2022 06:40:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1377415AbiEMGkU (ORCPT ); Fri, 13 May 2022 02:40:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1377407AbiEMGkS (ORCPT ); Fri, 13 May 2022 02:40:18 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D58372A1FFF for ; Thu, 12 May 2022 23:40:15 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id i1so7069819plg.7 for ; Thu, 12 May 2022 23:40:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LaI3WoJGVt28cor8mzjFeKlCPw870xKzF3pmeMk/wKY=; b=1gJPWxMu6fAn74+52mUuSodGzlFvKFuEgMUqbloqp59WvJjb6xTEweI2Z2zHtrcM1s XcCoA/VNkH3+xi6merS67PIjAx7vGEu+Ex8xTCBKXrZELtYoEjkf/2zZn2fi480yInKQ 8ft4R9UOMFubrtb3EOsLOl3kl482io/vmLoWYE3Tq4/ucIVlF6WW0iGUISquIcL4WJMu LEktIpp2UFH3rkD/ZJ43/BMYlcTTA1w6vJaT6uDYk0vssCH0/LIv5y8Vq092OykSImZ5 2pJnrI0MV18GSVxlkYCW2u49VYOjA0PjdrLpbyV6BxixRRZ0g32XxTDJikcYWWetrg6z HcCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LaI3WoJGVt28cor8mzjFeKlCPw870xKzF3pmeMk/wKY=; b=FnkLfkqVEn0Kr1bdB+qwM6B2nav14WbmnWAUPyEN/nKskgKjr3ZN6kmgBiIifpMzxg TzHTp08jXm5uTjG572wkU9WYpS801CqUhQDicnrjmtfjOs5nEX8gxNGNdjkd5Mo5hNih KTu63vGfW9ZQOkheLi0PpACo+GJ0BqOrqAUX25WJmsAmZfwCQ7sL50tFgnn5S3iyPQMC pWKc4VrWY+CuynNsnq7+R3WWccMW6buSFMHg8QudTWkiT+CyBDQNmlkN8exD/Cbf7WMq T5omm5cSBg4syGuRqsfZZDq0InRnEWCEC9oqHbBftUxZtDPayAiAdyLT4OcW9up4RDJK 8H6Q== X-Gm-Message-State: AOAM531xk151S1q4VHmRjDOSK9StWfg+iVwxmdEZM/CCLGwdsUSRM/mr h50IEfhyrn/cg8uTXFw1NZ9ZMQ== X-Google-Smtp-Source: ABdhPJwhRI5OXkfDJBHmwcDGLkf0atj9iTbEFxYaDKmfJTbmm+P0hMhc+5y2dD73KgbEZAHWFS4DLg== X-Received: by 2002:a17:902:c7cb:b0:15e:e0d0:c020 with SMTP id r11-20020a170902c7cb00b0015ee0d0c020mr3207389pla.51.1652424015088; Thu, 12 May 2022 23:40:15 -0700 (PDT) Received: from C02F52LSML85.bytedance.net ([139.177.225.225]) by smtp.gmail.com with ESMTPSA id 185-20020a6204c2000000b0050dc76281d8sm934990pfe.178.2022.05.12.23.40.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 May 2022 23:40:14 -0700 (PDT) From: Feng zhou To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, rostedt@goodmis.org, mingo@redhat.com, jolsa@kernel.org, davemarchevsky@fb.com, joannekoong@fb.com, geliang.tang@suse.com Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, songmuchun@bytedance.com, wangdongdong.6@bytedance.com, cong.wang@bytedance.com, zhouchengming@bytedance.com, zhoufeng.zf@bytedance.com, yosryahmed@google.com Subject: [PATCH bpf-next v3 1/2] bpf: add bpf_map_lookup_percpu_elem for percpu map Date: Fri, 13 May 2022 14:39:51 +0800 Message-Id: <20220513063952.41794-2-zhoufeng.zf@bytedance.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20220513063952.41794-1-zhoufeng.zf@bytedance.com> References: <20220513063952.41794-1-zhoufeng.zf@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Feng Zhou Add new ebpf helpers bpf_map_lookup_percpu_elem. The implementation method is relatively simple, refer to the implementation method of map_lookup_elem of percpu map, increase the parameters of cpu, and obtain it according to the specified cpu. Signed-off-by: Feng Zhou --- include/linux/bpf.h | 2 ++ include/uapi/linux/bpf.h | 9 +++++++++ kernel/bpf/arraymap.c | 15 +++++++++++++++ kernel/bpf/core.c | 1 + kernel/bpf/hashtab.c | 32 ++++++++++++++++++++++++++++++++ kernel/bpf/helpers.c | 18 ++++++++++++++++++ kernel/bpf/verifier.c | 17 +++++++++++++++-- kernel/trace/bpf_trace.c | 2 ++ tools/include/uapi/linux/bpf.h | 9 +++++++++ 9 files changed, 103 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 3ded8711457f..5061ccd8b2dc 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -89,6 +89,7 @@ struct bpf_map_ops { int (*map_push_elem)(struct bpf_map *map, void *value, u64 flags); int (*map_pop_elem)(struct bpf_map *map, void *value); int (*map_peek_elem)(struct bpf_map *map, void *value); + void *(*map_lookup_percpu_elem)(struct bpf_map *map, void *key, u32 cpu); =20 /* funcs called by prog_array and perf_event_array map */ void *(*map_fd_get_ptr)(struct bpf_map *map, struct file *map_file, @@ -2184,6 +2185,7 @@ extern const struct bpf_func_proto bpf_map_delete_ele= m_proto; extern const struct bpf_func_proto bpf_map_push_elem_proto; extern const struct bpf_func_proto bpf_map_pop_elem_proto; extern const struct bpf_func_proto bpf_map_peek_elem_proto; +extern const struct bpf_func_proto bpf_map_lookup_percpu_elem_proto; =20 extern const struct bpf_func_proto bpf_get_prandom_u32_proto; extern const struct bpf_func_proto bpf_get_smp_processor_id_proto; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index bc7f89948f54..0210f85131b3 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -5164,6 +5164,14 @@ union bpf_attr { * if not NULL, is a reference which must be released using its * corresponding release function, or moved into a BPF map before * program exit. + * + * void *bpf_map_lookup_percpu_elem(struct bpf_map *map, const void *key, = u32 cpu) + * Description + * Perform a lookup in *percpu map* for an entry associated to + * *key* on *cpu*. + * Return + * Map value associated to *key* on *cpu*, or **NULL** if no entry + * was found or *cpu* is invalid. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -5361,6 +5369,7 @@ union bpf_attr { FN(skb_set_tstamp), \ FN(ima_file_hash), \ FN(kptr_xchg), \ + FN(map_lookup_percpu_elem), \ /* */ =20 /* integer value in 'imm' field of BPF_CALL instruction selects which help= er diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 724613da6576..fe40d3b9458f 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -243,6 +243,20 @@ static void *percpu_array_map_lookup_elem(struct bpf_m= ap *map, void *key) return this_cpu_ptr(array->pptrs[index & array->index_mask]); } =20 +static void *percpu_array_map_lookup_percpu_elem(struct bpf_map *map, void= *key, u32 cpu) +{ + struct bpf_array *array =3D container_of(map, struct bpf_array, map); + u32 index =3D *(u32 *)key; + + if (cpu >=3D nr_cpu_ids) + return NULL; + + if (unlikely(index >=3D array->map.max_entries)) + return NULL; + + return per_cpu_ptr(array->pptrs[index & array->index_mask], cpu); +} + int bpf_percpu_array_copy(struct bpf_map *map, void *key, void *value) { struct bpf_array *array =3D container_of(map, struct bpf_array, map); @@ -725,6 +739,7 @@ const struct bpf_map_ops percpu_array_map_ops =3D { .map_lookup_elem =3D percpu_array_map_lookup_elem, .map_update_elem =3D array_map_update_elem, .map_delete_elem =3D array_map_delete_elem, + .map_lookup_percpu_elem =3D percpu_array_map_lookup_percpu_elem, .map_seq_show_elem =3D percpu_array_map_seq_show_elem, .map_check_btf =3D array_map_check_btf, .map_lookup_batch =3D generic_map_lookup_batch, diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 13e9dbeeedf3..76f68d0a7ae8 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -2619,6 +2619,7 @@ const struct bpf_func_proto bpf_map_delete_elem_proto= __weak; const struct bpf_func_proto bpf_map_push_elem_proto __weak; const struct bpf_func_proto bpf_map_pop_elem_proto __weak; const struct bpf_func_proto bpf_map_peek_elem_proto __weak; +const struct bpf_func_proto bpf_map_lookup_percpu_elem_proto __weak; const struct bpf_func_proto bpf_spin_lock_proto __weak; const struct bpf_func_proto bpf_spin_unlock_proto __weak; const struct bpf_func_proto bpf_jiffies64_proto __weak; diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 705841279d16..17fb69c0e0dc 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -2199,6 +2199,20 @@ static void *htab_percpu_map_lookup_elem(struct bpf_= map *map, void *key) return NULL; } =20 +static void *htab_percpu_map_lookup_percpu_elem(struct bpf_map *map, void = *key, u32 cpu) +{ + struct htab_elem *l; + + if (cpu >=3D nr_cpu_ids) + return NULL; + + l =3D __htab_map_lookup_elem(map, key); + if (l) + return per_cpu_ptr(htab_elem_get_ptr(l, map->key_size), cpu); + else + return NULL; +} + static void *htab_lru_percpu_map_lookup_elem(struct bpf_map *map, void *ke= y) { struct htab_elem *l =3D __htab_map_lookup_elem(map, key); @@ -2211,6 +2225,22 @@ static void *htab_lru_percpu_map_lookup_elem(struct = bpf_map *map, void *key) return NULL; } =20 +static void *htab_lru_percpu_map_lookup_percpu_elem(struct bpf_map *map, v= oid *key, u32 cpu) +{ + struct htab_elem *l; + + if (cpu >=3D nr_cpu_ids) + return NULL; + + l =3D __htab_map_lookup_elem(map, key); + if (l) { + bpf_lru_node_set_ref(&l->lru_node); + return per_cpu_ptr(htab_elem_get_ptr(l, map->key_size), cpu); + } + + return NULL; +} + int bpf_percpu_hash_copy(struct bpf_map *map, void *key, void *value) { struct htab_elem *l; @@ -2300,6 +2330,7 @@ const struct bpf_map_ops htab_percpu_map_ops =3D { .map_lookup_and_delete_elem =3D htab_percpu_map_lookup_and_delete_elem, .map_update_elem =3D htab_percpu_map_update_elem, .map_delete_elem =3D htab_map_delete_elem, + .map_lookup_percpu_elem =3D htab_percpu_map_lookup_percpu_elem, .map_seq_show_elem =3D htab_percpu_map_seq_show_elem, .map_set_for_each_callback_args =3D map_set_for_each_callback_args, .map_for_each_callback =3D bpf_for_each_hash_elem, @@ -2318,6 +2349,7 @@ const struct bpf_map_ops htab_lru_percpu_map_ops =3D { .map_lookup_and_delete_elem =3D htab_lru_percpu_map_lookup_and_delete_ele= m, .map_update_elem =3D htab_lru_percpu_map_update_elem, .map_delete_elem =3D htab_lru_map_delete_elem, + .map_lookup_percpu_elem =3D htab_lru_percpu_map_lookup_percpu_elem, .map_seq_show_elem =3D htab_percpu_map_seq_show_elem, .map_set_for_each_callback_args =3D map_set_for_each_callback_args, .map_for_each_callback =3D bpf_for_each_hash_elem, diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 3e709fed5306..d5f104a39092 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -119,6 +119,22 @@ const struct bpf_func_proto bpf_map_peek_elem_proto = =3D { .arg2_type =3D ARG_PTR_TO_UNINIT_MAP_VALUE, }; =20 +BPF_CALL_3(bpf_map_lookup_percpu_elem, struct bpf_map *, map, void *, key,= u32, cpu) +{ + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); + return (unsigned long) map->ops->map_lookup_percpu_elem(map, key, cpu); +} + +const struct bpf_func_proto bpf_map_lookup_percpu_elem_proto =3D { + .func =3D bpf_map_lookup_percpu_elem, + .gpl_only =3D false, + .pkt_access =3D true, + .ret_type =3D RET_PTR_TO_MAP_VALUE_OR_NULL, + .arg1_type =3D ARG_CONST_MAP_PTR, + .arg2_type =3D ARG_PTR_TO_MAP_KEY, + .arg3_type =3D ARG_ANYTHING, +}; + const struct bpf_func_proto bpf_get_prandom_u32_proto =3D { .func =3D bpf_user_rnd_u32, .gpl_only =3D false, @@ -1420,6 +1436,8 @@ bpf_base_func_proto(enum bpf_func_id func_id) return &bpf_map_pop_elem_proto; case BPF_FUNC_map_peek_elem: return &bpf_map_peek_elem_proto; + case BPF_FUNC_map_lookup_percpu_elem: + return &bpf_map_lookup_percpu_elem_proto; case BPF_FUNC_get_prandom_u32: return &bpf_get_prandom_u32_proto; case BPF_FUNC_get_smp_processor_id: diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index c27fee73a2cb..05c1b6656824 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -6137,6 +6137,12 @@ static int check_map_func_compatibility(struct bpf_v= erifier_env *env, map->map_type !=3D BPF_MAP_TYPE_BLOOM_FILTER) goto error; break; + case BPF_FUNC_map_lookup_percpu_elem: + if (map->map_type !=3D BPF_MAP_TYPE_PERCPU_ARRAY && + map->map_type !=3D BPF_MAP_TYPE_PERCPU_HASH && + map->map_type !=3D BPF_MAP_TYPE_LRU_PERCPU_HASH) + goto error; + break; case BPF_FUNC_sk_storage_get: case BPF_FUNC_sk_storage_delete: if (map->map_type !=3D BPF_MAP_TYPE_SK_STORAGE) @@ -6750,7 +6756,8 @@ record_func_map(struct bpf_verifier_env *env, struct = bpf_call_arg_meta *meta, func_id !=3D BPF_FUNC_map_pop_elem && func_id !=3D BPF_FUNC_map_peek_elem && func_id !=3D BPF_FUNC_for_each_map_elem && - func_id !=3D BPF_FUNC_redirect_map) + func_id !=3D BPF_FUNC_redirect_map && + func_id !=3D BPF_FUNC_map_lookup_percpu_elem) return 0; =20 if (map =3D=3D NULL) { @@ -13810,7 +13817,8 @@ static int do_misc_fixups(struct bpf_verifier_env *= env) insn->imm =3D=3D BPF_FUNC_map_pop_elem || insn->imm =3D=3D BPF_FUNC_map_peek_elem || insn->imm =3D=3D BPF_FUNC_redirect_map || - insn->imm =3D=3D BPF_FUNC_for_each_map_elem)) { + insn->imm =3D=3D BPF_FUNC_for_each_map_elem || + insn->imm =3D=3D BPF_FUNC_map_lookup_percpu_elem)) { aux =3D &env->insn_aux_data[i + delta]; if (bpf_map_ptr_poisoned(aux)) goto patch_call_imm; @@ -13859,6 +13867,8 @@ static int do_misc_fixups(struct bpf_verifier_env *= env) bpf_callback_t callback_fn, void *callback_ctx, u64 flags))NULL)); + BUILD_BUG_ON(!__same_type(ops->map_lookup_percpu_elem, + (void *(*)(struct bpf_map *map, void *key, u32 cpu))NULL)); =20 patch_map_ops_generic: switch (insn->imm) { @@ -13886,6 +13896,9 @@ static int do_misc_fixups(struct bpf_verifier_env *= env) case BPF_FUNC_for_each_map_elem: insn->imm =3D BPF_CALL_IMM(ops->map_for_each_callback); continue; + case BPF_FUNC_map_lookup_percpu_elem: + insn->imm =3D BPF_CALL_IMM(ops->map_lookup_percpu_elem); + continue; } =20 goto patch_call_imm; diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 2eaac094caf8..7141ca8a1c2d 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1197,6 +1197,8 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, cons= t struct bpf_prog *prog) return &bpf_map_pop_elem_proto; case BPF_FUNC_map_peek_elem: return &bpf_map_peek_elem_proto; + case BPF_FUNC_map_lookup_percpu_elem: + return &bpf_map_lookup_percpu_elem_proto; case BPF_FUNC_ktime_get_ns: return &bpf_ktime_get_ns_proto; case BPF_FUNC_ktime_get_boot_ns: diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index bc7f89948f54..0210f85131b3 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -5164,6 +5164,14 @@ union bpf_attr { * if not NULL, is a reference which must be released using its * corresponding release function, or moved into a BPF map before * program exit. + * + * void *bpf_map_lookup_percpu_elem(struct bpf_map *map, const void *key, = u32 cpu) + * Description + * Perform a lookup in *percpu map* for an entry associated to + * *key* on *cpu*. + * Return + * Map value associated to *key* on *cpu*, or **NULL** if no entry + * was found or *cpu* is invalid. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -5361,6 +5369,7 @@ union bpf_attr { FN(skb_set_tstamp), \ FN(ima_file_hash), \ FN(kptr_xchg), \ + FN(map_lookup_percpu_elem), \ /* */ =20 /* integer value in 'imm' field of BPF_CALL instruction selects which help= er --=20 2.20.1 From nobody Sun May 10 09:55:04 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EDEDC433F5 for ; Fri, 13 May 2022 06:40:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1377423AbiEMGka (ORCPT ); Fri, 13 May 2022 02:40:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1377417AbiEMGk2 (ORCPT ); Fri, 13 May 2022 02:40:28 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96DF62A28D1 for ; Thu, 12 May 2022 23:40:23 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id 31so6649893pgp.8 for ; Thu, 12 May 2022 23:40:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gEB3NbHEFaXkY+7O5LinimDmPz0C+lpCHDDyO2s1SG0=; b=j9XBSBxJAHXR/iQiKohExEY1bxn8g015nWIKUkNnn+CWX0u7t/ULIFexyBCN0iYoiV 1wcBiIUMECDQSY1AzqnU4H5qRenJI8E5o4F845ZL2WkkOXzUXnF2eBMMZlFq86dp6a8G hlFexIB7iFK4UTWYLukjtZt2s65cU+W3n968vljMcj6XzO5dUOopEZeoBEoZKqwZ+tYB f04kE2YYKNrXZeApMNRSnyEeNXeM0NWvJp4POhoG2O+FN2dFRwI4NffXdpoUA9afxWIO AhbOR2HMFdhkvcBPDgVGmeW6FJIwQG35fl4lWZRUwUtkis3EGSofHKJLnvW/BMdE+9B4 GlIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gEB3NbHEFaXkY+7O5LinimDmPz0C+lpCHDDyO2s1SG0=; b=5SIiELCfTc+v5JdzX9FdjFQrwRCM8zZDPRRTIa88ZLAL0ptNaXMCb4XqZcNN8NFI6Y smImVb7MfVEMYyaJnXT7nJAZNOhEWcv7q3Ath8eGtBkAEBXFYjEDLhEOHNbtpo8nd97X IxPExedgBuHFsf4Ell6ffaNWDWivD6hCabrSA2WkIr7TDyqyEvq9acse1Oopo3EUtb05 Ii9WwdTQgCR3EqJYpAANU4oQvfI8Hri/LHEU9dABbzpcThKu86nNCxe35OLA5bnJatFa jygCqPAoNl4wnxZly8rbrW60SfKdiH+jQosG1h/FZCjtKKt5FNrnFXJ8852MSbvX0wxf JRuQ== X-Gm-Message-State: AOAM532edLcBLodl/h/UhGi9jyd3dF61o2uNKb4ARXO/22VwNWCt+17k SgqqmSEwlS6h6X71tvKUOvSZHg== X-Google-Smtp-Source: ABdhPJwzIvuxlCkOB40IyPdY/7bfjpy1/4p8yogqsh2unHhjGNAhjAspnCJQ1CqLRP9JfB6CvKkyig== X-Received: by 2002:a63:2c4a:0:b0:3c1:df82:cf0e with SMTP id s71-20020a632c4a000000b003c1df82cf0emr2831567pgs.474.1652424022912; Thu, 12 May 2022 23:40:22 -0700 (PDT) Received: from C02F52LSML85.bytedance.net ([139.177.225.225]) by smtp.gmail.com with ESMTPSA id 185-20020a6204c2000000b0050dc76281d8sm934990pfe.178.2022.05.12.23.40.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 May 2022 23:40:22 -0700 (PDT) From: Feng zhou To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, rostedt@goodmis.org, mingo@redhat.com, jolsa@kernel.org, davemarchevsky@fb.com, joannekoong@fb.com, geliang.tang@suse.com Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, songmuchun@bytedance.com, wangdongdong.6@bytedance.com, cong.wang@bytedance.com, zhouchengming@bytedance.com, zhoufeng.zf@bytedance.com, yosryahmed@google.com Subject: [PATCH bpf-next v3 2/2] selftests/bpf: add test case for bpf_map_lookup_percpu_elem Date: Fri, 13 May 2022 14:39:52 +0800 Message-Id: <20220513063952.41794-3-zhoufeng.zf@bytedance.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20220513063952.41794-1-zhoufeng.zf@bytedance.com> References: <20220513063952.41794-1-zhoufeng.zf@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Feng Zhou test_progs: Tests new ebpf helpers bpf_map_lookup_percpu_elem. Signed-off-by: Feng Zhou --- .../bpf/prog_tests/map_lookup_percpu_elem.c | 59 +++++++++++++++ .../bpf/progs/test_map_lookup_percpu_elem.c | 71 +++++++++++++++++++ 2 files changed, 130 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/map_lookup_percp= u_elem.c create mode 100644 tools/testing/selftests/bpf/progs/test_map_lookup_percp= u_elem.c diff --git a/tools/testing/selftests/bpf/prog_tests/map_lookup_percpu_elem.= c b/tools/testing/selftests/bpf/prog_tests/map_lookup_percpu_elem.c new file mode 100644 index 000000000000..89ca170f1c25 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/map_lookup_percpu_elem.c @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2022 Bytedance */ + +#include + +#include "test_map_lookup_percpu_elem.skel.h" + +void test_map_lookup_percpu_elem(void) +{ + struct test_map_lookup_percpu_elem *skel; + __u64 key =3D 0, sum; + int ret, i; + int nr_cpus =3D libbpf_num_possible_cpus(); + __u64 *buf; + + buf =3D (__u64 *)malloc(nr_cpus*sizeof(__u64)); + if (!ASSERT_OK_PTR(buf, "malloc")) + return; + + for (i=3D0; irodata->nr_cpus =3D nr_cpus; + + ret =3D test_map_lookup_percpu_elem__load(skel); + if (!ASSERT_OK(ret, "test_map_lookup_percpu_elem__load")) + goto cleanup; + + ret =3D test_map_lookup_percpu_elem__attach(skel); + if (!ASSERT_OK(ret, "test_map_lookup_percpu_elem__attach")) + goto cleanup; + + ret =3D bpf_map_update_elem(bpf_map__fd(skel->maps.percpu_array_map), &ke= y, buf, 0); + ASSERT_OK(ret, "percpu_array_map update"); + + ret =3D bpf_map_update_elem(bpf_map__fd(skel->maps.percpu_hash_map), &key= , buf, 0); + ASSERT_OK(ret, "percpu_hash_map update"); + + ret =3D bpf_map_update_elem(bpf_map__fd(skel->maps.percpu_lru_hash_map), = &key, buf, 0); + ASSERT_OK(ret, "percpu_lru_hash_map update"); + + syscall(__NR_getuid); + + test_map_lookup_percpu_elem__detach(skel); + + ASSERT_EQ(skel->bss->percpu_array_elem_sum, sum, "percpu_array lookup per= cpu elem"); + ASSERT_EQ(skel->bss->percpu_hash_elem_sum, sum, "percpu_hash lookup percp= u elem"); + ASSERT_EQ(skel->bss->percpu_lru_hash_elem_sum, sum, "percpu_lru_hash look= up percpu elem"); + +cleanup: + test_map_lookup_percpu_elem__destroy(skel); +exit: + free(buf); +} diff --git a/tools/testing/selftests/bpf/progs/test_map_lookup_percpu_elem.= c b/tools/testing/selftests/bpf/progs/test_map_lookup_percpu_elem.c new file mode 100644 index 000000000000..75479180571f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_map_lookup_percpu_elem.c @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2022 Bytedance */ + +#include "vmlinux.h" +#include + +__u64 percpu_array_elem_sum =3D 0; +__u64 percpu_hash_elem_sum =3D 0; +__u64 percpu_lru_hash_elem_sum =3D 0; +const volatile int nr_cpus; + +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, __u64); +} percpu_array_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_HASH); + __uint(max_entries, 1); + __type(key, __u64); + __type(value, __u64); +} percpu_hash_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_PERCPU_HASH); + __uint(max_entries, 1); + __type(key, __u64); + __type(value, __u64); +} percpu_lru_hash_map SEC(".maps"); + +struct read_percpu_elem_ctx { + void *map; + __u64 sum; +}; + +static int read_percpu_elem_callback(__u32 index, struct read_percpu_elem_= ctx *ctx) +{ + __u64 key =3D 0; + __u64 *value; + value =3D bpf_map_lookup_percpu_elem(ctx->map, &key, index); + if (value) + ctx->sum +=3D *value; + return 0; +} + +SEC("tp/syscalls/sys_enter_getuid") +int sysenter_getuid(const void *ctx) +{ + struct read_percpu_elem_ctx map_ctx; + + map_ctx.map =3D &percpu_array_map; + map_ctx.sum =3D 0; + bpf_loop(nr_cpus, read_percpu_elem_callback, &map_ctx, 0); + percpu_array_elem_sum =3D map_ctx.sum; + + map_ctx.map =3D &percpu_hash_map; + map_ctx.sum =3D 0; + bpf_loop(nr_cpus, read_percpu_elem_callback, &map_ctx, 0); + percpu_hash_elem_sum =3D map_ctx.sum; + + map_ctx.map =3D &percpu_lru_hash_map; + map_ctx.sum =3D 0; + bpf_loop(nr_cpus, read_percpu_elem_callback, &map_ctx, 0); + percpu_lru_hash_elem_sum =3D map_ctx.sum; + + return 0; +} + +char _license[] SEC("license") =3D "GPL"; --=20 2.20.1