From nobody Thu Dec 18 20:18:00 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9A8CCA0EC3 for ; Tue, 12 Sep 2023 07:02:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231211AbjILHCe (ORCPT ); Tue, 12 Sep 2023 03:02:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231328AbjILHCa (ORCPT ); Tue, 12 Sep 2023 03:02:30 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5195810EC for ; Tue, 12 Sep 2023 00:02:03 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1bf6ea270b2so35558715ad.0 for ; Tue, 12 Sep 2023 00:02:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502123; x=1695106923; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HDkGVzbyVwp5CP2Bj0/gBikBLcGAUp0LBh7IxC9dYSw=; b=FuKgP0rdrCNA8BL2+5RQShyltCoZSh6tDMZSCrIaPkjDxlemdkC06K0GeEsnkYypnY 1P5q0ytA/Wa9uX9rOokYy3bVL7QhC4/mnOg5lGyldv6U9E3UXW94xGBr1iTzzLPwCzNp vznLxsYaTmrIn2+UPKGpAB3ua88fhLAR4+0r37N84H7yhRbk2Y8X8wnRkRlL7Blnn+vG V8N3ePT6EE0isFWsHU6pdYes5qSfPgtFNHjU3wgHZAimBWV7WSApZ0byr5HZCSU1p/qN Hb4+qGeskPP6wsbsiPvxmJNhKV8BG1yop2cson0Os3mCblUf0qc6FplKx6Wq5kZbdgSI 40ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502123; x=1695106923; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HDkGVzbyVwp5CP2Bj0/gBikBLcGAUp0LBh7IxC9dYSw=; b=gjbCGCmsE5VQXP4mh1yCyrCL4CFY8HoRmVym+osn7fGXEz96aykUR+dsjyuigc5J6J Ele/2vMiorNsYXrTuuk6K8+ZqCROQ2qGYW5dOoxb+qkgJvO3g9MsMzQ9pcRHBat4u3tZ ba6f3jFurtl+ZedMEYepaG6O5AkhwrXkXYVFOVS652uTBr1f9TSgAnkFfcKOgylDLkqk 4kMI8PI4nqoB/tjK2jMBOunNVqu5CHoBHASz8Xr+BUM86Le2hfEmu5wyUrGUGW9WDx11 jO8nTZsWibu1SdRDrPMe1z5Nrs+1Xnz2Ce1x0fRJ0LUaof6p90h2WaCKP4wC30+aRg8z HRCg== X-Gm-Message-State: AOJu0YzdFzlISD+Ts/TKcVaa8/mo0FarsW6F7HMbVD/eGzsuedq+Jbuu aGARLN1VaXt5lT/s+F6d6g8awg== X-Google-Smtp-Source: AGHT+IFckTbjOlPAIMiqK0BYx/GUeBDEJpj6eWIM+X+1r3sNOUrdk6v7pTIn0Cta0Ln/MF1jph7UfQ== X-Received: by 2002:a17:903:2443:b0:1c3:d864:d6b3 with SMTP id l3-20020a170903244300b001c3d864d6b3mr482903pls.57.1694502122788; Tue, 12 Sep 2023 00:02:02 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:02 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 1/6] cgroup: Prepare for using css_task_iter_*() in BPF Date: Tue, 12 Sep 2023 15:01:44 +0800 Message-Id: <20230912070149.969939-2-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch makes some preparations for using css_task_iter_*() in BPF Program. 1. Flags CSS_TASK_ITER_* are #define-s and it's not easy for bpf prog to use them. Convert them to enum so bpf prog can take them from vmlinux.h. 2. In the next patch we will add css_task_iter_*() in common kfuncs which is not safe. Since css_task_iter_*() does spin_unlock_irq() which might screw up irq flags depending on the context where bpf prog is running. So we should use irqsave/irqrestore here and the switching is harmless. Signed-off-by: Chuyi Zhou Acked-by: Tejun Heo --- include/linux/cgroup.h | 12 +++++------- kernel/cgroup/cgroup.c | 18 ++++++++++++------ 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index b307013b9c6c..0ef0af66080e 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -40,13 +40,11 @@ struct kernel_clone_args; #define CGROUP_WEIGHT_DFL 100 #define CGROUP_WEIGHT_MAX 10000 =20 -/* walk only threadgroup leaders */ -#define CSS_TASK_ITER_PROCS (1U << 0) -/* walk all threaded css_sets in the domain */ -#define CSS_TASK_ITER_THREADED (1U << 1) - -/* internal flags */ -#define CSS_TASK_ITER_SKIPPED (1U << 16) +enum { + CSS_TASK_ITER_PROCS =3D (1U << 0), /* walk only threadgroup leaders */ + CSS_TASK_ITER_THREADED =3D (1U << 1), /* walk all threaded css_sets in t= he domain */ + CSS_TASK_ITER_SKIPPED =3D (1U << 16), /* internal flags */ +}; =20 /* a css_task_iter should be treated as an opaque object */ struct css_task_iter { diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 1fb7f562289d..b6d64f3b8888 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4917,9 +4917,11 @@ static void css_task_iter_advance(struct css_task_it= er *it) void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int fla= gs, struct css_task_iter *it) { + unsigned long irqflags; + memset(it, 0, sizeof(*it)); =20 - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); =20 it->ss =3D css->ss; it->flags =3D flags; @@ -4933,7 +4935,7 @@ void css_task_iter_start(struct cgroup_subsys_state *= css, unsigned int flags, =20 css_task_iter_advance(it); =20 - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } =20 /** @@ -4946,12 +4948,14 @@ void css_task_iter_start(struct cgroup_subsys_state= *css, unsigned int flags, */ struct task_struct *css_task_iter_next(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_task) { put_task_struct(it->cur_task); it->cur_task =3D NULL; } =20 - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); =20 /* @it may be half-advanced by skips, finish advancing */ if (it->flags & CSS_TASK_ITER_SKIPPED) @@ -4964,7 +4968,7 @@ struct task_struct *css_task_iter_next(struct css_tas= k_iter *it) css_task_iter_advance(it); } =20 - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); =20 return it->cur_task; } @@ -4977,11 +4981,13 @@ struct task_struct *css_task_iter_next(struct css_t= ask_iter *it) */ void css_task_iter_end(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_cset) { - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); list_del(&it->iters_node); put_css_set_locked(it->cur_cset); - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } =20 if (it->cur_dcset) --=20 2.20.1 From nobody Thu Dec 18 20:18:00 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FDEBCA0EC3 for ; Tue, 12 Sep 2023 07:02:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231271AbjILHCh (ORCPT ); Tue, 12 Sep 2023 03:02:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231327AbjILHCa (ORCPT ); Tue, 12 Sep 2023 03:02:30 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B01B0E76 for ; Tue, 12 Sep 2023 00:02:06 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-68fb46f38f9so1916831b3a.1 for ; Tue, 12 Sep 2023 00:02:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502126; x=1695106926; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FNQpGQe4u79p1mkZVJD58U2KIJEFL4MZuGQeXTwOQ8w=; b=Dow5IJyKwkOy28TWDR9DDnjAogp9ctHCGYgTIiopVm61CJBX4KtSrMGSU1tt3U8dZX t8jMpD60rg5/hDwiqUJeq9z4WKLQwBXfV7ALggUQeh7PqPCrti/rKseXZU4/2/ivg1/M t5mdw24PCiXIwotA7LSzjM5CpbpQpODL/Dn/30HR64J/9e4rJhIpvl9l4hPrgOJbqsfJ RkROP7y53WJoqvfKDUMZYjz0QnDf7EWX/eM2mjGuGXMgnfMuOkyDw0Xe/nQTbeKaVznR bLX1GuNfRU/ADul/SXbTD7ZuEl25jbAoJdvDAO7ZRl2/CLRyqKPi+dt4vKy/NLOY/Y8K a6lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502126; x=1695106926; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FNQpGQe4u79p1mkZVJD58U2KIJEFL4MZuGQeXTwOQ8w=; b=irHQftaPoXH9vhy8VehLJyMHtdI3khBIhmK9tK+lpibJLLJsFQxwTMGxnRvi1fmhO4 07MhtE0VnTTzHvsdUePg4AhjdMubAoitVGkZ4TQk3iMf/1+Q4a3CSWv/sc3UZez1qQAr Q+Oyr6sLh1kpcj2aei/4Y8Ieayl4iDWJb+/Yf6sfgIGvc3m4hwccrTSJvZUlrhXvXK1B sP2l/9/W/cg0VD6VX94n+AHfEeTXkrBqa5AzEL0ChUJoxNBhJugcYCH7gFNhdiUpi7rN C7HnGb5gr+DiqADJdt+a7tIrpPF4dDl5yD/uWD/AFkn0YpYV2b3k4ZEd8i3ZFKAjGTjY uR3Q== X-Gm-Message-State: AOJu0YyOI7V4N1DGe/XBiaiUs8PlqDXgtPtCkjBuO/OGFW+PG/Xb3cQB ZPYTg4ZzrAQZKszLEJqJ8CXdcw== X-Google-Smtp-Source: AGHT+IFrGkPdm7n165iwKopet18YYKmVo1jjoqtm9QF2jj2jLPZw8NkCsAYe/iI/yLg8hGzAh6dENw== X-Received: by 2002:a05:6a20:ce9f:b0:153:c9fa:4ed6 with SMTP id if31-20020a056a20ce9f00b00153c9fa4ed6mr8870801pzb.10.1694502126142; Tue, 12 Sep 2023 00:02:06 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:05 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 2/6] bpf: Introduce css_task open-coded iterator kfuncs Date: Tue, 12 Sep 2023 15:01:45 +0800 Message-Id: <20230912070149.969939-3-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch adds kfuncs bpf_iter_css_task_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_css_task in open-coded iterator style. These kfuncs actually wrapps css_task_iter_{start,next, end}. BPF programs can use these kfuncs through bpf_for_each macro for iteration of all tasks under a css. css_task_iter_*() would try to get the global spin-lock *css_set_lock*, so the bpf side has to be careful in where it allows to use this iter. Currently we only allow it in bpf_lsm and bpf iter-s. Signed-off-by: Chuyi Zhou --- include/uapi/linux/bpf.h | 4 +++ kernel/bpf/helpers.c | 3 +++ kernel/bpf/task_iter.c | 48 ++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 23 ++++++++++++++++ tools/include/uapi/linux/bpf.h | 4 +++ tools/lib/bpf/bpf_helpers.h | 7 +++++ 6 files changed, 89 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 73b155e52204..de02c0971428 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7318,4 +7318,8 @@ struct bpf_iter_num { __u64 __opaque[1]; } __attribute__((aligned(8))); =20 +struct bpf_iter_css_task { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index b0a9834f1051..d6a16becfbb9 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2504,6 +2504,9 @@ BTF_ID_FLAGS(func, bpf_dynptr_slice_rdwr, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_num_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_num_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index 7473068ed313..d8539cc05ffd 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -803,6 +803,54 @@ const struct bpf_func_proto bpf_find_vma_proto =3D { .arg5_type =3D ARG_ANYTHING, }; =20 +struct bpf_iter_css_task_kern { + struct css_task_iter *css_it; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) +{ + struct bpf_iter_css_task_kern *kit =3D (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_task_kern) !=3D sizeof(struct bpf= _iter_css_task)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_task_kern) !=3D + __alignof__(struct bpf_iter_css_task)); + + switch (flags) { + case CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED: + case CSS_TASK_ITER_PROCS: + case 0: + break; + default: + return -EINVAL; + } + + kit->css_it =3D kzalloc(sizeof(struct css_task_iter), GFP_KERNEL); + if (!kit->css_it) + return -ENOMEM; + css_task_iter_start(css, flags, kit->css_it); + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css= _task *it) +{ + struct bpf_iter_css_task_kern *kit =3D (void *)it; + + if (!kit->css_it) + return NULL; + return css_task_iter_next(kit->css_it); +} + +__bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) +{ + struct bpf_iter_css_task_kern *kit =3D (void *)it; + + if (!kit->css_it) + return; + css_task_iter_end(kit->css_it); + kfree(kit->css_it); +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); =20 static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index dbba2b806017..2367483bf4c2 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10332,6 +10332,7 @@ enum special_kfunc_type { KF_bpf_dynptr_clone, KF_bpf_percpu_obj_new_impl, KF_bpf_percpu_obj_drop_impl, + KF_bpf_iter_css_task_new, }; =20 BTF_SET_START(special_kfunc_set) @@ -10354,6 +10355,7 @@ BTF_ID(func, bpf_dynptr_slice_rdwr) BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) +BTF_ID(func, bpf_iter_css_task_new) BTF_SET_END(special_kfunc_set) =20 BTF_ID_LIST(special_kfunc_list) @@ -10378,6 +10380,7 @@ BTF_ID(func, bpf_dynptr_slice_rdwr) BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) +BTF_ID(func, bpf_iter_css_task_new) =20 static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { @@ -10902,6 +10905,20 @@ static int process_kf_arg_ptr_to_rbtree_node(struc= t bpf_verifier_env *env, &meta->arg_rbtree_root.field); } =20 +static bool check_css_task_iter_allowlist(struct bpf_verifier_env *env) +{ + enum bpf_prog_type prog_type =3D resolve_prog_type(env->prog); + + switch (prog_type) { + case BPF_PROG_TYPE_LSM: + return true; + case BPF_TRACE_ITER: + return env->prog->aux->sleepable; + default: + return false; + } +} + static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc= _call_arg_meta *meta, int insn_idx) { @@ -11152,6 +11169,12 @@ static int check_kfunc_args(struct bpf_verifier_en= v *env, struct bpf_kfunc_call_ break; } case KF_ARG_PTR_TO_ITER: + if (meta->func_id =3D=3D special_kfunc_list[KF_bpf_iter_css_task_new]) { + if (!check_css_task_iter_allowlist(env)) { + verbose(env, "css_task_iter is only allowed in bpf_lsm and bpf iter-s= \n"); + return -EINVAL; + } + } ret =3D process_iter_arg(env, regno, insn_idx, meta); if (ret < 0) return ret; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 73b155e52204..de02c0971428 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7318,4 +7318,8 @@ struct bpf_iter_num { __u64 __opaque[1]; } __attribute__((aligned(8))); =20 +struct bpf_iter_css_task { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index 77ceea575dc7..f48723c6c593 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -303,6 +303,13 @@ extern int bpf_iter_num_new(struct bpf_iter_num *it, i= nt start, int end) __weak extern int *bpf_iter_num_next(struct bpf_iter_num *it) __weak __ksym; extern void bpf_iter_num_destroy(struct bpf_iter_num *it) __weak __ksym; =20 +struct bpf_iter_css_task; +struct cgroup_subsys_state; +extern int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) __weak __ksym; +extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task= *it) __weak __ksym; +extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak= __ksym; + #ifndef bpf_for_each /* bpf_for_each(iter_type, cur_elem, args...) provides generic construct f= or * using BPF open-coded iterators without having to write mundane explicit --=20 2.20.1 From nobody Thu Dec 18 20:18:00 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F6EDCA0EC3 for ; Tue, 12 Sep 2023 07:02:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231239AbjILHCS (ORCPT ); Tue, 12 Sep 2023 03:02:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231216AbjILHCN (ORCPT ); Tue, 12 Sep 2023 03:02:13 -0400 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02061E7C for ; Tue, 12 Sep 2023 00:02:10 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1c0ecb9a075so37537735ad.2 for ; Tue, 12 Sep 2023 00:02:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502129; x=1695106929; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+2pL38sdR1zkqvND06hEOkj6aCyDQM/YMk3Q7fUlrh4=; b=M2IuHdH8p6VL/T+r4KhufKVPTFU9Qdd3Vj8fRX16Ps/8Jex8HpP2UTkOUzS3bE7/oW p0lcRAwO7+0E3dt+h7uqMRwB2GqCI5g1PuSu6FZ9Ct0E7OnEX2gq5qyVLx3IIsoPkP6Q /2dD5jdLkzIsjH5bpUst9xbVHww97Jsxzgzlp5UJ2M4/+FHtUSdVd/tN+dxIcr/kq/bS GO2r9XE86jJyy7IkIzNpEanGMne3qAOb/KEHS2MlvGdiAEYv9VkI7cR7SuirsS019hcM DYFOZjhfY7g19waibSAbfQyTU+hR1nhYAnO2Bmms4oPTE+dMd4ikYAN0QIGgiINaHiuJ Vnzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502129; x=1695106929; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+2pL38sdR1zkqvND06hEOkj6aCyDQM/YMk3Q7fUlrh4=; b=gu0csuK4shNgeEb8z7dGU+8cshjShm4OUeaO2UinoPJGvKajXjjZBOfaNTKaj9D60l epgXi+lP1UNRBKUKWKwM7ZEpKj/m2Os3pMwhCBWC+n2yRkylBNDJSKMTSZelFyUQM48E TfRpJS47uQDzab+7b9yti0M0xHDBZ8mLGCJtM35YfYFLoPtfydYt7pksEtzucwqGB4Sx zgmF7KwxT9dhXN4dApQVUXNX+A1JviKsr4JaMmJbvoGRv42zZtqVAcBGz8AaS9iZWFIv PmxRunnJ0g0knSfUXOhOJUx3GsLPOff5ghtxMhz67gLQb4rAz0+4G8mwVqMGaTErLYV2 FyCg== X-Gm-Message-State: AOJu0Yz+CrGjL0u+fFLhui7t384WpRDulcnx8fQPQn3VcT/XFCROTiHn xl+MwZMGjlCv5YExSCOuewE2Pg== X-Google-Smtp-Source: AGHT+IFLHzQBh59ISNd1Ew9UXrAaH+IQXBWi+Di1oQTdwgCpmr79ELmZXckxp25ZL61NyEOj/fQqLg== X-Received: by 2002:a17:903:2302:b0:1bb:e74b:39ff with SMTP id d2-20020a170903230200b001bbe74b39ffmr12134008plh.0.1694502129564; Tue, 12 Sep 2023 00:02:09 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:09 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 3/6] bpf: Introduce process open coded iterator kfuncs Date: Tue, 12 Sep 2023 15:01:46 +0800 Message-Id: <20230912070149.969939-4-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch adds kfuncs bpf_iter_process_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_process in open-coded iterator style. BPF programs can use these kfuncs or through bpf_for_each macro to iterate all processes in the system. Signed-off-by: Chuyi Zhou --- include/uapi/linux/bpf.h | 4 ++++ kernel/bpf/helpers.c | 3 +++ kernel/bpf/task_iter.c | 29 +++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 4 ++++ tools/lib/bpf/bpf_helpers.h | 5 +++++ 5 files changed, 45 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index de02c0971428..befa55b52e29 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7322,4 +7322,8 @@ struct bpf_iter_css_task { __u64 __opaque[1]; } __attribute__((aligned(8))); =20 +struct bpf_iter_process { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index d6a16becfbb9..9b7d2c6f99d1 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2507,6 +2507,9 @@ BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DEST= ROY) BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_process_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_process_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_process_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index d8539cc05ffd..9d1927dc3a06 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -851,6 +851,35 @@ __bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_= iter_css_task *it) kfree(kit->css_it); } =20 +struct bpf_iter_process_kern { + struct task_struct *tsk; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_process_new(struct bpf_iter_process *it) +{ + struct bpf_iter_process_kern *kit =3D (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_process_kern) !=3D sizeof(struct bpf_= iter_process)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_process_kern) !=3D + __alignof__(struct bpf_iter_process)); + + kit->tsk =3D &init_task; + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_process_next(struct bpf_iter_proc= ess *it) +{ + struct bpf_iter_process_kern *kit =3D (void *)it; + + kit->tsk =3D next_task(kit->tsk); + + return kit->tsk =3D=3D &init_task ? NULL : kit->tsk; +} + +__bpf_kfunc void bpf_iter_process_destroy(struct bpf_iter_process *it) +{ +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); =20 static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index de02c0971428..befa55b52e29 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7322,4 +7322,8 @@ struct bpf_iter_css_task { __u64 __opaque[1]; } __attribute__((aligned(8))); =20 +struct bpf_iter_process { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index f48723c6c593..858252c2641c 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -310,6 +310,11 @@ extern int bpf_iter_css_task_new(struct bpf_iter_css_t= ask *it, extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task= *it) __weak __ksym; extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak= __ksym; =20 +struct bpf_iter_process; +extern int bpf_iter_process_new(struct bpf_iter_process *it) __weak __ksym; +extern struct task_struct *bpf_iter_process_next(struct bpf_iter_process *= it) __weak __ksym; +extern void bpf_iter_process_destroy(struct bpf_iter_process *it) __weak _= _ksym; + #ifndef bpf_for_each /* bpf_for_each(iter_type, cur_elem, args...) provides generic construct f= or * using BPF open-coded iterators without having to write mundane explicit --=20 2.20.1 From nobody Thu Dec 18 20:18:00 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 585B7CA0ECA for ; Tue, 12 Sep 2023 07:02:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231365AbjILHCr (ORCPT ); Tue, 12 Sep 2023 03:02:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231253AbjILHCj (ORCPT ); Tue, 12 Sep 2023 03:02:39 -0400 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6625EE75 for ; Tue, 12 Sep 2023 00:02:13 -0700 (PDT) Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1c336f3f449so45836235ad.3 for ; Tue, 12 Sep 2023 00:02:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502133; x=1695106933; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zbzquHAnwok7U8VH5ASiuRWtN2QtgmEjZHX4A6sEdQ0=; b=EgE/SEvhM6y7H0wOdoJWkDzV29CKcPkS3iWL8GssBkcc0O9ymUk5IhMnFKhvYVz7M0 OhAQpdsl5k1naQB1oRrSa/OXFxDDPGjdfgZG/ECrAQa2+x71LlfB7pXbFUe4Ox/t9F77 cLIcHTy2RaBomVkoR/Kljy5ollN4wNcDVT6/jAE5Xi/aZtEhd7wzp8WKSN2XV4qi4hSs 7+fs9Ifk/Us6R8GOVpjdC57+grmarGBjpPdJIKcUIgqzTPIyM5tMTYV4ZqmplsSyMVsH bQC6P6th9E06JuHAEDCV018vo8zzU5d4N6htKV8QC2a9LDXHUL8vF3u7yKRamWZKRcZT S5GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502133; x=1695106933; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zbzquHAnwok7U8VH5ASiuRWtN2QtgmEjZHX4A6sEdQ0=; b=PU+0fSV6O57JJVoAyImX/GKH/zcZpQn3CUjyOIeKr9w47axpbg4mm623g5DG2M5ItI d7DHvgVB6y6iixhkwQ6WI+Wnc8vLGly8poOjqetvq2/zdIEajtQoHLDJZ2wAc7K4+ME3 goN8WT8Cz0rVc79Xo/KjEWrFCaSW0ckm3LBYue2R4LPC1GqTLS3iOPA55dQnycmWjUjP 882tbZK1VxwCmfJbIyQB3QzooElJL2sRQGVjO4/2p8uxwduEy8MKsxekMz5ash1tQEBq V9JySNdx/oxhdpFBdc+Ip5E3zX2l7yFeCPL3U+xImsqUphmRL4Q6MfdnucoQUnN1P0DJ Dz7A== X-Gm-Message-State: AOJu0YwxNJ5bO26ZeQQtvqVIUJVD6gWyDnYOqWNaWiT6VfjIhSm8Eanu 5rMDsTzdyHkSIldgOfn8oK/YdD+JrzOAqfmacv8OBg== X-Google-Smtp-Source: AGHT+IH3nN6SdkGHHenKlH89H8T3xLnbtavEibYfCQL2fxVa7I7VNXrgt1h0+smga0DjUw8E5wv8fA== X-Received: by 2002:a17:902:8693:b0:1b8:66f6:87a3 with SMTP id g19-20020a170902869300b001b866f687a3mr10957357plo.52.1694502132925; Tue, 12 Sep 2023 00:02:12 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:12 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 4/6] bpf: Introduce css_descendant open-coded iterator kfuncs Date: Tue, 12 Sep 2023 15:01:47 +0800 Message-Id: <20230912070149.969939-5-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This Patch adds kfuncs bpf_iter_css_{pre,post}_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_css in open-coded iterator style. These kfuncs actually wrapps css_next_descendant_{pre, post}. BPF programs can use these kfuncs through bpf_for_each macro for iteration of all descendant css under a root css. Signed-off-by: Chuyi Zhou --- include/uapi/linux/bpf.h | 8 +++++ kernel/bpf/helpers.c | 6 ++++ kernel/bpf/task_iter.c | 53 ++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 8 +++++ tools/lib/bpf/bpf_helpers.h | 12 ++++++++ 5 files changed, 87 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index befa55b52e29..57760afc13d0 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7326,4 +7326,12 @@ struct bpf_iter_process { __u64 __opaque[1]; } __attribute__((aligned(8))); =20 +struct bpf_iter_css_pre { + __u64 __opaque[2]; +} __attribute__((aligned(8))); + +struct bpf_iter_css_post { + __u64 __opaque[2]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 9b7d2c6f99d1..ca1f6404af9e 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2510,6 +2510,12 @@ BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITE= R_DESTROY) BTF_ID_FLAGS(func, bpf_iter_process_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_process_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_process_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_pre_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_css_pre_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_pre_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_post_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_css_post_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_post_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index 9d1927dc3a06..8963fc779b87 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -880,6 +880,59 @@ __bpf_kfunc void bpf_iter_process_destroy(struct bpf_i= ter_process *it) { } =20 +struct bpf_iter_css_kern { + struct cgroup_subsys_state *root; + struct cgroup_subsys_state *pos; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_css_pre_new(struct bpf_iter_css_pre *it, + struct cgroup_subsys_state *root) +{ + struct bpf_iter_css_kern *kit =3D (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_kern) !=3D sizeof(struct bpf_iter= _css_pre)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_kern) !=3D __alignof__(struc= t bpf_iter_css_pre)); + kit->root =3D root; + kit->pos =3D NULL; + return 0; +} + +__bpf_kfunc struct cgroup_subsys_state *bpf_iter_css_pre_next(struct bpf_i= ter_css_pre *it) +{ + struct bpf_iter_css_kern *kit =3D (void *)it; + + kit->pos =3D css_next_descendant_pre(kit->pos, kit->root); + return kit->pos; +} + +__bpf_kfunc void bpf_iter_css_pre_destroy(struct bpf_iter_css_pre *it) +{ +} + +__bpf_kfunc int bpf_iter_css_post_new(struct bpf_iter_css_post *it, + struct cgroup_subsys_state *root) +{ + struct bpf_iter_css_kern *kit =3D (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_kern) !=3D sizeof(struct bpf_iter= _css_post)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_kern) !=3D __alignof__(struc= t bpf_iter_css_post)); + kit->root =3D root; + kit->pos =3D NULL; + return 0; +} + +__bpf_kfunc struct cgroup_subsys_state *bpf_iter_css_post_next(struct bpf_= iter_css_post *it) +{ + struct bpf_iter_css_kern *kit =3D (void *)it; + + kit->pos =3D css_next_descendant_post(kit->pos, kit->root); + return kit->pos; +} + +__bpf_kfunc void bpf_iter_css_post_destroy(struct bpf_iter_css_post *it) +{ +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); =20 static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index befa55b52e29..57760afc13d0 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7326,4 +7326,12 @@ struct bpf_iter_process { __u64 __opaque[1]; } __attribute__((aligned(8))); =20 +struct bpf_iter_css_pre { + __u64 __opaque[2]; +} __attribute__((aligned(8))); + +struct bpf_iter_css_post { + __u64 __opaque[2]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index 858252c2641c..6e5bd9ef14d6 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -315,6 +315,18 @@ extern int bpf_iter_process_new(struct bpf_iter_proces= s *it) __weak __ksym; extern struct task_struct *bpf_iter_process_next(struct bpf_iter_process *= it) __weak __ksym; extern void bpf_iter_process_destroy(struct bpf_iter_process *it) __weak _= _ksym; =20 +struct bpf_iter_css_pre; +extern int bpf_iter_css_pre_new(struct bpf_iter_css_pre *it, + struct cgroup_subsys_state *root) __weak __ksym; +extern struct cgroup_subsys_state *bpf_iter_css_pre_next(struct bpf_iter_c= ss_pre *it) __weak __ksym; +extern void bpf_iter_css_pre_destroy(struct bpf_iter_css_pre *it) __weak _= _ksym; + +struct bpf_iter_css_post; +extern int bpf_iter_css_post_new(struct bpf_iter_css_post *it, + struct cgroup_subsys_state *root) __weak __ksym; +extern struct cgroup_subsys_state *bpf_iter_css_post_next(struct bpf_iter_= css_post *it) __weak __ksym; +extern void bpf_iter_css_post_destroy(struct bpf_iter_css_post *it) __weak= __ksym; + #ifndef bpf_for_each /* bpf_for_each(iter_type, cur_elem, args...) provides generic construct f= or * using BPF open-coded iterators without having to write mundane explicit --=20 2.20.1 From nobody Thu Dec 18 20:18:00 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E06A6CA0ECA for ; Tue, 12 Sep 2023 07:02:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231336AbjILHCu (ORCPT ); Tue, 12 Sep 2023 03:02:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231382AbjILHCl (ORCPT ); Tue, 12 Sep 2023 03:02:41 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4347AF for ; Tue, 12 Sep 2023 00:02:16 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1c35ee3b0d2so35295345ad.2 for ; Tue, 12 Sep 2023 00:02:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502136; x=1695106936; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zZ7auXMwt2HG2eO79EZb79CGVaa+fy/IcLCkXgQTU2E=; b=gJntY8cV6wapICpdZfEIKXOai/sUbNM/O2PEKnDVSxn2gbWVIXA5t6cb7ba/zTwtwZ WPKEc9Sb6BUUld9t3MOlZYSn36CNY7tArLtbbMwnTUTx3Mf5iaTfiZltVlVMV5xD4xQY X0FW9FZVCyndsjXUA8mh1czh/TU1HNC0/w83YQ7H09Et5h9x0ZmpeyHEFeji9O+0Vmxl xlJ0cfto8eS7a4We7zUxCHvM91RjLFelbmxD1yMP5jZN/ZD7K0XFwIEDw+SW7VfCMNOT Jep7/hQOIfpGe4m+m5bjhWqY0GvnE9Fwk1FcBmB1GqSj2HWUlw3/QSPGHW5JjFwi6oqI xDyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502136; x=1695106936; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zZ7auXMwt2HG2eO79EZb79CGVaa+fy/IcLCkXgQTU2E=; b=Fv8wJ4ueJ6Hi+kcxAfo1obxMcUAFTShKxRFrUceNSO4yD8KEhlM5W2dE068gfoEDv+ +zy1oE6P8bx7uN2ea6/sDZFSK0/Jdt/ImmDpnIfxLMo/DBCjNp2KrS/htRbpgdck4M4q WCpdGLwlaZcgBo9DsqZFfR0bqO87z4xPN+qzxAjNpqeuN3nSTO8ZWKRgNHPltRGLQTlc L6L2s7tAaZBlieF2vyO0g/q+YGTUXzSb2Q9DXKQNT1DyVXC9uDVnXh/ugzqARLUMQtzB 6b/WRnsDsob326GQSf63scFzwoIdQ69Glbnfx1je65b1g9kTJh8PRcqLFSkg/+p7kw5h jVwQ== X-Gm-Message-State: AOJu0YwP9kbOJH7IfwreSUNeeZKPu/UuRoOhp2xT6q8RIde1AQ5lxl/X xKx/07lVmUGipamV+1JROkqJTw== X-Google-Smtp-Source: AGHT+IF1L3zszd0Qt7OImIjEsl7I2HCtreNc7f6XF9vPUvB7JGyRpTNFMRSxt9GUYpLv2X335et5TA== X-Received: by 2002:a17:902:e883:b0:1bc:506a:58f2 with SMTP id w3-20020a170902e88300b001bc506a58f2mr10371670plg.46.1694502136153; Tue, 12 Sep 2023 00:02:16 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:15 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 5/6] bpf: teach the verifier to enforce css_iter and process_iter in RCU CS Date: Tue, 12 Sep 2023 15:01:48 +0800 Message-Id: <20230912070149.969939-6-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" css_iter and process_iter should be used in rcu section. Specifically, in sleepable progs explicit bpf_rcu_read_lock() is needed before use these iters. In normal bpf progs that have implicit rcu_read_lock(), it's OK to use them directly. This patch checks whether we are in rcu cs before we want to invoke bpf_iter_process_new and bpf_iter_css_{pre, post}_new in mark_stack_slots_iter(). If the rcu protection is guaranteed, we would let st->type =3D PTR_TO_STACK | MEM_RCU. is_iter_reg_valid_init() will reject if reg->type is UNTRUSTED. Signed-off-by: Chuyi Zhou --- kernel/bpf/verifier.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 2367483bf4c2..6a6827ba7a18 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1172,7 +1172,13 @@ static bool is_dynptr_type_expected(struct bpf_verif= ier_env *env, struct bpf_reg =20 static void __mark_reg_known_zero(struct bpf_reg_state *reg); =20 +static bool in_rcu_cs(struct bpf_verifier_env *env); + +/* check whether we are using bpf_iter_process_*() or bpf_iter_css_*() */ +static bool is_iter_need_rcu(struct bpf_kfunc_call_arg_meta *meta); + static int mark_stack_slots_iter(struct bpf_verifier_env *env, + struct bpf_kfunc_call_arg_meta *meta, struct bpf_reg_state *reg, int insn_idx, struct btf *btf, u32 btf_id, int nr_slots) { @@ -1193,6 +1199,12 @@ static int mark_stack_slots_iter(struct bpf_verifier= _env *env, =20 __mark_reg_known_zero(st); st->type =3D PTR_TO_STACK; /* we don't have dedicated reg type */ + if (is_iter_need_rcu(meta)) { + if (in_rcu_cs(env)) + st->type |=3D MEM_RCU; + else + st->type |=3D PTR_UNTRUSTED; + } st->live |=3D REG_LIVE_WRITTEN; st->ref_obj_id =3D i =3D=3D 0 ? id : 0; st->iter.btf =3D btf; @@ -1281,6 +1293,8 @@ static bool is_iter_reg_valid_init(struct bpf_verifie= r_env *env, struct bpf_reg_ struct bpf_stack_state *slot =3D &state->stack[spi - i]; struct bpf_reg_state *st =3D &slot->spilled_ptr; =20 + if (st->type & PTR_UNTRUSTED) + return false; /* only main (first) slot has ref_obj_id set */ if (i =3D=3D 0 && !st->ref_obj_id) return false; @@ -7503,13 +7517,13 @@ static int process_iter_arg(struct bpf_verifier_env= *env, int regno, int insn_id return err; } =20 - err =3D mark_stack_slots_iter(env, reg, insn_idx, meta->btf, btf_id, nr_= slots); + err =3D mark_stack_slots_iter(env, meta, reg, insn_idx, meta->btf, btf_i= d, nr_slots); if (err) return err; } else { /* iter_next() or iter_destroy() expect initialized iter state*/ if (!is_iter_reg_valid_init(env, reg, meta->btf, btf_id, nr_slots)) { - verbose(env, "expected an initialized iter_%s as arg #%d\n", + verbose(env, "expected an initialized iter_%s as arg #%d or without bpf= _rcu_read_lock()\n", iter_type_str(meta->btf, btf_id), regno); return -EINVAL; } @@ -10382,6 +10396,18 @@ BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) BTF_ID(func, bpf_iter_css_task_new) =20 +BTF_SET_START(rcu_protect_kfuns_set) +BTF_ID(func, bpf_iter_process_new) +BTF_ID(func, bpf_iter_css_pre_new) +BTF_ID(func, bpf_iter_css_post_new) +BTF_SET_END(rcu_protect_kfuns_set) + +static inline bool is_iter_need_rcu(struct bpf_kfunc_call_arg_meta *meta) +{ + return btf_id_set_contains(&rcu_protect_kfuns_set, meta->func_id); +} + + static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { if (meta->func_id =3D=3D special_kfunc_list[KF_bpf_refcount_acquire_impl]= && --=20 2.20.1 From nobody Thu Dec 18 20:18:00 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6232DCA0EC3 for ; Tue, 12 Sep 2023 07:02:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231424AbjILHCy (ORCPT ); Tue, 12 Sep 2023 03:02:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231392AbjILHCl (ORCPT ); Tue, 12 Sep 2023 03:02:41 -0400 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B42B910C7 for ; Tue, 12 Sep 2023 00:02:19 -0700 (PDT) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1c336f3f449so45837165ad.3 for ; Tue, 12 Sep 2023 00:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502139; x=1695106939; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WHvChKGnnZi2VbMoC/1ghgca6iZmmMi6EKlv8J2G2fg=; b=Pz94yrZEm1gAewdNBZNyHYGilUQYcv52N/6CM7ofREcH9Azf91rO5OrplltOLjm/vK 2IjyqTcsuiHZR2OZlAWJlsf3XCAkc07jw5hrHF3czY017l5o1m+shrQ07R8ALKrHbDsM sqOozaHKk1//v8czq1/Et6jCeIAuhLqU8nfj+qGH4NUWsHn0unUwp4tQAmEIH0PA7Oyg F4tDd+v3U65B5PQXAz+S65m2l8CQVvAKbW20jgwvksYdCtctjwy30kapQemgdGTTH4Bl e/siybWVV/ldUfRAqjdF0+W9DLCPMZtZ++C9BLwYUypljfkYj9xhPuaPoz0TD54DAI2V 4p5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502139; x=1695106939; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WHvChKGnnZi2VbMoC/1ghgca6iZmmMi6EKlv8J2G2fg=; b=xBx6etKHWxG+Sb+SXLseIvGaY4W37g47KhQswVxq0vJFkCOfmK4vgKPJ/hMTXs3B7j Oks52tzHAYTLiVETbEzyZivcwHUxeLLpWmfSXzHSdZxW9+q74PCfz++El2CRPb3SetvV sRKl0QbEdNc508V0+YJTCBny/T5QcumD1rYFFMDh738fEF8XEBZ7b+GPCZ8T5EsXBDMi hY0/oevgU574Mj+Nx65Lg4Tf5it8DRV4mWd7OVMxeeyOcIRtd0Bs5cnzJKbO1hDpYsui LJTPdLfD38J1E1JdtyhMkVCeeENTtU2hXanPslIOOWL+2lZ+VK6rdvgRSo2e1oCVp0k1 rsiw== X-Gm-Message-State: AOJu0YzZ2qMq+i9S2eSYzEmmG5akluiePOst4fK/wfI0bPrAKj0lTRub 5uPiok79EMT/2T1gWGhap43cwQ== X-Google-Smtp-Source: AGHT+IGAzzXLvSkqyFtsQ81mmEO/NsorZtgGazng17R/SZuKMut3kM4aZbbVuYOInENRSv12IHsRgQ== X-Received: by 2002:a17:902:b287:b0:1c3:7628:fcbb with SMTP id u7-20020a170902b28700b001c37628fcbbmr10807814plr.43.1694502139227; Tue, 12 Sep 2023 00:02:19 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:19 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 6/6] selftests/bpf: Add tests for open-coded task and css iter Date: Tue, 12 Sep 2023 15:01:49 +0800 Message-Id: <20230912070149.969939-7-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch adds three tests to demonstrate these patterns and validating correctness. test1: We use bpf_for_each(process, task) to iterate all process in the system and search for the current process with a given pid. test2: We create a cgroup and add the current task to the cgroup. In the BPF program, we would use bpf_for_each(css_task, task, css) to iterate all tasks under the cgroup. As expected, we would find the current process. test3: We create a cgroup tree. In the BPF program, we use bpf_for_each(css_{pre,post}, pos, root) to iterate all descendant under the root with pre and post order. As expected, we would find all descendant and the last iterating cgroup in post-order is root cgroup, the first iterating cgroup in pre-order is root cgroup. Signed-off-by: Chuyi Zhou --- .../testing/selftests/bpf/prog_tests/iters.c | 138 ++++++++++++++++++ .../testing/selftests/bpf/progs/iters_task.c | 104 +++++++++++++ 2 files changed, 242 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/iters_task.c diff --git a/tools/testing/selftests/bpf/prog_tests/iters.c b/tools/testing= /selftests/bpf/prog_tests/iters.c index 10804ae5ae97..f4e69a506509 100644 --- a/tools/testing/selftests/bpf/prog_tests/iters.c +++ b/tools/testing/selftests/bpf/prog_tests/iters.c @@ -1,13 +1,21 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ =20 +#include +#include +#include +#include +#include +#include #include +#include "cgroup_helpers.h" =20 #include "iters.skel.h" #include "iters_state_safety.skel.h" #include "iters_looping.skel.h" #include "iters_num.skel.h" #include "iters_testmod_seq.skel.h" +#include "iters_task.skel.h" =20 static void subtest_num_iters(void) { @@ -90,6 +98,130 @@ static void subtest_testmod_seq_iters(void) iters_testmod_seq__destroy(skel); } =20 +static void subtest_process_iters(void) +{ + struct iters_task *skel; + int err; + + skel =3D iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + bpf_program__set_autoload(skel->progs.iter_task_for_each_sleep, true); + err =3D iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + skel->bss->target_pid =3D getpid(); + err =3D iters_task__attach(skel); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + syscall(SYS_getpgid); + iters_task__detach(skel); + ASSERT_EQ(skel->bss->process_cnt, 1, "process_cnt"); + +cleanup: + iters_task__destroy(skel); +} + +extern int stack_mprotect(void); + +static void subtest_css_task_iters(void) +{ + struct iters_task *skel; + int err, cg_fd, cg_id; + const char *cgrp_path =3D "/cg1"; + + err =3D setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + cg_fd =3D create_and_get_cgroup(cgrp_path); + if (!ASSERT_GE(cg_fd, 0, "cg_create")) + goto cleanup; + cg_id =3D get_cgroup_id(cgrp_path); + err =3D join_cgroup(cgrp_path); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + + skel =3D iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + + bpf_program__set_autoload(skel->progs.iter_css_task_for_each, true); + err =3D iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + + skel->bss->target_pid =3D getpid(); + skel->bss->cg_id =3D cg_id; + err =3D iters_task__attach(skel); + + err =3D stack_mprotect(); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + + iters_task__detach(skel); + ASSERT_EQ(skel->bss->css_task_cnt, 1, "css_task_cnt"); + +cleanup: + cleanup_cgroup_environment(); + iters_task__destroy(skel); +} + +static void subtest_css_dec_iters(void) +{ + struct iters_task *skel; + struct { + const char *path; + int fd; + } cgs[] =3D { + { "/cg1" }, + { "/cg1/cg2" }, + { "/cg1/cg2/cg3" }, + { "/cg1/cg2/cg3/cg4" }, + }; + int err, cg_nr =3D ARRAY_SIZE(cgs); + int i; + + err =3D setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + for (i =3D 0; i < cg_nr; i++) { + cgs[i].fd =3D create_and_get_cgroup(cgs[i].path); + if (!ASSERT_GE(cgs[i].fd, 0, "cg_create")) + goto cleanup; + } + + skel =3D iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + bpf_program__set_autoload(skel->progs.iter_css_dec_for_each, true); + err =3D iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + + skel->bss->target_pid =3D getpid(); + skel->bss->cg_id =3D get_cgroup_id(cgs[0].path); + + err =3D iters_task__attach(skel); + + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + + syscall(SYS_getpgid); + ASSERT_EQ(skel->bss->css_dec_cnt, cg_nr, "pre order search dec count"); + ASSERT_EQ(skel->bss->first_cg_id, get_cgroup_id(cgs[0].path), + "pre order search first cgroup id"); + skel->bss->css_dec_cnt =3D 0; + skel->bss->is_post_order =3D 1; + syscall(SYS_getpgid); + ASSERT_EQ(skel->bss->css_dec_cnt, cg_nr, "post order search dec count"); + ASSERT_EQ(skel->bss->last_cg_id, get_cgroup_id(cgs[0].path), + "post order search last cgroup id"); + iters_task__detach(skel); +cleanup: + cleanup_cgroup_environment(); + iters_task__destroy(skel); +} + void test_iters(void) { RUN_TESTS(iters_state_safety); @@ -103,4 +235,10 @@ void test_iters(void) subtest_num_iters(); if (test__start_subtest("testmod_seq")) subtest_testmod_seq_iters(); + if (test__start_subtest("process")) + subtest_process_iters(); + if (test__start_subtest("css_task")) + subtest_css_task_iters(); + if (test__start_subtest("css_dec")) + subtest_css_dec_iters(); } diff --git a/tools/testing/selftests/bpf/progs/iters_task.c b/tools/testing= /selftests/bpf/progs/iters_task.c new file mode 100644 index 000000000000..cf24a3b177f2 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_task.c @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +char _license[] SEC("license") =3D "GPL"; + +pid_t target_pid =3D 0; +int process_cnt =3D 0; +int css_task_cnt =3D 0; +int css_dec_cnt =3D 0; + +char is_post_order; +u64 cg_id; +u64 last_cg_id; +u64 first_cg_id; + +struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; +struct cgroup *bpf_cgroup_acquire(struct cgroup *cgrp) __ksym; +void bpf_cgroup_release(struct cgroup *p) __ksym; +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_task_for_each_sleep(void *ctx) +{ + struct task_struct *task; + struct task_struct *cur_task =3D bpf_get_current_task_btf(); + + if (cur_task->pid !=3D target_pid) + return 0; + bpf_rcu_read_lock(); + bpf_for_each(process, task) { + if (task->pid =3D=3D target_pid) + process_cnt +=3D 1; + } + bpf_rcu_read_unlock(); + return 0; +} + + +SEC("?lsm/file_mprotect") +int BPF_PROG(iter_css_task_for_each) +{ + struct task_struct *task; + struct task_struct *cur_task =3D bpf_get_current_task_btf(); + + if (cur_task->pid !=3D target_pid) + return 0; + + struct cgroup *cgrp =3D bpf_cgroup_from_id(cg_id); + + if (cgrp =3D=3D NULL) + return 0; + struct cgroup_subsys_state *css =3D &cgrp->self; + + bpf_for_each(css_task, task, css, CSS_TASK_ITER_PROCS) { + if (!task) + continue; + if (task->pid =3D=3D target_pid) + css_task_cnt +=3D 1; + } + bpf_cgroup_release(cgrp); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_css_dec_for_each(const void *ctx) +{ + struct task_struct *cur_task =3D bpf_get_current_task_btf(); + + if (cur_task->pid !=3D target_pid) + return 0; + + struct cgroup *cgrp =3D bpf_cgroup_from_id(cg_id); + + if (cgrp =3D=3D NULL) + return 0; + struct cgroup_subsys_state *root =3D &cgrp->self; + struct cgroup_subsys_state *pos =3D NULL; + + bpf_rcu_read_lock(); + if (is_post_order) { + bpf_for_each(css_post, pos, root) { + struct cgroup *cur_cgrp =3D pos->cgroup; + + css_dec_cnt +=3D 1; + if (cur_cgrp) + last_cg_id =3D cur_cgrp->kn->id; + } + } else { + bpf_for_each(css_pre, pos, root) { + struct cgroup *cur_cgrp =3D pos->cgroup; + + css_dec_cnt +=3D 1; + if (cur_cgrp && !first_cg_id) + first_cg_id =3D cur_cgrp->kn->id; + } + } + bpf_rcu_read_unlock(); + bpf_cgroup_release(cgrp); + return 0; +} + --=20 2.20.1