From nobody Sat Feb 7 11:52:39 2026 Received: from out-181.mta1.migadu.com (out-181.mta1.migadu.com [95.215.58.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48332350D6E for ; Tue, 27 Jan 2026 16:34:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769531658; cv=none; b=XrqyV1is4qwn0IRyZDBhqzeVWk/aq/7gyyCsFCx+B15W4viRX6ZNyhPd4wzY8RVsUZ4rsv1u4hdEjB5iAoHF6b8BG40wvk2bt3q3y/9U4L2VniqXOomFZ+0tsNdQ3cp9qSCChW0laS3lgndOdtw2wJ/S6VE4NanDLxgSL4VYls4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769531658; c=relaxed/simple; bh=1Wl0HSaQlEpTHhLn1sEl13pYO6+HmJSNslydlD9iNbU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=d6esOnr1BXOUcBiIFMoWSNZJvrfqbiuhRxeNZTQWkIXuVwf6lkH7XZ63Povou8mOGiauGk1GXeX4RZE99xOuHM46nEJuKKKPT7ZU/3Y573sS3LIjSa7y/Col6fU64tRxQX4xr+BfmNKRuTefmB/boOZDjQJBaUIxbdW6aELQda8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Xieb4/5I; arc=none smtp.client-ip=95.215.58.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Xieb4/5I" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1769531654; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2CS6NuVA0swIbhooY5O62ghjEkLWzlf9aXLW2T2B2NU=; b=Xieb4/5ItdoV608vLYsgdVcXovWlyKFlh+rXLfiyuurux9uTRQKe5tvoD4XbOaRwL92a9W G5b5bZGK/D/gO1IA0gIIOVillYjUsm/QaKFoQlT8iX8jwkC/zB9aC28RkQdV4NjUlE55O5 MU+8qtmD1oWxXVv0JhH3G/lmXSXwI8E= From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Puranjay Mohan , Xu Kuohai , Catalin Marinas , Will Deacon , Shuah Khan , Menglong Dong , Leon Hwang , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com Subject: [PATCH bpf-next 1/2] bpf, arm64: Add fsession support Date: Wed, 28 Jan 2026 00:33:43 +0800 Message-ID: <20260127163344.92819-2-leon.hwang@linux.dev> In-Reply-To: <20260127163344.92819-1-leon.hwang@linux.dev> References: <20260127163344.92819-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Implement fsession support in the arm64 BPF JIT trampoline. Extend the trampoline stack layout to store function metadata and session cookies, and pass the appropriate metadata to fentry and fexit programs. This mirrors the existing x86 behavior and enables call session cookies on arm64. Signed-off-by: Leon Hwang --- arch/arm64/net/bpf_jit_comp.c | 66 ++++++++++++++++++++++++++++++----- include/linux/bpf.h | 7 +++- 2 files changed, 63 insertions(+), 10 deletions(-) diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 0c4d44bcfbf4..3272dbfc3aa7 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -2510,6 +2510,12 @@ static bool is_struct_ops_tramp(const struct bpf_tra= mp_links *fentry_links) fentry_links->links[0]->link.type =3D=3D BPF_LINK_TYPE_STRUCT_OPS; } =20 +static void store_func_meta(struct jit_ctx *ctx, u64 func_meta, int func_m= eta_off) +{ + emit_a64_mov_i64(A64_R(10), func_meta, ctx); + emit(A64_STR64I(A64_R(10), A64_SP, func_meta_off), ctx); +} + /* Based on the x86's implementation of arch_prepare_bpf_trampoline(). * * bpf prog and function entry before bpf trampoline hooked: @@ -2533,7 +2539,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, st= ruct bpf_tramp_image *im, int regs_off; int retval_off; int bargs_off; - int nfuncargs_off; + int func_meta_off; int ip_off; int run_ctx_off; int oargs_off; @@ -2544,6 +2550,9 @@ static int prepare_trampoline(struct jit_ctx *ctx, st= ruct bpf_tramp_image *im, bool save_ret; __le32 **branches =3D NULL; bool is_struct_ops =3D is_struct_ops_tramp(fentry); + int cookie_off, cookie_cnt, cookie_bargs_off; + int fsession_cnt =3D bpf_fsession_cnt(tlinks); + u64 func_meta; =20 /* trampoline stack layout: * [ parent ip ] @@ -2562,10 +2571,14 @@ static int prepare_trampoline(struct jit_ctx *ctx, = struct bpf_tramp_image *im, * [ ... ] * SP + bargs_off [ arg reg 1 ] for bpf * - * SP + nfuncargs_off [ arg regs count ] + * SP + func_meta_off [ regs count, etc ] * * SP + ip_off [ traced function ] BPF_TRAMP_F_IP_ARG flag * + * [ stack cookie N ] + * [ ... ] + * SP + cookie_off [ stack cookie 1 ] + * * SP + run_ctx_off [ bpf_tramp_run_ctx ] * * [ stack arg N ] @@ -2582,13 +2595,18 @@ static int prepare_trampoline(struct jit_ctx *ctx, = struct bpf_tramp_image *im, /* room for bpf_tramp_run_ctx */ stack_size +=3D round_up(sizeof(struct bpf_tramp_run_ctx), 8); =20 + cookie_off =3D stack_size; + /* room for session cookies */ + cookie_cnt =3D bpf_fsession_cookie_cnt(tlinks); + stack_size +=3D cookie_cnt * 8; + ip_off =3D stack_size; /* room for IP address argument */ if (flags & BPF_TRAMP_F_IP_ARG) stack_size +=3D 8; =20 - nfuncargs_off =3D stack_size; - /* room for args count */ + func_meta_off =3D stack_size; + /* room for function metadata, such as regs count */ stack_size +=3D 8; =20 bargs_off =3D stack_size; @@ -2646,9 +2664,9 @@ static int prepare_trampoline(struct jit_ctx *ctx, st= ruct bpf_tramp_image *im, emit(A64_STR64I(A64_R(10), A64_SP, ip_off), ctx); } =20 - /* save arg regs count*/ - emit(A64_MOVZ(1, A64_R(10), nfuncargs, 0), ctx); - emit(A64_STR64I(A64_R(10), A64_SP, nfuncargs_off), ctx); + /* save function metadata */ + func_meta =3D nfuncargs; + store_func_meta(ctx, func_meta, func_meta_off); =20 /* save args for bpf */ save_args(ctx, bargs_off, oargs_off, m, a, false); @@ -2666,10 +2684,27 @@ static int prepare_trampoline(struct jit_ctx *ctx, = struct bpf_tramp_image *im, emit_call((const u64)__bpf_tramp_enter, ctx); } =20 - for (i =3D 0; i < fentry->nr_links; i++) + if (fsession_cnt) { + /* clear all the session cookies' value */ + emit(A64_MOVZ(1, A64_R(10), 0, 0), ctx); + for (int i =3D 0; i < cookie_cnt; i++) + emit(A64_STR64I(A64_R(10), A64_SP, cookie_off + 8 * i), ctx); + /* clear the return value to make sure fentry always gets 0 */ + emit(A64_STR64I(A64_R(10), A64_SP, retval_off), ctx); + } + + cookie_bargs_off =3D (bargs_off - cookie_off) / 8; + for (i =3D 0; i < fentry->nr_links; i++) { + if (bpf_link_prog_session_cookie(fentry->links[i])) { + u64 meta =3D func_meta | (cookie_bargs_off << BPF_TRAMP_COOKIE_INDEX_SH= IFT); + + store_func_meta(ctx, meta, func_meta_off); + cookie_bargs_off--; + } invoke_bpf_prog(ctx, fentry->links[i], bargs_off, retval_off, run_ctx_off, flags & BPF_TRAMP_F_RET_FENTRY_RET); + } =20 if (fmod_ret->nr_links) { branches =3D kcalloc(fmod_ret->nr_links, sizeof(__le32 *), @@ -2701,9 +2736,22 @@ static int prepare_trampoline(struct jit_ctx *ctx, s= truct bpf_tramp_image *im, *branches[i] =3D cpu_to_le32(A64_CBNZ(1, A64_R(10), offset)); } =20 - for (i =3D 0; i < fexit->nr_links; i++) + /* set the "is_return" flag for fsession */ + func_meta |=3D (1ULL << BPF_TRAMP_IS_RETURN_SHIFT); + if (fsession_cnt) + store_func_meta(ctx, func_meta, func_meta_off); + + cookie_bargs_off =3D (bargs_off - cookie_off) / 8; + for (i =3D 0; i < fexit->nr_links; i++) { + if (bpf_link_prog_session_cookie(fexit->links[i])) { + u64 meta =3D func_meta | (cookie_bargs_off << BPF_TRAMP_COOKIE_INDEX_SH= IFT); + + store_func_meta(ctx, meta, func_meta_off); + cookie_bargs_off--; + } invoke_bpf_prog(ctx, fexit->links[i], bargs_off, retval_off, run_ctx_off, false); + } =20 if (flags & BPF_TRAMP_F_CALL_ORIG) { im->ip_epilogue =3D ctx->ro_image + ctx->idx; diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 4427c6e98331..b299d1206bfc 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2192,13 +2192,18 @@ static inline int bpf_fsession_cnt(struct bpf_tramp= _links *links) return cnt; } =20 +static inline bool bpf_link_prog_session_cookie(struct bpf_tramp_link *lin= k) +{ + return link->link.prog->call_session_cookie; +} + static inline int bpf_fsession_cookie_cnt(struct bpf_tramp_links *links) { struct bpf_tramp_links fentries =3D links[BPF_TRAMP_FENTRY]; int cnt =3D 0; =20 for (int i =3D 0; i < links[BPF_TRAMP_FENTRY].nr_links; i++) { - if (fentries.links[i]->link.prog->call_session_cookie) + if (bpf_link_prog_session_cookie(fentries.links[i])) cnt++; } =20 --=20 2.52.0 From nobody Sat Feb 7 11:52:39 2026 Received: from out-176.mta1.migadu.com (out-176.mta1.migadu.com [95.215.58.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3197F360746 for ; Tue, 27 Jan 2026 16:34:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769531666; cv=none; b=r1QigEMg4KOPQmBX5l7D8RY2uITflWOuH0IEalpy7JOH/uZgeUSf6mU/m/8CzNVnIF+hNwrUzr3I6frdTmfsX4Vyw8ddIaf6d3slK0/nyqPonSVY4Q/ZUaZWP+D0mirfsISMUrFaTchtvZv7lnhV+rMSc4y5j6h8Izy0x+ERGfE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769531666; c=relaxed/simple; bh=W0Cbpf0C4qMtUBveQ/36bOrYZCTa24nxvp5lxfIYjQw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nCNONHmoBcozBzocQzK2Vs+CAVk8UkheuLPa1lXHlvNlGOP9YiYOudgGJdE6aBvmbSc5OvGXZeccpQYj/bKiQQIQnrihXkGciUpwYgZa/QfLV8xvFyim7PA+7u6hE5ZV3PKdFhRIP+6DYbh7JD48ha+QSXgbGk8nrvg2EQPnvh4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Eorp8ZaS; arc=none smtp.client-ip=95.215.58.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Eorp8ZaS" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1769531663; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=74wISzY+I2QQWizqfpjLw+tcNfcYrDh+eakCr6Jmco0=; b=Eorp8ZaSdCSRoAjQbof+94IZ9zp2MDPOWzukFJnbQY3XvfAAIncEUZxp0wMjL2iTdd6B4i m4eFdbLoscvu/8JefPTCCxGIj6x4aqiz3bEUC4TkyAHAURVF3KenJisoONoBn3pTFfileU YF5lcs50xT0TkGlGl11qSkGmkto2nNo= From: Leon Hwang To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Puranjay Mohan , Xu Kuohai , Catalin Marinas , Will Deacon , Shuah Khan , Menglong Dong , Leon Hwang , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-patches-bot@fb.com Subject: [PATCH bpf-next 2/2] bpf/selftests: Enable fsession and get_func_ tests on arm64* Date: Wed, 28 Jan 2026 00:33:44 +0800 Message-ID: <20260127163344.92819-3-leon.hwang@linux.dev> In-Reply-To: <20260127163344.92819-1-leon.hwang@linux.dev> References: <20260127163344.92819-1-leon.hwang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Allow fsession, get_func_args, and get_func_ip selftests to run on arm64. Signed-off-by: Leon Hwang --- tools/testing/selftests/bpf/prog_tests/fsession_test.c | 2 +- tools/testing/selftests/bpf/progs/get_func_args_test.c | 2 +- tools/testing/selftests/bpf/progs/get_func_ip_test.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/fsession_test.c b/tools= /testing/selftests/bpf/prog_tests/fsession_test.c index 0c4b428e1cee..d0cc6b1e29bf 100644 --- a/tools/testing/selftests/bpf/prog_tests/fsession_test.c +++ b/tools/testing/selftests/bpf/prog_tests/fsession_test.c @@ -111,7 +111,7 @@ static void test_fsession_cookie(void) =20 void test_fsession_test(void) { -#if !defined(__x86_64__) +#if !defined(__x86_64__) && !defined(__aarch64__) test__skip(); return; #endif diff --git a/tools/testing/selftests/bpf/progs/get_func_args_test.c b/tools= /testing/selftests/bpf/progs/get_func_args_test.c index 0a3236a7a109..180ba5098ca1 100644 --- a/tools/testing/selftests/bpf/progs/get_func_args_test.c +++ b/tools/testing/selftests/bpf/progs/get_func_args_test.c @@ -167,7 +167,7 @@ int BPF_PROG(tp_test2) } =20 __u64 test7_result =3D 0; -#ifdef __TARGET_ARCH_x86 +#if defined(bpf_target_x86) || defined(bpf_target_arm64) SEC("fsession/bpf_fentry_test1") int BPF_PROG(test7) { diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_test.c b/tools/t= esting/selftests/bpf/progs/get_func_ip_test.c index 65f7e1f182bf..43ff836a8ed8 100644 --- a/tools/testing/selftests/bpf/progs/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/progs/get_func_ip_test.c @@ -106,7 +106,7 @@ int BPF_URETPROBE(test8, int ret) =20 __u64 test9_entry_result =3D 0; __u64 test9_exit_result =3D 0; -#ifdef __TARGET_ARCH_x86 +#if defined(bpf_target_x86) || defined(bpf_target_arm64) SEC("fsession/bpf_fentry_test1") int BPF_PROG(test9, int a) { --=20 2.52.0