From nobody Sat Feb 7 08:44:12 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A36FD3161A6; Mon, 5 Jan 2026 10:53:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610404; cv=none; b=S0iFgqHW04wryWMULKsINui3TvTpCwxZRF88QmQeSw5exOuCENPoe0TL5RqzNQE3pcj0DOLZEjkCDI2uOd2msL5/8M+0BkcZ03rDnPhaCqRyUMQefUN4po/hxrOI/XAzIaWywyTxtl+ZS2KBFd75F/DaTEHzzLuN+uT5vT1457g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610404; c=relaxed/simple; bh=Pf6uq9Sk3kzC39d7Q4RTA3JzF0t3R8suCXTkzOTbqgk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ngTVOGMbrMTYDbKQZ/7vekBwS8em7g01y9uaBYcVQii3bbzshVjBi/VYcAJmsy8hfC18IZobvoYr8XC6NX5UKcPTnLx2PZE56gg7XvgNNgcxWlz0nl1gaMdF/02dnEMBu20E7JtzdWxp32CWyBM+UJTgbty/GslruDHGFn6+WGA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Xek50DoO; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Xek50DoO" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 605ACpi3012994; Mon, 5 Jan 2026 10:52:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=zv5IYtFTVrb5C5g07 3T1pvBbb4HNP4RHEYsB7V1X9gI=; b=Xek50DoOgHY+RkzFYKl1UfJ78Sc56taQE fipbFX7frH+vSPvv3ONSHQCWxd8OW7DFs+MeUtoMj7mPe3lD9D04lBMnXVnXCshp G9yJ7ipHc7V9Krv+7+q4h5rGsdRxYgkvOGPG9+5t3eA6fS/TOXMjCM3sk4qByuLX 1/0umV1Zq0vHMBHbIEF5NxMfIVrYipNhfPy+HexZtytiBWPiTKS4gFCJ69XZ+QfY 9CikwikLUQUVcKF0Xt+0mCgUXLcR9Zde05ou3tD4UPUlcgmm1T0HztwQTe21obdH DLj/635yl5FoOwHsadKlrtbmBinZ15nYMaP8gyC0P7FaqNlKq+XrQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betrte673-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:52:38 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 605Aqb7F010937; Mon, 5 Jan 2026 10:52:37 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betrte671-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:52:37 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 605AFxdL015233; Mon, 5 Jan 2026 10:52:36 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4bfdes5djq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:52:36 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 605AqXEK48038354 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Jan 2026 10:52:33 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0737720040; Mon, 5 Jan 2026 10:52:33 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F124A20049; Mon, 5 Jan 2026 10:52:26 +0000 (GMT) Received: from abhi.. (unknown [9.124.213.127]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 5 Jan 2026 10:52:26 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH 1/6] powerpc64/bpf: Support tailcalls with subprogs Date: Mon, 5 Jan 2026 16:22:07 +0530 Message-ID: <20260105105212.136645-2-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260105105212.136645-1-adubey@linux.ibm.com> References: <20260105105212.136645-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=aaJsXBot c=1 sm=1 tr=0 ts=695b97f6 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=kwRpZc0szV6nT_PZ_DcA:9 X-Proofpoint-GUID: lF21zfAvmDGrwsYBVwLpxdNGAqGXBLP- X-Proofpoint-ORIG-GUID: 7TdGhl2ur51Ib35N2yxIsvsi8Vu6YzC9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTA1MDA5NSBTYWx0ZWRfX4710Hz/rHE7G H7H19ND+7Uc0qhAcuGTAYLyV+/xY7kg3k97+W1/45LO0uxuePBrHNVenco7QeWwPurP8lF1D8AZ soZWaj3N7hGBy8snaNVB+iIHFDvpQmGrVpHd+GpUiaJlzLOtzYxuwm0CIFDQBLVCfiupqDPwI3V vTiBQqxzHDD5j0AtPsqlF1oOiLhGeW5wfZkx0LPKl0UhJHzJZUOLrUOikeZJaf7Vguzg/f8IS4Q 7hXXPnQme+u7mb60Buc1FFP0jA27Dw2WHbYEKhp+Ew0LplspH+MOQxqSNrbk5xzewn3/39LS1kB wpI1jlmFpAAR2rX5iKduOTuRqts8pI/5MKvrU5ukY/cMxXguPDQ7fzjFTntPZDC25PBfE7gprKn Ht3iDoR8Ou8XsOUl30XAioU4O3nKa9/xUKqtXwtdbChPUv/xIx+qY32p+sECkDAYTQtJXQ/4Lx7 TjF51T1IkSS+q3Zc/lg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-05_01,2025-12-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 lowpriorityscore=0 bulkscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 phishscore=0 spamscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601050095 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey Enabling tailcalls with subprog combinations by referencing method. The actual tailcall count is always maintained in the tail_call_info variable present in the frame of main function (also called entry function). The tail_call_info variables in the frames of all other subprog contains reference to the tail_call_info present in frame of main function. Dynamic resolution interprets the tail_call_info either as value or reference depending on the context of active frame while tailcall is invoked. Following is selftest run: #./test_progs -t tailcalls #425/1 tailcalls/tailcall_1:OK #425/2 tailcalls/tailcall_2:OK #425/3 tailcalls/tailcall_3:OK #425/4 tailcalls/tailcall_4:OK #425/5 tailcalls/tailcall_5:OK #425/6 tailcalls/tailcall_6:OK #425/7 tailcalls/tailcall_bpf2bpf_1:OK #425/8 tailcalls/tailcall_bpf2bpf_2:OK #425/9 tailcalls/tailcall_bpf2bpf_3:OK #425/10 tailcalls/tailcall_bpf2bpf_4:OK #425/11 tailcalls/tailcall_bpf2bpf_5:OK #425/12 tailcalls/tailcall_bpf2bpf_6:OK #425/13 tailcalls/tailcall_bpf2bpf_fentry:OK #425/14 tailcalls/tailcall_bpf2bpf_fexit:OK #425/15 tailcalls/tailcall_bpf2bpf_fentry_fexit:OK #425/16 tailcalls/tailcall_bpf2bpf_fentry_entry:OK #425/17 tailcalls/tailcall_poke:OK #425/18 tailcalls/tailcall_bpf2bpf_hierarchy_1:OK #425/19 tailcalls/tailcall_bpf2bpf_hierarchy_fentry:OK #425/20 tailcalls/tailcall_bpf2bpf_hierarchy_fexit:OK #425/21 tailcalls/tailcall_bpf2bpf_hierarchy_fentry_fexit:OK #425/22 tailcalls/tailcall_bpf2bpf_hierarchy_fentry_entry:OK #425/23 tailcalls/tailcall_bpf2bpf_hierarchy_2:OK #425/24 tailcalls/tailcall_bpf2bpf_hierarchy_3:OK #425/25 tailcalls/tailcall_freplace:OK #425/26 tailcalls/tailcall_bpf2bpf_freplace:OK #425/27 tailcalls/tailcall_failure:OK #425/28 tailcalls/reject_tail_call_spin_lock:OK #425/29 tailcalls/reject_tail_call_rcu_lock:OK #425/30 tailcalls/reject_tail_call_preempt_lock:OK #425/31 tailcalls/reject_tail_call_ref:OK #425 tailcalls:OK Summary: 1/31 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit.h | 14 ++++++- arch/powerpc/net/bpf_jit_comp.c | 10 ++++- arch/powerpc/net/bpf_jit_comp64.c | 67 +++++++++++++++++++++++-------- 3 files changed, 71 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 8334cd667bba..98e8b1f9c2f9 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -51,6 +51,12 @@ EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc)= ); \ } while (0) =20 +/* Same as PPC_BCC_SHORT, except valid dest is known prior to call. */ +#define PPC_COND_BRANCH(cond, dest) = \ + do { \ + long offset =3D (long)(dest) - CTX_NIA(ctx); \ + EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (of= fset & 0xfffc)); \ + } while (0) /* * Sign-extended 32-bit immediate load * @@ -72,6 +78,10 @@ } } while (0) =20 #ifdef CONFIG_PPC64 + +/* for gpr non volatile registers BPG_REG_6 to 10 */ +#define BPF_PPC_STACK_SAVE (6*8) + /* If dummy pass (!image), account for maximum possible instructions */ #define PPC_LI64(d, i) do { \ if (!image) \ @@ -166,6 +176,7 @@ struct codegen_context { unsigned int alt_exit_addr; u64 arena_vm_start; u64 user_vm_start; + bool is_subprog; }; =20 #define bpf_to_ppc(r) (ctx->b2p[r]) @@ -200,11 +211,10 @@ void bpf_jit_build_epilogue(u32 *image, struct codege= n_context *ctx); void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx); void bpf_jit_realloc_regs(struct codegen_context *ctx); int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tm= p_reg, long exit_addr); - int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, in= t pass, struct codegen_context *ctx, int insn_idx, int jmp_off, int dst_reg, u32 code); - +int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx); #endif =20 #endif diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 5e976730b2f5..069a8822c30d 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -206,6 +206,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *f= p) cgctx.stack_size =3D round_up(fp->aux->stack_depth, 16); cgctx.arena_vm_start =3D bpf_arena_get_kern_vm_start(fp->aux->arena); cgctx.user_vm_start =3D bpf_arena_get_user_vm_start(fp->aux->arena); + cgctx.is_subprog =3D bpf_is_subprog(fp); =20 /* Scouting faux-generate pass 0 */ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) { @@ -435,6 +436,11 @@ void bpf_jit_free(struct bpf_prog *fp) bpf_prog_unlock_free(fp); } =20 +bool bpf_jit_supports_subprog_tailcalls(void) +{ + return IS_ENABLED(CONFIG_PPC64); +} + bool bpf_jit_supports_kfunc_call(void) { return true; @@ -604,7 +610,7 @@ static void bpf_trampoline_setup_tail_call_cnt(u32 *ima= ge, struct codegen_contex int func_frame_offset, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { - /* See bpf_jit_stack_tailcallcnt() */ + /* See bpf_jit_stack_tailcallinfo_offset() */ int tailcallcnt_offset =3D 7 * 8; =20 EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); @@ -619,7 +625,7 @@ static void bpf_trampoline_restore_tail_call_cnt(u32 *i= mage, struct codegen_cont int func_frame_offset, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { - /* See bpf_jit_stack_tailcallcnt() */ + /* See bpf_jit_stack_tailcallinfo_offset() */ int tailcallcnt_offset =3D 7 * 8; =20 EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset)); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 1fe37128c876..37c547b49da8 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -24,17 +24,19 @@ * Ensure the top half (upto local_tmp_var) stays consistent * with our redzone usage. * + * tail_call_info - stores tailcall count value in main program's + * frame, stores reference to tail_call_info of + * main's frame in sub-prog's frame. + * * [ prev sp ] <------------- * [ nv gpr save area ] 6*8 | - * [ tail_call_cnt ] 8 | + * [ tail_call_info ] 8 | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | * [ frame header ] 32/112 | * sp (r1) ---> [ stack pointer ] -------------- */ =20 -/* for gpr non volatile registers BPG_REG_6 to 10 */ -#define BPF_PPC_STACK_SAVE (6*8) /* for bpf JIT code internal usage */ #define BPF_PPC_STACK_LOCALS 32 /* stack frame excluding BPF stack, ensure this is quadword aligned */ @@ -93,7 +95,7 @@ static inline bool bpf_has_stack_frame(struct codegen_con= text *ctx) * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- * [ nv gpr save area ] 6*8 - * [ tail_call_cnt ] 8 + * [ tail_call_info ] 8 * [ local_tmp_var ] 24 * [ unused red zone ] 224 */ @@ -105,7 +107,7 @@ static int bpf_jit_stack_local(struct codegen_context *= ctx) return -(BPF_PPC_STACK_SAVE + 32); } =20 -static int bpf_jit_stack_tailcallcnt(struct codegen_context *ctx) +int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) { return bpf_jit_stack_local(ctx) + 24; } @@ -138,17 +140,31 @@ void bpf_jit_build_prologue(u32 *image, struct codege= n_context *ctx) #endif =20 /* - * Initialize tail_call_cnt if we do tail calls. - * Otherwise, put in NOPs so that it can be skipped when we are - * invoked through a tail call. + * Tail call count(tcc) is saved & updated only in main + * program's frame and the address of tcc in main program's + * frame (tcc_ptr) is saved in subprogs frame. + * + * Offset of tail_call_info on any frame will be interpreted + * as either tcc_ptr or tcc value depending on whether it is + * greater than MAX_TAIL_CALL_CNT or not. */ - if (ctx->seen & SEEN_TAILCALL) { + if (!ctx->is_subprog) { EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0)); /* this goes in the redzone */ EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_STACK_SAVE + 8))); } else { - EMIT(PPC_RAW_NOP()); - EMIT(PPC_RAW_NOP()); + /* + * if tail_call_info < MAX_TAIL_CALL_CNT + * main prog calling first subprog -> copy reference + * else + * subsequent subprog calling another subprog -> directly copy content + */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, 0)); + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), -(BPF_PPC_= STACK_SAVE+8))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + PPC_COND_BRANCH(COND_GT, CTX_NIA(ctx) + 8); + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), -(BPF_PP= C_STACK_SAVE + 8))); + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_STACK_SAVE + 8))); } =20 if (bpf_has_stack_frame(ctx)) { @@ -343,19 +359,38 @@ static int bpf_jit_emit_tail_call(u32 *image, struct = codegen_context *ctx, u32 o EMIT(PPC_RAW_CMPLW(b2p_index, bpf_to_ppc(TMP_REG_1))); PPC_BCC_SHORT(COND_GE, out); =20 + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallinfo_of= fset(ctx))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + PPC_COND_BRANCH(COND_LE, CTX_NIA(ctx) + 8); + + /* dereference TMP_REG_1 */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 0)); + /* - * if (tail_call_cnt >=3D MAX_TAIL_CALL_CNT) + * if (tail_call_info =3D=3D MAX_TAIL_CALL_CNT) * goto out; */ - EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallcnt(ctx= ))); EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); - PPC_BCC_SHORT(COND_GE, out); + PPC_COND_BRANCH(COND_EQ, out); =20 /* - * tail_call_cnt++; + * tail_call_info++; <- Actual value of tcc here */ EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 1)); - EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallcnt(ct= x))); + + /* + * Before writing updated tail_call_info, distinguish if current frame + * is storing a reference to tail_call_info or actual tcc value in + * tail_call_info. + */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, bpf_jit_stack_tailcallinfo_of= fset(ctx))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_2), MAX_TAIL_CALL_CNT)); + PPC_COND_BRANCH(COND_GT, CTX_NIA(ctx) + 8); + + /* First get address of tail_call_info */ + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_2), _R1, bpf_jit_stack_tailcallinfo_= offset(ctx))); + /* Writeback updated value to tail_call_info */ + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), 0)); =20 /* prog =3D array->ptrs[index]; */ EMIT(PPC_RAW_MULI(bpf_to_ppc(TMP_REG_1), b2p_index, 8)); --=20 2.48.1 From nobody Sat Feb 7 08:44:12 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 429EF31A54A; Mon, 5 Jan 2026 10:54:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610470; cv=none; b=hTfG3sKqtRNTEx/wyllxJR3Di9x/jwL2Jje955gvEX5oUK3LYz175S0aHMYx07Zd3U8TeGSUsQTxGB2n+Z7YnZSf2AaGaTrY1PD5HXp64Cr/xaO/8UHCDbURwmzM5O4AGnPIfAbkwwAMhxDrnpXOMTt376eXvcYO3yTU5JVfmUA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610470; c=relaxed/simple; bh=MqjdVdkKaUm1LXo8BXJHQMlT6vzqATPPCpqfOOlYdRA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=P+79XEhbnMyNx6NKhgQjKk0N1tBKHJ/Fx++oumeKRe/CQId3FkaHTTlvjkVAqIEGCc2vvJygtwhLVgy/gb0T1Zma/irynk8tJ4Ny+2x44Pkk7FumOtBaOjpP33dVXUQMQwQDLyxNFUKJiy+zvuwL28cOkc3MOQ5mbSG1LNgUKrs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=cHk/za8b; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="cHk/za8b" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 604LMujJ020721; Mon, 5 Jan 2026 10:52:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=oELxnmOGoZA7NMRFk o1gjMFUS0erHG3/oMRsIrRf/Sk=; b=cHk/za8b46wa4PXdDHinlnyYwosmhKYtL a2/4Klje3XhqMV8/XrUV5B8Gb2OEM3oif2KNk76wXdUAJVjJRXs4PAZUF05Xkoq0 ecBpoVdVuQB2lQRb/gWDBa+nky6Aw9dKTL/oz7RnpZokFs9DO0FI0trsm+nji6y3 ed/kvsr6UZiswuNtWZw3hkLWD9FERd6F3H6DU1BzOh89PEAFoc5SUIMKqN15sXFs RN9Saa494Oian0BSayV6MBxioX7NZWFyFGuY+gV9Q3Ibs1beOkZyQc04b6RVZnVY 6LG8vHoal++89f8HtYAA2R5OGukOLxlzQvsKFJvJQ4ClYJVz9OlAw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betu5xs4h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:52:48 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 605AgUGD005433; Mon, 5 Jan 2026 10:52:48 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betu5xs4d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:52:48 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 605AdmCZ014511; Mon, 5 Jan 2026 10:52:46 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bfeemn8x8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:52:46 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 605AqhGY62259544 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Jan 2026 10:52:43 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DE93420040; Mon, 5 Jan 2026 10:52:42 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E788420049; Mon, 5 Jan 2026 10:52:36 +0000 (GMT) Received: from abhi.. (unknown [9.124.213.127]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 5 Jan 2026 10:52:36 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH 2/6] powerpc64/bpf: Tailcall handling with trampolines Date: Mon, 5 Jan 2026 16:22:08 +0530 Message-ID: <20260105105212.136645-3-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260105105212.136645-1-adubey@linux.ibm.com> References: <20260105105212.136645-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: oC44ZUBBUuDlwQj2SFPA2Q6q-5-12fUR X-Authority-Analysis: v=2.4 cv=QbNrf8bv c=1 sm=1 tr=0 ts=695b9800 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=vfcL6nHFD5ed_ebL8FkA:9 X-Proofpoint-ORIG-GUID: DSRSrrc2IQlsxKvIz-MPgwF59IZVlWb2 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTA1MDA5NSBTYWx0ZWRfX4sONKJqhI0Mo BT3bq+2ldmDKk+EoGjCkkFMVs21iP4Qy7kLIgRJA2vA3wDB19yHFyw0z8RKdSwnf7Oj4AimXNy/ Dx3URNsOOc5UL3+zd13lRVXu/M7ZCi/pijj01opjNHa9NPEvWq8zd0+o9EMwp2bcqL5/heEqNTm VJG5hMoNzKwm0YsQNtXScyWaYiirJgT+bgIwULaGrEtmHQr8H7mXMyjgyDKrmHYoTFqpa/FhMPZ xtZNDbQZrwTZ7rra14y5HP6yiKd2bodFbBrre1qVr7k3YseybELxb2OnIR4+L0KHCS4PxDTdtTW D8z8T/n032HimVFfFw/Q8yV0kpoDX7zy3hAT3MeZAw3AOCsPuSysVeg6LftVHgyEFkC5HVdHkXN khbetUDVlMWCYksjeTkuLGIVX7p1cw== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-05_01,2025-12-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 lowpriorityscore=0 phishscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601050095 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The trampoline mechanism sets up its own stack frame and an additional dummy frame. We need to have additional JIT instructions handling tailcall dereferencing in the trampoline's context. We don't add the two stack frames pointed above, rather add space for conventional 'non-volatile register save area' and tail_call_info in trampoline's frame for ppc64. This makes the trampoline's frame consistent with layout of all other frames. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp.c | 48 ++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 069a8822c30d..4aaa0a287a45 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -606,15 +606,42 @@ static int invoke_bpf_mod_ret(u32 *image, u32 *ro_ima= ge, struct codegen_context return 0; } =20 -static void bpf_trampoline_setup_tail_call_cnt(u32 *image, struct codegen_= context *ctx, - int func_frame_offset, int r4_off) +/* + * Refer the label 'Generated stack layout' in this file for actual stack + * layout during trampoline invocation. + * + * Refer __arch_prepare_bpf_trampoline() for stack component details. + * + * The tailcall count/reference is present in caller's stack frame. Its re= quired + * to copy the content of tail_call_info before calling the actual function + * to which the trampoline is attached. + * + */ + +static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen= _context *ctx, + int func_frame_offset, + int bpf_dummy_frame_size, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { /* See bpf_jit_stack_tailcallinfo_offset() */ - int tailcallcnt_offset =3D 7 * 8; + int tailcallinfo_offset =3D BPF_PPC_STACK_SAVE + SZL; + /* + * func_frame_offset =3D + * bpf_dummy_frame_size + trampoline_frame_size + */ + EMIT(PPC_RAW_LD(_R4, _R1, func_frame_offset)); + EMIT(PPC_RAW_LD(_R3, _R4, -tailcallinfo_offset)); + + /* + * Setting the tail_call_info in trampoline's frame + * depending on if previous frame had value or reference. + */ + EMIT(PPC_RAW_CMPLWI(_R3, MAX_TAIL_CALL_CNT)); + PPC_COND_BRANCH(COND_GT, CTX_NIA(ctx) + 8); + EMIT(PPC_RAW_ADDI(_R3, _R4, bpf_jit_stack_tailcallinfo_offset(ctx))); + EMIT(PPC_RAW_STL(_R3, _R1, func_frame_offset + - bpf_dummy_frame_size - tailcallinfo_offset)); =20 - EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); - EMIT(PPC_RAW_STL(_R3, _R1, -tailcallcnt_offset)); } else { /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ EMIT(PPC_RAW_LL(_R4, _R1, r4_off)); @@ -721,6 +748,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im * [ r0 save (32-bit) ] | * dummy frame for unwind [ back chain 1 ] -- * [ padding ] align stack frame + * [ r26..r31 ] nvr save : BPF_PPC_= STACK_SAVE + * [ tail_call_info ] non optional - 64-b= it powerpc * r4_off [ r4 (tailcallcnt) ] optional - 32-bit p= owerpc * alt_lr_off [ real lr (ool stub)] optional - actual lr * [ r26 ] @@ -801,6 +830,12 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tr= amp_image *im, void *rw_im } } =20 + /* Room for 64-bit tail_call_cnt */ + bpf_frame_size +=3D SZL; + + /* Room for nvr save area */ + bpf_frame_size +=3D BPF_PPC_STACK_SAVE; + /* Padding to align stack frame, if any */ bpf_frame_size =3D round_up(bpf_frame_size, SZL * 2); =20 @@ -902,7 +937,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im =20 /* Replicate tail_call_cnt before calling the original BPF prog */ if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) - bpf_trampoline_setup_tail_call_cnt(image, ctx, func_frame_offset, r4_of= f); + bpf_trampoline_setup_tail_call_info(image, ctx, func_frame_offset, + bpf_dummy_frame_size, r4_off); =20 /* Restore args */ bpf_trampoline_restore_args_stack(image, ctx, func_frame_offset, nr_regs= , regs_off); --=20 2.48.1 From nobody Sat Feb 7 08:44:12 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA3123168F2; Mon, 5 Jan 2026 10:53:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610414; cv=none; b=QAxuD3i3OrNEI6jUqiFiFCxh/blGBh7DdS/91smpJAqjHYFbEOY3PEmLGT9LPU487cI46+Y8KNF3EGuG50/kn5IkkcNADvL7AZUfwRykNDMewko1hzkmK5297hLsHJF4tNWQmvUO85UCgDeiuFR6HSRKP8hH4AbyCCM51NE98ow= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610414; c=relaxed/simple; bh=70ZsFIqBdRKJuYFjeHJbJKo32FL+Ckf1UULhaWKLoTg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NKTQTK3yETRUmuLDudZNsHyneXO5L0uGiQ7YGyKdkJpdNHC9KEwlxJDRNacwWPu4wh3ukMbkQvsW2H+ukuiLYIJT4+i3YUXamizmvxUom2VUR0MVgYAVgKi5JVDPuKwDAsM0g+zbkBewlET8JjFX3DvdyksRDtM4nZZNQd7lQpk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=J1YxI8Va; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="J1YxI8Va" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 604NPsu2016119; Mon, 5 Jan 2026 10:52:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=A7Gf/k /vc3DEKJhPZHuVcE8mdIGmWo71nev2g6/ii8s=; b=J1YxI8Va7uF9MtXQ5+Vtof nofKO70/Sv6omNq0oJOBa6pndzqgnspR3g7lsmvgX08Og8yljce6ppzK0rvvJ2WZ oSwLNpTXUc9vyPwphbg3rxIJ0SULP92G1fmXPNRYMPSxxHYaPTvjl1qWGvwQKpre 4tzdZXzNSfXgdB202KuhlYWrlR6dUBWrOufM7hF4kha8gecFaT07HY/GOfTJwHrZ JVJlNvV6+9Ue09KOnBR/cLqX0AtB33Hh5C/o8QNhguKevvx2wC5TXYJZkzEUMcDQ 2X+UyDOLFdyu3UtMKfHC6u+HjAown5qBIvXi91FLnkMR35wEV6KZcv8GI8nyey1Q == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4berhjxep1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:52:56 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 605AqtaZ004600; Mon, 5 Jan 2026 10:52:55 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4berhjxeny-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:52:55 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 605ABkfV023511; Mon, 5 Jan 2026 10:52:54 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bg3rm1vw8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:52:54 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 605Aqo4W13369606 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Jan 2026 10:52:50 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 88D712004B; Mon, 5 Jan 2026 10:52:50 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A8E1920040; Mon, 5 Jan 2026 10:52:44 +0000 (GMT) Received: from abhi.. (unknown [9.124.213.127]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 5 Jan 2026 10:52:44 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH 3/6] powerpc/bpf: use BPF_PPC_STACK_SAVE to spill trampoline NVRs Date: Mon, 5 Jan 2026 16:22:09 +0530 Message-ID: <20260105105212.136645-4-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260105105212.136645-1-adubey@linux.ibm.com> References: <20260105105212.136645-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=P4s3RyAu c=1 sm=1 tr=0 ts=695b9808 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=IkcTkHD0fZMA:10 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=M2TuCUmZdyp0mqlJdecA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: FjQ5vILT0664H1imdvC6YBdywT-tZgbg X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTA1MDA5NSBTYWx0ZWRfX2+9gxk17nLan oYCnzLdctkB7oSyuWYeIQEUVa3R8JvZrX3SKabKYJRBl0dquzbrTW6Umo2Jz8SUcGb8dOL/2WAR ASM4MbwoY06hFByM29cUL9DYMXbIIWLC69c+WzDXOpzVDU8GQ8nbj9uDmfoy1OzQLq1pUeZnQUx QwR+BvktbskDzmUur9jl9VnAKTUfnJYG7dOkv6F4mt1LPFBb++pa6lRKeVBc3dTiXrcumKbSAQP 5bY/Lr4hdzf4Dv/S8GBzqfzW9TL95AyXdPtaYC+aQAb/98guXnxl0X1B9ZPkiaR/UdxZ5PELs3v v21UBIAsBvhTetWakg6hI3VfbKTCzw2Gxbh7LcqDYfHZyjZtwFvPksKJajho9LEla+9knYM2cIT OnV/e3rOTQ1kSAuYsGZXbu45a4IU9RFeHFrOKRRqm7QWyYRPJI/2LUWaSo41wNLBd83CBPbOyYN gtqNm+JfXxfRp+JwHLQ== X-Proofpoint-GUID: YiDoRGGScWVrF20A9dn9vuEzpMfhiYIt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-05_01,2025-12-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 bulkscore=0 priorityscore=1501 clxscore=1015 suspectscore=0 phishscore=0 adultscore=0 spamscore=0 impostorscore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601050095 From: Abhishek Dubey In the previous patch, we introduced BPF_PPC_STACK_SAVE into the trampoline frame to make its layout consistent with a conventional stack frame. The trampoline JIT currently uses NVRs R25 and R26 and allocates dedicated stack slots to save them. This dedicated space can be eliminated by reusing the BPF_PPC_STACK_SAVE area instead, reducing overall stack footprint. The BPF_PPC_STACK_SAVE area corresponds to the register range R26=E2=80=93R31. By remapping registers in the trampoline JIT code (R25 =E2=86=92 R26 and R26 =E2=86=92 R27), we can spill these registers into the existing save area and utilize the stack more efficiently. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp.c | 27 +++++++++++---------------- 1 file changed, 11 insertions(+), 16 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 4aaa0a287a45..b09d294084d4 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -512,7 +512,7 @@ static int invoke_bpf_prog(u32 *image, u32 *ro_image, s= truct codegen_context *ct =20 /* __bpf_prog_enter(p, &bpf_tramp_run_ctx) */ PPC_LI_ADDR(_R3, p); - EMIT(PPC_RAW_MR(_R25, _R3)); + EMIT(PPC_RAW_MR(_R26, _R3)); EMIT(PPC_RAW_ADDI(_R4, _R1, run_ctx_off)); ret =3D bpf_jit_emit_func_call_rel(image, ro_image, ctx, (unsigned long)bpf_trampoline_enter(p)); @@ -520,7 +520,7 @@ static int invoke_bpf_prog(u32 *image, u32 *ro_image, s= truct codegen_context *ct return ret; =20 /* Remember prog start time returned by __bpf_prog_enter */ - EMIT(PPC_RAW_MR(_R26, _R3)); + EMIT(PPC_RAW_MR(_R27, _R3)); =20 /* * if (__bpf_prog_enter(p) =3D=3D 0) @@ -543,7 +543,7 @@ static int invoke_bpf_prog(u32 *image, u32 *ro_image, s= truct codegen_context *ct image[ctx->idx] =3D ppc_inst_val(branch_insn); ctx->idx++; } else { - EMIT(PPC_RAW_LL(_R12, _R25, offsetof(struct bpf_prog, bpf_func))); + EMIT(PPC_RAW_LL(_R12, _R26, offsetof(struct bpf_prog, bpf_func))); EMIT(PPC_RAW_MTCTR(_R12)); EMIT(PPC_RAW_BCTRL()); } @@ -560,8 +560,8 @@ static int invoke_bpf_prog(u32 *image, u32 *ro_image, s= truct codegen_context *ct } =20 /* __bpf_prog_exit(p, start_time, &bpf_tramp_run_ctx) */ - EMIT(PPC_RAW_MR(_R3, _R25)); - EMIT(PPC_RAW_MR(_R4, _R26)); + EMIT(PPC_RAW_MR(_R3, _R26)); + EMIT(PPC_RAW_MR(_R4, _R27)); EMIT(PPC_RAW_ADDI(_R5, _R1, run_ctx_off)); ret =3D bpf_jit_emit_func_call_rel(image, ro_image, ctx, (unsigned long)bpf_trampoline_exit(p)); @@ -748,12 +748,10 @@ static int __arch_prepare_bpf_trampoline(struct bpf_t= ramp_image *im, void *rw_im * [ r0 save (32-bit) ] | * dummy frame for unwind [ back chain 1 ] -- * [ padding ] align stack frame - * [ r26..r31 ] nvr save : BPF_PPC_= STACK_SAVE + * nvr_off [ r26..r31 ] nvr save : BPF_PPC_= STACK_SAVE * [ tail_call_info ] non optional - 64-b= it powerpc * r4_off [ r4 (tailcallcnt) ] optional - 32-bit p= owerpc * alt_lr_off [ real lr (ool stub)] optional - actual lr - * [ r26 ] - * nvr_off [ r25 ] nvr save area * retval_off [ return value ] * [ reg argN ] * [ ... ] @@ -811,10 +809,6 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tr= amp_image *im, void *rw_im if (save_ret) bpf_frame_size +=3D SZL; =20 - /* Room for nvr save area */ - nvr_off =3D bpf_frame_size; - bpf_frame_size +=3D 2 * SZL; - /* Optional save area for actual LR in case of ool ftrace */ if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { alt_lr_off =3D bpf_frame_size; @@ -834,6 +828,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im bpf_frame_size +=3D SZL; =20 /* Room for nvr save area */ + nvr_off =3D bpf_frame_size; bpf_frame_size +=3D BPF_PPC_STACK_SAVE; =20 /* Padding to align stack frame, if any */ @@ -897,8 +892,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im EMIT(PPC_RAW_STL(_R3, _R1, nregs_off)); =20 /* Save nv regs */ - EMIT(PPC_RAW_STL(_R25, _R1, nvr_off)); - EMIT(PPC_RAW_STL(_R26, _R1, nvr_off + SZL)); + EMIT(PPC_RAW_STL(_R26, _R1, nvr_off)); + EMIT(PPC_RAW_STL(_R27, _R1, nvr_off + SZL)); =20 if (flags & BPF_TRAMP_F_CALL_ORIG) { PPC_LI_ADDR(_R3, (unsigned long)im); @@ -999,8 +994,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im EMIT(PPC_RAW_LL(_R3, _R1, retval_off)); =20 /* Restore nv regs */ - EMIT(PPC_RAW_LL(_R26, _R1, nvr_off + SZL)); - EMIT(PPC_RAW_LL(_R25, _R1, nvr_off)); + EMIT(PPC_RAW_LL(_R27, _R1, nvr_off + SZL)); + EMIT(PPC_RAW_LL(_R26, _R1, nvr_off)); =20 /* Epilogue */ if (IS_ENABLED(CONFIG_PPC64_ELF_ABI_V2) && !IS_ENABLED(CONFIG_PPC_KERNEL_= PCREL)) --=20 2.48.1 From nobody Sat Feb 7 08:44:12 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA2533164B8; Mon, 5 Jan 2026 10:53:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610414; cv=none; b=sPqpzRKXJgCMP9PuhGa+TBUIWMu4MYwD4riU3RtAa7+xuLu2B5rgo8I1lff1RUjObPNTqpW4rvO0MO1MQHLmpkx5VAz25z88cssRHg1zUOREy+mIswbtXLQr6cvqtv5QR4dSUyHJpFygLa7Oub7GUd4ivEGgM/GNy4swmr2pPB8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610414; c=relaxed/simple; bh=lRtu9kw1ZhjXWN+KoM32YsPv1tHn0e8qSptl/UDCCP4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kOmW/JRpBsTR7SBmaRDSWBXem82rLxj5Vd2hHl+tMGOR1/r/6iBF0toRlScZh3FDYu11TXfBHQsFUQWWF0iWWGZIRCXu4MuOZtkvcDcPTiQGjDKN0JYCsxb+Q1/Mm4YSbdNznCBaqInr+T+GrBhY4rwR/roNOOY7WjufMCgZftw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=XWSIeIo0; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="XWSIeIo0" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 604GPB0w026573; Mon, 5 Jan 2026 10:53:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=wzBaK8qWCdkqUY3g3 Sj7Uoi09hD+8UHUGtCXe7xrww4=; b=XWSIeIo0bW23L4gDlGvrs5W+b77vSVXhJ K0m+iqZK9PbxvJOhDyt9bpOEQl2yXRr800GY93lyP/SkAe+/MtCyeP4Av9ulC34+ 8HKeselPnsN56tiNTYAyCgROnHmSOa7Q1rJXgCzD/6Atpq0PToasqvqqndEXQijK 6VBR01d/LUx5/tdFSGLTcqCq5QoX+VMePHq++YnCWtc4lujmoR1fMh4qXCBNTMM7 9/0N1wwWgKShkeRc9hNm1wx2mb4IPR3mW8eLnBr6KLGQG4OoGTlSNwzXHfwXp3Ek 3upNx9Eno3yZasVsS0zRwyN8MZQemtIpb5m+VLfvGWVTIcqhgr2Fg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betrte68c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:53:04 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 605Aflws021848; Mon, 5 Jan 2026 10:53:03 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betrte68a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:53:03 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 6058CIJB019161; Mon, 5 Jan 2026 10:53:02 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4bfg50w08b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:53:02 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 605AqwGm35717390 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Jan 2026 10:52:58 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B296E20049; Mon, 5 Jan 2026 10:52:58 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D39E520040; Mon, 5 Jan 2026 10:52:52 +0000 (GMT) Received: from abhi.. (unknown [9.124.213.127]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 5 Jan 2026 10:52:52 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH 4/6] powerpc64/bpf: Add arch_bpf_stack_walk() for BPF JIT Date: Mon, 5 Jan 2026 16:22:10 +0530 Message-ID: <20260105105212.136645-5-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260105105212.136645-1-adubey@linux.ibm.com> References: <20260105105212.136645-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=aaJsXBot c=1 sm=1 tr=0 ts=695b9810 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=ovcAVf-9gG4EF5hqTukA:9 X-Proofpoint-GUID: 1uSL2weSR1s7cYHIVMHughtrBm0l5nTR X-Proofpoint-ORIG-GUID: tpHHNSAoaRePyi43WYGB6w5odCvUn63K X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTA1MDA5NSBTYWx0ZWRfX24Ob40dEZRU3 4OeIU1sWWHJj6V9Ero72BF35800d8+4vHVBb5/GIWkh6/A8enM+2p3FpVBtboHL7CXeLyHHfjRa e+vsHPd4noSEadDT4UhV0ZdKRJcnhgLDE0wM9RO0dtrt0cRAAkUj8xOf48bd9nykQcr/aUaJSbM +ATcObndzJHpS18pdsTW9Js2Dm2BYaDmMszLAjm0Xna0fb+3PpuNLfTdyILL0tHFphqMbL+yZq8 jtdBTVE/OvLB19fnSklIaDf5LduA/O6Z8EQAc9mNgRkjr4/h19ws/d/UnVzU2fPfT/vC9l/qNaV OpXlxhCPA6yKWI2tMAMznNBhj3cr/S6XemSthsGMcJl2avlrK4X90bVnoCC+VTOAcVjUP1OdJlf Fz9weRbgutZEG+S508m7SEsKchu4VSqtbDBXrJw6txutUxu4UVYEwKIPAZcPYev8Lgou8gwcjEX OEWHq7uYip4Xt6eUMqg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-05_01,2025-12-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 lowpriorityscore=0 bulkscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 phishscore=0 spamscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601050095 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey This function is used by bpf_throw() to unwind the stack until frame of exception-boundary during BPF exception handling. This function is necessary to support BPF exceptions on PowerPC. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp64.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 37c547b49da8..0f3af67914d6 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -237,6 +237,34 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen= _context *ctx) bpf_jit_build_fentry_stubs(image, ctx); } =20 +void arch_bpf_stack_walk(bool (*consume_fn)(void *, u64, u64, u64), void *= cookie) +{ + // callback processing always in current context + unsigned long fp =3D current_stack_frame(); + + for (;;) { + unsigned long *frame =3D (unsigned long *) fp; + unsigned long ip; + + if(!validate_sp(fp, current)) + return; + + ip =3D frame[STACK_FRAME_LR_SAVE]; + if (!ip) + break; + + /* + * consume_fn common code expects stack pointer(sp) in third + * argument. There is no sp in ppc64, rather pass frame + * pointer. + */ + if (ip && !consume_fn(cookie, ip, fp, fp)) + break; + + fp =3D frame[0]; + } +} + int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_con= text *ctx, u64 func) { unsigned long func_addr =3D func ? ppc_function_entry((void *)func) : 0; --=20 2.48.1 From nobody Sat Feb 7 08:44:12 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B70FD2DCF71; Mon, 5 Jan 2026 10:53:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610400; cv=none; b=HZ00l9F+6TJsCRI0VN6wB8av3Afi3+vtDxBdVg3ukfd4WF24qqZNX8yAUNy2V452byaNXezjpJW6ZYpupHTPnYS8hNCwSYH4tfqOrUSG6K4ydEt0FdKV0iZB8joR7EQAauxGhr4wIcUGI2ZVGtAVMZ8butAQdW7ltc/CS5WD53A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610400; c=relaxed/simple; bh=g/rvjxr97oeBAAZ4NA8UMlem82QIm8p2umtrhrFDOWE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bQgoBhHIGuwZyMCktoZ9ueldW/EWyNyW7glw37FVPD3YJWjxlu1wTL2tL67j5hfVx/GDHrMY6SQe8XNilbkPFfUxR5Ml02IRUSESeVTyMF/dzuI6iT4EObEtG1ZCMetu2k67ENrjvl8BkM4oD5+6ZWkYGfh/L0yWfRkl7HK0ruA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=eXk1t0wH; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="eXk1t0wH" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 605AKF0n028393; Mon, 5 Jan 2026 10:53:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=1Ch459TmqXBIPON/F rdLOd7rTgxg4aff4Ts6ksixOXE=; b=eXk1t0wH+Or63fDYasmaV9vcP4d9zX+SX pA6HXoTSORZeCrDThc8DBtVPLj8ODlRzytSmjd7MvJM8b3H3cADH8AB8MzkvYrsk 8Tvq84H95Qgwo0gpi80xi4XlC+dVEXjtVoz+qszIYPQmAQcbl3HARQnlYu1e+7T1 oM6WwLZrLRvtLk3qLWfhHtMnLSyP8JWUIls2CSpbFLImKBDyN67Y5fLheb8vKhZW FC0clK/Dizexk0E/zeoYw02omaNAMpGE/Pk4fvSXqRcpK4MjF8JvBeGxRNNOUts8 PLIExMmpTKoyglTcHD/9BgXPSdjH7cguopKmnlZwc7WmJBr+9eGgw== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betrte68t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:53:10 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 605ACNBw023487; Mon, 5 Jan 2026 10:53:10 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bg3rm1vxq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:53:09 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 605Ar8dI50397668 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Jan 2026 10:53:08 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DDCC82004B; Mon, 5 Jan 2026 10:53:07 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F16AC20040; Mon, 5 Jan 2026 10:53:01 +0000 (GMT) Received: from abhi.. (unknown [9.124.213.127]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 5 Jan 2026 10:53:01 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH 5/6] powerpc64/bpf: Support exceptions Date: Mon, 5 Jan 2026 16:22:11 +0530 Message-ID: <20260105105212.136645-6-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260105105212.136645-1-adubey@linux.ibm.com> References: <20260105105212.136645-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=aaJsXBot c=1 sm=1 tr=0 ts=695b9816 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=DU0EyFXtAAAA:8 a=VnNF1IyMAAAA:8 a=6hNtxff-4WC3rVPnFXsA:9 a=UCR5be5CC-YrbG9FbbB0:22 X-Proofpoint-GUID: 2tmgi0fkjpV-W-oQLTgztyuvNsfasF8d X-Proofpoint-ORIG-GUID: 2tmgi0fkjpV-W-oQLTgztyuvNsfasF8d X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTA1MDA5NSBTYWx0ZWRfX1dM2ENzClrAH HERnCj62mrLk7oS+7Sbx0MIbm/Ql0UpSdX/1dB6055lXdHA2bkCb+/fJWboP0W4FBN6xxk27lDo c7cPMDHN5vN8a/wTtcISqBs5RPopGGFP1DEviJlkTMYIWgxtT0EOgf/ooNir7KLRlx4IZkZbRFC 5wCjzayns0KxUTGYZuLJZswhiBRjVTWfoBx/AMaRlf+PUaEyMG30kScdkz9bqPa9modIbVrvHYQ l1GyPfMafGyH4/GL4gH0lQsRPZVWo1LhBVDkHzXic7lEq6pDG9+W2/qrGUaF7k+Bkgy7PTZGsaR xcdR7kjzBaRkdlnOkQQh+Bv9l2Ni2pnVCtDvVKPhn634gFM/LYp1NEex75NZmKYdSmAdW5GV1dB cEQxwuVUsEZx0NN8TrVZWa32/e/cabFUpjSJUX4zLoJijbtIsd3p4QYJJ8DltR9tGPL2p23MWLR Yme2OdpVnf8y+ASjB8w== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-05_01,2025-12-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 lowpriorityscore=0 bulkscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 phishscore=0 spamscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601050095 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The modified prologue/epilogue generation code now enables exception-callback to use the stack frame of the program marked as exception boundary, where callee saved registers are stored. As per ppc64 ABIv2 documentation[1], r14-r31 are callee saved registers. BPF programs on ppc64 already saves r26-r31 registers. Saving the remaining set of callee saved registers(r14-r25) is handled in the next patch. [1] https://ftp.rtems.org/pub/rtems/people/sebh/ABI64BitOpenPOWERv1.1_16Jul= y2015_pub.pdf Following is exceptions selftest result on ppc64le: # ./test_progs -t exceptions #100/1 exceptions/exception_throw_always_1:OK #100/2 exceptions/exception_throw_always_2:OK #100/3 exceptions/exception_throw_unwind_1:OK #100/4 exceptions/exception_throw_unwind_2:OK #100/5 exceptions/exception_throw_default:OK #100/6 exceptions/exception_throw_default_value:OK #100/7 exceptions/exception_tail_call:OK #100/8 exceptions/exception_ext:OK #100/9 exceptions/exception_ext_mod_cb_runtime:OK #100/10 exceptions/exception_throw_subprog:OK #100/11 exceptions/exception_assert_nz_gfunc:OK #100/12 exceptions/exception_assert_zero_gfunc:OK #100/13 exceptions/exception_assert_neg_gfunc:OK #100/14 exceptions/exception_assert_pos_gfunc:OK #100/15 exceptions/exception_assert_negeq_gfunc:OK #100/16 exceptions/exception_assert_poseq_gfunc:OK #100/17 exceptions/exception_assert_nz_gfunc_with:OK #100/18 exceptions/exception_assert_zero_gfunc_with:OK #100/19 exceptions/exception_assert_neg_gfunc_with:OK #100/20 exceptions/exception_assert_pos_gfunc_with:OK #100/21 exceptions/exception_assert_negeq_gfunc_with:OK #100/22 exceptions/exception_assert_poseq_gfunc_with:OK #100/23 exceptions/exception_bad_assert_nz_gfunc:OK #100/24 exceptions/exception_bad_assert_zero_gfunc:OK #100/25 exceptions/exception_bad_assert_neg_gfunc:OK #100/26 exceptions/exception_bad_assert_pos_gfunc:OK #100/27 exceptions/exception_bad_assert_negeq_gfunc:OK #100/28 exceptions/exception_bad_assert_poseq_gfunc:OK #100/29 exceptions/exception_bad_assert_nz_gfunc_with:OK #100/30 exceptions/exception_bad_assert_zero_gfunc_with:OK #100/31 exceptions/exception_bad_assert_neg_gfunc_with:OK #100/32 exceptions/exception_bad_assert_pos_gfunc_with:OK #100/33 exceptions/exception_bad_assert_negeq_gfunc_with:OK #100/34 exceptions/exception_bad_assert_poseq_gfunc_with:OK #100/35 exceptions/exception_assert_range:OK #100/36 exceptions/exception_assert_range_with:OK #100/37 exceptions/exception_bad_assert_range:OK #100/38 exceptions/exception_bad_assert_range_with:OK #100/39 exceptions/non-throwing fentry -> exception_cb:OK #100/40 exceptions/throwing fentry -> exception_cb:OK #100/41 exceptions/non-throwing fexit -> exception_cb:OK #100/42 exceptions/throwing fexit -> exception_cb:OK #100/43 exceptions/throwing extension (with custom cb) -> exception_cb:OK #100/44 exceptions/throwing extension -> global func in exception_cb:OK #100/45 exceptions/exception_ext_mod_cb_runtime:OK #100/46 exceptions/throwing extension (with custom cb) -> global func in e= xception_cb:OK #100/47 exceptions/exception_ext:OK #100/48 exceptions/non-throwing fentry -> non-throwing subprog:OK #100/49 exceptions/throwing fentry -> non-throwing subprog:OK #100/50 exceptions/non-throwing fentry -> throwing subprog:OK #100/51 exceptions/throwing fentry -> throwing subprog:OK #100/52 exceptions/non-throwing fexit -> non-throwing subprog:OK #100/53 exceptions/throwing fexit -> non-throwing subprog:OK #100/54 exceptions/non-throwing fexit -> throwing subprog:OK #100/55 exceptions/throwing fexit -> throwing subprog:OK #100/56 exceptions/non-throwing fmod_ret -> non-throwing subprog:OK #100/57 exceptions/non-throwing fmod_ret -> non-throwing global subprog:OK #100/58 exceptions/non-throwing extension -> non-throwing subprog:OK #100/59 exceptions/non-throwing extension -> throwing subprog:OK #100/60 exceptions/non-throwing extension -> non-throwing subprog:OK #100/61 exceptions/non-throwing extension -> throwing global subprog:OK #100/62 exceptions/throwing extension -> throwing global subprog:OK #100/63 exceptions/throwing extension -> non-throwing global subprog:OK #100/64 exceptions/non-throwing extension -> main subprog:OK #100/65 exceptions/throwing extension -> main subprog:OK #100/66 exceptions/reject_exception_cb_type_1:OK #100/67 exceptions/reject_exception_cb_type_2:OK #100/68 exceptions/reject_exception_cb_type_3:OK #100/69 exceptions/reject_exception_cb_type_4:OK #100/70 exceptions/reject_async_callback_throw:OK #100/71 exceptions/reject_with_lock:OK #100/72 exceptions/reject_subprog_with_lock:OK #100/73 exceptions/reject_with_rcu_read_lock:OK #100/74 exceptions/reject_subprog_with_rcu_read_lock:OK #100/75 exceptions/reject_with_rbtree_add_throw:OK #100/76 exceptions/reject_with_reference:OK #100/77 exceptions/reject_with_cb_reference:OK #100/78 exceptions/reject_with_cb:OK #100/79 exceptions/reject_with_subprog_reference:OK #100/80 exceptions/reject_throwing_exception_cb:OK #100/81 exceptions/reject_exception_cb_call_global_func:OK #100/82 exceptions/reject_exception_cb_call_static_func:OK #100/83 exceptions/reject_multiple_exception_cb:OK #100/84 exceptions/reject_exception_throw_cb:OK #100/85 exceptions/reject_exception_throw_cb_diff:OK #100/86 exceptions/reject_set_exception_cb_bad_ret1:OK #100/87 exceptions/reject_set_exception_cb_bad_ret2:OK #100/88 exceptions/check_assert_eq_int_min:OK #100/89 exceptions/check_assert_eq_int_max:OK #100/90 exceptions/check_assert_eq_zero:OK #100/91 exceptions/check_assert_eq_llong_min:OK #100/92 exceptions/check_assert_eq_llong_max:OK #100/93 exceptions/check_assert_lt_pos:OK #100/94 exceptions/check_assert_lt_zero:OK #100/95 exceptions/check_assert_lt_neg:OK #100/96 exceptions/check_assert_le_pos:OK #100/97 exceptions/check_assert_le_zero:OK #100/98 exceptions/check_assert_le_neg:OK #100/99 exceptions/check_assert_gt_pos:OK #100/100 exceptions/check_assert_gt_zero:OK #100/101 exceptions/check_assert_gt_neg:OK #100/102 exceptions/check_assert_ge_pos:OK #100/103 exceptions/check_assert_ge_zero:OK #100/104 exceptions/check_assert_ge_neg:OK #100/105 exceptions/check_assert_range_s64:OK #100/106 exceptions/check_assert_range_u64:OK #100/107 exceptions/check_assert_single_range_s64:OK #100/108 exceptions/check_assert_single_range_u64:OK #100/109 exceptions/check_assert_generic:OK #100/110 exceptions/check_assert_with_return:OK #100 exceptions:OK Summary: 1/110 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit.h | 2 ++ arch/powerpc/net/bpf_jit_comp.c | 7 ++++ arch/powerpc/net/bpf_jit_comp64.c | 53 +++++++++++++++++++++---------- 3 files changed, 45 insertions(+), 17 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 98e8b1f9c2f9..b9316780a501 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -177,6 +177,8 @@ struct codegen_context { u64 arena_vm_start; u64 user_vm_start; bool is_subprog; + bool exception_boundary; + bool exception_cb; }; =20 #define bpf_to_ppc(r) (ctx->b2p[r]) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index b09d294084d4..3c030a7d8e73 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -207,6 +207,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *f= p) cgctx.arena_vm_start =3D bpf_arena_get_kern_vm_start(fp->aux->arena); cgctx.user_vm_start =3D bpf_arena_get_user_vm_start(fp->aux->arena); cgctx.is_subprog =3D bpf_is_subprog(fp); + cgctx.exception_boundary =3D fp->aux->exception_boundary; + cgctx.exception_cb =3D fp->aux->exception_cb; =20 /* Scouting faux-generate pass 0 */ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) { @@ -436,6 +438,11 @@ void bpf_jit_free(struct bpf_prog *fp) bpf_prog_unlock_free(fp); } =20 +bool bpf_jit_supports_exceptions(void) +{ + return IS_ENABLED(CONFIG_PPC64); +} + bool bpf_jit_supports_subprog_tailcalls(void) { return IS_ENABLED(CONFIG_PPC64); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 0f3af67914d6..5ec8e3654098 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -85,7 +85,9 @@ static inline bool bpf_has_stack_frame(struct codegen_con= text *ctx) * - the bpf program uses its stack area * The latter condition is deduced from the usage of BPF_REG_FP */ - return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, bpf_to_ppc(BPF_= REG_FP)); + return ctx->seen & SEEN_FUNC || + bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)) || + ctx->exception_cb; } =20 /* @@ -180,23 +182,32 @@ void bpf_jit_build_prologue(u32 *image, struct codege= n_context *ctx) EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size))); } =20 - /* - * Back up non-volatile regs -- BPF registers 6-10 - * If we haven't created our own stack frame, we save these - * in the protected zone below the previous stack frame - */ - for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) - if (bpf_is_seen_register(ctx, bpf_to_ppc(i))) - EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to= _ppc(i)))); + if (!ctx->exception_cb) { + /* + * Back up non-volatile regs -- BPF registers 6-10 + * If we haven't created our own stack frame, we save these + * in the protected zone below the previous stack frame + */ + for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) + if (ctx->exception_boundary || bpf_is_seen_register(ctx, bpf_to_ppc(i))) + EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, + bpf_jit_stack_offsetof(ctx, bpf_to_ppc(i)))); =20 - if (ctx->arena_vm_start) - EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1, + if (ctx->exception_boundary || ctx->arena_vm_start) + EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 - /* Setup frame pointer to point to the bpf stack area */ - if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP))) - EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1, + /* Setup frame pointer to point to the bpf stack area */ + if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP))) + EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1, STACK_FRAME_MIN_SIZE + ctx->stack_size)); + } else { + /* + * Exception callback receives Frame Pointer of main + * program as third arg + */ + EMIT(PPC_RAW_MR(_R1, _R5)); + } =20 if (ctx->arena_vm_start) PPC_LI64(bpf_to_ppc(ARENA_VM_START), ctx->arena_vm_start); @@ -208,17 +219,25 @@ static void bpf_jit_emit_common_epilogue(u32 *image, = struct codegen_context *ctx =20 /* Restore NVRs */ for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) - if (bpf_is_seen_register(ctx, bpf_to_ppc(i))) + if (ctx->exception_cb || bpf_is_seen_register(ctx, bpf_to_ppc(i))) EMIT(PPC_RAW_LD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_= ppc(i)))); =20 - if (ctx->arena_vm_start) + if (ctx->exception_cb || ctx->arena_vm_start) EMIT(PPC_RAW_LD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 + if (ctx->exception_cb) { + /* + * LR value from boundary-frame is received as second parameter + * in exception callback. + */ + EMIT(PPC_RAW_MTLR(_R4)); + } + /* Tear down our stack frame */ if (bpf_has_stack_frame(ctx)) { EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size)); - if (ctx->seen & SEEN_FUNC) { + if (ctx->seen & SEEN_FUNC || ctx->exception_cb) { EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF)); EMIT(PPC_RAW_MTLR(_R0)); } --=20 2.48.1 From nobody Sat Feb 7 08:44:12 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F3FA319606; Mon, 5 Jan 2026 10:54:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610445; cv=none; b=m9YQB7wWVXd3rew2Vl02Qah3zsn1hlA4MxYAUypzbDwYxr6W6kZvEgY/VWSb6pIsZHrwtYlRkJFGbKwDFMefOFPFsYCjw8rONISQej4AIQN1bkSe4sWzMi6u0fVZy4dT6lnFnrr+wVkbSny7VqsbgfFH84HwzofNM6x01uBvnhA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767610445; c=relaxed/simple; bh=Kce6HvTDmAbb5YmvD1yfmA+DaTSrS2gscSGSNwUAXVE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nOOUyrv0+UXkyMfUP4ztogJQcEWF4AUOwCvFjP120SfPxE+oiHvwoEYGFId35i/vxO9W5rmoIiYYM0jtZYvaYrzD5Efzbfu1FJnleKtt9c7G2A8Nh6L+AiW72/T+72pUxOxjnUleatEhRrf3v2CpLBw1qog36s1VxzXWojsb0xA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=q2BP2/si; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="q2BP2/si" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 604IkMXQ017255; Mon, 5 Jan 2026 10:53:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=4a8GXlB0hIJi+ZqSu KSoS6yEsvz6r4jZx+42LoHxxEY=; b=q2BP2/si+bRzOzXtLtjQGBs50P2Lvx778 nNq23tmodvoKnJZvTt7TEqj/WHlRbK9dBJ4mTgACLJMyq5N30sSDikRsciYl7ZJo 46TkWNdL1+0yXkWBOB3gK+i8uMvp6BA4Tjsf6o5IfpdjWM/iM4HEb9o4Ox40Ybtj PkdyauTCOhUuHng50obXWuVuNKFm11SOiJuewhREeJTH9vzTGcI2QdHox74EAuDU PE2y7kE32lHsfb5YQiOgSu28mzKCyZ5jIhlClT52OkGoyn7Gi+it83/zsLO9NyiH Sqsd0fskBT5bkhgNHTijO/T7zechlCMiFHJv844QCyCNUEdY7Y19A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betu5xs72-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:53:22 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 605Aotts022664; Mon, 5 Jan 2026 10:53:21 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betu5xs70-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:53:21 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 605AdmCl014511; Mon, 5 Jan 2026 10:53:20 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bfeemn90k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:53:20 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 605ArGud56230150 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Jan 2026 10:53:16 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9B1F420049; Mon, 5 Jan 2026 10:53:16 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7F2B020040; Mon, 5 Jan 2026 10:53:10 +0000 (GMT) Received: from abhi.. (unknown [9.124.213.127]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 5 Jan 2026 10:53:10 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH 6/6] powerpc64/bpf: Additional NVR handling for bpf_throw Date: Mon, 5 Jan 2026 16:22:12 +0530 Message-ID: <20260105105212.136645-7-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260105105212.136645-1-adubey@linux.ibm.com> References: <20260105105212.136645-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=QbNrf8bv c=1 sm=1 tr=0 ts=695b9822 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=49GENK-6fEsES3x69YYA:9 X-Proofpoint-ORIG-GUID: 4_UlJfNwuW3c3BE7I9RGgLB1eAGa3mjR X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTA1MDA5NSBTYWx0ZWRfX2N7RUlbM3ADW IImIXUOI57HUj3Nf54mlmDiH9MdzPuudWy9elfn7kaC21SbwqJbmflSevdNob2QVyBDdR2hBY6M I/uzCaSiJVu8ztjSOkHFNSBJ0ifBW5tZWc72WbcauZuW0LOHWLNJV0LOc5LEqg/Ki5jQ8jVNvWQ JSY/gcg1z4P9zOXu6CnEIlGzGF/qFgwhWwQUf80m50sPeCqiSl3R3yMWg89MvqCZHQuPJoDZcO8 2bzfj+iNpYfs6FcylYohE0hmIu/4gk0XyyGnmhq05vbwkUmFr80TCKKsPFQO8xYpCJ0s6AZaQOu 8yMWor4B5Kcyd+hUAS3EmzS8MKuaf6RHIbWjEUYdRHxEnTd9rPf1+ikVjup/ck/wZCbUB89fYzc n8M/ofaw/br0lgre2fzaQ/+44jdpQTeo+uHrlPbuZOuQtCDLzPr/uHbYOfabYE9cRTSmjYF+1hE /mdl3hlMzlq/0hfQKhw== X-Proofpoint-GUID: BZd-Rfx6MedGH2tL0nj2PfN_tUuzlzgE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-05_01,2025-12-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015 bulkscore=0 suspectscore=0 priorityscore=1501 adultscore=0 lowpriorityscore=0 impostorscore=0 phishscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601050095 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The bpf_throw() function never returns, if it has clobbered any callee-saved register, those will remain clobbered. The prologue must take care of saving all callee-saved registers in the frame of exception boundary program. Later these additional non volatile registers R14-R25 along with other NVRs are restored back in the epilogue of exception callback. To achieve above objective the frame size is determined dynamically to accommodate additional non volatile registers in exception boundary's frame. For non-exception boundary program, the frame size remains optimal. The additional instructions to save & restore r14-r25 registers are emitted only during exception boundary and exception callback respectively. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp64.c | 70 +++++++++++++++++++++++++++---- 1 file changed, 62 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 5ec8e3654098..21f04eb57605 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -29,19 +29,33 @@ * main's frame in sub-prog's frame. * * [ prev sp ] <------------- - * [ nv gpr save area ] 6*8 | + * [ nv gpr save area ] 6*8 + (12*8) | * [ tail_call_info ] 8 | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | * [ frame header ] 32/112 | * sp (r1) ---> [ stack pointer ] -------------- + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ =20 /* for bpf JIT code internal usage */ #define BPF_PPC_STACK_LOCALS 32 +/* + * for additional non volatile registers(r14-r25) to be saved + * at exception boundary + */ +#define BPF_PPC_EXC_STACK_SAVE (12*8) /* stack frame excluding BPF stack, ensure this is quadword aligned */ #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SAVE) +/* + * same as BPF_PPC_STACKFRAME with save area for additional + * non volatile registers saved at exception boundary. + * This is quad-word aligned. + */ +#define BPF_PPC_EXC_STACKFRAME BPF_PPC_STACKFRAME + BPF_PPC_EXC_STACK_SAVE =20 /* BPF register usage */ #define TMP_REG_1 (MAX_BPF_JIT_REG + 0) @@ -96,17 +110,23 @@ static inline bool bpf_has_stack_frame(struct codegen_= context *ctx) * [ prev sp ] <------------- * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- - * [ nv gpr save area ] 6*8 + * [ nv gpr save area ] 6*8 + (12*8) * [ tail_call_info ] 8 * [ local_tmp_var ] 24 * [ unused red zone ] 224 + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ static int bpf_jit_stack_local(struct codegen_context *ctx) { if (bpf_has_stack_frame(ctx)) return STACK_FRAME_MIN_SIZE + ctx->stack_size; else - return -(BPF_PPC_STACK_SAVE + 32); + return -(BPF_PPC_STACK_SAVE + + (ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACK_SAVE : 0) + + 32) /* tail_call_info + local_tmp_var */; } =20 int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) @@ -116,10 +136,20 @@ int bpf_jit_stack_tailcallinfo_offset(struct codegen_= context *ctx) =20 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg) { - if (reg >=3D BPF_PPC_NVR_MIN && reg < 32) + int min_valid_nvreg =3D BPF_PPC_NVR_MIN; + /* Default frame size for all cases except exception boundary */ + int frame_nvr_size =3D BPF_PPC_STACKFRAME; + + /* Consider all nv regs for handling exceptions */ + if (ctx->exception_boundary || ctx->exception_cb) { + min_valid_nvreg =3D _R14; + frame_nvr_size =3D BPF_PPC_EXC_STACKFRAME; + } + + if (reg >=3D min_valid_nvreg && reg < 32) return (bpf_has_stack_frame(ctx) ? - (BPF_PPC_STACKFRAME + ctx->stack_size) : 0) - - (8 * (32 - reg)); + frame_nvr_size + ctx->stack_size : 0) + - (8 * (32 - reg)); =20 pr_err("BPF JIT is asking about unknown registers"); BUG(); @@ -179,7 +209,20 @@ void bpf_jit_build_prologue(u32 *image, struct codegen= _context *ctx) EMIT(PPC_RAW_STD(_R0, _R1, PPC_LR_STKOFF)); } =20 - EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size))); + int stack_expand =3D ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACKFRAME : BPF_PPC_STACKFRAME; + EMIT(PPC_RAW_STDU(_R1, _R1, -(stack_expand + ctx->stack_size))); + } + + /* + * Program acting as exception boundary pushes R14..R25 in addition to + * BPF callee-saved non volatile registers. Exception callback uses + * the boundary program's stack frame, recover additionally saved + * registers in epilogue of exception callback. + */ + if (ctx->exception_boundary) { + for (i =3D _R14; i <=3D _R25; i++) + EMIT(PPC_RAW_STD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); } =20 if (!ctx->exception_cb) { @@ -227,6 +270,14 @@ static void bpf_jit_emit_common_epilogue(u32 *image, s= truct codegen_context *ctx bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 if (ctx->exception_cb) { + /* + * Recover additionally saved non volatile registers from stack + * frame of exception boundary program. + */ + for (i =3D _R14; i <=3D _R25; i++) { + EMIT(PPC_RAW_LD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); + } + /* * LR value from boundary-frame is received as second parameter * in exception callback. @@ -236,7 +287,10 @@ static void bpf_jit_emit_common_epilogue(u32 *image, s= truct codegen_context *ctx =20 /* Tear down our stack frame */ if (bpf_has_stack_frame(ctx)) { - EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size)); + int stack_shrink =3D ctx->exception_cb || ctx->exception_boundary ? + BPF_PPC_EXC_STACKFRAME : BPF_PPC_STACKFRAME; + EMIT(PPC_RAW_ADDI(_R1, _R1, stack_shrink + ctx->stack_size)); + if (ctx->seen & SEEN_FUNC || ctx->exception_cb) { EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF)); EMIT(PPC_RAW_MTLR(_R0)); --=20 2.48.1