From nobody Sun Feb 8 05:07:45 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FAA32C0F6D; Sat, 24 Jan 2026 07:54:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241255; cv=none; b=P78Iq2vzkJ+phDWU4CBNJyweeBkjR20vRiWlGu9oL7Cq8DPNw+NQ25pXTP6hJN6KAmF3ojBfM9jEqcVVhs63Es/odIQPxnPRsInSPww12Jw/sc0zbeL3k1Kj6EeX4VSdEAFAZe0U+yWRUBZq/geERMyByQ8qy9qv8h9pwn/I2ZQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241255; c=relaxed/simple; bh=feP4cI73aQ+5/rIlCrDHWW7cRk0x0Gn9TYzNVeOr0AQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p6l0c/fuTPS24wBPW8nGL8IHRfjT3ecXqT14N+I4R6AKNlf7WWFcyxq6um/H2WYnxouDPqY1EX/fP6i6GOYYjXntvQKSVRqVAKB1zbOaXwlxygkvtw9XW9uXjkfN7q/W7grTVUhR1MP3vJTXSINXC85c2slNiePDk97bWtVWOfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=IdevFVZv; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="IdevFVZv" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60O6qn5J007835; Sat, 24 Jan 2026 07:53:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=xSguuhn/XN2sPmGDH 4t1n1olL+gYWGXSgbvhcVrXBHI=; b=IdevFVZve+y4lWUAEIkgEV54Cq6DzE7Ql nySJQUBtNKxySGPxXWpniDkPpk15PacKDJP0F4Y1TEO6bItoEdPZ/vkO9JDmkXmg SFT66s41aRdRYDb2ptMN39bIGtqbPlvn1ZvwgsX6bP6aYlVkqxF2GdtVxYHWrxEj /iX+du2zRakum9u4QM7N43NrnBT1/YBW14d8dqQAPqlgEM8WwSRnPIZ7XiD6xfxs OGecJehFud06Gxjmg4MMmnC3tFP0WJ/EwoXrKzEA2JpPmK962tivaWRpMNk0CwyT pEyvrMGDiNNVc45yT/bMvflonWoB5K+2+7zkEDnXg3CTUCyhHsGzw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnr5rjbu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:35 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60O7rYco011228; Sat, 24 Jan 2026 07:53:35 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnr5rjbr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:34 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60O3CI3R001162; Sat, 24 Jan 2026 07:53:33 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4brpykcr80-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:33 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60O7rUhj15532296 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Jan 2026 07:53:30 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B46E20043; Sat, 24 Jan 2026 07:53:30 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 741BA20040; Sat, 24 Jan 2026 07:53:23 +0000 (GMT) Received: from abhi.. (unknown [9.124.223.59]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 24 Jan 2026 07:53:23 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v5 6/6] powerpc64/bpf: Additional NVR handling for bpf_throw Date: Sat, 24 Jan 2026 13:22:23 +0530 Message-ID: <20260124075223.6033-7-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260124075223.6033-1-adubey@linux.ibm.com> References: <20260124075223.6033-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI0MDA2MCBTYWx0ZWRfXwIMTuDrWGuT6 /H2850GJCYMc6zD3tXA/jN/XZx5iqY7ibSbDzJYvbSRnZQ3hnVqGu1kiVOG1DUZlu7LRfkwWU81 mwYkymFs9vEYswaZ8SUAWsqaj2D5CfX+gmQidyVZLuevmd8vKjj6D5vor5U6iRtQoKUOQEICKA7 NuMKIocWLn/36KDj+xcxBcAuhwbKiCleffxgJZoU3l8xmZvapEjNUxpPP5N1Ovuwr1gPB0UoWCk 4hEKBmKAbgWc8t4/JQdDeUYuLNhe7w7FYz58KayWL+6UtspJCIoZ1UTfFAhSPYSoKKbbiHX2DgQ CMSXLywMD6FCgMyFqX+MNuAZDAAUMQXpT39Ab3tTzhtEj8ewQgiw39S6vplNyAH4Pr3pyJGzKxx CZqRf9f2IP8fAbeYSJeAmz9V3wY7w82WUDqF3tDV9Q5+wGy0Z3zymLmQxwuGiotycjx5jRt+t3O J3oICFxoDgqnGpV39dA== X-Proofpoint-GUID: LVkydNDPMIYNuIrXjVl4Kgp5S0ijs0rA X-Proofpoint-ORIG-GUID: 2eZVcQPTNMY13b4vd8jIcqExKuNMJY0B X-Authority-Analysis: v=2.4 cv=X+Vf6WTe c=1 sm=1 tr=0 ts=69747a7f cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=2yuXeg4sgMS8061G0UsA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-24_02,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 impostorscore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 adultscore=0 malwarescore=0 bulkscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601240060 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The bpf_throw() function never returns, if it has clobbered any callee-saved register, those will remain clobbered. The prologue must take care of saving all callee-saved registers in the frame of exception boundary program. Later these additional non volatile registers R14-R25 along with other NVRs are restored back in the epilogue of exception callback. To achieve above objective, the frame size is determined dynamically to accommodate additional non volatile registers in exception boundary's frame. For non-exception boundary program, the frame size remains optimal. The additional instructions to save & restore r14-r25 registers are emitted only during exception boundary and exception callback program respectively. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp64.c | 89 ++++++++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index db121b1404fe..17de8b53a962 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -32,21 +32,37 @@ * * [ prev sp ] <------------- * [ tail_call_info ] 8 | - * [ nv gpr save area ] 6*8 | + * [ nv gpr save area ] 6*8 + (12*8) | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | * [ frame header ] 32/112 | * sp (r1) ---> [ stack pointer ] -------------- + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ =20 /* for bpf JIT code internal usage */ #define BPF_PPC_STACK_LOCALS 24 +/* + * for additional non volatile registers(r14-r25) to be saved + * at exception boundary + */ +#define BPF_PPC_EXC_STACK_SAVE (12*8) + /* stack frame excluding BPF stack, ensure this is quadword aligned */ #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ BPF_PPC_STACK_LOCALS + \ BPF_PPC_STACK_SAVE + \ BPF_PPC_TAILCALL) =20 +/* + * same as BPF_PPC_STACKFRAME with save area for additional + * non volatile registers saved at exception boundary. + * This is quad-word aligned. + */ +#define BPF_PPC_EXC_STACKFRAME (BPF_PPC_STACKFRAME + BPF_PPC_EXC_STACK_SAV= E) + /* BPF register usage */ #define TMP_REG_1 (MAX_BPF_JIT_REG + 0) #define TMP_REG_2 (MAX_BPF_JIT_REG + 1) @@ -88,10 +104,16 @@ static inline bool bpf_has_stack_frame(struct codegen_= context *ctx) * - we call other functions (kernel helpers), or * - the bpf program uses its stack area * The latter condition is deduced from the usage of BPF_REG_FP + * + * bpf_throw() leads to exception callback from a BPF (sub)program. + * The (sub)program is always marked as SEEN_FUNC, creating a stack + * frame. The exception callback uses the frame of the exception + * boundary, so the exception boundary program must have a frame. */ return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)) || - ctx->exception_cb; + ctx->exception_cb || + ctx->exception_boundary; } =20 /* @@ -103,9 +125,12 @@ static inline bool bpf_has_stack_frame(struct codegen_= context *ctx) * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- * [ tail_call_info ] 8 - * [ nv gpr save area ] 6*8 + * [ nv gpr save area ] 6*8 + (12*8) * [ local_tmp_var ] 24 * [ unused red zone ] 224 + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ static int bpf_jit_stack_local(struct codegen_context *ctx) { @@ -114,7 +139,12 @@ static int bpf_jit_stack_local(struct codegen_context = *ctx) return STACK_FRAME_MIN_SIZE + ctx->stack_size; } else { /* Stack layout with redzone */ - return -(BPF_PPC_TAILCALL + BPF_PPC_STACK_SAVE + BPF_PPC_STACK_LOCALS); + return -(BPF_PPC_TAILCALL + +BPF_PPC_STACK_SAVE + +(ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACK_SAVE : 0) + +BPF_PPC_STACK_LOCALS + ); } } =20 @@ -125,9 +155,19 @@ int bpf_jit_stack_tailcallinfo_offset(struct codegen_c= ontext *ctx) =20 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg) { - if (reg >=3D BPF_PPC_NVR_MIN && reg < 32) + int min_valid_nvreg =3D BPF_PPC_NVR_MIN; + /* Default frame size for all cases except exception boundary */ + int frame_nvr_size =3D BPF_PPC_STACKFRAME; + + /* Consider all nv regs for handling exceptions */ + if (ctx->exception_boundary || ctx->exception_cb) { + min_valid_nvreg =3D _R14; + frame_nvr_size =3D BPF_PPC_EXC_STACKFRAME; + } + + if (reg >=3D min_valid_nvreg && reg < 32) return (bpf_has_stack_frame(ctx) ? - (BPF_PPC_STACKFRAME + ctx->stack_size) : 0) + (frame_nvr_size + ctx->stack_size) : 0) - (8 * (32 - reg)) - BPF_PPC_TAILCALL; =20 pr_err("BPF JIT is asking about unknown registers"); @@ -138,6 +178,17 @@ void bpf_jit_realloc_regs(struct codegen_context *ctx) { } =20 +/* + * For exception boundary & exception_cb progs: + * return increased size to accommodate additional NVRs. + */ +static int bpf_jit_stack_size(struct codegen_context *ctx) +{ + return ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACKFRAME : + BPF_PPC_STACKFRAME; +} + void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx) { int i; @@ -198,7 +249,19 @@ void bpf_jit_build_prologue(u32 *image, struct codegen= _context *ctx) EMIT(PPC_RAW_STD(_R0, _R1, PPC_LR_STKOFF)); } =20 - EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size))); + EMIT(PPC_RAW_STDU(_R1, _R1, + -(bpf_jit_stack_size(ctx) + ctx->stack_size))); + } + + /* + * Program acting as exception boundary pushes R14..R25 in addition to + * BPF callee-saved non volatile registers. Exception callback uses + * the boundary program's stack frame, recover additionally saved + * registers in epilogue of exception callback. + */ + if (ctx->exception_boundary) { + for (i =3D _R14; i <=3D _R25; i++) + EMIT(PPC_RAW_STD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); } =20 if (!ctx->exception_cb) { @@ -248,9 +311,19 @@ static void bpf_jit_emit_common_epilogue(u32 *image, s= truct codegen_context *ctx EMIT(PPC_RAW_LD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 + if (ctx->exception_cb) { + /* + * Recover additionally saved non volatile registers from stack + * frame of exception boundary program. + */ + for (i =3D _R14; i <=3D _R25; i++) + EMIT(PPC_RAW_LD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); + } + /* Tear down our stack frame */ if (bpf_has_stack_frame(ctx)) { - EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size)); + EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_jit_stack_size(ctx) + ctx->stack_size)); + if (ctx->seen & SEEN_FUNC || ctx->exception_cb) { EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF)); EMIT(PPC_RAW_MTLR(_R0)); --=20 2.48.1