From nobody Sun Feb 8 05:59:54 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05F1C38B993; Thu, 22 Jan 2026 21:20:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116821; cv=none; b=LoYC+J0T7ncgbd62cuiUZ2lg0oCvwmiDpCQTVVelhV0CUsENmJ9k6xAyKFvO9mA1QoYYinbmmtSkdcfHKHLwsAw69+30A9zsRJbmKS+nzy/Pg6ulNx/t+lfqlKtkyYC3PFHaKoDaf1p5nDnUbjrSwibz+M0fPj47e8UvtvNCQ2A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116821; c=relaxed/simple; bh=5WogfuFbuvtQytL1wJYkBo1PFQJCL06nLi0Oc5DEItk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZrvDPjxHlN3NFURpbP2NzK317tEIpR81pxHTvAQUZ3VdNYVj94kRYby1N1UoDeY6GQeipP2I3x+6icYAB9riMIRcdg6mHn4XYb23zSVd4ZI/v7m8jSTpdeVZZQn1OF4DZR9QBpQt/IITX+EWd6iFH1e8FV3p5eIziPVOrS3dzNQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=rK7guGYC; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="rK7guGYC" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60MIHTON005600; Thu, 22 Jan 2026 21:19:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=MPzPgODU9C1SaV13C IbivxyR0sJHglUD9W4BPeaLFVM=; b=rK7guGYCdxa7p78VyELOyl8R4J8kojmhS X8zf84E07KVqYM5x51cvfIghHtd49AvfqzcAHDcIT7d2Ober4yxjbH1jTSZsf/BM ewdsBd/5a0uMEzVe1Klq9pWD4UbpcKI5eFjjFE3p3M4P9qGArvKkdBW10yPt5sHD Lkc2xFFHCRHyAeApLMuRbvEUJtGp0o/7BQLKXLvV8zvtUcm4coYzTXglYftUHZz6 3M08j+Kqsa8VLP/lUzblUYBEr/aUh7ubll/pGYAFXBI1mVEE6T8IQxpknw5w9M5t 3n5Gb+042fMble+t0bJb48KZT0aWnJ/TA6WRZH989/hPH7AfKoEaw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bus1prq5v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:19:25 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60MLJPwp003674; Thu, 22 Jan 2026 21:19:25 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bus1prq5t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:19:25 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60ML5sWv027298; Thu, 22 Jan 2026 21:19:24 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4brnrncy15-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:19:24 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60MLJKfm50332150 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jan 2026 21:19:20 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1E9C22004D; Thu, 22 Jan 2026 21:19:20 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E288020040; Thu, 22 Jan 2026 21:19:12 +0000 (GMT) Received: from abhi.. (unknown [9.124.222.159]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 22 Jan 2026 21:19:12 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org Subject: [PATCH v4 1/6] powerpc64/bpf: Moving tail_call_cnt to bottom of frame Date: Fri, 23 Jan 2026 02:48:49 +0530 Message-ID: <20260122211854.5508-2-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260122211854.5508-1-adubey@linux.ibm.com> References: <20260122211854.5508-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: zq2mBy5psqrjac-qqNpGJXmSd-qqHTfr X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTIyMDE2MiBTYWx0ZWRfX2hqJ/04LgnoL QE/pDWYCaCSDdeKNL6DWd/tv2P5Td/j6B5NuPraMCSzd6C27W5AWVLTWCkkKLVAooR2HhbaI68Y nE5ixadIAtaMdyM4EtmEcaSbXy2EvjdhJMSDeznHJjQ9X8nEBiHgTgxTKYUaiFsyAP/0Ws8EHKo yU8xNqAD6KOTQwDk8Az5JIeBJ9ok/dznl86ZIpSbK6Wt+S5M4HUequ2eDFz8hLaey4E2FirHcxe 8Tn0Uu6QXt2kbWzolUBtJpAUU4stxPZ+WoSVnlALpwds2Xb+yFI6q0+4f6SRMvsL+9lHvSMur+D FxdADlgO7b+dyHZIV7JI1cEDfob31vcVHOAYj88IajWrSgzq+7fHFB4Go9St3R7HWAR5qzwjI4i pQEma7/n93dO9YU1ThwaSLmBX3JhACR6/T+mCw3z3rKDIRmiRvgR7KfBT82x7vbcy0KBt4rkFVS 0pElWas25xg1pYq0wkA== X-Proofpoint-GUID: 8F6ZPkCXg0Uqm5ii68_r5aQrXU8mfR4S X-Authority-Analysis: v=2.4 cv=GY8aXAXL c=1 sm=1 tr=0 ts=6972945d cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=ESdMh1-tTcp0eb2Uk-UA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-22_05,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 impostorscore=0 phishscore=0 lowpriorityscore=0 spamscore=0 malwarescore=0 priorityscore=1501 bulkscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601220162 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey In the conventional stack frame, the position of tail_call_cnt is after the NVR save area (BPF_PPC_STACK_SAVE). Whereas, the offset of tail_call_cnt in the trampoline frame is after the stack alignment padding. BPF JIT logic could become complex when dealing with frame-sensitive offset calculation of tail_call_cnt. Having the same offset in both frames is the desired objective. The trampoline frame does not have a BPF_PPC_STACK_SAVE area. Introducing it leads to under-utilization of extra memory meant only for the offset alignment of tail_call_cnt. Another challenge is the variable alignment padding sitting at the bottom of the trampoline frame, which requires additional handling to compute tail_call_cnt offset. This patch addresses the above issues by moving tail_call_cnt to the bottom of the stack frame at offset 0 for both types of frames. This saves additional bytes required by BPF_PPC_STACK_SAVE in trampoline frame, and a common offset computation for tail_call_cnt serves both frames. The changes in this patch are required by the second patch in the series, where the 'reference to tail_call_info' of the main frame is copied into the trampoline frame from the previous frame. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit.h | 1 + arch/powerpc/net/bpf_jit_comp.c | 15 ++++++++++++--- arch/powerpc/net/bpf_jit_comp64.c | 31 ++++++++++++++++++++----------- 3 files changed, 33 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 8334cd667bba..9f6ec00bd02e 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -24,6 +24,7 @@ =20 #define SZL sizeof(unsigned long) #define BPF_INSN_SAFETY 64 +#define BPF_PPC_TAILCALL 8 =20 #define PLANT_INSTR(d, idx, instr) \ do { if (d) { (d)[idx] =3D instr; } idx++; } while (0) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 5e976730b2f5..d51c696221d7 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -604,8 +604,8 @@ static void bpf_trampoline_setup_tail_call_cnt(u32 *ima= ge, struct codegen_contex int func_frame_offset, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { - /* See bpf_jit_stack_tailcallcnt() */ - int tailcallcnt_offset =3D 7 * 8; + /* See Generated stack layout */ + int tailcallcnt_offset =3D BPF_PPC_TAILCALL; =20 EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); EMIT(PPC_RAW_STL(_R3, _R1, -tailcallcnt_offset)); @@ -620,7 +620,7 @@ static void bpf_trampoline_restore_tail_call_cnt(u32 *i= mage, struct codegen_cont { if (IS_ENABLED(CONFIG_PPC64)) { /* See bpf_jit_stack_tailcallcnt() */ - int tailcallcnt_offset =3D 7 * 8; + int tailcallcnt_offset =3D BPF_PPC_TAILCALL; =20 EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset)); EMIT(PPC_RAW_STL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); @@ -714,6 +714,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im * LR save area [ r0 save (64-bit) ] | header * [ r0 save (32-bit) ] | * dummy frame for unwind [ back chain 1 ] -- + * [ tail_call_cnt ] optional - 64-bit p= owerpc * [ padding ] align stack frame * r4_off [ r4 (tailcallcnt) ] optional - 32-bit p= owerpc * alt_lr_off [ real lr (ool stub)] optional - actual lr @@ -795,6 +796,14 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tr= amp_image *im, void *rw_im } } =20 + /* + * Save tailcall count pointer at the same offset on the + * stack where subprogs expect it + */ + if ((flags & BPF_TRAMP_F_CALL_ORIG) && + (flags & BPF_TRAMP_F_TAIL_CALL_CTX)) + bpf_frame_size +=3D BPF_PPC_TAILCALL; + /* Padding to align stack frame, if any */ bpf_frame_size =3D round_up(bpf_frame_size, SZL * 2); =20 diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 1fe37128c876..296e9ea14f2e 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -20,13 +20,15 @@ #include "bpf_jit.h" =20 /* - * Stack layout: + * Stack layout with frame: + * Layout when setting up our own stack frame. + * Note: r1 at bottom, component offsets positive wrt r1. * Ensure the top half (upto local_tmp_var) stays consistent * with our redzone usage. * * [ prev sp ] <------------- - * [ nv gpr save area ] 6*8 | * [ tail_call_cnt ] 8 | + * [ nv gpr save area ] 6*8 | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | * [ frame header ] 32/112 | @@ -36,10 +38,12 @@ /* for gpr non volatile registers BPG_REG_6 to 10 */ #define BPF_PPC_STACK_SAVE (6*8) /* for bpf JIT code internal usage */ -#define BPF_PPC_STACK_LOCALS 32 +#define BPF_PPC_STACK_LOCALS 24 /* stack frame excluding BPF stack, ensure this is quadword aligned */ #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ - BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SAVE) + BPF_PPC_STACK_LOCALS + \ + BPF_PPC_STACK_SAVE + \ + BPF_PPC_TAILCALL) =20 /* BPF register usage */ #define TMP_REG_1 (MAX_BPF_JIT_REG + 0) @@ -87,27 +91,32 @@ static inline bool bpf_has_stack_frame(struct codegen_c= ontext *ctx) } =20 /* + * Stack layout with redzone: * When not setting up our own stackframe, the redzone (288 bytes) usage i= s: + * Note: r1 from prev frame. Component offset negative wrt r1. * * [ prev sp ] <------------- * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- - * [ nv gpr save area ] 6*8 * [ tail_call_cnt ] 8 + * [ nv gpr save area ] 6*8 * [ local_tmp_var ] 24 * [ unused red zone ] 224 */ static int bpf_jit_stack_local(struct codegen_context *ctx) { - if (bpf_has_stack_frame(ctx)) + if (bpf_has_stack_frame(ctx)) { + /* Stack layout with frame */ return STACK_FRAME_MIN_SIZE + ctx->stack_size; - else - return -(BPF_PPC_STACK_SAVE + 32); + } else { + /* Stack layout with redzone */ + return -(BPF_PPC_TAILCALL + BPF_PPC_STACK_SAVE + BPF_PPC_STACK_LOCALS); + } } =20 static int bpf_jit_stack_tailcallcnt(struct codegen_context *ctx) { - return bpf_jit_stack_local(ctx) + 24; + return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SA= VE; } =20 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg) @@ -115,7 +124,7 @@ static int bpf_jit_stack_offsetof(struct codegen_contex= t *ctx, int reg) if (reg >=3D BPF_PPC_NVR_MIN && reg < 32) return (bpf_has_stack_frame(ctx) ? (BPF_PPC_STACKFRAME + ctx->stack_size) : 0) - - (8 * (32 - reg)); + - (8 * (32 - reg)) - BPF_PPC_TAILCALL; =20 pr_err("BPF JIT is asking about unknown registers"); BUG(); @@ -145,7 +154,7 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_= context *ctx) if (ctx->seen & SEEN_TAILCALL) { EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0)); /* this goes in the redzone */ - EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_STACK_SAVE + 8))); + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } else { EMIT(PPC_RAW_NOP()); EMIT(PPC_RAW_NOP()); --=20 2.48.1 From nobody Sun Feb 8 05:59:54 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 321033A2ACB; Thu, 22 Jan 2026 21:21:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116889; cv=none; b=FMJwxO3wHXIVk52xTNRB1nKH8M2CJZRKqOEu4k7nsgriYNv57eypqad2CfLVDG7VgqsjFqUqRR/X6sYv3liSrNzdCUDtPa6ijOyes7sjHlsAA7MTp3jVXibx/j5a0uJzjlgIpLL8ylmNyl8GdSJn4+CM4C2BWJpCilG0oVBjjRw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116889; c=relaxed/simple; bh=JT+XY6dPXrZevB96PyhaUGjT5k7sQ6A0mkXXTVFUJOk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=M+I0oYB9MWtLYH5Q8HKXSnxuoYkGR0kt0QO3R4UtnMUNnJ1Lyiut5uPECBvaP0UwlZTpzIN4DfUjvDxqBnbT0Hpat+mFIML7bcUJpC4jVwTT4N+SMzCIykV6eY+Wq7G/mkh6+WsYUM+KIZ37Szt74Nha0oz0i+cShYrd+vMcpos= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=FswGjdde; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="FswGjdde" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60MDm7Jo028639; Thu, 22 Jan 2026 21:19:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=lIXJpPOz6M8n5l/5x rS+RIZGbVNIZ9gJkeFEAK0Jml0=; b=FswGjddeoolbPnzdKoBbJpOYXf1cfVm67 3vlkdj0C6g6OjAU/UK16ZB2FNkcXUHVKR61ZcxfFKA467tlMIuDP1OEqKto2iI46 Na603O44AiNokOBDlo1t5diYngZUmFFdv+UZx6q5GvMt5QEbr6s3IxEW/yMMdo4O w/w27gBBhxir9oDrWOynS/+F65UQLtRJCIeQmGrBBIQ+tYzuFBfKpQH6rZeYnXlZ LIv5NZxyalEey/arnGXrtnFPMfvyBkUsBQarMG223EgcIpq21HPCkwb8OkHham7Y U4AmvWykOiwyCcGRFSwY6PZg/zMFurmvwuNwYy0dAdZ9iqGauS97w== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bt60eys1s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:19:37 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60MLGN2v018099; Thu, 22 Jan 2026 21:19:36 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bt60eys1p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:19:36 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60MLCoHm024572; Thu, 22 Jan 2026 21:19:35 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4brxas39ck-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:19:35 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60MLJW0r48628080 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jan 2026 21:19:32 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id ED66020043; Thu, 22 Jan 2026 21:19:31 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5275020040; Thu, 22 Jan 2026 21:19:25 +0000 (GMT) Received: from abhi.. (unknown [9.124.222.159]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 22 Jan 2026 21:19:25 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org Subject: [PATCH v4 2/6] powerpc64/bpf: Support tailcalls with subprogs Date: Fri, 23 Jan 2026 02:48:50 +0530 Message-ID: <20260122211854.5508-3-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260122211854.5508-1-adubey@linux.ibm.com> References: <20260122211854.5508-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=WMdyn3sR c=1 sm=1 tr=0 ts=69729469 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=aLVHvtoiTwlSXXnS9I8A:9 X-Proofpoint-GUID: inYvG4gxFFXM2SGktPvXODiYVtIAwuik X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTIyMDE2MiBTYWx0ZWRfXyKA5WYEbZili qL8F6PMOmiZ9luRYRP/dWmEM6yuXOVeB1kNBvSB3cNCNHkogp2B/vGjoVmdVhU79fslJwI4GewV NvFQfgTcGZbfXnF0zvFoSQ3bK7MecJdlHYkInCE0uLMKuu/TGYdq1ZakbXFxua4wLnJoVDrlL1g 9QljcwwCF6bzChGLzLPhUMV8BBLtJJU3hv+8Sp3y+W7B9Rt0zaKEXknFgxb9YQ8Tc2bhY2OcccM juG6L7LYTr5P3101Ly5qXKfVMFa9l+86WP++jkAndzjfxjMUnikPDcrMNyQu9WnJAfkS/jkiF9k gy9mjm93CRH2AtrsyWT7XPYEM1hSkDY5gY5m36pH302J3C6jhgz3+YkVFut/8C4llhN0TvHHZi6 uffTOOH6gq+lyHtPHFsSQs2dtKb/raw2u91QdNTJdWs6ajINyHFqM3lCJ1j+QVPraiIK4kTwtTm J5WE06fYFy/NsN5wkAg== X-Proofpoint-ORIG-GUID: _VUCRqpOhEklSNHNicensI3tOp8aVDsh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-22_05,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 suspectscore=0 bulkscore=0 adultscore=0 impostorscore=0 spamscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601220162 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey Enabling tailcalls with subprog combinations by referencing method. The actual tailcall count is always maintained in the tail_call_info variable present in the frame of main function (also called entry function). The tail_call_info field in the stack frame of subprogs contains reference to the tail_call_info field in the stack frame of main BPF program. Dynamic resolution interprets the tail_call_info either as value or reference depending on the context of active frame while tailcall is invoked. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit.h | 13 ++++++ arch/powerpc/net/bpf_jit_comp.c | 59 +++++++++++++++++++++++---- arch/powerpc/net/bpf_jit_comp64.c | 68 +++++++++++++++++++++++-------- 3 files changed, 117 insertions(+), 23 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 9f6ec00bd02e..56f56fdd4969 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -52,6 +52,13 @@ EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc)= ); \ } while (0) =20 +/* When constant jump offset is known prior */ +#define PPC_BCC_CONST_SHORT(cond, offset) \ + do { \ + BUILD_BUG_ON(offset < -0x8000 || offset > 0x7fff || (offset & 0x3)); \ + EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc)= ); \ + } while (0) + /* * Sign-extended 32-bit immediate load * @@ -73,6 +80,10 @@ } } while (0) =20 #ifdef CONFIG_PPC64 + +/* for gpr non volatile registers BPG_REG_6 to 10 */ +#define BPF_PPC_STACK_SAVE (6 * 8) + /* If dummy pass (!image), account for maximum possible instructions */ #define PPC_LI64(d, i) do { \ if (!image) \ @@ -167,6 +178,7 @@ struct codegen_context { unsigned int alt_exit_addr; u64 arena_vm_start; u64 user_vm_start; + bool is_subprog; }; =20 #define bpf_to_ppc(r) (ctx->b2p[r]) @@ -206,6 +218,7 @@ int bpf_add_extable_entry(struct bpf_prog *fp, u32 *ima= ge, u32 *fimage, int pass struct codegen_context *ctx, int insn_idx, int jmp_off, int dst_reg, u32 code); =20 +int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx); #endif =20 #endif diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index d51c696221d7..93355ba5382a 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -206,6 +206,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *f= p) cgctx.stack_size =3D round_up(fp->aux->stack_depth, 16); cgctx.arena_vm_start =3D bpf_arena_get_kern_vm_start(fp->aux->arena); cgctx.user_vm_start =3D bpf_arena_get_user_vm_start(fp->aux->arena); + cgctx.is_subprog =3D bpf_is_subprog(fp); =20 /* Scouting faux-generate pass 0 */ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) { @@ -435,6 +436,11 @@ void bpf_jit_free(struct bpf_prog *fp) bpf_prog_unlock_free(fp); } =20 +bool bpf_jit_supports_subprog_tailcalls(void) +{ + return IS_ENABLED(CONFIG_PPC64); +} + bool bpf_jit_supports_kfunc_call(void) { return true; @@ -600,15 +606,53 @@ static int invoke_bpf_mod_ret(u32 *image, u32 *ro_ima= ge, struct codegen_context return 0; } =20 -static void bpf_trampoline_setup_tail_call_cnt(u32 *image, struct codegen_= context *ctx, - int func_frame_offset, int r4_off) +/* + * Refer the label 'Generated stack layout' in this file for actual stack + * layout during trampoline invocation. + * + * Refer __arch_prepare_bpf_trampoline() for stack component details. + * + * The tailcall count/reference is present in caller's stack frame. Its re= quired + * to copy the content of tail_call_info before calling the actual function + * to which the trampoline is attached. + */ +static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen= _context *ctx, + int func_frame_offset, + int bpf_dummy_frame_size, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { /* See Generated stack layout */ - int tailcallcnt_offset =3D BPF_PPC_TAILCALL; + int tailcallinfo_offset =3D BPF_PPC_TAILCALL; + + /* + * func_frame_offset =3D ...(1) + * bpf_dummy_frame_size + trampoline_frame_size + */ + EMIT(PPC_RAW_LD(_R4, _R1, func_frame_offset)); + EMIT(PPC_RAW_LD(_R3, _R4, -tailcallinfo_offset)); =20 - EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); - EMIT(PPC_RAW_STL(_R3, _R1, -tailcallcnt_offset)); + /* + * Setting the tail_call_info in trampoline's frame + * depending on if previous frame had value or reference. + */ + EMIT(PPC_RAW_CMPLWI(_R3, MAX_TAIL_CALL_CNT)); + PPC_BCC_CONST_SHORT(COND_GT, 8); + EMIT(PPC_RAW_ADDI(_R3, _R4, bpf_jit_stack_tailcallinfo_offset(ctx))); + /* + * From ...(1) above: + * trampoline_frame_bottom =3D ...(2) + * func_frame_offset - bpf_dummy_frame_size + * + * Using ...(2) derived above: + * trampoline_tail_call_info_offset =3D ...(3) + * trampoline_frame_bottom - tailcallinfo_offset + * + * From ...(3): + * Use trampoline_tail_call_info_offset to write reference of main's + * tail_call_info in trampoline frame. + */ + EMIT(PPC_RAW_STL(_R3, _R1, (func_frame_offset - bpf_dummy_frame_size) + - tailcallinfo_offset)); } else { /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ EMIT(PPC_RAW_LL(_R4, _R1, r4_off)); @@ -714,7 +758,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im * LR save area [ r0 save (64-bit) ] | header * [ r0 save (32-bit) ] | * dummy frame for unwind [ back chain 1 ] -- - * [ tail_call_cnt ] optional - 64-bit p= owerpc + * [ tail_call_info ] optional - 64-bit p= owerpc * [ padding ] align stack frame * r4_off [ r4 (tailcallcnt) ] optional - 32-bit p= owerpc * alt_lr_off [ real lr (ool stub)] optional - actual lr @@ -905,7 +949,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im =20 /* Replicate tail_call_cnt before calling the original BPF prog */ if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) - bpf_trampoline_setup_tail_call_cnt(image, ctx, func_frame_offset, r4_of= f); + bpf_trampoline_setup_tail_call_info(image, ctx, func_frame_offset, + bpf_dummy_frame_size, r4_off); =20 /* Restore args */ bpf_trampoline_restore_args_stack(image, ctx, func_frame_offset, nr_regs= , regs_off); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 296e9ea14f2e..18da5a866447 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -26,8 +26,12 @@ * Ensure the top half (upto local_tmp_var) stays consistent * with our redzone usage. * + * tail_call_info - stores tailcall count value in main program's + * frame, stores reference to tail_call_info of + * main's frame in sub-prog's frame. + * * [ prev sp ] <------------- - * [ tail_call_cnt ] 8 | + * [ tail_call_info ] 8 | * [ nv gpr save area ] 6*8 | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | @@ -35,8 +39,6 @@ * sp (r1) ---> [ stack pointer ] -------------- */ =20 -/* for gpr non volatile registers BPG_REG_6 to 10 */ -#define BPF_PPC_STACK_SAVE (6*8) /* for bpf JIT code internal usage */ #define BPF_PPC_STACK_LOCALS 24 /* stack frame excluding BPF stack, ensure this is quadword aligned */ @@ -98,7 +100,7 @@ static inline bool bpf_has_stack_frame(struct codegen_co= ntext *ctx) * [ prev sp ] <------------- * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- - * [ tail_call_cnt ] 8 + * [ tail_call_info ] 8 * [ nv gpr save area ] 6*8 * [ local_tmp_var ] 24 * [ unused red zone ] 224 @@ -114,7 +116,7 @@ static int bpf_jit_stack_local(struct codegen_context *= ctx) } } =20 -static int bpf_jit_stack_tailcallcnt(struct codegen_context *ctx) +int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) { return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SA= VE; } @@ -147,17 +149,32 @@ void bpf_jit_build_prologue(u32 *image, struct codege= n_context *ctx) #endif =20 /* - * Initialize tail_call_cnt if we do tail calls. - * Otherwise, put in NOPs so that it can be skipped when we are - * invoked through a tail call. + * Tail call count(tcc) is saved & updated only in main + * program's frame and the address of tcc in main program's + * frame (tcc_ptr) is saved in subprogs frame. + * + * Offset of tail_call_info on any frame will be interpreted + * as either tcc_ptr or tcc value depending on whether it is + * greater than MAX_TAIL_CALL_CNT or not. */ - if (ctx->seen & SEEN_TAILCALL) { + if (!ctx->is_subprog) { EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0)); /* this goes in the redzone */ EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } else { - EMIT(PPC_RAW_NOP()); - EMIT(PPC_RAW_NOP()); + /* + * if tail_call_info < MAX_TAIL_CALL_CNT + * main prog calling first subprog -> copy reference + * else + * subsequent subprog calling another subprog -> directly copy cont= ent + */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, 0)); + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), -(BPF_PPC_= TAILCALL))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + PPC_BCC_CONST_SHORT(COND_GT, 8); + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), + -(BPF_PPC_TAILCALL))); + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } =20 if (bpf_has_stack_frame(ctx)) { @@ -352,19 +369,38 @@ static int bpf_jit_emit_tail_call(u32 *image, struct = codegen_context *ctx, u32 o EMIT(PPC_RAW_CMPLW(b2p_index, bpf_to_ppc(TMP_REG_1))); PPC_BCC_SHORT(COND_GE, out); =20 + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallinfo_of= fset(ctx))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + PPC_BCC_CONST_SHORT(COND_LE, 8); + + /* dereference TMP_REG_1 */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 0)); + /* - * if (tail_call_cnt >=3D MAX_TAIL_CALL_CNT) + * if (tail_call_info =3D=3D MAX_TAIL_CALL_CNT) * goto out; */ - EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallcnt(ctx= ))); EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); - PPC_BCC_SHORT(COND_GE, out); + PPC_BCC_SHORT(COND_EQ, out); =20 /* - * tail_call_cnt++; + * tail_call_info++; <- Actual value of tcc here */ EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 1)); - EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallcnt(ct= x))); + + /* + * Before writing updated tail_call_info, distinguish if current frame + * is storing a reference to tail_call_info or actual tcc value in + * tail_call_info. + */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, bpf_jit_stack_tailcallinfo_of= fset(ctx))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_2), MAX_TAIL_CALL_CNT)); + PPC_BCC_CONST_SHORT(COND_GT, 8); + + /* First get address of tail_call_info */ + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_2), _R1, bpf_jit_stack_tailcallinfo_= offset(ctx))); + /* Writeback updated value to tail_call_info */ + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), 0)); =20 /* prog =3D array->ptrs[index]; */ EMIT(PPC_RAW_MULI(bpf_to_ppc(TMP_REG_1), b2p_index, 8)); --=20 2.48.1 From nobody Sun Feb 8 05:59:54 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1E6634B1B6; Thu, 22 Jan 2026 21:21:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116882; cv=none; b=UpXZNDuj8RNMtgZ/I8HmrNzPFqZfdJ56Bv6pyYGvuwWe1gL+3ji3YL2iL2U3htb8Hkda34yGlI/3b1HBo+oZT2LIinXgzi3j6EU/QUixWc2IoDOFujaI4u5Wlp2DHymYGP2UEmodO1dnRIL2JMZk4hvdHFeKw0XrKJGMT6dnKrU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116882; c=relaxed/simple; bh=okLPwzpehcD1gB91punSYSgMEs8dxU5bAyjroI+uESg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nMHPZIgbHLYHGW2n8rEAFQCHSKNivBUE5RxdvrxBJ8h2vDaaqmcOnfSMgim31xlsRRmwqdeqVgZBkI6jSApqD3pYv/ZWtjJQ92n2JxtQBrMCWBbyCAeAaTm15A038boCcmzObdUQrtDL+4NnjS1FK0JvDj315GR+ZgyGNhZHZLQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=g7HXFva3; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="g7HXFva3" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60MD86fQ006766; Thu, 22 Jan 2026 21:19:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=15+B6CbLt2ChyR8qt ylg18EYlouONEheH96W2YTGceI=; b=g7HXFva3akx2SufG19+76Q1rZ1c9r8ax0 VvRQVmfcFNgF7HHenYskfuvnCQ3wqHYF7vf9wYFHfAI+cd9yC60JWg93o6onQ7YD TJ61mMCYm2sNEPIH6gftldH/oLlsfP8oWX5s6qgjvjd1e7LLt2PGLeJKzvz97r1n wnODdVbpEyZI5YreAOvxNV2Bl8SYHnCIYpP99tPa9tjaRCjjWPi8NC8fhjulSA/p q5Nkro9tnYhY7lctId4q38iQaA9K3tJbUVCjuTPr5F5n96MWJmo3tNeoVzevmQr9 N/MEpcrBjgVpADjMOS/l9OITXRJcd9kNfiHw62QUCAJIIhc/kP+Ig== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4br23sc90w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:19:50 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60MLJn9Y018809; Thu, 22 Jan 2026 21:19:49 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4br23sc90u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:19:49 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60MKKfuU016611; Thu, 22 Jan 2026 21:19:48 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4brn4yd3v9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:19:47 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60MLJh7745744428 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jan 2026 21:19:43 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C03E32004B; Thu, 22 Jan 2026 21:19:43 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6508420040; Thu, 22 Jan 2026 21:19:37 +0000 (GMT) Received: from abhi.. (unknown [9.124.222.159]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 22 Jan 2026 21:19:37 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org Subject: [PATCH v4 3/6] powerpc64/bpf: Avoid tailcall restore from trampoline Date: Fri, 23 Jan 2026 02:48:51 +0530 Message-ID: <20260122211854.5508-4-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260122211854.5508-1-adubey@linux.ibm.com> References: <20260122211854.5508-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: GPwLLMBfru7JZbASjB3ZK79lwNgowuaC X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTIyMDE2MiBTYWx0ZWRfX7jrp/6j5IKgq nkjEL6fWOb1cwK6VUJk2BEww/LuWV96XbkNN0i7ejG0M/GhsC9E+NsBaKloHr/sRiJGEVQZQA/V qqrWeKhE9ZWOefMjeOYb/tzRHNtGOmpj0bEnsmJW1cC/ITFSi4+i9/AipNsVBvxOi08x5EzjUbK +kG4NCcELPN9S1dfqNNCRC/eykiqaVxlqGRFbpxC+SXgul13RYro8Pa82bsi07EbdnLX/yathMd lC2GtEjPCcem9jhIKe6iGU4epwgA8Pg1zLft6jOc/kw2JrlzxvG09EQPU9XMKdD1C+gZTQrulrZ uFAafwsOlpp0Yss+TlsjXT0bnauFpYWDtW3Plh4UmzOjwIMPkQlAyvwz/dX43onoaVE8rn7yqG3 9V1PIE9I5A61UAiq22tRRW4BX62u+/DxPlqenqxeayhza7bTRg5miXwbmPo9s4jUjlFZMnTAbIC WaxSpc8101QXgN1Q+Pw== X-Authority-Analysis: v=2.4 cv=J9SnLQnS c=1 sm=1 tr=0 ts=69729476 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=yUNxoNuUyiXgUDPvJtcA:9 X-Proofpoint-ORIG-GUID: qfme069ZT8EvMbnwutMdsEnYi-f7akUY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-22_05,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 impostorscore=0 adultscore=0 suspectscore=0 spamscore=0 lowpriorityscore=0 malwarescore=0 clxscore=1015 bulkscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601220162 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey Back propagation of tailcall count is no longer needed for powerpc64 due to use of reference, which updates the tailcall count in the tail_call_info field in the frame of the main program only. Back propagation is still required for 32-bit powerpc. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 93355ba5382a..1a305f0fed27 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -662,14 +662,11 @@ static void bpf_trampoline_setup_tail_call_info(u32 *= image, struct codegen_conte static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codege= n_context *ctx, int func_frame_offset, int r4_off) { - if (IS_ENABLED(CONFIG_PPC64)) { - /* See bpf_jit_stack_tailcallcnt() */ - int tailcallcnt_offset =3D BPF_PPC_TAILCALL; - - EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset)); - EMIT(PPC_RAW_STL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); - } else { - /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ + if (IS_ENABLED(CONFIG_PPC32)) { + /* + * Restore tailcall for 32-bit powerpc + * See bpf_jit_stack_offsetof() and BPF_PPC_TC + */ EMIT(PPC_RAW_STL(_R4, _R1, r4_off)); } } --=20 2.48.1 From nobody Sun Feb 8 05:59:54 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C1763ACF04; Thu, 22 Jan 2026 21:26:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769117167; cv=none; b=YrZLjzOM829GJjBHBVuLjzM4oEEB5OMdzQdA9jCZv7dZbIf0j44hxd/2YJlhzYK4CZvnuOim+hyBisIdL9hYRodQ4AfcBJPgkQoUyZFGeaYlhhDIPNnFWMSsxKwrG7p9OpIGW+jXecigSuTeeUQHj4cq7lIO6J2Sq+AJJaOv2UI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769117167; c=relaxed/simple; bh=ZZiPNtMVFac7HGXvfmtml7hvEu08J1Eucvq6mX/4TTY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ugY1PIoCzJX8eO+6YuBOT7SlKv0v+fT6/CLbDU4iOXTf8gZSZpBsiKNpYUJzKu22DI/QMQPifEVRguzLThTf+7ZAXwVV6VVqkEvO35IWqxGAlaSYKhjeqXQ/5V6IsK9w0v4iQRj/rC+FduFpUrD97lIq8+DWM4mNfkC/q+P4vGY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=i7GWayn/; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="i7GWayn/" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60MEXF3c021316; Thu, 22 Jan 2026 21:20:02 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=TXgmDXs4JFjAGlSMt ms1ah2TwLO40IrSCwqYBvl/j9w=; b=i7GWayn/QQwrubpnUNbMqu4jZdNn+nOW/ 1u/nQ4+BAjDVOFfg5rwqxGhMSttGrVVVO1tjXVAh9FjppGepgnFO2PDX3D1m6zZD Sze1KGVN5ZtE5/kJ/Wt54VTfnFscF/UExm7I9xgcOEFkI5uGDMWLQWLOzUdT/Cvp dZClv8TL3aCCvSgpGLRZjxEy32h1pbstqvDkSWbmq950ep8gqyXpGPptxt0izSav nvld4lqeGmcQkOyTWZTb+wPLlKbbagR4vqRF162JyCSqMOrBWuqhZLHq5fkJ+m+O Hyrz/q+RAhVQ31AQeY96HiJ1sKlA2OEsYq0ZlQKyBqFN0oYkq/5nQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bt612f1ma-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:20:01 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60MLJPSt031579; Thu, 22 Jan 2026 21:20:01 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bt612f1m6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:20:01 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60ML8q7c027233; Thu, 22 Jan 2026 21:20:00 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4brnrncy3b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:20:00 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60MLJuLT61669880 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jan 2026 21:19:56 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 312A32004B; Thu, 22 Jan 2026 21:19:56 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6406120040; Thu, 22 Jan 2026 21:19:49 +0000 (GMT) Received: from abhi.. (unknown [9.124.222.159]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 22 Jan 2026 21:19:49 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org Subject: [PATCH v4 4/6] powerpc64/bpf: Add arch_bpf_stack_walk() for BPF JIT Date: Fri, 23 Jan 2026 02:48:52 +0530 Message-ID: <20260122211854.5508-5-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260122211854.5508-1-adubey@linux.ibm.com> References: <20260122211854.5508-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Bi1LIpXrrGVye6rfzAQkPQovYN59Ekfz X-Authority-Analysis: v=2.4 cv=LaIxKzfi c=1 sm=1 tr=0 ts=69729481 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=ovcAVf-9gG4EF5hqTukA:9 X-Proofpoint-ORIG-GUID: BWYecn5M-kVPH1btpTP8sK0d8HdbdFG4 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTIyMDE2MiBTYWx0ZWRfX2YhzRuJhLt/I eK5YHBqr1K76OrTkancm/KzY8JobRFCIAG2GZIZma4pigrSKQMMaRcWRIZ+ogDcc+tlH/CiRAzQ /J41SqKdvOQ8JgrGk/umy8YEB1Tg3RNC163EyWLw1Kj0qUrgft3ZS7wAAyvYYLPHKdn2xIcYltX V1/+kP6Eorgy8Frp/a3whSjC+g7Cv5u5W+Wx7ebStVxYACvwe2n9Dzud1ow45PefvmPqEykE9OA tVUZ0LJmrwWtlVv3KY9FLOppMDIzGfZ55RnSXuZahrhZTuuqLcXDvK1leu2oOfjchZbTTkgi/BU MR8N05Zuj7IKxvhwHy1I32YBt4bG4CkHs6yGNwvQ/CicRTJP6HjHoGTtFHoupd+7TIpVDdlokEr UHzGobbJTVv138CugPrhbk6LlCBKxz+61gap0IO1SMbxUqsLDTUbdK+U+nkECIiTGuUt0BNVyGV H8BGWO9eKYRmLT6PsVg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-22_05,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 adultscore=0 phishscore=0 priorityscore=1501 lowpriorityscore=0 suspectscore=0 clxscore=1015 impostorscore=0 spamscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601220162 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey This function is used by bpf_throw() to unwind the stack until frame of exception-boundary during BPF exception handling. This function is necessary to support BPF exceptions on PowerPC. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp64.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 18da5a866447..c25ba1ad587a 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -247,6 +247,34 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen= _context *ctx) bpf_jit_build_fentry_stubs(image, ctx); } =20 +void arch_bpf_stack_walk(bool (*consume_fn)(void *, u64, u64, u64), void *= cookie) +{ + // callback processing always in current context + unsigned long fp =3D current_stack_frame(); + + for (;;) { + unsigned long *frame =3D (unsigned long *) fp; + unsigned long ip; + + if (!validate_sp(fp, current)) + return; + + ip =3D frame[STACK_FRAME_LR_SAVE]; + if (!ip) + break; + + /* + * consume_fn common code expects stack pointer(sp) in third + * argument. There is no sp in ppc64, rather pass frame + * pointer. + */ + if (ip && !consume_fn(cookie, ip, fp, fp)) + break; + + fp =3D frame[0]; + } +} + int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_con= text *ctx, u64 func) { unsigned long func_addr =3D func ? ppc_function_entry((void *)func) : 0; --=20 2.48.1 From nobody Sun Feb 8 05:59:54 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95FE72EA743; Thu, 22 Jan 2026 21:21:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116899; cv=none; b=uKC3QaSSmROR76/R65+R8pvPu/n5QDTpUrTkk8rvherLoQtTmopgJnUbKU7q3IekH3WT6QExh7wlheFeefQF6dhDd7OySzWpOluADGIH3H+gw6f5QYfxA12eGGQlHnlt2sL40TWfBZEoiHenCU85ZywQJwDrYBt6MC6xhGwPUZ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116899; c=relaxed/simple; bh=FLrGt7/H8RsZJUbH+j7GhPRlDassB/VgNd1J0Mv6p2c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=f7Jk3NrpP74RC5ujWlu+6A/mETXE9Jl85AW2jK/WhMMhIpAtgTae8A4ykHPTjD5c3njMc4qX/POV8QLH+rdo9ldsykFFHn5K+eDsM65CQ5J57z67NIYu1G787vjkSL6ZRKfuS9DkDBpn2tYO3r7rz6CZTWihdgSVnqDrQt4mIP0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=abuErt/Y; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="abuErt/Y" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60MEI3h5007031; Thu, 22 Jan 2026 21:20:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=2aqmAFlIDX6Vm3pGI zlQzZebzSwq7r1k6HwxWg2oJvg=; b=abuErt/YqhJbfR6mAO8DyLVUkS+FoE35l WZ/7hcd6kU4Atx9QenJZGV2bqlj8aBU/opvxzzQCFjyrQrJ8yEWkbJ8F43GNi89L 4g+vEY0OYRjeaKGA/DhnNdSPKHJSlXinXZa8lrp7XfptYfTF07tIl1mguQA7nnc+ 52gYFnF4PxYGJcza/Jldgn5aQ5bW9bwM0fWfEqCgvlORvd0G2xpzBsTUsyfRBgSs dj5sA2YzA7bCL5RnZJAV5oGiqcku99i1DkVUxEGdCF4/Porg8rcpnBoIS3jZ5ib9 tUqxk3qtzMtX9dmA4AnyR/qd6xmzSw0Cvuz5NZLO4TmGZK3MZ+ipQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4br23sc92f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:20:15 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60MLKFRj019799; Thu, 22 Jan 2026 21:20:15 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4br23sc92c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:20:15 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60ML7XoG027334; Thu, 22 Jan 2026 21:20:13 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4brnrncy4p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:20:13 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60MLK9vv50135372 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jan 2026 21:20:09 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9BDAA20043; Thu, 22 Jan 2026 21:20:09 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B0AAE20040; Thu, 22 Jan 2026 21:20:02 +0000 (GMT) Received: from abhi.. (unknown [9.124.222.159]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 22 Jan 2026 21:20:02 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org Subject: [PATCH v4 5/6] powerpc64/bpf: Support exceptions Date: Fri, 23 Jan 2026 02:48:53 +0530 Message-ID: <20260122211854.5508-6-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260122211854.5508-1-adubey@linux.ibm.com> References: <20260122211854.5508-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: fB_JUKWn7jaJvHv9t3cJaviFoNxob3IK X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTIyMDE2MiBTYWx0ZWRfX9TCu8XKoVDhb Nou+G9uM6dV7GpLjVsQP0H2x8/NqZRgbn8EP4rS2bzI0aat7ztjg8+c/FoNMYRvBsI6nEl42a4d HX/5Kb+aQhJeuuz2qWcKPqH8a9kjPDfFlkwGgXGbOPTfqMNOx4aic2SwHL1rJCKpoBNFQ5niAWn JRoSiP2FxN7tRg/ZHIPF5M3MEkzBdbWdP5kJUOkQKh6kP6pqlv3ZrnCYNm897KNuiZXQ3DT2Z+i 8sWxzR5fizaYEVGeJnNPcEIz8EWg2uNoj1QFWi0CAsLL2Wv8xbbnfPgb37d6I+SP/XttIEdCSpj /dsBajzE6jn0TVZT9XTb//pXV8f1O1gljFxPS9ofcX5xK2bLyX/jI+i8J8queHvNvh3N50vzFdq u7pdJAbmeImZ9eojdsye9IZXL80ybk8b5ABoK4PfMzUHfM4knT1W5pp6FZuyIxguh+tpMxAPzN3 p/hdie0anNKYcrGOoUw== X-Authority-Analysis: v=2.4 cv=J9SnLQnS c=1 sm=1 tr=0 ts=6972948f cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=DU0EyFXtAAAA:8 a=VnNF1IyMAAAA:8 a=xcpHmmOOOUdXZ9sPVCMA:9 a=UCR5be5CC-YrbG9FbbB0:22 X-Proofpoint-ORIG-GUID: t-e6rztzxNzvTrLnK7oFJBPZfU5OD4Lt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-22_05,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 impostorscore=0 adultscore=0 suspectscore=0 spamscore=0 lowpriorityscore=0 malwarescore=0 clxscore=1015 bulkscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601220162 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The modified prologue/epilogue generation code now enables exception-callback to use the stack frame of the program marked as exception boundary, where callee saved registers are stored. As per ppc64 ABIv2 documentation[1], r14-r31 are callee saved registers. BPF programs on ppc64 already saves r26-r31 registers. Saving the remaining set of callee saved registers(r14-r25) is handled in the next patch. [1] https://ftp.rtems.org/pub/rtems/people/sebh/ABI64BitOpenPOWERv1.1_16Jul= y2015_pub.pdf Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit.h | 2 ++ arch/powerpc/net/bpf_jit_comp.c | 7 ++++ arch/powerpc/net/bpf_jit_comp64.c | 58 +++++++++++++++++++++---------- 3 files changed, 48 insertions(+), 19 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 56f56fdd4969..82bbf63f0e57 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -179,6 +179,8 @@ struct codegen_context { u64 arena_vm_start; u64 user_vm_start; bool is_subprog; + bool exception_boundary; + bool exception_cb; }; =20 #define bpf_to_ppc(r) (ctx->b2p[r]) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 1a305f0fed27..2607ea0bedef 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -207,6 +207,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *f= p) cgctx.arena_vm_start =3D bpf_arena_get_kern_vm_start(fp->aux->arena); cgctx.user_vm_start =3D bpf_arena_get_user_vm_start(fp->aux->arena); cgctx.is_subprog =3D bpf_is_subprog(fp); + cgctx.exception_boundary =3D fp->aux->exception_boundary; + cgctx.exception_cb =3D fp->aux->exception_cb; =20 /* Scouting faux-generate pass 0 */ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) { @@ -436,6 +438,11 @@ void bpf_jit_free(struct bpf_prog *fp) bpf_prog_unlock_free(fp); } =20 +bool bpf_jit_supports_exceptions(void) +{ + return IS_ENABLED(CONFIG_PPC64); +} + bool bpf_jit_supports_subprog_tailcalls(void) { return IS_ENABLED(CONFIG_PPC64); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index c25ba1ad587a..d7cd8ab6559c 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -89,7 +89,9 @@ static inline bool bpf_has_stack_frame(struct codegen_con= text *ctx) * - the bpf program uses its stack area * The latter condition is deduced from the usage of BPF_REG_FP */ - return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, bpf_to_ppc(BPF_= REG_FP)); + return ctx->seen & SEEN_FUNC || + bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)) || + ctx->exception_cb; } =20 /* @@ -161,8 +163,13 @@ void bpf_jit_build_prologue(u32 *image, struct codegen= _context *ctx) EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0)); /* this goes in the redzone */ EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); - } else { + } else if (!ctx->exception_cb) { /* + * Tailcall jitting for non exception_cb progs only. + * exception_cb won't require tail_call_info to be setup. + * + * tail_call_info interpretation logic: + * * if tail_call_info < MAX_TAIL_CALL_CNT * main prog calling first subprog -> copy reference * else @@ -177,8 +184,12 @@ void bpf_jit_build_prologue(u32 *image, struct codegen= _context *ctx) EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } =20 - if (bpf_has_stack_frame(ctx)) { + if (bpf_has_stack_frame(ctx) && !ctx->exception_cb) { /* + * exception_cb uses boundary frame after stack walk. + * It can simply use redzone, this optimization reduces + * stack walk loop by one level. + * * We need a stack frame, but we don't necessarily need to * save/restore LR unless we call other functions */ @@ -190,23 +201,32 @@ void bpf_jit_build_prologue(u32 *image, struct codege= n_context *ctx) EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size))); } =20 - /* - * Back up non-volatile regs -- BPF registers 6-10 - * If we haven't created our own stack frame, we save these - * in the protected zone below the previous stack frame - */ - for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) - if (bpf_is_seen_register(ctx, bpf_to_ppc(i))) - EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to= _ppc(i)))); + if (!ctx->exception_cb) { + /* + * Back up non-volatile regs -- BPF registers 6-10 + * If we haven't created our own stack frame, we save these + * in the protected zone below the previous stack frame + */ + for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) + if (ctx->exception_boundary || bpf_is_seen_register(ctx, bpf_to_ppc(i))) + EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, + bpf_jit_stack_offsetof(ctx, bpf_to_ppc(i)))); =20 - if (ctx->arena_vm_start) - EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1, + if (ctx->exception_boundary || ctx->arena_vm_start) + EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 - /* Setup frame pointer to point to the bpf stack area */ - if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP))) - EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1, + /* Setup frame pointer to point to the bpf stack area */ + if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP))) + EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1, STACK_FRAME_MIN_SIZE + ctx->stack_size)); + } else { + /* + * Exception callback receives Frame Pointer of main + * program as third arg + */ + EMIT(PPC_RAW_MR(_R1, _R5)); + } =20 if (ctx->arena_vm_start) PPC_LI64(bpf_to_ppc(ARENA_VM_START), ctx->arena_vm_start); @@ -218,17 +238,17 @@ static void bpf_jit_emit_common_epilogue(u32 *image, = struct codegen_context *ctx =20 /* Restore NVRs */ for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) - if (bpf_is_seen_register(ctx, bpf_to_ppc(i))) + if (ctx->exception_cb || bpf_is_seen_register(ctx, bpf_to_ppc(i))) EMIT(PPC_RAW_LD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_= ppc(i)))); =20 - if (ctx->arena_vm_start) + if (ctx->exception_cb || ctx->arena_vm_start) EMIT(PPC_RAW_LD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 /* Tear down our stack frame */ if (bpf_has_stack_frame(ctx)) { EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size)); - if (ctx->seen & SEEN_FUNC) { + if (ctx->seen & SEEN_FUNC || ctx->exception_cb) { EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF)); EMIT(PPC_RAW_MTLR(_R0)); } --=20 2.48.1 From nobody Sun Feb 8 05:59:54 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA0F83BE4A7; Thu, 22 Jan 2026 21:22:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116945; cv=none; b=DJfroC6AMNGeDCqxHJrD7HWCsmUv1pstZqQf8FrLt+vmbs+z6nksJIeIFHftQPU1QdZhWYpORnGoLWXbjB60ISfOgH/ax3ExXzYqW687bnBU8I0nhKOdomfI/KwszgQvobK/p6x/cMGIC2ShuX9Rx/4gWh+0XfLCWNQKhO53u0w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769116945; c=relaxed/simple; bh=CNS2Z8bc8NyCrEiQNl7W6YGktA1/CnW0zWVYs1cpeqY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TZnJd3C8Hs5+FoZpzd5jmJUfS39aSyJUAJlIB9LITR6ULNVfEGvgk1husHEzzq317o4eqTN4CtVVSdUNfgRavIQOhnmhhyF6bYazQwtYGFMqxFILv8QpDkq63piWMvFncyT9ZId6NsUCgvtsTKpyc6GoOymZNY1RSDVjY8UpOV8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=A5ERINqP; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="A5ERINqP" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60MCo28c028543; Thu, 22 Jan 2026 21:20:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=8PUggZjcaHIEZpeT3 Xm7CvXtCNJbJJE4E5IaGpaIdxM=; b=A5ERINqPLJMFJ3BtCRM5tW5y0my/PU0tY WrdNc4HmVNax5GQtzeOOi0+6vMZWbYmsJyE3ML5Fi3+dgOsDIjZIx1Y+RzSf+o+4 R8b6LnLDd8h1Q1gMm1kmeYx56kAcNUwx8gz6LmmmwkuWAXrH2Bk7Sx0ZKLP0YpbD /rvwNh4n8UFkOWVQaFlt/7g8W4gw2nOp5QQGW7+uVngrvRs1l4Am249pz5yrcOO5 3V6KVQUju71gme/s5Mw9zfzrLmeuwaD9RTlT6LAxLxjVBs6TeEc6zvczR+AKcAq9 fY7TXEq09TPiV5rYbf7UGVVXsoG35w+N9tj1XAGNlFn3pkjdf2TvA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bt60eys4j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:20:31 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60MLKUZb025862; Thu, 22 Jan 2026 21:20:30 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bt60eys4d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:20:30 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60MJ0H7V009265; Thu, 22 Jan 2026 21:20:29 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4brp8kmwbv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 21:20:29 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60MLKPIg36438284 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jan 2026 21:20:25 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E806420043; Thu, 22 Jan 2026 21:20:24 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DDD4E20040; Thu, 22 Jan 2026 21:20:17 +0000 (GMT) Received: from abhi.. (unknown [9.124.222.159]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 22 Jan 2026 21:20:17 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org Subject: [PATCH v4 6/6] powerpc64/bpf: Additional NVR handling for bpf_throw Date: Fri, 23 Jan 2026 02:48:54 +0530 Message-ID: <20260122211854.5508-7-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260122211854.5508-1-adubey@linux.ibm.com> References: <20260122211854.5508-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=WMdyn3sR c=1 sm=1 tr=0 ts=6972949f cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=2yuXeg4sgMS8061G0UsA:9 X-Proofpoint-GUID: _Lx3ftOdMDZ15M_8Hk44IxMJ4xH2OnF9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTIyMDE2MiBTYWx0ZWRfXxPpzeltT1tSM Zc4JbfEdsKK5mD2llE/AVliQXeHnEYraJWbJP0SQq13gj6ACMP2HkM0kQvGZxn9a76UQRfsZ5Rn 8+/vujiR7fVIF7a1EAVaKJZz3YmQ/D/fvYq7q5Hrt4VUyxAY9S/t4YAYas5QQ0HmjLULohd8drX WH0JfZCxRrBS2WG/QI9OxlruuCZLt4X9J768O8aRqurueGXccfqQYhxm/ZiLxG7xmOQPyE0oLzI 774lIxWch8dGK3fAY6J3Sfb/WILsKGDRwTOYPF4cYRcfspyAM44YaNz5l5JjhFYwrjlg5NMURAi +FS9ddk5+NX+Pc6Z413BpWIABM1GZjQV6UbcNRtmkukphf/B6JUXgX2xuDq2o79Jj0yEEtV5P3s cDgGLR5MGX2JzllqVHGuVHQB5Xe83TlzXUAGGf/oYb972N2h3mUhpyB8kx4fcvC/69QZ4YnZL7m hD6yR+k0Nb5kanpvhjA== X-Proofpoint-ORIG-GUID: WiNFPQIcmRUQosg3U2XlDqFLDYNf107f X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-22_05,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 suspectscore=0 bulkscore=0 adultscore=0 impostorscore=0 spamscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601220162 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The bpf_throw() function never returns, if it has clobbered any callee-saved register, those will remain clobbered. The prologue must take care of saving all callee-saved registers in the frame of exception boundary program. Later these additional non volatile registers R14-R25 along with other NVRs are restored back in the epilogue of exception callback. To achieve above objective, the frame size is determined dynamically to accommodate additional non volatile registers in exception boundary's frame. For non-exception boundary program, the frame size remains optimal. The additional instructions to save & restore r14-r25 registers are emitted only during exception boundary and exception callback program respectively. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp64.c | 81 ++++++++++++++++++++++++++++--- 1 file changed, 74 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index d7cd8ab6559c..df07375d1595 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -32,21 +32,37 @@ * * [ prev sp ] <------------- * [ tail_call_info ] 8 | - * [ nv gpr save area ] 6*8 | + * [ nv gpr save area ] 6*8 + (12*8) | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | * [ frame header ] 32/112 | * sp (r1) ---> [ stack pointer ] -------------- + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ =20 /* for bpf JIT code internal usage */ #define BPF_PPC_STACK_LOCALS 24 +/* + * for additional non volatile registers(r14-r25) to be saved + * at exception boundary + */ +#define BPF_PPC_EXC_STACK_SAVE (12*8) + /* stack frame excluding BPF stack, ensure this is quadword aligned */ #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ BPF_PPC_STACK_LOCALS + \ BPF_PPC_STACK_SAVE + \ BPF_PPC_TAILCALL) =20 +/* + * same as BPF_PPC_STACKFRAME with save area for additional + * non volatile registers saved at exception boundary. + * This is quad-word aligned. + */ +#define BPF_PPC_EXC_STACKFRAME (BPF_PPC_STACKFRAME + BPF_PPC_EXC_STACK_SAV= E) + /* BPF register usage */ #define TMP_REG_1 (MAX_BPF_JIT_REG + 0) #define TMP_REG_2 (MAX_BPF_JIT_REG + 1) @@ -103,9 +119,12 @@ static inline bool bpf_has_stack_frame(struct codegen_= context *ctx) * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- * [ tail_call_info ] 8 - * [ nv gpr save area ] 6*8 + * [ nv gpr save area ] 6*8 + (12*8) * [ local_tmp_var ] 24 * [ unused red zone ] 224 + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ static int bpf_jit_stack_local(struct codegen_context *ctx) { @@ -114,7 +133,12 @@ static int bpf_jit_stack_local(struct codegen_context = *ctx) return STACK_FRAME_MIN_SIZE + ctx->stack_size; } else { /* Stack layout with redzone */ - return -(BPF_PPC_TAILCALL + BPF_PPC_STACK_SAVE + BPF_PPC_STACK_LOCALS); + return -(BPF_PPC_TAILCALL + +BPF_PPC_STACK_SAVE + +(ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACK_SAVE : 0) + +BPF_PPC_STACK_LOCALS + ); } } =20 @@ -125,9 +149,19 @@ int bpf_jit_stack_tailcallinfo_offset(struct codegen_c= ontext *ctx) =20 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg) { - if (reg >=3D BPF_PPC_NVR_MIN && reg < 32) + int min_valid_nvreg =3D BPF_PPC_NVR_MIN; + /* Default frame size for all cases except exception boundary */ + int frame_nvr_size =3D BPF_PPC_STACKFRAME; + + /* Consider all nv regs for handling exceptions */ + if (ctx->exception_boundary || ctx->exception_cb) { + min_valid_nvreg =3D _R14; + frame_nvr_size =3D BPF_PPC_EXC_STACKFRAME; + } + + if (reg >=3D min_valid_nvreg && reg < 32) return (bpf_has_stack_frame(ctx) ? - (BPF_PPC_STACKFRAME + ctx->stack_size) : 0) + (frame_nvr_size + ctx->stack_size) : 0) - (8 * (32 - reg)) - BPF_PPC_TAILCALL; =20 pr_err("BPF JIT is asking about unknown registers"); @@ -138,6 +172,17 @@ void bpf_jit_realloc_regs(struct codegen_context *ctx) { } =20 +/* + * For exception boundary & exception_cb progs: + * return increased size to accommodate additional NVRs. + */ +static int bpf_jit_stack_size(struct codegen_context *ctx) +{ + return ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACKFRAME : + BPF_PPC_STACKFRAME; +} + void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx) { int i; @@ -198,7 +243,19 @@ void bpf_jit_build_prologue(u32 *image, struct codegen= _context *ctx) EMIT(PPC_RAW_STD(_R0, _R1, PPC_LR_STKOFF)); } =20 - EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size))); + EMIT(PPC_RAW_STDU(_R1, _R1, + -(bpf_jit_stack_size(ctx) + ctx->stack_size))); + } + + /* + * Program acting as exception boundary pushes R14..R25 in addition to + * BPF callee-saved non volatile registers. Exception callback uses + * the boundary program's stack frame, recover additionally saved + * registers in epilogue of exception callback. + */ + if (ctx->exception_boundary) { + for (i =3D _R14; i <=3D _R25; i++) + EMIT(PPC_RAW_STD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); } =20 if (!ctx->exception_cb) { @@ -245,9 +302,19 @@ static void bpf_jit_emit_common_epilogue(u32 *image, s= truct codegen_context *ctx EMIT(PPC_RAW_LD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 + if (ctx->exception_cb) { + /* + * Recover additionally saved non volatile registers from stack + * frame of exception boundary program. + */ + for (i =3D _R14; i <=3D _R25; i++) + EMIT(PPC_RAW_LD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); + } + /* Tear down our stack frame */ if (bpf_has_stack_frame(ctx)) { - EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size)); + EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_jit_stack_size(ctx) + ctx->stack_size)); + if (ctx->seen & SEEN_FUNC || ctx->exception_cb) { EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF)); EMIT(PPC_RAW_MTLR(_R0)); --=20 2.48.1