From nobody Sat Feb 7 21:15:19 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 654895478D; Sat, 24 Jan 2026 07:54:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241266; cv=none; b=JwA11a+3cnoBza9vHDaZHu0uPG3I0T4ftiVwzuEzaAbxTDJhpsdI+5dwLvO1chVBoXN0Y/dx2owSyU7MmZhUKdeQzLbHVKi+89YvdZ8+uvMQjae5NxLxfUY3J4GvBIB7+sD1GEDYYj77CYmUkHFM6Hd1WZsGObF8PlLRGn+eo6I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241266; c=relaxed/simple; bh=F2nbp4pKojZHx/h0Qn12bcgpLGoIcA3okIqyKVp7ybs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HFX7y0qS8ijZ+chzyy3/c2Z48ThSeXi+i/xHty+LLIbA49v9pfqzAVx1UEEN7ugAWG6k1Ui/Rs11EY3r4gG0EZZ3QO6sjYScbpuqE+gazaDtuha260XAxgfOUECfwNjKlbDQ4O4MzhaHMvl7INhSY9/Tpfbuu8zetLiTZUEahRU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=iZODewMP; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="iZODewMP" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60O4GdUq001887; Sat, 24 Jan 2026 07:52:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ZQvpxZmGfyMOLJXKl M5mxfJFZy3MusBPwW32+h9PHUk=; b=iZODewMPvE1+mEcEDI7Zj24ohOCg+n+uX Ee/nVLUTsDqNdAvXScZFLwT8wsUPdKk/MNdxG2aJ+cICd6E4CpEQfFOt5xigrLwo MSAxVENnCnBYJFW9sUVdkealTl0V7J+2NC1Ijwz+SCVDWwHHeuadtp0vYNW5HZEU 9ZJgNDxZTK/8dWLX9B+Hg95tb4RStFCT0S17XdhbEcGcU1mRzP4f4jO612pDYyd9 5aTLbuC/rPEvvqDz+hW4Z/sqkn0Uu8AVd2w5Azlq39aV+JV0+Kzgc7MIjKyyw1Qm DnAGbqKX2fTB6yO6EoH2Fb127/agSmpGb9Q4jvAOtVtoJzattGCrQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnrt0qc5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:52:53 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60O7qqW4000441; Sat, 24 Jan 2026 07:52:52 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnrt0qc3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:52:52 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60O2j3U5001168; Sat, 24 Jan 2026 07:52:51 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4brpykcr6j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:52:51 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60O7qmCt28836464 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Jan 2026 07:52:48 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E31DD20043; Sat, 24 Jan 2026 07:52:47 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 53E6520040; Sat, 24 Jan 2026 07:52:41 +0000 (GMT) Received: from abhi.. (unknown [9.124.223.59]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 24 Jan 2026 07:52:41 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v5 1/6] powerpc64/bpf: Moving tail_call_cnt to bottom of frame Date: Sat, 24 Jan 2026 13:22:18 +0530 Message-ID: <20260124075223.6033-2-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260124075223.6033-1-adubey@linux.ibm.com> References: <20260124075223.6033-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: nviSkfJrbQ322s3xSIEJIbDqwWDFVxuT X-Authority-Analysis: v=2.4 cv=Uptu9uwB c=1 sm=1 tr=0 ts=69747a55 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=CgeYeB0JMTkxdtrR0HIA:9 X-Proofpoint-ORIG-GUID: LmBQxAACTNRXwkePuXyfuLglS-jFqN_P X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI0MDA2MCBTYWx0ZWRfX+0fHpFAdGe/T +scmTfvhsW7uyQhWpneI58+Q+E4FlFpbBTU2Plq6VzbrfMrMCrj9ONa6+Lhv7H2zWFAFDDfI+b9 5O3YmwLuR4Kz3zXkZSbVbnNiXiKE5UhXE2k+RSC9nkKvhTJvLhQT/fspxi9dvtrg03awtJ26goz A5uwz2Fqb5gOn0TpWwVCH7rSSBZg9bTu6Y7hWh5UCYHklLcAZhjKWQPPtwz574BuGNZL4DioYb6 MV1E3P9mc81CX+4wHyZm7T0rgJQNVzy/LDZaWVIl1txZi9jRrYYN0E8BK62TKpmnBN+4tBQqY2w of1g+lY19426eO2VNf7ff2T+7w+9LV8XOe6IKmhkOSrK3V150W9eYnTyNXfQ/ecVRg3JygsPV+j puhCCTUG/nplnkyaXcQXb76+5a7rPHiQRVmXVk94Ajb6gkYR94y470sphMm48IV/0b6yQoF6LM2 i/r0bH/jDaP6w8K5fFw== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-24_02,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 priorityscore=1501 clxscore=1015 lowpriorityscore=0 phishscore=0 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601240060 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey To support tailcalls in subprogs, tail_call_cnt needs to be on the BPF trampoline stack frame. In a regular BPF program or subprog stack frame, the position of tail_call_cnt is after the NVR save area (BPF_PPC_STACK_SAVE). To avoid complex logic in deducing offset for tail_call_cnt, it has to be kept at the same offset on the trampoline frame as well. But doing that wastes nearly all of BPF_PPC_STACK_SAVE bytes on the BPF trampoline stack frame as the NVR save area is not the same for BPF trampoline and regular BPF programs. Address this by moving tail_call_cnt to the bottom of the frame. This change avoids the need to account for BPF_PPC_STACK_SAVE bytes in the BPF trampoline stack frame when support for tailcalls in BPF subprogs is added later. Also, this change makes offset calculation of tail_call_cnt field simpler all across. Signed-off-by: Abhishek Dubey Tested-by: Venkat Rao Bagalkote --- arch/powerpc/net/bpf_jit.h | 1 + arch/powerpc/net/bpf_jit_comp.c | 6 +++--- arch/powerpc/net/bpf_jit_comp64.c | 31 ++++++++++++++++++++----------- 3 files changed, 24 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 8334cd667bba..9f6ec00bd02e 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -24,6 +24,7 @@ =20 #define SZL sizeof(unsigned long) #define BPF_INSN_SAFETY 64 +#define BPF_PPC_TAILCALL 8 =20 #define PLANT_INSTR(d, idx, instr) \ do { if (d) { (d)[idx] =3D instr; } idx++; } while (0) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 5e976730b2f5..de1adc0b1157 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -604,8 +604,8 @@ static void bpf_trampoline_setup_tail_call_cnt(u32 *ima= ge, struct codegen_contex int func_frame_offset, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { - /* See bpf_jit_stack_tailcallcnt() */ - int tailcallcnt_offset =3D 7 * 8; + /* See Generated stack layout */ + int tailcallcnt_offset =3D BPF_PPC_TAILCALL; =20 EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); EMIT(PPC_RAW_STL(_R3, _R1, -tailcallcnt_offset)); @@ -620,7 +620,7 @@ static void bpf_trampoline_restore_tail_call_cnt(u32 *i= mage, struct codegen_cont { if (IS_ENABLED(CONFIG_PPC64)) { /* See bpf_jit_stack_tailcallcnt() */ - int tailcallcnt_offset =3D 7 * 8; + int tailcallcnt_offset =3D BPF_PPC_TAILCALL; =20 EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset)); EMIT(PPC_RAW_STL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 1fe37128c876..296e9ea14f2e 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -20,13 +20,15 @@ #include "bpf_jit.h" =20 /* - * Stack layout: + * Stack layout with frame: + * Layout when setting up our own stack frame. + * Note: r1 at bottom, component offsets positive wrt r1. * Ensure the top half (upto local_tmp_var) stays consistent * with our redzone usage. * * [ prev sp ] <------------- - * [ nv gpr save area ] 6*8 | * [ tail_call_cnt ] 8 | + * [ nv gpr save area ] 6*8 | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | * [ frame header ] 32/112 | @@ -36,10 +38,12 @@ /* for gpr non volatile registers BPG_REG_6 to 10 */ #define BPF_PPC_STACK_SAVE (6*8) /* for bpf JIT code internal usage */ -#define BPF_PPC_STACK_LOCALS 32 +#define BPF_PPC_STACK_LOCALS 24 /* stack frame excluding BPF stack, ensure this is quadword aligned */ #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ - BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SAVE) + BPF_PPC_STACK_LOCALS + \ + BPF_PPC_STACK_SAVE + \ + BPF_PPC_TAILCALL) =20 /* BPF register usage */ #define TMP_REG_1 (MAX_BPF_JIT_REG + 0) @@ -87,27 +91,32 @@ static inline bool bpf_has_stack_frame(struct codegen_c= ontext *ctx) } =20 /* + * Stack layout with redzone: * When not setting up our own stackframe, the redzone (288 bytes) usage i= s: + * Note: r1 from prev frame. Component offset negative wrt r1. * * [ prev sp ] <------------- * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- - * [ nv gpr save area ] 6*8 * [ tail_call_cnt ] 8 + * [ nv gpr save area ] 6*8 * [ local_tmp_var ] 24 * [ unused red zone ] 224 */ static int bpf_jit_stack_local(struct codegen_context *ctx) { - if (bpf_has_stack_frame(ctx)) + if (bpf_has_stack_frame(ctx)) { + /* Stack layout with frame */ return STACK_FRAME_MIN_SIZE + ctx->stack_size; - else - return -(BPF_PPC_STACK_SAVE + 32); + } else { + /* Stack layout with redzone */ + return -(BPF_PPC_TAILCALL + BPF_PPC_STACK_SAVE + BPF_PPC_STACK_LOCALS); + } } =20 static int bpf_jit_stack_tailcallcnt(struct codegen_context *ctx) { - return bpf_jit_stack_local(ctx) + 24; + return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SA= VE; } =20 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg) @@ -115,7 +124,7 @@ static int bpf_jit_stack_offsetof(struct codegen_contex= t *ctx, int reg) if (reg >=3D BPF_PPC_NVR_MIN && reg < 32) return (bpf_has_stack_frame(ctx) ? (BPF_PPC_STACKFRAME + ctx->stack_size) : 0) - - (8 * (32 - reg)); + - (8 * (32 - reg)) - BPF_PPC_TAILCALL; =20 pr_err("BPF JIT is asking about unknown registers"); BUG(); @@ -145,7 +154,7 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_= context *ctx) if (ctx->seen & SEEN_TAILCALL) { EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0)); /* this goes in the redzone */ - EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_STACK_SAVE + 8))); + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } else { EMIT(PPC_RAW_NOP()); EMIT(PPC_RAW_NOP()); --=20 2.48.1 From nobody Sat Feb 7 21:15:19 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65684288C3D; Sat, 24 Jan 2026 07:53:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241220; cv=none; b=KU8sv6aat/AENUz3B3vh1XaNjDiRXXJ5E43Lx/qLAqdWGevuGZgW+Mx3qOuLuy6YMpvfv3qj+FAx/hDJnpysTTacwQeC8nexIQ3jWTyX2BlEKFC3tyEfvfU7QI+PJwQ+wVXS3aq8A9wIQjm7cW8U9RroN/pbHM0cOuitbnAl9K0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241220; c=relaxed/simple; bh=JvKmuxPPrceNU/ahe49AfglNWXW/vxS+I/ptHFPhR/Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oHrKbQLUAvk/+YfS86Q+qQSqygROuaG75ZB2CCD/y7OEs6LrgUgv2HtTCl6FScUbP7S85JE9vsOSeEk4r+LlVsUeFQv/B9XwzDXQFe7Sp7fU0xK74xJRiG9OArookS99wUssO8w+8KeJe2snhtFmlZeWrhwVsjsb7Kc/sqIYin8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=STdSqVsS; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="STdSqVsS" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60O7mBQQ027202; Sat, 24 Jan 2026 07:53:02 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=cqhK3eE5u6cz/6sB9 gU2oMvs86G2G2g2ijzWa+XLki4=; b=STdSqVsSbr5hJ4RlFkV5LTr3LUxoUHuor J6O8jFZuVIQ98BVR0M+SOGdrmWHw0TB5EyLAbtcnQdXQm+yN6NtqXiWvwrOYB7IG xBu+7Hh5k4pMasMYee/cddoRS3io2LaXw1Cau534WNJN3FIamZvFCImSyCBOH23K qMr22HwaaOzxwhFY+vkmDqTruHzKjNW5HjAk6EnUX4zN9WqjFa6ZOgwdM4wCudqm ZYzRFX3GmZh2wN3ZRINGf3hCkikq1pKC8CfmXZ1FAYSVfjXmnmXY+x6zDTalQsbY qzAO4inx+cICSKmi5Gns6gJDA/xyMQVhAcyAQC7hlto7Kk6SO5fig== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnrt0qcd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:01 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60O7r1Va000916; Sat, 24 Jan 2026 07:53:01 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnrt0qcb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:01 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60O4D3Dq006427; Sat, 24 Jan 2026 07:53:00 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4brqf24n1k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:00 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60O7qu1J41222534 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Jan 2026 07:52:56 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1F90920043; Sat, 24 Jan 2026 07:52:56 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8B1C420040; Sat, 24 Jan 2026 07:52:49 +0000 (GMT) Received: from abhi.. (unknown [9.124.223.59]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 24 Jan 2026 07:52:49 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v5 2/6] powerpc64/bpf: Support tailcalls with subprogs Date: Sat, 24 Jan 2026 13:22:19 +0530 Message-ID: <20260124075223.6033-3-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260124075223.6033-1-adubey@linux.ibm.com> References: <20260124075223.6033-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: D5kswbOd7w7mYhxRyknAsqGIWqc4166s X-Authority-Analysis: v=2.4 cv=Uptu9uwB c=1 sm=1 tr=0 ts=69747a5d cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=aLVHvtoiTwlSXXnS9I8A:9 X-Proofpoint-ORIG-GUID: UY6qxMp2h8366d0qLIuqp8FhIPzkCZAr X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI0MDA2MCBTYWx0ZWRfX4wdD/H1X3ZmQ +siSjx5igiL6z7KzjO8iNisXPsgDjRihEJJRpzFvHl2CF+PSdwHMXL5BpG+Q1/uUUqDwgBOps5J Zzw5keWOlcDYKVEyjfK1XkkkP7XHp9ZY0r45Qc1VU65joyTs9bn6e7CcHI8dG3Ap4u7A1QIMhP4 JZfKgm8ceviCOF6BWjERLUAWFDUexIHZW4kdCLvrxR0+S5Bu1cSZNnRLNj1NTPDSaevTQqNwtFh YZMpLdx9TQUMJC0ENjZ487jYmeM7KqDOD1fCaB3k2ajLw24DhEEbln+IvbW28N45daUp62gWVy2 c2EVnjhnQ6XhtrfhP+5L7yDPHRW1xAATJ3AsvxRRDjYDxuP84Ox/00ntF5naZzQMXzeTOTv2Q91 AIerA+L0/yQb9pRkxOSil94n8uHmE8VTsaSxzDk5EQ4LS54uKxN7VGnzR1844Wez3bOmIHAeLQI YLudq3smpuZuD7hzkRA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-24_02,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 priorityscore=1501 clxscore=1015 lowpriorityscore=0 phishscore=0 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601240060 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey Enable tailcalls support in subprogs by passing tail call count as reference instead of value. The actual tailcall count is always maintained in the tailcall field present in the frame of main function (also called entry function). The tailcall field in the stack frame of subprogs contains reference to the tailcall field in the stack frame of main BPF program. Accordingly, rename tail_call_cnt field in the stack layout to tail_call_info. Signed-off-by: Abhishek Dubey Tested-by: Venkat Rao Bagalkote --- arch/powerpc/net/bpf_jit.h | 13 ++++++ arch/powerpc/net/bpf_jit_comp.c | 63 +++++++++++++++++++++++++--- arch/powerpc/net/bpf_jit_comp64.c | 68 +++++++++++++++++++++++-------- 3 files changed, 122 insertions(+), 22 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 9f6ec00bd02e..56f56fdd4969 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -52,6 +52,13 @@ EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc)= ); \ } while (0) =20 +/* When constant jump offset is known prior */ +#define PPC_BCC_CONST_SHORT(cond, offset) \ + do { \ + BUILD_BUG_ON(offset < -0x8000 || offset > 0x7fff || (offset & 0x3)); \ + EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc)= ); \ + } while (0) + /* * Sign-extended 32-bit immediate load * @@ -73,6 +80,10 @@ } } while (0) =20 #ifdef CONFIG_PPC64 + +/* for gpr non volatile registers BPG_REG_6 to 10 */ +#define BPF_PPC_STACK_SAVE (6 * 8) + /* If dummy pass (!image), account for maximum possible instructions */ #define PPC_LI64(d, i) do { \ if (!image) \ @@ -167,6 +178,7 @@ struct codegen_context { unsigned int alt_exit_addr; u64 arena_vm_start; u64 user_vm_start; + bool is_subprog; }; =20 #define bpf_to_ppc(r) (ctx->b2p[r]) @@ -206,6 +218,7 @@ int bpf_add_extable_entry(struct bpf_prog *fp, u32 *ima= ge, u32 *fimage, int pass struct codegen_context *ctx, int insn_idx, int jmp_off, int dst_reg, u32 code); =20 +int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx); #endif =20 #endif diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index de1adc0b1157..4b18daed054a 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -206,6 +206,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *f= p) cgctx.stack_size =3D round_up(fp->aux->stack_depth, 16); cgctx.arena_vm_start =3D bpf_arena_get_kern_vm_start(fp->aux->arena); cgctx.user_vm_start =3D bpf_arena_get_user_vm_start(fp->aux->arena); + cgctx.is_subprog =3D bpf_is_subprog(fp); =20 /* Scouting faux-generate pass 0 */ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) { @@ -435,6 +436,11 @@ void bpf_jit_free(struct bpf_prog *fp) bpf_prog_unlock_free(fp); } =20 +bool bpf_jit_supports_subprog_tailcalls(void) +{ + return IS_ENABLED(CONFIG_PPC64); +} + bool bpf_jit_supports_kfunc_call(void) { return true; @@ -600,15 +606,50 @@ static int invoke_bpf_mod_ret(u32 *image, u32 *ro_ima= ge, struct codegen_context return 0; } =20 -static void bpf_trampoline_setup_tail_call_cnt(u32 *image, struct codegen_= context *ctx, - int func_frame_offset, int r4_off) +/* + * Refer __arch_prepare_bpf_trampoline() for stack component details. + * + * The tailcall count/reference is present in caller's stack frame. The + * tail_call_info is saved at the same offset on the trampoline frame + * for the traced function (BPF subprog/callee) to fetch it. + */ +static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen= _context *ctx, + int func_frame_offset, + int bpf_dummy_frame_size, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { /* See Generated stack layout */ - int tailcallcnt_offset =3D BPF_PPC_TAILCALL; + int tailcallinfo_offset =3D BPF_PPC_TAILCALL; + + /* + * func_frame_offset =3D ...(1) + * bpf_dummy_frame_size + trampoline_frame_size + */ + EMIT(PPC_RAW_LD(_R4, _R1, func_frame_offset)); + EMIT(PPC_RAW_LD(_R3, _R4, -tailcallinfo_offset)); =20 - EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); - EMIT(PPC_RAW_STL(_R3, _R1, -tailcallcnt_offset)); + /* + * Setting the tail_call_info in trampoline's frame + * depending on if previous frame had value or reference. + */ + EMIT(PPC_RAW_CMPLWI(_R3, MAX_TAIL_CALL_CNT)); + PPC_BCC_CONST_SHORT(COND_GT, 8); + EMIT(PPC_RAW_ADDI(_R3, _R4, bpf_jit_stack_tailcallinfo_offset(ctx))); + /* + * From ...(1) above: + * trampoline_frame_bottom =3D ...(2) + * func_frame_offset - bpf_dummy_frame_size + * + * Using ...(2) derived above: + * trampoline_tail_call_info_offset =3D ...(3) + * trampoline_frame_bottom - tailcallinfo_offset + * + * From ...(3): + * Use trampoline_tail_call_info_offset to write reference of main's + * tail_call_info in trampoline frame. + */ + EMIT(PPC_RAW_STL(_R3, _R1, (func_frame_offset - bpf_dummy_frame_size) + - tailcallinfo_offset)); } else { /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ EMIT(PPC_RAW_LL(_R4, _R1, r4_off)); @@ -714,6 +755,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im * LR save area [ r0 save (64-bit) ] | header * [ r0 save (32-bit) ] | * dummy frame for unwind [ back chain 1 ] -- + * [ tail_call_info ] optional - 64-bit p= owerpc * [ padding ] align stack frame * r4_off [ r4 (tailcallcnt) ] optional - 32-bit p= owerpc * alt_lr_off [ real lr (ool stub)] optional - actual lr @@ -795,6 +837,14 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tr= amp_image *im, void *rw_im } } =20 + /* + * Save tailcall count pointer at the same offset on the + * stack where subprogs expect it + */ + if ((flags & BPF_TRAMP_F_CALL_ORIG) && + (flags & BPF_TRAMP_F_TAIL_CALL_CTX)) + bpf_frame_size +=3D BPF_PPC_TAILCALL; + /* Padding to align stack frame, if any */ bpf_frame_size =3D round_up(bpf_frame_size, SZL * 2); =20 @@ -896,7 +946,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im =20 /* Replicate tail_call_cnt before calling the original BPF prog */ if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) - bpf_trampoline_setup_tail_call_cnt(image, ctx, func_frame_offset, r4_of= f); + bpf_trampoline_setup_tail_call_info(image, ctx, func_frame_offset, + bpf_dummy_frame_size, r4_off); =20 /* Restore args */ bpf_trampoline_restore_args_stack(image, ctx, func_frame_offset, nr_regs= , regs_off); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 296e9ea14f2e..18da5a866447 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -26,8 +26,12 @@ * Ensure the top half (upto local_tmp_var) stays consistent * with our redzone usage. * + * tail_call_info - stores tailcall count value in main program's + * frame, stores reference to tail_call_info of + * main's frame in sub-prog's frame. + * * [ prev sp ] <------------- - * [ tail_call_cnt ] 8 | + * [ tail_call_info ] 8 | * [ nv gpr save area ] 6*8 | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | @@ -35,8 +39,6 @@ * sp (r1) ---> [ stack pointer ] -------------- */ =20 -/* for gpr non volatile registers BPG_REG_6 to 10 */ -#define BPF_PPC_STACK_SAVE (6*8) /* for bpf JIT code internal usage */ #define BPF_PPC_STACK_LOCALS 24 /* stack frame excluding BPF stack, ensure this is quadword aligned */ @@ -98,7 +100,7 @@ static inline bool bpf_has_stack_frame(struct codegen_co= ntext *ctx) * [ prev sp ] <------------- * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- - * [ tail_call_cnt ] 8 + * [ tail_call_info ] 8 * [ nv gpr save area ] 6*8 * [ local_tmp_var ] 24 * [ unused red zone ] 224 @@ -114,7 +116,7 @@ static int bpf_jit_stack_local(struct codegen_context *= ctx) } } =20 -static int bpf_jit_stack_tailcallcnt(struct codegen_context *ctx) +int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) { return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SA= VE; } @@ -147,17 +149,32 @@ void bpf_jit_build_prologue(u32 *image, struct codege= n_context *ctx) #endif =20 /* - * Initialize tail_call_cnt if we do tail calls. - * Otherwise, put in NOPs so that it can be skipped when we are - * invoked through a tail call. + * Tail call count(tcc) is saved & updated only in main + * program's frame and the address of tcc in main program's + * frame (tcc_ptr) is saved in subprogs frame. + * + * Offset of tail_call_info on any frame will be interpreted + * as either tcc_ptr or tcc value depending on whether it is + * greater than MAX_TAIL_CALL_CNT or not. */ - if (ctx->seen & SEEN_TAILCALL) { + if (!ctx->is_subprog) { EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0)); /* this goes in the redzone */ EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } else { - EMIT(PPC_RAW_NOP()); - EMIT(PPC_RAW_NOP()); + /* + * if tail_call_info < MAX_TAIL_CALL_CNT + * main prog calling first subprog -> copy reference + * else + * subsequent subprog calling another subprog -> directly copy cont= ent + */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, 0)); + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), -(BPF_PPC_= TAILCALL))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + PPC_BCC_CONST_SHORT(COND_GT, 8); + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), + -(BPF_PPC_TAILCALL))); + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } =20 if (bpf_has_stack_frame(ctx)) { @@ -352,19 +369,38 @@ static int bpf_jit_emit_tail_call(u32 *image, struct = codegen_context *ctx, u32 o EMIT(PPC_RAW_CMPLW(b2p_index, bpf_to_ppc(TMP_REG_1))); PPC_BCC_SHORT(COND_GE, out); =20 + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallinfo_of= fset(ctx))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + PPC_BCC_CONST_SHORT(COND_LE, 8); + + /* dereference TMP_REG_1 */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 0)); + /* - * if (tail_call_cnt >=3D MAX_TAIL_CALL_CNT) + * if (tail_call_info =3D=3D MAX_TAIL_CALL_CNT) * goto out; */ - EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallcnt(ctx= ))); EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); - PPC_BCC_SHORT(COND_GE, out); + PPC_BCC_SHORT(COND_EQ, out); =20 /* - * tail_call_cnt++; + * tail_call_info++; <- Actual value of tcc here */ EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 1)); - EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallcnt(ct= x))); + + /* + * Before writing updated tail_call_info, distinguish if current frame + * is storing a reference to tail_call_info or actual tcc value in + * tail_call_info. + */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, bpf_jit_stack_tailcallinfo_of= fset(ctx))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_2), MAX_TAIL_CALL_CNT)); + PPC_BCC_CONST_SHORT(COND_GT, 8); + + /* First get address of tail_call_info */ + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_2), _R1, bpf_jit_stack_tailcallinfo_= offset(ctx))); + /* Writeback updated value to tail_call_info */ + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), 0)); =20 /* prog =3D array->ptrs[index]; */ EMIT(PPC_RAW_MULI(bpf_to_ppc(TMP_REG_1), b2p_index, 8)); --=20 2.48.1 From nobody Sat Feb 7 21:15:19 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51A0A2848A0; Sat, 24 Jan 2026 07:53:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241224; cv=none; b=GhsYjbndowRsjsntsh4h02c/nnlNG8q07ud0N7KxVZBN87zLEi3svrKefW+gmpuG4PhennFtri8I5IAFvguo1ipJj1EAdMvDHAhZ47Lpex3Lf9G1XqzW3Ygc2hpHR1j8+h4nEGaZ4xqrHaB9on7mASjjjSwa5IuIWuCpdU3x/m4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241224; c=relaxed/simple; bh=oeJO3hczzmPQy7vube3UOCTKQb3K5zRr8MH4GQUFHxU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n0lfmXx9gJLwsFlKuvri9wtnUWMp1D7Nj+5JC+SCi2YRyYh8iarDS0ABzMUbW8kjAz8mfBtOrLPvRxJIAbFeB5IDvJmAJGAmPvH9NfhnR8214/EWMQASxnDhjtSRhdlI4VKj+kg+gRT2rTsfd2rK4w29lf0yULUsxGBOv4PmmUo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=mm1vQQqA; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="mm1vQQqA" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60O5RSev024443; Sat, 24 Jan 2026 07:53:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=UXdPxN0yzQYC1zXtR qHrrTIjvlERtEZxVMk7HfF01xU=; b=mm1vQQqAipr4Psx80BtReaB7rzUE7oQO/ VebSDCwbT/OVCBZ8YPUxNTeSLvVUHF/yv+bGzlizW6UC/VhnCGzHXWc+1Hxva9lP b7+AiKadjrQ51eWygQ1TtwQhDKuaPl8l0csisQlgRBmyHROIVubiajiZJUXHae+y zfQU/v+OrQNG8lA/6siEfb7RTrnXoWr8tfHXMrW3T1qr9YXgwcuxWZC3rZfTzj7D 5MXBrNo/rWs6tebOc70GzkHpXnuOJNopaRMh5OI8fhHTu8CKVaxp56mld69cCTWw qgNoS9vFxlcz8DQy5ftlGYfKcWzbns0VxfDey/RjU8hv8ccbzZrfA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnr5rjb6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:09 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60O7r8R7010462; Sat, 24 Jan 2026 07:53:08 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnr5rjb4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:08 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60O24kpu006424; Sat, 24 Jan 2026 07:53:07 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4brqf24n1q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:07 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60O7r4Vo47514030 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Jan 2026 07:53:04 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E9BA620043; Sat, 24 Jan 2026 07:53:03 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9A36120040; Sat, 24 Jan 2026 07:52:57 +0000 (GMT) Received: from abhi.. (unknown [9.124.223.59]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 24 Jan 2026 07:52:57 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v5 3/6] powerpc64/bpf: Avoid tailcall restore from trampoline Date: Sat, 24 Jan 2026 13:22:20 +0530 Message-ID: <20260124075223.6033-4-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260124075223.6033-1-adubey@linux.ibm.com> References: <20260124075223.6033-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI0MDA2MCBTYWx0ZWRfXyKAdOv1HA5FA 84DoEfCC1SloNkLxLa+syX9gZPZbEJSl5o8fxx3mKyikDijirNWaPPQ5TVfZpIBasV2o0eWT/EA shBgEZL5FSNF3ZkCCes2VZtpPHrp6xzoOi7mGGZcXm9Z+9I2Jy4fnOOFYN+tXZST8vnOO+4Z451 KbWrWRJOaeNrh/kT9GQ0DU07hOfsQHkAbAmPoRZOe/R7R3kuT6M2OGpZ3S+L87RrEb9gRPV/dgI O1QmMVI3KD+Z327VPP/7UsoDHooIHw47dzhanrGTeHcqcdIpYpyK9K48xXR4Xmya82dez/o2319 PtbOibXjCFZ9c6RHhM2ESjFuDCzckCXfDjGOSqWSMwrnQ/AS1YqCQVbm0P2oEE946Fx0shmeQRn FVOMc56dpQPyd78e2030QACHnBblQyHoq5hFQx8SZCd/ixVRyyJH0iZCmjX2RoLjQa52qApLVgW 3FCFXlFuOW2a84+f5/A== X-Proofpoint-GUID: zttSLy4Wpu_vInN_j3ZKCS3aXrDsCSxt X-Proofpoint-ORIG-GUID: PCDTYKd1zuPlntCIjQHjWNKfHLnepz56 X-Authority-Analysis: v=2.4 cv=X+Vf6WTe c=1 sm=1 tr=0 ts=69747a65 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=yUNxoNuUyiXgUDPvJtcA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-24_02,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 impostorscore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 adultscore=0 malwarescore=0 bulkscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601240060 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey Back propagation of tailcall count is no longer needed for powerpc64 due to use of reference, which updates the tailcall count in the tail_call_info field in the frame of the main program only. Back propagation is still required for 32-bit powerpc. Signed-off-by: Abhishek Dubey Tested-by: Venkat Rao Bagalkote --- arch/powerpc/net/bpf_jit_comp.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 4b18daed054a..f8769d785123 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -659,14 +659,11 @@ static void bpf_trampoline_setup_tail_call_info(u32 *= image, struct codegen_conte static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codege= n_context *ctx, int func_frame_offset, int r4_off) { - if (IS_ENABLED(CONFIG_PPC64)) { - /* See bpf_jit_stack_tailcallcnt() */ - int tailcallcnt_offset =3D BPF_PPC_TAILCALL; - - EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset)); - EMIT(PPC_RAW_STL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); - } else { - /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ + if (IS_ENABLED(CONFIG_PPC32)) { + /* + * Restore tailcall for 32-bit powerpc + * See bpf_jit_stack_offsetof() and BPF_PPC_TC + */ EMIT(PPC_RAW_STL(_R4, _R1, r4_off)); } } --=20 2.48.1 From nobody Sat Feb 7 21:15:19 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B2F2D1A9B46; Sat, 24 Jan 2026 07:54:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241300; cv=none; b=tdc5Kz2ibm7hGO92DDlg/p8tTH+BNwI3UIPycPXIoTxq04Wxt2KMtr4F8EYmSNNGyDFzcFWKOShIaBQ6VN1T2Ea7qI+V5Es/bn5IEVSZoDM/8uIpUnLve9GH9Y5I1CEyvV357hJnWnzMajmYAoZHwb1T/Zl+1N2t+IRyDM8eypg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241300; c=relaxed/simple; bh=Zd0m7rUk7beURgdwtoStwocQjmWcEg/kis8HVPDJfy0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m3yHR62vxOFEH5LLGDNBkuuok7TJ/E1YjutAgR0nHq4D6B3XSPUVWn5IB9d7yrOYV2BDDqCxwWC1rZ28LszeIpe8R6UMiqGm3xIuwGmybWujyFNCEIckpzYc6FSPRMkTHaTWsBMmyhRxN82QGV/vJ/IAVhz91t8KywmL8DtDd3w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=azM8mUfV; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="azM8mUfV" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60O7l5Hj029709; Sat, 24 Jan 2026 07:53:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=whHUJNNJZ6qz3J1Px /WYh0SIfZcWl0bjN4IlDjqDEKw=; b=azM8mUfVXj26cYN9cnZwqeqe/xWX86Lng 0I0re3LXQhzqZEAF75kWw/9ksZcZfUI6ryr2rSSfKbZAoZtdkOh+DV2700cNtqLb G/zGFnBxiFyALglOTBqo5VxXIjn0SHzQD+HMrf2CdPnMxMs1788gn4ADxPcU/m3b C6vWSdzEx4dEay7b8EA0ABhp5RyivQsXOas+ARKVJf1DYV4quKGuMIWyisJMJ0/Z QrqILN0DL2bmIbH2d3q5uJWODXKx0amwyxYdSHT+yBqG0x0JZqnnfU0jbjs/4FUu Eu83XqfIlRJjJtTJhK3LNnvOXPRqERS7Z2g42R13X/eXUaKjLyGKw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvkgm8urq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:18 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60O7rHli007337; Sat, 24 Jan 2026 07:53:17 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvkgm8urm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:17 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60O70MOj017070; Sat, 24 Jan 2026 07:53:16 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4brn4yn392-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:16 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60O7rCqp56885562 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Jan 2026 07:53:12 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BCC4E20043; Sat, 24 Jan 2026 07:53:12 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E4F6120040; Sat, 24 Jan 2026 07:53:05 +0000 (GMT) Received: from abhi.. (unknown [9.124.223.59]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 24 Jan 2026 07:53:05 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v5 4/6] powerpc64/bpf: Add arch_bpf_stack_walk() for BPF JIT Date: Sat, 24 Jan 2026 13:22:21 +0530 Message-ID: <20260124075223.6033-5-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260124075223.6033-1-adubey@linux.ibm.com> References: <20260124075223.6033-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=Gr1PO01C c=1 sm=1 tr=0 ts=69747a6e cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=lmbXRN_Zm_tNO3HBYvgA:9 X-Proofpoint-GUID: X_2ZuYxAxidKUGW62CFsspbQMqss3Qmk X-Proofpoint-ORIG-GUID: _Qr-1v1S46W0pKlaNUWMIFLkb5WedFBn X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI0MDA2MCBTYWx0ZWRfX42sCkNmsDp+1 3dIc6Gqf94nZ00lqSkC023Ww1kjBYMpWPOk6s9nTx5yUXX9Nc/2sSKfp0+CS0hAyAX8fJZ66Zrz DMeHHp9ubn96lt+FL/qG3GcPEhvIvfi5x/+6uiWenTinlw/agbZqPCgMkV7W5Qui1tj3oAU/O0G CwlWd3RhqVrIVg9t6mF6Zx2Vd/dwxqtkqysNf3bacD8IpH1PsF02gnew0SFuwMKUDYbQZKZJrnN 6qZuMqzO7P9m1kP4Ef/3TN7eL+aBagxGfCkFIG6ttKN2xzm+8E3QZ/r6RHr072Hlli6wxV3ja6g vvQ5blVo83B8zA8bu+jqdfmyiFMviz1NfYF0IlccP1npIS3+JZPTr5z+QzJF3Fx3VdhK8tfm/yx /6H7mtYOMundk6hDBGkB6KaqwS8gX/wipvlum/qF7LZMhSBGXjYYq0iW09tJyEuuO334uCgrsU/ VLnorIx4teRG0Iq/jGg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-24_02,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 clxscore=1015 lowpriorityscore=0 suspectscore=0 impostorscore=0 phishscore=0 malwarescore=0 adultscore=0 spamscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601240060 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey This function is used by bpf_throw() to unwind the stack until frame of exception-boundary during BPF exception handling. This function is necessary to support BPF exceptions on PowerPC. Signed-off-by: Abhishek Dubey Tested-by: Venkat Rao Bagalkote --- arch/powerpc/net/bpf_jit_comp64.c | 41 +++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 18da5a866447..1bba068f2c4c 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -247,6 +247,47 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen= _context *ctx) bpf_jit_build_fentry_stubs(image, ctx); } =20 +/* + * arch_bpf_stack_walk() - BPF stack walker for PowerPC + * + * Based on arch_stack_walk() from stacktrace.c. + * PowerPC uses stack frames rather than stack pointers. See [1] for + * the equivalence between frame pointers and stack pointers. + * Additional reference at [2]. + * TODO: refactor with arch_stack_walk() + * + * [1]: https://lore.kernel.org/all/20200220115141.2707-1-mpe@ellerman.id.= au/ + * [2]: https://lore.kernel.org/bpf/20260122211854.5508-5-adubey@linux.ibm= .com/ + */ + +void arch_bpf_stack_walk(bool (*consume_fn)(void *, u64, u64, u64), void *= cookie) +{ + // callback processing always in current context + unsigned long sp =3D current_stack_frame(); + + for (;;) { + unsigned long *stack =3D (unsigned long *) sp; + unsigned long ip; + + if (!validate_sp(sp, current)) + return; + + ip =3D stack[STACK_FRAME_LR_SAVE]; + if (!ip) + break; + + /* + * consume_fn common code expects stack pointer in third + * argument. There is no sp in ppc64, rather pass frame + * pointer(named sp here). + */ + if (ip && !consume_fn(cookie, ip, sp, sp)) + break; + + sp =3D stack[0]; + } +} + int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_con= text *ctx, u64 func) { unsigned long func_addr =3D func ? ppc_function_entry((void *)func) : 0; --=20 2.48.1 From nobody Sat Feb 7 21:15:19 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BB3C225A3D; Sat, 24 Jan 2026 07:54:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241251; cv=none; b=L3t0Y389jU+dfCdxjxCMdHlidPUbQ+0GpL5a1+RRyK+N4Mzio+LeW+3G64OLCmyJm5iGMcrfmMSEIM4WBpTHGbEpWT87UtUa6EvYp3yd5ZhoZypF1VJVPTZhGKNv1NQV/9X+tYCbPwMX1wwTJy+qUv0TjFRD8PwqVdGgeES4kJ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241251; c=relaxed/simple; bh=TCTKfshPCV8SO4rny2nEmpkQ7KnE8QDxHggBE04FCkY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VWEy80Gz8rOqWfuGbY5Ps9X3s0RORc8SLTsFEeiml8hLoGa9U131bc/WJ675OXpm2X039QpIHaNSZVBZbmtxOxs0XtYrLmCXiTqdtm91BDaOzB9z0wHHmLZzMQU5NlF49Xr0pHpy1FuG/yBzx5dd5gcknJ6Ral7Lq+ILphdNKj0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=FuZAuNKT; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="FuZAuNKT" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60O7jwrA001740; Sat, 24 Jan 2026 07:53:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=X9MwwHemcZiPszY3K p91dfFv3DKy9H/iTxZM5Y0gVkE=; b=FuZAuNKTsb9cJDD+teyQTTAq06yKLBFHc VCjkY9Pd3qIJJgkj99mE54nPmHjavQZCd7GF3iXGUrdTP28AuSJGfWg65znyD/DD G8klMFeZlc6nluKWrrM7C6SqcWxSsSlXkNeeKyJUSqkogRk06IJ9KI4icmfIbS1f vt3IauexTPAFlFaq0sX5GCxeyZaf5Yrj4GSJQQf1r2eLCwZhmiGY8e5CIidmHHFj FdcH4yJekB7BsKJONLw+a0SOIPRKaOWzFv6Ut7U8DphCihrsYo+l0FCg5uPL/kf3 IXvJ92AQfCClEBOUHx5xFJMw0KY1ux/lEpb2hR6UrFYS5a3l1+xRQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnk6grmk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:27 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60O7rR9A014049; Sat, 24 Jan 2026 07:53:27 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnk6grmg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:26 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60O6Seeq027298; Sat, 24 Jan 2026 07:53:25 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4brnrnmx74-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:25 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60O7rLFT17301860 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Jan 2026 07:53:21 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 79DC020043; Sat, 24 Jan 2026 07:53:21 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 16DB320040; Sat, 24 Jan 2026 07:53:15 +0000 (GMT) Received: from abhi.. (unknown [9.124.223.59]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 24 Jan 2026 07:53:14 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v5 5/6] powerpc64/bpf: Support exceptions Date: Sat, 24 Jan 2026 13:22:22 +0530 Message-ID: <20260124075223.6033-6-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260124075223.6033-1-adubey@linux.ibm.com> References: <20260124075223.6033-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: wwG8h5J6O3egLS2U5yNf7qxG-JWX_uuo X-Proofpoint-ORIG-GUID: ktkxY51okytgQPuHJcU8WUsOEYkCq1Xp X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI0MDA2MCBTYWx0ZWRfX6iLnNr0OKdHW pxkNi08Y+VOfmcrxUbAe5BA+YcFbA3DIdApvLN7Ck4l3CGym2YMwu2wjdU23TUmNBQ5o8KVmJPd efNWq8dKndLHP0R9AAxpoC+gZBB1kjZveeGC7CqeQG8xCt7UDNk8qaoPbaLyaI33pSnZeAAekoT jj9RaMftvTBXXdDEEGyV8uNtetKkbqsehRq84rCC8D+x/E5xnZulrT5PeTgucPaI12u8EfB1twh 9jgO+FXu0pwXSDJRWZBE+Z+iXCwv2SLB09uswO+NuJYU20u0Cr2qb+nLmL6zt4FFJg+3G/RO1it VGwwKLTaWmrEaq0ZxmmHT2qo81YYSX7ZN/w99uWjm5xzBbpnmnDZRrze6QPDjbxc2YdDXh5VsAK Wp70LG1vVPfqIdMQMo4dt7HJN9hqBNjC0Fwfaw/ffYbleqBqVT4rVyVgqcz+nDZtsbu2HLkDGGb C9YvJ1Zy4JWgtg/Gwyw== X-Authority-Analysis: v=2.4 cv=AMiVTGgp c=1 sm=1 tr=0 ts=69747a77 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=DU0EyFXtAAAA:8 a=VnNF1IyMAAAA:8 a=xcpHmmOOOUdXZ9sPVCMA:9 a=UCR5be5CC-YrbG9FbbB0:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-24_02,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 phishscore=0 bulkscore=0 suspectscore=0 adultscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601240060 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The modified prologue/epilogue generation code now enables exception-callback to use the stack frame of the program marked as exception boundary, where callee saved registers are stored. As per ppc64 ABIv2 documentation[1], r14-r31 are callee saved registers. BPF programs on ppc64 already saves r26-r31 registers. Saving the remaining set of callee saved registers(r14-r25) is handled in the next patch. [1] https://ftp.rtems.org/pub/rtems/people/sebh/ABI64BitOpenPOWERv1.1_16Jul= y2015_pub.pdf Signed-off-by: Abhishek Dubey Tested-by: Venkat Rao Bagalkote --- arch/powerpc/net/bpf_jit.h | 2 ++ arch/powerpc/net/bpf_jit_comp.c | 7 ++++ arch/powerpc/net/bpf_jit_comp64.c | 59 +++++++++++++++++++++---------- 3 files changed, 50 insertions(+), 18 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 56f56fdd4969..82bbf63f0e57 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -179,6 +179,8 @@ struct codegen_context { u64 arena_vm_start; u64 user_vm_start; bool is_subprog; + bool exception_boundary; + bool exception_cb; }; =20 #define bpf_to_ppc(r) (ctx->b2p[r]) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index f8769d785123..3a2e6fa3ac7c 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -207,6 +207,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *f= p) cgctx.arena_vm_start =3D bpf_arena_get_kern_vm_start(fp->aux->arena); cgctx.user_vm_start =3D bpf_arena_get_user_vm_start(fp->aux->arena); cgctx.is_subprog =3D bpf_is_subprog(fp); + cgctx.exception_boundary =3D fp->aux->exception_boundary; + cgctx.exception_cb =3D fp->aux->exception_cb; =20 /* Scouting faux-generate pass 0 */ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) { @@ -436,6 +438,11 @@ void bpf_jit_free(struct bpf_prog *fp) bpf_prog_unlock_free(fp); } =20 +bool bpf_jit_supports_exceptions(void) +{ + return IS_ENABLED(CONFIG_PPC64); +} + bool bpf_jit_supports_subprog_tailcalls(void) { return IS_ENABLED(CONFIG_PPC64); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 1bba068f2c4c..db121b1404fe 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -89,7 +89,9 @@ static inline bool bpf_has_stack_frame(struct codegen_con= text *ctx) * - the bpf program uses its stack area * The latter condition is deduced from the usage of BPF_REG_FP */ - return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, bpf_to_ppc(BPF_= REG_FP)); + return ctx->seen & SEEN_FUNC || + bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)) || + ctx->exception_cb; } =20 /* @@ -161,8 +163,13 @@ void bpf_jit_build_prologue(u32 *image, struct codegen= _context *ctx) EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0)); /* this goes in the redzone */ EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); - } else { + } else if (!ctx->exception_cb) { /* + * Tailcall jitting for non exception_cb progs only. + * exception_cb won't require tail_call_info to be setup. + * + * tail_call_info interpretation logic: + * * if tail_call_info < MAX_TAIL_CALL_CNT * main prog calling first subprog -> copy reference * else @@ -177,8 +184,12 @@ void bpf_jit_build_prologue(u32 *image, struct codegen= _context *ctx) EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } =20 - if (bpf_has_stack_frame(ctx)) { + if (bpf_has_stack_frame(ctx) && !ctx->exception_cb) { /* + * exception_cb uses boundary frame after stack walk. + * It can simply use redzone, this optimization reduces + * stack walk loop by one level. + * * We need a stack frame, but we don't necessarily need to * save/restore LR unless we call other functions */ @@ -190,23 +201,35 @@ void bpf_jit_build_prologue(u32 *image, struct codege= n_context *ctx) EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size))); } =20 - /* - * Back up non-volatile regs -- BPF registers 6-10 - * If we haven't created our own stack frame, we save these - * in the protected zone below the previous stack frame - */ - for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) - if (bpf_is_seen_register(ctx, bpf_to_ppc(i))) - EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to= _ppc(i)))); + if (!ctx->exception_cb) { + /* + * Back up non-volatile regs -- BPF registers 6-10 + * If we haven't created our own stack frame, we save these + * in the protected zone below the previous stack frame + */ + for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) + if (ctx->exception_boundary || bpf_is_seen_register(ctx, bpf_to_ppc(i))) + EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, + bpf_jit_stack_offsetof(ctx, bpf_to_ppc(i)))); =20 - if (ctx->arena_vm_start) - EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1, + if (ctx->exception_boundary || ctx->arena_vm_start) + EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); + } else { + /* + * Exception callback receives Frame Pointer of boundary + * program(main prog) as third arg + */ + EMIT(PPC_RAW_MR(_R1, _R5)); + } =20 - /* Setup frame pointer to point to the bpf stack area */ + /* + * Exception_cb not restricted from using stack area or arena. + * Setup frame pointer to point to the bpf stack area + */ if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP))) EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1, - STACK_FRAME_MIN_SIZE + ctx->stack_size)); + STACK_FRAME_MIN_SIZE + ctx->stack_size)); =20 if (ctx->arena_vm_start) PPC_LI64(bpf_to_ppc(ARENA_VM_START), ctx->arena_vm_start); @@ -218,17 +241,17 @@ static void bpf_jit_emit_common_epilogue(u32 *image, = struct codegen_context *ctx =20 /* Restore NVRs */ for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) - if (bpf_is_seen_register(ctx, bpf_to_ppc(i))) + if (ctx->exception_cb || bpf_is_seen_register(ctx, bpf_to_ppc(i))) EMIT(PPC_RAW_LD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_= ppc(i)))); =20 - if (ctx->arena_vm_start) + if (ctx->exception_cb || ctx->arena_vm_start) EMIT(PPC_RAW_LD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 /* Tear down our stack frame */ if (bpf_has_stack_frame(ctx)) { EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size)); - if (ctx->seen & SEEN_FUNC) { + if (ctx->seen & SEEN_FUNC || ctx->exception_cb) { EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF)); EMIT(PPC_RAW_MTLR(_R0)); } --=20 2.48.1 From nobody Sat Feb 7 21:15:19 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FAA32C0F6D; Sat, 24 Jan 2026 07:54:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241255; cv=none; b=P78Iq2vzkJ+phDWU4CBNJyweeBkjR20vRiWlGu9oL7Cq8DPNw+NQ25pXTP6hJN6KAmF3ojBfM9jEqcVVhs63Es/odIQPxnPRsInSPww12Jw/sc0zbeL3k1Kj6EeX4VSdEAFAZe0U+yWRUBZq/geERMyByQ8qy9qv8h9pwn/I2ZQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769241255; c=relaxed/simple; bh=feP4cI73aQ+5/rIlCrDHWW7cRk0x0Gn9TYzNVeOr0AQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p6l0c/fuTPS24wBPW8nGL8IHRfjT3ecXqT14N+I4R6AKNlf7WWFcyxq6um/H2WYnxouDPqY1EX/fP6i6GOYYjXntvQKSVRqVAKB1zbOaXwlxygkvtw9XW9uXjkfN7q/W7grTVUhR1MP3vJTXSINXC85c2slNiePDk97bWtVWOfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=IdevFVZv; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="IdevFVZv" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60O6qn5J007835; Sat, 24 Jan 2026 07:53:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=xSguuhn/XN2sPmGDH 4t1n1olL+gYWGXSgbvhcVrXBHI=; b=IdevFVZve+y4lWUAEIkgEV54Cq6DzE7Ql nySJQUBtNKxySGPxXWpniDkPpk15PacKDJP0F4Y1TEO6bItoEdPZ/vkO9JDmkXmg SFT66s41aRdRYDb2ptMN39bIGtqbPlvn1ZvwgsX6bP6aYlVkqxF2GdtVxYHWrxEj /iX+du2zRakum9u4QM7N43NrnBT1/YBW14d8dqQAPqlgEM8WwSRnPIZ7XiD6xfxs OGecJehFud06Gxjmg4MMmnC3tFP0WJ/EwoXrKzEA2JpPmK962tivaWRpMNk0CwyT pEyvrMGDiNNVc45yT/bMvflonWoB5K+2+7zkEDnXg3CTUCyhHsGzw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnr5rjbu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:35 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60O7rYco011228; Sat, 24 Jan 2026 07:53:35 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnr5rjbr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:34 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60O3CI3R001162; Sat, 24 Jan 2026 07:53:33 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4brpykcr80-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 24 Jan 2026 07:53:33 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60O7rUhj15532296 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Jan 2026 07:53:30 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B46E20043; Sat, 24 Jan 2026 07:53:30 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 741BA20040; Sat, 24 Jan 2026 07:53:23 +0000 (GMT) Received: from abhi.. (unknown [9.124.223.59]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 24 Jan 2026 07:53:23 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v5 6/6] powerpc64/bpf: Additional NVR handling for bpf_throw Date: Sat, 24 Jan 2026 13:22:23 +0530 Message-ID: <20260124075223.6033-7-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260124075223.6033-1-adubey@linux.ibm.com> References: <20260124075223.6033-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI0MDA2MCBTYWx0ZWRfXwIMTuDrWGuT6 /H2850GJCYMc6zD3tXA/jN/XZx5iqY7ibSbDzJYvbSRnZQ3hnVqGu1kiVOG1DUZlu7LRfkwWU81 mwYkymFs9vEYswaZ8SUAWsqaj2D5CfX+gmQidyVZLuevmd8vKjj6D5vor5U6iRtQoKUOQEICKA7 NuMKIocWLn/36KDj+xcxBcAuhwbKiCleffxgJZoU3l8xmZvapEjNUxpPP5N1Ovuwr1gPB0UoWCk 4hEKBmKAbgWc8t4/JQdDeUYuLNhe7w7FYz58KayWL+6UtspJCIoZ1UTfFAhSPYSoKKbbiHX2DgQ CMSXLywMD6FCgMyFqX+MNuAZDAAUMQXpT39Ab3tTzhtEj8ewQgiw39S6vplNyAH4Pr3pyJGzKxx CZqRf9f2IP8fAbeYSJeAmz9V3wY7w82WUDqF3tDV9Q5+wGy0Z3zymLmQxwuGiotycjx5jRt+t3O J3oICFxoDgqnGpV39dA== X-Proofpoint-GUID: LVkydNDPMIYNuIrXjVl4Kgp5S0ijs0rA X-Proofpoint-ORIG-GUID: 2eZVcQPTNMY13b4vd8jIcqExKuNMJY0B X-Authority-Analysis: v=2.4 cv=X+Vf6WTe c=1 sm=1 tr=0 ts=69747a7f cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=2yuXeg4sgMS8061G0UsA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-24_02,2026-01-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 impostorscore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 adultscore=0 malwarescore=0 bulkscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601240060 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The bpf_throw() function never returns, if it has clobbered any callee-saved register, those will remain clobbered. The prologue must take care of saving all callee-saved registers in the frame of exception boundary program. Later these additional non volatile registers R14-R25 along with other NVRs are restored back in the epilogue of exception callback. To achieve above objective, the frame size is determined dynamically to accommodate additional non volatile registers in exception boundary's frame. For non-exception boundary program, the frame size remains optimal. The additional instructions to save & restore r14-r25 registers are emitted only during exception boundary and exception callback program respectively. Signed-off-by: Abhishek Dubey Tested-by: Venkat Rao Bagalkote --- arch/powerpc/net/bpf_jit_comp64.c | 89 ++++++++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index db121b1404fe..17de8b53a962 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -32,21 +32,37 @@ * * [ prev sp ] <------------- * [ tail_call_info ] 8 | - * [ nv gpr save area ] 6*8 | + * [ nv gpr save area ] 6*8 + (12*8) | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | * [ frame header ] 32/112 | * sp (r1) ---> [ stack pointer ] -------------- + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ =20 /* for bpf JIT code internal usage */ #define BPF_PPC_STACK_LOCALS 24 +/* + * for additional non volatile registers(r14-r25) to be saved + * at exception boundary + */ +#define BPF_PPC_EXC_STACK_SAVE (12*8) + /* stack frame excluding BPF stack, ensure this is quadword aligned */ #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ BPF_PPC_STACK_LOCALS + \ BPF_PPC_STACK_SAVE + \ BPF_PPC_TAILCALL) =20 +/* + * same as BPF_PPC_STACKFRAME with save area for additional + * non volatile registers saved at exception boundary. + * This is quad-word aligned. + */ +#define BPF_PPC_EXC_STACKFRAME (BPF_PPC_STACKFRAME + BPF_PPC_EXC_STACK_SAV= E) + /* BPF register usage */ #define TMP_REG_1 (MAX_BPF_JIT_REG + 0) #define TMP_REG_2 (MAX_BPF_JIT_REG + 1) @@ -88,10 +104,16 @@ static inline bool bpf_has_stack_frame(struct codegen_= context *ctx) * - we call other functions (kernel helpers), or * - the bpf program uses its stack area * The latter condition is deduced from the usage of BPF_REG_FP + * + * bpf_throw() leads to exception callback from a BPF (sub)program. + * The (sub)program is always marked as SEEN_FUNC, creating a stack + * frame. The exception callback uses the frame of the exception + * boundary, so the exception boundary program must have a frame. */ return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)) || - ctx->exception_cb; + ctx->exception_cb || + ctx->exception_boundary; } =20 /* @@ -103,9 +125,12 @@ static inline bool bpf_has_stack_frame(struct codegen_= context *ctx) * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- * [ tail_call_info ] 8 - * [ nv gpr save area ] 6*8 + * [ nv gpr save area ] 6*8 + (12*8) * [ local_tmp_var ] 24 * [ unused red zone ] 224 + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ static int bpf_jit_stack_local(struct codegen_context *ctx) { @@ -114,7 +139,12 @@ static int bpf_jit_stack_local(struct codegen_context = *ctx) return STACK_FRAME_MIN_SIZE + ctx->stack_size; } else { /* Stack layout with redzone */ - return -(BPF_PPC_TAILCALL + BPF_PPC_STACK_SAVE + BPF_PPC_STACK_LOCALS); + return -(BPF_PPC_TAILCALL + +BPF_PPC_STACK_SAVE + +(ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACK_SAVE : 0) + +BPF_PPC_STACK_LOCALS + ); } } =20 @@ -125,9 +155,19 @@ int bpf_jit_stack_tailcallinfo_offset(struct codegen_c= ontext *ctx) =20 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg) { - if (reg >=3D BPF_PPC_NVR_MIN && reg < 32) + int min_valid_nvreg =3D BPF_PPC_NVR_MIN; + /* Default frame size for all cases except exception boundary */ + int frame_nvr_size =3D BPF_PPC_STACKFRAME; + + /* Consider all nv regs for handling exceptions */ + if (ctx->exception_boundary || ctx->exception_cb) { + min_valid_nvreg =3D _R14; + frame_nvr_size =3D BPF_PPC_EXC_STACKFRAME; + } + + if (reg >=3D min_valid_nvreg && reg < 32) return (bpf_has_stack_frame(ctx) ? - (BPF_PPC_STACKFRAME + ctx->stack_size) : 0) + (frame_nvr_size + ctx->stack_size) : 0) - (8 * (32 - reg)) - BPF_PPC_TAILCALL; =20 pr_err("BPF JIT is asking about unknown registers"); @@ -138,6 +178,17 @@ void bpf_jit_realloc_regs(struct codegen_context *ctx) { } =20 +/* + * For exception boundary & exception_cb progs: + * return increased size to accommodate additional NVRs. + */ +static int bpf_jit_stack_size(struct codegen_context *ctx) +{ + return ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACKFRAME : + BPF_PPC_STACKFRAME; +} + void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx) { int i; @@ -198,7 +249,19 @@ void bpf_jit_build_prologue(u32 *image, struct codegen= _context *ctx) EMIT(PPC_RAW_STD(_R0, _R1, PPC_LR_STKOFF)); } =20 - EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size))); + EMIT(PPC_RAW_STDU(_R1, _R1, + -(bpf_jit_stack_size(ctx) + ctx->stack_size))); + } + + /* + * Program acting as exception boundary pushes R14..R25 in addition to + * BPF callee-saved non volatile registers. Exception callback uses + * the boundary program's stack frame, recover additionally saved + * registers in epilogue of exception callback. + */ + if (ctx->exception_boundary) { + for (i =3D _R14; i <=3D _R25; i++) + EMIT(PPC_RAW_STD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); } =20 if (!ctx->exception_cb) { @@ -248,9 +311,19 @@ static void bpf_jit_emit_common_epilogue(u32 *image, s= truct codegen_context *ctx EMIT(PPC_RAW_LD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 + if (ctx->exception_cb) { + /* + * Recover additionally saved non volatile registers from stack + * frame of exception boundary program. + */ + for (i =3D _R14; i <=3D _R25; i++) + EMIT(PPC_RAW_LD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); + } + /* Tear down our stack frame */ if (bpf_has_stack_frame(ctx)) { - EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size)); + EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_jit_stack_size(ctx) + ctx->stack_size)); + if (ctx->seen & SEEN_FUNC || ctx->exception_cb) { EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF)); EMIT(PPC_RAW_MTLR(_R0)); --=20 2.48.1