From nobody Sat Feb 7 22:21:03 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30D3238946E; Wed, 14 Jan 2026 11:46:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391174; cv=none; b=mm5m39bvjEHk3ABV7S+OL91mCG6rLOuoVzz35Yx2BW4razLvzNW2SpRGwD/9e+xgAHyMMjvGUxldgq0l700vhCBt/GfIhUDj9Wiaib2dsGGPXXGAbm2/JkFfynsWgVXK5A9Tr6F8W6J8wYomNhj/m0OYv/z3vHn51dQx/S/nIPw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391174; c=relaxed/simple; bh=NKE3sgZj5OUww0WKMLWZQX1nBW/XxdZKqirXNsXcPRI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Zl6q/JIR28h4HiyluWVHbwfRoYe9HUsTYzqZe1rfg23ItCEpge7nwtpk3b+bd9pWUOdosprICY5wjs3RzgFb6Rl2zmxwHo+MXAwk2J06mW5fux2F+NQu3JkVsajxGgz+j4C3PeB0opdIKATWcfy556eQeLMD8c/VjUbIsoR1os0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=aBxUDYRC; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="aBxUDYRC" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60E5lWfp014533; Wed, 14 Jan 2026 11:45:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=c8eQyPKRkd8c6VjdL eEJNDRNEf6o7S8YixflG/EnmM4=; b=aBxUDYRCXlC8u2/A7SYkVASiqkCKVCKan b90kL9HSFcIACCwZn3dVcIUMaAAtna5BN7EVMUUpy7nXil8wQVzDKZlC+Wvcg8dF zNBEefgk2Myd+Amaa3KAo743mygOnUJLDRGBfNQf+4ONcJpBRMr2qO92xSECqbQD iV/PLxLuhAlkJypAam1ldPd5Ra7tbYbcjOn6NnOcaFI2xpyRcVMVOF/dfe0xCsFa wzPqjAMJcOhLDbhXXe8kfxyjdVSokw6x89rMao1Bt9ClWFnjuxAYShDEL5hmDoFY ZZOm1bxEEQl1bq8oReCdxxW2IwPz2s5Fka/wG63jVui32dQWJ7ttA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bkeeq1609-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:45:29 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60EBbs0V014054; Wed, 14 Jan 2026 11:45:28 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bkeeq1604-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:45:28 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60E861nE030153; Wed, 14 Jan 2026 11:45:27 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4bm3ajskpe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:45:27 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60EBjNwp36438300 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Jan 2026 11:45:23 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8A06F20043; Wed, 14 Jan 2026 11:45:23 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2B27020040; Wed, 14 Jan 2026 11:45:17 +0000 (GMT) Received: from abhi.. (unknown [9.124.217.65]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 14 Jan 2026 11:45:16 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v2 1/6] powerpc64/bpf: Move tail_call_cnt to bottom of stack frame Date: Wed, 14 Jan 2026 17:14:45 +0530 Message-ID: <20260114114450.30405-2-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260114114450.30405-1-adubey@linux.ibm.com> References: <20260114114450.30405-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=DI6CIiNb c=1 sm=1 tr=0 ts=696781d9 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=Rb3EU3fDdKp9F2ZL6qEA:9 X-Proofpoint-GUID: duOXHOgd6VJMcHvJXhnqevS465Fy2KGf X-Proofpoint-ORIG-GUID: cNXRZn3RtYxd44cGYlgZnjNCbQ-82IKv X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTE0MDA5MyBTYWx0ZWRfX4CtxT/ZD5X2s tcdTVzLp+Sr/51KXX4z2mBk4D9/pFaY8f/44K+lWNpxHFtaM/awMceJuwpCkuYjAeiHkmwpxYNe UCuhUF4gQqQxlnR3gOLHb6OSytuj6P/c9dhG8bdn14zQztIc4nMf/jaWOZcMcD7WL+VZC26MLfU 7gMhrFSvm+KVvvIz1z3y9Qm9/CPpsCbmtBTcHFuTL+F4ulnjs7LWzZPZ38BGj6i+t9yCUwNYGSU po6YeHmkyuKWdp2fmuqsXxHYLey1tXVuRKMovVI3A7KWS/CcsJmN+mXsK8yM12SqVMl3Y59AQ7K 9ZtAkOJChw8nXpQ3VpybM45D+bVgHFdyEnMr9kKGczyOQOmN4ELsRi0Jg2ib2332IhykUWBsTj6 K0+4metr3Qdv2gdsRWnC7+lBoT4yQUKTjYh5PLXHOt3xzgFGO4KBllcwcQkYuzZTuNsmFZky6nh dkP6gSOX635PVHrQ68w== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-14_03,2026-01-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 adultscore=0 malwarescore=0 spamscore=0 suspectscore=0 phishscore=0 impostorscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601140093 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey In the conventional stack frame, the position of tail_call_cnt is after the NVR save area (BPF_PPC_STACK_SAVE). Whereas, the offset of tail_call_cnt in the trampoline frame is after the stack alignment padding. BPF JIT logic could become complex when dealing with frame-sensitive offset calculation of tail_call_cnt. Having the same offset in both frames is the desired objective. The trampoline frame does not have a BPF_PPC_STACK_SAVE area. Introducing it leads to under-utilization of extra memory meant only for the offset alignment of tail_call_cnt. Another challenge is the variable alignment padding sitting at the bottom of the trampoline frame, which requires additional handling to compute tail_call_cnt offset. This patch addresses the above issues by moving tail_call_cnt to the bottom of the stack frame at offset 0 for both types of frames. This saves additional bytes required by BPF_PPC_STACK_SAVE in trampoline frame, and a common offset computation for tail_call_cnt serves both frames. The changes in this patch are required by the third patch in the series, where the 'reference to tail_call_info' of the main frame is copied into the trampoline frame from the previous frame. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit.h | 4 ++++ arch/powerpc/net/bpf_jit_comp64.c | 31 ++++++++++++++++++++----------- 2 files changed, 24 insertions(+), 11 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 8334cd667bba..45d419c0ee73 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -72,6 +72,10 @@ } } while (0) =20 #ifdef CONFIG_PPC64 + +/* for tailcall counter */ +#define BPF_PPC_TAILCALL 8 + /* If dummy pass (!image), account for maximum possible instructions */ #define PPC_LI64(d, i) do { \ if (!image) \ diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 1fe37128c876..39061cd742c1 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -20,13 +20,15 @@ #include "bpf_jit.h" =20 /* - * Stack layout: + * Stack layout 1: + * Layout when setting up our own stack frame. + * Note: r1 at bottom, component offsets positive wrt r1. * Ensure the top half (upto local_tmp_var) stays consistent * with our redzone usage. * * [ prev sp ] <------------- - * [ nv gpr save area ] 6*8 | * [ tail_call_cnt ] 8 | + * [ nv gpr save area ] 6*8 | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | * [ frame header ] 32/112 | @@ -36,10 +38,12 @@ /* for gpr non volatile registers BPG_REG_6 to 10 */ #define BPF_PPC_STACK_SAVE (6*8) /* for bpf JIT code internal usage */ -#define BPF_PPC_STACK_LOCALS 32 +#define BPF_PPC_STACK_LOCALS 24 /* stack frame excluding BPF stack, ensure this is quadword aligned */ #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ - BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SAVE) + BPF_PPC_STACK_LOCALS + \ + BPF_PPC_STACK_SAVE + \ + BPF_PPC_TAILCALL) =20 /* BPF register usage */ #define TMP_REG_1 (MAX_BPF_JIT_REG + 0) @@ -87,27 +91,32 @@ static inline bool bpf_has_stack_frame(struct codegen_c= ontext *ctx) } =20 /* + * Stack layout 2: * When not setting up our own stackframe, the redzone (288 bytes) usage i= s: + * Note: r1 from prev frame. Component offset negative wrt r1. * * [ prev sp ] <------------- * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- - * [ nv gpr save area ] 6*8 * [ tail_call_cnt ] 8 + * [ nv gpr save area ] 6*8 * [ local_tmp_var ] 24 * [ unused red zone ] 224 */ static int bpf_jit_stack_local(struct codegen_context *ctx) { - if (bpf_has_stack_frame(ctx)) + if (bpf_has_stack_frame(ctx)) { + /* Stack layout 1 */ return STACK_FRAME_MIN_SIZE + ctx->stack_size; - else - return -(BPF_PPC_STACK_SAVE + 32); + } else { + /* Stack layout 2 */ + return -(BPF_PPC_TAILCALL + BPF_PPC_STACK_SAVE + BPF_PPC_STACK_LOCALS); + } } =20 static int bpf_jit_stack_tailcallcnt(struct codegen_context *ctx) { - return bpf_jit_stack_local(ctx) + 24; + return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SA= VE; } =20 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg) @@ -115,7 +124,7 @@ static int bpf_jit_stack_offsetof(struct codegen_contex= t *ctx, int reg) if (reg >=3D BPF_PPC_NVR_MIN && reg < 32) return (bpf_has_stack_frame(ctx) ? (BPF_PPC_STACKFRAME + ctx->stack_size) : 0) - - (8 * (32 - reg)); + - (8 * (32 - reg)) - BPF_PPC_TAILCALL; =20 pr_err("BPF JIT is asking about unknown registers"); BUG(); @@ -145,7 +154,7 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_= context *ctx) if (ctx->seen & SEEN_TAILCALL) { EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0)); /* this goes in the redzone */ - EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_STACK_SAVE + 8))); + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } else { EMIT(PPC_RAW_NOP()); EMIT(PPC_RAW_NOP()); --=20 2.48.1 From nobody Sat Feb 7 22:21:03 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78594396B6E; Wed, 14 Jan 2026 11:46:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391177; cv=none; b=bqmG4ORfKrdwIaOMtM+BFCFy5d9PELy6awUa4y4S8qKt4IkDa30T4KE3U0CiVFm4VcRrj7GWvVnFl5t2ApY6OL4tA9SD3JpL0Difqlomoac0ETFiJ3RvAZ2XuG6DXKe/yQuQwnHlD4dEC+SLOUEsfa8z6RahyooPG42l/b1pd1c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391177; c=relaxed/simple; bh=IMv6UmyPHA/NZcSIkxk3yn+ujdh6Ok1Cl08vKSOlN5Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NlmwGPHWqlKcu1bHDXt6vNBFA9pBchrTn8kaURqc7wedbBIFa7VRJGgShkrazlnVCptIsQbLomcnJ2+KhEeIpWwlkGXRpVXitQ7pU5kwFjC1Sf8UScq7tcNdG7w4UDU34rhQYhZk0DeQL9gQ6kZgKhDMzS/+MbnO56Z0oAhxEx4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=cDWmPANQ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="cDWmPANQ" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60E9atH2027468; Wed, 14 Jan 2026 11:45:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ahiomznzEkOpa4WVW ml1SafgPWOLVG69UuUQ4udIaAU=; b=cDWmPANQ+3AH6WRJJBvFM4kGaGW1LIBBd /DaFsJzMgeXGE4edxerQVTue6xFY7eOwN7zXre6Wk0ea73EwFmoaiJlETewSkOVr X9BSgCWY5dIIVqHwDjhtj4HovZDrbpJA8PI3xLObOPGLsPWTPi9PAqRKhjRUSyrn U9D4quO0fkpCuOh5K3QFzV66IIfHbQ9Ok020yVNRGj+wKy8oZGP36tW6c/LyCwDu yo1g+D1kBZ7cNf5KKC6cNxefJk/dcWpZeU6rlPKDTu8gEcJLkEfovOLrn6DU7nkU mtY5BVIdE5k44DZP6Wn8fiQcWqr/voJxoHSfmmnGxwVgVUqkjezOg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bkc6h8xgr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:45:44 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60EBji7C010497; Wed, 14 Jan 2026 11:45:44 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bkc6h8xgm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:45:44 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60EBKNgE029743; Wed, 14 Jan 2026 11:45:43 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4bm3ajskq7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:45:43 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60EBjdQA24379876 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Jan 2026 11:45:39 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 525EA20043; Wed, 14 Jan 2026 11:45:39 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CE04220040; Wed, 14 Jan 2026 11:45:32 +0000 (GMT) Received: from abhi.. (unknown [9.124.217.65]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 14 Jan 2026 11:45:32 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v2 2/6] powerpc64/bpf: Support tailcalls with subprogs Date: Wed, 14 Jan 2026 17:14:46 +0530 Message-ID: <20260114114450.30405-3-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260114114450.30405-1-adubey@linux.ibm.com> References: <20260114114450.30405-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: fqc1pdcJsBmUjo_Vq3js5Wb5bV5Uf0jK X-Proofpoint-ORIG-GUID: V5sq_PHPvX459JuqANG2H2X6SN6_3mgS X-Authority-Analysis: v=2.4 cv=TaibdBQh c=1 sm=1 tr=0 ts=696781e8 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=hRgTkoRQdUDsKxgB9FcA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTE0MDA5MyBTYWx0ZWRfX8FqpTjiJCaxb rhtl6LeNQv2Lh/yDse/lU3h0ssEdjEahzEiOkDVqMWRQw9IBd8ef4zepcR/ByVqr7aQjfyn6RPa BVa5A38QEjKH1h6kesG/77JXmAkO6jRIlrBlxrw7caI3okRpHzUiLPHqqxL/1dp4zBvzPCj9wkU S0umLjFYVso2d7iw+aZLLaszSJTPXYtHWJsBKoaHS9FFh7T92astYWB228Qb35DzEtGib89XTXr n3Ins5/ur5OC1OeyQ64oOzSHxnSnUwO4p6UT0M2uySgFGDCjNa5bEJ81lFAfwVhruVIBwxpAr4l US+DpnGCUsesa6TXSZhUhRS4RaBhzc+I5HWvBSrVV/67/uV6k6NKDIy/1eZ6BwKTMArXR0QOmEm thu9SIm4FihmmRPsKum15PoH2536I2lLy80AEatCpHozQHrmZk5lUM9KIHbMuRCoip+C3/Al7U8 InvulGMOGpY8iuCIgQw== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-14_03,2026-01-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 phishscore=0 impostorscore=0 bulkscore=0 clxscore=1015 suspectscore=0 priorityscore=1501 malwarescore=0 lowpriorityscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601140093 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey Enabling tailcalls with subprog combinations by referencing method. The actual tailcall count is always maintained in the tail_call_info variable present in the frame of main function (also called entry function). The tail_call_info variables in the frames of all other subprog contains reference to the tail_call_info present in frame of main function. Dynamic resolution interprets the tail_call_info either as value or reference depending on the context of active frame while tailcall is invoked. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit.h | 12 +++++- arch/powerpc/net/bpf_jit_comp.c | 10 ++++- arch/powerpc/net/bpf_jit_comp64.c | 68 +++++++++++++++++++++++-------- 3 files changed, 70 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 45d419c0ee73..5d735bc5e6bd 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -51,6 +51,12 @@ EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc)= ); \ } while (0) =20 +/* Same as PPC_BCC_SHORT, except valid dest is known prior to call. */ +#define PPC_COND_BRANCH(cond, dest) = \ + do { \ + long offset =3D (long)(dest) - CTX_NIA(ctx); \ + EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc)= ); \ + } while (0) /* * Sign-extended 32-bit immediate load * @@ -75,6 +81,8 @@ =20 /* for tailcall counter */ #define BPF_PPC_TAILCALL 8 +/* for gpr non volatile registers BPG_REG_6 to 10 */ +#define BPF_PPC_STACK_SAVE (6*8) =20 /* If dummy pass (!image), account for maximum possible instructions */ #define PPC_LI64(d, i) do { \ @@ -170,6 +178,7 @@ struct codegen_context { unsigned int alt_exit_addr; u64 arena_vm_start; u64 user_vm_start; + bool is_subprog; }; =20 #define bpf_to_ppc(r) (ctx->b2p[r]) @@ -204,11 +213,10 @@ void bpf_jit_build_epilogue(u32 *image, struct codege= n_context *ctx); void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx); void bpf_jit_realloc_regs(struct codegen_context *ctx); int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tm= p_reg, long exit_addr); - int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, in= t pass, struct codegen_context *ctx, int insn_idx, int jmp_off, int dst_reg, u32 code); - +int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx); #endif =20 #endif diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 5e976730b2f5..069a8822c30d 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -206,6 +206,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *f= p) cgctx.stack_size =3D round_up(fp->aux->stack_depth, 16); cgctx.arena_vm_start =3D bpf_arena_get_kern_vm_start(fp->aux->arena); cgctx.user_vm_start =3D bpf_arena_get_user_vm_start(fp->aux->arena); + cgctx.is_subprog =3D bpf_is_subprog(fp); =20 /* Scouting faux-generate pass 0 */ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) { @@ -435,6 +436,11 @@ void bpf_jit_free(struct bpf_prog *fp) bpf_prog_unlock_free(fp); } =20 +bool bpf_jit_supports_subprog_tailcalls(void) +{ + return IS_ENABLED(CONFIG_PPC64); +} + bool bpf_jit_supports_kfunc_call(void) { return true; @@ -604,7 +610,7 @@ static void bpf_trampoline_setup_tail_call_cnt(u32 *ima= ge, struct codegen_contex int func_frame_offset, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { - /* See bpf_jit_stack_tailcallcnt() */ + /* See bpf_jit_stack_tailcallinfo_offset() */ int tailcallcnt_offset =3D 7 * 8; =20 EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); @@ -619,7 +625,7 @@ static void bpf_trampoline_restore_tail_call_cnt(u32 *i= mage, struct codegen_cont int func_frame_offset, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { - /* See bpf_jit_stack_tailcallcnt() */ + /* See bpf_jit_stack_tailcallinfo_offset() */ int tailcallcnt_offset =3D 7 * 8; =20 EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset)); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index 39061cd742c1..cebf81fbd59f 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -26,8 +26,12 @@ * Ensure the top half (upto local_tmp_var) stays consistent * with our redzone usage. * + * tail_call_info - stores tailcall count value in main program's + * frame, stores reference to tail_call_info of + * main's frame in sub-prog's frame. + * * [ prev sp ] <------------- - * [ tail_call_cnt ] 8 | + * [ tail_call_info ] 8 | * [ nv gpr save area ] 6*8 | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | @@ -35,8 +39,6 @@ * sp (r1) ---> [ stack pointer ] -------------- */ =20 -/* for gpr non volatile registers BPG_REG_6 to 10 */ -#define BPF_PPC_STACK_SAVE (6*8) /* for bpf JIT code internal usage */ #define BPF_PPC_STACK_LOCALS 24 /* stack frame excluding BPF stack, ensure this is quadword aligned */ @@ -98,7 +100,7 @@ static inline bool bpf_has_stack_frame(struct codegen_co= ntext *ctx) * [ prev sp ] <------------- * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- - * [ tail_call_cnt ] 8 + * [ tail_call_info ] 8 * [ nv gpr save area ] 6*8 * [ local_tmp_var ] 24 * [ unused red zone ] 224 @@ -114,7 +116,7 @@ static int bpf_jit_stack_local(struct codegen_context *= ctx) } } =20 -static int bpf_jit_stack_tailcallcnt(struct codegen_context *ctx) +int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) { return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SA= VE; } @@ -147,17 +149,32 @@ void bpf_jit_build_prologue(u32 *image, struct codege= n_context *ctx) #endif =20 /* - * Initialize tail_call_cnt if we do tail calls. - * Otherwise, put in NOPs so that it can be skipped when we are - * invoked through a tail call. + * Tail call count(tcc) is saved & updated only in main + * program's frame and the address of tcc in main program's + * frame (tcc_ptr) is saved in subprogs frame. + * + * Offset of tail_call_info on any frame will be interpreted + * as either tcc_ptr or tcc value depending on whether it is + * greater than MAX_TAIL_CALL_CNT or not. */ - if (ctx->seen & SEEN_TAILCALL) { + if (!ctx->is_subprog) { EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0)); /* this goes in the redzone */ EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } else { - EMIT(PPC_RAW_NOP()); - EMIT(PPC_RAW_NOP()); + /* + * if tail_call_info < MAX_TAIL_CALL_CNT + * main prog calling first subprog -> copy reference + * else + * subsequent subprog calling another subprog -> directly copy conte= nt + */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, 0)); + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), -(BPF_PPC_= TAILCALL))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + PPC_COND_BRANCH(COND_GT, CTX_NIA(ctx) + 8); + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), + -(BPF_PPC_TAILCALL))); + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL))); } =20 if (bpf_has_stack_frame(ctx)) { @@ -352,19 +369,38 @@ static int bpf_jit_emit_tail_call(u32 *image, struct = codegen_context *ctx, u32 o EMIT(PPC_RAW_CMPLW(b2p_index, bpf_to_ppc(TMP_REG_1))); PPC_BCC_SHORT(COND_GE, out); =20 + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallinfo_of= fset(ctx))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + PPC_COND_BRANCH(COND_LE, CTX_NIA(ctx) + 8); + + /* dereference TMP_REG_1 */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 0)); + /* - * if (tail_call_cnt >=3D MAX_TAIL_CALL_CNT) + * if (tail_call_info =3D=3D MAX_TAIL_CALL_CNT) * goto out; */ - EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallcnt(ctx= ))); EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); - PPC_BCC_SHORT(COND_GE, out); + PPC_COND_BRANCH(COND_EQ, out); =20 /* - * tail_call_cnt++; + * tail_call_info++; <- Actual value of tcc here */ EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 1)); - EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallcnt(ct= x))); + + /* + * Before writing updated tail_call_info, distinguish if current frame + * is storing a reference to tail_call_info or actual tcc value in + * tail_call_info. + */ + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, bpf_jit_stack_tailcallinfo_of= fset(ctx))); + EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_2), MAX_TAIL_CALL_CNT)); + PPC_COND_BRANCH(COND_GT, CTX_NIA(ctx) + 8); + + /* First get address of tail_call_info */ + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_2), _R1, bpf_jit_stack_tailcallinfo_= offset(ctx))); + /* Writeback updated value to tail_call_info */ + EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), 0)); =20 /* prog =3D array->ptrs[index]; */ EMIT(PPC_RAW_MULI(bpf_to_ppc(TMP_REG_1), b2p_index, 8)); --=20 2.48.1 From nobody Sat Feb 7 22:21:03 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FE26389E1B; Wed, 14 Jan 2026 11:46:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391192; cv=none; b=LhdAw1MRDxRGq1bqlQw925p1ZNF9nrLWUgUSvb5DSXRrqh+OIDpwK7w+TPPuIBv/o2caOZgNq2edArFXHROMIM6JoeKvosRm5nzJoguWY9WsDoAnbTBGkbZTx3/5KNAKzVPwQCSAMfUNvgC8uXqP7MgABS/Gpq811AkvGjbhrJM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391192; c=relaxed/simple; bh=G0piVQb9VJSyvY13aAZfv4i8eKvpKqz5rBImUcWOakY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QfpEz0ssIJ6Rf1/rfWl1EjZC98M0eSeszdPZXeNSYSaXxvNWliIfemsK6wtRbpbNSqSlWGH3ubpkGoR9Jaf+T36OUBspBfDCJwixp0BFKKzHBhfiPQ/DobuSbRYJf83Vk2NNdEDczkgM3zkU2ICxOgCkSRXuMcKvbBQO4Zo7lPc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=qP0P1vxI; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="qP0P1vxI" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60E97NBw002548; Wed, 14 Jan 2026 11:45:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ByuJMcarr0a2cFX1G TvgZGdkfern0JYYktb62wOJqc4=; b=qP0P1vxIHzTOdDF19dEog1lfaiIRH4F9o q8TeMfoouuf5cglSH0wG5UCFemSkbQkvT1GltCY8ndMljy0A5+35S6bO3qCJLLA1 tH/mR6fFlSqV3VFFLNURfkhldFcjFWu12w3JofxnFXgpAnctBoJhPkOQIjTv9JS8 hD0RlFyTkfiubftSdbc0HP3zQXurI/2rVFeEZ5X2Qs++FJTFtsaEY11O2G1dC5+5 mOGhSQNENsVNr57OcznCdHiBVsrK3UIoBJLRZgTtC9M3JZcX3GS9/ATR1ifmL8OX SjI9v6PUBRBXR7T4jh2uKowTNxiQefW4TkdE2JJsl59n7BSb3jmTw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bke93154r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:45:55 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60EBWDP0020741; Wed, 14 Jan 2026 11:45:55 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bke93154m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:45:55 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60E8Y3Pv031269; Wed, 14 Jan 2026 11:45:54 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4bm3t1sfm4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:45:53 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60EBjo7142467824 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Jan 2026 11:45:50 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2572720043; Wed, 14 Jan 2026 11:45:50 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 912BD20040; Wed, 14 Jan 2026 11:45:43 +0000 (GMT) Received: from abhi.. (unknown [9.124.217.65]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 14 Jan 2026 11:45:43 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v2 3/6] powerpc64/bpf: Tailcall handling with trampolines Date: Wed, 14 Jan 2026 17:14:47 +0530 Message-ID: <20260114114450.30405-4-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260114114450.30405-1-adubey@linux.ibm.com> References: <20260114114450.30405-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 4tXRySjWdU3uBBoqa_Q3gm6Xvk9V0hL1 X-Authority-Analysis: v=2.4 cv=dYyNHHXe c=1 sm=1 tr=0 ts=696781f3 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=H7N5BJ4JIw_s2QcKL7IA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTE0MDA5MyBTYWx0ZWRfXyKA8YkaVhvfS ggsszs6fwc/lYLx6gJ0HlxwFixH2csRghsRkXyYUQTu5Ipief6luj8YqEshM8LUoRHEiIjzxCQe ElJktFHnKJ3GUbS+Piua49zY++JKepRYFqSGUCRIeFs3YlqLeXeFhUy7tzz+xmKqznZotsGb+Cg FDgG2pI1vcS+znpm7BRNfBuM83pynfa7CKQNFR+FgL+kqKy4M+e18/B7wEC4H5mGjCloTWVynjx NzaswYPRIsM+/f23jJCawsNUUmd23o5a3uP2C+Jayz8uufND/G900qw6MSH8HCidWtnqFuDCV7M ZMssU+0BZiZACh/xCeOaD7KLmEvXlCkUZ7zd9sVe2pf4cBhRY10f2j1rGdAUEhnI2hgmW1v8TiX COdfZZra9rRJk5cCyopBX5Uo3qFQPFL11jMc14HLDbYz3x3xzNT6xU72f0a3p2U/Zc9bHUdZAJW 2TyWu4YeejRilnNTWwQ== X-Proofpoint-GUID: aAPXFZ5XM8R5C7Daaa3Ir7Dvb0raVItE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-14_03,2026-01-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0 adultscore=0 priorityscore=1501 suspectscore=0 bulkscore=0 phishscore=0 clxscore=1015 lowpriorityscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601140093 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The trampoline mechanism sets up its own stack frame and an additional dummy frame. We need to have additional JIT instructions handling tailcall dereferencing in the trampoline's context. We don't add the two stack frames pointed above, rather add space for tail_call_info at bottom in trampoline frame for ppc64. This makes the trampoline's frame consistent with layout of all other frames wrt tail_call_info offset. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp.c | 83 ++++++++++++++++++++++----------- 1 file changed, 56 insertions(+), 27 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index 069a8822c30d..e3088cf089d1 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -606,33 +606,58 @@ static int invoke_bpf_mod_ret(u32 *image, u32 *ro_ima= ge, struct codegen_context return 0; } =20 -static void bpf_trampoline_setup_tail_call_cnt(u32 *image, struct codegen_= context *ctx, - int func_frame_offset, int r4_off) -{ - if (IS_ENABLED(CONFIG_PPC64)) { - /* See bpf_jit_stack_tailcallinfo_offset() */ - int tailcallcnt_offset =3D 7 * 8; - - EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); - EMIT(PPC_RAW_STL(_R3, _R1, -tailcallcnt_offset)); - } else { - /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ - EMIT(PPC_RAW_LL(_R4, _R1, r4_off)); - } -} +/* + * Refer the label 'Generated stack layout' in this file for actual stack + * layout during trampoline invocation. + * + * Refer __arch_prepare_bpf_trampoline() for stack component details. + * + * The tailcall count/reference is present in caller's stack frame. Its re= quired + * to copy the content of tail_call_info before calling the actual function + * to which the trampoline is attached. + * + */ =20 -static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codege= n_context *ctx, - int func_frame_offset, int r4_off) +static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen= _context *ctx, + int func_frame_offset, + int bpf_dummy_frame_size, int r4_off) { if (IS_ENABLED(CONFIG_PPC64)) { /* See bpf_jit_stack_tailcallinfo_offset() */ - int tailcallcnt_offset =3D 7 * 8; + int tailcallinfo_offset =3D BPF_PPC_TAILCALL; + /* + * func_frame_offset =3D ...(1) + * bpf_dummy_frame_size + trampoline_frame_size + */ + EMIT(PPC_RAW_LD(_R4, _R1, func_frame_offset)); + EMIT(PPC_RAW_LD(_R3, _R4, -tailcallinfo_offset)); + + /* + * Setting the tail_call_info in trampoline's frame + * depending on if previous frame had value or reference. + */ + EMIT(PPC_RAW_CMPLWI(_R3, MAX_TAIL_CALL_CNT)); + PPC_COND_BRANCH(COND_GT, CTX_NIA(ctx) + 8); + EMIT(PPC_RAW_ADDI(_R3, _R4, bpf_jit_stack_tailcallinfo_offset(ctx))); + /* + * From ...(1) above: + * trampoline_frame_bottom =3D ...(2) + * func_frame_offset - bpf_dummy_frame_size + * + * Using ...(2) derived above: + * trampoline_tail_call_info_offset =3D ...(3) + * trampoline_frame_bottom - tailcallinfo_offset + * + * From ...(3): + * Use trampoline_tail_call_info_offset to write reference of main's + * tail_call_info in trampoline frame. + */ + EMIT(PPC_RAW_STL(_R3, _R1, (func_frame_offset - bpf_dummy_frame_size) + - tailcallinfo_offset)); =20 - EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset)); - EMIT(PPC_RAW_STL(_R3, _R1, func_frame_offset - tailcallcnt_offset)); } else { /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ - EMIT(PPC_RAW_STL(_R4, _R1, r4_off)); + EMIT(PPC_RAW_LL(_R4, _R1, r4_off)); } } =20 @@ -720,6 +745,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im * LR save area [ r0 save (64-bit) ] | header * [ r0 save (32-bit) ] | * dummy frame for unwind [ back chain 1 ] -- + * [ tail_call_info ] non optional - 64-b= it powerpc * [ padding ] align stack frame * r4_off [ r4 (tailcallcnt) ] optional - 32-bit p= owerpc * alt_lr_off [ real lr (ool stub)] optional - actual lr @@ -801,8 +827,14 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tr= amp_image *im, void *rw_im } } =20 - /* Padding to align stack frame, if any */ - bpf_frame_size =3D round_up(bpf_frame_size, SZL * 2); + if (!(bpf_frame_size % (2 * SZL))) { + /* Stack is 16-byte aligned */ + /* Room for padding followed by 64-bit tail_call_info */ + bpf_frame_size +=3D SZL + BPF_PPC_TAILCALL; + } else { + /* Room for 64-bit tail_call_info */ + bpf_frame_size +=3D BPF_PPC_TAILCALL; + } =20 /* Dummy frame size for proper unwind - includes 64-bytes red zone for 64= -bit powerpc */ bpf_dummy_frame_size =3D STACK_FRAME_MIN_SIZE + 64; @@ -902,7 +934,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tra= mp_image *im, void *rw_im =20 /* Replicate tail_call_cnt before calling the original BPF prog */ if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) - bpf_trampoline_setup_tail_call_cnt(image, ctx, func_frame_offset, r4_of= f); + bpf_trampoline_setup_tail_call_info(image, ctx, func_frame_offset, + bpf_dummy_frame_size, r4_off); =20 /* Restore args */ bpf_trampoline_restore_args_stack(image, ctx, func_frame_offset, nr_regs= , regs_off); @@ -917,10 +950,6 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tr= amp_image *im, void *rw_im /* Store return value for bpf prog to access */ EMIT(PPC_RAW_STL(_R3, _R1, retval_off)); =20 - /* Restore updated tail_call_cnt */ - if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) - bpf_trampoline_restore_tail_call_cnt(image, ctx, func_frame_offset, r4_= off); - /* Reserve space to patch branch instruction to skip fexit progs */ if (ro_image) /* image is NULL for dummy pass */ im->ip_after_call =3D &((u32 *)ro_image)[ctx->idx]; --=20 2.48.1 From nobody Sat Feb 7 22:21:03 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87A18396B7F; Wed, 14 Jan 2026 11:47:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391279; cv=none; b=HqUF1zfYKjeZmlQ48/EEPVAy42k/SQPVapp0FjvT+YRflcfCeH1PSih2XLzJZ5IdAjnACHIN0jGnk2FDdh0LRBzTkg/PcHpmThEIhT/36Kn1NOlGxalf23t/VuwMg9jLNfen6ozcofkeDj0Ei9o0WGLYJAoaFIXZpvJbKvy4Q68= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391279; c=relaxed/simple; bh=+mVN/1k32p3Q/oqWuuEyICSthLwnkmAMh/wSUMCC16w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hxr6tTtJD7Sz6+bkN8kSLTcVE337w8GENI90rwjl/fMA0YMtxt1UTvlBmUUC14Zv1lbxs6V660wi0n8FmvtnlSR06c+8ROb0zDeKXy+umKveViVLwtDevHA+vg3z7ieRonVuy2fqC6bdcSpX5uD8fa07brPK53N5BiwbY+sE4ZM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=cyFgbgaB; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="cyFgbgaB" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60DMsvIE023555; Wed, 14 Jan 2026 11:46:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=j407n7q4cCyhc5s3H s0Bcd7yRqelf1smKMXBcLjWqAY=; b=cyFgbgaBn3USLLJXlZnEJraRsZfWkuti4 kAN/CR6uAx9gIbCFLfZ0NJqYeP5w2OKwyEO7R6VQ4DSPuh/R1Aqq77sJPIoNFx8D BCMJCHBBtC8o5dQvaaAVd0C36yAf58Qx6IwRfOPeYOWwnYJnLfm9zJkPlPDcawYt z/uRaa98Wo0EPXy32VxKYgDoA+cfIBGWQFRhEZLq/hUemx1jKW7IcfrlRkfQ6jhf ndS/ErJiiB/wdJaWaLPdKT42UgaZ6HwkFPL74FmAVngLSzJng+RAycgRElmpNUEF ateMm+wZRdaVUBcFkAgIE3+WYu4l8Ry+LVZGf6SemcmUGxgk1cVew== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bkd6e8u87-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:46:06 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60EBcH2D005594; Wed, 14 Jan 2026 11:46:06 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bkd6e8u84-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:46:05 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60EAXcj4025809; Wed, 14 Jan 2026 11:46:05 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bm2kkhs9e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:46:05 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60EBk1xu45809954 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Jan 2026 11:46:01 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 021DC20043; Wed, 14 Jan 2026 11:46:01 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 54A7820040; Wed, 14 Jan 2026 11:45:54 +0000 (GMT) Received: from abhi.. (unknown [9.124.217.65]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 14 Jan 2026 11:45:54 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v2 4/6] powerpc64/bpf: Add arch_bpf_stack_walk() for BPF JIT Date: Wed, 14 Jan 2026 17:14:48 +0530 Message-ID: <20260114114450.30405-5-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260114114450.30405-1-adubey@linux.ibm.com> References: <20260114114450.30405-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: W4hPKjBIUSTGUSaemCmJiRVMBSg8nNW_ X-Authority-Analysis: v=2.4 cv=LLxrgZW9 c=1 sm=1 tr=0 ts=696781fe cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=ovcAVf-9gG4EF5hqTukA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTE0MDA5MyBTYWx0ZWRfXyhbIde78SoWS W2wMLVm2FpxNcaokZK+IIx5youxA5OiBrYd6okxwWZPm85L8+gDTDZDOFi7g0Tmpt+Yob6Ly1dz jUSFc4C7GU4hqU7Ekg9QJWC7xRUVj3mCvqGEN10nwBeQ+doZrSy5zcCK3rpXti3gJQXzzowomBg 0v3vAkq6N7dNh9aTb6TDEFQ4giqfETDFrgkTHNNAnXIrFPigEoKocw9URbO0ucmPE0ZvUTyDeU8 ZsB4eW2i8ANdAOL3RkYrXycdVgY9uwGuZ9zLnErXjFV21HMAyP29RIqH/2M95pTgCgWtTRNO3tO eb0f9KlrFb3id2YeHj4EgnosWHKgX33Eq3Bg9VxjbmQZBRqITtI3EohrTYE8Ro1F8y1uGQmOafR f+c/i0h7eqoQBguRTZkae7I81G9y/lGBMGr/aB3DVE2YwPRvOV0eUls9qEsYftgo98+snjpgWSX 21a3oexPIYvgW8IeF2Q== X-Proofpoint-ORIG-GUID: AVX5I48SCigD4bDqptliSEZSYfkJu5GH X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-14_03,2026-01-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 clxscore=1015 spamscore=0 impostorscore=0 malwarescore=0 phishscore=0 adultscore=0 lowpriorityscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601140093 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey This function is used by bpf_throw() to unwind the stack until frame of exception-boundary during BPF exception handling. This function is necessary to support BPF exceptions on PowerPC. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp64.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index cebf81fbd59f..ec58395f74f7 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -247,6 +247,34 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen= _context *ctx) bpf_jit_build_fentry_stubs(image, ctx); } =20 +void arch_bpf_stack_walk(bool (*consume_fn)(void *, u64, u64, u64), void *= cookie) +{ + // callback processing always in current context + unsigned long fp =3D current_stack_frame(); + + for (;;) { + unsigned long *frame =3D (unsigned long *) fp; + unsigned long ip; + + if (!validate_sp(fp, current)) + return; + + ip =3D frame[STACK_FRAME_LR_SAVE]; + if (!ip) + break; + + /* + * consume_fn common code expects stack pointer(sp) in third + * argument. There is no sp in ppc64, rather pass frame + * pointer. + */ + if (ip && !consume_fn(cookie, ip, fp, fp)) + break; + + fp =3D frame[0]; + } +} + int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_con= text *ctx, u64 func) { unsigned long func_addr =3D func ? ppc_function_entry((void *)func) : 0; --=20 2.48.1 From nobody Sat Feb 7 22:21:03 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3ED4395DBE; Wed, 14 Jan 2026 11:46:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391210; cv=none; b=E4/Is6yufSPsrVh2DEb7LuRdk9R5vmbMMwyjb9Bm7Zhj4TmTFNYNyjnUSV7BnYd/ZK4Wu8utY5a/beDixyNqPGHtI6yTrbbEsLnyXkNru352/NoqRgJZL+9zoIQXglx4F95aEEp/t/FaioIeuaCurcLjAgvoA3TsgzcsDti8r4E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391210; c=relaxed/simple; bh=tQZNFgrbg2PtUm7ISJ2VVRAYxlrVEz8pWBRoCM4lntk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=W89XnJ96tpwQsMp5R6TaJGtsi6LfXUdFxJyK1N1tqA4meVW2HC26GYhQvH0FsKNfmTIpgrqVVZd6BBs7rKfKbQVoR4bPPEK9PDLdzodGM3dp0zR2aFe+nK6c5eAvagxMSicMSiEdvq99mu9l3jmTNsDJU/0Yn4P2zBGdkIoRmfc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=lTT2E0Ym; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="lTT2E0Ym" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60EBVIFw009441; Wed, 14 Jan 2026 11:46:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=aXJUn3bNZD9fOETwc zZ/IV1t0kDAXX4Muw0AQio7os4=; b=lTT2E0YmAM/iOpFUA42h/oorTWU+K10D5 5u7CBO1rDajGULasgPTZ0zr+CFZJeEktvFOsc1RTKH2R7Z+OeT5IPHQaZqlsU/J+ XSQYM07S8aG8IMh18xM4nKiA4iG04RaZiODQ2bSh0+CZn8dyk9suGQp28DPbJ2J8 DPoE36VgaOlLxanh2ki5oDtft9v0dCf0+YNCnVMlmtuL5RFYb2PotsbjXi7dKfa2 VQ0Uhw61FMlIgb+ydoN7DD5zdeirrB/hzKpG+V6upyxiSKP86DIxbFM5qveZEBGo thJ30N18Z9XvXk/icC+LVGxetzpsx8a/e9ID5suGTydyfPwddCgmg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bke93155t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:46:17 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60EBWDP2020741; Wed, 14 Jan 2026 11:46:17 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bke93155q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:46:17 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60EBMTux025961; Wed, 14 Jan 2026 11:46:15 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bm23n9ra2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:46:15 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60EBkBUq46072088 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Jan 2026 11:46:11 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3C25E20040; Wed, 14 Jan 2026 11:46:11 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AB4B520043; Wed, 14 Jan 2026 11:46:04 +0000 (GMT) Received: from abhi.. (unknown [9.124.217.65]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 14 Jan 2026 11:46:04 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v2 5/6] powerpc64/bpf: Support exceptions Date: Wed, 14 Jan 2026 17:14:49 +0530 Message-ID: <20260114114450.30405-6-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260114114450.30405-1-adubey@linux.ibm.com> References: <20260114114450.30405-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: Zt7Vyr5IxOTsYF3YS_KghMXvGQFRnPuJ X-Authority-Analysis: v=2.4 cv=dYyNHHXe c=1 sm=1 tr=0 ts=69678209 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=DU0EyFXtAAAA:8 a=VnNF1IyMAAAA:8 a=8rlfmSvwrT77M3U47S8A:9 a=UCR5be5CC-YrbG9FbbB0:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTE0MDA5MyBTYWx0ZWRfX9216ceHfLNTy f2q9BP5brJFX1ng4HTTga5/7fd8UuEVNs/evVTdD6cNNyz4BKRooftukmA95roVZIZP0QO3EzC0 aYkkHQ2wc1hCNVx/7lltVXShDgSImtUN4HbzUDUG8WdFb76NSO3in+ciBq3sI26BIjm2hpDtUTB MH9FP35PVrLJ/7IXGiUw94U5HY2cx3kSNsqX9phRKHQRuEoq6hGqGSmaHKq1jH7rjfLQhBzgsrm /LPoZEi6sllTrhP/2Nhkp3FsRiwhMK5FVj1dxcYUscn4GhqmnKth92EaAQm85vV4Pu5xEoPqNeO PHS+DjfSCLCckAO6PlMA69u9lTNDo9Dk/QtdZJtKHNopxAqg6fV8claIqY3ANO85dO+fa0Zep3L R2r7tKW/pYye5/jB7yO78mOIw1cmyvgBVa4VGQEdASel8X9L8P5UVczf/ttcW+j6lvocfYAYwgZ Of673mxUYFhLLpcij1A== X-Proofpoint-GUID: mX_nWFvyyTcU6KkGbHjUDlMtSJl3gZNE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-14_03,2026-01-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0 adultscore=0 priorityscore=1501 suspectscore=0 bulkscore=0 phishscore=0 clxscore=1015 lowpriorityscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601140093 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The modified prologue/epilogue generation code now enables exception-callback to use the stack frame of the program marked as exception boundary, where callee saved registers are stored. As per ppc64 ABIv2 documentation[1], r14-r31 are callee saved registers. BPF programs on ppc64 already saves r26-r31 registers. Saving the remaining set of callee saved registers(r14-r25) is handled in the next patch. [1] https://ftp.rtems.org/pub/rtems/people/sebh/ABI64BitOpenPOWERv1.1_16Jul= y2015_pub.pdf Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit.h | 2 ++ arch/powerpc/net/bpf_jit_comp.c | 7 ++++ arch/powerpc/net/bpf_jit_comp64.c | 53 +++++++++++++++++++++---------- 3 files changed, 45 insertions(+), 17 deletions(-) diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 5d735bc5e6bd..fb548ae5d143 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -179,6 +179,8 @@ struct codegen_context { u64 arena_vm_start; u64 user_vm_start; bool is_subprog; + bool exception_boundary; + bool exception_cb; }; =20 #define bpf_to_ppc(r) (ctx->b2p[r]) diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_com= p.c index e3088cf089d1..26991940d36e 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -207,6 +207,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *f= p) cgctx.arena_vm_start =3D bpf_arena_get_kern_vm_start(fp->aux->arena); cgctx.user_vm_start =3D bpf_arena_get_user_vm_start(fp->aux->arena); cgctx.is_subprog =3D bpf_is_subprog(fp); + cgctx.exception_boundary =3D fp->aux->exception_boundary; + cgctx.exception_cb =3D fp->aux->exception_cb; =20 /* Scouting faux-generate pass 0 */ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) { @@ -436,6 +438,11 @@ void bpf_jit_free(struct bpf_prog *fp) bpf_prog_unlock_free(fp); } =20 +bool bpf_jit_supports_exceptions(void) +{ + return IS_ENABLED(CONFIG_PPC64); +} + bool bpf_jit_supports_subprog_tailcalls(void) { return IS_ENABLED(CONFIG_PPC64); diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index ec58395f74f7..a6083dd9786c 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -89,7 +89,9 @@ static inline bool bpf_has_stack_frame(struct codegen_con= text *ctx) * - the bpf program uses its stack area * The latter condition is deduced from the usage of BPF_REG_FP */ - return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, bpf_to_ppc(BPF_= REG_FP)); + return ctx->seen & SEEN_FUNC || + bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)) || + ctx->exception_cb; } =20 /* @@ -190,23 +192,32 @@ void bpf_jit_build_prologue(u32 *image, struct codege= n_context *ctx) EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size))); } =20 - /* - * Back up non-volatile regs -- BPF registers 6-10 - * If we haven't created our own stack frame, we save these - * in the protected zone below the previous stack frame - */ - for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) - if (bpf_is_seen_register(ctx, bpf_to_ppc(i))) - EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to= _ppc(i)))); + if (!ctx->exception_cb) { + /* + * Back up non-volatile regs -- BPF registers 6-10 + * If we haven't created our own stack frame, we save these + * in the protected zone below the previous stack frame + */ + for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) + if (ctx->exception_boundary || bpf_is_seen_register(ctx, bpf_to_ppc(i))) + EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, + bpf_jit_stack_offsetof(ctx, bpf_to_ppc(i)))); =20 - if (ctx->arena_vm_start) - EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1, + if (ctx->exception_boundary || ctx->arena_vm_start) + EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 - /* Setup frame pointer to point to the bpf stack area */ - if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP))) - EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1, + /* Setup frame pointer to point to the bpf stack area */ + if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP))) + EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1, STACK_FRAME_MIN_SIZE + ctx->stack_size)); + } else { + /* + * Exception callback receives Frame Pointer of main + * program as third arg + */ + EMIT(PPC_RAW_MR(_R1, _R5)); + } =20 if (ctx->arena_vm_start) PPC_LI64(bpf_to_ppc(ARENA_VM_START), ctx->arena_vm_start); @@ -218,17 +229,25 @@ static void bpf_jit_emit_common_epilogue(u32 *image, = struct codegen_context *ctx =20 /* Restore NVRs */ for (i =3D BPF_REG_6; i <=3D BPF_REG_10; i++) - if (bpf_is_seen_register(ctx, bpf_to_ppc(i))) + if (ctx->exception_cb || bpf_is_seen_register(ctx, bpf_to_ppc(i))) EMIT(PPC_RAW_LD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_= ppc(i)))); =20 - if (ctx->arena_vm_start) + if (ctx->exception_cb || ctx->arena_vm_start) EMIT(PPC_RAW_LD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 + if (ctx->exception_cb) { + /* + * LR value from boundary-frame is received as second parameter + * in exception callback. + */ + EMIT(PPC_RAW_MTLR(_R4)); + } + /* Tear down our stack frame */ if (bpf_has_stack_frame(ctx)) { EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size)); - if (ctx->seen & SEEN_FUNC) { + if (ctx->seen & SEEN_FUNC || ctx->exception_cb) { EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF)); EMIT(PPC_RAW_MTLR(_R0)); } --=20 2.48.1 From nobody Sat Feb 7 22:21:03 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5367399A4F; Wed, 14 Jan 2026 11:48:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391294; cv=none; b=be7182PWqcz9SLg+qSPVT21OC/FB7exCQXrruJ6zinF7UfC1HJEBZ4kdVBCjuO94Btl5wkGDJSMeWCqy17bF8s6x+/1p0rH3faNozO0Xj2042Z3mu6OuRv6CaadtwTpgFsNYKGNzKOGsWPZ/aK3DOVGtLcL1pfprBmd8ZNEd1Io= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768391294; c=relaxed/simple; bh=VgU0z7sN9/IwFuRXqpQn+y0dKGAvt44/dIl+fF/SZxE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dkP3J+ElNwhcHvirBMYAVZDiSPwBwzEPrqZ4dw4VAlO3TQueD5OVLqHpEEQQ00jDzTokooHmAg/liCZMx/jS+7+u4Noh1kmTTxORMZIC3fR8PNnwOQPhfkhnQLao4exr25X/IfGfTRgAjR7pFtm5ouhECal7ur4UO84P0kunpKE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Z70Lk7pl; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Z70Lk7pl" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60E3EkWG027978; Wed, 14 Jan 2026 11:46:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=/+4EsIgqtPyteIyyv wZeUaR3Ld9Je3XCr84TTqvedvQ=; b=Z70Lk7plYxb+RCTiLxZ1BcTzZA7IEUEfw SulQMTheJX+u2xDSCXAYGowgETrk70IsokxiTNzttm6i8rQKJKqoT8WaVjkhfTNn 6hdhhoAv7DXcxaP1bdK1ENLuxKU2LGJwIsYZR4tBd7jM10ri0OlC524pJbRwJG8s u1fzshYIvftZMxp9ET0AghpVnrXspMl2I30mbw7k1Fd1QU2kTHkvwezwg+tB5Lv2 XHM/ZH7wsqW5Ltoe28yc9j2ybIIGmWWVNJGN/Qe5pzMWpe4SCwYFf70sJhC8Oxpw m8yf8Hvb0r+eKkEpwt4/ot0LzxxZ9INy4baT1MidGpjV0ZWTkhl7A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bkc6h8xk8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:46:32 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60EBkWmv012664; Wed, 14 Jan 2026 11:46:32 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bkc6h8xk3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:46:32 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60EAmaKW025877; Wed, 14 Jan 2026 11:46:28 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bm2kkhsb8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Jan 2026 11:46:28 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60EBkOPS28115610 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Jan 2026 11:46:24 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 66A2C20043; Wed, 14 Jan 2026 11:46:24 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 194C920040; Wed, 14 Jan 2026 11:46:18 +0000 (GMT) Received: from abhi.. (unknown [9.124.217.65]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 14 Jan 2026 11:46:17 +0000 (GMT) From: adubey@linux.ibm.com To: bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Cc: hbathini@linux.ibm.com, sachinpb@linux.ibm.com, venkat88@linux.ibm.com, andrii@kernel.org, eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, christophe.leroy@csgroup.eu, naveen@kernel.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, memxor@gmail.com, iii@linux.ibm.com, shuah@kernel.org, Abhishek Dubey Subject: [PATCH v2 6/6] powerpc64/bpf: Additional NVR handling for bpf_throw Date: Wed, 14 Jan 2026 17:14:50 +0530 Message-ID: <20260114114450.30405-7-adubey@linux.ibm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260114114450.30405-1-adubey@linux.ibm.com> References: <20260114114450.30405-1-adubey@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 6JGKqhTsQQV73jaKPlA3g_liPYpb3I-p X-Proofpoint-ORIG-GUID: AiIBpe9NOnSwn7VhFh1r6A0TWPxAGobS X-Authority-Analysis: v=2.4 cv=TaibdBQh c=1 sm=1 tr=0 ts=69678218 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=2yuXeg4sgMS8061G0UsA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTE0MDA5OCBTYWx0ZWRfXwAirCVMD28l5 0Hzm3HZnR0koNk2GPehBaQaxIZZtT/nsws70Gj6IfYTjbdNtGD+TnZf6CM4uIPzQnh7a2X/CsXQ Y3jlcy5eKBgJ363tpRhhcIgHipxoWEGCb9btKvbUUj9x8kIhI3CIYsCSkfJLs/Ge/I4na8iAqlS tPSAdH6ozj3VeTjlQtc0rrbCgvLGcws5I4sBqoKpSCjn8SyAcV1IGoA9PESOF/p08ZvsgYJ2yen tbpmvXTlEzNNskfXAOVD/nYkJktIklIbDanU6sVHltpRaPmDys0WiAdSiqLPwq2beZxm2ut/kk9 C1zfD7b+1yf334ykCVWYXbHtyoarIq7/V+ANamD7FP/cZcagbbZIDXDVAFiV/nhLv1FDqS3hsji 4Grnyp8A8itcZeApP2iJq9Eres3pBqQhZZK9FddxSQA4gBF0qnJgKE0xGoN4Fsx9bB12I1lPJ64 WR8oqVG84ErG66y34Nw== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-14_03,2026-01-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 phishscore=0 impostorscore=0 bulkscore=0 clxscore=1015 suspectscore=0 priorityscore=1501 malwarescore=0 lowpriorityscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601140098 Content-Type: text/plain; charset="utf-8" From: Abhishek Dubey The bpf_throw() function never returns, if it has clobbered any callee-saved register, those will remain clobbered. The prologue must take care of saving all callee-saved registers in the frame of exception boundary program. Later these additional non volatile registers R14-R25 along with other NVRs are restored back in the epilogue of exception callback. To achieve above objective the frame size is determined dynamically to accommodate additional non volatile registers in exception boundary's frame. For non-exception boundary program, the frame size remains optimal. The additional instructions to save & restore r14-r25 registers are emitted only during exception boundary and exception callback respectively. Signed-off-by: Abhishek Dubey --- arch/powerpc/net/bpf_jit_comp64.c | 70 +++++++++++++++++++++++++++---- 1 file changed, 63 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_c= omp64.c index a6083dd9786c..941e0818c9ec 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -32,21 +32,37 @@ * * [ prev sp ] <------------- * [ tail_call_info ] 8 | - * [ nv gpr save area ] 6*8 | + * [ nv gpr save area ] 6*8 + (12*8) | * [ local_tmp_var ] 24 | * fp (r31) --> [ ebpf stack space ] upto 512 | * [ frame header ] 32/112 | * sp (r1) ---> [ stack pointer ] -------------- + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ =20 /* for bpf JIT code internal usage */ #define BPF_PPC_STACK_LOCALS 24 +/* + * for additional non volatile registers(r14-r25) to be saved + * at exception boundary + */ +#define BPF_PPC_EXC_STACK_SAVE (12*8) + /* stack frame excluding BPF stack, ensure this is quadword aligned */ #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ BPF_PPC_STACK_LOCALS + \ BPF_PPC_STACK_SAVE + \ BPF_PPC_TAILCALL) =20 +/* + * same as BPF_PPC_STACKFRAME with save area for additional + * non volatile registers saved at exception boundary. + * This is quad-word aligned. + */ +#define BPF_PPC_EXC_STACKFRAME (BPF_PPC_STACKFRAME + BPF_PPC_EXC_STACK_SAV= E) + /* BPF register usage */ #define TMP_REG_1 (MAX_BPF_JIT_REG + 0) #define TMP_REG_2 (MAX_BPF_JIT_REG + 1) @@ -103,9 +119,12 @@ static inline bool bpf_has_stack_frame(struct codegen_= context *ctx) * [ ... ] | * sp (r1) ---> [ stack pointer ] -------------- * [ tail_call_info ] 8 - * [ nv gpr save area ] 6*8 + * [ nv gpr save area ] 6*8 + (12*8) * [ local_tmp_var ] 24 * [ unused red zone ] 224 + * + * Additional (12*8) in 'nv gpr save area' only in case of + * exception boundary. */ static int bpf_jit_stack_local(struct codegen_context *ctx) { @@ -114,7 +133,11 @@ static int bpf_jit_stack_local(struct codegen_context = *ctx) return STACK_FRAME_MIN_SIZE + ctx->stack_size; } else { /* Stack layout 2 */ - return -(BPF_PPC_TAILCALL + BPF_PPC_STACK_SAVE + BPF_PPC_STACK_LOCALS); + return -(BPF_PPC_TAILCALL + + BPF_PPC_STACK_SAVE + + (ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACK_SAVE:0) + + BPF_PPC_STACK_LOCALS); } } =20 @@ -125,9 +148,19 @@ int bpf_jit_stack_tailcallinfo_offset(struct codegen_c= ontext *ctx) =20 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg) { - if (reg >=3D BPF_PPC_NVR_MIN && reg < 32) + int min_valid_nvreg =3D BPF_PPC_NVR_MIN; + /* Default frame size for all cases except exception boundary */ + int frame_nvr_size =3D BPF_PPC_STACKFRAME; + + /* Consider all nv regs for handling exceptions */ + if (ctx->exception_boundary || ctx->exception_cb) { + min_valid_nvreg =3D _R14; + frame_nvr_size =3D BPF_PPC_EXC_STACKFRAME; + } + + if (reg >=3D min_valid_nvreg && reg < 32) return (bpf_has_stack_frame(ctx) ? - (BPF_PPC_STACKFRAME + ctx->stack_size) : 0) + (frame_nvr_size + ctx->stack_size) : 0) - (8 * (32 - reg)) - BPF_PPC_TAILCALL; =20 pr_err("BPF JIT is asking about unknown registers"); @@ -189,7 +222,20 @@ void bpf_jit_build_prologue(u32 *image, struct codegen= _context *ctx) EMIT(PPC_RAW_STD(_R0, _R1, PPC_LR_STKOFF)); } =20 - EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size))); + int stack_expand =3D ctx->exception_boundary || ctx->exception_cb ? + BPF_PPC_EXC_STACKFRAME : BPF_PPC_STACKFRAME; + EMIT(PPC_RAW_STDU(_R1, _R1, -(stack_expand + ctx->stack_size))); + } + + /* + * Program acting as exception boundary pushes R14..R25 in addition to + * BPF callee-saved non volatile registers. Exception callback uses + * the boundary program's stack frame, recover additionally saved + * registers in epilogue of exception callback. + */ + if (ctx->exception_boundary) { + for (i =3D _R14; i <=3D _R25; i++) + EMIT(PPC_RAW_STD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); } =20 if (!ctx->exception_cb) { @@ -237,6 +283,13 @@ static void bpf_jit_emit_common_epilogue(u32 *image, s= truct codegen_context *ctx bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); =20 if (ctx->exception_cb) { + /* + * Recover additionally saved non volatile registers from stack + * frame of exception boundary program. + */ + for (i =3D _R14; i <=3D _R25; i++) + EMIT(PPC_RAW_LD(i, _R1, bpf_jit_stack_offsetof(ctx, i))); + /* * LR value from boundary-frame is received as second parameter * in exception callback. @@ -246,7 +299,10 @@ static void bpf_jit_emit_common_epilogue(u32 *image, s= truct codegen_context *ctx =20 /* Tear down our stack frame */ if (bpf_has_stack_frame(ctx)) { - EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size)); + int stack_shrink =3D ctx->exception_cb || ctx->exception_boundary ? + BPF_PPC_EXC_STACKFRAME : BPF_PPC_STACKFRAME; + EMIT(PPC_RAW_ADDI(_R1, _R1, stack_shrink + ctx->stack_size)); + if (ctx->seen & SEEN_FUNC || ctx->exception_cb) { EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF)); EMIT(PPC_RAW_MTLR(_R0)); --=20 2.48.1