From nobody Sun Apr 5 20:00:08 2026 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0AE92283CBF; Fri, 3 Apr 2026 13:49:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775224174; cv=none; b=PBFth9hlCvvgQdFlAORw4vtal3KbqDKqNLsDuaVoNFBCo/6oUuWEgVyrBAvJR9i5iUDDZ2Xdup/X4SqANVV6+4BtKtutBtewdHPns/lgNxqOBLrCrz8D5gtQ7Z58q6+lBpTlgOI88/o2uPuPXQB1/HZp9XpmUvq/8Hzk4yWHYLo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775224174; c=relaxed/simple; bh=EGvj1WLvIVtzwmCmgE4aiPiNY2kUopCEEbO6tRNSmkg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cVvC/+aA8FIY+pjBl48F+uJYJBOLj/Ynzt6AHSnMkqtkig9E4iTRtW7ce0+TkrrF9IY3C6AFMvfHbT8E5DQd32RgjXihzY6fwNVTxiEAyP41IxPv/yDa3+f6USB3Li28ma63lcqEki66CF6QpSdmCVnoLQjkQU7pLEQApUUOFvs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.170]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4fnKnH0tYXzKHMWD; Fri, 3 Apr 2026 21:48:35 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.252]) by mail.maildlp.com (Postfix) with ESMTP id F0CD440570; Fri, 3 Apr 2026 21:49:27 +0800 (CST) Received: from localhost.huawei.com (unknown [10.67.174.243]) by APP3 (Coremail) with SMTP id _Ch0CgAnB1Rjxc9pPbfoDA--.34698S6; Fri, 03 Apr 2026 21:49:27 +0800 (CST) From: Xu Kuohai To: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Yonghong Song , Puranjay Mohan , Anton Protopopov , =?UTF-8?q?Alexis=20Lothor=C3=A9?= , Shahab Vahedi , Russell King , Tiezhu Yang , Hengqi Chen , Johan Almbladh , Paul Burton , Hari Bathini , Christophe Leroy , Naveen N Rao , Luke Nelson , Xi Wang , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Pu Lehui , Ilya Leoshkevich , Heiko Carstens , Vasily Gorbik , "David S . Miller" , Wang YanQing Subject: [PATCH bpf-next v12 4/5] bpf, x86: Emit ENDBR for indirect jump targets Date: Fri, 3 Apr 2026 13:28:10 +0000 Message-ID: <20260403132811.753894-5-xukuohai@huaweicloud.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260403132811.753894-1-xukuohai@huaweicloud.com> References: <20260403132811.753894-1-xukuohai@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _Ch0CgAnB1Rjxc9pPbfoDA--.34698S6 X-Coremail-Antispam: 1UD129KBjvJXoWxZr4DXF1DAFW8Jw13WFW3KFg_yoWrtw48pa 9xJ3sIyrZ8WFs8KF1kXF47CFy2yF4qgryfGr4rt3yruw42grnYgF1Yka4FqFyYkrWrKrn7 XFyjkr1DWw1kurDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPvb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUAV Cq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0 rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJwA2z4x0Y4vE2Ix0cI8IcVCY1x0267 AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E 14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7 xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Y z7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lFIxGxcIEc7CjxVA2Y2ka0xkIwI1lc7CjxVAaw2 AFwI0_GFv_Wryl42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAq x4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r4a6r W5MIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6xkF 7I0E14v26r4UJVWxJr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14 v26r4j6F4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuY vjxUI-eODUUUU X-CM-SenderInfo: 50xn30hkdlqx5xdzvxpfor3voofrz/ Content-Type: text/plain; charset="utf-8" From: Xu Kuohai On CPUs that support CET/IBT, the indirect jump selftest triggers a kernel panic because the indirect jump targets lack ENDBR instructions. To fix it, emit an ENDBR instruction to each indirect jump target. Since the ENDBR instruction shifts the position of original jited instructions, fix the instruction address calculation wherever the addresses are used. For reference, below is a sample panic log. Missing ENDBR: bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1 ------------[ cut here ]------------ kernel BUG at arch/x86/kernel/cet.c:133! Oops: invalid opcode: 0000 [#1] SMP NOPTI ... ? 0xffffffffc00fb258 ? bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1 bpf_prog_test_run_syscall+0x110/0x2f0 ? fdget+0xba/0xe0 __sys_bpf+0xe4b/0x2590 ? __kmalloc_node_track_caller_noprof+0x1c7/0x680 ? bpf_prog_test_run_syscall+0x215/0x2f0 __x64_sys_bpf+0x21/0x30 do_syscall_64+0x85/0x620 ? bpf_prog_test_run_syscall+0x1e2/0x2f0 Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps") Reviewed-by: Anton Protopopov Acked-by: Leon Hwang Signed-off-by: Xu Kuohai Reviewed-by: Emil Tsalapatis --- arch/x86/net/bpf_jit_comp.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 72d9a5faa230..ea9e707e8abf 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -58,8 +58,8 @@ static u8 *emit_code(u8 *ptr, u32 bytes, unsigned int len) #define EMIT_ENDBR() EMIT(gen_endbr(), 4) #define EMIT_ENDBR_POISON() EMIT(gen_endbr_poison(), 4) #else -#define EMIT_ENDBR() -#define EMIT_ENDBR_POISON() +#define EMIT_ENDBR() do { } while (0) +#define EMIT_ENDBR_POISON() do { } while (0) #endif =20 static bool is_imm8(int value) @@ -1649,8 +1649,8 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *i= p, return 0; } =20 -static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw= _image, - int oldproglen, struct jit_context *ctx, bool jmp_padding) +static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog,= int *addrs, u8 *image, + u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_paddin= g) { bool tail_call_reachable =3D bpf_prog->aux->tail_call_reachable; struct bpf_insn *insn =3D bpf_prog->insnsi; @@ -1663,7 +1663,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *add= rs, u8 *image, u8 *rw_image void __percpu *priv_stack_ptr; int i, excnt =3D 0; int ilen, proglen =3D 0; - u8 *prog =3D temp; + u8 *ip, *prog =3D temp; u32 stack_depth; int err; =20 @@ -1734,6 +1734,11 @@ static int do_jit(struct bpf_prog *bpf_prog, int *ad= drs, u8 *image, u8 *rw_image dst_reg =3D X86_REG_R9; } =20 + if (bpf_insn_is_indirect_target(env, bpf_prog, i - 1)) + EMIT_ENDBR(); + + ip =3D image + addrs[i - 1] + (prog - temp); + switch (insn->code) { /* ALU */ case BPF_ALU | BPF_ADD | BPF_X: @@ -2440,8 +2445,6 @@ st: if (is_imm8(insn->off)) =20 /* call */ case BPF_JMP | BPF_CALL: { - u8 *ip =3D image + addrs[i - 1]; - func =3D (u8 *) __bpf_call_base + imm32; if (src_reg =3D=3D BPF_PSEUDO_CALL && tail_call_reachable) { LOAD_TAIL_CALL_CNT_PTR(stack_depth); @@ -2465,7 +2468,8 @@ st: if (is_imm8(insn->off)) if (imm32) emit_bpf_tail_call_direct(bpf_prog, &bpf_prog->aux->poke_tab[imm32 - 1], - &prog, image + addrs[i - 1], + &prog, + ip, callee_regs_used, stack_depth, ctx); @@ -2474,7 +2478,7 @@ st: if (is_imm8(insn->off)) &prog, callee_regs_used, stack_depth, - image + addrs[i - 1], + ip, ctx); break; =20 @@ -2639,7 +2643,7 @@ st: if (is_imm8(insn->off)) break; =20 case BPF_JMP | BPF_JA | BPF_X: - emit_indirect_jump(&prog, insn->dst_reg, image + addrs[i - 1]); + emit_indirect_jump(&prog, insn->dst_reg, ip); break; case BPF_JMP | BPF_JA: case BPF_JMP32 | BPF_JA: @@ -2729,8 +2733,6 @@ st: if (is_imm8(insn->off)) ctx->cleanup_addr =3D proglen; if (bpf_prog_was_classic(bpf_prog) && !ns_capable_noaudit(&init_user_ns, CAP_SYS_ADMIN)) { - u8 *ip =3D image + addrs[i - 1]; - if (emit_spectre_bhb_barrier(&prog, ip, bpf_prog)) return -EINVAL; } @@ -3791,7 +3793,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verif= ier_env *env, struct bpf_pr for (pass =3D 0; pass < MAX_PASSES || image; pass++) { if (!padding && pass >=3D PADDING_PASSES) padding =3D true; - proglen =3D do_jit(prog, addrs, image, rw_image, oldproglen, &ctx, paddi= ng); + proglen =3D do_jit(env, prog, addrs, image, rw_image, oldproglen, &ctx, = padding); if (proglen <=3D 0) { out_image: image =3D NULL; --=20 2.43.0