From nobody Thu Apr 2 15:41:44 2026 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 294833F23A2 for ; Fri, 27 Mar 2026 14:16:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774620986; cv=none; b=j1E8Luc54ODDLa7nRiN8zSToK5Uag0lJGJUgsXPNY7TiAEJQi3c3DnHzEhiUB19yb6HXa6p75pqSGyIgWH3PNSetfAP3wIh1/qHC0sp4v/HYUnTyAPwZvYxm8wzsHZT+1baajgrPWgsbsOSit1S9ltoalcivbJJmGiQ3LtU0Zd4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774620986; c=relaxed/simple; bh=mCB+uW4a761n0d2aVwv1tBGBLpbO8o3tzqc2Q88DMSk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=APt/6Wx7jk5OGzohK6WrPlotasLnEkPO7n2XCO2QIo7OjTgOWO9PpxkzW2oqSuVuS0obP1MryGKYSN5q3ajhdreaoUmKA3XTsYes2oAidE5jvizFFNBMBA1+tC2/Yai9rXIj2qax3vzNsasa84BOIqlr0A7PTWEy6USWAKtu5e8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=U7PXC/Ch; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="U7PXC/Ch" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2a9296b3926so14229775ad.1 for ; Fri, 27 Mar 2026 07:16:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774620982; x=1775225782; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CI8/ynCoj9QXIX8q6/wyTK1N5QESD0qh826RNi2z3ds=; b=U7PXC/Ch3zV8dvp/VxQPIvRhLHyFzL/oy+H9Aej02b05/i4Ye+anDKZN2SxcldSVRU aleJIqVhpyr66fmWWxiRSVujS6r7hE4B8TQ6ifc6ZQhDzn7ueYIM/TEEM8HCyrVN6v4W UI8l63ac1s/bxmIuuqTmcy6U8EfABI6k4fV15MFMaqvIzC1OkcI6RzQ+TmzjK7WAc2Zg On3F83godTBHtduKMqrdCVSCV5a0lOE76BroXoLPHIfP2wH55lYKfoaNgxgms9YS3eEe an5xjMqSvhEPCP9F4/JytFG3HtPBTXVAjrudTkDHrIxAI/O+ftDFocT0EwtTpY9kuKRJ p3ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774620982; x=1775225782; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CI8/ynCoj9QXIX8q6/wyTK1N5QESD0qh826RNi2z3ds=; b=M6xFrBJcZlxWyhvdWKr8AjGXHh31lk6+gcm7WGEURtQsnzz28ibakIBrPg352Cqeq4 rpSv8VB26Y8J/dq8DZxgABa3pRtzINiAv1N/HrU0l3ze/djulKGbkqulqwNay+4hE4vO RD2go8W+j1ILo0dtJOX/gJV5ZihT+wy87BkdzxfAZlk8MFnwLbE51vw8/Q1AAhmse9VZ lQC7ZpNqLxllzUNK1jAPlTTOerVJH1BJ37rGfEHxCQ09mmK6xbk+bRS+endF/bJX9AE/ LZi7H4Q1sIj9ePAOzy59u8Cq13wFsk4Wko59O37c00KVlZaLy4pVT56vNQ5/DxwzKhKl KRig== X-Forwarded-Encrypted: i=1; AJvYcCUXucz3Ysw4WkC14rWk/THmOW0Zk94SgbZdItLnINJcT1njii4uoUXgS66mIX14z0/eXXyv0qi+UrKZkds=@vger.kernel.org X-Gm-Message-State: AOJu0YzJmS/0Slk2Cw1ACqyml6TYFC1NWcFvtCX2L+p/a/Dm9+CJs+mu 9xUfrRZ6SQBltzMhgkv4VCZoa4GAt1j29pG4WARypPwEVNZHW7iQBGLS X-Gm-Gg: ATEYQzzr+xrze6Jte+LECDwEb7NhI8kZrJt62WAcUX7kTpsLCMYXoHET6mutJ/IfiiY 2bX4i0XMhY3UKIG6bvSkCgyP9uhVMORvSvFKrnyprAaek9ZP4PMT76h9BhkuY89dgphnOX+M+uq wMm4640o8G42G5hrwsPOGEhh3fBsD+rFT2FjSQgH/OapJcH2VczEewkzj8O4HiTDhm4XROMV+2n MzfPEtVjpcHhSj+LlvStQJg6GBnRdeB2OcHv3XbSaERgcHaqwpi6MFAnN8JAQCTTim9dyzDmdYj imsEic4OAql5r58w8bsT+GMgUpqwYghUKHqppyyUowPEA4xsinaa5ANDwYjds+PQCSVefT3o8aL gEMmfj3KZAKCtS9+rtnLLtuJM3odES2MqQqtrPhTEXFKwm1/MvhhTeXphC71OMTFvaa/1be/gXN GcOTCz54fgazZoQ08wi608bGIeFQfciiR8bCKPhrQBDQCl8zv+N6pQBsO2laRor4VM9W6642KK/ 1uq0w8ecTxhx96SAEt5RjYYPhyVxfp9+Q== X-Received: by 2002:a17:902:ccc7:b0:2b0:615f:9c2b with SMTP id d9443c01a7336-2b0cdd3fb5amr29434635ad.24.1774620982187; Fri, 27 Mar 2026 07:16:22 -0700 (PDT) Received: from lab-kiba-ocxma-dut-01.. (191.68.231.218.rev.ocx2915.net. [218.231.68.191]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2b0bc8e958csm61371365ad.66.2026.03.27.07.16.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Mar 2026 07:16:21 -0700 (PDT) From: Takeru Hayasaka To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org Cc: bpf@vger.kernel.org, x86@kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next 1/2] bpf, x86: patch tail-call fentry slot on non-IBT JITs Date: Fri, 27 Mar 2026 14:16:01 +0000 Message-ID: <20260327141616.1961457-2-hayatake396@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260327141616.1961457-1-hayatake396@gmail.com> References: <20260327141616.1961457-1-hayatake396@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" x86 tail-call fentry patching mirrors CALL text pokes to the tail-call landing slot. The helper that locates that mirrored slot assumes an ENDBR-prefixed landing, which works on IBT JITs but fails on non-IBT JITs where the landing starts directly with the 5-byte patch slot. As a result, the regular entry gets patched but the tail-call landing remains NOP5, so fentry never fires for tail-called programs on non-IBT kernels. Anchor the lookup on the landing address, verify the short-jump layout first, and only check ENDBR when one is actually emitted. Signed-off-by: Takeru Hayasaka --- arch/x86/net/bpf_jit_comp.c | 47 ++++++++++++++++++++++++++++++++++--- 1 file changed, 44 insertions(+), 3 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index e9b78040d703..fe5fd37f65d8 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -325,8 +325,10 @@ struct jit_context { =20 /* Number of bytes emit_patch() needs to generate instructions */ #define X86_PATCH_SIZE 5 +/* Number of bytes used by the short jump that skips the tail-call hook. */ +#define X86_TAIL_CALL_SKIP_JMP_SIZE 2 /* Number of bytes that will be skipped on tailcall */ -#define X86_TAIL_CALL_OFFSET (12 + ENDBR_INSN_SIZE) +#define X86_TAIL_CALL_OFFSET (12 + X86_TAIL_CALL_SKIP_JMP_SIZE + ENDBR_INS= N_SIZE) =20 static void push_r9(u8 **pprog) { @@ -545,8 +547,15 @@ static void emit_prologue(u8 **pprog, u8 *ip, u32 stac= k_depth, bool ebpf_from_cb EMIT3(0x48, 0x89, 0xE5); /* mov rbp, rsp */ } =20 + if (!is_subprog) { + /* Normal entry skips the tail-call-only trampoline hook. */ + EMIT2(0xEB, ENDBR_INSN_SIZE + X86_PATCH_SIZE); + } + /* X86_TAIL_CALL_OFFSET is here */ EMIT_ENDBR(); + if (!is_subprog) + emit_nops(&prog, X86_PATCH_SIZE); =20 /* sub rsp, rounded_stack_depth */ if (stack_depth) @@ -632,12 +641,33 @@ static int __bpf_arch_text_poke(void *ip, enum bpf_te= xt_poke_type old_t, return ret; } =20 +static void *bpf_tail_call_fentry_ip(void *ip) +{ + u8 *tail_ip =3D ip + X86_TAIL_CALL_OFFSET; + u8 *landing =3D tail_ip - ENDBR_INSN_SIZE; + + /* ip points at the regular fentry slot after the entry ENDBR. */ + if (landing[-X86_TAIL_CALL_SKIP_JMP_SIZE] !=3D 0xEB || + landing[-X86_TAIL_CALL_SKIP_JMP_SIZE + 1] !=3D + ENDBR_INSN_SIZE + X86_PATCH_SIZE) + return NULL; + + if (ENDBR_INSN_SIZE && !is_endbr((u32 *)landing)) + return NULL; + + return tail_ip; +} + int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t, enum bpf_text_poke_type new_t, void *old_addr, void *new_addr) { + void *tail_ip =3D NULL; + bool is_bpf_text =3D is_bpf_text_address((long)ip); + int ret, tail_ret; + if (!is_kernel_text((long)ip) && - !is_bpf_text_address((long)ip)) + !is_bpf_text) /* BPF poking in modules is not supported */ return -EINVAL; =20 @@ -648,7 +678,18 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_ty= pe old_t, if (is_endbr(ip)) ip +=3D ENDBR_INSN_SIZE; =20 - return __bpf_arch_text_poke(ip, old_t, new_t, old_addr, new_addr); + if (is_bpf_text && (old_t =3D=3D BPF_MOD_CALL || new_t =3D=3D BPF_MOD_CAL= L)) + tail_ip =3D bpf_tail_call_fentry_ip(ip); + + ret =3D __bpf_arch_text_poke(ip, old_t, new_t, old_addr, new_addr); + if (ret < 0 || !tail_ip) + return ret; + + tail_ret =3D __bpf_arch_text_poke(tail_ip, old_t, new_t, old_addr, new_ad= dr); + if (tail_ret < 0) + return tail_ret; + + return ret && tail_ret; } =20 #define EMIT_LFENCE() EMIT3(0x0F, 0xAE, 0xE8) --=20 2.43.0 From nobody Thu Apr 2 15:41:44 2026 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B38EF3F23D1 for ; Fri, 27 Mar 2026 14:16:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774620989; cv=none; b=ELc2K9rnCMYM91fcDHYdPyqtdrbiI2xewEDf8KjggZGK19UUQw29jgnecRn+I1NniKsMz5qltevMPtR8RVXlyokTUbgvg4A8ipCTfkNSHch5BKtq3A1oze+5+hX68m6xo52Av84iKqfaF8JfbZ0TDmoq6d4hR9AXovey2RRmmE8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774620989; c=relaxed/simple; bh=nk/5ZakMJ+t2bnACkG2W+99kDbR1MO4cQq0DjDs269o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=S+m0Sue45GweJnJ7ACmPgPo5V+788TSDW0INe5ejTKn2rw+XIztNofe45IO87LsCdKUSidItnSSQrTn38n/ZKNsDuc20l0xdVwKNSEhJF0bAKGaCnG2yBK43Pv21/dHUC/cjY2axHiTW+pi8mPYU22xjrpCwyqBtCVz61dobOoA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bBVk2nHT; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bBVk2nHT" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2b062069f80so12798545ad.0 for ; Fri, 27 Mar 2026 07:16:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774620985; x=1775225785; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6aupF+DuW1/DcLr6AsgbIe6z0Vi4ttYW8cK5LaEjq3w=; b=bBVk2nHTC40dS0AyeaC0MCoeEtN5Bf+tnn3sfNcMhXGXO+XfJqN4Fn6KVL8sdrWkwC gnnaYgu8oQomOxYvq4SaUXaCF81lut0Z3ewvBFNCR9QOEsoWc4bFbcZkc6AXV4GCeM7L 9HCOrLEk1gQOimME7WZbhZjWU7PbOkCTXwp5iqfK7P62rs+yFbeHHpNTC/d4/ZpYfrhq OdnzUPmTOh+opu9hcBmCR+6NnMySZv4SwX77hB2xtwQAmeIjjUMmZd81DuhtZlYEzjhM e2iwu0Ow3v53ROZbJ7vafnm89nTFBOspkLGHzHw+4kRXsOHanJqSPox+GqsJLqTtgr/C 0NIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774620985; x=1775225785; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=6aupF+DuW1/DcLr6AsgbIe6z0Vi4ttYW8cK5LaEjq3w=; b=ZYJpMmPjreoglCkBvb2xzjnpiTtxIUcGqsAWNuZ/woT1orpqxePaENoHAk78MAQHkF rEkMx68qya61ELXbSqxE2Um2aSz/nZ2MVFKmet/ql0eNlcC/2c4X3UTTqQ+BOBTSIKp3 K0i9rtTvJNlO7vNfLvyUhnB/pbREsv739X9VTArajik4QBiXp12Aggeldc3DMFUHxlPS Z1hs2BIlEf8fX3mGl6t60HAwPgOPq2TfCz68N+Lvhj6Yrp3VQ/1SL63jLTRfMBTBEB2e /Kv7C2JCjXbcwtFgg+yfWpUP+23+TXLDXmAYS8h8lTcliqFfZU3Lh06e9yb2xhKB570U tcdQ== X-Forwarded-Encrypted: i=1; AJvYcCVBpmcQNm3Rjvu50BxjYD4MBq6oqfQ8U4mEQdiRD7xyPseTyu4ljvJMOM+4CJu1Q4UQOLvI5gGOFuBWgqY=@vger.kernel.org X-Gm-Message-State: AOJu0YwWn3evWc+77OgoDIXVf4GXSm5o9VQnKiDpKduBkgDUENzAbB00 +xB4qHl2AUVBd53ZHjrSzmRHi2xXL2X0sntPKbHRNwLiASj53cIT0rr3 X-Gm-Gg: ATEYQzznbwx0PB199IcGtQN36WOIQ61cZzoBfXJx4PWPeyWNNAgVBxLFq1nboD3uFNT jFniHqcD0CHHQupxUiSchB6Txjs1A9SMf5kcb4ZU98yU8drKkyWdTVr0NNw3ct6usJR0Cv8kH6m ZlXUenEZvBa2Z11yzOdaCVDsByEP7f3IPP0YehoHCh74cXnK+hJNwQFun/DqKk3OFSJcgYRbWHn NkbygbSecRCckH3OH7/B5QJgUN1jJ7QQHi6UMFCZhoO+R4cgl3SrYX8v4pemFN18/6xg9NgMZ61 sjpUMt5RXA/eOPyb7/bzF2nsf3uOIYRZtzHtAhkitIo2YlSDZqpyobbRwKsQ8T9Jq2BIAuPwegK 5lXJGUNWYQGD8DjHltVDV8/7LlR6jOHUCxYSFy+QxuzFIccPxJM+yX0WeGY0EYmPMMPFP7bM3iP 8mcHAMMQi/AR1D+QFKjiPNtRUB94VNSqv5PzUuwhOf+xIoyWE1S0uW+AicEVKrRoVNixDXnuMj/ OLTzJPPrlywfdHO7vuR+HNZmEvVkm29/ww6NoK1QnX5 X-Received: by 2002:a17:902:ce06:b0:2b0:5a4c:7266 with SMTP id d9443c01a7336-2b0cdc29022mr30209125ad.16.1774620984287; Fri, 27 Mar 2026 07:16:24 -0700 (PDT) Received: from lab-kiba-ocxma-dut-01.. (191.68.231.218.rev.ocx2915.net. [218.231.68.191]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2b0bc8e958csm61371365ad.66.2026.03.27.07.16.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Mar 2026 07:16:23 -0700 (PDT) From: Takeru Hayasaka To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org Cc: bpf@vger.kernel.org, x86@kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next 2/2] selftests/bpf: cover fentry on tailcalled programs Date: Fri, 27 Mar 2026 14:16:02 +0000 Message-ID: <20260327141616.1961457-3-hayatake396@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260327141616.1961457-1-hayatake396@gmail.com> References: <20260327141616.1961457-1-hayatake396@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a small tailcall target/probe pair and a tailcalls subtest that attaches fentry to the callee program. The test first runs the callee directly and checks that fentry fires exactly once, then runs the entry program, tail-calls into the callee and checks that the same fentry also fires exactly once on the tail-call path. This covers both sides of the x86 change: direct entry must not double-fire, while tail-called entry must now be observable. Signed-off-by: Takeru Hayasaka --- .../selftests/bpf/prog_tests/tailcalls.c | 110 ++++++++++++++++++ .../bpf/progs/tailcall_fentry_probe.c | 16 +++ .../bpf/progs/tailcall_fentry_target.c | 27 +++++ 3 files changed, 153 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/tailcall_fentry_probe= .c create mode 100644 tools/testing/selftests/bpf/progs/tailcall_fentry_targe= t.c diff --git a/tools/testing/selftests/bpf/prog_tests/tailcalls.c b/tools/tes= ting/selftests/bpf/prog_tests/tailcalls.c index 7d534fde0af9..ac05df65b666 100644 --- a/tools/testing/selftests/bpf/prog_tests/tailcalls.c +++ b/tools/testing/selftests/bpf/prog_tests/tailcalls.c @@ -1113,6 +1113,114 @@ static void test_tailcall_bpf2bpf_fentry_entry(void) bpf_object__close(tgt_obj); } =20 +static void test_tailcall_fentry_tailcallee(void) +{ + struct bpf_object *tgt_obj =3D NULL, *fentry_obj =3D NULL; + struct bpf_map *prog_array, *data_map; + struct bpf_link *fentry_link =3D NULL; + struct bpf_program *prog; + int err, map_fd, callee_fd, main_fd, data_fd, i, val; + char buff[128] =3D {}; + + LIBBPF_OPTS(bpf_test_run_opts, topts, + .data_in =3D buff, + .data_size_in =3D sizeof(buff), + .repeat =3D 1, + ); + + err =3D bpf_prog_test_load("tailcall_fentry_target.bpf.o", + BPF_PROG_TYPE_SCHED_CLS, + &tgt_obj, &main_fd); + if (!ASSERT_OK(err, "load tgt_obj")) + return; + + prog_array =3D bpf_object__find_map_by_name(tgt_obj, "jmp_table"); + if (!ASSERT_OK_PTR(prog_array, "find jmp_table map")) + goto out; + + map_fd =3D bpf_map__fd(prog_array); + if (!ASSERT_FALSE(map_fd < 0, "find jmp_table map fd")) + goto out; + + prog =3D bpf_object__find_program_by_name(tgt_obj, "entry"); + if (!ASSERT_OK_PTR(prog, "find entry prog")) + goto out; + + main_fd =3D bpf_program__fd(prog); + if (!ASSERT_FALSE(main_fd < 0, "find entry prog fd")) + goto out; + + prog =3D bpf_object__find_program_by_name(tgt_obj, "callee"); + if (!ASSERT_OK_PTR(prog, "find callee prog")) + goto out; + + callee_fd =3D bpf_program__fd(prog); + if (!ASSERT_FALSE(callee_fd < 0, "find callee prog fd")) + goto out; + + i =3D 0; + err =3D bpf_map_update_elem(map_fd, &i, &callee_fd, BPF_ANY); + if (!ASSERT_OK(err, "update jmp_table")) + goto out; + + fentry_obj =3D bpf_object__open_file("tailcall_fentry_probe.bpf.o", NULL); + if (!ASSERT_OK_PTR(fentry_obj, "open fentry_obj file")) + goto out; + + prog =3D bpf_object__find_program_by_name(fentry_obj, "fentry_callee"); + if (!ASSERT_OK_PTR(prog, "find fentry prog")) + goto out; + + err =3D bpf_program__set_attach_target(prog, callee_fd, "callee"); + if (!ASSERT_OK(err, "set_attach_target callee")) + goto out; + + err =3D bpf_object__load(fentry_obj); + if (!ASSERT_OK(err, "load fentry_obj")) + goto out; + + fentry_link =3D bpf_program__attach_trace(prog); + if (!ASSERT_OK_PTR(fentry_link, "attach_trace")) + goto out; + + data_map =3D bpf_object__find_map_by_name(fentry_obj, ".bss"); + if (!ASSERT_FALSE(!data_map || !bpf_map__is_internal(data_map), + "find tailcall_fentry_probe.bss map")) + goto out; + + data_fd =3D bpf_map__fd(data_map); + if (!ASSERT_FALSE(data_fd < 0, + "find tailcall_fentry_probe.bss map fd")) + goto out; + + err =3D bpf_prog_test_run_opts(callee_fd, &topts); + ASSERT_OK(err, "direct callee"); + ASSERT_EQ(topts.retval, 7, "direct callee retval"); + + i =3D 0; + err =3D bpf_map_lookup_elem(data_fd, &i, &val); + ASSERT_OK(err, "direct fentry count"); + ASSERT_EQ(val, 1, "direct fentry count"); + + val =3D 0; + err =3D bpf_map_update_elem(data_fd, &i, &val, BPF_ANY); + ASSERT_OK(err, "reset fentry count"); + + err =3D bpf_prog_test_run_opts(main_fd, &topts); + ASSERT_OK(err, "tailcall"); + ASSERT_EQ(topts.retval, 7, "tailcall retval"); + + i =3D 0; + err =3D bpf_map_lookup_elem(data_fd, &i, &val); + ASSERT_OK(err, "fentry count"); + ASSERT_EQ(val, 1, "fentry count"); + +out: + bpf_link__destroy(fentry_link); + bpf_object__close(fentry_obj); + bpf_object__close(tgt_obj); +} + #define JMP_TABLE "/sys/fs/bpf/jmp_table" =20 static int poke_thread_exit; @@ -1759,6 +1867,8 @@ void test_tailcalls(void) test_tailcall_bpf2bpf_fentry_fexit(); if (test__start_subtest("tailcall_bpf2bpf_fentry_entry")) test_tailcall_bpf2bpf_fentry_entry(); + if (test__start_subtest("tailcall_fentry_tailcallee")) + test_tailcall_fentry_tailcallee(); if (test__start_subtest("tailcall_poke")) test_tailcall_poke(); if (test__start_subtest("tailcall_bpf2bpf_hierarchy_1")) diff --git a/tools/testing/selftests/bpf/progs/tailcall_fentry_probe.c b/to= ols/testing/selftests/bpf/progs/tailcall_fentry_probe.c new file mode 100644 index 000000000000..b784aeffb316 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/tailcall_fentry_probe.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" + +#include +#include + +int count =3D 0; + +SEC("fentry/callee") +int BPF_PROG(fentry_callee, struct sk_buff *skb) +{ + count++; + return 0; +} + +char _license[] SEC("license") =3D "GPL"; diff --git a/tools/testing/selftests/bpf/progs/tailcall_fentry_target.c b/t= ools/testing/selftests/bpf/progs/tailcall_fentry_target.c new file mode 100644 index 000000000000..46da06ce323c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/tailcall_fentry_target.c @@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0 +#include + +#include +#include "bpf_legacy.h" + +struct { + __uint(type, BPF_MAP_TYPE_PROG_ARRAY); + __uint(max_entries, 1); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(__u32)); +} jmp_table SEC(".maps"); + +SEC("tc") +int callee(struct __sk_buff *skb) +{ + return 7; +} + +SEC("tc") +int entry(struct __sk_buff *skb) +{ + bpf_tail_call_static(skb, &jmp_table, 0); + return 0; +} + +char __license[] SEC("license") =3D "GPL"; --=20 2.43.0