From nobody Sun Feb 8 09:17:06 2026 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 345073816EF for ; Tue, 20 Jan 2026 07:06:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768892788; cv=none; b=kG3wWeRbgt/+jSYIlv1dH2vOVL+ecVDrgTzHoBLXQ3CCqiuk3tuHU7RE7gL55CSXt8cyqto7oL6s2W6HAR1y1sU8CVzNqcd0Dw/97k/xC+mhHmia07MfixPKqcP9VllNidTGkNmRDs/H/AWqfwNx346PfSy8ozZW/u5PV1YMqgU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768892788; c=relaxed/simple; bh=bkEnpN35ROgLb1TaATahFCU+nYmJUOdqiUA5VNDzRiY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dYJ1NSgyBqegDm79dqJGiFB2DiQ2BECsecISoesipGUxyvv99kZTe2vZwUAK4Gav3ma5pr/WbF8rk42VBH9+5MMVezL8/rT6cZDP3OkPrEuTzbowsXt/RuckYNj+8fteEEt/LfD3/2xU09tfX+uxaPMk+0R1TY/1n2XOTDHZYgg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BIpbwsK8; arc=none smtp.client-ip=209.85.214.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BIpbwsK8" Received: by mail-pl1-f193.google.com with SMTP id d9443c01a7336-2a09757004cso44953255ad.3 for ; Mon, 19 Jan 2026 23:06:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768892778; x=1769497578; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AoyD4uV9eCRwWiRSuwU/fXn0oAa1lUGxNCw5PqJhjBU=; b=BIpbwsK8IFYvCZ5UM6BF2qr/i7osYi3MA2+M0lEvSqHQZMB7VpEkGSrGTrqxQPMsVt xffOdLinNu8OWRhyJ02aZ2eDeS/nugwYSGWlw4KLjNsfPG8vynnNzWi66SIHDvK5V/GD /2qipx/jpeol5mIg1oe05eSfOmtJ45bAC/z/VQ8Gp0HSo6uDxpTIxoatzullIsxzb+0y tNRiF1E/KG/FwyJcNl2ZYpkv0fWN5hiYmV70w5qznyUqmZebvIb0OVjvsCVzPJkWLWeU 7BWhuIjM7tT//vqyvXofo4qckw0DTktmhTr//XRKFRttrFkIDvbAEOFRL//2jm1/HV3e q23g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768892778; x=1769497578; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=AoyD4uV9eCRwWiRSuwU/fXn0oAa1lUGxNCw5PqJhjBU=; b=ly8lmpVxiyUCFs1BCKR70EyiUZjfE1C2ZCNIRKD66VvLGePsq0VzUx4Xa//up3d6ox cMj1tyUgm/3uKND2wq+8UZ6WBtOEadIyep7F6hc8TWyZTEtMG0hNoIcc34sz3Xbe85su rOxUNReBONWxXVlNp6UWq1vCe1gQJo8cu3Lf4KZIJ++/EEXXQGok3LUR5tTVi+cGUYXd D4hlQuWLshLjq4Q3zXNntK8hpq4989l/JxomKzTPLmIuVg5U+npGCvF8k/U2mgi0m51T LMTV4pf77riI817i5Dn+T7H/Zsr9GUWImS5vPBJSoIpCVQ+q1IfZbGIWyDVe1oM7iO2S 3Jog== X-Forwarded-Encrypted: i=1; AJvYcCXRvukVoQM8NM5D+vjQPO90oIHf7KmzF4hr13OrMvxkoKdZfmhVGFBOCG20rviqXyluuT8jI8RQvg+jmko=@vger.kernel.org X-Gm-Message-State: AOJu0YwYgHECUQF6BA5wd6B5XquXudv722E2klo+E8085589IcqVRc+O aaIPRQ/BDAb5EIhoM9peto4oLGwGBs1RcbGxMrAcF0hGfFnKy2mRdlDj X-Gm-Gg: AZuq6aLzNCBQcQ6yaiACV9++zgldjL3XTqsiAgKY8JxXR8ROPDxF1CE5zitJC9geerc lw5HATLS0pjP9ifJeFsxwCwmIe47dawSt1lkb1EFvBZbdM94zgd3Misq20T3Ls8LouHk9qh0B3d yOFpSlm492cjUfa1/tL5xEj+6gcpg6BydIORLNW8Oqg6B8vvZzQE4ILrKakb132uVIQthMNENKm st/KX7pSH5JQx6om3JAGoXeeNdfuBF7xX9k66t/7/BUi6LJ+Xrni4OEndRT6A8agl0xwT4aPR/U Bcalr5I51c1llga7q2LcHfTtUTwpqFze2BXflXLW945PnAVoP7GhTW6zRxvt0YEN2Nl7vXU1QXz i8/6P1N2WvWwva+UbAaw3+3CnjVQ857EGMZNKy0Q766nsYBDNu0yONe+opzNJ/Nn5nRGF4jatr3 qRhq3EDV3M X-Received: by 2002:a17:903:1a2e:b0:298:6a79:397b with SMTP id d9443c01a7336-2a7176cc35bmr123268025ad.56.1768892778110; Mon, 19 Jan 2026 23:06:18 -0800 (PST) Received: from 7950hx ([103.173.155.241]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a7190ce534sm111695665ad.27.2026.01.19.23.06.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Jan 2026 23:06:17 -0800 (PST) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, eddyz87@gmail.com Cc: davem@davemloft.net, dsahern@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v6 1/2] bpf, x86: inline bpf_get_current_task() for x86_64 Date: Tue, 20 Jan 2026 15:05:54 +0800 Message-ID: <20260120070555.233486-2-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260120070555.233486-1-dongml2@chinatelecom.cn> References: <20260120070555.233486-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 to obtain better performance. Signed-off-by: Menglong Dong Acked-by: Eduard Zingerman --- v5: - don't support the !CONFIG_SMP case v4: - handle the !CONFIG_SMP case v3: - implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in x86_64 JIT. --- kernel/bpf/verifier.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 9de0ec0c3ed9..c4e2ffadfb1f 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -17739,6 +17739,10 @@ static bool verifier_inlines_helper_call(struct bp= f_verifier_env *env, s32 imm) switch (imm) { #ifdef CONFIG_X86_64 case BPF_FUNC_get_smp_processor_id: +#ifdef CONFIG_SMP + case BPF_FUNC_get_current_task_btf: + case BPF_FUNC_get_current_task: +#endif return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); #endif default: @@ -23319,6 +23323,24 @@ static int do_misc_fixups(struct bpf_verifier_env = *env) insn =3D new_prog->insnsi + i + delta; goto next_insn; } + + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inlin= e. */ + if ((insn->imm =3D=3D BPF_FUNC_get_current_task || insn->imm =3D=3D BPF_= FUNC_get_current_task_btf) && + verifier_inlines_helper_call(env, insn->imm)) { + insn_buf[0] =3D BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_t= ask); + insn_buf[1] =3D BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0); + insn_buf[2] =3D BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); + cnt =3D 3; + + new_prog =3D bpf_patch_insn_data(env, i + delta, insn_buf, cnt); + if (!new_prog) + return -ENOMEM; + + delta +=3D cnt - 1; + env->prog =3D prog =3D new_prog; + insn =3D new_prog->insnsi + i + delta; + goto next_insn; + } #endif /* Implement bpf_get_func_arg inline. */ if (prog_type =3D=3D BPF_PROG_TYPE_TRACING && --=20 2.52.0 From nobody Sun Feb 8 09:17:06 2026 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F8EC38A2B4 for ; Tue, 20 Jan 2026 07:06:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.196 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768892797; cv=none; b=kqbBsvAa1V0WljZfk3ilczFSTTl3Msh7fNIzSaHZthuZE+YLEXWNmO6OEEb+UxOSV6n7Vxm9U52BLYMZdVfVSEvLAEJEpvGXn9QVnJQuamTkKSmaAVCQkPpCInThRfT4s/vQoxn1+FUPQ+E0CDnZ4ZDfQIXaXxAIJG75puc7VKA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768892797; c=relaxed/simple; bh=N2XC72YuwbVooj49gSUC23yr6OLWGEsBRVB0n5dcYww=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BHfXkBMVT1R1YFGUWDdfGjgtf3kYn1Rz7s7cp383vFTvA1n6snlbUPAiGj7nE5NWyVpyxCu1BEIhDFEFFdnySIrBAils8qRrc1UauY5gZw65see5dRiIS9mLQzuqYllQDkGjmX/LDBf/OFoPYjOifWEYTcHOj3zsmOunlyE27Ms= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mjTn3U7R; arc=none smtp.client-ip=209.85.214.196 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mjTn3U7R" Received: by mail-pl1-f196.google.com with SMTP id d9443c01a7336-2a77c1d5c3bso168905ad.0 for ; Mon, 19 Jan 2026 23:06:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768892787; x=1769497587; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=h8e96Wx1n0qpGbVjK3mBfOV2yV5DDoKmdYehuYIVGJg=; b=mjTn3U7R+YpbF4E4Uwyr2sfKTXoqfgg5d5jg4DI9l2NI245ny0GbuiRF7Cjo6IbKjo tES1nUVdTFWq4JMrC5q6EoL4B47I1S4FZ2Iu9oJDXTVL3k+juLmQ9V2Pu0B/3h4XUtMz idsKnU2a3Tb4Z2nb6eKlWwA4Z1sEWHCv3z6VfjBYJWdmJfHTr1Zi/xEpB+6dtsHgXy3n SBnXGJyhDDqISt4Co3atOsA4tnnEsJFQ1TcPA5Xkgaw7wgDSApXOBz/BtIiJCOU0oB0y hjllQf6aSG9DFXX8+gsxS9yQzQzVGq5FhS5x5nsMUfrCOGAa4TMf68f5VdjlREkS2x8x kF+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768892787; x=1769497587; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=h8e96Wx1n0qpGbVjK3mBfOV2yV5DDoKmdYehuYIVGJg=; b=jsqdp8KSXjgYvWdXxlsL4Kh296roW+0IiledVr0+gfTJhWK3Y/WEA5fMY/nYoBuKtK ysEF+HMAyeQ/8PlgOiHnP43IMavg2P8slA1JveHdh8op8mg+0/A+m+FYM3OnY5KLT+WV fcA2tXWvxoBbDDTHzQwEsUgkk40uwMHVywAhj/XxZqOAtS+EUg9Uucno7u/+/TiVTrzi 3yHzXLwxLZx1krYEM/l2YXAoi9hVOnhAsSn7Ma4mbEKtrRUprHZgDRbh6ikBvlRYMnH7 y/ha0b1PXvfythnSa1Aa32wg1b9shSBv4h0QfiXxWEzY7LOkBPUvsQp/KWAThAoOISXi Ak9A== X-Forwarded-Encrypted: i=1; AJvYcCUO7zheWJAZTFd78UDB4AcBFNJDb3/SHn2XPFy8olkISZEdrt5vNXGJ7yF0BtGOrJ39495WWa9f+QCPdOQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzGZYoSCuolT/zQ0iF2/FvuSnip4GtgJtFbFqQbAiXCnp171Kkv Z65Ht0yVhGqvjjR4wH7ZHf5uVEhbexzuUuZCYh8H0o3llKar0+SQxmx4 X-Gm-Gg: AZuq6aKckip4gTvZimnQH5qHAj4LsWTUXqYjpyu968FITgYSap9rY3vQi8sUZWJ3JOI 6VjJnP8iG+dc+ptXsBlua+3g0i/GxA2tuUGaxR0wStI4K2WiMbgWkt5py4bSyKOnRBbC7pnpvP4 wFjoOvUme3DOjpTO9XwspmgEf0g4EAxtSbDWPXQWQxCrCvdBKTB/UdXPrZgCwwwgA1+LxvHcTwZ Bf9/ZAo54wqPngmo+Aw7zBl/jkpwZuSrk4OLNAMJLb/Mbiyx+XB7EttE/756KuAT3nwIJJnTo1/ M+2j4vbY7jCbZLxGIjKiqCQ5C1clV+zR45No10HQ8oxtMUm/6xzM7finoTbpVNPnrMwNxmIdumu lhj96Oog7LatgQCWF1YYNleSM4lDT6sPGVonGEHoC/yN5ZybFdcJphOx+ZPb0uMnlhLR/cBXWIJ 5h589mbHxC X-Received: by 2002:a17:902:da88:b0:2a0:d364:983b with SMTP id d9443c01a7336-2a76b39df9dmr7779905ad.60.1768892786963; Mon, 19 Jan 2026 23:06:26 -0800 (PST) Received: from 7950hx ([103.173.155.241]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a7190ce534sm111695665ad.27.2026.01.19.23.06.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Jan 2026 23:06:26 -0800 (PST) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, eddyz87@gmail.com Cc: davem@davemloft.net, dsahern@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v6 2/2] selftests/bpf: test the jited inline of bpf_get_current_task Date: Tue, 20 Jan 2026 15:05:55 +0800 Message-ID: <20260120070555.233486-3-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260120070555.233486-1-dongml2@chinatelecom.cn> References: <20260120070555.233486-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add the testcase for the jited inline of bpf_get_current_task(). Signed-off-by: Menglong Dong Acked-by: Eduard Zingerman --- v6: * remove unnecessary 'ifdef' and __description --- .../selftests/bpf/prog_tests/verifier.c | 2 ++ .../selftests/bpf/progs/verifier_jit_inline.c | 20 +++++++++++++++++++ 2 files changed, 22 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/verifier_jit_inline.c diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/test= ing/selftests/bpf/prog_tests/verifier.c index 38c5ba70100c..2ae7b096bd64 100644 --- a/tools/testing/selftests/bpf/prog_tests/verifier.c +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c @@ -111,6 +111,7 @@ #include "verifier_xdp_direct_packet_access.skel.h" #include "verifier_bits_iter.skel.h" #include "verifier_lsm.skel.h" +#include "verifier_jit_inline.skel.h" #include "irq.skel.h" =20 #define MAX_ENTRIES 11 @@ -253,6 +254,7 @@ void test_verifier_bits_iter(void) { RUN(verifier_bits_= iter); } void test_verifier_lsm(void) { RUN(verifier_lsm); } void test_irq(void) { RUN(irq); } void test_verifier_mtu(void) { RUN(verifier_mtu); } +void test_verifier_jit_inline(void) { RUN(verifier_jit_inlin= e); } =20 static int init_test_val_map(struct bpf_object *obj, char *map_name) { diff --git a/tools/testing/selftests/bpf/progs/verifier_jit_inline.c b/tool= s/testing/selftests/bpf/progs/verifier_jit_inline.c new file mode 100644 index 000000000000..4ea254063646 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include "bpf_misc.h" + +SEC("fentry/bpf_fentry_test1") +__success __retval(0) +__arch_x86_64 +__jited(" addq %gs:{{.*}}, %rax") +__arch_arm64 +__jited(" mrs x7, SP_EL0") +int inline_bpf_get_current_task(void) +{ + bpf_get_current_task(); + + return 0; +} + +char _license[] SEC("license") =3D "GPL"; --=20 2.52.0