From nobody Wed Feb 11 13:28:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDBCEC77B7F for ; Mon, 8 May 2023 16:47:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233237AbjEHQrD (ORCPT ); Mon, 8 May 2023 12:47:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229464AbjEHQrB (ORCPT ); Mon, 8 May 2023 12:47:01 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74B3E420C for ; Mon, 8 May 2023 09:46:57 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-307664010fdso4190416f8f.0 for ; Mon, 08 May 2023 09:46:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1683564416; x=1686156416; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=5cVAaSqZcwrY3lv18J8fx0bI9NmqBYTO127bLBIYsww=; b=DCFLZgJnzXzm0FAcaXt6WCQsUg/rqY4EEq95pI+rlLe01PXHEWk0Q21Y/rEyZ57P2H Y7HcZHK8Z3i7dbT48uTqAjqtwrm/Wslv79y1AtpOQ+A13bzypcquD+ijrlgpHjZiFV5h 6CQF0MH1ajACIiPQXp3JzCuvuiXmxfLXsgd6I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683564416; x=1686156416; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=5cVAaSqZcwrY3lv18J8fx0bI9NmqBYTO127bLBIYsww=; b=WBBuaDy9x2MM426TlocM5Xb7l7Fb4x7V5Pbm4AiFtQ2WEAPiq0kkb/IzM+Ls72eeQM a83icyc6UXwrKvt5/GW1oFE1N7aVRZdSTxttJQgKK4O/nGVccLUkLC3Afm9i4xwltMGk 8vv1aJxu761NO2b6q5DiCYVl/jg3o1xcQk8Wz0Ie4tXCcLaBQMMzlN6E/Fxf/wyNejXO Yi6yEBEMA6cc8gbhmMDM+r+8kL3rFtvR8YrWr6N62XxElQvXFZP2nwIX6XwYnJiyJQHt 2kbg+ntexVGc+xfR53nnLCfWZatqZhVJmTcXMF9yH1Laa0pPpVAPCSKZTQ2ISQDf4xbD t+3w== X-Gm-Message-State: AC+VfDwx2pYYza4nwcmhrG29tHcsYdw3+NhCUpC0PKbZ7R7qSACJfMbr klthLY7u0fQRc4Qyg3cGZzkPD79yrwRyMNMwGL0= X-Google-Smtp-Source: ACHHUZ6z/144coiyfJCABFVRO65PSS8c91Hiqx5GQz2BFQLtTLi01LSze14EWbuvk/blQzBF+ZSD0Q== X-Received: by 2002:a5d:62c1:0:b0:307:8548:f793 with SMTP id o1-20020a5d62c1000000b003078548f793mr6560202wrv.53.1683564415896; Mon, 08 May 2023 09:46:55 -0700 (PDT) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:8c6a:3ec9:317:bd3]) by smtp.gmail.com with ESMTPSA id a6-20020a056000050600b003079c402762sm2250702wrf.19.2023.05.08.09.46.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 May 2023 09:46:55 -0700 (PDT) From: Florent Revest To: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, mark.rutland@arm.com, xukuohai@huaweicloud.com, zlim.lnx@gmail.com, Florent Revest Subject: [PATCH bpf-next] arm64,bpf: Support struct arguments in the BPF trampoline Date: Mon, 8 May 2023 18:46:50 +0200 Message-ID: <20230508164650.3217164-1-revest@chromium.org> X-Mailer: git-send-email 2.40.1.521.gf1e218fcd8-goog MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This extends the BPF trampoline JIT to support attachment to functions that take small structures (up to 128bit) as argument. This is trivially achieved by saving/restoring a number of "argument registers" rather than a number of arguments. The AAPCS64 section 6.8.2 describes the parameter passing ABI. "Composite types" (like C structs) below 16 bytes (as enforced by the BPF verifier) are provided as part of the 8 argument registers as explained in the section C.12. Signed-off-by: Florent Revest Acked-by: Yonghong Song --- arch/arm64/net/bpf_jit_comp.c | 51 ++++++++++---------- tools/testing/selftests/bpf/DENYLIST.aarch64 | 1 - 2 files changed, 26 insertions(+), 26 deletions(-) diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index b26da8efa616..22e3c456554f 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -1731,21 +1731,21 @@ static void invoke_bpf_mod_ret(struct jit_ctx *ctx,= struct bpf_tramp_links *tl, } } =20 -static void save_args(struct jit_ctx *ctx, int args_off, int nargs) +static void save_args(struct jit_ctx *ctx, int args_off, int nregs) { int i; =20 - for (i =3D 0; i < nargs; i++) { + for (i =3D 0; i < nregs; i++) { emit(A64_STR64I(i, A64_SP, args_off), ctx); args_off +=3D 8; } } =20 -static void restore_args(struct jit_ctx *ctx, int args_off, int nargs) +static void restore_args(struct jit_ctx *ctx, int args_off, int nregs) { int i; =20 - for (i =3D 0; i < nargs; i++) { + for (i =3D 0; i < nregs; i++) { emit(A64_LDR64I(i, A64_SP, args_off), ctx); args_off +=3D 8; } @@ -1764,7 +1764,7 @@ static void restore_args(struct jit_ctx *ctx, int arg= s_off, int nargs) */ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image = *im, struct bpf_tramp_links *tlinks, void *orig_call, - int nargs, u32 flags) + int nregs, u32 flags) { int i; int stack_size; @@ -1772,7 +1772,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, st= ruct bpf_tramp_image *im, int regs_off; int retval_off; int args_off; - int nargs_off; + int nregs_off; int ip_off; int run_ctx_off; struct bpf_tramp_links *fentry =3D &tlinks[BPF_TRAMP_FENTRY]; @@ -1799,7 +1799,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, st= ruct bpf_tramp_image *im, * [ ... ] * SP + args_off [ arg1 ] * - * SP + nargs_off [ args count ] + * SP + nregs_off [ arg regs count ] * * SP + ip_off [ traced function ] BPF_TRAMP_F_IP_ARG flag * @@ -1816,13 +1816,13 @@ static int prepare_trampoline(struct jit_ctx *ctx, = struct bpf_tramp_image *im, if (flags & BPF_TRAMP_F_IP_ARG) stack_size +=3D 8; =20 - nargs_off =3D stack_size; + nregs_off =3D stack_size; /* room for args count */ stack_size +=3D 8; =20 args_off =3D stack_size; /* room for args */ - stack_size +=3D nargs * 8; + stack_size +=3D nregs * 8; =20 /* room for return value */ retval_off =3D stack_size; @@ -1865,12 +1865,12 @@ static int prepare_trampoline(struct jit_ctx *ctx, = struct bpf_tramp_image *im, emit(A64_STR64I(A64_R(10), A64_SP, ip_off), ctx); } =20 - /* save args count*/ - emit(A64_MOVZ(1, A64_R(10), nargs, 0), ctx); - emit(A64_STR64I(A64_R(10), A64_SP, nargs_off), ctx); + /* save arg regs count*/ + emit(A64_MOVZ(1, A64_R(10), nregs, 0), ctx); + emit(A64_STR64I(A64_R(10), A64_SP, nregs_off), ctx); =20 - /* save args */ - save_args(ctx, args_off, nargs); + /* save arg regs */ + save_args(ctx, args_off, nregs); =20 /* save callee saved registers */ emit(A64_STR64I(A64_R(19), A64_SP, regs_off), ctx); @@ -1897,7 +1897,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, st= ruct bpf_tramp_image *im, } =20 if (flags & BPF_TRAMP_F_CALL_ORIG) { - restore_args(ctx, args_off, nargs); + restore_args(ctx, args_off, nregs); /* call original func */ emit(A64_LDR64I(A64_R(10), A64_SP, retaddr_off), ctx); emit(A64_ADR(A64_LR, AARCH64_INSN_SIZE * 2), ctx); @@ -1926,7 +1926,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, st= ruct bpf_tramp_image *im, } =20 if (flags & BPF_TRAMP_F_RESTORE_REGS) - restore_args(ctx, args_off, nargs); + restore_args(ctx, args_off, nregs); =20 /* restore callee saved register x19 and x20 */ emit(A64_LDR64I(A64_R(19), A64_SP, regs_off), ctx); @@ -1967,24 +1967,25 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_im= age *im, void *image, void *orig_call) { int i, ret; - int nargs =3D m->nr_args; + int nregs =3D m->nr_args; int max_insns =3D ((long)image_end - (long)image) / AARCH64_INSN_SIZE; struct jit_ctx ctx =3D { .image =3D NULL, .idx =3D 0, }; =20 - /* the first 8 arguments are passed by registers */ - if (nargs > 8) - return -ENOTSUPP; - - /* don't support struct argument */ + /* extra registers needed for struct argument */ for (i =3D 0; i < MAX_BPF_FUNC_ARGS; i++) { + /* The arg_size is at most 16 bytes, enforced by the verifier. */ if (m->arg_flags[i] & BTF_FMODEL_STRUCT_ARG) - return -ENOTSUPP; + nregs +=3D (m->arg_size[i] + 7) / 8 - 1; } =20 - ret =3D prepare_trampoline(&ctx, im, tlinks, orig_call, nargs, flags); + /* the first 8 registers are used for arguments */ + if (nregs > 8) + return -ENOTSUPP; + + ret =3D prepare_trampoline(&ctx, im, tlinks, orig_call, nregs, flags); if (ret < 0) return ret; =20 @@ -1995,7 +1996,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_imag= e *im, void *image, ctx.idx =3D 0; =20 jit_fill_hole(image, (unsigned int)(image_end - image)); - ret =3D prepare_trampoline(&ctx, im, tlinks, orig_call, nargs, flags); + ret =3D prepare_trampoline(&ctx, im, tlinks, orig_call, nregs, flags); =20 if (ret > 0 && validate_code(&ctx) < 0) ret =3D -EINVAL; diff --git a/tools/testing/selftests/bpf/DENYLIST.aarch64 b/tools/testing/s= elftests/bpf/DENYLIST.aarch64 index cd42e2825bd2..08adc805878b 100644 --- a/tools/testing/selftests/bpf/DENYLIST.aarch64 +++ b/tools/testing/selftests/bpf/DENYLIST.aarch64 @@ -10,4 +10,3 @@ kprobe_multi_test/link_api_addrs # link_f= d unexpected link_fd: a kprobe_multi_test/link_api_syms # link_fd unexpected link= _fd: actual -95 < expected 0 kprobe_multi_test/skel_api # libbpf: failed to load = BPF skeleton 'kprobe_multi': -3 module_attach # prog 'kprobe_multi': fa= iled to auto-attach: -95 -tracing_struct # tracing_struct__attach = unexpected error: -524 (errno 524) --=20 2.40.1.521.gf1e218fcd8-goog