From nobody Mon Dec 1 23:34:59 2025 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1E8731A7F0 for ; Wed, 26 Nov 2025 08:52:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.195 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764147179; cv=none; b=GRuWNNTrthuH3FR15VcQ1WRxyfwADs8QSXuXPnPz1utwTmPxo1cmDiOhKwbuKyr915TJ73eCbc67VeymUAllPF5rIeyZbalI8nG4nBGtjKwaZ9WMJCXSgsVr5g/da59fAayXlHTJpa9DdziC2VDVZgMSDw8ShpOFlP7QtzBwLRU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764147179; c=relaxed/simple; bh=J4BoZTcMCIwSrHMIm1Wg115n46MGz6eizwWMi1v8YGA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type; b=tVpBs1kPuzSxXHb5kC6Tw053K97RSougyIDSsNm7bIzUpIkyKedXb92GvlxKcY185RiG6IJZ9ZLVFlPpaJUuAmLCSNrvJt7j4nuKUYOYALTNXSAC+2vWDRZm38/32lolROdUDiGZixYhOtVMpFZrxtfzSoamRMnaV/rzHs0ZzbU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=O5/NUMYY; arc=none smtp.client-ip=209.85.214.195 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O5/NUMYY" Received: by mail-pl1-f195.google.com with SMTP id d9443c01a7336-29558061c68so86242345ad.0 for ; Wed, 26 Nov 2025 00:52:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764147176; x=1764751976; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Jgf6QzPNkLnaNG6eo7VvZMmMgNGAzeOfWslTIH6cEEA=; b=O5/NUMYYTPeuduAqlaPClPrzDrbfkLKttv/VLV6vTviO5yr7ianENsD/ikOOvxJAj/ iLlisLfZa5FFqOD0GTgKj2SIIClcdIGJGuBAig/WAqm2139NdTX+mvotQ+Psb6cGQco4 JzOOlmYI5Z+ZMw+aksufSjk+BCrDUY80c0pi7ryCWYxHNi9eqp63Nu0SWgEESR0vo9yw 1LO8t/x6bofMI/eokskywivT1ySYSY/9tH4au3xfjCGfOpjiQRnad88MjYFTJrQzTmQm d22GkJCYbEzdOs1/hKneuzpx/bZhzRouvTtd2BQSVy2DFebLbgQm1FkEBgKuLqhK9X6P KGdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764147176; x=1764751976; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Jgf6QzPNkLnaNG6eo7VvZMmMgNGAzeOfWslTIH6cEEA=; b=h7T9gQd5T5DN43pNqvhGyzXSmjphZRmIu/RJvMClI3Kh9dpRGA23xCcWf2Jgg+9Sdg emQ2AqiHtH0XufSB0XMXC+QnsOCl7NoxAgMEwgi5fCjD5vNDJnmp6rge7sgyILqt1fMe ruIcoO6L4KJ6a0nzLmgYpM1OgwMxNoeFeZeBOHdbi747j20owJSbEO5o0S0HB5GUjMy0 3NCPrGT1qF63/76Xb8j8YS86yoZ3Oee5TCfsKrw8kFU3tQ1lnbdIxZJ3K33R1UYs/72a 2zpM44tUmCOYUDVWigVjDxYZPB1S/E6SG6fD5CoRn0oP4B145W253rUnNYfrc+Tymwwt ejsQ== X-Forwarded-Encrypted: i=1; AJvYcCWOcnN/NQ36t9YsOm/kFWmZvz6OSb26AKKu0dVz+PMvbf4A1XWyJrrL3Z8BSoEgwEbK/QdESAWIKO8tcUM=@vger.kernel.org X-Gm-Message-State: AOJu0Yywv8VdNAWtl3ZcXuK88VTM1Fkt0z7KvyxUlhjajmiWZlKmZiSS +VoyC8fx6j4W71Eo/4xipGdrJo97Ux9JgP8PQM3W4FHQUnVZQoSezh9CCP6Fovf0qqE= X-Gm-Gg: ASbGnctk/F/m1EqK+iMCtxltiOJdrkpdOkgKs17EL0Mgjb9hBJyO5N/+Q5WSjAbzE7P QAmp2uGFVl3c5GUO/CLpby0IvrxwlYxOYqsYCH34iMcTcx4boBed1kdfHKvX4m4hE/sPG2BgtBo nped/hXK4vNJ1bvy5e4W/lB3gu69jc7YzJSPEEPKSV1djQa8q7Y3BhFY8QFYA4D+1XIuo01TI+U gPWwmXENQtn3ean7VxfnllEiwJpRDA6TUaklx/AVNRuu3aNb8hZ+wjGglFE+MomOuZ67fVmdvnn Q2dw0a1QYklOEQCwYFF65UtvNubu4nTEErq1XjCIrJZ8kPqKY0ZJUNgmnBUlcWRjPVr6y/gRPEz MSSspzFKY3MzRaEVlAbII5I997x1igADURgsy11fkTYksWXq7VkoBaIQ1siBWBtQc61+hFGrXyJ ynFsZvtUU= X-Google-Smtp-Source: AGHT+IFOTPUQKLezoW1PtOgUH7V/2VHPRcXFHERr2qyquuXGdL9D0nyn1gqUNf+kzQfWgYx9ll5PSw== X-Received: by 2002:a17:903:110c:b0:298:ea9:5732 with SMTP id d9443c01a7336-29b6bf64a07mr88019525ad.41.1764147175929; Wed, 26 Nov 2025 00:52:55 -0800 (PST) Received: from 7950hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29b5b274d39sm190678305ad.77.2025.11.26.00.52.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Nov 2025 00:52:55 -0800 (PST) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.or, jolsa@kernel.org Cc: kpsingh@kernel.org, mattbobrowski@google.com, song@kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, yonghong.song@linux.dev, john.fastabend@gmail.com, sdf@fomichev.me, haoluo@google.com, rostedt@goodmis.org, mhiramat@kernel.org, mathieu.desnoyers@efficios.com, jiang.biao@linux.dev, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH bpf-next] bpf: make kprobe_multi_link_prog_run always_inline Date: Wed, 26 Nov 2025 16:52:46 +0800 Message-ID: <20251126085246.309942-1-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.52.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Make kprobe_multi_link_prog_run() always inline to obtain better performance. Before this patch, the bench performance is: ./bench trig-kprobe-multi Setting up benchmark 'trig-kprobe-multi'... Benchmark 'trig-kprobe-multi' started. Iter 0 ( 95.485us): hits 62.462M/s ( 62.462M/prod), [...] Iter 1 (-80.054us): hits 62.486M/s ( 62.486M/prod), [...] Iter 2 ( 13.572us): hits 62.287M/s ( 62.287M/prod), [...] Iter 3 ( 76.961us): hits 62.293M/s ( 62.293M/prod), [...] Iter 4 (-77.698us): hits 62.394M/s ( 62.394M/prod), [...] Iter 5 (-13.399us): hits 62.319M/s ( 62.319M/prod), [...] Iter 6 ( 77.573us): hits 62.250M/s ( 62.250M/prod), [...] Summary: hits 62.338 =C2=B1 0.083M/s ( 62.338M/prod) And after this patch, the performance is: Iter 0 (454.148us): hits 66.900M/s ( 66.900M/prod), [...] Iter 1 (-435.540us): hits 68.925M/s ( 68.925M/prod), [...] Iter 2 ( 8.223us): hits 68.795M/s ( 68.795M/prod), [...] Iter 3 (-12.347us): hits 68.880M/s ( 68.880M/prod), [...] Iter 4 ( 2.291us): hits 68.767M/s ( 68.767M/prod), [...] Iter 5 ( -1.446us): hits 68.756M/s ( 68.756M/prod), [...] Iter 6 ( 13.882us): hits 68.657M/s ( 68.657M/prod), [...] Summary: hits 68.792 =C2=B1 0.087M/s ( 68.792M/prod) As we can see, the performance of kprobe-multi increase from 62M/s to 68M/s. Signed-off-by: Menglong Dong --- kernel/trace/bpf_trace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index a795f7afbf3d..d57727abaade 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -2529,7 +2529,7 @@ static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_c= tx *ctx) return run_ctx->entry_ip; } =20 -static int +static __always_inline int kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link, unsigned long entry_ip, struct ftrace_regs *fregs, bool is_return, void *data) --=20 2.52.0