From nobody Fri Dec 19 19:01:32 2025 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23C4330FF28 for ; Tue, 14 Oct 2025 11:26:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.196 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760441221; cv=none; b=NubbK7OsGYFAuYWr3b081KYKo2+ehAGlk71jAgvPnv1nkIhcicIdLziH3t/+iCedDYt4IJ0h2i3cKSJa/KZ0ovWsYlXEKpw1dq7DLcyfrCexcuS9McSAqOWhq0Og+sHGYhD8oghUEK7M8gVVwpG/r6HgaZTktyNLFM2UeHBI8+Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760441221; c=relaxed/simple; bh=9UBTPaJM795jgL9QFzpBm8/hFhK0/nTuD4q5tVTvhPw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OuVe27w/+71PDf+t80VNdN6Seti50YX/urVFEc55sN76oEmAeDaEtjxrQDG2u349XYWFQnYWbGIqRImZ0To+dzVuKv37RwNyfRogyMRGkcEqiq5MstzWt2LsQOGgzSrzWdjeSnSBEIN9GQC3YCAJmq7JBuX/6pysyHvt9dVDTdE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Fa28oB77; arc=none smtp.client-ip=209.85.215.196 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Fa28oB77" Received: by mail-pg1-f196.google.com with SMTP id 41be03b00d2f7-b63148d25c3so3427432a12.1 for ; Tue, 14 Oct 2025 04:26:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1760441217; x=1761046017; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UUUWsY7ufR6oqn2huC+LiTtkNemBc7wwNq571Svxhxg=; b=Fa28oB775qTKKYLCutJH9HB1NJGHWdnuHe8YyZEzlnCGkAtNn82UtCSnR2bpes+Y2p qce5MNCf/qhajwosxaXhIujakgaD1Kx3eVqAjX0kKVc8wrb0epRpBh8i6xkd2wQcJFkT ppUBAzEGdfgOhST6vKBb4/Y6j6/2p8gHzCufzRMwFpE4B+PaLsEVC6GZMwhrzdqz/ufG e36eKh0FZCOxnqx5gRIaiJLgR4wytTJU3e8U7G1jfCthQac7geQYJdXfi9C6juJFvx+N Vy07JN9+NheSS/d5FpAuLFkSw5bRx1Zdinb4jF3gHu1UE7/LGNIRhSAEDGaPb4mYDGIQ FwEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1760441217; x=1761046017; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UUUWsY7ufR6oqn2huC+LiTtkNemBc7wwNq571Svxhxg=; b=xEnd0KlFfJ1dJHkvpcbk7MzBLa7hkyBkip1mWWAUUWtxj3MNEXiSeRj3GT0b+cEtoO VYRPCo8SA6hjkm8v8229myrDu80cIadX3GWwUYwkX5RpVhmziMB1RkOpYWqzlY9TRYQQ ZsvaMFmiTR545vvUJPKbTz3/968/nwcl6NorJTTfblyo/NiUe6kjBjKM89Wnw3Hgg610 hVYlsNKcLTSD0KdU5omactANAE5Ib/8SYxVsfSJbyy25p4sOF1K3q6/EM6yu70okR1bQ f6ZsOJNmZZrAMVyK73ykP6PNtfwHLwk8QHXfFygaVV33tzWLipUZXz6/0VBnYmy6g9Yq Tk6Q== X-Forwarded-Encrypted: i=1; AJvYcCUv1Cy1DlcknrM4ssKmiQHmVmlphJd3G4IXpmH4wBaP2c7YEMmpNQaqL4T/g8npkg8PBRZnOcTclDRj/iU=@vger.kernel.org X-Gm-Message-State: AOJu0YzT71e/pEdtWUAWs8h2gl7pygjofEStXTf4n/guaj8M9NIh18Q+ DTtggP7S9Fpf9AdDqSzVau3UeHQhmBomObYmMxUQuBP23gllELRGIaFv X-Gm-Gg: ASbGnct9E3rG3q9jDclNRNhjRmvTzZYzfjlEwE+apnRZtwsvZdjoDvmR5aTKaC3ttf1 Rv6Buy9W+jzOcxcI63Oqk/O1ibn+3es0gAzNDkeiOnU2+RxIY+viuxOTD2FSwfqN6x0pi6ErSB7 D2yPEWLOHM+JZ9MEBGsxkR/A7ubbtgf2QqKy03B8XuT76A2Kzp/eKmlbC7P8yEx6EMZie5GpQz4 vu5hpjKBBa8wvLZd7aGAwZfNAiJWvJKRAydc3iz44ZnHAM1lRRgf+TpBVSiNZd3uaMuVhks1ouZ fqtufYceuaQ2qwGqMGpJ8Dg+hUsEUQavTc+37k7g8TKOOMKQGv26DbSjGmQhUTM+K2N4Yx1L5C0 iZibRLJ1p8p6Uzr+NjvctYhqIw6azV8hRu4IQ+tTcrUWVAkEwASOH X-Google-Smtp-Source: AGHT+IHwj+Dz4L8DnGND7olSFpVsnMTn5Lw/TEHpx72gcX3uQG8ce7dMeM/FZb/y/sUkqHY1JHLYcw== X-Received: by 2002:a17:902:f788:b0:274:506d:7fcc with SMTP id d9443c01a7336-28ec9c0c7e1mr394581375ad.6.1760441217252; Tue, 14 Oct 2025 04:26:57 -0700 (PDT) Received: from 7950hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29034e2062fsm161807285ad.48.2025.10.14.04.26.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Oct 2025 04:26:56 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, jakub@cloudflare.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, jiang.biao@linux.dev, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 1/4] rcu: factor out migrate_enable_rcu and migrate_disable_rcu Date: Tue, 14 Oct 2025 19:26:37 +0800 Message-ID: <20251014112640.261770-2-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251014112640.261770-1-dongml2@chinatelecom.cn> References: <20251014112640.261770-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Factor out migrate_enable_rcu/migrate_disable_rcu from rcu_read_lock_dont_migrate/rcu_read_unlock_migrate. These functions will be used in the following patches. It's a little weird to define them in rcupdate.h. Maybe we should move them to sched.h? Signed-off-by: Menglong Dong --- include/linux/rcupdate.h | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index c5b30054cd01..43626ccc07e2 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -988,18 +988,32 @@ static inline notrace void rcu_read_unlock_sched_notr= ace(void) preempt_enable_notrace(); } =20 -static __always_inline void rcu_read_lock_dont_migrate(void) +/* This can only be used with rcu_read_lock held */ +static inline void migrate_enable_rcu(void) +{ + WARN_ON_ONCE(!rcu_read_lock_held()); + if (IS_ENABLED(CONFIG_PREEMPT_RCU)) + migrate_enable(); +} + +/* This can only be used with rcu_read_lock held */ +static inline void migrate_disable_rcu(void) { + WARN_ON_ONCE(!rcu_read_lock_held()); if (IS_ENABLED(CONFIG_PREEMPT_RCU)) migrate_disable(); +} + +static __always_inline void rcu_read_lock_dont_migrate(void) +{ rcu_read_lock(); + migrate_disable_rcu(); } =20 static inline void rcu_read_unlock_migrate(void) { + migrate_enable_rcu(); rcu_read_unlock(); - if (IS_ENABLED(CONFIG_PREEMPT_RCU)) - migrate_enable(); } =20 /** --=20 2.51.0 From nobody Fri Dec 19 19:01:32 2025 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6EE9F3101AB for ; Tue, 14 Oct 2025 11:27:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760441226; cv=none; b=SoZzLX4XVtPYDuk4n7J1QwoWajwGcWtDkPtRbpThLE5V09ya3oh6JTNiK82szyyNtveuyZeS+YFKxHG/IDtIy/gdgYwJvYu9uMol02f91A0NJkke9cAVPqmo9jq6KHppudytjzI0/OnOnZwwFKNo5DaPoFKQZaIKiFguiTt2gcU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760441226; c=relaxed/simple; bh=T1Nmm/R6LFBiTFjOpeGfSLW0d5l/K2N7bsDjPiEWPmo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=opAkia9kKsGGFs9wkS3Hl41ueyG5Rlj6Y2cJKtRtyKltDuKX4xc1ytI4Kh5kalsXkxY4D0YAXo4pPG9g5iiC6f0/PevvHMT+jdDuzROwWsDW/98+1AfsZbFXYqgmcwH0fKJtpiPQlhHV7tnumIp6Gwohj4j7DqMfNmigAzo+4Tw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Zi+GOJLa; arc=none smtp.client-ip=209.85.210.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Zi+GOJLa" Received: by mail-pf1-f193.google.com with SMTP id d2e1a72fcca58-7841da939deso4570758b3a.2 for ; Tue, 14 Oct 2025 04:27:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1760441223; x=1761046023; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7Q3iRfExEJqSXZ9Zc1Icg4TjF6EtAHQb2pCdXlTokh4=; b=Zi+GOJLaSWqNICUjGYoVC6y93s4bW9axrH/dehEFEQGplsW2aYB30xT8OWdZlqTDbc 64ZoCrnWHzrl7bhLdPPfC4PG7DkCIJQnXy2aQFR02K2ZMSj3m6sq9G/5s3iG8Ea1jvxO ICqhYT1H8smExYECw1vXUxFfWPuRMVqnDUftDBZuMDA6NlVmtkUQQIdcvLBf4AvvcdKy vioJ+4zKYqJtAlZYiXw2/BeE1ANXEO+4blwEpSOkiT8LU2pERusPaXob4ScEgZjH1yKY LiMcc0hmEUuv1n2PlZnAqhhcgkDr1vvtGiUSgC0S6Vfzlxz/KEmL0zK2GfHQVh7N3sHi xrjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1760441223; x=1761046023; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7Q3iRfExEJqSXZ9Zc1Icg4TjF6EtAHQb2pCdXlTokh4=; b=ruw7/p/saPy9HY3CFWWwuFhVy8cnmRVV8Bq6vBvSdDEHZuuWAno5elfTWFr7o9X8Xu pmPbJtWbEdhrrE47EkBElglxq1GVczDz2mGDI7PpskeTmx+mdqufPn/hb/4O2/QSuMG2 n7Db72v3UHI9iEWFqGQXcn0Zf50TrohxyTb0qa6v68+UjSLyvmYqR25GPsKNibVs2a2E uZXPCQ/c13UGccFtQPyAFSG7+1HTw30ny2JFqTpCSYmTglP7Suay6LyQgdEHCiFhhYge eKaZmbFN3sOc7VesGRRgQRm9V/UcaTLNgIthJaxPWC72UFxxEcCq0M4ibhQSoqHgh9xQ F8GQ== X-Forwarded-Encrypted: i=1; AJvYcCXhqT8Z3Ov1cutB5apbcuQcowS1N/8Y06gYUkaAIZ7mGFK66ps/BwREznGQu5GMUobOgzBYFpNDN3++s2g=@vger.kernel.org X-Gm-Message-State: AOJu0YyQMmtnwEu9cxbmCbtyufxKD8y+UUSyLBEyToW9BOw6KoThoYsJ kd6ArWKFpCPEPgIGSNgXtRFgOg4e0QvmoMiX4HIuVQBe45Sa9/v2gVdZ X-Gm-Gg: ASbGncshcethojkZZXAH82MgWByvfn7XVYMepE1T84zlWpP45ghTDwJef9ATkIZp1IL W9a+5rXjoEJhZIKZH2niWnBGdzWbR9qzYxxWtUQ7nBu42XBBB1vrs8fAdjQkkjX3Zq9JOPohvKf KzUc7YTu4s6WLLGflcR0MsgslepUZ/Jzmfb69CqR98oqvsci2+kodn1TwVBVC/TeqE87/wfrBEx LZq2GoW97DTDemU69xbfCPxKjN/LdBA3vB8pkqjNqkxkbvMO39z2TmHi/6vHUaJROTvmtpb1cIn Sm+TuemnJpBcgpHnAQx9UVFEziHSI1KMlwpXwlpjixOUCPORgC5W/67sN0DkmpR2UShaqweGsgA MIcJFL9e8X75kJ8w9CnDO2GNbi0nEIgs4HqFOVO5e7g== X-Google-Smtp-Source: AGHT+IESAHoQT5qoaRUX6Sg1Z4sRZoIKiFEVF1CVFjtoN9V7EZsWdw8UMx2i81q1k2h2feHx/dvUDQ== X-Received: by 2002:a17:902:e550:b0:268:15f:8358 with SMTP id d9443c01a7336-29027402ca0mr302264415ad.42.1760441223089; Tue, 14 Oct 2025 04:27:03 -0700 (PDT) Received: from 7950hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29034e2062fsm161807285ad.48.2025.10.14.04.26.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Oct 2025 04:27:02 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, jakub@cloudflare.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, jiang.biao@linux.dev, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 2/4] bpf: introduce bpf_prog_run_pin_on_cpu_rcu() Date: Tue, 14 Oct 2025 19:26:38 +0800 Message-ID: <20251014112640.261770-3-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251014112640.261770-1-dongml2@chinatelecom.cn> References: <20251014112640.261770-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce bpf_prog_run_pin_on_cpu_rcu(), which will be called with rcu_read_lock. migrate_disable_rcu and migrate_enable_rcu are used to get better performance when CONFIG_PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- include/linux/filter.h | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/include/linux/filter.h b/include/linux/filter.h index f5c859b8131a..48eb42358543 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -747,6 +747,18 @@ static inline u32 bpf_prog_run_pin_on_cpu(const struct= bpf_prog *prog, return ret; } =20 +/* The same as bpf_prog_run_pin_on_cpu, except rcu_read_lock should be hel= d */ +static inline u32 bpf_prog_run_pin_on_cpu_rcu(const struct bpf_prog *prog, + const void *ctx) +{ + u32 ret; + + migrate_disable_rcu(); + ret =3D bpf_prog_run(prog, ctx); + migrate_enable_rcu(); + return ret; +} + #define BPF_SKB_CB_LEN QDISC_CB_PRIV_LEN =20 struct bpf_skb_data_end { --=20 2.51.0 From nobody Fri Dec 19 19:01:32 2025 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F71D30F959 for ; Tue, 14 Oct 2025 11:27:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.195 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760441232; cv=none; b=GOecU0exCpx2ivjYwcrMC6e9U7dOWh6ZpBCWdhrxYlj/RSGjtKazFeqaFPSd1JWHRGgUjMOimzqMznzdlJT9olpN0ZvuimomTSkDdBvT0kI+CghFqo30gk9K2KZP9HtLOCpd7KwCfnSIWt8w5SGVye9670Qjr8kjDt3gIvOQhXk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760441232; c=relaxed/simple; bh=LpUckPTb7jH/83BNuJSOxN+YPu0vw1zspxIFiooWH8Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Cea6QhhxA4MgwdyLF5xr1Hr3yhjhamyvP7+Fjdu6bcTaXL97yDKXYdKTnnrKnf1jvvq+Gg4WUFce04T+hpMM3tjbWhfhbgHTXq1HzKdC4+P1Caw2IEZyidcaCutgQOOFzetYkraMp0MwCvHnGQ+9xMsSg7eNqcEUdJPe7J2cFxE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=J/uM4FQO; arc=none smtp.client-ip=209.85.214.195 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J/uM4FQO" Received: by mail-pl1-f195.google.com with SMTP id d9443c01a7336-27ee41e074dso60262545ad.1 for ; Tue, 14 Oct 2025 04:27:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1760441230; x=1761046030; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hixVqDtG5kfwkDmbrI9+bLiGthHT6YGDBRV0eBz8fPk=; b=J/uM4FQOeU2neSw6b2Q3CXLQX41mnJa4xiREIUXU/k5ttB6Ai89oCC3oJO/zi0i4Em smD+9BI6AdsTdwtFbhYrqYUHdSxdu9dcNP/wQ81Q4ZN95YKF/kny8Pp3UJclJf57e+aD b7TYSrlIeHMzacKNdtSqkGLP2CpEOFMsLkedE6a+J0ft8tZrS9or9RTRHDoW2oaXIsu8 2094ih3A3111ggguyWvHMGbKpekHw1efANGxwJKdFHhUrP3tu83hZO1WTPFMC8UjtYRo oYbfT7V9oBMni/HJZNScjSnxxfiTEAO5EJjhxCc9yM4nub+CBCDZxF5kvaAGqfkNrVxV 6dRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1760441230; x=1761046030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hixVqDtG5kfwkDmbrI9+bLiGthHT6YGDBRV0eBz8fPk=; b=Z7QV227xZCcdweMGn3WcLYrmkWwFXmjHIHRKXOBvBATGdhU2NZwW76mfjGM83h/f0j zH978EemAvWNwKnTNKkP1XqV0HeYxsVTlDISuecL/P6hXZR1nHrn5KuOqPw+/cL0B+ko AYEQuChU2KNTMF94h+tAeEmsai2Ujqq7HLxd+LWDxeaolNsNd6+zecEHDtyzLqQrMkYs BOD70Z7gOVxt2cfLSQT+8k5AjHNLSI2i2mwGQjHavOTiViBeN4VD1QeBjCLyVfo7lzE4 S9D6hxkbluKwTSr80WADnpXOPqdMr0Yssc2/jjipBeSftxuFBGn+vmTR/fkcJ5HhvlCg gzAw== X-Forwarded-Encrypted: i=1; AJvYcCWd6h4yhuQRNc9/1LYICLB1W36iTXjB0Cx54OhwjpSnwNtOtV78/UUyMtLPepcQwH6UCuh192ZR2OghzN0=@vger.kernel.org X-Gm-Message-State: AOJu0Yw6PLRJ8me1oM9xJj1enFnYVT/v3pGI+Eg7K4jFcDPaqnS/NhVK to7xm0hD5ARzC7sXp0ytg5+WDwDgxcTGYLKGTyY/NUBsZR4fidZrQF/N X-Gm-Gg: ASbGnctLmpD0YSlN3SOLQ1NTfkUrfFUwdc7tBr85yUD85hrkUxIr8Oy0Ela3ZiV3YGP hqc4eX8Bzi/0l5r9BXNmu2nxQvcM5NY95/7J3GnmO4YnMPQl2slaNj+w5i/uyKFtpw+GFk3bgi5 iNRGnPCsI3J6vJ+jRCLd/vrnEDdeLiVM/idFUK4WX7NwHmETZP31lurwLT6kDyDsdQRbQgTSedV MJnrL9Cqn1n7xeI5Hftj1B+BeHacvY/pWI8oE6Y/zIRF8EY1MvtztkvXASYapbOH5A0FXxwRNBo G6tnQydYAk8onboOrkOLZwc/eiOLyUjQWIWeFxLX/1u6SI3g2ji7ZGAOuncBHyyftb3wp4MQd08 a/gdqejSGFZlRk1aHVbMg9SjqVy6VyETHg7NxkL3uHA== X-Google-Smtp-Source: AGHT+IGSGyLntq28lnbnZr4P0EjlrHeHsCeSBY4tltUAlQ5AUSXAYfEcRW/aomPwYfhGFSurWqTmGA== X-Received: by 2002:a17:902:ef06:b0:24c:ed95:2725 with SMTP id d9443c01a7336-290272135camr265404925ad.4.1760441229532; Tue, 14 Oct 2025 04:27:09 -0700 (PDT) Received: from 7950hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29034e2062fsm161807285ad.48.2025.10.14.04.27.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Oct 2025 04:27:09 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, jakub@cloudflare.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, jiang.biao@linux.dev, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 3/4] bpf: use bpf_prog_run_pin_on_cpu_rcu() in skmsg.c Date: Tue, 14 Oct 2025 19:26:39 +0800 Message-ID: <20251014112640.261770-4-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251014112640.261770-1-dongml2@chinatelecom.cn> References: <20251014112640.261770-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace bpf_prog_run_pin_on_cpu() with bpf_prog_run_pin_on_cpu_rcu() in following functions to obtain better performance: sk_psock_msg_verdict sk_psock_tls_strp_read sk_psock_strp_read sk_psock_strp_parse sk_psock_verdict_recv Signed-off-by: Menglong Dong --- net/core/skmsg.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/net/core/skmsg.c b/net/core/skmsg.c index 2ac7731e1e0a..1d3f47b07659 100644 --- a/net/core/skmsg.c +++ b/net/core/skmsg.c @@ -908,7 +908,7 @@ int sk_psock_msg_verdict(struct sock *sk, struct sk_pso= ck *psock, =20 sk_msg_compute_data_pointers(msg); msg->sk =3D sk; - ret =3D bpf_prog_run_pin_on_cpu(prog, msg); + ret =3D bpf_prog_run_pin_on_cpu_rcu(prog, msg); ret =3D sk_psock_map_verd(ret, msg->sk_redir); psock->apply_bytes =3D msg->apply_bytes; if (ret =3D=3D __SK_REDIRECT) { @@ -993,7 +993,7 @@ int sk_psock_tls_strp_read(struct sk_psock *psock, stru= ct sk_buff *skb) skb->sk =3D psock->sk; skb_dst_drop(skb); skb_bpf_redirect_clear(skb); - ret =3D bpf_prog_run_pin_on_cpu(prog, skb); + ret =3D bpf_prog_run_pin_on_cpu_rcu(prog, skb); ret =3D sk_psock_map_verd(ret, skb_bpf_redirect_fetch(skb)); skb->sk =3D NULL; } @@ -1101,7 +1101,7 @@ static void sk_psock_strp_read(struct strparser *strp= , struct sk_buff *skb) skb->sk =3D sk; skb_dst_drop(skb); skb_bpf_redirect_clear(skb); - ret =3D bpf_prog_run_pin_on_cpu(prog, skb); + ret =3D bpf_prog_run_pin_on_cpu_rcu(prog, skb); skb_bpf_set_strparser(skb); ret =3D sk_psock_map_verd(ret, skb_bpf_redirect_fetch(skb)); skb->sk =3D NULL; @@ -1126,7 +1126,7 @@ static int sk_psock_strp_parse(struct strparser *strp= , struct sk_buff *skb) prog =3D READ_ONCE(psock->progs.stream_parser); if (likely(prog)) { skb->sk =3D psock->sk; - ret =3D bpf_prog_run_pin_on_cpu(prog, skb); + ret =3D bpf_prog_run_pin_on_cpu_rcu(prog, skb); skb->sk =3D NULL; } rcu_read_unlock(); @@ -1230,7 +1230,7 @@ static int sk_psock_verdict_recv(struct sock *sk, str= uct sk_buff *skb) if (likely(prog)) { skb_dst_drop(skb); skb_bpf_redirect_clear(skb); - ret =3D bpf_prog_run_pin_on_cpu(prog, skb); + ret =3D bpf_prog_run_pin_on_cpu_rcu(prog, skb); ret =3D sk_psock_map_verd(ret, skb_bpf_redirect_fetch(skb)); } ret =3D sk_psock_verdict_apply(psock, skb, ret); --=20 2.51.0 From nobody Fri Dec 19 19:01:32 2025 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E397C30F959 for ; Tue, 14 Oct 2025 11:27:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760441237; cv=none; b=C9Z6DMLILL/GJaYMSB5JRm8E1qEEj1QPZJSeK6yvoLaueaV3ohBeKq0xoChW2otay9s4qGgx6aunnap84453cqIqLo4q/zmOI7WGKyMoKclajIm0PttyqKxEeCq20FCRyBD2AX4CHiXDuFg2C0J4Z0OA0buFRQWRtqLfdJRi3jc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760441237; c=relaxed/simple; bh=Rx/YhkNxvLHTn1Rxzb2qa4X0/4hclpnv8IPkM9Xg0Oc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XngAyPjnBOke2PM/L2cfmjuD9EWD1nkJj/YIYUfwYSifM9nB9gkaUyu3qEjnmstjyfUPIMHMfxqHOjuIowTilV89Cs8dvRMWIHc6oSusQHEdRM+qZLxokw8yRf+73ZXtY8fAxl9ddmY7UaKRE1wc+ORIqGOQ+8uXRjyoxmBjjw8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fZm/6l8e; arc=none smtp.client-ip=209.85.215.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fZm/6l8e" Received: by mail-pg1-f193.google.com with SMTP id 41be03b00d2f7-b5515eaefceso4505185a12.2 for ; Tue, 14 Oct 2025 04:27:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1760441235; x=1761046035; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iE4VzJhkO405E1xXiO2HovH6fVbTOd8AlZfv3OpV7tQ=; b=fZm/6l8etXCz+IX9EaYFXrINr+/EC5xO/KA/D/goPOt31MUJzDyet/ZxEgiA2sXwqP eo9ah2BfbtNGmFHVYg6leGsD6dSKaazLJBV8X5eJ8dZe6pJO8R+b9KzERmuoszWzy7xq EWKsaO9BpT57wDBvCdvLbDyqbaCk3dClrxpI/cuqgoIbEj8Z0Ame/e0gD1oBB/g1/rXA AGTSaU3PHg/qhnP93TsNC0VyiNpcmZKig5aOxq/I77v3nPd47l6Ec69LwUnTdATpGx1h URjqRZnr7Xh1ZPF+sUgVbVsvnhgUon5CCsQYhMKZGoylwSUVSOPJMa45YV1uV+9MHn0n Oq/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1760441235; x=1761046035; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iE4VzJhkO405E1xXiO2HovH6fVbTOd8AlZfv3OpV7tQ=; b=P3NC60bTC8TRL+pAsTITxRsvy0Tb/ZX5GdbCioky5kjxfpgnb52PNQ80czKZQUk5AZ 86LE2sEAdfJT7ccbYiTSl2LaqA54gF/T08m7hYDFWzEfhdhv5yl0UCOxBD2AmO84fFA1 fv27bFtmJkJPcAe24W2Kt+gptePeIH6C2RGoVMvMX+BSIctlml8dQ4gobTxB6q0IhEzf zrqY5400W1tH2TlgBkREi+efjjz9hYYS7fnXzuWeXXTY6/2h/vQexaYZZbcqllXXNiK0 d+csK2n1hbl62BqaNQ1oXfjvEfmfiY+ax14cbDEUqz8z4mNnfx4n7ScqwD/ieQUtWHpk dgKw== X-Forwarded-Encrypted: i=1; AJvYcCXDuP0J5dJQ4FuWEVvyMv8QxttVPEOIumAWRXpRIZ3fOE35iI0MT/kYnJ6IgNggpGIaUq8bS0cvddYtc64=@vger.kernel.org X-Gm-Message-State: AOJu0YxXfSXSNbgXKzzlc/VQjEF6rhUBzPKsieWXneHYomnNxoAcBods YzHg/W2t9ro01WRlQM4dj9jfRoD05AlbQvPGpOJx9/2iAPBvX9Odm6TM X-Gm-Gg: ASbGncvB7bILPkmrU+fk/ndopDjdopvTxDAw25XziKkxKgKgYFySZO260GYpAMZe1s5 DHB2MegIZY9nbBpzXJA09sBi36dKN5Xc92/3oQChSGpaD3PKDt5vFMd6DntHf2h7SKTvipKQ5X9 SR0GBqrrb0Bcc6LXX4F4xGA1lM11dyAaWEgSV8rTO2k5QP5BuqHiBW6MiB+c1nicorrHkd9NJyN TAcCOjCrmSETObVV264GaeKD41XQQw7cXNn/+m0SkPMmGyvbvfAEh6+r46CzqkLwVxoCouOjqCc BKqCwDTn6OPdd0Dk3Xxgwbbz1PN18omiKmjXdifjOjH61OT0zJrWCvRvvDbF2RRcULNrW+lKSxr UKOsISkQMMP7t7baH3H8/l+Ru8cCaSpJNyp/ivArId7IC4jmc X-Google-Smtp-Source: AGHT+IFYag3HNohwuOOUkgMrRymk2W/3k72l93HvciN+it3ayDICZ/f/o7hlITPi8oCOkannkiormQ== X-Received: by 2002:a17:903:246:b0:24c:9309:5883 with SMTP id d9443c01a7336-290273ecb35mr316032095ad.28.1760441235366; Tue, 14 Oct 2025 04:27:15 -0700 (PDT) Received: from 7950hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29034e2062fsm161807285ad.48.2025.10.14.04.27.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Oct 2025 04:27:15 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, jakub@cloudflare.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, jiang.biao@linux.dev, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next 4/4] bpf: use bpf_prog_run_pin_on_cpu_rcu() in bpf_prog_run_clear_cb Date: Tue, 14 Oct 2025 19:26:40 +0800 Message-ID: <20251014112640.261770-5-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251014112640.261770-1-dongml2@chinatelecom.cn> References: <20251014112640.261770-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" All the calling of bpf_prog_run_clear_cb() is protected with rcu_read_lock, so we can replace bpf_prog_run_pin_on_cpu() with bpf_prog_run_pin_on_cpu_rcu() for it. Signed-off-by: Menglong Dong --- include/linux/filter.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index 48eb42358543..5ec5b16538f4 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -995,7 +995,7 @@ static inline u32 bpf_prog_run_clear_cb(const struct bp= f_prog *prog, if (unlikely(prog->cb_access)) memset(cb_data, 0, BPF_SKB_CB_LEN); =20 - res =3D bpf_prog_run_pin_on_cpu(prog, skb); + res =3D bpf_prog_run_pin_on_cpu_rcu(prog, skb); return res; } =20 --=20 2.51.0