From nobody Sat Oct 4 01:47:33 2025 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F5452EBB84; Thu, 21 Aug 2025 09:06:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.65 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767189; cv=none; b=LF5OpRkb14HR5nWK4U3bkJmmcT2cxh+xsGXAnF7WaRuCja7DEBcJhlpaMGKH7PHcfRkiB4557pQeKnQTR6vIs2aJTeLu0/AwIsSn61O+NUy11e1+dFwt9A6emaTJANJ9mk6TTtscovxJFw2lezPJ0Nci/BIcLOlH/DsyN+jFClI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767189; c=relaxed/simple; bh=onO1kdG0wfR8fcTpUtHZzKTzoXwXEO6zkb9GsAGsMvY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KBxTNtjOHvFgKlxv8zaCFelH4Uyt9Om/4otIZlr7dE+1nXX8q5QRzdzjJCDDxgsbztz5x5EMC9POIzklmATOX2S5kgVTrDu3XFU89HzUBkItaMx7GvCvH6l0peVnce9OZBpc7RUKPH9dpV4qtQebxFcmM3kTDSJYTwYxBeerp3A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nmUkwFpL; arc=none smtp.client-ip=209.85.216.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nmUkwFpL" Received: by mail-pj1-f65.google.com with SMTP id 98e67ed59e1d1-323266d6f57so811587a91.0; Thu, 21 Aug 2025 02:06:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755767187; x=1756371987; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=81UsXzxUVJ+ajSffOPLVSZMBWfzn2NrgIut+TwVub3g=; b=nmUkwFpLxZu74y5pRQ+aOjBSVl+xQrwKhEuunylvstDtJARN3sqCXRpccYkEQ0QFjp Qq/gpTsWOmyLooA8TPQTGlkz98wlMFdqdxBIhYTR/U0Cxds6EFEWCZxlFQOD1ZORCEnh hz9KItA2KZan1S7AeX52qWn7YSW4DJ5eEY6jvy4VnRdEkCWE7QNbjmtE4073HOdSMruu EPM449mcg+fvNETXPrwhkYIV9XUSfog1bTGwbgR1tH22sfCfYCFQktZrdSLg5vpITRyf lhLIRy3OzkLZcqyidTSML2XgYaoVHH0muoKq9Xm3sMCwAJbCpJVKYW/7cz4srsHX35Pt YsMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755767187; x=1756371987; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=81UsXzxUVJ+ajSffOPLVSZMBWfzn2NrgIut+TwVub3g=; b=ht3Esj0XB1pjAxyrGF57kmwpbFdthSfjEUnFjAlEOn1CaUKRbtQnexdEZiTwHk+Vv9 fjFzIyT5zE6HMG324x9IGoB9lPbSSnd7lZ0LplgwrZ48Sx99ezszhzekXpXDkG9Bud0z Bo2xOjwahQXYPj/nTO16Yof2VZLyTC8z2DtoYph0hbhaBuUrzKIs+U/x0oQoBjJTaI13 FiPqxF/+d/xnl5xIheW15pS/Q0p7oo3mY0Gmn/cVZaLCwh7Hrvva58DMlu4K5v7sQqHR y12Tp5wqEdR/E3qIOXGw16YX7MQAvEkGv/2OzYye9yi6WTYGygC4X1+/dgeDpirLYIMq lQDg== X-Forwarded-Encrypted: i=1; AJvYcCVdCjFuw1rF7NWvOgmNrFv3WP1p0i/E0BoULhqk5lbgW6+XhuMaGqia3Wfh46ztVYhnT5a2XABOqpT2beWp@vger.kernel.org, AJvYcCVjym/G1VvxfGgBckVDMDQnmrsIKC62S30sOE0oziW9FrWLGJ769sKgshfTQcQRO9raQTjz@vger.kernel.org, AJvYcCWO0oZlxh61xY309ndf+UXijcB281dxJkEH1tfQJGGGKojSPEmUDmdtoEOt6vbPsBRbHjo=@vger.kernel.org X-Gm-Message-State: AOJu0Ywf8Cj1xg/jD9XT73Sj3DfudmrOPDtqMYfIvVmUhRYstOZKVRQ/ riyP/Dj/LCrw1t/k2z52LmThS0sY3iYdrKfHQC+u5IDJPPYgsCvyILiI X-Gm-Gg: ASbGncs/X4HaFyDJHzIizfEaNS5rcasDnAZ5BxMM4JJ8NKwRhk9hagzISwQZGkEp15o Jy19RYHDmnVCkp2EFSkWSuVUe8m79O6rZMdkmhT0Yn1TSAwGgrLSBbeTm1cyqomujJTODcM4+ZD XinlcT/2XPUnexIqXAlBzqxwjKijosJ2sN25quhjvTABK5KeC/t0JCwnwmPBuqP6nRcuwU+1cVg IelJDKpO3f/R8AKSG1DPewfJRp1HqMGS+2olYz0Y3zifkTkA7KrcydOpuU0Xj1aePYA/p5QJpmN Lm4P+j7lqV2YH3DfkJx8hPWuFxQKcDgKqLK3IaZydd0suP4Yx/JECn9e/dJ5uteH4Z/hassnkT3 Wno4Mo4pBLRKI5Ql2S9DkK2M= X-Google-Smtp-Source: AGHT+IHiOLaegPCqFhRjfODnSsdY9SrWFYGEQbdp54vFBIfPCpuzu4ZDNs0sWLbQVDRFdlPHmxWDbA== X-Received: by 2002:a17:90b:54c8:b0:321:2b8a:4304 with SMTP id 98e67ed59e1d1-324ed0f42c0mr2906974a91.10.1755767187389; Thu, 21 Aug 2025 02:06:27 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76ea0c16351sm1708937b3a.14.2025.08.21.02.06.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Aug 2025 02:06:27 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 1/7] rcu: add rcu_read_lock_dont_migrate() Date: Thu, 21 Aug 2025 17:06:03 +0800 Message-ID: <20250821090609.42508-2-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250821090609.42508-1-dongml2@chinatelecom.cn> References: <20250821090609.42508-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" migrate_disable() is called to disable migration in the kernel, and it is often used together with rcu_read_lock(). However, with PREEMPT_RCU disabled, it's unnecessary, as rcu_read_lock() will always disable preemption, which will also disable migration. Introduce rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate(), which will do the migration enable and disable only when PREEMPT_RCU. Signed-off-by: Menglong Dong Reviewed-by: Paul E. McKenney --- v3: - make rcu_read_lock_dont_migrate() more compact v2: - introduce rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- include/linux/rcupdate.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 120536f4c6eb..9691ca380a4f 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -962,6 +962,20 @@ static inline notrace void rcu_read_unlock_sched_notra= ce(void) preempt_enable_notrace(); } =20 +static __always_inline void rcu_read_lock_dont_migrate(void) +{ + if (IS_ENABLED(CONFIG_PREEMPT_RCU)) + migrate_disable(); + rcu_read_lock(); +} + +static inline void rcu_read_unlock_migrate(void) +{ + rcu_read_unlock(); + if (IS_ENABLED(CONFIG_PREEMPT_RCU)) + migrate_enable(); +} + /** * RCU_INIT_POINTER() - initialize an RCU protected pointer * @p: The pointer to be initialized. --=20 2.50.1 From nobody Sat Oct 4 01:47:33 2025 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F2FE2D6E53; Thu, 21 Aug 2025 09:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.195 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767196; cv=none; b=jCpa5PpAlvuKqYtyzbiHIeXnN/ix4pOmn5GtU/uthWaXCYZ04jhD8g/6IsuPGSBGj30vbDLk6xTDfPidipji7LNEThmMskesTI5IsMk4pcDuhU5Huo2egqLO+A7jPPAE3lPcvQXYfrpGJYoHx1pEJAXJcrmj3+y0Dkdboh7jXcw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767196; c=relaxed/simple; bh=bTIHaC7j+7J/bMwGU/BkRIOcDtOKJIMEeRQy/KliUC4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KLd5xaxcjxS7SfToPEa8Gp04UQmMgGsdRGVO4oaeoBnSI38yvEFXqKH8lrsnMKDFNlC7xO46c7hUPoSe7g/f/aqp7e+60jkmma4wLFkKOOGEGxKyINE6+QnCNUt96Td560tiqZUkbG1OkUk4FUkEiCSZcZL1aLWcbPgbmaL7si4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dg8cLrXb; arc=none smtp.client-ip=209.85.210.195 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dg8cLrXb" Received: by mail-pf1-f195.google.com with SMTP id d2e1a72fcca58-76e2ebe86ecso991800b3a.3; Thu, 21 Aug 2025 02:06:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755767194; x=1756371994; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XuCltBuqWN/DSCHw6nPVx35dtf5n5wtufyyXilS2WWQ=; b=dg8cLrXb88fj0GD0nLVyeZoh1ISA3k/YSAX8pigBKV9P5z072mKEkNIWlQsKwsxzh4 sd81YDEk0Zvy/UPrIC5S77noRdQeM37Xy2OnXECQNVqdzaJgzmaNy18qS1H6giBujD2Z OeL7piz4QwvHiRw2UUvK11cxooGG9NZiLAOFAOAvlZLdJ/Q+JsbhitqEv5foklpfmylY 5VodKGDCrmTZjewURCi9IX2GrCYA1HPuubTn9WdGcAeNCLV2lmwNCkPh2DP/gQpPaEEX mc3xTBdKSRqKfn/uPy4TkNdU8z9FWo7eKkuRDy726XebXI1xf6KFaYQAxf1FBDtfR9mP FR6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755767194; x=1756371994; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XuCltBuqWN/DSCHw6nPVx35dtf5n5wtufyyXilS2WWQ=; b=xDufCYeQczNX0YHkQ22AhRa10WkZ9hwTIXqZ9d+h0PoGee1EuQ40QRCsbquBD30ln7 3QQ+jasBTo8bOjW2XjzKexiuqIyfs2+xkxKrcZZ+vAjME6GojEBycgYxxWEOZsgPTtPF CEybN1p35Vg3hYX6AHPF8s8/+m8hl+aZbYv1ZUfBrD+qdSGyx7rwvBLuKznuwbEubNLQ +GYj5buKMO5Y9EFqoZaXZWBdeF17VN+DF4Tso2qUmDPPeTY+eB9mq572d6r0syTRsCtu Cx5Wwl7yGLFI/ympQ7ZPnSsx8WwKBfE3GJuPz3uotwUR6rIuBIQoCywfIAL2Yfg+AkJ5 lrMg== X-Forwarded-Encrypted: i=1; AJvYcCVjRRGpXpxaa7mHAv0SH46VcPVHj8YV2ONdqLwQMqpiawZylQizJXWiUZ5+Nh/6xhg9glo=@vger.kernel.org, AJvYcCVksyFx3wy/ZMCeNw9Q8QolEV+5WPN9V56pL43ais+mtZ98BplS9CxOLGIjMVeSoeUCcxzj@vger.kernel.org, AJvYcCX+ObHcdWrWeFjDrmFxWi5glyGEfAv55EgIMLAC6GvYQ1p6LKRBUNU6xOrDpkPwH+Qpy/LIgcfRwLQcdfzq@vger.kernel.org X-Gm-Message-State: AOJu0YzECJ7WcPHpc2SJjvsfpzebveq+4IQTH0pRpNc9no9+bbOkxA8L 742QnK3ZlrReOSGnSHne58+mkW6TepkY7wZhpq2wenVViAhFOLCjNrbf X-Gm-Gg: ASbGncvzfEDNPCjJcSyjWKXolXKczfHzIOTYo6C90abE2Gf5XmGxey4RzxmeivasqyG KakCMyIUn+P9wVMRnCvhb8Yu7j7xKSjdRZ3GDkbCOZz8LfbiHKuEeAxW2F5QMA+WH5KLs6nBt2b Tgju+d+jnO3og6yyCr3h40He34I9j6/WhK4IyAUqmmACKhIFn1nQEzYFFrgjm24gREWENr4Pc8p J/IbVzxtL4n0dXCIBjtmGZuFaMYoaGejnOLWwtQgiKw6w0N4wiQTB7OS9xYEA3nF4qx3gwYmfrY NvOpJSXDjPyL44Qr3Gi1lkTkcu25krC4lT85SSEqHAAXRphXIo0KabMjK8VVH/PYsN8z9LWBODr sb8HDanJ41MBKU4PUp6BEVFw= X-Google-Smtp-Source: AGHT+IEvGma+2JUsOCtxjSXnlbSq374x1K7erjquy/3NUbEf1gradL10rCdgT7ko5FE78MonWWLHBA== X-Received: by 2002:a05:6a00:928d:b0:769:93fb:210a with SMTP id d2e1a72fcca58-76ea324adfbmr2504056b3a.21.1755767194490; Thu, 21 Aug 2025 02:06:34 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76ea0c16351sm1708937b3a.14.2025.08.21.02.06.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Aug 2025 02:06:34 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 2/7] bpf: use rcu_read_lock_dont_migrate() for bpf_cgrp_storage_free() Date: Thu, 21 Aug 2025 17:06:04 +0800 Message-ID: <20250821090609.42508-3-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250821090609.42508-1-dongml2@chinatelecom.cn> References: <20250821090609.42508-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_cgrp_storage_free to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/bpf_cgrp_storage.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/bpf_cgrp_storage.c b/kernel/bpf/bpf_cgrp_storage.c index 148da8f7ff36..0687a760974a 100644 --- a/kernel/bpf/bpf_cgrp_storage.c +++ b/kernel/bpf/bpf_cgrp_storage.c @@ -45,8 +45,7 @@ void bpf_cgrp_storage_free(struct cgroup *cgroup) { struct bpf_local_storage *local_storage; =20 - migrate_disable(); - rcu_read_lock(); + rcu_read_lock_dont_migrate(); local_storage =3D rcu_dereference(cgroup->bpf_cgrp_storage); if (!local_storage) goto out; @@ -55,8 +54,7 @@ void bpf_cgrp_storage_free(struct cgroup *cgroup) bpf_local_storage_destroy(local_storage); bpf_cgrp_storage_unlock(); out: - rcu_read_unlock(); - migrate_enable(); + rcu_read_unlock_migrate(); } =20 static struct bpf_local_storage_data * --=20 2.50.1 From nobody Sat Oct 4 01:47:33 2025 Received: from mail-pj1-f67.google.com (mail-pj1-f67.google.com [209.85.216.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C1522EE616; Thu, 21 Aug 2025 09:06:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.67 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767203; cv=none; b=iAb4aspZGIKzVRgXro+QyYXrxAkJLDjrRBwQee981+OEv87CTTLlUn1Wi5RqVLcaIMSo6nV5H/9rm6K1mLEMGkCKDSC2kHBMTTPDycbz1L6WaZXToveEtSgQ4pMnndSxHiCR3tcJl8JKURtru/mNfDNMaPnwnXTN4vCVLL9gfzM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767203; c=relaxed/simple; bh=Ta+KXGP5P/G/yZ89XMbuCBtvz/po1cUgAXq+JKoB+A8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HXMD+yNMP/wg4DiVihXK4ujLnjXuYq/DrawaPZyeyxO6G7wqeZ+l6HfgKEMiRRcmcOPLIrImDJUqhlIYiw+a2IkmQWRep3N5Tpu+2PffLy4S0oC7KLn7cYeesRrVChx4BWAi20aludfLWLXdxZVvcaa0rAr7GTrpzrX3XT5JFT4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=DfoGFXys; arc=none smtp.client-ip=209.85.216.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DfoGFXys" Received: by mail-pj1-f67.google.com with SMTP id 98e67ed59e1d1-32326bd4f4dso630106a91.1; Thu, 21 Aug 2025 02:06:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755767201; x=1756372001; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9QVMhbhOtLzLkzu3HrcXzL+1NJ3SqUwSRILS3h61pRg=; b=DfoGFXysZDZZnLuHGedmbGJZi2uHWauEQ28ju98ABZjHIJwBBtuoxX1ngO19WZsjM1 uyY4neEwz+7OU0a3JmFuKWMoBQ5b8QTxpCUPZKD/mFCeXmicI+i0UGsHz/1t+qdAZnte yG8/UV6/tLKUWhjzB1n0h4Qlq6JUc3jIY30SIlHgjjAFXAaeb6wLB5KJ9ChnhvtWAdT2 Q7DUSVFPLcM7911jhl95c+pFSJP8tNGIME0qRbyPB8LDy6VL8SVLQTcm0R2DvGksn8t5 MMpY/8fFTyrc7jl9diSuyj0IKwDm1Jz4+vwnzRkFpXqWTs3KfOaqaCsQ6nYc4xPcwqZc N1Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755767201; x=1756372001; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9QVMhbhOtLzLkzu3HrcXzL+1NJ3SqUwSRILS3h61pRg=; b=TTMNVMgTIK76qJYDhFambL2alLFUfkHQFUjF5h7rljwTlC6QxZivxrnXK4QF8eY+yT AfJt2nOjgHxs9NPU0N1tBB97PxWTs9uTrQp+bAZa1SPOhOzFygoaXkvUJMhMr4IuZRz+ T5Vfrgdd8FE0J2VEAt8MS9fpKL0kGHL7RGbj9RZxPimI41uGnQHUWzB+ayC2e42nXeHh TKg98GYz0lYhVg10QyzLrTgMydGeQo4gaNpM+8YF31+RjxNqA5r0ywPpK2iasZYRaHhp V300bPfAN3S2i7z5afa4jHf+uBfien7s6anBAnZTKpORIUijhzlCy4g+7xibA22GGGY5 Y3PQ== X-Forwarded-Encrypted: i=1; AJvYcCUi5cnrRoqxICxFfn5XBPHM69cxkU88yJfUQAPlao+91QDYetSHV1QhPoiD5vA68pW/lN4=@vger.kernel.org, AJvYcCW0C2xo9QSkXe83munmGSo3lkoeOGM3VixV3bIXHiuuLlHMj8piGqsYkIo40BsB1p9Bui8b@vger.kernel.org, AJvYcCXmxXehteNUF/ctzJJz/2y+MCs6I3BmnI16kCgHtl4kBOLRUf/vk0yTWYdgQqQFwiUjCSsibkXsj5QLoCV9@vger.kernel.org X-Gm-Message-State: AOJu0Yy4CgBVXX9PuSrKPOYMiXJelLhX+8FIXFDgZQ0M3mdNbE6pG10g phnKOnuYakbcHJxkuLANyvy5TNRJWWsehId7xfEo/Rmqx15vLaIp9cd8 X-Gm-Gg: ASbGncu7Jpsb7Gs/tv4XHA2mtgTuvFKUc9YyGn5wHOdXR/hyYJ5WCM/ZTNhQ6+dHaLS YTdI3ZoZ6BSSRmqvN2rR4FadEbgWQ8KXADv1+bR+fkUJ/6nVqcqXSKkDKOmqRd8TqY92sM3J3OL KWwxJ+CP3QheqgHycsXMLz7D3rNqZCHAqtzmf5usarojgm3aBxs+z7mGy2SsJxU7Y2zgNsTtzG/ 0aFKnHw+AJ384LdBtBtDOtemflB6i+WaWoq34KYH7onW/y6d973px3eqPQaA5A/KefJ15yWnVDa B8YheDslBwbxVwTNMyKoxuONBA0YzHKZ24Hb7XrH9cKbn3lBogfQ4h/4jaVbc8u7wT8fDp/TMaJ RJd/9s39ENlTBzciejiPoEzKtG/TcvWj46Q== X-Google-Smtp-Source: AGHT+IHQaq8GeJuFXhkTC5l/8f8O+joWaAvZZ7nHcCehzsCAPrzZWRziMF5VtxqCODLpV0qfOtCVzw== X-Received: by 2002:a17:90b:3c08:b0:311:be43:f09a with SMTP id 98e67ed59e1d1-324ef3c6639mr1985454a91.9.1755767201325; Thu, 21 Aug 2025 02:06:41 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76ea0c16351sm1708937b3a.14.2025.08.21.02.06.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Aug 2025 02:06:41 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 3/7] bpf: use rcu_read_lock_dont_migrate() for bpf_inode_storage_free() Date: Thu, 21 Aug 2025 17:06:05 +0800 Message-ID: <20250821090609.42508-4-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250821090609.42508-1-dongml2@chinatelecom.cn> References: <20250821090609.42508-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_inode_storage_free to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/bpf_inode_storage.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/bpf_inode_storage.c b/kernel/bpf/bpf_inode_storage.c index 15a3eb9b02d9..e54cce2b9175 100644 --- a/kernel/bpf/bpf_inode_storage.c +++ b/kernel/bpf/bpf_inode_storage.c @@ -62,8 +62,7 @@ void bpf_inode_storage_free(struct inode *inode) if (!bsb) return; =20 - migrate_disable(); - rcu_read_lock(); + rcu_read_lock_dont_migrate(); =20 local_storage =3D rcu_dereference(bsb->storage); if (!local_storage) @@ -71,8 +70,7 @@ void bpf_inode_storage_free(struct inode *inode) =20 bpf_local_storage_destroy(local_storage); out: - rcu_read_unlock(); - migrate_enable(); + rcu_read_unlock_migrate(); } =20 static void *bpf_fd_inode_storage_lookup_elem(struct bpf_map *map, void *k= ey) --=20 2.50.1 From nobody Sat Oct 4 01:47:33 2025 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5AFA2D3743; Thu, 21 Aug 2025 09:06:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767210; cv=none; b=Bmvanx0boGZQ/T3hJTmO+ddevgXhwAwZOKSn2nDplRMljKkiQiyPKesGFh26AeqkKvykrpYyDmjRrEoHwhIMc+rsFyPglkG6HiFE//XaZ0aeHQRqWLk55czJ1YGY5cXMZA2Z3B9rtc9cE78rBlS48H3qfMnhgJXTlsMRdZOzfAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767210; c=relaxed/simple; bh=eqO46c3pAN3PfhGNOqY8l83f0ewMbPj8kQ9DK6CPRUU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Z32dPQvA2z2922eAxX7MgduoICS8j6QEPsQh/k8S/vBY3mo2trMs5YPQDVKWi75o93sLYfMcAZZdRhKUPhCFbNV7tRhD+KwICp4ICOBYrUgSjmbTxKpBV0XVAInNTSvlBOqB1Ygf6lmcPFnIXNC02mNOiq2lrcYvrvf+KlXlU8A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EQokETEg; arc=none smtp.client-ip=209.85.210.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EQokETEg" Received: by mail-pf1-f193.google.com with SMTP id d2e1a72fcca58-76e2ebe86ecso992122b3a.3; Thu, 21 Aug 2025 02:06:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755767208; x=1756372008; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=861OdxVnWv8IWFvxkcYe5tkgp3QAasS1lAYJfLFgMx4=; b=EQokETEg+O4W+MHbMDIvL+HCMw2eNf/XnJphfRNDMCNhvRCG75l86FgHhYdI2H8jG7 FuCiDIL3UoSsGH4UpQ7jREBZCHuamNYST2DP4oJhI1JqrYdKifEwGrMb8nlDWRqytDo0 D2yn3rfwzWQhSMfc0xqWsGU0+NtBpJr/tMO579srAh9uofFW4JISedVFh9mAb8m2znJM JUb6yHdHrOyzxC9bKm20a5CpBGITUuM5iyArFkNIDoY2xcQk08MkZtYeJCRJ9Kr+GJkf C/kF8vZ4TQ9w42Rcm4Ly2mfXKuGYsrFOHWLBNDB/OUc8OKuV8g3IB/Lz/kBpY0ue6gjV yetQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755767208; x=1756372008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=861OdxVnWv8IWFvxkcYe5tkgp3QAasS1lAYJfLFgMx4=; b=uqj3OboPvgUriaTThGg1pbIXKMSxsnRyf8Ppmf5TDOIQeTLtjCyWaCr62hopqAovdZ GrVTUfTZHbd9d8WXgH1H0+8I2+kzpJimwOnRCwcMaixogBqY/5+9yOTN81k0tnODVpya 9SdDCK0lll/w+wjTI5p6f0UZlGhfmY0b9dTRnZ44FqFprDv1ohTnH9fBPfBiEtyzNex+ qVaaZKa6t1rWu/bIYhhl0Y0L+EUoPALdhKCtNM8mfGKuLVo6uhMbQ77sSwU4G94pQnWh V58YDbLLYym2eESKEH9/QQWlZ9xlj6a6WmYKA33937xcEjQRItftkUAa88QRhiY4vXqR hPwQ== X-Forwarded-Encrypted: i=1; AJvYcCVuaLAiwaP/O4eAGxEjiFw/xTify3RMcSWQip7fQq7oKSaZO7UWenDQvjJVZ6w6U01PzPdG@vger.kernel.org, AJvYcCWCNcLLuKOjxW/RS9fpmWZAjY1EoxYKieeaNhaOBY9831suk3v0RfeAe34mSm2to4HqgJFKW7fQXR5hOYdK@vger.kernel.org, AJvYcCWnc1if8N+VSThDWnG7pxLNC1gprH2ImwkQ3iPdZLTt5Z7UaAa3ZhcKsq7oHr5Pgu18H7c=@vger.kernel.org X-Gm-Message-State: AOJu0YwIA4TColM9R8MbQ08nWMp5M7X20BlnIFWahc40DiwtOXRD9sls oeU4a1XKuKlwu8Dw3JyH3diIJnMnqEuXvds2HgVB2SY2mKfEEJMVqjfx X-Gm-Gg: ASbGncs79xzwrmP6ZoVOaU1+ctma+6YwHC44kXL82AYlJIakLK5EaZz2atLz/O8lbjP BR0IhLJHQnV5IbuQcT7pSwUpfYyembgtp6+wbQ8WWxnl2YXqARM11isdEb+hA59pOBtCRCnEmsw Bzsio/aEqq0P6Cv3o0/BlwP1nNfXX9soljKgaSJCBTCrZch3kp1/i2I6L7Kqg/dIHyij8x9m9PS r0SOJP94pNgU1Vqrc/pUh47OvgEkID+4FRQnhHqvRmVz7fc8eKeb3Q/jU3ANOjLqqYWpE2Y/OSS DmTInLf9TPrOHnlsLAmiib0mKudKLna/ASBhGY18qlpfeN4fQVnsiaYjV2pV4FyJKjTJuDAGUVn pIjFZBvjIkH/kTccxBevS8dpJCysxH78iiA== X-Google-Smtp-Source: AGHT+IHbFHBs2bkiQYDlXGS3VRmYlPSUwRBoe4kn5EMWdgE3xPMdKnwf0IZRHj5qQa2vZc81N7rGDg== X-Received: by 2002:a05:6a20:5483:b0:240:1ad8:17f1 with SMTP id adf61e73a8af0-243308364bcmr2503050637.21.1755767208009; Thu, 21 Aug 2025 02:06:48 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76ea0c16351sm1708937b3a.14.2025.08.21.02.06.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Aug 2025 02:06:47 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 4/7] bpf: use rcu_read_lock_dont_migrate() for bpf_iter_run_prog() Date: Thu, 21 Aug 2025 17:06:06 +0800 Message-ID: <20250821090609.42508-5-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250821090609.42508-1-dongml2@chinatelecom.cn> References: <20250821090609.42508-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_iter_run_prog to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/bpf_iter.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/bpf_iter.c b/kernel/bpf/bpf_iter.c index 0cbcae727079..6ac35430c573 100644 --- a/kernel/bpf/bpf_iter.c +++ b/kernel/bpf/bpf_iter.c @@ -705,13 +705,11 @@ int bpf_iter_run_prog(struct bpf_prog *prog, void *ct= x) migrate_enable(); rcu_read_unlock_trace(); } else { - rcu_read_lock(); - migrate_disable(); + rcu_read_lock_dont_migrate(); old_run_ctx =3D bpf_set_run_ctx(&run_ctx); ret =3D bpf_prog_run(prog, ctx); bpf_reset_run_ctx(old_run_ctx); - migrate_enable(); - rcu_read_unlock(); + rcu_read_unlock_migrate(); } =20 /* bpf program can only return 0 or 1: --=20 2.50.1 From nobody Sat Oct 4 01:47:33 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D79852D3743; Thu, 21 Aug 2025 09:06:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767216; cv=none; b=tp46+X27drdjKdtY8AEFU1sviHGZN6tHbRIe8nV67Y7qU9SJt9BMCgUwuMsY/ApGtMBrK5yxCKqd7K06S0W1Fs6WYWEB63+rnrdMrA1HdEIOtl0U+uAXGWopHapnMq6jm9G4a7ET7LEqn1Te97ibKCy+iJKWFYCzwv0ynx20ZtM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767216; c=relaxed/simple; bh=THRCv3ozIqTTxq6bSRfB2pdWOOnyKsRGYogE8fXxDkM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EAquyXcyNNjqrEl6Lp8YHEvrAfNmv1l6f2/R3Icft2erm1WIMofi5HPalrKR+JknehacGiWwQCtunNYe7yLaoPfuxjUWo0fbqHLJqZD63u1Eb2m1FwCEA76Slod1oj5AoLuWaq+thX1/wh4EdMV7TFEr37LvfjLofsUEJSqDT9M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KreR87Pk; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KreR87Pk" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-76e2e614b84so809227b3a.0; Thu, 21 Aug 2025 02:06:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755767214; x=1756372014; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6Bptfhu7onpt0MqrEIJlEVMcozR6INxKRYWqNZ+YESQ=; b=KreR87Pk/8Ik1Bu6Ox2657csl6HLtmbJiP8isLy8t0WmxN0HEwI10fiuAuLGRWCSnU ogrv2glUrXqfmulfrweLmgmjk0DTqkhKl5b7C83vD5GSlXAKs819JVJIVKBo7/QyQHEx Fl7FObv34ImKWFac0jhpdcKr8rzYHapQ1zm5cRTaBomKbdNPLQtKJV4LkgJ8LpXU8CND W2Q8Oirwqt5dWh+KkqKhd88EjAY8DiY8urU8BKI2EiIFI+YJkFcnIJmA6VSdo7UjwP01 qjzOZFvwZT87eet6rjxxLtxxoUVXbhvGzZ+kwGC7Pdk84NCn5fPdtqguxf5k1rffE7pw enqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755767214; x=1756372014; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6Bptfhu7onpt0MqrEIJlEVMcozR6INxKRYWqNZ+YESQ=; b=CSbAwkrBOGEOqaZeBCnhRadUgRopc52C+E+zfT4dPtDSTK7Qcn0eqE/VFc/i6JOKau ENz6zeoDTExiMuMpq7a+idQeyDf1FNbdgsKIXuwYJRRlsyzrgrYWzEe4xGYWn8cFwDrL LFwUiBOUmJZA1fmhe/pqgzXVll3Ahs3EmNm9bq0jwlI8vxGsuWcX8VW8PRpbSSyk7O8t UlpsBD4hWaseSBadV12H4Oh03GJ/Y0HR3FqL2ZGS6ATpK5DBXGp71gk4iwTXU/YDFveD 5P3Em65LzMhsuUzK7AvSwF+zykw68HWhhQySyX0HLYfXkqNextG4tBQ4EDdbablXGui5 aHmQ== X-Forwarded-Encrypted: i=1; AJvYcCVz/rlguJ0IlJ497BbLINfFhwEcVYassGS1Cwwpo/jiMJmfdai9OOWjKo1uk5RSYHBHOPmy@vger.kernel.org, AJvYcCW4XnJlqttQM+Pp3TR3UDLfX6aTrMkt7DwlhPjmEdW2tCQyba+j6psbmpZalZSLnrwOSB0=@vger.kernel.org, AJvYcCXM5cO1LdFWHmcHlSbYo6yWaRDtu3pZGOgPHE3hnT0rFvKMbqmu3/4HHYsUQmQ44n/3Q+WXT2E6p8aJOLCt@vger.kernel.org X-Gm-Message-State: AOJu0Yw4dOi4DnjQSXVvgKuMh3t/jNFD5jSu9tKg1gvVJmpLncvlczr3 dTCgeFBpdj1j2HDZvgI42ZaeE8HTNS2HfUmEXbs1h/zPsgpjo82l/VbTrWhVh4tm X-Gm-Gg: ASbGncvK7lNn9gsKbegsD/AFquLXzE/eT0q+/VRNiFtC81K8thm0THKS8ypNGgJ8okS NTM9rz+1m/B37niM0nWUt3NXsXCLFkYxpoU99/YF6dSK4Q0RKYb7yGdx4EVVqQDwZqLb/qSkD0J zQMTRTc9hnCRscnFpy+JxgHDnI4vJRqEMXABcqynwhKw13YQjT6nG1zCC193NXnYrG7ZCDiTaIh v4JPEwUowxRNSuPJQ8TPUQfAwlxIVld+nH7rFR/bkh89d4m8j3jPzSEilTLbYiqWmeGlbfE49VM ZrUa7N2kEiOgXR+4ZiXEQthePpJ/u05cXQ3SaqLu7GfTUTdV0vnACeCF8RDyIh7bkXglcaMXxd4 su3GYHH2vlsBG+zasY93WLK8= X-Google-Smtp-Source: AGHT+IHtehPLjKwQkbfI80ZsP1KmEWzAjUsow/SIdVqqBlEBCTQwPkmBZ6twbGxRWssp77QHtXhg5Q== X-Received: by 2002:a05:6a00:1823:b0:749:93d:b098 with SMTP id d2e1a72fcca58-76ea326368dmr1707755b3a.22.1755767214033; Thu, 21 Aug 2025 02:06:54 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76ea0c16351sm1708937b3a.14.2025.08.21.02.06.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Aug 2025 02:06:53 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 5/7] bpf: use rcu_read_lock_dont_migrate() for bpf_task_storage_free() Date: Thu, 21 Aug 2025 17:06:07 +0800 Message-ID: <20250821090609.42508-6-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250821090609.42508-1-dongml2@chinatelecom.cn> References: <20250821090609.42508-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_task_storage_free to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/bpf_task_storage.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/bpf_task_storage.c b/kernel/bpf/bpf_task_storage.c index 1109475953c0..a1dc1bf0848a 100644 --- a/kernel/bpf/bpf_task_storage.c +++ b/kernel/bpf/bpf_task_storage.c @@ -70,8 +70,7 @@ void bpf_task_storage_free(struct task_struct *task) { struct bpf_local_storage *local_storage; =20 - migrate_disable(); - rcu_read_lock(); + rcu_read_lock_dont_migrate(); =20 local_storage =3D rcu_dereference(task->bpf_storage); if (!local_storage) @@ -81,8 +80,7 @@ void bpf_task_storage_free(struct task_struct *task) bpf_local_storage_destroy(local_storage); bpf_task_storage_unlock(); out: - rcu_read_unlock(); - migrate_enable(); + rcu_read_unlock_migrate(); } =20 static void *bpf_pid_task_storage_lookup_elem(struct bpf_map *map, void *k= ey) --=20 2.50.1 From nobody Sat Oct 4 01:47:33 2025 Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33D88285073; Thu, 21 Aug 2025 09:07:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.196 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767222; cv=none; b=jlA0+6PPK2MjV1FezMuO/mJ+nCVlxJqOpLDWPOJbR6kub0gtMp2leXg9AbHW2B7NKb7LP1k9I/fVIaTwnmnVBG19p6eUcW+baiAcIyPFbw3jygK3llrCQIVqyZ6SwBouCjaXujyjFfdJa+XfgpLwoC+L5/ofzFUFfUu+1Yup0pM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767222; c=relaxed/simple; bh=fhjRoedQgU3UpqJ/vxJWw+P3CekwicN1Qc+dzHfUufo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NWLAqSIvz6J7pmCEWWu5AFnehqZImPkIhyiil9UFPryfp/dYHBSurV6JnkvK3XCzJI8/MRxU8fitoXl6mn+kUMq62X8GeMuqJ8vlwZuMCWXuA4z2Nkl/guwkrrRGUj57zqUi5/Nt184vmc3GKiX2vDyJ/F0Ko8GQjhidL5+f7C8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Nh8tueuh; arc=none smtp.client-ip=209.85.210.196 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Nh8tueuh" Received: by mail-pf1-f196.google.com with SMTP id d2e1a72fcca58-76e2e613e90so603815b3a.0; Thu, 21 Aug 2025 02:07:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755767220; x=1756372020; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j5H/4foa1XV+6AOUTU8S+hYtTzGYqXPj/gnpmsQWpYc=; b=Nh8tueuhLrbkJFwOikavq1lcfq2THU3ddm/qpLYO3M/KxtOTlg+AC+pYVVEXsYivbp iMtYw7RMo4gcTyTapA8F7DKFtWXvEPdKNe7YYVx8Uy5Ui1hb3EsR0RvrJeSljJBFRjL3 nAH6g/pdSXCynSSxZVP5EMGj77LfQKksxIVwRO8vsk6mVK6xZqkDLzFXGEBcVTfz1jP0 1SudHmXAf0IwUPfH7+X5+KHZv0G7GVMyzg+xeLp0tV2qS5pswqwuVQsp4L5WFX8r/eO3 XDOUVsWuSaywiabWiwHzJ+MfZZDuTxELoUmXrMleEMSZ5SjyC0FYYJxPL6vAYPbnv814 0WJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755767220; x=1756372020; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j5H/4foa1XV+6AOUTU8S+hYtTzGYqXPj/gnpmsQWpYc=; b=Cy7IzdQT5OMfyb8LP2dIFvSmQnnBDoGo/c+bgvcIPBnwfeZelGMZo7lHZFUA04JWkg HOQMtDMSrAWdkA8w9YFii21YavHzNx/jPEyqAZTOeB1aK/xl2MUrIBloOy0l/oqXeOWd eiBdaDVN88mVtHGG6KVsLANOE6deMqWripKRZCvTeWH4XQ1bvvaUtOC125aV0PgUrKWD sDd+KAmX1I5iX5AGCc092HbHUhAvVyZ350gYe5eMTJpg9r9HJB7v1iLF2YscoAZBWf1/ 8GzngJ94J/Nv0PZf6XPT2ck2JdmPrMAkkGHCbAJvKw12aEzU9tjWGVDZhD64vX/XIyDp RLQw== X-Forwarded-Encrypted: i=1; AJvYcCU16x/9Qhg/dd1FJXxkWXS+u4LqZYDIYqUSXo2x3iNvBkJ/VY+f7P/9uK4tNL4lWpjzlqg=@vger.kernel.org, AJvYcCUbnjJ6FakY8ymdMGv8pvt+GtZ3MdduSx7U9Vs8uHr+EHLhLO/KieZM8oK6UulNeN708F9S@vger.kernel.org, AJvYcCWkD74OkcU4cJcZJEe8y4W6oDu/wHWl0C5fPsGCvwwetYTePQEj/Q7cjoqWzLgrai7Fh/AkIPr99faXufTi@vger.kernel.org X-Gm-Message-State: AOJu0YzyAbmfgOdVzJIx8/9I07kCJA22BtuyymydrX3+VBGjTkIKWq0e lNEfXNm0lOdIuQiERYy686VQPp6HZYNRutP7yn/dfDvCCviqYh5xcih6 X-Gm-Gg: ASbGnctHeqj7w1yccpvZzXnkIqfjhsMBHOZu5rpajKSTtEqYfhapNDxVYgAxxUilsRq SwAUYkYzLaYkNn9HIyOWQlVaydEn38BNlOqYpZwWTklEbefzvr15s+f8kbBXizQy98WibUHpWo8 7ZAA2FS99nK3iSxjxEGjAgaAx1x45908mD4Fom9S5I0/miyiPAZp46iee3IRhgs/uaeumVAcpdh S6leJnYLuB54hHhk/xSMOHoIPTqO+GXGK58dUMkE59/9Rc8X9/pjsLykT6gkj/c8BQ1ODgzQEEk zpFUdnShd9vcuO8PD1nuxYf/j2gL7Yq1aThxQA/g5HlHTJAHNi4qMi28zrWUdRrOobEb/Az8CWf 5zp1Sqhn5+FIvqEdPgWUGZj8= X-Google-Smtp-Source: AGHT+IGe40Ad65Vhn/3nzydrqmDL8HgKANG9G/saMkqzFmwS45QqlSxHnNhcDOi3+PI23Wqs6ssU4g== X-Received: by 2002:a05:6a00:c90:b0:76e:885a:c3f1 with SMTP id d2e1a72fcca58-76ea3283b23mr2208384b3a.29.1755767220200; Thu, 21 Aug 2025 02:07:00 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76ea0c16351sm1708937b3a.14.2025.08.21.02.06.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Aug 2025 02:06:59 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 6/7] bpf: use rcu_read_lock_dont_migrate() for bpf_prog_run_array_cg() Date: Thu, 21 Aug 2025 17:06:08 +0800 Message-ID: <20250821090609.42508-7-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250821090609.42508-1-dongml2@chinatelecom.cn> References: <20250821090609.42508-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_prog_run_array_cg to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/cgroup.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c index 180b630279b9..9912c7b9a266 100644 --- a/kernel/bpf/cgroup.c +++ b/kernel/bpf/cgroup.c @@ -71,8 +71,7 @@ bpf_prog_run_array_cg(const struct cgroup_bpf *cgrp, u32 func_ret; =20 run_ctx.retval =3D retval; - migrate_disable(); - rcu_read_lock(); + rcu_read_lock_dont_migrate(); array =3D rcu_dereference(cgrp->effective[atype]); item =3D &array->items[0]; old_run_ctx =3D bpf_set_run_ctx(&run_ctx.run_ctx); @@ -88,8 +87,7 @@ bpf_prog_run_array_cg(const struct cgroup_bpf *cgrp, item++; } bpf_reset_run_ctx(old_run_ctx); - rcu_read_unlock(); - migrate_enable(); + rcu_read_unlock_migrate(); return run_ctx.retval; } =20 --=20 2.50.1 From nobody Sat Oct 4 01:47:33 2025 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F0572D3A6A; Thu, 21 Aug 2025 09:07:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767228; cv=none; b=f22Uy2s8e6LRcRnd4xfUKAELwWSEWVKDT6x0/oJbV1sE/FwchapFTLsLnRsMF5qmewaDMbZMerHjLgX4U8pRLP8eOkeUXuyO+qrs3/bUlAREPSQ1oPPBzTqXASL2o+DeOOQk2WeDpdf/m0IxCksiCsWFXBJ6QCeTsZHRfqUKrMg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755767228; c=relaxed/simple; bh=6Lh4b5LMBBzYETFNUKgsMw+Mg8slBT6PgN9WlZyKi/I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RQ3OlDwRM0txVK9HnyN0N8pfzQfWCxaxriKaIl39MvMw/P4UmDziwqxMAV3RKsMi6SfZDt6+1It/KrA1yNou0ST5MnJ7XiBqWw63sWTKFtStuIcgjGfi77q9fcT+40dToHEku0epKoSC5kxxkHNCJd1q4eU+lt/+8qD9y8ZMwo0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Ao1n+ecf; arc=none smtp.client-ip=209.85.210.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ao1n+ecf" Received: by mail-pf1-f193.google.com with SMTP id d2e1a72fcca58-76e6cbb991aso729468b3a.1; Thu, 21 Aug 2025 02:07:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755767226; x=1756372026; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qA6vzGkbuWAEnBwZb/6LL/xPx2BmZVoZP3qls2M6vww=; b=Ao1n+ecfl6Q43XMXVjN2memkN7Ay8VLhMyhZTFIxkqu1Sm2suh84xXLQtFCi2Vr8zr HT/Yd4XG6d4m5YwPZDdYggk5YMNPIpaSPID1/Nnoko098FdZjqevmgthdcuDoV14hqDe Jnxj32Qyg/nwcFdTzyEnnZXodufdm7/9N7EWpf4SvBic4+pY5FULUXzOEw4C/4eVsos7 AwYOT5QSVtR+xZT+2TDFlcT3tFyJk/cYY6KzAvm4sHQ0QxSfZErgdl2FviRObjwUFhsw neLyJabsR9g0chB+yAKl+w2bFPmipQcgs2a8eDWZruPIqF0EuuczEkC1RcIPJKz78LjM 8C2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755767226; x=1756372026; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qA6vzGkbuWAEnBwZb/6LL/xPx2BmZVoZP3qls2M6vww=; b=YV3lyBvM8y6OxXLro501VdU8zGXDasO2rxYSJ1WHZe88jrJTcsEXk/Z8hlLerKzK7d eQaJSWshQSpzFVjfJTz5GaLv8vIdTqsfltbx+zfSwRZf22hmIbVLOjr2rAF8QE9Vhumh cB6pgaTIpoZhkfwKlBGv/T2dZK9qVro/lcUagLCqf8usX1ZGd3lNHV5kMsae7YBAWsKf 4VYTc6mlIeLMXCTAl+ElsXnI3pkGrN8JKKsLv6h2/g/IaFcMrbyHhaQOLy+kpanlMowX RarpPuMp+TpGIdCdin+5SmHQ3Sr5S47fU3r9YuoaLIg274l/Urx79HxeCVixCUF2ixtw Mjug== X-Forwarded-Encrypted: i=1; AJvYcCUYt4qww6PttfpzUV+SLtHqtLjXuxLnCyuIrCUt3HiB6lyOFS/KwVYuUr6GSh3+W5PRlIAxJjC5yFW3jRkc@vger.kernel.org, AJvYcCVrv4vD8oO6IHt/KpDSEFLo0eIbXTbdaDFDgauOkhJpn3GkxxpDUKAHZNWI6GNp9F1duEJy@vger.kernel.org, AJvYcCWjtACkGMJW1xXQSFoWWr7XstjfP0DcQMpST0k/E//5oj/hoSdHJfQHdNj7rJBiyE9Fbx8=@vger.kernel.org X-Gm-Message-State: AOJu0YyuG/fo4nY0nXKYjlyNAxHXodcHJddhTYBpTcCOHaBKI8MnpSq7 Cl0RxoP5+sofzdKiEV31b0R7h6D5zCxWAab8pMRo53AwDpuk9B5gWhCj X-Gm-Gg: ASbGncuvSx11eqtFIzzq8ij2cbn17cpNlFygZ42so8QHWtEf7egdLAD20mikjKroZ7/ 1A4NMRs5h1wMsrKvK4picAzjW/aHa5IG105U0yByjY30G5xMN9m33kiRyD37R8uS1ZCEI1kSW2l TodcAhK/aDMCA0yVXLp1nAWc2y2yXRG/bGY3T6hqObsmj2eDSl0633zmK0N6MoTo2oFNvjIwrF5 sYKiQ2BohAUWxOh2ynrp3CdQWov062l4iBwH9o5PGheKgBiXbDjts0AWZGpNHZIHo1houGRoQ5c +oBIVqm/RHq1PmAGffGp29DBUA/L9YQmvySsilu/DMU39nKNivbmHOgbXPntkBm1Uz2U+b9C0RG IbooPvRXEeSjdd361LgeKazA= X-Google-Smtp-Source: AGHT+IFAxs155WkEAovxKXV2Qp7o7Ce69E4eJSaP9vrtG7bqjEQH//jftMOatslb46rp0Rv4pxscWQ== X-Received: by 2002:a05:6a00:190a:b0:76b:f73a:4457 with SMTP id d2e1a72fcca58-76ea314f59fmr2162223b3a.6.1755767226418; Thu, 21 Aug 2025 02:07:06 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76ea0c16351sm1708937b3a.14.2025.08.21.02.07.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Aug 2025 02:07:06 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v3 7/7] bpf: use rcu_read_lock_dont_migrate() for trampoline.c Date: Thu, 21 Aug 2025 17:06:09 +0800 Message-ID: <20250821090609.42508-8-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250821090609.42508-1-dongml2@chinatelecom.cn> References: <20250821090609.42508-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in trampoline.c to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/trampoline.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index 0e364614c3a2..5949095e51c3 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -899,8 +899,7 @@ static __always_inline u64 notrace bpf_prog_start_time(= void) static u64 notrace __bpf_prog_enter_recur(struct bpf_prog *prog, struct bp= f_tramp_run_ctx *run_ctx) __acquires(RCU) { - rcu_read_lock(); - migrate_disable(); + rcu_read_lock_dont_migrate(); =20 run_ctx->saved_run_ctx =3D bpf_set_run_ctx(&run_ctx->run_ctx); =20 @@ -949,8 +948,7 @@ static void notrace __bpf_prog_exit_recur(struct bpf_pr= og *prog, u64 start, =20 update_prog_stats(prog, start); this_cpu_dec(*(prog->active)); - migrate_enable(); - rcu_read_unlock(); + rcu_read_unlock_migrate(); } =20 static u64 notrace __bpf_prog_enter_lsm_cgroup(struct bpf_prog *prog, @@ -960,8 +958,7 @@ static u64 notrace __bpf_prog_enter_lsm_cgroup(struct b= pf_prog *prog, /* Runtime stats are exported via actual BPF_LSM_CGROUP * programs, not the shims. */ - rcu_read_lock(); - migrate_disable(); + rcu_read_lock_dont_migrate(); =20 run_ctx->saved_run_ctx =3D bpf_set_run_ctx(&run_ctx->run_ctx); =20 @@ -974,8 +971,7 @@ static void notrace __bpf_prog_exit_lsm_cgroup(struct b= pf_prog *prog, u64 start, { bpf_reset_run_ctx(run_ctx->saved_run_ctx); =20 - migrate_enable(); - rcu_read_unlock(); + rcu_read_unlock_migrate(); } =20 u64 notrace __bpf_prog_enter_sleepable_recur(struct bpf_prog *prog, @@ -1033,8 +1029,7 @@ static u64 notrace __bpf_prog_enter(struct bpf_prog *= prog, struct bpf_tramp_run_ctx *run_ctx) __acquires(RCU) { - rcu_read_lock(); - migrate_disable(); + rcu_read_lock_dont_migrate(); =20 run_ctx->saved_run_ctx =3D bpf_set_run_ctx(&run_ctx->run_ctx); =20 @@ -1048,8 +1043,7 @@ static void notrace __bpf_prog_exit(struct bpf_prog *= prog, u64 start, bpf_reset_run_ctx(run_ctx->saved_run_ctx); =20 update_prog_stats(prog, start); - migrate_enable(); - rcu_read_unlock(); + rcu_read_unlock_migrate(); } =20 void notrace __bpf_tramp_enter(struct bpf_tramp_image *tr) --=20 2.50.1