From nobody Sat Oct 4 08:05:11 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A18531AF01; Tue, 19 Aug 2025 09:34:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596083; cv=none; b=P8QgVzSMRPw5zD1pcnXPcy8EM5WA703IQjuTszcCc62Pk5jJGjivIWw16WX44Ss0WYBMwePnQPRt3G3H0FE/oowhMuxgFXoBNLfgTW4p7H4TpVEzrD9GusPEXgLTG9iSNeYF4rCbdvStXYj1hOrIQ+0C1ueAKPwZRIbl2am6OjY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596083; c=relaxed/simple; bh=4DAxj5XfxLfONQDlEFKob/8R6S47MeAwkTppdmAYlP8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V1h6VfYfNpfCvLBGxhTp8fDBPHYb3TOduHwBroT3xoKz5TWBzTQ1Pbt9pprRsYoKLkdRYnfetFVMBC/B6WvqMsfA3RILpQg/8Ldr+MpihN1jsT7sw2iK23sBwtVnm1MHckIpgqfCiE85JKmwrccqpuRG1GzXIigjKJxdzCOTsXk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TF3H8SXQ; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TF3H8SXQ" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-76e34c4ce54so3805481b3a.0; Tue, 19 Aug 2025 02:34:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755596081; x=1756200881; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NaB0vGBq5NJv4vKYct/lqW2Woq/641IZ+CLzkcU8pg8=; b=TF3H8SXQPpyqfprqCk/goOCjXVj5X7vBtzBcR/YeMx9xyAwacixtLlu8GE0H254sD9 lWVvSCRmvdbkJFesvEMqUxGCubzc/ymFYsvGS5EcqCNOaxnkGrHI2OFxnaleluzFfTQn FYKcBzpVRK9B5tRsm6tOZaCKxWRe6tO3mrnjLosbfzWZdytCbw8Ymo9RbcNDaRzl+Uhn OeCUTM1EAb472hSENxewRzUuHauIwR6/HoeP0CdmKhfMuua3sZZsdKlvt6XUgb8X5da5 gflvVprFOyCuQJer0ys1KYK6pRppweCSWweP6OFyE7Xy8rjtJlvllnsBi0FfmqRnmktD bc8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755596081; x=1756200881; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NaB0vGBq5NJv4vKYct/lqW2Woq/641IZ+CLzkcU8pg8=; b=j074gRwmAMUsjoyyepDpcw3hDq/HTveOqmQQu+JQqBE+/m9bLiAApesxLzWe+mYEDm Lr597FBXSbut90XN2iuZWnrKEBGG01393F6N9WCAfIP5o6EQk9SxGfmMWk1fmSP7fjk+ xh+B7apzO1eQbmANL3bxd3047Y3Yw/s2EHXNqdHLiJ0wLGXr+s628wTFxAISB8MwBfrQ mb1hJUenm6/YaSiIMHhF9Nc6EhBGpt38ru6186JdEQk/zsRjhnCixLSFx5B+fklMgp5h fg1Cnd94PWQNEcrUYZ94jbKH3FRILAL61+6cuTr8g1Gku99WU2/pcJVNzBAM+pOKMGQV IbZg== X-Forwarded-Encrypted: i=1; AJvYcCVrNnEVWIjrH5Meb42/dvKYmIICqXN33pjOKlGX+ruAKUNEguDdbGOUdxLyjjy8t8GTt30N@vger.kernel.org, AJvYcCWD3sVq7KgiSW4myhOgzysY0zVPNaTgf8elrvj/q6ZNfoGnUlVKmMso8eSrXAPlX/6eG7Y=@vger.kernel.org, AJvYcCXxPc5rRCzpObcaW2PH3pMmfEGjlHV1wNKMJhYDdVXGlmC2DyG6nTTZTLIwAnAbFePQbEsVfqusHEgcj/kg@vger.kernel.org X-Gm-Message-State: AOJu0Yxd7tt1TtkfmzODNChk8qmRtGH9E8UuePOkoLipMem5GQN2wW3H Zw4vrizTeVfdziX08pb37bKh8etdrpn3MzaAWN26+jrHvtxS4G+zoopd X-Gm-Gg: ASbGncv6uodApQT7ppZvqN/qUvDuQZVZiHGy8puY4vYK2zeqv8rRB7VHjds8BPwsTLz 5MPiNIVy8/Fz4AudKXFMvYR/pNXyJw9bBzvV2a+78q6zY0XXN8UplgAXauJ0r4HhT9xEkvGTo/3 0vW1jbyo8XOS8epGVhEB9IB0+cWMnVSMU4jV5165IfUoGlNfG1gwGQL9WbCP+Z0gcpelOEUnv87 HySNcPCjJaWfcfJQ4/VYS0/SqBjziuQhaC0aQOtp6zlv1H52XdlZGnxWTnLljSZWfVNQ+qcF9Fo gzeluXF5hfwe9CDwJDyHAsZMhNBL+lG+HMBAFpJ3k7CvIFpR5hmpDDbtNW27sQNsVRT0G1YWZt5 1tJVoYnrpvpJqvZgRg8A= X-Google-Smtp-Source: AGHT+IGonyZpH32TOEzDsZQXDq0zzesmgfi6wekH2vYDybV2Qk5abK+OPuJSljOJbLVWwf3+VArj0Q== X-Received: by 2002:a05:6a00:8707:b0:746:1d29:5892 with SMTP id d2e1a72fcca58-76e8142b14emr1367579b3a.4.1755596081269; Tue, 19 Aug 2025 02:34:41 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76e7d1314a4sm1990945b3a.41.2025.08.19.02.34.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Aug 2025 02:34:40 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v2 1/7] rcu: add rcu_read_lock_dont_migrate() Date: Tue, 19 Aug 2025 17:34:18 +0800 Message-ID: <20250819093424.1011645-2-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250819093424.1011645-1-dongml2@chinatelecom.cn> References: <20250819093424.1011645-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" migrate_disable() is called to disable migration in the kernel, and it is often used together with rcu_read_lock(). However, with PREEMPT_RCU disabled, it's unnecessary, as rcu_read_lock() will always disable preemption, which will also disable migration. Introduce rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate(), which will do the migration enable and disable only when !PREEMPT_RCU. Signed-off-by: Menglong Dong --- v2: - introduce rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- include/linux/rcupdate.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 120536f4c6eb..8918b911911f 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -962,6 +962,30 @@ static inline notrace void rcu_read_unlock_sched_notra= ce(void) preempt_enable_notrace(); } =20 +#ifdef CONFIG_PREEMPT_RCU +static __always_inline void rcu_read_lock_dont_migrate(void) +{ + migrate_disable(); + rcu_read_lock(); +} + +static inline void rcu_read_unlock_migrate(void) +{ + rcu_read_unlock(); + migrate_enable(); +} +#else +static __always_inline void rcu_read_lock_dont_migrate(void) +{ + rcu_read_lock(); +} + +static inline void rcu_read_unlock_migrate(void) +{ + rcu_read_unlock(); +} +#endif + /** * RCU_INIT_POINTER() - initialize an RCU protected pointer * @p: The pointer to be initialized. --=20 2.50.1 From nobody Sat Oct 4 08:05:11 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C43F31B125; Tue, 19 Aug 2025 09:34:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596089; cv=none; b=Gcen1ArMwF9FMtAta27iz+poavD+KR2BUKjSq7wLNJ3LhIzmErgxFVzx5un69/3dwSmXt+4kNt0UR5deiVx3hHWfiLPP3f5+vjG65ltpE+BPHseWdKz7fpT5Z2wlRF+OZnfVuQ/lVQdVwXklblQbDMOWzdx74MBa7fhmYPm438s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596089; c=relaxed/simple; bh=bTIHaC7j+7J/bMwGU/BkRIOcDtOKJIMEeRQy/KliUC4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aHMYkYN+Y2UcqssHMUXZ9j1fuqcYXZrHPX/TB4rYAXeakr5zVjBQ7BY+vA792aZa7f+za67HriYoc+pcZoVNqUN97NXNTN21EIENv95OIra6Cj6CnrVhrwC4F9GItPyniixxOa4XZslYpx2kppqXuMYMfiMRijxRKBKaJotjYTc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NSgVELyh; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NSgVELyh" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-76e2e6038cfso6032632b3a.0; Tue, 19 Aug 2025 02:34:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755596088; x=1756200888; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XuCltBuqWN/DSCHw6nPVx35dtf5n5wtufyyXilS2WWQ=; b=NSgVELyhU3tpSs5vW9/N8FjFZm13ogYnwZoKxqNvoiGOuTOPr06IXGMXEh/mSDOspo PpilO5J3AwS+oMvZXgUvFXmV83aSvJv6M0A/mRoj1317iFZrxkfJ07BJi9Zsj1Cklhqg f8YUYC1r6zpSPyaOMD60cbgGK83Ottjk/mRkZifluRJhHe/5nfhUmXk1h1FuHBl/cn0q DGJO4XuTNv7pclCRLN4hYDgOxceS+M7LJ+ugOa04W/KO0KcreIlKUxmyJKioo5I2CH5L ipNyemhLpZIdyahrqa7W7Cwirb16GBnDfXQ7TFEiMQd8xbwyWI2is6bC9R4Lewu2R8KP /7sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755596088; x=1756200888; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XuCltBuqWN/DSCHw6nPVx35dtf5n5wtufyyXilS2WWQ=; b=Vz1t48adS5kzsMQ20I+nMxpU5v5AJhD1q+M9QVs2ysrBut3cV3gjV+GbwPnxgMBYmy VDYPw50PRfhcFIwV2AZ4uffU5h3W0Wu171pzGeAzNX3cjjkNv3xvcZrjzv3aqJd+jO2j ZYpxJYyEjK5wLDzCXdDrbB9A+2OFsAPZe6EdU/X2Z9DuDq6h1NsjBXGfKaBNw5WdsEi1 jmi/F7tR7t3DybX8HtFVBFShIIKqKUsrckzC/3mdEGqLrjVtAPU9bC5nL92SP0qO3zKu /Lql2b6zwM3FHW+vXRL8i/r06Ryn2+XOTcZrnpc9EYALqpWPORqpueuKmfTqyb8LFVtT CKYQ== X-Forwarded-Encrypted: i=1; AJvYcCUQY9ac5CEnyXtXfctzFQ595GIczXnobfBpV4uUzmcH+nlZX6VpUOZuHYXzVGwf5YAqpts=@vger.kernel.org, AJvYcCWMJKedk924rAT/hVZxOVesDRoKWeFQJok5+5Yw3c2s94i0ryfq5Ryrvyw4hT634sWGsaM33A4hrT7fJ+ks@vger.kernel.org, AJvYcCXpfTn/BhWB0bpW4B1cgI3KtcV+k6ErTn8bAn3MBeZUjOl44Pd8+nI7khHkh1sR882ZB9Gp@vger.kernel.org X-Gm-Message-State: AOJu0YwMn35otF6nZb5xYWFPhkV7RArWeHTjdnM5UhbmKzd6cTg8zXAP VFuIraWAGsYNRfBg2qGZpwymXBA6h0SMS3Ff71bNYDwytFeg1xhbLQ7p X-Gm-Gg: ASbGnctUqVDeKJh075fiL/2S9ZVjuFVkfQDo5nVwn57duFF5zhbmHiYqQA8s18fTG+s B+IATxoWxn7umrYOuDWaxLZop7BWh6buqRtg+3kF+82kglB5GzvJtEqwwNEUyis0wotDqIEHkwu f6+7z328bV/zahIq7hZr8TvjsBiYEabb7c5D1EXpwCsVdb3sJeWC3U3DW792Njg8TlLGawANAY0 9fzEXGnNt2aukQF2f74OtcLm1EQQQR+S8P9Igr96oWzg6dj/gpb0AhRAhI5bMNCGU8n1u87RW1c cqak6T2h7J8g4lD56LfsPA4kGzBR8q88APOERVElkE3c6Fu1npCjVpxV1xgdnC2RgxenDzOZl2T //YhR76kx0tyOLCCtxBQ= X-Google-Smtp-Source: AGHT+IEWDXMb2Z+W99ZoDxKTIlJ16JU5fmBRmoDh0U1K3FBxgJBIF0YoVmhCPXKCDJPP62gTuqMl4w== X-Received: by 2002:a05:6a00:1399:b0:73c:a55c:6cdf with SMTP id d2e1a72fcca58-76e80eacd4cmr2178971b3a.1.1755596087532; Tue, 19 Aug 2025 02:34:47 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76e7d1314a4sm1990945b3a.41.2025.08.19.02.34.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Aug 2025 02:34:47 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v2 2/7] bpf: use rcu_read_lock_dont_migrate() for bpf_cgrp_storage_free() Date: Tue, 19 Aug 2025 17:34:19 +0800 Message-ID: <20250819093424.1011645-3-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250819093424.1011645-1-dongml2@chinatelecom.cn> References: <20250819093424.1011645-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_cgrp_storage_free to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/bpf_cgrp_storage.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/bpf_cgrp_storage.c b/kernel/bpf/bpf_cgrp_storage.c index 148da8f7ff36..0687a760974a 100644 --- a/kernel/bpf/bpf_cgrp_storage.c +++ b/kernel/bpf/bpf_cgrp_storage.c @@ -45,8 +45,7 @@ void bpf_cgrp_storage_free(struct cgroup *cgroup) { struct bpf_local_storage *local_storage; =20 - migrate_disable(); - rcu_read_lock(); + rcu_read_lock_dont_migrate(); local_storage =3D rcu_dereference(cgroup->bpf_cgrp_storage); if (!local_storage) goto out; @@ -55,8 +54,7 @@ void bpf_cgrp_storage_free(struct cgroup *cgroup) bpf_local_storage_destroy(local_storage); bpf_cgrp_storage_unlock(); out: - rcu_read_unlock(); - migrate_enable(); + rcu_read_unlock_migrate(); } =20 static struct bpf_local_storage_data * --=20 2.50.1 From nobody Sat Oct 4 08:05:11 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AAB5D31A046; Tue, 19 Aug 2025 09:34:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596096; cv=none; b=irpZw8eVeRazZK9H0pDHaAlSW7ayN24BOC9wX0Xbf2cHgY0tkCpUV/B51x0sI9mlC1KH9JBYiX/f/qsiQ+J7gZD7Kdhp70sP/lTqFXMuNOzQwSeD3ct6hx7zpgKMWJvPawPDNseHA6Ey6K4EWXxd07DKmdVDcyzLCOPiosgfzSM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596096; c=relaxed/simple; bh=Ta+KXGP5P/G/yZ89XMbuCBtvz/po1cUgAXq+JKoB+A8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DXmD+aTR1z+n0jQ8HdXCzaTCIK5TTFSTYVujSkM6YJvKsKpee42lOb31a4iAGu7h4lzZZm7LdC5qVSCt20zS+e+t/UpM26CDhSM4+vup1XmmIowS2TRDzJ0+LT4b+kjnm2kcSrLH4IumdLk9Kb03GiBwouTfaBf8DDG9ig4z48M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KIsn3gC0; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KIsn3gC0" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-76e2ea887f6so3768545b3a.2; Tue, 19 Aug 2025 02:34:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755596094; x=1756200894; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9QVMhbhOtLzLkzu3HrcXzL+1NJ3SqUwSRILS3h61pRg=; b=KIsn3gC0ISMUzggiwttifJVezAwLjLV15prPytig2Lo41OHZ0ra4eZUHqdo7fXru+5 m33qWrhDSeCks0bLtAx4ygHRlH3Yj8SZXpyQhUQ5evL2brDee/4oM/liu/oWXTmVH/Fa VN5BzUES2rtsGB1weY+spNDTTQnuGLRKaW/Gid/G+ZN6nz0abugo34OItahm20bmC/RQ Ohb4j3Dy419Y7XVyVaQyk6TMHyVNP/+LOCqrRB4GpFjtovoAYMFWUw/l6c5pcZFzfYDr 7Dd6K36iy2slJidcdNP1Vhg0smQCxjhljdExZCgvFxa73yGyHNNDVonQYbAkGdV3bwE6 fgjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755596094; x=1756200894; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9QVMhbhOtLzLkzu3HrcXzL+1NJ3SqUwSRILS3h61pRg=; b=hFa07F5YrPuR7QFAswj1sJ64weOiGaI0pQub4RbB0BdZiLW3hSrMr7LSdviCYJXfYw SvcxTwvP1KJ93h2A8+wJmkEWqZgS63ZSkPf76U+f1B4sgfuUO81TI+uDEoAPuSS4IT8L hne34cLD5tllHcdaMHHT3n9kdkJnalfZwelQQ+C+9C9JRZX2dBmh83FudmOktfhAfB7c gaQ6hC5HD/badxDeeQVklpfNOAObsXMFmkio0hX7/n+fTwJqIqjOFPR3HRBxnuM0l7SX rXTmY1Vc3uQ83bPPuNiPI3ItdLwTuLyZBz64/V7dYpdol03ZYcbQLWTUEJcviSCcJ3GF tLgw== X-Forwarded-Encrypted: i=1; AJvYcCV+rTxniB1pnr0yi355njExTphwvgktJ6tzxmn/g4D27kJj5rs1HibQZyitLocNboVsaAmksKJO5vimv/7q@vger.kernel.org, AJvYcCVQwUxYdDNMhwmsUbT/t7icu+8I1HsUYkHPWHRYPywMcxrf/tjycHRHUDXe6nowg2OiHME=@vger.kernel.org, AJvYcCWNcNR9AiCTjlGhJDPniK9WNwhJbdV0GXn16i3BtX+666kBMHSPCwc2XqFgG3GAtATPW1H4@vger.kernel.org X-Gm-Message-State: AOJu0Yy0BRust2iUuVYRH5eicxzpg1XeqmfSZTmwG6slu6VCNYEnesse eC/emyUfC2WnBcN6iRDEdig6/NUsFgm5KNLNRMS9Y1/TcYzrcKMS5Tp/ X-Gm-Gg: ASbGncvyKuKo8tA8ueHZMseUont9+1e6t/d9bSxkv/OBgIncy1fbNh4fyClzS3w34Cd XqVij1cXu5xrZ9TAOuT8qQX/HdgrYnjV0Ne54OKtg6ghbsNejvPrAaudTwfGzg/ygXf/Ol/ZE0X 7XCBkFqGiXXR5JskR9GYUEwi8aeC8v3JmoYS7E//72V8hDiRcn5k1BvdAXpi5LsHg5mgDxcPJeu Y3OJUXgxvIHvVyLDFwyPoTuSLuKswKs6lOPGwz4/vDUvT0Tpv8qlECuewNjqajsGXQzPJqKHBqm ZjVJwELs2yoRxswn3zHPXRxeobok3VxSdR8ikalM2ZI53AQrD0SchJiw6SQE6X5to7vO0/sFJRR 5QbQBJ//jxNhvOqvgSgw= X-Google-Smtp-Source: AGHT+IESLmw4DD5Ki9GcI9bHIcRKZU1nYhicBWlVNkdY+RJM1g1BAYjGa55ISfDgFrhmGnv9FdFmIw== X-Received: by 2002:a05:6a00:1d8d:b0:76e:8535:53d9 with SMTP id d2e1a72fcca58-76e85355e49mr1210485b3a.18.1755596093808; Tue, 19 Aug 2025 02:34:53 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76e7d1314a4sm1990945b3a.41.2025.08.19.02.34.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Aug 2025 02:34:53 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v2 3/7] bpf: use rcu_read_lock_dont_migrate() for bpf_inode_storage_free() Date: Tue, 19 Aug 2025 17:34:20 +0800 Message-ID: <20250819093424.1011645-4-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250819093424.1011645-1-dongml2@chinatelecom.cn> References: <20250819093424.1011645-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_inode_storage_free to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/bpf_inode_storage.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/bpf_inode_storage.c b/kernel/bpf/bpf_inode_storage.c index 15a3eb9b02d9..e54cce2b9175 100644 --- a/kernel/bpf/bpf_inode_storage.c +++ b/kernel/bpf/bpf_inode_storage.c @@ -62,8 +62,7 @@ void bpf_inode_storage_free(struct inode *inode) if (!bsb) return; =20 - migrate_disable(); - rcu_read_lock(); + rcu_read_lock_dont_migrate(); =20 local_storage =3D rcu_dereference(bsb->storage); if (!local_storage) @@ -71,8 +70,7 @@ void bpf_inode_storage_free(struct inode *inode) =20 bpf_local_storage_destroy(local_storage); out: - rcu_read_unlock(); - migrate_enable(); + rcu_read_unlock_migrate(); } =20 static void *bpf_fd_inode_storage_lookup_elem(struct bpf_map *map, void *k= ey) --=20 2.50.1 From nobody Sat Oct 4 08:05:11 2025 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAF0931A068; Tue, 19 Aug 2025 09:35:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596102; cv=none; b=tnX5zCxdNOXaonow4mQpYm808dxfTOvQOBwaEQl5RAm3TM3y4rgMVzw5twPygNVTDfoBJFWnjlCu+y+1o4V3bIRH/e1KhXcRXNvOuWWzFgFcxegef/K5tIDbvTXFc3Ne+DqpR27pOhEhYvEzCMctT1U9k19pTsa8E5Oml4lObu8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596102; c=relaxed/simple; bh=eqO46c3pAN3PfhGNOqY8l83f0ewMbPj8kQ9DK6CPRUU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p79ZyrpLVgptZs3Z4ww8WlGfUo28PsaeGwh0u3Z8do25Pa4bqZNpyDcxOHc5+lnm7Oi+asOEp+24aiOotwcqL6Qi3oCwX1KYiAMTynzZ5GCoi6WriH7c+cGkXKzHkiBt6nNp/AGOMWQYR+M+R0Ap7mb2ZNcLTLi87iCn8iThJTQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ENRY3CDd; arc=none smtp.client-ip=209.85.215.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ENRY3CDd" Received: by mail-pg1-f193.google.com with SMTP id 41be03b00d2f7-b471738daabso4659582a12.1; Tue, 19 Aug 2025 02:35:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755596100; x=1756200900; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=861OdxVnWv8IWFvxkcYe5tkgp3QAasS1lAYJfLFgMx4=; b=ENRY3CDdzcpfmOaHbiaDjgNllGhR1XD10QkQuL/LaozKJfIonkKhb8Ir+1T+G5mehp h6248DqbJ8fIaRx7bW9B3Ytim6Zedd/ZP2qvularElpUP71/aROxn+tU0ntIH5RMbOy0 8kjFUD+0UgT1enwRiqOmtqxsexy63RhtElyLCeq9S7YYh/++SVUJsrxbUv5mlKPjQ6xJ kB8PumEPJFdiNZY/6aNPoOby29a5BBEQEc8KFVt/nP/2+j2BkiyGjL+oA6JtXQL4OVle z6JzFXmQn0OyxdxD4YqsY7tIppTMuRSkhQURxz3GJWs6uBPqsDtFolpfqoUc8/7I9pay ok5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755596100; x=1756200900; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=861OdxVnWv8IWFvxkcYe5tkgp3QAasS1lAYJfLFgMx4=; b=F03kzSP9naLzlk51swKZ5B/85LstiM6r8YGIJTHxUPiJmNMNKo2l+ILF6vLSj7XDdd i7kBjqhj1TmL8STZpU4QHg3lJR37Ied5H7QtRBQFs0Mrqzxd44PTa5JOV9M3trEizsKX dK3FU0cWqx+uPqKEkc0+gEpH5zTC0rF1B4TWsC/G560CIYzcUWNpihUjtOdh/r++5TKi tm/ZvGDDaC2eeth0y80Swg+JD1pva/0fZ4IsAzJdbJzG6JXKeMssqPGnwKyrp8ni9mIc +BGi8PQQCheTL5YuGVX871K2+SivNWnJaAwfju03sHHnFtg5WWzyrKVp9iS3u+o7jvsz Ip9w== X-Forwarded-Encrypted: i=1; AJvYcCW6tT5NhZStLxd3FuWydaB7wcY6WFEEUsXBBm9mIy4qLAUb1lv1+qiHBFgknAola1cqXFo=@vger.kernel.org, AJvYcCWJ559SHxkiO3Y5S4SIaYvgy/MomBL3Jfochq5nhuEHuMMJuG+kSyrRFu8xg5OoznUAYnHK@vger.kernel.org, AJvYcCWTLtjC+YUZhVnzEUUpmSAQUFEqXNxd03IXNtkRm2ZhMNUgyLTJ37BzRe8JHWecPmYYAqPfhjwgKV+VGfGN@vger.kernel.org X-Gm-Message-State: AOJu0Yw4jUlX7nBfpTLfaktWUzFJ9j4mA4G6LdgXnER8gi9HquLaMPql Ir6/5quUvJb2A0iUrbF5qKXLcAWiJjTQ5HcNu72jSVniZ9AHwE1YAh0b X-Gm-Gg: ASbGncudF84teGr3ZU2OLyNsAbMPkVV4yclSZGegX1jTz32QtRBiJnWdDKVd0rL+2Sl yHN4jncYXhaTn6VGdXRTq64eEAKpzKcKi81ueryQOztP0mYFe3FP+Hu2qWvVyf8XchcA6jyrgHe 8Zmwy3iKDwlM9J9G1rcA25Pd/aqBpz9tAkEFgXKq9Hgv+U7RoUkQZW87ZXg1gS5r4dKySi6mluC 3LzOqC5IP5yrqeJqkg+2ONXemKQe+V+vVU1gYnlOhROq6zyw5JuzVrSofUiqXSqeHIVc3eQs3UL OlcTg/eJEYDv8mMSqDD7xCUczvrdsgIOV8larvoA7NC5sEyjcgU1mh5mvtEmSuuKbH4XJARpioS wT4DgkkRJBxqh7Wwaqpw= X-Google-Smtp-Source: AGHT+IE93gbHJgco21HqxguBff+5KzWKClvBeNlC34ItAW+lLtR8pGebvcMZTs8BzYxwIby0rAMFHQ== X-Received: by 2002:a05:6a20:a122:b0:203:bb3b:5f1d with SMTP id adf61e73a8af0-2430d330f66mr2785284637.6.1755596100201; Tue, 19 Aug 2025 02:35:00 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76e7d1314a4sm1990945b3a.41.2025.08.19.02.34.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Aug 2025 02:34:59 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v2 4/7] bpf: use rcu_read_lock_dont_migrate() for bpf_iter_run_prog() Date: Tue, 19 Aug 2025 17:34:21 +0800 Message-ID: <20250819093424.1011645-5-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250819093424.1011645-1-dongml2@chinatelecom.cn> References: <20250819093424.1011645-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_iter_run_prog to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/bpf_iter.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/bpf_iter.c b/kernel/bpf/bpf_iter.c index 0cbcae727079..6ac35430c573 100644 --- a/kernel/bpf/bpf_iter.c +++ b/kernel/bpf/bpf_iter.c @@ -705,13 +705,11 @@ int bpf_iter_run_prog(struct bpf_prog *prog, void *ct= x) migrate_enable(); rcu_read_unlock_trace(); } else { - rcu_read_lock(); - migrate_disable(); + rcu_read_lock_dont_migrate(); old_run_ctx =3D bpf_set_run_ctx(&run_ctx); ret =3D bpf_prog_run(prog, ctx); bpf_reset_run_ctx(old_run_ctx); - migrate_enable(); - rcu_read_unlock(); + rcu_read_unlock_migrate(); } =20 /* bpf program can only return 0 or 1: --=20 2.50.1 From nobody Sat Oct 4 08:05:11 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD1D0311958; Tue, 19 Aug 2025 09:35:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596109; cv=none; b=QlvUVAk+9pj13fBIvwd4WRjOSaamU2FwQNvzkyQZggsVYCquag5AW6DgYkBbqU7Pv4HTeCmUD9C4Mz0MgspmSOwWBNOdr+LNk+CCXwP1nIuMy0tc8YOlKaGrRoPuMfxXpBs/S+MFdr1GjLVZPHYdOPJGovpYOzsTDIhVLP4UPlM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596109; c=relaxed/simple; bh=THRCv3ozIqTTxq6bSRfB2pdWOOnyKsRGYogE8fXxDkM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hFDrXny0O1ccqzz8I+d1XN1cI4OagrDTtRT0PYwVWv7KIIjjnALSkvL4IUi+H/vuAU1x3CfA5z17sFH8vkGsUOXQpqelD6WkIN6siCeGssW2UyR5B6j03ggJxmt6j6E7L/408lat9wkCUhlvUQ8tg6nX8Akd5cLrA54tRuPCLUo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Aex/QL+l; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Aex/QL+l" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-76e39ec6e05so3200311b3a.2; Tue, 19 Aug 2025 02:35:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755596107; x=1756200907; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6Bptfhu7onpt0MqrEIJlEVMcozR6INxKRYWqNZ+YESQ=; b=Aex/QL+l+6kT2q0gBfiKeIka0ghIjUwlB4PTJoSSuglWux3YsI6HmIaZhYMnbbXjjI oS/dLzeENpPymerw+IdFikYmKueC08eiao4wQRflLrfWz5nqUDoApOLxSdszSXg7heXy N0vvaS7j0dLEQQ7MViZW4GAJe7GvuP4TwdOtT+vAASoa5hnC7ICylz8Fcyen14W85TUq xFAJUTHt5ZDgpEGBIOMAQQjMjY4AjxdRhkRVlt5za/3CvgfTDBKvFRSRQemA8Umipncu NCV9QuwoMgUlHTivLtvo1RBxcQoL9Mpsfr5JhZEYqmtDjqbaqmX8Mv9mam5EEXsFhvtb 0xsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755596107; x=1756200907; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6Bptfhu7onpt0MqrEIJlEVMcozR6INxKRYWqNZ+YESQ=; b=BZsejjf5PGwc+8pQguT1OtSxfCGagDeUNOEKo2UFAprCnzeTC69guinlBraG60xXBd Bl+ccUrxnQZxE63+VEfGd0p4FEcnAJII5R69ZlaPR3RWvtKIx/Mqt1ULst7pVKAKEfsy rCupaW0N1ZDhXkNLFN4BPrg4m6kSlnWtoHzqKZCN9pKxbaFZulVYk2OPZV0wmvPNbSjz YeAkIqoDlWWs4bupBIGiQerHrDbx/Sj+N56HHYHQdE101wV0eavSmI6RutY5UmKBVluQ mAxY3smRKPTxwvndi6yDKGWlDHj14Bc22m2ZQ4HwLb5srFYFyXkCwOdQAgcWzydHgcxn 7UqA== X-Forwarded-Encrypted: i=1; AJvYcCVM8xlfcqX/3Jwp481AMhMUQ7nva+isC6pwTy5PVMRZcd+PUAAdlqhnKTvi4HXcmxYDtZNH@vger.kernel.org, AJvYcCX7j+st6cQHLuIsK3St9mCZRWgHPOl0wd6BQG4EasHq+ySC4ba4+04WHkqbubJlSpOjhGqwsoyKiFrlSm+M@vger.kernel.org, AJvYcCX94FuGb/3piyz3NhNVJkLY3hJwnpuzAQKPc/WYAiqdP1go4hxCEfWY2V2IMQHPxJVShRY=@vger.kernel.org X-Gm-Message-State: AOJu0Yz6eEvcFxqTD33h691P59QkGCLpHgkVvEJT+LLsPVWSNNVdbqca 4Hpy0fLsbguciEPQhI6yFLFj2a+ftvUKj+rAqxyGh//RjnhGCAiljDXk X-Gm-Gg: ASbGnct9TCXfy0EiSG94hD+TD+5/rGVQM5gazH2QlLHhnGlQ82bn9M7ClTN5YFmpa6K NkozWM6sWy3FYYf/4QyAxO1ygtprKKgtOUr8fIX+StmN314HHzFbBEY3vB4+he8zH9gJK+2oxkT dt/imlDZv334WmH7mGp6h0VQmLZiwIK84s2eiJBrsAxBRT3WV+Dc2z/p8viOiXquMJnr7UzW49p RN+e8erhWTAKSRudTS6Gbv6Vwtv+Xpvt3f2gJTZELf3k3lb3E/tkXE1zfZuklbcpjsh1bYjJ0Mr TZUqpunGTp0YS+Pp4t1xP2HC5B6ImxFNWvwa5UiQRG9e9CYYI9tQo9wflCyxxmYWmwsFdU3a11/ AqzGq0EP8bimlN0iLsxM= X-Google-Smtp-Source: AGHT+IHjJOb9KVBoRIx0dMtdn5Tm/ZyMgaKw1QrpeMKOEmjf5u4O9taZJ8hoZaFTbIO3uMnSpJXanw== X-Received: by 2002:a05:6a00:180e:b0:76c:503:180d with SMTP id d2e1a72fcca58-76e80ebcca8mr2354947b3a.8.1755596106880; Tue, 19 Aug 2025 02:35:06 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76e7d1314a4sm1990945b3a.41.2025.08.19.02.35.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Aug 2025 02:35:06 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v2 5/7] bpf: use rcu_read_lock_dont_migrate() for bpf_task_storage_free() Date: Tue, 19 Aug 2025 17:34:22 +0800 Message-ID: <20250819093424.1011645-6-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250819093424.1011645-1-dongml2@chinatelecom.cn> References: <20250819093424.1011645-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_task_storage_free to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/bpf_task_storage.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/bpf_task_storage.c b/kernel/bpf/bpf_task_storage.c index 1109475953c0..a1dc1bf0848a 100644 --- a/kernel/bpf/bpf_task_storage.c +++ b/kernel/bpf/bpf_task_storage.c @@ -70,8 +70,7 @@ void bpf_task_storage_free(struct task_struct *task) { struct bpf_local_storage *local_storage; =20 - migrate_disable(); - rcu_read_lock(); + rcu_read_lock_dont_migrate(); =20 local_storage =3D rcu_dereference(task->bpf_storage); if (!local_storage) @@ -81,8 +80,7 @@ void bpf_task_storage_free(struct task_struct *task) bpf_local_storage_destroy(local_storage); bpf_task_storage_unlock(); out: - rcu_read_unlock(); - migrate_enable(); + rcu_read_unlock_migrate(); } =20 static void *bpf_pid_task_storage_lookup_elem(struct bpf_map *map, void *k= ey) --=20 2.50.1 From nobody Sat Oct 4 08:05:11 2025 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19FCD311965; Tue, 19 Aug 2025 09:35:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596115; cv=none; b=EirWQVuQl1oRKAdpUFq/cmVn1HA90DCE4um5/PxUMW8AqbRdTOxj+nvP2Y9xLjY41EP5lWH1DzmtzxMQb2cAbpJjhNua+D6ThXBPEItGke0KmseNhwm/Muz7CvEWU43R/OyF/4HEsqVePUfOf6alBq7eomLDMwB+qhQDsAz/tNg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596115; c=relaxed/simple; bh=fhjRoedQgU3UpqJ/vxJWw+P3CekwicN1Qc+dzHfUufo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cZpYQuFxuD6UyKx6Mz0GN9euf1whS/C+GUCk/besMq51bc9H+mi0w8dRsR8FTX+KtvOxh1M3TmzVgWgfK7C+qvcNkkKDTX2RUUXAWrEvDvd/29DVrYaNT9Bo8oqnuJbOnGjhwLoogm45EeoN4Rb/IV1Qjpp20tPEZh+tbi3XZ/w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PQfBnwJc; arc=none smtp.client-ip=209.85.210.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PQfBnwJc" Received: by mail-pf1-f193.google.com with SMTP id d2e1a72fcca58-76e2e613e90so3667023b3a.0; Tue, 19 Aug 2025 02:35:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755596113; x=1756200913; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j5H/4foa1XV+6AOUTU8S+hYtTzGYqXPj/gnpmsQWpYc=; b=PQfBnwJczbZWjoLhpFk83aCVMDLZBglDATymkmJ26CpH9En9BWu0QtFuS8tF71LSBw 9GYLRWJ1LP9FWVF0px71s1CstPR6guIT8+NRym934zGvYVyAx8jkp2vzV1BEXuMES3y5 UQ9v6vnJGyKV32STtp0Dr8CSxhwv+WrSs8Rez2b1/R8li3gc2Ew5yDEXbnHaIQVoKVPi 1rpxmhtUZLz7PYgKqjP46epGX0xrecZw9T2p0KFJRY8jTMqwG6ODU8+vfuHL70n5YHLx 1ZQcH2300J4pnRrYQe2IXh/h27Ueyki1jYXlHkfZP6HNkSj1XslIZ4u7VH2xA3WhOJ6I EToA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755596113; x=1756200913; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j5H/4foa1XV+6AOUTU8S+hYtTzGYqXPj/gnpmsQWpYc=; b=H1Uj2WlIDKD6W0sj5d/fVE55i4ONV/sClLgJ+hgV5bsVsB+ToqJ6LR4GSdNjv2wH1V AOafYWXM6RXmJobQIAnKkdu1gBdKWVoxY3YclZXXxzf8Kaqs8jrQa4O5nxBmtfXPoyKn bvSUX0JpWWXhsMB7gXJSM2dhg7C68+qehNgxz3bInwYgAD52mgh4TSHtXmSN9GmbJ4n4 8JKzSuhhxEFD9e+VKsi3ZDyABiTw3xEMr3bq5sEEaV8b/Q43LuuWYqzv7iFTkrzSwPot eqtjhpbSWQIWPq3+wvJOyR34SrwYRpxlBLQJbrazMLFYt0xCv+HG/hlMvFFxntCRVqEF cM5w== X-Forwarded-Encrypted: i=1; AJvYcCUvDPkstDP2tWv1ia74vqa2Xr8VcQk/0POf/yw1CvB357/jXvn/msG0+/r3Ikz4yDuJp9+P@vger.kernel.org, AJvYcCVN/04G/jZ9nPf4332gb0ADnbzxR/rzKtLbxsFe+j+upZsO5F/fvq1cc70UPqqXH5nnJtJS7LXAwRTBQUsq@vger.kernel.org, AJvYcCW7dS/GWg0I3iwG9g0jbRvSLV/tnJTR3e5llQ9kGWtahQwXTZQPtsJn2WUIo0wGlBlDzqM=@vger.kernel.org X-Gm-Message-State: AOJu0YxiszStWswIjF9ljGyZr0SXxhxPGO9Z8Iv8edRgK3EvmKN5S17u A/S99tCITTpgHCIyx8ok2WQH92NKa7og1YcWbTNe7azCzjrN6vk8oLUf X-Gm-Gg: ASbGnct2dIgH2GE11ItGUvYszqEgg1Khzn7cNzcMP9IEbjQhMZKPI5peMQA1Ebursbv 9LU4uaZiAfC0O75lBDQpgPr7/Xve7v8L5NUQqYtmfkmpPaOo63UG+mPeuhg32x4cryU8d78k4JV bmygdYRTXZ2VyzKwThxGiSVXtH1QONV76CJpQSd1C3jqz0VcekxSj+T1DxNfv79g/3yH5VzUy73 mVXzsAZsUq4JhjWieP1m9dYZy5ZvNMmPKikYj2JNhEMbC/eyDoCS4kXQmOelX31FVne3LhAH3kV YRD8iFU4X4bmc/bVVfAH27eiqCyNuNIASTd9v5AIdTDHOks/bwtj51IeavAn9XKDm0zPwJz8G7R CtA/QW9+RT4P2VCv/sFg= X-Google-Smtp-Source: AGHT+IFcH7bSLcl4HGDbOr/umRDtxmIwhRc2AfH/G+OzMma/+aubmcK+Df1k4i0oK/PDDhk+wqoT6g== X-Received: by 2002:a05:6a00:21c4:b0:76b:ee57:986c with SMTP id d2e1a72fcca58-76e80f0511amr2192612b3a.14.1755596113278; Tue, 19 Aug 2025 02:35:13 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76e7d1314a4sm1990945b3a.41.2025.08.19.02.35.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Aug 2025 02:35:12 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v2 6/7] bpf: use rcu_read_lock_dont_migrate() for bpf_prog_run_array_cg() Date: Tue, 19 Aug 2025 17:34:23 +0800 Message-ID: <20250819093424.1011645-7-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250819093424.1011645-1-dongml2@chinatelecom.cn> References: <20250819093424.1011645-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in bpf_prog_run_array_cg to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/cgroup.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c index 180b630279b9..9912c7b9a266 100644 --- a/kernel/bpf/cgroup.c +++ b/kernel/bpf/cgroup.c @@ -71,8 +71,7 @@ bpf_prog_run_array_cg(const struct cgroup_bpf *cgrp, u32 func_ret; =20 run_ctx.retval =3D retval; - migrate_disable(); - rcu_read_lock(); + rcu_read_lock_dont_migrate(); array =3D rcu_dereference(cgrp->effective[atype]); item =3D &array->items[0]; old_run_ctx =3D bpf_set_run_ctx(&run_ctx.run_ctx); @@ -88,8 +87,7 @@ bpf_prog_run_array_cg(const struct cgroup_bpf *cgrp, item++; } bpf_reset_run_ctx(old_run_ctx); - rcu_read_unlock(); - migrate_enable(); + rcu_read_unlock_migrate(); return run_ctx.retval; } =20 --=20 2.50.1 From nobody Sat Oct 4 08:05:11 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A94A317718; Tue, 19 Aug 2025 09:35:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596121; cv=none; b=M9RjjsEvcZiLNBwmzbwqD6SeDo/wi15q5aZvn4XDvRpBqtZTrgynrnju6Tw99HHzskrVQt25XnbDzR2gvz9zR0qlYHoAdHSBWYCt1pP+/iIEHg8TgZ4tVZ0acdD1gg0QU449uWl9JhuGHUJr6RTWxYuMUiq44N74fL9ltRUOsTw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755596121; c=relaxed/simple; bh=6Lh4b5LMBBzYETFNUKgsMw+Mg8slBT6PgN9WlZyKi/I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oGhpqJBUh7uUrc3Y0CNzbWCMyhKFTGbOibCw+ngcM8IUvkRyfwiBmSe6WdGGEoIrbOa6TrxOotEh5CKFuvdInvj5ckTk6oDzFxoItT70jH1fA1+Z5Pj32B/X054gLzhO51E9yJHBH3SmnDAE85Y8SwDzjnmdjbMkYtPKzu8ibPk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=g9vBKoTf; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="g9vBKoTf" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-76e39ec6e05so3200455b3a.2; Tue, 19 Aug 2025 02:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755596120; x=1756200920; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qA6vzGkbuWAEnBwZb/6LL/xPx2BmZVoZP3qls2M6vww=; b=g9vBKoTfHBG+JtoN2O9z6FEuX4tLbyr/H4MhGzfUtW7vTI+y7wIRklJMzHDSrefN4y V1TwK+Dv/dwXVqNYpn1Ds6PdX0pEzMlQLvfDfBiTtDAo0GybTCbUq/Utb08xMrmDHdC6 zIxzH37USPWVD0LohFNz08rT0bTk0b0hriktlCddZrLeU0gDgff9ccDf/4wm1uEuszLO g+uKVykUmekDCtbEdIWetc6Ciq46teXJei4WrGwfvIC0JCdV2eM4valrV9lZpstR6koW qmVo2bmERYkhyiRyQ2r+SJeweAKtY0Jk4JhrU3uJNDe+GvAy7Vj5NopqngxFojt97pDC 7jQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755596120; x=1756200920; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qA6vzGkbuWAEnBwZb/6LL/xPx2BmZVoZP3qls2M6vww=; b=FbCxCubkl3+8dd4fzPo0/h919LF+h2fHHxfVbYxhDGMDletKvPfEFwrRrHA4AJkmjK nOYLaIo3Hwtfy4cnKySMqMdt334gXsdzNwoyzJ+PxKYvkMnie472j60B85FAtMiFfI2P WM/ti6qOwe/arvsDuvkVX+k9mQy4CbfqXOSdvnXs0KRAbfmi0RI/bHxL2BNLm5ARev9T 6TC6Gu/x570/GH8QYnQZt3EI0aCVKPOYrMThtb/CxDnBmAA/3J4/pgp/jvcpCHfmzZSq i08J/Q16UYICiuPqSFCvphTRdRRSzSLZqH6+ou8k7W8MkxaU/1EcKNGW3iMd7RP51ho3 Nh2w== X-Forwarded-Encrypted: i=1; AJvYcCU72TMCiM4wna0UWFZCTgOWcdHW8QWdtyQtMzglmSmgABma7J5aSH1s6trr7KkvQP0yFps0@vger.kernel.org, AJvYcCUgxE49tPkchbIpylyFgIvEr3pyOFJ/Tp65+kldq9CFyL2q/sBl+WuV+fHiW6hhG740Zh+n7e7O83qyExC+@vger.kernel.org, AJvYcCXxyDHj/H1/KFeD3X7LTsHFXk8gd1jZ8AB7aPL0JhLZJFEPAGNtrDLa/7FZGzSCNG/4PNM=@vger.kernel.org X-Gm-Message-State: AOJu0YzUq1UxJ5orHoqZKpbiaPTVfkOBzUkNvVvApOoX07G3dtVAZz2Q wgEHx8iM/zEhIkgM4CF09GgzddnULe0AXvGGiQcBgVXbti3FOkCwTReb X-Gm-Gg: ASbGnctj+px5LeH9y5hFFtnIc8NGPvexhn2qHXQYaF+hi4I35wzQYZeoc1VZN9BBAoT KOgHabS+b/JnSJC4tMGb7sNR/DCHH7jYqV4q+aRxtHf2K6mFkeUq4Ka3YpGSXuHv5m5IC272Vuj SFYZqpQK4pukEzJxk9eDCHgMfVU9DOcBSYzLGtN9mqx8eSHGXLyfRJOtG1sBwcZn1i2S05Ei5IG KD2hIe94YwwKNi6lbvoVrYal2kd6AIArr0AhQe26xKNTlyGvDbABfA7S5/EAd2UiNF9s/QS0+aE 0xjLF6E4fcnxoqmdjOEr5UVRDzXa4UMzTacxcGtUWM1mTJgddFNbpex+R8Z7mTa0reKPd4rNL3C Ljk8fCfgLIrmERKtmVEI= X-Google-Smtp-Source: AGHT+IHyvkNhcELUFuNuzdPlQf7oSQL8a7T9dDNgx5dj4V4tyR5eYQmpfnvhrcwDfRqeq+KLqRJInw== X-Received: by 2002:a05:6a00:4fc2:b0:76b:d93a:6a02 with SMTP id d2e1a72fcca58-76e80da230bmr2035364b3a.0.1755596119672; Tue, 19 Aug 2025 02:35:19 -0700 (PDT) Received: from 7940hx ([43.129.244.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-76e7d1314a4sm1990945b3a.41.2025.08.19.02.35.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Aug 2025 02:35:19 -0700 (PDT) From: Menglong Dong X-Google-Original-From: Menglong Dong To: ast@kernel.org, paulmck@kernel.org Cc: frederic@kernel.org, neeraj.upadhyay@kernel.org, joelagnelf@nvidia.com, josh@joshtriplett.org, boqun.feng@gmail.com, urezki@gmail.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next v2 7/7] bpf: use rcu_read_lock_dont_migrate() for trampoline.c Date: Tue, 19 Aug 2025 17:34:24 +0800 Message-ID: <20250819093424.1011645-8-dongml2@chinatelecom.cn> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250819093424.1011645-1-dongml2@chinatelecom.cn> References: <20250819093424.1011645-1-dongml2@chinatelecom.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use rcu_read_lock_dont_migrate() and rcu_read_unlock_migrate() in trampoline.c to obtain better performance when PREEMPT_RCU is not enabled. Signed-off-by: Menglong Dong --- v2: - use rcu_read_lock_dont_migrate() instead of rcu_migrate_disable() --- kernel/bpf/trampoline.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index 0e364614c3a2..5949095e51c3 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -899,8 +899,7 @@ static __always_inline u64 notrace bpf_prog_start_time(= void) static u64 notrace __bpf_prog_enter_recur(struct bpf_prog *prog, struct bp= f_tramp_run_ctx *run_ctx) __acquires(RCU) { - rcu_read_lock(); - migrate_disable(); + rcu_read_lock_dont_migrate(); =20 run_ctx->saved_run_ctx =3D bpf_set_run_ctx(&run_ctx->run_ctx); =20 @@ -949,8 +948,7 @@ static void notrace __bpf_prog_exit_recur(struct bpf_pr= og *prog, u64 start, =20 update_prog_stats(prog, start); this_cpu_dec(*(prog->active)); - migrate_enable(); - rcu_read_unlock(); + rcu_read_unlock_migrate(); } =20 static u64 notrace __bpf_prog_enter_lsm_cgroup(struct bpf_prog *prog, @@ -960,8 +958,7 @@ static u64 notrace __bpf_prog_enter_lsm_cgroup(struct b= pf_prog *prog, /* Runtime stats are exported via actual BPF_LSM_CGROUP * programs, not the shims. */ - rcu_read_lock(); - migrate_disable(); + rcu_read_lock_dont_migrate(); =20 run_ctx->saved_run_ctx =3D bpf_set_run_ctx(&run_ctx->run_ctx); =20 @@ -974,8 +971,7 @@ static void notrace __bpf_prog_exit_lsm_cgroup(struct b= pf_prog *prog, u64 start, { bpf_reset_run_ctx(run_ctx->saved_run_ctx); =20 - migrate_enable(); - rcu_read_unlock(); + rcu_read_unlock_migrate(); } =20 u64 notrace __bpf_prog_enter_sleepable_recur(struct bpf_prog *prog, @@ -1033,8 +1029,7 @@ static u64 notrace __bpf_prog_enter(struct bpf_prog *= prog, struct bpf_tramp_run_ctx *run_ctx) __acquires(RCU) { - rcu_read_lock(); - migrate_disable(); + rcu_read_lock_dont_migrate(); =20 run_ctx->saved_run_ctx =3D bpf_set_run_ctx(&run_ctx->run_ctx); =20 @@ -1048,8 +1043,7 @@ static void notrace __bpf_prog_exit(struct bpf_prog *= prog, u64 start, bpf_reset_run_ctx(run_ctx->saved_run_ctx); =20 update_prog_stats(prog, start); - migrate_enable(); - rcu_read_unlock(); + rcu_read_unlock_migrate(); } =20 void notrace __bpf_tramp_enter(struct bpf_tramp_image *tr) --=20 2.50.1