From nobody Thu Apr 2 17:16:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBBA9354ADD for ; Wed, 11 Feb 2026 22:32:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770849143; cv=none; b=MWVWad7+2OcjStNHTvjHtUC970Bf01yhKZTlNvO+95LeVbao5PoLzO5AV/Qo5uk+2B309q55qiPGKY0WI7QickcQoGZG1Vvp7B7iBkoLFn/ecTmtk8Hk/xMKyveTxusT8Tpx8tMOG9vDK6dsLfLpIze3sWVdbSeQCJpHRjWbzIo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770849143; c=relaxed/simple; bh=YbtmwtPYpvW2DZ4YKyZJLQ3yQtQa2n6KsyOyBeW40Ao=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YPaihNW04r7m2Ol2wvVSxYVPYVg+SQ6izWWKp5RGSuPsa863CrqRay0XaZNZ+SVqzpiMH0mmc7P4x6y2DkTS9eKvVbFycdVToRXe/SFrxVGgz7nL5x6FMuRrtq5XPCa/J1m7eUnJz/0OYoAEwIp83jYBRklToO6bVds0ljBCL4I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NX4pn0dQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NX4pn0dQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 51C6DC4AF09; Wed, 11 Feb 2026 22:32:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1770849143; bh=YbtmwtPYpvW2DZ4YKyZJLQ3yQtQa2n6KsyOyBeW40Ao=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NX4pn0dQjv3YBXeebBkmnOrQwYkRwVampr0Ck1TaecVBGHShdxyuMwjPvnedUrqlY WVhqzMPL53Lm//tTIDtcUrwqkBTDUmoTGqPK0k7LTWOiEhYjxYWCA9gFdpc6//gPH/ ABMW7OAiQKpyG4TYFY1g5l0Ymers2i8onm/y0hoqDoHvRc+ZH5SmZjO3sg8ay8e6ld 6YaQeCzgELaj6x8azZ2g2Rw38HXnqIo9kgMf1grHS99S7WKXS9QjJi6gsebp2Ed3c5 +1XRwsfukgwvwBgdZBXvS59HjQfNLF3K9VJ4KYDB2MiuVLRDO6106XslaSZVVERiT2 RQNvwuyEbDtZA== From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , LKML , Guenter Roeck Subject: [PATCH 1/3] perf/core: Pass GFP flags to attach_task_ctx_data() Date: Wed, 11 Feb 2026 14:32:19 -0800 Message-ID: <20260211223222.3119790-2-namhyung@kernel.org> X-Mailer: git-send-email 2.53.0.273.g2a3d683680-goog In-Reply-To: <20260211223222.3119790-1-namhyung@kernel.org> References: <20260211223222.3119790-1-namhyung@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This is a preparation for the next change to reduce the computational complexity in the global context data handling for LBR callstacks. Signed-off-by: Namhyung Kim --- kernel/events/core.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index da013b9a595fcebe..b8498e9891e21c18 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -5370,15 +5370,15 @@ static void unaccount_freq_event(void) =20 =20 static struct perf_ctx_data * -alloc_perf_ctx_data(struct kmem_cache *ctx_cache, bool global) +alloc_perf_ctx_data(struct kmem_cache *ctx_cache, bool global, gfp_t gfp_f= lags) { struct perf_ctx_data *cd; =20 - cd =3D kzalloc(sizeof(*cd), GFP_KERNEL); + cd =3D kzalloc(sizeof(*cd), gfp_flags); if (!cd) return NULL; =20 - cd->data =3D kmem_cache_zalloc(ctx_cache, GFP_KERNEL); + cd->data =3D kmem_cache_zalloc(ctx_cache, gfp_flags); if (!cd->data) { kfree(cd); return NULL; @@ -5412,11 +5412,11 @@ static inline void perf_free_ctx_data_rcu(struct pe= rf_ctx_data *cd) =20 static int attach_task_ctx_data(struct task_struct *task, struct kmem_cache *ctx_cach= e, - bool global) + bool global, gfp_t gfp_flags) { struct perf_ctx_data *cd, *old =3D NULL; =20 - cd =3D alloc_perf_ctx_data(ctx_cache, global); + cd =3D alloc_perf_ctx_data(ctx_cache, global, gfp_flags); if (!cd) return -ENOMEM; =20 @@ -5499,7 +5499,7 @@ attach_global_ctx_data(struct kmem_cache *ctx_cache) =20 return 0; alloc: - ret =3D attach_task_ctx_data(p, ctx_cache, true); + ret =3D attach_task_ctx_data(p, ctx_cache, true, GFP_KERNEL); put_task_struct(p); if (ret) { __detach_global_ctx_data(); @@ -5519,7 +5519,7 @@ attach_perf_ctx_data(struct perf_event *event) return -ENOMEM; =20 if (task) - return attach_task_ctx_data(task, ctx_cache, false); + return attach_task_ctx_data(task, ctx_cache, false, GFP_KERNEL); =20 ret =3D attach_global_ctx_data(ctx_cache); if (ret) @@ -9231,7 +9231,7 @@ perf_event_alloc_task_data(struct task_struct *child, =20 return; attach: - attach_task_ctx_data(child, ctx_cache, true); + attach_task_ctx_data(child, ctx_cache, true, GFP_KERNEL); } =20 void perf_event_fork(struct task_struct *task) --=20 2.53.0.273.g2a3d683680-goog From nobody Thu Apr 2 17:16:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8ECB13563F1 for ; Wed, 11 Feb 2026 22:32:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770849144; cv=none; b=R7PSp2lz4vMXOwMOuRtC4xSOhJypuhcFZKilCEG3lrFB8VO7A8iA1LT1MfaHvMAJb0wFlVe0d2d10nR7OL6pXPqAmRIIZhETxtD84f9Dw/X9wqemr5dmSgI/hVDDUf+wKJVHbrSMpj+sovceJnbNawMnQ05pRz8n+QKy/viNkTE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770849144; c=relaxed/simple; bh=h01KiB6pCoCVMtu+V2L+yDlsgIpolKK800+M0ocd0oo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mQ/RMLsTBeiqv1Dmf1F8ZPst0zOl9ZqVDJVMNWn0A4yGfrMVzphGFmZAQWOMe7yMfD0YjL08EF5hYLOGnUC6BPPvk+P47YjhPXjst0cTSsN/5n1zpcyEMsSoPFGAhFKIAxnIXPrvV/3sBKfMgL6+iHUUT8685+x48/gxdAKQLsE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JaGHAZa5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JaGHAZa5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CDA3DC16AAE; Wed, 11 Feb 2026 22:32:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1770849144; bh=h01KiB6pCoCVMtu+V2L+yDlsgIpolKK800+M0ocd0oo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JaGHAZa5Pm4v3VXoOBI/5pfytK17uUmLtI7xZ+Cs6KtHW953ajdh24FYelP9tPFxK /jszR4Fu5rZ+Hf2Gw9OfOFoQ+RSU9WQVAAcadLgebVx/UgHe+Gi/OS6IrFLTT7y4+E PPrORUsAnskCL7O040JRR5cRKeg8u7Ld+AOEd1RxDhzzrdwI8bvZy2n/PbnMP01lpd LlVw/v9E7lOxLlzATNQ/4sLJ06+ITAYaHyqbij5DHMYgpN5eh/mwZqmktxRmIwOckV 0x6jLGtg0uhvcNIr7dKk5gW8gtHa6nTLwfZzOdDXhS6UOzIpRxOaAcAcFtLI/nrfGd /4OaozgkH74Vg== From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , LKML , Guenter Roeck Subject: [PATCH 2/3] perf/core: Try to allocate task_ctx_data quickly Date: Wed, 11 Feb 2026 14:32:20 -0800 Message-ID: <20260211223222.3119790-3-namhyung@kernel.org> X-Mailer: git-send-email 2.53.0.273.g2a3d683680-goog In-Reply-To: <20260211223222.3119790-1-namhyung@kernel.org> References: <20260211223222.3119790-1-namhyung@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The attach_global_ctx_data() has O(N^2) algorithm to allocate the context data for each thread. This caused perfomance problems on large systems with O(100k) threads. Because kmalloc(GFP_KERNEL) can go sleep it cannot be called under the RCU lock. So let's try with GFP_NOWAIT first so that it can proceed in normal cases. Signed-off-by: Namhyung Kim --- kernel/events/core.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/kernel/events/core.c b/kernel/events/core.c index b8498e9891e21c18..5b05a71edeb47955 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -5489,6 +5489,13 @@ attach_global_ctx_data(struct kmem_cache *ctx_cache) cd =3D NULL; } if (!cd) { + /* + * Try to allocate context quickly before + * traversing the whole thread list again. + */ + if (!attach_task_ctx_data(p, ctx_cache, true, + GFP_NOWAIT)) + continue; get_task_struct(p); goto alloc; } --=20 2.53.0.273.g2a3d683680-goog From nobody Thu Apr 2 17:16:00 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C785A356A0B for ; Wed, 11 Feb 2026 22:32:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770849144; cv=none; b=VVK0/Wxwn9EyT05ukHnur9KoLompeH4EzgYl9ODjK3doimolaNbIuZjvwXI5YzokkbECIHpCvcxVwydL3O6Ls/i102u1TWqcrJ6Bbtey8twGM/irAQlCN8Ryh4UTNZGUbbcOAkEMwCqvDKrWi9pQt0snK/LUcihwV1zBgiYv6ZA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770849144; c=relaxed/simple; bh=s295lL6q8K8Ne/0PfNFhb/twpqM7r7QP3z2ozzUVaK0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uUduIXPnio7Hl/rT6y0nFU2NX1P/GVRz2ghjXVxofHvWJE3R8Cvc6iMpOPAdEKd0UO9z8ob7NncDMP26JPjAul97Qu4zsWSGwdqKcGmbZIDKpiCBjlK2rtNORdVFblZeimIaTN7fWDy1zjIHphPVPq4cyVHcdWK0cEahXD1UTSs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Mg4vgdHv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Mg4vgdHv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 502F1C2BC87; Wed, 11 Feb 2026 22:32:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1770849144; bh=s295lL6q8K8Ne/0PfNFhb/twpqM7r7QP3z2ozzUVaK0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Mg4vgdHvcDsbdwiCFdda34V/XJE/MfJFRDj+BVBcj7eDJgU75grHtwH/HXfbtXh+f gOeYyZCA8r7ohjVS7ba2zHu8xezBX5tX51Wf4JoTt2HxkZ6RaA2FzZJwAM73khcsXP O3BBnvDe1TvVT2XosWSqnh7f4U02MJGqW7JFngQXnMN2LfvVRkuTOGYbRy1ag8H3E7 6MglqP9z8hOgB3rk6gh9DEtk3MQ/Wd5ZlAEc8VXk6Ma0EhbTTQPThLuIJk9XwBebhR bQKILu3ry6TeeCgXV5j+LorKBTI4um9CkEsD/o5ZAHlnVLPuX60GHUoE5aAhgMGYsE qdrVNLeMSh4Wg== From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , LKML , Guenter Roeck Subject: [PATCH 3/3] perf/core: Simplify __detach_global_ctx_data() Date: Wed, 11 Feb 2026 14:32:21 -0800 Message-ID: <20260211223222.3119790-4-namhyung@kernel.org> X-Mailer: git-send-email 2.53.0.273.g2a3d683680-goog In-Reply-To: <20260211223222.3119790-1-namhyung@kernel.org> References: <20260211223222.3119790-1-namhyung@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Like in the attach_global_ctx_data() it has a O(N^2) loop to delete task context data for each thread. But perf_free_ctx_data_rcu() can be called under RCU read lock, so just calls it directly rather than iterating the whole thread list again. Signed-off-by: Namhyung Kim --- kernel/events/core.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 5b05a71edeb47955..e67e1baa99d1e5a9 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -5561,22 +5561,15 @@ static void __detach_global_ctx_data(void) struct task_struct *g, *p; struct perf_ctx_data *cd; =20 -again: scoped_guard (rcu) { for_each_process_thread(g, p) { cd =3D rcu_dereference(p->perf_ctx_data); - if (!cd || !cd->global) - continue; - cd->global =3D 0; - get_task_struct(p); - goto detach; + if (cd && cd->global) { + cd->global =3D 0; + detach_task_ctx_data(p); + } } } - return; -detach: - detach_task_ctx_data(p); - put_task_struct(p); - goto again; } =20 static void detach_global_ctx_data(void) --=20 2.53.0.273.g2a3d683680-goog