From nobody Mon Feb 9 01:46:24 2026 Received: from mail.nppct.ru (mail.nppct.ru [195.133.245.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B37021D88C3 for ; Sun, 2 Feb 2025 07:50:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.133.245.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738482635; cv=none; b=NYbiArtJjIiv/Z8/gZURpuBqwWGrEWer8P5QZ0oyfQ7T6a9zRIOTC7SdwzdMHX2CBh5U+0p2KAK7vfeG4AkNlq3/xWJBizdFQMxyb7moKyY/zIU0TQu2gX4O/k52FKCq3/lVHOEpctBWVGtvskCQ6RBvzU874Uk7AyfM5OOhXgA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738482635; c=relaxed/simple; bh=Ldo/Ykq4wKZ/UskHvq7gtWqHWtFrRYbBgy/43Zu43LU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pDAS2znWSFUCFawNOu0ZjUCFRcT8QgHBoKhr5MzrrZxln2Gpnmh4F2UgU/0YNMXU4rHqz3xvZXcWjeSD4OLHpuiUJwse5666MZEiEM0L1JsE6RbUDp8X2db2FbSKk25eUAHvD3caKhedMZTrY54+WcVCwy5GHwTWaUqkn8DgM/A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=nppct.ru; spf=pass smtp.mailfrom=nppct.ru; dkim=pass (1024-bit key) header.d=nppct.ru header.i=@nppct.ru header.b=ahWj8A9G; arc=none smtp.client-ip=195.133.245.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=nppct.ru Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nppct.ru Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=nppct.ru header.i=@nppct.ru header.b="ahWj8A9G" Received: from mail.nppct.ru (localhost [127.0.0.1]) by mail.nppct.ru (Postfix) with ESMTP id EB7861C2431 for ; Sun, 2 Feb 2025 10:50:31 +0300 (MSK) Authentication-Results: mail.nppct.ru (amavisd-new); dkim=pass (1024-bit key) reason="pass (just generated, assumed good)" header.d=nppct.ru DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nppct.ru; h= content-transfer-encoding:mime-version:references:in-reply-to :x-mailer:message-id:date:date:subject:subject:to:from:from; s= dkim; t=1738482631; x=1739346632; bh=Ldo/Ykq4wKZ/UskHvq7gtWqHWtF rRYbBgy/43Zu43LU=; b=ahWj8A9GJizeRd+sTjFlUHOSkpFY4AM0L0/JpLsU8XC CbC4tMlvzN9O89nv3JlMq66ZZLbyQ8vWY+1qwVFcEfYloGsJYudvjkxkNCvMk4U6 4hLCabHwizhbgCs8wX2kVGSSEMKFdSaXxEQdL6HsOk5rWfhhv769EU0uZjDBTymw = X-Virus-Scanned: Debian amavisd-new at mail.nppct.ru Received: from mail.nppct.ru ([127.0.0.1]) by mail.nppct.ru (mail.nppct.ru [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id kfUP9qmd3doW for ; Sun, 2 Feb 2025 10:50:31 +0300 (MSK) Received: from localhost.localdomain (unknown [87.249.24.51]) by mail.nppct.ru (Postfix) with ESMTPSA id 74D741C242C; Sun, 2 Feb 2025 10:50:19 +0300 (MSK) From: Alexey Nepomnyashih To: stable@vger.kernel.org, Greg Kroah-Hartman Cc: Alexey Nepomnyashih , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , bpf@vger.kernel.org, "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Josh Triplett , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , rcu@vger.kernel.org, linux-kernel@vger.kernel.org, lvc-project@linuxtesting.org, Hou Tao Subject: [PATCH 6.1 08/16] bpf: Further refactor alloc_bulk(). Date: Sun, 2 Feb 2025 07:46:45 +0000 Message-ID: <20250202074709.932174-9-sdl@nppct.ru> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250202074709.932174-1-sdl@nppct.ru> References: <20250202074709.932174-1-sdl@nppct.ru> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alexei Starovoitov commit 7468048237b8a99c03e1325b11373f9b29ef4139 upstream. In certain scenarios alloc_bulk() might be taking free objects mainly from free_by_rcu_ttrace list. In such case get_memcg() and set_active_memcg() are redundant, but they show up in perf profile. Split the loop and only set me= mcg when allocating from slab. No performance difference in this patch alone, b= ut it helps in combination with further patches. Signed-off-by: Alexei Starovoitov Signed-off-by: Daniel Borkmann Acked-by: Hou Tao Link: https://lore.kernel.org/bpf/20230706033447.54696-7-alexei.starovoitov= @gmail.com Signed-off-by: Alexey Nepomnyashih --- kernel/bpf/memalloc.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/kernel/bpf/memalloc.c b/kernel/bpf/memalloc.c index 9a4fb4ea73ef..bbd3fa2bf119 100644 --- a/kernel/bpf/memalloc.c +++ b/kernel/bpf/memalloc.c @@ -196,8 +196,6 @@ static void alloc_bulk(struct bpf_mem_cache *c, int cnt= , int node) void *obj; int i; =20 - memcg =3D get_memcg(c); - old_memcg =3D set_active_memcg(memcg); for (i =3D 0; i < cnt; i++) { /* * free_by_rcu_ttrace is only manipulated by irq work refill_work(). @@ -212,16 +210,24 @@ static void alloc_bulk(struct bpf_mem_cache *c, int c= nt, int node) * numa node and it is not a guarantee. */ obj =3D __llist_del_first(&c->free_by_rcu_ttrace); - if (!obj) { - /* Allocate, but don't deplete atomic reserves that typical - * GFP_ATOMIC would do. irq_work runs on this cpu and kmalloc - * will allocate from the current numa node which is what we - * want here. - */ - obj =3D __alloc(c, node, GFP_NOWAIT | __GFP_NOWARN | __GFP_ACCOUNT); - if (!obj) - break; - } + if (!obj) + break; + add_obj_to_free_list(c, obj); + } + if (i >=3D cnt) + return; + + memcg =3D get_memcg(c); + old_memcg =3D set_active_memcg(memcg); + for (; i < cnt; i++) { + /* Allocate, but don't deplete atomic reserves that typical + * GFP_ATOMIC would do. irq_work runs on this cpu and kmalloc + * will allocate from the current numa node which is what we + * want here. + */ + obj =3D __alloc(c, node, GFP_NOWAIT | __GFP_NOWARN | __GFP_ACCOUNT); + if (!obj) + break; add_obj_to_free_list(c, obj); } set_active_memcg(old_memcg); --=20 2.43.0