From nobody Sun Nov 24 01:44:02 2024 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C2C1215C6A; Thu, 7 Nov 2024 15:57:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730995042; cv=none; b=clz+2q8wyLWkR0K8pRYjkdpWs85XL6ffjvmGKU8I8MjeMz88ufTY0UAjjO8rhUX31JlehXp/PDEW/O0FywHTRZALXLqG/AQmSuYBlEexpfNROmmcuzoILfGEKbHlt1F2Z/FEzwIJn37+nakc/8QihahEAO0pv1WNvE2MBgUeHoc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730995042; c=relaxed/simple; bh=l/0wZIy7SkdSRdFTGTTWstgdulRmA7dbmH3XYkTeIRU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=t/vAitWMM/pxG0pjxlLvYh3wsbBQUO/Cf63rxW2uY6/qbZYjfBElsAnqo7T0LiZTJeXF1RUyqYxyfHXxEU43WewbhddkotewzkcbbA1cjQkwsui+iRZrlFGhkFFggp6Bfox9VFWPZiUZK3BCwmF7mzZI6Vuoi2nVd39gyN1GdpM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-a86e9db75b9so177496366b.1; Thu, 07 Nov 2024 07:57:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730995036; x=1731599836; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hzst0PFb8vN+5CtSNyjHKngR6IOzkyemapq2li6Vfbs=; b=wxDuwfgAQTd8m4CUQgTYAJXPYo5naLWCdcEscIVkLyFNnnbIcmhnwUL1dek9kQWkuh 31nPFrfTDMwH3SIbUovpfQxSH0IRsaKAI22yxz4zKbLQP4ceoliA/gcKnX0ARlD49Qv/ pfImuv2yzjvdDRZUm/y+x5AuPmFz7/QkIQ0cfCEpVIeZuhDfv3TE30F6bc5hON6IcCah qufJjruETK0LZOYBpezsn01QNNna3aob0uH4hXgvtjID8/UJzZDAMAXsTgoMhuy3bHnz f3G1q58dYhk871vNwaiBJjoenHg4HtU/zPaWa/7hOH6jJ1JzaLiMvvKaogGwAZ9BHYFp XgFQ== X-Forwarded-Encrypted: i=1; AJvYcCWZb4oh8/D8uDvNPDbxjAEtgUoO/bfXzwcvUct55XJgP0zCT2cDd6khuhmW2u4u2UXq6Z8RxmlZCv8mUyw=@vger.kernel.org X-Gm-Message-State: AOJu0YxfyU5d8evPyaI2NKhJK+/FQw4Zj/hhZpidWhCGO9oAdqorgNOb tkMWBzpGTyTfaplhK0oHqiWmgQGnC4OY8HGhvYgCTFiZ2ouxJg9m08A/Yg== X-Google-Smtp-Source: AGHT+IEzDwoi95xhMMuHZgubLehCalpiaR8rUAzXf/DRrhLMuddonNKFMl/C25ZnFX2V4O7hQG+2+Q== X-Received: by 2002:a17:907:2687:b0:a9a:2afc:e4ed with SMTP id a640c23a62f3a-a9eeaefc7c7mr41016266b.32.1730995036463; Thu, 07 Nov 2024 07:57:16 -0800 (PST) Received: from localhost (fwdproxy-lla-115.fbsv.net. [2a03:2880:30ff:73::face:b00c]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9ee0defb61sm108826666b.150.2024.11.07.07.57.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Nov 2024 07:57:15 -0800 (PST) From: Breno Leitao Date: Thu, 07 Nov 2024 07:57:06 -0800 Subject: [PATCH net-next v2 1/2] net: netpoll: Individualize the skb pool Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20241107-skb_buffers_v2-v2-1-288c6264ba4f@debian.org> References: <20241107-skb_buffers_v2-v2-0-288c6264ba4f@debian.org> In-Reply-To: <20241107-skb_buffers_v2-v2-0-288c6264ba4f@debian.org> To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3542; i=leitao@debian.org; h=from:subject:message-id; bh=l/0wZIy7SkdSRdFTGTTWstgdulRmA7dbmH3XYkTeIRU=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBnLONZVOc3C8bsORLiLKniO8jahyMHqRR4634kF KH5CID95xaJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCZyzjWQAKCRA1o5Of/Hh3 bRvND/9JDVObUiBBbAkmjSkcfz+vTq0fof9B/MUryFzYByklPMc8Svu2T4jmc2IwSraWH5wKlWQ xAgpn4j6+CtLwj5FSRiGtJ1yIh/70cR3kyify7e1m+Hx9xFU8ZtiJq3tshxm5p4sX5KJ+dbmeQm bkZ9It7XgBiuzLRaijlMtYi/oVExw8USNDbqZDfa+L1ibXKoo5ufA3ef6HyYo2tttQ2YjWK1K7P mGEBmbWG8uuKxv9wajBdgTl0YiSLAFQflT+GSq7Ta3V20bE7bjXUePO48Bc4pQOqpRD1FsEpe9G kJF8Y7JDg83SlHu2f0OlKLnVXsa542AOt6j7/yqHNKkYQ4aLTFLvEZCVvBzCQnNEz5dT/wa5MDO ZkBjqFx7fVufOvQl7PYTiPpAAwuDOtcC8eV/3j+KTgSfxakhamwyaIZU4Rhc/oGaGRoh7gRMcVH rGVOiFfKTAvlL0sq6QgMNQBCzyK4a7VZiYBBpRbtZRq1qyR5oWm0QhcKRoVB7UNjg2kyIBSwzMj hrMfEZUm1q4OG6IOKQhocqRiUmDOwEZ+UkS3HtlAXp0jsvqj5kkEbHnnaItyq8ENuZH60Hf1k/X XDyiSayAwzj7QvoiGH0f1yY0X+jqbyLTi6ZvZPsqSxWjPoM8Emofn19zAWzl8VUk6nVBkjy22QJ QWXbu3ixtS6Ijug== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D The current implementation of the netpoll system uses a global skb pool, which can lead to inefficient memory usage and waste when targets are disabled or no longer in use. This can result in a significant amount of memory being unnecessarily allocated and retained, potentially causing performance issues and limiting the availability of resources for other system components. Modify the netpoll system to assign a skb pool to each target instead of using a global one. This approach allows for more fine-grained control over memory allocation and deallocation, ensuring that resources are only allocated and retained as needed. Signed-off-by: Breno Leitao --- include/linux/netpoll.h | 1 + net/core/netpoll.c | 31 +++++++++++++------------------ 2 files changed, 14 insertions(+), 18 deletions(-) diff --git a/include/linux/netpoll.h b/include/linux/netpoll.h index cd4e28db0cbd77572a579aff2067b5864d1a904a..77635b885c18b7d405642c2e7f3= 9f5ff2c7d469d 100644 --- a/include/linux/netpoll.h +++ b/include/linux/netpoll.h @@ -32,6 +32,7 @@ struct netpoll { bool ipv6; u16 local_port, remote_port; u8 remote_mac[ETH_ALEN]; + struct sk_buff_head skb_pool; }; =20 struct netpoll_info { diff --git a/net/core/netpoll.c b/net/core/netpoll.c index 94b7f07a952fff3358cc609fb29de33ae8ae8626..719c9aae845fbeb6f5b53a2bef6= 75d3cb8cd44a7 100644 --- a/net/core/netpoll.c +++ b/net/core/netpoll.c @@ -45,9 +45,6 @@ =20 #define MAX_UDP_CHUNK 1460 #define MAX_SKBS 32 - -static struct sk_buff_head skb_pool; - #define USEC_PER_POLL 50 =20 #define MAX_SKB_SIZE \ @@ -234,20 +231,23 @@ void netpoll_poll_enable(struct net_device *dev) up(&ni->dev_lock); } =20 -static void refill_skbs(void) +static void refill_skbs(struct netpoll *np) { + struct sk_buff_head *skb_pool; struct sk_buff *skb; unsigned long flags; =20 - spin_lock_irqsave(&skb_pool.lock, flags); - while (skb_pool.qlen < MAX_SKBS) { + skb_pool =3D &np->skb_pool; + + spin_lock_irqsave(&skb_pool->lock, flags); + while (skb_pool->qlen < MAX_SKBS) { skb =3D alloc_skb(MAX_SKB_SIZE, GFP_ATOMIC); if (!skb) break; =20 - __skb_queue_tail(&skb_pool, skb); + __skb_queue_tail(skb_pool, skb); } - spin_unlock_irqrestore(&skb_pool.lock, flags); + spin_unlock_irqrestore(&skb_pool->lock, flags); } =20 static void zap_completion_queue(void) @@ -284,12 +284,12 @@ static struct sk_buff *find_skb(struct netpoll *np, i= nt len, int reserve) struct sk_buff *skb; =20 zap_completion_queue(); - refill_skbs(); + refill_skbs(np); repeat: =20 skb =3D alloc_skb(len, GFP_ATOMIC); if (!skb) - skb =3D skb_dequeue(&skb_pool); + skb =3D skb_dequeue(&np->skb_pool); =20 if (!skb) { if (++count < 10) { @@ -673,6 +673,8 @@ int netpoll_setup(struct netpoll *np) struct in_device *in_dev; int err; =20 + skb_queue_head_init(&np->skb_pool); + rtnl_lock(); if (np->dev_name[0]) { struct net *net =3D current->nsproxy->net_ns; @@ -773,7 +775,7 @@ int netpoll_setup(struct netpoll *np) } =20 /* fill up the skb queue */ - refill_skbs(); + refill_skbs(np); =20 err =3D __netpoll_setup(np, ndev); if (err) @@ -792,13 +794,6 @@ int netpoll_setup(struct netpoll *np) } EXPORT_SYMBOL(netpoll_setup); =20 -static int __init netpoll_init(void) -{ - skb_queue_head_init(&skb_pool); - return 0; -} -core_initcall(netpoll_init); - static void rcu_cleanup_netpoll_info(struct rcu_head *rcu_head) { struct netpoll_info *npinfo =3D --=20 2.43.5 From nobody Sun Nov 24 01:44:02 2024 Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1CEA215C73; Thu, 7 Nov 2024 15:57:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730995042; cv=none; b=ElxX38etUUep2ep2BAHiHmHUsstBkm12XPo3xld4St7MT0uBdQnwReX84xH4V78qh4tmkCKYesvLOHK8BTFwRNCo/GrTRs3TgUC0uOnxX11GxlGj8WnvYiHsc1h81UIYbIIRmybUUDAWnkIzPq3zjnO/o6VqpbiO/qobdNxl35E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730995042; c=relaxed/simple; bh=O6G3z98wXMU2YHskw+iWUsDZqJS9nuyfK8uqaz/Y0D4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PrM37apsp0grTUOk5TuTi2O+nJUJmqseaApV3j3MYDQ0zHwElB3Pv0Yd0YQx3KcqRd1u2sxVEbmwvpfbwGLko95Kla/lqatOSesDKx0gSPthEs11+gtADVCtxrHEPLhVxB9AP4xDMkYRJ8wvov2yfuw8RvusxuzS6El6udYWMso= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.218.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-a99f1fd20c4so157879566b.0; Thu, 07 Nov 2024 07:57:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730995038; x=1731599838; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IqS89/NeNKi4uydQWItWI4zKvSn/L7HP4SPCkznjfEg=; b=Ir5k2JdPzAIewW5ggXfQiPw+A9VHI/kDcsWQ2l6K3uz7K1ajW4lsbPtNTc0rcDekUj FuDJTaOZdeQQP6yHtZOI64CwCux2IJJoEFNWasylq9D5LMH8S4cRCUNQwom7q7tbm5hM O6vjyVi8PGeTs8Aqo2YrAZtWqDaaHrrlnCBWCp+1DT46H+hpxtNEFCmlj3h00WN4jIzH zD8gm33tL9AJXRMYpxx6iJXKzJv9pIumRo0XYaIrm/EA2lKlDtc2Gmt8obZZ2Y4gXM+t scOwM08j75BMwZpr3Uk84qEjif93gFHApX1KcNRBLINYQFXdSLtXPN4hopOEgyYeuc8B cPcw== X-Forwarded-Encrypted: i=1; AJvYcCXSwFSSk5IVhFLPGz0yBWp/N9n9dBqsvz2DdrnOcJdFvpYBoTjnClO2vDksznopXE8khDNm78qlQKalNbI=@vger.kernel.org X-Gm-Message-State: AOJu0Yy6QmEvzA5e7lu2N63Cn7hDqVZQUmtJfk+lf077aKaXc0Bg6I1K 2N6gLadnL1qBoPLghd+HgVC1galSnHf9bDxOj6sYI4mzjaO9auTgouUkHA== X-Google-Smtp-Source: AGHT+IFZsXP79pPfpJZ4CyW5PRH8wvT1NzyJot5WN9TK02OPh2rk1OeSeDDBjvHvwRHy2R4aGqRH4Q== X-Received: by 2002:a17:907:d0c:b0:a99:fb56:39cc with SMTP id a640c23a62f3a-a9eeafc8be9mr48057866b.38.1730995038081; Thu, 07 Nov 2024 07:57:18 -0800 (PST) Received: from localhost (fwdproxy-lla-116.fbsv.net. [2a03:2880:30ff:74::face:b00c]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9ee0e0faaesm107761666b.175.2024.11.07.07.57.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Nov 2024 07:57:17 -0800 (PST) From: Breno Leitao Date: Thu, 07 Nov 2024 07:57:07 -0800 Subject: [PATCH net-next v2 2/2] net: netpoll: flush skb pool during cleanup Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20241107-skb_buffers_v2-v2-2-288c6264ba4f@debian.org> References: <20241107-skb_buffers_v2-v2-0-288c6264ba4f@debian.org> In-Reply-To: <20241107-skb_buffers_v2-v2-0-288c6264ba4f@debian.org> To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Breno Leitao X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1806; i=leitao@debian.org; h=from:subject:message-id; bh=O6G3z98wXMU2YHskw+iWUsDZqJS9nuyfK8uqaz/Y0D4=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBnLONZV4J75wiXQl5WtSLzAYYRB+uRtpMLkAIcC NUfli+1ivOJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCZyzjWQAKCRA1o5Of/Hh3 bd1BD/9/rVU0n1pYI4bVOAX3Nfg3FE+c7qVkbcY3wGSSAj5h2aPYGM6NL/zBw1bjIxGiG6Z6j5e t/XxJMEk8Y+HqipaV0uj4yjl2aYXT0S6RHkrh69b4fotmzTtRzct6bbD1tqUGU9RVZeOaq8z4UG hvd3s1ypGU/94F6PkticqmdUDUdvlIE8wimHSl7FLtlavSvSt0ys0y6NFgNyRKn70ON3orbBc2V q50UMc7qfYIWsNll281x3f8lm0gCFLqjALrQpytx4FID+y2SRMxz85q8ePkOdKY9bGoEtPTbTa3 s2SVSEE73DkgFgkJOzUnpeEJYpjkRLhBIZz2hwW5GD7fxskv4vMnSmrm9hL6XOELU/g1oFPJDRi 7Wp+UM6a7fCV7XGkZAIDf+22Df29M+RyJVvFGyZrozBA8yvuxunBeLQGmRo+LVFQ4wq/ORgXu7G HmGTOC5/41SFxfP0qRcH3Ows4QVniQpInCdLyvQ9Ao3Oete+ShN5m+ZK+NztxDCHi07jcnFwCKI rhIQ0rs5KSPHBtmBBwkRNgzaSH1B6rSYa99klzGK6H4KYv4mln62Hr0TLmkPO3p6BOAWq1ftKGP Q1/talQ+ckI+SASkT2vI1C5nydrotzSmHx8cD0oy4OZa/ka4h4tmpxt6rc0Xps/rBIxHoONLgW4 PGsB7Ikpn+Z1aNA== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D The netpoll subsystem maintains a pool of 32 pre-allocated SKBs per instance, but these SKBs are not freed when the netpoll user is brought down. This leads to memory waste as these buffers remain allocated but unused. Add skb_pool_flush() to properly clean up these SKBs when netconsole is terminated, improving memory efficiency. Signed-off-by: Breno Leitao --- net/core/netpoll.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/net/core/netpoll.c b/net/core/netpoll.c index 719c9aae845fbeb6f5b53a2bef675d3cb8cd44a7..498cc38073a3dc9f829f74f254b= c70b26ef3856c 100644 --- a/net/core/netpoll.c +++ b/net/core/netpoll.c @@ -531,6 +531,22 @@ static int netpoll_parse_ip_addr(const char *str, unio= n inet_addr *addr) return -1; } =20 +static void skb_pool_flush(struct netpoll *np) +{ + struct sk_buff_head *skb_pool; + struct sk_buff *skb; + unsigned long flags; + + skb_pool =3D &np->skb_pool; + + spin_lock_irqsave(&skb_pool->lock, flags); + while (skb_pool->qlen > 0) { + skb =3D __skb_dequeue(skb_pool); + kfree_skb(skb); + } + spin_unlock_irqrestore(&skb_pool->lock, flags); +} + int netpoll_parse_options(struct netpoll *np, char *opt) { char *cur=3Dopt, *delim; @@ -779,10 +795,12 @@ int netpoll_setup(struct netpoll *np) =20 err =3D __netpoll_setup(np, ndev); if (err) - goto put; + goto flush; rtnl_unlock(); return 0; =20 +flush: + skb_pool_flush(np); put: DEBUG_NET_WARN_ON_ONCE(np->dev); if (ip_overwritten) @@ -830,6 +848,8 @@ void __netpoll_cleanup(struct netpoll *np) call_rcu(&npinfo->rcu, rcu_cleanup_netpoll_info); } else RCU_INIT_POINTER(np->dev->npinfo, NULL); + + skb_pool_flush(np); } EXPORT_SYMBOL_GPL(__netpoll_cleanup); =20 --=20 2.43.5