From nobody Wed Nov 27 06:32:26 2024 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3BE51D014E for ; Fri, 11 Oct 2024 18:45:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672344; cv=none; b=MxcZmfbkwSIqc7Y6ARpXjix19DCj5/YjslYhV+ngF3Et4rfPT3I04Vlei++6NsvztbfqH/y46caGTqGMM+H8zVeaRtVEm6yJy3hP3pXCIJI7uBCnkmbZtU5lQSBD//2xRFiLUhXrAEzddUSJ50GMhrv898qR1TTuOYlkZXbokZg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672344; c=relaxed/simple; bh=1O1kGSWJK9mXzhgtA1OKFr6nhAfyWnBI/4+yFGVv6WA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iVVEN+2ZpKOXijE5gNSlo5xoNGnUAFojlXCq+u6JMoswweX7YYxui9gA6TQfxTudhufwFH7gp+VPZreRhwwsjTJ7EJoK/v+4p9BGtprBUde75xhwXro7OtrClKU27MKTYDwdk9s88Ah/4clVxBoTmFAlOHDC1iO1Md76G20nrOU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=X1i2cc/O; arc=none smtp.client-ip=209.85.216.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="X1i2cc/O" Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-2e2e23f2931so1465676a91.0 for ; Fri, 11 Oct 2024 11:45:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1728672342; x=1729277142; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/UOc67eyBhG5gGRkb0LgZtml97A93vYaVKIgdUp4Lkw=; b=X1i2cc/OjqWwdU1AJ8Km7VetM8d1QFES0C7sXjBwvNvmDvyjKasUDwiy8Bb6mlTcol AaJpNcbDfoH1FW1Uq7WVJ+gvPzhE8XizxRiX6qjAoJuWvNDZ7x7ZPnmhGDMBvqnejqqw XHC31VIkTadfb2ROEK6s+5fVc5LfSSOXTqIho= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728672342; x=1729277142; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/UOc67eyBhG5gGRkb0LgZtml97A93vYaVKIgdUp4Lkw=; b=FJD/FI+jyqG9ZnBY7npLlXGBTNtmrCvYg1pImneDapd4kjIpqINjLRz3ybzeWVKujI Y5bbSgb/cUnNkMtiJEX/WR9vJ/33/dQw0WBEIqZwZuUV1oUOTTyHNZrwPItL+8/wzemw C0MeA/yDQ3cu3AXH4kCdjdj+r8avT5rG1KjYlrNM3ZLKaIJiR7Fyfub7cs+TwgJZyXXM FIpTl+NgvhVGSVnrEidU/75I3hSJd9D0F9b7HAUk+uvgXh9riB28Lix+uz8ON1N/3guz /PUvS5ECZajVai+e3QJEJzDz8wgaJZgh+4oGUmao0T5iinCqEoZKggYNyiySenK3+7kM Po2g== X-Forwarded-Encrypted: i=1; AJvYcCUwW2CIIyGeHrF8W9RnZ/WJRWNzri6xKRPge8M9AyCiNIjdsspG4OuL5kW0OFwrWP0hMXh9Ud1h8ut2mBU=@vger.kernel.org X-Gm-Message-State: AOJu0YxPUT92FkLBs2vdIgs2sytmvvHqwRTI2OkCj01H8V1sNDBAtVYB EX8vzmhJ3MO/KKFyOP9wrtme22tBPoLpHAPm9jUd7o9yexvwnS9bxfVGvCCmfdo= X-Google-Smtp-Source: AGHT+IErkyicZp3WCEfq9BlQNkR5KZMGrmgYeItXsbJJyUYKuK1vGNK9wwxluDQVA+T0yl36htpl2A== X-Received: by 2002:a17:90b:1005:b0:2e2:e0c1:4452 with SMTP id 98e67ed59e1d1-2e31539036fmr406881a91.41.1728672341941; Fri, 11 Oct 2024 11:45:41 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e30d848e1csm687625a91.42.2024.10.11.11.45.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 11:45:41 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: mkarsten@uwaterloo.ca, skhawaja@google.com, sdf@fomichev.me, bjorn@rivosinc.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, willemdebruijn.kernel@gmail.com, edumazet@google.com, Joe Damato , Jakub Kicinski , "David S. Miller" , Paolo Abeni , Jonathan Corbet , Jiri Pirko , Sebastian Andrzej Siewior , Lorenzo Bianconi , Johannes Berg , Breno Leitao , Alexander Lobakin , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-kernel@vger.kernel.org (open list) Subject: [net-next v6 1/9] net: napi: Make napi_defer_hard_irqs per-NAPI Date: Fri, 11 Oct 2024 18:44:56 +0000 Message-Id: <20241011184527.16393-2-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241011184527.16393-1-jdamato@fastly.com> References: <20241011184527.16393-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add defer_hard_irqs to napi_struct in preparation for per-NAPI settings. The existing sysfs parameter is respected; writes to sysfs will write to all NAPI structs for the device and the net_device defer_hard_irq field. Reads from sysfs show the net_device field. The ability to set defer_hard_irqs on specific NAPI instances will be added in a later commit, via netdev-genl. Signed-off-by: Joe Damato Reviewed-by: Eric Dumazet Reviewed-by: Jakub Kicinski --- .../networking/net_cachelines/net_device.rst | 1 + include/linux/netdevice.h | 3 +- net/core/dev.c | 10 +++--- net/core/dev.h | 36 +++++++++++++++++++ net/core/net-sysfs.c | 2 +- 5 files changed, 45 insertions(+), 7 deletions(-) diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Docum= entation/networking/net_cachelines/net_device.rst index 1b018ac35e9a..5a7388b2ab6f 100644 --- a/Documentation/networking/net_cachelines/net_device.rst +++ b/Documentation/networking/net_cachelines/net_device.rst @@ -186,4 +186,5 @@ struct dpll_pin* dpll_pin struct hlist_head page_pools struct dim_irq_moder* irq_moder u64 max_pacing_offload_horizon +u32 napi_defer_hard_irqs =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index e6b93d01e631..2e7bc23660ec 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -373,6 +373,7 @@ struct napi_struct { unsigned int napi_id; struct hrtimer timer; struct task_struct *thread; + u32 defer_hard_irqs; /* control-path-only fields follow */ struct list_head dev_list; struct hlist_node napi_hash_node; @@ -2085,7 +2086,6 @@ struct net_device { unsigned int real_num_rx_queues; struct netdev_rx_queue *_rx; unsigned long gro_flush_timeout; - u32 napi_defer_hard_irqs; unsigned int gro_max_size; unsigned int gro_ipv4_max_size; rx_handler_func_t __rcu *rx_handler; @@ -2413,6 +2413,7 @@ struct net_device { struct dim_irq_moder *irq_moder; =20 u64 max_pacing_offload_horizon; + u32 napi_defer_hard_irqs; =20 /** * @lock: protects @net_shaper_hierarchy, feel free to use for other diff --git a/net/core/dev.c b/net/core/dev.c index b590eefce3b4..fbaa9eabf77f 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6233,7 +6233,7 @@ bool napi_complete_done(struct napi_struct *n, int wo= rk_done) if (work_done) { if (n->gro_bitmask) timeout =3D READ_ONCE(n->dev->gro_flush_timeout); - n->defer_hard_irqs_count =3D READ_ONCE(n->dev->napi_defer_hard_irqs); + n->defer_hard_irqs_count =3D napi_get_defer_hard_irqs(n); } if (n->defer_hard_irqs_count > 0) { n->defer_hard_irqs_count--; @@ -6371,7 +6371,7 @@ static void busy_poll_stop(struct napi_struct *napi, = void *have_poll_lock, bpf_net_ctx =3D bpf_net_ctx_set(&__bpf_net_ctx); =20 if (flags & NAPI_F_PREFER_BUSY_POLL) { - napi->defer_hard_irqs_count =3D READ_ONCE(napi->dev->napi_defer_hard_irq= s); + napi->defer_hard_irqs_count =3D napi_get_defer_hard_irqs(napi); timeout =3D READ_ONCE(napi->dev->gro_flush_timeout); if (napi->defer_hard_irqs_count && timeout) { hrtimer_start(&napi->timer, ns_to_ktime(timeout), HRTIMER_MODE_REL_PINN= ED); @@ -6653,6 +6653,7 @@ void netif_napi_add_weight(struct net_device *dev, st= ruct napi_struct *napi, INIT_HLIST_NODE(&napi->napi_hash_node); hrtimer_init(&napi->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED); napi->timer.function =3D napi_watchdog; + napi_set_defer_hard_irqs(napi, READ_ONCE(dev->napi_defer_hard_irqs)); init_gro_hash(napi); napi->skb =3D NULL; INIT_LIST_HEAD(&napi->rx_list); @@ -11059,7 +11060,7 @@ void netdev_sw_irq_coalesce_default_on(struct net_d= evice *dev) =20 if (!IS_ENABLED(CONFIG_PREEMPT_RT)) { dev->gro_flush_timeout =3D 20000; - dev->napi_defer_hard_irqs =3D 1; + netdev_set_defer_hard_irqs(dev, 1); } } EXPORT_SYMBOL_GPL(netdev_sw_irq_coalesce_default_on); @@ -12003,7 +12004,6 @@ static void __init net_dev_struct_check(void) CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, real= _num_rx_queues); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, _rx); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, gro_= flush_timeout); - CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, napi= _defer_hard_irqs); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, gro_= max_size); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, gro_= ipv4_max_size); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, rx_h= andler); @@ -12015,7 +12015,7 @@ static void __init net_dev_struct_check(void) #ifdef CONFIG_NET_XGRESS CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, tcx_= ingress); #endif - CACHELINE_ASSERT_GROUP_SIZE(struct net_device, net_device_read_rx, 104); + CACHELINE_ASSERT_GROUP_SIZE(struct net_device, net_device_read_rx, 100); } =20 /* diff --git a/net/core/dev.h b/net/core/dev.h index d3ea92949ff3..0716b1048261 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -148,6 +148,42 @@ static inline void netif_set_gro_ipv4_max_size(struct = net_device *dev, WRITE_ONCE(dev->gro_ipv4_max_size, size); } =20 +/** + * napi_get_defer_hard_irqs - get the NAPI's defer_hard_irqs + * @n: napi struct to get the defer_hard_irqs field from + * + * Return: the per-NAPI value of the defar_hard_irqs field. + */ +static inline u32 napi_get_defer_hard_irqs(const struct napi_struct *n) +{ + return READ_ONCE(n->defer_hard_irqs); +} + +/** + * napi_set_defer_hard_irqs - set the defer_hard_irqs for a napi + * @n: napi_struct to set the defer_hard_irqs field + * @defer: the value the field should be set to + */ +static inline void napi_set_defer_hard_irqs(struct napi_struct *n, u32 def= er) +{ + WRITE_ONCE(n->defer_hard_irqs, defer); +} + +/** + * netdev_set_defer_hard_irqs - set defer_hard_irqs for all NAPIs of a net= dev + * @netdev: the net_device for which all NAPIs will have defer_hard_irqs s= et + * @defer: the defer_hard_irqs value to set + */ +static inline void netdev_set_defer_hard_irqs(struct net_device *netdev, + u32 defer) +{ + struct napi_struct *napi; + + WRITE_ONCE(netdev->napi_defer_hard_irqs, defer); + list_for_each_entry(napi, &netdev->napi_list, dev_list) + napi_set_defer_hard_irqs(napi, defer); +} + int rps_cpumask_housekeeping(struct cpumask *mask); =20 #if defined(CONFIG_DEBUG_NET) && defined(CONFIG_BPF_SYSCALL) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 05cf5347f25e..25125f356a15 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -429,7 +429,7 @@ static int change_napi_defer_hard_irqs(struct net_devic= e *dev, unsigned long val if (val > S32_MAX) return -ERANGE; =20 - WRITE_ONCE(dev->napi_defer_hard_irqs, val); + netdev_set_defer_hard_irqs(dev, (u32)val); return 0; } =20 --=20 2.25.1 From nobody Wed Nov 27 06:32:26 2024 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EC7A1D0E0F for ; Fri, 11 Oct 2024 18:45:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672348; cv=none; b=eT1CIwNXJ3pOluS05ssK+BeRlR/7Fb1uc4r3LUICFekjBOzWm+ZN3m4GH1iEPzx2DRdxMoTTRqReJEe5I6D0ca5vM6Xa4dv1GAEzr0HuUop2YFV70lxc8FTdsk0QGT+QsU9cMpZqmoKtsHxjqLzmupKFXgSlU/rmuM0ScyejiMM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672348; c=relaxed/simple; bh=q+b9jZGAidjuh8GtSlIIKJ1LqbGOzaugQLHfnQm1lqE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AG0bMzRHxUh76jXXjFym1C9NpyR0+gnHMe83EQ6wR8ilCMEG2q1uuZLVS/NUwpc9bjo/Qr7Nw78ANWcJ0jHc9gwxAaNzcl3TDV1M9OBFYzyINnRICVRH6/IJMWRxMqGBP2cLu4F514QlxyOwYOa9+hL/N9G+FXYWFipY/c+YhIM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=ZgSUi6K+; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="ZgSUi6K+" Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2e28b75dbd6so1791145a91.0 for ; Fri, 11 Oct 2024 11:45:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1728672346; x=1729277146; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yXIej0C4eoHUHQ4nEdeNbNdLeJSUvgnrgxPSPMx1EiE=; b=ZgSUi6K+fg/sFQvrgP4gqFZWja7bxeRtBjGIIsSYjz8p0bbbRpKi4XeAUlLVNNjkzR 9ha4BKc8FTRCLUGKEUoXjOM+FuuS0GZ466Jo6Q0LlOEYtfKaFnrgaCIFbtsSs9HrUQmJ FELVyiIjfwI/LwPBoQJBMFCDN0dSs/dOM15tc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728672346; x=1729277146; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yXIej0C4eoHUHQ4nEdeNbNdLeJSUvgnrgxPSPMx1EiE=; b=I23UpZhvt8aB2ZvBpCAIplvRlrhrUM0BhUJZvXH9MnIobpX8eeZQHRq5asJT9PU8rW 6Pw+7g9V2LjAneX9/LJzBMA1fG4omlOAW815ciKgPqK3eLmgh023r/9dxHn6YJe3nKp+ Up6dstcH3GQe47nflOiqZsH+B5ikgpXVmQ7o/HH5si4IwXWDpZZQCLdFG+CFRPAQSshq w7x7Ah6pBz2L1X3R2fagmnwf0u1EC9fl0hphqaoblLPuk8Khj+zCyHsfBw+qMNgjLMPC 2J3A7IBT8Y00W9bovKovk111Mn0V5ji01seSeJVjkmezO4HkEhaLuWCCLgqE0jAwVYLw APJQ== X-Forwarded-Encrypted: i=1; AJvYcCUUxwUTKpW2X9QjvTO16Kj/QxH5kJ4mhBdAcFc/8GXLo8b9WVHlTUbxeCfHBpxEFIXPZK7Mlom4uxkTITE=@vger.kernel.org X-Gm-Message-State: AOJu0YyvBi4oMOquszKKexzPLh7A0QQQJp0Bo+lKggjReMWf8adIw4Sz Gr/QtosF0HEyF8L7xKTh5XTa8svSf2+9++SFii0NG4Zy2nL4mlVmseb1G7Rl2/8= X-Google-Smtp-Source: AGHT+IFnsyEs1/CwI3pWxWqWY7TURNss9/odTFhLrbhqMJEXea7TzszaE0iBWvpXWaM9N7sLxA6ERg== X-Received: by 2002:a17:90b:4ac3:b0:2e2:e2f8:104 with SMTP id 98e67ed59e1d1-2e3152b0097mr445535a91.8.1728672346549; Fri, 11 Oct 2024 11:45:46 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e30d848e1csm687625a91.42.2024.10.11.11.45.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 11:45:46 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: mkarsten@uwaterloo.ca, skhawaja@google.com, sdf@fomichev.me, bjorn@rivosinc.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, willemdebruijn.kernel@gmail.com, edumazet@google.com, Joe Damato , Jakub Kicinski , "David S. Miller" , Paolo Abeni , Donald Hunter , Jesper Dangaard Brouer , Mina Almasry , Xuan Zhuo , Larysa Zaremba , linux-kernel@vger.kernel.org (open list) Subject: [net-next v6 2/9] netdev-genl: Dump napi_defer_hard_irqs Date: Fri, 11 Oct 2024 18:44:57 +0000 Message-Id: <20241011184527.16393-3-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241011184527.16393-1-jdamato@fastly.com> References: <20241011184527.16393-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Support dumping defer_hard_irqs for a NAPI ID. Signed-off-by: Joe Damato Reviewed-by: Eric Dumazet Reviewed-by: Jakub Kicinski --- Documentation/netlink/specs/netdev.yaml | 8 ++++++++ include/uapi/linux/netdev.h | 1 + net/core/netdev-genl.c | 6 ++++++ tools/include/uapi/linux/netdev.h | 1 + 4 files changed, 16 insertions(+) diff --git a/Documentation/netlink/specs/netdev.yaml b/Documentation/netlin= k/specs/netdev.yaml index 08412c279297..585e87ec3c16 100644 --- a/Documentation/netlink/specs/netdev.yaml +++ b/Documentation/netlink/specs/netdev.yaml @@ -248,6 +248,13 @@ attribute-sets: threaded mode. If NAPI is not in threaded mode (i.e. uses nor= mal softirq context), the attribute will be absent. type: u32 + - + name: defer-hard-irqs + doc: The number of consecutive empty polls before IRQ deferral ends + and hardware IRQs are re-enabled. + type: u32 + checks: + max: s32-max - name: queue attributes: @@ -636,6 +643,7 @@ operations: - ifindex - irq - pid + - defer-hard-irqs dump: request: attributes: diff --git a/include/uapi/linux/netdev.h b/include/uapi/linux/netdev.h index 7c308f04e7a0..13dc0b027e86 100644 --- a/include/uapi/linux/netdev.h +++ b/include/uapi/linux/netdev.h @@ -122,6 +122,7 @@ enum { NETDEV_A_NAPI_ID, NETDEV_A_NAPI_IRQ, NETDEV_A_NAPI_PID, + NETDEV_A_NAPI_DEFER_HARD_IRQS, =20 __NETDEV_A_NAPI_MAX, NETDEV_A_NAPI_MAX =3D (__NETDEV_A_NAPI_MAX - 1) diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c index 358cba248796..f98e5d1d0d21 100644 --- a/net/core/netdev-genl.c +++ b/net/core/netdev-genl.c @@ -161,6 +161,7 @@ static int netdev_nl_napi_fill_one(struct sk_buff *rsp, struct napi_struct *napi, const struct genl_info *info) { + u32 napi_defer_hard_irqs; void *hdr; pid_t pid; =20 @@ -189,6 +190,11 @@ netdev_nl_napi_fill_one(struct sk_buff *rsp, struct na= pi_struct *napi, goto nla_put_failure; } =20 + napi_defer_hard_irqs =3D napi_get_defer_hard_irqs(napi); + if (nla_put_s32(rsp, NETDEV_A_NAPI_DEFER_HARD_IRQS, + napi_defer_hard_irqs)) + goto nla_put_failure; + genlmsg_end(rsp, hdr); =20 return 0; diff --git a/tools/include/uapi/linux/netdev.h b/tools/include/uapi/linux/n= etdev.h index 7c308f04e7a0..13dc0b027e86 100644 --- a/tools/include/uapi/linux/netdev.h +++ b/tools/include/uapi/linux/netdev.h @@ -122,6 +122,7 @@ enum { NETDEV_A_NAPI_ID, NETDEV_A_NAPI_IRQ, NETDEV_A_NAPI_PID, + NETDEV_A_NAPI_DEFER_HARD_IRQS, =20 __NETDEV_A_NAPI_MAX, NETDEV_A_NAPI_MAX =3D (__NETDEV_A_NAPI_MAX - 1) --=20 2.25.1 From nobody Wed Nov 27 06:32:26 2024 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A53D91D0160 for ; Fri, 11 Oct 2024 18:45:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672354; cv=none; b=iOZHyWJnltgRs1oHyVPNbQ1R+8GVDGqVwad/ge7o1k/sCpStbb2dR4aPk7Xq17hFZGk8BMK7LgPtYoT6wRknVI6xfyyxXaUBpkU6gi2ssyBg9s0V8n3zvtjqAOTbu4o0319bnVpiZGFqdYW38w39t9/UGWSz86885IpqBN1QGZI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672354; c=relaxed/simple; bh=jq2l+fxGXKoykb2l2iMxd+SFdR3dmpqdfUhhGDXCRWQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iGmFcHyDB/NEt7F7+jOL5LdpM4Cdx33iGySR0DMHlc7IUFHOHtZHEP0ihRsBuHrPbOteLb1d1zJzWYZ7bW5c5h8Btl/EwaJGZJhYZeD8WjVDfp9VhJpSl8m1XmM99W7+inx3H/z05oXUopgyGHg5AGqjK9+ZHadoH2gZmiwmP9w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=ssItfq8g; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="ssItfq8g" Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2e2bb1efe78so1685470a91.1 for ; Fri, 11 Oct 2024 11:45:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1728672352; x=1729277152; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=80z0m3ADJ29bsKYQWP0ErWo8cSlJeLlTsXMs/xhftIw=; b=ssItfq8gtTOaiM+A3T2CvbdGjVxfdLYPTuBBjTiUo39PUwSG4S0YxJpvVBe1D5RG76 sUKJ2Q9Jx9CmlA0WbrO95GnsntbgYL41Zf0FWQYtE2h1f6VKbx1YJmnAcuEFFtqRjJB1 D3x+oNQUpIaCUfVmRjOB+1h91y6u4+9jZxfNE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728672352; x=1729277152; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=80z0m3ADJ29bsKYQWP0ErWo8cSlJeLlTsXMs/xhftIw=; b=P5m29nx/DC61hZjpGEWwPTtitzKOdYZ+HcEQb9+VSXHyBmi7Q6a9mmOnjacAoAwkYI +4Jw7MP1BEGOWIO1kwCiz0mTcS8Okg7WCzAbVBMvNrEcUoTzmORgMxUnEOQq+w88OEko A0NRjqFye/PkHtRAKj41w1cWssftAp7U1sVkaVz6s5qkQpXFF9O7sge/XT0r2TL4k660 Uf/KaHkKip/2HRIC2p9CE/LTkEkbbB/ps8zB/2KCqQo+W1+n/JBEMe9KmV7ipvyH9Ir4 ojgq8ptSQbLrlj7HmnWELOMsTZ9kCadzXjD9A3B2P5is2anxY/gG+9J5Z7BI01ShN/Kw DfUA== X-Forwarded-Encrypted: i=1; AJvYcCWSiPA7aLxWNU8C+x/bdx0N/LktI/Q29OXBVR2aQgZUMPngKoswvEG1mf9juar9rGrQCPlYpd/7Q+FlAeY=@vger.kernel.org X-Gm-Message-State: AOJu0YwQKJUgUaF+Icym4TmLXbP154Z0uhUMQlXh92f9IEFRoI71Z6jP fv4bXy1+qlZJAWZ6t75k1Z9mjryptEdZkkBWTn0fhkXDZH/PJJNghCQPpDxyGEc+tW59NnHu/a5 K X-Google-Smtp-Source: AGHT+IFYcs3KxuO+CWTjDrHOAcPtxCILOLE7nbtAh6uSh43xs0LBvyu0sXaZNFv82HkWTgZFGAsk5A== X-Received: by 2002:a17:90b:897:b0:2e2:e086:f5c0 with SMTP id 98e67ed59e1d1-2e2e086f663mr9255734a91.5.1728672352013; Fri, 11 Oct 2024 11:45:52 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e30d848e1csm687625a91.42.2024.10.11.11.45.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 11:45:51 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: mkarsten@uwaterloo.ca, skhawaja@google.com, sdf@fomichev.me, bjorn@rivosinc.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, willemdebruijn.kernel@gmail.com, edumazet@google.com, Joe Damato , Jakub Kicinski , "David S. Miller" , Paolo Abeni , Jonathan Corbet , Jiri Pirko , Sebastian Andrzej Siewior , Lorenzo Bianconi , Johannes Berg , Breno Leitao , Alexander Lobakin , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-kernel@vger.kernel.org (open list) Subject: [net-next v6 3/9] net: napi: Make gro_flush_timeout per-NAPI Date: Fri, 11 Oct 2024 18:44:58 +0000 Message-Id: <20241011184527.16393-4-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241011184527.16393-1-jdamato@fastly.com> References: <20241011184527.16393-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allow per-NAPI gro_flush_timeout setting. The existing sysfs parameter is respected; writes to sysfs will write to all NAPI structs for the device and the net_device gro_flush_timeout field. Reads from sysfs will read from the net_device field. The ability to set gro_flush_timeout on specific NAPI instances will be added in a later commit, via netdev-genl. Signed-off-by: Joe Damato Reviewed-by: Eric Dumazet Reviewed-by: Jakub Kicinski --- .../networking/net_cachelines/net_device.rst | 1 + include/linux/netdevice.h | 3 +- net/core/dev.c | 12 +++--- net/core/dev.h | 40 +++++++++++++++++++ net/core/net-sysfs.c | 2 +- 5 files changed, 50 insertions(+), 8 deletions(-) diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Docum= entation/networking/net_cachelines/net_device.rst index 5a7388b2ab6f..67910ea49160 100644 --- a/Documentation/networking/net_cachelines/net_device.rst +++ b/Documentation/networking/net_cachelines/net_device.rst @@ -186,5 +186,6 @@ struct dpll_pin* dpll_pin struct hlist_head page_pools struct dim_irq_moder* irq_moder u64 max_pacing_offload_horizon +unsigned_long gro_flush_timeout u32 napi_defer_hard_irqs =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 2e7bc23660ec..93241d4de437 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -373,6 +373,7 @@ struct napi_struct { unsigned int napi_id; struct hrtimer timer; struct task_struct *thread; + unsigned long gro_flush_timeout; u32 defer_hard_irqs; /* control-path-only fields follow */ struct list_head dev_list; @@ -2085,7 +2086,6 @@ struct net_device { int ifindex; unsigned int real_num_rx_queues; struct netdev_rx_queue *_rx; - unsigned long gro_flush_timeout; unsigned int gro_max_size; unsigned int gro_ipv4_max_size; rx_handler_func_t __rcu *rx_handler; @@ -2413,6 +2413,7 @@ struct net_device { struct dim_irq_moder *irq_moder; =20 u64 max_pacing_offload_horizon; + unsigned long gro_flush_timeout; u32 napi_defer_hard_irqs; =20 /** diff --git a/net/core/dev.c b/net/core/dev.c index fbaa9eabf77f..e21ace3551d5 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6232,12 +6232,12 @@ bool napi_complete_done(struct napi_struct *n, int = work_done) =20 if (work_done) { if (n->gro_bitmask) - timeout =3D READ_ONCE(n->dev->gro_flush_timeout); + timeout =3D napi_get_gro_flush_timeout(n); n->defer_hard_irqs_count =3D napi_get_defer_hard_irqs(n); } if (n->defer_hard_irqs_count > 0) { n->defer_hard_irqs_count--; - timeout =3D READ_ONCE(n->dev->gro_flush_timeout); + timeout =3D napi_get_gro_flush_timeout(n); if (timeout) ret =3D false; } @@ -6372,7 +6372,7 @@ static void busy_poll_stop(struct napi_struct *napi, = void *have_poll_lock, =20 if (flags & NAPI_F_PREFER_BUSY_POLL) { napi->defer_hard_irqs_count =3D napi_get_defer_hard_irqs(napi); - timeout =3D READ_ONCE(napi->dev->gro_flush_timeout); + timeout =3D napi_get_gro_flush_timeout(napi); if (napi->defer_hard_irqs_count && timeout) { hrtimer_start(&napi->timer, ns_to_ktime(timeout), HRTIMER_MODE_REL_PINN= ED); skip_schedule =3D true; @@ -6654,6 +6654,7 @@ void netif_napi_add_weight(struct net_device *dev, st= ruct napi_struct *napi, hrtimer_init(&napi->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED); napi->timer.function =3D napi_watchdog; napi_set_defer_hard_irqs(napi, READ_ONCE(dev->napi_defer_hard_irqs)); + napi_set_gro_flush_timeout(napi, READ_ONCE(dev->gro_flush_timeout)); init_gro_hash(napi); napi->skb =3D NULL; INIT_LIST_HEAD(&napi->rx_list); @@ -11059,7 +11060,7 @@ void netdev_sw_irq_coalesce_default_on(struct net_d= evice *dev) WARN_ON(dev->reg_state =3D=3D NETREG_REGISTERED); =20 if (!IS_ENABLED(CONFIG_PREEMPT_RT)) { - dev->gro_flush_timeout =3D 20000; + netdev_set_gro_flush_timeout(dev, 20000); netdev_set_defer_hard_irqs(dev, 1); } } @@ -12003,7 +12004,6 @@ static void __init net_dev_struct_check(void) CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, ifin= dex); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, real= _num_rx_queues); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, _rx); - CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, gro_= flush_timeout); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, gro_= max_size); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, gro_= ipv4_max_size); CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, rx_h= andler); @@ -12015,7 +12015,7 @@ static void __init net_dev_struct_check(void) #ifdef CONFIG_NET_XGRESS CACHELINE_ASSERT_GROUP_MEMBER(struct net_device, net_device_read_rx, tcx_= ingress); #endif - CACHELINE_ASSERT_GROUP_SIZE(struct net_device, net_device_read_rx, 100); + CACHELINE_ASSERT_GROUP_SIZE(struct net_device, net_device_read_rx, 92); } =20 /* diff --git a/net/core/dev.h b/net/core/dev.h index 0716b1048261..7d0aab7e3ef1 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -184,6 +184,46 @@ static inline void netdev_set_defer_hard_irqs(struct n= et_device *netdev, napi_set_defer_hard_irqs(napi, defer); } =20 +/** + * napi_get_gro_flush_timeout - get the gro_flush_timeout + * @n: napi struct to get the gro_flush_timeout from + * + * Return: the per-NAPI value of the gro_flush_timeout field. + */ +static inline unsigned long +napi_get_gro_flush_timeout(const struct napi_struct *n) +{ + return READ_ONCE(n->gro_flush_timeout); +} + +/** + * napi_set_gro_flush_timeout - set the gro_flush_timeout for a napi + * @n: napi struct to set the gro_flush_timeout + * @timeout: timeout value to set + * + * napi_set_gro_flush_timeout sets the per-NAPI gro_flush_timeout + */ +static inline void napi_set_gro_flush_timeout(struct napi_struct *n, + unsigned long timeout) +{ + WRITE_ONCE(n->gro_flush_timeout, timeout); +} + +/** + * netdev_set_gro_flush_timeout - set gro_flush_timeout of a netdev's NAPIs + * @netdev: the net_device for which all NAPIs will have gro_flush_timeout= set + * @timeout: the timeout value to set + */ +static inline void netdev_set_gro_flush_timeout(struct net_device *netdev, + unsigned long timeout) +{ + struct napi_struct *napi; + + WRITE_ONCE(netdev->gro_flush_timeout, timeout); + list_for_each_entry(napi, &netdev->napi_list, dev_list) + napi_set_gro_flush_timeout(napi, timeout); +} + int rps_cpumask_housekeeping(struct cpumask *mask); =20 #if defined(CONFIG_DEBUG_NET) && defined(CONFIG_BPF_SYSCALL) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 25125f356a15..2d9afc6e2161 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -409,7 +409,7 @@ NETDEVICE_SHOW_RW(tx_queue_len, fmt_dec); =20 static int change_gro_flush_timeout(struct net_device *dev, unsigned long = val) { - WRITE_ONCE(dev->gro_flush_timeout, val); + netdev_set_gro_flush_timeout(dev, val); return 0; } =20 --=20 2.25.1 From nobody Wed Nov 27 06:32:26 2024 Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 625181D0421 for ; Fri, 11 Oct 2024 18:45:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672358; cv=none; b=Ckj9PH83ahxD/Hfg4/lKaNKTlxl109fqZwStg+sgqmwbFmDsVVi1ClbpDzkbrB5T4GCBGgBW+yzY8A1o7CY/rEnBetZjDUtviiU4kZuAXzfXtd2AICaX0HKUKtNvRAoM1WOP8UPBW0duKnjdg5Gv8QNR7XxYuuD0Zk1w0HbWIQo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672358; c=relaxed/simple; bh=8bCROnq391qhvbK6vLiYM25EQ9c1S2j14ZF+/SPGKz8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cWI0xxadVYJZLHTBSxFwAAOjn3cy2sCxsL4LjLjqhHBPOeaAwnidkTluKkkk4x2qf61Oc74zu6I7vkffJOlpVyYl7m8GdvzJr3RsCyACPXCQ6E0ZKcgKbg+pyX1XsOvrA/KsvfY+Km0Eb6CyGhENglIa4J+4xNfSYzwVrY/FcVY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=jtOOcDhR; arc=none smtp.client-ip=209.85.215.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="jtOOcDhR" Received: by mail-pg1-f173.google.com with SMTP id 41be03b00d2f7-7db908c9c83so1400116a12.2 for ; Fri, 11 Oct 2024 11:45:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1728672356; x=1729277156; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N45oz9QM8Oj1BURj2520V1UATs/lARHOECpxZOQBaPI=; b=jtOOcDhRoBxz6Ckk4ImR5Yu/Wbqf6sdRb8TdJlMBP7OSKkhXWdy/ohaziUA8ymxNXN gmR8g8WIQUIwvtUg30lHWoksBMdUQaKRuz4XUwJ+s/FpFhkhZFJOwq38Lp+3K3LHyLZ4 M0Gl4qDD/WG1jgCetrIqk4YzLa0kbO7JTjLEY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728672356; x=1729277156; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N45oz9QM8Oj1BURj2520V1UATs/lARHOECpxZOQBaPI=; b=H44M4vsLud5X9PqLn3FLAfpFaR4qy9s9pcruQoZ0m7qM9+v92yhRrbSGJTdNQ9eB89 YWluIKkIUuliM/6CnQS0XUwB+guYhAIYGHcQ6snVbl7B36hUa03t0MIwhuplsxn6/8Fb mFd8NRfQg4MEB6s4VYROT7MN0T/kc2E6n1GceLDffYzutOL+mbGzVXZ2xun9e82ptots oE3n+gLkHbZSv0O5Av648OcN3Gtv8qYJoGq67eH9rdaS0QAKNAqhaZwElRVbITAjQ5Wx wxOz4SsQr1HDGyaYHqjEBu0umW0v2zkLqKJkBxbkVpfo2WUErTUuKN4u1ftSPGo583Em JtKw== X-Forwarded-Encrypted: i=1; AJvYcCXK9Mfzo6TnKF/KwU2JedWAIdDWk2H9G+JHNHSr4GoXgTZa00oVbC0XzLMuxKoHWVzeVp8ckio8g6DWR38=@vger.kernel.org X-Gm-Message-State: AOJu0YwUyW4KX6LDdUS9SOuCKAa/IVUYjn1DYCzK6nqVMh9L20fFg69v BKld6eeqqAWI/j9XoXimokuc/zIvLcC8UNiFLQWWIY7lbKrBDL6ye2iXQ1wE+NuBJ2e03fMu5yF T X-Google-Smtp-Source: AGHT+IFCPoOYnokvNQ/2QoiFXULBQEDHRLHivUxnHE1zewct7LR//wwx5wrmKun/pN8nw2xNKOpV+A== X-Received: by 2002:a17:90a:2f21:b0:2e2:8299:2701 with SMTP id 98e67ed59e1d1-2e3152eb849mr492826a91.20.1728672356576; Fri, 11 Oct 2024 11:45:56 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e30d848e1csm687625a91.42.2024.10.11.11.45.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 11:45:56 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: mkarsten@uwaterloo.ca, skhawaja@google.com, sdf@fomichev.me, bjorn@rivosinc.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, willemdebruijn.kernel@gmail.com, edumazet@google.com, Joe Damato , Jakub Kicinski , "David S. Miller" , Paolo Abeni , Donald Hunter , Jesper Dangaard Brouer , Mina Almasry , Xuan Zhuo , Larysa Zaremba , linux-kernel@vger.kernel.org (open list) Subject: [net-next v6 4/9] netdev-genl: Dump gro_flush_timeout Date: Fri, 11 Oct 2024 18:44:59 +0000 Message-Id: <20241011184527.16393-5-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241011184527.16393-1-jdamato@fastly.com> References: <20241011184527.16393-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Support dumping gro_flush_timeout for a NAPI ID. Signed-off-by: Joe Damato Reviewed-by: Jakub Kicinski Reviewed-by: Eric Dumazet --- Documentation/netlink/specs/netdev.yaml | 9 +++++++++ include/uapi/linux/netdev.h | 1 + net/core/netdev-genl.c | 6 ++++++ tools/include/uapi/linux/netdev.h | 1 + 4 files changed, 17 insertions(+) diff --git a/Documentation/netlink/specs/netdev.yaml b/Documentation/netlin= k/specs/netdev.yaml index 585e87ec3c16..7b47454c51dd 100644 --- a/Documentation/netlink/specs/netdev.yaml +++ b/Documentation/netlink/specs/netdev.yaml @@ -255,6 +255,14 @@ attribute-sets: type: u32 checks: max: s32-max + - + name: gro-flush-timeout + doc: The timeout, in nanoseconds, of when to trigger the NAPI watc= hdog + timer which schedules NAPI processing. Additionally, a non-ze= ro + value will also prevent GRO from flushing recent super-frames= at + the end of a NAPI cycle. This may add receive latency in exch= ange + for reducing the number of frames processed by the network st= ack. + type: uint - name: queue attributes: @@ -644,6 +652,7 @@ operations: - irq - pid - defer-hard-irqs + - gro-flush-timeout dump: request: attributes: diff --git a/include/uapi/linux/netdev.h b/include/uapi/linux/netdev.h index 13dc0b027e86..cacd33359c76 100644 --- a/include/uapi/linux/netdev.h +++ b/include/uapi/linux/netdev.h @@ -123,6 +123,7 @@ enum { NETDEV_A_NAPI_IRQ, NETDEV_A_NAPI_PID, NETDEV_A_NAPI_DEFER_HARD_IRQS, + NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT, =20 __NETDEV_A_NAPI_MAX, NETDEV_A_NAPI_MAX =3D (__NETDEV_A_NAPI_MAX - 1) diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c index f98e5d1d0d21..ac19f2e6cfbe 100644 --- a/net/core/netdev-genl.c +++ b/net/core/netdev-genl.c @@ -161,6 +161,7 @@ static int netdev_nl_napi_fill_one(struct sk_buff *rsp, struct napi_struct *napi, const struct genl_info *info) { + unsigned long gro_flush_timeout; u32 napi_defer_hard_irqs; void *hdr; pid_t pid; @@ -195,6 +196,11 @@ netdev_nl_napi_fill_one(struct sk_buff *rsp, struct na= pi_struct *napi, napi_defer_hard_irqs)) goto nla_put_failure; =20 + gro_flush_timeout =3D napi_get_gro_flush_timeout(napi); + if (nla_put_uint(rsp, NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT, + gro_flush_timeout)) + goto nla_put_failure; + genlmsg_end(rsp, hdr); =20 return 0; diff --git a/tools/include/uapi/linux/netdev.h b/tools/include/uapi/linux/n= etdev.h index 13dc0b027e86..cacd33359c76 100644 --- a/tools/include/uapi/linux/netdev.h +++ b/tools/include/uapi/linux/netdev.h @@ -123,6 +123,7 @@ enum { NETDEV_A_NAPI_IRQ, NETDEV_A_NAPI_PID, NETDEV_A_NAPI_DEFER_HARD_IRQS, + NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT, =20 __NETDEV_A_NAPI_MAX, NETDEV_A_NAPI_MAX =3D (__NETDEV_A_NAPI_MAX - 1) --=20 2.25.1 From nobody Wed Nov 27 06:32:26 2024 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 145A21D131F for ; Fri, 11 Oct 2024 18:46:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672363; cv=none; b=dQMbBMffoEc4C+v19gtorPVyG5ncPFyGMB7l9fMvZh6ZrV8lptyZP6Nta1HOhm/8tuCnjb0hCLH2v3V1njYVgJo5AY6nHjYRRKkuSsne/l/CCgRZIEJ7j10+u+gWpDoBnsrSm3KVpPvLN21CVplHbwjjSmdd9CzdwrNpkNV67jA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672363; c=relaxed/simple; bh=bfDh3MXLufnPabGOrxrrmjox6Eci5oAwJ24pd1w+jdc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eJNybUQSoXB0M2wFboGCWExFy74FkZtc3IS2XN9IXCWXgBZujkocy2gbboso7jckaisqmVdeCkNfP4iHe6rO6QliPMCeYr4XLZT4b3ZwOrAKXp4ITCIz2jcj02bN2K2cat5DDLLxZ4xcWeJmbpny8zhRRKiyYiqNCMKzFqadTm8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=R89x13gR; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="R89x13gR" Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2e2a999b287so1972654a91.0 for ; Fri, 11 Oct 2024 11:46:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1728672361; x=1729277161; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aLUfaAOkJl+Vde+WJrmLFXZ+vNL5nfp8Zqg9U19iPew=; b=R89x13gRd0p82JSg3vfhEqeKAxaOBilF66tHSsTCS/aBXbox4SZ7Z5/0t3vIXZlOHt AQRGIN9Ubof6CsoSDtUyRMMdHWceQOAaRs5wQjW+ioopcTK/sV47PO7nPnWV+65e1moT GPO/OCZldFMyh9ChPy+rdkt+7HtgfNuSwIcuw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728672361; x=1729277161; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aLUfaAOkJl+Vde+WJrmLFXZ+vNL5nfp8Zqg9U19iPew=; b=v2V8JnQO104fuGBwPZEwHoN1Y9zDY+/H680GDJhv4KdH1SwNliJ/5qXQ2AgdSWd7+n cBGb23Qr8XY/BrepQWvByFsburqDUDfOqtR6zA+03QSQzYNXRfOgsF6Qni2ycU1bjZi6 k9LFqMSCkTv5GlT+JhdFILrdOdWcEsX7BqIKADFsJz4o7OvMmQDHvURDiY9Om9GRwaAI itdRlohMcWIvAS60sNx7rZUBkOvCXBXGsPb7nsw0KV7fM7mGDtgqvqquesCJpPcmIows 8g7NfP3dxpGsL8oXF+94L0+6GiB/5vMtYqiTv8tZTWfBMrSB5ewvd6zWp/mk/BuG8eGA R4cQ== X-Forwarded-Encrypted: i=1; AJvYcCXpTn4wi+hjOEMRk3bKZQ7sqWO0R7ky7+EZAmjb8FJT8Qpfu9oAu8n8uXU4//bwUsvapP3xRvy8OpwoSf8=@vger.kernel.org X-Gm-Message-State: AOJu0YwNZEtEFlXZlhYNFLiyeE1BUmiyPUk4HNieXWTW7sOdH7Q3ufL5 PYqN0CviVFAkq5OMA6SuTFwnHKiCXXDMx4bhzbaOFFdiqcpx25wjV9WIp75UF4k= X-Google-Smtp-Source: AGHT+IFz4xe7dCBlsvUD75EtcMO0wHJOv7cpsrcGSU55pc1PZ9LqNUwYvebSdnHdxbEcvS1Kcwo9Cg== X-Received: by 2002:a17:90b:4a0b:b0:2e2:d879:7cfc with SMTP id 98e67ed59e1d1-2e2f0affbffmr4257847a91.21.1728672361222; Fri, 11 Oct 2024 11:46:01 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e30d848e1csm687625a91.42.2024.10.11.11.45.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 11:46:00 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: mkarsten@uwaterloo.ca, skhawaja@google.com, sdf@fomichev.me, bjorn@rivosinc.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, willemdebruijn.kernel@gmail.com, edumazet@google.com, Joe Damato , Jakub Kicinski , "David S. Miller" , Paolo Abeni , Jonathan Corbet , Jiri Pirko , Sebastian Andrzej Siewior , Lorenzo Bianconi , Johannes Berg , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-kernel@vger.kernel.org (open list) Subject: [net-next v6 5/9] net: napi: Add napi_config Date: Fri, 11 Oct 2024 18:45:00 +0000 Message-Id: <20241011184527.16393-6-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241011184527.16393-1-jdamato@fastly.com> References: <20241011184527.16393-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a persistent NAPI config area for NAPI configuration to the core. Drivers opt-in to setting the persistent config for a NAPI by passing an index when calling netif_napi_add_config. napi_config is allocated in alloc_netdev_mqs, freed in free_netdev (after the NAPIs are deleted). Drivers which call netif_napi_add_config will have persistent per-NAPI settings: NAPI IDs, gro_flush_timeout, and defer_hard_irq settings. Per-NAPI settings are saved in napi_disable and restored in napi_enable. Co-developed-by: Martin Karsten Signed-off-by: Martin Karsten Signed-off-by: Joe Damato Reviewed-by: Jakub Kicinski Reviewed-by: Eric Dumazet --- .../networking/net_cachelines/net_device.rst | 1 + include/linux/netdevice.h | 36 ++++++++- net/core/dev.c | 80 +++++++++++++++++-- net/core/dev.h | 12 +++ 4 files changed, 119 insertions(+), 10 deletions(-) diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Docum= entation/networking/net_cachelines/net_device.rst index 67910ea49160..db6192b2bb50 100644 --- a/Documentation/networking/net_cachelines/net_device.rst +++ b/Documentation/networking/net_cachelines/net_device.rst @@ -186,6 +186,7 @@ struct dpll_pin* dpll_pin struct hlist_head page_pools struct dim_irq_moder* irq_moder u64 max_pacing_offload_horizon +struct_napi_config* napi_config unsigned_long gro_flush_timeout u32 napi_defer_hard_irqs =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 93241d4de437..8feaca12655e 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -342,6 +342,15 @@ struct gro_list { */ #define GRO_HASH_BUCKETS 8 =20 +/* + * Structure for per-NAPI config + */ +struct napi_config { + u64 gro_flush_timeout; + u32 defer_hard_irqs; + unsigned int napi_id; +}; + /* * Structure for NAPI scheduling similar to tasklet but with weighting */ @@ -379,6 +388,8 @@ struct napi_struct { struct list_head dev_list; struct hlist_node napi_hash_node; int irq; + int index; + struct napi_config *config; }; =20 enum { @@ -1868,9 +1879,6 @@ enum netdev_reg_state { * allocated at register_netdev() time * @real_num_rx_queues: Number of RX queues currently active in device * @xdp_prog: XDP sockets filter program pointer - * @gro_flush_timeout: timeout for GRO layer in NAPI - * @napi_defer_hard_irqs: If not zero, provides a counter that would - * allow to avoid NIC hard IRQ, on busy queues. * * @rx_handler: handler for received packets * @rx_handler_data: XXX: need comments on this one @@ -2020,6 +2028,11 @@ enum netdev_reg_state { * where the clock is recovered. * * @max_pacing_offload_horizon: max EDT offload horizon in nsec. + * @napi_config: An array of napi_config structures containing per-NAPI + * settings. + * @gro_flush_timeout: timeout for GRO layer in NAPI + * @napi_defer_hard_irqs: If not zero, provides a counter that would + * allow to avoid NIC hard IRQ, on busy queues. * * FIXME: cleanup struct net_device such that network protocol info * moves out. @@ -2413,6 +2426,7 @@ struct net_device { struct dim_irq_moder *irq_moder; =20 u64 max_pacing_offload_horizon; + struct napi_config *napi_config; unsigned long gro_flush_timeout; u32 napi_defer_hard_irqs; =20 @@ -2678,6 +2692,22 @@ netif_napi_add_tx_weight(struct net_device *dev, netif_napi_add_weight(dev, napi, poll, weight); } =20 +/** + * netif_napi_add_config - initialize a NAPI context with persistent config + * @dev: network device + * @napi: NAPI context + * @poll: polling function + * @index: the NAPI index + */ +static inline void +netif_napi_add_config(struct net_device *dev, struct napi_struct *napi, + int (*poll)(struct napi_struct *, int), int index) +{ + napi->index =3D index; + napi->config =3D &dev->napi_config[index]; + netif_napi_add_weight(dev, napi, poll, NAPI_POLL_WEIGHT); +} + /** * netif_napi_add_tx() - initialize a NAPI context to be used for Tx only * @dev: network device diff --git a/net/core/dev.c b/net/core/dev.c index e21ace3551d5..c682173a7642 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6505,6 +6505,23 @@ EXPORT_SYMBOL(napi_busy_loop); =20 #endif /* CONFIG_NET_RX_BUSY_POLL */ =20 +static void __napi_hash_add_with_id(struct napi_struct *napi, + unsigned int napi_id) +{ + napi->napi_id =3D napi_id; + hlist_add_head_rcu(&napi->napi_hash_node, + &napi_hash[napi->napi_id % HASH_SIZE(napi_hash)]); +} + +static void napi_hash_add_with_id(struct napi_struct *napi, + unsigned int napi_id) +{ + spin_lock(&napi_hash_lock); + WARN_ON_ONCE(napi_by_id(napi_id)); + __napi_hash_add_with_id(napi, napi_id); + spin_unlock(&napi_hash_lock); +} + static void napi_hash_add(struct napi_struct *napi) { if (test_bit(NAPI_STATE_NO_BUSY_POLL, &napi->state)) @@ -6517,10 +6534,8 @@ static void napi_hash_add(struct napi_struct *napi) if (unlikely(++napi_gen_id < MIN_NAPI_ID)) napi_gen_id =3D MIN_NAPI_ID; } while (napi_by_id(napi_gen_id)); - napi->napi_id =3D napi_gen_id; =20 - hlist_add_head_rcu(&napi->napi_hash_node, - &napi_hash[napi->napi_id % HASH_SIZE(napi_hash)]); + __napi_hash_add_with_id(napi, napi_gen_id); =20 spin_unlock(&napi_hash_lock); } @@ -6643,6 +6658,28 @@ void netif_queue_set_napi(struct net_device *dev, un= signed int queue_index, } EXPORT_SYMBOL(netif_queue_set_napi); =20 +static void napi_restore_config(struct napi_struct *n) +{ + n->defer_hard_irqs =3D n->config->defer_hard_irqs; + n->gro_flush_timeout =3D n->config->gro_flush_timeout; + /* a NAPI ID might be stored in the config, if so use it. if not, use + * napi_hash_add to generate one for us. It will be saved to the config + * in napi_disable. + */ + if (n->config->napi_id) + napi_hash_add_with_id(n, n->config->napi_id); + else + napi_hash_add(n); +} + +static void napi_save_config(struct napi_struct *n) +{ + n->config->defer_hard_irqs =3D n->defer_hard_irqs; + n->config->gro_flush_timeout =3D n->gro_flush_timeout; + n->config->napi_id =3D n->napi_id; + napi_hash_del(n); +} + void netif_napi_add_weight(struct net_device *dev, struct napi_struct *nap= i, int (*poll)(struct napi_struct *, int), int weight) { @@ -6653,8 +6690,6 @@ void netif_napi_add_weight(struct net_device *dev, st= ruct napi_struct *napi, INIT_HLIST_NODE(&napi->napi_hash_node); hrtimer_init(&napi->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED); napi->timer.function =3D napi_watchdog; - napi_set_defer_hard_irqs(napi, READ_ONCE(dev->napi_defer_hard_irqs)); - napi_set_gro_flush_timeout(napi, READ_ONCE(dev->gro_flush_timeout)); init_gro_hash(napi); napi->skb =3D NULL; INIT_LIST_HEAD(&napi->rx_list); @@ -6672,7 +6707,13 @@ void netif_napi_add_weight(struct net_device *dev, s= truct napi_struct *napi, set_bit(NAPI_STATE_SCHED, &napi->state); set_bit(NAPI_STATE_NPSVC, &napi->state); list_add_rcu(&napi->dev_list, &dev->napi_list); - napi_hash_add(napi); + + /* default settings from sysfs are applied to all NAPIs. any per-NAPI + * configuration will be loaded in napi_enable + */ + napi_set_defer_hard_irqs(napi, READ_ONCE(dev->napi_defer_hard_irqs)); + napi_set_gro_flush_timeout(napi, READ_ONCE(dev->gro_flush_timeout)); + napi_get_frags_check(napi); /* Create kthread for this napi if dev->threaded is set. * Clear dev->threaded if kthread creation failed so that @@ -6704,6 +6745,11 @@ void napi_disable(struct napi_struct *n) =20 hrtimer_cancel(&n->timer); =20 + if (n->config) + napi_save_config(n); + else + napi_hash_del(n); + clear_bit(NAPI_STATE_DISABLE, &n->state); } EXPORT_SYMBOL(napi_disable); @@ -6719,6 +6765,11 @@ void napi_enable(struct napi_struct *n) { unsigned long new, val =3D READ_ONCE(n->state); =20 + if (n->config) + napi_restore_config(n); + else + napi_hash_add(n); + do { BUG_ON(!test_bit(NAPI_STATE_SCHED, &val)); =20 @@ -6748,7 +6799,11 @@ void __netif_napi_del(struct napi_struct *napi) if (!test_and_clear_bit(NAPI_STATE_LISTED, &napi->state)) return; =20 - napi_hash_del(napi); + if (napi->config) { + napi->index =3D -1; + napi->config =3D NULL; + } + list_del_rcu(&napi->dev_list); napi_free_frags(napi); =20 @@ -11085,6 +11140,8 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv= , const char *name, unsigned int txqs, unsigned int rxqs) { struct net_device *dev; + size_t napi_config_sz; + unsigned int maxqs; =20 BUG_ON(strlen(name) >=3D sizeof(dev->name)); =20 @@ -11098,6 +11155,8 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv= , const char *name, return NULL; } =20 + maxqs =3D max(txqs, rxqs); + dev =3D kvzalloc(struct_size(dev, priv, sizeof_priv), GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL); if (!dev) @@ -11174,6 +11233,11 @@ struct net_device *alloc_netdev_mqs(int sizeof_pri= v, const char *name, if (!dev->ethtool) goto free_all; =20 + napi_config_sz =3D array_size(maxqs, sizeof(*dev->napi_config)); + dev->napi_config =3D kvzalloc(napi_config_sz, GFP_KERNEL_ACCOUNT); + if (!dev->napi_config) + goto free_all; + strscpy(dev->name, name); dev->name_assign_type =3D name_assign_type; dev->group =3D INIT_NETDEV_GROUP; @@ -11237,6 +11301,8 @@ void free_netdev(struct net_device *dev) list_for_each_entry_safe(p, n, &dev->napi_list, dev_list) netif_napi_del(p); =20 + kvfree(dev->napi_config); + ref_tracker_dir_exit(&dev->refcnt_tracker); #ifdef CONFIG_PCPU_DEV_REFCNT free_percpu(dev->pcpu_refcnt); diff --git a/net/core/dev.h b/net/core/dev.h index 7d0aab7e3ef1..7881bced70a9 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -177,11 +177,17 @@ static inline void napi_set_defer_hard_irqs(struct na= pi_struct *n, u32 defer) static inline void netdev_set_defer_hard_irqs(struct net_device *netdev, u32 defer) { + unsigned int count =3D max(netdev->num_rx_queues, + netdev->num_tx_queues); struct napi_struct *napi; + int i; =20 WRITE_ONCE(netdev->napi_defer_hard_irqs, defer); list_for_each_entry(napi, &netdev->napi_list, dev_list) napi_set_defer_hard_irqs(napi, defer); + + for (i =3D 0; i < count; i++) + netdev->napi_config[i].defer_hard_irqs =3D defer; } =20 /** @@ -217,11 +223,17 @@ static inline void napi_set_gro_flush_timeout(struct = napi_struct *n, static inline void netdev_set_gro_flush_timeout(struct net_device *netdev, unsigned long timeout) { + unsigned int count =3D max(netdev->num_rx_queues, + netdev->num_tx_queues); struct napi_struct *napi; + int i; =20 WRITE_ONCE(netdev->gro_flush_timeout, timeout); list_for_each_entry(napi, &netdev->napi_list, dev_list) napi_set_gro_flush_timeout(napi, timeout); + + for (i =3D 0; i < count; i++) + netdev->napi_config[i].gro_flush_timeout =3D timeout; } =20 int rps_cpumask_housekeeping(struct cpumask *mask); --=20 2.25.1 From nobody Wed Nov 27 06:32:26 2024 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CCEC1D150C for ; Fri, 11 Oct 2024 18:46:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672369; cv=none; b=OCJ6OKfLPMoy2uCHpYhRfew5nEUXs8OWLZca7CMUTingydQwqByqGFv8KeXYELsmwVPRB4vmosZjNqvyMA+QpuJeneJfgzO5d5nUh8IvWow8+5AqRWWwXMNdhyqmVzmu860Uu+glAxZF+GVfSWbH9M+DU6PcptGna/DM4StrMaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672369; c=relaxed/simple; bh=zvOKoSWpCtfq2DkhujLZNrhtz2v6Zmec/fDGJRPrwF4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=B8Dm4B7WHwVxSz4jxlIDYRhyO8dM12FXWBtP4rBDGjLy7acw+4yHRxLnsRt78OyCLDuk20Qg3nd8RJjgVpc2azqwXZcVl/Gd+CANQgkF8d2efALL03imgJL+lvj+V7a/xEVtydfF+uOxBYHmbYFDnFbI3Wo1u+r+cB9C0WhvPcc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=kEFLNMUP; arc=none smtp.client-ip=209.85.216.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="kEFLNMUP" Received: by mail-pj1-f49.google.com with SMTP id 98e67ed59e1d1-2e18856feb4so2178655a91.3 for ; Fri, 11 Oct 2024 11:46:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1728672368; x=1729277168; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hjPADzwi5V5cwtkOzcpnu/nnt8Tp7+ZZXtsz7wNAOXY=; b=kEFLNMUPCMI+d85la67QUclsgDekU++B+Utmmv/HEpMixlZXKlXFt6nTEkmoFFbu0f HmZ88d6PHPEGUE9XdmyjaOsMTBrFl2Pn+71jlaBaWQJbb/aKr/t7rwuAuxs8rtfgWiLI UdobHG3QPs3UFHU5ms6qXu/WJD/dUYjrgfkGM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728672368; x=1729277168; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hjPADzwi5V5cwtkOzcpnu/nnt8Tp7+ZZXtsz7wNAOXY=; b=bPcBGGKrU4GuL4n+Y2F9j6JkaspfhHLccbIauHgv/EyMed7ZANFK4OsKUqaamGBqjf b6ypue2WKhnupteQVOvwKVZMIjIpMKY8G/P81kkO8z1lq06b5/W/+uJXA32foCioLrH5 EaaTlx1kljbl31kvURbQyg+hhqgnPzqpGGnEGz27YI7fMW7trG3AO5P2169v2F8+hREb NuiGoxSMDE5PrZx+forJZWow+7SjaKsBrgncyTZtvDQHsvJMl/s5u9Re58P4U5QaaaK0 k3ZrTljJm9RjvlHddRcBvF86N0hG/0LXgKpF/m2NW0NB5FPfBibpdmjWW+3KK6oW4srX qqkA== X-Forwarded-Encrypted: i=1; AJvYcCVJqOJU4glB3gY9Xmh0l835+dFGGZFJNvQV5CqrSivLVZq9ui70rmXKo5LEhO0JXWea85XchnbQJ4MomAs=@vger.kernel.org X-Gm-Message-State: AOJu0YxLES89J/KqleoTmasspzfI9zEOREQm14MTxMogzou7cTFVM0ik fOSaEc4lyd9zNchwHFNAAYwMAzXmo7V8I4S18TmMN5cQygg2SNlfJLWvkbaILmE= X-Google-Smtp-Source: AGHT+IFbpC6LdWulF15xz3jNJey+lj89xVnhKvnZb2szYPjzAcm84CGLWMoW7WgyJIzp7pZGdv4myg== X-Received: by 2002:a17:90b:4009:b0:2e2:bd68:b8d8 with SMTP id 98e67ed59e1d1-2e2f0a065b8mr4688076a91.8.1728672367752; Fri, 11 Oct 2024 11:46:07 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e30d848e1csm687625a91.42.2024.10.11.11.46.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 11:46:07 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: mkarsten@uwaterloo.ca, skhawaja@google.com, sdf@fomichev.me, bjorn@rivosinc.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, willemdebruijn.kernel@gmail.com, edumazet@google.com, Joe Damato , Jakub Kicinski , Donald Hunter , "David S. Miller" , Paolo Abeni , Jesper Dangaard Brouer , Mina Almasry , Xuan Zhuo , linux-kernel@vger.kernel.org (open list) Subject: [net-next v6 6/9] netdev-genl: Support setting per-NAPI config values Date: Fri, 11 Oct 2024 18:45:01 +0000 Message-Id: <20241011184527.16393-7-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241011184527.16393-1-jdamato@fastly.com> References: <20241011184527.16393-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add support to set per-NAPI defer_hard_irqs and gro_flush_timeout. Signed-off-by: Joe Damato Reviewed-by: Jakub Kicinski Reviewed-by: Eric Dumazet --- Documentation/netlink/specs/netdev.yaml | 11 ++++++ include/uapi/linux/netdev.h | 1 + net/core/netdev-genl-gen.c | 18 ++++++++++ net/core/netdev-genl-gen.h | 1 + net/core/netdev-genl.c | 45 +++++++++++++++++++++++++ tools/include/uapi/linux/netdev.h | 1 + 6 files changed, 77 insertions(+) diff --git a/Documentation/netlink/specs/netdev.yaml b/Documentation/netlin= k/specs/netdev.yaml index 7b47454c51dd..f9cb97d6106c 100644 --- a/Documentation/netlink/specs/netdev.yaml +++ b/Documentation/netlink/specs/netdev.yaml @@ -693,6 +693,17 @@ operations: reply: attributes: - id + - + name: napi-set + doc: Set configurable NAPI instance settings. + attribute-set: napi + flags: [ admin-perm ] + do: + request: + attributes: + - id + - defer-hard-irqs + - gro-flush-timeout =20 kernel-family: headers: [ "linux/list.h"] diff --git a/include/uapi/linux/netdev.h b/include/uapi/linux/netdev.h index cacd33359c76..e3ebb49f60d2 100644 --- a/include/uapi/linux/netdev.h +++ b/include/uapi/linux/netdev.h @@ -201,6 +201,7 @@ enum { NETDEV_CMD_NAPI_GET, NETDEV_CMD_QSTATS_GET, NETDEV_CMD_BIND_RX, + NETDEV_CMD_NAPI_SET, =20 __NETDEV_CMD_MAX, NETDEV_CMD_MAX =3D (__NETDEV_CMD_MAX - 1) diff --git a/net/core/netdev-genl-gen.c b/net/core/netdev-genl-gen.c index b28424ae06d5..e197bd84997c 100644 --- a/net/core/netdev-genl-gen.c +++ b/net/core/netdev-genl-gen.c @@ -22,6 +22,10 @@ static const struct netlink_range_validation netdev_a_pa= ge_pool_ifindex_range =3D .max =3D 2147483647ULL, }; =20 +static const struct netlink_range_validation netdev_a_napi_defer_hard_irqs= _range =3D { + .max =3D 2147483647ULL, +}; + /* Common nested types */ const struct nla_policy netdev_page_pool_info_nl_policy[NETDEV_A_PAGE_POOL= _IFINDEX + 1] =3D { [NETDEV_A_PAGE_POOL_ID] =3D NLA_POLICY_FULL_RANGE(NLA_UINT, &netdev_a_pag= e_pool_id_range), @@ -87,6 +91,13 @@ static const struct nla_policy netdev_bind_rx_nl_policy[= NETDEV_A_DMABUF_FD + 1] [NETDEV_A_DMABUF_QUEUES] =3D NLA_POLICY_NESTED(netdev_queue_id_nl_policy), }; =20 +/* NETDEV_CMD_NAPI_SET - do */ +static const struct nla_policy netdev_napi_set_nl_policy[NETDEV_A_NAPI_GRO= _FLUSH_TIMEOUT + 1] =3D { + [NETDEV_A_NAPI_ID] =3D { .type =3D NLA_U32, }, + [NETDEV_A_NAPI_DEFER_HARD_IRQS] =3D NLA_POLICY_FULL_RANGE(NLA_U32, &netde= v_a_napi_defer_hard_irqs_range), + [NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT] =3D { .type =3D NLA_UINT, }, +}; + /* Ops table for netdev */ static const struct genl_split_ops netdev_nl_ops[] =3D { { @@ -171,6 +182,13 @@ static const struct genl_split_ops netdev_nl_ops[] =3D= { .maxattr =3D NETDEV_A_DMABUF_FD, .flags =3D GENL_ADMIN_PERM | GENL_CMD_CAP_DO, }, + { + .cmd =3D NETDEV_CMD_NAPI_SET, + .doit =3D netdev_nl_napi_set_doit, + .policy =3D netdev_napi_set_nl_policy, + .maxattr =3D NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT, + .flags =3D GENL_ADMIN_PERM | GENL_CMD_CAP_DO, + }, }; =20 static const struct genl_multicast_group netdev_nl_mcgrps[] =3D { diff --git a/net/core/netdev-genl-gen.h b/net/core/netdev-genl-gen.h index 8cda334fd042..e09dd7539ff2 100644 --- a/net/core/netdev-genl-gen.h +++ b/net/core/netdev-genl-gen.h @@ -33,6 +33,7 @@ int netdev_nl_napi_get_dumpit(struct sk_buff *skb, struct= netlink_callback *cb); int netdev_nl_qstats_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb); int netdev_nl_bind_rx_doit(struct sk_buff *skb, struct genl_info *info); +int netdev_nl_napi_set_doit(struct sk_buff *skb, struct genl_info *info); =20 enum { NETDEV_NLGRP_MGMT, diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c index ac19f2e6cfbe..b49c3b4e5fbe 100644 --- a/net/core/netdev-genl.c +++ b/net/core/netdev-genl.c @@ -303,6 +303,51 @@ int netdev_nl_napi_get_dumpit(struct sk_buff *skb, str= uct netlink_callback *cb) return err; } =20 +static int +netdev_nl_napi_set_config(struct napi_struct *napi, struct genl_info *info) +{ + u64 gro_flush_timeout =3D 0; + u32 defer =3D 0; + + if (info->attrs[NETDEV_A_NAPI_DEFER_HARD_IRQS]) { + defer =3D nla_get_u32(info->attrs[NETDEV_A_NAPI_DEFER_HARD_IRQS]); + napi_set_defer_hard_irqs(napi, defer); + } + + if (info->attrs[NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT]) { + gro_flush_timeout =3D nla_get_uint(info->attrs[NETDEV_A_NAPI_GRO_FLUSH_T= IMEOUT]); + napi_set_gro_flush_timeout(napi, gro_flush_timeout); + } + + return 0; +} + +int netdev_nl_napi_set_doit(struct sk_buff *skb, struct genl_info *info) +{ + struct napi_struct *napi; + unsigned int napi_id; + int err; + + if (GENL_REQ_ATTR_CHECK(info, NETDEV_A_NAPI_ID)) + return -EINVAL; + + napi_id =3D nla_get_u32(info->attrs[NETDEV_A_NAPI_ID]); + + rtnl_lock(); + + napi =3D napi_by_id(napi_id); + if (napi) { + err =3D netdev_nl_napi_set_config(napi, info); + } else { + NL_SET_BAD_ATTR(info->extack, info->attrs[NETDEV_A_NAPI_ID]); + err =3D -ENOENT; + } + + rtnl_unlock(); + + return err; +} + static int netdev_nl_queue_fill_one(struct sk_buff *rsp, struct net_device *netdev, u32 q_idx, u32 q_type, const struct genl_info *info) diff --git a/tools/include/uapi/linux/netdev.h b/tools/include/uapi/linux/n= etdev.h index cacd33359c76..e3ebb49f60d2 100644 --- a/tools/include/uapi/linux/netdev.h +++ b/tools/include/uapi/linux/netdev.h @@ -201,6 +201,7 @@ enum { NETDEV_CMD_NAPI_GET, NETDEV_CMD_QSTATS_GET, NETDEV_CMD_BIND_RX, + NETDEV_CMD_NAPI_SET, =20 __NETDEV_CMD_MAX, NETDEV_CMD_MAX =3D (__NETDEV_CMD_MAX - 1) --=20 2.25.1 From nobody Wed Nov 27 06:32:26 2024 Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFB341CCB3B for ; Fri, 11 Oct 2024 18:46:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672371; cv=none; b=B2gAZFI+VIikW8uTR3PuaO3SBtNH6cVRioAmO+zcLDH9YeZpWezALqnqZR3Ej3hijef0Q71BHf8Vk9hWdkSG88b6gl1d9HhXwe++Iz5NKrERxOscjtj7kUbeofc1sql8gE0mOfk3t2hfr7NgEiyiUZde9eOJ3kgR9mpMN0rHocE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672371; c=relaxed/simple; bh=Ay+s0guDrHAq8MnuDnxM0ZdAsgqJCoes87n91IKUUZY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aGSBKFG55u9KX5jJXGK02l+hxwC32z2E4sreLIhy/1WybHvO6AuEAp5a8XKBIeo5bpuJcinK4BOEhG+ab0shBnxBYkYzCzpx28K9AnY5R9OjUNrMAN380dlGEX2Ge1HCuh5x3Id9Vw+pyEtCa+jQjz56vSdhdvB82A8/N/3Ap+s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=NmSiXLfZ; arc=none smtp.client-ip=209.85.215.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="NmSiXLfZ" Received: by mail-pg1-f179.google.com with SMTP id 41be03b00d2f7-7e9ad969a4fso1691920a12.3 for ; Fri, 11 Oct 2024 11:46:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1728672369; x=1729277169; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dn2sSEQQaNvDLDNmmYALnWN9i1EK+aNS4wMNnm/Kh+Q=; b=NmSiXLfZeNa1txwLYlXh7WYgk7as8D1IKm1A+Yp2I1q1Xc3Y7jGMSfzjKJKiQ0CThg nTPr3/O/ZBLadBtVWq/GB/zDqddRwobx0SfmUP6GBD0KmUiOYJwQbAHp1IKRg/SFF1Bc /nRD2XaCHmzSOPydRtZ/ca/TllAqP8gmMWnCo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728672369; x=1729277169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dn2sSEQQaNvDLDNmmYALnWN9i1EK+aNS4wMNnm/Kh+Q=; b=j3D/oWdAVRRlPw9XUVJDvQsG5xuVselk6K3YhPcn8QnEpDyQm8vKlISXoJZo8OAQIG /JCjfOaAMx6DhkHtg4PGxRz+u90QKTbVb0g6Jy5dXnxMOJwz6PA5ZDoN+j8nVKW/WDwQ AsoMUQG26JVdm5NO2iysfndSar8IIfnCNg5cxFH6K199upDJzyvpGuT2hggGUa9VoR8M x/noc6qvWOreBjBJh6jqA7eWqrRU3KuY4nRIUpyzZU0Y7CPCiQJMrljcstk+hqxqLXwq CfAUypgcZpCrg8W5GRRdXWTnTc1jkb1kUhhM+kbzIXAkosPhFPcjcMHqABbhwIBWh3K0 TdGg== X-Forwarded-Encrypted: i=1; AJvYcCW7MiWowMEF+doZn/rrLZKNO43+GD99Ut7VHHbcuJ/t+UQxzf13KQiLWnf9SbiZ6ZddRQcfsx8uT0m0XoA=@vger.kernel.org X-Gm-Message-State: AOJu0YxqmhwXCDQmqYgtY/7mA4/CV8C6obWbBdw+C9poNgMus0TElJTV 5LnomxkmKcGcfnZ5N2HUQ5Xb8ia31x1/g5OEBOEXM9ecHuAh4ReQgZJU+HCOqXM= X-Google-Smtp-Source: AGHT+IFmnoI7zdMShGQdX0K5KPw1Wu5OvLwhLQCL8LnCffu2Y9WFXd0TLNvEcwdmpwS926sN5ZEeUQ== X-Received: by 2002:a17:90b:2246:b0:2e2:da81:40ca with SMTP id 98e67ed59e1d1-2e2f0a4d957mr4649758a91.2.1728672369296; Fri, 11 Oct 2024 11:46:09 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e30d848e1csm687625a91.42.2024.10.11.11.46.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 11:46:08 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: mkarsten@uwaterloo.ca, skhawaja@google.com, sdf@fomichev.me, bjorn@rivosinc.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, willemdebruijn.kernel@gmail.com, edumazet@google.com, Joe Damato , Jakub Kicinski , Michael Chan , "David S. Miller" , Paolo Abeni , linux-kernel@vger.kernel.org (open list) Subject: [net-next v6 7/9] bnxt: Add support for persistent NAPI config Date: Fri, 11 Oct 2024 18:45:02 +0000 Message-Id: <20241011184527.16393-8-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241011184527.16393-1-jdamato@fastly.com> References: <20241011184527.16393-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use netif_napi_add_config to assign persistent per-NAPI config when initializing NAPIs. Signed-off-by: Joe Damato Reviewed-by: Eric Dumazet Reviewed-by: Jakub Kicinski --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethern= et/broadcom/bnxt/bnxt.c index 6e422e24750a..f5da2dace982 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -10986,7 +10986,8 @@ static void bnxt_init_napi(struct bnxt *bp) cp_nr_rings--; for (i =3D 0; i < cp_nr_rings; i++) { bnapi =3D bp->bnapi[i]; - netif_napi_add(bp->dev, &bnapi->napi, poll_fn); + netif_napi_add_config(bp->dev, &bnapi->napi, poll_fn, + bnapi->index); } if (BNXT_CHIP_TYPE_NITRO_A0(bp)) { bnapi =3D bp->bnapi[cp_nr_rings]; --=20 2.25.1 From nobody Wed Nov 27 06:32:26 2024 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2FF91D1E67 for ; Fri, 11 Oct 2024 18:46:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672374; cv=none; b=IshaRJO0UIBPF+5hiEf0PZbLBC5q/Ab4ZJKNb6ATzfABecTYD2vGwjG4TANwKNrjwvWhPJHysmG3LcxtStl0AHO3dIYW5UOTFxt2FXd5oZ8yiXv4gp0yK3poOFt7jUdEATgi8u4TbWT+Wrao6EpgyX1rFGls8VuuKAlLvg0yng4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672374; c=relaxed/simple; bh=rdUwUl0O8uACIgxUtWrf+uGT//W8R20vX+DxgJVlwC8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=J45dyLwfSFtk1iM/NpPdOwJd5kbsiCFUQXAbawJPZjKKQrO6+Mw5znt8B3xiliBJx9wkryXTHRuRQpJPpmjDqxlwMzqWKwCNkjlbEDSyaVJrjB3B2LJmoYId7UZvwSWyIEtZzoSK8SJLuuCbA0zSb+eVQ/MbkP6nOzE/Lvog+3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=foKSwG9Z; arc=none smtp.client-ip=209.85.216.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="foKSwG9Z" Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-2e2ad9825a7so1707511a91.0 for ; Fri, 11 Oct 2024 11:46:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1728672372; x=1729277172; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SnL/nRVpUbVr0QJP8g9nrYJJ88Tc0QHRfi4pl34q3/U=; b=foKSwG9Z5R+oVSEEsAsDF6CMRqzlGN11KZofIQGq9q3IyINWyrlipI4wHK6oIMhgF/ ggef884BilgLINlz0fAlFJn+wCZ2D0SEZO8sjJy4tYyIRluddptQn04h0v33ephS2m0r H2d6e47LHM7fi1TWDXRVxkD+TtEKhtHfUo23E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728672372; x=1729277172; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SnL/nRVpUbVr0QJP8g9nrYJJ88Tc0QHRfi4pl34q3/U=; b=haYmaSmSyxgvexgdvgFM012/BkkE4QSQKcYI/FOGpcs62gxdFnIgSx5o+C3Q8jfpE2 QjvZerXzq0YcEm0NhOzdEGnQ7PJGL6q8eQZDxEvFyE4DDxpq9Km5HsUDxGPE+Gsrdj0j O7yGRw6Xkxo1JQ7NkyICsAEKCB9VXTO67GroGMR8rBrixcI7SyVm4fWfG/7+vpYKOrlw t5n5g+gcmUs+UdfRUFMLx54iI3GJu+CWXgRnW6oZ7IzEvW3XD67mqgJveYQGwuAugdRm UEh0YAeBW3fx4yexwN7YksaW9Fzz7G5hDuZ9t5fa8GYiZHUCN03YZWt7PKm74093eeKI TbHA== X-Forwarded-Encrypted: i=1; AJvYcCU25Vbst2Ud38qFev9cMScmatmma9IzHDElg8JlPLcwiYbfxwEAIPCFwjV/bUMD2j3xfJjhkw39XjRr4Cc=@vger.kernel.org X-Gm-Message-State: AOJu0Yy6zQsBLyeb3kqgTbMBbuemnRHlA1NjTuGGr7yBaUnBGX6OizwT bGSGeRsOnK/jDPFv0bAZnwIboQwmI2Ww97Ft5DMgfd/l1L+h1Mg3wraeRlUpQhE= X-Google-Smtp-Source: AGHT+IEFsFDVLhRfbEt3rKS6YFF2OdRWJzbou3HLQ3bnnhFginEdMVv+nVTJNPgJ7nFKxYZB38UDGw== X-Received: by 2002:a17:90a:ee8f:b0:2e2:cf5c:8ee3 with SMTP id 98e67ed59e1d1-2e2f0a48877mr5057995a91.10.1728672372104; Fri, 11 Oct 2024 11:46:12 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e30d848e1csm687625a91.42.2024.10.11.11.46.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 11:46:10 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: mkarsten@uwaterloo.ca, skhawaja@google.com, sdf@fomichev.me, bjorn@rivosinc.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, willemdebruijn.kernel@gmail.com, edumazet@google.com, Joe Damato , Jakub Kicinski , Saeed Mahameed , Tariq Toukan , Leon Romanovsky , "David S. Miller" , Paolo Abeni , linux-rdma@vger.kernel.org (open list:MELLANOX MLX5 core VPI driver), linux-kernel@vger.kernel.org (open list) Subject: [net-next v6 8/9] mlx5: Add support for persistent NAPI config Date: Fri, 11 Oct 2024 18:45:03 +0000 Message-Id: <20241011184527.16393-9-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241011184527.16393-1-jdamato@fastly.com> References: <20241011184527.16393-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use netif_napi_add_config to assign persistent per-NAPI config when initializing NAPIs. Signed-off-by: Joe Damato Reviewed-by: Eric Dumazet Reviewed-by: Jakub Kicinski --- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/ne= t/ethernet/mellanox/mlx5/core/en_main.c index a5659c0c4236..09ab7ac07c29 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -2697,7 +2697,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv= , int ix, c->aff_mask =3D irq_get_effective_affinity_mask(irq); c->lag_port =3D mlx5e_enumerate_lag_port(mdev, ix); =20 - netif_napi_add(netdev, &c->napi, mlx5e_napi_poll); + netif_napi_add_config(netdev, &c->napi, mlx5e_napi_poll, ix); netif_napi_set_irq(&c->napi, irq); =20 err =3D mlx5e_open_queues(c, params, cparam); --=20 2.25.1 From nobody Wed Nov 27 06:32:26 2024 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5AE731D1E97 for ; Fri, 11 Oct 2024 18:46:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672375; cv=none; b=QWz12FUMHpfF+g/bE25hEh0e0UsM79uMHTP+HsVAx+ijhEGnv8oP67gy3kto989Nzd/6xXrfFbH3nF3BhszoR0bAYLKsOMBA/I43PtfWnyIBNv0UIdcdgxiFrpnFhC/qm8vcY32+u6G/PxdkLLbbuN/twJbhw49S5Wydl+LJ21U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728672375; c=relaxed/simple; bh=RIFYUWJWd7bN5cBhz+koUNJF3wVi5EIvLaiIKSbtO7Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kQ+Xq/QhhOFs7QJu4PVxm/CssR597s3Plz3/l0aeeWAOyYhMgwhbAzGDXOGHjWNC5wMCiNZFXn+wKGpRri05HtGKBoG82i131bZ4lcRKWdg9337FNjEsbMOWjHT15frdKnpofG5xyYaOriimG88SB4eaxlmGrKYk0E6ITwcrpCs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=yKZY1rdR; arc=none smtp.client-ip=209.85.216.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="yKZY1rdR" Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-2e06acff261so1632923a91.2 for ; Fri, 11 Oct 2024 11:46:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1728672374; x=1729277174; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qutnxq+9ITQDx+pSqPE3IDGupovZNJJRS/Je9OZgZtA=; b=yKZY1rdReVg9EwvUDrb1RBtzzj36Fswmrf8LZvEUd57owNUUHS7k785/TEtPZopCJr lKjFUT1frA/bwPFQ59W9n28WeLpoDNQAVwcuDKQkBWcBrUNYgjs+wIy9OVCzKabxkPGN dhjnS2YLbXw0xaQ40w27kDUj3MWHZKa/QhQW8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728672374; x=1729277174; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qutnxq+9ITQDx+pSqPE3IDGupovZNJJRS/Je9OZgZtA=; b=IwBg31ahzCtyozahM8Sf4K1zrYLTeDh98UPLtSrYsrnPmvXsgiOFyqO94bWnZcVNsf LvoWAi0pJQWVEpLSxWadOxs0CyZpYLk56Yq3S6Ouea4llnpgWWqXXMfmD7a6oDWcWeHC WaVXfa2inlfomCiPoG+gCX1DUzs0H0WWuZYsMdU+ecGLzfu+Pszui+9I94nrlfmvS6ZB ojXVoyRXtdAAbk/AhaVJdWl/d1A3gqpViJa3IiQbIAxKD/0breEJ54tqW/McgTua/TsQ KFXLBTMMq5loH+RuBFuJqHQV2zn4Fc8rhIcSdWbv/JsIwJDBVPIuZuWD+mehfNBIMZlJ BYmg== X-Forwarded-Encrypted: i=1; AJvYcCUd6FTww/0VleL2qepw1zykZHHTYUNZfiI7kNwu7jotctLrAxiTkxugHHXBKZrF5GO+6yfXynzEFRqtdkI=@vger.kernel.org X-Gm-Message-State: AOJu0YwmX+ZnH/7/NqPDZSybU2loWAmGnPCpXa86VyHGJ4RHY7hLpKQW KZCbWyrxwEuqV1rUzKoBXzXvGfwpcpAkS9hHyriE/6eI0NWbCzSKmsPsvKZmTzQ= X-Google-Smtp-Source: AGHT+IGSvRqPkZgaNg6prJZocJDduEg5S2p6+ufKif1tZ3kYV8SUhwDPKkp2QencXRLwI99kDBw/Xw== X-Received: by 2002:a17:90b:3b8c:b0:2e2:ac13:6f7 with SMTP id 98e67ed59e1d1-2e2f0a4d6b8mr5021240a91.4.1728672373888; Fri, 11 Oct 2024 11:46:13 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e30d848e1csm687625a91.42.2024.10.11.11.46.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Oct 2024 11:46:13 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: mkarsten@uwaterloo.ca, skhawaja@google.com, sdf@fomichev.me, bjorn@rivosinc.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, willemdebruijn.kernel@gmail.com, edumazet@google.com, Joe Damato , Jakub Kicinski , Tariq Toukan , "David S. Miller" , Paolo Abeni , linux-rdma@vger.kernel.org (open list:MELLANOX MLX4 core VPI driver), linux-kernel@vger.kernel.org (open list) Subject: [net-next v6 9/9] mlx4: Add support for persistent NAPI config to RX CQs Date: Fri, 11 Oct 2024 18:45:04 +0000 Message-Id: <20241011184527.16393-10-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241011184527.16393-1-jdamato@fastly.com> References: <20241011184527.16393-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use netif_napi_add_config to assign persistent per-NAPI config when initializing RX CQ NAPIs. Presently, struct napi_config only has support for two fields used for RX, so there is no need to support them with TX CQs, yet. Signed-off-by: Joe Damato Reviewed-by: Eric Dumazet Reviewed-by: Jakub Kicinski --- drivers/net/ethernet/mellanox/mlx4/en_cq.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_cq.c b/drivers/net/ether= net/mellanox/mlx4/en_cq.c index 461cc2c79c71..0e92956e84cf 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_cq.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_cq.c @@ -156,7 +156,8 @@ int mlx4_en_activate_cq(struct mlx4_en_priv *priv, stru= ct mlx4_en_cq *cq, break; case RX: cq->mcq.comp =3D mlx4_en_rx_irq; - netif_napi_add(cq->dev, &cq->napi, mlx4_en_poll_rx_cq); + netif_napi_add_config(cq->dev, &cq->napi, mlx4_en_poll_rx_cq, + cq_idx); netif_napi_set_irq(&cq->napi, irq); napi_enable(&cq->napi); netif_queue_set_napi(cq->dev, cq_idx, NETDEV_QUEUE_TYPE_RX, &cq->napi); --=20 2.25.1