From nobody Sat Nov 23 23:00:17 2024 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2CE013E02A for ; Sat, 9 Nov 2024 05:03:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128586; cv=none; b=CZehfOodD8ik/px13X5yo6RcbvwzC2uc/HhS+fCxL8Y29XKeCmQO0dx/kjsJqtuUlK/HrzTGdKPwa5SHC4ABzJMomQVg9xYlQZjeya7PfJ3Mj+J0hEARNThGTOxBv2KW2ZOYXCaG+1B4ndcxru6yCOQRDjKbBu1RIttSnJKxh/Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128586; c=relaxed/simple; bh=vhpZbxXd7C7mjQJupVwVecfi6bQG/CCh2CT3lQxFliI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gFr6F8b/kf8ylnty0RR7aYMqih64unoFfV6NtgwrOUANfDXbYArQoEcw/47f1BF8k518l5cSAfVpfbJWK+DF4sOIV+nbxdN7hxQVEdq8moZzqtwJqknDRgmI22YjUkKVJ3A47k8XwH/fzmV7fKyfZXHjyJnfZVDRik1VrybVSjU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=WI1sWUiN; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="WI1sWUiN" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-20cdda5cfb6so30095695ad.3 for ; Fri, 08 Nov 2024 21:03:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1731128584; x=1731733384; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q8AKrkYZkK/XfTl4qhIfRCQST4FHZONknN91eVwQ+ak=; b=WI1sWUiN2yH0aOZgztdgojR9GUpwIBNXTdbOk4ZM9tiNLL4a1p9dTKtzVWR2g4BBgR 3VGZKdDcZwMhETN6N+nPPGw5eiPvaTwe6DYfbC1Imk7aB9VrZYK1cwJd63WNDIA6tNSp 2PVqEVKwu93dTSTtkXApCAPLQPk+w8Z/Oc/7E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731128584; x=1731733384; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q8AKrkYZkK/XfTl4qhIfRCQST4FHZONknN91eVwQ+ak=; b=V8Ph6e86E/U4pxqBf05DtWMzj+k90HDbt7zF9pQSie6tTSXIo4wto058KT0Wut60Kh j/Hfvoe5jq5zy1cm2R2b3p8QNmj5ksRl9n/5lowe/PJ36Pjn/b1w5ThpnsZJ/M6ifng5 IqmJZJyszssTMOXMR/Nty+6l//EQLa1IKgYxfMQcWRc/NE0fAhXPhGDeP5BuijI+1cpT XNewtoA/4eU2ciG5iZu3eWHvY/clWBd6UeC8BaWYuHU3K6t+TyaPZr5U4qKbjVXK/YMU 1Jx2hPh8SOVe6eVCpXhBvDQStrIZ4ZDUAxDqAcEihD/B5bq08/ADikQooTMLmTqcEgjE yNpA== X-Forwarded-Encrypted: i=1; AJvYcCX9nt2msGLMwr0FaOcZMcxQ9NhB38T+kxZLmAEdM22CTNgGciTEq2ebPhTMg8qnbf9okWWLwvfYVocsioQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yx/HUlpQzZgbGsegvBm3sf5rhfJe/PTXNxEnPOVWQ4tYsZF4A/4 SHyuz7kpcXKmYnBdWFwzJBduOJbFevjghG88U4HnAr7B1YFLYX0y1Yus3l52KPg= X-Google-Smtp-Source: AGHT+IEMq37tVi4c91DfzloTLjCXTo3L29zqRwrccgH8BsbRJ99xG3u5r4ZOjuhT0issZiqTBHnn8g== X-Received: by 2002:a17:902:e88b:b0:20c:5263:247a with SMTP id d9443c01a7336-2118359a478mr66866515ad.38.1731128583817; Fri, 08 Nov 2024 21:03:03 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e5853csm39182305ad.186.2024.11.08.21.03.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 21:03:03 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: corbet@lwn.net, hdanton@sina.com, bagasdotme@gmail.com, pabeni@redhat.com, namangulati@google.com, edumazet@google.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, sdf@fomichev.me, peter@typeblog.net, m2shafiei@uwaterloo.ca, bjorn@rivosinc.com, hch@infradead.org, willy@infradead.org, willemdebruijn.kernel@gmail.com, skhawaja@google.com, kuba@kernel.org, Martin Karsten , Joe Damato , "David S. Miller" , Simon Horman , Donald Hunter , Andrew Lunn , Jesper Dangaard Brouer , Mina Almasry , Xuan Zhuo , David Ahern , Sebastian Andrzej Siewior , Lorenzo Bianconi , Alexander Lobakin , Jiri Pirko , Johannes Berg , linux-kernel@vger.kernel.org (open list) Subject: [PATCH net-next v9 1/6] net: Add napi_struct parameter irq_suspend_timeout Date: Sat, 9 Nov 2024 05:02:31 +0000 Message-Id: <20241109050245.191288-2-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241109050245.191288-1-jdamato@fastly.com> References: <20241109050245.191288-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Martin Karsten Add a per-NAPI IRQ suspension parameter, which can be get/set with netdev-genl. This patch doesn't change any behavior but prepares the code for other changes in the following commits which use irq_suspend_timeout as a timeout for IRQ suspension. Signed-off-by: Martin Karsten Co-developed-by: Joe Damato Signed-off-by: Joe Damato Tested-by: Joe Damato Tested-by: Martin Karsten Acked-by: Stanislav Fomichev Reviewed-by: Sridhar Samudrala --- v1 -> v2: - rewrote this patch to make irq_suspend_timeout per-napi via netdev-genl. rfc -> v1: - removed napi.rst documentation from this patch; added to patch 6. Documentation/netlink/specs/netdev.yaml | 7 +++++++ include/linux/netdevice.h | 2 ++ include/uapi/linux/netdev.h | 1 + net/core/dev.c | 2 ++ net/core/dev.h | 25 +++++++++++++++++++++++++ net/core/netdev-genl-gen.c | 5 +++-- net/core/netdev-genl.c | 12 ++++++++++++ tools/include/uapi/linux/netdev.h | 1 + 8 files changed, 53 insertions(+), 2 deletions(-) diff --git a/Documentation/netlink/specs/netdev.yaml b/Documentation/netlin= k/specs/netdev.yaml index f9cb97d6106c..cbb544bd6c84 100644 --- a/Documentation/netlink/specs/netdev.yaml +++ b/Documentation/netlink/specs/netdev.yaml @@ -263,6 +263,11 @@ attribute-sets: the end of a NAPI cycle. This may add receive latency in exch= ange for reducing the number of frames processed by the network st= ack. type: uint + - + name: irq-suspend-timeout + doc: The timeout, in nanoseconds, of how long to suspend irq + processing, if event polling finds events + type: uint - name: queue attributes: @@ -653,6 +658,7 @@ operations: - pid - defer-hard-irqs - gro-flush-timeout + - irq-suspend-timeout dump: request: attributes: @@ -704,6 +710,7 @@ operations: - id - defer-hard-irqs - gro-flush-timeout + - irq-suspend-timeout =20 kernel-family: headers: [ "linux/list.h"] diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 3c552b648b27..c8ab5f08092b 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -347,6 +347,7 @@ struct gro_list { */ struct napi_config { u64 gro_flush_timeout; + u64 irq_suspend_timeout; u32 defer_hard_irqs; unsigned int napi_id; }; @@ -383,6 +384,7 @@ struct napi_struct { struct hrtimer timer; struct task_struct *thread; unsigned long gro_flush_timeout; + unsigned long irq_suspend_timeout; u32 defer_hard_irqs; /* control-path-only fields follow */ struct list_head dev_list; diff --git a/include/uapi/linux/netdev.h b/include/uapi/linux/netdev.h index e3ebb49f60d2..e4be227d3ad6 100644 --- a/include/uapi/linux/netdev.h +++ b/include/uapi/linux/netdev.h @@ -124,6 +124,7 @@ enum { NETDEV_A_NAPI_PID, NETDEV_A_NAPI_DEFER_HARD_IRQS, NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT, + NETDEV_A_NAPI_IRQ_SUSPEND_TIMEOUT, =20 __NETDEV_A_NAPI_MAX, NETDEV_A_NAPI_MAX =3D (__NETDEV_A_NAPI_MAX - 1) diff --git a/net/core/dev.c b/net/core/dev.c index 6a31152e4606..4d910872963f 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6666,6 +6666,7 @@ static void napi_restore_config(struct napi_struct *n) { n->defer_hard_irqs =3D n->config->defer_hard_irqs; n->gro_flush_timeout =3D n->config->gro_flush_timeout; + n->irq_suspend_timeout =3D n->config->irq_suspend_timeout; /* a NAPI ID might be stored in the config, if so use it. if not, use * napi_hash_add to generate one for us. It will be saved to the config * in napi_disable. @@ -6680,6 +6681,7 @@ static void napi_save_config(struct napi_struct *n) { n->config->defer_hard_irqs =3D n->defer_hard_irqs; n->config->gro_flush_timeout =3D n->gro_flush_timeout; + n->config->irq_suspend_timeout =3D n->irq_suspend_timeout; n->config->napi_id =3D n->napi_id; napi_hash_del(n); } diff --git a/net/core/dev.h b/net/core/dev.h index 7881bced70a9..d043dee25a68 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -236,6 +236,31 @@ static inline void netdev_set_gro_flush_timeout(struct= net_device *netdev, netdev->napi_config[i].gro_flush_timeout =3D timeout; } =20 +/** + * napi_get_irq_suspend_timeout - get the irq_suspend_timeout + * @n: napi struct to get the irq_suspend_timeout from + * + * Return: the per-NAPI value of the irq_suspend_timeout field. + */ +static inline unsigned long +napi_get_irq_suspend_timeout(const struct napi_struct *n) +{ + return READ_ONCE(n->irq_suspend_timeout); +} + +/** + * napi_set_irq_suspend_timeout - set the irq_suspend_timeout for a napi + * @n: napi struct to set the irq_suspend_timeout + * @timeout: timeout value to set + * + * napi_set_irq_suspend_timeout sets the per-NAPI irq_suspend_timeout + */ +static inline void napi_set_irq_suspend_timeout(struct napi_struct *n, + unsigned long timeout) +{ + WRITE_ONCE(n->irq_suspend_timeout, timeout); +} + int rps_cpumask_housekeeping(struct cpumask *mask); =20 #if defined(CONFIG_DEBUG_NET) && defined(CONFIG_BPF_SYSCALL) diff --git a/net/core/netdev-genl-gen.c b/net/core/netdev-genl-gen.c index 21de7e10be16..a89cbd8d87c3 100644 --- a/net/core/netdev-genl-gen.c +++ b/net/core/netdev-genl-gen.c @@ -92,10 +92,11 @@ static const struct nla_policy netdev_bind_rx_nl_policy= [NETDEV_A_DMABUF_FD + 1] }; =20 /* NETDEV_CMD_NAPI_SET - do */ -static const struct nla_policy netdev_napi_set_nl_policy[NETDEV_A_NAPI_GRO= _FLUSH_TIMEOUT + 1] =3D { +static const struct nla_policy netdev_napi_set_nl_policy[NETDEV_A_NAPI_IRQ= _SUSPEND_TIMEOUT + 1] =3D { [NETDEV_A_NAPI_ID] =3D { .type =3D NLA_U32, }, [NETDEV_A_NAPI_DEFER_HARD_IRQS] =3D NLA_POLICY_FULL_RANGE(NLA_U32, &netde= v_a_napi_defer_hard_irqs_range), [NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT] =3D { .type =3D NLA_UINT, }, + [NETDEV_A_NAPI_IRQ_SUSPEND_TIMEOUT] =3D { .type =3D NLA_UINT, }, }; =20 /* Ops table for netdev */ @@ -186,7 +187,7 @@ static const struct genl_split_ops netdev_nl_ops[] =3D { .cmd =3D NETDEV_CMD_NAPI_SET, .doit =3D netdev_nl_napi_set_doit, .policy =3D netdev_napi_set_nl_policy, - .maxattr =3D NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT, + .maxattr =3D NETDEV_A_NAPI_IRQ_SUSPEND_TIMEOUT, .flags =3D GENL_ADMIN_PERM | GENL_CMD_CAP_DO, }, }; diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c index b49c3b4e5fbe..765ce7c9d73b 100644 --- a/net/core/netdev-genl.c +++ b/net/core/netdev-genl.c @@ -161,6 +161,7 @@ static int netdev_nl_napi_fill_one(struct sk_buff *rsp, struct napi_struct *napi, const struct genl_info *info) { + unsigned long irq_suspend_timeout; unsigned long gro_flush_timeout; u32 napi_defer_hard_irqs; void *hdr; @@ -196,6 +197,11 @@ netdev_nl_napi_fill_one(struct sk_buff *rsp, struct na= pi_struct *napi, napi_defer_hard_irqs)) goto nla_put_failure; =20 + irq_suspend_timeout =3D napi_get_irq_suspend_timeout(napi); + if (nla_put_uint(rsp, NETDEV_A_NAPI_IRQ_SUSPEND_TIMEOUT, + irq_suspend_timeout)) + goto nla_put_failure; + gro_flush_timeout =3D napi_get_gro_flush_timeout(napi); if (nla_put_uint(rsp, NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT, gro_flush_timeout)) @@ -306,6 +312,7 @@ int netdev_nl_napi_get_dumpit(struct sk_buff *skb, stru= ct netlink_callback *cb) static int netdev_nl_napi_set_config(struct napi_struct *napi, struct genl_info *info) { + u64 irq_suspend_timeout =3D 0; u64 gro_flush_timeout =3D 0; u32 defer =3D 0; =20 @@ -314,6 +321,11 @@ netdev_nl_napi_set_config(struct napi_struct *napi, st= ruct genl_info *info) napi_set_defer_hard_irqs(napi, defer); } =20 + if (info->attrs[NETDEV_A_NAPI_IRQ_SUSPEND_TIMEOUT]) { + irq_suspend_timeout =3D nla_get_uint(info->attrs[NETDEV_A_NAPI_IRQ_SUSPE= ND_TIMEOUT]); + napi_set_irq_suspend_timeout(napi, irq_suspend_timeout); + } + if (info->attrs[NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT]) { gro_flush_timeout =3D nla_get_uint(info->attrs[NETDEV_A_NAPI_GRO_FLUSH_T= IMEOUT]); napi_set_gro_flush_timeout(napi, gro_flush_timeout); diff --git a/tools/include/uapi/linux/netdev.h b/tools/include/uapi/linux/n= etdev.h index e3ebb49f60d2..e4be227d3ad6 100644 --- a/tools/include/uapi/linux/netdev.h +++ b/tools/include/uapi/linux/netdev.h @@ -124,6 +124,7 @@ enum { NETDEV_A_NAPI_PID, NETDEV_A_NAPI_DEFER_HARD_IRQS, NETDEV_A_NAPI_GRO_FLUSH_TIMEOUT, + NETDEV_A_NAPI_IRQ_SUSPEND_TIMEOUT, =20 __NETDEV_A_NAPI_MAX, NETDEV_A_NAPI_MAX =3D (__NETDEV_A_NAPI_MAX - 1) --=20 2.25.1 From nobody Sat Nov 23 23:00:17 2024 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24FBE13E03E for ; Sat, 9 Nov 2024 05:03:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128589; cv=none; b=u6hvAms/1FuE2NLhBSr4lB8aO3Q6jf8zmQrVqHshywAEnFGyb+RmS8Iae1JCT0Rjz8E2+/wh1BTfI7+obhWRnPgsc0zrI16bt8TYaOWRbEoSRl9/4ZBISx5A/m1wPq2UpiCcCnIl2Fw9AVL97urOMK4sCR+uGb7D3cXlbbfWSAo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128589; c=relaxed/simple; bh=7YkKWB/VoyUXO1lGGm/ESxIvFChf/k/+Rh3/h3kzr9I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Jqq71nN8XKhCiD0EEZ0gU2VB2vYMQ/y6YdhiGaYmHSU8Mwm9qOQOczuZ8H3lcFFKbgZ3RukJn0nkvhYQ2tTyaiLUGofhKB+eUK5TiWnaWizNyzeyMuCtnMR5BR0V5+asVDIZZoPttoBSqIfGKK1JkSdYCOQqY8/UiWf3zLLpq+U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=t/evmyu8; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="t/evmyu8" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-20cdb889222so31088555ad.3 for ; Fri, 08 Nov 2024 21:03:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1731128587; x=1731733387; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7HzDb2SxhUo/TAU2Mmq5phfYmvXMmtRzJHslebclKIo=; b=t/evmyu8G4ny8Jj6n3S6uWHTe17BFJZAhatSr6DDRc8Ln8hL4O8YtaDMM8ulvpLiN5 q5oo7IOkdsp1eakLyNNeQyfN/M7HuPlGdM4/fjkCWddtoz9DDfUUeztN+pvZrwGu8yBE MSffHIoLcc9KMa/BQ44jQLGKXwJpciRdqdGDU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731128587; x=1731733387; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7HzDb2SxhUo/TAU2Mmq5phfYmvXMmtRzJHslebclKIo=; b=IIn9H4P3gReH0v1dkQ6HRp51Cw4OgCTr5tGYdiRmfWcQywtK/t1fTGXayBqr4ZBr2f bBx0TFvB5ZlYGEgcWaCfjWLvb9wWQFO3u27/SNGyFQkZAyWGPEMehOGM9Oz4BxEozKH3 R8kHFT8KquEFo6334Mhq8Rb/ztelH7b7cxr71KxajWrI2OX1EDPv6vsqyEjG8GaFWtMK WHT4FXBORvDXOxWCiRrRRfWp6gcr6B6pLwvZYiuAcxUk2FL6rthybM8vlsZXO75Fbukm DCZlKvOLCN2YMXs2VbwfP5hK/734w9v7hz5WljvIR1c+ZVoCZ8CrLt1Cxj+n5cCk6vo5 3bAA== X-Forwarded-Encrypted: i=1; AJvYcCUEVAghGz/yjeB37rVwjA8gZoxImEDpL3Po2gZ1DRi7jtNz7RZdrZNoEKItuSeF9jKy8FidX0g2PZ/ZiLg=@vger.kernel.org X-Gm-Message-State: AOJu0Yye9v9j4NBbQovzoZuhWWINeO65MS8tMRZpxutlgf14XzeJQL6z YQ6l0G4LbDNnJt200h9meEiZ3tV1wLVSLVsdBcB8I0MK/jid1fRzXfutXKOnQm8= X-Google-Smtp-Source: AGHT+IHzB6H5fYYWLx9Q/0EgCmqqmoSLpb0RpBfHo0MP/hNXaD4wRBqJGO7LLl2PREd6yNCUXUCBSQ== X-Received: by 2002:a17:903:234c:b0:20c:8907:90a with SMTP id d9443c01a7336-211834f3291mr77516525ad.5.1731128587405; Fri, 08 Nov 2024 21:03:07 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e5853csm39182305ad.186.2024.11.08.21.03.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 21:03:06 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: corbet@lwn.net, hdanton@sina.com, bagasdotme@gmail.com, pabeni@redhat.com, namangulati@google.com, edumazet@google.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, sdf@fomichev.me, peter@typeblog.net, m2shafiei@uwaterloo.ca, bjorn@rivosinc.com, hch@infradead.org, willy@infradead.org, willemdebruijn.kernel@gmail.com, skhawaja@google.com, kuba@kernel.org, Martin Karsten , Joe Damato , "David S. Miller" , Simon Horman , David Ahern , Sebastian Andrzej Siewior , Lorenzo Bianconi , Alexander Lobakin , linux-kernel@vger.kernel.org (open list) Subject: [PATCH net-next v9 2/6] net: Add control functions for irq suspension Date: Sat, 9 Nov 2024 05:02:32 +0000 Message-Id: <20241109050245.191288-3-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241109050245.191288-1-jdamato@fastly.com> References: <20241109050245.191288-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Martin Karsten The napi_suspend_irqs routine bootstraps irq suspension by elongating the defer timeout to irq_suspend_timeout. The napi_resume_irqs routine effectively cancels irq suspension by forcing the napi to be scheduled immediately. Signed-off-by: Martin Karsten Co-developed-by: Joe Damato Signed-off-by: Joe Damato Tested-by: Joe Damato Tested-by: Martin Karsten Acked-by: Stanislav Fomichev Reviewed-by: Sridhar Samudrala --- v8: - Drop unneeded exports as pointed out by Jakub. v7: - This is now patch 2 instead of patch 3; patch 2 from v6 was dropped. v1 -> v2: - Added a comment to napi_resume_irqs. include/net/busy_poll.h | 3 +++ net/core/dev.c | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/include/net/busy_poll.h b/include/net/busy_poll.h index f03040baaefd..c858270141bc 100644 --- a/include/net/busy_poll.h +++ b/include/net/busy_poll.h @@ -52,6 +52,9 @@ void napi_busy_loop_rcu(unsigned int napi_id, bool (*loop_end)(void *, unsigned long), void *loop_end_arg, bool prefer_busy_poll, u16 budget); =20 +void napi_suspend_irqs(unsigned int napi_id); +void napi_resume_irqs(unsigned int napi_id); + #else /* CONFIG_NET_RX_BUSY_POLL */ static inline unsigned long net_busy_loop_on(void) { diff --git a/net/core/dev.c b/net/core/dev.c index 4d910872963f..13d00fc10f55 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6507,6 +6507,43 @@ void napi_busy_loop(unsigned int napi_id, } EXPORT_SYMBOL(napi_busy_loop); =20 +void napi_suspend_irqs(unsigned int napi_id) +{ + struct napi_struct *napi; + + rcu_read_lock(); + napi =3D napi_by_id(napi_id); + if (napi) { + unsigned long timeout =3D napi_get_irq_suspend_timeout(napi); + + if (timeout) + hrtimer_start(&napi->timer, ns_to_ktime(timeout), + HRTIMER_MODE_REL_PINNED); + } + rcu_read_unlock(); +} + +void napi_resume_irqs(unsigned int napi_id) +{ + struct napi_struct *napi; + + rcu_read_lock(); + napi =3D napi_by_id(napi_id); + if (napi) { + /* If irq_suspend_timeout is set to 0 between the call to + * napi_suspend_irqs and now, the original value still + * determines the safety timeout as intended and napi_watchdog + * will resume irq processing. + */ + if (napi_get_irq_suspend_timeout(napi)) { + local_bh_disable(); + napi_schedule(napi); + local_bh_enable(); + } + } + rcu_read_unlock(); +} + #endif /* CONFIG_NET_RX_BUSY_POLL */ =20 static void __napi_hash_add_with_id(struct napi_struct *napi, --=20 2.25.1 From nobody Sat Nov 23 23:00:17 2024 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29E6B145A16 for ; Sat, 9 Nov 2024 05:03:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128591; cv=none; b=S58xUvZOgc8xuUScUfy44hPSMmqiQya2kcMyJ4VkvcCFz3eEyUQGqOI26QrbhSaXL1/P0aiQzsgLkpq/gR33IhDBr4dPXxQTgWIs7dYUoj5ySg8MZKLU/YCSp/yS+iSR/fTFHSVOCzLL/xNX/BRCySqHkN2m4LaiBJGA84j+xBc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128591; c=relaxed/simple; bh=QMGh9V+fCN/j+D0uLjcnIHw3s7shm5dxlUbsYTeYPOY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Xor7m7QhAgbXmZprTKbcZPPNnoCRpF1APjmE2lscB0mr1A/r89vK0EdSvJmmhDvXZ90rNt/om9w+TKcBag+I5lv3YvHD4QeX7/4W3JYe/knH90R0QOE/lCVcym2TlrrwL8Ec6YCELgGlosfJL9sO97wCIKPzTfjFLlvnSOae1N4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=IFnhriQn; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="IFnhriQn" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-20cb7139d9dso27821525ad.1 for ; Fri, 08 Nov 2024 21:03:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1731128590; x=1731733390; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OCBpeWwUMYPpQhLDdrb3cMWwRM83RuSfCI6YlvdtYdM=; b=IFnhriQnBCTn70E//Jl2nn7JfCbUV9ayUVk6rayOUdaQstJCVSb2cqiOAMssJK2NPa DCAMemM5+bnH0DPL0jtuZuuqiMjFrHQM2ZnruRJB8ow80ExTJOp9Vgkdd/HEjB7MTGHr adYFlcPQZBNC2QOXdKNB9d4Y8GKI4D6CYaaAI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731128590; x=1731733390; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OCBpeWwUMYPpQhLDdrb3cMWwRM83RuSfCI6YlvdtYdM=; b=OsEU4g2JAxUWMNirp2qwu0DVflbOaHjE3JKO0hHWVC9GxVc6fYWpAv7DC5r9qFwRxa KoDkKhsA/svXZ1+eC/f2SoNZJykOT1M9g90akSjqhIKZQaPXdrmRXsXMllY7+Ui5wR3y Ms8rZjiUtEEPipEgI0D/rn2c9SA9MZ6PDa2nzX9pv10HIZFvTZt9ZQXJoJ0xaOWgp4n4 8N0KknC4MGvxC9TyGbAiBgtTeW/n2tt2b9Ex0OJqBvvn2WFUY3kx5fD52SMaupQ5anW7 YGAALpazzIuJRIwYx3Fh4OQKOdq2ppUykpP8kqKsQ4gr4XAveZorkaGAFW2BTYR7/Q36 Dm5w== X-Forwarded-Encrypted: i=1; AJvYcCUMLe3jnuQFzFcnFiUojnqhJwAIu7AodYwe/KhU0l/6B1Q7CCm99dADSFFjRDtJfNB4n7QDvGNSAe6HIjQ=@vger.kernel.org X-Gm-Message-State: AOJu0YwDqVnIGtMaqqcgGvfdd4io4TcpMElJcXbYwJeWbxmvZCYJ8NCQ L/YDJ8NuJKMdpFPh9/S3mzRnHFHOjrdVdydecrHodK88VBf05jK9200s0Ps94Nw= X-Google-Smtp-Source: AGHT+IGzmRuVCERnRF29spRVxccerJag0s3xixY3WGrpRZL96j0U/eDHAABtUMvfvWyq5b9qA5cGxA== X-Received: by 2002:a17:903:41c3:b0:20c:e65c:8c6c with SMTP id d9443c01a7336-21183d34368mr57616805ad.19.1731128589606; Fri, 08 Nov 2024 21:03:09 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e5853csm39182305ad.186.2024.11.08.21.03.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 21:03:09 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: corbet@lwn.net, hdanton@sina.com, bagasdotme@gmail.com, pabeni@redhat.com, namangulati@google.com, edumazet@google.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, sdf@fomichev.me, peter@typeblog.net, m2shafiei@uwaterloo.ca, bjorn@rivosinc.com, hch@infradead.org, willy@infradead.org, willemdebruijn.kernel@gmail.com, skhawaja@google.com, kuba@kernel.org, Martin Karsten , Joe Damato , Alexander Viro , Christian Brauner , Jan Kara , linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH net-next v9 3/6] eventpoll: Trigger napi_busy_loop, if prefer_busy_poll is set Date: Sat, 9 Nov 2024 05:02:33 +0000 Message-Id: <20241109050245.191288-4-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241109050245.191288-1-jdamato@fastly.com> References: <20241109050245.191288-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Martin Karsten Setting prefer_busy_poll now leads to an effectively nonblocking iteration though napi_busy_loop, even when busy_poll_usecs is 0. Signed-off-by: Martin Karsten Co-developed-by: Joe Damato Signed-off-by: Joe Damato Tested-by: Joe Damato Tested-by: Martin Karsten Acked-by: Stanislav Fomichev Reviewed-by: Sridhar Samudrala --- v1 -> v2: - Rebased to apply now that commit b9ca079dd6b0 ("eventpoll: Annotate data-race of busy_poll_usecs") has been picked up from VFS. fs/eventpoll.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 1ae4542f0bd8..f9e0d9307dad 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -420,7 +420,9 @@ static bool busy_loop_ep_timeout(unsigned long start_ti= me, =20 static bool ep_busy_loop_on(struct eventpoll *ep) { - return !!READ_ONCE(ep->busy_poll_usecs) || net_busy_loop_on(); + return !!READ_ONCE(ep->busy_poll_usecs) || + READ_ONCE(ep->prefer_busy_poll) || + net_busy_loop_on(); } =20 static bool ep_busy_loop_end(void *p, unsigned long start_time) --=20 2.25.1 From nobody Sat Nov 23 23:00:17 2024 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 989451474CF for ; Sat, 9 Nov 2024 05:03:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128594; cv=none; b=PlQppRpqd/+cAxzto0CS647Xsg+hjV46pN0Ivt2hFNdzdLOlPdEvrZC9CpSAXDwgEtAjbSJxjBxvAN3OiYzH17lqilch3w0gClrMp8SYOb0WqBfXG5uYuyRl1HA9nceqlJKJDC1Z5njNRmr4cvMSjm7d1MRRYwwBNWX9mca7VUY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128594; c=relaxed/simple; bh=ZvawQy978AWawDyo2L5Npe51s7GUMbBNxiumv6HwfAk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=drB5F4am1wq4VJtecD66H4EZNYCP/Jsf4OM3cTcnEfD2uAipY57Ubc4jBaN97UB4GaNOvOToDj76tF5/4d8yzncXxO10UH7vuKiGRGqSdYdntDPwCcOvfikJSRdGkwRVq+ELLe6v6+uS2yNGlTEid1nwdVpfVC1ogYWNmtEz270= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=Uw7saS7t; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="Uw7saS7t" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-20c803787abso24473225ad.0 for ; Fri, 08 Nov 2024 21:03:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1731128592; x=1731733392; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U/mZBN4ImS6YyPeWvLF6rIBEjo5ZB0WxVpPyUomkuEI=; b=Uw7saS7t6nDjDRO/ouEsGgh12nKYkz2eVEu3kVJrFEsTjF7VKN2+cEJrOn6gH+G2VZ SehZ6JocVSOLlrEjgNFlPfvf4YEau2R9RYxBiOz+Du2bs5XmCG/ah7r19A4DQB9kh/0T 1HR7ZOWtetrC9HUGbXfaGU79f1lA8QdaA1iSc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731128592; x=1731733392; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U/mZBN4ImS6YyPeWvLF6rIBEjo5ZB0WxVpPyUomkuEI=; b=CGHU82f2T+6awNZaWBzXteWV5pU749K8zAxcD2iSmGCfsh7sSeT1RdPOkHu68ZbbOQ /I54eNmqF0c8e8fUKutgI8Q9X3/c/MrUNp3hEwo/+awvnh8Ih7nmnGIUrNeD6LKVmJGQ YVBoBG2h0NrfBcvxss/mYIP38RNNXw2WvJAbEgxsZjPT78svfFsMyK3gG1mulV4g+IWk EAKhXfTts17g5vbIjvvpXh+Q81FbqbtzGEn/I6vZdhixUZqtFiut9VMDkDmQAIQaAhfg +wWw9TFo5sTuouX/PDbCUNlNeokRu4WvtLJk+gydCfIU1lWeBW3MdjvcR663Y8Vq0mrz xRxA== X-Forwarded-Encrypted: i=1; AJvYcCXt3aVI5Kv945+Z2BLw0eVJUAL66AiY18nsv8spZMI9qhhUyM2TBB2jPPscWZ4r35sIleI7mFCBuOBQB0I=@vger.kernel.org X-Gm-Message-State: AOJu0YwGvcNWIItzaBholI/LvH1xGA9C9IC9Aq3PwQDInChoPvUVmrpi c7V1qf+8ElBv8yW9lGF6Pves3eDSRLKIWTTZIDwu4f02Gjj65vMIDO87CYCGYyM= X-Google-Smtp-Source: AGHT+IFwub+z7xSI5MikFyIEn8D0noXN6MBqpDdrXm/q6wfIDmeAVnqBQrYEukM/mMoyt3Kx3OMjPA== X-Received: by 2002:a17:902:d4c9:b0:20c:c482:1d72 with SMTP id d9443c01a7336-2118377ebd2mr82909855ad.20.1731128591788; Fri, 08 Nov 2024 21:03:11 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e5853csm39182305ad.186.2024.11.08.21.03.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 21:03:11 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: corbet@lwn.net, hdanton@sina.com, bagasdotme@gmail.com, pabeni@redhat.com, namangulati@google.com, edumazet@google.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, sdf@fomichev.me, peter@typeblog.net, m2shafiei@uwaterloo.ca, bjorn@rivosinc.com, hch@infradead.org, willy@infradead.org, willemdebruijn.kernel@gmail.com, skhawaja@google.com, kuba@kernel.org, Martin Karsten , Joe Damato , Alexander Viro , Christian Brauner , Jan Kara , linux-fsdevel@vger.kernel.org (open list:FILESYSTEMS (VFS and infrastructure)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH net-next v9 4/6] eventpoll: Control irq suspension for prefer_busy_poll Date: Sat, 9 Nov 2024 05:02:34 +0000 Message-Id: <20241109050245.191288-5-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241109050245.191288-1-jdamato@fastly.com> References: <20241109050245.191288-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Martin Karsten When events are reported to userland and prefer_busy_poll is set, irqs are temporarily suspended using napi_suspend_irqs. If no events are found and ep_poll would go to sleep, irq suspension is cancelled using napi_resume_irqs. Signed-off-by: Martin Karsten Co-developed-by: Joe Damato Signed-off-by: Joe Damato Tested-by: Joe Damato Tested-by: Martin Karsten Acked-by: Stanislav Fomichev Reviewed-by: Sridhar Samudrala --- v5: - Only call ep_suspend_napi_irqs when ep_send_events returns a positive value. IRQs are not suspended in error (e.g. EINTR) cases. This issue was pointed out by Hillf Danton. rfc -> v1: - move irq resume code from ep_free to a helper which either resumes IRQs or does nothing if !defined(CONFIG_NET_RX_BUSY_POLL). fs/eventpoll.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index f9e0d9307dad..83bcb559b89f 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -457,6 +457,8 @@ static bool ep_busy_loop(struct eventpoll *ep, int nonb= lock) * it back in when we have moved a socket with a valid NAPI * ID onto the ready list. */ + if (prefer_busy_poll) + napi_resume_irqs(napi_id); ep->napi_id =3D 0; return false; } @@ -540,6 +542,22 @@ static long ep_eventpoll_bp_ioctl(struct file *file, u= nsigned int cmd, } } =20 +static void ep_suspend_napi_irqs(struct eventpoll *ep) +{ + unsigned int napi_id =3D READ_ONCE(ep->napi_id); + + if (napi_id >=3D MIN_NAPI_ID && READ_ONCE(ep->prefer_busy_poll)) + napi_suspend_irqs(napi_id); +} + +static void ep_resume_napi_irqs(struct eventpoll *ep) +{ + unsigned int napi_id =3D READ_ONCE(ep->napi_id); + + if (napi_id >=3D MIN_NAPI_ID && READ_ONCE(ep->prefer_busy_poll)) + napi_resume_irqs(napi_id); +} + #else =20 static inline bool ep_busy_loop(struct eventpoll *ep, int nonblock) @@ -557,6 +575,14 @@ static long ep_eventpoll_bp_ioctl(struct file *file, u= nsigned int cmd, return -EOPNOTSUPP; } =20 +static void ep_suspend_napi_irqs(struct eventpoll *ep) +{ +} + +static void ep_resume_napi_irqs(struct eventpoll *ep) +{ +} + #endif /* CONFIG_NET_RX_BUSY_POLL */ =20 /* @@ -788,6 +814,7 @@ static bool ep_refcount_dec_and_test(struct eventpoll *= ep) =20 static void ep_free(struct eventpoll *ep) { + ep_resume_napi_irqs(ep); mutex_destroy(&ep->mtx); free_uid(ep->user); wakeup_source_unregister(ep->ws); @@ -2005,8 +2032,11 @@ static int ep_poll(struct eventpoll *ep, struct epol= l_event __user *events, * trying again in search of more luck. */ res =3D ep_send_events(ep, events, maxevents); - if (res) + if (res) { + if (res > 0) + ep_suspend_napi_irqs(ep); return res; + } } =20 if (timed_out) --=20 2.25.1 From nobody Sat Nov 23 23:00:17 2024 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA69E149C7B for ; Sat, 9 Nov 2024 05:03:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128598; cv=none; b=TAF9InRi4RtCI1bi90aZ/BPALUTJuA2rw/RNQS+4Hhj3mIZhtjIfqvWChHSn1og1v78W/oImfUsuSfzepTXJxS9kC67dVo+HzFNQDi4ys9RuJ0tyhbd7pPmhaiND0JnPQPuyyeq+Y0+Uhfh+7zNkylBLawEUZjk+TdXw5yRNwto= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128598; c=relaxed/simple; bh=TA6+09JxyYx8N2BZC0DSuoqm08//c3a9ac1Xh2PsxS4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=W9DXUjyrzmwjvzS+BBh4JmxbAr+sdQBiV0S6y07wXaTGbYLqJeuJFSNuGdes0AJTk9tAsVARksDYT1L67Xzd4WzukGZPzyO0d7amzkjwp1f5cXj8Xm45d0RTFHolN5eq5sFHNA83WJ0KAo/zY7xRmbmIv8ahTyAkpiMYYw1YHms= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=viebMlJ1; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="viebMlJ1" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-20c803787abso24473385ad.0 for ; Fri, 08 Nov 2024 21:03:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1731128595; x=1731733395; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mJdyutUzcWvTw2QE9fsDIn1WWIYvPPCHPpjqEF9VBFc=; b=viebMlJ1RxSh9aHBCySoSogvvujBkrWFAIk8aoIN05r16DTSqR8hqgYJIrCEM6xO4z EfQNsYs1FOtcqCgTpgfrY030Qyc1TCdn1IqYKuGLNnmRQvWhaYv93E+mWa4JM6AiqBo3 7tdih7RWQFxvbpl/1IYB/QyfX1eEP1QvaVze8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731128595; x=1731733395; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mJdyutUzcWvTw2QE9fsDIn1WWIYvPPCHPpjqEF9VBFc=; b=NszXDOuqbUJ/BkW+yppvXc7doLbid7B1wXvnz2JLMfMC6mVRNERsqExuGn+Vd4CM2m Qk01aHk0j1Fv53Ge7om27XEVnD0GIRJQb53O+xlt542q27mJ+pLl7fp0TjTttbwTw49+ agQi7KJelPy3yxN1/PeaG7V+88EsbDu9ub1nHeD0lte3hX8XHQ43son7aOh6+XhvF0bV TmIEmHEla/wRooNcTw+THmHQYy3LIEUhh3kgOD7qFRrOEW/3DZ9oRoFyz9WzudKrbEUo 0Gcul7i1xVeu3hqh1pqFbRhkKYQ7XW+syrCT04XvFrosk2w7q+a3+4ru3aztQpIfHWCm FZIA== X-Forwarded-Encrypted: i=1; AJvYcCXepHcsN4zj74AUiJ8I3nswJK2g4iCQfq+tip4ViV9wHXrq6kAZvLYZjDJT3mDhgEYPkaX+8fRMngCR0dw=@vger.kernel.org X-Gm-Message-State: AOJu0Yx1U5NJkvstCajM8YDBcU98xKKZBgSb/mniPuh5ZD2bnrX9edq8 mkURZMcbtcOUiQrriPTVNDUfUZy7pXR1HOUHRRtCWxEIodyDIuBba174uao7ob4= X-Google-Smtp-Source: AGHT+IH4dp3m5Hhrx1CbDvczLpmKYpjGDCdqo0CFI5blSrbPFJ7dwmmY7nsWa4yDiLld1HRmLlBmKQ== X-Received: by 2002:a17:903:228d:b0:20c:5d5a:af6f with SMTP id d9443c01a7336-211836e6dcemr76079345ad.10.1731128595150; Fri, 08 Nov 2024 21:03:15 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e5853csm39182305ad.186.2024.11.08.21.03.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 21:03:14 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: corbet@lwn.net, hdanton@sina.com, bagasdotme@gmail.com, pabeni@redhat.com, namangulati@google.com, edumazet@google.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, sdf@fomichev.me, peter@typeblog.net, m2shafiei@uwaterloo.ca, bjorn@rivosinc.com, hch@infradead.org, willy@infradead.org, willemdebruijn.kernel@gmail.com, skhawaja@google.com, kuba@kernel.org, Joe Damato , Martin Karsten , "David S. Miller" , Simon Horman , Shuah Khan , linux-kernel@vger.kernel.org (open list), linux-kselftest@vger.kernel.org (open list:KERNEL SELFTEST FRAMEWORK) Subject: [PATCH net-next v9 5/6] selftests: net: Add busy_poll_test Date: Sat, 9 Nov 2024 05:02:35 +0000 Message-Id: <20241109050245.191288-6-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241109050245.191288-1-jdamato@fastly.com> References: <20241109050245.191288-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an epoll busy poll test using netdevsim. This test is comprised of: - busy_poller (via busy_poller.c) - busy_poll_test.sh which loads netdevsim, sets up network namespaces, and runs busy_poller to receive data and socat to send data. The selftest tests two different scenarios: - busy poll (the pre-existing version in the kernel) - busy poll with suspend enabled (what this series adds) The data transmit is a 1MiB temporary file generated from /dev/urandom and the test is considered passing if the md5sum of the input file to socat matches the md5sum of the output file from busy_poller. netdevsim was chosen instead of veth due to netdevsim's support for netdev-genl. For now, this test uses the functionality that netdevsim provides. In the future, perhaps netdevsim can be extended to emulate device IRQs to more thoroughly test all pre-existing kernel options (like defer_hard_irqs) and suspend. Signed-off-by: Joe Damato Co-developed-by: Martin Karsten Signed-off-by: Martin Karsten Acked-by: Stanislav Fomichev --- v9: - Based on feedback from Willem, in busy_poll_test.sh: - shortened long lines, - used more reader friendly variable names - moved constants into variables - fixed the SPDX-License-Identifier - reduced code duplication - In busy_poller.c: - Added a comment explaining the ifdef blob - Fixed some types for strtoul and added explicit casts v5: - Updated commit message to replace netcat with socat and fixed misspelling of netdevsim. No functional/code changes. v4: - Updated busy_poll_test.sh: - use socat instead of nc - drop cli.py usage from the script - removed check_ynl - Updated busy_poller.c: - use netlink to configure napi parameters v3: - New in v3 tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 3 +- tools/testing/selftests/net/busy_poll_test.sh | 165 +++++++++ tools/testing/selftests/net/busy_poller.c | 346 ++++++++++++++++++ 4 files changed, 514 insertions(+), 1 deletion(-) create mode 100755 tools/testing/selftests/net/busy_poll_test.sh create mode 100644 tools/testing/selftests/net/busy_poller.c diff --git a/tools/testing/selftests/net/.gitignore b/tools/testing/selftes= ts/net/.gitignore index 217d8b7a7365..85b0c4a2179f 100644 --- a/tools/testing/selftests/net/.gitignore +++ b/tools/testing/selftests/net/.gitignore @@ -2,6 +2,7 @@ bind_bhash bind_timewait bind_wildcard +busy_poller cmsg_sender diag_uid epoll_busy_poll diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests= /net/Makefile index 26a4883a65c9..3ccfe454db1a 100644 --- a/tools/testing/selftests/net/Makefile +++ b/tools/testing/selftests/net/Makefile @@ -96,9 +96,10 @@ TEST_PROGS +=3D fdb_flush.sh TEST_PROGS +=3D fq_band_pktlimit.sh TEST_PROGS +=3D vlan_hw_filter.sh TEST_PROGS +=3D bpf_offload.py +TEST_PROGS +=3D busy_poll_test.sh =20 # YNL files, must be before "include ..lib.mk" -YNL_GEN_FILES :=3D ncdevmem +YNL_GEN_FILES :=3D ncdevmem busy_poller TEST_GEN_FILES +=3D $(YNL_GEN_FILES) =20 TEST_FILES :=3D settings diff --git a/tools/testing/selftests/net/busy_poll_test.sh b/tools/testing/= selftests/net/busy_poll_test.sh new file mode 100755 index 000000000000..7db292ec4884 --- /dev/null +++ b/tools/testing/selftests/net/busy_poll_test.sh @@ -0,0 +1,165 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +source net_helper.sh + +NSIM_SV_ID=3D$((256 + RANDOM % 256)) +NSIM_SV_SYS=3D/sys/bus/netdevsim/devices/netdevsim$NSIM_SV_ID +NSIM_CL_ID=3D$((512 + RANDOM % 256)) +NSIM_CL_SYS=3D/sys/bus/netdevsim/devices/netdevsim$NSIM_CL_ID + +NSIM_DEV_SYS_NEW=3D/sys/bus/netdevsim/new_device +NSIM_DEV_SYS_DEL=3D/sys/bus/netdevsim/del_device +NSIM_DEV_SYS_LINK=3D/sys/bus/netdevsim/link_device +NSIM_DEV_SYS_UNLINK=3D/sys/bus/netdevsim/unlink_device + +SERVER_IP=3D192.168.1.1 +CLIENT_IP=3D192.168.1.2 +SERVER_PORT=3D48675 + +# busy poll config +MAX_EVENTS=3D8 +BUSY_POLL_USECS=3D0 +BUSY_POLL_BUDGET=3D16 +PREFER_BUSY_POLL=3D1 + +# IRQ deferral config +NAPI_DEFER_HARD_IRQS=3D100 +GRO_FLUSH_TIMEOUT=3D50000 +SUSPEND_TIMEOUT=3D20000000 + +setup_ns() +{ + set -e + ip netns add nssv + ip netns add nscl + + NSIM_SV_NAME=3D$(find $NSIM_SV_SYS/net -maxdepth 1 -type d ! \ + -path $NSIM_SV_SYS/net -exec basename {} \;) + NSIM_CL_NAME=3D$(find $NSIM_CL_SYS/net -maxdepth 1 -type d ! \ + -path $NSIM_CL_SYS/net -exec basename {} \;) + + # ensure the server has 1 queue + ethtool -L $NSIM_SV_NAME combined 1 2>/dev/null + + ip link set $NSIM_SV_NAME netns nssv + ip link set $NSIM_CL_NAME netns nscl + + ip netns exec nssv ip addr add "${SERVER_IP}/24" dev $NSIM_SV_NAME + ip netns exec nscl ip addr add "${CLIENT_IP}/24" dev $NSIM_CL_NAME + + ip netns exec nssv ip link set dev $NSIM_SV_NAME up + ip netns exec nscl ip link set dev $NSIM_CL_NAME up + + set +e +} + +cleanup_ns() +{ + ip netns del nscl + ip netns del nssv +} + +test_busypoll() +{ + suspend_value=3D${1:-0} + tmp_file=3D$(mktemp) + out_file=3D$(mktemp) + + # fill a test file with random data + dd if=3D/dev/urandom of=3D${tmp_file} bs=3D1M count=3D1 2> /dev/null + + timeout -k 1s 30s ip netns exec nssv ./busy_poller \ + -p${SERVER_PORT} \ + -b${SERVER_IP} \ + -m${MAX_EVENTS} \ + -u${BUSY_POLL_USECS} \ + -P${PREFER_BUSY_POLL} \ + -g${BUSY_POLL_BUDGET} \ + -i${NSIM_SV_IFIDX} \ + -s${suspend_value} \ + -o${out_file}& + + wait_local_port_listen nssv ${SERVER_PORT} tcp + + ip netns exec nscl socat -u $tmp_file TCP:${SERVER_IP}:${SERVER_PORT} + + wait + + tmp_file_md5sum=3D$(md5sum $tmp_file | cut -f1 -d' ') + out_file_md5sum=3D$(md5sum $out_file | cut -f1 -d' ') + + if [ "$tmp_file_md5sum" =3D "$out_file_md5sum" ]; then + res=3D0 + else + echo "md5sum mismatch" + echo "input file md5sum: ${tmp_file_md5sum}"; + echo "output file md5sum: ${out_file_md5sum}"; + res=3D1 + fi + + rm $out_file $tmp_file + + return $res +} + +test_busypoll_with_suspend() +{ + test_busypoll ${SUSPEND_TIMEOUT} + + return $? +} + +### +### Code start +### + +modprobe netdevsim + +# linking + +echo $NSIM_SV_ID > $NSIM_DEV_SYS_NEW +echo $NSIM_CL_ID > $NSIM_DEV_SYS_NEW +udevadm settle + +setup_ns + +NSIM_SV_FD=3D$((256 + RANDOM % 256)) +exec {NSIM_SV_FD} \ + $NSIM_DEV_SYS_LINK + +if [ $? -ne 0 ]; then + echo "linking netdevsim1 with netdevsim2 should succeed" + cleanup_ns + exit 1 +fi + +test_busypoll +if [ $? -ne 0 ]; then + echo "test_busypoll failed" + cleanup_ns + exit 1 +fi + +test_busypoll_with_suspend +if [ $? -ne 0 ]; then + echo "test_busypoll_with_suspend failed" + cleanup_ns + exit 1 +fi + +echo "$NSIM_SV_FD:$NSIM_SV_IFIDX" > $NSIM_DEV_SYS_UNLINK + +echo $NSIM_CL_ID > $NSIM_DEV_SYS_DEL + +cleanup_ns + +modprobe -r netdevsim + +exit 0 diff --git a/tools/testing/selftests/net/busy_poller.c b/tools/testing/self= tests/net/busy_poller.c new file mode 100644 index 000000000000..99b0e8c17fca --- /dev/null +++ b/tools/testing/selftests/net/busy_poller.c @@ -0,0 +1,346 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include +#include +#include + +#include +#include + +#include "netdev-user.h" + +/* The below ifdef blob is required because: + * + * - sys/epoll.h does not (yet) have the ioctl definitions included. So, + * systems with older glibcs will not have them available. However, + * sys/epoll.h does include the type definition for epoll_data, which is + * needed by the user program (e.g. epoll_event.data.fd) + * + * - linux/eventpoll.h does not define the epoll_data type, it is simply an + * opaque __u64. It does, however, include the ioctl definition. + * + * Including both headers is impossible (types would be redefined), so I've + * opted instead to take sys/epoll.h, and include the blob below. + * + * Someday, when glibc is globally up to date, the blob below can be remov= ed. + */ +#if !defined(EPOLL_IOC_TYPE) +struct epoll_params { + uint32_t busy_poll_usecs; + uint16_t busy_poll_budget; + uint8_t prefer_busy_poll; + + /* pad the struct to a multiple of 64bits */ + uint8_t __pad; +}; + +#define EPOLL_IOC_TYPE 0x8A +#define EPIOCSPARAMS _IOW(EPOLL_IOC_TYPE, 0x01, struct epoll_params) +#define EPIOCGPARAMS _IOR(EPOLL_IOC_TYPE, 0x02, struct epoll_params) +#endif + +static uint32_t cfg_port =3D 8000; +static struct in_addr cfg_bind_addr =3D { .s_addr =3D INADDR_ANY }; +static char *cfg_outfile; +static int cfg_max_events =3D 8; +static int cfg_ifindex; + +/* busy poll params */ +static uint32_t cfg_busy_poll_usecs; +static uint32_t cfg_busy_poll_budget; +static uint32_t cfg_prefer_busy_poll; + +/* IRQ params */ +static uint32_t cfg_defer_hard_irqs; +static uint64_t cfg_gro_flush_timeout; +static uint64_t cfg_irq_suspend_timeout; + +static void usage(const char *filepath) +{ + error(1, 0, + "Usage: %s -p -b -m -u -P<= prefer_busy_poll> -g -o -d -r -s -i", + filepath); +} + +static void parse_opts(int argc, char **argv) +{ + int ret; + int c; + + if (argc <=3D 1) + usage(argv[0]); + + while ((c =3D getopt(argc, argv, "p:m:b:u:P:g:o:d:r:s:i:")) !=3D -1) { + switch (c) { + case 'u': + cfg_busy_poll_usecs =3D strtoul(optarg, NULL, 0); + if (cfg_busy_poll_usecs =3D=3D ULONG_MAX || + cfg_busy_poll_usecs > UINT32_MAX) + error(1, ERANGE, "busy_poll_usecs too large"); + break; + case 'P': + cfg_prefer_busy_poll =3D strtoul(optarg, NULL, 0); + if (cfg_prefer_busy_poll =3D=3D ULONG_MAX || + cfg_prefer_busy_poll > 1) + error(1, ERANGE, + "prefer busy poll should be 0 or 1"); + break; + case 'g': + cfg_busy_poll_budget =3D strtoul(optarg, NULL, 0); + if (cfg_busy_poll_budget =3D=3D ULONG_MAX || + cfg_busy_poll_budget > UINT16_MAX) + error(1, ERANGE, + "busy poll budget must be [0, UINT16_MAX]"); + break; + case 'p': + cfg_port =3D strtoul(optarg, NULL, 0); + if (cfg_port > UINT16_MAX) + error(1, ERANGE, "port must be <=3D 65535"); + break; + case 'b': + ret =3D inet_aton(optarg, &cfg_bind_addr); + if (ret =3D=3D 0) + error(1, errno, + "bind address %s invalid", optarg); + break; + case 'o': + cfg_outfile =3D strdup(optarg); + if (!cfg_outfile) + error(1, 0, "outfile invalid"); + break; + case 'm': + cfg_max_events =3D strtol(optarg, NULL, 0); + + if (cfg_max_events =3D=3D LONG_MIN || + cfg_max_events =3D=3D LONG_MAX || + cfg_max_events <=3D 0) + error(1, ERANGE, + "max events must be > 0 and < LONG_MAX"); + break; + case 'd': + cfg_defer_hard_irqs =3D strtoul(optarg, NULL, 0); + + if (cfg_defer_hard_irqs =3D=3D ULONG_MAX || + cfg_defer_hard_irqs > INT32_MAX) + error(1, ERANGE, + "defer_hard_irqs must be <=3D INT32_MAX"); + break; + case 'r': + cfg_gro_flush_timeout =3D strtoull(optarg, NULL, 0); + + if (cfg_gro_flush_timeout =3D=3D ULLONG_MAX) + error(1, ERANGE, + "gro_flush_timeout must be < ULLONG_MAX"); + break; + case 's': + cfg_irq_suspend_timeout =3D strtoull(optarg, NULL, 0); + + if (cfg_irq_suspend_timeout =3D=3D ULLONG_MAX) + error(1, ERANGE, + "irq_suspend_timeout must be < ULLONG_MAX"); + break; + case 'i': + cfg_ifindex =3D strtoul(optarg, NULL, 0); + if (cfg_ifindex =3D=3D ULONG_MAX) + error(1, ERANGE, + "ifindex must be < ULONG_MAX"); + break; + } + } + + if (!cfg_ifindex) + usage(argv[0]); + + if (optind !=3D argc) + usage(argv[0]); +} + +static void epoll_ctl_add(int epfd, int fd, uint32_t events) +{ + struct epoll_event ev; + + ev.events =3D events; + ev.data.fd =3D fd; + if (epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev) =3D=3D -1) + error(1, errno, "epoll_ctl add fd: %d", fd); +} + +static void setnonblock(int sockfd) +{ + int flags; + + flags =3D fcntl(sockfd, F_GETFL, 0); + + if (fcntl(sockfd, F_SETFL, flags | O_NONBLOCK) =3D=3D -1) + error(1, errno, "unable to set socket to nonblocking mode"); +} + +static void write_chunk(int fd, char *buf, ssize_t buflen) +{ + ssize_t remaining =3D buflen; + char *buf_offset =3D buf; + ssize_t writelen =3D 0; + ssize_t write_result; + + while (writelen < buflen) { + write_result =3D write(fd, buf_offset, remaining); + if (write_result =3D=3D -1) + error(1, errno, "unable to write data to outfile"); + + writelen +=3D write_result; + remaining -=3D write_result; + buf_offset +=3D write_result; + } +} + +static void setup_queue(void) +{ + struct netdev_napi_get_list *napi_list =3D NULL; + struct netdev_napi_get_req_dump *req =3D NULL; + struct netdev_napi_set_req *set_req =3D NULL; + struct ynl_sock *ys; + struct ynl_error yerr; + uint32_t napi_id; + + ys =3D ynl_sock_create(&ynl_netdev_family, &yerr); + if (!ys) + error(1, 0, "YNL: %s", yerr.msg); + + req =3D netdev_napi_get_req_dump_alloc(); + netdev_napi_get_req_dump_set_ifindex(req, cfg_ifindex); + napi_list =3D netdev_napi_get_dump(ys, req); + + /* assume there is 1 NAPI configured and take the first */ + if (napi_list->obj._present.id) + napi_id =3D napi_list->obj.id; + else + error(1, 0, "napi ID not present?"); + + set_req =3D netdev_napi_set_req_alloc(); + netdev_napi_set_req_set_id(set_req, napi_id); + netdev_napi_set_req_set_defer_hard_irqs(set_req, cfg_defer_hard_irqs); + netdev_napi_set_req_set_gro_flush_timeout(set_req, + cfg_gro_flush_timeout); + netdev_napi_set_req_set_irq_suspend_timeout(set_req, + cfg_irq_suspend_timeout); + + if (netdev_napi_set(ys, set_req)) + error(1, 0, "can't set NAPI params: %s\n", yerr.msg); + + netdev_napi_get_list_free(napi_list); + netdev_napi_get_req_dump_free(req); + netdev_napi_set_req_free(set_req); + ynl_sock_destroy(ys); +} + +static void run_poller(void) +{ + struct epoll_event events[cfg_max_events]; + struct epoll_params epoll_params =3D {0}; + struct sockaddr_in server_addr; + int i, epfd, nfds; + ssize_t readlen; + int outfile_fd; + char buf[1024]; + int sockfd; + int conn; + int val; + + outfile_fd =3D open(cfg_outfile, O_WRONLY | O_CREAT, 0644); + if (outfile_fd =3D=3D -1) + error(1, errno, "unable to open outfile: %s", cfg_outfile); + + sockfd =3D socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); + if (sockfd =3D=3D -1) + error(1, errno, "unable to create listen socket"); + + server_addr.sin_family =3D AF_INET; + server_addr.sin_port =3D htons(cfg_port); + server_addr.sin_addr =3D cfg_bind_addr; + + /* these values are range checked during parse_opts, so casting is safe + * here + */ + epoll_params.busy_poll_usecs =3D cfg_busy_poll_usecs; + epoll_params.busy_poll_budget =3D (uint16_t)cfg_busy_poll_budget; + epoll_params.prefer_busy_poll =3D (uint8_t)cfg_prefer_busy_poll; + epoll_params.__pad =3D 0; + + val =3D 1; + if (setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val))) + error(1, errno, "poller setsockopt reuseaddr"); + + setnonblock(sockfd); + + if (bind(sockfd, (struct sockaddr *)&server_addr, + sizeof(struct sockaddr_in))) + error(0, errno, "poller bind to port: %d\n", cfg_port); + + if (listen(sockfd, 1)) + error(1, errno, "poller listen"); + + epfd =3D epoll_create1(0); + if (ioctl(epfd, EPIOCSPARAMS, &epoll_params) =3D=3D -1) + error(1, errno, "unable to set busy poll params"); + + epoll_ctl_add(epfd, sockfd, EPOLLIN | EPOLLOUT | EPOLLET); + + for (;;) { + nfds =3D epoll_wait(epfd, events, cfg_max_events, -1); + for (i =3D 0; i < nfds; i++) { + if (events[i].data.fd =3D=3D sockfd) { + conn =3D accept(sockfd, NULL, NULL); + if (conn =3D=3D -1) + error(1, errno, + "accepting incoming connection failed"); + + setnonblock(conn); + epoll_ctl_add(epfd, conn, + EPOLLIN | EPOLLET | EPOLLRDHUP | + EPOLLHUP); + } else if (events[i].events & EPOLLIN) { + for (;;) { + readlen =3D read(events[i].data.fd, buf, + sizeof(buf)); + if (readlen > 0) + write_chunk(outfile_fd, buf, + readlen); + else + break; + } + } else { + /* spurious event ? */ + } + if (events[i].events & (EPOLLRDHUP | EPOLLHUP)) { + epoll_ctl(epfd, EPOLL_CTL_DEL, + events[i].data.fd, NULL); + close(events[i].data.fd); + close(outfile_fd); + return; + } + } + } +} + +int main(int argc, char *argv[]) +{ + parse_opts(argc, argv); + setup_queue(); + run_poller(); + return 0; +} --=20 2.25.1 From nobody Sat Nov 23 23:00:17 2024 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3EFE514AD29 for ; Sat, 9 Nov 2024 05:03:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128601; cv=none; b=HydpNd+1AP20WOibp2G3y+ch2HcoK+Xb2YozM8CzQHRb8Q0VgkA95HjepUP0wIhIQy4eoiejjxPIN6SZML0Yj75P9Lqd2xxn1n/QvRzR0J0gXNV9OnqEHTEVWmYU5N1teQh7AFJoRFIDOV6DYPTlLIdFEi4UhcbCgCyPm9vVEMQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731128601; c=relaxed/simple; bh=dIGeynGkMLjGNXXYNRqQVjC5Ja6EgwuB0H/jBnVmRu0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RPCFqyaRc/4npuFG90ghdUP8h7PDmzNM4KFIfrvIGjEjJka/DIgg033s5dqFFJCOSclax3pLCrPg9j5JhRUAVGmeJc3knHcKL0WhrJ7miD5SLTCIKYICQ4AMEbKepL8tcX9YeekUC0VKBwLsXH64zzY1PKMLn2ylzSjfsPR0YlU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=PwfxpWhD; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="PwfxpWhD" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-20cd76c513cso26624585ad.3 for ; Fri, 08 Nov 2024 21:03:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1731128597; x=1731733397; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UJgan6o2RSMXdq36KFGAXLuMMgpefFNHI2RLTAg8Qn4=; b=PwfxpWhDv7sjkTj77IkvNGmCvI9OW9LPBx6lofxJY4Dhj3mm1WddJcqYT5f2LgdRNj A+vGo+hQVr2bCoRCbN7KAngRwxgytzWbpTK8HX2FfqQl5K/kFpReIjxqSUCMh3tXs9LL +3D6T9Lyht8mn3wv94Gr+pn3GQiznibgx922Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731128597; x=1731733397; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UJgan6o2RSMXdq36KFGAXLuMMgpefFNHI2RLTAg8Qn4=; b=cnuvaHAyCZGk4+6DDD7CruPWJb+6ewKQ+uiWEmIuo6cRF4j3pvq0YmQYFclF6/uPMC GT+gUq7JWBOHVTSXRkRlFN5zeUIPtdzzjg/HdOYUP+Q6yVHtb8HwYCeZx0sSwJ6Jcb34 s4doIz9LHveTHW5rQwH1CKzKsBSKA8rqia47vlz91aTftvNfYvfjOU+OSP33QfsRrvaQ a2fK4OoES9lnos3i5IO7b7xfGUfsYumRvUv23wh6fq8v1XF6rtkjnnlilGN2Xj+53AVX 3C6SGi9wtMRX0oiHKocXIlVLXr7/++W1SL9BEJ5MgYrIgadqR0AWdJ2LoQyMfwAnpTgq uJWA== X-Forwarded-Encrypted: i=1; AJvYcCVBcpggvFQcPSIZaL/M+Kxy35yImmXJzdha4mmbEp/lo/h8uHX84eADTREv7az75WQscCepvGtTo5bJ2No=@vger.kernel.org X-Gm-Message-State: AOJu0Yxovse1Byk+OZq8bEDAG3Z3mlwNl9ia9lEYeWIwifm1/spwCc5l 9IPI9iW4vPECbp/kbqjC1VBDtvHwZ0e/Hgfru33Gpr76Pgbq2xdyGmyuNCzBns8= X-Google-Smtp-Source: AGHT+IEjn+xcAUceHGud+HMcQkcqQ12yJi0EZgs+YV9Lxlbj7Jgw/sVFn4oeHmzdyGkeNYnEiaY7TA== X-Received: by 2002:a17:902:d509:b0:20c:a8cf:fa25 with SMTP id d9443c01a7336-211835df588mr81407165ad.46.1731128597393; Fri, 08 Nov 2024 21:03:17 -0800 (PST) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e5853csm39182305ad.186.2024.11.08.21.03.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 21:03:16 -0800 (PST) From: Joe Damato To: netdev@vger.kernel.org Cc: corbet@lwn.net, hdanton@sina.com, bagasdotme@gmail.com, pabeni@redhat.com, namangulati@google.com, edumazet@google.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, sdf@fomichev.me, peter@typeblog.net, m2shafiei@uwaterloo.ca, bjorn@rivosinc.com, hch@infradead.org, willy@infradead.org, willemdebruijn.kernel@gmail.com, skhawaja@google.com, kuba@kernel.org, Joe Damato , Martin Karsten , "David S. Miller" , Simon Horman , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-kernel@vger.kernel.org (open list), bpf@vger.kernel.org (open list:BPF [MISC]:Keyword:(?:\b|_)bpf(?:\b|_)) Subject: [PATCH net-next v9 6/6] docs: networking: Describe irq suspension Date: Sat, 9 Nov 2024 05:02:36 +0000 Message-Id: <20241109050245.191288-7-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241109050245.191288-1-jdamato@fastly.com> References: <20241109050245.191288-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Describe irq suspension, the epoll ioctls, and the tradeoffs of using different gro_flush_timeout values. Signed-off-by: Joe Damato Co-developed-by: Martin Karsten Signed-off-by: Martin Karsten Reviewed-by: Sridhar Samudrala Reviewed-by: Bagas Sanjaya --- v6: - Fixed packet processing loop description based on feedback from Bagas Sanjaya so that it renders properly when generated as html v5: - Fixed a minor typo in the epoll-based busy polling section - Removed short paragraph referring to experimental data as that data is not included in the documentation v4: - Updated documentation to further explain irq suspension - Dropped Stanislav's Acked-by tag because of the doc changes - Dropped Bagas' Reviewed-by tag because of the doc changes v1 -> v2: - Updated documentation to describe the per-NAPI configuration parameters. Documentation/networking/napi.rst | 170 +++++++++++++++++++++++++++++- 1 file changed, 168 insertions(+), 2 deletions(-) diff --git a/Documentation/networking/napi.rst b/Documentation/networking/n= api.rst index dfa5d549be9c..02720dd71a76 100644 --- a/Documentation/networking/napi.rst +++ b/Documentation/networking/napi.rst @@ -192,6 +192,33 @@ is reused to control the delay of the timer, while ``napi_defer_hard_irqs`` controls the number of consecutive empty polls before NAPI gives up and goes back to using hardware IRQs. =20 +The above parameters can also be set on a per-NAPI basis using netlink via +netdev-genl. When used with netlink and configured on a per-NAPI basis, the +parameters mentioned above use hyphens instead of underscores: +``gro-flush-timeout`` and ``napi-defer-hard-irqs``. + +Per-NAPI configuration can be done programmatically in a user application +or by using a script included in the kernel source tree: +``tools/net/ynl/cli.py``. + +For example, using the script: + +.. code-block:: bash + + $ kernel-source/tools/net/ynl/cli.py \ + --spec Documentation/netlink/specs/netdev.yaml \ + --do napi-set \ + --json=3D'{"id": 345, + "defer-hard-irqs": 111, + "gro-flush-timeout": 11111}' + +Similarly, the parameter ``irq-suspend-timeout`` can be set using netlink +via netdev-genl. There is no global sysfs parameter for this value. + +``irq-suspend-timeout`` is used to determine how long an application can +completely suspend IRQs. It is used in combination with SO_PREFER_BUSY_POL= L, +which can be set on a per-epoll context basis with ``EPIOCSPARAMS`` ioctl. + .. _poll: =20 Busy polling @@ -207,6 +234,46 @@ selected sockets or using the global ``net.core.busy_p= oll`` and ``net.core.busy_read`` sysctls. An io_uring API for NAPI busy polling also exists. =20 +epoll-based busy polling +------------------------ + +It is possible to trigger packet processing directly from calls to +``epoll_wait``. In order to use this feature, a user application must ensu= re +all file descriptors which are added to an epoll context have the same NAP= I ID. + +If the application uses a dedicated acceptor thread, the application can o= btain +the NAPI ID of the incoming connection using SO_INCOMING_NAPI_ID and then +distribute that file descriptor to a worker thread. The worker thread woul= d add +the file descriptor to its epoll context. This would ensure each worker th= read +has an epoll context with FDs that have the same NAPI ID. + +Alternatively, if the application uses SO_REUSEPORT, a bpf or ebpf program= can +be inserted to distribute incoming connections to threads such that each t= hread +is only given incoming connections with the same NAPI ID. Care must be tak= en to +carefully handle cases where a system may have multiple NICs. + +In order to enable busy polling, there are two choices: + +1. ``/proc/sys/net/core/busy_poll`` can be set with a time in useconds to = busy + loop waiting for events. This is a system-wide setting and will cause a= ll + epoll-based applications to busy poll when they call epoll_wait. This m= ay + not be desirable as many applications may not have the need to busy pol= l. + +2. Applications using recent kernels can issue an ioctl on the epoll conte= xt + file descriptor to set (``EPIOCSPARAMS``) or get (``EPIOCGPARAMS``) ``s= truct + epoll_params``:, which user programs can define as follows: + +.. code-block:: c + + struct epoll_params { + uint32_t busy_poll_usecs; + uint16_t busy_poll_budget; + uint8_t prefer_busy_poll; + + /* pad the struct to a multiple of 64bits */ + uint8_t __pad; + }; + IRQ mitigation --------------- =20 @@ -222,12 +289,111 @@ Such applications can pledge to the kernel that they= will perform a busy polling operation periodically, and the driver should keep the device IRQs permanently masked. This mode is enabled by using the ``SO_PREFER_BUSY_POL= L`` socket option. To avoid system misbehavior the pledge is revoked -if ``gro_flush_timeout`` passes without any busy poll call. +if ``gro_flush_timeout`` passes without any busy poll call. For epoll-based +busy polling applications, the ``prefer_busy_poll`` field of ``struct +epoll_params`` can be set to 1 and the ``EPIOCSPARAMS`` ioctl can be issue= d to +enable this mode. See the above section for more details. =20 The NAPI budget for busy polling is lower than the default (which makes sense given the low latency intention of normal busy polling). This is not the case with IRQ mitigation, however, so the budget can be adjusted -with the ``SO_BUSY_POLL_BUDGET`` socket option. +with the ``SO_BUSY_POLL_BUDGET`` socket option. For epoll-based busy polli= ng +applications, the ``busy_poll_budget`` field can be adjusted to the desire= d value +in ``struct epoll_params`` and set on a specific epoll context using the `= `EPIOCSPARAMS`` +ioctl. See the above section for more details. + +It is important to note that choosing a large value for ``gro_flush_timeou= t`` +will defer IRQs to allow for better batch processing, but will induce late= ncy +when the system is not fully loaded. Choosing a small value for +``gro_flush_timeout`` can cause interference of the user application which= is +attempting to busy poll by device IRQs and softirq processing. This value +should be chosen carefully with these tradeoffs in mind. epoll-based busy +polling applications may be able to mitigate how much user processing happ= ens +by choosing an appropriate value for ``maxevents``. + +Users may want to consider an alternate approach, IRQ suspension, to help = deal +with these tradeoffs. + +IRQ suspension +-------------- + +IRQ suspension is a mechanism wherein device IRQs are masked while epoll +triggers NAPI packet processing. + +While application calls to epoll_wait successfully retrieve events, the ke= rnel will +defer the IRQ suspension timer. If the kernel does not retrieve any events +while busy polling (for example, because network traffic levels subsided),= IRQ +suspension is disabled and the IRQ mitigation strategies described above a= re +engaged. + +This allows users to balance CPU consumption with network processing +efficiency. + +To use this mechanism: + + 1. The per-NAPI config parameter ``irq-suspend-timeout`` should be set t= o the + maximum time (in nanoseconds) the application can have its IRQs + suspended. This is done using netlink, as described above. This timeo= ut + serves as a safety mechanism to restart IRQ driver interrupt processi= ng if + the application has stalled. This value should be chosen so that it c= overs + the amount of time the user application needs to process data from its + call to epoll_wait, noting that applications can control how much data + they retrieve by setting ``max_events`` when calling epoll_wait. + + 2. The sysfs parameter or per-NAPI config parameters ``gro_flush_timeout= `` + and ``napi_defer_hard_irqs`` can be set to low values. They will be u= sed + to defer IRQs after busy poll has found no data. + + 3. The ``prefer_busy_poll`` flag must be set to true. This can be done u= sing + the ``EPIOCSPARAMS`` ioctl as described above. + + 4. The application uses epoll as described above to trigger NAPI packet + processing. + +As mentioned above, as long as subsequent calls to epoll_wait return event= s to +userland, the ``irq-suspend-timeout`` is deferred and IRQs are disabled. T= his +allows the application to process data without interference. + +Once a call to epoll_wait results in no events being found, IRQ suspension= is +automatically disabled and the ``gro_flush_timeout`` and +``napi_defer_hard_irqs`` mitigation mechanisms take over. + +It is expected that ``irq-suspend-timeout`` will be set to a value much la= rger +than ``gro_flush_timeout`` as ``irq-suspend-timeout`` should suspend IRQs = for +the duration of one userland processing cycle. + +While it is not stricly necessary to use ``napi_defer_hard_irqs`` and +``gro_flush_timeout`` to use IRQ suspension, their use is strongly +recommended. + +IRQ suspension causes the system to alternate between polling mode and +irq-driven packet delivery. During busy periods, ``irq-suspend-timeout`` +overrides ``gro_flush_timeout`` and keeps the system busy polling, but when +epoll finds no events, the setting of ``gro_flush_timeout`` and +``napi_defer_hard_irqs`` determine the next step. + +There are essentially three possible loops for network processing and +packet delivery: + +1) hardirq -> softirq -> napi poll; basic interrupt delivery +2) timer -> softirq -> napi poll; deferred irq processing +3) epoll -> busy-poll -> napi poll; busy looping + +Loop 2 can take control from Loop 1, if ``gro_flush_timeout`` and +``napi_defer_hard_irqs`` are set. + +If ``gro_flush_timeout`` and ``napi_defer_hard_irqs`` are set, Loops 2 +and 3 "wrestle" with each other for control. + +During busy periods, ``irq-suspend-timeout`` is used as timer in Loop 2, +which essentially tilts network processing in favour of Loop 3. + +If ``gro_flush_timeout`` and ``napi_defer_hard_irqs`` are not set, Loop 3 +cannot take control from Loop 1. + +Therefore, setting ``gro_flush_timeout`` and ``napi_defer_hard_irqs`` is +the recommended usage, because otherwise setting ``irq-suspend-timeout`` +might not have any discernible effect. =20 .. _threaded: =20 --=20 2.25.1