From nobody Mon Nov 25 02:02:19 2024 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDE721369AE for ; Fri, 1 Nov 2024 00:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730422175; cv=none; b=HFRa8JLusoHVebylJpJlr2Ph9PeEW0DrxcsK/TAPzOl16WiUSgyqq7iKfGsIp99qHpSf1jAubu3DM9jpS+raG1wzPLrUEqPnsbl20ADetxO+1dQ3tyYFlntThtB+dWJCm/QbvCDa69bluUjWj2TaReoiJH0e5kIV8Ml9m0Ca644= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730422175; c=relaxed/simple; bh=JIqIEaw/Xkb4IhEOecWCp/j56eekf1dIpHWraAxZfLU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rDZ9JCXG+IGxUyhejE1lP5Vi46XlMA3yt1vA5iQN6IK3R8hHF1CXm1FisZ3Rni8D1zMXKPNokUg8CvfOO5s2mOB6JFVbFwuWOQGsnhVji5OV2k+JXlWeZkSRIYhXDskE9JVNyq/JJW9Mo2kNYiKPOIwQgyEsfeyFlRkGS3QlOt4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=KUpcA0EH; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="KUpcA0EH" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-71e52582cf8so1232448b3a.2 for ; Thu, 31 Oct 2024 17:49:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1730422167; x=1731026967; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eTHWnISoB9bhQ055WvkbHwZ+dmeJ9HCmVk36PR/yRAs=; b=KUpcA0EH5KESLUq6Zved1VHltKSZzJoyLEFDaCVF4wniN6HN69TRFBkuNgbsQ1vSue fRcijUitgqKc6Og5lz/+qNMfA9MlW39MrksklNL4/H+6cYPb+m5Me0sMGkrdEJRGFV1O BFe9GJWW6tvgGm9CkCqIiOz8RqxBm1F8XWDn0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730422167; x=1731026967; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eTHWnISoB9bhQ055WvkbHwZ+dmeJ9HCmVk36PR/yRAs=; b=E1M452a/54euICJ4pLhiZOPnkCfg6BgMtC/Xxas4AWZ6oex+93abZRTnGwnuuesO/w FPpCziUgAA2RD1QvBiAtbbgToNqmCCJQtwiCsTtGz6ThBa/3aUu/zU+lfGMp6sTTKY1h 97e8Z1jma1L4AZ3sMW4Xajbdn9KXXDBRHCzjlHZs0doPJAEjIYKrJq/vvldn5fSywuvC WoNdc1vreT/yVa0aCy0BvfIecs/PAAJsVPA2pUxH3v/uQLqj3w8wYDzY9bme7YaaNCn0 mEI2oxgCfkQxGJL/ywGjpGPApv6ayN7f3XtJQPgVCYqKbYoiVnMEmKHh0pfMvqYvyuL3 +vZQ== X-Forwarded-Encrypted: i=1; AJvYcCV3gyhqEIl+Bk7e3UL2kzFIfL0fq5FuTaIST8T6quQhUjR7v54+0oQLiSzE5AvSOEwPiyl5fPq+A1uvCAY=@vger.kernel.org X-Gm-Message-State: AOJu0YzYflZ5GXGn13TXgWfpkEPe64ixzwPBcgI/GgO8C6cLI22092vR BWthtQYl6ngHJ4f92m9B5/QtcI6bEL24BEZPMooSKJ0tyocGAjIyBrUIVgsFz1k= X-Google-Smtp-Source: AGHT+IEoY1KMHd8KZb8qMlUl9iT2A/57KkBUfF8xPOGHBaPR4o7uPXqExc76fhPtwVsPduFrWv4bRQ== X-Received: by 2002:a05:6a21:670f:b0:1d9:266e:8216 with SMTP id adf61e73a8af0-1dba53bf472mr1794797637.22.1730422166902; Thu, 31 Oct 2024 17:49:26 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7ee45a12c27sm1585365a12.93.2024.10.31.17.49.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Oct 2024 17:49:26 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: pabeni@redhat.com, namangulati@google.com, edumazet@google.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, sdf@fomichev.me, peter@typeblog.net, m2shafiei@uwaterloo.ca, bjorn@rivosinc.com, hch@infradead.org, willy@infradead.org, willemdebruijn.kernel@gmail.com, skhawaja@google.com, kuba@kernel.org, Joe Damato , Martin Karsten , Bagas Sanjaya , "David S. Miller" , Simon Horman , Jonathan Corbet , linux-doc@vger.kernel.org (open list:DOCUMENTATION), linux-kernel@vger.kernel.org (open list), bpf@vger.kernel.org (open list:BPF [MISC]:Keyword:(?:\b|_)bpf(?:\b|_)) Subject: [PATCH net-next v3 7/7] docs: networking: Describe irq suspension Date: Fri, 1 Nov 2024 00:48:34 +0000 Message-Id: <20241101004846.32532-8-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241101004846.32532-1-jdamato@fastly.com> References: <20241101004846.32532-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Describe irq suspension, the epoll ioctls, and the tradeoffs of using different gro_flush_timeout values. Signed-off-by: Joe Damato Co-developed-by: Martin Karsten Signed-off-by: Martin Karsten Acked-by: Stanislav Fomichev Reviewed-by: Bagas Sanjaya --- v1 -> v2: - Updated documentation to describe the per-NAPI configuration parameters. Documentation/networking/napi.rst | 132 +++++++++++++++++++++++++++++- 1 file changed, 130 insertions(+), 2 deletions(-) diff --git a/Documentation/networking/napi.rst b/Documentation/networking/n= api.rst index dfa5d549be9c..3b43477a52ce 100644 --- a/Documentation/networking/napi.rst +++ b/Documentation/networking/napi.rst @@ -192,6 +192,28 @@ is reused to control the delay of the timer, while ``napi_defer_hard_irqs`` controls the number of consecutive empty polls before NAPI gives up and goes back to using hardware IRQs. =20 +The above parameters can also be set on a per-NAPI basis using netlink via +netdev-genl. This can be done programmatically in a user application or by +using a script included in the kernel source tree: ``tools/net/ynl/cli.py`= `. + +For example, using the script: + +.. code-block:: bash + + $ kernel-source/tools/net/ynl/cli.py \ + --spec Documentation/netlink/specs/netdev.yaml \ + --do napi-set \ + --json=3D'{"id": 345, + "defer-hard-irqs": 111, + "gro-flush-timeout": 11111}' + +Similarly, the parameter ``irq-suspend-timeout`` can be set using netlink +via netdev-genl. There is no global sysfs parameter for this value. + +``irq_suspend_timeout`` is used to determine how long an application can +completely suspend IRQs. It is used in combination with SO_PREFER_BUSY_POL= L, +which can be set on a per-epoll context basis with ``EPIOCSPARAMS`` ioctl. + .. _poll: =20 Busy polling @@ -207,6 +229,46 @@ selected sockets or using the global ``net.core.busy_p= oll`` and ``net.core.busy_read`` sysctls. An io_uring API for NAPI busy polling also exists. =20 +epoll-based busy polling +------------------------ + +It is possible to trigger packet processing directly from calls to +``epoll_wait``. In order to use this feature, a user application must ensu= re +all file descriptors which are added to an epoll context have the same NAP= I ID. + +If the application uses a dedicated acceptor thread, the application can o= btain +the NAPI ID of the incoming connection using SO_INCOMING_NAPI_ID and then +distribute that file descriptor to a worker thread. The worker thread woul= d add +the file descriptor to its epoll context. This would ensure each worker th= read +has an epoll context with FDs that have the same NAPI ID. + +Alternatively, if the application uses SO_REUSEPORT, a bpf or ebpf program= be +inserted to distribute incoming connections to threads such that each thre= ad is +only given incoming connections with the same NAPI ID. Care must be taken = to +carefully handle cases where a system may have multiple NICs. + +In order to enable busy polling, there are two choices: + +1. ``/proc/sys/net/core/busy_poll`` can be set with a time in useconds to = busy + loop waiting for events. This is a system-wide setting and will cause a= ll + epoll-based applications to busy poll when they call epoll_wait. This m= ay + not be desirable as many applications may not have the need to busy pol= l. + +2. Applications using recent kernels can issue an ioctl on the epoll conte= xt + file descriptor to set (``EPIOCSPARAMS``) or get (``EPIOCGPARAMS``) ``s= truct + epoll_params``:, which user programs can define as follows: + +.. code-block:: c + + struct epoll_params { + uint32_t busy_poll_usecs; + uint16_t busy_poll_budget; + uint8_t prefer_busy_poll; + + /* pad the struct to a multiple of 64bits */ + uint8_t __pad; + }; + IRQ mitigation --------------- =20 @@ -222,12 +284,78 @@ Such applications can pledge to the kernel that they = will perform a busy polling operation periodically, and the driver should keep the device IRQs permanently masked. This mode is enabled by using the ``SO_PREFER_BUSY_POL= L`` socket option. To avoid system misbehavior the pledge is revoked -if ``gro_flush_timeout`` passes without any busy poll call. +if ``gro_flush_timeout`` passes without any busy poll call. For epoll-based +busy polling applications, the ``prefer_busy_poll`` field of ``struct +epoll_params`` can be set to 1 and the ``EPIOCSPARAMS`` ioctl can be issue= d to +enable this mode. See the above section for more details. =20 The NAPI budget for busy polling is lower than the default (which makes sense given the low latency intention of normal busy polling). This is not the case with IRQ mitigation, however, so the budget can be adjusted -with the ``SO_BUSY_POLL_BUDGET`` socket option. +with the ``SO_BUSY_POLL_BUDGET`` socket option. For epoll-based busy polli= ng +applications, the ``busy_poll_budget`` field can be adjusted to the desire= d value +in ``struct epoll_params`` and set on a specific epoll context using the `= `EPIOCSPARAMS`` +ioctl. See the above section for more details. + +It is important to note that choosing a large value for ``gro_flush_timeou= t`` +will defer IRQs to allow for better batch processing, but will induce late= ncy +when the system is not fully loaded. Choosing a small value for +``gro_flush_timeout`` can cause interference of the user application which= is +attempting to busy poll by device IRQs and softirq processing. This value +should be chosen carefully with these tradeoffs in mind. epoll-based busy +polling applications may be able to mitigate how much user processing happ= ens +by choosing an appropriate value for ``maxevents``. + +Users may want to consider an alternate approach, IRQ suspension, to help = deal +with these tradeoffs. + +IRQ suspension +-------------- + +IRQ suspension is a mechanism wherein device IRQs are masked while epoll +triggers NAPI packet processing. + +While application calls to epoll_wait successfully retrieve events, the ke= rnel will +defer the IRQ suspension timer. If the kernel does not retrieve any events +while busy polling (for example, because network traffic levels subsided),= IRQ +suspension is disabled and the IRQ mitigation strategies described above a= re +engaged. + +This allows users to balance CPU consumption with network processing +efficiency. + +To use this mechanism: + + 1. The per-NAPI config parameter ``irq_suspend_timeout`` should be set t= o the + maximum time (in nanoseconds) the application can have its IRQs + suspended. This is done using netlink, as described above. This timeo= ut + serves as a safety mechanism to restart IRQ driver interrupt processi= ng if + the application has stalled. This value should be chosen so that it c= overs + the amount of time the user application needs to process data from its + call to epoll_wait, noting that applications can control how much data + they retrieve by setting ``max_events`` when calling epoll_wait. + + 2. The sysfs parameter or per-NAPI config parameters ``gro_flush_timeout= `` + and ``napi_defer_hard_irqs`` can be set to low values. They will be u= sed + to defer IRQs after busy poll has found no data. + + 3. The ``prefer_busy_poll`` flag must be set to true. This can be done u= sing + the ``EPIOCSPARAMS`` ioctl as described above. + + 4. The application uses epoll as described above to trigger NAPI packet + processing. + +As mentioned above, as long as subsequent calls to epoll_wait return event= s to +userland, the ``irq_suspend_timeout`` is deferred and IRQs are disabled. T= his +allows the application to process data without interference. + +Once a call to epoll_wait results in no events being found, IRQ suspension= is +automatically disabled and the ``gro_flush_timeout`` and +``napi_defer_hard_irqs`` mitigation mechanisms take over. + +It is expected that ``irq_suspend_timeout`` will be set to a value much la= rger +than ``gro_flush_timeout`` as ``irq_suspend_timeout`` should suspend IRQs = for +the duration of one userland processing cycle. =20 .. _threaded: =20 --=20 2.25.1