From nobody Sat Jul 25 06:02:56 2026 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F23C3EE1C0 for ; Fri, 24 Jul 2026 21:21:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784928108; cv=none; b=NSq6HfIjKjeIGBkaC5r7mkuy92QydRz+n7jIpUIVScZe2NfXknOOu18e1AKcNDtSOPHl071YQtXK9mcEBmWKVritGT1jxjeYenil48RVtZ/TQsAQKyFXpa1uGdGxciYTXxLfTUUdFwwmE68GZhyBMix46mICLUcc5j4i75qrLcQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784928108; c=relaxed/simple; bh=7TOa2VV1TbNrnyXqytel2nuAlMHR9rwZc8Qp57r1p1E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PIRINdQfbYtOjryBRizT9rXKgOFkAY+vBU94a/4JC2K327onKjHsBcxn8znGq35ccJmQOJM6FoNZmPEqfh2tK0BnaPZEKhcCiuEwx0OZI3LzzHRxwG/Kzbz+IxJuf4Dv7um8MFtF62J7hhC/meNOR3Qmevp61jrZRgwjfpM/ECs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EA3REyvu; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EA3REyvu" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2cee9b74ee1so7804085ad.3 for ; Fri, 24 Jul 2026 14:21:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784928105; x=1785532905; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=oeflXoru0Mfnf5XrqEAO+FerIpvmIxL3GIy1D3aexGs=; b=EA3REyvuYu6rfjzgXVckP9zcFemPOI7MwVRyx2V6n2zyc3zdT166DDZUki6p61/8/O 8sxCsypkslauZNuyFRDf079lYiUon1d3M5fMoonP+SRwwUSkgwysaCPZtl/AycsP/qfb zz/zC71jYEqDQa5NCXrPB9S8CAJ5+Jz9rcQ9W9U5QBvouRlHGW9maR41pqqbkOhDn8gi /Voh9ruk6YuPJvBFIuwxd44mropnVFrufBivTUbgBO9jRuNClrHU+Q2mFU2FS3o7GIPk JY7VEbcrtCo7xl8HHbBKMKmVkrLtouKsCssy9DRI3jCNe+tVYUfA1edIzIreP6+VjNuG GVMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784928105; x=1785532905; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=oeflXoru0Mfnf5XrqEAO+FerIpvmIxL3GIy1D3aexGs=; b=fNJDO0lUEehfpvAKsiuVEt4wOVlYOT4iaxcuabTI0K6PmQLclwak/5Z0Oqx1y2GIL1 YDEVj7MlBmtp0pK1rE2ms5FpNzqnaQ12zspb2GCjbcXTZGmI+5V0AG3mrSOL30spQoXw TRo3KuzPqE/lGgf+9pi0pZ6QYnNhZnKKn7uU6RI1Xp8DUB41Ju7esEoji4TsdCCnlAms g2ntJEH+p5IeQi6dCyiff5Pg4dl8XK7RY/K/kJE3s+KC9aOvSozkVbiPWzKpFR++cBcb 3y6b+IyDNW/PUgruk24hEcLjS0MrkKshLLzg7h/GbJXGUnaxJkZuF2ExEBL3/xm3OgOH MP+g== X-Forwarded-Encrypted: i=1; AHgh+Rpda4QVSUOHNkWkFRMDOiYeiBeyLJ5QIONXTaEqQmKmgF1AMPHnIHHLm02eCETKbtmGpRX8NrtwAbhKq4A=@vger.kernel.org X-Gm-Message-State: AOJu0Yyu2F9UsaZdwxJk1dWwK6okT8jH/02Gi1aOtRE1NW3ZU0b+QsXc 3Zl0TszCqMmm6TnyNLW4GRsPVSI5mf4Yy/xIjoaAwFJgaa/3Ee6HSND4 X-Gm-Gg: AR+sD11GDtNDE+oiuEmrTnOWqKrPHpT2oivntN4oQCsIWoDF4z05YMXsY5S46wWqgu2 QuhsEG7qE68yJXLgDBF+nI0Kh1VjtlCs1fn9tOm3oxHduC9zA8YXdB32x+cT5eXeJEse6ChyK/V Y1UJ6QjvYxaYfZley9bQs/Bvqusexr2czRSrOSYhFf61LOgAK8LPoboaVnUr686ZYQKWH4ncP3d Dc+AChiNZwFse+QJskgVkZqdNJZ7dkS3+qxmnynIWdHdpjbt7f5jTkMiPVwWmmw4C3ZphEe6tl/ BQrLxrDcdoBUtVyLIgmVhkTQHVlRkrFI1l63421sx8RIiBCWHJIHWKB8vDMCjq2tUWPYGR6PCc2 /ftmbHVcvzpdvYO77V5NtXMw9w+gdBQT4j0Vz3spA/x+PcUZtXtRsVFf/lEV3j7bSeByuS3/kgw == X-Received: by 2002:a17:902:da4d:b0:2cc:6817:d9b1 with SMTP id d9443c01a7336-2cfde8a52a7mr925145ad.41.1784928104766; Fri, 24 Jul 2026 14:21:44 -0700 (PDT) Received: from localhost ([2a03:2880:ff:49::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cfde7ee6b0sm335535ad.63.2026.07.24.14.21.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jul 2026 14:21:44 -0700 (PDT) From: Bobby Eshleman Date: Fri, 24 Jul 2026 14:21:15 -0700 Subject: [PATCH net-next v6 1/3] net: devmem: allow rx-page-size > PAGE_SIZE per dmabuf binding Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-tcpdm-large-niovs-v6-1-7975f62cdfbb@meta.com> References: <20260724-tcpdm-large-niovs-v6-0-7975f62cdfbb@meta.com> In-Reply-To: <20260724-tcpdm-large-niovs-v6-0-7975f62cdfbb@meta.com> To: Donald Hunter , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Simon Horman , Andrew Lunn , Gerd Hoffmann , Vivek Kasireddy , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-kselftest@vger.kernel.org, sdf@fomichev.me, razor@blackwall.org, daniel@iogearbox.net, almasrymina@google.com, matttbe@kernel.org, skhawaja@google.com, dw@davidwei.uk, Joe Damato , Bobby Eshleman X-Mailer: b4 0.14.3 From: Bobby Eshleman Every devmem dmabuf binding today hands the page_pool PAGE_SIZE niovs. This caps a single RX descriptor at PAGE_SIZE, burning CPU on buffer churn for large flows. Add a bind-time netlink attribute, NETDEV_A_DMABUF_RX_PAGE_SIZE, that lets userspace request a larger niov size. The value must be a power of two >=3D PAGE_SIZE. Measurements: Setup: kperf in devmem RX/TX cuda mode, 4 flows, 64 MB messages, 60s, dctcp, num-rx-queues=3D4, dmabuf-rx/tx-size-mb=3D2048, 10 runs per niov size, mlx5. CPU Util: niov net sirq % net idle % app sys % app id= le % ----- ---------------- ---------------- ---------------- ------------= ---- 4K 62.38 +/- 8.27 33.40 +/- 7.51 54.15 +/- 10.23 43.67 +/- 1= 0.53 16K 58.91 +/- 5.35 35.23 +/- 5.88 41.05 +/- 8.87 56.42 +/- = 9.24 32K 64.12 +/- 0.68 31.09 +/- 1.48 44.54 +/- 3.51 52.63 +/- = 3.65 64K 54.69 +/- 5.54 39.67 +/- 5.81 35.47 +/- 3.11 61.97 +/- = 3.27 RX app sys % drops ~19% from 4K to 64K. Throughput: niov RX dev Gbps RX flow avg Gbps ----- ---------------- ----------------- 4K 300.63 +/- 53.21 75.16 +/- 13.30 16K 321.35 +/- 28.20 80.34 +/- 7.05 32K 347.63 +/- 2.20 86.91 +/- 0.55 64K 332.11 +/- 14.26 83.03 +/- 3.56 Throughput seems to increase, but the stdev is pretty wide so could just be noise. kperf support (not yet merged): https://github.com/facebookexperimental/kperf/commit/8837577f920876bce6986e= c18869ac04439ebcd2 Signed-off-by: Bobby Eshleman Acked-by: Stanislav Fomichev Reviewed-by: Mina Almasry --- Documentation/netlink/specs/netdev.yaml | 17 ++++++++++++ include/uapi/linux/netdev.h | 1 + net/core/devmem.c | 49 +++++++++++++++++++----------= ---- net/core/devmem.h | 13 ++++++--- net/core/netdev-genl-gen.c | 11 ++++++-- net/core/netdev-genl-gen.h | 1 + net/core/netdev-genl.c | 18 ++++++++++-- tools/include/uapi/linux/netdev.h | 1 + 8 files changed, 83 insertions(+), 28 deletions(-) diff --git a/Documentation/netlink/specs/netdev.yaml b/Documentation/netlin= k/specs/netdev.yaml index 5f143da7458c..1ee64342df11 100644 --- a/Documentation/netlink/specs/netdev.yaml +++ b/Documentation/netlink/specs/netdev.yaml @@ -6,6 +6,12 @@ doc: >- netdev configuration over generic netlink. =20 definitions: + - + type: const + name: page-size + value: 0 # dummy: codegen needs a number, real value is the PAGE_SIZE= macro (header) + header: asm/page.h + scope: kernel - type: flags name: xdp-act @@ -598,6 +604,16 @@ attribute-sets: type: u32 checks: min: 1 + - + name: rx-page-size + doc: | + Size in bytes of each device page the NIC writes into from the b= ound + dmabuf. Must be a power of two and >=3D PAGE_SIZE; defaults to + PAGE_SIZE. + type: u32 + checks: + min: page-size + max: u32-max =20 operations: list: @@ -812,6 +828,7 @@ operations: - ifindex - fd - queues + - rx-page-size reply: attributes: - id diff --git a/include/uapi/linux/netdev.h b/include/uapi/linux/netdev.h index 2f3ab75e8cc0..35ff083221c7 100644 --- a/include/uapi/linux/netdev.h +++ b/include/uapi/linux/netdev.h @@ -219,6 +219,7 @@ enum { NETDEV_A_DMABUF_QUEUES, NETDEV_A_DMABUF_FD, NETDEV_A_DMABUF_ID, + NETDEV_A_DMABUF_RX_PAGE_SIZE, =20 __NETDEV_A_DMABUF_MAX, NETDEV_A_DMABUF_MAX =3D (__NETDEV_A_DMABUF_MAX - 1) diff --git a/net/core/devmem.c b/net/core/devmem.c index 957d6b96216b..b036f14cb23a 100644 --- a/net/core/devmem.c +++ b/net/core/devmem.c @@ -46,7 +46,7 @@ static dma_addr_t net_devmem_get_dma_addr(const struct ne= t_iov *niov) =20 owner =3D net_devmem_iov_to_chunk_owner(niov); return owner->base_dma_addr + - ((dma_addr_t)net_iov_idx(niov) << PAGE_SHIFT); + ((dma_addr_t)net_iov_idx(niov) << owner->binding->niov_shift); } =20 static void net_devmem_dmabuf_binding_release(struct percpu_ref *ref) @@ -93,13 +93,14 @@ net_devmem_alloc_dmabuf(struct net_devmem_dmabuf_bindin= g *binding) ssize_t offset; ssize_t index; =20 - dma_addr =3D gen_pool_alloc_owner(binding->chunk_pool, PAGE_SIZE, + dma_addr =3D gen_pool_alloc_owner(binding->chunk_pool, + 1UL << binding->niov_shift, (void **)&owner); if (!dma_addr) return NULL; =20 offset =3D dma_addr - owner->base_dma_addr; - index =3D offset / PAGE_SIZE; + index =3D offset >> binding->niov_shift; niov =3D &owner->area.niovs[index]; =20 niov->desc.pp_magic =3D 0; @@ -113,12 +114,13 @@ void net_devmem_free_dmabuf(struct net_iov *niov) { struct net_devmem_dmabuf_binding *binding =3D net_devmem_iov_binding(niov= ); unsigned long dma_addr =3D net_devmem_get_dma_addr(niov); + size_t niov_size =3D 1UL << binding->niov_shift; =20 if (WARN_ON(!gen_pool_has_addr(binding->chunk_pool, dma_addr, - PAGE_SIZE))) + niov_size))) return; =20 - gen_pool_free(binding->chunk_pool, dma_addr, PAGE_SIZE); + gen_pool_free(binding->chunk_pool, dma_addr, niov_size); } =20 void net_devmem_unbind_dmabuf(struct net_devmem_dmabuf_binding *binding) @@ -163,6 +165,9 @@ int net_devmem_bind_dmabuf_to_queue(struct net_device *= dev, u32 rxq_idx, u32 xa_idx; int err; =20 + if (binding->niov_shift !=3D PAGE_SHIFT) + mp_params.rx_page_size =3D 1U << binding->niov_shift; + err =3D netif_mp_open_rxq(dev, rxq_idx, &mp_params, extack); if (err) return err; @@ -184,10 +189,12 @@ struct net_devmem_dmabuf_binding * net_devmem_bind_dmabuf(struct net_device *dev, void *vdev, struct device *dma_dev, enum dma_data_direction direction, - unsigned int dmabuf_fd, struct netdev_nl_sock *priv, + unsigned int dmabuf_fd, unsigned int niov_shift, + struct netdev_nl_sock *priv, struct netlink_ext_ack *extack) { struct net_devmem_dmabuf_binding *binding; + size_t niov_size =3D 1UL << niov_shift; static u32 id_alloc_next; struct scatterlist *sg; struct dma_buf *dmabuf; @@ -213,6 +220,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *vd= ev, =20 binding->dev =3D dev; binding->vdev =3D vdev; + binding->niov_shift =3D niov_shift; xa_init_flags(&binding->bound_rxqs, XA_FLAGS_ALLOC); =20 err =3D percpu_ref_init(&binding->ref, @@ -248,18 +256,14 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *= vdev, goto err_unmap; } binding->tx_vec =3D kvmalloc_objs(struct net_iov *, - dmabuf->size / PAGE_SIZE); + dmabuf->size >> niov_shift); if (!binding->tx_vec) { err =3D -ENOMEM; goto err_unmap; } } =20 - /* For simplicity we expect to make PAGE_SIZE allocations, but the - * binding can be much more flexible than that. We may be able to - * allocate MTU sized chunks here. Leave that for future work... - */ - binding->chunk_pool =3D gen_pool_create(PAGE_SHIFT, + binding->chunk_pool =3D gen_pool_create(niov_shift, dev_to_node(&dev->dev)); if (!binding->chunk_pool) { err =3D -ENOMEM; @@ -273,9 +277,12 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *v= dev, size_t len =3D sg_dma_len(sg); struct net_iov *niov; =20 - if (!IS_ALIGNED(len, PAGE_SIZE)) { + if (!IS_ALIGNED(dma_addr, niov_size) || + !IS_ALIGNED(len, niov_size)) { err =3D -EINVAL; - NL_SET_ERR_MSG(extack, "dma-buf SG length must be PAGE_SIZE aligned"); + NL_SET_ERR_MSG_FMT(extack, + "dmabuf sg entry (addr=3D%pad, len=3D%zu) not aligned to niov size= %zu", + &dma_addr, len, niov_size); goto err_free_chunks; } =20 @@ -288,7 +295,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *vd= ev, =20 owner->area.base_virtual =3D virtual; owner->base_dma_addr =3D dma_addr; - owner->area.num_niovs =3D len / PAGE_SIZE; + owner->area.num_niovs =3D len >> niov_shift; owner->binding =3D binding; =20 err =3D gen_pool_add_owner(binding->chunk_pool, dma_addr, @@ -313,7 +320,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *vd= ev, page_pool_set_dma_addr_netmem(net_iov_to_netmem(niov), net_devmem_get_dma_addr(niov)); if (direction =3D=3D DMA_TO_DEVICE) - binding->tx_vec[owner->area.base_virtual / PAGE_SIZE + i] =3D niov; + binding->tx_vec[(owner->area.base_virtual >> niov_shift) + i] =3D niov; } =20 virtual +=3D len; @@ -430,13 +437,15 @@ struct net_iov * net_devmem_get_niov_at(struct net_devmem_dmabuf_binding *binding, size_t virt_addr, size_t *off, size_t *size) { + size_t niov_size =3D 1UL << binding->niov_shift; + if (virt_addr >=3D binding->dmabuf->size) return NULL; =20 - *off =3D virt_addr % PAGE_SIZE; - *size =3D PAGE_SIZE - *off; + *off =3D virt_addr & (niov_size - 1); + *size =3D niov_size - *off; =20 - return binding->tx_vec[virt_addr / PAGE_SIZE]; + return binding->tx_vec[virt_addr >> binding->niov_shift]; } =20 /*** "Dmabuf devmem memory provider" ***/ @@ -454,7 +463,7 @@ int mp_dmabuf_devmem_init(struct page_pool *pool) pool->dma_sync =3D false; pool->dma_sync_for_cpu =3D false; =20 - if (pool->p.order !=3D 0) + if (pool->p.order !=3D binding->niov_shift - PAGE_SHIFT) return -E2BIG; =20 net_devmem_dmabuf_binding_get(binding); diff --git a/net/core/devmem.h b/net/core/devmem.h index 3852a56036cb..4a293a7d1149 100644 --- a/net/core/devmem.h +++ b/net/core/devmem.h @@ -71,6 +71,8 @@ struct net_devmem_dmabuf_binding { */ struct net_iov **tx_vec; =20 + unsigned int niov_shift; + struct work_struct unbind_w; }; =20 @@ -93,7 +95,8 @@ struct net_devmem_dmabuf_binding * net_devmem_bind_dmabuf(struct net_device *dev, void *vdev, struct device *dma_dev, enum dma_data_direction direction, - unsigned int dmabuf_fd, struct netdev_nl_sock *priv, + unsigned int dmabuf_fd, unsigned int niov_shift, + struct netdev_nl_sock *priv, struct netlink_ext_ack *extack); struct net_devmem_dmabuf_binding *net_devmem_lookup_dmabuf(u32 id); void net_devmem_unbind_dmabuf(struct net_devmem_dmabuf_binding *binding); @@ -122,10 +125,11 @@ static inline u32 net_devmem_iov_binding_id(const str= uct net_iov *niov) =20 static inline unsigned long net_iov_virtual_addr(const struct net_iov *nio= v) { - struct net_iov_area *owner =3D net_iov_owner(niov); + struct dmabuf_genpool_chunk_owner *co =3D + net_devmem_iov_to_chunk_owner(niov); =20 - return owner->base_virtual + - ((unsigned long)net_iov_idx(niov) << PAGE_SHIFT); + return net_iov_owner(niov)->base_virtual + + ((unsigned long)net_iov_idx(niov) << co->binding->niov_shift); } =20 static inline bool @@ -175,6 +179,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *vd= ev, struct device *dma_dev, enum dma_data_direction direction, unsigned int dmabuf_fd, + unsigned int niov_shift, struct netdev_nl_sock *priv, struct netlink_ext_ack *extack) { diff --git a/net/core/netdev-genl-gen.c b/net/core/netdev-genl-gen.c index d18c89b5a6c7..f83790341eae 100644 --- a/net/core/netdev-genl-gen.c +++ b/net/core/netdev-genl-gen.c @@ -11,6 +11,7 @@ =20 #include #include +#include =20 /* Integer value ranges */ static const struct netlink_range_validation netdev_a_page_pool_id_range = =3D { @@ -27,6 +28,11 @@ static const struct netlink_range_validation netdev_a_na= pi_defer_hard_irqs_range .max =3D S32_MAX, }; =20 +static const struct netlink_range_validation netdev_a_dmabuf_rx_page_size_= range =3D { + .min =3D PAGE_SIZE, + .max =3D U32_MAX, +}; + /* Common nested types */ const struct nla_policy netdev_lease_nl_policy[NETDEV_A_LEASE_NETNS_ID + 1= ] =3D { [NETDEV_A_LEASE_IFINDEX] =3D NLA_POLICY_MIN(NLA_U32, 1), @@ -106,10 +112,11 @@ static const struct nla_policy netdev_qstats_get_nl_p= olicy[NETDEV_A_QSTATS_SCOPE }; =20 /* NETDEV_CMD_BIND_RX - do */ -static const struct nla_policy netdev_bind_rx_nl_policy[NETDEV_A_DMABUF_FD= + 1] =3D { +static const struct nla_policy netdev_bind_rx_nl_policy[NETDEV_A_DMABUF_RX= _PAGE_SIZE + 1] =3D { [NETDEV_A_DMABUF_IFINDEX] =3D NLA_POLICY_MIN(NLA_U32, 1), [NETDEV_A_DMABUF_FD] =3D { .type =3D NLA_U32, }, [NETDEV_A_DMABUF_QUEUES] =3D NLA_POLICY_NESTED(netdev_queue_id_nl_policy), + [NETDEV_A_DMABUF_RX_PAGE_SIZE] =3D NLA_POLICY_FULL_RANGE(NLA_U32, &netdev= _a_dmabuf_rx_page_size_range), }; =20 /* NETDEV_CMD_NAPI_SET - do */ @@ -219,7 +226,7 @@ static const struct genl_split_ops netdev_nl_ops[] =3D { .cmd =3D NETDEV_CMD_BIND_RX, .doit =3D netdev_nl_bind_rx_doit, .policy =3D netdev_bind_rx_nl_policy, - .maxattr =3D NETDEV_A_DMABUF_FD, + .maxattr =3D NETDEV_A_DMABUF_RX_PAGE_SIZE, .flags =3D GENL_UNS_ADMIN_PERM | GENL_CMD_CAP_DO, }, { diff --git a/net/core/netdev-genl-gen.h b/net/core/netdev-genl-gen.h index d71b435d72c1..3fae88e8f5c5 100644 --- a/net/core/netdev-genl-gen.h +++ b/net/core/netdev-genl-gen.h @@ -12,6 +12,7 @@ =20 #include #include +#include =20 /* Common nested types */ extern const struct nla_policy netdev_lease_nl_policy[NETDEV_A_LEASE_NETNS= _ID + 1]; diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c index c15d8d4ca1f8..0eea4ee22f24 100644 --- a/net/core/netdev-genl.c +++ b/net/core/netdev-genl.c @@ -1013,6 +1013,7 @@ netdev_nl_get_dma_dev(struct net_device *netdev, unsi= gned long *rxq_bitmap, int netdev_nl_bind_rx_doit(struct sk_buff *skb, struct genl_info *info) { struct net_devmem_dmabuf_binding *binding; + unsigned int niov_shift =3D PAGE_SHIFT; u32 ifindex, dmabuf_fd, rxq_idx; struct netdev_nl_sock *priv; struct net_device *netdev; @@ -1030,6 +1031,18 @@ int netdev_nl_bind_rx_doit(struct sk_buff *skb, stru= ct genl_info *info) ifindex =3D nla_get_u32(info->attrs[NETDEV_A_DEV_IFINDEX]); dmabuf_fd =3D nla_get_u32(info->attrs[NETDEV_A_DMABUF_FD]); =20 + if (info->attrs[NETDEV_A_DMABUF_RX_PAGE_SIZE]) { + u32 rx_page_size =3D nla_get_u32(info->attrs[NETDEV_A_DMABUF_RX_PAGE_SIZ= E]); + + if (!is_power_of_2(rx_page_size)) { + NL_SET_ERR_MSG_ATTR(info->extack, + info->attrs[NETDEV_A_DMABUF_RX_PAGE_SIZE], + "rx-page-size must be a power of 2"); + return -EINVAL; + } + niov_shift =3D ilog2(rx_page_size); + } + priv =3D genl_sk_priv_get(&netdev_nl_family, NETLINK_CB(skb).sk); if (IS_ERR(priv)) return PTR_ERR(priv); @@ -1080,7 +1093,8 @@ int netdev_nl_bind_rx_doit(struct sk_buff *skb, struc= t genl_info *info) } =20 binding =3D net_devmem_bind_dmabuf(netdev, NULL, dma_dev, DMA_FROM_DEVICE, - dmabuf_fd, priv, info->extack); + dmabuf_fd, niov_shift, priv, + info->extack); if (IS_ERR(binding)) { err =3D PTR_ERR(binding); goto err_rxq_bitmap; @@ -1221,7 +1235,7 @@ int netdev_nl_bind_tx_doit(struct sk_buff *skb, struc= t genl_info *info) binding =3D net_devmem_bind_dmabuf(bind_dev, bind_dev !=3D netdev ? netdev : NULL, dma_dev, DMA_TO_DEVICE, dmabuf_fd, - priv, info->extack); + PAGE_SHIFT, priv, info->extack); if (IS_ERR(binding)) { err =3D PTR_ERR(binding); goto err_unlock_bind_dev; diff --git a/tools/include/uapi/linux/netdev.h b/tools/include/uapi/linux/n= etdev.h index 2f3ab75e8cc0..35ff083221c7 100644 --- a/tools/include/uapi/linux/netdev.h +++ b/tools/include/uapi/linux/netdev.h @@ -219,6 +219,7 @@ enum { NETDEV_A_DMABUF_QUEUES, NETDEV_A_DMABUF_FD, NETDEV_A_DMABUF_ID, + NETDEV_A_DMABUF_RX_PAGE_SIZE, =20 __NETDEV_A_DMABUF_MAX, NETDEV_A_DMABUF_MAX =3D (__NETDEV_A_DMABUF_MAX - 1) --=20 2.53.0-Meta From nobody Sat Jul 25 06:02:56 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16BB245DF78 for ; Fri, 24 Jul 2026 21:21:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784928110; cv=none; b=onat0OAmhiZmXno+r1O2td+qAsIG0Vp4d1kJ1QO5cwRzT9SZjwxQ1eSfzMbqVVDq+a5e724QPckR3cEpDDPTS1aD68LMNEc8C9puFf5Ljp/iqIoZ6/7eLu4R/vKj6Y2/fSM7g7jndR1qt7hO3WlKNFQzrw9tYT5i3Du91QVtrwA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784928110; c=relaxed/simple; bh=NRtHEkVZJAVGq67dgJbG7XlxpkURYn25ZP++g1FdIE4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fs/icyUKx4e4rGyrEE2f4CziFF+2GSB5P9R9cqcmQHNlLZWHZI/sAM+3fBVHXxEr7X+8rXlL9cfTwsEU/WDK/eDqe9QCH/MCWXPL6L/VtlYKyxZSuBJ5niYwrYh22cIHUz/2kj36HPBzaHKJlPmtjpJvf9I6sqeLMQF7HF2aiAQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Q8n5NaaW; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q8n5NaaW" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-845b6d9bf39so940537b3a.1 for ; Fri, 24 Jul 2026 14:21:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784928108; x=1785532908; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=bwbrRJxFtRMglJHsKStZHneRkjfPxcqBpbuMqDOPwOI=; b=Q8n5NaaWwcIMBjq4rUlizDWClldOofwauDxPQSgtmkkUBXGE0HsiTagy1Fww+54HQR B8jlJT0lUvRyEWCfc6YpenBH7dbXvCMHh0dwsy+uvGcNe6+qvjukG/WpVDeY72RZGMWA HCQV27/V9qx8KG0nkNgYMm4mU2EGi+aLR3uKZWrSWBHyebYUDnVzQRK1uyOGmx8xvSvc s/IfIn/tHo73qRCqYAWGnNMRTy60xi3Bj6MyuVbVpNkE3xWOX1EvsIVX/d3OHWCZQCGk Zq3Fzvn8IlNgg4QZaGMJ/rFjiobuzheOeom3qkf3Td6PkT+BPQJjzRcAx4QRkuI/BgRZ WLJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784928108; x=1785532908; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=bwbrRJxFtRMglJHsKStZHneRkjfPxcqBpbuMqDOPwOI=; b=BDkEh8T1LtQqK6isIwGPal5tKKeK39oPyk3p4gpttrqs/4oDFDAxNDJpue0ftJ097s AadMTtAqFMXmbqEbCgtzZKWZ9GGgRenwGugMTVO5cw9sLAZjY1moD+gbFgeBPLHvufpN r7svBUj4yNlYLZDfUsOluVfSPGO0a40At+mG2jiHHHGIxMdGSEq8hak1kEsHKWHNYOiQ IqrwVgz7xGsALMvAj5kAHo6HBsrApKthGzFY51RvoiJ9WDt5dJIgtW/dIGeS4L7voIuZ XY4o9lXMzjWacuC3zIT4LsKZL29fbbYkiLzgoMI4vym7uu8yIwWkA3l9Lz0KIowWJxJ3 cq4A== X-Forwarded-Encrypted: i=1; AHgh+RpZcmvvqepzsvSNM08B2YTGIBO1DdwWmV7wa5Ej2vysLvpMp6JgA5eGd5erqWUiujOEoB+OQBce64dnWqk=@vger.kernel.org X-Gm-Message-State: AOJu0YzvOnYPqju3Cr9ux5SVxizNAq4Oxt3NXhDWUyCfTcqs9dGN4cAI cjEIpalDDoYCHTNOpZpoIWhSn6NouFYurbccjbE5xpGUFkY1Q+qBr0hB X-Gm-Gg: AR+sD13EEE7/ph96PHlXtNcV1x0nw2oHHm0bQpfKe6tWrSFnzyvCs+BQQtkCGtc5b+8 +shidJqGgEuSxDMzFyjiRtU5hqiw9x5+gVnHxkIx6Ai35HujYABRzMd/uaYBGRtavWfdlVDeHK+ qVYTbHqyClTpPul7zwjdJlH+277Uj8fEQZOQaTLSxfKlTOx5TkOgvSnH2K3+aQves9BPTrChvTU QUv0gb8jplTQshldyWcdDl+Tfb7G08d8dCn9rRMfhLnwiPxShFedcWNWIvfRxm6VlCc4G/JhqI/ EkTUUTcIJNZUy7hSPUJ2fh7huu6saYKZxfuwrfJXFfkyZew2VfSbj5Bo/gjVLRl4GQzE8fBdwh7 Uq9FNW3s2iZ9phCFB4OUwdFBx5p+sizrpgnfzxlRaI1IvfcuDb88+PNeSFB6I66ouWlII7tYl X-Received: by 2002:a05:6a00:8007:b0:848:40f5:ff5e with SMTP id d2e1a72fcca58-84e4e4283b9mr1491520b3a.37.1784928108422; Fri, 24 Jul 2026 14:21:48 -0700 (PDT) Received: from localhost ([2a03:2880:ff:8::]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e5341d68asm411468b3a.46.2026.07.24.14.21.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jul 2026 14:21:46 -0700 (PDT) From: Bobby Eshleman Date: Fri, 24 Jul 2026 14:21:16 -0700 Subject: [PATCH net-next v6 2/3] selftests/net: ncdevmem: add -b option to set rx-page-size on bind Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-tcpdm-large-niovs-v6-2-7975f62cdfbb@meta.com> References: <20260724-tcpdm-large-niovs-v6-0-7975f62cdfbb@meta.com> In-Reply-To: <20260724-tcpdm-large-niovs-v6-0-7975f62cdfbb@meta.com> To: Donald Hunter , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Simon Horman , Andrew Lunn , Gerd Hoffmann , Vivek Kasireddy , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-kselftest@vger.kernel.org, sdf@fomichev.me, razor@blackwall.org, daniel@iogearbox.net, almasrymina@google.com, matttbe@kernel.org, skhawaja@google.com, dw@davidwei.uk, Joe Damato , Bobby Eshleman X-Mailer: b4 0.14.3 From: Bobby Eshleman Add -b to request a non-default niov size via NETDEV_A_DMABUF_RX_PAGE_SIZE. When the value exceeds PAGE_SIZE, udmabuf_alloc() switches to an MFD_HUGETLB-backed memfd so each 2 MB hugepage produces one naturally-aligned sg entry. Signed-off-by: Bobby Eshleman Acked-by: Stanislav Fomichev --- tools/testing/selftests/drivers/net/hw/ncdevmem.c | 36 +++++++++++++++++++= ++-- 1 file changed, 33 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/drivers/net/hw/ncdevmem.c b/tools/test= ing/selftests/drivers/net/hw/ncdevmem.c index d96e8a3b5a65..ffe8eb05fac0 100644 --- a/tools/testing/selftests/drivers/net/hw/ncdevmem.c +++ b/tools/testing/selftests/drivers/net/hw/ncdevmem.c @@ -40,6 +40,7 @@ =20 #include #include +#include #include #include #include @@ -61,6 +62,7 @@ #include =20 #include +#include #include #include #include @@ -79,6 +81,7 @@ #define PAGE_SHIFT 12 #define TEST_PREFIX "ncdevmem" #define NUM_PAGES 16000 +#define MB(x) ((x) << 20) =20 #ifndef MSG_SOCK_DEVMEM #define MSG_SOCK_DEVMEM 0x2000000 @@ -100,6 +103,7 @@ static unsigned int dmabuf_id; static uint32_t tx_dmabuf_id; static int waittime_ms =3D 500; static bool fail_on_linear; +static uint32_t rx_page_size; =20 /* System state loaded by current_config_load() */ #define MAX_FLOWS 8 @@ -142,6 +146,7 @@ static struct memory_buffer *udmabuf_alloc(size_t size) { struct udmabuf_create create; struct memory_buffer *ctx; + unsigned int memfd_flags; int ret; =20 ctx =3D malloc(sizeof(*ctx)); @@ -156,9 +161,14 @@ static struct memory_buffer *udmabuf_alloc(size_t size) goto err_free_ctx; } =20 - ctx->memfd =3D memfd_create("udmabuf-test", MFD_ALLOW_SEALING); + memfd_flags =3D MFD_ALLOW_SEALING; + if (rx_page_size > getpagesize()) + memfd_flags |=3D MFD_HUGETLB | MFD_HUGE_2MB; + + ctx->memfd =3D memfd_create("udmabuf-test", memfd_flags); if (ctx->memfd < 0) { - pr_err("[skip,no-memfd]"); + pr_err("[skip,no-memfd%s]", + (memfd_flags & MFD_HUGETLB) ? " (need hugepages)" : ""); goto err_close_dev; } =20 @@ -168,6 +178,11 @@ static struct memory_buffer *udmabuf_alloc(size_t size) goto err_close_memfd; } =20 + if (memfd_flags & MFD_HUGETLB) { + size =3D roundup(size, MB(2)); + ctx->size =3D size; + } + ret =3D ftruncate(ctx->memfd, size); if (ret =3D=3D -1) { pr_err("[FAIL,memfd-truncate]"); @@ -699,6 +714,8 @@ static int bind_rx_queue(unsigned int ifindex, unsigned= int dmabuf_fd, netdev_bind_rx_req_set_ifindex(req, ifindex); netdev_bind_rx_req_set_fd(req, dmabuf_fd); __netdev_bind_rx_req_set_queues(req, queues, n_queue_index); + if (rx_page_size) + netdev_bind_rx_req_set_rx_page_size(req, rx_page_size); =20 rsp =3D netdev_bind_rx(*ys, req); if (!rsp) { @@ -1411,7 +1428,7 @@ int main(int argc, char *argv[]) int is_server =3D 0, opt; int ret, err =3D 1; =20 - while ((opt =3D getopt(argc, argv, "Lls:c:p:v:q:t:f:z:n")) !=3D -1) { + while ((opt =3D getopt(argc, argv, "Lls:c:p:v:q:t:f:z:nb:")) !=3D -1) { switch (opt) { case 'L': fail_on_linear =3D true; @@ -1446,6 +1463,19 @@ int main(int argc, char *argv[]) case 'n': skip_config =3D 1; break; + case 'b': { + unsigned long val; + + errno =3D 0; + val =3D strtoul(optarg, NULL, 0); + if ((val =3D=3D ULONG_MAX && errno =3D=3D ERANGE) || + val > UINT32_MAX) { + pr_err("invalid rx_page_size: %s", optarg); + return 1; + } + rx_page_size =3D val; + break; + } case '?': fprintf(stderr, "unknown option: %c\n", optopt); break; --=20 2.53.0-Meta From nobody Sat Jul 25 06:02:56 2026 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9001646A5F7 for ; Fri, 24 Jul 2026 21:21:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784928112; cv=none; b=UdWuentZwh4A7KhUH0zgo3bVePZ6nF9utKrcT91JNHik9WjFupUq/4MxqqQlCvyMLooP0hV2XPpfm8cdjyQnw9onrmtCPxh6d7/ZrknE1KEz28tbASjnay8AMaqv/qQX1S/eHXHRYb1l2nsrhwqOel3xX/E/pgOZ+toQxr6oqqg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784928112; c=relaxed/simple; bh=RG1zMOAjscVettyJyIUMDCMbHN2ZmcaGTf0gdEBIwgA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=i5S9hS9kAhiVMU5FM1Hv16ZKxNQS1SYwZjGmVe9IOdx4RlqdqybxaCzh87/f5U3Oen+8bdVuMMfPDMXqAu1KNNzQjD0bFP6SLzr7ivKuqmsKiDTlM8kg7VFu7Q8IhFoyoJYsTnAJ2MEh6/6P6ip/LdzqOzLdk62340AOKMyll7M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=pYuhkWGE; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pYuhkWGE" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-2caced6038eso12570015ad.0 for ; Fri, 24 Jul 2026 14:21:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784928110; x=1785532910; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=wVE9mezfwx2DgerLJfcGNMZoP6+QarHJ8jyXD0AzZd8=; b=pYuhkWGEdMplHguPsYCrXBQKufIJDypB6WhFgtenVoIpsYKo2PvULNfdiSUZ5XZsOn g7xu0CoZk9JmQZAjmLDAW3nEDu8oITArYrw5EkWgM/8eCtSwWTDjiSEiQzP0uulc5rEz gZt+Iy9F/4fvZW8xyXPIvpuxP0NjO1wvVusw3KHZYv5mHiMLasZQvjgoOAi6c6uHkPMT SPuWtDOJLG5UbtrA4EnIK37Sqy8YtMidBWQrV5RnrCscPLTMpY+D0/12SE812pOF61cM ZboJlN6/dZHGCZlVyB1/mMrHdf6DdLSs2X9owLxRQvqSRpKTvUzpDXGPdPKxvwFnEqo0 QowA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784928110; x=1785532910; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=wVE9mezfwx2DgerLJfcGNMZoP6+QarHJ8jyXD0AzZd8=; b=cmwObqowlc8tFPKXOyWd2SSl8mofM2boapCIDYxpG4hSRXzjtOSDzhaWUdCvpxi0K0 lxc+EtuvQbadBin9jK3GeoiCxt0AyLNl/aqsO/gWoGa5fQ8ATfxwm7DuP9Wply6rRjTw yigYhLcsDz4BGJVvMWbl9I1nUlIQErcZTBJSweYF2/7QDnv1YVJcfhK7lS91fQyKuwE1 bnNEbbrJd67zsvxP2KXTeHacsIzBB19l4MX4nhKygiSrsJ6r9buhbsbQ4RlfAcJ8yVgQ lt7vglqOXPUXqMSSp0N1m7k/WQKP/w8a1LtJ6AS/4VlfaC3lOoLt2iQXVMpg8k1+aW9Q kS1w== X-Forwarded-Encrypted: i=1; AHgh+Rrw8L80fzde3XEdKsDIUBWFulSj8I5RILmyauF4eOSjBZNgJ/TCYNj57vGJ/9Qy5DxwuP01q3jXRJz/xsA=@vger.kernel.org X-Gm-Message-State: AOJu0Yxz7DSyyAsL88bf5keNeYhd/KNKJEFwEAuef8Jbeb37Pm7KEi9/ bk3YbSYKsxhUm2Sd9Wd3yms0Y0L4ehsCIBAmlz6Vhwd4xice2zyX81pL X-Gm-Gg: AR+sD10vizyoCj5UQf7btcDctn6MvbaZqj84Gi4AIZcWFuMqRVb2mQQ6/mqmtf58wU+ CoMg778EkIoW853X6qNb7tKhdqCVFagaoRaOED6vkibfzgBtZzEUu1rb7DQiyOFCw9O59wbApZt c95DlTl0TjlyuTiaJMgpZAmES8er3RijHh/Ev7YjyA1URnNn86hn0p3pXmkqw2X6rx3jErTB27o 5SXgzpkRrQ5HDSZpYK9PzC3QNega4VFb8I+jIXj0hHnA01QA9G5NRdhynpwoLE00QjO4ykTcAuX WnpjDaEAagxErPQAea3pAPvoB6ZY3GpznkvBTXul9ubsFkAlQAxZptReTcM3lg8ILqLxdZR1KZ0 Bj34X78/L8j4ALcQiSyhqGKjM0bKBubiVMubfdbacE8L8cY/olIY+zlhCcGwGtecDVwDdrlLIFI s8q5XV+nAF1vPhmQVtUNnMeIAAUX1y5I/CkovS6+vU3wCAexz6nnqZviFQ X-Received: by 2002:a17:903:1c2:b0:2bc:e299:4b3f with SMTP id d9443c01a7336-2cfd7953a96mr18792475ad.10.1784928109829; Fri, 24 Jul 2026 14:21:49 -0700 (PDT) Received: from localhost ([2a03:2880:ff:73::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cfde81dcb3sm330815ad.82.2026.07.24.14.21.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jul 2026 14:21:49 -0700 (PDT) From: Bobby Eshleman Date: Fri, 24 Jul 2026 14:21:17 -0700 Subject: [PATCH net-next v6 3/3] selftests/net: devmem.py: add check_rx_large_niov Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-tcpdm-large-niovs-v6-3-7975f62cdfbb@meta.com> References: <20260724-tcpdm-large-niovs-v6-0-7975f62cdfbb@meta.com> In-Reply-To: <20260724-tcpdm-large-niovs-v6-0-7975f62cdfbb@meta.com> To: Donald Hunter , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Simon Horman , Andrew Lunn , Gerd Hoffmann , Vivek Kasireddy , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , Shuah Khan Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-kselftest@vger.kernel.org, sdf@fomichev.me, razor@blackwall.org, daniel@iogearbox.net, almasrymina@google.com, matttbe@kernel.org, skhawaja@google.com, dw@davidwei.uk, Joe Damato , Bobby Eshleman X-Mailer: b4 0.14.3 From: Bobby Eshleman Add a new devmem test case for binding the dmabuf with rx-page-size=3D16K. The test sweeps RX payload sizes straddling the niov boundary to cover the sub-niov, exact-niov, and multi-niov RX paths. Silence pylint invalid-name (`with open() as f`) and too-many-arguments (ncdevmem_rx grew to 6 args) at file scope. Signed-off-by: Bobby Eshleman Acked-by: Stanislav Fomichev --- tools/testing/selftests/drivers/net/hw/devmem.py | 11 +++- .../testing/selftests/drivers/net/hw/devmem_lib.py | 59 ++++++++++++++++++= +++- .../testing/selftests/drivers/net/hw/nk_devmem.py | 10 +++- 3 files changed, 74 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/drivers/net/hw/devmem.py b/tools/testi= ng/selftests/drivers/net/hw/devmem.py index 031cf9905f65..82c11ffc4add 100755 --- a/tools/testing/selftests/drivers/net/hw/devmem.py +++ b/tools/testing/selftests/drivers/net/hw/devmem.py @@ -2,7 +2,8 @@ # SPDX-License-Identifier: GPL-2.0 =20 from os import path -from devmem_lib import setup_test, run_rx, run_tx, run_tx_chunks, run_rx_h= ds +from devmem_lib import (setup_test, run_rx, run_tx, run_tx_chunks, run_rx_= hds, + run_rx_large_niov) from lib.py import ksft_run, ksft_exit, ksft_disruptive from lib.py import NetDrvEpEnv =20 @@ -30,11 +31,17 @@ def check_rx_hds(cfg) -> None: run_rx_hds(cfg) =20 =20 +def check_rx_large_niov(cfg) -> None: + """Run the devmem RX test with rx-page-size =3D 16 KiB.""" + run_rx_large_niov(cfg) + + def main() -> None: """Run the devmem test cases.""" with NetDrvEpEnv(__file__) as cfg: setup_test(cfg, path.abspath(path.dirname(__file__) + "/ncdevmem")) - ksft_run([check_rx, check_tx, check_tx_chunks, check_rx_hds], + ksft_run([check_rx, check_tx, check_tx_chunks, check_rx_hds, + check_rx_large_niov], args=3D(cfg,)) ksft_exit() =20 diff --git a/tools/testing/selftests/drivers/net/hw/devmem_lib.py b/tools/t= esting/selftests/drivers/net/hw/devmem_lib.py index 0921ff03eb81..6c932018b463 100644 --- a/tools/testing/selftests/drivers/net/hw/devmem_lib.py +++ b/tools/testing/selftests/drivers/net/hw/devmem_lib.py @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 +# pylint: disable=3Dinvalid-name,too-many-arguments """Shared helpers for devmem TCP selftests.""" =20 import re @@ -8,7 +9,7 @@ from lib.py import (bkg, cmd, defer, ethtool, rand_port, wa= it_port_listen, NetdevFamily) =20 =20 -def require_devmem(cfg): +def require_devmem(cfg, rx_page_size=3D0): """Probe ncdevmem on cfg.ifname and SKIP the test if devmem isn't supp= orted.""" if not hasattr(cfg, "devmem_probed"): probe_command =3D f"{cfg.bin_local} -f {cfg.ifname}" @@ -18,6 +19,19 @@ def require_devmem(cfg): if not cfg.devmem_supported: raise KsftSkipEx("Test requires devmem support") =20 + if rx_page_size > 0: + if not hasattr(cfg, "devmem_rx_page_size_probed"): + cfg.devmem_rx_page_size_probed =3D {} + + if rx_page_size not in cfg.devmem_rx_page_size_probed: + probe_command =3D f"{cfg.bin_local} -f {cfg.ifname} -b {rx_pag= e_size}" + cfg.devmem_rx_page_size_probed[rx_page_size] =3D \ + cmd(probe_command, fail=3DFalse, shell=3DTrue).ret =3D=3D 0 + + if not cfg.devmem_rx_page_size_probed[rx_page_size]: + raise KsftSkipEx( + f"Test requires devmem rx-page-size=3D{rx_page_size} suppo= rt") + =20 def configure_nic(cfg): """Channels, rings, RSS, queue lease for netkit devmem.""" @@ -76,7 +90,8 @@ def set_flow_rule(cfg, port): return int(re.search(r'ID (\d+)', output).group(1)) =20 =20 -def ncdevmem_rx(cfg, port, verify=3DTrue, fail_on_linear=3DFalse, flow_ste= er=3DFalse): +def ncdevmem_rx(cfg, port, verify=3DTrue, fail_on_linear=3DFalse, flow_ste= er=3DFalse, + rx_page_size=3D0): """Build the ncdevmem RX listener command.""" if hasattr(cfg, 'netns'): flow_rule_id =3D set_flow_rule(cfg, port) @@ -96,6 +111,8 @@ def ncdevmem_rx(cfg, port, verify=3DTrue, fail_on_linear= =3DFalse, flow_steer=3DFalse): extras.append("-v 7") if fail_on_linear: extras.append("-L") + if rx_page_size > 0: + extras.append(f"-b {rx_page_size}") =20 parts =3D [cfg.bin_local, "-l", f"-f {ifname}", f"-s {addr}", f"-p {port}", *extras] @@ -202,6 +219,44 @@ def run_tx_chunks(cfg): ksft_eq(socat.stdout.strip(), "hello\nworld") =20 =20 +def _restore_nr_hugepages(hp_file, nr_hugepages): + with open(hp_file, 'w', encoding=3D'utf-8') as f: + f.write(str(nr_hugepages)) + + +def run_rx_large_niov(cfg): + """Run the devmem RX test with a large niov (rx-page-size > PAGE_SIZE). + + Sweep payload sizes that straddle the niov boundary: below, equal to, + and above rx_page_size, to exercise sub-niov, exact-niov, and multi-ni= ov + RX paths. + """ + hp_file =3D "/proc/sys/vm/nr_hugepages" + with open(hp_file, 'r+', encoding=3D'utf-8') as f: + nr_hugepages =3D int(f.read().strip()) + if nr_hugepages < 64: + f.seek(0) + f.write("64") + defer(_restore_nr_hugepages, hp_file, nr_hugepages) + require_devmem(cfg, rx_page_size=3D16384) + configure_nic(cfg) + netns =3D getattr(cfg, "netns", None) + + for size in [1024, 4096, 8192, 16384, 32768, 65536]: + port =3D rand_port() + socat =3D socat_send(cfg, port) + listen_cmd =3D ncdevmem_rx(cfg, port, + flow_steer=3Dnot netns, + rx_page_size=3D16384) + data_pipe =3D (f"yes $(echo -e \x01\x02\x03\x04\x05\x06) | " + f"head -c {size} | {socat}") + with bkg(listen_cmd, exit_wait=3DTrue, ns=3Dnetns) as ncdevmem: + wait_port_listen(port, proto=3D"tcp", ns=3Dnetns) + cmd(data_pipe, host=3Dcfg.remote, shell=3DTrue) + ksft_eq(ncdevmem.ret, 0, + f"large-niov failed for payload size {size}") + + def run_rx_hds(cfg): """Run the HDS test by running devmem RX across a segment size sweep."= "" require_devmem(cfg) diff --git a/tools/testing/selftests/drivers/net/hw/nk_devmem.py b/tools/te= sting/selftests/drivers/net/hw/nk_devmem.py index 300ed2a70ab4..61c6f31f01e5 100755 --- a/tools/testing/selftests/drivers/net/hw/nk_devmem.py +++ b/tools/testing/selftests/drivers/net/hw/nk_devmem.py @@ -3,7 +3,8 @@ """Test devmem TCP with netkit.""" =20 import os -from devmem_lib import setup_test, run_rx, run_tx, run_tx_chunks, run_rx_h= ds +from devmem_lib import (setup_test, run_rx, run_tx, run_tx_chunks, run_rx_= hds, + run_rx_large_niov) from lib.py import ksft_run, ksft_exit, ksft_disruptive from lib.py import NetDrvContEnv =20 @@ -31,6 +32,11 @@ def check_nk_rx_hds(cfg) -> None: run_rx_hds(cfg) =20 =20 +def check_nk_rx_large_niov(cfg) -> None: + """Run the devmem RX large-niov test through netkit.""" + run_rx_large_niov(cfg) + + def main() -> None: """Run the netkit devmem test cases.""" with NetDrvContEnv(__file__, rxqueues=3D2, primary_rx_redirect=3DTrue)= as cfg: @@ -38,7 +44,7 @@ def main() -> None: os.path.join(os.path.dirname(os.path.abspath(__file__)), "ncdevmem")) ksft_run([check_nk_rx, check_nk_tx, check_nk_tx_chunks, - check_nk_rx_hds], args=3D(cfg,)) + check_nk_rx_hds, check_nk_rx_large_niov], args=3D(cfg,)) ksft_exit() =20 =20 --=20 2.53.0-Meta