From nobody Thu Sep 24 14:26:25 2026 Received: from mail-pj2-f5.google.com (mail-pj2-f5.google.com [74.125.227.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1EB851AFF8 for ; Tue, 22 Sep 2026 20:43:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790109853; cv=none; b=XbyPwouICDoP93+4Sje0B7dutzuIAAd6mGL63BQj0kFLusRxGxL1UKdhgurKRzsIDO2TFh9MZcG5wrkxE63LeWFILe20bLaA+OJoZIgOyY86lgrIomvcM73UDzyN7JCWlywqF7hruI0We68bGmZHXB0NBWemHb8vI69hC3JpJf8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790109853; c=relaxed/simple; bh=Q5TkZR39NYgiwWRYBslQEVd5kt9MNO8iAajWHVC2zJ0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cibQitg0nQUugUFz2NLu6J3o80Y2Yh+IzNA48OsIybBAHclr0S5nuuJBi0/Ok5Bq8dZ+R6LaM3BVwvLLkoaTQkQiZIaonj72l/1U5tDCptrJoDw3caM7tM3hziORIhEaEHwpgOhrOVJ2wJBY8MtZykWlHLpMQHUoygiAFUtvY+w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GULKsRoL; arc=none smtp.client-ip=74.125.227.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GULKsRoL" Received: by mail-pj2-f5.google.com with SMTP id 98e67ed59e1d1-3965683e9e9so167359a91.1 for ; Tue, 22 Sep 2026 13:43:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1790109831; x=1790714631; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=cxDLGte0XFPc20u8C8dvpS8ZTxM5ZLPAP2KRl1LHufI=; b=GULKsRoL3B1k5oOgZqEvQzT6kYYf+gJ1F6N4aKCxIFoxxrOicumRa3xneYsIakhvqL B2MaRSiIDD4kjUKGR7ljFPGYWgf28+egxyweSDkcFXMHo6sj9TI3GgtKmXqO7498KtOV vXpcBIMrZRUV7u/LWJniRvpL5tNCllyncQlaLoLRLH+3Iupk+2w30hjHP2Udflr5P2jF oKcNMLpHwUVD3Z3sv7Aj5cB1kxju3pM4HsCjqMzXlEa9g3skDpNqxqLJ3XZwMD4uQPHq xv9a3obnnlTMizaTRv1q5NRhPcsmaqTkY4WW0HNGrxkzcH9K29+UsmfpYhmsaB79R8Z5 L2TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790109831; x=1790714631; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=cxDLGte0XFPc20u8C8dvpS8ZTxM5ZLPAP2KRl1LHufI=; b=EzXQgm/hKrkzK0RY+td+WAx6Gm3Uj7K3jY5RwOtOc1gsDOqeZH5ZgHIutEwFl1RKk0 BKtgoL2IWr14l4vjuDgK1+1nOEAkWu5teHKn92jdsM6Qc6g9+wzeJLWV0P2yjJS1+TtO a5xuqSgUa5e/w4WWDRqJB85l7P47c+JyE9tz9Uabq7njMC2FxvADtfkASA/3eVDzVxtc SZNYxknlXUamGRP4srG6661FMAgqLn9bL+l/95jPZNCyQ2piM5jK6AbruS/b/Zy8NRnM CM6NbA/V6J//sDzx6dZqjVWcj1lwKunKA5ek3CBOyVOrMMGhUAm2ri5jku0OFhNRG1n9 qYNg== X-Forwarded-Encrypted: i=1; AKwUvByhll/kziB1q8D63giWQ6uS7gT7ZsMlOiNKcIwwym/IfIjN6KXwOvM8yDYfbGLlgNA61NKO7wC6bHJ+0zs=@vger.kernel.org X-Gm-Message-State: AFuF++l0ERVfU5mAET3B45vrK14J93jvWhYa7cfE2Em5b+QQsqOx0VZN PNe+0/FpnhYSwbqTDdBKRE0tk/msLS7mCzOEbE2XIlIzeNr1teK1LKXB X-Gm-Gg: AYBFou2m2B9vMO4BKKTqReSwxd5gVaACKrNYv8VlkcXw7GyOqT0iAsCgt3WL1irlXCz pC/QVIITNCJZucJij21JFNgxL1IJThAvFTHc8gN0KGWoId2L4f2plP+FhRlo4xX0DHIpPOP4bFr McCNs9eF1lEVNE+E/2zcJ1IaazB1QMIlKScwOQkCyrl0B5r1rw0RHfNhJrJtfOAweTlqy9XdvNN xxw18W+iBcc1F39zeJ5ZNjATuCanmJIlZ+9kKS6uvMFwF+LT3NCUamfknKMrkK0BfXCdQ24wTZW hSgGgxuK1Wrb9ji4VBYLJqXi84JhnSTUueDsML2+mSoQdd0Ce7g5lsMshNVxbErrn940Ov1Dzzt +wo7Vw1XFBvkipNOQMRlvA2ZkWoz78TKNiPLOXysEJBvKWu27fU3ocQT8u/xe1zxUtmaeDDuxo+ Kg9h1S6TSFStC+pcJEDkKoI3a72CX8zZWfXNm3QN2mQUsYqHpA4FyA5F0DcYxGg+w= X-Received: by 2002:a05:6a21:398f:b0:3c3:7179:a640 with SMTP id adf61e73a8af0-3ddf7ccd4cfmr658486637.3.1790109830841; Tue, 22 Sep 2026 13:43:50 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:5::]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-cc75f3d5ce8sm99133a12.17.2026.09.22.13.43.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 13:43:50 -0700 (PDT) From: Stanislav Fomichev X-Google-Original-From: Stanislav Fomichev To: netdev@vger.kernel.org Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, hawk@kernel.org, ilias.apalodimas@linaro.org, asml.silence@gmail.com, axboe@kernel.dk, sdf@fomichev.me, bobbyeshleman@meta.com, almasrymina@google.com, kaiyuanz@google.com, linux-kernel@vger.kernel.org, io-uring@vger.kernel.org Subject: [PATCH net-next 1/3] net: netmem: add net_iov_area freelist helpers Date: Tue, 22 Sep 2026 13:43:46 -0700 Message-ID: <20260922204348.717198-2-sdf@fomichev.me> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260922204348.717198-1-sdf@fomichev.me> References: <20260922204348.717198-1-sdf@fomichev.me> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" io_uring zero-copy receive and devmem both maintain a bounded LIFO for net_iovs in a contiguous area. Store the freelist in struct net_iov_area and provide common push and pop helpers. Leave synchronization to area owners. Keep devmem's area adjacent to its lock. Use u32 indices and counts, which halves devmem's freelist storage on 64-bit systems. Reject devmem areas with more than U32_MAX entries before narrowing the count. With 4 KiB chunks, the limit is almost 16 TiB. Signed-off-by: Stanislav Fomichev --- include/net/netmem.h | 28 ++++++++++++++++++++++++++- io_uring/zcrx.c | 25 +++++++++--------------- io_uring/zcrx.h | 4 ---- net/core/devmem.c | 46 ++++++++++++++++++++++---------------------- net/core/devmem.h | 7 +++---- 5 files changed, 62 insertions(+), 48 deletions(-) diff --git a/include/net/netmem.h b/include/net/netmem.h index bccacd21b6c3..da885d95ea63 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -101,10 +101,15 @@ struct net_iov { struct net_iov_area { /* Array of net_iovs for this area. */ struct net_iov *niovs; - size_t num_niovs; + + /* Stack of free net_iov indices. */ + u32 *freelist; =20 /* Offset into the dma-buf where this chunk starts. */ unsigned long base_virtual; + + u32 num_niovs; + u32 free_count; }; =20 static inline struct net_iov_area *net_iov_owner(const struct net_iov *nio= v) @@ -117,6 +122,27 @@ static inline unsigned int net_iov_idx(const struct ne= t_iov *niov) return niov - net_iov_owner(niov)->niovs; } =20 +static inline struct net_iov *net_iov_area_pop(struct net_iov_area *area) +{ + u32 idx; + + if (unlikely(!area->free_count)) + return NULL; + + idx =3D area->freelist[--area->free_count]; + return &area->niovs[idx]; +} + +static inline void net_iov_area_push(struct net_iov_area *area, + struct net_iov *niov) +{ + if (WARN_ON_ONCE(net_iov_owner(niov) !=3D area || + area->free_count >=3D area->num_niovs)) + return; + + area->freelist[area->free_count++] =3D net_iov_idx(niov); +} + /* Initialize a niov: stamp the owning area, the memory provider type. */ static inline void net_iov_init(struct net_iov *niov, diff --git a/io_uring/zcrx.c b/io_uring/zcrx.c index 86d580d4410d..fa6061127b9d 100644 --- a/io_uring/zcrx.c +++ b/io_uring/zcrx.c @@ -462,7 +462,7 @@ static void io_zcrx_free_area(struct io_zcrx_ifq *ifq, io_unaccount_mem(ifq->user, ifq->mm_account, area->mem.account_pages); =20 - kvfree(area->freelist); + kvfree(area->nia.freelist); kvfree(area->nia.niovs); kvfree(area->user_refs); kfree(area); @@ -548,9 +548,10 @@ static int __zcrx_create_area(struct io_zcrx_ifq *ifq, if (!area->nia.niovs) goto err; =20 - area->freelist =3D kvmalloc_array(nr_iovs, sizeof(area->freelist[0]), - GFP_KERNEL_ACCOUNT | __GFP_ZERO); - if (!area->freelist) + area->nia.freelist =3D kvmalloc_array(nr_iovs, + sizeof(area->nia.freelist[0]), + GFP_KERNEL_ACCOUNT | __GFP_ZERO); + if (!area->nia.freelist) goto err; =20 area->user_refs =3D kvmalloc_objs(area->user_refs[0], nr_iovs, @@ -562,7 +563,7 @@ static int __zcrx_create_area(struct io_zcrx_ifq *ifq, struct net_iov *niov =3D &area->nia.niovs[i]; =20 net_iov_init(niov, &area->nia, NET_IOV_IOURING); - area->freelist[i] =3D i; + area->nia.freelist[i] =3D i; atomic_set(&area->user_refs[i], 0); } =20 @@ -572,7 +573,7 @@ static int __zcrx_create_area(struct io_zcrx_ifq *ifq, goto err; } =20 - area->free_count =3D nr_iovs; + area->nia.free_count =3D nr_iovs; /* we're only supporting one area per ifq for now */ area->area_id =3D zcrx_next_area_id(ifq); area_reg->rq_area_token =3D zcrx_area_id_to_token(area->area_id); @@ -689,22 +690,14 @@ static void io_zcrx_return_niov_freelist(struct net_i= ov *niov) struct io_zcrx_ifq *ifq =3D area->ifq; =20 guard(spinlock_bh)(&ifq->alloc_lock); - if (WARN_ON_ONCE(area->free_count >=3D area->nia.num_niovs)) - return; - area->freelist[area->free_count++] =3D net_iov_idx(niov); + net_iov_area_push(&area->nia, niov); } =20 static struct net_iov *zcrx_get_free_niov(struct io_zcrx_area *area) { - unsigned niov_idx; - lockdep_assert_held(&area->ifq->alloc_lock); =20 - if (unlikely(!area->free_count)) - return NULL; - - niov_idx =3D area->freelist[--area->free_count]; - return &area->nia.niovs[niov_idx]; + return net_iov_area_pop(&area->nia); } =20 static void io_zcrx_return_niov(struct net_iov *niov) diff --git a/io_uring/zcrx.h b/io_uring/zcrx.h index d4a54b4e17fd..a923291defde 100644 --- a/io_uring/zcrx.h +++ b/io_uring/zcrx.h @@ -36,10 +36,6 @@ struct io_zcrx_area { bool is_mapped; u16 area_id; =20 - /* freelist */ - u32 free_count; - u32 *freelist; - struct io_zcrx_mem mem; }; =20 diff --git a/net/core/devmem.c b/net/core/devmem.c index a9d86b5a5588..c1c1872b88de 100644 --- a/net/core/devmem.c +++ b/net/core/devmem.c @@ -42,10 +42,10 @@ void __net_devmem_dmabuf_binding_free(struct work_struc= t *wq) { struct net_devmem_dmabuf_binding *binding =3D container_of(wq, typeof(*bi= nding), unbind_w); =20 - if (binding->freelist) - WARN(binding->free_count !=3D binding->area.num_niovs, - "destroying dmabuf binding with outstanding net_iovs: total=3D%zu, = free=3D%zu", - binding->area.num_niovs, binding->free_count); + if (binding->area.freelist) + WARN(binding->area.free_count !=3D binding->area.num_niovs, + "destroying dmabuf binding with outstanding net_iovs: total=3D%u, f= ree=3D%u", + binding->area.num_niovs, binding->area.free_count); =20 kvfree(binding->area.niovs); dma_buf_unmap_attachment_unlocked(binding->attachment, binding->sgt, @@ -54,7 +54,7 @@ void __net_devmem_dmabuf_binding_free(struct work_struct = *wq) dma_buf_put(binding->dmabuf); xa_destroy(&binding->bound_rxqs); percpu_ref_exit(&binding->ref); - kvfree(binding->freelist); + kvfree(binding->area.freelist); kvfree(binding->tx_vec); kfree(binding); } @@ -67,12 +67,10 @@ net_devmem_alloc_dmabuf_bulk(struct net_devmem_dmabuf_b= inding *binding, =20 spin_lock_bh(&binding->freelist_lock); =20 - count =3D min_t(size_t, count, binding->free_count); - for (i =3D 0; i < count; i++) { - struct net_iov *niov =3D binding->freelist[--binding->free_count]; - - netmems[i] =3D net_iov_to_netmem(niov); - } + count =3D min(count, binding->area.free_count); + for (i =3D 0; i < count; i++) + netmems[i] =3D + net_iov_to_netmem(net_iov_area_pop(&binding->area)); =20 spin_unlock_bh(&binding->freelist_lock); =20 @@ -84,12 +82,7 @@ void net_devmem_free_dmabuf(struct net_iov *niov) struct net_devmem_dmabuf_binding *binding =3D net_devmem_iov_binding(niov= ); =20 spin_lock_bh(&binding->freelist_lock); - if (WARN_ON_ONCE(binding->free_count >=3D binding->area.num_niovs)) { - spin_unlock_bh(&binding->freelist_lock); - return; - } - - binding->freelist[binding->free_count++] =3D niov; + net_iov_area_push(&binding->area, niov); spin_unlock_bh(&binding->freelist_lock); } =20 @@ -228,6 +221,12 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *v= dev, goto err_unmap; } =20 + if ((dmabuf->size >> niov_shift) > U32_MAX) { + err =3D -E2BIG; + NL_SET_ERR_MSG(extack, "dmabuf contains too many net_iovs"); + goto err_unmap; + } + binding->area.base_virtual =3D 0; binding->area.num_niovs =3D dmabuf->size >> niov_shift; if (direction =3D=3D DMA_TO_DEVICE) { @@ -239,10 +238,11 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *= vdev, } } else { spin_lock_init(&binding->freelist_lock); - binding->freelist =3D kvmalloc_array(binding->area.num_niovs, - sizeof(binding->freelist[0]), - GFP_KERNEL); - if (!binding->freelist) { + binding->area.freelist =3D + kvmalloc_array(binding->area.num_niovs, + sizeof(binding->area.freelist[0]), + GFP_KERNEL); + if (!binding->area.freelist) { err =3D -ENOMEM; goto err_unmap; } @@ -279,7 +279,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *vd= ev, if (direction =3D=3D DMA_TO_DEVICE) binding->tx_vec[niov_idx] =3D niov; else - binding->freelist[binding->free_count++] =3D niov; + net_iov_area_push(&binding->area, niov); dma_addr +=3D niov_size; } } @@ -297,7 +297,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *vd= ev, err_free_niovs: kvfree(binding->area.niovs); err_free_freelist: - kvfree(binding->freelist); + kvfree(binding->area.freelist); kvfree(binding->tx_vec); err_unmap: dma_buf_unmap_attachment_unlocked(binding->attachment, binding->sgt, diff --git a/net/core/devmem.h b/net/core/devmem.h index f71d9a2bcb3d..5edb763b89b2 100644 --- a/net/core/devmem.h +++ b/net/core/devmem.h @@ -17,6 +17,9 @@ struct netlink_ext_ack; =20 struct net_devmem_dmabuf_binding { struct net_iov_area area; + /* Protects area.freelist and area.free_count. */ + spinlock_t freelist_lock; + struct dma_buf *dmabuf; struct dma_buf_attachment *attachment; struct sg_table *sgt; @@ -57,10 +60,6 @@ struct net_devmem_dmabuf_binding { /* rxq's this binding is active on. */ struct xarray bound_rxqs; =20 - spinlock_t freelist_lock ____cacheline_aligned_in_smp; - size_t free_count; - struct net_iov **freelist; - /* ID of this binding. Globally unique to all bindings currently * active. */ --=20 2.53.0-Meta From nobody Thu Sep 24 14:26:25 2026 Received: from mail-pz2-f7.google.com (mail-pz2-f7.google.com [74.125.228.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0D5E4E2F0B for ; Tue, 22 Sep 2026 20:43:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790109853; cv=none; b=BDMAK/6NRJJobZP3hIu5Agv3vcOSAt1QLBzmsCI9KH5UjdgaKPxvF9qAJ/uSahLOHXlVyyH67/4F9GKJ7QKN9UhnpHZvvHIOUO4HKU7uR+INhabn+LkPr6n1UztdT8s5rI8ZJEmQ0eYEhnrA+IZRzRVQl/XPyRRxjFEKzVl72xs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790109853; c=relaxed/simple; bh=skCoc+C0hEQPVu18sCy3a39JKiwa3vnAiGPPaMNlYOU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fWcLypLBgr7HSEDowBXaGsm/cAb4T2qwZk/qzcBTXp4XqVwieN911Rn3dExlFT6Z2yfp+51/us4eUO2z8ATX2MQmU1G/g3o9HAJVdcby4RDRdzsL+z2m/59BGFEQ6HfioKhjVQA4WJDORG+CZo7UwMk2KT54VCLZf3LFgy+FHrs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XlfHmxgS; arc=none smtp.client-ip=74.125.228.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XlfHmxgS" Received: by mail-pz2-f7.google.com with SMTP id 41be03b00d2f7-cc7323540dcso17582a12.1 for ; Tue, 22 Sep 2026 13:43:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1790109832; x=1790714632; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Q2XSgCg/L+gcV4fc8+Srep5u/tQwTGnGLQScsN2YpZw=; b=XlfHmxgSl9Gx8kKDvp+jLClKQMTvp2SYLnbKzqwZ31Kb4Ja/CLHhgiqwBl+N0/0qMY Kqi0KRaktuTsVx+4Nj5+hv3XbUobKMoAdU3IL/ZTBakEsu3M+gRG5k8lYMt5f3GX2Cs7 dOqrTrTjAQDj1+WdKUC+oWp/xkigxsO+gWSj2cB3gEy/pH7KSReqVYK2Bfa5mElhKgho aXtOE8tl0onrPyj3ktK9suPZlnSkdkhJHf55WRWVk7EIyrvRg6zegZUaAPIjDDTxyAFh 7AwFZQXQdtP2FW5ZiqePHpyFRgZ3rL4jr9Zhi4RUwuy1YyNjsVj96b6XeYIDCjU8aBx1 Z4tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790109832; x=1790714632; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Q2XSgCg/L+gcV4fc8+Srep5u/tQwTGnGLQScsN2YpZw=; b=zZIJ6VzWofwi4ufed+3zc8h54Hp4uCg8206Jyh+Y3JQsfIvc2n9c1NYc3CiqDeS7Ij ebNfXmQy3aQEgk2Kq7rMMTZIgS6VxXG9TvFt4lumRHCxDW81K5P0ME2BKGp+WlYdOuzh lJaRQamf6ol1nEnQnIZUk8Fn76HZOh+K5/tEcrAjw2A7pFOCWynrxdQnxVq7Pjw5axlj ea5oKZ3m6dqzRnOFyjrGRC7szQQn9tL+RNOnmyR2k2eU0zxvvV3KpxHA/OyIym4mo5P6 rEfVHNlbev946OqqFWHbJYKbCmwr9vaD/LThxtzRyiN4ngjbZG85tBFV/RGne6W2mqQ+ 4a0g== X-Forwarded-Encrypted: i=1; AKwUvBwhXM0mfCHLDq+rq1pYK+U7QKVnaEN5Wv7kps9o+9YnO7BI3s3FR441LxsPmVYfSE5J7W6AYfu2cGhW+48=@vger.kernel.org X-Gm-Message-State: AFuF++ndwuleZi/3tSA345Wm2k2hb5bzLfzGhHgnA5T8cTN+qxechTQo V1BBBnFoMQQI9qrPG9IGfDuvNN3xGwQ5bXQLH0W+RWbBeAusn9GtzN70 X-Gm-Gg: AYBFou18Gz3JSVpdpunBp/LmNfXmAmgfojZJbTjd3vjSQyOeAYU/UVeE/roHjpD6J3d i9PXhvp3ZQ5zypXfOCxEsJjJUrcpzWSE6U4ChRs8hubGPafup0+qba25nlkXk7Urvl82/otsMq9 AFD7XXRGC2l48k5RJ2H09vyq1Kdc7lRE1XRTXpK6fjIptDfm7Vzvzc6QLdDj1VrfNB0JDL3nNeZ jSSTV4IaEmXlsfQSik5uiY14Gu4xBFVy6HZ5aQdXZ6Q+uLzn9zu3lq22Pmo65/yh9eWOGQ9V1u/ 0/BPn7XfCrPgsvNSGh9syqsd61xY/qCQzWfefqhuFQjubjIYS0gaXnjnghZ0JmzRUKsIU12RV46 iNVHvLlX5VM7Wjlmdd1Q+jYXkm/Yi7G2oTqbGS/XCV5t/ZI5UPG4XJlVA5dzM6PygCg1Lc6y4X0 lT1Z3tVdZfH1m/k6I+IZRDj/NdBfmd3+NipVBnjmzx7p/qNLkr06GfpguFZob6k7vc X-Received: by 2002:a05:6a00:b807:b0:857:72f8:dca6 with SMTP id d2e1a72fcca58-87d2374c350mr495409b3a.12.1790109831832; Tue, 22 Sep 2026 13:43:51 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:47::]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-87d1dff9769sm280202b3a.46.2026.09.22.13.43.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 13:43:51 -0700 (PDT) From: Stanislav Fomichev X-Google-Original-From: Stanislav Fomichev To: netdev@vger.kernel.org Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, hawk@kernel.org, ilias.apalodimas@linaro.org, asml.silence@gmail.com, axboe@kernel.dk, sdf@fomichev.me, bobbyeshleman@meta.com, almasrymina@google.com, kaiyuanz@google.com, linux-kernel@vger.kernel.org, io-uring@vger.kernel.org Subject: [PATCH net-next 2/3] net: devmem: use memory provider helpers for net_iovs Date: Tue, 22 Sep 2026 13:43:47 -0700 Message-ID: <20260922204348.717198-3-sdf@fomichev.me> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260922204348.717198-1-sdf@fomichev.me> References: <20260922204348.717198-1-sdf@fomichev.me> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use public memory-provider helpers for DMA address storage and page-pool association instead of open-coding page_pool state and accounting. Zero the net_iov array because net_mp_niov_set_page_pool() ORs PP_SIGNATURE into pp_magic. Reject DMA addresses that cannot be represented in net_iov metadata. Signed-off-by: Stanislav Fomichev --- net/core/devmem.c | 31 +++++++++++-------------------- 1 file changed, 11 insertions(+), 20 deletions(-) diff --git a/net/core/devmem.c b/net/core/devmem.c index c1c1872b88de..0fc77c1a2956 100644 --- a/net/core/devmem.c +++ b/net/core/devmem.c @@ -17,11 +17,9 @@ #include #include #include -#include =20 #include "devmem.h" #include "mp_dmabuf_devmem.h" -#include "page_pool_priv.h" =20 /* Device memory support */ =20 @@ -247,7 +245,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *vd= ev, goto err_unmap; } } - binding->area.niovs =3D kvmalloc_objs(*binding->area.niovs, + binding->area.niovs =3D kvzalloc_objs(*binding->area.niovs, binding->area.num_niovs); if (!binding->area.niovs) { err =3D -ENOMEM; @@ -274,8 +272,12 @@ net_devmem_bind_dmabuf(struct net_device *dev, void *v= dev, for (i =3D 0; i < nr_niovs; i++, niov_idx++) { niov =3D &binding->area.niovs[niov_idx]; net_iov_init(niov, &binding->area, NET_IOV_DMABUF); - page_pool_set_dma_addr_netmem(net_iov_to_netmem(niov), - dma_addr); + if (net_mp_niov_set_dma_addr(niov, dma_addr)) { + err =3D -EFAULT; + NL_SET_ERR_MSG(extack, + "dmabuf DMA address cannot be represented"); + goto err_free_niovs; + } if (direction =3D=3D DMA_TO_DEVICE) binding->tx_vec[niov_idx] =3D niov; else @@ -439,19 +441,9 @@ netmem_ref mp_dmabuf_devmem_alloc_netmems(struct page_= pool *pool, gfp_t gfp) if (unlikely(!allocated)) return 0; =20 - for (i =3D 0; i < allocated; i++) { - struct net_iov *niov =3D netmem_to_net_iov(netmems[i]); - - niov->desc.pp_magic =3D 0; - niov->desc.pp =3D NULL; - atomic_long_set(&niov->desc.pp_ref_count, 0); - - page_pool_set_pp_info(pool, netmems[i]); - - pool->pages_state_hold_cnt++; - trace_page_pool_state_hold(pool, netmems[i], - pool->pages_state_hold_cnt); - } + for (i =3D 0; i < allocated; i++) + net_mp_niov_set_page_pool(pool, + netmem_to_net_iov(netmems[i])); =20 /* Return the last one, the rest stay in the page_pool cache. */ allocated--; @@ -476,8 +468,7 @@ bool mp_dmabuf_devmem_release_page(struct page_pool *po= ol, netmem_ref netmem) if (WARN_ON_ONCE(refcount !=3D 1)) return false; =20 - page_pool_clear_pp_info(netmem); - + net_mp_niov_clear_page_pool(netmem_to_net_iov(netmem)); net_devmem_free_dmabuf(netmem_to_net_iov(netmem)); =20 /* We don't want the page pool put_page()ing our net_iovs. */ --=20 2.53.0-Meta From nobody Thu Sep 24 14:26:25 2026 Received: from mail-pj2-f10.google.com (mail-pj2-f10.google.com [74.125.227.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ADEF517BC5 for ; Tue, 22 Sep 2026 20:43:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.138 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790109853; cv=none; b=AiIZPdbloquYKF3qq6wzsDIejtvz4qBDi/vg06qQvUdh5NF/Z0eGgAYkH5qJTQfFxCPGlJlIMoRW/xdMu3R+NmZLtPTIHFluB+kwQKe8moahqj6NQoRrvbVeKpSvjEGRkXJqChWzWF3wraBCdV2zDCweJIdmzDSMl6xfVVPbHPg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790109853; c=relaxed/simple; bh=OWLnRy0Kyd8FdV/E9+5Qi5HN1AFVDCBPd909vqVd4d4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=t69M60cPPBFtWl4NfmYQEf15Tx6+wS2xdhs8cP7x2td+hBar0+DTNcirzfi6RfgThYhJSgtFc53yOm4EjtdLwV2U1KPPtgLgO9xWs47Qwvz9F+EtLbdXyHIhAeZeUYg/VzhXVJtmYuvheqtXxCIDdGMHyGinT+NT9p5U5U5GVrE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=LsW/y3zj; arc=none smtp.client-ip=74.125.227.138 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LsW/y3zj" Received: by mail-pj2-f10.google.com with SMTP id d9443c01a7336-2dd9bce707bso931325ad.1 for ; Tue, 22 Sep 2026 13:43:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1790109833; x=1790714633; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=NWtkhPsIR7rUvSSAcPaCXCFRGMGm2fhAVUCymwEWeyQ=; b=LsW/y3zjvB6uh21uuruh/Ri7fkZYm4WKusxPX4jKNVvoGlr21HDedY5cMenr46pwEP 9/gfmdW0U9WhuDu5TEqyCnkVctLyU16gjfA5hLkTySocRoAwBlRyWpAW4bpIiipShmqe F5PlUQlbGF3jZZS8+Gc9zPvuCxBSMHAbxn/FQezpxhTqumXa+e9jpx4/zLPQdKIqqwDI uFHDF4yqYA5qK7zc4fW2wF5dd7kJReZRgtWKUHQ42DkW7eQ5vugrqLsQRYnrhJSY/wR7 29Ags8MMbggZA1pmBWShQQCGLPrDUFBQRlt76jDkbKDHl7ebXPColxqRO1glX2ybNt6p GgHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790109833; x=1790714633; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=NWtkhPsIR7rUvSSAcPaCXCFRGMGm2fhAVUCymwEWeyQ=; b=yKto15zJ+C0yNLOshRiXIhVlRH4hDtjeCLT522kKz8f/IqcCYK3lXOZGpvrHxJ4FQC oZZJAKLm5QY+R+oR3r5KOWU1DQhUvsYTxrwhp+JHuoiV4b/8s6GNz8NBvc3+3LsG/oe4 rPOj+hYqC2wluCbV0T7dYlhchjUjB4o4nV7CJD3NzDLvQ4erizeCeOXwe50BjIJOKUOe gvjVWRVEbdJ3/fgeJvHgKGLgc6ZiN6QHMuk35WhpAophhV1+bVyk4eoa3yz8ykrKw53R yl1TOQIrOzdthd8UDvXdmM4u09Zn2E8Ucd5b6upIQV8UZRaMKWbJTucK/sdDYp+alQXN w2bA== X-Forwarded-Encrypted: i=1; AKwUvBz+ytA/KcXHPR+OJXBzM23od1IPJkwIL3uTF5vX4vQvD1HPH8UcPNEbgvPvYUqzp/xBO+pSJwQbZxkWrVM=@vger.kernel.org X-Gm-Message-State: AFuF++mlFRc0VM5o5NZAZqJQuv7um4QiTKpaWvAl9sXamlmOFTtRjlt7 HcfEnchHF65ssNN03qVCRLo4QNLEeihGMVwNH3VCmAuDq5O1VCPA7I9CANDYNnJK X-Gm-Gg: AYBFou2QDYOm3V0BHEw7mDskDcPxJSHF/08fo9OhFwgNjGf9kwiQbJZGHLc2Qcyd9hU 0nCKXzUUAUKFcGOCn16Ciw+VPkmkTSucR5rN40MACcQaNrk6vm6VubA4IyqBxnmxLdXCrwkMSt3 ln2/N4ueg4MNrDxuEvHadd5Vr7nlxUYnCZs2eytRVQOsxi4LsAWLEQERnvJjkkqCOQUXlVW/79X 9nW1EnFfJd85zl4QsyXC3bNnTa7aJHhvPmIfzwXXqCohO6O9LRgdv0JEWovuOVTSz0bEx42dvKA EKBrSUZ2jpRvjcSjKjSRExNjGmYNhJ2mkhnVLMoE3A8dS3YPZ/mXE4omQf+ze2JJKluX1/G2F4j meD77z+ta+n/VFPPIdZBCMh1CxmhiSypY4dNKvqoVRCrllpggknpBIWvJAmKgRTNp/pjJFSHyzj vP/IiqapdtX19u4puzT8zC0kmETvyj3tBYHV6x4r6AKKb1/cb3JkuyAX3Hqk9GFglB/SMOhhtHp 68= X-Received: by 2002:a17:903:1ac5:b0:2dd:c053:82f0 with SMTP id d9443c01a7336-2df69dc3fddmr4755005ad.39.1790109833271; Tue, 22 Sep 2026 13:43:53 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:72::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2df6a5f965esm879425ad.75.2026.09.22.13.43.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 13:43:52 -0700 (PDT) From: Stanislav Fomichev X-Google-Original-From: Stanislav Fomichev To: netdev@vger.kernel.org Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, hawk@kernel.org, ilias.apalodimas@linaro.org, asml.silence@gmail.com, axboe@kernel.dk, sdf@fomichev.me, bobbyeshleman@meta.com, almasrymina@google.com, kaiyuanz@google.com, linux-kernel@vger.kernel.org, io-uring@vger.kernel.org Subject: [PATCH net-next 3/3] net: devmem: decode DMA addresses for TX Date: Tue, 22 Sep 2026 13:43:48 -0700 Message-ID: <20260922204348.717198-4-sdf@fomichev.me> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260922204348.717198-1-sdf@fomichev.me> References: <20260922204348.717198-1-sdf@fomichev.me> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" On 32-bit architectures where dma_addr_t is wider than unsigned long, page_pool_set_dma_addr_netmem() stores page-aligned DMA addresses shifted by PAGE_SHIFT. The net_iov branch of __skb_frag_dma_map() adds byte offsets to the encoded value, so the NIC is programmed with an invalid DMA address. This can trigger an IOMMU fault or DMA from unintended memory. Consolidate DMA address encoding, decoding, and representability checks in netmem helpers. Use the common decoder from the page pool and net_iov TX paths so both interpret stored addresses consistently. Fixes: bd61848900bf ("net: devmem: Implement TX path") Signed-off-by: Stanislav Fomichev --- include/linux/skbuff.h | 7 +++++-- include/net/netmem.h | 27 +++++++++++++++++++++++++++ include/net/page_pool/helpers.h | 10 +--------- net/core/page_pool_priv.h | 14 ++------------ 4 files changed, 35 insertions(+), 23 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 421f6fc45451..3740d4b15b70 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3833,9 +3833,12 @@ static inline dma_addr_t __skb_frag_dma_map(struct d= evice *dev, size_t offset, size_t size, enum dma_data_direction dir) { + dma_addr_t addr; + if (skb_frag_is_net_iov(frag)) { - return netmem_to_net_iov(frag->netmem)->desc.dma_addr + - offset + frag->offset; + addr =3D netmem_dma_addr_decode( + netmem_get_dma_addr(frag->netmem)); + return addr + offset + frag->offset; } return dma_map_page(dev, skb_frag_page(frag), skb_frag_off(frag) + offset, size, dir); diff --git a/include/net/netmem.h b/include/net/netmem.h index da885d95ea63..4e06a647620a 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -384,6 +384,33 @@ static inline bool netmem_is_pfmemalloc(netmem_ref net= mem) return page_is_pfmemalloc(netmem_to_page(netmem)); } =20 +#define NETMEM_32BIT_ARCH_WITH_64BIT_DMA \ + (sizeof(dma_addr_t) > sizeof(unsigned long)) + +static inline unsigned long netmem_dma_addr_encode(dma_addr_t addr) +{ + if (NETMEM_32BIT_ARCH_WITH_64BIT_DMA) + addr >>=3D PAGE_SHIFT; + + return addr; +} + +static inline dma_addr_t netmem_dma_addr_decode(unsigned long addr) +{ + if (NETMEM_32BIT_ARCH_WITH_64BIT_DMA) + return (dma_addr_t)addr << PAGE_SHIFT; + + return addr; +} + +static inline bool netmem_dma_addr_fits(dma_addr_t addr) +{ + /* We assume page alignment to shave off bottom bits, + * if this "compression" doesn't work we need to drop. + */ + return addr =3D=3D netmem_dma_addr_decode(netmem_dma_addr_encode(addr)); +} + static inline unsigned long netmem_get_dma_addr(netmem_ref netmem) { return netmem_to_nmdesc(netmem)->dma_addr; diff --git a/include/net/page_pool/helpers.h b/include/net/page_pool/helper= s.h index 87a4e13886e1..cd021832c3fa 100644 --- a/include/net/page_pool/helpers.h +++ b/include/net/page_pool/helpers.h @@ -408,9 +408,6 @@ static inline void page_pool_recycle_direct_netmem(stru= ct page_pool *pool, page_pool_put_full_netmem(pool, netmem, true); } =20 -#define PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA \ - (sizeof(dma_addr_t) > sizeof(unsigned long)) - /** * page_pool_free_va() - free a va into the page_pool * @pool: pool from which va was allocated @@ -427,12 +424,7 @@ static inline void page_pool_free_va(struct page_pool = *pool, void *va, =20 static inline dma_addr_t page_pool_get_dma_addr_netmem(netmem_ref netmem) { - dma_addr_t ret =3D netmem_get_dma_addr(netmem); - - if (PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA) - ret <<=3D PAGE_SHIFT; - - return ret; + return netmem_dma_addr_decode(netmem_get_dma_addr(netmem)); } =20 /** diff --git a/net/core/page_pool_priv.h b/net/core/page_pool_priv.h index 2fb06d5f6d55..430b97cd88da 100644 --- a/net/core/page_pool_priv.h +++ b/net/core/page_pool_priv.h @@ -18,18 +18,8 @@ void page_pool_unlist(struct page_pool *pool); static inline bool page_pool_set_dma_addr_netmem(netmem_ref netmem, dma_addr_t addr) { - if (PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA) { - netmem_set_dma_addr(netmem, addr >> PAGE_SHIFT); - - /* We assume page alignment to shave off bottom bits, - * if this "compression" doesn't work we need to drop. - */ - return addr !=3D (dma_addr_t)netmem_get_dma_addr(netmem) - << PAGE_SHIFT; - } - - netmem_set_dma_addr(netmem, addr); - return false; + netmem_set_dma_addr(netmem, netmem_dma_addr_encode(addr)); + return !netmem_dma_addr_fits(addr); } =20 static inline bool page_pool_set_dma_addr(struct page *page, dma_addr_t ad= dr) --=20 2.53.0-Meta