[RFC net-next v3 7/7] net: devmem: allow binding on rx queues with same MA devices

Dragos Tatulea posted 7 patches 1 month, 2 weeks ago
There is a newer version of this series
[RFC net-next v3 7/7] net: devmem: allow binding on rx queues with same MA devices
Posted by Dragos Tatulea 1 month, 2 weeks ago
Multi-PF netdevs have queues belonging to different PFs which also means
different DMA devices. This means that the binding on the DMA buffer can
be done to the incorrect device.

This change allows devmem binding to multiple queues only when the
queues have the same DMA device. Otherwise an error is returned.

Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
---
 net/core/netdev-genl.c | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c
index 3e990f100bf0..649b62803529 100644
--- a/net/core/netdev-genl.c
+++ b/net/core/netdev-genl.c
@@ -903,6 +903,31 @@ static int netdev_nl_read_rxq_bitmap(struct genl_info *info,
 	return 0;
 }
 
+static struct device *netdev_nl_get_dma_dev(struct net_device *netdev,
+					    unsigned long *rxq_bitmap,
+					    struct netlink_ext_ack *extack)
+{
+	struct device *dma_dev = NULL;
+	u32 rxq_idx;
+
+	for_each_set_bit(rxq_idx, rxq_bitmap, netdev->num_rx_queues) {
+		struct device *rxq_dma_dev;
+
+		rxq_dma_dev = netdev_queue_get_dma_dev(netdev, rxq_idx);
+		/* Multi-PF netdev queues can belong to different DMA devoces.
+		 * Block this case.
+		 */
+		if (rxq_dma_dev && dma_dev && rxq_dma_dev != dma_dev) {
+			NL_SET_ERR_MSG(extack, "Can't bind to queues from different dma devices");
+			return NULL;
+		}
+
+		dma_dev = rxq_dma_dev;
+	}
+
+	return dma_dev;
+}
+
 int netdev_nl_bind_rx_doit(struct sk_buff *skb, struct genl_info *info)
 {
 	struct net_devmem_dmabuf_binding *binding;
@@ -962,7 +987,7 @@ int netdev_nl_bind_rx_doit(struct sk_buff *skb, struct genl_info *info)
 	}
 	netdev_nl_read_rxq_bitmap(info, rxq_bitmap);
 
-	dma_dev = netdev_queue_get_dma_dev(netdev, 0);
+	dma_dev = netdev_nl_get_dma_dev(netdev, rxq_bitmap, info->extack);
 	binding = net_devmem_bind_dmabuf(netdev, dma_dev, DMA_FROM_DEVICE,
 					 dmabuf_fd, priv, info->extack);
 	if (IS_ERR(binding)) {
-- 
2.50.1
Re: [RFC net-next v3 7/7] net: devmem: allow binding on rx queues with same MA devices
Posted by Jakub Kicinski 1 month, 2 weeks ago
On Fri, 15 Aug 2025 14:03:48 +0300 Dragos Tatulea wrote:
> +		rxq_dma_dev = netdev_queue_get_dma_dev(netdev, rxq_idx);
> +		/* Multi-PF netdev queues can belong to different DMA devoces.
> +		 * Block this case.
> +		 */
> +		if (rxq_dma_dev && dma_dev && rxq_dma_dev != dma_dev) {

Why rxq_dma_dev ? 🤔️
Don't we want to error out if the first queue gave us a DMA dev but the
second gave us a NULL ?

> +			NL_SET_ERR_MSG(extack, "Can't bind to queues from different dma devices");

_FMT the conflicting queue IDs into this?
Re: [RFC net-next v3 7/7] net: devmem: allow binding on rx queues with same MA devices
Posted by Dragos Tatulea 1 month, 2 weeks ago
On Fri, Aug 15, 2025 at 10:24:33AM -0700, Jakub Kicinski wrote:
> On Fri, 15 Aug 2025 14:03:48 +0300 Dragos Tatulea wrote:
> > +		rxq_dma_dev = netdev_queue_get_dma_dev(netdev, rxq_idx);
> > +		/* Multi-PF netdev queues can belong to different DMA devoces.
Just saw the "devoces" typo. Will fix this too.

> > +		 * Block this case.
> > +		 */
> > +		if (rxq_dma_dev && dma_dev && rxq_dma_dev != dma_dev) {
> 
> Why rxq_dma_dev ? 🤔️
> Don't we want to error out if the first queue gave us a DMA dev but the
> second gave us a NULL ?
>
Hmm, yes... I didn't take this case into account. Will fix.

> > +			NL_SET_ERR_MSG(extack, "Can't bind to queues from different dma devices");
> 
> _FMT the conflicting queue IDs into this?
Sure. Will print the last dma_dev index and the current rxq_dma_dev index.

Thanks,
Dragos