1. Patchew
  2. linux
  3. [PATCH 0/5] vsock/virtio: SKB allocation improvements
  4. View patch

[PATCH 5/5] vhost/vsock: Allocate nonlinear SKBs for handling large transmit buffers

Will Deacon posted 5 patches 3 months, 2 weeks ago
There is a newer version of this series
[PATCH 5/5] vhost/vsock: Allocate nonlinear SKBs for handling large transmit buffers
Posted by Will Deacon 3 months, 2 weeks ago 
When transmitting a vsock packet, virtio_transport_send_pkt_info() calls
virtio_transport_alloc_skb() to allocate and fill SKBs with the transmit
data. Unfortunately, these are always linear allocations and can
therefore result in significant pressure on kmalloc() considering that
the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE +
VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB
allocation for each packet.

Rework the vsock SKB allocation so that, for sizes with page order
greater than PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated
instead with the packet header in the SKB and the transmit data in the
fragments.

Signed-off-by: Will Deacon <will@kernel.org>
---
 net/vmw_vsock/virtio_transport_common.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
index 1b5d9896edae..424eb69e84f9 100644
--- a/net/vmw_vsock/virtio_transport_common.c
+++ b/net/vmw_vsock/virtio_transport_common.c
@@ -109,7 +109,8 @@ static int virtio_transport_fill_skb(struct sk_buff *skb,
 		return __zerocopy_sg_from_iter(info->msg, NULL, skb,
 					       &info->msg->msg_iter, len, NULL);
 
-	return memcpy_from_msg(skb_put(skb, len), info->msg, len);
+	virtio_vsock_skb_put(skb);
+	return skb_copy_datagram_from_iter(skb, 0, &info->msg->msg_iter, len);
 }
 
 static void virtio_transport_init_hdr(struct sk_buff *skb,
@@ -261,7 +262,11 @@ static struct sk_buff *virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *
 	if (!zcopy)
 		skb_len += payload_len;
 
-	skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL);
+	if (skb_len > SKB_WITH_OVERHEAD(PAGE_SIZE << PAGE_ALLOC_COSTLY_ORDER))
+		skb = virtio_vsock_alloc_skb_with_frags(skb_len, GFP_KERNEL);
+	else
+		skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL);
+
 	if (!skb)
 		return NULL;
 
-- 
2.50.0.714.g196bf9f422-goog
Re: [PATCH 5/5] vhost/vsock: Allocate nonlinear SKBs for handling large transmit buffers
Posted by Stefano Garzarella 3 months, 1 week ago 
nit: I'd use `vsock/virtio: ` prefix since we are touching the virtio 
transport common code. Maybe we can mention that this will affect both
virtio and vhost transports.

On Wed, Jun 25, 2025 at 02:15:43PM +0100, Will Deacon wrote:
>When transmitting a vsock packet, virtio_transport_send_pkt_info() calls
>virtio_transport_alloc_skb() to allocate and fill SKBs with the transmit
>data. Unfortunately, these are always linear allocations and can
>therefore result in significant pressure on kmalloc() considering that
>the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE +
>VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB
>allocation for each packet.
>
>Rework the vsock SKB allocation so that, for sizes with page order
>greater than PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated
>instead with the packet header in the SKB and the transmit data in the
>fragments.
>
>Signed-off-by: Will Deacon <will@kernel.org>
>---
> net/vmw_vsock/virtio_transport_common.c | 9 +++++++--
> 1 file changed, 7 insertions(+), 2 deletions(-)
>
>diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
>index 1b5d9896edae..424eb69e84f9 100644
>--- a/net/vmw_vsock/virtio_transport_common.c
>+++ b/net/vmw_vsock/virtio_transport_common.c
>@@ -109,7 +109,8 @@ static int virtio_transport_fill_skb(struct sk_buff *skb,
> 		return __zerocopy_sg_from_iter(info->msg, NULL, skb,
> 					       &info->msg->msg_iter, len, NULL);
>
>-	return memcpy_from_msg(skb_put(skb, len), info->msg, len);
>+	virtio_vsock_skb_put(skb);
>+	return skb_copy_datagram_from_iter(skb, 0, &info->msg->msg_iter, len);
> }
>
> static void virtio_transport_init_hdr(struct sk_buff *skb,
>@@ -261,7 +262,11 @@ static struct sk_buff *virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *
> 	if (!zcopy)
> 		skb_len += payload_len;
>
>-	skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL);
>+	if (skb_len > SKB_WITH_OVERHEAD(PAGE_SIZE << PAGE_ALLOC_COSTLY_ORDER))
>+		skb = virtio_vsock_alloc_skb_with_frags(skb_len, GFP_KERNEL);
>+	else
>+		skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL);
>+

As I mentioned in the other patch, we may avoid this code duplication 
hiding this in virtio_vsock_alloc_skb() or adding a new function that
we can use when we want to allocate frags or not.

Thanks,
Stefano

> 	if (!skb)
> 		return NULL;
>
>-- 
>2.50.0.714.g196bf9f422-goog
>
>
Re: [PATCH 5/5] vhost/vsock: Allocate nonlinear SKBs for handling large transmit buffers
Posted by Will Deacon 3 months, 1 week ago 
On Fri, Jun 27, 2025 at 12:50:27PM +0200, Stefano Garzarella wrote:
> nit: I'd use `vsock/virtio: ` prefix since we are touching the virtio
> transport common code. Maybe we can mention that this will affect both
> virtio and vhost transports.

Sure, I'll do that.

> On Wed, Jun 25, 2025 at 02:15:43PM +0100, Will Deacon wrote:
> > When transmitting a vsock packet, virtio_transport_send_pkt_info() calls
> > virtio_transport_alloc_skb() to allocate and fill SKBs with the transmit
> > data. Unfortunately, these are always linear allocations and can
> > therefore result in significant pressure on kmalloc() considering that
> > the maximum packet size (VIRTIO_VSOCK_MAX_PKT_BUF_SIZE +
> > VIRTIO_VSOCK_SKB_HEADROOM) is a little over 64KiB, resulting in a 128KiB
> > allocation for each packet.
> > 
> > Rework the vsock SKB allocation so that, for sizes with page order
> > greater than PAGE_ALLOC_COSTLY_ORDER, a nonlinear SKB is allocated
> > instead with the packet header in the SKB and the transmit data in the
> > fragments.
> > 
> > Signed-off-by: Will Deacon <will@kernel.org>
> > ---
> > net/vmw_vsock/virtio_transport_common.c | 9 +++++++--
> > 1 file changed, 7 insertions(+), 2 deletions(-)
> > 
> > diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
> > index 1b5d9896edae..424eb69e84f9 100644
> > --- a/net/vmw_vsock/virtio_transport_common.c
> > +++ b/net/vmw_vsock/virtio_transport_common.c
> > @@ -109,7 +109,8 @@ static int virtio_transport_fill_skb(struct sk_buff *skb,
> > 		return __zerocopy_sg_from_iter(info->msg, NULL, skb,
> > 					       &info->msg->msg_iter, len, NULL);
> > 
> > -	return memcpy_from_msg(skb_put(skb, len), info->msg, len);
> > +	virtio_vsock_skb_put(skb);
> > +	return skb_copy_datagram_from_iter(skb, 0, &info->msg->msg_iter, len);
> > }
> > 
> > static void virtio_transport_init_hdr(struct sk_buff *skb,
> > @@ -261,7 +262,11 @@ static struct sk_buff *virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *
> > 	if (!zcopy)
> > 		skb_len += payload_len;
> > 
> > -	skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL);
> > +	if (skb_len > SKB_WITH_OVERHEAD(PAGE_SIZE << PAGE_ALLOC_COSTLY_ORDER))
> > +		skb = virtio_vsock_alloc_skb_with_frags(skb_len, GFP_KERNEL);
> > +	else
> > +		skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL);
> > +
> 
> As I mentioned in the other patch, we may avoid this code duplication hiding
> this in virtio_vsock_alloc_skb() or adding a new function that
> we can use when we want to allocate frags or not.

That would be good. I had a crack at it in the diff I sent in reply to
the earlier patch, so please take a look.

Will

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.