From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82F33C433EF for ; Sun, 24 Apr 2022 02:40:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237735AbiDXCns (ORCPT ); Sat, 23 Apr 2022 22:43:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232948AbiDXCnr (ORCPT ); Sat, 23 Apr 2022 22:43:47 -0400 Received: from out30-56.freemail.mail.aliyun.com (out30-56.freemail.mail.aliyun.com [115.124.30.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EE907CDEB for ; Sat, 23 Apr 2022 19:40:48 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04423;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzbmqS_1650768045; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzbmqS_1650768045) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:46 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 01/16] virtio_ring: split: vring_unmap_one_split() get extra by arg Date: Sun, 24 Apr 2022 10:40:29 +0800 Message-Id: <20220424024044.94749-2-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" vring_unmap_one_split() is changed to pass the extra from the argument instead of getting it from vq. The purpose of this is that this function can be used later to unmap the extra from the detached vring. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 0516fdbd070e..0a2c58557027 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -426,32 +426,31 @@ static void vring_unmap_one_split_indirect(const stru= ct vring_virtqueue *vq, } =20 static unsigned int vring_unmap_one_split(const struct vring_virtqueue *vq, - unsigned int i) + struct vring_desc_extra *extra) { - struct vring_desc_extra *extra =3D vq->split.desc_extra; u16 flags; =20 if (!vq->use_dma_api) goto out; =20 - flags =3D extra[i].flags; + flags =3D extra->flags; =20 if (flags & VRING_DESC_F_INDIRECT) { dma_unmap_single(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, + extra->addr, + extra->len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } else { dma_unmap_page(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, + extra->addr, + extra->len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } =20 out: - return extra[i].next; + return extra->next; } =20 static struct vring_desc *alloc_indirect_split(struct virtqueue *_vq, @@ -679,7 +678,7 @@ static inline int virtqueue_add_split(struct virtqueue = *_vq, vring_unmap_one_split_indirect(vq, &desc[i]); i =3D virtio16_to_cpu(_vq->vdev, desc[i].next); } else - i =3D vring_unmap_one_split(vq, i); + i =3D vring_unmap_one_split(vq, &vq->split.desc_extra[i]); } =20 if (indirect) @@ -733,12 +732,12 @@ static void detach_buf_split(struct vring_virtqueue *= vq, unsigned int head, i =3D head; =20 while (vq->split.vring.desc[i].flags & nextflag) { - vring_unmap_one_split(vq, i); + vring_unmap_one_split(vq, &vq->split.desc_extra[i]); i =3D vq->split.desc_extra[i].next; vq->vq.num_free++; } =20 - vring_unmap_one_split(vq, i); + vring_unmap_one_split(vq, &vq->split.desc_extra[i]); vq->split.desc_extra[i].next =3D vq->free_head; vq->free_head =3D head; =20 --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BC27C433EF for ; Sun, 24 Apr 2022 02:41:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237770AbiDXCn4 (ORCPT ); Sat, 23 Apr 2022 22:43:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235851AbiDXCns (ORCPT ); Sat, 23 Apr 2022 22:43:48 -0400 Received: from out30-45.freemail.mail.aliyun.com (out30-45.freemail.mail.aliyun.com [115.124.30.45]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17F1F7DA8D for ; Sat, 23 Apr 2022 19:40:48 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04394;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzcuYK_1650768046; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzcuYK_1650768046) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:46 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 02/16] virtio_ring: split: introduce vring_virtqueue_detach_split() Date: Sun, 24 Apr 2022 10:40:30 +0800 Message-Id: <20220424024044.94749-3-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The function vring_virtqueue_detach_split() is introduced to detach the vring of the current vq. num_left records how many buffers there are, which can be used to check the recovery of buffers completed. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 0a2c58557027..f3ad9322b512 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -112,6 +112,8 @@ struct vring_virtqueue_split { */ u32 vring_align; bool may_reduce_num; + + u32 num_left; }; =20 struct vring_virtqueue_packed { @@ -982,6 +984,21 @@ static void virtqueue_reinit_split(struct vring_virtqu= eue *vq) virtqueue_vring_init_split(vq); } =20 +static void virtqueue_vring_detach_split(struct vring_virtqueue *vq, + struct vring_virtqueue_split *vring) +{ + vring->vring =3D vq->split.vring; + + vring->queue_dma_addr =3D vq->split.queue_dma_addr; + + vring->queue_size_in_bytes =3D vq->split.queue_size_in_bytes; + + vring->desc_extra =3D vq->split.desc_extra; + vring->desc_state =3D vq->split.desc_state; + + vring->num_left =3D vring->vring.num - vq->vq.num_free; +} + static void virtqueue_vring_attach_split(struct vring_virtqueue *vq, struct vring_virtqueue_split *vring) { --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F4216C433EF for ; Sun, 24 Apr 2022 02:41:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237394AbiDXCoC (ORCPT ); Sat, 23 Apr 2022 22:44:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237738AbiDXCnt (ORCPT ); Sat, 23 Apr 2022 22:43:49 -0400 Received: from out30-54.freemail.mail.aliyun.com (out30-54.freemail.mail.aliyun.com [115.124.30.54]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 253CA7DE3E for ; Sat, 23 Apr 2022 19:40:49 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04357;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzRZYU_1650768047; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzRZYU_1650768047) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:47 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 03/16] virtio_ring: split: extract virtqueue_update_split() Date: Sun, 24 Apr 2022 10:40:31 +0800 Message-Id: <20220424024044.94749-4-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Separate the logic for updating the vq state from virtqueue_add_split(). In this way, when the subsequent patch implements the logic of reusing the buffer when resize, we can share this function. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 95 ++++++++++++++++++++---------------- 1 file changed, 54 insertions(+), 41 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index f3ad9322b512..6fd45c9a3517 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -507,6 +507,51 @@ static inline unsigned int virtqueue_add_desc_split(st= ruct virtqueue *vq, return next; } =20 +static inline void virtqueue_update_split(struct vring_virtqueue *vq, + u32 descs_used, + u32 next, + struct vring_desc *desc, + void *data) +{ + struct virtqueue *_vq =3D &vq->vq; + u32 avail, head; + + head =3D vq->free_head; + + /* We're using some buffers from the free list. */ + vq->vq.num_free -=3D descs_used; + + /* Update free pointer */ + vq->free_head =3D next; + + /* Store token and indirect buffer state. */ + vq->split.desc_state[head].data =3D data; + vq->split.desc_state[head].indir_desc =3D desc; + + /* Put entry in available array (but don't update avail->idx until they + * do sync). + */ + avail =3D vq->split.avail_idx_shadow & (vq->split.vring.num - 1); + vq->split.vring.avail->ring[avail] =3D cpu_to_virtio16(_vq->vdev, head); + + /* Descriptors and available array need to be set before we expose the + * new available array entries. + */ + virtio_wmb(vq->weak_barriers); + vq->split.avail_idx_shadow++; + vq->split.vring.avail->idx =3D cpu_to_virtio16(_vq->vdev, + vq->split.avail_idx_shadow); + vq->num_added++; + + pr_debug("Added buffer head %i to %p\n", head, vq); + + /* This is very unlikely, but theoretically possible. Kick + * just in case. + */ + if (unlikely(vq->num_added =3D=3D (1 << 16) - 1)) + virtqueue_kick(_vq); +} + static inline int virtqueue_add_split(struct virtqueue *_vq, struct scatterlist *sgs[], unsigned int total_sg, @@ -519,7 +564,7 @@ static inline int virtqueue_add_split(struct virtqueue = *_vq, struct vring_virtqueue *vq =3D to_vvq(_vq); struct scatterlist *sg; struct vring_desc *desc; - unsigned int i, n, avail, descs_used, prev, err_idx; + unsigned int i, n, descs_used, prev, err_idx; int head; bool indirect; =20 @@ -619,50 +664,18 @@ static inline int virtqueue_add_split(struct virtqueu= e *_vq, if (vring_mapping_error(vq, addr)) goto unmap_release; =20 - virtqueue_add_desc_split(_vq, vq->split.vring.desc, - head, addr, - total_sg * sizeof(struct vring_desc), - VRING_DESC_F_INDIRECT, - false); + i =3D virtqueue_add_desc_split(_vq, vq->split.vring.desc, + head, addr, + total_sg * sizeof(struct vring_desc), + VRING_DESC_F_INDIRECT, + false); + } else { + desc =3D ctx; } =20 - /* We're using some buffers from the free list. */ - vq->vq.num_free -=3D descs_used; - - /* Update free pointer */ - if (indirect) - vq->free_head =3D vq->split.desc_extra[head].next; - else - vq->free_head =3D i; - - /* Store token and indirect buffer state. */ - vq->split.desc_state[head].data =3D data; - if (indirect) - vq->split.desc_state[head].indir_desc =3D desc; - else - vq->split.desc_state[head].indir_desc =3D ctx; - - /* Put entry in available array (but don't update avail->idx until they - * do sync). */ - avail =3D vq->split.avail_idx_shadow & (vq->split.vring.num - 1); - vq->split.vring.avail->ring[avail] =3D cpu_to_virtio16(_vq->vdev, head); - - /* Descriptors and available array need to be set before we expose the - * new available array entries. */ - virtio_wmb(vq->weak_barriers); - vq->split.avail_idx_shadow++; - vq->split.vring.avail->idx =3D cpu_to_virtio16(_vq->vdev, - vq->split.avail_idx_shadow); - vq->num_added++; - - pr_debug("Added buffer head %i to %p\n", head, vq); + virtqueue_update_split(vq, descs_used, i, desc, data); END_USE(vq); =20 - /* This is very unlikely, but theoretically possible. Kick - * just in case. */ - if (unlikely(vq->num_added =3D=3D (1 << 16) - 1)) - virtqueue_kick(_vq); - return 0; =20 unmap_release: --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A04AC433EF for ; Sun, 24 Apr 2022 02:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237814AbiDXCoK (ORCPT ); Sat, 23 Apr 2022 22:44:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237747AbiDXCnw (ORCPT ); Sat, 23 Apr 2022 22:43:52 -0400 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01DD17DE3E for ; Sat, 23 Apr 2022 19:40:51 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R741e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04357;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzpPIS_1650768048; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzpPIS_1650768048) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:48 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 04/16] virtio_ring: split: extract detach_from_vring_split() Date: Sun, 24 Apr 2022 10:40:32 +0800 Message-Id: <20220424024044.94749-5-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To handle freeing buf from the detached vring, do a split for detach_buf_split(). The split function detach_buf_from_vring_split() is used to release buf from vring, and the vq passed in is read-only. All modifications are for vring. In this way, detach_buf_from_vring_split() becomes a general function, which can be used for detach_buf_split() and also for handling detached vrings. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 54 +++++++++++++++++++++++------------- 1 file changed, 34 insertions(+), 20 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 6fd45c9a3517..aa85058978cb 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -734,54 +734,68 @@ static bool virtqueue_kick_prepare_split(struct virtq= ueue *_vq) return needs_kick; } =20 -static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, - void **ctx) +static int detach_buf_from_vring_split(struct vring_virtqueue_split *vring, + struct vring_virtqueue const *vq, + unsigned int head, + unsigned int free_head, + void **ctx) { - unsigned int i, j; + unsigned int i, j, num =3D 0; __virtio16 nextflag =3D cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT); =20 /* Clear data ptr. */ - vq->split.desc_state[head].data =3D NULL; + vring->desc_state[head].data =3D NULL; =20 /* Put back on free list: unmap first-level descriptors and find end */ i =3D head; =20 - while (vq->split.vring.desc[i].flags & nextflag) { - vring_unmap_one_split(vq, &vq->split.desc_extra[i]); - i =3D vq->split.desc_extra[i].next; - vq->vq.num_free++; + while (vring->vring.desc[i].flags & nextflag) { + vring_unmap_one_split(vq, &vring->desc_extra[i]); + i =3D vring->desc_extra[i].next; + ++num; } =20 - vring_unmap_one_split(vq, &vq->split.desc_extra[i]); - vq->split.desc_extra[i].next =3D vq->free_head; - vq->free_head =3D head; + vring_unmap_one_split(vq, &vring->desc_extra[i]); + vring->desc_extra[i].next =3D free_head; =20 - /* Plus final descriptor */ - vq->vq.num_free++; + ++num; =20 if (vq->indirect) { struct vring_desc *indir_desc =3D - vq->split.desc_state[head].indir_desc; + vring->desc_state[head].indir_desc; u32 len; =20 /* Free the indirect table, if any, now that it's unmapped. */ if (!indir_desc) - return; + return num; =20 - len =3D vq->split.desc_extra[head].len; + len =3D vring->desc_extra[head].len; =20 - BUG_ON(!(vq->split.desc_extra[head].flags & - VRING_DESC_F_INDIRECT)); + BUG_ON(!(vring->desc_extra[head].flags & VRING_DESC_F_INDIRECT)); BUG_ON(len =3D=3D 0 || len % sizeof(struct vring_desc)); =20 for (j =3D 0; j < len / sizeof(struct vring_desc); j++) vring_unmap_one_split_indirect(vq, &indir_desc[j]); =20 kfree(indir_desc); - vq->split.desc_state[head].indir_desc =3D NULL; + vring->desc_state[head].indir_desc =3D NULL; } else if (ctx) { - *ctx =3D vq->split.desc_state[head].indir_desc; + *ctx =3D vring->desc_state[head].indir_desc; } + + return num; +} + +static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, + void **ctx) +{ + int num; + + num =3D detach_buf_from_vring_split(&vq->split, vq, head, vq->free_head, + ctx); + + vq->vq.num_free +=3D num; + vq->free_head =3D head; } =20 static inline bool more_used_split(const struct vring_virtqueue *vq) --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8579C433F5 for ; Sun, 24 Apr 2022 02:41:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237769AbiDXCoE (ORCPT ); Sat, 23 Apr 2022 22:44:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237744AbiDXCnv (ORCPT ); Sat, 23 Apr 2022 22:43:51 -0400 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF9817EA09 for ; Sat, 23 Apr 2022 19:40:51 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04395;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzbmr0_1650768049; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzbmr0_1650768049) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:49 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 05/16] virtio_ring: split: support copy from vring Date: Sun, 24 Apr 2022 10:40:33 +0800 Message-Id: <20220424024044.94749-6-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To support reusing old buffers during resize. This patch implements copying a buffer from the detached vring to the vq where the new vring is attached. This process is similar to virtqueue_add_split(), but skips DMA. Use the function virtqueue_update_split() provided by the previous patch to update the state of the vq. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 60 ++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index aa85058978cb..167442cfdb2a 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -703,6 +703,66 @@ static inline int virtqueue_add_split(struct virtqueue= *_vq, return -ENOMEM; } =20 +static u32 vring_copy_desc_split(struct vring_virtqueue *vq, u32 i, + struct vring_virtqueue_split *vring, + u32 src) +{ + struct vring_desc_extra *extra =3D vq->split.desc_extra; + struct vring_desc *desc =3D vq->split.vring.desc; + u16 next; + + desc[i].flags =3D vring->vring.desc[src].flags; + desc[i].addr =3D vring->vring.desc[src].addr; + desc[i].len =3D vring->vring.desc[src].len; + + next =3D extra[i].next; + + desc[i].next =3D cpu_to_virtio16(vq->vq.vdev, next); + + extra[i].addr =3D vring->desc_extra[src].addr; + extra[i].len =3D vring->desc_extra[src].len; + extra[i].flags =3D vring->desc_extra[src].flags; + + return next; +} + +static int vring_copy_to_vq_split(struct vring_virtqueue *vq, + struct vring_virtqueue_split *vring, + u32 old_index) +{ + __virtio16 nextflag =3D cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT); + struct vring_desc_state_split *state; + u32 i, num =3D 1, old_idx; + + old_idx =3D old_index; + while (vring->vring.desc[old_idx].flags & nextflag) { + old_idx =3D vring->desc_extra[old_idx].next; + ++num; + } + + if (num > vq->vq.num_free) + return -ENOSPC; + + i =3D vq->free_head; + + old_idx =3D old_index; + while (vring->vring.desc[old_idx].flags & nextflag) { + i =3D vring_copy_desc_split(vq, i, vring, old_idx); + + old_idx =3D vring->desc_extra[old_idx].next; + } + + i =3D vring_copy_desc_split(vq, i, vring, old_idx); + + state =3D &vring->desc_state[old_index]; + + virtqueue_update_split(vq, num, i, state->indir_desc, state->data); + + state->data =3D NULL; + + return num; +} + static bool virtqueue_kick_prepare_split(struct virtqueue *_vq) { struct vring_virtqueue *vq =3D to_vvq(_vq); --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1883AC433EF for ; Sun, 24 Apr 2022 02:41:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237808AbiDXCoG (ORCPT ); Sat, 23 Apr 2022 22:44:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237748AbiDXCnw (ORCPT ); Sat, 23 Apr 2022 22:43:52 -0400 Received: from out30-56.freemail.mail.aliyun.com (out30-56.freemail.mail.aliyun.com [115.124.30.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9709F7F213 for ; Sat, 23 Apr 2022 19:40:52 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R331e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04407;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzpPIr_1650768049; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzpPIr_1650768049) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:50 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 06/16] virtio_ring: split: introduce vring_reuse_bufs_split() Date: Sun, 24 Apr 2022 10:40:34 +0800 Message-Id: <20220424024044.94749-7-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch will resubmit the buffers to the new vq in the order in which they were submitted. In order to get these buffers in order, the patch will get the buffers from the avail ring. We can know the current position of the avail ring from vring.avail->idx. First, check backward from idx. If a state appears repeatedly, it means that the buffer corresponding to this state has been consumed by the device and resubmitted. We will remove the subsequent state from the queue. Then move forward from the position where idx ends, the buffers encountered at this time are the order in which they were submitted. It is beneficial to ensure the order of buffers in the process of reuse. For example, under virtio-net, if the order is not guaranteed, it may lead to out-of-order tcp streams. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 65 ++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 167442cfdb2a..02d4ffcc0a3b 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -69,6 +69,7 @@ struct vring_desc_state_split { void *data; /* Data for callback. */ struct vring_desc *indir_desc; /* Indirect descriptor, if any. */ + bool checked; }; =20 struct vring_desc_state_packed { @@ -1007,6 +1008,70 @@ static bool virtqueue_enable_cb_delayed_split(struct= virtqueue *_vq) return true; } =20 +static void vring_reuse_bufs_split(struct vring_virtqueue *vq, + struct vring_virtqueue_split *vring, + void (*recycle)(struct virtqueue *vq, void *buf)) +{ + u32 head, num, idx, oidx, i, desc_num =3D 0; + u16 null, *p; + int err =3D 0; + void *buf; + + num =3D vring->vring.num; + + oidx =3D le16_to_cpu(vring->vring.avail->idx) - 1; + null =3D vring->vring.avail->ring[oidx & (num - 1)]; + + /* + * Check in the opposite direction in the avail ring. If a state appears + * repeatedly, it means that the state has been used and rejoined the + * avail ring. + */ + for (i =3D 0, idx =3D oidx; i < num; ++i, --idx) { + p =3D &vring->vring.avail->ring[idx & (num - 1)]; + + head =3D virtio32_to_cpu(vq->vq.vdev, *p); + + if (vring->desc_state[head].checked) { + *p =3D null; + continue; + } + + vring->desc_state[head].checked =3D true; + } + + /* + * Checking the avail ring forward, the non-null states encountered are + * the order in which they were added to the avail ring. + */ + for (i =3D 0, ++idx; i < num; ++i, ++idx) { + p =3D &vring->vring.avail->ring[idx & (num - 1)]; + if (*p =3D=3D null && idx !=3D oidx) + continue; + + head =3D virtio32_to_cpu(vq->vq.vdev, *p); + + if (!vring->desc_state[head].data) + continue; + + /* once add to vq fail, no more try add to vq. */ + if (err >=3D 0) { + err =3D vring_copy_to_vq_split(vq, vring, head); + if (err >=3D 0) { + desc_num +=3D err; + continue; + } + } + + buf =3D vring->desc_state[head].data; + desc_num +=3D detach_buf_from_vring_split(vring, vq, head, 0, + NULL); + recycle(&vq->vq, buf); + } + + WARN_ON(vring->num_left !=3D desc_num); +} + static void *virtqueue_detach_unused_buf_split(struct virtqueue *_vq) { struct vring_virtqueue *vq =3D to_vvq(_vq); --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B958AC433F5 for ; Sun, 24 Apr 2022 02:41:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237848AbiDXCoN (ORCPT ); Sat, 23 Apr 2022 22:44:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40160 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230382AbiDXCnx (ORCPT ); Sat, 23 Apr 2022 22:43:53 -0400 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F6F880201 for ; Sat, 23 Apr 2022 19:40:53 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04394;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzcuZE_1650768050; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzcuZE_1650768050) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:51 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 07/16] virtio_ring: split: resize support re-use buffers Date: Sun, 24 Apr 2022 10:40:35 +0800 Message-Id: <20220424024044.94749-8-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Split vring resize supports reusing the original buffer. The split vring resize function implemented earlier uses the method of letting the upper layer recycle all the buffers. This commit will first try to re-put it to the new vring in the order submitted to the old vring. The remaining buffers that cannot be submitted to the new vring will be called the recycle callback to release. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 02d4ffcc0a3b..fa4270e8c009 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1281,11 +1281,13 @@ static struct virtqueue *vring_create_virtqueue_spl= it( return vq; } =20 -static int virtqueue_resize_split(struct virtqueue *_vq, u32 num) +static int virtqueue_resize_split(struct virtqueue *_vq, u32 num, + void (*recycle)(struct virtqueue *vq, void *buf)) { + struct vring_virtqueue_split vring =3D {}, vring_old =3D {}; struct vring_virtqueue *vq =3D to_vvq(_vq); - struct vring_virtqueue_split vring =3D {}; struct virtio_device *vdev =3D _vq->vdev; + void *buf; int err; =20 err =3D vring_alloc_queue_split(&vring, vdev, num, vq->split.vring_align, @@ -1299,15 +1301,26 @@ static int virtqueue_resize_split(struct virtqueue = *_vq, u32 num) goto err; } =20 - vring_free(&vq->vq); + virtqueue_vring_detach_split(vq, &vring_old); =20 virtqueue_init(vq, vring.vring.num); virtqueue_vring_attach_split(vq, &vring); virtqueue_vring_init_split(vq); =20 + vring_reuse_bufs_split(vq, &vring_old, recycle); + vring_free_split(&vring_old, vdev); + return 0; =20 err: + /* + * In the case of failure to create vring, do not try to reuse the + * original buffer. Because the probability of this situation is not + * high, but we have to introduce new logic. + */ + while ((buf =3D virtqueue_detach_unused_buf(&vq->vq))) + recycle(&vq->vq, buf); + virtqueue_reinit_split(vq); return -ENOMEM; } @@ -2747,7 +2760,7 @@ int virtqueue_resize(struct virtqueue *_vq, u32 num, if (packed) err =3D virtqueue_resize_packed(_vq, num); else - err =3D virtqueue_resize_split(_vq, num); + err =3D virtqueue_resize_split(_vq, num, recycle); =20 if (vdev->config->enable_reset_vq(_vq)) return -EBUSY; --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 955DDC433F5 for ; Sun, 24 Apr 2022 02:41:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237864AbiDXCoW (ORCPT ); Sat, 23 Apr 2022 22:44:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237759AbiDXCnx (ORCPT ); Sat, 23 Apr 2022 22:43:53 -0400 Received: from out30-44.freemail.mail.aliyun.com (out30-44.freemail.mail.aliyun.com [115.124.30.44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A0BDD81182 for ; Sat, 23 Apr 2022 19:40:54 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R191e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04357;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzcuZa_1650768051; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzcuZa_1650768051) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:52 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 08/16] virtio_ring: packed: extract next_idx() Date: Sun, 24 Apr 2022 10:40:36 +0800 Message-Id: <20220424024044.94749-9-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Separate the logic of idx growth of packed into a function. It is convenient to share in multiple locations. Subsequent patches will also use this logic. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index fa4270e8c009..e3525d92f646 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1330,6 +1330,18 @@ static int virtqueue_resize_split(struct virtqueue *= _vq, u32 num, * Packed ring specific functions - *_packed(). */ =20 +static u32 next_idx(struct vring_virtqueue *vq, u32 idx) +{ + if ((unlikely(++idx >=3D vq->packed.vring.num))) { + idx =3D 0; + vq->packed.avail_used_flags ^=3D + 1 << VRING_PACKED_DESC_F_AVAIL | + 1 << VRING_PACKED_DESC_F_USED; + } + + return idx; +} + static void vring_unmap_extra_packed(const struct vring_virtqueue *vq, struct vring_desc_extra *extra) { @@ -1465,14 +1477,11 @@ static int virtqueue_add_indirect_packed(struct vri= ng_virtqueue *vq, vq->vq.num_free -=3D 1; =20 /* Update free pointer */ - n =3D head + 1; - if (n >=3D vq->packed.vring.num) { - n =3D 0; + n =3D next_idx(vq, head); + + if (n < head) vq->packed.avail_wrap_counter ^=3D 1; - vq->packed.avail_used_flags ^=3D - 1 << VRING_PACKED_DESC_F_AVAIL | - 1 << VRING_PACKED_DESC_F_USED; - } + vq->packed.next_avail_idx =3D n; vq->free_head =3D vq->packed.desc_extra[id].next; =20 @@ -1592,12 +1601,7 @@ static inline int virtqueue_add_packed(struct virtqu= eue *_vq, prev =3D curr; curr =3D vq->packed.desc_extra[curr].next; =20 - if ((unlikely(++i >=3D vq->packed.vring.num))) { - i =3D 0; - vq->packed.avail_used_flags ^=3D - 1 << VRING_PACKED_DESC_F_AVAIL | - 1 << VRING_PACKED_DESC_F_USED; - } + i =3D next_idx(vq, i); } } =20 --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 615D3C433F5 for ; Sun, 24 Apr 2022 02:41:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237825AbiDXCon (ORCPT ); Sat, 23 Apr 2022 22:44:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237790AbiDXCn5 (ORCPT ); Sat, 23 Apr 2022 22:43:57 -0400 Received: from out199-14.us.a.mail.aliyun.com (out199-14.us.a.mail.aliyun.com [47.90.199.14]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00D5882D07 for ; Sat, 23 Apr 2022 19:40:56 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01424;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzbmrg_1650768052; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzbmrg_1650768052) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:53 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 09/16] virtio_ring: packed: always update desc_extra Date: Sun, 24 Apr 2022 10:40:37 +0800 Message-Id: <20220424024044.94749-10-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" No longer determine whether to update desc_extra based on use_dma_api. Because desc will be modified by the device, in the process of resize, if you want to reuse buffers, you can only get len and flags from desc_extra. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index e3525d92f646..436b18184dfe 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1456,13 +1456,11 @@ static int virtqueue_add_indirect_packed(struct vri= ng_virtqueue *vq, sizeof(struct vring_packed_desc)); vq->packed.vring.desc[head].id =3D cpu_to_le16(id); =20 - if (vq->use_dma_api) { - vq->packed.desc_extra[id].addr =3D addr; - vq->packed.desc_extra[id].len =3D total_sg * - sizeof(struct vring_packed_desc); - vq->packed.desc_extra[id].flags =3D VRING_DESC_F_INDIRECT | - vq->packed.avail_used_flags; - } + vq->packed.desc_extra[id].addr =3D addr; + vq->packed.desc_extra[id].len =3D total_sg * + sizeof(struct vring_packed_desc); + vq->packed.desc_extra[id].flags =3D VRING_DESC_F_INDIRECT | + vq->packed.avail_used_flags; =20 /* * A driver MUST NOT make the first descriptor in the list @@ -1592,12 +1590,10 @@ static inline int virtqueue_add_packed(struct virtq= ueue *_vq, desc[i].len =3D cpu_to_le32(sg->length); desc[i].id =3D cpu_to_le16(id); =20 - if (unlikely(vq->use_dma_api)) { - vq->packed.desc_extra[curr].addr =3D addr; - vq->packed.desc_extra[curr].len =3D sg->length; - vq->packed.desc_extra[curr].flags =3D - le16_to_cpu(flags); - } + vq->packed.desc_extra[curr].addr =3D addr; + vq->packed.desc_extra[curr].len =3D sg->length; + vq->packed.desc_extra[curr].flags =3D le16_to_cpu(flags); + prev =3D curr; curr =3D vq->packed.desc_extra[curr].next; =20 --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DD30C433EF for ; Sun, 24 Apr 2022 02:41:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237839AbiDXCoj (ORCPT ); Sat, 23 Apr 2022 22:44:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237783AbiDXCn5 (ORCPT ); Sat, 23 Apr 2022 22:43:57 -0400 Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com [115.124.30.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9CFF82D01 for ; Sat, 23 Apr 2022 19:40:56 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04400;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzbmry_1650768053; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzbmry_1650768053) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:53 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 10/16] virtio_ring: packed: introduce vring_virtqueue_detach_packed() Date: Sun, 24 Apr 2022 10:40:38 +0800 Message-Id: <20220424024044.94749-11-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The function vring_virtqueue_detach_packed() is introduced to detach the vring of the current vq. Add two new members. last_used_idx is used to record the position where the current vring desc is used, which can be used to obtain buffers from the vring in order. Another num_left records how many buffers there are, which can be used to check the recovery of buffers completed. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 436b18184dfe..219e008a4633 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -154,6 +154,10 @@ struct vring_virtqueue_packed { dma_addr_t device_event_dma_addr; size_t ring_size_in_bytes; size_t event_size_in_bytes; + + /* for vring detach */ + u16 last_used_idx; + u32 num_left; }; =20 struct vring_virtqueue { @@ -2090,6 +2094,25 @@ static int vring_alloc_state_extra_packed(struct vri= ng_virtqueue_packed *vring) return -ENOMEM; } =20 +static void vring_virtqueue_detach_packed(struct vring_virtqueue *vq, + struct vring_virtqueue_packed *vring) +{ + vring->vring =3D vq->packed.vring; + + vring->ring_dma_addr =3D vq->packed.ring_dma_addr; + vring->driver_event_dma_addr =3D vq->packed.driver_event_dma_addr; + vring->device_event_dma_addr =3D vq->packed.device_event_dma_addr; + + vring->ring_size_in_bytes =3D vq->packed.ring_size_in_bytes; + vring->event_size_in_bytes =3D vq->packed.event_size_in_bytes; + + vring->desc_state =3D vq->packed.desc_state; + vring->desc_extra =3D vq->packed.desc_extra; + + vring->last_used_idx =3D vq->last_used_idx; + vring->num_left =3D vq->packed.vring.num - vq->vq.num_free; +} + static void virtqueue_vring_attach_packed(struct vring_virtqueue *vq, struct vring_virtqueue_packed *vring) { --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25E2CC433EF for ; Sun, 24 Apr 2022 02:41:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237614AbiDXCod (ORCPT ); Sat, 23 Apr 2022 22:44:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237788AbiDXCn5 (ORCPT ); Sat, 23 Apr 2022 22:43:57 -0400 Received: from out30-43.freemail.mail.aliyun.com (out30-43.freemail.mail.aliyun.com [115.124.30.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1119782D2E for ; Sat, 23 Apr 2022 19:40:56 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01424;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzv5F-_1650768054; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzv5F-_1650768054) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:54 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 11/16] virtio_ring: packed: extract virtqueue_update_packed() Date: Sun, 24 Apr 2022 10:40:39 +0800 Message-Id: <20220424024044.94749-12-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Separate the logic for updating the vq state from virtqueue_add_packed() and virtqueue_add_indirect_packed(). In this way, when the subsequent patch implements the logic of reusing the buffer when resize, we can share this function. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 96 ++++++++++++++++++------------------ 1 file changed, 47 insertions(+), 49 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 219e008a4633..5e6bd9a4e648 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1403,6 +1403,47 @@ static struct vring_packed_desc *alloc_indirect_pack= ed(unsigned int total_sg, return desc; } =20 +static inline void virtqueue_update_packed(struct vring_virtqueue *vq, + u32 descs_used, + u16 curr, + u16 prev, + u32 idx, + __le16 head_flags, + struct vring_packed_desc *desc, + void *data) +{ + u16 head, id; + + id =3D vq->free_head; + head =3D vq->packed.next_avail_idx; + + if (idx < head) + vq->packed.avail_wrap_counter ^=3D 1; + + /* We're using some buffers from the free list. */ + vq->vq.num_free -=3D descs_used; + + /* Update free pointer */ + vq->packed.next_avail_idx =3D idx; + vq->free_head =3D curr; + + /* Store token. */ + vq->packed.desc_state[id].num =3D descs_used; + vq->packed.desc_state[id].data =3D data; + vq->packed.desc_state[id].indir_desc =3D desc; + vq->packed.desc_state[id].last =3D prev; + + /* + * A driver MUST NOT make the first descriptor in the list + * available before all subsequent descriptors comprising + * the list are made available. + */ + virtio_wmb(vq->weak_barriers); + vq->packed.vring.desc[head].flags =3D head_flags; + vq->num_added +=3D descs_used; + +} + static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, struct scatterlist *sgs[], unsigned int total_sg, @@ -1414,6 +1455,7 @@ static int virtqueue_add_indirect_packed(struct vring= _virtqueue *vq, struct vring_packed_desc *desc; struct scatterlist *sg; unsigned int i, n, err_idx; + __le16 head_flags; u16 head, id; dma_addr_t addr; =20 @@ -1466,34 +1508,13 @@ static int virtqueue_add_indirect_packed(struct vri= ng_virtqueue *vq, vq->packed.desc_extra[id].flags =3D VRING_DESC_F_INDIRECT | vq->packed.avail_used_flags; =20 - /* - * A driver MUST NOT make the first descriptor in the list - * available before all subsequent descriptors comprising - * the list are made available. - */ - virtio_wmb(vq->weak_barriers); - vq->packed.vring.desc[head].flags =3D cpu_to_le16(VRING_DESC_F_INDIRECT | - vq->packed.avail_used_flags); - - /* We're using some buffers from the free list. */ - vq->vq.num_free -=3D 1; + head_flags =3D cpu_to_le16(VRING_DESC_F_INDIRECT | vq->packed.avail_used_= flags); =20 /* Update free pointer */ n =3D next_idx(vq, head); =20 - if (n < head) - vq->packed.avail_wrap_counter ^=3D 1; - - vq->packed.next_avail_idx =3D n; - vq->free_head =3D vq->packed.desc_extra[id].next; - - /* Store token and indirect buffer state. */ - vq->packed.desc_state[id].num =3D 1; - vq->packed.desc_state[id].data =3D data; - vq->packed.desc_state[id].indir_desc =3D desc; - vq->packed.desc_state[id].last =3D id; - - vq->num_added +=3D 1; + virtqueue_update_packed(vq, 1, vq->packed.desc_extra[id].next, id, n, + head_flags, desc, data); =20 pr_debug("Added buffer head %i to %p\n", head, vq); END_USE(vq); @@ -1605,31 +1626,8 @@ static inline int virtqueue_add_packed(struct virtqu= eue *_vq, } } =20 - if (i < head) - vq->packed.avail_wrap_counter ^=3D 1; - - /* We're using some buffers from the free list. */ - vq->vq.num_free -=3D descs_used; - - /* Update free pointer */ - vq->packed.next_avail_idx =3D i; - vq->free_head =3D curr; - - /* Store token. */ - vq->packed.desc_state[id].num =3D descs_used; - vq->packed.desc_state[id].data =3D data; - vq->packed.desc_state[id].indir_desc =3D ctx; - vq->packed.desc_state[id].last =3D prev; - - /* - * A driver MUST NOT make the first descriptor in the list - * available before all subsequent descriptors comprising - * the list are made available. - */ - virtio_wmb(vq->weak_barriers); - vq->packed.vring.desc[head].flags =3D head_flags; - vq->num_added +=3D descs_used; - + virtqueue_update_packed(vq, descs_used, curr, prev, i, head_flags, + ctx, data); pr_debug("Added buffer head %i to %p\n", head, vq); END_USE(vq); =20 --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B979EC433F5 for ; Sun, 24 Apr 2022 02:41:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237838AbiDXCor (ORCPT ); Sat, 23 Apr 2022 22:44:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237780AbiDXCoA (ORCPT ); Sat, 23 Apr 2022 22:44:00 -0400 Received: from out30-45.freemail.mail.aliyun.com (out30-45.freemail.mail.aliyun.com [115.124.30.45]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 181E88164C for ; Sat, 23 Apr 2022 19:40:57 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R851e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04407;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzcuah_1650768055; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzcuah_1650768055) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:55 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 12/16] virtio_ring: packed: extract detach_from_vring_packed() Date: Sun, 24 Apr 2022 10:40:40 +0800 Message-Id: <20220424024044.94749-13-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To handle freeing buf from the detached vring, do a split for detach_buf_packed(). The split function detach_buf_from_vring_packed() is used to release buf from vring, and the vq passed in is read-only. All modifications are for vring. In this way, detach_buf_from_vring_packed() becomes a general function, which can be used for detach_buf_packed() and also for handling detached vrings. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 36 +++++++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 5e6bd9a4e648..1efb47b88b40 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1703,28 +1703,28 @@ static bool virtqueue_kick_prepare_packed(struct vi= rtqueue *_vq) return needs_kick; } =20 -static void detach_buf_packed(struct vring_virtqueue *vq, - unsigned int id, void **ctx) +static int detach_buf_from_vring_packed(struct vring_virtqueue_packed *vri= ng, + struct vring_virtqueue const *vq, + unsigned int id, + unsigned int free_head, + void **ctx) { struct vring_desc_state_packed *state =3D NULL; struct vring_packed_desc *desc; unsigned int i, curr; =20 - state =3D &vq->packed.desc_state[id]; + state =3D &vring->desc_state[id]; =20 /* Clear data ptr. */ state->data =3D NULL; =20 - vq->packed.desc_extra[state->last].next =3D vq->free_head; - vq->free_head =3D id; - vq->vq.num_free +=3D state->num; + vring->desc_extra[state->last].next =3D free_head; =20 if (unlikely(vq->use_dma_api)) { curr =3D id; for (i =3D 0; i < state->num; i++) { - vring_unmap_extra_packed(vq, - &vq->packed.desc_extra[curr]); - curr =3D vq->packed.desc_extra[curr].next; + vring_unmap_extra_packed(vq, &vring->desc_extra[curr]); + curr =3D vring->desc_extra[curr].next; } } =20 @@ -1734,10 +1734,10 @@ static void detach_buf_packed(struct vring_virtqueu= e *vq, /* Free the indirect table, if any, now that it's unmapped. */ desc =3D state->indir_desc; if (!desc) - return; + return state->num; =20 if (vq->use_dma_api) { - len =3D vq->packed.desc_extra[id].len; + len =3D vring->desc_extra[id].len; for (i =3D 0; i < len / sizeof(struct vring_packed_desc); i++) vring_unmap_desc_packed(vq, &desc[i]); @@ -1747,6 +1747,20 @@ static void detach_buf_packed(struct vring_virtqueue= *vq, } else if (ctx) { *ctx =3D state->indir_desc; } + + return state->num; +} + +static void detach_buf_packed(struct vring_virtqueue *vq, + unsigned int id, void **ctx) +{ + int num; + + num =3D detach_buf_from_vring_packed(&vq->packed, vq, id, vq->free_head, + ctx); + + vq->free_head =3D id; + vq->vq.num_free +=3D num; } =20 static inline bool is_used_desc_packed(const struct vring_virtqueue *vq, --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B3EDC433F5 for ; Sun, 24 Apr 2022 02:42:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237823AbiDXCpE (ORCPT ); Sat, 23 Apr 2022 22:45:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237792AbiDXCoB (ORCPT ); Sat, 23 Apr 2022 22:44:01 -0400 Received: from out30-44.freemail.mail.aliyun.com (out30-44.freemail.mail.aliyun.com [115.124.30.44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3076C84EE1 for ; Sat, 23 Apr 2022 19:40:58 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01424;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzcub1_1650768055; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzcub1_1650768055) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:56 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 13/16] virtio_ring: packed: support copy from vring Date: Sun, 24 Apr 2022 10:40:41 +0800 Message-Id: <20220424024044.94749-14-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To support reusing old buffers during resize. This patch implements copying a buffer from the detached vring to the vq where the new vring is attached. This process is similar to virtqueue_add_packed(), but skips DMA. Use the function virtqueue_update_packed() provided by the previous patch to update the state of the vq. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 76 ++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 1efb47b88b40..8ca9985ffb4b 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1654,6 +1654,82 @@ static inline int virtqueue_add_packed(struct virtqu= eue *_vq, return -EIO; } =20 +static u32 vring_copy_desc_packed(struct vring_virtqueue *vq, + u32 idx, + u16 curr, + __le16 *head_flags, + struct vring_virtqueue_packed *vring, + u32 src) +{ + u16 old_flags =3D vring->desc_extra[src].flags; + u16 flags =3D vq->packed.avail_used_flags; + struct vring_packed_desc *desc; + struct vring_desc_extra *extra; + + if (old_flags & VRING_DESC_F_NEXT) + flags |=3D VRING_DESC_F_NEXT; + + if (old_flags & VRING_DESC_F_WRITE) + flags |=3D VRING_DESC_F_WRITE; + + if (old_flags & VRING_DESC_F_INDIRECT) + flags |=3D VRING_DESC_F_INDIRECT; + + desc =3D vq->packed.vring.desc + idx; + extra =3D vq->packed.desc_extra + curr; + + if (head_flags) + *head_flags =3D cpu_to_le16(flags); + else + desc->flags =3D cpu_to_le16(flags); + + desc->addr =3D cpu_to_le64(vring->desc_extra[src].addr); + desc->len =3D cpu_to_le32(vring->desc_extra[src].len); + desc->id =3D cpu_to_le16(vq->free_head); + + extra->addr =3D vring->desc_extra[src].addr; + extra->len =3D vring->desc_extra[src].len; + extra->flags =3D vring->desc_extra[src].flags; + + return vq->packed.desc_extra[curr].next; +} + +static int vring_copy_to_vq_packed(struct vring_virtqueue *vq, + struct vring_virtqueue_packed *vring, + u32 old_id) +{ + struct vring_desc_state_packed *state; + __le16 head_flags; + u16 prev, curr; + u32 i, n; + + state =3D &vring->desc_state[old_id]; + + if (state->num > vq->vq.num_free) + return -ENOSPC; + + i =3D vq->packed.next_avail_idx; + curr =3D vq->free_head; + + for (n =3D 0; n < state->num; n++) { + prev =3D curr; + curr =3D vring_copy_desc_packed(vq, i, curr, + n ? NULL : &head_flags, + vring, old_id); + + old_id =3D vring->desc_extra[old_id].next; + + i =3D next_idx(vq, i); + } + + virtqueue_update_packed(vq, state->num, curr, prev, i, head_flags, + state->indir_desc, state->data); + + state->data =3D NULL; + + return state->num; +} + static bool virtqueue_kick_prepare_packed(struct virtqueue *_vq) { struct vring_virtqueue *vq =3D to_vvq(_vq); --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 201CBC433F5 for ; Sun, 24 Apr 2022 02:42:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237892AbiDXCo4 (ORCPT ); Sat, 23 Apr 2022 22:44:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40858 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237796AbiDXCoB (ORCPT ); Sat, 23 Apr 2022 22:44:01 -0400 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F8AF85970 for ; Sat, 23 Apr 2022 19:40:59 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04407;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzbmsJ_1650768056; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzbmsJ_1650768056) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:57 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 14/16] virtio_ring: packed: introduce vring_reuse_bufs_packed() Date: Sun, 24 Apr 2022 10:40:42 +0800 Message-Id: <20220424024044.94749-15-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" vring_reuse_bufs_packed() checks vring.desc forward based on last_used_idx. The resulting buffers are the order in which they were committed. It is beneficial to ensure the order of buffers in the process of reuse. For example, under virtio-net, if the order is not guaranteed, it may lead to out-of-order tcp streams. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 45 ++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 8ca9985ffb4b..66f71e22ece0 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2041,6 +2041,51 @@ static bool virtqueue_enable_cb_delayed_packed(struc= t virtqueue *_vq) return true; } =20 +static void vring_reuse_bufs_packed(struct vring_virtqueue *vq, + struct vring_virtqueue_packed *vring, + void (*recycle)(struct virtqueue *vq, void *buf)) +{ + struct vring_desc_state_packed *state; + u32 last_used, id, desc_num =3D 0; + int err =3D 0; + void *buf; + + last_used =3D vring->last_used_idx; + + do { + id =3D le16_to_cpu(vring->vring.desc[last_used].id); + + state =3D &vring->desc_state[id]; + + if (!state->data) { + last_used++; + goto next; + } + + /* once add to vq fail, no more try add to vq. */ + if (err >=3D 0) { + err =3D vring_copy_to_vq_packed(vq, vring, id); + if (err >=3D 0) + goto ok; + } + + buf =3D state->data; + detach_buf_from_vring_packed(vring, vq, id, 0, NULL); + recycle(&vq->vq, buf); + +ok: + last_used +=3D state->num; + desc_num +=3D state->num; + +next: + if (unlikely(last_used >=3D vring->vring.num)) + last_used -=3D vring->vring.num; + + } while (last_used !=3D vring->last_used_idx); + + WARN_ON(vring->num_left !=3D desc_num); +} + static void *virtqueue_detach_unused_buf_packed(struct virtqueue *_vq) { struct vring_virtqueue *vq =3D to_vvq(_vq); --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 298C0C433EF for ; Sun, 24 Apr 2022 02:41:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237887AbiDXCot (ORCPT ); Sat, 23 Apr 2022 22:44:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237766AbiDXCoB (ORCPT ); Sat, 23 Apr 2022 22:44:01 -0400 Received: from out30-44.freemail.mail.aliyun.com (out30-44.freemail.mail.aliyun.com [115.124.30.44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B287582D01 for ; Sat, 23 Apr 2022 19:41:00 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R531e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04423;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzpPKB_1650768057; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzpPKB_1650768057) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:58 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 15/16] virtio_ring: packed: resize support re-use buffers Date: Sun, 24 Apr 2022 10:40:43 +0800 Message-Id: <20220424024044.94749-16-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Packed vring resize supports reusing the original buffer. The packed vring resize function implemented earlier uses the method of letting the upper layer recycle all the buffers. This commit will first try to re-put it to the new vring in the order submitted to the old vring. The remaining buffers that cannot be submitted to the new vring will be called the recycle callback to release. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 66f71e22ece0..730c8dded4c7 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -217,7 +217,6 @@ struct vring_virtqueue { }; =20 static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num); -static void vring_free(struct virtqueue *_vq); =20 /* * Helpers. @@ -2357,11 +2356,13 @@ static struct virtqueue *vring_create_virtqueue_pac= ked( return NULL; } =20 -static int virtqueue_resize_packed(struct virtqueue *_vq, u32 num) +static int virtqueue_resize_packed(struct virtqueue *_vq, u32 num, + void (*recycle)(struct virtqueue *vq, void *buf)) { - struct vring_virtqueue_packed vring =3D {}; + struct vring_virtqueue_packed vring =3D {}, vring_old =3D {}; struct vring_virtqueue *vq =3D to_vvq(_vq); struct virtio_device *vdev =3D _vq->vdev; + void *buf; int err; =20 if (vring_alloc_queue_packed(&vring, vdev, num)) @@ -2371,17 +2372,28 @@ static int virtqueue_resize_packed(struct virtqueue= *_vq, u32 num) if (err) goto err_state_extra; =20 - vring_free(&vq->vq); + vring_virtqueue_detach_packed(vq, &vring_old); =20 virtqueue_init(vq, vring.vring.num); virtqueue_vring_attach_packed(vq, &vring); virtqueue_vring_init_packed(vq); =20 + vring_reuse_bufs_packed(vq, &vring_old, recycle); + vring_free_packed(&vring_old, vdev); + return 0; =20 err_state_extra: vring_free_packed(&vring, vdev); err_ring: + /* + * In the case of failure to create vring, do not try to reuse the + * original buffer. Because the probability of this situation is not + * high, but we have to introduce new logic. + */ + while ((buf =3D virtqueue_detach_unused_buf(&vq->vq))) + recycle(&vq->vq, buf); + virtqueue_reinit_packed(vq); return -ENOMEM; } @@ -2914,7 +2926,7 @@ int virtqueue_resize(struct virtqueue *_vq, u32 num, recycle(_vq, buf); =20 if (packed) - err =3D virtqueue_resize_packed(_vq, num); + err =3D virtqueue_resize_packed(_vq, num, recycle); else err =3D virtqueue_resize_split(_vq, num, recycle); =20 --=20 2.31.0 From nobody Sun May 10 21:18:49 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1EE8C433F5 for ; Sun, 24 Apr 2022 02:42:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237856AbiDXCpA (ORCPT ); Sat, 23 Apr 2022 22:45:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237795AbiDXCoB (ORCPT ); Sat, 23 Apr 2022 22:44:01 -0400 Received: from out30-57.freemail.mail.aliyun.com (out30-57.freemail.mail.aliyun.com [115.124.30.57]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75C9D86E12 for ; Sat, 23 Apr 2022 19:41:01 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R191e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04357;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VAzpPKJ_1650768058; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VAzpPKJ_1650768058) by smtp.aliyun-inc.com(127.0.0.1); Sun, 24 Apr 2022 10:40:59 +0800 From: Xuan Zhuo To: linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , virtualization@lists.linux-foundation.org Subject: [RFC PATCH 16/16] virtio_ring: virtqueue_resize() no longer call recycle() directly Date: Sun, 24 Apr 2022 10:40:44 +0800 Message-Id: <20220424024044.94749-17-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> References: <20220424024044.94749-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: c42022d07dde Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" virtqueue_resize() no longer calls the recycle callback to release the buffer. These bufs are reused by virtqueue_resize_* first, and if they cannot be used, the buffers that cannot be reused will be released Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 730c8dded4c7..51dab35a54c9 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2895,7 +2895,6 @@ int virtqueue_resize(struct virtqueue *_vq, u32 num, struct vring_virtqueue *vq =3D to_vvq(_vq); struct virtio_device *vdev =3D vq->vq.vdev; bool packed; - void *buf; int err; =20 if (!vq->we_own_ring) @@ -2922,16 +2921,19 @@ int virtqueue_resize(struct virtqueue *_vq, u32 num, if (err) return err; =20 - while ((buf =3D virtqueue_detach_unused_buf(_vq)) !=3D NULL) - recycle(_vq, buf); - if (packed) err =3D virtqueue_resize_packed(_vq, num, recycle); else err =3D virtqueue_resize_split(_vq, num, recycle); =20 - if (vdev->config->enable_reset_vq(_vq)) + if (vdev->config->enable_reset_vq(_vq)) { return -EBUSY; + } else if (!err) { + num =3D packed ? vq->packed.vring.num : vq->split.vring.num; + + if (num !=3D vq->vq.num_free) + virtqueue_kick(_vq); + } =20 return err; } --=20 2.31.0