From nobody Sun Feb 8 10:34:08 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 560253624BD for ; Fri, 16 Jan 2026 11:42:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768563778; cv=none; b=OrQwl9Cl/CYZTMkstPCqqJ8ueWykewzuNTmdePpF65IxIc5ihc0SK146lgIHzB5BOVJ8Gz5eZj62Z2SsKzUOvUClElRick+34xhi4tSlA6T7uLBbBFyZEuNvNW0dFGO1QsNnPCnUxJsKXanYb4LogNxrKIBh0rVYtWNy58avUOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768563778; c=relaxed/simple; bh=68Rv2zV+IH8ejpZkWV3TYUJiEfMMJYeNAhvl7aY2Lsk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=eTWv2E5Sy0NPZV2UtOv/udVS4NlepS1qvWpwQJrgJSeQYfnN2Rtjla+w7NcO5KjNfyISerm09FC2/fKkTtb3AXmfBHR25zlbS55g9ZPEeUCR1Y6M1Bg8TQoaYcLkAjJyGfgaXWr9x9/x6js647VoIDQmvz4nQloDjk1y6XbLc5E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=RSwxkimC; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RSwxkimC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1768563773; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=URhy/eVdYPWYFT6arANROkh3If2bK/p/7S6HuKXpigM=; b=RSwxkimCQpYvh1SN9pUyQ9Shh86uMRTqSTlg5gCePbcnmizaJ9TIwN+kRXoAqORVNPAV77 60hy1fYUsGqt2AIqpkQl2L/9gyKh0MqXzSelJXWSHgEbCa8bSYua9Mjiko5NA8womJGDf3 NFbg1u3Ip1Liqu7d+anlITjejRz1ptw= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-616-VNoEPQnUMVOROrSVRLcVRg-1; Fri, 16 Jan 2026 06:42:52 -0500 X-MC-Unique: VNoEPQnUMVOROrSVRLcVRg-1 X-Mimecast-MFC-AGG-ID: VNoEPQnUMVOROrSVRLcVRg_1768563770 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D17271956046; Fri, 16 Jan 2026 11:42:50 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.82]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4707B180066A; Fri, 16 Jan 2026 11:42:47 +0000 (UTC) From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: "Michael S . Tsirkin " Cc: Stefano Garzarella , virtualization@lists.linux.dev, Xuan Zhuo , Maxime Coquelin , Laurent Vivier , Yongji Xie , Cindy Lu , linux-kernel@vger.kernel.org, =?UTF-8?q?Eugenio=20P=C3=A9rez?= , jasowang@redhat.com Subject: [PATCH v13 03/13] vduse: add vq group support Date: Fri, 16 Jan 2026 12:42:21 +0100 Message-ID: <20260116114231.1474306-4-eperezma@redhat.com> In-Reply-To: <20260116114231.1474306-1-eperezma@redhat.com> References: <20260116114231.1474306-1-eperezma@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 This allows separate the different virtqueues in groups that shares the same address space. Asking the VDUSE device for the groups of the vq at the beginning as they're needed for the DMA API. Allocating 3 vq groups as net is the device that need the most groups: * Dataplane (guest passthrough) * CVQ * Shadowed vrings. Future versions of the series can include dynamic allocation of the groups array so VDUSE can declare more groups. Acked-by: Jason Wang Reviewed-by: Xie Yongji Signed-off-by: Eugenio P=C3=A9rez --- v12: * Change the style of checking for vq group =3D=3D 0 in VDUSE_VQ_SETUP dev ioctl if api_version < 1 (MST). v11: * Rename vq->vq_group to vq->group (Jason). * Do not reset vq group at virtio reset (Jason). v6: * s/sepparate/separate (MST). * s/dev->api_version < 1/dev->api_version < VDUSE_API_VERSION_1 v5: * Revert core vdpa changes (Jason). * Fix group =3D=3D ngroup case in checking VQ_SETUP argument (Jason). v4: * Revert the "invalid vq group" concept and assume 0 if not set (Jason). * Make config->ngroups =3D=3D 0 invalid (Jason). v3: * Make the default group an invalid group as long as VDUSE device does not set it to some valid u32 value. Modify the vdpa core to take that into account (Jason). * Create the VDUSE_DEV_MAX_GROUPS instead of using a magic number v2: * Now the vq group is in vduse_vq_config struct instead of issuing one VDUSE message per vq. v1: * Fix: Remove BIT_ULL(VIRTIO_S_*), as _S_ is already the bit (Maxime) RFC v3: * Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs. * Remove the descs vq group capability as it will not be used and we can add it on top. * Do not ask for vq groups in number of vq groups < 2. * Move the valid vq groups range check to vduse_validate_config. RFC v2: * Cache group information in kernel, as we need to provide the vq map tokens properly. * Add descs vq group to optimize SVQ forwarding and support indirect descriptors out of the box. --- drivers/vdpa/vdpa_user/vduse_dev.c | 47 ++++++++++++++++++++++++++---- include/uapi/linux/vduse.h | 12 ++++++-- 2 files changed, 51 insertions(+), 8 deletions(-) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index ae357d014564..5bffc25a266e 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -39,6 +39,7 @@ #define DRV_LICENSE "GPL v2" =20 #define VDUSE_DEV_MAX (1U << MINORBITS) +#define VDUSE_DEV_MAX_GROUPS 0xffff #define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024) #define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024) #define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024) @@ -58,6 +59,7 @@ struct vduse_virtqueue { struct vdpa_vq_state state; bool ready; bool kicked; + u32 group; spinlock_t kick_lock; spinlock_t irq_lock; struct eventfd_ctx *kickfd; @@ -114,6 +116,7 @@ struct vduse_dev { u8 status; u32 vq_num; u32 vq_align; + u32 ngroups; struct vduse_umem *umem; struct mutex mem_lock; unsigned int bounce_size; @@ -592,6 +595,16 @@ static int vduse_vdpa_set_vq_state(struct vdpa_device = *vdpa, u16 idx, return 0; } =20 +static u32 vduse_get_vq_group(struct vdpa_device *vdpa, u16 idx) +{ + struct vduse_dev *dev =3D vdpa_to_vduse(vdpa); + + if (dev->api_version < VDUSE_API_VERSION_1) + return 0; + + return dev->vqs[idx]->group; +} + static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx, struct vdpa_vq_state *state) { @@ -789,6 +802,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_o= ps =3D { .set_vq_cb =3D vduse_vdpa_set_vq_cb, .set_vq_num =3D vduse_vdpa_set_vq_num, .get_vq_size =3D vduse_vdpa_get_vq_size, + .get_vq_group =3D vduse_get_vq_group, .set_vq_ready =3D vduse_vdpa_set_vq_ready, .get_vq_ready =3D vduse_vdpa_get_vq_ready, .set_vq_state =3D vduse_vdpa_set_vq_state, @@ -1252,12 +1266,24 @@ static long vduse_dev_ioctl(struct file *file, unsi= gned int cmd, if (config.index >=3D dev->vq_num) break; =20 - if (!is_mem_zero((const char *)config.reserved, - sizeof(config.reserved))) + if (dev->api_version < VDUSE_API_VERSION_1) { + if (config.group) + break; + } else { + if (config.group >=3D dev->ngroups) + break; + if (dev->status & VIRTIO_CONFIG_S_DRIVER_OK) + break; + } + + if (config.reserved1 || + !is_mem_zero((const char *)config.reserved2, + sizeof(config.reserved2))) break; =20 index =3D array_index_nospec(config.index, dev->vq_num); dev->vqs[index]->num_max =3D config.max_size; + dev->vqs[index]->group =3D config.group; ret =3D 0; break; } @@ -1737,12 +1763,20 @@ static bool features_is_valid(struct vduse_dev_conf= ig *config) return true; } =20 -static bool vduse_validate_config(struct vduse_dev_config *config) +static bool vduse_validate_config(struct vduse_dev_config *config, + u64 api_version) { if (!is_mem_zero((const char *)config->reserved, sizeof(config->reserved))) return false; =20 + if (api_version < VDUSE_API_VERSION_1 && config->ngroups) + return false; + + if (api_version >=3D VDUSE_API_VERSION_1 && + (!config->ngroups || config->ngroups > VDUSE_DEV_MAX_GROUPS)) + return false; + if (config->vq_align > PAGE_SIZE) return false; =20 @@ -1858,6 +1892,9 @@ static int vduse_create_dev(struct vduse_dev_config *= config, dev->device_features =3D config->features; dev->device_id =3D config->device_id; dev->vendor_id =3D config->vendor_id; + dev->ngroups =3D (dev->api_version < VDUSE_API_VERSION_1) + ? 1 + : config->ngroups; dev->name =3D kstrdup(config->name, GFP_KERNEL); if (!dev->name) goto err_str; @@ -1936,7 +1973,7 @@ static long vduse_ioctl(struct file *file, unsigned i= nt cmd, break; =20 ret =3D -EINVAL; - if (vduse_validate_config(&config) =3D=3D false) + if (!vduse_validate_config(&config, control->api_version)) break; =20 buf =3D vmemdup_user(argp + size, config.config_size); @@ -2017,7 +2054,7 @@ static int vduse_dev_init_vdpa(struct vduse_dev *dev,= const char *name) =20 vdev =3D vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev, &vduse_vdpa_config_ops, &vduse_map_ops, - 1, 1, name, true); + dev->ngroups, 1, name, true); if (IS_ERR(vdev)) return PTR_ERR(vdev); =20 diff --git a/include/uapi/linux/vduse.h b/include/uapi/linux/vduse.h index ccb92a1efce0..a3d51cf6df3a 100644 --- a/include/uapi/linux/vduse.h +++ b/include/uapi/linux/vduse.h @@ -31,6 +31,7 @@ * @features: virtio features * @vq_num: the number of virtqueues * @vq_align: the allocation alignment of virtqueue's metadata + * @ngroups: number of vq groups that VDUSE device declares * @reserved: for future use, needs to be initialized to zero * @config_size: the size of the configuration space * @config: the buffer of the configuration space @@ -45,7 +46,8 @@ struct vduse_dev_config { __u64 features; __u32 vq_num; __u32 vq_align; - __u32 reserved[13]; + __u32 ngroups; /* if VDUSE_API_VERSION >=3D 1 */ + __u32 reserved[12]; __u32 config_size; __u8 config[]; }; @@ -122,14 +124,18 @@ struct vduse_config_data { * struct vduse_vq_config - basic configuration of a virtqueue * @index: virtqueue index * @max_size: the max size of virtqueue - * @reserved: for future use, needs to be initialized to zero + * @reserved1: for future use, needs to be initialized to zero + * @group: virtqueue group + * @reserved2: for future use, needs to be initialized to zero * * Structure used by VDUSE_VQ_SETUP ioctl to setup a virtqueue. */ struct vduse_vq_config { __u32 index; __u16 max_size; - __u16 reserved[13]; + __u16 reserved1; + __u32 group; + __u16 reserved2[10]; }; =20 /* --=20 2.52.0